Tutorials

Better Testing of Microservices Using Consumer-Driven Contracts in Node.js


In this post we’ll look into using Pact for testing interactions between microservices. Pact is a family of frameworks that allows us to do Consumer-Driven Contract testing. We’ll also discuss the advantages of using Consumer-Driven Contracts over integration tests, which are the traditional way of testing interactions between microservices.

If you are working on a distributed system and have never heard of Consumer-Driven Contracts or think that the only way to test the interaction between services is with integration tests, I think you’ll find this post extremely useful.

Post Service

To demonstrate the concepts in this post, I’ve created a service example called PostService, which returns posts from a fictional blog. The service exposes two API endpoints:

  • /post/list: returns the list of all posts
  • /post/:id: returns post whose id is :id

Download source

Here is an example of a response returned by the /post/list endpoint:

{
  "posts": [
    {
      "id": 1,
      "date": "01/10/2016",
      "contents": "...."
    },
    {
      "id": 2,
      "date": "01/09/2016",
      "contents": "...."
    }
  ]
}

The example also has a client side that acts as a consumer of the PostService. In a real life application this client could either be another application, microservice or web client.

Interaction between Consumer and Provider

Interaction between Consumer and Provider

We can individually unit-test both the consumer and provider to verify that they work as expected. However obviously, this is not enough to guarantee that both sides interact correctly. For example:

  • The provider might change the endpoint /post/list to /posts
  • The provider might rename or remove certain fields in the payload
  • The provider might add new mandatory parameters
  • The provider might remove endpoints being used by the consumer

Those types of changes to the service interface are not things that isolated unit-test would catch. Also, not all problems are the provider’s fault, breaking backwards compatibility. There is also the possibility that the consumer is making the wrong assumptions about the provider or using outdated specifications.

The traditional way to test the interaction between two services, a provider and a consumer, is to use integration tests.

Using Integration Tests

The idea is to run both the consumer and provider services in some integration environment and check that they interact as expected. This type of testing simulates what the services would do in the production environment, so in theory integration tests make sense. However, there are a few problems with this approach.

First of all, integration tests are generally slow. They require setting up an integration environment, starting both consumer and provider services and initializing their dependencies. This might not seem like a problem at first, but as the number of integration tests starts to increase, the building process becomes slower and slower. This is especially true in a microservice architecture. Having integration tests between every pair of interacting microservices doesn’t scale.

Another issue with integration tests is that they are brittle. Sometimes they will fail for reasons not related to the services themselves – there might be issues with the network or with external dependencies like databases. This means that a failed integration test doesn’t necessarily mean that there is a problem with the code. Martin Fowler does a great job explaining the (bad) consequences of having non-deterministic tests in our test suite.

Even if an integration test fails because of an actual integration problem between the consumer and provider services, the high-level nature of integration tests makes it hard to pinpoint where the problem is: is it the consumer service’s fault? Or the provider service’s? Or both? Add this to the fact that integration tests are mostly performed by QA teams and not the developers themselves, which means having the extra overhead of going back and forth between teams when things go wrong. This also leads to the question of who is better qualified for testing the integration points between two services: the QA team? Or the actual developers of the services? Clearly knowing whether two services interact correctly at development time, before reaching QA, would save us a big chunk of pain and headache.

So you might be asking yourself, if integration tests are so bad (hopefully you’re convinced) why are they still so popular? Well, I think it’s because there haven’t been viable alternatives up until very recently.

Also note that in contrast with integration tests, unit tests are robust, reliable, they work fast and they are very specific in telling us where the problems are. It seems that replacing integration tests with better tests would certainly improve our development, testing and deployment experience.

Using Consumer-Driven Contracts

The idea of Consumer-Driven Contracts is to formalize the interactions between a consumer and a provider. The consumer creates a contract, which is an agreement between the consumer and a provider on the interactions that will take place between them. Or in other words, what the consumer expects from the provider. Once the provider has agreed on the contract, both consumer and provider can take a copy of the contract and use tests to verify that their corresponding implementations don’t violate the contract. The advantage of these new tests is that they are pretty much unit-tests: they can be run locally and independently, and they are fast and reliable.

If a given provider has several consumers, then it would have to verify several contracts – one for each consumer. This would ensure that changes to the provider haven’t broken any consumer services.

All this sounds good albeit a bit theoretical: How does the consumer define contracts? How does the provider verify them?

Pact

Pact is a set of frameworks that enable Consumer-Driven Contract testing. There are implementations of Pact for several platforms like Ruby, .NET and JVM. In this example we’re using Pact JS, which is a javascript implementation of Pact.

Consumer Side

The first step when using Pact is for the Consumer to define its expectations from the provider and create unit-tests which confirm that the consumer service client interacts correctly with the remote service provider. In our example, we’ll create a spec file called PostServiceClient.spec.js in which:

  1. We’ll create a mock web server that will act as the service provider. Pact JS provides this functionality out of the box.
  2. For each request that we want to verify, we’ll define the expected response that the mock service should return. In Pact terminology, these are called interactions: for a given request, what does the consumer expect the provider to return?
  3. Then we’ll create unit-tests, in which we’ll run our service client against the mock provider and verify that the client returns the expected values.
  4. Finally, we’ll create a Pact file containing the consumer expectations. More on this below.
Consumer Side of Pact Verification

Consumer Side of Pact Verification

In code it looks like this:

...
   // Configure mock server
    const mockServer = wrapper.createServer({
        port: 1234,
        log: path.resolve(process.cwd(), 'logs', 'mockserver-integration.log'),
        dir: path.resolve(process.cwd(), 'pacts'),
        spec: 2
    });

    // Define expected payloads
    const expectedBodyPostList = {
        posts: [
            {id: 1, date: '01/10/2016', contents: 'Bla bla bla'},
            {id: 2, date: '01/09/2016', contents: 'Microservice microservice'}
        ]
    };

    const expectedBodyPostGet = {
        post: {id: 1, date: '01/08/2016', contents: 'Bla'}
    };

    before((done) => {

        // Start mock server
        mockServer.start().then(() => {
            provider = Pact({consumer: 'My Consumer', provider: 'Posts Provider', port: 1234});

            // Add interactions
            provider.addInteraction({
                state: 'Has two posts',
                uponReceiving: 'a request for all posts',
                withRequest: {
                    method: 'GET',
                    path: '/post/list',
                    headers: {'Accept': 'application/json'}
                },
                willRespondWith: {
                    status: 200,
                    headers: {'Content-Type': 'application/json'},
                    body: expectedBodyPostList
                }
            }).then(() => {
                provider.addInteraction({
                    state: 'Has one post',
                    uponReceiving: 'a request for one post',
                    withRequest: {
                        method: 'GET',
                        path: '/post/1',
                        headers: {'Accept': 'application/json'}
                    },
                    willRespondWith: {
                        status: 200,
                        headers: {'Content-Type': 'application/json'},
                        body: expectedBodyPostGet
                    }
                }).then(() => done())
            })
        })
    });

    // Verify service client works as expected
    it('successfully receives all post', (done) => {
        const postServiceClient = new PostServiceClient('http://localhost:1234');
        const verificationPromise = postServiceClient.getAllPosts();
        const expectedPosts = [
            Post.fromJson(expectedBodyPostList.posts[0]),
            Post.fromJson(expectedBodyPostList.posts[1])
        ];

        expect(verificationPromise).to.eventually.eql(expectedPosts).notify(done);
    });

    it('successfully receives one post', (done) => {
        const postServiceClient = new PostServiceClient('http://localhost:1234');
        const verificationPromise = postServiceClient.getPostById(1);
        const expectedPost = Post.fromJson(expectedBodyPostGet.post);

        expect(verificationPromise).to.eventually.eql(expectedPost).notify(done);
    });

    after(() => {
        // Write pact files
        provider.finalize().then(() => {
            wrapper.removeAllServers()

        })
    });

...

We can run our spec file as follows: mocha app/client/spec/PostServiceClient.spec.js. Assuming that the provider will stick to the contract that the consumer defined, we can be sure that the consumer interacts correctly with the provider.

Pact File

As noted earlier, the script above will not only run the unit-tests but also generate a pact file called my_consumer-posts_provider.json. This file, in JSON format, is essentially the contract between consumer and provider and it contains all the interactions we defined above. Each interaction specifies what the service provider is supposed to return for a given request. This is what the file looks like:

{
  "consumer": {
    "name": "My Consumer"
  },
  "provider": {
    "name": "Posts Provider"
  },
  "interactions": [
    {
      "description": "a request for all posts",
      "provider_state": "Has two posts",
      "request": {
        "method": "GET",
        "path": "/post/list",
        "headers": {
          "Accept": "application/json"
        }
      },
      "response": {
        "status": 200,
        "headers": {
          "Content-Type": "application/json"
        },
        "body": {
          "posts": [
            {
              "id": 1,
              "date": "01/10/2016",
              "contents": "Bla bla bla"
            },
            {
              "id": 2,
              "date": "01/09/2016",
              "contents": "Microservice microservice"
            }
          ]
        }
      }
    },
    {
      "description": "a request for one post",
      "provider_state": "Has one post",
      "request": {
        "method": "GET",
        "path": "/post/1",
        "headers": {
          "Accept": "application/json"
        }
      },
      "response": {
        "status": 200,
        "headers": {
          "Content-Type": "application/json"
        },
        "body": {
          "post": {
            "id": 1,
            "date": "01/08/2016",
            "contents": "Bla"
          }
        }
      }
    }
  ],
  "metadata": {
    "pactSpecificationVersion": "2.0.0"
  }
}

Provider Side

The provider side verification is very straightforward. The provider receives the pact file from the consumer and needs to verify that it doesn’t violate the interactions that the consumer defined. Pact JS will read the pact file, execute the corresponding request from each interaction and confirm that the service returns that payload that the consumer expects.

Provider side verification

Provider side verification

The code is also very straightforward:

var pact = require('@pact-foundation/pact-node');
var path = require('path');
const PostService = require('../PostService');

var opts = {
    providerBaseUrl: 'http://localhost:8081',
    providerStatesUrl: 'http://localhost:8081/states',
    providerStatesSetupUrl: 'http://localhost:8081/setup',
    pactUrls: [path.resolve(__dirname, '../../../pacts/my_consumer-posts_provider.json')]
};

pact.verifyPacts(opts).then(() => {
    console.log('success');
    process.exit(0);
}).catch((error) => {
    console.log('failed', error);
    process.exit(1);
});

We include and start our PostService and then we tell Pact to verify that the service complies with the interactions defined in my_consumer-posts_provider.json file. We can run our script as follows: node app/service/spec/PostService.spec.js.

By verifying that the provider doesn’t violate any of its consumers’ contracts, we can be sure that the latest changes to the provider’s code haven’t broken any consumer code.

In this example, both consumer and provider are in the same codebase, so the provider has access to the pact file that the consumer generated. In most real-life distributed applications, consumer and provider are in different codebases, and generally, maintained by different teams. For these type of situations, you can use Pact Broker, which allows you to share pact files across different projects.

One thing I haven’t covered here is the use of provider states for injecting data into the provider. You can read more about provider states and how they work here.

In conclusion, by formalising the interaction between consumer and provider, and verifying that both services stick to the contract they’ve agreed upon, we see that we have no need to perform integration tests between them. Instead we’re using new tests that are specific, fast and robust. I encourage you to try Consumer-Driven Contracts and Pact and see how you can improve your testing process.

Software Engineering
Building a Serverless Mesh Processing Microservice in Node.js
Software Development
Are You Agile? 6 Signs Your Team May Not Be
System Architecture
Integration Patterns for Microservices Architectures: The good, the bad and the ugly
There are currently no comments.