Integrate API tests with Postman, Newman, and Travis CI

Joyce

August 23, 2017

API testing is hard, but it doesn’t have to be. Postman makes it easy to write tests, make sure they’re working, set up the testing environment, and then eventually hook it all up to your build system. More than ever before, developers are using Postman tests in their Continuous Integration or Continuous Delivery pipeline.

Continuous Integration (CI)

CI is a practice that requires developers to integrate code into a shared repository several times a day. By committing early and often, the team avoids a ton of technical debt by allowing teams to detect problems early while conflicts are relatively easy to fix. Every check-in kicks off an automated build process which typically includes testing and (if your commit hasn’t broken anything) might include deployment too.

Postman has a number of tools to make API testing easier. Postman has a testing sandbox that lets you write and execute tests for your API, and Newman is Postman’s command line tool that sends API requests, receives the response, and then runs your tests against the response. We won’t dive into the details of setting up a collection or writing tests, but here’s a tutorial on writing tests if you’re just getting started.

In general, integrating your Postman tests with your favorite continuous integration service is the same process if you’re running on Jenkins, Travis CI, AppVeyor, or any other build system. You will set up your CI configuration to run a shell command upon kicking off your build. The command is a Newman script that runs your collection with the tests, returning a pass or fail exit code that’s logged in your CI system.

In this example, we’ll walk through how to integrate Postman with Travis CI, a continuous integration service used to build and test projects hosted on GitHub. Travis CI will run your tests every time you commit to your GitHub repo, submit a pull request, or some other specified configuration.

Before we get started:

1. Start with a Postman collection with tests: For now, let’s assume you already have a Postman collection with tests. Download the sample collection and environment by clicking the Run in Postman button if you want to follow along with this example.
2. Set up a GitHub repository: Travis CI is free for open source projects on GitHub, so in this example, we will keep our Postman tests in a public GitHub repo.
3. Set up Travis CI: Getting started with Travis CI is simple and will take a few minutes. Follow the Travis CI getting started guide for the complete walk through. Sign in to Travis CI with your GitHub account. Go to your profile page and enable Travis CI for the public GitHub repo we set up in the previous step.

Hooking up Postman to Travis CI

1. Export the Postman Collection as a JSON file and move the file to your project directory. If you’re using an environment like in this example, download the Postman environment as a JSON file and move the file to your project directory as well. In this example, I’ve moved both files into a directory called tests placed in the root of the project repository. Remember to add and commit these 2 files to your repo.
   
2. Create a new file called .travis.yml and move it to the root of your project repository. Remember to add and commit it to your repo. This file tells Travis CI the programming language for your project and how to build it. Any step of the build can be customized. These scripts will execute the next time you commit and push a change to your repo.
   In the .travis.yml file, add a command to install Newman in the CI environment, and then add a script telling Newman to run the Postman tests (which I’ve placed in the tests directory). Since Travis CI doesn’t know where Newman is located, let’s update the PATH. In this node.js example, the newman tool is located in my .bin directory which is located in my node_modules directory.

Now, my .travis.yml file looks like this for this node.js example:
https://gist.github.com/loopDelicious/c8c43557e0229e5c20341913a42cf528

Travis CI is now set up to run your Postman tests every time you trigger a build, for example, by pushing a commit to your repo.

Let’s try it out. The Travis CI build status page will show if the build passes or fails:

Travis CI is running our Newman command, but we see a failed exit code (1). Boo.

Stay calm. Let’s review the logs in Travis CI. Newman ran our tests, we see the first and second tests passed, but the last test Updated in the last day failed.

Let’s go back to our Postman collection and fix our Updated in the last day test.

Once we fix the mistake in our test, let’s save the changes, update the repo with the latest collection file, and then trigger a Travis CI build once again by committing and pushing the change.  

And it’s working! All our tests passed and the command exited with a successful exit code (0).

For extra credit

During development, it’s kind of a pain in the butt to manually update your repo with the updated collection file every time you make a change to your collection. Here’s 2 tips to help ease that pain.

1. For Postman Pro users: add the Postman Pro to GitHub integration so that changes to your Postman collection are automatically synced to your GitHub repo.
2. For all Postman users: update your collection or environment using the Postman API.