API testing is hard, but it doesn’t have to be. Postman makes it easy to write tests, make sure they’re working, set up the testing environment, and then eventually hook it all up to your build system. More than ever before, developers are using Postman tests in their Continuous Integration or Continuous Delivery pipeline.
Continuous Integration (CI)
CI is a practice that requires developers to integrate code into a shared repository several times a day. By committing early and often, the team avoids a ton of technical debt by allowing teams to detect problems early while conflicts are relatively easy to fix. Every check-in kicks off an automated build process which typically includes testing and (if your commit hasn’t broken anything) might include deployment too.
Postman has a number of tools to make API testing easier. Postman has a testing sandbox that lets you write and execute tests for your API, and Newman is Postman’s command line tool that sends API requests, receives the response, and then runs your tests against the response. We won’t dive into the details of setting up a collection or writing tests, but here’s a tutorial on writing tests if you’re just getting started.
In general, integrating your Postman tests with your favorite continuous integration service is the same process if you’re running on Jenkins, Travis CI, AppVeyor, or any other build system. You will set up your CI configuration to run a shell command upon kicking off your build. The command is a Newman script that runs your collection with the tests, returning a pass or fail exit code that’s logged in your CI system.
In this example, we’ll walk through how to integrate Postman with Travis CI, a continuous integration service used to build and test projects hosted on GitHub. Travis CI will run your tests every time you commit to your GitHub repo, submit a pull request, or some other specified configuration.
Before we get started:
- Start with a Postman collection with tests: For now, let’s assume you already have a Postman collection with tests. Download the sample collection and environment by clicking the Run in Postman button if you want to follow along with this example.
- Set up a GitHub repository: Travis CI is free for open source projects on GitHub, so in this example, we will keep our Postman tests in a public GitHub repo.
- Set up Travis CI: Getting started with Travis CI is simple and will take a few minutes. Follow the Travis CI getting started guide for the complete walk through. Sign in to Travis CI with your GitHub account. Go to your profile page and enable Travis CI for the public GitHub repo we set up in the previous step.
Hooking up Postman to Travis CI
- Export the Postman Collection as a JSON file and move the file to your project directory. If you’re using an environment like in this example, download the Postman environment as a JSON file and move the file to your project directory as well. In this example, I’ve moved both files into a directory called
testsplaced in the root of the project repository. Remember to add and commit these 2 files to your repo.
- Create a new file called
.travis.ymland move it to the root of your project repository. Remember to add and commit it to your repo. This file tells Travis CI the programming language for your project and how to build it. Any step of the build can be customized. These scripts will execute the next time you commit and push a change to your repo.
.travis.ymlfile, add a command to
installNewman in the CI environment, and then add a
scripttelling Newman to run the Postman tests (which I’ve placed in the
testsdirectory). Since Travis CI doesn’t know where Newman is located, let’s update the
PATH. In this node.js example, the
newmantool is located in my
.bindirectory which is located in my
.travis.yml file looks like this for this node.js example:
Travis CI is now set up to run your Postman tests every time you trigger a build, for example, by pushing a commit to your repo.
Let’s try it out. The Travis CI build status page will show if the build passes or fails:
Travis CI is running our Newman command, but we see a failed exit code (1). Boo.
Stay calm. Let’s review the logs in Travis CI. Newman ran our tests, we see the first and second tests passed, but the last test
Updated in the last day failed.
Let’s go back to our Postman collection and fix our
Updated in the last day test.
Once we fix the mistake in our test, let’s save the changes, update the repo with the latest collection file, and then trigger a Travis CI build once again by committing and pushing the change.
And it’s working! All our tests passed and the command exited with a successful exit code (0).
For extra credit
During development, it’s kind of a pain in the butt to manually update your repo with the updated collection file every time you make a change to your collection. Here’s 2 tips to help ease that pain.