How to Catch Breaking Changes Before They Happen

This is a guest post written by Allen Helton, software engineering manager at Tyler Technologies. 

“Alexa, turn on the kitchen lights.”

I sat there in silence for a few seconds, waiting for the darkness around me to go away.

“Alexa, turn on the kitchen lights.”

A brief pause, and then the response, “I’m sorry, I can’t find a device named kitchen lights.”

Well, Alexa was wrong. I had set up my kitchen lights to be voice-controlled three years ago, and I had been turning them on and off multiple times every single day since. I successfully used that command a thousand times.

I tried again. Same response.

I opened the Alexa app and looked at the list of connected devices. Sure enough, “kitchen lights” was not listed as a connected device anymore. What was going on?

The lights were controlled via my SmartThings hub. Alexa would typically talk to SmartThings, then SmartThings would turn my kitchen lights on and off. Turns out, SmartThings had made what is called a “breaking change” in their API, forcing me to re-authenticate with Alexa and add all of my devices again.

This was frustrating, as you might imagine. It was something I had no control over, and it broke a favorite integration I’d set up years ago. Needless to say, it left me feeling a little helpless and mad at SmartThings for making me go back and set everything up again.

This can happen to any of us. As web developers, we must be careful when we make updates to a live API. Consumers use those APIs for every use case under the sun, and if we make a breaking change, we can cause some serious frustration.

So let’s talk a bit about breaking changes, which are a serious problem in the API industry today but can easily be prevented through automation.

What is a breaking change?

Simply put, a breaking change is an update that fails to maintain backward compatibility. With API development, this can be caused by a variety of things:

• Adding a new required property
• Removing support for an existing property
• Renaming something
• Changing the shape of the data in either the request or response
• Removing/changing a possible response code

These are just a few ways to make breaking changes to your API, and there are plenty more. All this goes to say, when you’re maintaining an API in production, it’s important to be intentional about every change you make.

What isn’t a breaking change?

Looking at the list above, you might be asking yourself, “So what can I do once my API is live?” To be honest, your choices are so limited that the only thing you can do is add optional fields.

If you must make a breaking change, like adding a new required field or grouping data in a different way, you will need to version your API. You can version a specific endpoint, or you can add a new major version to your entire API.

But whatever you do, remember this—you must continue supporting the old version of your API! At least until a deprecation date where you give your consumers an appropriate amount of time to prepare.

Versioning

If you use the API feature in Postman, you already get semantic API versioning out of the box. Postman will track all versions and schemas of your API for you, tracking specific monitors, mocks, and documentation related to it as well.

Of course, you are responsible for the actual implementation of the API—making sure you keep separate versions deployed—but the tracking and public-facing pieces are handled for you.

The two most common ways to version your API are by:

• Updating the URL to include the version
• Passing in a version header

When you update the URL to include the version, you might go from something like this:

api.mysite.com/resources to api.mysite.com/v2/resources

If you version a specific endpoint instead of the entire API, it might be:

api.mysite.com/resources to api.mysite.com/resources/v2

If you want to pass in a version header, you must start including the header on all your endpoints. If the header is not passed in, default to the current version. If it is passed in, use that version.

Note: Passing in a version header is essentially a breaking change. If your consumers do nothing, they will automatically be upgraded to the new version.

Preventing breaking changes

Now that you are appropriately nervous, let’s discuss how to prevent breaking changes from sneaking their way into your API.

You can always hope that reviewers on pull requests will catch any breaking changes, but that is unreliable and likely to fail at some point. Humans make mistakes. Whether they’ve done it one time or a thousand times, something will be forgotten and slip through the cracks. So we need automation.

To help with this, I’ve created a Postman Template that will automatically check the current version of your API for breaking changes. The template is a collection intended to be run by Newman in your CI pipeline (you can also run it manually if you choose to do so).

The template uses the Postman API to load the current and last version of your API and compare the two. The comparison checks for all the things I listed at the beginning of this article: no new required fields, not removing optional fields, no changing data structure, no removing response codes, etc.

If any breaking changes are detected, test assertions fail, causing your pipeline to fail. This means your pipeline will not complete and you’ve successfully prevented a breaking change!

Setting up the pipeline

Let’s get started with adding the breaking change detector in your CI pipeline:

1. First, import the template into your workspace.
2. With the template imported as a collection, we can now go get the environment on GitHub. Simply download that file and import it as an environment in Postman.
3. Next, we need to create a Postman API key so we can dynamically load our spec via the breaking change detector collection.
4. If you already have a CI pipeline configured, great! If you don’t, refer to the docs for wherever your repository is hosted (GitHub Actions, Bitbucket Pipelines, Jenkins, etc.).
5. To add the detector to the pipeline, we need to create a new step that will run newman. Here’s an example step using Bitbucket pipelines:

steps: 
 - step: detect-breaking-changes
    name: Detecting Breaking Changes 
    script: 
     - npm install -g newman 
     - newman run https://api.getpostman.com/collections/$BREAKING_CHANGE_COLLECTION_ID?apikey=$POSTMAN_API_KEY --environment 
https://api.getpostman.com/environments/$BREAKING_CHANGE_ENV_ID?apikey=$POSTMAN_API_KEY --env-var "env-apiKey=$POSTMAN_API_KEY" --env-var 
"env-workspaceId=$POSTMAN_WORKSPACE_ID"

In the step above, we are installing Newman in the build container, then running it against our breaking change detector collection with our environment. We are also overriding environment variables env-apiKey and env-workspaceId to contain our API key and workspace.

Variables

The command above makes use of pipeline variables. We use variables so we can build reusable pipelines across multiple repositories, and so we can prevent storing sensitive information like API keys in source code.

The variables we use to run Newman are:

• $BREAKING_CHANGE_COLLECTION_ID: The ID of the collection we imported from the template. This can be found in the URL if you navigate to the collection via the Postman web app.
• $BREAKING_CHANGE_ENV_ID: The ID of the environment we imported from GitHub. This can also be found in the URL if you navigate to the environment via the Postman web app.
• $POSTMAN_API_KEY: The key we generated earlier that allows us to use the Postman API.
• $POSTMAN_WORKSPACE_ID: The ID of the workspace that contains the API/spec we are testing. This can be found in the URL if you navigate to the workspace via the Postman web app.

With that, you are configured and ready to go! Now, whenever your pipeline runs, the breaking change detector will compare the current version of the API spec with the previous and alert you if something is broken.

Manually monitoring your API

If you don’t want to go through the hassle of setting up a CI pipeline, or think it might be a bit too intrusive, you always have the option to run it manually.

Simply configure the environment with the values from above, and you’re free to run the collection through the collection runner whenever you like. Tests will be generated dynamically as you build the spec, so no need to update anything in the collection itself.

Be a watcher

You also have the option to become a “watcher” of an API. If your team is updating the API spec through the Postman UI, you can easily watch for changes and be notified whenever a change is made.

When a change is made (not by you), you will get an in-app notification saying so. You will also get an email with a little more info: The email will contain who made the change, what version was edited, and what changes were made. This gives you the ability to do your own audit and catch potential problems before they even make it to the build.

You’re ready to try it out

From here, the next thing to do is to try it out. Follow the instructions above and either bring it into your pipeline or start running it manually.

Preventing breaking changes is important, and it should absolutely be a part of your governance model when it comes to APIs. You don’t want to get into production and be the one to break the app for everyone by accident.

We’re in the early stages of getting strong API governance built into tools like Postman. So if you have improvements or ideas on the processes, please share them. Drop me a note on Twitter, leave a comment below, or write on the Postman community forum. We’re here to make APIs better, stronger, and easier. Good luck, and have fun making great APIs without breaking changes!

Do you have your own experience or tips to share with the Postman community? We want to hear from you. Learn more about our guest blogger program and submit your idea here.