API contract testing: 4 things to validate to meet (and exceed) expectations
If you’ve ever received a report card that said “Meets Expectations,” your parents probably weren’t impressed. You might not be excited to see it on a performance review, either. But when it comes to API contract testing, meeting expectations might be exactly what you want.
At this year’s API Specifications Conference (ASC), Postman Senior Developer Advocate Ian Douglas shared some valuable insights about taking your API contract validations to the next level in Postman. Read on for some of the highlights, and be sure to check out the full video to catch all of the dad jokes (and some helpful demos).
Defining the API contract and building confidence
Related: Use the Contract Testing Template
Many of us think about an API contract as an agreement between the API producer and consumer, where the producer promises that if you call the endpoint a certain way, you’ll receive a certain response. Some developers see the API contract as the structure of the response, and others think the API contract describes how the API should be built. In reality, the API contract includes all of these things.
We test to build confidence between teams within our company, as well as between our company and our end users. Ian outlines four areas to focus on when developing API tests:
- API definition validation: ensure the examples you provide match what your API specification tells users to expect.
- Industry standards validation: ensure your API definition matches industry standards, like OpenAPI Specification guidelines.
- End-user schema validation: ensure that the user experience matches your documentation and API specification.
- Governance validation: ensure that the user experience matches your business needs.
1. Validating your API definition and design
API definition and design validation involve testing that your example responses match what’s in the API specification. Typically, the teams that implement the business rules in the specification file and the teams that document and test the API are the ones that pay the most attention to this type of testing.
In his 25 years of development experience, Ian has observed that it’s very common for these teams to be out of sync. While one team is working on writing the specification, the documentation and testing teams are trying to figure out how it’s supposed to work. Documentation is very important to end users, so the documentation team’s perspective is helpful when validating the API design. The development team needs to have a process for collecting and implementing feedback, and they also need tools that support merging ideas.
Ian demonstrated importing an OpenAPI specification into Postman that includes example responses. Postman’s linter compares your example to what the OpenAPI specification says it should look like. If there are differences, you can decide whether the specification needs to be changed, or if that’s not possible, the documentation needs to be changed. This process gives you better alignment between the teams that are developing and documenting the API, which leads to a better user experience because your API examples match what your API specification says it should do.
2. Validating against industry standards
After your API definition is validated and your teams are synchronized, the next thing you need to do is validate that your API definition conforms to industry standards and best practices. Does it matter if your API follows these standards? As Ian pointed out, by aligning to these standards, you build trust by making your API seem more familiar to your end users because it looks and feels like other APIs they’ve used. If you define an API that’s supposed to be RESTful, then it should feel like other RESTful APIs. If you define a GraphQL API, it should feel like other GraphQL APIs. This familiarity speeds up the time to first call. When your API feels intuitive, developers can implement it much faster.
Ian showed a demo of the Contract Tests simulator in Postman. This will read your OpenAPI specification file and then scrape the OpenAPI spec to make sure it conforms. This gives your users confidence that they can use your specification and that their tooling will be compatible with it.
3. Validating our end-user experience
If you remember the early days of Postman, it all started with API response testing. Since then, we’ve continued to improve the testing experience. When a user begins working with your API in Postman, they can import a specification file, read through your documentation, and write their own JavaScript tests. But, as Ian suggested, wouldn’t it be a better user experience if your organization wrote those tests for them? If they’re going to import your collection anyway, why not include pre-written tests? This gives your users an extra layer of confidence that the API is working as you say it will.
Check out the video below to watch Ian’s demo, where he adds a JSON Schema layout to a test that checks if the response you get in Postman matches the schema using the Ajv library.
4. Validating for governance
The point of standardizing APIs within your company is to build a common goal, a common interface, and a common voice. In Postman v10, we introduced governance rules to help you accomplish this within your organization. Using governance rules allows your team to add rules that help shape the design of your API. The teams developing APIs can check against those rules to make sure they’re building the API properly, and those on the business side can see that the development teams are following the standards they’ve set. Governance rules build confidence on the business side.
Ian demonstrated how to create governance rules using the API Governance interface in Postman v10. You can add rules recommended by Postman, but you can also create your own Spectral rules. You can also set up API security rules the same way. Ian also provides a demo of what the interface looks like to the user when these linters are enabled.
Building confidence and trust
At the end of all this testing, business leaders are more confident that the API does what they need it to do, has been designed well, and follows industry best practices. Your teams are more confident about what they’re building because they can run those tests. Your users have confidence in your ability to produce high-quality APIs, and client-side testing shows that your team is thorough and diligent. At the end of the day, putting these different layers of validation together builds confidence that you’re delivering a high-quality API.
What do you think about this topic? Tell us in a comment below.