API-first observability for the API era

Jean Yang

March 19, 2024

It may seem like no matter how much work your team puts into monitoring, it’s not enough. New functionalities and new incidents keep slipping past your alerts and dashboards!

It’s not your fault. Reliability is getting harder. The rise of APIs means it’s easier than ever before to build quickly. The flip side: it’s harder than ever to get ahead of production incidents. Today, many developers are drowning in a sea of logs and errors.

In this blog post, let’s examine why performance monitoring isn’t serving all needs anymore, what it looks like to keep up with the pace of software development, and the API-first observability solution we’ve built at Postman.

Undermonitored and overwhelmed

Today, teams are often unable to quickly find and fix issues for the following reasons:

• Manual upkeep leads to undermonitoring: A decade ago, we saw the rise of customizable, easy-to-use application performance monitoring that changed the game. The catch: monitoring relies on manual upkeep of code, libraries, or gateways. With the rise of APIs, software is growing more quickly than ever before, making manual upkeep a bottleneck.

• We’re drowning in logs: Every developer knows the crucial role of logs in helping root cause and fix issues. But developers also know that the more logs you have, the harder it is to narrow down which logs to look at. As software systems get larger and more complex, developers are looking at managing millions of logs or thousands of bug alerts. Logs are only as good as the effort teams put into organizing them, creating dashboards, and creating alerts.
• Frameworks won’t save us: Keeping up with production doesn’t have to be manual if we can automate it at the language or framework level. But in reality, people have only built tools for specific frameworks, and the increasing heterogeneity of tech stacks (thanks to service-oriented architectures and legacy subsystems) is making it increasingly impossible to consolidate on specific frameworks.

The API era needs a new approach to understanding production

If you’re optimizing performance or diving deep into debugging an issue, it’s always helpful to get complete logs or stack traces. But requiring that level of detailed monitoring across an entire system locks developers into unnecessarily confined ways of building. In order to set developer teams free, we need a solution for the API era that is:

• Automatic and comprehensive: In order for the developer not to have to keep logs, dashboards, and alerts up-to-date with every new change to the app, there needs to be a basic level of production visibility that is both automatic and covers the entire system.
• At the API level: If today’s logs and error alerts are too overwhelming, we’re going to need to move up the stack. The best candidate: APIs. Not only are APIs a functional boundary for the software, but they often demarcate an organizational boundary, the hand-off point between one team and another. Monitoring at the API level means identifying the issues that are most likely to impact others and matching the monitoring to the boundaries of responsibility. Getting comprehensive metrics across API endpoints allows teams to get ahead of alerts and quickly determine who may be responsible for investigations and fixes.
• Able to easily work across languages and frameworks: Modern software exists as a collage across tech stacks and partial migrations, where only some of the software is paged in by the current members of the organization. In order for a monitoring approach to be comprehensive, it needs to easily work across programming languages and frameworks, without requiring changes to the underlying system.

Setting the developer free with Postman Insights

At Postman, we have been building a solution to support production reliability needs in an API world. Here’s what we’ve built:

• Automatic, comprehensive agent-based endpoint discovery: The Postman Insights Agent passively watches API traffic across your entire system in order to provide automatic, comprehensive coverage across API endpoints. Our agent is minimally invasive, using technology based on network monitoring so you can avoid having to install SDKs or use proxies that add invasive performance or security overheads. One of the best parts: our agent only sends request/response metadata to the Postman cloud, meaning your data is never at risk.
• Endpoint-centric metrics and error detection within minutes: Postman Insights analyzes the API traffic that the agent sees in order to automatically infer endpoints and report on metrics and errors. This means that within 15 minutes of installing, users can automatically explore metrics and errors across all endpoints, easily identifying endpoints causing active errors without any instrumentation.
• Integration at the infrastructure level to work across languages and frameworks: The Postman Insights Agent installs at the infrastructure layer rather than the language or framework level, making it possible for the same agent to work across most of your tech stack. We have been hard at work smoothing installation, configuration, and maintenance for container platforms, starting with ECS. Up soon: Kubernetes and EC2.

While API monitoring is about alerting on API issues, API observability additionally helps developers get a better understanding of their system in order to get ahead of issues. Through capabilities like API discovery and the service graph, API observability solutions allow developers to not only react to incidents but also proactively address issues before they affect customers.

What’s next?

We’ve just released the latest version of our alpha to address the above. Currently, Postman Insights is integrated with Postman Collections and workspaces to help our users take advantage of Postman’s existing capabilities for managing and collaborating on APIs. We will continue to integrate with other capabilities on the platform, like the Public API Network and Private API Network, to deliver better and better API observability user experiences.

Sign up for early access to Postman Insights here.