How to craft a great, measurable developer experience for your APIs
I often hear the phrase “API as a Product” when talking to companies who have built successful API programs. A play on the “Software/Platform/Infrastructure as a Service” monikers, they’re basically saying, “We treat our APIs just like we treat other products that we offer (mobile app, website, or any physical device).” For any product, it’s important to understand who you’re building it for, what they are hoping to accomplish, and how they are doing it right now.
Subsequently, once you have built a product, it’s paramount that you look at the following things to ensure its continued success:
-
How you make your product discoverable to potential users and customers
-
How you onboard users onto your product
-
How you analyze usage patterns, recognize feedback, and iterate to continuously improve your API
Great products are about great user experiences. Developers, the primary users of APIs, benefit hugely from a thoughtful UX. A positive developer experience (DX) that makes an API more approachable, usable, valuable, and sticky can transform it from being just another functional tool into an empowering asset. When a developer is using an API, they’re essentially delegating a job to that service. Sometimes, they’re paying for that job to be done. This brings home the need for a great DX because, like any other product, it is one of the ways in which we choose between similar products.
In this blog post, I’m going to talk about crafting a great developer experience for your APIs. I’ll share some stories as I introduce four useful metrics (because where’s the fun without data!) that I have seen companies adopt in their successful API journeys.
Metric #1: Bounce Rate
Back in 2017, I went to an API conference where I met a Google product manager who introduced me to a metric that he was obsessing about for his API: Bounce Rate. I was familiar with this metric in the context of website analytics, but what does it mean for an API?
He (philosophically) defined Bounce Rate as a developer coming to your API portal, investing the time to generate authentication credentials (filling out forms, verifying emails, etc.) but then failing to make a single successful call against your API. Essentially, your API’s Bounce Rate is the number of generated API tokens that are not used to make even a single successful API call against a dimension of time.
After he started measuring this metric for a rather popular API whose success he was overseeing, he was surprised to see how high this number was. From then on, improving the API’s Bounce Rate had become an essential part of his team’s OKRs. Since I heard this story, I have asked this question of many API owners across companies who have been amazed to find out that their API’s Bounce Rate was sometimes as high as 40-50%.
(I’d be remiss if I didn’t point out the “successful” qualifier here that the Google PM had adopted measured through a 2xx response code. The number is often worse if you remove that qualifier, and it’s an important metric to analyze independently.)
Imagine this: you have painstakingly, carefully built this API, somehow convinced users to come to your API portal (topic for another post), convinced them to try your solution, and convinced them to go through the arduous steps of generating the credentials—but they ultimately failed to make a single successful API call. Not even one measly “Hello, world!” call. Why?
Here are three prominent reasons that I have observed both firsthand and in research:
1. Users couldn’t figure out how to use the authentication credentials. (Auth is hard; cURL what?)
2. Users couldn’t figure out where to start. (The “reference manual” documentation is overwhelming!)
3. Users were not comfortable using an API directly. (Give me a UI!)
Let’s go back to our “API as a Product” moniker for #1 and #2.
Good product teams often disambiguate between designing for first-time user experience (FTUX) and designing for continuous user experience. FTUX is where the user’s first impressions are formed, and the stage for a positive long-term relationship is set. Continuous user experience assumes some familiarity from users who are past the initial learning curve, and it aims to ensure the product remains relevant, useful, and enjoyable for users as their familiarity and expectations grow. In other words, this is the difference between optimizing for someone experiencing or learning how to use your product for the first time vs. helping someone who already knows how to use it.
Yet, API documentation on developer portals is often more like receiving a “reference manual,” where the onus is on you to figure out how to take the first step. They’re anchored more in being “correct and comprehensive” than helping you form the right first impression that will lead to a beneficial long-term relationship. I struggle to remember the last time I truly enjoyed, over a longer-term horizon, a physical/mobile/web product that didn’t guide me through a phenomenal first-time experience.
Enter Postman. Postman was introduced to the world 12 years ago, in this now-infamous Stack Overflow answer from our co-founder and CEO, Abhinav Asthana. Today, more than 30 million developers have used Postman to make API requests across as broad a variety of APIs as you can imagine. That’s more than 30 million developers in more than 500,000 organizations all around the world—all who use the Postman app and understand how it can be used to successfully call your API.
At Postman, we understand that authentication is hard. We’ve had developers tell us that for the last 12 years. In addition to simplifying authentication for API consumers within Postman, we also launched easier API authentication for API publishers in 2023. Any API publisher can now register their API within the Postman Public API Network and make it “click-through easy” for any Postman user in the world who is trying to call their API, joining the likes of OpenAI, Notion, Twilio in the process.
Let’s further address #2, “Users couldn’t figure out where to start.”
I’d like to introduce a key artifact that developers create with Postman—the Postman Collection. There are more than 100 million active Postman Collections in the world. Postman Collections are executable documentation for APIs and API workflows, designed for sharing. With a collection, you can design your documentation for execution, with the specific intent of getting your prospective user to make their API calls successfully. You can bundle in authentication helpers, combine reference documentation with Getting Started guides/recipes, or just create a simple, actionable way to help the user start using your API.
The WhatsApp team over at Meta has done a fantastic job of this, using Postman Collections to encapsulate the reference documentation with Getting Started guides for their users. Each part of the onboarding experience was thoughtfully designed and optimized for users to actually use the API.
Postman creates an amazing developer experience, and there's great demand for it.—Marco Wirasinghe, Head of Business Messaging API Platforms (Whatsapp, Instagram, Messenger), Meta
The Run in Postman button is a highly effective way to make your API documentation more executable. Embeddable in any HTML/Markdown page, the Run in Postman button allows your users to “fork” your Postman Collections, simultaneously helping them get started with your API and also giving you a persistent communication channel with them.
Square, for reference, chose to embed the Run in Postman button in their API documentation to help developers easily get started with their API:
Postman makes it especially easy for users to get started using Square APIs. Being able to try out an API as quickly as possible is important when learning about its features.—Tristan Sokol, Developer Evangelist, Square
Now, onto #3: “Users were not comfortable using an API directly.”
As APIs have exploded over the years, so have the job roles that work with APIs: product managers, executives, technical writers, business analysts, support teams, and more. In our more recent annual survey, Postman’s 2023 State of the API Report, as many as 53% of the participants were non-developers. During the early days of the COVID-19 pandemic, many APIs emerged to address the need for quick, easy access to real-time critical data for healthcare professionals, researchers, and government experts. And yes, not everyone is comfortable with command-line tooling, the intricacies of authentication, or JSON/XML data structures.
Cisco gets this. Cisco has some of the most popular collections in the Postman Public API Network. And, some of the primary consumers of Cisco’s APIs are network engineers with expertise in hardware and network protocols rather than in software development. As Cisco commits to an API-first motion to drive innovation, making APIs more accessible to these consumers becomes essential. Cisco uses the powerful Postman Visualizer to present API response data as charts, tables, and other visual elements to simplify the successful adoption of their API.
Here’s Steve Greenberg, technical solutions architect at Cisco, talking about how they do it:
Metric #2: Time to First Call (TTFC)
A complementary metric to Bounce Rate is the Time to First Call (TTFC). I see companies usually measuring this as the time taken between a new user generating their auth credentials to making their first successful call to your API.
All the things we’ve discussed so far—30 million developers’ familiarity with Postman, Postman Collections, simplified authentication, the Run in Postman button, and Postman Visualizer—hugely help reduce the time it takes to make that first successful API call.
Using Postman Collections, Paypal was able to reduce the Time to First Call from hours to 1 minute. As a result, Paypal’s Postman Collection, since launch, has been “forked” and “watched” collectively by over 100,000 Postman users.
PayPal's Public Postman Collection is in the top 10 most requested APIs in terms of popularity and quality. We're committed to investing in this next generation of technology.—Dan Schulman, President and CEO, PayPal
As with any metric, it’s important to assess it along different statistical measures (average, median, 95th percentile, 99th percentile, etc.). The relative importance among them will depend on the uniqueness of your API and business. Similarly, based on the use cases, the dimensions along which this metric is typically segmented include API version, user persona, geographical location, industry or sector, and platform or environment.
Metric #3: Time to Value (TTV)
Great! Your users are able to make a successful API call. What next?
Well, API calls, as you can gather, are rarely made in isolation. Any meaningful workflow or user journey has multiple API endpoints that need to be called together, sometimes from the same API, often times combining multiple different APIs.
For your users to see value from your API, it’s important for you to understand what job your users are trying to do and what their motivations and needs are. In turn, your API’s developer experience needs to showcase how your API fits in their workflow so that they see the value of your API faster.
Unlike the time to first successful call, which measures the initial technical interaction, Time to Value (TTV) is the duration it takes for a user to realize the tangible worth of your API in their work. In the realm of APIs, the Time to Value is critical because it directly impacts user satisfaction and retention; the quicker users can see the value, the more likely they are to continue using your API.
Postman Collections are pretty great for that! You can use Postman Collections to showcase your API capabilities with user-friendly “demos” that make your APIs accessible and instantly executable to all your consumers. Using the Postman Visualizer can add a tangible, visual touch to this value realization.
These demos can be “self-serve” for your API consumers or they can be driven by client-facing teams to easily showcase API functions and use cases to potential consumers.
Canopy, the loan management and servicing platform, uses Postman Collections to significantly bring this Time to Value down with their partners:
Postman is a great platform that helps people interact with APIs in a way that makes sense. It has an intuitive approach to organizing API interactions.—Anurag Angara, Head of Product, Canopy
We often see pre-sales engineering teams using Postman Collections to demo APIs to customers, showcasing value and shortening sales cycles as a result of the higher customer engagement.
Time to Value can be measured as the successful completion of this guided journey—the demo Postman Collection—that you curate for your users.
It’s a good tactic to cohort the users who have gone through these onboarding steps against those who haven’t, and measure your critical business metrics like retention, expansion, advocacy, and others. With our customers, I regularly hear about the stark significant difference in these metrics between the users who have been “value-onboarded” vs. those who haven’t. And, in turn, they double down on value-onboarding more users to shorten this critical “Time to Value.”
Metric #4: Time to Wow (TTW)
An adjacent but more psychological, emotional metric to Time to Value is what API teams sometimes call Time to Wow (TTW). Time to Wow focuses on the moment users experience a significant positive revelation, that “aha” moment that surpasses their expectations and solidifies their appreciation and engagement with your API.
Time to Wow is the duration between when a user first interacts with your API to the moment they experience a profound realization of its potential or benefit that exceeds their initial expectations. This moment is crucial because it transforms user perception from seeing your API as merely a tool to viewing it as an innovative solution that can significantly impact their work or project.
To craft this “aha” moment with your API, it’s important to first identify potential wow factors that can elicit this response. It could be an unexpectedly simple integration process, the speed of data processing, or the immediate applicability of the API in solving complex problems. Postman Collections are a powerful ally in this process.
Time to Wow is an extension of the concept of Time to Value. While delivering practical value is essential, creating memorable, impactful experiences can turn casual users into long-term advocates. It can be a critical unlock if you’re able to deliver moments of delight and surprise that make users think, “Wow, this is amazing!”
Turn developers into champions of your APIs
Crafting a great developer experience is a journey, and it’s driven by a commitment to understanding and meeting developer needs at every step. Thinking about your APIs as “products” necessitates thoughtful design, a robust understanding of use cases, and ongoing improvements.
By continuously assessing against these four metrics, you can not only enhance the usability and appeal of your APIs but also foster a community of engaged, satisfied developers who are eager to explore, use, and champion your APIs.
What do you think about this topic? Tell us in a comment below.