State of the API report data is now available via API
The world’s largest and most comprehensive survey of API usage just became even more useful. Today, we’re making data from the 2023 State of the API report available on Postman’s Public API Network. Users can query a public API to ask their own questions about survey responses from more than 40,000 API professionals.
For example, what are the top API development priorities in financial services? How does that compare to the retail sector? What is the preferred architecture for developers with ten years of experience? Who’s more bullish on API investment this year—CEOs, CTOs, or frontend developers?
We encourage you to experiment with the API and devise your own questions, stories, or visualizations. An important benefit of the API is that it allows you to integrate existing or new findings into your own websites, applications, or publications. Note that we have imposed some limitations on the data to protect the privacy of respondents.
Building the State of the API public workspace and API took a concerted effort from the Postman team. Here’s a little more about how we did it, and what data is available via the API.
The data
We collected the data with SurveyMonkey and exported it as an SPSS file (a proprietary format commonly used by statisticians), holding 407 variables and 40,261 records. Our data analysts then used the SPSS to produce the tables underlying the report. To interact with this data through an API, we converted it into a CSV file and loaded it into a PostgreSQL database. You can find more about the survey process here.
The metadata
To enable the API’s backend to dynamically query the data and deliver documentation to users alongside the results, we captured various metadata elements in a spreadsheet. We then loaded them alongside the data in the SQL database. This metadata included, for example, information about the variables, questions, their associations, and the classifications (coded responses). This step is essential to any data-centric project.
The backend
We implemented the API backend as an AWS Lambda function. In a nutshell, it took the query parameters (the topics of interest) and generated an SQL query to compute the aggregated data. The resulting table was combined with the relevant metadata to produce a simple JSON document returned by the API.
This piece was tricky, as it had to account for various types of questions in the survey (i.e., binary single/multiple choice, percentages single/multiple choice). The data was further constrained by the need to protect the respondents’ privacy, restricting certain topic combinations. For these reasons, we’ve limited the API to cross-tabulating two topics (two-dimensional tables) at this early stage.
The infrastructure
We’re hosting the API in a virtual private cloud (VPC) on our AWS infrastructure, and we’ve leveraged various services including API Gateway, Lambda, RDS and RDS Proxy, and CloudFront. We defined all of our infrastructure as code using the Cloud Development Kit library, enabling us to rebuild or update the entire stack on the fly. This is an approach we highly recommend.
The workspace
We naturally created a Postman public workspace to make this data available to the world. This workspace contains collections and requests that mirror the report’s underlying tables to help users get started with more complex analyses. We used the Postman Visualizer to provide user-friendly rendering of the JSON responses. Postbot was a big helper here, too, in seeding the basic templates.
What’s next?
Overall, we had a great experience setting up our API and its infrastructure. We continue to explore other solutions and have a few ideas on how we might improve this setup.
This State of the API workspace is experimental, and it’s the first time we’ve shared the underlying survey data. In the future, we may include the ability to create a time series using data from past years’ surveys.
We would love your feedback and suggestions on how we can make the workspace better. In the meantime, try out the API, and let us know what you think in the comments below!
This post is co-authored by Pascal Heus, Postman Open Technologies data lead, and Michael Claus, software engineer II.
this is awesome. now we can say – there is an API for that. BTW, I could not use it out-of-the-box because I am not sure what is the value for {{endpoint2023Tabulate}}. The blog, embedded doc, or collection variables do not provide any hint.