Performance Testing in the Cloud with Postman: What We Built

Load testing an API usually means one of two things. Either you spin up dedicated infrastructure, learn a new tool, and maintain scripts that drift from your actual API contract. Or you run a load generator from your laptop and hope your Wi-Fi holds up long enough to produce useful numbers.

We wanted a third option, and we built it. This post is from the engineering team behind cloud performance tests in Postman. We’ll walk through what shipped, what makes it different, and a few of the harder problems we had to solve under the hood. A deeper engineering write-up on the coordination layer is coming soon, so consider this the tour before the teardown.

The same Collection you already have

The starting point was a simple constraint: your load test should be the same Postman Collection you already use for functional testing. Not a translation of it. Not an exported script. The same Collection, with the same requests, the same authentication, the same environment variables, and the same test assertions.

That means if you have a Collection that hits POST /orders, validates the response schema, and checks that the status is 201, you can point cloud performance tests at it, dial up the virtual user count, and every pm.test block still runs on every iteration. The assertions fire under load. If the API starts returning 500s at 400 concurrent users, you see the failed tests, not just a throughput graph that dipped.

This alone changes the workflow. Functional tests and performance tests stop being two separate universes maintained by two separate teams. You write the Collection once. You run it against one user for a smoke test. You run it against two thousand users for a load test. Same artifact.

No infrastructure to set up

The other constraint was zero setup. You should not need to provision workers, configure a runner pool, or read a doc on how to distribute load across regions before your first test. You open the Collection Runner, switch to Performance, pick a virtual user count and duration, and click run.

Behind that button, we spin up a fleet of workers in our cloud, coordinate them, run your load profile, stream metrics back to your Postman client in real time, and tear everything down when the test ends. You do not see any of it. You see a chart.

Because the workers live in our infrastructure, you can push virtual user counts well beyond what a single laptop can produce. Your machine is not the bottleneck. Your home network is not the bottleneck. The load actually reaches the numbers you configured.

Real-time metrics while the test runs

We wanted the test to feel live. Not “wait ten minutes, then read a report.” You should watch average response time, error rate, and throughput change as the load ramps, and you should be able to spot the moment a specific endpoint starts to degrade.

So metrics stream continuously. As soon as a worker completes a request, the timing and outcome flow back to your client and update the charts. If you configured a ramp-up profile, you can literally watch the failure rate turn on at a specific concurrency level. That feedback loop is the whole point.

A few things we had to get right under the hood

Building this looked simple from the outside and turned out to have some genuinely interesting problems in the middle. A quick tour, without getting into the specific technology choices (that’s the follow-up post).

Synchronized starts across ephemeral workers. When you ask for 1,000 virtual users, we spin up a fleet of short-lived workers to produce that load. Getting them all to start hitting your API at the same instant is harder than it sounds — network delays and startup jitter mean “start now” arrives at different workers at different times. Our approach flips the problem: instead of trying to signal every worker at the same moment, we hand each worker a shared future timestamp and let each one wait locally against that shared clock. A distributed-timing problem becomes a local wait. Everyone starts together.

Faithful splitting of one load profile. A ramp from 0 to 1,000 users over five minutes has to look like a smooth ramp to your API, no matter how many workers are producing the load. If we naively divide the profile and round, small rounding errors compound and the fleet total drifts away from what you asked for. So the split anchors on the peak and works backward, and the fleet total matches the profile you configured. Not approximately. Exactly.

Metrics that survive a worker vanishing. Workers are ephemeral. Sometimes one dies mid-run. We stream metrics continuously to a columnar store as they are produced, so if a worker disappears, everything it already reported is safely persisted. Nothing is buffered on a worker only to be lost when the worker is.

Knowing when the test is really over. A test is bounded by its duration, but “duration” and “the last request came back” are not the same instant. If we cut the run off at the nominal end time, we miss the tail. So the watchdog that ends the run is armed off the true end-of-load moment, not the wall-clock duration you configured. You get the whole tail.

We had to be careful about the difference between “this worker finished” and “this worker died,” too. Marking a worker as finished only on acknowledgement, and refusing to cache a stale verdict, is the kind of detail that sounds pedantic until you see the alternative in production.

Try it now

If you have a Collection that hits an API you care about, you can run your first cloud performance test in about a minute:

1. Open your Collection in Postman.
2. Click Run and switch to the Performance tab.
3. Choose Cloud as the load source.
4. Set a virtual user count and duration. Start small (25 users, 1 minute) if you’re just kicking the tires.
5. Click Run, and watch the metrics stream in.

Your existing test scripts run against every iteration, so failures show up as failures, not just as latency spikes. Try a fixed load first, then try a ramp profile and see which endpoint hits its knee first. That’s usually the interesting one.

A deeper engineering post on the coordination layer, the ramp math, and how we keep the fleet honest is coming next. If any of the nuggets above made you curious about how they actually work, that’s the post you want.

Resources

• Configure and run performance tests in Postman — Official docs on setting up a performance test, choosing virtual user counts, and configuring load profiles.
• Simulate user traffic to test your API performance — How virtual users work in Postman and what metrics are captured during a run.
• Postman release notes for cloud performance tests — Release notes covering the availability of cloud-based performance runs.
• Writing test scripts in Postman — Reference for the pm.test assertions that continue to run under load.
• Postman Collection Runner overview — Background on the Runner, the same entry point used for cloud performance tests.