Ridiculously Easy API Testing: Introducing Jetpacks for Postman
Fred Brooks in his seminal book “The Mythical Man Month” gave the following ideal distribution of time spent on a software project:
- 1/3rd to planning
- 1/6th to coding
- 1/4th to component testing and early system testing
- 1/4th to system testing with all components in hand.
Even if the exact break up between tasks varies in different environments, it’s evident that testing software requires a great deal of more time than actually writing software.
Integration between different parts of a system is a big challenge. Still with limited time and a limited budget, the first thing that’s cut off from the development schedule is rigorous testing. This has massive hidden costs. If the current state of security flaws and breakdowns of apps and portals is any indication, even experienced teams face difficulties. And for startups and small development teams, testing becomes a luxury and often the first casualty.
Compared to the amount of time we spend in honing our development tools, we spend far lesser on testing tools. Setting up a testing environment can be time consuming. And sometimes really boring. There is a lot of boilerplate code required to start testing even simple things. Writing tons of code just to initialize stuff puts off beginners and experts alike and proper API testing is limited to a few privileged projects.
To solve these problems, Postman now features a brand new update called Jetpacks. The Jetpacks upgrade allows you to improve your workflow as well as provides a kick-ass environment for API testing:
Start adding tests to your requests with a single click: There is no set up required. Everything that you would need to test your API response is included. Postman includes several libraries to make sure that you don’t have to write extra code. Specifying tests can be as simple as a single line of code. Some of the most common scenarios are listed in the Snippets sidebar so that you don’t have to read the documentation. Though if you want to, it’s available here.
No limits on test or request iterations: In the Collection Runner window, you can choose a collection (a set of requests), an environment and then set the number of times you want to run a collection. There is no limit to the frequency and the iteration count.
Chain requests together: Environment and global variables can be set inside the JS test script. You can extract data from one request and use it inside another request. This allows you to chain multiple requests together.
Mock data for tests: Postman allows you to import a CSV or a JSON file while running a request. Data variables follow the same format as the environment and global variables. You can test all possible scenarios for your API with absolutely no change or repetition for your existing requests.
Make your collections self aware: If you are distributing your API through a collection, you can add checks for your API consumers to ensure that everything works properly when she tries out your API. Instead of a vague “it doesn’t work” email, your consumers can tell you exactly what’s not working.
Check out the documentation for more details on what you can do with Jetpacks.
The Jetpacks upgrade is now available in the packaged app version of Postman. To activate the upgrade, click “Collection runner” or Know more in the “Tests” tab and then press the buy button. The upgrade is available for $9.99. All future upgrades for features as part of Jetpacks would be free!
If you have donated to the Postman project you have the option to avail this upgrade for free. Shoot me an email at abhinav@getpostman.com for activation instructions.
There are several other exciting updates in the pipeline. Support for Continuous Integration and build systems is coming soon.
As Brooks noted in his “No Silver Bullet” essay, there might not be a magic solution for dealing with the complexity of the software development process. However, I am sure using Postman to write tests will make your development life easier, lead to increased productivity for your team, and result in robust APIs while reducing development costs as well as the ones which you incur when your API stops working.
If you have any suggestions, drop me a line at abhinav@getpostman.com or post an issue at Github. You can also reach us through Twitter @getpostman.
Great work Abhinav!
Thanks John! Let me know if you take it for a spin. Would love to know what you think. 🙂
This is an excellent feature and well worth the 10 bucks. In some of the examples, I see that you have used console.log. Where do I view the console in Postman? Is there any alternative way where I can “print” something and view it? I’m using tv4 to validate JSON and I would like to view the errors when a test fails.
Hi Hari! I thought I replied to this but for some reason it’s not showing up here. You can open console.log using the steps given in this article: https://blog.postman.com/index.php/2013/09/18/legacy-app-update-cookies-and-devtools-for-v0-9-x/
Thanks Abhinav. This pointed me in the right direction, although to get to it I still had to run some circles – enable debugging in chrome://flags and then go to chrome://inspect -> Apps and select Postman (requester.html). Thanks again for the help and this great product as well!
Oh great. I have noted down the steps. I will publish a blog post on this along with other tips. Thanks!
This helped a lot thanks everyone.
Note to help someone else. I followed your instructions and searched for “packaged” on the flags page. There is a typo on the debugging option and it says “packed”. Search for “debugging” instead.
This helped a lot! Thanks @hariharan:disqus and everyone.
This worked like a charm! Thanks @hariharan:disqus!
These test features are excellent! I had a few questions – like hariharan, I wanted to know how to access console.log output. Also, we can set global variables in the Javascript test code – but is there a way to *get* previously-set values in the Javascript tests? I couldn’t figure that out. Plus – documentation on the entire “postman” Javascript class? Anyway, nice job!
Hi Steve! Replied to your email. Updated documentation is coming soon. Right now only two functions available in the “postman” object but a lot more is coming soon. Thanks! Glad you love the upgrade.
We love postman – Jetpack is a Great feature – anyway to introduce a try-before-you-buy option for the upgrade? maybe a single test for free or something?
It’s not that the 10$ are expensive but the fact that we would have to pay before we even get things to work is just not how we expect things to go usually. Happy to buy multiple seats at my company for this feature. We’ve been looking for a humanly manageable way to test our APIs for so long …
Great to hear that Dali! Yes. A trial version is in the works and should be out soon. We have a separate beta channel which will be active for a couple of weeks. I can send you an invite for that so that you can try out the entire feature set before you buy. Can you mail me at abhinav@getpostman.com?
I’m going to purchase the jetpack option, but would like to know if I will have problems sharing collections with team members who don’t (yet) have the upgrade. Love your work so far and am happy to give you $10.
Thanks! Feels great to hear that. Collections can be shared as before but if your team members need to run tests, they would need the upgrade too.
Problem with Test Runner. (Or possibly I’m not understanding something)
When running a POST statement through Test Runner, I’m getting a 500 Internal Server Error. Sending that same request individually works. I suspect the form data isn’t being submitted via Test Runner, but I can’t figure out how to see the request that’s run by Test Runner.
I initially created a new collection with only the single POST request. It fails. I added a GET request to that collection and re-ran. GET works, but POST fails.
Apologies if I’m using the blog comments as a user forum. Let me know if there is a more appropriate place for these comments.
Just confirmed this and pushed a bug fix to the Web Store. You should get the update in a few mins. Thanks for pointing it out! I think it occurred due to an update that was pushed a few hours back. You can post issues on the Github tracker too: https://github.com/postmanlabs/postman-app-support/issues. Easier to track bugs for future releases.
Fantastic feature set.. looking forward to even more good stuff coming. Couple of *potential* issues. First one.. postman.setEnvironmentVariable(key,value)… but the environment NAME is not provided? I would think I can specify the name of the environment (localhost, dev, production, etc) PLUS the variable name and value. Unless I am missing how to do this, I would suggest the method have three parameters.. signature: (environmentName, variableName, value)
Also, something else I am not sure of.. is the only way to actually pull values from a response is to run it in the Collection Runner and have to run entire collection? I can edit the test for each request and save it with it, that is exactly what I was hoping for. I am just not sure if there is a way to run the test for one request so that I can make sure it works without having to run an entire collection every time? I suspect that the way it is now, this is why the postman.setEnvironmentVariable() only has key/value.. because you run these in the collection runner where you set the environment and collection. That would make sense.. but it would be substantially more useful and helpful to be able to run individual requests one-off and make sure they work.. and in some cases, sometimes I will run it one-off anyway, and not as part of a collection of requests.
Keep up the awesome work!
Hi Kevin,
Agree with the first feature request. The “postman” object is going to have a lot of stuff in the future. Planning it out.
And yes, you can pull values from a response and set it to an environment variable without using the collection runner. Just make sure to have an active environment while you fire requests from Postman. Tests inside Postman are run every time when you hit send.
Thanks!
FYI, there’s a “tests” tab on the response area that shows the tests result when you run the calls one at a time, i.e. not via the collection runner.
Very nice. Can I automate running these tests? I would like to possibly wire this into my automated build process if possible.
Working on the feature Mike. Should be available soon!
Awesome. Where’s the best place to keep up-to-date?
I would suggest following the blog as well as the Twitter account: http://www.twitter.com/postmanclient
I am sure this is already on the radar, but when running a collection of requests, it would be immensely helpful to provide some sort of visual into each request, response, variables set for each request (upon response JS code executing), etc. As well, I’d guess that if one of the requests fails, the rest stop executing per the chance that the response of the failed request set some variable that is used in subsequent requests. At any rate, more visual detail would be really nice, but as I said I am sure this is something you already have plans to add at some point.
Wow
very nice, i have a question, i also want to do smock testing web pages, so, in this case, the response content type is html, is there any way i can parse html response to do test verify?
Hi Jason, Yes. You can parse HTML too and write tests in Javascript.
Hi
Just a couple of quick questions
a.) Do you have a eval copy of the JetPack? maybe just 7 days is enough..
b.) Can the app do a condition branch testing? i.e
Call REST API “A”, if return consists of “B”, call REST API “B”, else Call REST API “C”
c.) Can the app extract data from a return REST call, and put it a parameters on followup REST call,
example
Call REST API “A”, return JSON, extract the ‘assignUserID’, and Call REST API “B” with the assignUserID
Hi Khoo,
1. An fully functional evaluation version of Postman is now available. It’ll run for 2 weeks.
2. Conditional branch testing would be coming soon.
3. Yes. This is possible. Check out this article: https://blog.postman.com/index.php/2014/01/27/extracting-data-from-responses-and-chaining-requests/
Hi Abhivan.
Great tool. Love it thanks.
Found one limitation… really a limitation of Javascript I feel but thought I should let you know. If Json returned from Server contains number and is greater than 2^53 (9007199254740992) then it starts replacing the right most numbers with 0.
Cheers,
Mahboob
Thanks for reporting this! Will look into fixing it.
Will be cool can decide the order of tests runing because y have in a same folder all “auth” methods (login, logout) and make login and logout one per one, but i have other folder to endpoint of manage users (create, list, remove) but this never can run because previously was logout 🙁
Requests are run in the order which you specify in the UI. What you can do here is have a login request in the beginning and a logout request in the end. This will also mirror your user’s API request flow.
Hi Abhinav, Postman is just great. It is simple and powerful. One thing I feel could be added in the new version is multiple tabs, this can increase the workflow really well. As of now I see a new window opens up to run another API.
Hi Deep,
Clicking on the Postman logo will open a new window for sending requests. We’ll add a shortcut for this in an upcoming version.
Like it! However, is there any more features upcoming to improve ways to work in a team? It is not really convenient to upload/download collections and environments on any change by one team member, even using a revision control system.
Yep. We are working on these. Stay tuned for updates!
= = Nice Work!
..But Sorry.. I change its style : D
Abhinav, I am having an issue. I have 1 environment & 1 collection setup. Yet when I open the collection runner the drop downs for both collection & environment do not have them. There is 1 error in my console coming from ga_details.js saying “analytics is not defined”. Any help would be appreciated. Thanks.
Hi,
Can you email us at help@getpostman.com so the team can look into this?
Abhinav
Just bought Jetpacks yesterday…Silly thing is that I don’t know how to activate it?
Hey, regarding automated testing JetPacks seems to be paid for and doing something thats also be done via SoapUI. Does anyone who has tried both, have an opinion on this?
Can I run another test from my javascript code?
I want to be able to loop on one test until certain condition is met.