Writing a behaviour driven API testing environment within Postman

Behaviour driven test frameworks like mocha and jasmine have certain unalienable advantages. They provide a clean way to organize tests as well as safe and (in this particular case,) isolated assertions to prevent runtime error of the testing spec itself.

For obvious reasons, having the same test specification style in Postman sounds just about right. After all, one may argue that they are testing the behavior of the response! And not to mention the endless benefits of well organized tests, seamless creation and teardown of fixtures, yada yada yada!

I would personally love to write tests within Postman that look like the following. (Never mind the aforementioned xkcd :-/ )

describe("response", function () {
  var responseJSON;
  
  it("must be a valid JSON", function () {
    responseJSON = JSON.parse(responseBody);
    expect(typeof responseJSON).toBe("object");
  });
  
  describe("json", function () {
    it("must not have error object", function () {
      expect(responseJSON.error).toNotBeOk();
    });
    
    it("must have a message", function () {
      expect(responseJSON.message).toBeOk();
    });
  });
});

Thus began my transit-time project to create a tiny (really tiny) test framework that I could include in my Postman Collections and write tests using it. I personally started using it and thought sharing it might help us BDD aficionados. The framework ended up being a gist that is about 0.7kb when minified. I named the script describe-it (such creativity!)

The “describe-it” test framework script

The following piece of minified code is the complete test framework that provides behavior driven testing functionalities (albeit basic) similar to mocha and jasmine. The script does look cryptic — but that is to ensure that it is as lightweight as possible.

(typeof tests!=='object')&&(tests={});var
it=((it=function(k,v){it.d.push(k);it.t[it.d]=1;it.b.forEach(it.c);try{v()}catch(e){it.t[it.d]=0;setTimeout&&setTimeout(function(){throw e;})}it.a.forEach(it.c);it.d.pop()}),
it.a=[],it.b=[],it.c=function(x){x()},it.d=[],it.d.toString=function(){return this.join(' ')},
it.t=tests,it.x=function(v){this.v=v},it.xp=it.x.prototype,it.xp.toBe=function(x){(this.v!==x)&&it._()},
it.xp.toNotBe=function(x){(this.v===x)&&it._()},it.xp.toEql=function(x){(this.v!=x)&&it._()},
it.xp.toNotEql=function(x){(this.v==x)&&it._()},it.xp.toBeOk=function(){!this.v&&it._()},
it.xp.toNotBeOk=function(){this.v&&it._()},it),describe=function(k,v){it.d.push(k);v();it.d.pop()},
expect=function(v){return new it.x(v)},beforeEach=it.b.push.bind(it.b),afterEach=it.a.push.bind(it.a);

The not-so-cryptic version of the script is in the gist: https://gist.github.com/shamasis/7b12f451374314500da3 (Although, my colleagues rightfully contradict that it’s still cryptic.)

The script exposes the following functions:

• describe (testgroup:string, tests:function);
• it (testname:string, test:function);
• beforeEach (setup:function);
• afterEach (teardown:function);
• expect (value); // assertion function

These functions work similar to mocha/jasmine and as such referring to their documentation is an obvious shortcut I would take.

The “expect” assertion function supports the following checks:

• expect(something).toBeOk(); // allows truthy variables
• expect(something).toNotBeOk(); // allows falsey variables
• expect(something).toBe(somethingElse); // checks equality
• expect(something).toNotBe(somethingElse); // checks inequality
• expect(something).toEql(somethingElse); // checks similarity
• expect(something).toNotEql(somethingElse); // checks dissimilarity

We might need more of these and especially ones that are related to API testing. (Would love to hear more in comments.)

Using the script within Postman

There are two parts to using this within Postman:

1. Save the script as a global variable
2. In the test scripts, eval the global variable before using the framework

Saving the script as a global variable

Saving the script as a global variable allows me to re-use the test framework script in multiple requests without having to manually add them. One can simply go and add the contents of the script to a global variable (or an environment variable) with name, say, describe-it-test-script. The documentation article Our Manage Environments documentation outlines how to set and use these variables.

Personally, I save the test script as part of the first request to ensure that the test script is bundled with Postman Collections while running them in Collection Runner or Newman.

The following collection fetches the test framework from the gist (so that I can keep it updated with latest content) and saves it in global variables. The subsequent requests use this global variable to get the test framework within each request’s test script.

Sample Collection that uses this test framework:
www.getpostman.com/collections/34163da95c4372e36c39

Import it in your postman and refer to the tests written in the collection called “describe-it” them.

Using the script in tests

Once you have the script stored in a variable, it is easy to use it in any test by running eval on the script. The first line of your test would look somewhat like the following:

eval(postman.getGlobalVariable("describe-it-test-script"));

I know — eval is evil! But that is a separate discussion altogether. If you want to avoid “eval” altogether, simply paste the minified script at the beginning of your tests. But that would make the tests look a bit ugly.

Post this, you can start using the describe, it, expect and other functions of the test framework normally.

Working with the test framework

Say, you have already saved the framework in a global variable, and you would like to run a test that ensures that a request responds with a 200 status code and serves HTML content type. The tests for the same would look like:

eval(postman.getGlobalVariable("describe-it-test-script"));
 
describe ("response", function () {
  it ("must have html content type header", function () {
    expect(responseHeaders['Content-Type'])
      .toBe('text/html; charset=utf-8');
  });
 
  it ("must return a 200 status code", function () {
    expect(responseCode.code).toEql(200);
  });
});

We can write as many tests as we want and nest as many describe blocks as we need. If there are runtime errors in one it block, the other tests would still continue to execute sequentially.

The collection shared in this post (named describe-it,) has examples of tests written using this micro framework. It has tests written with failing and passing scenarios. And upon running the collection in Postman Collection Runner, it outputs the test results grouped within their respective describe blocks.

How it works

The it function executes the test parameter function within a try-catch block. If the function executes without an error, it adds to the test description parameter (along with the description of it’s parent describe blocks) to the Postman tests variable and sets it to a truthy value. In the event that an expect assertion fails or some other error occurs, the tests object key is set to a falsey value.

In the above example, if all tests pass, the tests object looks like

tests: {
  "response must have html content type header": 1,
  "response must return a 200 status code": 1
}