The Postman CLI vs. Newman: choose the right tool for you

API testing is a big part of Postman’s history, and it continues to be a primary use of our platform. To extend our testing capabilities, we offer command-line interface (CLI) tools that can be used for local scripting or in CI/CD environments. Postman’s original CLI tool was called Newman (yes, it was named after the “Seinfeld” character), and in the fall of 2022, we introduced the new Postman CLI as part of Postman v10.

These CLI tools operate in very similar ways, as we will see in this blog post. For instance, both CLI tools allow you to run the requests in a Postman Collection. If any test code exists within the collection, that JavaScript code will be executed, as well. If an error occurs, or if some test expectations are not met within the JavaScript code, that collection request is halted and the CLI tool will move on to the next request.

Still, there are some important differences between the tools themselves and their capabilities that can help you determine which one will best suit your needs.

Let’s look at some of the similarities and differences between Newman and the Postman CLI.

Similarities between the Postman CLI and Newman

At a high level, both CLI tools operate in a similar fashion: they can run a collection that has been exported in JSON format, or they can use a Postman API key to retrieve the collection remotely. Additionally, both tools can use exported or remotely-retrieved environments from a Postman workspace.

The Postman application will allow you to export your collection in JSON format. You can export a Postman environment in JSON format as well, but you will need to follow best security practices for shared secret data, such as API keys in the export environment or collection variables.

Once your collection is exported, you can run it with either CLI tool. For example, if you called your exported collection file “my_collection.json”, then you would run the following commands:

• In the Postman CLI:
  $ postman collection run my_collection.json
• In Newman:
  $ newman run my_collection.json

The primary difference between these two commands is that the Postman CLI includes the word “collection” before the “run” instruction. This allows the Postman CLI to have a number of other base-level features, such as logging in or using API linting for governance and security.

If your JavaScript tests are built to accept incoming variables, you can set those variables in the same way with both tools. You can also choose between “global” variables and “environment” variables. For example, if you wanted to set a global variable called widget to a value of ABC, and set an environment variable called ServerURL to a value of http://1.2.3.4, you would run the following commands:

• In the Postman CLI:
  $ postman collection run my_collection.json --global-var “widget=ABC” --env-var “ServerURL=http://1.2.3.4”
• In Newman:
  $ newman run my_collection.json --global-var “widget=ABC” --env-var “ServerURL=http://1.2.3.4”

Similarly, Postman includes the capability to use CSV data for inputting large amounts of data to your tests. Both tools use the same command-line arguments for uploading data in CSV or JSON format:

• In the Postman CLI:
  $ postman collection run my_collection.json --iteration-data my_data.csv
• In Newman:
  $ newman run my_collection.json --iteration-data my_data.csv

Looking through the command-line arguments for the Postman CLI and Newman will show many similarities in the additional parameters that you can pass to both CLI tools.

Common platforms, such as CircleCI, Travis CI, and Jenkins, can use either the Postman CLI or Newman as part of an organization’s continuous integration (CI) or continuous delivery/deployment (CD) process for software releases. Both of these CLI tools will exit with an appropriate shell “exit code,” which these CI/CD platforms use to determine the success or failure of an application. Both applications will return a zero (0) if everything succeeded, and any other value will generally indicate an error or failure condition. Both the Postman CLI and Newman allow a command-line parameter to suppress the exit code, which can be helpful if you use other tools to check the reporting output in order to determine whether to proceed with your CI/CD process.

Postman wanted to maintain compatibility for people changing from Newman to the Postman CLI, so most of the command-line options will remain the same. However, the Postman CLI introduces new features that are not available with Newman, which we will discuss below.

Important considerations of exporting JSON vs. remote retrieval

Before we discuss the differences between the Postman CLI and Newman, let’s review some considerations when you’re deciding whether to export a collection or retrieve it remotely.

The examples above showed how the Postman CLI and Newman can use both exported collections and exported environments. You can also use your Postman API key to remotely retrieve a collection or an environment, but this will require you to use a known collection ID or environment ID. Remote retrieval will also use some of your allocated Postman API calls per month.

From the right context bar within the Postman application, you can click on a top-level collection and use the Info icon to find your collection ID, as shown in the screenshot below:

To use this collection ID, you would alter your Postman CLI command so that it looks like this:

• In the Postman CLI:
  $ postman collection run 12345678-12345ab-1234-1ab2-1ab2-ab1234112a12

With Newman, you will need to provide a Postman API key, whereas the Postman CLI will already have this API key available from its login procedure. In this example, we will imagine that the Postman API key is “1234-56789”:

• In Newman:
  $ newman run https://api.getpostman.com/collections/12345678-12345ab-1234-1ab2-1ab2-ab1234112a12?apikey=1234-56789

In most CI/CD platforms or other shell environments, a common best practice is to store the API key, in addition to other data like a collection ID or environment ID, in shell variables. For example, let’s look at an example where the collection ID is stored in a variable called COLLECTION_ID, the environment ID is stored in a variable called ENVIRONMENT_ID, and the Postman API key (not usually needed with the Postman CLI) is stored in a variable called POSTMAN_API_KEY:

• In the Postman CLI:
  $ postman collection run $COLLECTION_ID -e $ENVIRONMENT_ID
• In Newman:
  $ newman run “https://api.getpostman.com/collections/$COLLECTION_ID?apikey=$POSTMAN_API_KEY” -e $ENVIRONMENT_ID

Primary differences between the Postman CLI and Newman

Newman is an open source project that was started by Postman in 2014, and it is community-driven. It has an active GitHub community with about one hundred code contributors. You can install Newman using the Node Package Manager (npm) or the popular Homebrew package manager. There are a lot of tutorial videos available on how to install and use Newman because of its age and popularity. The application is not digitally “signed” by Postman, but it is actively compiled and released by the Postman team.

The Postman CLI was released in September 2022, and it is a closed source project built and maintained by employees of Postman. It has different installation instructions for Mac, Linux, and Windows systems, and the installation is digitally signed by Postman for authenticity. This tool also includes additional API linting for governance and security, which is available to enterprise customers.

As shown above, the Postman CLI introduces a way to log in to Postman from the command line, which can take an optional API key to immediately log you in, or provide a browser-based authentication check. After this step, the API key is stored in a local file, which should be kept safe.

There are no plans to deprecate Newman, but instructions for using the Postman CLI will be shown in the Postman application when using our CI/CD integrations. If you want to use Newman in your CI/CD platform, you will need to adjust the resulting instructions manually.

Results of Postman CLI executions will also be shown in the Postman application. To access these results, you can navigate to your collection in Postman and click on the Runs tab:

Newman will allow you to export collections, environments, and global variables before completing the execution of a collection, whereas the Postman CLI does not have this functionality.

By default, both CLI tools will run every collection request and report how many passed or failed. Postman allows the Postman CLI and Newman to halt any execution as soon as any single request fails using the “bail” option, but the Postman CLI allows for additional modifiers to halt a collection using a “folder” flag. It can also gracefully exit using a “failure” flag to allow for additional cleanup instructions you may want to execute.

One frequently asked question about Newman is how to run a single request. By default, this is not something that Newman supports, but you can use the “folder” flag with a unique request name to run that single request. The Postman CLI supports this functionality natively by allowing you to pass either a folder ID or an individual request ID using the “-i” flag, which you can include multiple times. For example, if we stored a request ID in a shell variable called REQUEST_ID and a folder ID in a shell variable called FOLDER_ID, we could run the following command in the Postman CLI:

$ postman run collection.json -i $REQUEST_ID -i $FOLDER_ID

The Postman CLI and Newman will run your collection requests in the order in which they are saved, but this “-i” flag for the Postman CLI will allow you to override this behavior and specify a custom order instead.

Which CLI tool should you choose?

Choosing which CLI tool to use will depend on your team and your API test automation needs.

The Postman CLI is newer and has a more compact command-line interface, as well as some features and extensibility that Newman does not have. The tool is created, digitally signed, and released by Postman, and it allows you to store a Postman API key for easier access to your collections and environments for remote retrieval.

Newman has been around for a long time and there are many videos and tutorials to help you get started, as well as a much larger open source community. The tool was created by Postman, but it is not digitally signed. Be sure that you are installing Newman from a trusted npm repository.

Both tools use very similar command-line arguments, and both offer rich reporting features. However, only Postman CLI executions will show up in the Postman application.