# Measure your Postman Collection's quality with the Postman API and Spectral rules

[Postman Collections](https://www.postman.com/collection/) are an excellent tool for collaborating with users in the [API design](https://www.postman.com/api-platform/api-design/), development, testing, and documentation processes. You can [share](https://learning.postman.com/docs/collections/using-collections/#sharing-a-collection) your collection with others, who can then help in the development process by providing their feedback in [comments](https://learning.postman.com/docs/collections/using-collections/#adding-comments-to-a-collection). To help your API users, you should always define the following items in a collection: - **Documentation**: Adding [descriptions](https://learning.postman.com/docs/publishing-your-api/authoring-your-documentation/) to the collection and its requests helps users understand the collection’s purpose and how your API works. At the collection’s top level, provide details for how users can get started. Describing requests in your collection helps users to understand how it works and what to expect from it.
- **Examples:** Providing [examples](https://learning.postman.com/docs/publishing-your-api/authoring-your-documentation/#including-examples) of different responses (such as success, unauthenticated errors, and bad requests) for each request helps users understand the expected responses and adapt their code.
- **Variables:** Defining generic requests with global, environment, or collection [variables](https://learning.postman.com/docs/sending-requests/variables/variables-intro/) lets users easily test APIs against different environments or conditions by simply changing the variables' values.
- **Authentication:** Defining [authentication](https://learning.postman.com/docs/publishing-your-api/setting-up-authentication-for-public-apis) at the global or request level makes it much easier for users to test the APIs.
- **Tests:** Providing [tests](https://learning.postman.com/docs/writing-scripts/test-scripts/) in your collection can help your users get started testing your API’s behavior.
 
 When a collection lacks these items, it becomes difficult for users to get any value from it or use it. If you intend to share your collection with users, whether internal or [public](https://blog.postman.com/exploring-public-apis-and-public-api-network/), it's good practice to properly define these items to improve the quality of your collections. In this blog post, we’ll show you how to create a tool for your automation that evaluates the quality of a collection using the [Postman API](https://www.postman.com/postman/workspace/postman-public-workspace/collection/12959542-c8142d51-e97c-46b6-bd77-52bb66712c9a),[ Spectral CLI](https://docs.stoplight.io/docs/spectral/9ffa04e052cc1-spectral-cli), and[ jq](https://jqlang.github.io/jq/). You can find the code in our [postman-collection-spectral-linter GitHub repository](https://github.com/postmanlabs/postman-collection-spectral-linter). ## Best practices for defining the quality rules for a Postman Collection

 The **postman-collection-spectral-linter** tool uses [Spectral rules](https://docs.stoplight.io/docs/spectral/d3482ff0ccae9-rules) to detect any quality issues in a Postman Collection. Spectral rules are a method for defining the conditions that a JSON or YAML file must meet. The Spectral CLI tool processes a given JSON or YAML file and checks if that file meets all the rules in this file. For example, we define a `collection-description-is-mandatory` rule that checks the `collection.info` property in the collection’s JSON for a `description` property: ```
rules:
  collection-description-is-mandatory:
    description: Collection must have a description.
    given: $.collection.info
    severity: error
    then:
      field: description
      function: truthy

```

### Measuring quality

 Based on our experience as Postman users and our experience working with Postman users around the world, we have seen several key elements that a collection should include to be considered a high-quality collection. We've defined these features in the tool’s [*rules.yaml*](https://github.com/postmanlabs/postman-collection-spectral-linter/blob/main/rules.yaml) file: #### **Errors**

- All requests must have a description.
- Requests must have a URL.
- All requests have at least a 2XX or 3XX example.
- All responses must have a status code.
- DELETE responses must have a 200 or 204 status code.
- Requests with path variables require a 404 response.
- All request query parameters must have a description.
- All request path parameters must have a description.
- All PATCH requests require a request body.
- All responses must have a status code.
- All authenticated requests require a 401 response.
 
#### **Warnings**

- HTTP should not be used
- Response bodies should be in JSON
- All folders should have a description
- Request names should not end in "Copy"
 
 This ruleset is not exhaustive and may not necessarily meet your use case. You can add your own customized rules to fit your needs, extend any of the default rules, or remove rules you don’t need in the *rules.yaml* file. ## **How it works**

 When you run the tool’s shell script (Bash) command, it: - Retrieves the collection in JSON format with the Postman API’s [GET /collections/{collectionId}](https://www.postman.com/postman/workspace/postman-public-workspace/request/12959542-a6a282df-907e-438b-8fe6-e5efaa60b8bf) endpoint
- Uses Spectral CLI to validate a set of defined rules against the collection
- Checks whether the Spectral results contain errors and returns them the script’s exit status. If any of the rules fail, then the command fails
 
 The script is very straightforward: ```
# Perform a curl request against the Postman API. Save the request body in a file called collection.json
curl -s -H "x-api-key: $API_KEY" https://api.postman.com/collections/$COLLECTION_UID | jq . > _collection.json

# Lint the collection using spectral. Save the result in a JSON file
spectral lint _collection.json --ruleset $RULES_PATH -f json --quiet > _result.json

# search for errors (severity=0) in the result file. If there are errors, exit with a non-zero status code
jq '.[] | select(.severity==0)' -e _result.json >/dev/null

if [ $? -eq 0 ]
  then
    exit 1
fi

```

- In line 2, the tool uses the Postman API to retrieve the collection in JSON format. It then uses jq to beautify the output and saves it as a temporary *\_collection.json* file.
- In line 5, Spectral CLI checks the rules in the rules.yaml file, then dumps the results to a temporary *\_results.json* file in JSON format.
- In line 8, jq performs a search for errors with a `0` (error) severity.
- In line 10, the tool checks if the previous command succeeded with an execution status of `0`. In that case, it returns an error execution status of `1`. You can use this status code in scripts to evaluate if the collection contains any errors. Additionally, you can use the `_result.json` file contents with the error details to perform any kind of notification.
 
### Using the tool

 Once you clone the GitHub repository, you can run the shell script with the following command: ```
./linter.sh -c <collection_uid> [-k <postman_api_key>] [-r path_to_ruleset_file]

```

 In this command, the `postman_api_key` parameter defaults to a `POSTMAN_API_KEY` environment variable and `path_to_ruleset_file` defaults to the `rules.yaml` ruleset. You can find the ruleset in our[ postman-collection-spectral-linter](https://github.com/postmanlabs/postman-collection-spectral-linter) GitHub repository, and you can [extend it](https://docs.stoplight.io/docs/spectral/83527ef2dd8c0-extending-rulesets) by adding new rules or turning off any existing rules you don’t need. ## Summary

 The Postman API provides full CRUD capabilities that let you manage your collections. You can easily retrieve the collection JSON using the Postman API and a valid API key. To continuously validate whether your collection follows your quality standards, you can define your Spectral rules and check those rules automatically with the Spectral CLI. It’s important to integrate quality checks in your CI/CD workflows to improve the overall quality of your collections. ## Next steps

- Define your own rules, adapted to the quality level you want in your collections. You can extend the [best practices](https://blog.postman.com/document-your-api-like-a-pro-postman-collection-best-practices/) rules we provided in this blog post.
- Use the tool to automatically notify users when they don’t meet the quality standards to help your team maintain great collections.
- Use the detailed results of the Spectral CLI to send notifications or add comments to a collection with the Postman API.
 
 Do you have any use cases related to collection quality? We would love to hear about it—please let us know in a comment below!