Smart API Security: The Power of LLMs and Postman MCP

APIs are the nervous system of modern software. They power apps, SaaS platforms, fintech products, and even your coffee machine. Yet, APIs remain one of the most common attack surfaces.

Every new endpoint, every schema change, every microservice deployed adds another vector that needs to be tested, reviewed, and secured.

Now imagine this challenge multiplied across dozens of teams and hundreds of APIs shipping updates daily. Traditional security reviews can’t keep up. Static scanners miss logic flaws. Manual reviews don’t scale. And even well-meaning security gates often become bottlenecks for developers.

What if AI could read your OpenAPI spec, understand security implications, generate Postman security test collections dynamically, and execute them against a live server, all automatically within your release pipeline?

That’s exactly what this is about: An AI-augmented API security review tool that merges LLMs, Postman tools, and automated CI orchestration; turning OpenAPI diffs (for now) into live security test Postman collections.

Architecture: Turning OpenAPI Diffs into Security Tests

Note: The current implementation supports only OpenAPI specifications, other API definition formats such as GraphQL schemas, gRPC proto files, etc are not part of this yet. The tool relies on OpenAPI format for structure, endpoint metadata, and diff generation.

File Structure:

 smart-api-security-agent/
├── app/
│   └── app.py                   # Flask mock API with deliberate security flaws (You need to integrate with your server URL)
├── openapi/
│   ├── api.v1.yaml              # Baseline OpenAPI
│   └── api.yaml                 # New version to trigger diff
├── tools/
│   ├── agent_openapi_to_postman.py # LLM orchestrator + MCP collection builder
│   ├── postman_json_to_sarif.py    # Postman JSON → SARIF converter
│   ├── cli_report_to_sarif.py      # Legacy fallback converter for SARIF conversion
│   └── mcp_client.py               # Minimal Postman MCP HTTP client
└── .github/
    └── workflows/
        └── openapi-security-mcp.yml # GitHub Actions workflow

Overview of what happens when a Pull Request that changes OpenAPI spec is submitted:

• OpenAPI diffing (to identify what changed)
• AI reasoning (to decide what security tests to run)
• Postman MCP + CLI (to actually create security test collection to execute the tests)
• SARIF translation (to integrate results into GitHub’s security dashboard)
• Pull request summaries (for fast triage and developer feedback)

Blueprint Breakdown: What Each File Does and Why

GitHub repo for the project: https://github.com/5h4d0wr007/smart-api-security-agent

#1. The Orchestration Engine (.github/workflows/openapi-security-mcp.yml)

This workflow is the backbone. It is what turns commits into live API security reviews. It:

• Triggers automatically on PRs when the OpenAPI files are modified
• Sets environment variables (POSTMAN_MCP_URL, POSTMAN_WORKSPACE_NAME, etc)
• Installs dependencies: Postman CLI, swagger-cli, oasdiff, and other Python packages
• Spins up the mock Flask app (app.py): you need to integrate with your API server URL
• Validates and diffs OpenAPI specs:

  swagger-cli validate api.v1.yaml
  oasdiff diff --format json api.v1.yaml api.yaml > diff.json

• Runs agent_openapi_to_postman.py to build a Postman collection and upload it via Postman MCP
• Executes that security contract test collection with Postman CLI:

  postman collection run ./generated-security-test.postman_collection.json \ --reporters cli,json --reporter-json-export run.json

• Converts results to SARIF and uploads to GitHub’s Code Scanning Dashboard (GHAS)
• Posts a custom Markdown summary in the PR, and grouping findings per endpoint

#2. The Mock Target API (app/app.py)

Note: The included Flask app (app.py) is a deliberately vulnerable mock designed purely to demonstrate the security workflow.

Each route in our demo app has one or more of the following issues injected:

• unauth (no token)
• owner (token t1)
• x-tenant (token t2)

1. Replace app.py with your the real code repo where API changes happen
2. Ensure the OpenAPI spec (api.yaml) points to the correct server URL (e.g., your API base URL)
3. Update environment variables in your CI (tokens, user IDs, etc.) under the workflow’s Postman environment section
4. The workflow will then automatically:

• Validate your OpenAPI spec
• Generate and run Postman security tests against your real endpoints
• Report results back to your pull request and GitHub Code Scanning dashboard (if enabled)

#3. The Smart Orchestrator Agent (tools/agent_openapi_to_postman.py)

This is where the real magic happens. It does five major things:

1. Validates both OpenAPI specs (base vs head)
2. Computes diff using oasdiff diff --format json
3. Calls OpenAI’s models (e.g., gpt-4o-mini) with the diff (diff.json) to generate a JSON security test plan (plan.json). Sets temperature=0.1 for better reproducibility and less randomness
4. Builds a Postman collection dynamically with test cases (you can customize prompt with temperature setting as per your need) with a focus on : (unauth) → expect 401, (owner) → expect 200, (x-tenant) → expect 403 or 404
5. Pushes the collection via Postman MCP, creating it in your actual Postman workspace

Key highlight: the collection includes automatic variable substitution (customize as per your app) logic:

setIfUsed('orderId', 'order_owner', 'order_other');

Finally, it saves generated-security-test.postman_collection.json which is an actual runnable security contract test Postman collection

#4. Postman MCP Client (tools/mcp_client.py)

This is the “transport layer” that makes the LLM’s “plan” actionable inside Postman.

The defined Python class:

• Authenticates using your MCP API key
• Sends requests like:

{ "jsonrpc": "2.0", "method": "tools/call", "params": {...} }

• Handles responses over both application/json and text/event-stream
• Interacts with Postman’s backend for listing workspaces, creating collections, and creating environments

Refer for your own implementation on self-hosted version: https://github.com/postmanlabs/postman-mcp-server

#5. Translating Execution Results into Security Findings (tools/postman_json_to_sarif.py & tools/cli_report_to_sarif.py)

After the Postman CLI run completes, it outputs run.json. This script converts that file into SARIF, the industry standard for static analysis findings (used by GitHub Advanced Security). There’s also a backup SARIF builder that handles non-standard Postman JSON formats (depends on CLI run like Newman).

It maps:

• Each failed test assertion → a SARIF result
• Each endpoint → a ruleId (postman.security.bola, etc.)
• Ensures safe file URIs so GHAS displays them properly
• Adds a “clean run” entry if no issues are found

Result → GitHub’s Security tab now lists findings like:

How to Integrate This For Your Use Case

• Replace the Flask app with your application URL
• Make sure your OpenAPI specs are stored in /openapi/ (or update the file structure)
• Configure your environment secrets
• Enable GitHub Advanced Security (GHAS) to see SARIF reports
• Adjust Postman environment variables (tokens, IDs) as per you auth system
• Fine-tune LLM model or rules to match internal access patterns

The Future of API Security

We’re entering a new age of “vibe coding”, where developers build with the help of AI copilots. The speed and abstraction that modern LLMs bring are reshaping how software is built, tested, and shipped.

But with this acceleration comes a challenge, security needs to move at the same velocity as development. APIs are shipped faster than ever. Threat models change, endpoints evolve, and human security reviewers can’t feasibly keep up with every schema diff, permission boundary, or missing guardrail.

That’s where AI-assisted security becomes a game-changer. By standardizing how AI agents talk to tools like Postman, we can bridge the gap between development and security. Instead of static checks, we get dynamic, LLM-driven reasoning over real APIs every time the code changes.

In the near future, we’ll see more of MCP-powered ecosystems where security isn’t just a gate, it’s an integrated, adaptive layer. Human security teams will be augmented by intelligent agents. These smart agents will act as force multipliers, letting engineers move fast while ensuring that every change still meets strong security standards.