Automating API Tests with Collection Runner and Newman

Automating API Tests with Collection Runner and Newman

Automating your API test suite is where Postman evolves from a testing tool into a continuous quality assurance platform. The Collection Runner executes your entire collection in sequence, and Newman — Postman's command-line companion — integrates those same tests into your CI/CD pipeline. This is the foundation of automated REST API testing at scale.

Collection Runner

The Collection Runner runs all requests in a collection automatically, in order, and reports results for every test assertion:

• Click the Run button (triangle icon) next to your collection name in the sidebar.
• Select which folders and requests to include.
• Choose the environment to run against.
• Set the number of Iterations (how many times to run the entire collection).
• Set a Delay between requests in milliseconds — important for rate-limited APIs.
• Click Run [Collection Name].

The runner shows each request in real time with green (pass) and red (fail) indicators for every test assertion. A summary at the end shows total pass/fail counts.

Data-Driven Testing with CSV and JSON Files

The Collection Runner supports running the same requests with different input data from a CSV or JSON file. This is powerful for testing an API with many different user inputs:

// users.csv:
username,email,role
alice,alice@test.com,admin
bob,bob@test.com,user
carol,carol@test.com,editor

In the request body, reference CSV columns as variables: {{username}}, {{email}}, {{role}}. The Collection Runner runs one iteration per row, with each iteration using the corresponding row's values.

Newman — Running Collections from the Command Line

Newman is the command-line interface for running Postman collections outside of the Postman app. Install it with npm:

npm install -g newman

Run a collection:

# Run with an environment file
newman run MyCollection.json -e Development.json

# Run with a data file
newman run MyCollection.json -e Production.json -d users.csv

# Generate an HTML report
newman run MyCollection.json -e Staging.json   --reporters html   --reporter-html-export report.html

Integrating Newman into CI/CD

Add Newman to your GitHub Actions workflow to run API tests on every push:

name: API Tests
on: [push, pull_request]

jobs:
  api-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Install Newman
        run: npm install -g newman
      - name: Run API Tests
        run: |
          newman run postman/collection.json             -e postman/staging-env.json             --reporters cli,junit             --reporter-junit-export results.xml
      - name: Publish Results
        uses: actions/upload-artifact@v3
        with:
          name: test-results
          path: results.xml

Interpreting Runner Results

• Green check: All test assertions in that request passed.
• Red X: One or more assertions failed — click to see which ones.
• Failed requests: Network errors or server errors — check the console for details.
• Iteration failures: If one request fails, subsequent requests in the same iteration still run (unless you stop on error).

Key Takeaways

• Collection Runner executes your entire test suite in sequence with full pass/fail reporting.
• Use CSV/JSON data files for data-driven testing — one iteration per data row.
• Newman runs collections from the command line — essential for CI/CD integration.
• Export collections and environments as JSON files to version-control them alongside your source code.