Skip to main content

API Test Generation

RoostGPT generates API tests from your Swagger / OpenAPI specification. It produces test collections covering all endpoints, HTTP methods, request payloads, authentication, and edge cases.

How It Works​

  1. Spec ingestion — RoostGPT reads your Swagger/OpenAPI file (.yaml or .json)
  2. Server setup — Optionally provide a live server URL; otherwise RoostGPT spins up a Prism mock server automatically
  3. Test generation — The AI model generates test cases per endpoint, including positive, negative, and boundary conditions
  4. Execution — Tests are run against the live or mock server to validate responses
  5. Output — Test files and CSV data files are written to your project directory

Supported Input Formats​

  • OpenAPI 3.x (.yaml, .json)
  • Swagger 2.0

Supported Output Frameworks​

FrameworkOutput
Postman.json collection files
Karate.feature files with CSV test data
Rest AssuredJava test classes

Using the CLI​

# Configure your test
TEST_NAME=weather-api-test
TEST_TYPE=api-spec-test
AI_TYPE=bedrock_ai

API_SPEC_TYPE=swagger
API_SPEC_SOURCE=file
API_SPEC_FILE_PATH=$HOME/workspace/weather.yaml

# Optional: provide a live server
# API_BASE_URL=https://api.example.com
# API_AUTH_TOKEN=your_token

roostgpt test create -c api-test.env

Using the VS Code Plugin​

  1. Right-click on your Swagger spec file in VS Code
  2. Select "Generate Test using RoostGPT" → "API tests using API Spec file"
  3. Select your output framework (Postman, Karate, Rest Assured)
  4. Optionally provide a live server URL and auth token
  5. Click Generate

If no live server is provided, RoostGPT automatically provisions a Prism mock server.

CLI Prompts Explained​

When running the CLI interactively, you may see:

Please enter the alias name for the api spec: weather.yaml: url1_weather
✔ Do you want to provide the url1_weather_BASE_URL? (default: Yes). No
✔ Do you want to provide the url1_weather_AUTH_TOKEN? (default: Yes). No
  • Alias name — A unique identifier for this spec. Accept the default.
  • BASE_URL — The live API server URL. Skip to use a mock server.
  • AUTH_TOKEN — Authentication token for the live server. Skip for mock.

Providing a live server URL and token enables validation against real API responses and produces more accurate internal reports.

Output Structure​

postman_collections/
└── postman-api-test/
    ├── Weather_Realtime_API.json
    ├── Weather_Forecast_API.json
    └── ...

# Or for Karate:
karate-tests/
├── WeatherApiTest.feature
├── WeatherApiTest_data.csv
└── ...

View Results​

roostgpt test result

AI Provider Recommendations​

  • OpenAI: gpt-4o
  • Claude: claude-sonnet-4-5
  • Google Gemini: gemini-2.5-pro, gemini-2.5-flash
  • AWS Bedrock: global.anthropic.claude-sonnet-4-20250514-v1:0

See AI Provider Support for full model comparison.