Skip to main content

Natural Language Tests

Don't want to wait for auto-discovery? Describe your test scenarios in plain English — or in any supported format — and RoostGPT generates complete, runnable Playwright tests from your descriptions.

Supported Input Formats​

FormatDescription
Plain textFree-form test scenario descriptions in natural language
MarkdownStructured test scenarios with headings and steps
GherkinGiven/When/Then syntax for BDD-style scenarios
WordTest requirements or user stories from .docx documents
PDFTest specifications or acceptance criteria
Excel / CSVTabular test scenarios with columns for steps, inputs, and expected results
JSONStructured scenario definitions

How It Works​

  1. Provide your test descriptions through the RoostGPT Web UI or via USER_SCENARIO_FILE_PATH in the CLI
  2. RoostGPT parses the input, identifies individual test scenarios, and validates which are automatable
  3. The system maps scenarios against the target application's pages and elements
  4. Complete Playwright test code is produced with proper navigation, interactions, and assertions

Upload a single file with multiple scenarios and they are automatically split into individual, self-contained tests.

Example: Plain Text Input​

Test that a user can log in with valid credentials and see the dashboard.

Test that submitting the contact form with all required fields shows a success message.

Test that searching for "wireless headphones" returns at least one product result.

Test that adding an item to the cart updates the cart count badge.

Each description produces a self-contained Playwright test with navigation, element interaction, and meaningful assertions.

Example: Gherkin Input​

Feature: Shopping Cart

  Scenario: Add item to cart
    Given I am on the product listing page
    When I click "Add to Cart" on the first product
    Then the cart badge should show "1"
    And I should see a confirmation message

  Scenario: Remove item from cart
    Given I have one item in my cart
    When I click "Remove" next to the item
    Then the cart should be empty

Smart Validation​

Before generating test code, the system evaluates each scenario:

StatusMeaning
AutomatableScenario can be fully automated as a Playwright test
Partially automatableSome steps require manual verification
Needs reviewScenario is ambiguous or references elements that cannot be located

Scenarios that need review are reported with specific guidance on what to clarify.

Custom Instructions​

Guide the generation with your team's coding standards, naming conventions, selector strategies, or any other team-specific requirements. Your words, real tests — the generated tests faithfully reflect your described intent.

Next Steps​