Skip to content

Regression Testing Agent

The Regression Testing Agent replays previously reported vulnerabilities against a scanned asset to verify whether they are still reproducible.

It is designed for retesting after a fix, before release, or during recurring security validation campaigns.

What It Does

  • Consumes uploaded pentest reports: Uses uploaded report files as replay source material
  • Builds executable replay plans: Extracts reproducible vulnerabilities and converts them into ordered action checklists
  • Replays each plan item: Executes one vulnerability at a time on the current target asset
  • Validates replay signals: Confirms findings with concrete evidence before reporting
  • Publishes replay issues: Produces findings with report context and execution evidence

Supported Assets

  • Frontend web applications (frontend)
  • REST API services (api_service_rest)
  • GraphQL API services (api_service_graphql)

The agent automatically switches to browser-driven replay for frontend assets and HTTP-driven replay for API assets.

Configuration

Enable AI Pentesting and provide replay files

automated_pentesting:
  enabled: true

experimental:
  regression_testing_file_upload_ids:
    - "file_upload_id_1"
    - "file_upload_id_2"

The uploaded file IDs must reference files already attached to the scan context.

Authentication and scope

Replay execution honors the standard authentication and scope configuration of the scanned asset:

  • Frontend replay uses frontend auth and frontend scope settings
  • API replay uses REST/GraphQL scope settings and authenticated HTTP client execution

Current Constraints

  • Only PDF replay source files are currently supported for plan extraction
  • Replay quality depends on how actionable the original report reproduction steps are
  • All actions stay within configured scope boundaries
  • Replay execution remains bounded by scan timeout and model budget limits

Best Practices

  • Upload pentest reports with clear, step-by-step reproduction instructions
  • Include expected signals (error messages, DOM changes, outbound callbacks) in reports
  • Run replay scans after remediation to confirm closure
  • Keep scope precise to reduce noisy or irrelevant replays