GitHub Action

Overview

DevAssure O2 is an AI-powered end-to-end (E2E) testing agent that integrates directly into GitHub workflows. It reads pull request diffs, automatically generates targeted UI tests, executes them on real browsers, and reports results back to the PR — with no test scripts to write or maintain.

Core workflow: reads code diff → maps blast radius → generates tests → executes → reports

---

How It Works

When a pull request is opened or updated:

1. The DevAssure O2 agent reads the code diff from the PR.
2. It identifies the impacted areas (blast radius) of the changes.
3. It generates comprehensive end-to-end UI tests in natural language to validate those changes.
4. It executes the generated tests on real browsers.
5. It reports bugs and test results back to the PR, along with downloadable reports.

---

Prerequisites

• Runner OS: Unix-based runners only (Linux or macOS). Windows runners are not supported.
• Checkout step: The actions/checkout step must include with.ref so the agent can resolve the code diff correctly.
• Node.js: Required for installing @devassure/cli. Node.js 24 is recommended.
• DevAssure API Token: Required for authentication. See Authentication below.

---

Authentication

Creating a DevAssure API Token

1. Log in at https://app.devassure.io or sign up for free at https://app.devassure.io/sign_up.
2. Navigate to Account → Settings → Token.
3. Generate a new API token.
4. Save it as a GitHub repository secret named DEVASSURE_TOKEN.

Configuring the Token in Workflows

The action resolves the token in order of precedence:

1. with.token input (explicit pass-through)
2. env.DEVASSURE_TOKEN environment variable

Recommended approach — use the environment variable via GitHub Secrets:

env:
  DEVASSURE_TOKEN: ${{ secrets.DEVASSURE_TOKEN }}

---

Installation & Basic Setup

The action installs @devassure/cli globally, configures the token, and runs the specified command automatically. No manual CLI installation is required in the workflow.

Minimal Workflow (Default: test on every push)

name: devassure
on: [push]

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
          ref: ${{ github.head_ref || github.ref_name }}
      - uses: actions/setup-node@v4
        with:
          node-version: "24"
      - name: Run DevAssure
        uses: devassure-ai/devassure-action@v1
        env:
          DEVASSURE_TOKEN: ${{ secrets.DEVASSURE_TOKEN }}

---

Commands

The action supports the following commands via the command input. The default command is test.

Command	Description
setup	Installs and configures the DevAssure CLI
test	Generates and runs E2E tests from a PR or branch diff
run	Executes existing tests with optional filters
summary	Retrieves a test session summary
archive / archive-report	Archives the test report for a session

test — Generate and Run Tests from a Diff

- name: Run DevAssure test
  uses: devassure-ai/devassure-action@v1
  with:
    command: test
    minimum_score: 80
    workers: 2
    path: e2e/tests
    source: staging
    target: production
    commit_id: ${{ github.sha }}
    url: https://example.com
    environment: staging
    headless: "true"
  env:
    DEVASSURE_TOKEN: ${{ secrets.DEVASSURE_TOKEN }}

run — Execute Existing Tests with Filters

- name: Run DevAssure run
  uses: devassure-ai/devassure-action@v1
  with:
    command: run
    workers: 2
    path: e2e
    filter: smoke
    query: "login flow"
    tag: nightly
    priority: high
    folder: reports
    url: https://example.com
    environment: staging
    headless: "false"
  env:
    DEVASSURE_TOKEN: ${{ secrets.DEVASSURE_TOKEN }}

summary — Retrieve Session Summary

- name: Run DevAssure summary
  uses: devassure-ai/devassure-action@v1
  with:
    command: summary
    session_id: sess_123
  env:
    DEVASSURE_TOKEN: ${{ secrets.DEVASSURE_TOKEN }}

If session_id is not provided, the action runs devassure summary --last.

archive — Archive a Test Report

- name: Run DevAssure archive-report
  uses: devassure-ai/devassure-action@v1
  with:
    command: archive
    session_id: sess_123
  env:
    DEVASSURE_TOKEN: ${{ secrets.DEVASSURE_TOKEN }}

If session_id is not provided, the action runs devassure archive-report --last.

setup — Explicit Token Pass-Through

- name: Run DevAssure with explicit token
  uses: devassure-ai/devassure-action@v1
  with:
    token: ${{ secrets.DEVASSURE_TOKEN }}
    command: setup

---

Inputs Reference

All inputs are optional. The default command is test.

Input	Default	Description
command	test	DevAssure command: setup, test, run, summary, archive, archive-report
token	(empty)	DevAssure API token. Falls back to DEVASSURE_TOKEN env var if unset
path	(empty)	Relative path to run from (useful when tests are in a subdirectory)
source	(empty)	Source branch for test command scope and branch checkout
target	(empty)	Target branch for test command baseline comparison
commit_id	(empty)	Commit SHA for test command scope
filter	(empty)	Filter expression for run command
query	(empty)	Query string for selecting tests in run command
tag	(empty)	Tag-based selector for run command
priority	(empty)	Priority-based selector for run command
folder	(empty)	Folder selector for run command
url	(empty)	Application URL under test (used by test and run)
headless	true	Headless browser mode ("true" or "false")
session_id	(empty)	Specific session ID for summary or archive (defaults to latest session)
archive	true	Set to false to skip archive-report --last and artifact upload for test/run
minimum_score	75	Minimum passing score threshold for test/run. Set to 0 to disable
workers	(empty)	Number of parallel workers (integer > 0)
environment	(empty)	Environment name passed to test/run (e.g., staging, qa, production)

---

Outputs Reference

Output	Description
archive_path	Path to the archived report, parsed from devassure archive-report output

---

Command Parameter Mapping

The action forwards supported inputs to the CLI using --<arg-name>="<arg-value>" format.

Command	Forwarded Parameters
setup	(none)
test	path, source, target, commit_id, url, workers, environment, headless
run	path, filter, query, tag, priority, folder, url, workers, environment, headless
summary	session_id (when set), otherwise --last
archive / archive-report	session_id (when set), otherwise --last

Note: headless always defaults to true and is always passed for test and run commands.

---

Score Validation

For test and run commands, the action automatically runs devassure summary --last after execution and validates the score against minimum_score.

Score validation behavior:

• If minimum_score is non-numeric or <= 0 → validation is skipped.
• If the score is missing or N/A → the job fails.
• If the score is below minimum_score → the job fails with:
  Test score '<score>' is less than the minimum expected score (<minimum_score>)
• If the score meets or exceeds minimum_score → the job passes.

To disable score validation, set minimum_score: 0.

---

Report Archiving & Artifacts

For test and run commands, report archiving is enabled by default (archive: true). The action:

1. Runs devassure archive-report --last to archive the report.
2. Uploads the archive as a GitHub Actions artifact.
3. Adds test result details to the GitHub Actions run summary using dorny/test-reporter.

To skip archiving, set archive: false.

The archived .zip report can be downloaded from the GitHub Actions run summary and opened using:

• DevAssure CLI (@devassure/cli)
• DevAssure VSCode Extension

---

Runner Sizing Recommendations

Workers / Concurrency	Recommended Spec
Baseline (default)	4 vCPU / 16 GB RAM
Higher concurrency	Scale vCPU and RAM proportionally to avoid CPU and memory contention

Increasing workers without scaling the runner will cause performance degradation and potential test failures due to resource contention.

---

Local Execution

The DevAssure O2 agent can also be invoked locally using the DevAssure CLI npm package:

npm install -g @devassure/cli
devassure test --url https://your-app.com

See the DevAssure CLI documentation for full local usage details.

---

FAQ

How does minimum_score work?

minimum_score is the threshold a test run must meet or exceed to pass. The default is 75. If the score falls below this value, the GitHub Actions job fails. Set minimum_score: 0 to disable score validation entirely.

How are credits consumed?

Credits are consumed based on the number of browser interactions performed and the complexity of tests executed. Usage details for each run are visible in the DevAssure web portal under Usage (/usage).

How do I view the complete test report?

The full report is archived as a .zip file and available for download from the GitHub Actions run summary. Open it using the DevAssure CLI or the DevAssure VSCode extension.

Can I run the agent locally?

Yes. Install @devassure/cli via npm and run devassure commands directly from your local machine. Full CLI documentation is available at https://app.devassure.io.

Can I use session_id with the summary command?

Yes. Pass a specific session_id to retrieve the summary for that session. If omitted, the action defaults to devassure summary --last (most recent session). The --last flag and session_id are mutually exclusive — the action ensures only one is passed.

---

Related Resources

• DevAssure Web Portal
• DevAssure CLI npm Package
• DevAssure GitHub Action
• Sign Up for Free