Skip to main content

GitLab CI/CD Catalog

Overview​

DevAssure O2 is an AI-powered end-to-end (E2E) testing agent that integrates directly into GitLab CI/CD pipelines. It reads merge request diffs, automatically generates targeted UI tests, executes them on real browsers, and surfaces results in the pipeline and MR — with no test scripts to write or maintain.

Core workflow: reads code diff → maps blast radius → generates tests → executes → reports

The component works on merge request pipelines, branch pipelines, and other GitLab CI jobs. It uses the DevAssure CLI — @devassure/cli.

How It Works​

When a merge request pipeline runs:

  1. The DevAssure O2 agent reads the code diff from the MR or branch.
  2. It identifies the impacted areas (blast radius) of the changes.
  3. It generates comprehensive end-to-end UI tests in natural language to validate those changes.
  4. It executes the generated tests in real browsers.
  5. It surfaces bugs in the pipeline and in the MR, with reports and GitLab Tests tab integration via JUnit XML.

Prerequisites​

  • Runners: Unix-like GitLab runners only — Linux container-based or macOS shell runners. Windows runners are not supported.
  • Git: The test command expects a normal git checkout of the repository. For merge requests, ensure enough history is available for diffs and branches to resolve. Set GIT_DEPTH: 0 in the job if needed.
  • Node.js: Version 20 or above. The component validates this automatically.
  • DevAssure API Token: Required for authentication. See Authentication below.

Authentication​

Creating a DevAssure API Token​

  1. Log in at https://app.devassure.io or sign up for free at https://app.devassure.io/sign_up.
  2. Navigate to Account → Settings → Token.
  3. Generate a new API token.
  4. Save it as a masked (and protected, if using protected branches) GitLab CI/CD variable named DEVASSURE_TOKEN at the project or group level.

Configuring the Token​

The component resolves the token in order of precedence:

  1. token component input (explicit pass-through)
  2. DEVASSURE_TOKEN CI/CD variable (recommended)

Recommended approach — store the token as a masked variable in Settings → CI/CD → Variables and leave the token input empty. The component reads it automatically.

Component Location​

The CI component is exposed from this repository at:

templates/devassure/template.yml   (component name: devassure)

Installation & Basic Setup​

Include the component in your .gitlab-ci.yml using a pinned tag for stability:

include:
  - component: $CI_SERVER_FQDN/devassure/devassure/devassure@1

Set DEVASSURE_TOKEN as a masked/protected CI/CD variable in your project or group.

Security note: Always pin to a specific tag (e.g., @1 or @1.0.0). Avoid @latest in component references to ensure pipeline stability.

Commands​

The component supports the following commands via the command input. The default command is test.

CommandDescription
setupInstalls and configures the DevAssure CLI
testGenerates and runs E2E tests from an MR or branch diff
runExecutes existing tests with optional filters
summaryRetrieves a test session summary
archive / archive-reportArchives the test report for a session

test — Generate and Run Tests from a Diff​

include:
  - component: $CI_SERVER_FQDN/devassure/devassure/devassure@1
    inputs:
      command: test
      minimum_score: 80
      workers: 2
      path: e2e/tests
      source: $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME
      target: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME
      commit_id: $CI_COMMIT_SHA
      url: https://example.com
      environment: staging
      headless: "true"

run — Execute Existing Tests with Filters​

include:
  - component: $CI_SERVER_FQDN/devassure/devassure/devassure@1
    inputs:
      command: run
      workers: 2
      path: e2e
      filter: smoke
      query: "login flow"
      tag: nightly
      priority: high
      folder: reports
      url: https://example.com
      environment: staging
      headless: "false"

summary — Retrieve Session Summary​

include:
  - component: $CI_SERVER_FQDN/devassure/devassure/devassure@1
    inputs:
      command: summary
      session_id: sess_123

If session_id is not provided, the component runs devassure summary --last.

archive — Archive a Test Report​

include:
  - component: $CI_SERVER_FQDN/devassure/devassure/devassure@1
    inputs:
      command: archive
      session_id: sess_123

If session_id is not provided, the component runs devassure archive-report --last.

setup — Explicit Token Pass-Through​

include:
  - component: $CI_SERVER_FQDN/devassure/devassure/devassure@1
    inputs:
      token: $DEVASSURE_TOKEN
      command: setup

Inputs Reference​

All inputs are optional.

InputDefaultDescription
stagetestPipeline stage where the generated DevAssure job runs
imagenode:24-bookwormContainer image used to execute the job
commandtestDevAssure command: setup, test, run, summary, archive, or archive-report
token(empty)DevAssure API token. Falls back to DEVASSURE_TOKEN CI/CD variable if empty
path(empty)Relative path to run from (useful when tests live in a subdirectory)
source(empty)Source branch for test command scope and branch checkout
target(empty)Target branch for test command baseline comparison
commit_id(empty)Commit SHA for test command scope
filter(empty)Filter expression for narrowing run command execution
query(empty)Query string for selecting tests in run
tag(empty)Tag-based selector for run
priority(empty)Priority-based selector for run
folder(empty)Folder selector for run
url(empty)Application URL under test (used by test and run)
headlesstrueHeadless browser mode for test and run ("true" or "false")
session_id(empty)Session ID for summary or archive (defaults to latest session)
archivetrueSet to false to skip archive-report --last and artifact archiving for test/run
minimum_score75Minimum passing score threshold for test/run. Set to 0 to disable
workers0Parallel worker count. Use 0 for CLI default; must be an integer > 0 when set
environment(empty)Environment name passed to test/run (e.g., staging, qa, production)
verbose(empty)Enable verbose logging (true / false)
debug(empty)Enable debug logging (true / false)

Command Parameter Mapping​

The component forwards supported inputs to the CLI using --<arg-name>="<arg-value>" format.

CommandForwarded Parameters
setup(none) — optional verbose / debug flags
testpath, source, target, commit_id, url, workers, environment, headless
runpath, filter, query, tag, priority, folder, url, workers, environment, headless
summarysession_id (when set), otherwise --last
archive / archive-reportsession_id (when set), otherwise --last

Note: headless always defaults to true and is always passed for test and run commands. session_id and --last are mutually exclusive — the component ensures only one is passed.

Score Validation​

For test and run commands, the component automatically runs devassure summary --last after execution and validates the score against minimum_score.

Score validation behavior:

  • If minimum_score is non-numeric or <= 0 → validation is skipped.
  • If the score is missing or N/A → the job fails.
  • If the score is below minimum_score → the job fails.
  • If the score meets or exceeds minimum_score → the job passes.

To disable score validation, set minimum_score: 0.

Artifacts & Reports​

Job Artifacts​

When archive is not false and the command is test or run, archived reports are written under:

.devassure-artifacts/

These are exposed as job artifacts with expire_in: 7 days and can be downloaded from Build → Job artifacts.

GitLab Tests Tab Integration​

For test and run commands, JUnit XML is published as artifacts:reports:junit, making test results visible directly in the GitLab Tests tab of the pipeline.

Viewing Reports​

Downloaded artifacts (.zip) can be opened using:

  • DevAssure CLI (@devassure/cli)
  • DevAssure VSCode Extension

Runner Sizing Recommendations​

Workers / ConcurrencyRecommended Spec
Baseline (default)4 vCPU / 16 GB RAM
Higher concurrencyScale vCPU and RAM proportionally to avoid CPU and memory contention

Increasing workers without scaling the runner will cause performance degradation and potential test failures due to resource contention.

Publishing to GitLab Catalog​

Recommended release flow for component maintainers:

  1. Merge changes to the default branch.
  2. Create a semantic version tag (e.g., 1 or 1.0.0).
  3. Let .gitlab-ci.yml run release:catalog on tags.
  4. Consumers pin the component include with @<tag>.

Security & Best Practices​

  • Store DEVASSURE_TOKEN as a masked and protected CI/CD variable — never hardcode it in .gitlab-ci.yml.
  • Pin component references to specific tags (e.g., @1). Avoid @latest.
  • Keep runner resources aligned with workers count (see Runner Sizing Recommendations).

Local Execution​

The DevAssure O2 agent can also be invoked locally using the DevAssure CLI npm package:

npm install -g @devassure/cli
devassure test --url https://your-app.com

See the DevAssure CLI documentation for full local usage details.

Troubleshooting​

GLIBC_2.35 not found​

Cause: The runner image is too old for the packaged DevAssure agent runtime.

Fix: Use a newer Linux base image such as node:24-bookworm, ubuntu:22.04, or newer. Override the component image with inputs.image if your organization provides a compatible custom base.

FAQ​

How does minimum_score work?​

minimum_score is the lowest test score that still passes the job. The default is 75. If the score falls below this value, the job fails; if it meets or exceeds the threshold, the job passes. Set minimum_score: 0 to disable score validation entirely.

How are credits consumed?​

Credits depend on browser interactions and test complexity. Per-run usage is visible in the DevAssure portal at app.devassure.io/usage.

How do I view the complete test report?​

Download the job artifacts from Build → Job artifacts and open them with the DevAssure CLI or the DevAssure VSCode extension.

Can I run the agent locally?​

Yes. Install @devassure/cli via npm and run devassure commands directly from your local machine.

Can I use session_id with the summary command?​

Yes. Pass a specific session_id to retrieve the summary for that session. If omitted, the component defaults to devassure summary --last (most recent session). The --last flag and session_id are mutually exclusive.

Related Resources​