Test-Driven Development with DevAssure O2: A Step-by-Step Guide

Test-Driven Development (TDD) is one of those practices everyone agrees with in principle and skips under deadline pressure. The reason is usually mechanical, not philosophical: writing the test first means maintaining test code before the feature exists, and that upfront cost is what teams cut when a sprint runs long.

This guide walks through a version of that workflow that removes most of the mechanical cost. Instead of writing test code, you write test cases as plain-English steps — in a spreadsheet or in TestRail — and DevAssure's O2 agent executes them against a real browser. You still get the core discipline of TDD (write the test first, watch it fail, build until it passes, refactor with a safety net), but the "test code" is a row in Excel or a case in TestRail instead of a Playwright script.

One clarification up front, in the interest of not overselling this: classic TDD, as Kent Beck described it, is a unit-level practice — a developer writes a small, fast, in-process test for a function or class before writing the function itself, and the red-green-refactor loop happens in seconds. What this guide describes is closer to acceptance TDD (ATDD) — writing the acceptance criteria for a user-facing flow before the flow is built, then using O2 to validate the whole feature end-to-end in a browser. It's a genuinely useful discipline, and it fits naturally with tools like TestRail that already store acceptance criteria in natural language. It is not a replacement for unit tests on business logic, and this guide isn't claiming it is — the two operate at different layers and different speeds, and most teams doing this well run both.

What TDD actually asks of you​

Three steps, repeated in a loop:

1. Red — Write a test for behavior that doesn't exist yet. Run it. It fails, because the behavior isn't built.
2. Green — Write the minimum implementation needed to make the test pass. Run it again. It passes.
3. Refactor — Clean up the implementation without changing behavior, using the passing test as a guardrail. Run the test once more to confirm nothing broke.

The value isn't the ceremony — it's that the test defines "done" before anyone starts guessing at it, and every later change to that code gets checked against the same definition automatically.

What DevAssure O2 is, in this context​

O2 is DevAssure's AI test-execution agent. It reads test cases written as natural-language steps ("Log in as admin," "Verify dashboard shows welcome message") and drives a real browser (Chromium, Chrome, or Edge) to carry them out, without anyone writing or maintaining Selenium/Playwright code. It's available two ways that share the same configuration: the @devassure/cli (terminal and CI/CD) and a VS Code extension (interactive, in-IDE runs). It can pull test cases from three places: local YAML files, a CSV file (which is how the "write tests in Excel" workflow works — export your spreadsheet to CSV), or directly from TestRail as a connected provider.

That last point is what makes the red-green-refactor loop practical here: you can keep authoring cases wherever your team already works — a shared spreadsheet or TestRail — and O2 becomes the executor for the "run the test" step of the loop, with no separate automation codebase to keep in sync.

Prefer a walkthrough before the written steps? This video covers the same O2 red-green-refactor flow:

Step-by-step: TDD with O2 using Excel/CSV​

Step 1 — Install and authenticate the CLI​

npm install -g @devassure/cli
devassure version
devassure login   # opens browser for OAuth2
devassure ping    # confirm auth/connectivity

For CI later, you'll use devassure add-token instead of interactive login. --no-ui disables the terminal UI and emits plain logs — useful in scripts; optional for local interactive runs.

Step 2 — Initialize the project​

devassure init

This creates a .devassure/ folder with app.yaml, test_data.yaml, personas.yaml, preferences.yaml, agent_instructions.yaml, and a sample test. test_data.yaml is where the target URL and any login credentials live, per environment — set this up before you run anything against a real app. See the CLI get-started guide for the full layout.

Step 3 (RED) — Write the acceptance test before the feature exists​

Open a spreadsheet and add one row per test case, with columns for at minimum a summary and the steps. For example, before a "password reset" feature has been built:

Export this as password-reset-tests.csv.

If your headers already match the defaults (Summary, Steps, Priority, Tags), you can skip mapping. If they differ — or you want the mapping explicit — add .devassure/csv_mapping.yaml:

summary: Summary
steps: Steps
priority: Priority
tags: Tags

Step 4 (RED) — Run it and watch it fail​

devassure run --csv=password-reset-tests.csv

Because the feature doesn't exist yet, this fails — correctly. That failure is the point: it's now a concrete, executable definition of what "password reset works" means, agreed on before a line of implementation code is written.

Step 5 (GREEN) — Build the feature, then re-run​

Implement the password reset flow. Then run the exact same command:

devassure run --csv=password-reset-tests.csv

O2 re-executes the same natural-language steps in a real browser. When it reports a pass, you have working software that satisfies the acceptance criteria you defined in Step 3 — not a guess that it probably works.

Step 6 (REFACTOR) — Change the implementation, keep the test as a guardrail​

Refactor the underlying code (switch the email provider, restructure the reset-token logic, whatever's needed) and re-run:

devassure run --csv=password-reset-tests.csv --headless=true

If it still passes, the refactor didn't change observable behavior. If it fails, you caught a regression before a person did.

Step 7 — Fold it into CI so the loop runs on every change​

devassure add-token "$DEVASSURE_TOKEN"
devassure run --csv=password-reset-tests.csv --archive=./test-reports
devassure summary --last --json

Placing this after your build/deploy step means every pull request re-proves the acceptance test, not just the developer's local run. For GitHub-native wiring, see O2 on GitHub Marketplace.

Step-by-step: TDD with O2 using TestRail​

If your team already plans and tracks cases in TestRail rather than a spreadsheet, the loop is the same — only where the test lives changes. Full provider details live in the TestRail × DevAssure docs.

Step 1 — Connect TestRail to DevAssure (one-time setup)​

You'll need your TestRail URL, username, and API key (found in TestRail under Administration → Site Settings → API). Then, from the CLI:

devassure add-provider-config

Choose provider testrail and enter the URL, username, API key, and (optionally) a default project ID. For CI or scripting, the non-interactive form:

devassure add-provider-config \
  --provider testrail \
  --url https://yourcompany.testrail.io \
  --username qa-user@company.com \
  --api-key YOUR_API_KEY \
  --project-id 42

Credentials are stored encrypted (testrail.enc.json), never committed as plain text. The VS Code extension has an equivalent flow in its Configuration view if you'd rather do this from the IDE.

Step 2 (RED) — Write the case in TestRail before building the feature​

Create the case in TestRail as you normally would, with steps written the same action-oriented way as the CSV example above ("Log in as admin," "Verify dashboard shows welcome message"). Tag it, set its priority and section, same as any other case in your suite.

Step 3 (RED) — Run it from O2 and confirm it fails​

devassure run --provider testrail \
  --project-id=42 \
  --case-ids=1001 \
  --post-results

--post-results tells DevAssure to create a run in TestRail and record the outcome against the case automatically — so the "red" state is visible in TestRail itself, not just your terminal. You can also target cases by section, label, or priority instead of by ID:

devassure run --provider testrail \
  --section=password_reset \
  --labels=smoke \
  --priority=P0 \
  --post-results

Step 4 (GREEN) — Build the feature, re-run the same case​

devassure run --provider testrail --case-ids=1001 --post-results

TestRail's dashboard now shows the case as passing, with no manual status update — DevAssure posted it.

Step 5 — Turn on defect logging and video for faster triage on failures​

devassure run --provider testrail \
  --case-ids=1001 \
  --post-results \
  --log-defects \
  --attach-videos=on-failure

If a case fails during a later refactor, DevAssure logs a defect in TestRail linked to that case and attaches a session recording, so whoever triages it can see exactly what the browser did without reproducing it locally first.

Step 6 (REFACTOR) — Re-run as a regression gate​

devassure run --provider testrail \
  --filter="section=Regression && priority=P0" \
  --post-results \
  --log-defects \
  --attach-videos=on-failure

Step 7 — Wire it into CI/CD​

devassure add-token "$DEVASSURE_TOKEN"
devassure run --provider testrail \
  --labels=smoke \
  --post-results \
  --attach-videos=on-failure \
  --archive=./test-reports
devassure summary --last --json

A note on code-change-scoped runs​

Beyond running a fixed set of pre-written cases, O2 also has a mode built specifically around code changes: devassure plan analyzes a git diff (branch-vs-branch or a specific commit) and generates a scoped test plan without executing anything; devassure run --plan-id=<id> executes that plan. devassure test does plan-and-run in one step. This is a useful complement to the TDD loop above once a feature has an established suite: instead of manually deciding what to re-run, you point O2 at the diff and let it determine blast radius — the same idea behind change-aware regression.

# Plan only, then execute
devassure plan --head feature/password-reset --base main
devassure run --plan-id=last

# Or plan-and-run in one step
devassure test --head feature/password-reset --base main

Where this fits, and where it doesn't​

Acceptance-level TDD with O2 is well suited to user-facing flows where the "test" is naturally something a person could describe in a sentence: login, checkout, password reset, form submission, navigation. It is not a substitute for fast, in-process unit tests around business logic, calculations, or edge-case-heavy functions — those still belong in a unit-testing framework where a red-green cycle takes milliseconds, not the seconds-to-minutes a real browser session takes.

The teams that get the most out of this approach tend to run both: unit-level TDD for logic (including when coding agents are writing the implementation), and O2-executed acceptance tests — authored in Excel or TestRail — as the outer feedback loop that confirms the feature actually works the way the ticket said it would.

Quick reference​

Command syntax was cross-checked against DevAssure's CLI reference, TestRail integration docs, and the @devassure/cli package. If any flag behaves differently in your installed CLI version, devassure help and devassure version are the fastest way to confirm.

Links​

DevAssure​

• DevAssure O2: https://www.devassure.io/o2-testing-agent
• Why TDD is having a second act with AI coding agents: https://www.devassure.io/blog/tdd-second-act-ai-coding-agents/
• Add power to TestRail cases with DevAssure: https://www.devassure.io/blog/testrail-devassure/
• Regression testing was never supposed to be this much work: https://www.devassure.io/blog/regression-testing-never-supposed-to-be-this-much-work/
• O2 on GitHub Marketplace: https://www.devassure.io/blog/devassure-o2-agent-github-marketplace/
• CLI get started: https://www.devassure.io/docs/DevAssure/Invisible%20Agent/CLI/overview/
• TestRail integration docs: https://www.devassure.io/docs/DevAssure/Invisible%20Agent/Integration/testrail-devassure/
• @devassure/cli on npm: https://www.npmjs.com/package/@devassure/cli
• DevAssure: https://www.devassure.io