39 posts tagged with "AI-Agentic Test Automation"

The target is demo.evershop.io/kids, a public EverShop demo store. Fourteen products, a price filter, a colour facet, a sort control, add-to-cart, a cart page, and a checkout. Enough surface area to hit every interesting case.

By the end you will have driven six flows through an LLM, measured what they cost, and converted one of them into a Playwright spec that runs without an agent.

There's a number nobody puts in their testing ROI deck: the cost of keeping the test infrastructure alive.

Not writing tests. Running them. The Selenium grid that needs a Chrome bump every six weeks. The CI runners that queue for twenty minutes at 5 PM. The Docker image someone built in 2023 that nobody dares touch. Ask any engineering team where their automation effort actually goes, and "maintaining the thing that runs the tests" is quietly near the top.

Today we're shipping the fix: O2 Cloud Agent. Your tests execute in real browsers on infrastructure we run — nothing to install, nothing to configure, nothing to maintain.

For most of software history, writing code was the bottleneck and testing was the thing squeezed around it. That balance has flipped. AI coding agents — tools like Claude Code, GitHub Copilot's agent mode, and Cursor — can now scaffold a feature, write the surrounding logic, and open a pull request in the time it used to take an engineer to read the ticket. The code isn't the constraint anymore. Verifying that the code actually does what it's supposed to is.

That's the real argument for AI in testing, and it's worth being precise about it: it isn't "AI is trendy, so apply it everywhere." It's a capacity problem. If code generation gets 5–10x faster and test creation doesn't, testing becomes the new bottleneck — or worse, it gets skipped, and the speed gains show up in production incidents instead of shipped features.

An AI coding agent does not know its code works. It infers this from signals — the same way a junior engineer infers a change is safe to merge because the test suite went green, not because they proved it mathematically. The quality gap between a coding agent that "looks done" and one that is actually done comes almost entirely from how well its architecture is built to gather, trust, and act on those signals. That architecture — not the underlying language model — is where most of the meaningful differentiation between coding agents currently sits, and it's worth understanding in some detail when building with these tools or evaluating which one to trust with production code.

Manual QA doesn't scale with a growing codebase. Every new feature adds another set of paths someone has to click through by hand, and every release means re-checking that nothing old broke. Test automation exists to break that tradeoff — but only if it's built the right way. Here's a practical, in-depth look at what test automation actually involves, where teams typically go wrong, how to think about tooling choices, and what's changed recently with AI-driven approaches.