4 posts tagged with "Build vs Buy"

For twenty years, automated testing meant writing more code. That era is ending — and most teams haven't noticed yet.

The first automated test I ever wrote was in Playwright. It was 2012. It launched a browser, filled in a login form, and checked that the dashboard loaded. It passed. I felt like a wizard.

More than a decade later, the fundamental contract hasn't changed. To test software, you write more software. You describe, in code, what your code is supposed to do. Then you maintain that second codebase forever.

We've built entire careers, conferences, certifications, and consultancies on this premise. Selenium. Cypress. Playwright. Test pyramids. BDD. Page object models. The whole apparatus rests on a single assumption: humans must specify, in writing, what to test.

That assumption is quietly dying.

A CTO told me last month, very pleased with himself:

"We're already paying $200/month per dev for Claude. Testing is basically free now — we just ask Claude to also write the tests."

I asked him to pull up his Anthropic bill. The number was 14x what he'd budgeted at the start of the quarter. And his team still hadn't shipped the regression suite.

This is the most expensive trap in the AI tooling stack right now, and it's expensive precisely because it looks free. If you've already bought a coding agent, asking it to do double duty as a testing agent feels like the obvious move. One subscription, one workflow, one bill.

Except there is no "one bill." There are six.

Last week, on a customer call, a CTO asked me the question I now get every single week:

"I'm already using Claude to write my code. Why can't I just point the same agent at the code and have it test itself?"

It's a fair question. If one AI can write a React component, surely it can write the test for that component too. The economics look seductive — one tool, one workflow, one bill.

But here's an insight:

Testing your own PR is like proofreading your own essay. You'll read it 10 times. You'll miss the same typo 10 times. Because your brain autocorrects what it wrote.

That insight is what this blog is about. And it explains why a coding agent, no matter how capable, is structurally the wrong tool to verify its own work.

If you have been paying attention to Anthropic's tooling ecosystem over the last 12 months, Claude's web testing capabilities are genuinely exciting. The Computer Use API, the Claude in Chrome extension, and Claude Code's browser integration represent a real shift in what an AI model can do inside a browser - and the instinct to build a testing layer on top of it is completely rational.