I Tested an API Client with AI—Without Writing a Single CSS Selector

I've spent years writing end-to-end tests. You know the drill: inspect element, copy that fragile CSS selector, watch it break the moment a frontend developer renames a class, rinse and repeat. It's tedious, brittle, and honestly, it's the part of testing I dread most.

So when I heard about Passmark, an open-source AI testing library that lets you write tests in plain English, I was skeptical but intrigued. When Bug0 announced the Breaking Apps Hackathon with $4,000 in prizes and free AI credits, I decided it was time to see if AI could really replace the selector grind.

Spoiler: It can. And it's kind of magical.

What Is Passmark? Passmark is built on top of Playwright, but it removes the need for page objects, XPaths, or CSS selectors. Instead, you describe what you want the browser to do in plain English, and an AI model interprets those instructions and drives the browser accordingly.

Here's what a typical test looks like:

javascript await runSteps({ page, userFlow: "Search for a product", steps: [ { description: "Navigate to the homepage" }, { description: "Type 'laptop' into the search bar", data: { value: "laptop" } }, { description: "Click the search button" }, ], assertions: [ { assertion: "The results page shows at least one product" }, ], test, expect, }); No selectors. No DOM traversal. Just intent.

Choosing My Target: Why Hoppscotch? The hackathon rules were simple: pick any public web app without CAPTCHAs or aggressive bot detection, write a Passmark test suite, and share your experience.

I could have picked a basic demo store. But I wanted to push the AI's limits. I chose Hoppscotch, an open-source API development platform—essentially a lightweight, browser-based Postman.

Why? Because Hoppscotch is complex. It has:

Multiple protocol modes (REST, GraphQL, WebSocket)

Dynamic panels and modals

Real-time response rendering

Environment variable interpolation

A stateful collection system

If Passmark could navigate this, it could handle anything.

Building the Test Suite I wrote five distinct test files, each targeting a different user flow. All tests are available in my GitHub repo: [Your GitHub Repo Link].

1. Basic REST Request The AI had to open Hoppscotch, ensure the REST tab was active, enter a public echo endpoint, click Send, and verify a 200 response.

2. Custom Headers The test added a custom header X-Custom-Header: BreakingAppsHackathon and validated that the echo server reflected it back.

3. GraphQL Query This was the real test. The AI had to locate the protocol dropdown, switch from REST to GraphQL, enter a public GraphQL endpoint, compose a query in the editor, execute it, and parse the JSON response to confirm country data was returned.

4. Environment Variables The test created a new environment named "HackathonTest," defined a baseURL variable, used it in a request template, and verified the interpolation worked.

5. Collections Finally, the AI saved a request to a collection and asserted that it appeared in the sidebar—a surprisingly nuanced interaction involving modals and state updates.

What Surprised Me (And What Broke) The Good:

The AI understood context remarkably well. When I wrote "Click the Send button," it didn't just look for any button—it seemed to grasp the spatial relationship between the URL bar and the adjacent Send button. It also handled dynamic loading states gracefully, waiting for the response panel to populate before running assertions.

The Tricky Part:

GraphQL mode was a challenge. Initially, the AI couldn't reliably locate the dropdown to switch protocols. The fix? More descriptive language. Changing "Select GraphQL" to "Locate the dropdown menu that currently shows REST, click it, and select GraphQL from the list" gave the AI the extra context it needed.

This taught me that while Passmark eliminates selectors, it rewards clarity and specificity in your step descriptions. It's less like coding and more like writing a manual for a very literal but intelligent assistant.

The Verdict After spending several hours with Passmark, I'm convinced this is the future of end-to-end testing. The speed at which I could write tests—without ever opening DevTools—was liberating. And the fact that my tests are immune to CSS class churn? That alone is worth the price of admission.

For teams tired of maintaining brittle test suites, Passmark offers a compelling path forward. The AI credits provided during the hackathon made it completely free to experiment, and the OpenRouter integration meant I didn't need to juggle multiple API keys.

Try It Yourself If you're curious, check out my code on GitHub: [https://github.com/Panther0508/my-hackathon-tests\]

Clone the repo, add your OpenRouter API key to a .env file, and run:

bash npm install npx playwright test You'll see the browser open and the AI take over—it's genuinely fun to watch.

Final Thoughts The Breaking Apps Hackathon pushed me to rethink how I approach testing. Passmark isn't perfect—it occasionally needs more explicit instructions than a human tester might—but the trade-off is a test suite that's faster to write, easier to maintain, and surprisingly robust.

Thank you to the Bug0 team for organizing this event and for building such an innovative tool. I'm excited to see where AI-driven testing goes next.

This post is part of the #BreakingAppsHackathon. All tests were written using Passmark and Playwright. No CSS selectors were harmed in the making of this submission.