Playwright Agent
Import a Playwright repo and let an agent read, edit and run the tests — then open a PR with the fix.
- Role
- Software Engineer
- Status
- shipped
Problem
Test suites rot faster than anyone repairs them. A selector moves, a flow changes, and the failure sits in CI for a week. This platform points an agent at the repo so the loop — read the test, run it, change it, prove it passes — happens without a human holding it.
Architecture
Read this diagram as text
- Web console connects to Agent orchestrator
- Agent orchestrator tool calls LLM
- Agent orchestrator executes Playwright runner
- Playwright runner results Run history
- Agent orchestrator commit + PR Git provider
- Web console reads Run history
Engineering decisions
The stack, reasoning not written up yet:
Playwright · TypeScript · Node.js · Azure OpenAI · Docker · GitHub API
The hard part
An agent given a failing test and told to make it pass will delete the assertion. That is not the model misbehaving, it is the honest reading of the instruction, and it is the one thing that would make this worthless. The loop is built so that route is closed: selectors, waits and setup are editable, the assertion is not, and a run only counts as fixed when the same assertion passes against the same flow. The other half is that it clones repositories it did not write and executes them, so the runner is sandboxed and the git integration only ever opens a pull request — nothing it does lands anywhere a person has not read it first.
Result
Point it at a Playwright repository and it reads the suite, runs it sandboxed, edits what is broken and opens a pull request with the fix and the run that proves it. Assertions sit outside what it may touch, so a green run means the flow passes rather than that the test stopped asking.