Shiftwright migrates Selenium and Cypress suites to Playwright: an agentic workflow does the migration, and you approve the pull request. This post is the build story — the pieces, what happens to one file, a real run beat by beat, and what broke on the way. Every label names its wire, so you can check it against the code.
We created Shiftwright and this post for the purposes of entering the All Things Agentic Hackathon. Published 2026-08-30.
Legacy Selenium suites rot: brittle XPath, arbitrary time.sleep(), CI
flakiness. Teams want Playwright's auto-waiting and parallelism, and they don't migrate,
because it's weeks of work that ships no features. Public GitHub holds ~1.78M Selenium and
Cypress source files; of the Python ones, 0.2% also import Playwright. Migrating is a
rewrite per test, so it doesn't happen.
Our starting position: migration is a runtime problem. The code has to run, the failure has to be read against live DOM, and the fix has to hold. So we built a system that executes what it writes — every migrated test runs in a container first, and what lands in your pull request is a suite that already passes.
The stack, up front: one Google ADK SequentialAgent with four agents (Planner, Modernizer, Diagnoser, Verifier), all on Gemini 3.5 Flash over Vertex AI, the Diagnoser multimodal; a private MCP server on Cloud Run that runs every test; Firestore as message bus and run record; Vertex AI Memory Bank as the cross-repository memory tier; three Cloud Run services in total.
What goes in and out, the agents that decide, the sandbox that executes, the memory both write to. No step order — that is next.
ADK SequentialAgent · all four on gemini-3.5-flash over Vertex AI, the Diagnoser multimodal
Isolated sandbox driven over JSON-RPC — never your machine, never your branches. Every call carries an ID token.
What one test learns, every later test reuses — the reason healing effort trends down across a run. Verified repairs also persist to Vertex AI Memory Bank, so the next repository starts with what earlier ones worked out.
Rewrite, run it for real, and on failure diagnose from the screenshot and DOM rather than the source. At most three attempts; every file ends in one of four recorded states.
The failure is the input, not the source that already failed. Each attempt gets a different brief, so a second pass knows what the first missed.
Green is not the finish line. The suite is timed against the one it replaces, and nothing lands without a person approving the diff.
16 beats in the order a run plays them, timed as they actually occurred.
The wizard writes a request document. No server call — the browser talks to Firestore.
A migration that edits assertions has destroyed the thing the
suite was for. Our first run against a repo we had not designed for came out
16/16. It should have been 13/16. The Diagnoser had made three genuinely-failing tests
pass by rewriting what they asserted — each diagnosis right that the test was wrong, and
each out of scope. patch_guard.py now extracts every test's expectation set
before and after each patch, and refuses any change to a comparison operator, an expected
literal, or the assertion count.
A verdict is only worth what the runner behind it did. Adding Java and Cypress created three ways to report an outcome nobody had measured. The
markdown-fence stripper matched ```python and nothing else, so ```java survived into the file handed to Maven — every Java file failed to
compile, and the benchmark timed those failures as a 5.1× "speedup". Playwright's default testMatch filtered out migrated Cypress specs named *.cy.ts,
which read as "no tests found". Both are now rules: pair only tests that passed on both
sides, and a runner that collected nothing reports harnessError rather than
a verdict.
A knowledge base can be dead and look alive. Ours had one
writer storing Playwright expressions and one reader querying By.ID=… strings — two vocabularies that could never match, so "reusing N locators" had never once
fired, while counters incremented locally for documents that were never written.
Serial migration looked like a framework constraint and was
not. The Modernizer is an LlmAgent driven through session state,
and concurrent turns race on it. The constraint was the single session, not the
agent. One ADK session per file, and the same agent parallelizes untouched.
Give an agent an oracle and generation stops being the hard part. Scope does. Knowing which failures are yours to fix is the real problem, and enforcing that boundary in code worked where prompting did not. The guard costs no legitimate heal, and it is why the third-party corpus lands at 13-of-16.
Structure beats model size. A curriculum, a typed hand-off and a scoped memory made Gemini 3.5 Flash sufficient for every role, multimodal diagnosis included. The wins came from the shape of the workflow.
MCP made the sandbox modular. Standardising execution on JSON-RPC tools decoupled reasoning from the environment. Running untrusted test code in an isolated Cloud Run container became a deployment detail rather than an agent concern.
Two memory tiers earn their keep differently. Firestore stays authoritative inside a migration, because a locator this run already learned has to come back verbatim. Vertex AI Memory Bank answers across repositories — what has ever worked for a widget like this, in any suite this project has migrated.
Browser and worker never call each other. Both read and write Firestore, and the dashboard subscribes — so the UI has no backend to lose.
Tests run in a private Cloud Run container, driven over JSON-RPC with an ID token — never a developer machine, never your branch.
Approve the plan, approve the draft. The most that reaches your repository is a draft pull request on a new branch.