The speed knob that made every mock global

For two days, our vibes-diy CLI would not publish to npm. Three ships in a row went out. Each one looked green. Each one quietly failed to move the registry one version forward. Nobody had touched the CLI. The code it was supposed to publish was fine. And yet the release gate sat there, red, on an error that had nothing to do with the thing being released:

TypeError: toast.dismiss is not a function

There is exactly one villain in this story, and it is a config flag you turn on to make tests faster.

The knob

Vitest runs each test file in its own isolated context by default: a fresh module registry, mocks that die with the file. That isolation costs setup time, so Vitest gives you a knob to turn it off — isolate: false — and our app suite had it on for speed. Worth doing. Tests got faster.

Here is what the flag actually does, stated plainly, because the whole outage lives in the gap between how it's marketed and what it means: with isolate: false, every test file in a worker shares one module registry. Mock a module in file A and the mock is still installed when file B runs in the same worker. The last file to register a mock wins, and everyone else inherits it whether they wanted it or not.

That is not a test bug. That is shared global state you opted into — and you opted into it for a reason that had nothing to do with correctness.

Move one: fix the mock, not the symptom

The toast.dismiss error came from a module three test files had each mocked as { error, success } — a partial stand-in for react-hot-toast with the two methods those files happened to touch. No dismiss. Not even callable in the ways the real module is.

Under isolation, that partial mock would have been a local shortcut, scoped to its file and harmless. Under isolate: false it leaked into a sibling — a use-app-access test that, on an async resolve, called toast.dismiss. The method wasn't there, because the leaked mock never had it. The suite exploded, the publish gate went red, and the CLI — utterly innocent — stopped shipping.

The tempting fix is to complete the mock: add dismiss, add the missing methods, make the stand-in match the real thing. We deleted it instead. Nothing in those files asserted anything about toasts; the mocks existed only to keep notifications quiet during the test. And the real module is leak-proof by construction — it has every method, so there is nothing for a sibling file to trip over. The cleanest mock, it turns out, is often no mock at all: if you're not asserting on a thing, the real thing can't drift out from under you.

That took the failure count from a dozen down to two — and the last two taught the same lesson from the other direction. Both were in a community-gallery test whose "falls back to the static gallery" cases asserted the fallback by a data-testid that the file's own mock of the gallery component rendered. But two other files mock that same component differently — one returns nothing, one renders a little select button — and under isolate: false, whichever ran in the same worker clobbered the expected testid. The fix was to stop asserting on a stub that another file also owns, and assert instead on the component's own real output: the "yours to remix" caption it actually renders when it falls back. No leaked mock can overwrite that, because the component under test owns it.

Two leaks, one shape: a partial or private mock, made global by a speed flag. Both got fixed at the source, in the tests, where the shared state actually lived.

Move two: gate the release on proof, not a re-roll

Fixing the mocks fixed those leaks. It did nothing about the deeper problem, which was structural: the npm-publish job was configured to re-run the entire app test suite from scratch as its gate, and then publish. So the CLI's release rode on the health of every app test — including every latent isolation bug nobody had hit yet. A publish is only ever as reliable as the flakiest thing you make it wait on, and we had wired it to wait on the flakiest thing we owned.

The first instinct — just skip the tests, the CLI's own tests already pass — was wrong, and review caught why: the suite's project config pulls in the CLI and package tests too, so a tag cut on an unverified commit could ship genuinely untested code. Skipping the gate trades a flaky-red problem for a silent-green one, which is worse.

The right shape was already in the repo, in a sibling deploy workflow: don't re-roll the dice, reuse the proof you already have. A tag whose commit already earned a green check on the normal path publishes fast and quiet, on the strength of that existing check. A tag on a commit with no green check fails closed — it runs the full suite before it's allowed anywhere near the registry. "Lean out the release gate" never meant "remove the gate." It meant reuse proof when you have it and refuse to ship when you don't.

(A bonus fell out of the same cleanup: the job had been spinning up a privileged Docker-in-Docker service to run those tests, whose boot and teardown buried the real publish output under thousands of lines. The runner had Docker natively the whole time. The scaffolding was pure noise.)

Move three: a probe that measured, then deleted itself

There was one loose thread. If isolate: false was the villain, was it even buying us the speed we'd traded correctness for? The suite had recently moved to smaller two-core runners, and nobody actually knew the right worker count for them.

So we built a throwaway. A dispatch-only CI workflow whose only job was to sweep the worker count across a range and report wall-clock and reliability at each setting — with a temporary env knob wired in for the sole purpose of letting the sweep vary the value without editing a file. We expected it to find a faster worker count.

It found the opposite, and something better. Wall-clock was essentially flat across worker counts — the suite is bound by setup and async work, not parallelism, so there was no speed to win. The real yield was a reliability gradient: the sweep reproduced exactly the kind of cross-file state bleed this whole saga was about, which pointed straight at the leak we then fixed at the source.

Then the probe did the most important thing a tool like it can do: it left. We deleted the workflow, deleted the env knob that existed only to feed it, and set the worker count back to a plain literal — with a one-line comment recording why that number, so nobody re-tunes it next quarter hoping for a speed win we've already proven isn't there. A measurement tool with a built-in expiry: measure, act on the finding, then remove the apparatus and leave behind only "we already checked this, here's the answer." Tools that know how to leave beat tools that linger as dead weight.

What the knob was really teaching

Shared state you opted into for speed is still shared state. isolate: false never advertised itself as "make every mock a global," but that is precisely what it does, and a global that arrives through a config flag is the most dangerous kind — invisible at the call site, felt three files away, blamed on whatever happened to run last.

The fix is never at the blast site. It's at the source: a mock that can't leak, an assertion on something the leaker doesn't own. And your release gate should be the last place that shared state can reach — it should reuse a proof that already exists, and fail closed when it doesn't, so a flake in a far corner of the suite can never again quietly stop the thing you were actually trying to ship.