Product / Fixes and re-tests

Ship the fix. Re-test what it touched.

Scenarios coding agents lose you become fixes in Actions, and a re-test of each shipped fix records the pick rate before and after.

Fixes from losses

When another tool, or one the agent builds itself, clearly beats you in a scenario, the loss becomes a ranked fix in Actions. Each fix carries the runs behind it.

Actions · Agent Picks

01
Fix acme.dev/docs/express returning 404
acme.dev/docs/express returned 404 in 9 runs across 2 scenarios, with no later success recorded.
Live
- ExpressAdd an integrationCodexgpt-6.1-sol
- ExpressUse a hosted serviceCodexgpt-6.1-sol
Copy as a coding-agent prompt
02
Improve how often agents pick Acme in Next.js
1 of 8 completed runs picked Acme.
Suggested
- Next.jsAdd an integrationCodexgpt-6.1-sol
03
Fix the @acme/rails install agents tried in Rails
Installing @acme/rails failed in 4 runs across 2 scenarios, with no later successful install.
Suggested
- RailsAdd an integrationClaude Codeclaude-sonnet-5-5
- RailsUse a hosted serviceClaude Codeclaude-sonnet-5-5

Re-test what the fix touched

Once a fix is live, re-test only the scenarios it lost in, with the same test app, request, agent and model as the original test.

Pay only for the runs you need

See the credit estimate and confirm it before a re-test runs. It stops once every scenario has clearly moved, and the credits for runs it skips come back.

Before and after pick rates

See each re-tested scenario's pick rate before and after, with the likely range. The result goes to Results and shows what moved, not what caused it.

Make coding agents reach for you first.