backtest-coroner
An agent skill for Claude Code, Codex and Cursor, plus a Python CLI with no dependencies. Ask your assistant whether a backtest is ready, and instead of encouragement it runs a falsification protocol: holdouts from a different market era, the breakeven cost the edge could survive, a random-direction placebo, a planted-edge probe that proves the engine can see an edge at all, and for prop-firm accounts, a comparison with a coin flip under the firm's own rules. It ends with a verdict that cites its numbers.
- Python
- Agent Skills
- Claude Code plugins
- GitHub Actions
Where the rules came from
The trading bot started with an opening-range breakout and a model on top. Taking it seriously meant writing down each new idea as a protocol before running it: the cells, the gates, the holdout, what would count as a failure. About 99 of those families later, nothing day-trade-legal had survived, and the useful output was the list of ways a backtest lies.
Each rule is kept with the measurement that produced it, because a rule without its number gets argued away within a few sessions. A breakout that read t = +3.73 over 2021-2024 returned nothing on 10.5 unseen years. A grid that looked like an edge on 13 instruments turned out to score its target on the same bar as its entry, with 71.7% of the wins printed before the trade existed.
Catching the false negatives too
Placebos and lookahead controls only protect against results that look better than they are. A bug that destroys a real edge looks exactly like an honest negative, and nothing flags it. So the CLI plants a known drift into the price series after each signal, and the user's own engine has to find it before any negative result counts.
The prop-firm null is not zero
Funded-account challenges have their own payoff geometry. Under a static floor with the balance reset after each payout, a book trading random direction withdrew more than its fee. Under a trailing drawdown it lost money at every size. The prop command runs the user's trades and a random-direction twin through the same account program, re-buying blown accounts, and reports the difference with 2,000 seeds.
What it does not do
It does not find strategies. It can say a strategy is dead and why; it never promises one is alive.