Validation Checklist

Work top to bottom. Each stage assumes the previous one passed. Chapter references in brackets.

B.1 Data

[ ] Single vendor, single instrument specification, documented [6]
[ ] Raw data stored immutably; all cleaning produces derived copies [6]
[ ] Dataset hashed, and the hash recorded with every result [6]
[ ] Timestamps in UTC; bar-labelling convention (open vs close) asserted in code [6]
[ ] Daylight-saving handling verified for any session filter [6]
[ ] validate_ohlcv passes: monotonic, no duplicates, OHLC internally consistent, gaps counted [6]
[ ] Outliers flagged and inspected, not silently deleted [6]
[ ] Adjustment status known: adjusted for returns, unadjusted for price-level rules [6]
[ ] For futures: roll method and roll dates documented [6]
[ ] For equity universes: point-in-time membership, delisted names included [6]

[ ] Strategy receives a view that cannot return future data [7]
[ ] Known-answer tests pass (straight-line P&L exact to the cent) [7]
[ ] Cost-accounting test passes (flat price, enter/exit → P&L equals minus costs) [7]
[ ] P&L conservation test passes (sum of trades equals equity change) [7]
[ ] Null test passes: random entries lose approximately the cost of trading [7]
[ ] Results are byte-identical across runs with the same seed [7]
[ ] Intrabar ambiguity resolution is explicit and known, not defaulted [7]
[ ] Optimistic / realistic / pessimistic fill models all run; the gap is measured [7]

[ ] The mechanism is written down: who is on the other side, and why they lose [2, 11-15]
[ ] Costs per round trip calculated as a percentage of the average winning trade [3]
[ ] Expected holding period is consistent with the effect being exploited [12]
[ ] Parameter count recorded; every one justified [8]
[ ] Hidden degrees of freedom counted: instruments tried, timeframes tried, ideas abandoned [8]

[ ] Trade count sufficient for the effect size sought (see the table in [4])
[ ] Expectancy reported in R, with its standard error and t-statistic [4]
[ ] Result survives removal of the single best trade [8]
[ ] Parameter sweeps show a plateau, not a peak; the chosen value is mid-plateau [8]
[ ] Two-parameter heatmap inspected for the primary pair [8]
[ ] ±10% parameter jitter keeps the outcome distribution positive [8]
[ ] Total number of configurations tested is recorded and reported alongside the result [8]
[ ] Permutation test run; p-value adjusted for the number of configurations tested [9]
[ ] Monte Carlo trade shuffling run; drawdown distribution inspected (treat as a floor) [9]
[ ] Out-of-sample run once, on data never previously inspected [9]
[ ] Walk-forward analysis complete; parameter stability across windows assessed [9]
[ ] Walk-forward efficiency computed [9]
[ ] Robustness across instruments, timeframes and regimes checked [9]

[ ] Risk per trade derived from a stated drawdown budget, not chosen by feel [10]
[ ] Drawdown distribution simulated; the 95th percentile is tolerable [4, 10]
[ ] Portfolio heat cap defined and enforced in code [10]
[ ] Correlations estimated from crisis periods, not full sample [10]
[ ] Position sizing reflects uncertainty in the edge estimate, not just its size [4]

[ ] Reconciliation on startup and periodically; divergence halts trading [17]
[ ] Client-generated idempotency key on every order [17]
[ ] Position state recoverable from the exchange, not local storage [17]
[ ] Data staleness watchdog halts trading past a threshold [17]
[ ] Clock synchronised; drift alerted [17]
[ ] Kill switch exists, is reachable from a phone, is out-of-process, and has been tested in production [17]
[ ] API keys: withdrawal disabled, IP allowlisted, scoped per environment [17]
[ ] Every decision logged with its inputs, not just its output [17]
[ ] Alerts on: liveness, staleness, reconciliation, rejection rate, drawdown, and unusually low trade frequency [17]
[ ] Shadow mode diff run before any logic deployment [17]