Backtest → Paper → Live

Three transitions, each of which reveals a different category of problem. Skipping any of them means discovering that category with real money.

What each stage actually tests

Backtest → paper trading tests your infrastructure, not your edge. Paper trading cannot validate a strategy - the fills are simulated, so it inherits most of the optimism from Chapter 7. What it does test:

• Does your data pipeline work in real time, or only on historical files?
• Does your signal fire when you expect, in wall-clock time?
• Do orders get accepted, or rejected for reasons your backtest never modelled (minimum size, tick size, price bands, insufficient margin)?
• Does the system survive a restart, a disconnect, a data gap?
• Is your live signal the same as your backtest signal on the same bar? This is the big one. Discrepancies here are almost always a bug in your backtest, and finding them is worth the entire paper stage.

Paper → live small tests execution economics and yourself. This is the first stage with real information about fill quality:

• What slippage do you actually get, versus what you assumed?
• Do your limit orders fill, or do you get adversely selected?
• Are borrow costs and availability what you modelled?
• What do you do at 2 a.m. when the position is down and you can see it?

Live small → live sized tests capacity and psychology under weight. Market impact appears, and so does the difference between a 1% drawdown and one that matters to you.

Set acceptance criteria before each transition

This is Chapter 9's pre-registration applied to deployment. Write down, before starting the stage:

• How many trades the stage runs for. Measure in trades, not weeks - 30-50 minimum per stage, and note from Chapter 4 that this is not enough to validate an edge, only to detect gross breakage.
• What result advances, what result halts, what result returns you to research.
• What degradation is acceptable. Live will be worse than backtest. Decide in advance how much worse is tolerable, because deciding afterwards means deciding while motivated.

A defensible rule: compute the confidence interval around your backtest expectancy. If live expectancy falls below its lower bound after N trades, stop and investigate. This converts "it feels like it's not working" into a decision with a threshold.

Measure implementation shortfall

The single most valuable instrument you can build at this stage: for every live trade, record what the backtest would have assumed, and diff it.

from dataclasses import dataclass


@dataclass
class ExecutionRecord:
    signal_time: float          # when the strategy decided
    signal_price: float         # price at decision
    order_sent_time: float
    fill_time: float
    fill_price: float
    intended_qty: float
    filled_qty: float
    side: int                   # +1 long, -1 short
    backtest_assumed_price: float


def implementation_shortfall(r: ExecutionRecord) -> dict:
    """
    Decompose the gap between backtest and reality into its causes,
    so you know which one to fix.
    """
    decision_slippage = r.side * (r.fill_price - r.signal_price) / r.signal_price
    model_error = r.side * (r.fill_price - r.backtest_assumed_price) / r.backtest_assumed_price
    fill_ratio = r.filled_qty / r.intended_qty if r.intended_qty else 0.0

    return {
        "decision_to_fill_bps": decision_slippage * 10_000,
        "vs_backtest_bps": model_error * 10_000,
        "latency_ms": (r.fill_time - r.signal_time) * 1000,
        "fill_ratio": fill_ratio,
    }

Aggregate vs_backtest_bps across trades. If the mean is materially negative, your fill model is optimistic and every backtest you have ever run is overstated by that amount. Feed the measured number back into the backtester and re-run your validation. Most people never close this loop, which is why their live results are a permanent surprise.

Watch fill_ratio too. If your limit orders fill only when the market is about to move against you, your realised fill rate will look fine while your realised P&L doesn't - that's adverse selection from Chapter 15, and it shows up here first.

Expect degradation, and know its sources

Live underperforms backtest essentially always. The honest sources, roughly in order of size:

1. Overfitting - the largest contributor, and the one Parts II covered at length.
2. Optimistic fill assumptions.
3. Costs not modelled - financing, borrow, exchange fees you forgot, tax.
4. Regime change between your sample and now.
5. Your own intervention.

If live is dramatically worse than backtest, suspect the first item before blaming the market. A strategy that degrades by 80% on deployment was probably never a strategy.