{adamcoding}
← The Machine and the Market
Part III
17
Chapter 17

Testing

The problem

Your moving average returns numbers. Are they the right numbers?

You could eyeball a few. You did that in Chapter 8, and it worked because the data was small enough to check by hand. Now you have ten functions, and every change to one might break another.

A test is a function

Put it in a file ending _test.go, in the same package:

go
package market

import "testing"

func TestMovingAverageSimple(t *testing.T) {
	prices := []float64{10, 20, 30, 40, 50}

	got, err := MovingAverage(prices, 3)
	if err != nil {
		t.Fatalf("unexpected error: %v", err)
	}

	want := []float64{20, 30, 40}      // (10+20+30)/3, (20+30+40)/3, ...
	if len(got) != len(want) {
		t.Fatalf("got %d values, want %d", len(got), len(want))
	}
	for i := range want {
		if math.Abs(got[i]-want[i]) > 1e-9 {
			t.Errorf("index %d: got %f, want %f", i, got[i], want[i])
		}
	}
}

Run:

go test ./...
go test -v ./market

t.Errorf records a failure and continues. t.Fatalf records it and stops the test immediately - use it when continuing would just produce noise, like after a length mismatch.

Note the tolerance comparison. Chapter 2's rule applies in tests more than anywhere: never assert exact equality on floats.

Table-driven tests

The Go idiom, and it's genuinely better than the alternatives:

go
func TestPositionSize(t *testing.T) {
	tests := []struct {
		name     string
		equity   float64
		riskPct  float64
		entry    float64
		stop     float64
		want     float64
		wantErr  bool
	}{
		{"long, 1% risk", 10000, 1.0, 100, 98, 50, false},
		{"short, 1% risk", 10000, 1.0, 100, 102, 50, false},
		{"2% risk doubles size", 10000, 2.0, 100, 98, 100, false},
		{"wider stop, smaller size", 10000, 1.0, 100, 96, 25, false},
		{"zero stop distance", 10000, 1.0, 100, 100, 0, true},
	}

	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			got, err := PositionSize(tt.equity, tt.riskPct, tt.entry, tt.stop)

			if tt.wantErr {
				if err == nil {
					t.Fatal("expected an error, got none")
				}
				return
			}
			if err != nil {
				t.Fatalf("unexpected error: %v", err)
			}
			if math.Abs(got-tt.want) > 1e-9 {
				t.Errorf("got %f, want %f", got, tt.want)
			}
		})
	}
}

The cases are data. Adding one is a line, not a function. t.Run gives each a name, so a failure says exactly which case broke:

--- FAIL: TestPositionSize/wider_stop,_smaller_size (0.00s)
    sizing_test.go:31: got 50.000000, want 25.000000

Notice that a well-chosen table is documentation. Reading those five lines tells you what PositionSize does more clearly than any comment: doubling risk doubles size, widening the stop shrinks it, a zero stop is an error.

Testing errors

go
func TestMovingAverageInsufficientData(t *testing.T) {
	_, err := MovingAverage([]float64{1, 2, 3}, 10)
	if !errors.Is(err, ErrInsufficientData) {
		t.Errorf("got %v, want ErrInsufficientData", err)
	}
}

Test the kind of error with errors.Is, not the message text. Message text is for humans and changes; sentinel errors are the contract.

See It Work: what haven't you tested?

go test -cover ./...
ok   github.com/yourname/tradebot/market    0.004s   coverage: 68.2% of statements

Which 68%? Find out:

go test -coverprofile=coverage.out ./market
go tool cover -html=coverage.out

A browser opens with your source, green for covered lines and red for lines no test has ever executed.

Do this once and you will find something uncomfortable. Usually it's the error branches - every guard clause you wrote in Chapter 6, never run. Those are exactly the paths that matter when things go wrong, and they're the ones nobody tests.

Don't chase 100%. Chase "no red on anything that handles money or errors."

See It Work: let the computer find your bugs

Go has built-in fuzzing, and it's the closest thing to magic in the toolchain:

go
func FuzzParsePrice(f *testing.F) {
	f.Add("100.50")
	f.Add("0.001")
	f.Add("-5")
	f.Add("")

	f.Fuzz(func(t *testing.T, input string) {
		price, err := ParsePrice(input)
		if err != nil {
			return          // rejecting bad input is correct
		}
		// If it accepted the input, these must hold.
		if price <= 0 {
			t.Errorf("accepted %q and returned non-positive %f", input, price)
		}
		if math.IsNaN(price) || math.IsInf(price, 0) {
			t.Errorf("accepted %q and returned %f", input, price)
		}
	})
}
go test -fuzz=FuzzParsePrice ./market

Go now generates millions of inputs, mutating your seed examples, hunting for something that breaks your assertions. It will try "1e999", "NaN", "+Inf", "0x1p-1", strings of null bytes, and things you'd never have thought of.

When it finds a failure it writes the input to testdata/fuzz/ and it becomes a permanent regression test.

You are not writing test cases here. You are writing rules that must always hold, and letting the machine attack them. For anything that parses external input - and market data is all external input - this finds bugs that hand-written tests never will. Try it on ParsePrice and see whether "NaN" gets through.

Known-answer tests

For the backtester specifically, from Trading Systems for Software Engineers Chapter 7:

go
func TestBacktestStraightLine(t *testing.T) {
	// Price rises by exactly 1.00 per bar. A long held for 10 bars,
	// with no costs, must make exactly 10.00 per unit.
	bars := makeLinearBars(100.0, 1.0, 20)
	result := RunBacktest(BuyAndHoldFor(10), bars, NoCosts{}, OneUnit{})

	if math.Abs(result.NetPnL-10.0) > 1e-9 {
		t.Errorf("got %f, want 10.0", result.NetPnL)
	}
}

func TestCostsAreCharged(t *testing.T) {
	// Flat price, enter and exit each bar. P&L must equal minus the costs.
	bars := makeFlatBars(100.0, 50)
	costs := PercentCosts{Commission: 0.001}
	result := RunBacktest(EnterExitEveryBar{}, bars, costs, OneUnit{})

	expected := -float64(result.TradeCount) * 2 * 100.0 * 0.001
	if math.Abs(result.NetPnL-expected) > 1e-6 {
		t.Errorf("got %f, want %f", result.NetPnL, expected)
	}
}

Synthetic data whose correct answer you can compute by hand. The second test catches an entire family of bugs - silently dropped costs - and it's the one people don't write.

Exercises

17.1 Write table-driven tests for Bar.Range(), Body(), IsUp() and IsDoji(), including zero-range bars.

17.2 Test RingBuffer including the wrap-around: fill it past capacity and assert Values() returns the right elements in the right order.

17.3 Run go test -coverprofile on market and open the HTML report. List every red branch. Write tests for the three you think matter most.

17.4 Write a fuzz test for parseRow. Seed it with one good row and let it run for a minute. Did it find anything?

17.5 Write the two known-answer backtester tests above and make them pass.

17.6 Write the null test: random entries with realistic costs, over 200 seeds, must have negative mean expectancy. Assert it.

17.7 Harder. Write a test asserting P&L conservation - the sum of individual trade P&Ls equals the change in equity - and run it over randomly generated strategies and price series.

Solutions

17.2 The wrap-around case is the one that breaks. Fill a 4-slot buffer with 6 values and assert Values() gives values 3, 4, 5, 6 in that order. If you wrote Values() naively it'll return them starting from index 0, which is wrong once pos has moved.

17.4 A common find is that strconv.ParseFloat happily accepts "NaN", "Inf" and "1e999". "NaN" is the nastiest - it propagates silently through every subsequent calculation, and NaN != NaN, so equality checks won't catch it either. Reject non-finite values explicitly.

17.6 This is the harness test from the other book. If it fails, either your costs aren't being applied or you have lookahead somewhere. Both are worth knowing about before you trust a single result.