Produce quant hypothesis and backtest review with inputs, checks, findings and limitations.
2026-10-26.2 · 60 minutes estimated; work in several sessions
Saved on this device; sign in to import explicitly.
Prerequisites and goal
Choose a target role and explain probability and conditional probability. Diagnose first, then revisit the checklist as needed.
Start with diagnostics; skip explanations you can already apply independently.
Probability and conditional probability
Use probability and conditional probability to make decisions inspectable.
Use a tree diagram for a two-stage conditional event.
Teaching example: P(A and B)=P(A)×P(B|A).. Predict the result, independently check the key step, and compare these assumptions with the actual role. Do not use demonstration results as real-world advice.
Expectation, variance and odds
Use expectation, variance and odds to make decisions inspectable.
Calculate expected payoff and distinguish it from variability.
Teaching example: A 50% chance of +4 and 50% of −2 gives expected payoff 1.. Predict the result, independently check the key step, and compare these assumptions with the actual role. Do not use demonstration results as real-world advice.
Research code and data checks
Use research code and data checks to make decisions inspectable.
Write pseudocode for missing values, timestamps and duplicates.
Teaching example: Misaligned timestamps can create spurious predictability.. Predict the result, independently check the key step, and compare these assumptions with the actual role. Do not use demonstration results as real-world advice.
Signals and leakage
Use signals and leakage to make decisions inspectable.
Mark training, validation and test periods and check future information.
Teaching example: Using information released after the close leaks future information.. Predict the result, independently check the key step, and compare these assumptions with the actual role. Do not use demonstration results as real-world advice.
Backtests and costs
Use backtests and costs to make decisions inspectable.
Calculate returns, costs and turnover on a fictional trade log.
Teaching example: Gross profit 30 less cost 12 gives net profit 18.. Predict the result, independently check the key step, and compare these assumptions with the actual role. Do not use demonstration results as real-world advice.
Robustness and repeated trials
Use robustness and repeated trials to make decisions inspectable.
Log every parameter trial and preserve an untouched test set.
Teaching example: Selecting the best of many trials can overstate performance.. Predict the result, independently check the key step, and compare these assumptions with the actual role. Do not use demonstration results as real-world advice.
Research reporting and risk review
Use research reporting and risk review to make decisions inspectable.
Report hypotheses, data scope, costs and failure scenarios.
Teaching example: A backtest is a historical test, not a promise of future returns.. Predict the result, independently check the key step, and compare these assumptions with the actual role. Do not use demonstration results as real-world advice.
Worked example · fictional
From a problem to inspectable work
Synthetic strategy: 100 assumed independent days, mean daily return 0.1%, daily SD 1%. Mean-test illustration only, before costs or multiple selection.
SE = 1%/√100 = 0.1%; t = 0.1%/0.1% = 1. Daily SD is not the mean's SE.
Further judgment: This strategy was selected from 500 backtests. How review it?
Audit selection, leakage, costs and an independent period; one t value is insufficient
Deliver quant hypothesis and backtest review: the calculation reference is 1; the recommendation must respect Audit selection, leakage, costs and an independent period; one t value is insufficient. Separate numbers, inference and unknowns.
Failure case and repair
In Signals and leakage, copy tool output without definitions, then retain the conclusion when asked: This strategy was selected from 500 backtests. How review it?
Even a result of 1 does not explain when Audit selection, leakage, costs and an independent period; one t value is insufficient is supported.
SE = 1%/√100 = 0.1%; t = 0.1%/0.1% = 1. Daily SD is not the mean's SE. Then test assumptions in Robustness and repeated trials.