Backtesting is a term used in modeling to refer to testing a predictive model on historical data. Backtesting is a type of retrodiction, and a special type of cross-validation applied to previous time period.
What is ‘Backtesting’
Backtesting is the process of testing a trading strategy on relevant historical data to ensure its viability before the trader risks any actual capital. A trader can simulate the trading of a strategy over an appropriate period of time and analyze the results for the levels of profitability and risk.
If the results meet the necessary criteria that are acceptable to the trader, the strategy can then be implemented with some degree of confidence that it will result in profits. If the results are less favorable, the strategy can be modified, adjusted and optimized to achieve the desired results, or it can be completely scrapped.
When done correctly, backtesting can be an invaluable tool for making decisions on whether to utilize a trading strategy. The sample time period on which a backtest is performed on is critical. The duration of the sample time period should be long enough to include periods of varying market conditions including uptrends, downtrends and range-bound trading. Performing a test on only one type of market condition may yield unique results that may not function well in other market conditions, which may lead to false conclusions.
Keeping it Real
A backtest should reflect reality to the best extent possible. Trading costs that may otherwise be considered to be negligible by traders when analyzed individually may have a significant impact when the aggregate cost is calculated over the entire backtesting period. These costs include commissions, spreads and slippage, and they could determine the difference between whether a trading strategy is profitable or not. Most backtesting software packages include methods to account for these costs.
- Emerging market sovereign bond spreads: Estimation and back-testing – www.sciencedirect.com [PDF]
- Pseudo-mathematics and financial charlatanism: The effects of backtest overfitting on out-of-sample performance – www.ams.org [PDF]
- Backtesting value-at-risk: A duration-based approach – academic.oup.com [PDF]
- A backtesting protocol in the era of machine learning – jfds.pm-research.com [PDF]
- On back‐testing “zero‐investment” strategies – www.jstor.org [PDF]
- Frontiers in VaR forecasting and backtesting – www.sciencedirect.com [PDF]
- Backtesting var models: An expected shortfall approach – papers.ssrn.com [PDF]
- Backtesting VaR Models: A Τwo-Stage Procedure – papers.ssrn.com [PDF]