Plain definitions of the terms used across these guides.
A simulation of a strategy or model on historical data. Useful as a sanity check, but easy to mislead yourself with if costs or future information are handled carelessly.
A testing method that repeatedly splits data to estimate how a model generalizes. For time series it must respect the order of time to avoid using future data.
Using information in training that would not have been available at decision time. A leading cause of results that look excellent and then fail live.
The decline from a peak to a trough in a strategy value. The depth and length of drawdowns describe the real pain of holding a strategy.
A measured input fed into a model, such as a return, a volatility estimate, or a sentiment score. Features define what a model can notice.
The craft of designing model inputs. Often more decisive than the choice of model itself.
A tree ensemble method that builds models in sequence to correct earlier errors. Popular for tabular data and still prone to overfitting.
A configuration value set before training, such as model depth. Tuning many of them on the same data is a common path to overfitting.
The known answer attached to a training example in supervised learning. Labeling future market moves is inherently noisy.
A form of data leakage where a calculation uses information from the future relative to the decision point.
A flexible model of connected layers. Powerful but data hungry and prone to overfitting on small, noisy market data.
Random variation in data that carries no durable signal. Markets contain a great deal of it, which is why overfitting is so easy.
Data the model never saw during training, used to estimate honest performance. Results on seen data are not trustworthy.
When a model learns the quirks of its training data rather than a durable pattern, scoring well on the past and poorly on new data.
An ensemble of decision trees averaged together. A reasonable workhorse for structured data that still needs careful validation.
Techniques that penalize complexity to reduce overfitting, as in ridge and lasso regression.
Training an agent through rewards from actions. Research heavy and easily fooled by imperfect simulations.
Estimating the mood of text such as news or filings. The output is an uncertain estimate, not a fact about what prices will do.
A risk adjusted measure relating return to volatility. One of several numbers used to judge a strategy, and gameable on its own.
A rule or condition that flags a moment of possible interest. It marks a point in time and carries no guarantee of outcome.
The difference between an expected trade price and the price actually obtained, especially when orders move the market.
A property where statistical behavior stays stable over time. Raw prices are usually not stationary, which is why returns are often used instead.
Learning a mapping from inputs to known labels. The main family used to attempt direction prediction, with sharp limits in noisy markets.
Testing only on instruments that still exist, which quietly inflates results by ignoring those that failed.
Finding structure in unlabeled data, such as grouping similar instruments. Often more honest because it makes fewer predictive promises.
A backtest that retrains and tests in moving time windows to mimic real use. Stronger than a single fixed split.