Quantitative Backtesting With Python: What the Tutorials Don't Tell You

The average Python backtesting tutorial covers roughly the same ground: load a CSV of historical data, compute a moving average or RSI indicator, define entry and exit rules, and plot an equity curve. If the curve goes up-right, the tutorial calls it a success. What it almost never covers is the list of silent mistakes that make that curve meaningless. Those mistakes are not edge cases — they are the default. If you just run vectorbt or backtrader on a strategy idea without thinking about each pitfall below, your backtest is probably lying to you. Here's what they leave out.
Look-Ahead Bias: One Line Can Change Everything
Look-ahead bias is when your code uses information it couldn't have had at the time of the trade. The tutorials usually mention it in passing. What they skip is showing you how extreme the effect is in practice.
Consider a trivially broken strategy: “be long SPY on days when SPY's return is positive.” Obviously useless in reality, because you only know the close after the market shuts. But in code, if you compute the signal from today's close and apply it to today's return without a shift, the backtest sees the future. On SPY from 2010 to 2025, this one-line bug produces a CAGR of 164.4% and a Sharpe ratio of 9.36. The correct version — shifting the signal by one bar so you trade on yesterday's information — produces 4.7% CAGR and a Sharpe of 0.47. Same idea. Same data. One .shift(1) apart.
The subtle version is harder to catch: rolling window calculations that include the current bar in their lookback, fillna calls that propagate future values backwards, or indicator libraries that compute signals end-of-bar and you assume they fired before the bar closed. Always verify that a signal computed at time t only drives a trade at t+1.

Survivorship Bias: Your Universe Is Already Cherry-Picked
If you backtest on the current S&P 500 constituents, you are backtesting on 500 companies that survived long enough to still be in the index today. Every company that went bankrupt, got delisted, or was acquired at a loss has already disappeared from your dataset before you even start. Your strategy never held Lehman Brothers, Enron, or any of the dozens of energy, biotech, or retail names that imploded in the years your test covers. The winners stay; the losers vanish. The result is a universe that is, by construction, pre-screened for success.
This matters most for strategies that touch individual equities with momentum or quality signals — exactly the strategies most likely to concentrate in strong recent performers. A momentum backtest on survivorship-biased historical data can overstate annualized returns by 1–3 percentage points or more, depending on how volatile the universe is and how far back you test. Point-in-time constituent data (the list of what was actually in the index on each historical date) is the fix, and most free data sources don't provide it.
Overfitting and Data Snooping: The Test Isn't Independent
Overfitting in a backtesting context — sometimes called curve fitting or data snooping — is what happens when you tune parameters until the historical result looks good. Every time you look at a backtest result and adjust something in response to it, you've made the strategy fit that specific history more tightly. Do it enough times and you can make any set of rules look profitable on the same dataset you optimized against. The historical data record is finite; a rule with enough parameters can be made to look brilliant on any history.
The diagnostic: a moving average lookback of 187 days instead of 200. A momentum rule that mysteriously excludes one specific calendar month. A result that shows a Sharpe ratio above 1.5 for a long-only equity strategy. If you round a parameter from 187 to 200 and results get noticeably worse, that's not a strategy — it's a curve fit to a number the data happened to like. Bailey, Borwein, López de Prado, and Zhu's paper “Pseudo-Mathematics and Financial Charlatanism” (2014) quantifies how few trials it takes to produce an impressive-looking backtest by chance alone; the number is much smaller than most people expect.
The practical fix is walk-forward optimization: split your historical data into rolling in-sample windows for parameter optimization and out-of-sample windows you never touch during fitting. Optimize on the in-sample window, evaluate on the out-of-sample window, then roll forward. The out-of-sample performance is the number that matters. If the strategy that looked excellent in-sample collapses out-of-sample, the backtest was telling you about that specific history, not about the strategy's durability.
The Cost Modeling Problem Most Tutorials Skip Entirely
Most beginner tutorials model transaction costs as zero, or add a flat $1 per trade and call it realistic. Real execution has three components: commission, slippage, and market impact. Getting any of these wrong doesn't just affect your P&L figure — it can reverse the sign of your edge entirely.
Commission is the easy one, but even here the backtesting library can silently mislead you. A 2025 arXiv paper analyzing implementation risk across five open-source engines found a documented defect in Backtrader's default configuration: it silently divides the user-specified commission rate by 100 before applying it. If you set commission to 0.001 (intending 0.1% per side), Backtrader applies 0.001% instead. Your costs look immaterial, your equity curve looks clean, and the strategy appears stronger than it is.
Slippage is harder. A simple slippage model adds half the bid-ask spread to each fill. A more realistic model considers order size relative to average volume: at 1% of average daily volume, market impact scales roughly with the square root of your participation rate. For high-turnover strategies, the gap between a flat commission assumption and a realistic slippage-plus-market-impact model can swing simulated annual returns by several full percentage points. The same arXiv paper found that across 15 benchmark strategies, switching from zero transaction costs to 60 basis points per side created divergences of up to 3.71% in total return between testing engines over five years — not because any engine computed wrong, but because their cost model implementations differed in ways that compound over hundreds of trades.
If you're running a strategy with 200 or more trades per year, model realistic costs including spread. Add at minimum one tick of slippage per side for futures strategies. Backtest with costs at zero, 10bps, and 30bps per side: if the edge disappears at 10bps, it was probably never there to begin with.
Adjusted Price Data: Returns Are Clean, Price Levels Are Not
Free data sources — Yahoo Finance, most broker APIs — give you dividend-adjusted closing prices by default. Returns computed from adjusted prices are accurate: the ratio of two adjusted closes cancels out every dividend paid between them, giving you correct total-return performance. But price levels from an adjusted series are not point-in-time data.
Here's the problem: today's adjusted level for a date six years ago reflects every dividend paid since that date, including dividends that didn't exist yet on that historical date. Download the same dataset today and again in six months and the historical adjusted levels will be different numbers. Any rule in your strategy that thresholds on a price level — “buy when price crosses above $400” — is implicitly consuming future-dividend information. The fix: derive returns from an adjusted series, but threshold on raw or split-adjusted levels only. This rarely matters for pure return-based signals, but it matters a great deal for value strategies, mean-reversion strategies, and any signal that directly compares today's price to a historical level.
Regime Fit: Your Backtest Covers One or Two Market Environments
The bulk of most retail backtests cover roughly the same stretch of historical data: 2010 to present, using whatever free source is easiest. That era is not a representative sample of market conditions. It includes the longest U.S. equity bull market on record, near-zero interest rates for most of the period, and then a high-inflation, rising-rate regime starting in 2022 that looked nothing like the prior decade. A strategy optimized — even informally, through repeated testing and parameter adjustments — against that specific window may look excellent because it fit the dominant regime, not because it has a durable structural edge.
The check is simple in concept: slice your historical data by regime and examine results separately. Did the strategy perform in both the low-rate era and the post-2022 environment? Did it survive both the 2018 Q4 selloff and the 2020 COVID drawdown? If it only works during trending, low-volatility bull markets, label it a trending strategy and size it accordingly — don't treat it as all-weather. A maximum drawdown figure calculated across a single regime doesn't tell you how bad things get when the regime shifts. Walk-forward testing helps here too, because rolling windows naturally cross regime boundaries if your in-sample and out-of-sample windows span enough time.
Position Sizing: Fixed-Size Tests Don't Tell You What You Actually Need to Know
Tutorials almost universally assume fixed position sizing: always invest 100% of capital, or always trade one unit. This sidesteps a meaningful problem. If you're going to run a strategy live, your position sizes need to adapt to account size and current volatility. A volatility-targeted or Kelly-fraction sizing model interacts with the equity curve in ways that fixed-size testing doesn't reveal: a strategy that looks manageable with 100% fixed allocation might concentrate risk badly during high-volatility regimes under real sizing. A deep drawdown also compounds the damage when you're sizing as a fraction of equity, because smaller positions make recovery slower.
More practically: if your backtest shows a Sharpe ratio of 1.1 under fixed sizing, that figure doesn't transfer directly to the Sharpe you'd realize in production with percent-of-equity sizing at a realistic leverage level. These effects only show up if you explicitly code sizing logic into the backtest. Most tutorials don't, because it requires thinking about position construction not just signal construction.
What a Realistic Backtesting Process Actually Looks Like
The practical rules are not complicated, but they produce less photogenic results than a smooth equity curve chart:
- Write the strategy logic in full before looking at any backtest results. Parameter values should have a stated rationale, not “I tried several and this one scored best.”
- Shift all signals by at least one bar relative to the data used to compute them. Audit any
rolling(),fillna(), or third-party indicator call for hidden look-ahead contamination. - Use point-in-time universe data if your strategy selects among multiple securities. Treat survivorship bias as a known optimism floor if you skip this step.
- Model costs at the realistic level for the asset class and your order size. For U.S. equity futures: $2–$4 per side in commission plus at minimum one-tick slippage is a reasonable floor. At 60bps all-in, your strategy needs to generate significantly more than 60bps average edge per trade to survive in production.
- Use walk-forward windows. If your strategy's out-of-sample Sharpe is meaningfully below its in-sample Sharpe, you have an overfitting problem, not a trading strategy.
- Test explicitly across regimes. A strategy with all its historical data from 2010–2019 is a one-regime strategy. Label it that way.
The reason tutorials skip all of this is that a backtest with every bias addressed often produces a modest, noisy, genuinely uncertain result instead of a clean equity curve. That's less shareable. A rigorous backtest isn't supposed to confirm your strategy is great — it's supposed to tell you whether it has a real edge at all. Most strategies don't survive honest scrutiny. The ones that do are worth building on.