What Actually Changes an MT5 Backtest — We Ran the Controlled Test

A seller shows you a backtest. You run it yourself. The numbers don't match. This is the most common frustration when buying an EA — and the easiest thing for a seller to wave away: "your environment is different."

That's true, but it isn't an explanation. Some backtest variables move the result by an order of magnitude. Others change nothing at all. Knowing which is which tells you what to actually push back on. Below, ranked by impact, each with real numbers measured on our own machines.

Start with the counterintuitive one: changing the timeframe changed nothing

Most people's first instinct is "they used H1, I used M1 — that's why." We isolated that variable and tested it.

Instrument: ArtQuant Gold. Same .set file, same window (2026.01.01 – 2026.07.23), same modelling (every tick based on real ticks), 50ms execution delay, 1:1000 leverage, $500 starting balance, same broker (Tickmill). The only change: chart timeframe from H1 to M1.

Nearly 60× the bar count. Every figure in the two reports is identical — the net profit doesn't differ by a cent.

The reason is the second-to-last row: the tick count is identical. Under "every tick based on real ticks" modelling, the tester feeds the EA the same tick stream regardless of chart period. This EA makes decisions on ticks, not on bar closes. The timeframe only changed how many candles get drawn — not the price sequence the EA sees.

We ran a third pass for good measure: the same H1 configuration on a second virtual machine returned $24,399.65. Across machines, a $0.33 gap — 0.0014% of net profit. Across timeframes, $0.00. The only noise this experiment could find came from the hardware, not the timeframe.

Worth scoping this properly: the finding holds for EAs that decide on ticks. If an EA explicitly states it evaluates only at the open of a new bar, then the timeframe is one of its strategy parameters — changing it changes the strategy, and of course the result moves. But that's a different strategy, not the same configuration producing different results. How to tell: check whether the documentation or inputs specify a timeframe. If nothing specifies one, it's almost certainly tick-driven.

So for the vast majority of EAs, "different timeframe" is not a valid excuse for a mismatch. If someone offers it, push on the items below instead.

What actually changes the number, ranked

1. Modelling quality — the biggest single lever

The MT5 Strategy Tester offers several modelling levels, from "open prices only" up to "every tick based on real ticks." Open-prices-only evaluates once per bar and skips everything that happened inside it — stops and targets fire where they shouldn't, and the resulting curve tends to look implausibly clean. A backtest that doesn't state its modelling mode is not worth reading.

Every param-set backtest we publish uses the highest setting. The 48-million-plus ticks in the table above are where that comes from.

2. Spread, commission, and broker

Gold EAs are especially exposed here. The same logic run on a 15-point spread versus a 35-point spread can flip a high-frequency strategy from profitable to negative. That's why we state which broker the backtest ran on — numbers can only reconcile if the environment is spelled out. For the arithmetic, see how gold EA spread costs actually add up.

3. Execution delay

The tester defaults to zero delay — instant fills. Live trading doesn't work that way. We standardise on 50ms, a realistic figure for a VPS colocated near the broker. Sniper and scalping strategies are extremely sensitive to this; anything that collapses when you move delay from 0 to 50ms will not survive live either.

4. Leverage

Leverage doesn't change entry and exit logic, but it determines how much floating loss the margin can absorb. For grid and averaging systems, that's the difference between riding out an extreme move and blowing up. Our two param sets use different leverage on purpose — each matches its own position-sizing rules, and they aren't comparable side by side.

5. Compounding vs. periodic withdrawal

This is the sneakiest one. If the EA or the tester is set to periodically withdraw profit, the balance gets shaved back on a schedule, the curve turns into a sawtooth, and the denominator behind every monthly return shifts — the same market can produce completely different percentages. Before reading any backtest curve, ask whether profit stayed in the account or was taken out. Our param detail pages state this explicitly.

6. The test window

Same parameters, different market, different number. Still ArtQuant Gold, still the same .set:

Fewer trades, more than triple the profit — because gold in 2026 hasn't behaved like gold in 2025. Showing you only the flattering window is the most common form of cherry-picking in this business. It's why we always run both a full prior year and the current year-to-date: one for the baseline, one for what's happening now.

One more: don't read balance drawdown alone

In that 2026 ArtQuant run, max balance drawdown was 3.85%. Max equity drawdown was 14.86%. Nearly 4× apart.

Balance drawdown counts closed trades only — floating losses are invisible to it. Equity drawdown includes them. For grid and averaging systems the floating loss is the risk, so reporting balance drawdown alone hides the dangerous part. We publish both. For how these strategies fail, see are grid and martingale EAs dangerous.

Eight things a reproducible backtest must disclose

• Symbol and chart timeframe
• Modelling mode (is it every-tick-based-on-real-ticks?)
• Broker and account type — this sets spread and commission
• Execution delay
• Leverage
• Starting balance
• Test window (start and end dates)
• Compounding or periodic withdrawal

With all eight, someone else stands a chance of reproducing your curve. Missing any one of them, the number is just a picture.

How we do it

Every .set in our param sets category is measured on our own machines under one fixed protocol: every tick based on real ticks, 50ms delay, $500 starting balance, Tickmill, full-year 2025 plus 2026 year-to-date. Every figure on those pages comes straight from the MT5 report — nothing derived, no window shopping.

One thing worth admitting: the A/B above exists because while preparing the ArtQuant param set, we changed an input we had no business touching and produced a dataset far off from what we eventually published. Tracing it down, the cause turned out to be that input — not the timeframe we'd assumed at the time. The controlled test was run specifically to falsify that assumption. We'd rather burn another test run than put an unverified causal claim on a product page.

To see what this looks like in practice, browse the param sets. To learn the tester itself, read how to backtest an EA in MT5. For what to verify beyond backtests, see verifying an EA with Myfxbook.

Risk note: this is a methodology article, not investment advice. Backtested results do not predict future returns, and every EA carries risk of loss. All figures here come from MT5 Strategy Tester reports generated on our own machines.