Most of us have done this on a Sunday afternoon.
You open the journal. The overall number is disappointing, so you start filtering. London only. Then London and trend days. Then London, trend days, and entries taken in the first half hour. And there it is — a green number, a decent win rate, a subset of your trades that finally looks like the trader you thought you were.
You write it down as a rule. You trade it for six weeks. It does nothing.
That is curve-fitting. Same mistake the algo guys get lectured about constantly, except nobody warns discretionary traders about it, because we assume it only happens inside a backtesting engine.
It doesn't. It happens in a spreadsheet, with two hundred trades and a filter dropdown, and honestly it's worse there.
What curve-fitting actually means
Quick definition, because the term gets thrown around without being explained.
Curve-fitting (or overfitting) is when a rule describes the past very well but has no reason to describe the future. The rule isn't picking up a real pattern in the market. It's picking up the specific bumps and accidents of the particular trades you happened to look at.
Think of it this way. Flip a coin 200 times and write down the results. Now go hunting. You'll find that flips 40 through 55 were 12 heads out of 16. That's a real fact about that data. It tells you nothing about flip 201.
A backtest overfits when someone tunes the moving average from 20 to 21 to 22 until the equity curve looks nice. A journal overfits when you keep adding filters until the remaining trades look nice. Same act. One automated, one done by hand.
Data mining is the neutral name for the searching itself. Searching isn't the crime. Searching and then reporting only the winner, as if it were the only thing you looked at, is.
What it looks like in a real journal
Let's make this concrete. Imagine 220 trades logged over roughly fourteen months. Everything in R, where 1R is the amount you risked per trade.
The headline number is +0.02R per trade. Basically flat. After costs, probably slightly negative. Not what you wanted to see.
So you start slicing:
- London session only: +0.11R over 90 trades. Better. Still not exciting.
- London and higher-timeframe trending: +0.31R over 41 trades. Now we're talking.
- London, trending, entry within the first 30 minutes: +0.68R over 23 trades, 61% win rate.
That last line looks like an edge. It feels like the answer to why you've been flat for a year: you were diluting a good setup with garbage trades.
Two things about that number.
First, 23 trades at +0.68R is 15.6R total. Pull out the best trade (+7.2R, that runner in March) and the second best (+5.1R). You're left with 3.3R across 21 trades, or +0.16R each. The "edge" was mostly two trades, and without them it's barely distinguishable from the flat baseline you started with.
Second, look at the spread. If your R outcomes have a standard deviation around 1.5R — normal for a setup that mixes small losses with occasional big runners — the uncertainty on an average of 23 trades is roughly 1.5 ÷ √23, about 0.31R. A rough two-standard-error range around +0.68R — meaning the band where the true long-run number plausibly sits, given how little data you have — runs from about +0.06R to +1.30R. That's not a measurement. That's a shrug with a positive sign on it.
None of this proves the filter is worthless. It might be real. You just can't tell from this data, and the way you found it makes noise more likely, not less. Anatomy of an overfit setup shows what a fitted rule looks like from the inside.
Three reasons the journal version is worse than a backtest
I want to be specific here, because "your journal can overfit too" is easy to nod along to and then ignore.
1. You have almost no data
A backtest on five years of 15-minute data might produce 3,000 trades. Your journal has 220. Maybe 400 if you've been at it a while and trade actively.
Smaller samples produce bigger accidents. In 3,000 trades a lucky streak gets averaged away. In 220, one good month is a big chunk of your whole history. And after three filters you're not looking at 220 trades, you're looking at 23, where almost anything can happen. How many trades it takes to validate a setup goes into where the honest thresholds sit, and they're higher than most people want them to be.
2. Nothing records how many things you tried
A decent backtesting workflow leaves a trail: parameter sweeps, optimization logs, the number of variants run. Even if the trader ignores that trail, it exists.
Your Sunday afternoon leaves nothing. You clicked through maybe thirty filter combinations, forgot the twenty-six that looked bad, remembered the one that looked good. By Monday the story in your head is "I noticed my London trend trades do better," not "I ran thirty tests and reported the best one."
That number matters enormously. Say you tag trades with 12 conditions — session, trend state, entry timing, volatility, day of week, that sort of thing. Testing each condition on its own is 12 tests. Every pair of them is another 66. Every triple, another 220. Search all of them and you've run 298 tests. Now apply the usual 5% significance threshold — that's just the common convention of accepting a 1-in-20 chance of calling a random result real. Across 298 tests, about 1 in 20 clears that bar by luck alone. Roughly 15 winning combinations, even if none of your tags mean anything at all.
You will find something. Finding something is guaranteed. The whole logic of counting your hypotheses before you trust a result rests on this, and it's the single most useful idea in this whole area.
3. You're the searcher and you're the one who wants a yes
This is the part that doesn't apply to a hired quant reviewing someone else's strategy.
You already believe you have an edge. You've spent years on this. You've told people. When a filter produces a green number, you don't evaluate it neutrally — you recognize it. That's the version of the setup I take when I'm sharp. The confirmation arrives dressed as insight.
And the filters you reach for aren't random. They flatter the trades you're proud of and quarantine the ones you're embarrassed by. Filtering out "revenge trades" or "trades I took while tilted" feels like cleaning the data. Often it's just removing your losses and calling the remainder your edge.
Nobody is immune, disciplined traders included. It isn't a willpower problem, it's a structure problem: the search and the wish are running in the same head at the same time.
The fix is boring: write the guess down first
Here's the whole method, and it fits in one sentence. Decide what you're testing before you look at the answer.
In practice:
Write the hypothesis in your notes, with a date, before you filter. Not "let's see what stands out." Something like: I think my losses cluster in the New York afternoon because liquidity thins out and my stops get run. I expect afternoon trades to underperform morning trades by at least 0.2R. Now you've committed, and the result can disappoint you. That's the entire point. A test that can't disappoint you isn't a test.
Keep a running list of every hypothesis you check, dead ones included. Twelve lines in a text file. When the thirteenth comes back positive, you know it's the thirteenth, not the first, and you can be suspicious accordingly.
Then split your data. Discover on the older trades, check on the newer ones — chronologically, never randomly shuffled, because markets change over time and a random split lets the future leak into the past. In-sample versus out-of-sample for traders covers how to do this split without fooling yourself.
This is the part we thought hardest about when building EdgeFlow. It does search your history — it tests combinations of the conditions you tagged, the exact thing I've spent this post warning you about. What matters is what happens around the search. You set the vocabulary up front by tagging each trade across four layers (technical setup, execution, environment, management), so the engine isn't inventing categories to fit your winners. Then it finds candidates on your older trades and checks them on your newer ones, ranks by the cautious lower end of the range rather than the headline average, and drops thin samples into a "needs more data" bucket instead of dressing them up as edges. Searching is fine. Reporting the prettiest slice as though it were the only slice is not.
What to do with the filter you already found
You've probably got one. Most of us do. Don't throw it away — it costs nothing to keep and it might be real.
Treat it as a candidate, not a rule. Write it down precisely enough that a stranger could apply it to your next 40 trades without asking you a question. Then log those 40 and compare. If the effect was real, it should still be there, probably weaker. If it was noise, it evaporates, and that costs you attention instead of six weeks of confused position sizing.
Also, cut the combination down. Three stacked filters on 23 trades is fragile. One condition on 90 trades is something you can actually measure. London alone at +0.11R across 90 trades is a weaker claim but a far more trustworthy one, and it's worth asking whether the extra two filters added information or just added specificity. Testing condition combinations properly is its own discipline once you go past a single filter.
The honest caveat
Some of your filters are real. Genuinely. Setups do behave differently across sessions and volatility regimes, and finding those differences is most of what building a trading edge actually consists of. I'm not telling you to stop segmenting your journal.
I'm telling you the searching costs something, and you pay it back with evidence you didn't use during the search. Nobody can separate the real patterns from the lucky ones with certainty. What you can do is stop pretending the search never happened.
The uncomfortable version: your flat +0.02R might just be your actual number right now, and no arrangement of filters will change it. Worth knowing early, because it points you at fixing the process instead of decorating the record of it.
If you want a journal that checks a candidate on trades it didn't use to find it, and quotes you the cautious end of the range instead of the prettiest slice, that's what we built EdgeFlow to do — there's a free interactive demo on the homepage if you want to see the idea before committing to anything.
Common questions
Isn't every filter I apply a form of overfitting?
No. A filter you decided on in advance, for a reason you can state, tested on data you hadn't already picked through, is normal analysis. Overfitting is what happens when the filter is chosen because it produced a good number.
How do I know if my subset result is real or noise?
Mostly you can't, not from the same data that produced it. Check the sample size, remove the two largest winners and look again, count how many combinations you tried, then wait for new trades. The waiting is what gives you the answer.
Does this apply if I only trade one setup?
Yes, and sometimes more. One setup usually means fewer trades, so every slice you take runs into small-sample problems faster.
Is 30 trades ever enough?
For a rough sanity check, sometimes. For a rule you'd size up on, no. High-variance strategies with occasional large winners need considerably more before the average means much.