<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Backtesting on Shane&apos;s Personal Blog</title><description>Recent content in Backtesting on Shane&apos;s Personal Blog</description><link>https://shanechang.com/tags/backtesting/</link><language>en-us</language><lastBuildDate>Wed, 02 Sep 2026 00:00:00 GMT</lastBuildDate><atom:link href="https://shanechang.com/tags/backtesting/index.xml" rel="self" type="application/rss+xml"/><item><title>Google&apos;s TimesFM vs the Stock Market: Every Way I Fooled Myself</title><link>https://shanechang.com/p/timesfm-stock-market-how-i-fooled-myself/</link><guid isPermaLink="true">https://shanechang.com/p/timesfm-stock-market-how-i-fooled-myself/</guid><description>&lt;img src=&quot;https://shanechang.com/_astro/cover.oJ6VoBWL_ZRmYpP.webp&quot; alt=&quot;Featured image of post Google&apos;s TimesFM vs the Stock Market: Every Way I Fooled Myself&quot; /&gt;&lt;h2 id=&quot;a-model-that-had-seen-a-billion-time-series&quot;&gt;A Model That Had Seen a Billion Time Series&lt;/h2&gt;
&lt;p&gt;At the end of August, Google Research released TimesFM 3.0.&lt;/p&gt;
&lt;p&gt;If you haven’t been following this corner of machine learning, here’s the pitch. Foundation models — the GPT-style approach of pretraining one enormous model on a mountain of data and then pointing it at problems it was never specifically trained for — had already eaten language, then images, then code. Time series were next. TimesFM is trained on billions of points drawn from electricity demand, web traffic, retail sales, weather, hospital admissions, traffic sensors. Then you hand it a sequence it has never seen, and it forecasts the continuation. Zero-shot. No fitting, no tuning, no domain expertise.&lt;/p&gt;
&lt;p&gt;I read the release notes and had the thought. You’ve probably already had it too, reading the paragraph above.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;A stock price is a time series.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;That’s not a stupid thought. That’s the whole appeal of foundation models — they generalize to domains nobody explicitly trained them for. If a model can pick up the rhythm of electricity consumption in a city it has never seen, why not the rhythm of a stock it has never seen? I had a single RTX 3090 Ti sitting under my desk, free historical price data from a broker API, and a weekend.&lt;/p&gt;
&lt;p&gt;So I started. And a few months later I have a codebase, sixty-one tests, a folder of results, and a conclusion I did not expect: &lt;strong&gt;essentially every promising thing I found was a mistake in my own measurement.&lt;/strong&gt; Not a small mistake. Not a rounding issue. Things like “my strategy made money because I accidentally let it see tomorrow’s prices.”&lt;/p&gt;
&lt;p&gt;This post is the tour. I’m going to show you the results that excited me, then show you exactly how each one died. The mistakes generalize far past finance — most of them are just ways humans fool themselves with data, wearing a costume. And at the end I’ll tell you what I actually think you should do with your money, which is a conclusion so boring that I’d never have believed it if I hadn’t spent months earning it.&lt;/p&gt;
&lt;h2 id=&quot;the-equity-curve-that-was-too-beautiful&quot;&gt;The Equity Curve That Was Too Beautiful&lt;/h2&gt;
&lt;p&gt;My first working backtest produced a chart that went up and to the right with an almost cartoonish steadiness. Not a rocket — I was suspicious of rockets — but a lovely, smooth, gently accelerating climb through 2018, through the COVID crash, through 2022. It survived everything.&lt;/p&gt;
&lt;p&gt;I sat and looked at it for a while with a feeling I can only describe as vertigo.&lt;/p&gt;
&lt;p&gt;The bug was one line. When you backtest a strategy, you compute a set of positions — buy this, sell that — and then multiply those positions by what the stocks actually did. The subtlety is &lt;em&gt;when&lt;/em&gt;. If you decide your positions using today’s closing price, you cannot also earn today’s return, because today already happened. You have to shift your positions forward by one day.&lt;/p&gt;
&lt;pre class=&quot;astro-code astro-code-themes github-light-default github-dark-default&quot; style=&quot;background-color:#ffffff;--shiki-dark-bg:#0d1117;color:#1f2328;--shiki-dark:#e6edf3; overflow-x: auto;&quot; tabindex=&quot;0&quot; data-language=&quot;python&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#6E7781;--shiki-dark:#8B949E&quot;&gt;# What I wrote:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#1F2328;--shiki-dark:#E6EDF3&quot;&gt;returns &lt;/span&gt;&lt;span style=&quot;color:#CF222E;--shiki-dark:#FF7B72&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color:#1F2328;--shiki-dark:#E6EDF3&quot;&gt; weights &lt;/span&gt;&lt;span style=&quot;color:#CF222E;--shiki-dark:#FF7B72&quot;&gt;*&lt;/span&gt;&lt;span style=&quot;color:#1F2328;--shiki-dark:#E6EDF3&quot;&gt; stock_returns&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#6E7781;--shiki-dark:#8B949E&quot;&gt;# What it should have been:&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#1F2328;--shiki-dark:#E6EDF3&quot;&gt;returns &lt;/span&gt;&lt;span style=&quot;color:#CF222E;--shiki-dark:#FF7B72&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;color:#1F2328;--shiki-dark:#E6EDF3&quot;&gt; weights.shift(&lt;/span&gt;&lt;span style=&quot;color:#0550AE;--shiki-dark:#79C0FF&quot;&gt;1&lt;/span&gt;&lt;span style=&quot;color:#1F2328;--shiki-dark:#E6EDF3&quot;&gt;) &lt;/span&gt;&lt;span style=&quot;color:#CF222E;--shiki-dark:#FF7B72&quot;&gt;*&lt;/span&gt;&lt;span style=&quot;color:#1F2328;--shiki-dark:#E6EDF3&quot;&gt; stock_returns&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Without that &lt;code&gt;.shift(1)&lt;/code&gt;, my simulated trader was deciding what to buy &lt;em&gt;after&lt;/em&gt; watching the price move. Of course it made money. It was a time traveler.&lt;/p&gt;
&lt;p&gt;I want to be precise about how it felt, because this is the part people skip when they tell these stories. It did not feel like a bug. It felt like a &lt;em&gt;discovery&lt;/em&gt;. I had spent a weekend on it and it worked, and there’s a specific pleasure in that which makes you not want to poke at it. I only found it because I made myself write down what I expected the numbers to look like before I ran the test, and the real numbers were much &lt;em&gt;too&lt;/em&gt; good — a strategy with those returns would be the best hedge fund in history, run by a guy who’d been at it for two days.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;First lesson, and the one that governs everything else: your excitement is a signal, and the signal is “check this again.”&lt;/strong&gt; Not “this is great.” The feeling that you’ve found something is exactly the feeling that stops you from looking closer.&lt;/p&gt;
&lt;p&gt;I fixed the line. The beautiful curve evaporated. And that turned out to be the &lt;em&gt;easy&lt;/em&gt; mistake — the kind that announces itself once you know to look. The rest of the year was spent on mistakes that don’t.&lt;/p&gt;
&lt;h2 id=&quot;sharpe-072-and-how-it-died&quot;&gt;Sharpe 0.72, and How It Died&lt;/h2&gt;
&lt;p&gt;With the time travel removed, I built something more serious.&lt;/p&gt;
&lt;p&gt;Not “predict whether Apple goes up” — that’s a coin flip with extra steps. Instead a &lt;strong&gt;cross-sectional&lt;/strong&gt; strategy: rank a basket of stocks against each other, buy the top slice, short the bottom slice, hold roughly equal amounts of each so the overall market direction cancels out. If the whole market drops 3%, your longs and shorts drop together and you’re roughly flat. You’re not betting on the market. You’re betting on the &lt;em&gt;ranking&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;This is real technique. It’s roughly what a large fraction of the systematic hedge fund industry does. And on my universe of sixty large US stocks, it produced a &lt;strong&gt;Sharpe ratio of 0.72.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Quick translation, because Sharpe ratio is the one number you need for the rest of this post: it’s return divided by volatility — how much reward you got per unit of stomach-churning. The S&amp;#x26;P 500 runs about 0.5 over the long haul. A 0.72 would mean I’d built something meaningfully better than the index on a risk-adjusted basis, alone, on a gaming GPU. That’s a real number. Hedge funds have been launched on less.&lt;/p&gt;
&lt;p&gt;Then I ran one more test, and it killed the whole thing.&lt;/p&gt;
&lt;p&gt;The test is embarrassingly simple. My sixty stocks were a list I’d written by hand — Apple, Microsoft, JPMorgan, the obvious ones. So: what happens if I run &lt;em&gt;the exact same strategy, unchanged&lt;/em&gt;, on sixty stocks picked at random? I did that two hundred times.&lt;/p&gt;
&lt;p&gt;Here’s what came back.&lt;/p&gt;





















&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Stock universe&lt;/th&gt;&lt;th&gt;Sharpe&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;My hand-written 60&lt;/td&gt;&lt;td&gt;&lt;strong&gt;0.72&lt;/strong&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Random 60, median&lt;/td&gt;&lt;td&gt;&lt;strong&gt;−0.03&lt;/strong&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Random 60, 95th percentile&lt;/td&gt;&lt;td&gt;0.55&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;My list landed at the &lt;strong&gt;97th percentile&lt;/strong&gt; of random draws. The typical random list &lt;em&gt;lost money&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;Sit with that for a second, because it took me a while. The signal wasn’t doing the work. &lt;strong&gt;The stock list was the strategy.&lt;/strong&gt; And I hadn’t picked those sixty companies by any clever process — I’d picked the ones I could name off the top of my head in 2026, which means I picked companies that had spent the previous decade becoming large enough for me to know their names.&lt;/p&gt;
&lt;p&gt;That’s called &lt;strong&gt;survivorship bias&lt;/strong&gt;, and it’s the most common way people accidentally cheat at investing research. If you build a strategy and test it on today’s big winners, you’ve smuggled the future into your test. You already know Nvidia worked out. In 2015, you’d have been staring at a list of five hundred names with no idea which ones were about to become household words.&lt;/p&gt;
&lt;p&gt;I rebuilt the universe properly: rank companies by trading volume during 2021 &lt;em&gt;only&lt;/em&gt;, freeze that list, and never let anything from 2022 onward influence membership. On that honest list, the same strategy scored &lt;strong&gt;0.17.&lt;/strong&gt; Which, after trading costs, is nothing at all.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;A full Sharpe point of my result was pure hindsight.&lt;/strong&gt; Not from a bug this time. From a list I’d typed in three minutes without thinking.&lt;/p&gt;
&lt;p&gt;The generalizable lesson: &lt;strong&gt;before you believe a result, find out what a random version of it scores.&lt;/strong&gt; Not a theoretical null — an actual, shuffled, run-it-two-hundred-times version. If your carefully engineered thing lands in the same neighborhood as the random things, you don’t have a result. You have a draw.&lt;/p&gt;
&lt;h2 id=&quot;four-more-ways-i-fooled-myself&quot;&gt;Four More Ways I Fooled Myself&lt;/h2&gt;
&lt;p&gt;That was the big one. But once you start looking for these, they’re everywhere, and they’re not really about finance at all. Every one of these is a way of counting that flatters you.&lt;/p&gt;
&lt;h3 id=&quot;counting-112000-things-that-were-really-1149-things&quot;&gt;Counting 112,000 things that were really 1,149 things&lt;/h3&gt;
&lt;p&gt;To claim a signal is real, you run a significance test, which asks: &lt;em&gt;if this had no skill whatsoever, how often would luck alone produce something this good?&lt;/em&gt; Below one-in-twenty, conventionally, and you’re allowed to be interested.&lt;/p&gt;
&lt;p&gt;I had 98 stocks over 1,149 days. So — 112,000 observations, right? Wonderfully large sample. My p-value came back below 0.0001. One in ten thousand. I remember feeling quite pleased.&lt;/p&gt;
&lt;p&gt;Except those aren’t 112,000 independent observations. When the market convulses, &lt;em&gt;every&lt;/em&gt; stock moves at once and every forecast misses at once. A day where I’m wrong about Apple is overwhelmingly a day where I’m also wrong about Microsoft. I hadn’t gathered 112,000 pieces of evidence. I’d gathered 1,149 pieces of evidence about the market, photocopied 98 times.&lt;/p&gt;
&lt;p&gt;Collapse each day to a single number, redo the test, and the same data gives &lt;strong&gt;p = 0.299.&lt;/strong&gt; Roughly one in three by luck. Nothing.&lt;/p&gt;
&lt;p&gt;Same data. Same model. &lt;strong&gt;Three orders of magnitude, entirely from how I counted.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;This one generalizes brutally far beyond stocks. Any time you have repeated measurements on correlated units — users in the same cohort, patients in the same hospital, sensors in the same building — the temptation to treat every row as independent evidence is enormous, and it will hand you spectacular p-values that mean nothing.&lt;/p&gt;
&lt;h3 id=&quot;trying-fifty-four-things-and-admiring-the-winner&quot;&gt;Trying fifty-four things and admiring the winner&lt;/h3&gt;
&lt;p&gt;If you flip fifty-four coins twenty times each, one of them will come up heads sixteen times, and it will look magical. It isn’t magical. It’s just what fifty-four coins do.&lt;/p&gt;
&lt;p&gt;Over the project I tested fifty-four variations — different signals, holding periods, universes, parameters. Then I looked at the best one and felt good about it. But the best of fifty-four attempts is &lt;em&gt;supposed&lt;/em&gt; to look good. That’s arithmetic, not skill.&lt;/p&gt;
&lt;p&gt;There’s a proper fix, called the &lt;strong&gt;Deflated Sharpe Ratio&lt;/strong&gt;, which asks: given that you ran N experiments, how good would the luckiest &lt;em&gt;worthless&lt;/em&gt; strategy have looked? Then it makes you beat that instead of beating zero.&lt;/p&gt;
&lt;p&gt;My headline result scored &lt;strong&gt;0.25&lt;/strong&gt; against that bar. It never cleared the best-of-noise threshold. Not once, at any point, on any version.&lt;/p&gt;
&lt;p&gt;And note the perverse incentive baked in: the harder I worked — the more ideas I tried — the &lt;em&gt;higher&lt;/em&gt; the bar became. That is correct and it is deeply annoying, and it’s why “I tested a hundred strategies and this one works” is a much weaker claim than it sounds. The published finance literature has this problem so badly that a large fraction of documented market anomalies stop working immediately after publication.&lt;/p&gt;
&lt;h3 id=&quot;a-bug-in-the-ruler&quot;&gt;A bug in the ruler&lt;/h3&gt;
&lt;p&gt;Deep in my statistics code was a function that measures whether one forecast is genuinely better than another. It needs to estimate how noisy the comparison is, which involves summing up a series of terms.&lt;/p&gt;
&lt;p&gt;I summed them the obvious way. The obvious way is wrong — the resulting estimate isn’t guaranteed to be a valid variance, and it can come out too small, or even negative, in which case my code silently returned “no difference detected” and moved on.&lt;/p&gt;
&lt;p&gt;I fixed it, re-ran everything, and every single p-value moved &lt;em&gt;against&lt;/em&gt; my model:&lt;/p&gt;

























&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Comparison&lt;/th&gt;&lt;th&gt;Before the fix&lt;/th&gt;&lt;th&gt;After&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;vs. a naive baseline&lt;/td&gt;&lt;td&gt;p = 0.004&lt;/td&gt;&lt;td&gt;p = 0.017&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;vs. a 2009 linear model&lt;/td&gt;&lt;td&gt;p = 0.060&lt;/td&gt;&lt;td&gt;p = 0.114&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;vs. a 1996 moving average&lt;/td&gt;&lt;td&gt;p = 0.101&lt;/td&gt;&lt;td&gt;p = 0.299&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;The verdict didn’t change — I’d already failed that test — but that tantalizing p = 0.060, the one that had me thinking &lt;em&gt;so close, just needs a bit more tuning&lt;/em&gt;, was an artifact. There was never anything there to tune toward.&lt;/p&gt;
&lt;p&gt;Here’s the part I actually want you to take away. &lt;strong&gt;I did not find that bug by being suspicious of the answer.&lt;/strong&gt; I found it by writing tests for my statistics code — the boring engineering hygiene I’d been skipping because it felt like it wasn’t the real work. Four genuine bugs surfaced within a day of writing those tests.&lt;/p&gt;
&lt;p&gt;Intuition is not a bug detector. Your intuition is on the bug’s side; it wants the answer to be good.&lt;/p&gt;
&lt;h3 id=&quot;a-model-that-may-have-already-read-the-future&quot;&gt;A model that may have already read the future&lt;/h3&gt;
&lt;p&gt;This one is specific to the AI era and it has no classical equivalent.&lt;/p&gt;
&lt;p&gt;TimesFM was released on August 31st, 2026. It was trained on an enormous corpus. Was there stock price data in that corpus? Google doesn’t fully disclose the mixture. Probably some. Possibly a lot.&lt;/p&gt;
&lt;p&gt;Think about what that means. The standard defense against fooling yourself in finance is out-of-sample testing: fit on 2015–2021, test on 2022–2026, and if it holds up on data the model never saw, you have something. &lt;strong&gt;But the model may have already seen 2022–2026.&lt;/strong&gt; Not as a lucky guess — as memorized training data. Every split I could construct lives entirely inside the model’s possible knowledge. The leak predates my dataset. There is no split that fixes it.&lt;/p&gt;
&lt;p&gt;The only clean evidence obtainable is to write predictions down &lt;em&gt;today&lt;/em&gt;, before the outcome exists, and score them later. So that’s running: a script that records the model’s forecasts every afternoon, committed to git so the timestamps are auditable. It needs one to two years to say anything. It is genuinely the most valuable thing in the project, and it consists of doing nothing but waiting.&lt;/p&gt;
&lt;p&gt;If you’re evaluating any AI system on historical data — trading, medicine, hiring, anything — this problem now applies to you. “It performed well on data from 2023” means something very different when your model was trained in 2025.&lt;/p&gt;
&lt;h2 id=&quot;why-the-chart-has-no-secret-in-it&quot;&gt;Why the Chart Has No Secret in It&lt;/h2&gt;
&lt;p&gt;At some point, staring at another dead result, I had the reframe that actually made everything make sense. And it’s the single idea I’d most want to hand to someone who spends their evenings looking for patterns in price charts.&lt;/p&gt;
&lt;p&gt;I had been thinking: &lt;em&gt;prices are basically random, and randomness can’t be predicted.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;That’s wrong, and the correct version has the opposite flavor. &lt;strong&gt;Prices are unpredictable precisely because they already contain everything anyone knows.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Every headline, every earnings report, every rumor, every analyst model — millions of participants, some with satellite imagery of parking lots and better data than I’ll ever have, have already looked and already traded. The price &lt;em&gt;is&lt;/em&gt; the summary of all of it. What’s left over, the wiggle from day to day, is the part nobody could anticipate. The randomness isn’t emptiness. &lt;strong&gt;It’s the residue left after everyone smart already acted.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Once you see it that way, a lot of things snap into focus.&lt;/p&gt;
&lt;p&gt;It explains why my foundation model found nothing: it wasn’t too weak, it was arriving at a scene that had already been picked clean. It explains why the pattern you find in a chart doesn’t repeat — if it were reliably there, someone with a colocated server found it years ago and traded it away. And it explains something that stung a little: in my own forward test, a &lt;strong&gt;moving-average rule from the 1990s&lt;/strong&gt; — one of the simplest technical indicators that exists, computable in a spreadsheet — held up out of sample slightly better than the foundation model did. Not because the moving average is good. Because neither is good, and the simple one at least isn’t pretending.&lt;/p&gt;
&lt;p&gt;There was one more hope, and its failure taught me the most about &lt;em&gt;why&lt;/em&gt; markets are hard.&lt;/p&gt;
&lt;p&gt;There’s a famous result in finance called the Fundamental Law of Active Management: your performance scales roughly as your skill times the square root of the number of independent bets you make. A tiny edge, applied across enough separate bets, becomes a real business. This is why casinos work — the house edge on a single spin is trivial, but they spin a million times.&lt;/p&gt;
&lt;p&gt;So: go from 60 stocks to 500. That’s 2.9× more square-root-of-breadth. My performance should improve materially.&lt;/p&gt;
&lt;p&gt;It did not move at all. Flat.&lt;/p&gt;
&lt;p&gt;The reason is the word &lt;strong&gt;independent&lt;/strong&gt;, and I’d skimmed right past it. Five hundred US stocks are not five hundred independent bets. When the market drops, they nearly all drop. When rates move, every bank in your book moves the same way. Underneath, you have maybe a handful of genuinely independent things — the market itself, a few sector factors, a size effect — and five hundred names is those same few bets wearing five hundred costumes.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;You can’t diversify your way out of a bad edge by buying more of the same market.&lt;/strong&gt; The casino’s spins are independent. Yours aren’t.&lt;/p&gt;
&lt;h2 id=&quot;so-how-does-anyone-actually-make-money&quot;&gt;So How Does Anyone Actually Make Money?&lt;/h2&gt;
&lt;p&gt;Fair question. The industry visibly exists, the buildings are very nice. So what’s happening in them?&lt;/p&gt;
&lt;h3 id=&quot;first-how-big-is-a-real-edge&quot;&gt;First: how big is a real edge?&lt;/h3&gt;
&lt;p&gt;Before the tour, calibrate your expectations, because mine were wildly off.&lt;/p&gt;
&lt;p&gt;I started this project imagining that success looked like being right 55% of the time. Maybe 60% on a good signal. That’s the intuition most people have — you need to be &lt;em&gt;meaningfully&lt;/em&gt; better than a coin flip for this to be worth doing.&lt;/p&gt;
&lt;p&gt;Renaissance Technologies’ Medallion fund has the best track record in the history of finance. Roughly 66% a year, before fees, for three decades. It is the single most successful investment operation ever assembled, staffed by physicists and codebreakers.&lt;/p&gt;
&lt;p&gt;It reportedly wins on about &lt;strong&gt;50.75%&lt;/strong&gt; of its trades.&lt;/p&gt;
&lt;p&gt;Not 55%. Not 60%. Barely better than a coin, applied millions of times with ferocious discipline and enough scale that the tiny margin compounds into the greatest fortune-generating machine ever built.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;That’s the actual size of real edges. Nobody has 55%.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Which reframes the whole problem. My failure was never that I couldn’t find a big edge — nobody has a big edge. It’s that at 50.75%, the signal is smaller than my measurement errors, smaller than my trading costs, and smaller than the amount by which a hand-picked stock list flatters a backtest. At that scale, &lt;strong&gt;the measurement is harder than the model.&lt;/strong&gt; That’s why this post is mostly about statistics instead of about neural networks.&lt;/p&gt;
&lt;p&gt;And one more detail worth more than it looks: Medallion has been &lt;strong&gt;closed to outside money since 1993.&lt;/strong&gt; They manage their own employees’ capital and nobody else’s.&lt;/p&gt;
&lt;p&gt;That’s the tell. If you find a genuine edge with limited capacity, you don’t sell it — you hoard it, because every additional dollar in the trade erodes the very margin you found. &lt;strong&gt;Real edge gets hoarded, not marketed.&lt;/strong&gt; Which means the strategy someone is trying to sell you is, by revealed preference, one that didn’t survive that test.&lt;/p&gt;
&lt;h3 id=&quot;the-five-businesses&quot;&gt;The five businesses&lt;/h3&gt;
&lt;p&gt;Roughly five, and only one is “predict the stock.”&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Market making.&lt;/strong&gt; Firms like Citadel Securities and Jane Street don’t forecast direction at all. They stand ready to buy and sell continuously, capturing the tiny spread between the two prices, thousands of times a second, hedging the leftover exposure. Enormously profitable, and it’s a technology-and-inventory business, not a prophecy business.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Statistical arbitrage.&lt;/strong&gt; The closest thing to what I was attempting. Thousands of tiny edges, each barely better than a coin flip, spread across thousands of instruments. It works through breadth — and the professionals do the step I skipped: they explicitly strip out the market factor, the sector factors, the size and value exposures, so what remains genuinely &lt;em&gt;is&lt;/em&gt; closer to independent.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Risk premia.&lt;/strong&gt; Firms like AQR and Dimensional. This isn’t prediction at all — it’s getting paid to hold things other people don’t want to hold. Boring, honest, high-capacity, and the risk-adjusted returns are modest by design.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Discretionary fundamental investing.&lt;/strong&gt; Humans reading filings and meeting management. Some are genuinely excellent. The average one loses to the index.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Fees.&lt;/strong&gt; A large share of what looks like “Wall Street makes money on stocks” is actually charging other people to manage their money, which is profitable whether or not the strategy works.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Do they use algorithms?&lt;/em&gt; Almost universally — but mind the distinction. Essentially all institutional orders are &lt;em&gt;executed&lt;/em&gt; by algorithms that slice a large order into pieces to avoid moving the price against themselves. That’s about minimizing your own footprint, not seeing the future. Prediction algorithms are rarer and far weaker than the movies suggest.&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Do they use AI?&lt;/em&gt; Increasingly, but as a &lt;strong&gt;feature extractor, not a decider&lt;/strong&gt; — reading filings and transcripts to turn prose into numbers, which then feed a small, boring, auditable model that makes the actual call. Nobody sane puts a large language model in the decision loop: too slow, non-deterministic, and when you lose money you have to explain why to a risk committee, and “the model felt bearish” is not an explanation.&lt;/p&gt;
&lt;h3 id=&quot;the-deepseek-story-which-is-not-the-story-you-think&quot;&gt;The DeepSeek story, which is not the story you think&lt;/h3&gt;
&lt;p&gt;Here’s my favorite illustration, and it’s one most people have backwards.&lt;/p&gt;
&lt;p&gt;DeepSeek — the Chinese lab whose models shook the industry — is a spinout of &lt;strong&gt;High-Flyer&lt;/strong&gt;, a Hangzhou quantitative hedge fund founded around 2015 by Liang Wenfeng. At its peak it was one of China’s largest quant managers.&lt;/p&gt;
&lt;p&gt;The intuitive story is: a hedge fund needed AI to trade better, so it built an AI lab.&lt;/p&gt;
&lt;p&gt;The actual story runs the other way. High-Flyer traded with machine learning on Chinese stocks, and to do that they built enormous GPU clusters — the “Fire-Flyer” series, the second reportedly around ten thousand A100s. Then Chinese regulators clamped down on quant funds in 2024, High-Flyer had a rough stretch, and its founder — who by all accounts was more interested in artificial general intelligence than in trading — pointed the cluster at a different problem. DeepSeek doesn’t trade. It was never meant to.&lt;/p&gt;
&lt;p&gt;But here’s the part relevant to you and me. A large share of High-Flyer’s trading edge came not from superior models but from &lt;strong&gt;who they were trading against.&lt;/strong&gt; Chinese A-shares are dominated by individual retail investors — historically something like 70–80% of trading volume, versus roughly the inverse in the US. Retail flow is more predictable, more momentum-chasing, more behaviorally patterned. Their opponent was different.&lt;/p&gt;
&lt;p&gt;Same techniques, radically different opponent. When I ran my experiments on US large-cap stocks, I was competing against the most thoroughly picked-over price series on the planet.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The edge lives in who you’re trading against, not in how clever you are.&lt;/strong&gt;&lt;/p&gt;
&lt;h2 id=&quot;warren-buffett-taken-apart&quot;&gt;Warren Buffett, Taken Apart&lt;/h2&gt;
&lt;p&gt;Which brings us to the counterexample everyone reaches for. If markets are this efficient, what about Buffett?&lt;/p&gt;
&lt;p&gt;First, though, a distinction that took me far too long to internalize, and that quietly invalidates most of the investing conversations you’ll ever overhear.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Buying a stock that goes up is not skill. It’s rent.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;US stocks have returned roughly 10% a year for a century. Anyone who bought almost anything and held it made money. That’s called &lt;strong&gt;beta&lt;/strong&gt; — payment for tolerating risk — and it’s available to literally everyone for about three basis points in an index fund. It requires no insight whatsoever.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Alpha&lt;/strong&gt; is the return &lt;em&gt;above&lt;/em&gt; what your risk exposure alone would have handed you. That’s the scarce thing, and it’s what I spent months failing to find.&lt;/p&gt;
&lt;p&gt;Now consider the story you hear at a dinner party: &lt;em&gt;“I bought Nvidia in 2016 and it went up twenty times.”&lt;/em&gt; That’s usually beta plus survivorship. The market went up. One stock went up a great deal more. And nobody at that table is going to tell you about the position that went to zero, because that story isn’t fun and because human memory is not an auditor. The stories that reach your ears are filtered by the same mechanism that made my hand-picked stock list look brilliant.&lt;/p&gt;
&lt;p&gt;So when you evaluate anyone’s track record — mine, a fund’s, your own — the question is never “did it go up?” It’s “did it go up more than simply owning the risk would have?”&lt;/p&gt;
&lt;p&gt;With that distinction in hand, Buffett gets much more interesting. Some academics took his sixty-year record apart, and the answer is more useful than either “he’s a genius” or “he got lucky.” What they found:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;His Sharpe ratio is about &lt;strong&gt;0.79.&lt;/strong&gt; Excellent. Not superhuman — plenty of funds beat it over shorter windows. But he sustained it for sixty years, which nobody else did.&lt;/li&gt;
&lt;li&gt;He ran roughly &lt;strong&gt;1.6× leverage&lt;/strong&gt;, financed by insurance float — premiums collected up front and paid out later — at an effective cost &lt;em&gt;below&lt;/em&gt; US Treasury rates. Borrowing more cheaply than the government, for decades.&lt;/li&gt;
&lt;li&gt;His picks load heavily on measurable characteristics: &lt;strong&gt;quality&lt;/strong&gt; (profitable, stable, low-debt businesses), low volatility, and value.&lt;/li&gt;
&lt;li&gt;He had &lt;strong&gt;permanent capital.&lt;/strong&gt; No investors could yank their money during a drawdown, so he was never a forced seller.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Take those apart and a large share of the magic is explained. Not all — there’s real residual skill — but the headline “20% a year versus the market’s 10%” comes substantially from &lt;em&gt;structure&lt;/em&gt;: cheap leverage, a measurable tilt, a thirty-year horizon, and never being forced to sell at the bottom.&lt;/p&gt;
&lt;p&gt;That’s the third time the theme has landed, so let me say it plainly. &lt;strong&gt;High-Flyer’s edge was their counterparty. Buffett’s edge was his balance sheet and his patience. Neither was primarily about being smarter than the other participants.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;And here’s the genuinely good news buried in there: &lt;strong&gt;two of Buffett’s four advantages are already yours, for free.&lt;/strong&gt; If you’re investing your own savings, you have permanent capital — nobody can redeem you, ever. And unless you’re near retirement, you have a horizon measured in decades. Those are the two that most professional investors would kill for and structurally cannot have, because their clients will pull money after two bad quarters.&lt;/p&gt;
&lt;p&gt;You don’t have his cheap leverage, and you don’t have his judgment. But you’re not starting from zero. You’re starting with the two hardest ones already in hand.&lt;/p&gt;
&lt;h2 id=&quot;the-one-honest-hope-news&quot;&gt;The One Honest Hope: News&lt;/h2&gt;
&lt;p&gt;There’s a version of this that I still think is worth trying, and it came out of a conversation where someone pushed back on my “it’s all priced in” fatalism with a genuinely sharp objection.&lt;/p&gt;
&lt;p&gt;Yes, when news breaks, the price adjusts almost instantly — for a major stock, in well under a second. Any plan that begins “I read a headline and then trade on it” is racing firms that parse the same newswire in microseconds from a rack next to the exchange. That race is not close.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;But the initial reaction isn’t the whole move — because afterward, people react to other people’s reactions.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;That’s the good insight, and it’s exactly right. The machines finish in a second. Then a portfolio manager sees the move and revises her model overnight. Then an analyst updates a rating on Thursday. Then a pension fund rebalances at month-end. Then someone reads about it in a Sunday newspaper. The initial splash is unwinnable. &lt;strong&gt;The ripples take weeks.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;This is a documented, named phenomenon — &lt;strong&gt;post-earnings announcement drift.&lt;/strong&gt; A company beats expectations, the stock jumps immediately, and then it keeps drifting in the same direction for weeks afterward. It’s been replicated since 1968. It survived publication. It’s weaker now than it was in the 1990s, but “weaker” is the right shape of prize for an individual — the fast part belongs to the professionals, and the slow part is a claim about human under-reaction, which is something you can actually trade from a laptop.&lt;/p&gt;
&lt;p&gt;There’s also a structural reason to like it. Remember the breadth failure — 500 stocks moving as one? Events don’t have that problem nearly as badly. Nvidia reporting in February and Costco reporting in May are genuinely separate bets, separated in time and cause. That’s real independence, which is the ingredient I couldn’t manufacture before.&lt;/p&gt;
&lt;p&gt;But there’s a trap sitting right in the middle of this idea, and it’s the reason most “news sentiment trading bots” fail:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Sentiment is not surprise.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Nvidia announces record earnings. Revenue up 60%. Any language model scoring that press release will confidently report &lt;em&gt;very positive&lt;/em&gt;. The stock drops 8%.&lt;/p&gt;
&lt;p&gt;Why? Because the market expected 70%. Only the &lt;em&gt;deviation from expectation&lt;/em&gt; moves a price — the expected part was already in there, which is the efficiency point from earlier, showing up as a practical problem. A model that scores “is this news good?” will be most confidently wrong exactly where the biggest moves are, because the biggest moves happen when good news disappoints.&lt;/p&gt;
&lt;p&gt;So the job isn’t sentiment analysis. It’s measuring surprise against what was already priced in — which is harder, and which is the difference between this and the hundreds of abandoned sentiment-trading repositories on GitHub.&lt;/p&gt;
&lt;p&gt;And there’s one more landmine, which is that you can’t honestly backtest this with a modern language model. A model trained through 2025, asked to score a 2019 press release, &lt;strong&gt;knows how it turned out.&lt;/strong&gt; Not as leaked numbers — as ambient knowledge of the entire subsequent story. It will find the 2016 “we’re betting everything on AI” memo visionary because it knows. Every backtest number will be inflated, and the harder you tune, the more inflated it gets.&lt;/p&gt;
&lt;p&gt;And a warning I have to include, because the version of this idea that feels most compelling is the one that reliably loses money: &lt;em&gt;“There’s big news, I can tell the market is about to move, let me act.”&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The evidence on that instinct is unkind. By the time it’s news you can read, it’s priced. And what feels like &lt;em&gt;I saw it coming&lt;/em&gt; is, in almost every case, the market having already moved while your memory quietly reorders the events afterward so that your reaction precedes the move. That’s not a character flaw, it’s how memory works — we reconstruct sequences to make ourselves the protagonist. It’s why everyone remembers calling the crash and nobody remembers the four crashes they called that never came.&lt;/p&gt;
&lt;p&gt;Which is the argument for writing your reasoning down &lt;em&gt;before&lt;/em&gt; you act, in a file with a timestamp you can’t edit. That’s exactly what I do to the model in my forward test, and it’s a fair thing to do to yourself. After fifty trades you’ll have an honest answer about whether your instincts add value. Most people find out they don’t. Some find out they do. Either way you’ll &lt;em&gt;know&lt;/em&gt;, which is worth more than the trades.&lt;/p&gt;
&lt;p&gt;I still think the news idea is worth testing. I’d put the odds of it becoming genuinely tradable at maybe one in four. But notice what happened to the plan: it stopped being “AI predicts the market” and became “measure whether a specific, documented, decades-old human behavior still persists.” That’s a much smaller claim, and much smaller claims are the only ones that ever survive.&lt;/p&gt;
&lt;h2 id=&quot;what-actually-beats-the-index-without-predicting-anything&quot;&gt;What Actually Beats the Index Without Predicting Anything&lt;/h2&gt;
&lt;p&gt;Here’s where I landed, and it took months of failure to be willing to accept it.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;For almost everyone, buying a broad index fund and leaving it alone is the correct answer.&lt;/strong&gt; Roughly 90% of professional US large-cap fund managers underperform the S&amp;#x26;P 500 over fifteen years. These are full-time specialists with Bloomberg terminals and research staff. The honest base rate is not encouraging for the rest of us.&lt;/p&gt;
&lt;p&gt;But “just buy the index” isn’t the end of the conversation, because there are things that reliably improve on it — and every one of them works by &lt;strong&gt;predicting nothing.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Not panicking.&lt;/strong&gt; The most valuable, least glamorous edge available. Studies of actual investor returns versus the returns of the very funds they held find a persistent gap of roughly one to two percent per year — entirely from buying after things go up and selling after they go down. Automation’s real advantage isn’t intelligence. It’s that a script doesn’t feel anything in March 2020.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Tax-loss harvesting.&lt;/strong&gt; In a taxable account, hold the individual stocks rather than the fund. When a position drops, sell it, book the loss, and immediately buy something statistically similar so you stay invested. The loss offsets gains elsewhere; if you have none, you can deduct $3,000 a year against income and carry the rest forward.&lt;/p&gt;
&lt;p&gt;Worth being honest about what this is, though, because it’s usually oversold: &lt;strong&gt;it’s a deferral machine, not free money.&lt;/strong&gt; Your replacement position has a lower cost basis, so your eventual gain is bigger. You’re borrowing from your future self at zero interest. The real benefit comes from three places — the deferral itself, a rate arbitrage (you harvest short-term losses that offset income at high rates and eventually pay lower long-term rates), and the possibility you never pay at all if the shares are donated or inherited. Call it half a percent to one and a half percent a year, and it decays as your positions age and there are fewer losers left to harvest.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Rebalancing mechanically.&lt;/strong&gt; Fixed target weights, rebalance when they drift past a band. Small, free, and it enforces selling what’s expensive and buying what’s cheap without requiring you to have an opinion.&lt;/p&gt;
&lt;p&gt;And here’s the argument for adding anything else at all — the honest version, with the arithmetic shown.&lt;/p&gt;
&lt;p&gt;Adding an &lt;em&gt;uncorrelated&lt;/em&gt; return stream improves your overall result even if that stream is individually worse. The S&amp;#x26;P runs about 0.5. Suppose you add a strategy at 0.4 — worse on its own. If the two were genuinely uncorrelated, the optimal blend gives about &lt;strong&gt;0.64.&lt;/strong&gt; A 28% improvement from adding an inferior strategy. That’s the whole theory of diversification and it’s real.&lt;/p&gt;
&lt;p&gt;Now the sobering part, which most people selling you a strategy will not do. Nothing equity-based is uncorrelated with equities. At a realistic correlation of 0.4, that same blend gives &lt;strong&gt;0.55.&lt;/strong&gt; About 9% better.&lt;/p&gt;
&lt;p&gt;Nine percent isn’t nothing across thirty years. But it’s a lot smaller than the clean version implies, and it has to clear one more hurdle that backtests essentially never include: &lt;strong&gt;taxes.&lt;/strong&gt; Active trading generates short-term gains taxed as ordinary income. Index shares held for decades are taxed at long-term rates, and not until you sell. An active overlay needs to beat the index by one to two percent a year &lt;em&gt;before tax&lt;/em&gt; just to break even after it.&lt;/p&gt;
&lt;p&gt;That is a very high bar, and I’ve now watched enough beautiful backtests dissolve to have real respect for it.&lt;/p&gt;
&lt;h2 id=&quot;the-most-sophisticated-thing-i-built-told-me-to-stop&quot;&gt;The Most Sophisticated Thing I Built Told Me to Stop&lt;/h2&gt;
&lt;p&gt;So the punchline of months of GPU time, a foundation model, and sixty-one unit tests is: &lt;em&gt;own the index, keep costs low, don’t panic, harvest losses if the account is taxable, and if you want to speculate, wall it off and size it small.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;You could have read that in any personal finance book. You’ve probably already read it somewhere and it slid right off, because it’s boring and it sounds like giving up.&lt;/p&gt;
&lt;p&gt;Here’s why I don’t think the months were wasted, and I mean this sincerely rather than as consolation.&lt;/p&gt;
&lt;p&gt;Start with the counterfactual. Most people who had run my project would be trading that Sharpe 0.72 right now. The backtest was clean, the strategy was sophisticated, the chart was gorgeous, and the one test that killed it — running the same code on two hundred random stock lists — is a test almost nobody thinks to run, because why would you deliberately try to destroy your own best result?&lt;/p&gt;
&lt;p&gt;That’s the whole job, it turns out. &lt;strong&gt;The most valuable thing I built was the thing that told me to stop.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;And there’s an enormous difference between &lt;em&gt;believing&lt;/em&gt; the boring advice and having &lt;em&gt;measured your way to it.&lt;/em&gt; I now know precisely why my chart pattern wasn’t real: I ran it against two hundred random versions of itself and watched it land in the middle. I know why the foundation model didn’t help: it wasn’t too weak, it was too late. I know why more stocks didn’t help: they weren’t independent bets. I know that when someone shows me a backtest, the first questions are &lt;em&gt;what universe, how many things did you try, and did you count correlated days as independent evidence&lt;/em&gt; — and I know that most of the time, one of those three questions is fatal.&lt;/p&gt;
&lt;p&gt;What I actually built wasn’t a trading strategy. It was a detector for a specific kind of self-deception — the kind where data tells you what you want to hear because you counted it slightly wrong. That thing transfers. I’ve caught myself with it twice since, in work that had nothing to do with money.&lt;/p&gt;
&lt;p&gt;And there’s a version of this I want to leave you with, because I think it reframes the whole enterprise.&lt;/p&gt;
&lt;p&gt;When you buy an index fund, you’re not failing to pick winners. You’re making a different bet — the one that says human beings will keep organizing themselves into companies that produce more next decade than they did this decade, that capital will keep flowing toward the useful ones and away from the dying ones, and that you get paid a fee for supplying the patience that makes it possible. &lt;strong&gt;You’re betting on capitalism, not on a company.&lt;/strong&gt; Individual companies die constantly. The index quietly buries them and moves on.&lt;/p&gt;
&lt;p&gt;That bet has paid roughly 10% a year for a century. It requires no edge, no GPU, no foundation model, and no cleverness whatsoever. It only requires that you be willing to own risk and then wait — which turns out to be the rarest quality of all, because waiting feels like doing nothing, and doing nothing feels like the one thing a smart person shouldn’t do.&lt;/p&gt;
&lt;p&gt;The market doesn’t pay you for being smart. There are far too many smart people, and they’ve already looked.&lt;/p&gt;
&lt;p&gt;It pays you for being patient. And patience, unlike a trading edge, doesn’t stop working when everyone finds out about it.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;&lt;em&gt;Everything above is what I learned from a research project, not financial advice. I hold no positions based on any of it, and the account I’ve been experimenting with is paper money. If you take one thing from this: before you trust any result — mine, yours, or a stranger’s — go find out what a random version of it would have scored.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Disclaimer: Written by Human, improved using AI where applicable.&lt;/em&gt;&lt;/p&gt;</description><pubDate>Wed, 02 Sep 2026 00:00:00 GMT</pubDate></item></channel></rss>