Starship
Conviction

My system won 69% of the time. My real alpha was zero.

Six months building a trading system on insider buying, and the measurement that proved I had no edge.

My dashboard showed 69% winning predictions across 42 verdicts. Statistically solid: less than a one-in-a-hundred chance it was luck. I could have stopped there, taken a screenshot, and started putting money behind it.

That number meant nothing. Here is why, and what it took to find out.

The thesis came from research, not intuition

The starting premise wasn't a market hunch. It came from a reasonably well-established body of academic work: when several executives at the same company buy its stock at the same time, the stock tends to outperform. The literature puts the effect at around 7.4% a year on US small caps, and indicates that roughly a quarter of that edge materialises in the first five days.

The edge I was after was structural, not intellectual. Small caps are too small to interest large funds: a manager moving a billion dollars cannot take a meaningful position in a four-hundred-million-dollar company without moving the price against himself. So a niche remains where public information is absorbed more slowly.

I wasn't betting on being smarter than the market. I was betting on being small enough to go where it doesn't.

One signal, and all the rigour spent on measurement

The previous system stacked more than ten unvalidated signals. That was a mistake: stacking weak signals only builds an edge if each one is valid in isolation and their errors are uncorrelated. I had verified neither condition.

Conviction was built the other way round. One signal, with the entire engineering effort spent on how to judge it.

The rule was fixed in advance and never moved: at least two distinct executives, open-market purchases — code P on the SEC's Form 4, not option exercises — a rolling ten-day window, and the S&P 600 universe exclusively. Three verification horizons: five, ten and twenty days.

Around that rule, the measurement apparatus:

  • Every prediction was sealed with a SHA-256 hash at creation, before its outcome was known. Impossible to rewrite after the fact, even unintentionally.
  • Performance was measured against IJR, the US small-cap ETF, not in absolute terms.
  • The signal was dated to the SEC filing date, not the transaction date. That correction had flipped the sign of the previous system.
  • A filter separated opportunistic insiders from routine ones, following Cohen, Malloy and Pomorski (2012).
  • A conservative cost model, calibrated by liquidity tier, with a page showing gross and net side by side.

All of it in Python on PostgreSQL, a daily pipeline at 08:00 UTC, and more than thirty-two tests.

The rising-market trap

Here is why 69% meant nothing.

A stock that rises while the whole market rises proves nothing about the quality of the choice. The only question that matters: did my signal do better than simply buying the index?

Measured against IJR, the share of winning predictions fell to 54.8%, with roughly a 44% chance of getting that result by pure luck. In other words: nothing.

The numbers, as they are

  • Winning in absolute terms: 69% — significant, p ≈ 0.8%
  • Winning against IJR: 54.8% — not significant, p ≈ 44%
  • Average gross alpha: +0.36%
  • Average net alpha, after costs: +0.08%
  • Version 1, honestly re-measured: −0.49%

The edge didn't vanish in a reasoning error. It vanished in transaction costs, on thinly traded stocks where the spread between buying and selling is expensive.

What stopping actually cost

I shut the system down. It's the correct decision, and the most unpleasant one to take when you've spent six months building the measuring instrument that tells you you're wrong.

A rigorous measurement system doesn't protect you from error: it makes error visible. That's why you build one, and that's why it hurts.

What continued after the shutdown

Stopping the system didn't stop the measurement. A testing bench took shape around one simple rule: every candidate is pre-registered with its hypothesis, its exact rule and its numerical promote-or-kill criteria, before its results are looked at. A modified candidate becomes a new candidate. The lab may kill freely; it never promotes.

Three candidates were registered, then all killed on 22 September by a historical replication rule: when the same signal, tested over a history ten times longer, fails to replicate, the candidate dies — whatever its running score.

One of them deserves a note. When I killed it, I had written a falsifiable prediction: its net alpha would be at or below +1.0% at forty verdicts. The threshold has been reached. The net alpha is +1.48%, with 72.5% success against the index.

My own prediction is contradicted. This is exactly the case the protocol is meant to make impossible to ignore: an officially abandoned signal outperforming what its own gravedigger announced.

Two readings remain open, and I haven't settled it. The candidate was killed on historical replication, not on its live verdicts. And forty observations are still forty observations.

What I never tested

  • The strictly opportunistic subgroup with enough data: it is stuck at nine verdicts.
  • Several weak, uncorrelated signals: post-earnings inattention, forced selling with no informational content.
  • Near-real-time detection, from eighteen hours after filing down to two.

What I take from it

An honest backtest is the real skill. Producing signals is cheap; knowing which one is worth anything is the only work with value, and therefore the one to industrialise.

The verdict at sixty will settle it. Until then I don't know, and I'd rather write that than pick the reading that suits me.

12 reactions

One project a week, in your inbox

What was built, the numbers it produced, and what did not work. No other email, and one click to leave.