Are derbies unpredictable?
Form goes out of the window in a derby, or so they say. Across 297 Scottish Premiership derbies with bookmakers' odds, the favourites won about as often as the odds said. Derbies are different in one way. They're feistier, with about three-quarters of a yellow card more a game.
Beginner
Contents
The claim
"Form goes out of the window in a derby." Local pride, a packed ground and a week of build-up are supposed to make derbies a lottery, where the underdog raises its game and anything can happen.
Why people believe it
Derby upsets are remembered for years, and they're retold every time the fixture comes round. A derby the favourite wins comfortably is just another result. And derbies often are close, because the two clubs are often not far apart, which makes a shock feel more likely than it is.
The data
Every Scottish Premiership match from 2000/01 to 2025/26, from the football-data.co.uk files: results and cards for every season, and Bet365 odds from 2002/03. Six derbies:
- Old Firm: Celtic and Rangers
- Edinburgh: Hearts and Hibernian
- Dundee: Dundee and Dundee United
- New Firm: Aberdeen and Dundee United
- Lanarkshire: Motherwell and Hamilton
- Highland: Inverness Caledonian Thistle and Ross County
The method
"Unpredictable" needs a yardstick, and the fairest one is the bookmakers' odds, which already know how good both sides are. Turn each match's odds into chances, removing the bookmaker's margin so they add up to 100%, as in do the bookmakers get Scottish football right?. Then compare how often the favourite won, and how often it was a draw, with how often the odds said it would be. If derbies really are a lottery, favourites should win them less often than the odds suggest. The same comparison on every other match shows what "normal" looks like.
The evidence
No more upsets than the odds expect
In derbies the favourite was expected to win 44.6% of the time and won 42.1%. With 297 matches, luck alone can move that by about ±5.8 points either way, so a 2.5-point shortfall is well inside it. Draws came in almost exactly as priced: 27.3% expected, 26.9% happened.
What's true is that derbies are closer on paper. The favourite's chance in a derby averages 44.6%, against 53.2% in other matches, because the two sides are usually nearer each other in strength. The bookmakers already know that. The derby itself doesn't add any extra chaos on top.
Derby by derby
| Derby (matches) | Favourite: odds → won | Yellow cards |
|---|---|---|
| Old Firm (78) | 45.7% → 51.3% | 4.74 |
| Edinburgh (71) | 43.1% → 38.0% | 4.29 |
| New Firm (64) | 44.7% → 40.6% | 3.25 |
| Lanarkshire (36) | 46.2% → 38.9% | 4.00 |
| Dundee (31) | 42.9% → 38.7% | 3.79 |
| Highland (17) | 45.2% → 35.3% | 4.24 |
The number in brackets is how many of each derby had bookmakers' odds, which start in 2002/03. Yellow cards are the average per match across every meeting since 2000/01.
Favourites did a little worse than priced in five of the six derbies, and better in the biggest, the Old Firm. But each derby on its own has few matches, and luck alone can move these figures by 11 to 24 points. None of the gaps is bigger than that.
Where derbies really are different
| Per match | Derbies | Other matches |
|---|---|---|
| Yellow cards | 4.09 | 3.31 |
| Red cards | 0.27 | 0.20 |
Derbies are feistier: about three-quarters of a yellow card more a game, roughly a quarter more, and luck can only account for about ±0.21 of that gap. There are slightly more red cards too, 0.27 against 0.20, though that difference is only just bigger than luck (±0.06). The Old Firm tops the list, at 4.74 yellow cards a match.
Verdict
Not supported. Derbies aren't a lottery. Favourites win them about as often as the odds say, and draws happen as often as expected. Derbies look unpredictable because they're usually between closely matched sides, and the odds already allow for that. What really does change in a derby is the temperature: about three-quarters of a yellow card more a game.
Caveats
- Which derbies. Six local derbies are included; others, such as Dundee United against St Johnstone, could be argued for. The Old Firm dominates the total.
- Odds from 2002/03. Results and cards cover every season from 2000/01, but the test against the odds starts two seasons later.
- The odds are the yardstick. If bookmakers systematically misprice derbies, this test would miss it; but the point of the test is that derbies don't surprise the people whose job is to price them.
- Cards measure refereeing too. More cards could mean more fouls, stricter refereeing in big matches, or both; the data can't separate them.
Reproduce the analysis
The results files are published by football-data.co.uk. Download the Premiership file (SC0) for each season from 2000/01 to 2025/26 and save each under its own name, such as SC0_2425.csv; they aren't rehosted on this site. Then:
Show the Python49 lines, ready to copy and run.
import csv
from collections import defaultdict
from math import sqrt
names = [f"{y % 100:02d}{(y + 1) % 100:02d}" for y in range(2000, 2026)]
DERBIES = {frozenset(pair): name for pair, name in [
(("Celtic", "Rangers"), "Old Firm"), (("Hearts", "Hibernian"), "Edinburgh"), (("Dundee", "Dundee United"), "Dundee"),
(("Aberdeen", "Dundee United"), "New Firm"), (("Motherwell", "Hamilton"), "Lanarkshire"), (("Inverness C", "Ross County"), "Highland")]}
groups = defaultdict(lambda: defaultdict(float))
for s in names:
with open(f"SC0_{s}.csv", encoding="latin-1") as f:
for r in csv.DictReader(f):
if r.get("FTR") not in ("H", "D", "A"):
continue
derby = DERBIES.get(frozenset((r["HomeTeam"], r["AwayTeam"])))
for key in ("All derbies", derby) if derby else ("Other matches",):
g = groups[key]
if r.get("HY") and r.get("AY"): # cards: every season
g["carded"] += 1
y, red = int(r["HY"]) + int(r["AY"]), int(r["HR"]) + int(r["AR"])
g["yellows"] += y
g["yellows_sq"] += y * y
g["reds"] += red
g["reds_sq"] += red * red
if r.get("B365H"): # odds: from 2002/03; the margin is removed so the three chances add up to 1
inv = {c: 1 / float(r[f"B365{c}"]) for c in "HDA"}
p = {c: v / sum(inv.values()) for c, v in inv.items()}
fav = max(p, key=p.get)
g["n"] += 1
g["fav_expected"] += p[fav]
g["fav_won"] += r["FTR"] == fav
g["draw_expected"] += p["D"]
g["draws"] += r["FTR"] == "D"
for key in ["All derbies", "Other matches"] + sorted(set(DERBIES.values())):
g = groups[key]
n = g["n"]
luck = 2 * sqrt(g["fav_expected"] / n * (1 - g["fav_expected"] / n) / n)
print(f"{key}: {n:.0f} matches with odds; favourite expected to win {g['fav_expected'] / n:.1%}, won {g['fav_won'] / n:.1%} "
f"(luck ±{luck:.1%}); draws expected {g['draw_expected'] / n:.1%}, happened {g['draws'] / n:.1%}; "
f"{g['yellows'] / g['carded']:.2f} yellow and {g['reds'] / g['carded']:.3f} red cards a match over {g['carded']:.0f}")
# are derbies really feistier, or could the card gap be luck? (two standard errors of the derby average)
d, o = groups["All derbies"], groups["Other matches"]
for card in ("yellows", "reds"):
mean = d[card] / d["carded"]
luck = 2 * sqrt((d[card + "_sq"] / d["carded"] - mean ** 2) / d["carded"])
print(f"{card}: derbies {mean:.2f} a match (luck ±{luck:.2f}), other matches {o[card] / o['carded']:.2f}")