The Prediction Market Scorecard That Counted Easy Calls as Proof

Saying the favorite won nine times out of ten proves very little when most races have an obvious favorite.

The midterms were billed as the first real test of whether prediction markets beat pollsters. The money was there. The Anti-Corruption Data Collective (ACDC), a nonprofit research and advocacy group, found $133 million bet on the 2026 midterms by August 10, already above the $92.4 million wagered across the entire 2024 congressional cycle, and Reuters later reported $220 million as of late September. Kalshi and Polymarket's combined monthly volume doubled from May to July to $53 billion, mostly on sports, according to a Pew analysis.

Going into November, the signals largely agreed. On October 30, Kalshi priced a Democratic House at 92% and Polymarket at 94%, with Democratic Senate control at 69% on both. A polls-based forecast model I checked, as of October 24, had Democrats winning the House in 94.7% of simulations and the Senate in 62.1%. A comparison from Action Network found markets and polls broadly aligned on the House, with the Senate more complicated and some individual races where the two weren't backing the same candidate.

Platforms have been making accuracy claims. Kalshi says candidates favored by its markets since 2024 have won roughly nine times out of ten, even three months before Election Day. Polymarket's spokesperson says its election markets are 98% accurate one day out.

Both signals said essentially the same thing about the House, so the House result can't tell you which one is smarter.

The 2024 election produced the founding myth. Kalshi had Trump at 57% and Polymarket at about 62% on election eve, while polls called it a toss-up, and the outcome was widely hailed as markets beating polls. But a 62% favorite wins about 62% of the time, and one outcome can't tell a calibrated forecaster from an overconfident one. As one market analyst put it afterward, the result was consistent with both accurate markets and an unremarkable polling error.

A Vanderbilt study by Joshua Clinton and TzuFeng Huang examined more than 2,500 political markets and $2.4 billion in transactions in the final five weeks of 2024. It reported accuracy of 93% for PredictIt, 78% for Kalshi, and 67% for Polymarket, so the largest platform ranked last. Clinton said the paper wasn't really about accuracy. Its concern was that traders weren't reacting to political reality so much as to each other.

Base rate neglect: Ignoring how common an outcome is in the reference class when judging evidence (Kahneman and Tversky, 1973).

Most House and Senate races are lopsided by partisan lean. Calling the favorite in them demonstrates little, since a naive rule of backing the party that won the district last time is also right most of the time. Only competitive races separate good forecasters from lucky ones, so a headline accuracy rate that pools easy and hard calls inflates the evidence.

Calibration and the Brier score: A forecaster is calibrated if events given X% probability occur about X% of the time. The Brier score measures the squared gap between stated probabilities and outcomes, penalizing both overconfidence and underconfidence (Brier, 1950).

Calibration is the fairer test, and Kalshi has begun publishing Brier scores for its election markets, which is a good step. Accuracy rates don’t show whether 90% calls come true 90% of the time.

Herding (information cascades): When people follow what others do instead of using their own information, early or loud signals get amplified and independent information is lost (Bikhchandani, Hirshleifer and Welch, 1992).

Markets and polls are not independent. Kalshi says polls inform its odds, so agreement between the two is partly an echo. That matters for the tempting claim that two methods “confirming” each other is stronger evidence than either alone.

Wisdom of crowds, and its conditions: A crowd’s average beats individual judgment when members have independent information, diverse views, and decentralized decision-making (Surowiecki, 2004).

The ACDC data raise a question about those conditions. The top 1% of Polymarket wallets accounted for 68% of congressional trading volume, a figure Reuters later updated to 70%. Ten wallets alone produced 17% of volume and traded in markets touching 91% of races. Only 39,820 Polymarket wallets bet on congressional races at all, and 80% of its congressional markets had fewer than 100 participating wallets. Kalshi held 95% of its volume in just 50 of 470 races. That is closer to a small set of heavy traders than to a diverse crowd, though the ACDC data come from Polymarket’s international platform, which publishes wallet data, so I can’t say the same forKalshi.

The odds are being cited as if they were public opinion. ACDC found that 87% of Polymarket's congressional markets were either under $10,000 in volume or had high volume held by very few traders, which means one well-funded order can move a displayed probability even if nothing about the race changed. A price from a thin market is a number, not a consensus.

There is also a feedback loop. ACDC notes that campaigns, media outlets, and donors increasingly treat these prices as signals, and warns that market odds could undermine trust in elections when results go the other way. A forecast that influences donations, coverage, and expectations isn't only measuring the election.

Finally, platforms are scoring themselves. Accuracy figures like "90%" and "98%" are company statements, built on definitions the companies choose. Prediction markets also drew scrutiny during primaries after a handful of races where probabilities were far off the final result.

• Score forecasters on competitive races only, with the race list and timestamps declared before the election.

• Benchmark every market against two baselines: a naive partisan-lean rule and a polls-based model at the same timestamps. Skill is the margin over those.

• Use proper scoring, such as Brier scores and calibration curves, not hit rates.

• Report participation with every price: number of independent traders, concentration, and spread. Don't present a thin-market price as a probability.

• Separate "more accurate" from "faster." Markets update instantly, and polls take time. Test both dimensions instead of blending them into one claim.

• For users of forecasts: treat a market as one input with known weaknesses, and check its depth before citing it.

• Markets may still add information. Money at stake, fast updating, and possibly informed traders are real strengths, and the 2024 result is real evidence. My point is narrower: a high hit rate on mostly easy races doesn't show a market is well calibrated, and it shouldn't be presented as if it does.

• One election is a small sample. A 70% favorite loses 30% of the time, and a handful of competitive races can't establish calibration either way.

• The critique applies to polls too. "The polls were wrong" is often scored the same lazy way, pooling easy calls with hard ones.

• The concentration data have limits. They come from one platform, from August and September, and from an advocacy group. As one analysis noted, the 68% figure alone can't show odds were wrong or manipulated, only what the number represents.

• The accuracy figures come from different sources. Kalshi and Polymarket claims are company statements, and the Vanderbilt numbers cover a five-week window of 2024 across many markets of varied difficulty, which shows how much the measure matters.

• The right comparison needs data I don't have yet, and I'm not claiming markets or polls won.

When you cite a forecast, do you know how it did in the races that were actually in doubt?