import {make_map} from "./components/make_map.js"
house_topology = FileAttachment("./data/house_districts_2026.json").json()
house_forecasts = FileAttachment("./data/house_2026_comparison_normalized.csv").csv()
house_outcomes = FileAttachment("./data/house_2026_outcomes.csv").csv()
make_map({
topology: house_topology,
forecasts: house_forecasts,
outcomes: house_outcomes,
mode: "comparison",
invalidation
})Polymarket vs FLIPR House predictions
Comparing predictions for the US House elections by Polymarket and the Silver Bulletin’s FLIPR model
Prediction markets, like Polymarket, have quickly become widely used to create probabilistic predictions for all sorts of interesting events. Sports, politics, finance - even the weather are all subjects where you can find market-based odds for a wide variety of events.
FLIPR is the latest election model by well established statistical analyst Nate Silver. His reputation was first set in 2009 when his model correctly predicted the results of the 2008 U.S. presidential election in 49 of 50 states. He’s continued to be an outstanding analyst ever since.
This post compares the predictions made by Polymarket and FLIPR for the recent US House elections and, surprisingly (to me), Polymarket comes out on top.
NoteDisclaimer
This analysis is based on one contest; it’s not my intention to declare Polymarket as superior to FLIPR more generally.
It’s also worth mentioning that FLIPR creator Nate Silver posted another comparison noting that Models led prediction markets in forecasting the blue wave. That post focuses on how the overall projected margins in the Senate evolved over time, though. He also discussed governor’s races.
This post, by contrast, focuses on the House, which has many more contests. We use the Brier score to compare how accurate the final predictions were just the day before the election.
Brier score
The Brier score is a common metric used to quantitatively assess two sets of predictions for a common set of events. Specializing to this post, we could define it as follows: If we’ve got a total of \(N\) elections in congressional districts, then the Brier score (BS) is defined as
\[ \operatorname{BS} = \frac{1}{N}\sum_{i=1}^{N}(p_i-y_i)^2, \]
where \(p_i\) is the predicted probability of a Democratic win in district \(i\), \(y_i\) is 1 if the Democrat wins and 0 otherwise. Probabilities are expressed on a scale from 0 to 1.
This is effectively the mean square error of the predictions. A lower Brier score indicates more accurate predictions.
The overall results
Not quite all of the results have been finalized, as of this writing. Additionally, there are four districts that Polymarket has no predictions for, namely CA-22, CA-27, CA-40, and ME-02.
The Brier scores based on the 415 results that we have so far are
| Predictor | FLIPR | Polymarket |
|---|---|---|
| Brier score | 0.028516 | 0.024896 |
Of course, this is just one set of contests and the difference between the Brier scores is \[0.028516 - 0.024896 = 0.00362.\] We can get a handle on the significance of this difference using a standard technique called the paired nonparametric bootstrap. This allows us to estimate the standard error associated with this process.
Based on 100,000 redraws with replacements from our sample, I estimated a standard error of \(0.00237\). Thus, the difference of \(0.00362\) is about \(1.5\) standard errors from the mean. I’d say that suggests that Polymarket has the advantage in this particular set of circumstances. The advantage is not decisive, however.
The predictions
Perhaps more interesting than the overall result is - where does one technique out perform the other?
The map below shows the US Congressional districts shaded by difference between Brier scores. More specifically, by \[ \text{FLIPR Brier score } - \text{Polymarket Brier score}. \] Since smaller is better,
- a negative difference means FLIPR performed better and
- a positive difference means Polymarket performed better.
The map
Most of the map is white, since most of the predictions are close to one another. In 289 out of the 415 districts where we’ve got data, the difference in Brier score is less than 0.001.
It looks like there are some clusters of significant differences, including in my own state of North Carolina. In fact, the top three Brier differences are in North Carolina and one more is in the top 10. Depending on the outcome of NC-5, one top NC Brier difference might join the top 10.
Tabular data
Here’s a tabular look at the same data set. You can use the search box to list the results for a state or a particular district. You can click a column header once or twice to sort the data by that column’s values in ascending or descending order.
This table includes only the called races with forecasts from both sources. FLIPR and Polymarket are Democratic win probabilities, and Result is 1 for a Democratic win and 0 otherwise.
Summary
Again, Polymarket outperformed FLIPR noticeably, if not decisively.
Polymarket, however, did not provide predictions for four districts. Three of those districts in California were easy predictions; ME-02, though, not so much. While these four districts probably didn’t affect the results here significantly, this should still be viewed as a weakness in the comparisons. One shouldn’t choose districts to avoid.
Generally, Polymarket was more bullish on Democrats’ chances. Among the 415 available districts, FLIPR’s predictions implied 236 Democratic wins, while Polymarket prices yielded about 241. There were, in fact, Democratic 252 wins in those districts so both sets of predictions were a little short.
Of course, the election was strong for Democrats overall. Since these trends are often correlated, this might explain Polymarket’s strong performance in this particular election.
Comments