Happy Groundhog Day!
The big prediction is in for 2018. Punxsutawney Phil—the star of the most popular annual Groundhog Day event—saw his own shadow this morning (as he does about 70% of the time).
According to tradition, this means that there will be 6 more weeks of Winter!
But how likely is that prediction to come true?
We bring you a statistical analysis of all the Groundhog Day predictions we could find (compared against weather data collected by the National Climactic Data Center).
And Punxsutawney Phil is just the beginning. We have been collecting results from more than 50 different Groundhog Day events, each with their own separate predictions. We now have “big data” including 213 Groundhog Day predictions from over the past 10 years.
Predictions: Punxsutawney Phil
Punxsutawney Phil’s predictions seem a lot like flipping a coin.
In our original analysis, we found that Punxsutawney Phil’s predictions were correct 48% of the time.
As of our analysis last year, his predictions were correct 52% of the time.
Punxsutawney Phil saw his shadow in 2017 (incorrectly predicting a long Winter). We saw another warm Spring in 2017, with some of the warmest temperatures on record.
Here’s what it looks like when we update the results (correct predictions are highlighted in green).
- Punxsutawney Phil has made a correct prediction 50% of the time.
- Punxsutawney Phil has made an incorrect prediction 50% of the time.
- Probability that this pattern is just random chance (Fisher’s Exact Test, 2-tailed): 37%.
- This pattern of data would definitely not be considered “statistically significant” because the probability that it’s due to random chance is much greater than 5%.
Predictions: Groundhog “Big Data”
Let’s combine data from all the Groundhog Day predictions.
What are the chances of Groundhog Day predictions coming true? Here are the results (correct predictions are highlighted in green).
- Overall, Groundhog Day predictions have been correct 59% of the time.
- Overall, Groundhog Day predictions have been incorrect 41% of the time.
- Probability that this pattern is just random chance (Fisher’s Exact Test, 2-tailed): 6%.
- This pattern of data wouldn’t quite be considered “statistically significant” but it’s getting really close. You could call it “marginally significant.”
This continues a trend that we have observed for several years now: Groundhog Day predictions actually are correct more often than not.
They certainly aren’t always correct, but there is something to it.
Why does Groundhog Day “Big Data” outperform Punxsutawney Phil?
The “base rates” could be working against Phil
The most obvious problem for Punxsutawney Phil is that he usually sees his shadow (about 70% of the time). Perhaps this bias is getting in the way.
It would have been less of a problem years ago, when an “early” Spring was less common. But “early” Spring is becoming more and more common (in our data as of last year, they happen 72% of the time).
If you look separately—just at the years Punxsutawney Phil didn’t see a shadow—the prediction of early Spring has been correct 8 out of 9 times. Phil might be able to improve by not seeing his shadow quite so often.
Strength in numbers
Sometimes, more information simply leads to better predictions.
Sir Francis Galton famously observed/discovered this phenomenon during a contest at a livestock fair, in which an audience of about 800 people was invited to guess the weight of an ox.
- The actual weight of the ox was 1,197 pounds.
- Most individual guesses were either too high or too low.
- When Galton stacked all the guesses the audience made, from lowest to highest, the one right in the middle (1,208 pounds) was the closest guess.
- When Galton took the average of all the guesses, it was 1,197 pounds!! Spot on.
It worked with an ox, seems to be the same with groundhogs.
Let’s just take last year as an example.
We were able to track down 24 new predictions reported in the news.
- 15 of them (62.5%) made a prediction that turned out to be correct (no shadow/early Spring).
- 9 of them (37.5%) made a prediction that turned out to be incorrect (shadow/6 more weeks of Winter).
If you are going to make a prediction based on Groundhog Day, you wouldn’t want to use just one groundhog.
The best strategy is to find out if a “groundhog consensus” exists, and to know what direction they’re leaning. We know that, for some reason, the groundhogs are right slightly more often than wrong, so we can use that consensus to make the best (groundhog-related) guess.
It doesn’t stop with groundhogs.
You can see this phenomenon almost anywhere predictions are made.
Improving reliability tends to improve validity. And finding consensus tends to boost predictions.