It’s Groundhog Day, and we’ve got data. Big Data!
Of course, we can’t ignore Punxsutawney Phil, the main attraction at one of the most popular and long-running Groundhog Day celebrations. His prediction from today is that the winter of 2015 will last until mid-March.
But how good is Phil at actually predicting the weather? We begin by analyzing his legendary career.
Then we go much bigger.
In possibly the largest analysis to date, we studied 134 predictions made by 42 groundhogs over the past 7 years…and you can only see the results right here.
How much wood would a woodchuck chuck? That’s a silly question.
Perhaps a better question: can they accurately predict the weather?
On February 2 of each year, people gather to observe a groundhog emerging from its burrow in the morning.
Its reactions are thought to predict the changing seasons.
According to the tradition, if it’s sunny and the groundhog sees its own shadow, it will often retreat back into the burrow—this apparently indicates that winter will continue until mid-March. On the other hand, if it is overcast and there is no visible shadow, it is predicted that springtime will come early.
We begin with Punxy Phil, the longest-running set of predictions available from a single groundhog.
Our comparison includes weather data provided by The National Climactic Data Center from 1988 to 2014. We considered above-average temperatures by March to indicate early spring in a given year, and average or below-average temperatures to indicate a regular winter.
There are 4 possible outcomes to consider:
- Shadow + Regular Winter (correct)
- No Shadow + Early Spring (correct)
- Shadow + Early Spring (incorrect)
- No Shadow + Regular Winter (incorrect)
Just looking at the numbers, you might see what looks like an amazing trend. When Phil doesn’t see his shadow, an early spring came 7 out of 8 times! That’s 87.5% accuracy, right?
Wrong. This only looks impressive if you ignore the base rates. Phil usually sees a shadow, and there have been many recent years with an unseasonably warm end of winter.
For example, if we just look at the years that springtime came early, Phil made the correct prediction (no shadow) 7 out of 20 times—that’s 13 incorrect predictions vs. 7 correct.
One of the best ways to understand this type of data is to calculate the pattern that would be predicted just by random chance (rounded to whole numbers for ease of interpretation).
As you can see, these numbers actually aren’t very different from Phil’s predictions.
Even easier, we can just break down what percentage of the time each outcome occurred. The correct predictions are highlighted in green.
As you can see, Phil was correct 22 + 26 = 48% of the time.
The probability that Phil’s pattern of results is due purely to random chance is about 63% (Fisher’s Exact Test, 2-tailed).
Groundhog Big Data
Punxy Phil is just one groundhog. Is that really a representative sample?
The Cangrade team is always looking to take data science to the next level, and today is no exception.
We collected data from the predictions of 42 groundhogs made between 2008 and 2014, and again compared the results with weather data provided by The National Climactic Data Center.
We again see what might look like somewhat promising patterns for the ‘hogs…
…until you notice the base rates. We again see a preponderance of early springs in this analysis, and this time around we see a different pattern of shadow vs. no shadow, with no-shadow results relatively more common.
Here is the pattern that would be predicted just by random chance (rounded to whole numbers for ease of interpretation). It’s a little bit different.
And here is percentage of the time each outcome occurred. Correct predictions are highlighted in green.
As you can see, the 42 groundhogs were on average correct 13 + 40 = 53% of the time.
The probability that this pattern of results is due purely to random chance is about 20% (Fisher’s Exact Test, 2-tailed).
As a general rule, a pattern of data would only be considered “statistically significant” if such probability was less than 5%.
There is still a very, very small chance that all those groundhog predictions, when put together, might be slightly better than just flipping a coin.
You never know…
Things aren’t looking too promising for groundhog predictions.
But if you’re still feeling optimistic, here are some hot new contenders to maybe look out for:
- French Creek Freddie (French Creek, WV) 100% prediction rate
- Chuckles (Manchester, Conn) 100% prediction rate
- Staten Island Chuck (New York, NY) 80% prediction rate
- Queen Charlotte (Charlotte, NC) 75% prediction rate
- Dover Doug (Dover, PA) 75% prediction rate
Just don’t hold your breath…or look up what the actual lifespan of a groundhog is. That would be depressing.
And if you still haven’t had enough, here are yesterday’s Super Bowl “predictions” from various animals:
Image credit: Matt Reinbold