Post by Rags on Oct 2, 2012 15:34:03 GMT
I don't want to bang on about whether we will get promotion or not, I'm simply using this quote as a "tag" as it was this post I thought of when I was reading about about Nate Silver's book "The Signal and the Noise: Why Most Predictions Fail – But Some Don't" over the weekend (available in hardback on e-bay but at a fairly hefty price so maybe worth getting from the library or waiting for the paperback edition):
Book blurb:
Drawing on his own groundbreaking work, [Nate] Silver examines the world of prediction, investigating how we can distinguish a true signal from a universe of noisy data. Most predictions fail, often at great cost to society, because most of us have a poor understanding of probability and uncertainty. Both experts and laypeople mistake more confident predictions for more accurate ones. But overconfidence is often the reason for failure. If our appreciation of uncertainty improves, our predictions can get better too. This is the "prediction paradox": The more humility we have about our ability to make predictions, the more successful we can be in planning for the futurewww.us.penguingroup.com/static/pages/features/the_signal_and_the_noise.html
Silver observes that the most accurate forecasters tend to have a superior command of probability, and they tend to be both humble and hardworking. They distinguish the predictable from the unpredictable, and they notice a thousand little details that lead them closer to the truth. Because of their appreciation of probability, they can distinguish the signal from the noise..
I first stumbled across Silver's work when he created the PECOTA forecasting system for Baseball Prospectus (which is subscriber-protected so if you really want to know what that is, look here: en.wikipedia.org/wiki/PECOTA).
It's one of the great things about baseball that a large number of meaningful KPIs can be assigned to performance which in turn means that the concept of forecasting performance is relatively straightforward as long as you only concentrate on the KPIs that are directly attributable to player actions, eg a hitter's contact rate is useful, his batting average is not.
It's partly a bad thing that the same can't be done with football players, although ProZone tries hard to give some semblance of accountability to player performance; but it's also a fantastic thing that football cannot be reduced to mere statistics and patterns.
If the baseball bounces one way or another different outcomes can happen but as soon as the ball is returned to the catcher all players return to their starting positions and that basic play is repeated. If a football bounces one way or another all 20 outfield players are dragged into different positions which gives another situation for the ball to move in an unpredictable way.
It's why I hate it when a manager says that if the ref had given the penalty his team wouldn't have lost the game. Really? How do we know that if Team A beat Team B 1-0, they wouldn't have beaten Team B 3-1 had Team B had been awarded the penalty that their manager is moaning on about? We don't, because ball- and player-movements mean that each minute of a game has a number of outcomes that then create more opportunities which have even more outcomes. The only real constant is at the point of kick-off which only happens once every half and after every goal.
What Silver is saying in his book (I think - I haven't actually read it yet) is that there is a lot of informational "noise" that finds its way into predicting: details that aren't relevant but are incorrectly considered to be useful so they are mistakenly used to fuel the argument.
For example, the statement that Man City haven't won at Swansea for 28 years is factually accurate but taking into consideration the fact that they have only played 1 game in Swansea since Apr 94, there is no usefulness in using that fact in any prediction. Or that Chelsea's record of only 4 home Premership wins against Arsenal in the past 10 years is irrelevant to their next meeting at Stamford Bridge as the comparison is not between the same sets of players or even managers, just the same team names.
As the Wall Street Journal says in their review
[Nate] Silver is a well-known forecaster and the founder of the New York Times political blog FiveThirtyEight.com, which accurately predicted the outcome of the last presidential election. Before he was a Times blogger, he was known as a careful analyst of (often widely unreliable) public-opinion polls and, not least, as the man who hit upon an innovative system for forecasting the performance of Major League Baseball players. In "The Signal and the Noise," he takes the reader on a whirlwind tour of the success and failure of predictions in a wide variety of fields and offers advice about how we might all improve our forecasting skill.
Mr. Silver reminds us that we live in an era of "Big Data," with "2.5 quintillion bytes" generated each day. But he strongly disagrees with the view that the sheer volume of data will make predicting easier. "Numbers don't speak for themselves," he notes. In fact, we imbue numbers with meaning, depending on our approach. We often find patterns that are simply random noise, and many of our predictions fail: "Unless we become aware of the biases we introduce, the returns to additional information may be minimal—or diminishing." The trick is to extract the correct signal from the noisy data. "The signal is the truth," Mr. Silver writes. "The noise is the distraction."
online.wsj.com/article/SB10000872396390444554704577644031670158646.html?KEYWORDS=nate+silver
In an interview with Fast Company.com, he says:
"We need to stop and admit it: we have a prediction problem," he writes in the introduction to his new book, The Signal and the Noise. "We love to predict things--and we aren't very good at it."
The diagnosis comes from a collective failure to foresee epochal events--say the September 11 attacks or the 2008 financial crisis--and a political culture rife with constantly forecasting (and consistently wrong) experts. The solution, he says, requires a change in attitude, one that emphasizes probability.
When human judgment and big data intersect there are some funny things that happen. On the one hand, we get access to more and more information that ought to help us make better decisions. On the other hand, the more information you have, the more selective you can be in which information you pick out to tell the narrative that might not be the true or accurate, or the one that helps your business, but the one that makes you feel good or that your friends agree with.
We see this in polls. After the conventions we've gone from having three polls a day to like 20. When you have 20, people get a lot angrier about things, because out of 20 polls you can find the three best Obama polls, or the three best Romney polls in any given day, and construct a narrative from that. But you're really just kind of looking at the outlier when you should be looking at what the consensus of the information says.
www.fastcompany.com/3001794/fivethirtyeights-nate-silver-explains-why-we-suck-predictions-and-how-improve
Personally, I think we have a problem in this country with the way the media manages to successfully convince us that anything they spout is true. It's not just that many predictions are wrong, it's that the media tell us that wrong predicion are right. I could go on and on, as if I haven't already, about the way that the football pundit is consistently held up as an expert when any close analysis shows quite clearly that the pundit is consistently wrong. And it doesn't matter whether the pundit is Chris Waddle, Andy Townsend, Alan Hansen, Glen Hoddle or whoever.
My point in all this is not that we are unable, or should not, try to predict what happens in League Two promotion or relegation race this season - I've already publicly stated that it is 2 from Barnet, AFC, Wycombe and Daggers to go down this season - but that in any form of prediction, it helps to separate the proverbial wheat from the chaff to get an accurate result. And too often we don't.
Anyway, James, I didn't know if you were familiar with Nate Silver, but I reckoned you might be interested in reading his stuff if you hadn't already.
And as the guy who correctly forecasted 49 out of the 50 states in the 2008 presidential election (and got the other one wrong by 1%) maybe it's worth reading his political blog to see who will be the next US President:
fivethirtyeight.blogs.nytimes.com
