Thursday 1 November 2012

In all probability........

We live in a world of information overload. Sorting out the wheat from the chaff is never easy particularly when so many of the 'facts' cannot be specified precisely. It is often difficult to make judgements from any set of data but when the information is known to be imprecise, drawing conclusions can be fraught with problems. Let's take the world of statistics and probabilities, starting with the former.

Benjamin Disraeli is attributed by many to have originated the phrase: "There are lies, damned lies, and statistics". Well perhaps before before exploring this statement further, it is worth drawing the distinction between statistics and probabilities. Statistics are numbers that represent facts and as such, deal with certainties, as long as good counting methods are employed. Probabilities, on the other hand, deal with the unknown. So Disraeli's statement is false if the implication is that statistics are a higher form of untruths than 'damned lies'. It is the misuse of statistics, very often linked with probabilities, that can result, intentionally or unintentionally, in something worse than 'damned lies'.

If the residents of a town, who are aged ten or over, are asked to report to a medical centre to have their heights and weights measured, then these measurements can be compared with the criteria for obesity and a statistic of the obesity of the population can be assessed. Let's assume the result was 34% of the population aged ten or over were, at the time the measurements were taken, technically obese. That measurement of 34% is a statistic. Now, if a government bureaucrat decides to use that statistic for other means, for example assessing the probability of obesity in the nation as a whole and publishes a statement to the effect: "it is likely that one in three of the nation's inhabitants over the age of ten, is obese", then the 34% is now a probability rather than a statistic. It would only become a statistic if it was supported by measurements taken of the entire nation's population. An assessment of probability based on the results of one town can be very misleading if the sample that was measured was not truly representative of the country as a whole.

One of the websites that I use for local weather forecasts, provides 'probability of precipitation' information, which is another way of saying the likelihood of rain. There is no indication of how that figure has been computed but there is likely to be a computer algorithm that might use current and historic data to assess the likelihood of rain. Note the last sentence used the words 'likely to be' and 'might use'; that sentence, therefore, has a probability of being correct and likewise a probability of being incorrect - just what can you believe these days?! Returning to the weather forecast, if the probability of rain is 70%, then the probability of no rain is 30%. This means that if I plan my day on the assumption that it is going to rain, then I might be (I've used the word 'might' again!!) very disappointed if it doesn't rain but I can't complain to the forecaster because he or she would point out that there was always a 30% probability of the rain not materialising. So what use is the forecast?!

I suppose our conversations would be very limited if all we communicated were certainties. There are very few certainties and none if we are talking about the future. Also the high probability, low impact events might be interesting but don't change our lives significantly. So if it doesn't rain tomorrow and I planned on it being wet, because the forecaster told me there was a 70% probability of precipitation, then my change of plan might be to do some gardening rather than work in the house. So what?! The real life-changers are the low probability, high impact events, very often not even considered. So if my house was hit and destroyed by a meteorite, oh boy that would be a life changer.

The more I think about probabilities, the more convinced I am that detailed long-term planning of our lives is a waste of time. We should live for the present, trying to improve our actions by learning from the past (positive and negative experiences) and attempt to use the present to make some positive impact on the future. In all probability you will agree with that........but then again, you might not! :-)

No comments:

Post a Comment