Earthquakes and the Limits of Predictive Analysis

This October, six Italian scientists and an Italian government employee were sentenced to six years in jail for what amounts to giving inadequate warning of a potential earthquake and the risks associated with such an event. The scientific community has strongly condemned this situation. From an IT perspective, this story illustrates the risks and limits of prediction.

On April 6, 2009 a 6.3 magnitude earthquake struck the Italian medieval city of L'Aquila. The quake wrecked tens of thousands of buildings, killed 308 people and injured more than 1,000 others. The city had previously been ravaged by earthquakes in 1349, 1461 and 1703. For some period of time before the earthquake, the city experienced dozens of lower-level tremors.

On Oct. 22, six Italian scientists and a government official who served on the National Commission for the Forecast and Prevention of Major Risks were convicted and sentenced to six years in prison (pending appeal) by an Italian court for criminal manslaughter and causing criminal injury. Basically, they were accused of negligence and malpractice for failing to evaluate the dangers of potential earthquakes in 2009 and to keep the public and government informed of related risks. According to news reports, the scientists downplayed, but did not exclude, the potential for a catastrophic quake. They were convicted despite the well-recognized fact that there is no reliable method for predicting earthquakes.

A Cautionary Tale on Predictive Analytics

On a personal note, I started my business career some time ago as an operations research/management science analyst, so I find the growing interest in business intelligence subjects--including predictive analysis and big data--fascinating. Still, there are some issues to note about predictive analytics:

• Recognize that predictive analysis is not new, even though new tools are being developed. For example, time series analysis, notably the Box-Jenkins model, has been around since 1976. Most importantly, even techniques that work well have limits and require the ability to interpret data correctly (which is a serious limitation in earthquake analytics, as gaining any measure of accuracy has proven elusive).

• Predictive analysis has philosophical limitations apart from its methodology that restricts its usefulness. If I had my way, the book The Black Swan: The Impact of the Highly Improbable, by Nassim Nicholas Taleb, would be required reading for anyone studying or promoting predictive analytics. Taleb discusses the extreme impact of certain kinds of rare and unpredictable events that, retrospectively, people try to fit within simplistic explanations. Even though L'Aquila has been struck by devastating earthquakes in the past, such events were rare and the current state of science made predicting earthquakes impossible.

Mesabi Musings

Consider what might have happened if better predictive analysis had been available for earthquake analysis. Well, first there is the matter of precision. Given past performance, one could logically predict that the stock market will go up over the course of the next 10 years, but what happens tomorrow and over the next 12 months eludes even the best forecasters (although some do better than others).

Precision doesn't always matter, but in the case of earthquake predictions it does. What might have happened if the Italian scientists had raised a warning? What could they have said? Evacuate? There wasn't sufficient evidence. Conditions are unsafe and you are at your own risk? Warnings would have been dismissed, with good reason.

We can deeply sympathize with the loss of life, health and property as a result of the L'Aquila earthquake. However, scapegoating seven people who, though they probably did not communicate as well as they should have, could not have changed the results of the disaster will not solve the problem.

As for IT, while the increased use of predictive analytics can have real value in many cases, organizations must keep things in perspective. What are limitations and risks of predictive analysis if applied inaccurately or improperly? While the consequences are not likely to be as severe as the L'Aquila earthquake, putting too much faith in predictive analytics can waste time and money. Proceed carefully, with eyes wide open.