Big Data and The Great Turkey Problem

Source: Taleb, N. (2012). Antifragile. London: Penguin

The Great Turkey Problem

A turkey is fed for a thousand days by a butcher; every day confirms to its staff of analysts that butchers love turkeys ‘with increased statistical confidence.’ The butcher will keep feeding the turkey until a few days before thanksgiving. Then comes the day when it is really not a very good idea to be a turkey. So with the butcher surprising it, the turkey will have a revision of belief – right when its confidence in the statement the butcher loves turkeys is maximal and ‘it is very quiet’ and soothingly predictable in the life of the turkey. The example builds on the adaptation of a metaphor by Bertrand Russell. The key here is that such a surprise will be a Black Swan event; but just for the turkey, not for the butcher. We can also see from the turkey story the mother of all harmful mistakes: mistaking absence of evidence (of harm) for evidence of absence, a mistake that we will see tends to prevail in intellectual circles and one that is grounded in the social sciences.

So our mission in life becomes simply ‘how not to be a turkey,’ or, if possible, how to be a turkey in
reverse – antifragile, that is. ‘Not being a turkey’ starts with figuring out the difference between true
and manufactured stability.

And Big Data.. (http://www.bigdatalandscape.com/blog/top-8-laws-of-big-data)

These are the Top 8 Laws of Big Data, based on hundreds of discussions with Big Data insiders.
1. The faster you analyze your data, the greater its predictive value. Companies are moving away from batch processing to real-time to gain competitive advantage.
2. Maintain one copy of your data, not dozens. The more you copy and move your data, the less reliable it becomes (example: banking crisis).
3. Use more diverse data, not just more data. More diverse data leads to greater insights. Combining multiple data sources can lead to the most interesting insights of all.
4. Data has value far beyond what you originally anticipate. Don’t throw it away.
5. Plan for exponential growth. The number of photos, emails, and IMs, while large, is limited by the number of people. Networked “sensor” data from mobile phones, GPS, and other devices is much larger.
6. Solve a real pain point. Don’t think of big data as a stand-alone shiny technology. Think about your core business problems and how to solve them by analyzing Big Data.
7. Put data and humans together to get the most insight. More data alone isn’t sufficient. Look for ways to broaden the use of data across your organization.
8. The focus in IT has shifted from Technology to Information. Those that fail to leverage the numerous internal and external data sources available will be leapfrogged by new entrants.

Understanding 

If we focus on tools, technologies and capabilities alone, we are doomed to assume that we know everything based on the evidence of absence or absence of evidence.  The best math in the world can’t explain the feeling of a hug.  We need to have a clear approach to data sciences that takes into account human perspective while considering the concepts or risk or antifragile / fragile systems.   (Gobble Gobble)