These days, it’s impossible to pick up a newspaper or magazine without reading about the tremendous changes that are about to be wrought upon our society. We’re not talking about the climate change or the surveillance state, but something that promises to improve our lives: let’s talk about Big Data.
First, a bit about what exactly we’re referring to when we talk about Big Data. We’re talking about the modern usage of extremely powerful supercomputers to store and analyze huge amounts of data on a macro scale. Though humans have almost always collected data, up until recently, we lacked the tools to analyze this data without a tremendous expenditure of effort. Data had to be combed though by hand, calculations worked out on paper, and finally compiled together, all in a coordinated effort so complicated as to be effectively impossible. Even after the invention of computers, it took decades before they were powerful enough and could store enough information to make analysis of these massive datasets possible. Even after sufficiently powerful computers were developed, they were out of reach for everyone save those with the deepest pockets. They were event called “supercomputers” to differentiate them from those used by everyone else.
But we’re in the midst of a sea of change: for the first time in history, analysis of these vast data stores is not only possible, but accessible to and feasibly for the average business. This access promises to change everything about the way we live. Welcome to the age of Big Data.
One of Big Data’s Earliest Success Stories Is in Sports
Keeping in mind the obsession that sports fans have with statistics of all kinds, it would seem natural that it would be one of the first places that Big Data would be applied. It has all the ingredients: the data is there and easily accessible, there is plenty of interest in the subject and there’s even a lot of money in it. And eventually, it happened in baseball (later, the story became the basis for the acclaimed film Moneyball).
But baseball isn’t the only place that Big Data can find applications, Indeed, hockey is another sport where it has begun to make a big impact. Scientists like Aaron Clauset have turned many erstwhile sports “rules” on their heads. In a paper published as an open access article in EPJ Data Science, Clauset debunked the established idea of “momentum,” the concept that players and teams have “hot streaks” during which many points are scored in quick succession. Clauset found this to be little more than random choice: people seeing patterns in the tea leaves.
But it doesn’t meant that Big Data can’t be used productively by coaches. A 2013 seminar on the applications of Big Data to hockey in Toronto found several ways that coaches can make use of their player data to enhance their game. By using a complicated analysis, players can be given different ratings for particularly characteristics they have, such as their ability to perform in a given position. This can help coaches make choices about where (and when) to deploy a player to maximize his or her chances of scoring. Though most sports have been more resistant to the Big Data Revolution than baseball was, it’s only a matter of time before all organized sports adopt these methods of data analysis.
Data: Better than a Gut Feeling About Someone
One of the strengths of Big Data is its ability to determine relationships between seemingly unrelated factors. Another place that data has the chance to really revolutionize the way business is conducted is in the recruiting industry. The recruiting industry is concerned with all kinds of success metrics, from talent sourcing to employee retention. A useful example is that of Gate Gourmet, a caterer for the airline industry. One of their human resources analysts noticed that their Chicago O’Hare Airport staff was experiencing a 50 percent turnover rate, and set out to determine the reason. In depth analysis found a strong connection between employee turnover and time spent commuting to work; Gate Gourmet was able to adjust their hiring strategies to find workers closer to the airport and cut their turnover rate almost in half.
Seeing the Forest for the Trees
In some ways, forest management, with its focus on long-term trends and relationships, predicted Big Data. But in those times, scientists were forced to deal with abstractions, and predictions had to be tailored to specific regions, or even specific forests. Now, foresters can get insights based on general tools just by inputting their own forest’s data. Software can account for types of trees, different growth rates, climates, weather and many other variables, and can predict outcomes over years, decades, or even centuries. This analysis has proved its value through its ability to identify high-risk areas for dangerous forest fires or even ecological collapse. That’s right: Big Data can even save lives.
A Healthy Impact
It will come as no surprise, then, that one of the most important areas that Big Data will influence the future is in the realm of healthcare. Finally, scientists have access to anonymous statistics about millions of people, providing (with due analysis) an unparalleled look into the details of people’s lives And the ways that this can help are multitudinous. One recent example was published in the Journal of American Medical Informatics Association, which analyzed people’s Google searches to identify a previously unknown drug interaction. They found that people who searched for both paroxetine (an anti-depressant) and pravastatin (a cholesterol drug) were more likely to search for symptoms of hyperglycemia, a link that was later established in a study.
Big Data: Big Conclusions
These four examples are only the tip of the iceberg. Truly, wherever you look, there are processes that can be improved by application of data, whether they be free business loan calculators or advanced hockey statistics. Here, we’ve only covered a few areas where its impact will be felt, but the real question might turn out to be: where won’t Big Data change everything?
Nick Rojas is a business consultant and write who lives in Los Angeles and Chicago. He has consulted small and medium-sized enterprises for over twenty years. He has contributed articles to Visual.ly, Entrepreneur and TechCrunch. You can follow him on Twitter @NickARojas, or you can reach him at [email protected].
