“Life moves pretty fast. If you don’t stop and look around once in a while, you could miss it.” – Ferris Bueller’s Day Off, 1986.
It feels like there is both more uncertainty and more mainstream adoption in the Big Data market in contrast to a year ago at this time. So, a month before the 2015 Strata/Hadoop World event in New York, it feels like the right time to stop and look around at Big Data then and now.
Industry analysts have an interesting perspective on the issue, speaking regularly to both technology vendors and buyers. So let’s look at milestones from the last year from the perspective of Gartner. [A couple of these reports require a Gartner subscription, but I have referenced publicly-accessible blog posts where possible.]
August 4, 2014: Gartner releases its annual “Hype Cycle” report on Big Data technologies. (“Hype Cycle for Big Data 2014,” Frank Buytendijk, Gartner.) [subscription required]
The hype cycle is a proprietary Gartner maturity model for emerging technologies within a specific category. These regularly-issued reports show where a specific technology is in its lifecycle and how long it will take to reach the next phase of its development. The stages are:
Innovation trigger -> Peak of inflated expectations -> Trough of disillusionment -> Slope of enlightenment -> Plateau of productivity.
The notion here (my interpretation) is that every new technology has a hype stage where the buzz is greater than actual adoption or business value achieved. This phase is generally followed by a trough were the buzz wears off, skepticism creeps in and adopters must consider business needs and realities before applying to specific use cases and proceeding to the plateau where technologies contribute to measurable business productivity.
According to the 2014 report, Big Data technologies were either at the peak of hype or well into the trough:
“Big data, as a whole, has crossed the Peak of Inflated Expectations, and is sliding into the Trough of Disillusionment. Once adoption increases, and reality sets in, based on first successes and failures, the peak of hype has passed. Innovation will continue, the innovation trigger is full, but the Trough of Disillusionment will be fast and brutal. Viable technologies will grow quickly, combined with a shakeout of all the vendors that simply jumped on the bandwagon. This is essentially good news. More robust and enterprise-ready solutions will appear; and big data implementations will move from being systems of innovation to mission-critical systems of differentiation.”
Further, these technologies are seen as a long way from adding business value.
“In many cases, transformation is at least two to five years away — or more. In addition, many technologies indicate that they will become obsolete before reaching the Plateau of Productivity.”
Fast-forward to August 2015 and the updated Big Data Hype Cycle report…
But there isn’t a new report. According to an August 20 blog post from Gartner analyst Nick Heudecker, “Big Data Isn’t Obsolete. It’s Normal.” [no subscription required]
“First, the big data technology profile dropped off a few Hype Cycles, but advanced into the Trough of Disillusionment in others. Second, we retired the very popular Hype Cycle for Big Data [emphasis mine]. The reason for both is simple: big data is no longer a topic unto itself. Instead, the various topics formerly encompassing big data evolved into other areas. What other areas?
- Advanced Analytics and Data Science
- Business Intelligence and Analytics
- Enterprise Information Management
- In-Memory Computing Technology
- Information Infrastructure
The characteristics that defined big data, those pesky 3 Vs [volume, variety, velocity], are no longer exotic. They’re common.”
So – one year and Big Data goes from a hyped set of technologies still in the realm of the early adopter, to common and simply part of long-standing tech sectors such as analytics, business intelligence and information infrastructure. Why is that? And what does it mean?
In part 2, I’ll delve into adoption drivers and barriers using Gartner’s annual Hadoop adoption surveys from 2014 and 2015.
