Call it what you want:
- The Nexus of Forces (Gartner)
- The Age of the Customer (Forrester)
- The Third Platform (IDC)
Whatever your terminology, we?ve clearly come to the end of one technology cycle and entered into a new cycle of innovation and business and IT transformation. Whether it?s Social, Mobile, Analytics and Big Data, Cloud Computing or the Internet of Things (SMACT), not to mention Bring Your Own Everything, these are exciting and challenging times for enterprise IT leaders seeking to transform their organizations while improving business alignment. CIOs have become ?change agents.” They’ve become Chief Innovation Officers and even Chief Integration Officers, as one of the first places they?re looking to initiate this change is by improving the connections between internal and external data, applications and APIs, and eliminating disconnected data silos. We?ve written about the Integrator?s Dilemma extensively in recent months, as well as enterprise “cloudification” and the resulting shift in “data gravity.” Increasingly, however, I’m hearing terms like “heavyweight,” “technical debt,” and “IT burden” used to describe the legacy middleware that was purchased and implemented well over 10 years ago. It was built for a different time and now often seen as the change blocker, not a change agent. On the other hand, in conversations with industry analysts, customers, and partners I’m hearing terms like ?fabric,? ?agility layer,? and “self-service” used to describe the desire for a more flexible approach to integration and data management.
Sound familiar? 
As integration platform as a service (iPaaS) gains market awareness and acceptance, it?s becoming more and more apparent that a new ?middle kingdom? is emerging to address the integration requirements of the modern enterprise. Using Old ETL or EAI to tackle the volume, variety and velocity of today?s data, application and API integration requirements (not to mention IoT) is a bit like driving a Tesla Model S on a dirt road. (Okay, I’m still working on the metaphor, but this image comes to mind.) Recent financial results aside (examples here and here), I’ve put together a list of 10 reasons why old extraction, transformation and loading (ETL) and enterprise application integration (EAI) tools (and solution providers) will continue to struggle in the SMACT era:
- Cannibalization of the Core On-Premises Business: This is the obvious one. Innovator’s Dilemma. It’s hard to “skate where the puck is going” without making big bets on the future and being willing to walk away from underperforming product lines. It can be done (here’s a great example), but I haven’t seen this level of commitment from the incumbents in the data and application integration market. Back in 2007, my friend Ken Rudin presented “SaaS Cannibalization and the Civil War Within.” It still rings true.
- Heritage Matters in the Cloud: I’m generalizing, but if you come from the EAI/ESB/SOA world you typically have an application-centric view of the world. If you come from the world of business intelligence, data warehousing and master data management, you naturally have a data-centric view. Re-thinking, or to use Mary Meeker’s word – re-imagining – integration requires the vision to tackle both equally. Message-oriented EAI tools struggle with bulk/batch data movement and transformations. Rows and columns-based ETL tools struggle with real-time, unstructured and hierarchical data. Modern, JSON-centric, RESTful platforms clearly have an advantage in the new era, whether your requirement is event-based, real-time, streaming or scheduled batch-oriented data integration.
- EAI without the ESB: Features like guaranteed delivery, triggers, monitoring and orchestration are just as important in the cloud as they were on-premises. But the enterprise service bus (ESB) does’t fly in the cloud. Here are some comments from Forrester’s Stefan Ried on the topic and here’s why SOA was DOA thanks to the ESB.
- Beyond ETL: Features like re-use, aggregation, joins, union, splitting, SCDs and scheduling are just as important in the cloud as they were on premises so I’m not going to jump in on the ETL is Dead debate, but multi-purpose or multimodal integration is clearly the future. This webinar with Dave Linthicum and Gaurav Dhillon provides some good perspective on the topic.
- Point to Point Misses the Point: For legacy ISVs, all roads lead back to point number one – cannibalization – but simply introducing a light-weight cloud (or cloud-washed) version of your on-premises software does not a cloud strategy make. While there is a growing demand for more approachable, easier-to-use cloud integration services, there must also be advanced functionality (see examples of iPaaS requirements here), broad connectivity and the ability to go beyond simple point-to-point integration scenarios. Otherwise, the only thing that’s going hybrid is your integration hairball.
- Franken-tegration: Speaking of hybrid, it’s a hot topic in enterprise IT and cloud computing, but when it comes to an iPaaS solution, hybrid shouldn’t mean half-baked. A cloud integration service must able to handle complex on-premises integration use cases (cloud to ground) and cloud to cloud. It also has to provide more than just monitoring, but also a sophisticated integration design and administration environment that does not require on-premises tools. More on this topic in the post Fluidity in Hybrid Deployments.
- Big Data Integration is not Core…or Cloud: For EAI vendors, big data integration isn’t an option. For legacy ETL vendors, built to scale up, not out, and designed to primarily handle structured, relational sources and targets, big data is a big problem. And that’s just for the on-premises technology. Cloud computing and big data, not to mention mobile, social, APIs and IoT, are changing the rules and requirements of integration and a modern, elastic platform is going to be necessary to keep up.
- Elastic Scale Out: I wrote about why this is important in this post, iPaaS Requirements: Elastic Scale. Re-purposing legacy technology for integration design or run-time processing guarantees that the solution wasn’t built for the cloud and elastic to the core.
- An On-Ramp to On-Prem: Some of this comes down to sales compensation plans (paying reps only on first year annual contract value (ACV) and not on multi-year agreements, for example), but often point-to-point cloud integration tools from legacy vendors are used as a loss leader to open doors in new accounts or expand into divisions of accounts. The longer term objective is a bait and switch, which leads me to my final point.
- Focus and DNA: As I mentioned, all roads lead back to point number one and the Innovator’s Dilemma, but there is an Innovator’s Solution and proven techniques to Escape Velocity. Of course, we believe there’s also an Integrator’s Solution, but for legacy ETL and EAI independent software vendors (ISVs), the question is whether they have the wherewithal to adapt and truly embrace the change that the SMACT era will require. It will not only require a complete re-focus on the future and a willingness to walk away from the past, which is particularly difficult for public companies, and require a shift to subscription pricing, a new approach to customer adoption and renewals and agile development. It will also require a DNA of innovation and a willingness to change. When you’re on the inside, I compare it to the frog in boiling water story, but I think Aaron Levie, the visionary CEO of Box.com, put it best when he recently tweeted:
When trying to change an industry from the inside, it’s easy to see why it won’t work. When you’re outside, you can’t believe it won’t work. ? Aaron Levie (@levie) August 8, 2014
