Start your third week with a new company by attending a trade show? Why not, right — how hard can it be?
Being the new boy in the SnapLogic booth at Big Data Minds Europe, I was eager to do two things: not make a complete fool of myself, and talk to as many of my fellow attendees as possible to understand their reasons for attending. With no preconceived notions, I wanted to hear directly from these leading practitioners about their concerns and what the path to big data analytics looked like from their point of view.
The one aspect that came up in every single conversation is the value of data, and the consequent need for ever more data. Over the two days of the event, I heard from companies in a variety of different situations, from well-established data teams with hundreds of analysts, to one group that was less than a month old. The strategies for gathering data were equally varied, from the straightforward API and database connections, to Wiener Linien, the public transport operator in Vienna, where getting data involves sending drones to survey the city from the air and robots into the tunnels underneath.
Beside that universal goal, an equally universal complaint was the difficulty of getting access to data in the first place — even when you don’t need to use drones! The data that analysts want access to *exists* within the companies and organizations; the problem is that it is distributed across any number of different systems, each one owned by a particular group which has its own goals, view of the world, and definition of success.
The stakes were also very different from one organization to another, with the International Committee of the Red Cross in particular laboring under constraints that most of us have never had to consider. All of the attendees, however, shared a desire to involve domain experts more in both their decision-making and in day-to-day operations, simply because the people closest to it have the best understanding of their own data.
Another concern, shared especially by the attendees from the financial sector, was around data security and the need to be in compliance with applicable laws and regulations. People were particularly wary of the cloud — not to the point of ruling it out, as the advantages are unarguable, but definitely taking a careful and selective approach as to which workloads and data are a good fit for different infrastructure options. The European Cloud User Coalition (ECUC) in particular is fostering nuanced discussions around this critical topic.
On a related topic, and especially among the more mature organizations, there is a concern with the cost of different IT options, and especially with runaway cloud bills. This emergent discipline of integrating cost planning within IT decisions is becoming known as FinOps.
How SnapLogic can help achieve the goals of Big Data projects
The good news is that we were able to have a number of productive conversations about these topics at the SnapLogic booth. The entire goal of a low-code/no-code data integration platform like ours is to enable the free flow of data across the organization, by working at a number of different levels that are well aligned to what we heard from practitioners attending the event. Here are a few suggestions that resonated with the big data professionals at the show:
- Empower domain experts. These are the people who understand where the data is coming from, and where it needs to go. In the past, their experience of getting involved in IT projects was of being on the outside, communicating with the people actually doing the work by writing (and endlessly revising) exhaustive requirements documents. With SnapLogic, they can get involved directly in the definition of the data flows, dragging and dropping components and even applying data transformations as they go, with a live preview of the data to guide them.
- Address cloud concerns. By offering both on-premise and cloud options, SnapLogic offers the best of both worlds: local and secure access to local data and systems, as well as flexible direct access to data in SaaS systems.
- Integrate IT best practices. Empowering of data users outside of IT enables deeper conversations between IT and the business. The question of the cost of different infrastructure options is neither a purely technical problem, nor a purely business problem. Cost must align to value, which in the end only the business knows. Vice versa, IT will be able to advise the business on how to achieve the best fit to that business value. What is even better is if this distributed knowledge can be encoded in automated flow, instead of requiring many iterations of back-and-forth requirements gathering.
- Run IT at the speed of business. Business users need results fast. The time scales can vary — some requirements are near real time, like customer experience personalisation, while others can tolerate longer time-scales, like quarterly sales reports and projections. At either end of the scale, though, faster is better — so a platform that can support thousands of API calls per second will help improve outcomes for everyone.
It was great talking to everyone in Berlin, but if you missed the event and would like to share your own goals and concerns, we would love to hear from you to understand your situation better and work out together how SnapLogic can help.
This is how we support you in achieving your business goals: by simplifying access to your data so that you can stop playing catch-up with your fragmented data stack and get on with delivering ever more business value.
