According to the 2023 State of Data and Analytics report by Snaplogic and Corinium, 86% of data leaders report an increase in demand for data and analytics expertise.
90% say AI will drive the single greatest change in their data strategy over the next three to five years.
Throughout my career, I’ve spent considerable time working with big data and using its potential to drive crucial business insights. I’ve seen many large shifts in the industry over the years, and now we are at another turning point.
Generative AI (GenAI) holds a lot of opportunities for organisations when it comes to data management but there are barriers to entry as some industries are slow to modernise.
I see this friction causing a few shifts in the data landscape in 2024. Here are my observations.
Data leaders know their role is becoming more important
GenAI can process and create insights around large sets of data including unstructured data.
To use GenAI correctly, organisations need to have the right systems in place.
According to PwC, 44% of business leaders said their companies are planning to implement data modernisation efforts in 2024 to take better advantage of GenAI
Data leaders will be relied on to modernise data strategies and create systems that unify and extract data in the simplest and most effective way. To deal with these increased demands we may see larger data teams with more resources at their disposal in 2024.
Technological maturity is still a problem
Modernising data strategies involves digitising data, moving it to the cloud, ensuring compliance and managing risks.
It sounds simple but it’s one of the biggest challenges data leaders face because modernising a data strategy requires a certain level of technological maturity.
In just one business day I encounter enterprises with high digital maturity and those who are still decades behind.
Those slow on the uptake usually have the right talent in place but struggle with legacy system inertia, operational silos and regulatory hurdles (to name a few issues).
Modern organisations face challenges too, mostly with implementation.
For example, organisations are investing in data warehouses and data lakes to help deliver on the promise of agile analytics and actionable business insights.
But, 83% of IT decision-makers are not completely satisfied with the performance and output of their data management and data warehousing solutions.
As a result of these challenges, I see more data leaders looking for ways to overcome challenges around technological maturity and implementation in 2024.
It’ll become clearer which organisations have an inconsistent data strategy
As organisations rush to modernise data systems, it’ll become obvious who is left behind.
Let’s look at the retail industry for example. There will be companies like Hush, that integrate data to keep accurate data on stock levels, personalise sales and gain insights for improvements.
Then there will be others that confuse customers with disjointed shopping experiences.
Online, customers may receive personalised recommendations and exclusive promotions. In-store, there may be no record of their prior purchases, payment details or preferences. The result? Missed sales.
As organisations move to keep up with innovations in the world of data, we will begin to more clearly see who has been successful and who still has work to do.
GenAI will push upskilling and smart shopping decisions for tech teams
Neural nets and predictive analytical models aren’t brand new to data scientists, but discussions around large language models and GenAI has exploded over the past year.
GenAI can help organisations scan, read, translate and analyse large amounts of data. It can also integrate data between platforms with little effort, a move that can lead to even more powerful insights.
What professionals will begin focusing on now is upskilling so they can understand and use GenAI, and other AI tools, effectively.
The challenge for IT leaders lies in finding the right talent and choosing where to focus.
With so many competing priorities leaders will need to decide on their priorities and choose the right products to help achieve their goals.
Of growing interest to IT leaders are Independent Software Vendors (ISVs) like Snowflake and native Amazon Web Services (AWS).
ISVs and cloud computing platforms will increasingly offer GenAI solutions to help make data management easier and more effective. Decision-makers navigate their purchasing choices with GenAI options and roadmaps in mind.
In the current business climate, I see choices also coming down to cost-effectiveness and return on investment.
Integrating data will be easy with GenAI
I’d need a series of reports to dive into all the opportunities GenAI presents to data experts. Instead, I’ll focus on data integration as one example of opportunities GenAI presents to data teams.
GenAI has the power to simplify data integration across ISVs and cloud platforms to create a powerful holistic view of activity. Integrating data in this way can radically change an organisation’s analytical capabilities.
With a few prompts, generative AI can:
- Automate data integration including automating data mapping, transformation, and matching, reducing the need for manual intervention.
- Create real-time data integration helping to integrate and analyse data in real-time, enabling organisations to make timely decisions and respond swiftly to changes in their environment.
- Integrate cloud-based platforms connecting data hosted on different platforms to offer deeper insights.
- Allow self-service data integration empowering non-technical users to integrate, analyse, and derive insights from data without extensive technical expertise.
- Enable serverless data integration so organisations can execute data integration functions without managing the underlying infrastructure.
SnapGPT is an example of a GenAI platform that integrates cloud-based platforms and offers self-service data integration. For instance, after one prompt SnapGPT can automatically extract Salesforce data and move it into Snowflake. Or create a data flow between Salesforce and Netsuite. All without the need for coding and long lead times.
GenAI will help data professionals do their jobs and help non-technical workers handle data without needing high-level expertise.
By allowing more employees to become citizen developers, organisations give them the power to harness data for their own means. Imagine what your marketing, sales and customer relations team could do with better data?
ITV is a great example of what can happen when data is accessible to more people.
The famous UK broadcast network empowered 90+ salespeople with real-time data allowing for faster administration, better customer service and a great boost in overall team performance.
Join our technical workshops
While citizen developers may not need to acquire new technical skills, data experts will, as mentioned, make this a priority in 2024. Learning about the best tools (AI and otherwise) to handle data, extract data and derive its maximum value is more important than ever.
That’s why we’re hosting technical workshops across APAC about AI-optimised data conversions. Suited to tech professionals with a foundation of data ingestion, processing, storage and visualisation techniques, this workshop is ideal for those hoping to extract maximum value from their data, as well as those who are simply curious about the best tools to do this.
Technical experts will explain key features and use cases, walk through demos and be available to answer your questions one-on-one.
If you’re interested in a short but impactful upskilling session I hope you will join us!
