Traditional relational data warehouses are increasingly being complemented by or transitioned to non-relational big data. With the change to big data, new skill sets, approaches, and technologies are required. Even with the changes that have to be implemented and roles impacted, there are benefits to transitioning from data warehousing to big data.
How to transition from data warehousing to big data
Big data, or cloud-based, storage provides quick and efficient processing and deployment, increased security, and automated disaster recovery. However, planning for the transition should be thorough. There are five essential steps in data migration:
1. Define business goals for data migration
Define the business goals for migrating your data warehouse to the cloud. Goals can include performance improvements to both the back- and front-end of applications, flexibility to scale cloud resources up, and/or savings on operational expenses like software, hardware, utilities, services, space, labor, systems administration, and maintenance and support staff.
2. Evaluate current data warehouse architecture
Evaluate the current data warehouse architecture to assess the need to redesign and/or restructure data models and processes before transitioning to the cloud. It is imperative that your data warehouse architecture is sufficient for both business intelligence (BI) and big data integration.
3. Define the migration strategy
Define the migration strategy, utilizing an incremental migration or hybrid migration strategy. Moving an entire data warehouse in a given instance is unnecessarily time-consuming, disruptive, tedious, and potentially error-prone.
When major design changes are part of the transition to the cloud, incremental data migration is necessary. With the hybrid strategy, both the on-premises data warehouse and the cloud data warehouse can operate simultaneously as the data between the two are synchronized.
4. Select migration tools and services
Select specific tools and services needed for the data migration including data integration, warehouse automation, and virtualization tools and system integrators.
5. Migration testing and execution
At the end of the planning and preparation process, final test planning and executing of the migration process is performed. Once tested, users and applications can be moved to the cloud data warehouse.
