Do we need a data warehouse or a data lake?
It’s a fair question, because often there is a lot of confusion in the marketplace about which option is best. When it comes to your organization’s data, the last thing you want is to feel confused.
So, let’s clear up the confusion you’re having today. Keep reading to understand the definitions of data warehousing and data lakes—and the differences between them. Then, you’ll feel more confident about which solution your team needs to succeed.
What is a Data Warehouse?
A data warehouse is a centralized repository for integrated data from numerous source data systems (Finance, Sales, Operations, etc.) for reporting and data analysis. The data warehouse is powered by a relational database engine.
Data warehouses support business users in answering business questions from the data. Because the data is structured to answer specific types of business questions, these questions must be considered in advance when designing the warehouse data model.
What is a Data Lake?
A data lake is a centralized repository for data extracts coming directly from source data systems. Data lakes enable data scientists and other data consumers to access large data sets quickly and efficiently.
Data stored in the lake may be unstructured, semi-structured, or fully structured. This flexibility means that the data lake supports many different use cases, including staging raw data that can be transformed and then loaded into the data warehouse.
Data Lake vs Data Warehouse
Data Lake
Think of customers at a lumber yard. This customer’s needs are varied. One day they might need to lay down new flooring, the next day they may be repairing a roof or want to build their own dining set.
The lumberyard supports them by offering lots of raw material inventory and (hopefully) organizing it in a way that’s easy for the customer to find what they want. The lumber yard is your data lake—technically-skilled customers are served up raw materials to take and create what they need.
Data Warehouse
Now think of customers at a furniture store. Their needs are much more focused; they need a chair, a couch, a coffee table, etc. The furniture store supports its customers by building and offering products that solve specific problems. The furniture store is your data warehouse—business-focused customers are served up answers to specific questions.
Do We Need a Data Warehouse or a Data Lake?
Drumroll, please…you likely need both a data warehouse AND a data lake. They are both integral components of a modern data infrastructure. A data lake is not a substitute for a data warehouse. The data lake and data warehouse go hand-in-hand.
As a general rule, if you currently have neither, consider creating the data lake first. While the design of the data lake is important, it is a less daunting task compared to the data warehouse design. Of course, there is no one size fits all—requirements should always drive design.
The modern data warehouse approach means you will need to provide both the lumber yard and the furniture store. Because modern enterprises rarely serve just one type of customer.
________
The CSG Pro team is happy to offer further guidance in your data warehousing journey. Get in touch if you need support.
