The 3Ds that every data engineer should know are:
- Data Warehouses
- Data Marts
- Data Lakes
What is a data warehouse?
A data warehouse is a system that aggregates data from one or more sources into a single consistent data store to support various data analytics requirements
Who needs data warehouse?
Practically every industry:
- E-commerce:
- To analyse and report on sales performance
- Create machine learning assisted shopping recommendations
- Healthcare:
- Apply AI to patient data to assist with diagnoses and treatment
- Transportation:
- Apply AI to optimise routes, travel times, equipment needs and staffing requirements
Benefits of data warehouse?
- Centralise data from disparate sources
- Creates a single source of truth
- Leverage all the company’s data while enhancing speed of access
- Facilitates smarter decision using BI
What is a Data Marts:
A data mart is an isolated part of the larger enterprise data warehouse that is built for a specific business function, purpose, or community of users.
For example, sales and finance departments in a company may have access to dedicated data marts that supply data for their quarterly sales reports and projections.