Data Warehouse Internal - sachit914/datawarehouse GitHub Wiki

why do we need staging layer?

image

ETL

  • how we extract data and load into datawarehouse

image

  • inside datawarehouse we have two layer (staging layer and user acess layer)
  • staging layer is landing zone for data, first we load data from external source to this layer and then we transform this data image

types of staging layer

  • persistent staging layer (A persistent staging layer involves storing the data in a way that allows it to persist over time. This typically means that once data is loaded into the staging area, it remains there until it is successfully loaded into the data acess layer or until it's no longer needed.)

  • non -persistent staging layer (A non-persistent staging layer involves handling data in a transient manner. Data is loaded into the staging area, processed or transformed, and then discarded once it has been successfully loaded into the data acess layer or downstream systems.)

  • user acess layer The user access layer in data warehousing manages how users securely interact with and access data