Home - ScalefreeCOM/datavault4dbt GitHub Wiki

Welcome to datavault4dbt

The following documentation sheds some light on the dbt-macros that have been developed by Scalefree to make your DataVault-Experience more comfortable! The documentation can be found by clicking the links in the sidebar on the right side. In the documentation, the macros and their parameters are explained and further exemplified.

Included Macros

  • Staging Area (For Hashing, prejoins and ghost records)
  • Hubs, Links & Satellites (allowing multiple deltas)
  • Non-Historized Links and Satellites
  • Multi-Active Satellites
  • Reference Data Entities
  • Virtualized End-Dating (in Satellites)
  • PIT Tables
    • Hook for Cleaning up PITs
  • Snapshot Control

Features

With datavault4dbt you will get a lot of awesome features, including:

  • A Data Vault 2.0 implementation congruent to the original Data Vault 2.0 definition by Dan Linstedt
  • Ready for both Persistent Staging Areas and Transient Staging Areas, due to the allowance of multiple deltas in all macros, without loosing any intermediate changes- Enforcing standards in naming conventions by implementing global variables for technical columns
  • A fully auditable solution for a Data Warehouse
  • Creating a centralized, snapshot-based Business interface by using a centralized snapshot table supporting logarithmic logic
  • A modern insert-only approach that avoids updating data
  • Optimizing incremental loads by implemting a high-water-mark that also works for entities that are loaded from multiple sources
  • A straight-forward, standardized approach to conduct agile datawarehouse development cycles

Requirements

To use the macros efficiently, there are a few prerequisites you need to provide:

  • Flat & Wide source data, available within your target database
  • Load Date column that represents the arriving time in the source data storage
  • Record Source column that gives information about where the source data is coming from (e.g. the file location inside a Data Lake)

Resources:

⚠️ **GitHub.com Fallback** ⚠️