Reference Sat v1 - ScalefreeCOM/datavault4dbt GitHub Wiki
This macro calculates a virtualized load end date on top of a version 0 reference satellite. This column is generated for usage in the reference tables, and only virtualized to follow the insert-only approach. Usually one version 1 sat would be created for each version 0 sat. A version 1 satellite should be materialized as a view by default.
Compared to a regular version 1 Satellite, a version 1 reference Satellite includes the parent reference keys, instead of a parents Hashkey. Besides that, it shares the same features as a standard v1 Satellite, which are:
- Calculating a virtualized load-end-date on top of a version 0 satellite
- Enforces insert-only approach without loosing time ranges for business vault entities
Parameters | Data Type | Required | Default Value | Explanation |
---|---|---|---|---|
ref_sat_v0 | string | mandatory | - | Name of the underlying version 0 refrence_satellite. |
ref_keys | string/list | mandatory | - | Name of the references key(s) inside the parent reference Hub. Should be the same as defined in the underlying version 0 reference Satellite. |
hashdiff | string | mandatory | - | Name of the hashdiff column inside the underlying version 0 reference satellite. Needs to be similar to the 'src_hashdiff' pararmeter inside the sat_v0 model. |
src_ldts | string | optional | datavault4dbt.ldts_alias | Name of the ldts column inside the source models. Needs to use the same column name as defined as alias inside the staging model. |
src_rsrc | string | optional | datavault4dbt.rsrc_alias | Name of the rsrc column inside the source models. Needs to use the same column name as defined as alias inside the staging model. |
ledts_alias | string | optional | datavault4dbt.ledts_alias | Desired alias for the load end date column. |
add_is_current_flag | boolean | optional | False | Optional parameter to add a new column to the v1 sat based on the load end date timestamp (ledts). If set to true it will add this is_current flag to the v1 sat. For each record this column will be set to true if the load end date time stamp is equal to the variable end of all times. If its not, then the record is not current therefore it will be set to false. |
{{ config(materialized='view',
schema='Core') }}
{%- set yaml_metadata -%}
ref_sat_v0: nation_0_rs
ref_keys: N_NATIONKEY
hashdiff: hd_nation_rs
add_is_current_flag: true
{%- endset -%}
{% set metadata_dict = fromyaml(yaml_metadata) %}
{{ datavault4dbt.ref_sat_v1(ref_sat_v0=metadata_dict.get('ref_sat_v0'),
ref_keys=metadata_dict.get('ref_keys'),
hashdiff=metadata_dict.get('hashdiff'),
add_is_current_flag=mmetadata_dict.get('add_is_current_flag')) }}
-
ref_sat_v0:
- nation_0_rs: This satellite would be the version 1 reference satellite of the underlying version 0 reference satellite for nation.
-
ref_keys:
- N_NATIONKEY: The satellite would be attached to the reference hub 'nation_rh', which has the column 'N_NATIONKEY' as a reference key column.
-
hashdiff:
- hd_nation_rs: Since we recommend naming the hashdiff column similar to the name of the satellite entity, just with a prefix, this would be the hashdiff column of the reference satellite for nation.
-
ledts_alias:
- loadenddate: The 'ledts' column will be called 'loadenddate'
-
add_is_current_flag:
- true: This will add a new column to the v1 sat based on the load end date timestamp (ledts). For each record this column will be set to true if the load end date time stamp is equal to the variable end of all times
Click Me!
WITH
end_dated_source AS (
SELECT
N_NATIONKEY,
hd_nation_rs,
rsrc,
ldts,
COALESCE(LEAD(ldts - INTERVAL '1 MICROSECOND') OVER (PARTITION BY N_NATIONKEY ORDER BY ldts),TO_TIMESTAMP('8888-12-31T23:59:59', 'YYYY-MM-DDTHH24:MI:SS')) as ledts,
N_COMMENT,
N_NAME,
N_REGIONKEY
FROM datavault4dbt_demo.core_Core.nation_rs
)
SELECT
N_NATIONKEY,
hd_nation_rs,
rsrc,
ldts,
ledts,
CASE WHEN ledts = TO_TIMESTAMP('8888-12-31T23:59:59', 'YYYY-MM-DDTHH24:MI:SS')
THEN TRUE
ELSE FALSE
END AS IS_CURRENT,
N_COMMENT,
N_NAME,
N_REGIONKEY
FROM end_dated_source