dbt Scaffolding - accelerate-data/migration-utility GitHub Wiki

dbt Scaffolding

ad-migration setup-target scaffolds the dbt project and generates staging source metadata from the catalog.

Invocation

ad-migration setup-target
Option Required Description
--source-schema no Physical schema for the canonical bronze dbt source (defaults to bronze)
--project-root no Defaults to current working directory

setup-target reads the target technology from manifest.json as runtime.target, which is seeded by /init-ad-migration.

Prerequisites

  • manifest.json must exist
  • catalog/tables/ must exist
  • the initial analyze stage must be complete for in-scope tables

Before ad-migration setup-target can proceed, extracted tables need to be in one of these states:

  • resolved to a writer
  • excluded from the migration
  • writerless and explicitly confirmed as a source

In practice, this means you should finish the scope/exclude/source decision first, then run ad-migration add-source-table <fqn> for every table that should remain a dbt source before invoking ad-migration setup-target.

Use /status if you want to verify which tables are confirmed as sources versus still pending before running ad-migration setup-target.

What it writes

dbt/
  dbt_project.yml
  profiles.yml
  packages.yml
  models/
    staging/
      _staging__sources.yml
      _staging__models.yml
      stg_bronze__<entity>.sql
    intermediate/
    marts/
  macros/
  seeds/
  tests/

Staging source behavior

This is the part that matters most operationally:

  • tables with is_source: true are included in models/staging/_staging__sources.yml
  • each included source table gets a pure pass-through models/staging/stg_bronze__<entity>.sql wrapper
  • writerless tables with scoping.status == "no_writer_found" but no is_source flag are left in the unconfirmed bucket
  • resolved migration targets are excluded from staging source metadata because they are expected to become dbt mart models
  • excluded tables do not appear in staging source metadata

So no_writer_found by itself is not enough. Source tables have to be explicitly confirmed with ad-migration add-source-table <fqn> before ad-migration setup-target. setup-target consumes those decisions; it should not be the step where you make them.

If you confirm additional source tables later, rerun ad-migration setup-target. The command is idempotent: it will regenerate staging source metadata and wrappers from the latest is_source flags and create only the missing target-side source tables.

Re-running

Re-running ad-migration setup-target is safe:

  • it regenerates staging source metadata and wrappers
  • it does not overwrite your edited profiles.yml
  • it does not overwrite generated models or snapshots
  • it creates any missing target-side source tables for items already marked is_source: true, but it does not backfill data or decide which tables are sources

Connection variables

setup-target requires the target runtime variables for the configured technology. See:

Next step

Proceed to Sandbox Setup.

⚠️ **GitHub.com Fallback** ⚠️