Unity Catalog with Terraform - razmipatel/Random GitHub Wiki

1. “Automate Distributed Unity Catalog Mgmt” – Databricks Blog
   This blog gives an overview of how to use the Terraform provider for Unity Catalog to manage governance patterns (delegated vs constrained) across teams. Databricks
   Highlights for your context:

   • How to create a metastore and assign storage/identity for managed tables. Databricks
   • Patterns for team-based governance across a shared metastore — relevant when you have Dev/Test/Prod or multiple business units.
   • Focus on AWS, but the architectural principles carry across.

2. “Getting Started with Azure Databricks Unity Catalog with Terraform” – Medium (Robbie Douglas)
   Focused on Azure, this is a more hands-on “getting started” article. Medium
   Highlights:

   • Walks through service principal setup and metastore creation in Azure.
   • Good for your Azure landing-zone context, especially since you’re working with Entra ID / Azure AD.
   • Less enterprise policy/governance depth than some others.

3. “Automate Unity Catalog setup using Terraform” – Official Databricks Documentation
   This is a more structured “deployment guide” rather than just a blog. docs.databricks.com+1
   Highlights:

   • Pre-reqs (storage accounts, identities, service principal, etc) for both AWS & Azure.
   • Linkages to the provider docs and reference material.
   • Good for building a repeatable template/landing-zone module.

4. “From S3 to Schema: Deploying Unity Catalog with Terraform on Databricks” – DataEngineerThings blog
   More depth into IaC code snippet, managing catalog/schema/table lifecycle. Data Engineer Things
   Highlights:

   • Detailed step-by-step: S3 bucket, IAM role, storage credential → external location → catalog → schema → table.
   • Strong for building your “non-human identity automation” work in Terraform.

5. “Terraforming Azure Databricks Part 2: Unlocking Unity Catalog…” – Mark Tinderholt
   Focuses on Azure + Terraform, including Unity Catalog config. marktinderholt.com
   Highlights:

   • How to configure the Databricks provider blocks, especially when dealing with account vs workspace scopes (important for your identity/governance roles).
   • Useful since you’re designing enterprise-scale landing zones.

6. “Managing Databricks with Terraform: A modular approach” – Valcon
   While not purely Unity Catalog, it covers modular Terraform patterns for Databricks incl. UC. Medium
   Highlights:

   • Emphasises modularisation (which you will appreciate given your landing-zone and multi-env setup).
   • Touches on modules for Unity Catalog, schemas, grants — useful when designing a scalable blueprint.

---

Why these matter for you:

• You’re working on a large-scale Azure landing-zone design with strong security/identity/governance focus — Unity Catalog + Terraform plays into that by enabling infrastructure as code for data governance.
• Several articles emphasise service principals, storage accounts, identity/role setup — tightly aligned with your expertise in Entra ID, RBAC, PIM etc.
• The modular and multi-environment aspects (dev/test/prod) appear in these blogs, which fits your current project context (Dev/Test/PreProd/Prod environments).

Strategic questions for you to consider:

• How will you integrate Unity Catalog IaC with your identity/privilege model (e.g., using PIM, least-privilege for service principals, etc)?
• What is your approach for segmenting UC objects (metastore, catalogs, schemas) across environments (Dev/Test/Prod) and possibly business units?
• How will you ensure auditability, version control, and secure secrets/storage for Terraform state and service principals in this pipeline?
• How will you tie in your toolset (e.g., SailPoint, CyberArk) into the management of identities/entitlements for Unity Catalog objects and the Terraform automation itself?