Governance Policies Azure Data Databricks - Azure/az-prototype GitHub Wiki
Governance policies for Databricks
Domain: azure-data
| Name | Description |
|---|---|
| Databricks Premium with VNet injection and private endpoints | Fully isolated Databricks workspace with no public IPs and workspace-level encryption |
| Description | Instead |
|---|---|
| Do not deploy Databricks without VNet injection | Use customVirtualNetworkId parameter with dedicated subnets |
| Do not use Standard SKU for production | Use Premium SKU for RBAC, audit logging, and CMK support |
| Check | Severity | Description |
|---|---|---|
| AZ-DBR-001 | Required | Deploy Databricks workspace with Premium SKU, VNet injection, and public access disabled |
| AZ-DBR-002 | Required | Create two dedicated subnets delegated to Databricks with required NSG rules |
| AZ-DBR-003 | Recommended | Create private endpoints for workspace UI/API and browser authentication |
| AZ-DBR-004 | Recommended | Enable diagnostic settings for Databricks workspace |
Deploy Databricks workspace with Premium SKU, VNet injection, and public access disabled
Severity: Required
Rationale: Premium SKU provides RBAC, audit logging, and CMK; VNet injection isolates cluster traffic
Agents: terraform-agent, bicep-agent, cloud-architect
- Microsoft.Databricks/workspaces
| Resource | Name | Purpose |
|---|---|---|
| Microsoft.Network/virtualNetworks/subnets | snet-dbr-public, snet-dbr-private | Two delegated subnets for Databricks cluster nodes (public and private) |
| Microsoft.Network/networkSecurityGroups | nsg-dbr | NSG with Databricks-required inbound/outbound rules on both subnets |
| Microsoft.Network/privateEndpoints | pe-dbr-ui-api, pe-dbr-browser | Private endpoints for Databricks workspace UI/API and browser authentication |
| Microsoft.Network/privateDnsZones | privatelink.azuredatabricks.net | Private DNS zone for Databricks workspace private endpoint resolution |
| Microsoft.Insights/diagnosticSettings | diag-databricks | Diagnostic settings routing workspace and cluster logs to Log Analytics |
Create two dedicated subnets delegated to Databricks with required NSG rules
Severity: Required
Rationale: Databricks requires separate public and private subnets with specific NSG rules for cluster communication
Agents: terraform-agent, bicep-agent, cloud-architect
- Microsoft.Databricks/workspaces
| Resource | Name | Purpose |
|---|---|---|
| Microsoft.Network/networkSecurityGroups | nsg-dbr | NSG with required Databricks communication rules for delegated subnets |
Create private endpoints for workspace UI/API and browser authentication
Severity: Recommended
Rationale: Private endpoints ensure all workspace access stays on the private network
Agents: terraform-agent, bicep-agent, cloud-architect
- Microsoft.Databricks/workspaces
| Resource | Name | Purpose |
|---|---|---|
| Microsoft.Network/privateDnsZones | privatelink.azuredatabricks.net | Private DNS zone for Databricks private endpoint name resolution |
| Microsoft.Network/privateEndpoints/privateDnsZoneGroups | default | DNS zone group registering Databricks private endpoint records |
Enable diagnostic settings for Databricks workspace
Severity: Recommended
Rationale: Track workspace access, job runs, cluster events, and notebook executions
Agents: terraform-agent, bicep-agent, cloud-architect, monitoring-agent
- Microsoft.Databricks/workspaces
| Resource | Name | Purpose |
|---|---|---|
| Microsoft.OperationalInsights/workspaces | log-analytics | Log Analytics workspace as destination for Databricks diagnostic logs |