Lifecycle: Initial Deployment via Notebook - GT-Analytics/fuam-basic GitHub Wiki

General

This page describes how to set up FUAM Basic step by step via Notebook.

The deployment of FUAM Basic can be done with very little effort, since we tried to automize as much as possible. image

Prerequisites

  • A user account with permanent Fabric Administrator Entra rights
  • Ability to create a Service Principal on your tenant
  • Ability to create a Workspace on your tenant
  • Power BI or Fabric capacity
  • Enabled admin setting 'Users can create Fabric items' for FUAM workspace admin(s)
  • Optional: Ability to access an Azure Key Vault on your tenant

Deployment process (Gen2)

The following steps need to be done:

1. Download Notebook

2. Prepare your environment

2.1 Create and authorize Service Principal

  • Create a new service principal with client secret within Azure Entra ID, without any API permissions.

Info: Enabling some of the Power BI API permissions might cause errors when executing pipelines later on.

  • Add the service principal to a group enabled for the following two admin settings:
    • Service Principals can use Fabric APIs
    • Service Principals can access read-only admin APIs

2.2 Create a Workspace

  • Create a new workspace "FUAM" (name can be changed), which is backed by a P or F-capacity
  • Download the Workspace logo
  • Add the logo to FUAM workspace

3. Import and Run Notebook

  • Import the recently downloaded Deploy_FUAM_Basic_Gen2.ipynb Notebook into your FUAM workspace

  • Run the Notebook

Info: The notebook will automatically create two new cloud connections (without credentials):

Connection 1 Connection 2
Connection Name pbi-service-api admin fabric-service-api admin
Connection Type Web v2 Web v2
Base Url https://api.powerbi.com/v1.0/myorg/admin https://api.fabric.microsoft.com/v1/admin
Token Audience Url https://analysis.windows.net/powerbi/api https://api.fabric.microsoft.com
Authentification Service Principal Service Principal

Error handling: In case of an error, you'll be able to run the notebook again. It has an update mechanism, which will act as an item update.

4. Add credentials to connections

  • Navigate under Settings to 'Manage connections and gateways' in Fabric

  • Set the credentials of the connections with the recently created service principal information:

Info: These connections are used in FUAM pipelines to retrieve data from REST APIs. If the credentials are incorrect or the secret has expired, the pipeline will fail.

5. Run orchestration Pipeline

Info: The Load_Basic_Package_Sequentially_E2E is the main end-to-end orchestration pipeline of FUAM Basic. It contains/triggers all other sub-pipelines (FUAM modules), which are implemented in the solution. The sub-pipelines download all the required data via APIs and the referenced Notebooks will transform/write the data to the final FUAM_Lakehouse delta tables.

  • Navigate to your FUAM Workspace
  • Search for the item 'Load_Basic_Package_Sequentially_E2E'
  • Open the Load_Basic_Package_Sequentially_E2E pipeline

The Pipeline has different parameters, which are controlling the data load flow:

Parameter Name Description Allowed values
has_tenant_domains If true, the tenant inventory is enriched with domain information. Use it only, if domains are in use at your tenant. Default is false true or false
extract_powerbi_artifacts_only If true, the tenant inventory contains only semantic models, dataflows, datamarts, reports, dashboard and apps. If false the pipeline extracts Power BI and Fabric items. Currently, first-party workloads are supported only. Default is false true or false
activity_days_in_scope It defines how many days in the past the activity must be retrieved from the API. Recommended to use 30 for the initial load and change the value to 2 for daily load. range between 1 and 30
display_data If true, the notebooks will display more information about each relevant step at runtime. This is useful for debugging. Default is false true or false
optional_keyvault_name Optional: If you have configured a key vault, enter the name of the key vault. Otherwise, simply leave this field blank. In this case, the Load_Inventory module will use the Notebook owner's identity. empty or string
optional_keyvault_sp_tenantId_secret_name Optional: If you have configured a key vault and its secretes, enter the name of the tenantId secret name. Otherwise, simply leave this field blank. In this case, the Load_Inventory module will use the Notebook owner's identity. empty or string
optional_keyvault_sp_clientId_secret_name Optional: If you have configured a key vault and its secretes, enter the name of the clientId secret name. Otherwise, simply leave this field blank. In this case, the Load_Inventory module will use the Notebook owner's identity. empty or string
optional_keyvault_sp_secret_secret_name Optional: If you have configured a key vault and its secretes, enter the name of the service principal's secret secret name. Otherwise, simply leave this field blank. In this case, the Load_Inventory module will use the Notebook owner's identity. empty or string

  • Run the Pipeline once

Error handling:

Make sure that you are signed-in with a User, which has 'Fabric Administrator' rights on the tenant

Make sure that the SP has granted access for the two tenant settings: 'Service Principals can use Fabric APIs' and 'Service Principals can access read-only admin APIs'

Review the credentials of the connections

Set the 'activity_days_in_scope' to 28, whenever the 'Load_Activities' pipeline got an error

Or visit the Remarks to FUAM wiki page.

6. Open FUAM Report

6.1 Refresh semantic model

  • Navigate to your FUAM Workspace
  • Search for the item 'FUAM_Basic_PBI_Overview_SM'
  • Click on 'Refresh' semantic model

6.2 Open PBI report

  • Navigate to your FUAM Workspace
  • Search for the item 'FUAM_Basic_Overview_Report'
  • Open the FUAM_Basic_Overview_Report report
  • Feel free to explore the report pages

Error handling: In case of errors like 'Visual can't be rendered', please visit the Remarks to FUAM wiki page.

7. Schedule Pipeline for daily load

  • Navigate to your FUAM Workspace
  • Search for the item 'Load_Basic_Package_Sequentially_E2E'
  • Open the Load_Basic_Package_Sequentially_E2E pipeline
  • Click on Run -> Schedule
  • Configure the schedule

Congratulations!

Your FUAM Basic solution is ready to use.

Other helpful resource