Product Requirements Document

Mission

In order to contribute to our objectives for customer success and growing business value, we want to integrate Fabric into Empower through both compute, data access, and PowerBI capability (as part of an upgrade to our publishing capability).

Background

Today, we limit our capability to Fabric through the use of PowerBI. Our handoff of PowerBI credentials is manual, through the use of our delivery team. As of 1-12-2024 please see Step 4 of our PowerBI docs: https://docs.empoweranalytics.io/empower/docs/setting-up-power-bi-publishing

Quote below:

"Empower will securely place these values into KeyVault:

TenantID as pbi-tenant-id

ClientID as pbi-app-reg-id

ClientSecret as pbi-app-reg-secret

WorkspaceID as pbi-workspace-id

DataSetID as pbi-dataset-id

Note: These changes may take a few hours for Azure to update"

We plan to capture some of these credentials inside our UI with the Fabric Configuration page in the Settings tab. Others will be inferred potentially (and needs research to understand and verify the API endpoints exist and work as anticipated) such as DataSetID, which may be called dynamically.

Targeted Personas

Who is the primary beneficiary of this initiative? Why? Are there secondary beneficiaries?

• Barry BI Director: who may use this to setup their teams BI reports for publishing.
• Dario Data Professional: who may use this for the easy toggle fabric/databricks compute configuration, will also setup publishing flows to push to reporting layer.
• Debbie Delivery: who will do the same as Dario or at least assist Dario in doing so.

Current Behavior

We ask clients to provide TenantID, ClientID, ClientSecret, WorkspaceID, and DataSetIDs after configuring them in PowerBI. They provide these manually to a delivery engineer, who then manually adds these values to keyvault (see background section above). The delivery engineer then manually adds a powerBI publishing phase to DBtoStep to trigger a pull for all the tables in the dataset.

As part of our PowerBI mainPL phase, we enable customers to schedule kicking off PowerBI imports as part of a load group. All tables within a given datasetID are pulled into BI when this phase is kicked off.

Outcomes / Goals

• Data Integration: Users can switch from unmanaged to managed Databricks tables, which can shortcut into OneLake, enabling Direct Query mode for PowerBI/Fabric
  • NEEDS MORE BUILDOUT
• Compute Integration: Users can toggle back and forth between Databricks compute and Fabric compute (both spark) for any Flow. This means the spark job will now run on that type of compute for each run of that Flow, regardless of Flow type (Acquisition, Analytics Engineering, Publishing)
• PowerBI Publishing: PowerBI publishing is no longer its own Phase (PowerBI) but is now wrapped up into the Publishing Flow capability. PowerBI will now be a data target, with its own set of configurations, allowing customers to take advantage of targeted tables to publish just as any other target type
  • Configurations are defined on the Organizational and Enterprise level
  • Organizations and Enterprises are allowed multiple tenant/clients
• This is a multiphased release across multiple quarters. Keep in mind that the completion of this initiative (and the Epics within) are highly coupled with Fabric's roadmap and the order of features they complete for general availability.
• One change from this is a rename of the formerly PowerBI settings page. This will now be renamed to Fabric, with several subtabs:
  • PowerBI - this will be the only available tab at initial release
  • Compute - initially this will not be visible to users
  • OneLake - initially this will not be visible to users

User Stories

Managing PowerBI Tenant/Client Details

• As an organizational or enterprise admin, I want to be able to view and manage multiple powerBI tenant and client secrets from the settings page.

Viewing PowerBI Tenant/Client Details

• As an user at any level, I am able to view the powerBI tentants and clients within the organization(s) and enterprise(s) of which I am a member.

Viewing/Toggling Fabric Compute "On"

• As an executor level user or higher, I want to be able to toggle between Databricks and Fabric Compute on a specific flow.
• As a reader level user, I want to be able to view the compute (Databricks or Fabric) of a given flow.

Viewing all tables in the Data Lake and making visible to OneLake via Managed Tables

• TBD: this is still highly experimental and sill being researched

Use cases

Fabric Configuration - PowerBI

1. Viewing PowerBI Configurations for an enterprise/organization
   1. Primary Actor: enterprise/organization-level-or-above users of any role
   2. Description: being able to view the PowerBi configurations within the Empower UI settings page for a given organization
   3. Preconditions: PowerBI tenants/clients have been configured already within the PowerBI app. Empower contains these secrets within the vault already, and these secrets can be grabbed/displayed in the UI.
   4. Postconditions: any-role env-level users can view all tenants in the org, as well as any datasets within them.
   5. Flow:
      1. Navigate to the settings page
      2. Navigate to the Fabric Tab
      3. Navigate to the PowerBI sub-tab
      4. Navigate to the enterprise/organization with reader or above access
      5. View the following fields:
         1. A list of tenants that have been configured for this enterprise/organization
         2. Within a tenant:
            1. the SP Tenant ID
            2. the SP App ID
            3. an obfuscated SP Secret
            4. the expiration date of the SP secret
            5. The count of workspaces accessible within this tenant (https://learn.microsoft.com/en-us/rest/api/power-bi/groups/get-groups). With View All to see all workspaces as a table
            6. The count of datasets within each workspace within this tenant (https://learn.microsoft.com/en-us/rest/api/power-bi/datasets/get-datasets). With View All to see datasets as a filterable table.
            7. The user should be able to click into View All for either workspaces or datasets
2. Editing PowerBI configuration for an enterprise/organization
   1. Primary Actor: enterprise/organization-level-or-above users of writer and admin
   2. Description: being able to view the PowerBi configurations within the Empower UI settings page for a given organization
   3. Preconditions: PowerBI tenants/clients have been configured already within the PowerBI app
   4. Postconditions: writer role and above env-level users have their changes saved
   5. Flow:
      1. Navigate to settings page
      2. Navigate to the Fabric tab
      3. Navigate to the PowerBI sub-tab
      4. Navigate to the enterprise/organization with writer or above level access
      5. View the following fields:
         1. A list of tenants that have been configured for this environment
         2. Within a tenant:
            1. the SP Tenant ID (editable)
            2. the SP App ID (editable)
            3. an obfuscated SP Secret (editable)
            4. the expiration date of the SP secret (editable)
            5. The count of workspaces accessible within this tenant (https://learn.microsoft.com/en-us/rest/api/power-bi/groups/get-groups)
            6. The count of datasets within each workspace within this tenant (https://learn.microsoft.com/en-us/rest/api/power-bi/datasets/get-datasets)
            7. The user should be able to click into View All for either workspaces or datasets

Requirements

Functional

• List of functional description of the feature

Non-functional requirements

• List of NFRs to focus on

Out of Scope

• Funtionality and cases out of scope for planned feature
• Might be treated as future updates that not yet planned and refined

Dependencies & Constraints

• Any kind of dependency

Assumptions

• Fabric roadmap will maintain its course:
  • First, PowerBI integration (available today)
  • Second, Data integration
  • Third, Compute integration

Risks

Timeline and Milestones

View Epic timelines within this Initiative

Questions?