Microsoft Fabric — The CSA-in-a-Box Platform Guide¶

TL;DR: Microsoft Fabric is the unified analytics platform that collapses lakehouse, warehouse, real-time analytics, data engineering, BI, and governance into a single SaaS boundary backed by OneLake. ADR-0010 declares Fabric the strategic target for CSA-in-a-Box: every primitive in the current PaaS stack (Delta, Purview, dbt, Spark, Power BI) maps forward into Fabric with low rewrite cost. This guide covers what Fabric is, how its components work, and where the Supercharge Microsoft Fabric companion site provides deeper hands-on material.

Why Fabric Matters for CSA¶

CSA-in-a-Box was designed from day one to be Fabric-forward. Every technology choice recorded in ADRs 0001 through 0009 was selected so it either matches a Fabric primitive directly or has a documented migration path to one (ADR-0010).

The value proposition is straightforward:

• Unified platform — One capacity, one security model, one catalog. No more stitching together ADF, Databricks, Synapse, Purview, and Power BI as independent services with separate billing, networking, and RBAC.
• OneLake as the single data lake — All workloads read and write to OneLake in open formats (Delta, Parquet, Iceberg). No data duplication across engines.
• Direct Lake — Power BI reads Delta tables in OneLake directly, eliminating scheduled imports and delivering sub-second analytics over fresh data.
• Elimination of service sprawl — Fewer Azure resources to deploy, monitor, and secure. A single Fabric capacity replaces multiple standalone services.
• Federal readiness — The current PaaS stack is production-ready for Azure Government today. When Fabric reaches Gov GA, the migration path is a connection swap, not a rewrite.

Fabric is GA in Azure Commercial. Azure Government availability (FedRAMP High, IL4+) is on the Microsoft roadmap but not yet GA as of 2026-Q2. Federal tenants continue on the PaaS stack and migrate when Gov GA lands. See ADR-0010 for the dual-track strategy.

Architecture Overview¶

Fabric organizes all analytics workloads around OneLake as the central storage layer. Each workload (Lakehouse, Warehouse, Real-Time Intelligence, Data Pipelines, Power BI, Copilot) operates on the same data without copies.

flowchart TD
    subgraph "Microsoft Fabric"
        OL[(OneLake<br/>Delta / Parquet / Iceberg)]

        subgraph Workloads
            LH[Lakehouse<br/>Spark + SQL Endpoint]
            WH[Warehouse<br/>T-SQL MPP]
            DP[Data Pipelines<br/>Orchestration]
            RTI[Real-Time Intelligence<br/>Eventstream + Eventhouse]
            NB[Notebooks<br/>PySpark / R / Scala]
            PBI[Power BI<br/>Direct Lake]
            COP[Copilot & AI Skills]
        end

        subgraph Governance
            PV[Purview<br/>Catalog + Lineage]
            SL[Sensitivity Labels]
            RBAC[Workspace RBAC]
        end
    end

    EXT_S3[AWS S3] -.->|Shortcut| OL
    EXT_GCS[Google GCS] -.->|Shortcut| OL
    EXT_ADLS[ADLS Gen2] -.->|Shortcut| OL
    EXT_DV[Dataverse] -.->|Shortcut| OL

    OL --> LH
    OL --> WH
    OL --> RTI
    OL --> NB
    OL --> PBI

    LH --> PV
    WH --> PV
    RTI --> PV

    DP --> LH
    DP --> WH
    COP --> NB
    COP --> PBI

OneLake¶

OneLake is the single, tenant-wide data lake underpinning every Fabric workload. It is to Fabric what OneDrive is to Microsoft 365 — one lake per tenant, organized by workspaces.

Core Concepts¶

Shortcuts¶

Shortcuts let you query data that lives outside of Fabric without moving it. This is the primary mechanism for hybrid architectures or incremental migration.

Open Formats¶

Fabric stores data in open formats by default:

• Delta Lake — the primary table format. Full ACID transactions, time travel, Z-order optimization, and liquid clustering.
• Parquet — read natively for any non-Delta files.
• Iceberg — OneLake exposes Delta tables via Iceberg-compatible metadata, enabling external engines (Spark, Trino, Snowflake) to read Fabric tables without conversion.

Security Model¶

OneLake security layers from coarse to fine:

1. Workspace RBAC — Admin, Member, Contributor, Viewer.
2. Item permissions — Per-lakehouse or per-warehouse grants.
3. OneLake data access roles — Folder-level read/write within a lakehouse (preview).
4. Row/Column-level security — Enforced at the SQL endpoint or semantic model layer.
5. Sensitivity labels — Purview MIP labels flow through to exports and downstream reports.

Deep dive: OneLake Security | Iceberg Interop | OneLake Catalog

Lakehouse¶

The Fabric Lakehouse combines a Delta Lake file store with an auto-generated SQL analytics endpoint for T-SQL querying. It is the natural home for medallion architecture workloads.

Medallion Architecture in Fabric¶

flowchart LR
    SRC[Source Systems] --> DP[Data Pipelines<br/>or Dataflows Gen2]
    DP --> B[(Bronze<br/>Raw Delta)]
    B --> NB[Notebooks<br/>PySpark / dbt]
    NB --> S[(Silver<br/>Cleansed Delta)]
    S --> NB2[Notebooks / dbt]
    NB2 --> G[(Gold<br/>Star Schema Delta)]
    G --> SQL[SQL Analytics Endpoint]
    G --> DL[Direct Lake<br/>Semantic Model]
    SQL --> ADHOC[Ad-hoc T-SQL]
    DL --> PBI[Power BI Reports]

SQL Analytics Endpoint¶

Every Lakehouse automatically exposes a read-only T-SQL endpoint over its Delta tables. SQL users and tools can query the lakehouse without Spark. This endpoint also serves as the backing store for Direct Lake semantic models.

When to Use Lakehouse vs Warehouse¶

Many production architectures use a Lakehouse for Bronze/Silver (data engineering) and a Warehouse for Gold (BI serving). Cross-database queries let you join across them seamlessly.

Deep dive: Lakehouse Setup | Bronze Layer Tutorial | Silver Layer Tutorial | Gold Layer Tutorial | Medallion Deep Dive

Warehouse¶

The Fabric Warehouse is a fully managed T-SQL data warehouse built on the same OneLake storage as the Lakehouse but optimized for SQL-first workloads.

Key Capabilities¶

• Full T-SQL surface — DDL, DML, stored procedures, views, functions.
• Cross-database queries — Query across Lakehouses and Warehouses in the same workspace without data movement.
• Automatic statistics and indexing — Fabric manages physical optimization automatically.
• Clone tables — Zero-copy clones for dev/test or point-in-time snapshots.
• Stored procedures — Business logic in T-SQL, callable from pipelines.

When to Use Warehouse¶

Choose the Warehouse when:

• The team's primary skill set is T-SQL.
• The workload is SQL-first DML (INSERT, UPDATE, DELETE, MERGE).
• Stored procedures encapsulate critical business logic.
• You need full transactional semantics beyond Delta ACID.

Avoid the Warehouse when the workload is Spark/Python-centric or requires heavy ML processing — that belongs in the Lakehouse with notebooks.

Deep dive: Warehouse Setup | Cross-Database Queries | Lakehouse vs Warehouse Decision Guide

Data Pipelines¶

Fabric Data Pipelines are the orchestration layer, built on the same engine as Azure Data Factory but running natively inside Fabric.

Core Patterns¶

Metadata-Driven Pipelines¶

The metadata-driven pattern uses a control table to define sources, targets, and transformation logic. A single generic pipeline reads the table and iterates over entries, reducing pipeline sprawl.

control_table
├── source_system: "crm"
├── source_table: "accounts"
├── target_lakehouse: "bronze"
├── load_type: "incremental"
├── watermark_column: "modified_date"
└── is_active: true

Comparison to ADF¶

Deep dive: Data Pipelines Tutorial | Metadata-Driven Pipelines | Pipelines & Data Movement

Real-Time Intelligence¶

Real-Time Intelligence (RTI) is Fabric's streaming and time-series analytics suite, combining event ingestion, storage, querying, and alerting in a single integrated experience.

Components¶

flowchart LR
    SRC1[IoT Devices] --> ES[Eventstream]
    SRC2[Event Hubs] --> ES
    SRC3[Kafka] --> ES
    SRC4[Custom Apps] --> ES

    ES --> EH[(Eventhouse<br/>KQL Database)]
    ES --> LH[(Lakehouse<br/>Delta)]
    ES --> DA[Data Activator<br/>Alerts]

    EH --> KQL[KQL Queries]
    EH --> RTD[Real-Time<br/>Dashboards]
    EH --> PBI[Power BI<br/>Real-Time Reports]

    RTH[Real-Time Hub] -.->|Discover| ES

When to Use RTI¶

• Sub-second ingestion and querying of time-series or event data.
• IoT telemetry, application logs, clickstream, or security events.
• Real-time dashboards that need refresh at ingestion speed, not scheduled intervals.
• Alert-driven workflows (Data Activator triggers Teams/email/Power Automate on threshold breach).

RTI vs Event Hubs + ADX¶

RTI is the Fabric-native evolution of the Event Hubs + Azure Data Explorer (ADX) pattern. If you are already using ADX standalone, RTI provides the same KQL engine inside Fabric with unified billing and governance.

Deep dive: Real-Time Intelligence Feature | Real-Time Analytics Tutorial | Eventhouse Vector Database | Alerting & Data Activator

Direct Lake & Power BI¶

Direct Lake is the connectivity mode that makes Fabric transformative for BI workloads. It reads Delta tables in OneLake directly into the Power BI in-memory engine — no scheduled import refresh, no DirectQuery backend round-trips.

How It Works¶

flowchart LR
    GOLD[(Gold Layer<br/>Delta in OneLake)] -->|Direct read| SM[Semantic Model<br/>VertiPaq in-memory]
    SM --> RPT[Reports & Dashboards]
    SM --> COP[Copilot in Power BI]

• Import-tier performance — Data is loaded into the VertiPaq engine on first query, then cached. Subsequent queries hit memory.
• Near real-time freshness — When the underlying Delta table changes, the cache is transparently refreshed. No scheduled refresh to manage.
• Fallback to DirectQuery — If the data exceeds memory or the Delta table is unavailable, queries fall back to DirectQuery automatically.

Trade-offs¶

Semantic Models¶

Build semantic models following the same star schema conventions used in the current PaaS Power BI guide. The DAX measures, calculation groups, and RLS definitions are identical — only the data source connection changes from Import/DirectQuery to Direct Lake.

Deep dive: Direct Lake Feature | Direct Lake & Power BI Tutorial | Power BI Best Practices | Semantic Link | Composite Models

Copilot & AI¶

Fabric integrates AI throughout the platform, from natural language querying to notebook code generation.

Capabilities¶

AI Skills¶

AI Skills let you create a natural language interface over your Gold layer. Users ask questions in plain English; the skill translates to SQL and returns results. Skills are versioned, testable, and shareable within a workspace.

Data Agents¶

Data Agents extend AI Skills with autonomous reasoning. An agent can decompose a complex question, query multiple tables, apply business logic, and return a synthesized answer — effectively a RAG pipeline over your lakehouse.

Deep dive: AI Copilot Configuration | Data Agents | Fabric IQ | Copilot & AI Tutorial | Advanced AI/ML Tutorial | AutoML & Model Endpoints

Spark & Notebooks¶

Fabric Spark (Synapse Data Engineering) provides managed Apache Spark for data engineering, ML, and exploration.

Key Features¶

• PySpark, Spark SQL, Scala, R — Full language support in notebooks.
• Environments — Reusable configurations bundling library versions, Spark properties, and pool sizes. Attach an environment to a notebook or a Spark job definition.
• Library management — Public PyPI/Maven packages, workspace libraries, and custom .whl / .jar uploads.
• Spark job definitions — Submit production Spark jobs outside of notebooks with configurable retry and alerting.
• V-Order optimization — Fabric applies V-Order write optimization to Delta tables automatically, accelerating Direct Lake reads.
• High concurrency mode — Share a single Spark session across multiple notebooks for cost-efficient interactive work.

Notebook Best Practices¶

1. Use environments — Pin library versions to an environment rather than running %pip install inline.
2. Optimize writes — Enable V-Order and liquid clustering on Gold layer tables for Direct Lake performance.
3. Parameterize — Use notebook parameters and pipeline notebook activities for reusable, testable transforms.
4. Monitor — Use the Spark monitoring hub to identify slow stages, skewed partitions, and spill.

Deep dive: Spark & Notebooks Best Practices | Notebooks | dbt Integration

Data Governance (Purview)¶

Fabric governance is built on Microsoft Purview, the same platform used in the current CSA-in-a-Box PaaS stack (Purview guide).

Governance Capabilities in Fabric¶

Migration from PaaS Purview¶

When migrating from the current PaaS stack to Fabric, Purview metadata carries forward:

• Glossary terms and classifications transfer directly.
• Sensitivity labels apply to Fabric items the same way they apply to ADLS and Databricks today.
• Lineage expands to cover Fabric-native items (Eventstreams, Direct Lake models).

Deep dive: Governance & Purview Tutorial | Data Governance Deep Dive | Identity & RBAC Patterns

Capacity Planning¶

Fabric uses a Capacity Unit (CU) model. All workloads — Spark, SQL, pipelines, Power BI — consume CUs from a shared pool.

F-SKU Comparison¶

Cost Optimization Levers¶

Begin with F2 or F4 for proof-of-concept work. Monitor CU consumption in the Fabric Capacity Metrics app and scale up only when utilization consistently exceeds 80%.

Deep dive: Capacity Planning & Cost Optimization | Cost Estimation | FinOps & Cost Governance | Cost Optimization Tutorial

When to Use Fabric vs Alternatives¶

The full decision tree lives in Fabric vs. Databricks vs. Synapse and the Reference Architecture. Below is a condensed comparison.

Decision Shortcuts¶

• Power BI-centric, Azure Commercial, greenfield -- Fabric.
• Heavy Spark/ML, multi-cloud, or deep MLflow investment -- Databricks.
• Azure Government, IL5/IL6, or existing Synapse dedicated pools -- Synapse.
• Hybrid -- Databricks for data engineering + Fabric for BI and real-time (common transitional architecture).

Fabric in CSA-in-a-Box¶

CSA-in-a-Box integrates Fabric at multiple levels. The current PaaS stack is designed as a Fabric on-ramp — every component has a documented forward path.

Integration Points¶

Bicep Deployment¶

CSA-in-a-Box Bicep modules provision Azure PaaS resources today. Fabric capacity can be provisioned via Bicep (Microsoft.Fabric/capacities), and Fabric items (lakehouses, warehouses, pipelines) are managed through the Fabric REST API or the fabric-cicd CLI tool.

dbt on Fabric¶

The dbt-fabric adapter connects dbt Core to the Fabric SQL analytics endpoint. Existing dbt models written for Databricks or Synapse require an adapter swap and minor SQL dialect adjustments — the business logic remains unchanged.

Power BI with Direct Lake¶

The transition from Import/DirectQuery to Direct Lake preserves the entire semantic model: star schema, DAX measures, calculation groups, RLS. Only the data source binding changes. See the Power BI guide for the detailed migration table.

Deep dive: dbt Integration | CI/CD with fabric-cicd | CI/CD & DevOps Tutorial

Migration Paths to Fabric¶

CSA-in-a-Box includes migration playbooks for common platform transitions. Several target Fabric directly or pass through the PaaS stack with a documented Fabric forward path.

Deep dive: Migration Planning Tutorial | Migration Patterns | Teradata Migration Tutorial | Snowflake to Fabric Tutorial | SAS Connectivity Tutorial

Deep Dive Resources¶

The Supercharge Microsoft Fabric companion site provides hands-on tutorials, feature guides, best practices, and POC materials for Fabric. Use the table below to navigate to the right section.

Selected Feature Guides¶

Related¶

• ADR: ADR-0010 — Fabric as strategic target
• Decision tree: Fabric vs. Databricks vs. Synapse
• Decision tree: Lakehouse vs. Warehouse vs. Lake
• Reference architecture: Fabric vs Synapse vs Databricks
• Pattern: Power BI & Fabric Roadmap
• Guide: Power BI (Direct Lake migration, semantic models)
• Guide: Purview (governance, sensitivity labels)
• Guide: Azure Data Explorer (RTI predecessor)
• Guide: Azure Synapse (current PaaS engine)
• Guide: Databricks Best Practices (current Spark engine)
• Migration: Databricks to Fabric
• Use case: Fabric Unified Analytics
• Companion site: Supercharge Microsoft Fabric