Home > Tutorials > Palantir Foundry to Fabric Migration

🟣 Tutorial 55: Palantir Foundry → Microsoft Fabric Migration¶

Last Updated: 2026-05-21 | Status: ✅ Final | Maintainer: Platform Team

📋 Table of Contents¶

• Overview
• Why migrate
• Component mapping
• Reference architecture
• Prerequisites
• Step-by-step migration
• Ontology migration deep dive
• Egress, network, and cost considerations
• Validation checklist
• Troubleshooting
• References

📖 Overview¶

Palantir Foundry is an end-to-end data platform with a strong ontology layer (semantic objects + properties + links), transforms (Python or SQL), datasets (Parquet + Delta in the platform's object store), pipelines, and apps. It is a capable, mature platform, and the ontology and operational-decision tooling it pioneered are genuine strengths. The descriptions of Foundry here are based on Palantir's publicly available documentation (as of this page's date); always verify against Palantir's current official docs. Organizations choose to migrate from Foundry to Microsoft Fabric for several reasons — usually some combination of:

1. License consolidation — collapsing Foundry, Power BI, and Azure analytics spend into a single Fabric F-SKU.
2. Microsoft-native tooling — preferring Power BI, Purview, Entra ID, and Microsoft Defender over Foundry-native equivalents.
3. OneLake gravity — operational systems already write to OneLake, and Foundry sits as an island.
4. AI estate consolidation — using Fabric IQ, Data Agents, and Azure OpenAI rather than Foundry AIP.

This tutorial covers the canonical migration path: export Foundry datasets and ontology to OneLake, rebuild the ontology in Fabric IQ, port transforms to PySpark / Dataflows Gen2, and republish dashboards in Power BI Direct Lake.

📝 Scope: This is a technical migration tutorial. Contract, licensing, and procurement work should be sequenced ahead of any of the steps below — Foundry is typically an enterprise contract with a fixed term, and the migration timeline must align with renewal windows.

🎯 Why migrate¶

The table below maps common drivers for choosing Fabric to the Fabric capability that addresses them. It reflects Foundry behavior as described in Palantir's publicly available documentation; where Foundry is the better fit for a given need, stay on Foundry. This is a comparison from a Microsoft Fabric perspective, not an authoritative assessment of Foundry.

🧭 Component mapping¶

The Foundry → Fabric translation below is based on Palantir's publicly documented component model (as of this page's date). Verify specifics against Palantir's current official documentation:

🏗️ Reference architecture¶

flowchart LR
    subgraph Foundry["🟣 Palantir Foundry (source)"]
        FD[Datasets<br/>Object Storage V2]
        FO[Ontology<br/>objects + props + links]
        FP[Pipelines<br/>Code workbooks + Pipeline Builder]
        FA[Apps<br/>Workshop / Forge / Quiver]
    end

    subgraph Bridge["🔄 Migration bridge"]
        EXP[Foundry Export API<br/>+ object storage downloader]
        SHC[OneLake shortcuts<br/>to S3/ADLS<br/>if Foundry is on AWS]
    end

    subgraph Fabric["🟦 Microsoft Fabric (target)"]
        OL[(OneLake<br/>Bronze · Silver · Gold)]
        IQ[Fabric IQ<br/>Ontology + Graph]
        NB[Fabric Notebooks<br/>PySpark transforms]
        DF[Dataflows Gen2<br/>Pipeline Builder replacement]
        PA[Power Apps + Translytical<br/>Apps replacement]
        PBI[Power BI<br/>Direct Lake reports]
        DA[Data Agents<br/>AIP replacement]
        PV[Microsoft Purview<br/>governance + lineage]
    end

    FD --> EXP --> OL
    FD -.shortcut.- SHC -.- OL
    FO --> IQ
    FP --> NB
    FP --> DF
    FA --> PA
    OL --> PBI
    OL --> DA
    Fabric --> PV

The bridge is built once and torn down at the end of migration. The steady state is OneLake → Fabric IQ → Power BI / Data Agents, with Purview wrapping everything for governance.

📋 Prerequisites¶

• ✅ Fabric F64 capacity provisioned with workspace identity (Tutorial 00)
• ✅ Bronze/Silver/Gold lakehouses in OneLake (Tutorials 01-03)
• ✅ Foundry Administrator role and Data Engineer in the source workspace
• ✅ Foundry API token with read access to source datasets and the ontology
• ✅ A target landing location in OneLake (recommend a dedicated lh_foundry_migration lakehouse during the cutover, then promote to Bronze)
• ✅ Migration assessment from Tutorial 13 signed off by stakeholders

🚀 Step-by-step migration¶

Step 1 — Inventory the Foundry estate¶

Pull a complete inventory before touching anything. Foundry's REST API lets you enumerate without exporting data.

# foundry_inventory.py
import requests, os, json

FOUNDRY = os.environ["FOUNDRY_HOSTNAME"]            # e.g. yourtenant.palantirfoundry.com
TOKEN   = os.environ["FOUNDRY_TOKEN"]
HDRS    = {"Authorization": f"Bearer {TOKEN}"}

# 1. Datasets
datasets = requests.get(f"https://{FOUNDRY}/api/v2/datasets", headers=HDRS).json()
# 2. Ontology object types
objects  = requests.get(f"https://{FOUNDRY}/api/v2/ontologies/default/objectTypes", headers=HDRS).json()
# 3. Code repos (transforms)
repos    = requests.get(f"https://{FOUNDRY}/api/v2/codeRepositories", headers=HDRS).json()

inventory = {
    "datasets":  [{"rid": d["rid"], "name": d["name"], "size_gb": d.get("sizeBytes", 0) / 1e9} for d in datasets["data"]],
    "object_types": [{"rid": o["rid"], "apiName": o["apiName"]} for o in objects["data"]],
    "repositories": [{"rid": r["rid"], "name": r["name"]} for r in repos["data"]],
}
print(json.dumps(inventory, indent=2))

Save this inventory — it drives the wave plan in Step 2.

Step 2 — Build a wave plan¶

Group datasets into migration waves. Recommended waves:

1. Wave 1 — Foundational reference data (small, low blast radius)
2. Wave 2 — Core operational datasets (the ones every dashboard depends on)
3. Wave 3 — Domain-specific marts
4. Wave 4 — Ontology and applications (must come after data lands)

For each dataset, decide: bulk export (Parquet/JSON dump → OneLake) or shortcut (point OneLake at the underlying S3/ADLS Foundry storage if available).

Step 3 — Land Bronze data in OneLake¶

Two patterns: shortcut (preferred when possible) and export.

3a. OneLake shortcut (zero copy)¶

If Foundry is deployed on Azure or AWS and your administrator can grant read access to the underlying object store, point OneLake shortcuts directly at it. This avoids any data movement.

# notebook cell — register a shortcut to the Foundry Parquet location
mssparkutils.fs.mssparkutils.fs.mount(
    "abfss://<container>@<foundry-storage>.dfs.core.windows.net/path/to/dataset",
    "/lakehouse/default/Files/foundry_<dataset>"
)

3b. Bulk export via Foundry API¶

When shortcuts aren't viable (different cloud, network constraints, or contractual reasons), bulk export.

# foundry_export.py — sketch
from foundry_dev_tools import FoundryContext
import shutil, os

ctx = FoundryContext(token=os.environ["FOUNDRY_TOKEN"],
                     foundry_url=f"https://{os.environ['FOUNDRY_HOSTNAME']}")

DATASETS = [
    "ri.foundry.main.dataset.<rid>",
    # ...
]

for rid in DATASETS:
    ds = ctx.dataset(rid)
    parquet_dir = ds.transactions.last_committed.download_to_local("./tmp/" + rid)
    # Upload to OneLake via Spark write
    # (run from inside a Fabric notebook — write directly to lh_bronze)
    df = spark.read.parquet("./tmp/" + rid)
    df.write.format("delta").mode("overwrite").save(f"abfss://workspace@onelake.dfs.fabric.microsoft.com/lh_bronze.Lakehouse/Tables/foundry_{rid}")

💡 Foundry Dev Tools (pip install foundry-dev-tools) is the open-source Palantir-published client library. It handles auth, transactions, and chunked downloads cleanly.

Step 4 — Port transforms¶

Foundry Code Workbooks are mostly PySpark with a few foundry_* SDK calls (mainly input/output decorators and Transforms lineage hooks).

Foundry transform:

from transforms.api import transform_df, Input, Output

@transform_df(
    Output("ri.foundry.main.dataset.gold_cust"),
    raw=Input("ri.foundry.main.dataset.bronze_cust"),
)
def compute(raw):
    return raw.filter(raw.status == "active").select("id", "name", "tier")

Fabric notebook equivalent:

# notebook cell — same logic, no Foundry decorators
df_raw = spark.read.format("delta").table("lh_bronze.bronze_cust")
df_gold = df_raw.filter(df_raw.status == "active").select("id", "name", "tier")
df_gold.write.format("delta").mode("overwrite").saveAsTable("lh_gold.gold_cust")

Patterns to handle during port:

1. Input / Output decorators → Replace with spark.read.format("delta").table(...) / df.write.saveAsTable(...).
2. Markings (PII / sensitivity) → Map to Purview sensitivity labels.
3. Profile definitions → Translate to Delta partition + Z-ORDER strategies.
4. Custom validation hooks → Move to Great Expectations suites (validation framework).
5. Foundry-native expectations package → Replace with Great Expectations or pyspark.sql.dataframe-level assertions.

Step 5 — Rebuild the ontology in Fabric IQ¶

This is the highest-value piece of the migration. The Foundry ontology becomes a Fabric IQ ontology + Knowledge Graph.

Export the Foundry ontology to JSON, then map:

See Fabric IQ for the authoring workflow. The ontology is defined in YAML and round-trips through Git.

Step 6 — Republish dashboards¶

Foundry Quiver / Contour / Workshop dashboards become Power BI Direct Lake reports.

1. Identify the source dataset for each Foundry dashboard.
2. Build the Fabric semantic model on top of the Gold lakehouse table.
3. Recreate visuals in Power BI Desktop.
4. Connect via Direct Lake (preferred) or DirectQuery if the model exceeds Direct Lake limits.

See Direct Lake and Tutorial 05.

Step 7 — Replace Foundry Apps (Workshop / Forge / Slate)¶

Three replacement patterns depending on the app complexity:

Step 8 — Cutover and decommission¶

1. Run both platforms in parallel for one reporting cycle.
2. Reconcile core KPIs to within an agreed tolerance (typically ≤ 0.5% drift).
3. Repoint downstream consumers (BI, ML, external APIs).
4. Freeze writes to Foundry.
5. Take a final archive snapshot to OneLake lh_archive.
6. Cancel the Foundry contract at the renewal boundary.

🧩 Ontology migration deep dive¶

The ontology is the highest-leverage piece of Foundry — and the trickiest piece to migrate. A naive table-level migration will lose the semantic layer entirely.

Pattern: Object Type → Fabric IQ Entity

# Fabric IQ ontology file (excerpt)
entities:
  - name: Customer
    primary_key: customer_id
    fields:
      - { name: customer_id, type: string, source: lh_gold.gold_customer.customer_id }
      - { name: name,        type: string, source: lh_gold.gold_customer.name }
      - { name: tier,        type: enum,   values: [bronze, silver, gold, platinum] }
    relationships:
      - { name: orders,   to: Order,   cardinality: one_to_many, fk: orders.customer_id }
      - { name: account,  to: Account, cardinality: one_to_one,  fk: account.customer_id }

This becomes queryable by Data Agents, Power BI semantic models, and Fabric REST APIs in one place.

💰 Egress, network, and cost considerations¶

✅ Validation checklist¶

• Every Foundry dataset has a corresponding OneLake Delta table
• Row counts match (Foundry vs. OneLake) for every Bronze table
• All transforms ported, tests added in validation/
• Ontology rebuilt in Fabric IQ — each object type mapped
• Sensitivity labels (formerly Foundry marks) applied via Purview
• Every Foundry dashboard has an equivalent Power BI report
• All Foundry apps (Workshop / Forge) replaced or formally retired
• Downstream consumers repointed
• Reconciliation report signed off by data stewards
• Foundry account frozen and archived

🛠️ Troubleshooting¶

📚 References¶

• Tutorial 13 — Migration Planning
• Fabric IQ — the ontology replacement for Foundry's ontology
• Data Agents — replacement for Foundry AIP
• OneLake Shortcuts — zero-copy bridging to AWS/GCP storage
• Translytical task flows — Workshop write-back replacement
• Best practices — Migration patterns
• Palantir Foundry Dev Tools (open source client)
• Fabric API for GraphQL — REST API replacement for Foundry's APIs

Navigation: ⬅️ 45 — On-Prem SSAS/SSIS/SSRS | Tutorials Home | 56 — Informatica → Fabric ➡️