Financial Services: Real-Time Fraud Detection & Regulatory Compliance¶

Executive Summary¶

This use case demonstrates how a mid-tier commercial bank with $45 billion in assets under management (AUM) and 2 million retail accounts leverages Microsoft Fabric to unify real-time fraud detection, anti-money laundering (AML) monitoring, credit risk scoring, and regulatory reporting into a single analytics platform. The architecture replaces a fragmented legacy stack of batch-oriented SAS models, siloed data warehouses, and manual reconciliation processes with a modern lakehouse that delivers sub-second fraud decisioning and continuous compliance.

Key Outcomes¶

Industry Context¶

The Financial Services Challenge¶

Financial institutions face a converging set of pressures:

1. Fraud escalation - Card-not-present (CNP) fraud grew 30% YoY to $9.49B in US losses (Nilson Report 2025). Real-time authorization decisioning is table stakes.
2. Regulatory burden - Banks must comply with BSA/AML (31 CFR 1020), SOX Section 404, PCI DSS v4.0, Basel III capital adequacy, and GLBA privacy requirements simultaneously.
3. Stress testing mandates - CCAR and DFAST require banks with >$10B assets to run quarterly macro-economic scenario analyses across the full loan portfolio.
4. Legacy modernization - Most mid-tier banks run 15-20 year old SAS/Teradata stacks that cannot support real-time ML scoring or streaming ingestion.

Regulatory Framework¶

Architecture¶

System Overview¶

+---------------------+     +--------------------+     +-------------------+
|   Card Networks     |     |   Core Banking     |     |   Wire/ACH        |
|   (Visa, MC, etc.)  |     |   (FIS/Fiserv)     |     |   (FedWire/NACHA) |
+--------+------------+     +--------+-----------+     +--------+----------+
         |                           |                          |
         v                           v                          v
+--------+---------------------------+---------------------------+----------+
|                        Azure Event Hub / Eventstream                       |
|                    (Card auth events, wire transfers, ACH)                 |
+--------+---------------------------+---------------------------+----------+
         |                           |                          |
         v                           v                          v
+--------+---------------------------+---------------------------+----------+
|                              EVENTHOUSE                                    |
|   Hot path: velocity checks, geo-anomaly detection, amount spike          |
|   KQL real-time queries with 30-second materialized views                 |
+--------+-----------------------------------------------------------------+
         |
         v
+--------+-----------------------------------------------------------------+
|                          MICROSOFT FABRIC LAKEHOUSE                       |
|                                                                           |
|  +-------------+    +-----------------+    +---------------------------+  |
|  |   BRONZE    |    |     SILVER      |    |          GOLD             |  |
|  |             |    |                 |    |                           |  |
|  | Raw txns    |--->| Enriched txns   |--->| fraud_scores              |  |
|  | Raw accounts|    | Velocity feats  |    | aml_alerts                |  |
|  | Raw wires   |    | Geo-distance    |    | ctr_filings               |  |
|  | Audit log   |    | MCC enrichment  |    | portfolio_risk            |  |
|  +-------------+    +-----------------+    | stress_test_scenarios     |  |
|                                            +---------------------------+  |
+--------------------------------------------------------------------------+
         |
         v
+--------+-----------------------------------------------------------------+
|                           POWER BI (Direct Lake)                          |
|   Fraud Operations Dashboard | AML Case Management | Risk Reporting      |
+-------------------------------------------------------------------------- +

Medallion Architecture Detail¶

Bronze Layer - Raw Ingestion¶

Key design decisions: - Append-only writes preserve the immutable audit trail required by SOX Section 404 - input_file_name() and _ingested_at metadata enable full data lineage - PAN (Primary Account Number) is tokenized at ingestion per PCI DSS Req 3.4 - Raw files retained in OneLake cold tier for regulatory examination requests

Silver Layer - Enrichment & Feature Engineering¶

Velocity features computed: - txn_count_1h - Transaction count in last 1 hour per card - txn_count_24h - Transaction count in last 24 hours per card - txn_count_7d - Transaction count in last 7 days per card - avg_amount_30d - Rolling 30-day average transaction amount - geo_distance_km - Haversine distance from previous transaction location - time_since_last_txn_sec - Seconds since last transaction

Gold Layer - Scoring & Alerts¶

Real-Time Fraud Detection Flow¶

Card Swipe/Online Purchase
        |
        v
  Event Hub (< 50ms)
        |
        v
  Eventstream Ingestion
        |
        +---> Eventhouse (KQL)
        |       |
        |       +---> Velocity check (txns in last 1h)
        |       +---> Geo-anomaly (impossible travel)
        |       +---> Amount spike (> 3x avg)
        |       |
        |       v
        |     Risk Score (rules-based, < 200ms)
        |       |
        |       +---> score > 0.85 → DECLINE + alert
        |       +---> score 0.50-0.85 → STEP-UP AUTH
        |       +---> score < 0.50 → APPROVE
        |
        +---> Bronze table (append, async)
                |
                v
          Silver enrichment (batch, 5-min micro-batch)
                |
                v
          Gold fraud_scores (ML model re-score, hourly)

Fraud Detection Models¶

Rule-Based Scoring (Tier 1 - Real-Time)¶

These rules execute in KQL against the Eventhouse with sub-second latency:

ML-Based Scoring (Tier 2 - Batch)¶

The hourly batch job re-scores all transactions using a gradient-boosted model trained on historical fraud labels:

• Model: LightGBM classifier trained on 24 months of labeled fraud data
• Features: 47 engineered features from Silver layer
• Performance: AUC-ROC 0.967, Precision@95%Recall = 0.82
• Retraining: Monthly with champion/challenger framework
• Hosting: Fabric ML model endpoint via AutoML or custom training

AML Transaction Monitoring¶

Compliance Control Mapping¶

SOX Section 404 - Internal Controls¶

PCI DSS v4.0 Compliance¶

Basel III Capital Adequacy¶

The Gold layer computes risk-weighted assets (RWA) for the credit portfolio:

RWA = Sum(EAD_i * RiskWeight_i)

Where:
  EAD = Exposure at Default (outstanding balance + undrawn commitments * CCF)
  RiskWeight = f(PD, LGD, maturity) per IRB formula or standardized buckets:
    - Sovereign: 0%
    - Bank: 20-150% (rating-dependent)
    - Corporate: 20-150%
    - Retail mortgage: 35%
    - Other retail: 75%
    - Past due: 150%

CET1 Ratio = CET1 Capital / RWA >= 4.5% (minimum)
Total Capital Ratio = Total Capital / RWA >= 8.0% (minimum)

Data Pipeline Implementation¶

Prerequisites¶

Notebooks¶

Data Generation¶

For POC demonstration, synthetic data is generated using the TransactionGenerator:

from data_generation.generators.financial.transaction_generator import TransactionGenerator

gen = TransactionGenerator(seed=42, num_customers=10000)
txns = gen.generate(num_records=100000)
gen.to_parquet(txns, "landing/financial/transactions.parquet")

The generator produces realistic card transactions with: - Configurable fraud rate (~0.15% baseline, matching industry averages) - PCI-compliant: PAN tokenized, only card_hash stored - Realistic MCC codes, amounts, and merchant names - Embedded fraud patterns: velocity bursts, geo-anomalies, amount spikes

Cost Model¶

Monthly Fabric Costs (F64 SKU)¶

ROI Analysis¶

Power BI Dashboards¶

Fraud Operations Dashboard¶

Purpose: Real-time fraud monitoring for the fraud operations team.

Visuals: - Fraud rate trend (hourly, daily) - line chart - Fraud by channel (card-present vs. CNP vs. ACH vs. wire) - stacked bar - Geographic heat map of fraud incidents - Top 10 merchants by fraud count - Alert queue with case assignment - KPI cards: fraud rate %, total losses today, false positive rate

Connectivity: Direct Lake to gold_fraud_scores and gold_daily_fraud_summary

AML Case Management¶

Purpose: BSA/AML compliance officers review and disposition alerts.

Visuals: - CTR filing queue (>$10K transactions) - Structuring pattern alerts with transaction timeline - Customer risk profile with account history - SAR narrative auto-generation data - Alert aging and SLA compliance

Connectivity: Direct Lake to gold_aml_alerts

Executive Risk Dashboard¶

Purpose: CRO and board-level risk reporting.

Visuals: - Basel III capital ratios (CET1, Tier 1, Total Capital) - RWA by portfolio segment - Credit quality migration matrix - Stress test scenario comparison (baseline/adverse/severely adverse) - Trend analysis of key risk indicators

Connectivity: Direct Lake to gold_portfolio_risk and gold_stress_scenarios

Security & Access Control¶

Workspace Architecture¶

Financial Services Workspaces
├── fin-dev          (developers - full access)
├── fin-staging      (CI/CD validation - automated)
├── fin-prod-ingest  (data engineers - contributor)
├── fin-prod-analytics (analysts - viewer + build)
└── fin-prod-compliance (BSA/AML officers - viewer)

Data Classification¶

Encryption¶

• At rest: Microsoft-managed keys (default) or customer-managed keys via Key Vault
• In transit: TLS 1.2+ enforced
• Column-level: PAN tokenized before entering Fabric; SSN hashed with HMAC-SHA-256
• Key rotation: Automatic 90-day rotation via Key Vault policy

Deployment Guide¶

Step 1: Infrastructure¶

az deployment sub create --location eastus2 \
  --template-file infra/main.bicep \
  --parameters infra/environments/prod/prod.bicepparam \
  --parameters enableCompliance=true enableCMK=true

Step 2: Data Generation (POC Only)¶

cd data_generation
python -m generators.financial.transaction_generator \
  --records 1000000 --output landing/financial/

Step 3: Notebook Execution Order¶

1. notebooks/bronze/51_financial_transactions.py
2. notebooks/silver/51_financial_enriched.py
3. notebooks/gold/51_financial_fraud_scoring.py

Step 4: Power BI¶

1. Create semantic model over Gold tables using Direct Lake
2. Import dashboard templates from poc-agenda/dashboards/
3. Configure row-level security for compliance roles

References¶

• 31 CFR 1020 - BSA Rules for Banks
• 12 CFR 21 - OCC Suspicious Activity Reports
• PCI DSS v4.0
• 12 CFR 217 - Basel III Capital Rules
• 15 USC 7262 - SOX Section 404
• 15 USC 6801 - GLBA Privacy
• 12 CFR 252 - DFAST/CCAR
• FinCEN BSA/AML Manual
• Nilson Report - Card Fraud Statistics
• Microsoft Fabric Documentation

Last Updated: 2026-04-27 Phase: 14 - Wave 6 (Commercial Verticals) Industry: Financial Services - Commercial Banking