Skip to content

Best Practices

Nine field-tested guides for running cloud-scale analytics + AI on Azure. Each one is independent — read the ones relevant to your role.

Guide When to read Length
Medallion Architecture Designing your bronze/silver/gold lakehouse 664 lines
Data Engineering Authoring ADF + dbt + Spark pipelines 800 lines
Data Governance Setting up Purview, contracts, lineage, classification 573 lines
Security & Compliance Hardening identities, secrets, network, encryption 654 lines
Infrastructure as Code & CI/CD Bicep, what-if, GitHub Actions, environment promotion 657 lines
Cost Optimization Tagging, reserved capacity, auto-pause, FinOps 518 lines
Monitoring & Observability Log Analytics, Workbooks, OTel, SLI/SLO 520 lines
Performance Tuning Spark configs, Synapse SQL pools, AI Search shards 705 lines
Disaster Recovery RPO/RTO targets, geo-replication, runbook drills 521 lines

How to use these

Each guide follows the same structure:

1. The problem (1-2 paragraphs)
2. The opinionated answer (this is what we do)
3. The reasoning (why — usually links to an ADR)
4. The mechanics (commands, code, configs)
5. The trade-offs (what we gave up to make this choice)
6. The escape hatches (when this advice does NOT apply)

If a guide ever reads as "do X because everyone does X," that's a bug — open an issue.