"In traditional data platforms, storage and compute were married. You bought a server — it stored data AND processed queries. Modern platforms divorced them. That divorce changed everything about how we scale, cost, and architect enterprise analytics."
The Old World — Tightly Coupled Storage and Compute
In traditional data warehouses (SQL Server, Oracle, Teradata on-premises), the same machine stored your data AND ran your queries. This created a fundamental problem:
TRADITIONAL ARCHITECTURE (Tightly Coupled)
[SQL Server Machine]
├── Storage: 10TB of data sitting on disk
└── Compute: CPU + RAM processing queries
Problem 1 — Scale storage = buy bigger server (also scales compute you don't need)
Problem 2 — Scale compute for month-end = idle CPU all other 29 days
Problem 3 — Two teams need different processing? Two separate servers, two copies of data
Problem 4 — Server offline for maintenance = BOTH storage AND compute unavailable
The New World — Decoupled Storage and Compute
Modern cloud platforms separate these completely.
MODERN ARCHITECTURE (Decoupled)
STORAGE LAYER (cheap, infinite, always on)
├── Azure Data Lake Storage Gen2 / OneLake
├── Stores data in open formats (Parquet, Delta)
├── Cost: ~$0.02 per GB per month
└── Never goes offline, scales automatically
↕️ Any compute can connect to any storage
COMPUTE LAYER (elastic, pay per use, disposable)
├── Spark Cluster for data engineering
├── SQL Engine for ad-hoc queries
├── Power BI Premium for report rendering
├── Machine Learning compute for models
└── Cost: Only when running
What This Means in Microsoft Fabric
Microsoft Fabric is built entirely on this principle. OneLake is the storage layer — one unified data lake for your entire organisation. Every Fabric workload (Spark, SQL, Power BI, Data Science) reads from the SAME OneLake without copying data.
MICROSOFT FABRIC — STORAGE vs COMPUTE SEPARATION
┌─────────────────────────────┐
│ ONELAKE │
│ (One copy of all data) │
│ Delta format, always on │
└──────────┬──────────────────┘
│ All workloads read same data
┌────────────────────┼────────────────────┐
↓ ↓ ↓
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Fabric │ │ SQL │ │ Power BI │
│ Spark │ │ Analytics │ │ Direct │
│ (Engineer) │ │ Endpoint │ │ Lake │
└─────────────┘ └─────────────┘ └─────────────┘
Scale up for ETL Query with SQL Report without
jobs, shut down like a warehouse importing data
when done
No data movement between workloads. No synchronisation. No copies. One data store, multiple compute engines accessing it simultaneously.
Enterprise Scenario — Insurance Company Quarter-End Processing
An insurance company runs routine analytics 9-to-5 on small compute. At quarter-end, they need to process 10x the normal data volume for regulatory reporting.
Common Misconceptions
- "Separating storage and compute means slower queries" — Not in modern platforms. Fabric's SQL Analytics Endpoint and Direct Lake are engineered specifically for this architecture. Direct Lake reads Delta tables from OneLake with performance comparable to in-memory import.
- "We need to copy data to Power BI to use it" — Not with Direct Lake. Power BI reads directly from Delta tables in OneLake — no import, no refresh cycle, no copy.
- "OneLake is just another storage account" — OneLake is a logical layer over Azure Data Lake Gen2 that adds unified governance, automatic metadata, and multi-engine access. It is not just a storage account with a different name.
🎯 Quick Check
Q1: A data team needs to run heavy Spark ETL jobs only on weekends. In a decoupled architecture, how does this affect storage costs during the week?
Show Answer
Storage costs are unaffected. Storage is always on and always charged (at low per-GB rates). Only compute costs change — Spark clusters are provisioned on weekends and shut down during the week. The team pays for storage 7 days, compute 2 days.
Q2: In Microsoft Fabric, if a data engineer transforms data using Spark and writes it to OneLake, can a Power BI report read that data immediately without a separate refresh?
Show Answer
Yes, via Direct Lake. Because Spark writes to Delta tables in OneLake and Power BI Direct Lake reads from those same Delta tables, the report reflects the new data as soon as the Spark job completes. No import refresh is needed because both tools read from the same physical storage.
Q3: What is the main cost advantage of decoupled architecture for a company with highly variable processing workloads?
Show Answer
Paying for peak compute only when needed. In coupled architecture, you provision for peak load and pay that cost 24/7. In decoupled architecture, you scale compute up during peak periods and back down (or off) otherwise. Storage cost is low and constant. Compute cost matches actual usage.
Key Takeaways — Part 2
- Traditional platforms couple storage and compute on the same machine — you pay for both even when you only need one
- Modern platforms separate them — storage is cheap and permanent, compute is elastic and pay-per-use
- Microsoft Fabric's OneLake is the storage layer — all Fabric workloads (Spark, SQL, Power BI) read from the same data without copying
- Direct Lake means Power BI reads live Delta tables from OneLake — no import, no refresh cycle
- The business benefit: right-size compute for normal operations, scale for peaks, return to normal — paying only for what you use