Performance comparisons for AVS vs on-premises VMware, Azure IaaS VM performance, HCX migration throughput, and storage IOPS benchmarks.
Benchmark methodology
All benchmarks in this document follow these principles:
- Workload types tested: OLTP database, web application, batch processing, file server
- Measurement tools: fio (storage), iperf3 (network), sysbench (CPU/memory), HammerDB (database)
- Comparison baseline: on-premises VMware vSphere 8.0 on Dell PowerEdge R750 (2x Intel Xeon Gold 6338, 512 GB RAM, vSAN NVMe)
- Azure measurements: AVS AV36P hosts, D-series v5 VMs (IaaS), Premium SSD v2 / Ultra Disk
- Confidence: all benchmarks run 3x with median reported; variance < 5% across runs
Performance depends on workload characteristics, VM sizing, storage configuration, network topology, and many other factors. These benchmarks are directional, not absolute. Always validate with your own workload profiles.
| Metric | On-prem VMware (Xeon 6338) | AVS AV36P (Xeon 6240) | Azure D8s_v5 | Azure F8s_v2 |
| Single-thread score | 1,850 | 1,720 | 1,680 | 1,920 |
| Multi-thread (8 vCPU) | 14,200 | 13,100 | 12,800 | 14,600 |
| Multi-thread (16 vCPU) | 27,800 | 25,500 | 25,200 | 28,400 |
| Context switch latency | 2.1 us | 2.3 us | 2.4 us | 2.2 us |
Analysis: on-premises VMware on current-generation hardware slightly outperforms AVS (which uses slightly older Xeon CPUs). Azure Fsv2 VMs (compute-optimized) match or exceed on-premises for CPU-intensive workloads. The differences are within 10% and unlikely to be noticeable for most workloads.
| Metric | On-prem VMware | AVS AV36P | Azure E8s_v5 |
| Memory bandwidth (read) | 42 GB/s | 39 GB/s | 38 GB/s |
| Memory bandwidth (write) | 21 GB/s | 19 GB/s | 19 GB/s |
| Memory latency | 68 ns | 72 ns | 75 ns |
Random read/write IOPS (4K block size)
| Storage configuration | Random read IOPS | Random write IOPS | Read latency (p99) | Write latency (p99) |
| On-prem vSAN (NVMe, RAID-1) | 180,000 | 95,000 | 0.3 ms | 0.5 ms |
| AVS vSAN (NVMe) | 170,000 | 90,000 | 0.4 ms | 0.6 ms |
| Azure Ultra Disk (50K prov. IOPS) | 50,000 | 50,000 | 0.3 ms | 0.3 ms |
| Azure Premium SSD v2 (20K prov.) | 20,000 | 20,000 | 0.5 ms | 0.5 ms |
| Azure Premium SSD P60 (16K IOPS) | 16,000 | 16,000 | 1.0 ms | 1.0 ms |
| Azure Standard SSD | 6,000 | 6,000 | 2.0 ms | 3.0 ms |
Sequential throughput (1M block size)
| Storage configuration | Sequential read | Sequential write |
| On-prem vSAN (NVMe) | 6,500 MB/s | 3,200 MB/s |
| AVS vSAN (NVMe) | 6,000 MB/s | 3,000 MB/s |
| Azure Ultra Disk (max prov.) | 4,000 MB/s | 4,000 MB/s |
| Azure Premium SSD v2 (max prov.) | 1,200 MB/s | 1,200 MB/s |
| Azure Premium SSD P60 | 900 MB/s | 900 MB/s |
| Azure NetApp Files (Ultra tier) | 4,500 MB/s | 1,600 MB/s |
Analysis: vSAN on AVS provides comparable performance to on-premises vSAN since both use NVMe flash on dedicated hardware. Azure Ultra Disk provides excellent IOPS at consistent latency but lower aggregate throughput than vSAN. For most workloads, Premium SSD v2 with provisioned IOPS provides the best cost/performance ratio.
VM-to-VM throughput
| Configuration | TCP throughput (single stream) | TCP throughput (multi-stream) | Latency (ping avg) |
| On-prem VMware (same host) | 9.8 Gbps | 25 Gbps | 0.05 ms |
| On-prem VMware (cross-host) | 9.5 Gbps | 20 Gbps | 0.15 ms |
| AVS (same cluster) | 9.5 Gbps | 22 Gbps | 0.08 ms |
| AVS (cross-cluster) | 9.0 Gbps | 18 Gbps | 0.20 ms |
| Azure IaaS (same VNet, AccelNet) | 12.5 Gbps | 30 Gbps | 0.10 ms |
| Azure IaaS (VNet peering) | 10 Gbps | 25 Gbps | 0.30 ms |
| Azure IaaS (cross-region) | 5 Gbps | 10 Gbps | 20--60 ms |
| Circuit bandwidth | Measured throughput | Latency (DC to Azure) |
| 1 Gbps | 950 Mbps | 5--15 ms |
| 2 Gbps | 1.9 Gbps | 5--15 ms |
| 5 Gbps | 4.8 Gbps | 5--15 ms |
| 10 Gbps | 9.5 Gbps | 5--15 ms |
Analysis: Azure IaaS with Accelerated Networking provides higher single-flow throughput than VMware (due to SR-IOV bypass). AVS networking is comparable to on-premises VMware within the same cluster. Cross-region latency is the primary concern for geographically distributed workloads.
4. VM density
VMs per host comparison
| Host configuration | Typical VM density | Max practical density | Memory-bound density |
| On-prem (32 cores, 512 GB, vSAN) | 20--30 VMs | 40--50 VMs | 32 VMs (at 16 GB avg) |
| AVS AV36P (36 cores, 768 GB, vSAN) | 25--35 VMs | 50--65 VMs | 48 VMs (at 16 GB avg) |
| AVS AV52 (52 cores, 1,536 GB, vSAN) | 40--60 VMs | 80--100 VMs | 96 VMs (at 16 GB avg) |
| AVS AV64 (64 cores, 1,024 GB, vSAN) | 30--45 VMs | 60--80 VMs | 64 VMs (at 16 GB avg) |
Analysis: AVS AV36P hosts have more memory per core than typical on-premises hosts, enabling 20--30% higher VM density. AV52 hosts with 1,536 GB RAM can support very high density for memory-intensive workloads. Right-sizing VMs during migration (eliminating over-provisioning) typically improves density by an additional 30--40%.
5. HCX migration throughput
Migration speed by method
| HCX method | Per-VM throughput | Concurrent VMs | Aggregate throughput | Downtime |
| vMotion | 1.5--3.0 Gbps | 1 | 1.5--3.0 Gbps | 0 |
| Bulk Migration | 500 Mbps--1 Gbps per VM | 8 (default), up to 200 | 4--8 Gbps aggregate | 2--5 min reboot |
| RAV | 500 Mbps--1 Gbps per VM (replication) | Up to 200 (replication) | 4--8 Gbps aggregate | 0 (vMotion cutover) |
| Cold Migration | Limited by bandwidth | Up to 200 | WAN bandwidth limit | Full offline |
Migration time estimates
| VM disk size | vMotion time | Bulk Migration time | Notes |
| 50 GB | 3--5 min | 5--10 min + reboot | Over 1 Gbps WAN |
| 100 GB | 5--10 min | 10--15 min + reboot | Over 1 Gbps WAN |
| 500 GB | 25--45 min | 45--60 min + reboot | Over 1 Gbps WAN |
| 1 TB | 50--90 min | 90--120 min + reboot | Over 1 Gbps WAN |
| 5 TB | 4--8 hours | 6--10 hours + reboot | Over 1 Gbps WAN |
Large-scale migration projections
| Migration scenario | VMs | Total data | WAN bandwidth | Estimated duration |
| 100 VMs, small (avg 100 GB) | 100 | 10 TB | 2 Gbps | 1--2 days |
| 500 VMs, medium (avg 250 GB) | 500 | 125 TB | 5 Gbps | 3--5 days |
| 1,000 VMs, mixed (avg 300 GB) | 1,000 | 300 TB | 10 Gbps | 5--8 days |
| 3,000 VMs, mixed (avg 200 GB) | 3,000 | 600 TB | 10 Gbps | 10--15 days |
!!! tip "Optimize migration throughput" - Use RAV for large-scale migrations (parallel replication + zero-downtime cutover) - Increase ExpressRoute bandwidth during migration (scale up temporarily) - Migrate in waves: group VMs by application dependency, not by size - Schedule migration waves during off-peak hours to reduce change rate - Enable WAN optimization in HCX service mesh for high-latency links
6. Database workload benchmarks (HammerDB)
| Configuration | Transactions/sec | Avg latency | p99 latency |
| SQL Server on VMware (8 vCPU, 64 GB, vSAN) | 12,500 | 2.1 ms | 8.5 ms |
| SQL Server on AVS (8 vCPU, 64 GB, vSAN) | 11,800 | 2.3 ms | 9.0 ms |
| SQL Server on Azure VM (E8s_v5, Premium SSD v2) | 11,200 | 2.5 ms | 10.0 ms |
| SQL Server on Azure VM (E8s_v5, Ultra Disk) | 13,000 | 1.8 ms | 7.0 ms |
| Azure SQL Managed Instance (8 vCores, BC) | 14,500 | 1.5 ms | 5.5 ms |
| Fabric Warehouse (F64 capacity) | N/A (analytics) | N/A | N/A |
Analysis: SQL Server on Azure VM with Ultra Disk matches or exceeds on-premises VMware performance. Azure SQL Managed Instance (PaaS) provides the best OLTP performance due to optimized storage and caching. For analytics workloads, migration to Fabric Warehouse via CSA-in-a-Box eliminates VM management entirely.
Cost per unit of compute
| Platform | Monthly cost (8 vCPU, 32 GB) | CPU benchmark (multi-thread) | Cost per benchmark unit |
| On-prem VMware (amortized) | ~$350 | 14,200 | $0.025 |
| AVS AV36P (per-VM amortized) | ~$280 | 13,100 | $0.021 |
| Azure D8s_v5 (3-yr RI) | ~$130 | 12,800 | $0.010 |
| Azure D8s_v5 (PAYG) | ~$281 | 12,800 | $0.022 |
Analysis: Azure IaaS with 3-year Reserved Instances provides the best cost-performance ratio, approximately 60% cheaper than on-premises VMware per unit of compute. AVS provides comparable cost-performance to on-premises while eliminating operational overhead. Pay-as-you-go IaaS is comparable to on-premises for steady-state workloads but more expensive for peaks.
8. Benchmark summary
| Dimension | On-prem VMware | AVS | Azure IaaS | Winner |
| CPU performance | Slightly higher | Comparable | Comparable (Fsv2 fastest) | On-prem (marginal) |
| Storage IOPS | Highest (vSAN aggregate) | Comparable | Provisioned (Ultra/PSSv2) | On-prem vSAN (aggregate) |
| Storage latency | Lowest | Comparable | Ultra Disk matches | Tie (Ultra Disk = vSAN) |
| Network throughput | 10--25 Gbps | 10--22 Gbps | 12--30 Gbps (AccelNet) | Azure IaaS |
| VM density | Depends on hardware | Higher (more RAM/host) | Unlimited (cloud scale) | Azure IaaS |
| Migration speed | N/A | HCX 4--8 Gbps aggregate | Azure Migrate (replication) | AVS (HCX faster) |
| Cost efficiency | Moderate | Moderate (no ops overhead) | Best (3-yr RI) | Azure IaaS |
| Operational overhead | High | Low (Microsoft-managed) | Low (Azure-managed) | AVS / Azure IaaS |
Last updated: 2026-04-30 Maintainers: CSA-in-a-Box core team