Live DB — analyst tables
Test
Microsoft 65536 H100 CX-7 Hyperscaler Infiniband Cluster Configuration

MSFT-H100-CX7-IB-DEF

Microsoft · H100 · CX-7

Rack type
NVL8
Cluster / location
Arizona H100s
Buyer profile
Hyperscaler
Networking type
Infiniband
Number of XPUs
65,536
NIC used
CX-7
NICs / XPU
1
Scale-out BW / XPU
400G
XPU / server
8

Scale-up network

1 rows
Assigned blockDefault
H100 - NVL8
Scale-UpXPU: H100 NVL81 lineshared by 17 SKUs

Scale-up Units are based on the scale-up world size (8 XPUs).

XPU World Size

2
WS
Scale-Up Network LayerPart / Model
Units#
Unit BWGbit/s uni-d
Port Count#
Port BWGbit/s uni-d
XPUs per Node·83,600··
Scale-Up World Size·828,800··

Switches

5
SW
Scale-Up Network LayerPart / Model
Units#
Unit BWGbit/s uni-d
Port Count#
Port BWGbit/s uni-d
Scale-Up Switch ANvidia NVSwitch 3.0 (NVLink 4)18,00020400
Scale-Up Switch BNvidia NVSwitch 3.0 (NVLink 4)16,40016400
Scale-Up Switch CNvidia NVSwitch 3.0 (NVLink 4)16,40016400
Scale-Up Switch DNvidia NVSwitch 3.0 (NVLink 4)18,00020400
Scale-Up Switch TotalNvidia NVSwitch 3.0 (NVLink 4)428,80072·

Scale-Up Parts Summary

1
ItemQty/ClusterQty/ServerAttach/XPU
Nvidia NVSwitch 3.0 (NVLink 4)32,7684.000.50

Backend network

4 rows
Assigned blockDefault
H100 - NVL8 - CX7 - 3L - IB - QM9700 - MSFT
BackendXPU: H100 NVL84 linesshared by 1 SKU

Backend Units are based on the cluster size (65,536 XPUs).

Switches

5
SW
 Network Layer
Switch and Server SpecAggregate Port UtilizationNetwork Bandwidth Aggregate
SwitchPart Name / Model
Units#
Unit BWGbit/s
Port BWGbit/s
Downlink Ports / SW#
Uplink Ports / SW#
Max Ports#
Downlink Ports#
Uplink Ports#
Total Ports#
Downlink BWTbit/s
Uplink BWTbit/s
OversubRatio
XPUs·65,536400400065,53665,536065,53665,536026,214.4·
NICs per NodeConnectX-7 NIC - 400G8400400··8······
LeafQuantum-2 InfiniBand (QM9700)2,04825,60080016163232,76832,76865,53626,214.426,214.41
SpineQuantum-2 InfiniBand (QM9700)2,04825,60080016163232,76832,76865,53626,214.426,214.41
CoreQuantum-2 InfiniBand (QM9700)1,02425,6008003203232,768032,76826,214.40·

Transceivers

6
CN
 Network Layer
Transceiver / CablePart Name / Model
Total Units#
Units per SW/GPU#
Unit BWGbit/s
Lanes#
BW per LaneGbit/s
Agg. BWTbit/s
XPU NodesGeneric 800G DR865,53665,53640085026,214.4
Leaf
DownlinkGeneric 800G DR832,76816800810026,214.4
UplinkGeneric 800G DR832,76816800810026,214.4
Spine
DownlinkGeneric 800G DR832,76816800810026,214.4
UplinkGeneric 800G DR832,76816800810026,214.4
Core
DownlinkGeneric 800G DR832,76832800810026,214.4
Uplink·······
Super-Core

Cables

6
CB
 Network Layer
Cable/ FiberType
Total Units#
Units per Transceiver#
Unit BWGbit/s
Lane Count#
SerDes BWGbit/s
XPU NodesGeneric 400G Single Mode65,53614004100
Leaf
DownlinkAlready counted above·····
UplinkGeneric 400G Single Mode65,53624004100
Spine
DownlinkAlready counted above·····
UplinkGeneric 400G Single Mode65,53624004100
Core
DownlinkAlready counted above·····
Uplink······
Super-Core

Backend Parts Summary

4
ItemQty/ClusterQty/ServerAttach/XPU
Generic 800G DR8229,37628.003.50
Generic 400G Single Mode196,60824.003.00
ConnectX-7 NIC - 400G65,5368.001.00
Quantum-2 InfiniBand (QM9700)5,1200.630.08

Frontend network

11 rows
Assigned blockDefault
H100 - NVL8 - SN5600 - Generic
FrontendXPU: H100 NVL811 linesshared by 10 SKUs

Converged Frontend + Storage Fabric

11
LayerPart / ModelUnitsAttach / XPU
·Spectrum-4 51.2T (SN5600)7870.0120
·Spectrum-3 12.8T (SN4700)30.0000
·Spectrum-2 6.4T (SN3700)1170.0018
·Spectrum-2 1GbE (SN2201)9620.0147
·Nvidia LinkX 800G SR838,0570.5807
·Nvidia LinkX 400G SR437,8590.5777
·Nvidia LinkX 100G SR4 QSFP2816,8000.2563
·Nvidia LinkX 800G Multimode52,0840.7947
·Nvidia LinkX 400G Multimode2670.0041
·Nvidia LinkX 100G Multimode7,4830.1142
·RJ45 Cables46,1870.7048

Frontend Parts Summary

11
ItemQty/ClusterQty/ServerAttach/XPU
Spectrum-4 51.2T (SN5600)78700.0120
Spectrum-3 12.8T (SN4700)300.0001
Spectrum-2 6.4T (SN3700)11700.0018
Spectrum-2 1GbE (SN2201)96200.0147
Nvidia LinkX 800G SR838,05750.5807
Nvidia LinkX 400G SR437,85950.5777
Nvidia LinkX 100G SR4 QSFP2816,80020.2563
Nvidia LinkX 800G Multimode52,08460.7947
Nvidia LinkX 400G Multimode26700.0041
Nvidia LinkX 100G Multimode7,48310.1142
RJ45 Cables46,18760.7048

Compute Tray Networking Content

1 rows

Networking components inside the compute tray — accelerators, board-to-board connectors, retimers, and their per-tray bandwidth.

Assigned blockDefault
H100 - NVL8 - Generic
Compute TrayXPU: H100 NVL81 lineshared by 16 SKUs
Tray layerPart / ModelUnits per Tray
Unit BWGb/s
Lanes
Lane BWGb/s
Per switch
XPUs per TrayHopper 1008····
H100 ServerNVLink PCIe 5.0 x16 Retimer8512···

Compute Tray Parts Summary

1
ItemQty/ClusterQty/ServerAttach/XPU
NVLink PCIe 5.0 x16 Retimer65,5368.001.00