Live DB — analyst tables
Test
Microsoft 157696 GB200 NVL72 CX-8 Hyperscaler Ethernet Cluster Configuration
MSFT-GB200-CX8-TH5U-AR-WIS
Microsoft · GB200 · CX-8
Rack type
NVL72
Cluster / location
Fairwater 2 - Atlanta, GA
Buyer profile
Hyperscaler
Networking type
Ethernet
Number of XPUs
157,696
NIC used
CX-8
NICs / XPU
1
Scale-out BW / XPU
800G
XPU / server
72
Scale-up network
8 rows
Assigned blockDefault
GB200 - NVL72
Scale-UpXPU: GB200 NVL728 linesshared by 10 SKUs
Scale-up Units are based on the scale-up world size (72 XPUs).
XPU World Size
2
WS| Scale-Up Network Layer | Part / Model | Units# | Unit BWGbit/s uni-d | Lane Count# | Lane BWGbit/s uni-d |
|---|---|---|---|---|---|
| XPUs per Node | · | 4 | 7,200 | 18 | 400 |
| Scale-Up World Size | · | 72 | 518,400 | · | · |
Switches
2
SW| Scale-Up Network Layer | Part / Model | Units# | Unit BWGbit/s uni-d | Lane Count# | Lane BWGbit/s uni-d |
|---|---|---|---|---|---|
| per Switch | Nvidia NVSwitch 5.0 (NVLink 5) | 18 | 28,800 | 72 | 400 |
| Scale-Up Switch Total | · | 18 | 518,400 | · | · |
Connections & cabling
8
CN| Scale-Up Network Layer | Part / Model | Units# | BW / PortGbit/s uni-d | Unit DP# | DP / Port# | Total DPs# | BW / Connector | Cable / Model | Cable Units# |
|---|---|---|---|---|---|---|---|---|---|
| Ultrapass to Switch | NV 2 UltraPass OverPass 144 DP 224Gb/s | 36 | 400 | 144 | 4 | 5,184 | 14,400 | · | · |
| Male Connectors | NV Paladin HD2 224Gb/s 144 DP (Male) | 36 | 400 | 144 | 4 | 5,184 | 14,400 | Already Counted Below | · |
| Female Connectors | NV Paladin HD2 224Gb/s 144 DP (Female) | 36 | 400 | 144 | 4 | 5,184 | 14,400 | Already Counted Below | · |
| Switch Total | ** Two connectors per switch | · | · | · | · | · | 28,800 | · | · |
| Male Connectors | NV Paladin HD2 224Gb/s 72 DP (Male) | 72 | 400 | 72 | 4 | 5,184 | 7,200 | Already Counted Below | · |
| Female Connectors | NV Paladin HD2 224Gb/s 72 DP (Female) | 72 | 400 | 72 | 4 | 5,184 | 7,200 | NV 224Gb/s SkewClear EXD Gen 2 Cable | 5,184 |
| XPU Total | ** One connector per GPU | · | · | · | · | · | 7,200 | · | · |
| · | NVL Spine Chassis (NvLink 5) | 2 | · | · | · | · | · | · | · |
Scale-Up Parts Summary
8
| Item | Qty/Cluster | Qty/Server | Attach/XPU |
|---|---|---|---|
| Nvidia NVSwitch 5.0 (NVLink 5) | 39,424 | 18.00 | 0.25 |
| NV 2 UltraPass OverPass 144 DP 224Gb/s | 78,848 | 36.00 | 0.50 |
| NV Paladin HD2 224Gb/s 144 DP (Male) | 78,848 | 36.00 | 0.50 |
| NV Paladin HD2 224Gb/s 144 DP (Female) | 78,848 | 36.00 | 0.50 |
| NV Paladin HD2 224Gb/s 72 DP (Male) | 157,696 | 72.00 | 1.00 |
| NV Paladin HD2 224Gb/s 72 DP (Female) | 157,696 | 72.00 | 1.00 |
| NV 224Gb/s SkewClear EXD Gen 2 Cable | 11,354,112 | 5,184.00 | 72.00 |
| NVL Spine Chassis (NvLink 5) | 4,380 | 2.00 | 0.03 |
Backend network
7 rows
Assigned blockDefault
GB200 - NVL72 - CX8 - 3L - ETH - TH5 - MSFT
BackendXPU: GB200 NVL727 linesshared by 1 SKU
Backend Units are based on the cluster size (157,696 XPUs).
Switches
5
SW Network Layer | Switch and Server Spec | Aggregate Port Utilization | Network Bandwidth Aggregate | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
SwitchPart Name / Model | Units# | Unit BWGbit/s | Port BWGbit/s | Downlink Ports / SW# | Uplink Ports / SW# | Max Ports# | Downlink Ports# | Uplink Ports# | Total Ports# | Downlink BWTbit/s | Uplink BWTbit/s | OversubRatio | |
| XPUs | · | 157,696 | 800 | 100 | 0 | 1,261,568 | 1,261,568 | 0 | 1,261,568 | 1,261,568 | 0 | 126,156.8 | · |
| NICs per Node | ConnectX-8 NIC - 800G | 4 | 800 | 100 | · | · | 8 | · | · | · | · | · | · |
| Leaf | Arista Tomahawk 5 51.2T 512 Radix | 4,928 | 51,200 | 100 | 256 | 256 | 512 | 1,261,568 | 1,261,568 | 2,523,136 | 126,156.8 | 126,156.8 | 1 |
| Spine | Arista Tomahawk 5 51.2T 512 Radix | 4,096 | 51,200 | 100 | 308 | 40 | 512 | 1,261,568 | 163,840 | 2,097,152 | 126,156.8 | 16,384 | 7.7 |
| Core | Optical Circuit Switch (OCS) | 142.222 | 115,200 | 100 | 1,152 | 0 | 1,152 | 163,840 | 0 | 163,840 | 16,384 | 0 | · |
Transceivers
8
CN Network Layer | Transceiver / CablePart Name / Model | Total Units# | Units per SW/GPU# | Unit BWGbit/s | Lanes# | BW per LaneGbit/s | Agg. BWTbit/s |
|---|---|---|---|---|---|---|---|
| XPU Nodes | Generic 800G DR8 | 157,696 | 157,696 | 800 | 8 | 100 | 126,156.8 |
| Leaf | |||||||
| Downlink | Generic 800G DR8 | 157,696 | 32 | 800 | 8 | 100 | 126,156.8 |
| Uplink | Generic 800G DR8 | 157,696 | 32 | 800 | 8 | 100 | 126,156.8 |
| Spine | |||||||
| Downlink | Generic 800G DR8 | 157,696 | 38.5 | 800 | 8 | 100 | 126,156.8 |
| Uplink | 800G FR8 with Circulator | 20,480 | 5 | 800 | 8 | 100 | 16,384 |
| Core | |||||||
| Downlink | 800G FR8 with Circulator | 20,480 | 144 | 800 | 8 | 100 | 16,384 |
| Uplink | 800G FR8 with Circulator | 20,480 | 144 | 800 | 8 | 100 | 16,384 |
| Super-Core | Note: The core will be shared with Fairwater 2 Building B, we will count it only here in Building A, and Building B will be missing the parts for Core layer. | · | · | · | · | · | · |
Cables
8
CB Network Layer | Cable/ FiberType | Total Units# | Units per Transceiver# | Unit BWGbit/s | Lane Count# | SerDes BWGbit/s |
|---|---|---|---|---|---|---|
| XPU Nodes | Generic 400G Single Mode | 315,392 | 2 | 400 | 4 | 100 |
| Leaf | ||||||
| Downlink | Already counted above | · | · | · | · | · |
| Uplink | Generic 400G Single Mode | 315,392 | 2 | 400 | 4 | 100 |
| Spine | ||||||
| Downlink | Already counted above | · | · | · | · | · |
| Uplink | Generic 800G Single Mode | 20,480 | 1 | 800 | 4 | 200 |
| Core | ||||||
| Downlink | Already counted above | · | · | · | · | · |
| Uplink | Generic 800G Single Mode | 20,480 | 1 | 800 | 8 | 100 |
| Super-Core | · | · | · | · | · | · |
Backend Parts Summary
7
| Item | Qty/Cluster | Qty/Server | Attach/XPU |
|---|---|---|---|
| Generic 800G DR8 | 630,784 | 288.00 | 4.00 |
| Generic 400G Single Mode | 630,784 | 288.00 | 4.00 |
| ConnectX-8 NIC - 800G | 157,696 | 72.00 | 1.00 |
| Arista Tomahawk 5 51.2T 512 Radix | 9,024 | 4.12 | 0.06 |
| 800G FR8 with Circulator | 61,440 | 28.05 | 0.39 |
| Generic 800G Single Mode | 40,960 | 18.70 | 0.26 |
| Optical Circuit Switch (OCS) | 142 | 0.06 | 0.00 |
Frontend network
11 rows
Assigned blockDefault
GB200 - NVL72 - Arista TH5 - Generic
FrontendXPU: GB200 NVL7211 linesshared by 3 SKUs
Converged Frontend + Storage Fabric
11
| Layer | Part / Model | Units | Attach / XPU |
|---|---|---|---|
| · | Arista Tomahawk 5 51.2T (7060X6-64PE) | 1,893 | 0.0120 |
| · | Arista Tomahawk 3 12.8T (7050PX4-32S) | 8 | 0.0000 |
| · | Arista Tomahawk 2 6.4T (7050CX3-32S) | 282 | 0.0018 |
| · | Arista Ethernet 1GbE (7048-A) | 2,315 | 0.0147 |
| · | Generic 800G DR8 | 91,575 | 0.5807 |
| · | Generic 400G DR4 | 91,098 | 0.5777 |
| · | Generic 100G DR4 QSFP28 | 40,425 | 0.2563 |
| · | Generic 800G Single Mode | 125,328 | 0.7947 |
| · | Generic 400G Single Mode | 641 | 0.0041 |
| · | Generic 100G Single Mode | 18,007 | 0.1142 |
| · | RJ45 Cables | 111,138 | 0.7048 |
Frontend Parts Summary
11
| Item | Qty/Cluster | Qty/Server | Attach/XPU |
|---|---|---|---|
| Arista Tomahawk 5 51.2T (7060X6-64PE) | 1,893 | 1 | 0.0120 |
| Arista Tomahawk 3 12.8T (7050PX4-32S) | 8 | 0 | 0.0001 |
| Arista Tomahawk 2 6.4T (7050CX3-32S) | 282 | 0 | 0.0018 |
| Arista Ethernet 1GbE (7048-A) | 2,315 | 1 | 0.0147 |
| Generic 800G DR8 | 91,575 | 42 | 0.5807 |
| Generic 400G DR4 | 91,098 | 42 | 0.5777 |
| Generic 100G DR4 QSFP28 | 40,425 | 18 | 0.2563 |
| Generic 800G Single Mode | 125,328 | 57 | 0.7947 |
| Generic 400G Single Mode | 641 | 0 | 0.0041 |
| Generic 100G Single Mode | 18,007 | 8 | 0.1142 |
| RJ45 Cables | 111,138 | 51 | 0.7048 |
Compute Tray Networking Content
3 rows
Networking components inside the compute tray — accelerators, board-to-board connectors, retimers, and their per-tray bandwidth.
Assigned blockDefault
GB200 - NVL72 - Generic
Compute TrayXPU: GB200 NVL723 linesshared by 10 SKUs
| Tray layer | Part / Model | Units per Tray | Unit BWGb/s | Lanes | Lane BWGb/s | Per switch |
|---|---|---|---|---|---|---|
| XPUs per Tray | Blackwell 200 | 4 | · | · | · | · |
| Bianca Board | NV MCIO PCIe x16 Connector | 2 | · | · | · | · |
| Bianca Board | NV MCIO PCIe x16 Cable | 2 | · | · | · | · |
| Mezz Board to OSFP | NV DensiLink OverPass to OSFP Cage | 4 | · | · | · | · |
Compute Tray Parts Summary
3
| Item | Qty/Cluster | Qty/Server | Attach/XPU |
|---|---|---|---|
| NV MCIO PCIe x16 Connector | 78,848 | 36.00 | 0.50 |
| NV MCIO PCIe x16 Cable | 78,848 | 36.00 | 0.50 |
| NV DensiLink OverPass to OSFP Cage | 157,696 | 72.00 | 1.00 |