Live DB — analyst tables
Test
Meta 16000 A100 CX-6 Hyperscaler Ethernet Cluster Configuration
META-A100-CX6-IB-NV-16KRSC
Meta · A100 · CX-6
Rack type
NVL8
Cluster / location
16k RSC Infiniband Cluster
Buyer profile
Hyperscaler
Networking type
Ethernet
Number of XPUs
16,000
NIC used
CX-6
NICs / XPU
1
Scale-out BW / XPU
200G
XPU / server
8
Scale-up network
1 rows
Assigned blockDefault
A100 - NVL8
Scale-UpXPU: A100 NVL81 lineshared by 9 SKUs
Scale-up Units are based on the scale-up world size (8 XPUs).
XPU World Size
2
WS| Scale-Up Network Layer | Part / Model | Units# | Unit BWGbit/s uni-d | Lane Count# | Lane BWGbit/s uni-d |
|---|---|---|---|---|---|
| XPUs per Node | · | 8 | 2,400 | 24 | 100 |
| Scale-Up World Size | · | 8 | 19,200 | · | · |
Switches
2
SW| Scale-Up Network Layer | Part / Model | Units# | Unit BWGbit/s uni-d | Lane Count# | Lane BWGbit/s uni-d |
|---|---|---|---|---|---|
| per Switch | Nvidia NVSwitch 2.0 (NVLink 3) | 6 | 2,400 | 24 | 100 |
| Scale-Up Switch Total | · | · | 14,400 | · | · |
Notes
- XPUs per Node — 600GB Bidi
Scale-Up Parts Summary
1
| Item | Qty/Cluster | Qty/Server | Attach/XPU |
|---|---|---|---|
| Nvidia NVSwitch 2.0 (NVLink 3) | 12,000 | 6.00 | 0.75 |
Backend network
4 rows
Assigned blockDefault
A100 - NVL8 - CX6 - 3L - IB - QM8700 - META
BackendXPU: A100 NVL84 linesshared by 1 SKU
Backend Units are based on the cluster size (16,000 XPUs).
Switches
5
SW Network Layer | Switch and Server Spec | Aggregate Port Utilization | Network Bandwidth Aggregate | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
SwitchPart Name / Model | Units# | Unit BWGbit/s | Port BWGbit/s | Downlink Ports / SW# | Uplink Ports / SW# | Max Ports# | Downlink Ports# | Uplink Ports# | Total Ports# | Downlink BWTbit/s | Uplink BWTbit/s | OversubRatio | |
| XPUs | · | 16,000 | 200 | 200 | 0 | 16,000 | 16,000 | 0 | 16,000 | 16,000 | 0 | 3,200 | · |
| NICs per Node | ConnectX-6 NIC - 200G | 8 | 200 | 200 | · | · | 1 | · | · | · | · | · | · |
| Leaf | Quantum InfiniBand HDR (QM8700) | 800 | 8,000 | 200 | 20 | 20 | 40 | 16,000 | 16,000 | 32,000 | 3,200 | 3,200 | 1 |
| Spine | Quantum InfiniBand HDR (QM8700) | 800 | 8,000 | 200 | 20 | 20 | 40 | 16,000 | 16,000 | 32,000 | 3,200 | 3,200 | 1 |
| Core | Quantum InfiniBand HDR (QM8700) | 400 | 8,000 | 200 | 40 | 0 | 40 | 16,000 | 0 | 16,000 | 3,200 | 0 | · |
Transceivers
6
CN Network Layer | Transceiver / CablePart Name / Model | Total Units# | Units per SW/GPU# | Unit BWGbit/s | Lanes# | BW per LaneGbit/s | Agg. BWTbit/s |
|---|---|---|---|---|---|---|---|
| XPU Nodes | Nvidia LinkX 200G DR4 QSFP56 | 16,000 | 16,000 | 200 | 4 | 50 | 3,200 |
| Leaf | |||||||
| Downlink | Nvidia LinkX 200G DR4 QSFP56 | 16,000 | 20 | 200 | 4 | 50 | 3,200 |
| Uplink | Nvidia LinkX 200G DR4 QSFP56 | 16,000 | 20 | 200 | 4 | 50 | 3,200 |
| Spine | |||||||
| Downlink | Nvidia LinkX 200G DR4 QSFP56 | 16,000 | 20 | 200 | 4 | 50 | 3,200 |
| Uplink | Nvidia LinkX 200G DR4 QSFP56 | 16,000 | 20 | 200 | 4 | 50 | 3,200 |
| Core | |||||||
| Downlink | Nvidia LinkX 200G DR4 QSFP56 | 16,000 | 40 | 200 | 4 | 50 | 3,200 |
| Uplink | · | · | · | · | · | · | · |
| Super-Core | |||||||
Cables
6
CB Network Layer | Cable/ FiberType | Total Units# | Units per Transceiver# | Unit BWGbit/s | Lane Count# | SerDes BWGbit/s |
|---|---|---|---|---|---|---|
| XPU Nodes | Nvidia LinkX 200G Single Mode | 16,000 | 1 | 200 | 4 | 50 |
| Leaf | ||||||
| Downlink | Already counted above | · | · | · | · | · |
| Uplink | Nvidia LinkX 200G Single Mode | 16,000 | 1 | 200 | 4 | 50 |
| Spine | ||||||
| Downlink | Already counted above | · | · | · | · | · |
| Uplink | Nvidia LinkX 200G Single Mode | 16,000 | 1 | 200 | 4 | 50 |
| Core | ||||||
| Downlink | Already counted above | · | · | · | · | · |
| Uplink | · | · | · | · | · | · |
| Super-Core | ||||||
Backend Parts Summary
4
| Item | Qty/Cluster | Qty/Server | Attach/XPU |
|---|---|---|---|
| Nvidia LinkX 200G DR4 QSFP56 | 96,000 | 48.00 | 6.00 |
| Nvidia LinkX 200G Single Mode | 48,000 | 24.00 | 3.00 |
| ConnectX-6 NIC - 200G | 16,000 | 8.00 | 1.00 |
| Quantum InfiniBand HDR (QM8700) | 2,000 | 1.00 | 0.13 |
Frontend network
11 rows
Assigned blockDefault
A100 - NVL8 - SN5600 - Generic
FrontendXPU: A100 NVL811 linesshared by 8 SKUs
Converged Frontend + Storage Fabric
11
| Layer | Part / Model | Units | Attach / XPU |
|---|---|---|---|
| · | Spectrum-4 51.2T (SN5600) | 192 | 0.0120 |
| · | Spectrum-3 12.8T (SN4700) | 1 | 0.0000 |
| · | Spectrum-2 6.4T (SN3700) | 29 | 0.0018 |
| · | Spectrum-2 1GbE (SN2201) | 235 | 0.0147 |
| · | Nvidia LinkX 800G SR8 | 9,291 | 0.5807 |
| · | Nvidia LinkX 400G SR4 | 9,243 | 0.5777 |
| · | Nvidia LinkX 100G SR4 QSFP28 | 4,102 | 0.2563 |
| · | Nvidia LinkX 800G Multimode | 12,716 | 0.7947 |
| · | Nvidia LinkX 400G Multimode | 65 | 0.0041 |
| · | Nvidia LinkX 100G Multimode | 1,827 | 0.1142 |
| · | RJ45 Cables | 11,276 | 0.7048 |
Frontend Parts Summary
11
| Item | Qty/Cluster | Qty/Server | Attach/XPU |
|---|---|---|---|
| Spectrum-4 51.2T (SN5600) | 19 | 0 | 0.0012 |
| Spectrum-3 12.8T (SN4700) | 0 | 0 | 0.0000 |
| Spectrum-2 6.4T (SN3700) | 3 | 0 | 0.0002 |
| Spectrum-2 1GbE (SN2201) | 23 | 0 | 0.0015 |
| Nvidia LinkX 800G SR8 | 929 | 0 | 0.0581 |
| Nvidia LinkX 400G SR4 | 924 | 0 | 0.0578 |
| Nvidia LinkX 100G SR4 QSFP28 | 410 | 0 | 0.0256 |
| Nvidia LinkX 800G Multimode | 1,272 | 1 | 0.0795 |
| Nvidia LinkX 400G Multimode | 7 | 0 | 0.0004 |
| Nvidia LinkX 100G Multimode | 183 | 0 | 0.0114 |
| RJ45 Cables | 1,128 | 1 | 0.0705 |
Compute Tray Networking Content
1 rows
Networking components inside the compute tray — accelerators, board-to-board connectors, retimers, and their per-tray bandwidth.
Assigned blockDefault
A100 - NVL8 - Generic
Compute TrayXPU: A100 NVL81 lineshared by 8 SKUs
| Tray layer | Part / Model | Units per Tray | Unit BWGb/s | Lanes | Lane BWGb/s | Per switch |
|---|---|---|---|---|---|---|
| XPUs per Tray | Ampere 100 | 8 | · | · | · | · |
| A100 Server | NVLink PCIe 4.0 x16 Retimer | 8 | 256 | · | · | · |
Compute Tray Parts Summary
1
| Item | Qty/Cluster | Qty/Server | Attach/XPU |
|---|---|---|---|
| NVLink PCIe 4.0 x16 Retimer | 16,000 | 8.00 | 1.00 |