Live DB — analyst tables
Test
Meta 18144 GB200 NVL36x2 CX-7 Hyperscaler DSF Ethernet Cluster Configuration
META-GB200NVL36x2-CX7-DSF-AR
Meta · GB200 · CX-7
Rack type
NVL36x2
Cluster / location
All Other Clusters
Buyer profile
Hyperscaler
Networking type
DSF Ethernet
Number of XPUs
18,144
NIC used
CX-7
NICs / XPU
1
Scale-out BW / XPU
400G
XPU / server
72
Scale-up network
8 rows
Assigned blockDefault
GB200 - NVL36x2
Scale-UpXPU: GB200 NVL36x28 linesshared by 6 SKUs
Scale-up Units are based on the scale-up world size (72 XPUs).
XPU World Size
2
WS| Scale-Up Network Layer | Part / Model | Units# | Unit BWGbit/s uni-d | Port Count# | Port BWGbit/s uni-d |
|---|---|---|---|---|---|
| XPUs per Node | · | 4 | 7,200 | 18 | 400 |
| Scale-Up World Size | · | 72 | 518,400 | · | · |
Switches
2
SW| Scale-Up Network Layer | Part / Model | Units# | Unit BWGbit/s uni-d | Port Count# | Port BWGbit/s uni-d |
|---|---|---|---|---|---|
| per Switch | Nvidia NVSwitch 5.0 (NVLink 5) | 1 | 28,800 | 72 | 400 |
| Scale-Up Switch Total | · | 36 | 1,036,800 | · | · |
Connections & cabling
10
CN| Scale-Up Network Layer | Part / Model | Units# | BW / PortGbit/s uni-d | Unit DP# | DP / Port# | Total DPs# | BW / Connector | Cable / Model | Cable Units# |
|---|---|---|---|---|---|---|---|---|---|
| Ultrapass to Switch | NV 1 UltraPass OverPass 72 DP 224Gb/s | 72 | 400 | 144 | 4 | 10,368 | 7,200 | · | · |
| Male Connectors | NV Paladin HD2 224Gb/s 72 DP (Male) | 144 | 400 | 72 | 4 | 10,368 | 7,200 | Already Counted Below | · |
| Female Connectors | NV Paladin HD2 224Gb/s 72 DP (Female) | 144 | 400 | 72 | 4 | 10,368 | 7,200 | Already Counted Below | · |
| Switch Total (Connectors) | ** Two connectors per switch | · | · | · | · | · | 14,400 | · | · |
| Male Connectors | NV Paladin HD2 224Gb/s 72 DP (Male) | 72 | 400 | 72 | 4 | 5,184 | 7,200 | Already Counted Below | · |
| Female Connectors | NV Paladin HD2 224Gb/s 72 DP (Female) | 72 | 400 | 72 | 4 | 5,184 | 7,200 | NV 224Gb/s SkewClear EXD Gen 2 Cable | 5,184 |
| XPU Total (Connectors) | ** One connector per GPU | · | · | · | · | · | 14,400 | · | · |
| Overpass to 1.6T OSFP Cage | NV OverPass DensiLink UltraPass (NVLink 5) | 324 | · | · | · | · | · | · | · |
| Cross Rack Cables | NV 1.6T Twin Port OSFP 2x800G LinkX NVLink5 ACC | 162 | · | · | · | · | · | · | · |
| · | NVL Spine Chassis (NvLink 5) | 4 | · | · | · | · | · | · | · |
Notes
- Male Connectors · NV Paladin HD2 224Gb/s 72 DP (Male) — >> 16 lanes or 32 DP are Redundant
- Female Connectors · NV Paladin HD2 224Gb/s 72 DP (Female) — >> Explains why the DPs avail at switch trays exceed
- per Switch · Nvidia NVSwitch 5.0 (NVLink 5) — >>> Only 50% of ports utilized for GPU-Switch Connection
- Switch Total (Connectors) · ** Two connectors per switch — ...DPs avail on the XPU Tray
Scale-Up Parts Summary
8
| Item | Qty/Cluster | Qty/Server | Attach/XPU |
|---|---|---|---|
| Nvidia NVSwitch 5.0 (NVLink 5) | 9,072 | 36.00 | 0.50 |
| NV 1 UltraPass OverPass 72 DP 224Gb/s | 18,144 | 72.00 | 1.00 |
| NV Paladin HD2 224Gb/s 72 DP (Male) | 54,432 | 216.00 | 3.00 |
| NV Paladin HD2 224Gb/s 72 DP (Female) | 54,432 | 216.00 | 3.00 |
| NV 224Gb/s SkewClear EXD Gen 2 Cable | 1,306,368 | 5,184.00 | 72.00 |
| NV OverPass DensiLink UltraPass (NVLink 5) | 81,648 | 324.00 | 4.50 |
| NV 1.6T Twin Port OSFP 2x800G LinkX NVLink5 ACC | 40,824 | 162.00 | 2.25 |
| NVL Spine Chassis (NvLink 5) | 1,008 | 4.00 | 0.06 |
Backend network
6 rows
Assigned blockDefault
GB200 - NVL36x2 - CX7 - 3L - ETH - 7700R4C - META
BackendXPU: GB200 NVL36x26 linesshared by 1 SKU
Backend Units are based on the cluster size (18,144 XPUs).
Switches
6
SW Network Layer | Switch and Server Spec | Aggregate Port Utilization | Network Bandwidth Aggregate | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
SwitchPart Name / Model | Units# | Unit BWGbit/s | Port BWGbit/s | Downlink Ports / SW# | Uplink Ports / SW# | Max Ports# | Downlink Ports# | Uplink Ports# | Total Ports# | Downlink BWTbit/s | Uplink BWTbit/s | OversubRatio | |
| XPUs | · | 18,144 | 400 | 400 | 0 | 18,144 | 18,144 | 0 | 18,144 | 18,144 | 0 | 7,257.6 | · |
| NICs per Node | ConnectX-7 NIC - 400G | 4 | 400 | 400 | · | · | 8 | · | · | · | · | · | · |
| Leaf (RDSW) | Arista 14.4T DSF Leaf (7700R4C) | 504 | 30,400 | 400 | 36 | 40 | 76 | 18,144 | 20,160 | 38,304 | 7,257.6 | 8,064 | 0.9 |
| Spine (FDSW) | Arista 102.4T DSF Spine (7720R4) | 160 | 102,400 | 400 | 126 | 128 | 256 | 20,160 | 20,480 | 40,960 | 8,064 | 8,192 | 0.984 |
| Core (SDSW) | Arista 102.4T DSF Spine (7720R4) | 128 | 102,400 | 400 | 160 | 0 | 256 | 20,480 | 0 | 32,768 | 8,192 | 0 | · |
| Super-Core | This is like the diagram for GB300 NVL72 with DSF - just using one plane only. | · | · | · | · | · | · | 0 | 0 | 0 | 0 | 0 | · |
Transceivers
6
CN Network Layer | Transceiver / CablePart Name / Model | Total Units# | Units per SW/GPU# | Unit BWGbit/s | Lanes# | BW per LaneGbit/s | Agg. BWTbit/s |
|---|---|---|---|---|---|---|---|
| XPU Nodes | Generic 400G DR4 | 18,144 | 18,144 | 400 | 4 | 100 | 7,257.6 |
| Leaf | |||||||
| Downlink | Generic 800G DR8 | 9,072 | 18 | 800 | 8 | 100 | 7,257.6 |
| Uplink | Generic 800G DR8 | 10,080 | 20 | 800 | 8 | 100 | 8,064 |
| Spine | |||||||
| Downlink | Generic 800G DR8 | 10,080 | 63 | 800 | 8 | 100 | 8,064 |
| Uplink | Generic 800G DR8 | 10,240 | 64 | 800 | 8 | 100 | 8,192 |
| Core | |||||||
| Downlink | Generic 800G DR8 | 10,240 | 80 | 800 | 8 | 100 | 8,192 |
| Uplink | · | · | · | · | · | · | · |
| Super-Core | |||||||
Cables
6
CB Network Layer | Cable/ FiberType | Total Units# | Units per Transceiver# | Unit BWGbit/s | Lane Count# | SerDes BWGbit/s |
|---|---|---|---|---|---|---|
| XPU Nodes | Generic 400G Single Mode | 18,144 | 1 | 400 | 4 | 100 |
| Leaf | ||||||
| Downlink | Already counted above | · | · | · | · | · |
| Uplink | Generic 400G Single Mode | 20,160 | 2 | 400 | 4 | 100 |
| Spine | ||||||
| Downlink | Already counted above | · | · | · | · | · |
| Uplink | Generic 400G Single Mode | 20,480 | 2 | 400 | 4 | 100 |
| Core | ||||||
| Downlink | Already counted above | · | · | · | · | · |
| Uplink | · | · | · | · | · | · |
| Super-Core | ||||||
Backend Parts Summary
6
| Item | Qty/Cluster | Qty/Server | Attach/XPU |
|---|---|---|---|
| Generic 400G DR4 | 18,144 | 72.00 | 1.00 |
| Generic 400G Single Mode | 58,784 | 233.27 | 3.24 |
| Generic 800G DR8 | 49,712 | 197.27 | 2.74 |
| ConnectX-7 NIC - 400G | 18,144 | 72.00 | 1.00 |
| Arista 14.4T DSF Leaf (7700R4C) | 504 | 2.00 | 0.03 |
| Arista 102.4T DSF Spine (7720R4) | 288 | 1.14 | 0.02 |
Notes
- Leaf (RDSW) · Arista 14.4T DSF Leaf (7700R4C) — >> Empty Ports
Frontend network
11 rows
Assigned blockDefault
GB200 - NVL36x2 - Minipack-3 - META
FrontendXPU: GB200 NVL36x211 linesshared by 1 SKU
Converged Frontend + Storage Fabric
11
| Layer | Part / Model | Units | Attach / XPU |
|---|---|---|---|
| · | Minipack-3 OCP Rack 51.2T | 218 | 0.0120 |
| · | Minipack-1 OCP Rack 12.8T | 1 | 0.0000 |
| · | WhiteBox Tomahawk 2 6.4T | 32 | 0.0018 |
| · | Generic 1GbE Ethernet | 266 | 0.0147 |
| · | Generic 800G DR8 | 10,536 | 0.5807 |
| · | Generic 400G DR4 | 10,481 | 0.5777 |
| · | Generic 100G DR4 QSFP28 | 4,651 | 0.2563 |
| · | Generic 800G Single Mode | 14,420 | 0.7947 |
| · | Generic 400G Single Mode | 74 | 0.0041 |
| · | Generic 100G Single Mode | 2,072 | 0.1142 |
| · | RJ45 Cables | 12,787 | 0.7048 |
Frontend Parts Summary
11
| Item | Qty/Cluster | Qty/Server | Attach/XPU |
|---|---|---|---|
| Minipack-3 OCP Rack 51.2T | 218 | 1 | 0.0120 |
| Minipack-1 OCP Rack 12.8T | 1 | 0 | 0.0001 |
| WhiteBox Tomahawk 2 6.4T | 32 | 0 | 0.0018 |
| Generic 1GbE Ethernet | 266 | 1 | 0.0147 |
| Generic 800G DR8 | 10,536 | 42 | 0.5807 |
| Generic 400G DR4 | 10,481 | 42 | 0.5777 |
| Generic 100G DR4 QSFP28 | 4,651 | 18 | 0.2563 |
| Generic 800G Single Mode | 14,420 | 57 | 0.7947 |
| Generic 400G Single Mode | 74 | 0 | 0.0041 |
| Generic 100G Single Mode | 2,072 | 8 | 0.1142 |
| RJ45 Cables | 12,787 | 51 | 0.7048 |
Compute Tray Networking Content
3 rows
Networking components inside the compute tray — accelerators, board-to-board connectors, retimers, and their per-tray bandwidth.
Assigned blockDefault
GB200 - NVL36x2 - Generic
Compute TrayXPU: GB200 NVL36x23 linesshared by 5 SKUs
| Tray layer | Part / Model | Units per Tray | Unit BWGb/s | Lanes | Lane BWGb/s | Per switch |
|---|---|---|---|---|---|---|
| XPUs per Tray | Blackwell 200 | 4 | · | · | · | · |
| Bianca Board | NV MCIO PCIe x16 Connector | 2 | · | · | · | · |
| Bianca Board | NV MCIO PCIe x16 Cable | 2 | · | · | · | · |
| Mezz Board to OSFP | NV DensiLink OverPass to OSFP Cage | 4 | · | · | · | · |
Compute Tray Parts Summary
3
| Item | Qty/Cluster | Qty/Server | Attach/XPU |
|---|---|---|---|
| NV MCIO PCIe x16 Connector | 9,072 | 36.00 | 0.50 |
| NV MCIO PCIe x16 Cable | 9,072 | 36.00 | 0.50 |
| NV DensiLink OverPass to OSFP Cage | 18,144 | 72.00 | 1.00 |