Live DB — analyst tables
Test
Google 16384 H100 RoCE NIC - 200G Hyperscaler Ethernet Cluster Configuration
GOOG-H100-Eth-a3mega-16k
Google · H100 · RoCE NIC - 200G
Rack type
NVL8
Cluster / location
Default H100
Buyer profile
Hyperscaler
Networking type
Ethernet
Number of XPUs
16,384
NIC used
RoCE NIC - 200G
NICs / XPU
1
Scale-out BW / XPU
200G
XPU / server
8
Scale-up network
1 rows
Assigned blockDefault
H100 - NVL8
Scale-UpXPU: H100 NVL81 lineshared by 17 SKUs
Scale-up Units are based on the scale-up world size (8 XPUs).
XPU World Size
2
WS| Scale-Up Network Layer | Part / Model | Units# | Unit BWGbit/s uni-d | Port Count# | Port BWGbit/s uni-d |
|---|---|---|---|---|---|
| XPUs per Node | · | 8 | 3,600 | · | · |
| Scale-Up World Size | · | 8 | 28,800 | · | · |
Switches
5
SW| Scale-Up Network Layer | Part / Model | Units# | Unit BWGbit/s uni-d | Port Count# | Port BWGbit/s uni-d |
|---|---|---|---|---|---|
| Scale-Up Switch A | Nvidia NVSwitch 3.0 (NVLink 4) | 1 | 8,000 | 20 | 400 |
| Scale-Up Switch B | Nvidia NVSwitch 3.0 (NVLink 4) | 1 | 6,400 | 16 | 400 |
| Scale-Up Switch C | Nvidia NVSwitch 3.0 (NVLink 4) | 1 | 6,400 | 16 | 400 |
| Scale-Up Switch D | Nvidia NVSwitch 3.0 (NVLink 4) | 1 | 8,000 | 20 | 400 |
| Scale-Up Switch Total | Nvidia NVSwitch 3.0 (NVLink 4) | 4 | 28,800 | 72 | · |
Scale-Up Parts Summary
1
| Item | Qty/Cluster | Qty/Server | Attach/XPU |
|---|---|---|---|
| Nvidia NVSwitch 3.0 (NVLink 4) | 8,192 | 4.00 | 0.50 |
Backend network
8 rows
Assigned blockDefault
H100 - NVL8 - 200G NIC - 3L - ETH - TH3 - GOOG
BackendXPU: H100 NVL88 linesshared by 1 SKU
Backend Units are based on the cluster size (16,384 XPUs).
Switches
5
SW Network Layer | Switch and Server Spec | Aggregate Port Utilization | Network Bandwidth Aggregate | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
SwitchPart Name / Model | Units# | Unit BWGbit/s | Port BWGbit/s | Downlink Ports / SW# | Uplink Ports / SW# | Max Ports# | Downlink Ports# | Uplink Ports# | Total Ports# | Downlink BWTbit/s | Uplink BWTbit/s | OversubRatio | |
| XPUs | · | 16,384 | 200 | 200 | 0 | 16,384 | 16,384 | 0 | 16,384 | 16,384 | 0 | 3,276.8 | · |
| NICs per Node | RoCE NIC - 200G | 8 | · | · | · | · | 8 | · | · | · | · | · | · |
| Leaf | WhiteBox Tomahawk 3 12.8T | 2,048 | 12,800 | 400 | 4 | 4 | 32 | 8,192 | 8,192 | 65,536 | 3,276.8 | 3,276.8 | 1 |
| Spine | WhiteBox Tomahawk 4 25.6T | 256 | 25,600 | 400 | 32 | 32 | 64 | 8,192 | 8,192 | 16,384 | 3,276.8 | 3,276.8 | 1 |
| Core | WhiteBox Tomahawk 4 25.6T | 128 | 25,600 | 400 | 64 | 0 | 64 | 8,192 | 0 | 8,192 | 3,276.8 | 0 | · |
Transceivers
7
CN Network Layer | Transceiver / CablePart Name / Model | Total Units# | Units per SW/GPU# | Unit BWGbit/s | Lanes# | BW per LaneGbit/s | Agg. BWTbit/s |
|---|---|---|---|---|---|---|---|
| GPU Nodes | None - DACs are Used -> | 16,384 | 16,384 | 200 | 4 | 50 | 3,276.8 |
| CPU Nodes | None - DACs are Used -> | · | · | · | · | · | · |
| Leaf | |||||||
| Downlink | None - DACs are Used -> | 16,384 | 8 | 200 | 4 | 50 | 3,276.8 |
| Uplink | Generic 200G DR4 QSFP56 | 16,384 | 8 | 200 | 4 | 50 | 3,276.8 |
| Spine | |||||||
| Downlink | Generic 400G SR8 | 8,192 | 32 | 400 | 8 | 50 | 3,276.8 |
| Uplink | Generic 400G SR8 | 8,192 | 32 | 400 | 8 | 50 | 3,276.8 |
| Core | |||||||
| Downlink | Generic 400G SR8 | 8,192 | 64 | 400 | 8 | 50 | 3,276.8 |
| Uplink | · | · | · | · | · | · | · |
| Super-Core | |||||||
Cables
7
CB Network Layer | Cable/ FiberType | Total Units# | Units per Transceiver# | Unit BWGbit/s | Lane Count# | SerDes BWGbit/s |
|---|---|---|---|---|---|---|
| GPU Nodes | Generic 200G 2-meter DAC | 16,384 | 1 | 200 | 4 | 50 |
| CPU Nodes | · | · | · | · | · | · |
| Leaf | ||||||
| Downlink | Already counted above | · | · | · | · | · |
| Uplink | Generic 200G Multimode | 16,384 | 1 | 200 | 4 | 50 |
| Spine | ||||||
| Downlink | Already counted above | · | · | · | · | · |
| Uplink | Generic 400G Multimode | 8,192 | 1 | 400 | 8 | 50 |
| Core | ||||||
| Downlink | · | · | · | · | · | · |
| Uplink | · | · | · | · | · | · |
| Super-Core | ||||||
Backend Parts Summary
8
| Item | Qty/Cluster | Qty/Server | Attach/XPU |
|---|---|---|---|
| Generic 200G 2-meter DAC | 16,384 | 8.00 | 1.00 |
| RoCE NIC - 200G | 16,384 | 8.00 | 1.00 |
| Generic 200G DR4 QSFP56 | 16,384 | 8.00 | 1.00 |
| Generic 200G Multimode | 16,384 | 8.00 | 1.00 |
| WhiteBox Tomahawk 3 12.8T | 2,048 | 1.00 | 0.13 |
| Generic 400G SR8 | 24,576 | 12.00 | 1.50 |
| WhiteBox Tomahawk 4 25.6T | 384 | 0.19 | 0.02 |
| Generic 400G Multimode | 8,192 | 4.00 | 0.50 |
Frontend network
11 rows
Assigned blockDefault
H100 - NVL8 - WhiteBox TH5 - GOOG
FrontendXPU: H100 NVL811 linesshared by 2 SKUs
Converged Frontend + Storage Fabric
11
| Layer | Part / Model | Units | Attach / XPU |
|---|---|---|---|
| · | WhiteBox Tomahawk 5 51.2T | 197 | 0.0120 |
| · | WhiteBox Tomahawk 3 12.8T | 1 | 0.0000 |
| · | WhiteBox Tomahawk 2 6.4T | 29 | 0.0018 |
| · | Generic 1GbE Ethernet | 241 | 0.0147 |
| · | Generic 800G DR8 | 9,514 | 0.5807 |
| · | Generic 400G SR8 | 9,465 | 0.5777 |
| · | Generic 100G SR4 QSFP28 | 4,200 | 0.2563 |
| · | Generic 800G Single Mode | 13,021 | 0.7947 |
| · | Generic 400G Multimode | 67 | 0.0041 |
| · | Generic 100G Multimode | 1,871 | 0.1142 |
| · | RJ45 Cables | 11,547 | 0.7048 |
Frontend Parts Summary
11
| Item | Qty/Cluster | Qty/Server | Attach/XPU |
|---|---|---|---|
| WhiteBox Tomahawk 5 51.2T | 197 | 0 | 0.0120 |
| WhiteBox Tomahawk 3 12.8T | 1 | 0 | 0.0001 |
| WhiteBox Tomahawk 2 6.4T | 29 | 0 | 0.0018 |
| Generic 1GbE Ethernet | 241 | 0 | 0.0147 |
| Generic 800G DR8 | 9,514 | 5 | 0.5807 |
| Generic 400G SR8 | 9,465 | 5 | 0.5777 |
| Generic 100G SR4 QSFP28 | 4,200 | 2 | 0.2563 |
| Generic 800G Single Mode | 13,021 | 6 | 0.7947 |
| Generic 400G Multimode | 67 | 0 | 0.0041 |
| Generic 100G Multimode | 1,871 | 1 | 0.1142 |
| RJ45 Cables | 11,547 | 6 | 0.7048 |
Compute Tray Networking Content
1 rows
Networking components inside the compute tray — accelerators, board-to-board connectors, retimers, and their per-tray bandwidth.
Assigned blockDefault
H100 - NVL8 - Generic
Compute TrayXPU: H100 NVL81 lineshared by 16 SKUs
| Tray layer | Part / Model | Units per Tray | Unit BWGb/s | Lanes | Lane BWGb/s | Per switch |
|---|---|---|---|---|---|---|
| XPUs per Tray | Hopper 100 | 8 | · | · | · | · |
| H100 Server | NVLink PCIe 5.0 x16 Retimer | 8 | 512 | · | · | · |
Compute Tray Parts Summary
1
| Item | Qty/Cluster | Qty/Server | Attach/XPU |
|---|---|---|---|
| NVLink PCIe 5.0 x16 Retimer | 16,384 | 8.00 | 1.00 |