Live DB — analyst tables
Test
Amazon 32768 P4d-A100 NVL8 Nitro-v3 Hyperscaler EFA Cluster Configuration

AWS-P4d-A100 NVL8-EFA-WB

Amazon · A100 · Nitro-v3

Rack type
P4d NVL8
Cluster / location
P4d-A100 NVL8 Clusters
Buyer profile
Hyperscaler
Networking type
EFA
Number of XPUs
32,768
NIC used
Nitro-v3
NICs / XPU
0.5
Scale-out BW / XPU
50G
XPU / server
8

Scale-up network

1 rows
Assigned blockDefault
A100 - NVL8
Scale-UpXPU: A100 NVL81 lineshared by 9 SKUs

Scale-up Units are based on the scale-up world size (8 XPUs).

XPU World Size

2
WS
Scale-Up Network LayerPart / Model
Units#
Unit BWGbit/s uni-d
Lane Count#
Lane BWGbit/s uni-d
XPUs per Node·82,40024100
Scale-Up World Size·819,200··

Switches

2
SW
Scale-Up Network LayerPart / Model
Units#
Unit BWGbit/s uni-d
Lane Count#
Lane BWGbit/s uni-d
per SwitchNvidia NVSwitch 2.0 (NVLink 3)62,40024100
Scale-Up Switch Total··14,400··
Notes
  • XPUs per Node — 600GB Bidi

Scale-Up Parts Summary

1
ItemQty/ClusterQty/ServerAttach/XPU
Nvidia NVSwitch 2.0 (NVLink 3)24,5766.000.75

Backend network

5 rows
Assigned blockDefault
A100 - P4d NVL8 - EFAv2 - 3L - ETH - TH2 - AWS
BackendXPU: A100 P4d NVL85 linesshared by 1 SKU

Backend Units are based on the cluster size (32,768 XPUs).

Switches

5
SW
 Network Layer
Switch and Server SpecAggregate Port UtilizationNetwork Bandwidth Aggregate
SwitchPart Name / Model
Units#
Unit BWGbit/s
Port BWGbit/s
Downlink Ports / SW#
Uplink Ports / SW#
Max Ports#
Downlink Ports#
Uplink Ports#
Total Ports#
Downlink BWTbit/s
Uplink BWTbit/s
OversubRatio
XPUs·32,76850100016,38416,384016,38416,38401,638.4·
NICs per NodeEFAv2 Nitro-v3 100G NIC4100100··8······
ToRWhiteBox Tomahawk 2 6.4T5126,40010032326416,38416,38432,7681,638.41,638.41
LeafWhiteBox Tomahawk 3 12.8T25612,8002003232648,1928,19216,3841,638.41,638.41
SpineWhiteBox Tomahawk 3 12.8T12812,800200640648,19208,1921,638.40·

Transceivers

8
CN
 Network Layer
Transceiver / CablePart Name / Model
Total Units#
Units per SW/GPU#
Unit BWGbit/s
Lanes#
BW per LaneGbit/s
Agg. BWTbit/s
GPU NodesNone - AECs are Used ->32,76832,768501501,638.4
CPU NodesNone - AECs are Used ->8,192·····
Leaf
DownlinkGeneric 200G DR4 QSFP568,192322004501,638.4
UplinkGeneric 200G DR4 QSFP568,192#ERR200#ERR500
Spine
DownlinkGeneric 200G DR4 QSFP568,192#ERR200#ERR500
Uplink·······
Core
Downlink·······
Uplink·······
Super-Core
ToR
DownlinkNone - AECs are Used ->16,384161004251,638.4
UplinkGeneric 200G DR4 QSFP5616,384161004251,638.4

Cables

8
CB
 Network Layer
Cable/ FiberType
Total Units#
Units per Transceiver#
Unit BWGbit/s
Lane Count#
SerDes BWGbit/s
GPU NodesGeneric 200G Y Cable AEC16,3840.5100250
CPU Nodes······
Leaf
DownlinkAlready counted above·····
UplinkGeneric 200G Single Mode8,1921200450
Spine
Downlink······
Uplink······
Core
Downlink······
Uplink······
Super-Core
ToR
DownlinkAlready counted above·····
UplinkGeneric 200G Single Mode8,1920.5200450

Backend Parts Summary

5
ItemQty/ClusterQty/ServerAttach/XPU
Generic 200G Y Cable AEC16,3844.000.50
Generic 200G DR4 QSFP5640,96010.001.25
Generic 200G Single Mode16,3844.000.50
WhiteBox Tomahawk 2 6.4T5120.130.02
WhiteBox Tomahawk 3 12.8T3840.090.01

Frontend network

No converged frontend + storage fabric data for this cluster.

Frontend Parts Summary

0
ItemQty/ClusterQty/ServerAttach/XPU

Compute Tray Networking Content

2 rows

Networking components inside the compute tray — accelerators, board-to-board connectors, retimers, and their per-tray bandwidth.

Assigned blockDefault
A100 - P4d NVL8 - AWS
Compute TrayXPU: A100 P4d NVL82 linesshared by 1 SKU
Tray layerPart / ModelUnits per Tray
Unit BWGb/s
Lanes
Lane BWGb/s
Per switch
XPUs per TrayBlackwell 2008····
Compute TrayBroadcom PEX89144 Gen 5.0 144L4····
GPU to CPU TrayGeneric PCIe Gen5 x16 2-meter DAC8····

Compute Tray Parts Summary

2
ItemQty/ClusterQty/ServerAttach/XPU
Broadcom PEX89144 Gen 5.0 144L16,3844.000.50
Generic PCIe Gen5 x16 2-meter DAC32,7688.001.00