SXM GPU Carrier PCB: NVIDIA NVLink Interconnect Deep Dive
The SXM (Socketed Mezzanine Module) is NVIDIA's proprietary GPU form factor for HGX platforms — a high-density mezzanine card that carries the GPU die, HBM stacks, and the NVLink interconnect fabric. Unlike PCIe add-in cards, SXM modules connect through a high-pin-count mezzanine socket directly to the HGX baseboard, enabling dramatically higher bandwidth. This article unpacks the PCB design challenges of SXM carrier boards.
SXM Generations: Evolution of Bandwidth
| Generation | GPU | NVLink BW | Lanes | TDP | Connector |
|---|---|---|---|---|---|
| SXM2 | V100 | 300 GB/s | 6 × 25 GB/s | 300W | Mezzanine, ~2,000 pins |
| SXM3 | A100 | 600 GB/s | 12 × 50 GB/s | 400W | Mezzanine, ~2,500 pins |
| SXM4 | A100 | 600 GB/s | 12 × 50 GB/s | 500W | Mezzanine, ~2,500 pins |
| SXM5 | H100 | 900 GB/s | 18 × 50 GB/s | 700W | Mezzanine, ~3,000 pins |
| SXM6 | B200 | 1,800 GB/s | 18 × 100 GB/s | 1,000W | Mezzanine, ~3,500 pins |
Each generation doubles NVLink bandwidth while increasing TDP — the SXM6 at 1,000W represents a staggering power density of 5.7 W/cm² across the module area.
SXM Connector: The Critical Interface
The SXM mezzanine connector is a proprietary NVIDIA-designed interface with approximately 3,000–3,500 pins. Key characteristics:
Signal pins: ~2,000 pins dedicated to NVLink high-speed differential pairs (18 ports × 4 lanes × 2 pairs/lane = 144 pairs per direction)
Power pins: ~1,000 pins for power delivery — each pin rated at 1.5A continuous. Total connector current capacity exceeds 1,500A for the combined voltage rails
Contact resistance: <15 mΩ per pin after mating, including connector body resistance
Mating height: ~5 mm stack height — the SXM module sits 5 mm above the baseboard
Alignment tolerance: ±0.15 mm in X/Y; the connector includes guide pins and a metal stiffener frame for precision alignment
PCB Stackup: 20-28 Layer Design
An SXM5 carrier PCB typically uses 22-26 layers:
Top layer: GPU BGA (0.4 mm pitch, ~55 mm × 55 mm), HBM3 stacks (×6), connector placement zone
Signal layers (12–16): NVLink routing to connector, HBM PHY breakout, management signals
Power planes (6–8): Vcore (0.75–0.85V at 600–800A), Vmem-HBM (1.1V at 60–80A), Vddq (1.2V at 30A)
Material: Megtron 7 for all signal layers (NVLink at 100 GB/s PAM4 demands ultra-low loss); high-Tg FR-4 for power layers only
NVLink Signal Integrity at 100 GB/s PAM4
NVLink 4 (Blackwell) operates at 100 GB/s per lane using PAM4 signaling at 50 Gbaud. The Nyquist frequency is 25 GHz. Critical SI requirements:
Insertion loss: Must stay below −25 dB at 25 GHz across the complete channel (GPU die → SXM PCB → SXM connector → baseboard PCB → NVSwitch). SXM PCB allocation is ~6 dB
Via design: All NVLink vias must be backdrilled to within 6 mil of the target layer. Via stub resonance at 25 GHz (λ/4 ≈ 2.5 mm in Megtron 7) is catastrophic if not removed
Pair-to-pair skew: <1 ps within each differential pair. Trace length matching within 3 mil
Impedance: 100Ω differential ±7%
Return loss: < −15 dB across 10 MHz–25 GHz — the connector is typically the dominant discontinuity
Power Delivery at 1,000W
The SXM6's 1,000W TDP pushes power delivery to extremes:
GPU core current: At 0.75V Vcore, 1,000W means 1,333A — though approximately 30% is HBM power, making actual Vcore current ~900A
VRM placement: Multi-phase VRM on the baseboard (not the SXM module). The SXM connector carries 0.8V core voltage directly — the PCB just distributes it to the GPU BGA
IR drop: Total path resistance from connector pins to GPU die bumps must be below 50 μΩ. At 900A, 50 μΩ = 45 mV drop — 5.6% of a 0.8V rail
Decoupling: Thousands of MLCCs on the SXM PCB in a dense grid under the GPU — reverse-geometry 0306 capacitors with <0.15 nH mounted inductance
Plane count: 6–8 copper planes at 3–4 oz weight dedicated to Vcore distribution
Thermal and Mechanical Design
SXM modules are the hottest components in the server, requiring direct liquid cooling or massive vapor chamber heatsinks:
Heatsink mounting: The heatsink bolts to the baseboard stiffener, clamping the SXM module between the heatsink and connector — not bolted to the SXM PCB itself
Thermal interface: Phase-change TIM or liquid metal (e.g., Thermal Grizzly Conductonaut) between GPU lid and heatsink cold plate, achieving <0.05 °C·cm²/W thermal resistance
PCB warpage: At 2.2 mm thickness with 0.4 mm pitch BGA, warpage must stay below 0.2% post-reflow. Asymmetric copper distribution causes severe warpage — stackup must be carefully balanced
Underfill: Capillary underfill under the GPU BGA for mechanical reliability; must be compatible with the high-temperature liquid cooling environment (85°C continuous)