The switch line card is the beating heart of modern data center fabrics. A single line card based on Broadcom's Tomahawk 5 (BCM78900) switches 25.6 Tbps, with 12.8 Tbps allocated to front-panel ports and 12.8 Tbps to the fabric interface. Routing 512 lanes of 112 Gbps PAM4 on a PCB that fits into a 2RU chassis slot is one of the most demanding PCB design challenges in the industry.

Switch ASIC: The Center of the Universe

With 512 SerDes lanes radiating from a 60 mm BGA, the routing density is staggering. Each lane is a differential pair — that's 1,024 individual traces just for the SerDes, ignoring power, ground, management, and clock signals.

Layer Stackup: 28-36 Layers

A Tomahawk 5 line card typically uses 30–34 layers:

• L1: Top — ASIC, front-panel connectors (QSFP-DD/OSFP), VRM, clocking

• L2–L3: GND planes — continuous reference for top-side routing

• L4–L9: SerDes routing quadrant 1 & 2 (256 lanes to front-panel ports)

• L10–L11: PWR planes — Vcore (0.75V, 500A), Vserdes (0.9V, 100A)

• L12–L17: SerDes routing quadrant 3 & 4 (256 lanes to fabric interface)

• L18–L19: GND + additional PWR

• L20–L23: Management plane — CPU complex (x86 or ARM), BMC, timing

• L24–L25: DDR5 memory routing (for packet buffer)

• L26–L29: Additional SerDes breakout / interposer routing

• L30–L31: GND planes

• L32: Bottom — decoupling capacitors, DC-DC converters

112G PAM4 Routing: 512 Lanes

The sheer volume of SerDes lanes forces aggressive routing density:

• Trace width/spacing: 3.0 mil trace, 3.5 mil space for 100Ω differential pairs — roughly 10 mil per pair, 5 pairs per inch of routing channel width

• Routing channels: The ASIC is divided into 4 quadrants, each routing 128 lanes (64 differential pairs) to a specific region of the board. This quadrant-based floorplanning is essential to avoid routing congestion

• Layer assignment: Each routing layer carries 16–32 pairs organized in parallel buses. Orthogonal routing between adjacent signal layers prevents inter-layer crosstalk

• Length matching: All lanes within a 400G port group (8 lanes) must be matched to within 10 ps (~1.5 mm in Megtron 7) to meet PAM4 skew requirements

• Via count: Each differential pair traverses 4–8 vias (ASIC BGA break-out → routing layer → connector break-out). At 512 pairs × 6 vias/pair, that's >3,000 high-speed vias, each backdrilled

Power Delivery: 600W at 0.75V

The Tomahawk 5's 600W TDP at 0.75V core voltage means 800A of current. Power delivery requirements:

• VRM topology: 20–24 phase multiphase buck converter, DrMOS power stages at 80A each. Phases are physically distributed around the ASIC to spread heat

• Remote sensing: Differential kelvin sense traces from center of ASIC BGA back to VRM feedback node — compensates for ~30 mV IR drop across the PCB

• Decoupling: Thousands of MLCCs: 10 μF × 7R (0805) for bulk; 0.1 μF × 7R (0402) for mid-frequency; 0.01 μF × 7R (0201) for high-frequency. Total mounted capacitance ~5,000 μF

• Plane thickness: 4 oz copper on Vcore planes. At 800A, current density is manageable with 3–4 parallel planes

Front-Panel Connector Density

32 ports of 400G QSFP-DD on a single line card requires 32 cage connectors in a 2RU × 17" form factor. Each QSFP-DD connector is ~18 mm wide — 32 × 18 mm = 576 mm, but a 2RU card is only ~430 mm wide. This forces:

• Belly-to-belly mounting: Connectors on both sides of the PCB, doubling density. The PCB becomes a mezzanine between two rows of cages

• Stacked cages: 2×1 or 2×2 stacked QSFP-DD cages, providing 2 ports in a single 18 mm width

• Thermal: 32 × 12W per QSFP-DD optical module = 384W of front-panel heat. Dedicated airflow channels and cage-integrated heatsinks are mandatory

Clocking and Synchronization

The 512 SerDes lanes require precise clock distribution with <300 fs RMS jitter:

• Reference clock: 156.25 MHz or 312.5 MHz low-jitter oscillator (e.g., SiT9365, <100 fs RMS jitter)

• Clock distribution: Differential LVDS/HCSL clock tree with matched-length traces (±1 ps) to each SerDes quad. Clock routing on inner layers with ground plane isolation

• SyncE / PTP: For timing-sensitive applications, the line card supports Synchronous Ethernet and IEEE 1588 PTP with hardware timestamping — additional clock routing and FPGA logic