Co-Packaged Optics: Replacing Copper Traces in AI Datacenters

The Electrical Copper Wall in Megawatt AI Clusters

As distributed foundation model training clusters scale to hundreds of thousands of accelerators, network interconnect throughput has become the primary physical barrier to linear scaling. In modern supercomputing fabrics, networking switches must handle aggregate switching bandwidths of 51.2 Terabits per second (Tbps) today, with next-generation platforms targeting 102.4 Tbps.

Historically, networking racks transferred high-speed signals across printed circuit board (PCB) traces using standard copper conductors. However, at serialization-deserialization (SerDes) lane rates of 112 Gbps PAM4 and 224 Gbps PAM4, the laws of classical electromagnetism impose severe penalties: 1. High-Frequency Attenuation (Skin Effect): At 28 to 56 GHz Nyquist frequencies, alternating electrical current concentrates on the outermost microscopic skin of copper traces, causing resistive losses to escalate exponentially. 2. Dielectric Insertion Loss: Signal energy bleeds into PCB dielectric substrates (such as Megtron-7 or Tachyon 100G) as heat. Over trace lengths of just 30 to 40 centimeters from the switch ASIC to front-panel optical transceivers, electrical signal loss routinely exceeds 25 to 30 decibels (dB). 3. The Transceiver Power Trap: To overcome this massive signal degradation, retimer chips and power-hungry DSPs are packed into pluggable optical transceivers (QSFP-DD and OSFP modules). An 800G optical transceiver consumes between 25 and 30 watts. In a 51.2 Tbps switch chassis containing 64 transceiver cages, optical transceivers alone consume over 1,800 watts—accounting for more than 35% of total switch electrical power purely on moving bits from the chip to the faceplate.

To bypass this electrical copper wall, the semiconductor industry is executing a fundamental architectural migration: Co-Packaged Optics (CPO) (*Optical Internetworking Forum (OIF) Co-Packaged Optics Implementation Agreement*, 2024).

┌────────────────────────────────────────────────────────────────────────┐ │ Conventional Pluggable Switch vs Co-Packaged Optics (CPO) Architecture │ ├────────────────────────────────────────────────────────────────────────┤ │ │ │ Conventional Pluggable Architecture: │ │ ┌──────────────────────────────────────────────────────────────┐ │ │ │ Monolithic Switch ASIC (51.2 Tbps) │ │ │ └──────────────────────────────┬───────────────────────────────┘ │ │ │ Lossy PCB Copper Traces (30-40 cm) │ │ │ (25-30 dB Insertion Loss & Retimers)│ │ ▼ │ │ ┌──────────────────────────────────────────────────────────────┐ │ │ │ Faceplate Pluggable Transceivers (64x OSFP @ 25W each) │ │ │ └──────────────────────────────┬───────────────────────────────┘ │ │ ▼ Optical Fibers to Adjacent Racks │ ├────────────────────────────────────────────────────────────────────────┤ │ Co-Packaged Optics (CPO) Architecture: │ │ ┌──────────────────────────────────────────────────────────────┐ │ │ │ Common Advanced Substrate / Interposer │ │ │ │ ┌───────────────────────┐ ┌─────────────────────────┐ │ │ │ │ │ Switch Core ASIC │◄──►│ 8x Optical Engines │ │ │ │ │ │ (51.2 Tbps Logic) │ │ (Integrated Silicon │ │ │ │ │ └───────────────────────┘ │ Photonics PIC + EIC) │ │ │ │ │ (Millimeter-scale XSR └────────────┬────────────┘ │ │ │ │ copper die-to-die traces: < 3 dB loss)│ │ │ │ └─────────────────────────────────────────────┼────────────────┘ │ │ ▼ Direct Fiber Ribbons │ │ (Faceplate MPO Connections) │ └────────────────────────────────────────────────────────────────────────┘

---

Silicon Photonics and Optical Engine Integration

Co-Packaged Optics eliminates long PCB copper traces by co-mounting the optics directly on the same multi-chip substrate or organic interposer as the switch logic ASIC: - Photonic Integrated Circuits (PIC): Fabricated using specialized semiconductor processes that etch waveguides, optical modulators (Mach-Zehnder or micro-ring resonators), and germanium photodetectors directly into silicon. - Electronic Integrated Circuits (EIC): Companion silicon containing low-power driver amplifiers and transimpedance amplifiers (TIA) bonded to the PIC via micro-bumps. - Extra-Short Reach (XSR / USR) Interconnect: By placing the optical engine within millimeters of the switch silicon, high-loss PCB traces are replaced with ultra-short reach die-to-die interconnects. Signal loss drops from 30 dB down to less than 3 dB. - Energy Metric Transformation: Traditional pluggable optics consume 15 to 20 picojoules per bit (pJ/bit). Co-Packaged Optics reduces this energy expenditure to less than 5 pJ/bit—and down to 2 to 3 pJ/bit in advanced prototypes—slashing switch interconnect energy consumption by up to 70%.

---

Vendor Roadmaps vs Actual Commercial Deployment

While the physics of silicon photonics are compelling, commercial adoption has been governed by manufacturing yields, testability, and hyperscaler risk tolerance:

1. Broadcom: - In 2023 and 2024, Broadcom led commercial CPO deployment with the Tomahawk 4-CPO (25.6 Tbps) and Tomahawk 5-CPO (Bailly, 51.2 Tbps) platforms. - The Bailly system integrates eight 6.4 Tbps optical engines co-packaged alongside the central 5nm Tomahawk 5 switch die, coupled with optical fiber pig-tails routed to front-panel multi-fiber push-on (MPO) optical connectors. 2. Cisco: - Cisco has invested billions into silicon photonics acquisitions (Luxtera and Acacia), integrating optics into its Silicon One architecture to target hyperscale cloud providers deploying massive distributed AI fabrics. 3. TSMC: - TSMC introduced its COUPE (Compact Universal Photonic Engine) platform, utilizing 3D Chip-on-Wafer (CoW) packaging to stack EICs directly atop PICs, targeting low-resistance copper-to-copper micro-pad bonding for volume production in 2025 and 2026. 4. NVIDIA's Deliberate Divergence in Blackwell NVL72: - Notably, for the Blackwell NVL72 rack-scale architecture (analyzed in NVLink 5 vs PCIe Gen 6 Interconnect), NVIDIA deliberately chose passive copper over optical interconnects for intra-rack communication. - By engineering custom internal copper cartridge backplanes containing over 5,000 NVLink copper wires spanning under 2 meters, NVIDIA avoided optical transceivers entirely within the rack, saving an estimated 20 kilowatts of optical power per 120 kW rack. Optics are reserved exclusively for spine-level multi-rack scaling across InfiniBand and Ethernet.

---

The Laser Reliability Dilemma: Internal vs External Laser Sources (ELS)

The primary engineering obstacle that delayed CPO adoption was laser diode physics: - Semiconductor lasers—specifically Distributed Feedback (DFB) lasers fabricated from Indium Phosphide (InP)—are extremely sensitive to ambient thermal stress. - According to the Arrhenius reliability equation, the Mean Time Between Failures (MTBF) of a laser diode roughly halves for every $10^circ ext{C}$ increase in operating junction temperature. - A 51.2 Tbps switch ASIC runs at operating temperatures between $85^circ ext{C}$ and $105^circ ext{C}$. If laser diodes were placed directly inside the co-packaged multi-chip module, a single failed microscopic laser diode would render an entire $$50,000+$ CPO switch package unusable.

┌────────────────────────────────────────────────────────┐ │ External Laser Small Form-Factor Pluggable (ELSFP) │ ├────────────────────────────────────────────────────────┤ │ Front Panel (Chilled Air Intake @ 25°C - 35°C): │ │ [ Hot-Swappable ELSFP Laser Module ] │ │ │ │ │ ▼ Optical Polarization-Maintaining Fibers (PMF)│ │ Hot ASIC Substrate (Center of Chassis @ 95°C): │ │ [ Co-Packaged Optical Engine (Silicon Modulators) ] │ │ (Zero on-die active lasers; completely heat immune) │ └────────────────────────────────────────────────────────┘

To solve this, the industry standardized on External Laser Small Form-Factor Pluggable (ELSFP) modules: - High-power continuous-wave (CW) lasers are housed in hot-swappable cartridges mounted on the chilled front panel faceplate. - Unmodulated light is piped into the switch interior through polarization-maintaining optical fibers. - The optical engine on the hot ASIC substrate merely modulates the incoming light using silicon Mach-Zehnder or micro-ring modulators. If a laser fails, a datacenter technician simply swaps the front-panel cartridge in seconds without taking the switch offline.

Co-Packaged Optics represents the inevitable thermodynamic future of megawatt AI computing on RunPod GPU Cloud and Together AI Inference, steadily replacing copper backplanes as lane speeds cross into 224 Gbps.