A photonic chip performing a matrix multiplication doesn’t heat up the way a GPU does. The photons carrying the signal travel through waveguides etched into silicon, interfere with each other, and produce a result — at the speed of light, consuming a fraction of the energy. That’s not a metaphor. That’s physics, and a growing number of teams are now engineering it into real hardware that runs neural network inference without the thermal wall that has defined the GPU era.
The core idea has been circulating in research for years: optical interference networks can implement the weighted sums at the heart of matrix-vector multiplication using photonic beam splitters and phase shifters rather than transistors switching current. What’s changed recently is the engineering maturity. Companies like Lightmatter and Luminous Computing have moved from proof-of-concept chips to systems designed for actual deployment. Lightmatter’s Passage interconnect fabric, which uses light to shuttle data between chips at bandwidth densities electrical interconnects struggle to match, is already shipping to select partners. The next step — doing the compute optically too, not just the communication — is where the field is racing.
Why does this matter so much right now? Because the efficiency crisis in AI hardware is real and acute. Running a frontier inference workload on dense GPU clusters is extraordinarily power-hungry. A data center housing tens of thousands of H100-class GPUs can consume hundreds of megawatts, and electrical interconnects between chips increasingly bottleneck the whole system. Photonics attacks both problems simultaneously. Optical matrix multiplications don’t dissipate heat the way CMOS arithmetic does, and optical interconnects move data with far less energy per bit than copper at equivalent bandwidth.
The technical obstacles have always been formidable. Optical components are analog, which means numerical precision is limited by physical noise rather than digital bit depth. Programming a photonic network means tuning thousands of phase shifters to represent learned weights, and those weights drift with temperature. Early systems required cryogenic stabilization or constant recalibration, which undermined the efficiency gains they were supposed to deliver. But the engineering progress on thermal stabilization, on-chip monitoring circuits, and hybrid optical-electronic designs that let digital electronics handle the parts they’re good at while optics handles the linear algebra — that progress has been substantial enough to push photonic inference from “fascinating curiosity” toward “plausible production path.”
One particularly exciting direction is edge inference. A photonic inference chip running at milliwatt power levels could deploy sophisticated models in environments where a GPU is simply impossible: implantable medical devices, satellite payloads, sensor nodes in remote industrial systems. The energy-per-operation advantage of optical matrix multiplication is most dramatic at small scale, where the overhead of a full digital compute stack is prohibitive. Researchers at MIT and Stanford have demonstrated photonic accelerators for convolutional operations small enough to fit on a chip the area of a thumbnail, achieving inference on image classification tasks at power budgets measured in microwatts.
The hybrid architecture is probably where the near-term wins land: photonic chips handling the matrix-multiply-intensive layers of a transformer or CNN, with conventional digital logic managing memory, non-linearities, and control flow. That’s not a compromise — it’s sensible engineering. It plays to the strengths of both substrates. And it means photonics doesn’t need to replace CMOS entirely to be transformative. It just needs to be good enough at one thing to reshape the economics of the whole stack.
We are still early. Production-scale photonic AI accelerators are not yet a commodity, and the precision and programming challenges are genuine. But the trajectory is steep. As photonic foundry processes mature — driven in part by the enormous investment in silicon photonics for datacom — the cost of fabricating these chips is falling. The question is shifting from whether optical AI compute works to how fast the engineering catches up with the physics. Given how compelling that physics is, the answer is probably: faster than most people expect.