Every GPU running inference today is fighting the same war it has always fought: moving data. Not computing — moving. The arithmetic itself is almost incidental. The real bottleneck is the constant, energy-hungry shuttle between processing cores and memory, a gap so fundamental to conventional chip design that engineers call it the Von Neumann bottleneck, and for decades it was simply accepted as the price of doing business. Neuromorphic computing is now making a serious bid to make that bottleneck obsolete.
The core insight is almost insultingly simple in retrospect. The brain does not separate memory from computation. A synapse stores a weight and applies it simultaneously, in the same physical location, using almost no energy. Silicon chips have spent seventy years doing the opposite: compute here, store there, shuttle constantly between the two. Neuromorphic architectures collapse that distinction. Memory elements — often resistive RAM, phase-change memory, or ferroelectric devices — sit directly at the point of computation. The operation and the storage happen in the same device, at the same moment.
Intel’s Hala Point system, the largest neuromorphic deployment to date, packs over 1,100 Loihi 2 chips into a single rack and sustains roughly 20 quadrillion synaptic operations per second at a fraction of the energy a comparable GPU cluster would consume for certain sparse, event-driven workloads. That last qualifier matters enormously — neuromorphic chips are not general-purpose replacements for GPUs yet. But the workloads where they excel, sparse inference, always-on sensory processing, edge pattern recognition, are exactly the workloads that need to be cheap and fast if embodied AI is going to run anywhere beyond a server room.
What makes the current moment genuinely exciting is that analog in-memory computing is maturing alongside the neuromorphic architectures that can exploit it. Analog crossbar arrays perform matrix-vector multiplication — the foundational operation of nearly every neural network — by exploiting Ohm’s law and Kirchhoff’s current law directly in hardware. The numbers add up physically, in parallel, across an entire array of memory cells simultaneously. IBM’s research groups have demonstrated analog AI chips achieving inference at energy efficiencies that digital approaches cannot approach for the same operation, because the computation quite literally costs the energy of moving current through a wire rather than encoding numbers, shipping them, decoding them, and operating on them in sequence.
The challenge the field has been attacking is precision. Analog devices drift. They have noise floors. The weights stored in phase-change or resistive RAM cells shift over time, which is catastrophic if you need the model to stay calibrated. Recent work on iterative write-verify schemes, drift compensation algorithms, and hybrid analog-digital architectures has pushed the practical accuracy of analog inference on quantized models to within striking distance of their digital equivalents, close enough that the energy savings start justifying serious engineering investment rather than remaining a laboratory curiosity.
Mythic, BrainChip, and a growing cluster of well-funded startups are betting their companies on exactly this crossover point arriving soon. So, notably, are larger players — Samsung’s research labs have published extensively on ferroelectric FET-based in-memory computing, and TSMC’s process roadmaps are beginning to include memory-integrated compute layers that would have seemed exotic five years ago.
The longer arc here is profound. If memory and computation can truly merge at the device level, the entire stack above it — how models are trained, how weights are quantized, how inference is scheduled — gets redesigned around different physical constraints. We have been optimizing for a bottleneck that does not have to exist. Neuromorphic and analog architectures are not merely faster chips; they are a different theory of what a chip is for. And that theory is starting to win in silicon.