Compute Bottlenecks Are Shifting From Chips to Data Movement, Memory, and Control of the Stack

Read together, these six stories share a common thread: the AI hardware conversation is moving from who has the fastest accelerator to where the real bottlenecks sit. The most concrete items are systems papers. PriorityFEC addresses how KV-cache state gets handed between edge agents under tight time-to-first-token deadlines and lossy links, treating the network as a first-class constraint on multi-agent inference. TensorDex attacks a different cost curve: model hubs filled with fine-tuned variants that share most of their weights, where tensor-level delta compression reports 70.5% storage savings on a sample of 2,890 public Hugging Face models. Both reflect a maturing field where memory footprint, transfer latency, and storage economics increasingly determine whether an AI system is viable, not just peak FLOPS.

The Apple Neural Engine work, ANEForge, fits the same pattern from the software side. By bypassing CoreML to program the NPU directly, it signals that practitioners want lower-level control over accelerators that vendor frameworks abstract away. Its reported 0.33ms ResNet-18 forward pass is a useful reference point, though framework-versus-direct comparisons need scrutiny on workload and measurement conditions. The GNN-guided layout decomposition paper, applying learning to DSA and multiple patterning in semiconductor manufacturing, shows the same trend reaching upstream into fabrication, where graph-structured optimization is replacing heuristic methods.

The market-structure stories carry the tensions. Nvidia reportedly turning away neoclouds from GTC Berlin for adding AMD and TPU hardware suggests that ecosystem access is being used as leverage. For neoclouds, multi-vendor strategies are increasingly a hedge against both supply constraints and pricing power, and the report tests how far a dominant vendor will tolerate that diversification. The Shkreli photonics claim is the counterweight: a podcast assertion about three private startups with no public benchmarks or financials. Its useful content is the underlying question, whether photonic matrix-multiply units can handle full workloads including nonlinearities, memory access, and training, not the hype around it.

What to watch: published benchmarks from photonic and alternative accelerator startups under workload-level scrutiny; whether neocloud vendor-diversification drives changes in Nvidia's partner programs; adoption of delta-compression or deadline-aware transport in production model hubs and inference stacks; and whether direct-access programming frameworks for NPUs gain traction beyond researchers. The common signal is that the next performance gains will likely come from the system around the chip, including memory, networking, storage, and the software and business relationships that control access to them.