AI's Bottleneck Is Migrating: From Silicon Scarcity to Memory, Efficiency, Capital, and Novel Compute

Read together, these six stories sketch a clear pattern: the AI infrastructure conversation is moving from 'how many accelerators can we get' to 'where does the cost and friction actually sit.' That shift shows up in four directions at once.

The first is storage and state. The TensorDex paper addresses a cost problem that rarely makes headlines: model hubs filled with fine-tuned LLM variants that share most of their weights. Its tensor-level, lossless delta compression, reporting 70.5% storage savings on a sample of 2,890 public Hugging Face models, treats redundancy as the enemy. The PriorityFEC work points the same way for inference. Moving KV-cache state between edge agents under time-to-first-token deadlines and lossy links is a systems problem in which the cache, not the GPU, becomes the thing to protect and ship. Both papers assume that data movement and persistence now dominate the economics.

The second is software control of existing silicon. ANEForge exposes Apple's Neural Engine through Python, bypassing CoreML, and reports roughly 90 microsecond small-program calls and a 0.33 ms ResNet-18 forward pass. Whatever its ultimate reach, it reflects a demand for lower-level access to accelerators that vendors have abstracted away. The GNN-guided layout decomposition paper, applying learned models to multiple patterning with directed self-assembly, shows the same instinct in semiconductor manufacturing: use learned methods to squeeze better results from constrained physical processes.

The third is the speculative frontier. The Shkreli story is a useful counterweight. Claims that new photonic chips could outrun Nvidia came via a podcast, not benchmarks, and the named startups have no public financials. The substantive question is whether photonic matrix-multiply units can handle the full workload, including the nonlinear, memory-heavy operations that dominate transformer inference. Until that is demonstrated, photonics should be tracked as a research direction rather than a near-term substitute.

The fourth, and arguably the most consequential, is capital. Broadcom's CEO argued that financing, not chips, may be the binding constraint, and that chip stocks sliding on that view could feed back into funding costs. Multi-year data center buildouts depend on investor sentiment as much as on engineering. If capital tightens faster than demand grows, the winners may be those who extract the most useful work per watt and per stored byte, which is exactly what the compression, caching, and low-level programming papers pursue.

Why this matters now is that the easy phase of scaling, buying more GPUs and renting space, is becoming expensive in every dimension at once. Key tensions include whether efficiency gains will be absorbed by ever-larger models (a Jevons-style rebound) and whether novel hardware can escape the software ecosystem lock-in that made incumbents dominant.

Signals to watch: adoption of delta or deduplicated storage in major model hubs; whether low-level accelerator access becomes a supported vendor feature rather than a workaround; independent benchmarks for photonic or other non-GPU matmul hardware on full transformer workloads; hyperscaler capex guidance and debt financing terms for data centers; and whether edge multi-agent deployments produce real latency gains outside lab conditions. The bottleneck is migrating, and the teams that track where it lands next will have the advantage.