analiza aiqbench
This paper addresses a critical hardware-algorithm mismatch in digital Predictive Coding by introducing error-based PC (ePC), which eliminates exponential signal decay inherent in state-based formulations. The proposed reparameterization achieves training speeds orders of magnitude faster than standard methods while matching backpropagation performance on deep architectures. This work provides a viable path for scaling brain-inspired learning models on current digital hardware, bridging the gap between theoretical energy minimization and practical efficiency.
ePC: Fast and Deep Predictive Coding in Digital Simulation
arXiv:2505.20137v5 Announce Type: replace Abstract: Predictive Coding (PC) offers a brain-inspired alternative to backpropagation for neural network training, described as a physical system minimizing its internal energy. However, in practice, PC is predominantly digitally simulated, requiring excessive amounts of compute while struggling to scale to deeper architectures. This paper reformulates PC to overcome this hardware-algorithm mismatch. First, we uncover how the canonical state-based formulation of PC (sPC) is, by design, deeply inefficient in digital simulation, inevitably resulting in exponential signal decay that stalls the entire minimization process. Then, to overcome this fundamental limitation, we introduce error-based PC (ePC), a novel reparameterization of PC which does not suffer from signal decay. Though no longer biologically plausible, ePC numerically computes exact PC weights gradients and runs orders of magnitude faster than sPC. Experiments across multiple architectures and datasets demonstrate that ePC matches backpropagation's performance even for deeper models where sPC struggles. Besides practical improvements, our work provides theoretical insight into PC dynamics and establishes a foundation for scaling PC-based learning to deeper architectures on digital hardware and beyond.
Abstract:Predictive Coding (PC) offers a brain-inspired alternative to backpropagation for neural network training, described as a physical system minimizing its internal energy. However, in practice, PC is predominantly digitally simulated, requiring excessive amounts of compute while struggling to scale to deeper architectures. This paper reformulates PC to overcome this hardware-algorithm mismatch. First, we uncover how the canonical state-based formulation of PC (sPC) is, by design, deeply inefficient in digital simulation, inevitably resulting in exponential signal decay that stalls the entire minimization process. Then, to overcome this fundamental limitation, we introduce error-based PC (ePC), a novel reparameterization of PC which does not suffer from signal decay. Though no longer biologically plausible, ePC numerically computes exact PC weights gradients and runs orders of magnitude faster than sPC. Experiments across multiple architectures and datasets demonstrate that ePC matches backpropagation's performance even for deeper models where sPC struggles. Besides practical improvements, our work provides theoretical insight into PC dynamics and establishes a foundation for scaling PC-based learning to deeper architectures on digital hardware and beyond.
Accepted at ICML 2026 - Main Track. All code available at this https URL
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2505.20137 [cs.LG]
(or arXiv:2505.20137v5 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 199 cuvinte
Subiecte conexe
News
Quantum Computing Report · 25 iul. 2026
Vertical Advantage: Transforming Industries with Lakebase and Agentic AI
Databricks · 27 aug. 2026
News Archive 2026
Quantum Computing Report · 25 iul. 2026
Automatic Patient-Specific Microwave Ablation Planning Accelerated by a Physics-Guided Deep Learning Model
arXiv · 5 aug. 2026