aiqbenchInformații despre AI & calcul cuantic

...

Română · beta. Traducerea a fost făcută automat și este încă în curs de revizuire. Citește pagina în engleză

perspectiva aiqbenchMachine learning

Sakana AI researchers have introduced PC-ALM, an augmented Lagrangian method that enables layer-local learning in deep networks while recovering backpropagation-aligned gradients. This approach successfully trains 1000-layer residual MLPs with performance nearly matching standard backpropagation on small benchmarks, addressing the credit assignment problem inherent in local learning theories. The method offers a promising theoretical bridge between biologically plausible local updates and the high performance of global gradient descent for deep architectures.

Sursă care acoperăMarkTechPost

Ce este aceastaLectura unui model asupra articolului și a acoperirii conexe, scrisă pentru aiqbench. Nu sunt cuvintele publicației.

Sakana AI Researchers Introduce PC-ALM, a Layer-Local Alternative to Backpropagation That Trains 1000-Layer Networks

MarkTechPost14 septembrie 20264 min de cititArticolul și perspectiva sunt în engleză.

Sakana AI researchers Jeffrey Seely and Julian Gould introduce Augmented Lagrangian Predictive Coding (PC-ALM), a local-learning alternative to backpropagation. By attaching a Lagrange multiplier to each layer constraint, PC-ALM keeps predictive coding's layer-local updates while recovering exact backprop gradients in linear networks. It matches BP across widths and depths from 8 to 128 at an inference budget of T = 2L, lifts gradient cosine to BP from 0.604 to 0.909 in the reference cell, and trains 1000-layer residual MLPs within about 2 points of backprop on MNIST. MIT-licensed JAX code is available. The post Sakana AI Researchers Introduce PC-ALM, a Layer-Local Alternative to Backpropagation That Trains 1000-Layer Networks appeared first on MarkTechPost.

Backpropagation is a global algorithm: a forward pass, then a backward pass, then a weight update, each locked behind the previous one. Brains have no known mechanism for that kind of network-wide phase locking, which is why local-learning alternatives such as predictive coding (PC) keep drawing research interest. Sakana AI researchers propose Augmented Lagrangian Predictive Coding (PC-ALM), a variant of PC that keeps every update layer-local yet recovers backprop-aligned credit…

Extras afișat conform termenilor RSS ai publicației · 70 cuvinte · textul integral și figurile sunt pe site-ul publicației

Citește pe MarkTechPost ↗Întreabă despre acest articol