Benchtalks #3: We taught AI everything except how to learn

For our third Benchtalks, the series dedicated to the researchers building the measurement toolkits that frontier labs hill-climb on, Snorkel AI co-founder Vincent Sunn Chen sat down with Parth Asawa, a PhD student at UC Berkeley advised by Matei Zaharia and Joey Gonzalez. Parth leads research on continual learning and is the creator of Continual Learning Bench, developed in collaboration with Snorkel AI — the first standardized benchmark for measuring whether AI systems actually learn from experience over time. Coding agents re-read your entire codebase every session. Every single time. Any time you start a fresh session, a coding agent ls-es through your codebase and cats all the files just to understand what it looks like. Then you start a new session and it does the same thing all over again. After one or two tasks, a human builds a rough mental model of the dependencies and code structure and stops re-exploring. As Asawa puts it: “That capability just doesn’t exist in coding agents today.” The field got so good at scaling, it forgot the original question. Pre-training on the internet gave models a massive jump in capabilities, so the field quietly put the learning question aside. Continual Learning Bench…