Presentation: Adaptive Recommenders in the Real World: Inference, Evals, and System Design

Mallika Rao: I'm very excited to be talking about one of my very close to heart topics, adaptive recommenders. When people hear the phrase, recommendation systems, typically they're thinking about models, maybe the stages that go into it, like ranking, retrieval systems, embeddings, your stores, offline, metrics, and all of that good stuff. Maybe recently, perhaps large language models. After spending many years in the area of search and discovery, personalization, recommendation systems, candidate generation systems, what I have realized and come to be convinced of is slightly different. The hardest part about building recommendation systems and engines isn't building just a model. It's so much about building the system and everything that goes end to end in building a system like that that can continuously learn, adapt, evolve, and actually deliver that value, while operating under real-world constraints like latency, cost, observability, experimentation, customer trust, compliance, and the hard constraints that really make great products. That's what I want to talk to you all about today. I want to start with a simple observation. Like many of the conversations that we are having around AI today and most of the conversations that have been happening over the last year,…