The State of Context Engineering

The Job is Knowing When to Be Deterministic & When to Be Probabilistic This week, I gave a short talk at AGNTCon titled, “Context Fails & Fixes,” that functioned as a sort of “State of Context Engineering.” How did context engineering emerge, where has it gone, and where might it go in the future? As model makers target new capabilities, they inadvertently create challenges we, as users and harness builders, must overcome. Below are the slides and a lightly edited transcript. As always, shoot me a note with any feedback, thoughts, or questions. I really appreciate it. Hi, everyone. Thanks for joining me today. We’re going to walk through context fails and fixes. I have a book coming out in a bit, The Context Engineering Handbook. When I first talked to the team at O’Reilly, back in June 2025, they asked me a very reasonable question: “How do we know this won’t become obsolete in a few weeks?” The trouble with AI textbooks is that many age extraordinarily quickly. I said, I don’t know, but I’m optimistic. Giving clear instructions about what to do and what not to do, and ensuring those instructions are consistent and not noisy, will forever be a challenge with AI, no matter how good it gets. We have coworkers, partners, and other people we know and interact with who are sentient. They pass the Turing test, they are “AGI,” and yet: we still have a hard time conveying instructions, ideas, and concepts among each other. Context engineering will continue to be hard. But it will be hard in unique ways, because models aren’t people. You can make assumptions about how models think or interpret things (use whatever anthropomorphizing word you like), but it helps to understand how they’re built, optimized, and trained. Their unique attributes and construction create a unique set of failure modes we need to reckon with. So let’s review how we got here. Let’s flash back to the end of 2024. If you remember, this was the height of the “RAG is dead” moment. The hope was: if we can just make the context window large enough, we can put the entire internet in it. We can put everything in it. We don’t have to be selective. We don’t have to build agentic systems. We can fit it all in there, and AI will be solved. Model builders started aiming at long context capabilities and we had a big leap from ~128,000-token context window models to million-token context window models, starting with Gemini 1.5. If you look at the marketing material for Gemini 1.5 Pro, it was mostly about the million-token context window. They were even teasing a 10-million-token context window model they had in the lab, that would come out soon. But they got beaten to the punch. Llama 4 Scout came out with a 10-million-token context window, and well, it wasn’t quite what we hoped. Inference was a challenge, and Scout never really caught on, and we never got another Llama. But we still got a million tokens. That’s pretty good. We’ll take it. So this was the brief at the time: just put everything in the context and let the model sort it out. But with more context, we had room for more problems. Long contexts are not a panacea. When I wrote about this, I identified four key failure modes. Context poisoning is when something gets into the context that shouldn’t be there. It could be a hallucination or a bad fact, and the model can’t get past it. Context distraction is when the context grows so long that the model starts to focus on the context over its base knowledge. We can see this in some really interesting ways. Context confusion is when extraneous facts—things that don’t relate to the problem I’m working on—come in and influence the model in ways we don’t fully understand. This one’s really pernicious. We’re not going to dig into it today, but it’s one of my favorite failure modes. The model makes assumptions about who you are, and those assumptions influence the answer. Finally, context clash is when tool calls, skills, and other information within your harness start to conflict and work against you in ways you didn’t expect. (How many of you keep memory on in ChatGPT or Claude? I turn it off. I hate it. Sometimes I’m talking about work conversations; sometimes I’m talking about a project I’m doing with my son. The model will take notes on both, mix up those contexts, and the confusion grows from there.) Let me give you my favorite example of context poisoning. I’m going to date myself: I played the game on the left. The one from 1996, for the original Game Boy. This is the first Pokémon Red. It was in black and white. In 2004, Nintendo released Pokémon FireRed for the Game Boy Advance. These were essentially the same game, except one had better colors and audio. But there was one subtle change. At some point in both Pokémon Reds, to get past a security guard, you have to give them something to drink. A bribe. Quite the practical lesson for children. In the original game, you need to give the thirsty guard lemonade. In FireRed, for some reason, t

Excerpt shown under the publisher's RSS terms · 200 words · full text and figures are on the publisher's site