Agent memory the human can still audit: primary sources, epistemic status markers, and user sign-off

Over the last months, many different tools for giving memory to agents have been released, such as Karpathy's popular LLM wiki, or the many tools built on Obsidian. I have been developing one more such tool, but with a focus on AI safety, alignment and stability. This research was funded by Foresight Institute. In the course of this research I have independently arrived at many of the same ideas also used by other projects. However, I have also developed a few features that none of the tools I am aware of are using. I want to share these features with others: Primary Sources to prevent Memory Drift: As agents edit the wiki or the memory, they keep making small changes to the content that accumulate over time. To prevent this, you need a primary source that is never altered and can be compared against. Many existing projects technically have this, through git history, but this is not very agent-friendly. It is a lot of effort for agents to review git, and so this work is often skipped. My system keeps track of per-topic chronological LOG files that are never altered and can be used to prevent drift and find details. Unlike Karpathy's log.md file, which is just a single list of operations, these files keep track of each conversation between user and agent in detail. Epistemic Status Markers to prevent Hallucinations: When agents write their wiki entries, they don't have an automatic, intuitive way to differentiate between "it's plausibly X" and "we discussed it and decided on X". My system uses epistemic status markers ([decided], [verified], [concluded], [assumed], [superseded]) to prevent this. Markers like [superseded], when combined with the LOG files that keep track of human input, allow agents to remember very compactly (1) what the current status is, (2) if there were any alternative or preceding options, and (3) which discussions and arguments led us to our conclusions. Explicit Signoff by the User to prevent Runaway Reasoning: Ratifying an idea as [decided] can only ever be done by the user, never the agent. Agents are instructed to ask for explicit signoffs for important decisions instead of approving them silently. This prevents the failure mode where long running agents start bypassing the user because they silently assumed that a suggestion they made was accepted, when the user just didn't notice. All three of these are important for safety as well as productivity: Agents that lose track of their goals over time because their content drifts are dangerous. The system I have developed prevents this: summaries can always be compared to primary sources and agents do not mistakenly assume an idea has been ratified by a user. Current trends in memory system design go towards increased automation. As agents become faster and smarter, reviewing their output manually becomes increasingly intractable. The safety features I add mitigate this risk: They keep agents easier to audit and make them less likely to develop a life of their own that was never intended. My project lives here: https://github.com/FlorianDietz/EpistemicMemory. It is a set of file conventions and instructions in a GitHub template repo, not software, so no installation is required. If you like the features I just described but you are already committed to a different memory management system, it should be easy enough to ask your agent to scan my project and copy only those rules that you are missing. Discuss

Text integral sub licența deschisă a publicației · 566 cuvinte