Researcher finds LLM agents lose track of facts in long vulnerability sessions
A security researcher's essay describes how LLM agents for vulnerability research gradually lose track of established facts during multi-hour sessions, leading to repeated suggestions of ruled-out approaches or reasoning from invalid observations. The researcher notes that telling an LLM something is wrong does not necessarily stop it from relying on dependent beliefs, prompting exploration of memory systems to reduce such hallucinations.
Coverage timeline
Hacker Newsmatt_d
Over the past few months I have been playing around quite a bit with LLM agents, particularly for vulnerability research. They are becoming surprisingly good at navigating large codebases, explaining unfamiliar subsystems and helping explore potential attack surfaces. However, once an investigation starts taking a few hours, I kept running into the same problem: the model would slowly lose track of what we had actually established. It might suggest an approach that we had already ruled out, forget that an assumption turned out to be false, or confidently continue reasoning from an observation that was no longer valid. Obviously, telling an LLM that something is wrong does not necessarily mean that it will stop believing all of the things that depended on it :) I initially started looking into memory systems because I wanted to make LLMs more useful for complex vulnerability research and reduce this type of hallucination. There are of course already plenty of solutions for giving LLMs m