MemGhost: One Email Can Poison Your AI's Memory Forever

Memory poisoning happens when an AI agent with email access reads a message containing hidden instructions and saves them as a 'learned fact' in its persistent memory. Unlike one-off phishing, that instruction survives every future conversation and needs no further action from the attacker: a single email is enough to corrupt the agent for good.

Memory poisoning is a variant of prompt injection that, instead of manipulating a single response, permanently manipulates what the agent 'knows'. An agent with persistent memory — whether an assistant connected to Outlook or Gmail, a CRM copilot, or a support bot that remembers past conversations — stores facts in a database or vector store it queries on every future turn. If that agent processes external content (an email, an attachment, a webpage) and that content contains an instruction disguised as legitimate information, the agent can end up saving it as if it were a real fact learned from the user. Security researcher Johann Rehberger documented a concrete case of this in 2024: ChatGPT's memory feature could be manipulated through externally shared content to implant false memories that persisted across sessions, a finding he dubbed 'SpAIware'. The OWASP GenAI Security Project, in its 2025 report on threats to agentic AI systems, explicitly lists memory poisoning as one of the priority risk categories for any agent capable of persisting information between sessions. This isn't a lab hypothesis: it's an attack category already recognized by the AI offensive security community.

The Trick Isn't the Email — It's the 'Remember This' Tool

Most teams deploying AI agents with email access rightly worry about filtering what the agent can do outward: who it can email, whether it can trigger a payment, whether it can modify a CRM record. Those are the actions that get audited, logged, and often require human confirmation. But the internal 'save to memory' tool almost never gets the same scrutiny, because it's perceived as harmless: the agent is just noting something down for itself, not acting on the outside world. That's exactly the blind spot memory poisoning exploits. When the content that triggers that memory write comes from an unverified external email, the 'remember' tool becomes a write channel effectively controlled by whoever wrote that email. The agent doesn't distinguish between 'my user said this' and 'an unknown sender said this in an email I'm summarizing', unless someone explicitly designed that distinction in.

• The attacker sends an email with a hidden instruction disguised as a routine note (white text, an HTML comment, or simply a sentence that reads like administrative information).
• The agent processes the email to summarize it, classify it, or extract tasks, treating that content as legitimate input data rather than a possible command.
• The agent itself calls its internal memory tool and persists the instruction as if it were a normally learned fact.
• In future conversations, that memory is retrieved automatically, without the user ever seeing the original email again or knowing that 'rule' exists.
• The agent acts on the poisoned instruction silently and repeatedly, every time the context triggers it.

Classic phishing depends on the attacker repeating the trick every time: if the user doesn't fall for that specific email, the attempt fails and leaves no trace beyond that inbox. Memory poisoning breaks that logic because it turns a single success into a permanent compromise: one email the agent processes and saves is enough for the instruction to survive conversation resets, session changes, and, in poorly designed systems, even memory migrations or exports between agent versions. There's also a harder-to-trace variant when memory is implemented as a semantic vector store (RAG): the poisoned instruction doesn't need to literally match a future conversation to resurface — it just needs to be semantically similar to whatever topic is being discussed at that moment. That means a poisoned fact inserted while, say, handling an invoice, can reappear weeks later in a completely unrelated conversation about that same vendor, with no one ever connecting it back to the original email.

The Blind Spot in Most Deployments

More and more Spanish tech and services SMEs are connecting email copilots, CRM assistants, or support bots with persistent memory to save time on repetitive management and customer service tasks. INCIBE has for years published guidance for companies on the risks of adopting AI without a minimum security layer, and memory poisoning fits exactly the kind of threat those guides try to anticipate: it doesn't require breaching any system, only that the agent process one email. The problem isn't having an agent with memory — that memory is precisely what makes it useful, able to remember a client's preferences or a vendor's history without being told every time. The problem is never having explicitly decided what can be written to that memory, and from which sources. Most IT teams we audit at Blurtek have thought about filtering the agent's outbound actions, but very few have ever reviewed the actual contents of its persistent memory.

It's worth being honest about the real scope of this risk: not every AI agent has persistent memory, and many one-off chatbots or assistants are completely stateless between sessions, so this specific threat doesn't apply to them. You also don't need to hire an external audit to harden a simple agent: separating the instruction channel from the data channel in the system prompt, and requiring explicit confirmation before writing to memory from external content, is an afternoon's work for a competent internal dev team. Where specialized help genuinely makes sense is when the agent already touches sensitive actions — payments, sending email on the company's behalf, modifying CRM records — and no one has yet designed what happens when that agent's memory quietly turns into a remote-control channel.

• Treat any email, web, or document content as data, never as instructions, in the agent's system prompt.
• Require human confirmation before writing to memory from unverified external content.
• Limit what kind of 'facts' memory can store: formatting preferences, yes; financial data or permissions, no.
• Log every memory write with its source — email, sender, date — so it can be audited and reverted.
• Periodically review the actual contents of the agent's memory, the same way you'd review the forwarding rules of a mailbox you suspect is compromised.
• Isolate the memory of agents with access to sensitive actions from the memory of purely conversational agents.