Manipulating AI memory for personal gain: AI Recommendation Poisoning

Microsoft security researchers have identified a rising wave of attacks that manipulate AI memory to influence recommendations for promotional gain. A tactic known as AI Recommendation Poisoning.

What is AI Recommendation Poisoning?

That “Summarize with AI/ChatGPT” button you just clicked might have done more than generate a quick overview.

It may have quietly altered the long-term behaviour of your AI assistant and potentially shaping what it recommends to you from that point forward.

The implications are significant. Technically anyone who relies on AI assistants for research and decision-making can be affected.

This is important because compromised AI can provide biased recommendations on critical topics. And user would not know that this happened.

Attacks that target our trust quietly and persistently create a new frontier in both cybersecurity and information integrity.

The numbers are interesting:

• 50+ unique poisoning prompts discovered
• 31 companies caught doing this
• 14 industries involved, including health, finance, and security
• Freely available tooling makes this a copy-paste attack

One of the companies doing this was a cybersecurity vendor, just insane! Below is taken from blog post:

The next AI recommendation you receive might not be a recommendation at all, but a manipulated response.

So what can we do as users?

1. ‘Check before you click’ or event avoid clicking those buttons. Hover over every “Summarize with AI” button to view URL.
2. Check your AI assistant’s memory settings. Delete anything you don’t remember adding.
3. Question any AI recommendation that seem suspicious. Ask your AI assistant to explain why it’s recommending it.
4. Copied prompts from untrusted sources might contain hidden memory instructions. Do not do that.

👉 Full research and highly recommended: arxiv.org/html/2510.01171v3