Defending Against Dormant Poisoning Attacks Across Language and Multimodal Agents

Abstract

Recent work has shown that an attacker can implant dormant malicious behaviors through poisoning that remain hidden at release but emerge later after benign downstream fine-tuning. As foundation models are increasingly used as agents, whether dormant poisoning extends to harmful agent actions and multimodal settings remains unclear. We show two risks of dormant poisoning in agentic settings: (1) poison activation can extend to tools and actions not optimized during poisoning; and (2) multiple harmful behaviors can coexist within one model and be jointly activated by fine-tuning. Existing defenses often rely on externally provided safe responses, which are hard to obtain for agent actions, or a trusted clean reference model as a safety anchor. In dormant poisoning, however, the released model appears safe before fine-tuning, so downstream users may be unaware of the poisoning and may not know which model to trust as a clean reference. We address this issue by exploiting a key property of dormant poisoning: the attacker must preserve safe behavior at release to conceal the poisoning, and this preserved behavior can itself serve as a safety reference. Leveraging this property, we automatically construct a Safety Buffer that pairs each harmful input with the released model's response and a benign version generated by the same model to remove harmful intent. The Safety Buffer can also replace external safe responses required by existing defenses. However, these defenses do not distinguish updates that weaken safety from those that preserve it, often increasing over-refusal or reducing utility. To address this limitation, we lastly introduce LookAhead Defense, which previews each candidate update using the Safety Buffer and penalizes only updates predicted to weaken safe behavior on the harmful input or make the released model's response more likely on the benign version, allowing us to preserve safety while avoiding unnecessary changes to benign behavior. Across language models, language agents, and multimodal agents, LookAhead Defense reduces harmful behavior in agents from up to 93.9% to at most 14.2% and harmful responses in language models from up to 92.5% to at most 2.0% compared with vanilla fine-tuning, while maintaining competitive utility and low over-refusal. It also remains effective against other poisoning methods.

Contributions

1. 1

   We define dormant poisoning over agent actions, showing that language and multimodal agents that appear safe at release can later execute targeted harmful actions and that multiple harmful behaviors can coexist in one agent.

2. 2

   We construct a Safety Buffer from the released model itself to avoid relying on an external model or safe responses, and show that it performs comparably to buffers built from them.

3. 3

   We introduce LookAhead Defense, which penalizes a candidate update only when it is predicted to weaken safe behavior.

Key Results

How It Works

Results

Agents. Poison activation (Act., lower is better) and utility (Util., higher is better), each averaged over two benchmarks. Bold is best.

Language models. FAB-poisoned models after GSM8K fine-tuning. Avg. is the mean of ASR and over-refusal (OR), and Util. is GSM8K accuracy. Bold is the best Avg.

• Utility stays close to vanilla fine-tuning in every setting.
• Compared with existing defenses, LookAhead Defense gives the lowest average of attack success rate and over-refusal on all three language models.
• It also remains effective against Sleeper and Benign DPO poisoning.

BibTeX

@misc{lee2026defendingdormantpoisoningattacks,
      title={Defending Against Dormant Poisoning Attacks Across Language and Multimodal Agents},
      author={Wonjun Lee and Vaidehi Patil and Zaid Khan and Yue Zhang and Hyunji Lee and Suhyun Kim and Mohit Bansal},
      year={2026},
      eprint={2610.24379},
      archivePrefix={arXiv},
      primaryClass={cs.CR},
      url={https://arxiv.org/abs/2610.24379},
}