CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents
Hanna Foerster, Robert Mullins, Tom Blanchard, Nicolas Papernot, Kristina Nikolić, Florian Tramèr, Ilia Shumailov, Cheng Zhang and Yiren Zhao
The surprising failure modes of machine learning systems threaten their viability in security-critical settings. In this paper, we address security vulnerabilities in AI agents that automate computer tasks. We propose "Single-Shot Planning," where a trusted planner generates a complete execution blueprint with conditional branches before encountering potentially malicious content. This approach prevents instruction injection attacks while maintaining reasonable performance levels—retaining up to 57% of frontier model performance while improving smaller models by up to 19% on the OSWorld benchmark. We also identify an additional threat called "Branch Steering attacks" that manipulates UI elements to trigger unintended plan paths.
| @misc{FMBP+26, | |||
| author | = | {Foerster, Hanna and Mullins, Robert and Blanchard, Tom and Papernot, Nicolas and Nikolić, Kristina and Tramèr, Florian and Shumailov, Ilia and Zhang, Cheng and Zhao, Yiren}, | |
| title | = | {{CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents}}, | |
| year | = | {2026}, | |
| howpublished | = | {arXiv preprint arXiv:2601.09923}, | |
| url | = | {https://arxiv.org/abs/2601.09923} | |
| } | |||