Physical Adversarial Examples for Object Detectors

Kevin Eykholt*, Ivan Evtimov*, Earlence Fernandes*, Bo Li*, Amir Rahmati*, Florian Tramèr*, Atul Prakash, Tadayoshi Kohno and Dawn Song   (*joint first authors)

USENIX Workshop on Offensive Technologies (WOOT) 2018


Deep neural networks (DNNs) are vulnerable to adversarial examples, maliciously crafted inputs that cause DNNs to make incorrect predictions. Recent work has shown that these attacks can generalize to the physical domain, to create perturbations on physical objects that fool image classifiers under a variety of real-world conditions. These attacks pose a risk to deep learning models used in safety-critical cyber-physical systems. In this work, we extend physical attacks to object detection models, a broader class of deep learning algorithms widely used to detect and label multiple objects within a scene. Improving upon a previous physical attack on image classifiers, we create perturbed physical objects that are either ignored or mislabeled by object detection models. We implement a Disappearance Attack, in which we cause a Stop sign to "disappear" according to the detector—either by covering the sign with an adversarial Stop sign poster, or by adding adversarial stickers onto the sign. In a video recorded in a controlled lab environment, the state-of-the-art YOLO v2 detector failed to recognize these adversarial Stop signs in over 85% of the video frames. In an outdoor experiment, YOLO was fooled by the poster and sticker attacks in 72.5% and 63.5% of the video frames respectively. We also use Faster R-CNN, a different object detection model, to demonstrate the transferability of our adversarial perturbations. The created poster perturbation is able to fool Faster R-CNN in 85.9% of the video frames in a controlled lab environment, and 40.2% of the video frames in an outdoor environment. Finally, we present preliminary results with a new Creation Attack, wherein innocuous physical stickers fool a model into detecting nonexistent objects.

  author   =   {Eykholt, Kevin and Evtimov, Ivan and Fernandes, Earlence and Li, Bo and Rahmati, Amir and Tram{\`e}r, Florian and Prakash, Atul and Kohno, Tadayoshi and Song, Dawn},
  title   =   {{Physical Adversarial Examples for Object Detectors}},
  booktitle   =   {USENIX Workshop on Offensive Technologies (WOOT)},
  year   =   {2018},
  howpublished   =   {arXiv preprint arXiv:1807.07769},
  url   =   {}