Published on Tue Mar 23 2021

The Hammer and the Nut: Is Bilevel Optimization Really Needed to Poison Linear Classifiers?

Antonio Emanuele Cinà, Sebastiano Vascon, Ambra Demontis, Battista Biggio, Fabio Roli, Marcello Pelillo

Data poisoning is a particularly worrisome subset of poisoning attacks where the attacker aims to cause a Denial-of-Service (DoS) attack. We propose a counter-intuitive but efficient heuristic.

1
4
3
Abstract

One of the most concerning threats for modern AI systems is data poisoning, where the attacker injects maliciously crafted training data to corrupt the system's behavior at test time. Availability poisoning is a particularly worrisome subset of poisoning attacks where the attacker aims to cause a Denial-of-Service (DoS) attack. However, the state-of-the-art algorithms are computationally expensive because they try to solve a complex bi-level optimization problem (the "hammer"). We observed that in particular conditions, namely, where the target model is linear (the "nut"), the usage of computationally costly procedures can be avoided. We propose a counter-intuitive but efficient heuristic that allows contaminating the training set such that the target system's performance is highly compromised. We further suggest a re-parameterization trick to decrease the number of variables to be optimized. Finally, we demonstrate that, under the considered settings, our framework achieves comparable, or even better, performances in terms of the attacker's objective while being significantly more computationally efficient.

Fri Feb 28 2020
Machine Learning
Regularisation Can Mitigate Poisoning Attacks: A Novel Analysis Based on Multiobjective Bilevel Optimisation
Machine Learning (ML) algorithms are vulnerable to poisoning attacks. A fraction of the training data is manipulated to deliberately degrade the algorithms' performance. Optimal poisoning attacks can be formulated as multi-objective optimisation problems.
0
0
0
Sat Sep 08 2018
Machine Learning
Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning Attacks
The ability of an attack to be effective against a different, potentially unknown, model is not yet well understood. We present a framework for understanding the transferability of such attacks.
1
1
1
Tue Jun 30 2020
Machine Learning
Model-Targeted Poisoning Attacks with Provable Convergence
In a poisoning attack, an adversary with control over a small fraction of the training data attempts to select that data in a way that induces a corrupted model that misbehaves in favor of the adversary. We propose an efficient poisoning attack designed to induce a specified model. Unlike previous model-
1
5
6
Mon Mar 19 2018
Machine Learning
Technical Report: When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attacks
Recent results suggest that attacks against supervised machine learning systems are quite effective. The existing attacks make diverse, potentially unrealistic assumptions about the strength of the adversary who launches them. We propose the FAIL attacker model, which describes the adversary's knowledge and control.
0
0
0
Sun May 23 2021
Machine Learning
Regularization Can Help Mitigate Poisoning Attacks... with the Right Hyperparameters
Machine learning algorithms are vulnerable to poisoning attacks. A small fraction of the training data is manipulated to degrade the algorithms' performance. We show that current approaches lead to an overly pessimistic view of algorithms' robustness and the impact of regularization.
1
0
0
Fri Jun 09 2017
Machine Learning
Certified Defenses for Data Poisoning Attacks
Machine learning systems trained on user-provided data are susceptible to data poisoning attacks. malicious users inject false training data with the aim of corrupting the learned model. While recent work has proposed a number of attacks and defenses, little is understood about the worst-case loss.
0
0
0
Sat Dec 20 2014
Machine Learning
Explaining and Harnessing Adversarial Examples
Machine learning models consistently misclassify adversarial examples. We argue that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature. This explanation is supported by new quantitative results.
8
4
25
Sat Dec 21 2013
Neural Networks
Intriguing properties of neural networks
Deep neural networks are highly expressive models that have recently achieved state of the art performance on speech and visual recognition tasks. While expressiveness is the reason they succeed, it also causes them to learn uninterpretable solutions.
2
4
16
Tue Mar 14 2017
Machine Learning
Understanding Black-box Predictions via Influence Functions
Influence functions are a classic technique from robust statistics. They can be used to trace a model's prediction through the learning algorithm and back to its training data. They are useful for understanding model behavior, debugging models, detecting dataset errors.
3
0
4
Thu Nov 10 2016
Machine Learning
Understanding deep learning requires rethinking generalization
Conventional wisdom attributes small generalization error to properties of the model family or to the regularization techniques used during training. We show how these traditional approaches fail to explain why large neural networks generalize well in practice.
4
1
2
Tue Oct 24 2017
Machine Learning
One pixel attack for fooling deep neural networks
Recent research has revealed that the output of Deep Neural Networks (DNN) can be easily altered by adding relatively small perturbations to the input vector. In this paper, we analyze an attack in an extremely limited scenario where only one pixel can be modified.
2
0
1
Tue Nov 08 2016
Machine Learning
Delving into Transferable Adversarial Examples and Black-box Attacks
An intriguing property of deep neural networks is the existence of transferable adversarial examples. These examples may severely hinder deep neural network-based applications. This work is the first to conduct an extensive study of the transferability over large models.
0
0
0