Train a Model and Simulate a Data Poisoning Attack

Instructions

Step 1: Model Training

Choose a dataset:

Options: chose any opensource or available datasets of your choice example, MNIST (handwritten digit classification), CIFAR-10 (image classification) IMDB Reviews (text classification for sentiment analysis) or You may use any other publicly available dataset suitable for classification tasks.

Build and train a classification model:

Use Python with a framework of your choice (e.g., TensorFlow, PyTorch, Scikit-learn). Document the steps for preprocessing, model selection, and training. Evaluate the model and report performance metrics (e.g., accuracy, precision, recall).

Step 2: Simulate a Data Poisoning Attack

Define the attack scenario:

• Backdoor attack: Inject malicious patterns into a subset of the training data that triggers incorrect predictions during inference.
• Label flipping attack: Modify labels for a subset of training samples to degrade overall accuracy.

Implement the attack:

Add the poisoned data to the training dataset. Retrain the model on the poisoned dataset. Show the impact of the attack on model performance, such as reduced accuracy or targeted misclassification.

Step 3: Mitigation Strategy

Identify possible defenses:

Propose a basic mitigation technique (e.g., robust training, outlier detection, data validation).

Implement the mitigation:

Apply the chosen defense to the poisoned dataset. Retrain the model and show improved performance compared to the attacked model.

Step 4: Lab Documentation

Prepare a Jupyter Notebook or any other mean of delivery, a detailed lab document with:

Clear instructions for each step, including code snippets and explanations. A description of the data poisoning attack and its impact. Visualizations (e.g., confusion matrix, accuracy graphs) to illustrate results. Step-by-step guidance on implementing the mitigation.

Include discussion points:

How the attack works and its implications. How the defense strategy mitigates the risk. Open questions for interviewers to further the candidate research and implementation.

Note:

There is not right or wrong answer, we do not even have a template answer for each of the three steps requested above. We are primarily interested in your ability to research and analyze then provide or at least suggest possible approach and recommendation to the problem at Hand.