AI Cyber lab2 - Hiresh12/UMKC GitHub Wiki

Lab Assignment -2

Introduction:

In today’s world, privacy in machine learning becomes a major concern. How we are going to ensure the security for the user’s data in real world. There are so many ways to attack a machine learning model. We are going to discuss about Membership Inference attack. The term membership inference attack determines if particular data was present in training dataset of model while black box access is provided.

Objective:-

Implementing Membership Inference attack on target model here

  1. Creating first shadow model which holds same architecture as of target model but trained with different dataset .
  2. Creating second shadow model which holds same dataset used to train target model, but architecture is different in comparison to target

Methods:-

• Creation of shadow model • Creation of attack model • Attacking target using attack model • Calculating the precision and recall

Datasets:-

• CIFAR10 from torch vision datasets • STL10 from torch vision datasets

Workflow:-

Training the target, shadow model on common dataset and common architecture. Dataset Change for Shadow Model: Downloading STL10 dataset for shadow model training Creating Data loader: Model layout: Training Target model:

Training the shadow model

Attack model due to shadow model trained on STL10 dataset Calculating the recall and precision now 2. Architecture Change for Shadow Model - Dataset CIFAR10 same as target model Changing the architecture for Shadow Model Attack the model Calculating precision and recall

Evaluation & Discussion:-

Target Shadow Attack Precision Recall Dataset CIFAR10 CIFAR10 52.08 100 Dataset CIFAR10 STL10 37.5 100 Architecture CNN(Same) CNN(Same) 52.08 100 Architecture CNN Fully connected 53.73 30.04

Conclusion:-

Membership inference attack against one deep learning model was successfully implemented. The shadow training model can be used to evaluate other scenarios , mimicking variety of models. We can also evaluate foreign models which is not trained at our place , provided dataset used. Our next venture will be trying a technique from scratch where dataset and architecture seems haywire. The data survival technique reviewed here seems resolving most of data leakage issues.