Lierature Review - liyinnbw/deepfake-detector GitHub Wiki
Overviews
- 2019 The State of Deepfakes 2019
- 2019 Deep Learning for Deepfakes Creation and Detection: A Survey great survey on current deepfake techniques and detection methods
- 2020 Media Forensics and DeepFakes: an overview survey on media forensics with a focus on deepfake
- 2020 DeepFake Detection: Current Challenges and Next Steps
Faking
- 2014 GAN NOTE: GAN will improve fake quality but is not must-have. The key architecture for generating deepfake is the Deepfake Autoencoder (DFAE) structure.
- 2016 Face2Face: Real-Time Face Capture and Reenactment of RGB Videos real-time face reenactment
- 2017 Synthesizing obama: learning lip sync from audio generate obama speaking video from audio (needs to generate 3D face of the subject, a lot of manual work in masking)
- 2017 Deepfakes/faceswap face swap, source mimic target expression, superceeded by DeepFaceLab
- 2017 On Face Segmentation, Face Swapping, and Face Perception face swap, poor result
- 2018 StyleGAN image generation as a style transfer (not limited to face)
- 2019 StyleGAN2 improved image quality
- 2019 FSGAN face swap and face reenactment
- 2019 FaceShifter face swap that handles occlusion
- 2019 First Order Motion Model for Image Animation Another type of deepfake which animates a single image using a video
- 2020 DeepFaceLab similar to Deepfakes/faceswap, better quality, handles occlusion and bad lighting. claimed to be hard to detect even with frequency domain analysis (Fig.7 in the paper).
Detecting
Single-frame Facial Landmarks Analysis
- 2018 In ictu oculi: exposing AI created fake videos by detecting eye blinking No longer useful, deepfakes can now blink.
- 2018 Exposing Deep Fakes Using Inconsistent Head Poses Inconsistent fake face landmark location when head pose changes
- 2018 Detecting GAN-generated Imagery using Color Cues Fake image has limited frequency of saturated pixels (due to GAN normalization) and different RGB weights than produced by camera.
- 2019 Exposing DeepFake Videos By Detecting Face Warping Artifacts
- 2019 Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations missing reflections and missing details in the eye and teeth
- 2020 Forensics and Analysis of Deepfake Videos detect fake at the mouth region
Single-frame Signal Analysis
WARNING: May not work for video frame because video frame is highly compressed
- 2018 Learning to Detect Fake Face Images in the Wild Contrastive loss, fake region visualization
- 2018 Detection of deep network generated images using disparities in color components artifacts in non-RGB color space
- 2018 Detection of deepfake video manipulation photo response non uniformity (PRNU) analysis
- 2020 Effective and fast deepfake detection method based on Haar wavelet transform blur inconsistencies
- 2020 Preliminary Forensics Analysis of DeepFake Images detection using fourier transformation
- Error Level Analysis A technique used to detect modification associated with inconsistent JPEG compression levels
Single-frame Direct Deep Learning
WARNING: Detector neural net can be fooled, see these paper1 paper2
- 2020 HcDFDNet - A lightweight real-time AI based Facial Forgery Detector for Compressed Videos
- 2018 MesoNet: a Compact Facial Video Forgery Detection Network
- 2015 Deep learning for steganalysis via convolutional neural networks
- 2016 A deep learning approach to universal image manipulation detection using a new convolutional layer
- 2017 Automated face swapping and its detection use old machine learning models like MLP, SVM
- 2017 Recapture Image Forensics Based on Laplacian Convolutional Neural Networks used Laplacian Convolutional Neural Network
- 2018 Mesonet: a compact facial video forgery detection network use a standard image neural net
- 2018 Capsuleforensics: Using capsule networks to detect forged images and videos used capsule network
Cross-frame Analysis
- 2018 Deepfake video detection using recurrent neural networks uses CNN+LSTM
- 2019 Recurrent convolutional strategies for face manipulation detection in videos uses recurrent convolutional network (RCN)
- 2020 Interpretable Deepfake Detection via Dynamic Prototypes
- 2020 Deepfake detection using spatiotemporal convolutional networks
- 2020 Exploiting prediction error inconsistencies through LSTM-based classifiers to detect deepfake videos
Behaviour Analysis
- 2019 Protecting World Leaders Against Deep Fakes facial expression, speaking pattern analysis, but only for that person
Statistical
WARNING: May not work for video frame because video frame is highly compressed
- 2019 Limits of Deepfake Detection: A Robust Estimation Viewpoint hypothesis testing, also points out the high resolution image is harder to fake.
Origin Tracing
- 2019 Combating deepfake videos using blockchain and smart contracts use block chain to trace video origin
Others
- Deepface Online Detector A cloud solution with API (formerly Deeptrace)
Datasets
- 2019 FaceForensics++ Google's dataset
- 2020 Deep Fake Detection Challenge (DFDC) Facebook's dataset, the largest so far
- 2020 DeeperForensics-1.0 Another recent larget dataset