NIPS

[nips 17.6612] Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks [PDF] [notes]

Jian Zhao, Lin Xiong, Karlekar Jayashree, Jianshu Li, Fang Zhao, Zhecan Wang

read 04/25/2017

Objective

Balance face recognition datasets with synthesized extreme poses. Reduce the domain gap using a generative adverserial network

Synthesis

DA-GAN : dual agent generative adverserial network is introduced to both discriminate fake vs real and recognize identity

• Detect facial landmarks form picture of person
• Use a morphable 3d face simulator to generate synthetic faces with mapped texture
• Render faces in arbitrary poses
• Use a Fully Convolutional Network (U-net likenetwork) to operate on pixel level.
• Add discriminator which is an auto-encoder (from HxWxC to HxWxC with downpooling to bottleneck and uppooling to original size using deconvolutional layers)

Loss

• "Pose perception loss" is a simple pixel-wise l1 loss to enforce pose consistency between the generated and synthetic image
• "Identity loss" which encourages features of bottleneck of auto-encoder discriminator to be close accross same identity

Quotes

"synthetic data is often not realistic enough with artifacts and severe texture losses. The low-quality synthesis face images would mislead the learned face recognition model to overfit to fake information only presented in synthetic images and fail to generalize well on real faces"