Variational Autoencoders - Valentyn1997/xray GitHub Wiki

Variational Autoencoder

Variational Autoencoder (VAE) is a convolutional autoencoder with distributional assumptions on data. Similar to a classic (vanilla) autoencoder, Variational Autoencoder's architecture includes encoder and decoder layers with bottleneck layers between them. The difference hides in the mentioned bottleneck layers. When image passes encoder layers, we get mean and variance vectors associated with Gaussian distribution, which as a result form a so-called hidden representation. Then, the sampled output is passed to the decoder network to do the reconstruction job. The figure below demonstrates the described architecture.

[figure here]

Loss

The loss function used in Variational Autoencoder is a sum of two components: binary cross-entropy loss between original and reconstructed images and Kulback-Leiber divergence between the current distribution of the image and Normal distribution.

[loss formula here]

Choice of binary cross-entropy loss was motivated by the similar works in the frame of reconstructing images. However, tried out MSE and L1-loss did not make significant (noticeable) changes in the ROC-AUC score.

Architecture

In order to solve the task different network sizes for VAE were tried out. As a result of experiments, the following network was selected as the most successful one (according to ROC-AUC metric and reconstruction quality):

[network architecture here]

References

Disentangled version, which might bring improvement to vanilla VAE

Disentangled Variational Autoencoder

Disentangled Variational Auto-Encoder for Semi-supervised Learning

Nice Youtube video on VAE

• Variational Autoencoder based Anomaly Detection using Reconstruction Probability
  • "We propose an anomaly detection method using the reconstruction probability from the variational autoencoder. The reconstruction probability is a probabilistic measure that takes into account the variability of the distribution of variables. The reconstruction probability has a theoretical background making it a more principled and objective anomaly score than the reconstruction error, which is used by autoencoder and principal components based anomaly detection methods"

VAE for medical images

• Unsupervised pathology detection in medical images using conditional variational autoencoders

  • Pathology detection in medical image data
  • "Our method is based on learning the entire variability of healthy data and detect pathologies by their differences to the learned norm. For this purpose, we use conditional variational autoencoders which learn the reconstruction and encoding distribution of healthy images and also have the ability to integrate certain prior knowledge about the data (condition)."
  • "different 2D and 3D datasets show that the approach is suitable for the detection of pathologies and deliver reasonable Dice coefficients and AUCs. Also this method can estimate missing correspondences in pathological images and thus can be used as a pre-step to a registration method."
  • "variational autoencoder (VAE) [5] is a form of autoencoders that assumes a prior distribution of the latent space (typically normal distribution) motivated by the logical distribution of the input data on some particular domain"
  • "VAEs can be conditioned (CVAE) [18], by adding an extra prior semantic information about the data, e.g., a label. This way the z-space distribution is learned per condition and more control over the learned space and recon- struction of images is possible."
  • Comparison to "**AnoGAN **is a GAN-based method, also following the idea of learning the representation of healthy images. Similarly to the approach in this work, the authors combine a z-space distance and an image reconstruction dis- tance of the images in test phase, defining an anomaly score to differentiate between pathologies and healthy tissue."

• Unsupervised Detection of Lesions in Brain MRI using constrained adversarial auto-encoders

  • "In this work, we approached the challenge of lesion detection in an unsupervised-learning manner by learning prior knowledge from healthy data and detect abnormalities according to the learned healthy data distribution. We investigated the detection performance abnormality based methods, namely VAE and AAE using brain MRI images. We then analyzed the behavior of these models and proposed a latent constraint to ensure latent consistency and enable more accurate detection of abnormal regions. We showed that the abnormal lesions can be detected with the investigated models and the accuracy of detection can be improved with our proposed latent constraint. A natural competitive to the models we presented is the AnoGAN model."

• Different architectures mostly for brain images

  • Unsupervised Lesion Detection in Brain CT using Bayesian Convolutional Autoencoders uses baysian convolutional autoencoder
  • Deep Generative Models in the Real-World: An Open Challenge from Medical Imaging " Results indicate that convolutional VAE, Bayesian VAE and AAE have great potential to be further studied and developed to gain higher detection accuracy." for abnormality detection

Constrain

• Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images
  • "Previous approaches towards deep unsupervised anomaly detection model patches of normal anatomy with variants of Autoen- coders or GANs, and detect anomalies either as outliers in the learned feature space or from large reconstruction errors. In contrast to these patch-based approaches, we show that deep spatial autoencoding models can be efficiently used to capture normal anatomical variability of entire 2D brain MR images. "
  • "Our experiments show that AE & VAE models with dense bottlenecks cannot reconstruct anomalies, but at the same time lack the capability to reconstruct important fine details in brain MR images such as brain convolutions."