FlowGMM - ZYL-Harry/Machine_Learning_study GitHub Wiki

• Definition: an end-to-end approach to generative semi-supervised learning with normalizing flows, using a latent Gaussian mixture model
• Paper: Semi-Supervised Learning with Normalizing Flows

Normalizing flow

Given data X and latent feature Z, an invertible mapping f: X→Z, which is formulated as

• The change of variable theorem:
  

  Detailed proof: https://0809zheng.github.io/2022/04/30/variable.html

• Parameterize likelihood as flow:
  To maximize the likelihood p(x) as shown above
  
  First, take the log of both sides and get
  
  This setting allows us to directly train the model via maximum likelihood as we can compute the log marginal likelihood exactly.
  As the latent Z is a normalized Gaussian distribution, this method is called "Normalizing flow". By the way, in FlowGMM, the amount of flow K is 1.

FlowGMM

Maximum object

As the latent Z is a Gaussian distribution as

The likelihood for labeled data and unlabeled data are calculated respectively:

• For labeled data,
  
• For unlabeled data,
  

Then, the object is to maximize:

Code

• Official code: https://github.com/izmailovpavel/flowgmm