Normalizing flow - ZYL-Harry/Machine_Learning_study GitHub Wiki
Generative model
Generative model is a kind of model used in self-supervised learning, it is mainly based on the idea of "generate x based on given feature z with a network G", which is shown in the following picture.
In theory, the object is to maximize P(x) as
For flow-based models, they directly optimize the objective function.
Introduction
Given data X and latent feature Z, an invertible mapping f: X→Z, which is formulated as
- The change of variable theorem:
Detailed proof: https://0809zheng.github.io/2022/04/30/variable.html
- Parameterize likelihood as flow:
To maximize the likelihood p(x) as shown above
First, take the log of both sides and get
This setting allows us to directly train the model via maximum likelihood as we can compute the log marginal likelihood exactly.
As the latent Z is a normalized Gaussian distribution, this method is called "Normalizing flow".