Generative Adversarial Networks or GANs is a deep learning method that allows generative modeling. In a GAN, two neural networks contest with each other in the form of a zero-sum game, where one agent's gain equals another agent's loss. These two neural networks are called generator and discriminator.

• The generator learns to generate plausible data. The generated instances become negative training examples for the discriminator.
• The discriminator learns to distinguish the generator's fake data from real data. The discriminator penalizes the generator for producing implausible results.

Figure 1.1: Initial generated images are easily distinguishable by the discriminator.

Figure 1.2: After many iterations generator will be able to fool the discriminator.

Since the generator and discriminator are trained in an adversarial manner, these models becomes as good as possible at their respective tasks. The adversarial training process forces the generator to create increasingly realistic data. This is because the discriminator is constantly getting better at distinguishing between real and fake data, so the generator has to work harder to fool it.

The generator output is connected directly to the discriminator input. Through backpropagation, a backward pass to adjust a neural network model's parameters, the discriminator's classification provides a signal that the generator uses to update its weights.

Figure 2: Whole architecture of the GAN.

The generator component of a GAN continuously refines its ability to produce synthetic data by incorporating feedback from the discriminator. It strives to produce outputs that can deceive the discriminator into classifying them as genuine. Training the generator necessitates a more intimate connection between the generator and the discriminator than training the discriminator does. The generator training component of a GAN encompasses:

• Random input
• Generator network, which transforms the random input into a data instance
• Discriminator network, which classifies the generated data
• Discriminator output
• Generator loss, which penalizes the generator for failing to fool the discriminator

Figure 3: GAN: Backpropagation in Generator Training.

Generator is trained using following procedures:

• Using sample random noise.
• Producing generator output from sampled random noise.
• Getting discriminator "Real" or "Fake" classification for generator output.
• Calculating loss from discriminator classification.
• Backpropagating through both the discriminator and generator to obtain gradients.
• Using gradients to change only the generator weight.

The discriminator's training data comes from two sources:

• Real data instances, such as real pictures of people. The discriminator uses these instances as positive examples during training.
• Fake data instances created by the generator. The discriminator uses these instances as negative examples during training.

Figure 4: GAN: Backpropagation in Discriminator Training.

During discriminator training:

• The discriminator classifies both real data and fake data from the generator.
• The discriminator loss penalizes the discriminator for misclassifying a real instance as fake or a fake instance as real.
• The discriminator updates its weights through backpropagation from the discriminator loss through the discriminator network.

Since GAN contains two separately trained neural networks, its training has to be done in alternating periods.

• The discriminator trains for one or more epochs.
• The generator trains for one or more epochs.

During discriminator training, the generator remains unchanged. The discriminator must learn to identify the generator's flaws as it attempts to distinguish between genuine and artificial data. This presents a different challenge for a well-trained generator compared to an untrained generator that produces random output. Similarly, the discriminator remains unchanged during generator training. Otherwise, the generator would be pursuing a moving target and might never converge.

As the generator improves with training, the discriminator's performance deteriorates because it becomes more difficult to distinguish between real and fake data. If the generator achieves perfect accuracy, the discriminator's accuracy drops to 50%. This progression presents a convergence issue for the GAN as a whole: The discriminator's feedback loses significance over time. If GAN training continues beyond the point where the discriminator provides entirely random feedback, the generator begins to train on useless feedback, and its own quality may deteriorate.

Figure 5: Train Loss graph for Generator and Discriminator.

However, if the discriminator becomes too good, the generator will not improve its gradient which will lead to vanishing gradient. In other words, this will hinder the generator learning and producing high-quality outputs.

Solutions for these bottlenecks GAN’s convergence problem can be solved using methods like,

• Adding noise to discriminator inputs
• Penalizing discriminator weights

To solve the vanishing gradient in GANs,

• Use Wasserstein loss: This loss is designed to prevent vanishing gradients even when the rain the discriminator to optimality.
• Modified minimax loss: Modification to minimax loss will be used to deal with vanishing gradients.

GANs have become really popular these days due to its ability to generate a variety of impressive results, including:

• Photorealistic images: GANs can be used to generate images that are indistinguishable from real photographs.

Figure 6: Realistic images generated using GANs.

• Realistic videos: GANs can also be used to generate realistic videos.

• New music: GANs can even be used to generate new music.

  GANSynth: Making music with GANs

In addition to these creative applications, GANs are also being used in a variety of research applications, such as:

• Drug discovery: GANs can be used to generate new molecules that may have desirable properties.

Figure 7: Using GANs to discover novel and diverse compounds.

• Medical imaging: GANs can be used to improve the quality of medical images.

There are a number of open-source GAN implementations available such as:

• Tensor Flow GAN
• PyTorch GAN

GANs are a rapidly developing field, and there are a number of exciting research directions being pursued. Some of these directions include:

• Improved training stability: GANs can be difficult to train, and often suffer from instability.
• New GAN architectures: New GAN architectures are being developed that may improve the performance of GANs.
• Applications of GANs to new domains: GANs are being applied to a variety of new domains, such as natural language processing and robotics.

1. https://developers.google.com/machine-learning/gan/gan_structure (figures 1.1, 1.2, 2, 3, 4)
2. https://en.wikipedia.org/wiki/Generative_adversarial_network
3. https://machinelearningmastery.com/what-are-generative-adversarial-networks-gans/
4. https://neptune.ai/blog/gan-failure-modes (figure 5)
5. https://medium.com/dc-gan/how-to-generate-realistic-pictures-with-deep-convolutional-gans-328beb40c14 (figure 6)
6. https://wires.onlinelibrary.wiley.com/doi/full/10.1002/wcms.1608 (figure 7)