A Keras implementation of a custom Deep Convolutional Generative Adversarial Networks (DCGAN) on MNIST dataset. The code is based on the Francois Chollet’s Deep Learning with Python, page 308.
DCGAN is an extension of the GAN architecture, where the generator and discriminator are deep convolutional networks. The discriminator that takes an image as input and outputs a single scalar value representing the probability that the input image is real (as opposed to fake). The generator takes a random vector as input and decodes it into a synthetic image, and is trained to generate images that the discriminator identifies as real.
The code is available on my GitHub.
What worked
- Using dropout in the discriminator
- Adding noise to the labels
- Using strided convolutions
- Normalizing the data to \([-1, 1]\)
- RMSprop optimizer
What didn’t work
- Using high learning rates (>1e-3)
- I initially normalized the data between 0 and 1, but since I was using “tanh” as the last layer of the generator’s output, it made the convergence a lot harder
- Constructing separate batches for real and fake images
- Training the discriminator with more steps than the generator
- Using SGD optimizer
Results
This is the first 5000 iterations of the training history. The random vectors are fixed at the beginning of the training. As the training progresses, the generator learns to generate more realistic images.