The contest at the centre of it
Training a GAN sets up an adversarial game between two networks. The generator maps random noise to an image. The discriminator receives a mix of real images and generated ones and has to label each one. The discriminator is trained to catch fakes; the generator is trained to fool it.
Neither can win permanently. Every time the discriminator gets sharper, the signal it sends back tells the generator exactly which cues gave it away, so the generator improves. The equilibrium, in theory, is a generator whose output is indistinguishable from the training distribution, at which point the discriminator is reduced to guessing.
Two properties fall out of this design and explain everything about where GANs are used today:
- Inference is a single pass. No iterative denoising, no step count. Once trained, output is essentially instant, which matters for anything interactive or batch-heavy.
- Training is fragile. The two networks have to improve at a similar pace. If the discriminator gets too strong, the generator receives no useful gradient and stalls. If the generator finds a cheap trick, it stops exploring. This is why GAN training earned a reputation for being finicky in a way diffusion training is not.
Why diffusion displaced it for synthesis
The shift was not about final image quality on a narrow domain. StyleGAN faces were extraordinary. It was about three things production work cares about more:
- Coverage. Diffusion learns the whole distribution and can be prompted across an enormous range of subjects. A GAN trained on faces produces faces.
- Steerability. Because diffusion generates over many steps, you can intervene between steps: guidance scale, negative prompts, masks, control inputs, denoising strength. A single-pass architecture offers far fewer intervention points.
- Training stability at scale. Diffusion objectives train predictably on billions of images. Adversarial objectives at that scale are much harder to keep balanced.
| GAN | Diffusion | |
|---|---|---|
| Inference | One pass, milliseconds | Many steps, seconds |
| Steerability | Limited | High |
| Subject coverage | Narrow domain | Very broad |
| Training | Unstable, needs balance | Predictable |
| Still used for | Upscale, restore, enhance | Text to image, video |
Where GANs are still the right choice
If you use AI tools daily, you are already using them, probably without noticing. The pattern is the same in every case: a well-defined input-to-output mapping, a need for speed, and no requirement for open-ended prompting.
- Upscaling. Real-ESRGAN and its descendants are still the practical default for fast, cheap resolution increases.
- Face restoration. GFPGAN and similar models recover blurred or damaged faces from low-resolution sources.
- Portrait and skin enhancement. A narrow, repeatable transform where one-pass speed is the entire point.
- Latent editing. StyleGAN's well-behaved latent space still supports smooth attribute edits (age, expression, pose) with a precision diffusion does not match as cleanly.
- On-device and real-time work. Anything that has to run inside a video call or a phone camera pipeline.
The practical caution
Because these models were trained to produce plausible texture rather than to recover truth, an upscaler will confidently invent what is not there. Eyelashes, teeth, fabric weave, distant faces, small lettering: all get filled with something convincing. Two habits help. Compare against the source at 100 percent before you accept a result, and on faces prefer a lower upscale factor with two passes over one aggressive jump, since the invented detail compounds with each factor of enlargement.