The transformers behind ChatGPT and Claude are built for sequences like text. The AI that generates images works on a completely different and rather beautiful principle: diffusion. If the rest of this series explained how machines handle language, this one explains how they handle pictures.
Diffusion: two directions
The core idea: learn to undo noise
A diffusion model is trained in two directions. In the forward process, it takes a real image and adds a little random noise, again and again, until the image is pure static. That part is easy and needs no learning. The clever part is the reverse process: the model learns to predict and remove the noise at each step, so it can run the whole thing backwards.
Generating from nothing
Once trained, the model can start from a fresh field of random noise and denoise it step by step into a brand-new image that never existed. Each denoising step nudges the pixels toward something more coherent — like sculpting a figure out of static by removing everything that does not belong.
Where the prompt comes in
Your text prompt guides the denoising. The words are turned into a representation (often using embeddings, the subject of our embeddings guide) that steers each step toward an image matching your description. Change the prompt and you steer the same noise toward a different picture.
Latent diffusion: why it runs on your laptop
Denoising full-resolution pixels directly is slow. Latent diffusion — popularised by Stable Diffusion — does the work in a compressed “latent” space and only expands to full pixels at the end. This cut the cost so dramatically that high-quality image generation now runs on ordinary hardware.
Beyond images
The same denoising principle now drives AI video, audio and 3D generation, and researchers increasingly combine diffusion with transformers for even better control. It is a reminder that “generative AI” is not one technology but several — and understanding the difference is exactly the kind of fluency our AI training is built to give teams.
Diffusion vs autoregressive: two ways to generate
The split is worth remembering. Text is generated autoregressively — one token after another, each depending on the last (see inference). Images are generated by diffusion — the whole canvas is refined in parallel over several denoising passes. That is why an LLM “types” its answer while an image model reveals the whole picture at once, sharpening with each step.
From 1,000 steps to a handful
Early diffusion models needed around 1,000 denoising steps to make one image — slow and costly. Two advances changed that: latent diffusion (denoising in a compressed space, popularised by Stable Diffusion) and step distillation, which trains models to reach a good image in as few as 1–10 steps. Together they turned image generation from a data-centre task into something that runs in seconds on a laptop.