Why models do not work on pixels
A 1024 by 1024 RGB image is about three million numbers. Running dozens of denoising steps on that directly is possible but wasteful, because most of those numbers are redundant: neighbouring pixels agree, and fine texture carries little of the meaning.
So a latent diffusion model puts an autoencoder in front. A VAE encoder squeezes the image down by a factor of eight per side into a latent tensor, roughly 16k numbers instead of 3M. The model does all its work there, and a decoder expands the result back into pixels at the very end. Video models do the same in three dimensions, compressing time as well, which is why a video latent is a small stack rather than a sequence of independent frames.
Two things worth knowing about that trade: the encode and decode steps are lossy (this is where slightly waxy skin and softened fine text come from, before the sampler is even involved), and a latent has real spatial layout. A latent is a coarse map of the image, not a scrambled hash of it, which is why masks and control inputs can be applied in latent space at all.
What a seed really is
The sampler starts from a field of random noise. The seed is the key that makes that field reproducible. Fix it and you fix the starting point, so with the same prompt and model version you get the same output every time.
This also explains the most common misunderstanding in the entire toolset. People treat seeds as a dial to be nudged. They are not ordered: consecutive seeds produce unrelated noise fields, so seed 4001 tells you nothing about seed 4000. If you want a small variation on an image you liked, the moves that work are:
- keep the seed and edit a small part of the prompt
- keep the seed and prompt and lower the guidance slightly
- feed the image back in at a low denoising strength
- blend the latent with a second one, if the tool exposes latents directly
The knobs, translated into movement
Every parameter in a diffusion interface is a statement about a path through latent space.
| Parameter | What it does to the path |
|---|---|
| Seed | Chooses the starting point |
| Steps | How many increments the path is split into |
| Guidance (CFG) | How hard the prompt pulls the path off the unconditioned route |
| Denoising strength | How far from your input image the path begins |
| Scheduler | The shape of the step sizes along the path |
Read that way, some familiar failures become obvious. Very high guidance produces burnt, oversaturated results because you are yanking the path outside the region where decoded images look natural. Too few steps leave the path short of a settled point, so output looks soft or half formed. A denoising strength of 0.9 on an image to image run ignores your input because the path starts almost at pure noise.
Using this on real work
For variations, move short distances. Low strength image to image (0.2 to 0.35) is a nudge inside the neighbourhood. It is the reliable way to get ten cousins of a frame you already like.
For transitions, expect a morph. Interpolating between two latent vectors gives smooth in-betweens, which is great for abstract or textural transitions and bad for two different characters, where you get an unwanted blend. If the two ends are semantically far apart, cut instead.
Concepts live in directions, not places. Prompt terms tend to shift latents along consistent directions, which is why the same style phrase applied to different subjects has a recognisable effect. It is also why two strong style words fight: they pull the same path in different directions and the winner is arbitrary.
Aspect ratio is not neutral. Latent dimensions have to be multiples of the compression factor, and the model was trained on a limited set of shapes. Unusual ratios push the latent into territory it saw less of, which is a frequent cause of duplicated subjects and drifting composition.