What a flow field is
For each pixel in frame N, flow gives you the horizontal and vertical offset to where that same content sits in frame N+1. Visualized, it is usually a color image where hue means direction and saturation means speed.
Three properties are worth internalizing, because every artifact traces back to one of them:
- Flow assumes correspondence. A pixel must exist in both frames. Anything that appears, disappears, or hides behind something else has no valid vector.
- Flow is directional. Forward flow (N to N+1) and backward flow are computed separately and will disagree. Comparing them is the standard way to detect occlusions, because disagreement means one of the two is guessing.
- Flow measures, it does not understand. It does not know a limb is a limb. Two visually identical regions are interchangeable to it.
Where flow shows up in generation
- Frame interpolation. Slow motion and frame rate conversion warp both neighbors halfway along their vectors and blend.
- Video-to-video and stylization. The previous stylized frame gets warped into the current frame's geometry and used as a starting point, so the style persists instead of being re-invented every frame. This is the single most important use.
- Per-frame edits. A mask, a relight, or an inpaint applied on one frame is carried forward along the flow rather than recomputed.
- Upscaling and denoising. Detail from neighboring frames is aligned and pooled, which is why video upscaling can resolve detail a single-frame upscaler cannot invent.
How flow fails, and what it looks like on screen
- Displacement too large. Fast pans and whip motion exceed the search range. Result: vectors collapse toward zero and the output stutters instead of moving.
- Occlusion boundaries. Around edges of moving objects there is nothing to match. Result: ghost trails and transparent smears, most visible on hands, hair, and thin objects.
- Repeated texture. Grass, water, brick, and picket fences look the same everywhere, so the match is ambiguous. Result: flow wanders and the region shimmers.
- Motion blur and low light. Blur destroys the features flow needs, noise invents features that are not there.
- Cuts. Two unrelated frames produce garbage flow. Always split a timeline at cuts before running any flow-based process, since one bad boundary can poison everything downstream.
Reading a temporal consistency complaint
When someone says a clip is inconsistent, the specific symptom tells you which part of the pipeline to touch.
- Texture that boils or crawls means frames were generated independently, with no flow guidance at all.
- Ghost trails behind moving edges mean flow error at occlusions, so reduce the propagation step or interpolate from closer frames.
- A jelly-like warp that grows over the clip means flow-guided propagation is drifting, so re-anchor with a fresh keyframe every few seconds.
- A subject whose face or clothing changes identity is not a flow issue at all. Nothing in the flow field controls appearance, so the fix belongs to the conditioning, usually a reference frame.