AI Generation Tasks Explained

A depth map is a grayscale image where each pixel encodes distance from the camera instead of color, conventionally with near surfaces bright and far ones dark. Generation pipelines use a depth map as a structural control signal, because it carries the layout and volume of a scene without carrying its style, so you can change how an image looks while keeping where everything is.

depth estimation · monocular depth · depth pass · z-depth

Frame interpolation synthesizes new frames between existing ones, either to raise a clip's frame rate or to stretch it into slow motion. Modern methods estimate how every pixel moves between two frames and render the in-between along that motion, rather than cross-fading the two images.

video frame interpolation · keyframe interpolation · interpolation · frame blending · fps boost

Image segmentation divides a picture into labeled regions at the pixel level, producing a mask rather than a bounding box. Inside a generation pipeline it is the step that decides which pixels an edit is allowed to touch, which puts it underneath background removal, object removal, masked inpainting, and any per-subject adjustment.

semantic segmentation · instance segmentation · masking · cutout

Image to image, usually shortened to i2i, is the generation task where an input image conditions the output. The model rewrites an existing picture instead of starting from noise alone, so composition, color, and pose carry over to whatever degree the strength setting allows.

img2img · image to image translation · what is image to image · reference image

Image to video, usually shortened to i2v, is the generation task where a still image becomes the starting frame of a clip and the model only has to invent motion. Appearance is locked by the picture you supply, which removes the largest source of randomness in video generation.

img2vid · image2video · what is image to video · still to video

Inpainting regenerates the pixels inside a mask you draw and leaves everything outside it untouched. It is how you remove an object, swap a detail, or repair a mistake without re-rendering the whole picture, and the quality of the result depends far more on the mask than on the prompt.

what is inpainting · ai inpainting · video inpainting · generative fill · masked editing

A keyframe is a frame you fix in advance so the model has to pass through it. In AI video the two that matter are the first frame and the last frame, and supplying either one converts an open-ended generation into a constrained one. It is the single biggest lever on consistency.

first frame · last frame · start frame · end frame · first-last frame

Lip sync is the task of driving a face in a photo or video so its mouth matches a supplied audio track. The model repaints the mouth, jaw, and usually the whole lower face frame by frame so the visemes line up with the sound, while the rest of the shot is left as it was.

lipsync · lip syncing · audio driven face · mouth sync

Motion transfer copies the movement of a subject in a driving video onto a different character, so a still image or a new identity performs the same action. The pipeline extracts a pose track from the driver, retargets it to the target's proportions, then generates frames conditioned on that track plus your reference identity, which is why it is also called pose transfer.

pose transfer · motion reference · character animation · driving video

Optical flow is a per-pixel map of how content moved between two consecutive frames, stored as a 2D displacement vector for every pixel. It is the measurement that interpolation, video-to-video, stabilization, and video upscaling pipelines rely on to know what should stay the same across frames, which makes it the machinery behind temporal consistency.

flow field · motion estimation · temporal consistency · flow map

Outpainting extends an image beyond its original borders, generating new pixels outside the frame so they continue the existing scene. It is the same masked fill mechanism as inpainting, just pointed outward: you enlarge the canvas, mark the empty area as the region to fill, and the model paints what would plausibly have been there.

generative fill · image extension · uncrop · canvas expand

Rotoscoping is the practice of isolating a moving subject from its background frame by frame, producing an alpha matte that can be composited over something else. It started as literal tracing over live-action footage and is now mostly done by segmentation and matting models that propose the mask automatically.

alpha matte · green screen · roto · ai rotoscoping

Style transfer takes the look of one image, its palette, mark-making, texture, and light quality, and applies it to the content of another. Classical neural style transfer optimized a single output against two loss targets. Diffusion pipelines now do the same job from a style reference image or a prompt, which is faster and more flexible but less literal about copying texture.

neural style transfer · style reference · stylization · style matching

Text to image is the generation task where a model turns a written prompt into a still picture with no reference image. A text to image model samples from what it learned rather than retrieving anything, so the same words with a different seed produce a different but equally valid image.

t2i · txt2img · text2img · what is text to image

Text to video (T2V) is a generation task where a model turns a written prompt into a video clip with no reference image. The model invents the subject, the framing, and the motion at once, which makes it the least controllable and most exploratory of the video tasks.

t2v · txt2vid · text2video · text to video model

Upscaling increases the pixel dimensions of an image or video. Classic resampling enlarges what is already there, while AI upscaling (super resolution) uses a model to synthesize detail that the source never contained, which is why it can look sharper than the original and also why it can quietly change a face.

super resolution · super resolution ai · ai upscaling · image enlargement · what is upscaling

Video to video ai takes an existing clip as its main input and re-renders it, restyled, relit, or with objects changed, while the original motion and timing are preserved. The hard part is not the new look, it is holding that look identical across every frame.

vid2vid · v2v · video restyle · video style transfer

Voice cloning builds a synthetic voice from a recording of a real speaker, then reads new text in that voice. Current zero-shot systems need seconds rather than hours of audio: the model extracts a speaker embedding from your reference clip and conditions speech synthesis on it, so nothing has to be retrained for each new voice.

voice clone · speaker cloning · zero-shot tts · voice replication