What the model sees
An inpainting pass gets three inputs: the image, a mask marking the region to replace, and a prompt describing what should be there. The model generates new content for the masked region while conditioning on the surrounding pixels, so it can match lighting, texture, and perspective at the boundary.
That conditioning is why inpainting behaves so differently from a whole-frame edit. The model is not deciding what the picture is about, it is solving a much smaller problem: what would plausibly occupy this hole, given everything around it. Small problems have better answers.
The mask is most of the work
Nearly every bad inpainting result is a mask problem wearing a prompt problem's clothes.
- Mask larger than the object. A tight outline tells the model the object's exact silhouette, which it will helpfully preserve by inventing something of the same shape.
- Include the evidence. Shadows, reflections, and contact points are what convince the model the object existed. Remove a lamp but leave its cast light and something lamp-shaped comes back.
- Feather the edge. A hard mask edge creates a visible seam. A few pixels of falloff lets the blend land inside the transition.
- One object per pass. Two unrelated regions in one mask makes the model try to relate them.
- Do not mask across a strong structural line such as a horizon, table edge, or architectural corner, unless you include enough of it on both sides for the model to continue it correctly.
Choosing the right task
| Goal | Task |
|---|---|
| Remove or replace something inside the frame | Inpainting |
| Extend the frame outward | Outpainting |
| Change the look of the whole picture | Image to image |
| Cut a subject out for compositing | Matting and rotoscoping |
| Make a small image larger | Upscaling |
Reach for the narrowest task that solves the problem. Running a whole-frame pass to fix one hand means re-rolling every part of the picture you already approved.
Prompting an inpaint
Describe what should be in the region, not what you want gone. The model has no concept of removal; a removal is just an inpaint whose prompt describes background.
- ✅
empty wet asphalt continuing the road markings, same overcast light - ⚠️
remove the carsometimes works, but it leaves the model guessing what replaces it
Two more habits. Keep the prompt short, because a long prompt inside a small mask encourages the model to cram a whole scene into a patch. And when a fill is nearly right, rerun with a different seed before rewriting the prompt: variance inside a small mask is usually cheaper than reasoning about wording.
Video inpainting
Video inpainting extends the same idea across time, and it is the standard method for wire removal, rig removal, sign replacement, and taking a modern object out of a period shot. Two things change. The mask has to travel with the object, which means tracking rather than drawing, and the fill has to stay consistent frame to frame, which means the model needs to reference neighboring frames rather than solve each one alone.
It works best when the region behind the object is revealed elsewhere in the shot, because then the fill is reconstruction rather than invention. A locked-off camera with a moving subject is the easy case. A static object in front of a static background, never revealed, is the hard one, and it is where the patch tends to breathe.