Models & Parameters

Foundation Model: What It Means for Generative Media

Also called base model, foundation models, pretrained model, base checkpoint

A foundation model is a large model pretrained on broad data and then adapted to many downstream jobs rather than built for one. In generative media it is the base checkpoint an entire ecosystem hangs off, since fine-tunes, LoRAs, control adapters, distilled fast variants and task specific versions are all built against a particular base.

Pretrain once, adapt many times

The term came out of academic work describing a shift in how models get built: instead of training a separate network per task, you pretrain one large model on broad data and adapt it. Everything downstream is adaptation.

In generative media that pattern is unusually visible, because the adaptations are all things you can name. A base image or video checkpoint gets turned into a style LoRA, a full fine-tune, a control adapter for depth or pose, an inpainting variant with an extra mask channel, a first-and-last-frame variant, a distilled four-step sibling, and a handful of product features. All of them point back at the same pretrained weights.

Which is why choosing a foundation model is a bigger commitment than choosing a model for one shot. You are choosing which ecosystem of adapters you can use, which fine-tunes you can build on, and which failure modes you will be working around for the length of the project.

The base sets the ceiling

Everything that feels like intelligence in a generative model comes from pretraining:

  • Prompt understanding. How many clauses survive, and whether spatial relationships are honoured or averaged.
  • World knowledge. Whether it knows what a Bolex looks like, how a dovetail joint fits, what a 1970s Tokyo street sign says.
  • Physical plausibility. For video, whether cloth, water, hair and weight behave, which is the hardest thing to fake and the fastest tell of a weak base.
  • Text rendering. Almost entirely a pretraining property.
  • Resolution and duration range. Set by what it was trained to produce.

Adaptation moves style, subject and taste. It does not move any of the above. That asymmetry is the most useful thing to internalise about the term, because it tells you when to switch models rather than keep prompting: if the problem is a look, adapt; if the problem is a capability, change the base.

Evaluating one before you commit

Judging a base from a gallery is useless, because galleries are selected output. Build a small fixed probe set and run it on every candidate, unchanged, at each model's own recommended settings. Eight prompts is enough if they are chosen to hurt:

  1. Four distinct objects with specified colours and positions, to test counting and layout.
  2. A short phrase that must be spelled correctly in frame.
  3. Hands doing something specific, holding a tool, tying a knot.
  4. A named aesthetic you actually need, not a generic one.
  5. A low light scene with a single practical source, to test noise handling and grade.
  6. A long prompt with a clause buried in the middle, to test adherence decay.
  7. For video, one shot with real physics: pouring, fabric in wind, a person sitting down.
  8. Your own recurring subject, whatever you generate most often.

Keep the outputs. A probe set only earns its keep when you rerun it against the next release and can see what actually changed instead of trusting a claim.

What the term does not promise

Broad pretraining is not the same as being good at your work. A model with wide general competence will still lose to a narrow fine-tune on a specific illustration style, and a large video base will still lose to a purpose built lip sync model on a talking head. The word describes where the weights came from, not the quality of the result, and treating it as a quality tier is how workflows end up standardised on the wrong checkpoint.

The prompt for this

A starting point that reliably produces the effect. Adjust the subject and setting; keep the technical clauses.

Four glass bottles on a windowsill, each a different colour, handwritten label reading SPRING 24 on the second bottle from the left

Try Foundation Model yourself

Open the generator with a starting point already filled in.

Frequently asked questions

What is the difference between a foundation model and a fine-tuned model?
The base is pretrained on broad data at enormous cost and knows a great deal about a great many things. A fine-tune takes those weights and pushes them toward a narrower target, gaining accuracy on that target and usually losing range elsewhere. A specialist fine-tune will beat its own base on the look it was trained for, and lose to it on everything else.
Is this the same thing as a large language model?
A large language model is one kind of foundation model, the text kind. The term is deliberately broader, because the same pretrain-then-adapt pattern now covers image, video, audio and multimodal systems. When someone in a production context says base model, they usually mean the image or video checkpoint their adapters target.
Can a LoRA fix a weak base?
It can add a style, a character or a concept. It cannot add capability. If the base cannot render legible text, count objects, or hold a subject for ten seconds, no adapter fixes that, because adapters nudge existing behaviour rather than teaching new competence. Weak base plus good adapter is still a weak result.
Why do all the variants of one model share the same weaknesses?
Because they are the same weights wearing different interfaces. The image-to-image, inpainting and first-last-frame versions of a family usually sit on one pretrain, so a bias in the base, a warm colour cast, a tendency to centre subjects, sluggish motion, appears in every one of them. Evaluate at family level, then pick the task variant.
How many should I keep in a working pipeline?
Two or three, chosen for complementary failure modes rather than for overlapping strengths. A typical set is one strong instruction follower for anything with text or exact requirements, one with the photographic look you like, and one fast tier for exploration. Adding a fourth similar model adds decisions, not capability.

Related terms