WayanPalmieri
← Writing/·3 min read

Keeping Characters Consistent Across AI Shots

The consistency problem in AI video — why a character drifts shot to shot — and the practical, repeatable steps I use to lock a look across a whole piece.

By Wayan Palmieri

Character consistency in AI video is engineered, not assumed. Left alone, a model re-invents a face every time you generate, so the same character drifts across shots — different features, wardrobe, proportions. You solve it with trained references, locked settings, node-based control, and a QC pass against a fixed look. Here is how I do it.

This is the hardest problem in AI-native work, and it is where most reels fall apart. One striking shot is easy. A character who is unmistakably the same person across forty shots is the actual craft.

Why do characters drift across AI shots?

Each generation samples fresh from the model. Unless you constrain it, there is nothing tying shot two to shot one — the model has no memory of the face it just made. Small differences compound: a slightly different jaw here, wardrobe that shifts there, and the audience stops believing it is one character.

So the goal is to remove randomness everywhere it does not serve you, and reintroduce control at every step.

Step 1: Build a reference set and train a LoRA

Start with the identity. Gather a clean, varied reference set for the character — angles, expressions, lighting — and train a LoRA on it. A LoRA fine-tunes the model on that specific identity so the face and features carry across every generation instead of being reinvented each time.

Quality of references decides quality of the LoRA. Garbage or too-similar inputs produce a brittle character that only works from one angle. Invest here; everything downstream depends on it.

Step 2: Lock seeds, prompts, and settings

With the identity trained, remove the rest of the noise. Standardize seeds, prompts, and generation settings so a shot is reproducible on demand. When a note comes back — "same shot, warmer" — you want to regenerate the exact setup, not gamble for it again.

Locked settings do not guarantee consistency by themselves, but paired with a LoRA they turn generation from a slot machine into a controllable process.

Step 3: Control generation with node graphs

Move the work into node-based control — ComfyUI and similar. A node graph lets you parameterize a shot: the same pipeline, references, and controls applied deliberately rather than hoped for. You can hold identity fixed while varying only camera, motion, or environment.

This is the difference between generating and directing. The graph is your repeatable setup, the way a lighting plot is repeatable on a stage.

Step 4: QC every shot against the look bible

Before anything reaches the edit, run a dedicated quality-control pass against a look bible — the reference frames, palette, and rules that define the piece. Compare each shot's character, wardrobe, and proportions to the reference. Flag drift, artifacts, and anything off-model, then regenerate the misses.

TechniqueWhat it locksWhen it matters most
Trained LoRACharacter identityRecurring characters, brand assets
Locked reference setLook and wardrobeShort runs, one-off characters
Standardized seeds/settingsReproducibilityRegenerating notes and revisions
Node-based controlRepeatable pipelineScaling across many shots
QC vs look bibleFinal consistencyEvery piece, before the edit

None of these works alone. Consistency is the stack, not any single trick.

The discipline behind the tools

The tooling is new; the instinct is not. Continuity has always been a production job — script supervisors, wardrobe, and editors have protected it for a century. In AI-native work you protect it with LoRAs and look bibles instead of a continuity photo, but the goal is identical: the audience should never notice the seams.

If you are standing this up for a team, I help studios and brands build repeatable consistency pipelines, and you can see the results in selected AI Artist work. For where this fits in the larger workflow, see how to build an AI-native production pipeline.

WAYAN AI// online
$ ask anything about Wayan's work, AI production, or hiring.
tip: try /director, /producer, /artist, /pitch