Why Your Claude Code + Remotion Videos Keep Breaking

Ask an AI coding agent to "build me a Remotion video" and the first render usually looks great. The second one, on the same clip with a slightly different prompt, has muted audio. The third one is missing the caption track entirely. The fourth one renders fine but the callout appears two seconds late.

This isn't bad luck, and it isn't a Remotion problem. It's what happens every time you ask a language model to write the same class of code from scratch, over and over, with no fixed reference to check itself against.

Why the same prompt produces a different bug each time

A coding agent like Claude Code or Codex doesn't have a single "correct" Remotion composition memorized. Every time you ask it to build a video edit, it's generating new code — new component structure, new timing math, new audio handling — based on your prompt and whatever it infers from your footage. Two runs on the same brief can diverge in dozens of small decisions: how it computes a Sequence offset, whether it wraps <Audio> inside the right <Sequence>, how it resolves overlapping z-index on two text layers.

Most of the time this doesn't matter. In video rendering, it matters a lot, because a video composition has failure modes that don't throw an error — they just render wrong:

• Muted audio. The audio track gets nested inside a Sequence whose duration doesn't cover the full clip, or the volume prop gets set to 0 during a "fade" that never resolves back up. The render completes. It's just silent.
• Missing elements. A caption, a callout badge, or a B-roll cutaway gets planned in the code's structure but never actually mounted for any frame range, because the interpolate() window the agent wrote doesn't overlap with anything.
• Broken timing. A callout appears based on an estimated timestamp instead of a real one, so it lands half a second after the line was actually spoken.

None of these are "the AI being bad at coding." They're the predictable cost of asking a probabilistic system to produce a deterministic, frame-accurate artifact from scratch, every single time.

The fix isn't a smarter prompt — it's a smaller job

The instinct is to write a better prompt, or add more instructions, or paste in an example. That helps marginally and caps out fast, because you're still asking the model to write structural code and get it right blind.

The actual fix is to stop asking it to write structural code at all. Split the job into two parts that require completely different kinds of correctness:

1. Structure — the composition itself: sequences, timing math, audio wiring, layout, z-index, animation curves. This should be written once by a human, rendered, watched, fixed, and then frozen.
2. Content — what actually goes into that structure for a specific clip: the transcript, which line becomes the callout, what timestamp a caption starts at. This is exactly the kind of narrow, well-defined task a language model is reliable at, because there's a right answer it can check itself against (the real transcript).

Once you separate the two, the failure surface shrinks enormously. The model can't accidentally break the audio wiring, because it never touches the audio wiring — that code was tested once and never changes. All it does is decide which words go where, using real word-level timestamps from the actual audio instead of estimating.

A fixed template that an AI fills in beats an AI that writes a new template every time — not because the AI is unreliable in general, but because "write correct video-rendering code from a blank page" and "pick the right three lines from a transcript" are different-sized problems.

What this looks like in practice

A locked template means: the <Composition>, the sequence structure, the audio track, the caption renderer, the animation timing curves — all of it is fixed, pre-tested code. The AI agent's job is reduced to something it's actually good at: reading a transcript, picking the real content (which checklist items to show, which line deserves a callout, where a cutaway should land), and writing that content — plus timestamps — into slots the template already defines.

Before handing the render back, a good setup also has the agent verify its own output: does the audio track actually have signal, do any two text elements overlap, does the render duration match the source clip. That step catches the handful of failures that slip through even a locked template, before you ever open the file.

If you're building this yourself

You don't need to buy anything to apply this. If you're already prompting Claude Code or Codex to edit your videos:

• Build the composition once, by hand, and get it fully working on a real clip before you ask an AI to reuse it.
• Never let the agent regenerate the composition code on a new video — only regenerate the content that fills it.
• Get real word-level timestamps (a proper transcription pass, not the model guessing from context) before asking it to place anything.
• Add a verification step — even a simple one — before you trust a render.

It's a few weeks of work to build and harden a template this way, mostly spent on the edge cases nobody enjoys: a caption one word too long, a callout landing on a cut word, a clip four seconds shorter than the one you tested on.