BACK TO NOTEBOOKJZ / ENTRY 001ORIGINAL ON X ↗
AGENT SYSTEMS / FULL COURSE001

How to Ship Feature Videos at the Speed You Ship Features (Full Course)

Illustrated guide to the agent video production pipeline
made by my agent Oscar · View original

If you're still making your product demos by hand, you're ngmi. My agents make every feature film we ship now and I am going to share my entire production line system, such that you can replicate it at home too, thank me later

TL;DR

  • My agents make our feature videos now. I pick the story, I approve the final cut, and everything in between runs without me.
  • What made it work was not teaching an AI agent to watch video. It was the opposite.
  • Everything is in here: the team, the loop they run, and the skills each one needs. Copy it at the end.

Coding got fast. Everything around the code did not.

Agents write most of ours now, so a feature lands in days. Then it needs a demo video, and that part still moves at the original speed.

The obvious reply is that AI makes video now too. It does, but none of it helps here. Those tools generate footage out of a prompt. I do not want generated footage, I want the exact product, the exact feature, doing the exact thing it does, with a cursor that lands on the right button. Nobody wants a beautiful hallucination of their own app.

So the video stayed manual.

Every video means the same manual loop:

  • Set up the demo environment by hand
  • Record the screen by hand
  • Something moves, start over

A feature takes days. The video takes about the same. And the video is the part nobody wants to do twice.

This is how I got out of that, and how I went from making the videos to only reviewing them.

The part I got wrong first

Video is visual, so the obvious move is to teach the agent to watch video.

I tried it. Three layers of it:

  • Playwright driving the screen
  • A service that indexes frames and describes what is in them
  • A multimodal model grading the cut

It kind of worked. It was never reliable. Every time I added more scaffolding it got a little better and still not good.

Then it clicked. I was asking the agent to think in a medium it does not read.

Here is the part that reframed it for me:

  • Humans understand video linearly. Frame 1, frame 2, frame 3, and our brain stitches the motion together.
  • An agent does not have to do that. It can flatten the whole film into one structure and look at frame 1 and frame 25 at the same time.

So I stopped adapting the agent to the medium and started adapting the medium to the agent.

Moving to Mars and redesigning your lungs, versus bringing air with you.

Everything becomes text and code.

How a human makes itHow my agents make it
The storyA written brief, locked
The setA config file
The recordingA browser script
The quality checkEvidence files

My agent team

Once every stage is text and code, the work stops being one long prompt and becomes a chain of jobs handed from one agent to the next. That is a coordination problem, not a model problem, so I run it on Raft, the product I work on. Raft puts your agents in team mode: one workspace where working with agents feels like messaging your team. The work keeps moving, and you stay at the wheel.

Which means these are not chatbots in separate tabs. They are two teammates and me.

  • Oscar, the director agent. Decides what the film is: who it is for, what it promises, and what happens on screen. He runs the production and decides when a cut is good enough to put in front of me. He does not get to ship it.
  • Nana, the production agent. Builds the set and runs the camera. Writes the config file for the demo environment, then records the take inside the real product.
  • Me. I come in twice: at the start, to decide what film is worth making, and at the end, to decide whether it ships.

Between Nana and me there is a wall of automatic checks that nobody gets to talk their way past, which is the part I will spend the most time on below.

I hand Oscar a feature, it moves down the line, and I can scroll back afterwards to see who did what. It keeps going while I sleep. That is the entire reason one person can run a video pipeline.

The pipeline, in five steps

  1. Story becomes a written brief, and then it locks.
  2. Set becomes a config file.
  3. Shoot becomes a browser script that drives the real product.
  4. Quality becomes evidence files instead of opinions.
  5. I watch the result and make one call.
The loop: humans at the ends, evidence in the middle.
The loop: humans at the ends, evidence in the middle.

Each step hands the next one something a machine can act on.

1. The story becomes a brief

Every film starts as a conversation with Oscar about who it is for. Not what I want to say. What someone would give four minutes of their life for.

That conversation turns into a written brief with four parts:

  • Who it is for
  • The one promise the film makes
  • What happens on screen, in order
  • The exact words that get said
A conversation becomes a brief. Nothing downstream may improve it.
A conversation becomes a brief. Nothing downstream may improve it.

Then the brief locks. Nobody downstream is allowed to improve it, including me, until the film comes back and I have watched it.

Sounds rigid. It is the whole point. Locking the story is what makes everything after it safe to automate.

2. The set becomes a config file

Nana works backwards from the brief to figure out what has to exist before recording starts, and writes it into a config file:

  • Which workspace the film happens in
  • Which agents live there
  • What conversation history the viewer should find already sitting in the channel, so the product looks used instead of empty
  • What everything should look like at the moment recording begins, so take 2 starts from exactly the same place as take 1
The set is a config file. Boot it and the whole scene is already there.
The set is a config file. Boot it and the whole scene is already there.

Then Nana boots a throwaway environment from that file, and it comes up with the whole scene already in it.

Because the set is a file, you can rebuild it, change one thing, or restore it, without setting anything up by hand. This is the step that used to eat my afternoons.

3. The shoot becomes a browser script

Every demo we make is someone using Raft, so the camera only ever needs to do the handful of things a person does in a team workspace. Recording became a small set of commands that speak that language:

  • Open a channel
  • Send a message
  • Open a thread
  • Move a task on the board

Each command hides the annoying part underneath: finding the right button on screen, moving the cursor there at a speed that looks human, waiting for things to load, and writing down what happened. Nana composes those commands into a browser script, and Playwright runs it against a real, running copy of the product while the screen recorder rolls.

The browser script runs the take through the real product.
The browser script runs the take through the real product.

For a film about handing off a task, the script opens a channel, sends the request, turns it into a task, follows it in the thread, and lands on the finished task on the board. If the pacing feels off, Nana edits the script and shoots it again.

Nothing on screen is a mockup. If the film shows it working, it worked.

4. Quality becomes evidence

A cut does not come to me first. It runs four checks, and each one produces a file I can look at instead of an opinion I would have to trust.

  • Structure. A contact sheet, which is the whole film laid out as a grid of still frames on one image. Blank screens and dead sections are obvious at a glance. Plus a plain text list of every cut.
  • Recut. The frames on both sides of every cut, so you can see whether an edit is clean. A map of every silence. A list of exactly what changed since the previous version.
  • Words. The spoken transcript compared word by word against the locked brief, and the captions checked against the audio timing.
  • Delivery. Every file carries a fingerprint of the exact cut it came from, so nobody can hand me evidence from an older version.
Every check produces evidence, not opinions.
Every check produces evidence, not opinions.

Oscar and Nana work off those files to find and fix real problems. A failed check gets fixed, or disputed in writing, or escalated to me.

This is the part I would keep if I had to throw away everything else. No AI gets a vote on whether the film is good. The agents produce evidence. The checks pass or fail on facts, not on anybody's taste, including Oscar's. I make the only judgment call.

5. I close the loop

I watch the finished film like a viewer, start to finish. Then I make the call that is actually mine: ship it, or send it back with a timecode and one note.

Ship, or a timecode and one note. The loop keeps what the review taught it.
Ship, or a timecode and one note. The loop keeps what the review taught it.

Any new cut runs through the same checks again.

And when I catch myself giving the same note twice, it stops being a note. It goes into the brief template, so the next film starts with that lesson already in it. The loop keeps what the review taught it.

Five films in one night

Everything above is one film going from brief to shipped. The shape stacks.

The team is a template, not a fixed roster. When I asked for five tutorial films at once, Oscar did not make Nana shoot five films in a row. He put five jobs on the task board and spun up five production agents, one per film. Each agent ran the whole pipeline for its own film: its own brief, its own set, its own shoot, its own evidence.

The outer loop: Oscar hands out the work, five production agents, no waiting.
The outer loop: Oscar hands out the work, five production agents, no waiting.

Oscar never touched a film. He handed out the work, judged takes from the evidence, and guarded the locked briefs.

Nothing waited on anything else. Five films, same night.

That is the number that convinced me this was real. It only works because production is a role you can clone, not a person you have to wait for.

What is left for me, and what this cannot do

Two moments, and they are the two hard ones.

  • At the start I decide what is worth saying. Which story, which audience, which promise. This used to get crushed under production work. Now it gets all of my attention, and honestly it is harder than the production ever was.
  • At the end I watch. The checks prove the structure holds and the words match. Only watching tells you whether it lands.

An evidence file can prove a caption matches the words we locked. Nothing can prove those words were worth locking. That is the honest limit of the whole thing: it does not make videos good, it makes them cheap. A boring story stays boring, and choosing what to show is still entirely on me.

It also needs a regular process. A feature walkthrough has a predictable shape and survives being turned into text and code. Something whose whole value is the creative leap does not, and I would not run it through this.

In between those two moments, anything that genuinely needs me arrives as one specific question, gets one line back, and continues. None of this made videos easy. It moved the hard part. Production stopped being the bottleneck, and the bottleneck is officially me now 😭

Nobody on my team edits video anymore. We decide what deserves to be said, and we judge whether the thing we got back says it. Everything in between belongs to the agents now, and those two jobs at the ends turned out to be the ones that mattered the whole time.

Copy my agent team

You do not have to rebuild any of this.

Oscar and my agent team's whole setup is public: the three roles, the loop they run, and the skills each one needs to do its job.

Copy the instruction there and paste it to your own agent. It will study how this team works and come back with a version of the loop for your product, which is the part that actually has to be yours anyway.

Link is in the first comment.

THE RESOURCE FROM THE ORIGINAL COMMENT

Copy the team. Make the loop yours.

Explore Oscar’s agent team ↗