#447 – Workflow Tips for AI-Generated Video
This is harder than it looks.
Deliberate experimentation with the prompts, visual references and (constantly improving) tools is the only way to get any good at this.
That and hitting ‘regenerate’ a few times.
The same prompt + the same references + the randomness of an LLM will generate different results each and every time.
What am I talking about?
Creating useful AI-generated video content.
If you have a better way of doing this – Hit Reply and let me know!
Here's what's ahead:
- How to judge what you make
- 10 Gen-AI Video Tools
- A Pro Workflow
- Crafting Prompts
- Perceptual Fluency
For context, Marvel's Fantastic Four: First Steps trailer (the 5th most anticipated movie of the year) gained 16.9M views in those 7 days.
I found that astounding. But maybe that's why I run TWMT.
Categories of Judgement
When reviewing an AI-generated image, audio or video, here are a few categories to help you with critical evaluation before deciding whether to run with it.
Most important of all, will be your instinctive first response to experiencing it.
Your gut won't lie.
- Accuracy — How many errors are present?
- Adherence — How close is it to the prompt?
- Realism — Does it look, move, and feel realistic?
- Consistency — Of characters/worlds/objects across shots/generations
- Creativity — Does it look creatively interesting and useful?
Generative AI Video Workflow Best Practices
Test the different models to see which one you like. Most have free trials, which allow you to generate a couple of videos, sometimes with watermarks, to get a sense of their capabilities.
Here is a starter for ten:
- Kling (2.2)
- Runway.ml (Gen 4)
- OpenAI Sora (No video for new accounts)
- Pika
- Adobe FireFly
- Hailou Minimax
- Luma Labs Dream Machine
- Artlist
- Google Veo 2 (Join a waitlist)
- MidJourney (v7)
Depending on the type of shot you're creating you might get better results with one model over another. Experimentation is necessary!
However, with some of the smaller players in the market, it's hard to know if they're really just a wrapper with an API call to a 'foundation model' or if they have actually built their own video generation LLM (unlikely).
Count the cost
While AI video generations are not expensive, per se, they are not free.
If you're planning on creating a whole film or even a short sequence, you may end up spending more than you'd expect to get the right/useful results.
Also, credits are not created equal.
100 credits on one platform might be the same expense in dollars and cents as 1000 credits on another. So you'll also have to track your miles per gallon (to strain a metaphor) to know where you might get the best deal.
Most of the bigger models indicate how many credits any single generation will burn before you commit to it.