The question that settles your workflow
There are two routes. Text-to-video: you describe the scene and the model builds it out of nothing. Image-to-video: you hand over a finished picture as the starting point and the model sets it in motion. At most providers both are available, on the same interface, for the same money.
The choice is not a matter of taste, it is a matter of which carries the bigger risk: the look or the motion. If a particular product, face or setting has to look exactly the way you pictured it, the look is the risk. If a complicated run of movement has to land, the motion is.
What happens under the bonnet
One provider's documentation contains a sentence that settles most of the argument: generation started from text, in their system, actually makes an image first and then a video out of it. Which means text-to-video in many places is not a separate capability at all but an automated image-to-video, where the model picks the starting frame on your behalf.
If that is how it works, the question rephrases itself: do you want to see and approve that starting frame, or not? Supply it yourself and twenty seconds of work rules out most of the errors that come from the look — and you avoid regenerating a whole video because of one bad composition. If you do not care exactly how it looks, the text route is faster.
Where the picture wins
- An exact look. The product really does look the way it has to, the face is the one you want, the colours are right.
- Cheaper mistakes. Generating an image is a matter of seconds and cents, a video is minutes and dollars. Catch the error where it is cheap.
- Brand. If the starting frame comes out of your own brand template, the video stays inside it too. On a series that is what decides it.
- Consistency across clips. The same starting frame — or a sequence of images built on one another — gives a far more unified set. The detail is in keeping a character the same on video, a post devoted to that alone.
Where the text wins
The counter-argument is worth knowing too, and it also comes from a vendor source: a picture does improve consistency, but a written description handles complex runs of movement more efficiently.
- Complicated motion. "She turns around, picks up the glass and walks out of frame" — that is easier to state in words than to derive from a still.
- Camera movement as the main event. When the point is what the camera does, not what is in the picture.
- Ideation. Twenty directions fast, to see which one works. The direction you pick you are going to remake from a picture anyway.
- Abstract or non-literal visuals. Where there is no specific "correct" look, there is nothing for a starting frame to defend.
The recipe that works most of the time
1. Make the starting frame in an image generator, and keep fixing
it until it is genuinely good. Mistakes are cheap here.
2. The starting frame should be WIDER than what you want on screen:
motion happens at the edges, and the model sometimes crops in.
3. The video prompt should be ONLY about motion. Do not describe
again what is already in the picture — that just confuses it:
"the camera slowly pushes in, the steam rises, the light shifts"
4. If you know how the clip ends, give that as a closing frame too,
where the provider allows it. Motion then runs between two points.
5. Do not deal with sound here. Separate track, afterwards — always.
Starting-frame mistakes the video magnifies
- Too tight a crop. If the subject fills the frame, the motion has nowhere to go, and the model either hands you a still or distorts things.
- Motion blur. If the starting frame is already "in motion", the model makes that permanent, and you end up with a shaky video.
- Too many people in shot. What was fine in a still image merges together once it moves.
- A caption in the picture. In the video it will change from frame to frame, and that is far more noticeable than a misspelt word on a still.
Common misconceptions
- "Text-to-video is the advanced one, image-to-video is for beginners." The other way round: professional workflows almost always start from a picture, because that is where things can be checked.
- "If I give it an image, the prompt stops mattering." It still matters — just for something else. From then on the prompt's job is the motion.
- "The starting frame should be the highest quality possible." Good, yes, but above all correct: sound composition, room for the movement, a clear subject.
- "A photograph works too." It does, and often it is the best option of all: a real photo plus generated motion. Just watch the rights if there is a person in it.
What to take away
By default, start from a picture: it is cheaper, more predictable, and it takes out exactly the risk that causes most of the disappointment. Start from text when the motion is the point, or while you are still exploring. For the mechanism behind all of this, the length limits and the prices, how AI video works is the place to look next.