AI Video Generation Benchmark
15 LLMs took 7 prompts through FrameCall's production pipeline. Every video started from the same brief and ran through the same renderer, with no hand-fixing in between. What came out is what you see here, failures included.
Three of the four briefs, generated by Claude Fable 5, the newest model in the lineup, with zero hand edits. Keep scrolling to watch all 15 models attempt the same videos side by side.
Leaderboard
The order comes from watching all four videos from every model and judging the output quality by hand.
Swipe the table sideways for all columns
| # | Model | Ran via | Videos rendered | Tries | Avg gen time | Avg code size | Total cost |
|---|---|---|---|---|---|---|---|
| 1 | Gemini 3.1 Pro Google | OpenRouter | 7/7 | 16 | 121.0s | 16.4k | $1.467 |
| 2 | GPT-5.6 Sol Pro OpenAI | OpenRouter | 7/7 | 8 | 119.5s | 29.1k | $9.304 |
| 3 | GPT-5.6 Terra Pro OpenAI | OpenRouter | 7/7 | 7 | 46.1s | 19.4k | $3.123 |
| 4 | GLM 5.2 Z.ai | OpenRouter | 6/7 | 25 | 305.7s | 28.2k | $0.399 |
| 5 | Claude Fable 5 Anthropic | Claude Code | 4/4 | 4 | — | 23.8k | ~$2.055 |
| 6 | Claude Opus 4.8 Anthropic | Claude Code | 7/7 | 7 | — | 27.4k | ~$2.036 |
| 7 | GPT-5.6 Luna Pro OpenAI | OpenRouter | 7/7 | 7 | 43.3s | 22.1k | $1.304 |
| 8 | GPT-5.5 OpenAI | OpenRouter | 7/7 | 7 | 261.2s | 29.3k | $2.807 |
| 9 | Gemini 3.5 Flash Google | OpenRouter | 7/7 | 9 | 69.2s | 29.4k | $0.996 |
| 10 | Claude Sonnet 4.6 Anthropic | Claude Code | 7/7 | 7 | — | 24.4k | ~$1.093 |
| 11 | Kimi K2.6 Moonshot AI | OpenRouter | 7/7 | 28 | 379.5s | 22.4k | $0.551 |
| 12 | GLM 5.1 Z.ai | OpenRouter | 7/7 | 19 | 337.8s | 27.2k | $0.665 |
| 13 | MiMo V2.5 Pro Xiaomi | OpenRouter | 5/7 | 46 | 413.5s | 27.7k | $0.053 |
| 14 | DeepSeek V4 Pro DeepSeek | OpenRouter | 5/7 | 10 | 148.2s | 17.7k | $0.166 |
| 15 | DeepSeek V4 Flash DeepSeek | OpenRouter | 7/7 | 8 | 165.5s | 31.2k | $0.033 |
1. YouTube Channel Intro
1920×1080 · 16:9 · 30fpsShow the exact prompt every model received
Create a polished 12-second YouTube channel intro for a tech review channel called "TECHPULSE". Include: logo reveal (a "TP" monogram), channel name animation, tagline "honest reviews • weekly uploads", social handles (@techpulse, /techpulse, techpulse.com), and a subscribe CTA with bell icon. Should look like something a channel with 500k+ subs would use. Professional but approachable vibe. Dark theme with blue accents.
2. SaaS Product Launch Teaser
1080×1080 · 1:1 · 30fpsShow the exact prompt every model received
Create a hype 12-second product launch teaser for a SaaS app called "FlowBoard" - a team productivity tool. Show off the product name with dramatic typography (gradient text looks good), highlight 3 key features: "Real-time Collaboration", "AI Task Prioritization", "Team Analytics". Include an app mockup, tagline "Built for teams that ship", waitlist CTA, and the URL flowboard.io. Modern startup aesthetic, square format for social. Make it feel like a Product Hunt launch video.
3. Instagram Story Ad
1080×1920 · 9:16 · 30fpsShow the exact prompt every model received
Create a 15-second Instagram Story ad for wireless earbuds called "SoundPods Pro". Product showcase with: product visual (earbud silhouette), name, price ($79, crossed out $149 showing discount), features (40hr battery, ANC, wireless charging), social proof (5-star rating, "2,847 reviews", customer quote), and Shop Now CTA with swipe-up prompt. Should feel premium like an Apple or Samsung ad. Clean light background, product is the hero. Vertical story format, readable text at mobile size.
4. App Feature Walkthrough
1920×1080 · 16:9 · 30fpsShow the exact prompt every model received
Create a 15-second feature walkthrough for a task app's "Smart Scheduling" feature. Show the user flow: phone mockup with task list, adding a new task "Finish quarterly report", toggling on "Smart Schedule" (with AI sparkle), watching it auto-slot into the calendar, getting a reminder notification. End with tagline "Smart Scheduling — Let AI handle the when". Should look like a real landing page demo video. Clean modern UI, satisfying interactions, professional feel.
5. Flagship Smartwatch Reveal
1920×1080 · 16:9 · 30fpsShow the exact prompt every model received
Create a cinematic 20-second flagship product reveal for a premium smartwatch called "PULSE One". This should feel like an Apple keynote film, with multiple scenes and confident transitions, not a single static shot.
Open on a dark, moody stage with a single beam of light and a teaser line ("Your day, redefined") as the watch rises into frame. Move into a hero shot: the watch rotating slowly in 3D, its screen glowing to life with an animated watch face. Then break it apart into an exploded view, components floating with three feature callouts arriving one by one: "OLED Always-On", "7-Day Battery", "ECG + Blood Oxygen". Reassemble the watch and cut to a live health dashboard scene where activity rings fill, a heart-rate line pulses, and sleep and step counters tick up. Land on a clean logo lockup with the tagline "PULSE One. Feel everything." and a "Pre-order now" CTA.
Premium and cinematic. Deep blacks, brushed metal, one vivid accent color. Make the product the hero and the motion feel expensive.6. Music App Year in Review
1080×1920 · 9:16 · 30fpsShow the exact prompt every model received
Create a 20-second personalized "year in review" recap for a music streaming app called "Sonus", in the punchy, shareable style of a Spotify Wrapped story. Vertical format, fast-paced, one bold stat per beat with playful full-screen scene changes.
Run these beats: a hook intro ("Your 2025, on repeat"), a huge number counting up to "47,210 minutes listened", a top-genre reveal where animated bars race and "Indie Pop" pulls ahead and wins, a "top artist" card that flips to reveal "Phoebe Vale" with a play count, a listening-mood breakdown as a colorful animated donut chart that draws itself, and a "top song" moment with a kinetic typography lyric flash. End on a confetti payoff with the Sonus logo and a "Share your Wrapped" CTA.
Give each scene its own vivid gradient and personality. Bold type, energetic motion, satisfying counters and snappy transitions. It should feel personal, celebratory, and instantly screenshot-worthy.7. Sneaker Drop Hype Ad
1080×1080 · 1:1 · 30fpsShow the exact prompt every model received
Create a 15-second hype ad for a limited-edition sneaker drop called the "Velocity AERO", square format for social. High-energy streetwear vibe with quick cuts synced to an implied beat.
Open with a hard-hitting countdown ("DROPS IN 3... 2... 1...") over a bold graphic backdrop, then a dramatic reveal of the sneaker as the hero, shown as a clean side profile that sweeps across frame with motion-blur energy. Fire in three feature callouts as punchy labels: "AeroWeave Upper", "Carbon Plate", "React Foam". Flash through two colorway swatches that snap the shoe's accents to new colors. Build to the release moment: "LIMITED. 500 PAIRS.", the price "$180", and a "Get yours" CTA with a drop date.
Aggressive kinetic typography, bold heavy sans-serif, neon-on-dark palette with one electric accent. Draw the sneaker stylistically in code rather than relying on a photo. Make it feel like a sneakerhead's must-cop hype reel.Bonus: recreating a real music video
960×720 · 4:3 · 30fps · 21sOff the leaderboard, we pushed the top Claude model further: recreate the opening of a real lyric video — “...Baby One More Time” — shot for shot, as a single Remotion composition. Claude Fable 5 designed an original mascot, synced every lyric to the word it lands on, and rebuilt the scene cuts, whip pans, and camera punch-ins from a written spec of the original.
Want videos like these for your own product?
Every one of the 97 videos on this page came out of FrameCall. Type a prompt, pick a model, and get a real motion-design video you can edit, export, and ship. No timeline editor and no After Effects required.