Making AI storyboard sheets reliable: five rounds of experiments with Seedream 4.5

Before anything expensive renders, a film needs its storyboard: six angles on a scene that agree with each other on faces, light, and geography. We are building a flow where one generated image, a 2×3 storyboard sheet, is cropped into six panels, and the user clicks the panels they want as exact visual references for video generation with Seedance 2.0.

That only works if the sheets are reliable: a correct grid, no sub-images inside a panel, six genuinely different setups, and one consistent cast across all six. So before wiring it into the product we ran five rounds of controlled experiments, roughly a hundred generations and under $10 of image spend, to find out what actually holds up. Sheets were generated with ByteDance's Seedream 4.5, with Google's Nano Banana Pro as a challenger in one round. This post reports everything we measured.

What to try first

For a landscape sheet, start with a photographer's contact-sheet prompt, explicitly request a uniform grid, and describe the cast once. Generate several repeats and inspect the actual crops before using them downstream. For vertical sheets, validate panel boundaries instead of assuming the requested grid was returned.

Scope: these are small, iterative tests, mostly on Seedream 4.5, judged with a vision model and spot-checked visually. The landscape result is 29/30 in the revised-prompt test, not an expected success rate for every project. Detection recovered useful choices from existing vertical outputs; it did not demonstrate better all-panels reliability.

The results, up front

The rest of this post is how we got those numbers.

What good looks like: one generation, six camera setups, one consistent performer. Crowd scenario, contact-sheet register, Seedream 4.5, about 25 seconds.

How we tested

Every round used the same harness. We generated sheets across deliberately hard scenarios (night-rain lighting, crowds, macro product shots, creatures, vertical formats, scenes with character reference images attached), cropped them exactly the way production would, and graded the crops with a vision-language-model judge. The judge checked explicit gates (every crop is one complete, coherent shot; no annotation text; six distinct camera setups) and scored character, location/lighting, and style consistency from 0 to 10.

Two lessons about method, learned the hard way:

1. Look at the pixels before trusting the judge. Our first reliability run scored 55%. Visual verification put the true rate near 70%. The judge was failing sheets for diegetic text (a clipboard reading "FAILED INSPECTION" in a scene about a failed inspection, a painted "B3" in a parking garage set on level B3) and for cosmetic gutters that never touch a crop. The rubric needed an explicit exemption for in-world text, and needed it stated twice.
2. Repeats are the point. Grid failures are probabilistic. A scenario that passes once can fail the next run, so a single-sample comparison measures luck.

Round 1: six approaches, head to head

We screened four prompt variants, a 2×2 board, and Seedream's native sequential mode (sequential_image_generation, which returns a set of separate images instead of a grid) across five hard scenarios.

Content quality was not the problem anywhere. With character reference images attached, both test characters stayed on-model in every panel of every sheet. The differences were all in layout discipline and shot diversity:

• The contact-sheet register ("a photographer's contact sheet of 6 cinematic film stills…") outperformed the word "storyboard" in this screen. Storyboard prompts produced more borders, panel numbers, and caption bars; an association learned during training is a possible explanation, not something this experiment measured.
• Trait-locking, the popular community technique of repeating each character's full description inside every panel line, hurt crop usability and bought nothing on consistency. One possible explanation is that repeated descriptions add competing instructions. We did not isolate that mechanism, and these five runs do not establish a general rule about identity drift.
• Sequential mode produced flawless full-frame geometry and collapsed creatively. Four of five runs returned near-duplicate framings: six similar medium shots of the same subject instead of wide/medium/close coverage. At 160s versus 25s per board and about 5× the cost, it lost on every axis we care about except geometry, which round 5 explores recovering through detection.

Sequential mode's failure, laid flat. Same scenario as the sheet above, but the six separately generated images collapse toward one framing. Consistency and shot diversity pull in opposite directions, and the set generator gives up the diversity.

Reference images carry hardest. Both characters here are generated from attached identity plates and hold across all six panels, down to the diegetic prop text on the inspection report that our first judge rubric wrongly penalized.

The register generalizes beyond characters. Product scenarios keep the same object, lighting, and grade across the coverage ladder. Macro-only scenes remained the hardest distinctness case through every round.

Rounds 2–3: 29 of 30 landscape sheets passed after iteration

We locked the winning register and measured it properly: twelve scenarios, three repeats each. The verified first pass sat around 70%, with three real failure modes. Comic-page layouts, where the model draws merged and irregular panels: beautiful manga pages, useless for fixed-grid cropping. Near-duplicate panels. And the occasional sub-composite, two little images inside one cell.

The fix was to name the failure modes in the prompt (every still identical in size, a perfectly regular lattice, NOT a comic page, NOT a manga page, no merged or spanning panels) and add an explicit distinctness ladder (at least one wide, one medium, one close-up). One iteration took the overall pass rate to 89%, and landscape, the dominant film format, to 29/30 (97%), our ship bar. Square held at 2/3, and its one failure was a single weak panel rather than a layout collapse.

Vertical did not follow. That became its own investigation.

Round 4: vertical failures persisted on both tested models

On portrait canvases, both models fit panels to the canvas rather than to the request. Seedream drew full-width webtoon-style rows. Nano Banana Pro drew immaculate grids with the wrong count, a clean 4×2 when asked for 3×2. With the identical anti-comic prompt, Seedream passed 2/6 vertical sheets and Nano Banana Pro 1/6. These failures suggest a portrait-layout tendency on these endpoints, but that is an interpretation, not a measured training mechanism. Switching between these two models did not solve our tested cases. Other prompts, models, and future versions remain untested.

Vertical failure shape one, Seedream's comic-page prior: a gorgeous manga layout that fixed-grid cropping slices to pieces.

Vertical failure shape two, Nano Banana Pro: flawless discipline, wrong count. A clean 4×2 when 3×2 was requested. Both sheets are full of good panels; only the grid assumption failed. That is what pointed us at round 5.

Round 5: stop asking, start detecting

The reframe that solved it: in almost every "failed" vertical sheet, the panels themselves were good. Only our assumption about where they sat was wrong. So instead of demanding a rigid grid, we detect the gutter lines the model actually drew and crop the cells that exist, accepting whatever regular count comes back. A sheet with eight good panels instead of six is not a failure. It is two free extra angles.

We re-processed every vertical sheet from the earlier rounds with a prototype detector. The wrong-count Nano Banana Pro sheets were fully recovered (8/8 usable panels each), and 73% of the 15 vertical sheets yielded four or more usable, distinct panels. That is a partial-recovery measure. Under the stricter requirement that every panel be usable, detection scored 27%, compared with roughly 30% for blind cropping. We did not measure the blind-cropping baseline for the four-or-more criterion, so these numbers do not quantify an improvement on a common success metric. The production version adds plausibility guardrails, falls back to the uniform grid for flush sheets, and doubles as a safety net for the rare landscape sheet with drawn gutters.

How to read the examples

The crowd sheet illustrates the goal: different framings that still look like one performer in one setting. The sequential example shows why clean geometry alone is insufficient: similar medium shots offer little coverage. The vertical examples show two separate issues—irregular geometry and a regular grid with the wrong count. A detector may recover the latter without fixing shot diversity or identity, which still need review.

What shipped

The measured recipe is now our production configuration:

• the contact-sheet register with uniform-lattice and anti-comic-page language, numbered per-panel lines (which users can override with their own shot descriptions: the 6/6 adherence result), and one global cast description;
• sheet dimensions computed so each cell is exactly the film's aspect ratio, with grid orientation matched to cell orientation;
• crops with a small inner margin, because every working grid pipeline we found, ours included, needs one to absorb boundary bleed;
• gutter detection with variable panel counts, so the flow keeps every good panel the model draws;
• and the panels feed Seedance 2.0 as ordered reference images, which honors their composition shot by shot.

Every number

For readers who want the full picture, the round-by-round results. "Crops usable" = every cropped panel is one complete coherent shot; "distinct" = six clearly different camera setups; "char" = judged character consistency (0–10). All generations Seedream 4.5 unless noted.

Round 1 — approach screen (5 hard scenarios × 1 repeat per arm):

Rounds 2–3 — reliability runs (12 scenarios × 3 repeats; pass = all product gates):

¹ Remaining repeats lost to transient network errors, not generation failures. ² Across both grid orientations tested. ³ Raw judge failures later verified as diegetic-text false positives.

Round 4 — vertical showdown (identical prompt, 2 scenarios × 3 repeats):

Round 5 — panel detection on the vertical sheets (15 sheets from rounds 3–4):

⁴ Strict all-panels bar with the unguarded prototype; the production detector adds plausibility checks and a uniform-grid fallback. The product-relevant bar is the 4+ row: users select panels, so a sheet with four strong options and two rejects is a working sheet.

A 2×2 vertical probe (6 sheets) mostly held geometry (4 clean cells in 4/6) but lost shot diversity, consistent with the round-1 finding that fewer panels invite repetition.

---

The through-line of all five rounds: current image models are already excellent cinematographers and inconsistent typesetters. Give them one canvas and one cast, name the layout priors you don't want, verify with your own eyes before trusting a judge, and when the model insists on drawing the grid its own way, measure where it drew the lines instead of insisting on yours.