How to work through it

01

Define what counts as the same result

Separate invariants from allowed variation. A product shot might require stable identity, label geometry, action order, and final composition while allowing small changes in texture or timing. Write pass, fail, and review-needed criteria before any attempt. Without that contract, ten attractive clips can still represent ten different interpretations rather than one repeatable production behavior.

  • Required invariants
  • Allowed variation
  • Automatic failure conditions
  • Human review conditions
02

Freeze the test context and log every attempt

Record the interface, visible model label, date, region, account context, prompt, references, settings, seed behavior if exposed, and output order. Keep rejected attempts instead of reporting only selected clips. When a system or prompt changes, start a new test version. A claim cannot be generalized across configurations that were not held constant and disclosed.

03

Report observations with their denominator

Describe how many attempts were run, which passed each criterion, the recurring failure types, and who reviewed them. Do not turn one small set into a permanent product percentage. A transparent conclusion sounds like an observation tied to this brief and date, followed by unresolved questions and a retest trigger—not a universal guarantee about future generations.