Synthetic film from open weights: memory, compute and licences, stage by stage

A minute of film

Generative models have become an artistic medium: voices that were never recorded, scores composed from a sentence, shots that no camera filmed. The same models now make product films and internal explainers. Watching a good result, it is tempting to think the hard part is a prompt, or a script that a coding agent could write in an afternoon.

It is not. Sixty seconds of 720p video at 24 frames per second is 1,440 frames, or 1.33 billion pixels. Open video models do not generate a minute. They generate clips of about five seconds, each one a long computation on a large GPU, and each one has to agree with the next. Around them sit separate models for speech, music, lip movement, frame interpolation and upscaling, each with its own weights, memory footprint, failure modes and licence.

This part walks through that open-weights stack, the first way, with the arithmetic in the open. Part two takes the second way: the managed services that run comparable stacks behind an API.

How a video model makes a frame

A diffusion model learns to undo noise. Training mixes a clean example x₀ with Gaussian noise ε according to a schedule, and teaches a network to recover what was added:

Generation runs the other way: start from pure noise and take tens of steps, each one a full pass of a network with billions of parameters. Wan, the open video model used for the numbers below, is trained with flow matching, which learns a velocity from noise to data rather than the noise itself, but the cost structure is the same: steps, times passes per step, times the cost of one pass. Video models also never denoise pixels. A variational autoencoder first compresses the clip into a latent tensor, and a transformer works on that latent, cut into patches.

The token count of one clip

The latent is where the cost is decided. Wan 2.1, whose code and weights are published under Apache 2.0, compresses time by 4 and each side of the frame by 8, then cuts the latent into 1 × 2 × 2 patches. A five-second 720p clip, 81 frames at 16 fps, becomes 21 latent frames of 90 × 160, then 21 × 45 × 80 = 75,600 tokens. Every transformer layer lets each token attend to every other: 5.7 billion pairs, per layer, per step. The same clip at 480p is 32,760 tokens, so going from 480p to 720p multiplies the attention work by 5.3.

The open-weights stack, layer by layer

A film is not one model. The diagram is one reasonable open-weights pipeline for a narrated film with a presenter and generated shots. Each box is a separate model or tool, with its own weights, its own input format and its own way of failing. Select a box for its candidates, memory class and failure modes.

Memory and compute per clip

Two walls come first: whether the weights fit in GPU memory, and how long the arithmetic takes once they do. The Wan 2.1 14B transformer ships as 57 GB of FP32 safetensors; inference casts it to BF16, 28.6 GB, already more than a 24 GB card before a single activation. Its text encoder, the encoder half of umT5-XXL, is an 11.4 GB file of its own. Wan 2.2 pushes both ways: its A14B model splits denoising between two experts of 14.3 billion parameters, one for high noise and one for low, so 28.6 billion must be held for 14.3 billion active per step, and the README asks for at least 80 GB of GPU memory to run it on one card; its 5B model takes the other lever, a VAE that compresses each frame 16 × 16, which leaves a quarter of the tokens per latent frame.

For a diffusion transformer with P active parameters, L layers of width d and N tokens, one forward pass costs about:

The first term is the matrix multiplications, the second is self-attention, and at video token counts the second dominates: for the 14B model at 720p it is more than twice the first. Classifier-free guidance, on by default in the Wan scripts with a scale of 5, runs the network twice per step, once with the prompt and once without.

Two published timings anchor the formula. The Wan 2.1 authors report about 4 minutes for a five-second 480p clip from the 1.3B model on an RTX 4090; with the default 50 steps and guidance, the formula gives that time at about 73 % of the card’s dense BF16 peak. For Wan 2.2’s 5B model they report under 9 minutes for five seconds of 720p on “a single consumer-grade GPU” with at least 24 GB, which on an RTX 4090 means at least 61 %. The figure below starts at 60 %. A run made for this article gives the other end: on an NVIDIA L4, an off-the-shelf diffusers pipeline took 12 minutes 3 seconds for the 50 steps of the same 480p clip, 14.4 seconds a step, which is 33 % of the L4’s dense BF16 peak. Set the slider to 33 % to see what that does to a finished minute.

The constants behind Figures 4 and 5, with their sources:

Location constrains the hardware too. In the AWS price list of 23 October 2026, the on-demand GPU instances in Paris (eu-west-3) were the g4dn, g5, g6, gr6 and p6-b200 families: no H100 or L40S instances. The single-L4 g6.xlarge cost $1.0216 an hour, and a team that must keep scripts and footage in that region starts from that card.

One GPU, many models

The stack in Figure 3 holds more weights than any single GPU has memory. Unless every stage gets its own GPU, models take turns: read from storage, moved to the device, used, evicted. The read alone depends on where the checkpoints live. Batching by stage, all keyframes, then all shots, then every lip-sync pass, pays each load once. Going clip by clip pays it twelve times per minute.

Which one matters depends on the rest of the setup. With the defaults of Figure 4, a 14B model on an L4, the video computation takes about 32 hours per finished minute and the reads are noise. Make the computation fast, an H100 with an 8-step distilled sampler, and put the checkpoints on a network disk at 125 MiB/s read clip by clip: the reads then take longer than the computing, about 60 % of the total under the assumptions shown. Every speed-up on the GPU moves the bottleneck somewhere else.

The licence matrix

“Open” says the weights can be downloaded, not what may be done with them. Several of the best-known checkpoints forbid commercial use, cap it by revenue, or exclude whole territories, sometimes including the reader’s own. The table summarises the licence files as read on 25 October 2026. It is not legal advice: read the file of the exact version you deploy.

How far behind is open

On the Artificial Analysis text-to-video arena, where people vote blind between two clips, the ranking read on 25 October 2026 put a closed model first: Wan 3.0, at 1175 Elo. The best open-weights model, MiniMax H3, was fourth at 1158, inside the margin of error. Its licence excludes the European Union, the United Kingdom, South Korea and the United States. The best open models that a company in the EU may run under their licence, Lightricks’ LTX-2.5, free below US$10 million of annual revenue, stood at 949, in 24th place.

The model behind the numbers in this article has followed the same path: Wan published weights up to version 2.2; versions 2.5 to 2.7 are offered through Alibaba Cloud’s API, with no weights on the Wan-AI model hub, and the arena lists 3.0 as closed. So the open way, for a European team, starts from models a clear step behind the services it would compete with.

Samples made for this article

Estimates are one thing, outputs another. The samples below were generated for this article with open-weights models, from scripts kept with it, with every setting and every time recorded.

Figure 7 · Image and video: SDXL and Wan 2.1 T2V 1.3B on one NVIDIA L4

“A moonlit valley at night, layered deep-blue hills, a large pale-gold moon, rivers of luminous ink flowing across the sky in long curved strokes, painterly contemporary digital art.”

“Rivers of luminous gold ink swirling and flowing fast across a deep-blue night sky, strokes curling and unfurling…”

“Ocean waves of deep blue and gold paint crashing and rolling toward the viewer under a full moon…”

“Clouds racing across a moonlit sky above layered deep-blue hills, time-lapse…”

Keyframe, SDXL, 1344 × 768, 30 steps

19.1 s

Denoising, per five-second shot (81 frames, 50 steps)

12 min 13–23 s

Decoding, per shot

about 30 s

Peak GPU memory

11.5 GB

Loading the models, once

3 min 51 s

The part you cannot download

A pipeline that runs is not a pipeline that produces something you would show. The failure modes are specific, and most of them only a person sees:

• Continuity between clips. Each clip is sampled on its own. Faces, clothing and light drift from one clip to the next. Conditioning every shot on a shared keyframe, or on the last frame of the previous clip, or training an identity adapter, each adds a stage with its own failures.
• Lip-sync at resolution. Audio-driven mouth models work on a cropped face at low resolution, which is then pasted back. On a 4K frame the mouth is upscaled from a few hundred pixels, and long sentences drift.
• Text, hands, brand assets. Generated frames misspell words and invent logos. Anything that must be exact is composited afterwards.
• Speech. Acronyms, numbers, product names and words from another language are where synthesis breaks, and where a listener notices first.
• Music. Generated tracks lose structure over long durations, drift in key or tempo, and seam audibly when looped.
• Levels and colour. Every stage returns its own loudness and its own grade. A film needs one: for broadcast, −23 LUFS under EBU R 128.

Metrics help but do not decide. Video benchmarks score dimensions such as subject consistency and motion smoothness; intelligibility and loudness can be measured. None of them says whether a shot is good. In practice someone reviews every second, and most generations are thrown away: that is the “generations per kept clip” in Figure 4.

What a coding agent changes, and what it does not

This article is a small test of the question. An AI coding agent wrote the code of its figures under my direction, and AI agents found and summarised the licence files, datasheets and price lists in the sources, each one linked so that every number can be checked. That is the part of a generative-media project that agents speed up: code and reading.

None of that work produced a GPU hour, a model weight, a licence or a reviewed shot, and those are the costs of the pipeline in Figure 3: on the defaults of Figure 4, 127 GPU hours on an L4 for one finished minute, a review of every row of the licence matrix for the exact versions used, and a person watching every generation. A prompt or a skill can describe the process. It does not come with the infrastructure, the models or the review.

The second way

Managed services run comparable stacks on their own fleets and sell the result per second, per character or per seat. The GPUs, the licences of their models and much of the quality work become their problem; the brief, the review and the terms of use of what you generate remain yours. Part two, in preparation, compares them on price and on the same short brief, with what each run actually cost.

What this part does not establish

• The compute figures are estimates from published configurations and an assumed utilisation, anchored on two published timings; optimised implementations can be several times faster. The models run for this article are listed with their samples.
• Licences and prices are those published on the dates given in the sources. Both change; check them before a decision.
• The licence table summarises licence files. It is not legal advice.
• The pipeline in Figure 3 is one reasonable design among several, not a recommendation.

Sources

• Nichol and Dhariwal, Improved Denoising Diffusion Probabilistic Models, 2021: the cosine schedule.
• Wan team, Wan: Open and Advanced Large-Scale Video Generative Models, 2025, and the configurations and generation scripts of Wan2.1 and Wan2.2, read on 25 October 2026: dimensions, layers, VAE strides, patch size, default steps and guidance. Checkpoint sizes from the model repositories of Wan-AI on Hugging Face.
• AWS Price List for Amazon EC2, eu-west-3, version of 23 October 2026: instance families and the g6.xlarge price.
• EBU R 128 (2023 edition): the −23 LUFS target.
• Licence files and model cards read on 25 October 2026: FLUX.1 [schnell], FLUX.1 [dev], FLUX.2 [dev], FLUX.2 [klein] 4B and 9B, SDXL, SD 3.5, Stability AI licence, HunyuanImage 3.0, Qwen-Image, MusicGen weights, Stable Audio Open, YuE, DiffRhythm VAE, Demucs weights, Real-ESRGAN, Practical-RIFE; for video, HunyuanVideo and HunyuanVideo 1.5, LTX-2.x, MiniMax H3, Mochi 1, MAGI-1, Step-Video-T2V; for speech, XTTS-v2, F5-TTS, Kokoro-82M, StyleTTS 2, Piper voices, Orpheus; for faces, LatentSync, LivePortrait, Hallo3, InsightFace; for recognition, Whisper, Parakeet TDT 0.6B v3, MMS forced aligner.
• Artificial Analysis text-to-video leaderboard, board with audio, read on 25 October 2026; the page shows no update date. Wan versions: Alibaba Cloud Model Studio and the Wan-AI organisation.
• NVIDIA datasheets and architecture whitepapers for L4, L40S, A100, H100, RTX 4090 and RTX 5090: dense tensor throughput and memory.