AI & Tech··1 sources verified
DECOUPAGE Adds Directorial Audio-Visual Control to Long Multi-Shot Video Generation
Summarised by Relevant News AI · Read time: 3 min
Researchers introduced DECOUPAGE, a screenplay-driven framework that generates joint audio and video for 15- to 60-second multi-shot, multi-character scenes. It separates visual and acoustic timing into independently bounded tracks, so dialogue can bridge cuts (J- and L-cuts) and silences can be held. A directorial lip-sync mechanism stops speech conditioning from driving mouth movements during off-screen dialogue, which addresses hallucinated speech and false lip motion in existing systems.
Why it matters: It shows how audio-visual diffusion models can be given explicit editorial control over timing and lip-sync across long, multi-shot narratives, a capability current video generators lack.
All sources