Tutorial: Video
Generative AI video is still emerging: while there are many tools available already using the capabilities of the technology to assist in the modification of video content, relatively few models have been released that allow for full image-to-video or text-to-video content directly to the consumer market. Precursors to these tools, such as deepfakes video creators for working from photos or video to create false content featuring particular people, generally require dataset preparation of a type we won’t be engaging here. Instead, we’ll be running a limited experiment with generative video this week, then looking at the larger claims and outputs from companies developing tools in this space.
Image to Video Exploration
For this exercise, you’ll need a few images that are either in the public domain or that you hold the rights to, and a free account with a generative video tool. Given the current coverage of Runway, I suggest using it as at least one of the tools you try for this exercise. However, there are several options - explore and select one that fits your interests. This example was generated using a static photo (shown below) and the Runway Gen-3 alpha model with a simple prompt: “Monarch butterfly, taking off, Florida garden” - in this case, the image provided most of the content, but notice how the wings change as the sequence progresses.
Source image:
Output:
Use up your free attempts and try a range of prompts - there’s no need to subscribe this time, unless you are interested in exploring this further as part of your own work. Generate 2-3 short videos to share with your peers, using either one tool, or a range.
Critiquing Generated Video
Now that you’ve explored the tools within some limitations, choose a generated video to analyze and critique, with particular attention to the tensions of perception, vision, and labor drawn out through this week’s readings. Keep in mind the concerns raised in Brett Halperin’s talk, “Hollywood Film Workers Strike Against AI: Understanding Algorithmic Resistance to Generative Cinematography”, as well as the current news coverage around greater integration of AI into cinematic productions.
I recommend beginning with explorations by new media artists probing the technology through the form. Here’s a few to get you started, but feel free to look further or to pick another piece discussed in The Perception Machine.
- Refik Anadol - Unsupervised
- Will Luers, Mark America, Chad Mossholder - Posthuman Cinema
- Kelly Boesch - AI Art
- Alan Warburton - The Wizard of AI
- Jon Finger et al - Summoner’s Rune
You could also critique a more commercial example. Be warned that these are drawn from AI hype posts, and thus tend to present these videos uncritically (and don’t necessarily offer much transparency about the tools of their production):
Research the video you analyze: determine the tools involved and look for any statements regarding the creator’s intention or approach, and think about how that relates to the broader discussion of the impact on cinema going forward.
Discussion
Your discussion post should combine your knowledge (and samples) from your own exploration of generative video with your observations about the form drawing on the example you’ve selected for analysis. Make sure to include the link to the video you’ve selected as well as relevant context, so that we can all build a better sense of the state of the field from the range of works critiqued.