Remember the ‘Success Kid’ meme from 2007? Yes, the one with a baby clenching a fistful of sand. What about making the baby run? It is possible to make it happen using Dream Machine, a new AI system from Luma AI?
The tool is open to everyone but the output is limited to a few seconds for free users. For example, upload a photograph of Winston Churchill standing and then use keywords like: “Make the man walk with a gentle pace.” The tool takes a few minutes to create a short video of Churchill taking a stroll.
There is an alternate way of coming up with videos: Enter a descriptive prompt, like: “Make a cat jump around the room while chasing a ball.” The prompt is enough to generate a five-second video clip matching the scene. The rendition is more or less faithful to the prompt and there is some level of fluid motion. While creating the video for the ‘Success Kid’, we didn’t mention that he should be seen around the sea but the AI bot understood the background and made sure there is storytelling involved.
Luma AI is not the only one
There are a number of rivals to the generative chatbot from Luma AI, like OpenAI’s Sora and Kuaishou’s Kling.
Sora is not available to the general public but it is already being used by a select group of filmmakers. In the latest edition of Tribeca Film Festival, five short films (by Bonnie Discepolo, Ellie Foumbi, Nikyatu Jusu, Reza Sixo Safai and Michaela Ternasky-Holland) were screened that were made using Sora, a text-to-video artificial intelligence model.
Luma AI previously released an app that helps one shoot 3D photos with the iPhone. To activate the free tier of the company’s latest tool, all that one needs to do is log in using a Google account. The clips it produces are five seconds long and in 1360x752 resolution. (We suggest you use an email ID that doesn’t contain personal or official emails.)
The race to offer high-quality videos using text prompts is on, prompting concern. Earlier this week, Runway, a company building generative AI tools geared toward film and image content creators, unveiled Gen-3 Alpha.
Available soon for Runway subscribers, the company said in a blog post: “Gen-3 Alpha excels at generating expressive human characters with a wide range of actions, gestures and emotions. It was designed to interpret a wide range of styles and cinematic terminology [and enable] imaginative transitions and precise key-framing of elements in the scene.” The limitation of the chatbot at the moment is that its footage is limited to 10 seconds.
Recently, Google showcased its new tool, Veo, at its recent I/O developers conference. Filmmaker Donald Glover was present to showcase the tool. But the big name that many are looking forward to is Sora, which needs to compete with Runway, Pika and Stability AI.
Also on the horizon is Adobe and its generative AI video model for its Firefly family that will bring new tools to its Premiere Pro video editing platform. The company enjoys an advantage over competitors — it is building its own Firefly models into its software ecosystem that is already used by professional content creators.
Controversies and concerns
The problem with AI-generated video and photographs remains unsolved — IP-related lawsuits if the vendor trained on public data, including copyrighted data from the web. There are several lawsuits being filed across the globe around vendors’ fair use training data defenses, arguing that generative AI tools replicate artists’ styles without the artists’ permission.
AI-generated videos can fuel disinformation. Last year, fake AI photos of former US president Donald Trump running from police went viral. Now, what if these photos are brought to life using video AI tools? It’s not difficult to imagine lifelike fake videos turning up on social media to erode public trust in political leaders and the media.
Apple recently announced its AI plans with Apple Intelligence but the company wants to restrict itself to tools that help users rather than create fake videos or photos.
