MY KOLKATA EDUGRAPH
ADVERTISEMENT
regular-article-logo Wednesday, 09 October 2024

Meta has an AI-powered video clip generator — Movie Gen

Movie Gen is a 'cast' of foundation models, the largest of which is the text-to-video bit

Mathures Paul Published 09.10.24, 08:11 AM
Meta’s Movie Gen model puts out realistic videos with sound.  

Meta’s Movie Gen model puts out realistic videos with sound.   Meta

Meta, the company behind Facebook and Instagram, has a new artificial intelligence model called Movie Gen that can come up with realistic-seeming video and audio clips depending on user prompts.

Movie Gen is a “cast” of foundation models, the largest of which is the text-to-video bit. The company claims it outperforms the likes of Runway’s Gen3, LumaLabs’ latest, and Kling1.5, though none of this can be confirmed.

ADVERTISEMENT

The audio that gets added to videos is also AI-generated, matching the imagery with ambient noise, sound effects, and background music. Further, the videos can come in different aspect ratios.

Meta says Movie Gen can also create custom videos from images or take an existing video and change its elements. In an example shared by Meta, a runner is edited using AI in different ways: in one frame, he’s holding pompoms while in another, the background has been edited to depict a desert and in a third, the runner is wearing a dinosaur costume.

Videos created by Movie Gen can be up to 16 seconds long, while audio can be up to 45 seconds long. The announcement comes at a time when Hollywood is struggling to control the power of generative AI video technology after OpenAI in February first showed off how its product Sora could create feature film-like videos in response to text prompts.

Meta’s chief product officer, Chris Cox, writes on Threads that the company “[isn’t] ready to release this as a product anytime soon”, as it’s still expensive and generation time is too long.

Lawmakers are expressing concern about how AI-generated fakes, or deepfakes are being used, especially during elections.

AI image generators are raising concerns about ownership. There have been reports about AI startup Runaway training its video generator on thousands of scraped YouTube videos, something YouTube CEO Neal Mohan has already said would violate the platform’s terms of use. In a blog post, Meta says it trained Movie Gen on “a combination of licensed and publicly available datasets” but didn’t specify which.

RELATED TOPICS

Follow us on:
ADVERTISEMENT
ADVERTISEMENT