Bringing generative AI to video editing workflows in Adobe Premiere Pro | Adobe Blog

🌈 Abstract

The article discusses Adobe's plans to bring generative AI to video editing workflows in Adobe Premiere Pro. It covers the following key points:

• Adobe believes generative AI is the next accelerant to human creativity and is integrating it across Creative Cloud.
• The Premiere Pro team has been working with the professional video community to understand how generative AI can solve real problems in their workflows.
• Adobe is developing a new Firefly Video Model and exploring incorporating third-party generative AI tools and models like OpenAI's Sora, RunwayML, and Pika into Premiere Pro.
• New generative AI-powered tools are being introduced in Premiere Pro, including Generative Extend, Object Addition, Object Removal, and Generative B-Roll.
• Adobe is committed to an open approach, allowing customers to choose the right AI models for their use cases, and supporting Content Credentials to ensure transparency.

🙋 Q&A

[01] Bringing Generative AI to Video Editing Workflows in Adobe Premiere Pro

1. What are the key requirements that editors have for using generative AI in their workflows?

• AI must be integrated into their workflows and the tools they use every day, like Premiere Pro.
• The real value is in editing existing content faster with Generative AI - adding objects, removing distractions, or extending clips seamlessly on the timeline, all with their content and precise control.
• The ability to use different AI models that are great at different things, providing them with the power of choice.

2. What is Adobe's approach to delivering AI models for video editing?

• Adobe is developing a new Firefly Video Model to join their existing Firefly Image, Vector, Design, and Text Effects Models.
• Adobe is also exploring incorporating third-party generative AI tools and models, such as OpenAI's Sora, RunwayML, and Pika, into Premiere Pro.
• Adobe is committed to an open approach, allowing customers to choose the right AI models for their use cases.
• Adobe will support Content Credentials, which are like a nutrition label for content creation, to ensure transparency on the models used in the creation of media.

[02] New Generative AI-Powered Tools in Premiere Pro

1. What are the new generative AI-powered tools being introduced in Premiere Pro?

• Generative Extend: Allows users to make shots a little bit longer by generating entirely new frames at the beginning or end of a clip.
• Object Addition and Object Removal: Enables editors to replace moving objects in a shot, remove unwanted items, or quickly add set dressings like a painting or a photorealistic flower vase.
• Generative B-Roll: Allows users to generate video clips that illustrate difficult-to-shoot or fantastical concepts, or create placeholders to assist with planning and pacing.

2. How do these new tools leverage generative AI to enhance video editing workflows?

• The tools use AI-based smart masking and tracking to make it easier and faster to select moving objects over time.
• The Generative Extend, Object Addition, and Object Removal tools allow editors to manipulate and enhance their existing video content in new ways.
• The Generative B-Roll tool enables editors to create custom video clips to fit their storytelling needs, without having to source difficult-to-find footage.