The field of generative AI is experiencing a rapid evolution, offering promising prospects for enhancing human creativity and self-expression. In a significant stride forward, the transition from image to video generation was accomplished within a mere few months in 2022. The recent Meta Connect event showcased groundbreaking developments, prominently featuring Emu, a foundational model for image generation. Emu technology serves as the cornerstone for numerous generative AI applications, including advanced AI image editing tools for Instagram and the innovative Imagine feature within Meta AI, facilitating the creation of photorealistic images directly in messages or group chats across various apps.
Continuing to push the boundaries of this dynamic field, a new research endeavor has been announced, focusing on controlled image editing based solely on text instructions and a novel method for text-to-video generation utilizing diffusion models.
Emu Video: Pioneering High-Quality Video Generation
Emu Video introduces a simple yet highly effective method for generating videos from text using diffusion models. This unified architecture for video generation accommodates various inputs, such as text only, image only, or a combination of both. The process involves two distinct steps: generating images based on a text prompt and subsequently creating videos based on both the text and the generated image. This innovative "factorized" approach to video generation streamlines model training, enabling the direct generation of higher-resolution videos with unprecedented efficiency. Notably, our approach utilizes just two diffusion models to generate 512x512 four-second videos at 16 frames per second, significantly surpassing previous methodologies. Human evaluations have overwhelmingly favored our model, with 96% of respondents preferring it based on quality and 85% based on faithfulness to the text prompt. Additionally, our model excels in animating user-provided images based on a text prompt, setting a new benchmark in video generation capabilities.
Emu Edit: Revolutionizing Precise Image Editing
Emu Edit introduces a novel approach to streamline image manipulation tasks and enhance precision in image editing. By enabling free-form editing through instructions, Emu Edit empowers users to perform a wide range of editing tasks with unparalleled precision. Unlike existing generative AI models, Emu Edit accurately follows instructions, ensuring that only relevant pixels are altered, thus preserving the integrity of the original image. Leveraging computer vision tasks as instructions, Emu Edit offers unprecedented control in image generation and editing, setting a new standard for precision and fidelity.
Looking Ahead: A New Era of Creativity
While the current developments represent fundamental research, the potential applications are vast and diverse. Imagine the ability to create personalized animated stickers or GIFs effortlessly, edit photos with precision and ease, or enhance social media posts with dynamic effects. While not intended to replace professional artists and animators, technologies like Emu Video and Emu Edit have the potential to empower individuals to express themselves in new and exciting ways. From creative professionals exploring new concepts to friends sharing personalized greetings, these innovations herald a new era of creativity and self-expression.
Download paper: https://ai.meta.com/blog/emu-text-to-video-generation-image-editing-research/
0 Comentarios