Today, once again it’s Google’s turn to present us with its latest AI-powered innovation. Is about Lumiereand amazing video creation toolthat we owe it to Google Research, the research and development arm of the web giant in terms of AI.
Presented in a website, This tool – which at the moment is just a project – holds very nice surprises for its future users. In fact, demo videos posted on the Internet offer an overview of its capabilities, which are impressive and surpass those of tools available to date, such as Photoshop. The launch of Lumière IA Therefore, it is eagerly awaited, but at the moment Google Research has not indicated any date.
Wait, we invite you to discover the operation and capabilities of this revolutionary tool.
How does this AI work? What are your skills?
Lumiere is an AI model designed by Google research, who participated in the development of WALT, an artificial intelligence for generating videos developed by researchers at Stanford University (south of San Francisco). Introduced at the end of December 2023, WALT impressed with its performance, without matching Runway or Pika Labs, two other video-generating AIs. To differentiate themselves from these tools, Google Research teams have chosen an innovative approach.
Thus, Lumière is based on Space-Time-U-Net (STUNet)a model that uses very advanced technology.
An innovative model created by Google Research
Video generation involves two dimensions: temporal and spatial. This means that an AI model must:
- generate each pixel correctly;
- predict its evolution to ensure the fluidity and coherence of a video.
Therefore, a more complicated task than image generation. To achieve such a result, Google Research has developed a new AI model, Space-Time-U-Net (STUNet), capable of understanding:
- where the elements of a video are located;
- the way they move, they evolve.
Specifically, with STUNet, an AI can generate a first image and then its movements by referring to the approximate locations where the elements go. Its simultaneous management of elements and their movement allows you to create, in one go, a video from start to finish.
The Google Research teams have thus managed to face a great challenge, as they explain in a research work published on the Internet: the creation of “realistic, diverse and coherent movements.”
What can Lumière do?
Lumiere is an AI specialized in generating and editing videos. From a simple text message (a request) or an image, you can create a video. 1024×1024 format and a duration of 5 seconds (16 frames per second). We are, therefore, light years away from short or feature films, and we cannot deny the sometimes artificial side of videos, but the result is still impressive and realistic. Especially since the Google Research teams have not finished working on this tool.
To date, Lumiere is capable, for example, of:
- animate parts of a static image to transform it into a cinemagraph;
- generate missing or damaged parts of a video;
- Dynamically modify people (for example, their clothing) and objects that appear.
On the site dedicated to this tool, several examples are presented that give us an idea of its capabilities and performance.
Lumiere, today’s most advanced video creation tool
Unlike other technologies available, the videos generated by Lumiere are fluid. Hash sequences and anomalies are avoided. This results in a much better viewing experience and makes the videos actionable.
Thus, with Lumiere, Google consolidates itself in an area where we did not necessarily expect it. In fact, the web giant has fallen a little behind in terms of image generation. It was surpassed by other players (OpenAI in the lead), who launched very efficient tools such as DALL-E, Midjourney or Stable Diffusion. But with Lumiere it has caught up and is now in first place although, remember, this AI is still in the development phase.
For this tool, Google researchers innovated by introducing “a spatio-temporal U-Net architecture that generates the entire temporal duration of the video at once, through a single pass through the model.” A difference from other existing models, which improves the general coherence of the videos and gives them a realistic appearance.
However, the innovative nature of Lumiere raises a question. By generating fluid and realistic videos, don’t you run the risk of promoting massive spread of deepfakes ? (Deepfakes are AI-created videos that make people say or do things they never said or did.) For now the tool is not public, so this solves the problem of deepfakes. But the question remains, especially since Google admits to being concerned about the risks. misuse to create false or harmful content ».
An innovation still in research and development
Starting in early 2024, Lumiere will not be available online. This AI is not used in any Google tools and no beta version is currently offered. To evaluate its capabilities, only demonstrations, a video and research documents have been made public on the Internet.
Therefore, this tool remains a research project that Google nevertheless wanted to reveal. A way to warn your competitors? To remind us that it is still the giant of the web and that it does not intend to be surpassed by others? Or did you want to share the work and exploits of your teams? Mystery…
On the other hand, Google has been transparent in saying that it does not prefer to make this AI available to the general public at the moment, for fear of misuse. But we can’t help but think that he didn’t show up for no reason. In what format will it be available? Who can benefit from it? At what price? How to control its uses to avoid abuse (particularly deepfakes)? Once again, we don’t have the answers. Google probably doesn’t have them either, hence this simple presentation of Lumiere, not to mention a beta or public version. Therefore, you will have to be patient before you can try this video generating AI.
