Runway launches its first global model, and adds original audio to its latest video model

The race to launch global models has begun as AI-powered image and video generation company Runway joins a growing number of startups and major technology companies by launching its first model. The model, called GWM-1, works by making frame-by-frame predictions, creating a simulation with an understanding of physics and how the world actually behaves over time, the company said.

A world model is an artificial intelligence system that learns an internal simulation of how the world works so that it can think, plan, and act without having to train for every possible scenario in real life.

Runway, which earlier this month launched a Gen 4.5 video model that surpassed both Google and OpenAI on the Video Arena leaderboard, said its GWM-1 global model is more “general” than Google’s Genie-3 and other competitors. The company is promoting it as a model that can create simulations to train customers in various fields such as robotics and life sciences.

“To build a global model, we first needed to build a really great video model. We believe the way to build a global model is that teaching models to predict pixels directly is the best way to achieve general-purpose simulation. At enough scale and with the right data, you can build a model that has enough understanding of how the world works,” Anastasis Germanides, the company’s chief technology officer, said during the live stream.

Runway has released specific directions or versions of the new world model called GWM-Worlds, GWM-Robotics, and GWM-Avatars.

GWM-Worlds is a template application that allows you to create an interactive project. Users can set a scene through a prompt or image reference, and as you explore the space, the model creates the world with an understanding of geometry, physics, and lighting. The company stated that the simulation runs at a rate of 24 frames per second and at a resolution of 720 pixels. While Worlds may be good for gaming, it’s also well-positioned to teach customers how to navigate and behave in the physical world, Runway said.

With GWM-Robotics, the company aims to use synthetic data augmented with new parameters such as changing weather conditions or obstacles. Runway says this method can also reveal when and how bots might violate policies and instructions in different scenarios.

Runway also builds realistic avatars within GWM-Avatars to simulate human behavior. Companies like D-ID, Synthesia, Soul Machines, and even Google have worked to create human avatars that look real and work in areas like communications and training.

The company notes that worlds, bots, and avatars are technically separate models, but it plans to eventually merge all of these models into one.

Along with the launch of a new global model, the company is also updating the base Gen 4.5 model that was released earlier in the month. The new update brings native audio and long-shot and multi-shot generation capabilities to the model. Using this model, users can create one-minute videos with character consistency, original dialogue, background audio, and complex shots from different angles, the company said. The company said that you can also edit existing audio and add dialogues. Additionally, you can edit multi-shot videos of any length.

The Gen 4.5 update moves Runway closer to competitor Kling’s all-in-one video suite, which also launched earlier this month, particularly around native audio and multi-shot storytelling. It also indicates that video generation models are moving from prototype to production-ready tools. Runway’s updated Gen 4.5 model is available to all paid plan users.

The company said it will make GWM-Robotics available through the SDK. It added that it is in active talks with several robotics companies and enterprises to use GWM-Robotics and GWM-Avatars.