Waymo says it has built a better standard for comparing robots to humans

Waymo has created a new computer model designed to more accurately answer a fundamental question: How can its self-driving software outperform humans?

The Alphabet-owned robotaxi company, which developed the computer model of human driving abilities in collaboration with TU Delft, published a paper on the topic in the journal Nature Communications on Wednesday.

Waymo said it expects the new model to be more accurate than the previous version it has used for the past several years. The new model was built using a framework called active inference, a theory that a driver constantly imagines possible futures and takes action to reach the safest, most predictable future.

Waymo said the new model will help it better understand how humans behave in crash scenarios faced by its automated robot.

“For decades, the auto industry has used physical and virtual crash dummies to evaluate a vehicle’s safety features, including its hardware and structural integrity,” Waymo wrote in a blog post on Wednesday. Waymo said the new model “advances this concept, serving as a behavioral standard for self-driving systems capable of realistically representing reasonable expectations of how a careful and competent human driver will respond to traffic conflicts.”

A more accurate model of human driving behavior is betting on autonomous vehicle companies needing to understand and evaluate the performance of their automated cars in collision situations. It comes at a critical juncture for Waymo, which is expanding into more cities and facing greater scrutiny from regulators and the public.

In January, when a Waymo robotaxi hit a child near a school in Santa Monica, California, the company relied on its previous computer model to claim that an alert human driver could have caused a collision at about 14 mph. The Waymo robo-taxi hit the child at just 6mph, after slowing down from 17mph, and the company said she suffered minor injuries. (The crash remains under investigation by the National Highway Traffic Safety Administration and the National Transportation Safety Board.)

The biggest difference between this new model — which Waymo calls the “Reference Driver” — and its predecessor is that it is able to reproduce the behavior of a human driver in the lead-up to an accident. Previously, Waymo’s models (and other industry models) focused on replicating “last-second reactive” human maneuvers, according to the company.

Meanwhile, the reference driver can “simulate the internal ‘surprise’ felt by the driver during conflict, providing a more human-like standard for autonomous driving systems that were previously impossible to automate at scale,” Arkady Zhgunikov, an associate professor at TU Delft, said in a statement.

Waymo says this new driver model can be adapted to model “a wide range of road user behaviors beyond collision avoidance,” and that it is better equipped to be applied to “large test sets including thousands of scenarios.”

“The model can represent and evaluate many complex real-world failures in a virtual environment, and identify performance improvements with unprecedented speed and efficiency,” the company wrote.

Waymo wants others to collaborate on pushing the reference driver program further, too. The company said on Wednesday that it is making the model’s research code available under a non-commercial academic license that allows it to be used in research, teaching, personal experiments and scientific publishing.