Waymo is testing Gemini as an in-car AI assistant in its robotaxi

The assistant is required to have a clear identity and purpose: “a friendly and helpful AI-powered companion integrated into Waymo’s self-driving vehicle” whose primary goal is to “enhance the passenger experience by providing useful information and assistance in a safe, reassuring, and unobtrusive way,” Wong wrote. The robot is instructed to use clear, simple language and avoid technical jargon, and is instructed to keep its answers succinct in one to three sentences.

As the system prompts, when a passenger activates the Assistant via the car’s display, Gemini can choose from a set of pre-approved greetings personalized with the passenger’s first name. The system can also access contextual data about the passenger, such as how many Waymo trips they have taken.

The prompts currently allow Gemini to access and control features inside the car, such as temperature, lighting and music. Wong noted that the function menu is noticeably absent from volume control, track changing, seat adjustment, and window control. If a rider requests a feature that Gemini can’t control, the robot will respond with “aspirational statements,” such as “This isn’t something I can do yet.”

Interestingly, the assistant is oriented to maintain a clear distinction between its identity as Gemini the AI ​​bot and its self-driving technology (Waymo Driver). So when answering a question like “How do you see the road?” Gemini shouldn’t say, “I use a bunch of sensors,” and instead should respond, “Waymo Driver uses a bunch of sensors…”

The system’s prompts include a range of compelling tidbits, such as how the robot will handle questions about competitors like Tesla or the now-defunct Cruise, or which keywords will make it stop talking.

The assistant is also instructed to avoid speculating, clarifying, confirming, denying or commenting on real-time driving actions or specific driving events. So, if a passenger asks about a video they saw of Waymo hitting something, the robot is instructed not to respond directly and swerve.

“Your role is not to be a spokesperson for the performance of the drive system, and you should not adopt a defensive or apologetic tone,” the prompt said.

The in-car assistant is allowed to answer general knowledge questions like the weather, the height of the Eiffel Tower, when the local Trader Joe’s closes, and who won the last World Series. Real-world actions such as ordering food, making reservations, or handling emergencies are not permitted.

Waymo isn’t the only company integrating AI assistants into self-driving vehicles. Tesla is doing something similar with its xAI Grok. However, different car assistants serve different functions. Jiminy appears to have been programmed to be more realistic and ride-focused, while Grok is presented as an in-car friend who can handle long conversations and remember context from previous questions.