Nomadic raises $8.4 million to discuss data streaming from autonomous vehicles

To build the autonomous machines of the future, sometimes your model needs a model.

Companies developing self-driving cars, robots that manipulate the physical environment, or autonomous construction equipment collect thousands, if not millions, of hours of video data for evaluation and training.

Organizing and indexing this video is now a task for humans, who have to watch it in its entirety. Even fast forwarding, it cannot be measured. NomadicML, a startup founded by CEO Mustafa Pal and CTO Varun Krishnan, wants to solve problems for customers who have 95% of their fleet data archived.

The challenge becomes even more difficult when looking for edge cases – the most valuable data depicts events that rarely occur and can confound inexperienced physical AI models.

Nomadic solves this problem with a platform that turns snapshots into a structured, searchable dataset through a set of vision language models. This, in turn, allows for better fleet monitoring and the creation of unique data sets to promote faster learning and iteration.

The company announced an $8.4 million seed round on Tuesday at a post-money valuation of $50 million. The round was led by TQ Ventures, with participation from Pear VC and Jeff Dean, and will allow the company to onboard more clients and continue improving its platform. Nomadic also won first prize in the Nvidia GTC demo competition last month.

The founders, who met when they were computer science undergraduates at Harvard, “kept facing the same technical challenges over and over in our jobs” at companies like Lyft and Snowflake, Ball told TechCrunch.

“We’re giving people insight into their footage, whatever is driving their autonomous vehicles [and] “Robots,” he said. “That’s what drives autonomous system builders forward, not random data.”

Imagine, for example, trying to improve an autonomous vehicle’s understanding of its ability to run a red light if a police officer directs it to do so, or isolating it every time vehicles travel under a certain type of bridge. The Nomadic platform allows these incidents to be identified for compliance purposes, and fed directly into training pipelines.

Customers such as Zoox, Mitsubishi Electric, Natix Network and Zendar are already using the platform to develop smart machines. Antonio Bolelli, vice president of engineering at Zendar, said Nomadic’s tool has allowed the company to scale its business much faster than an outsourcing alternative, and that its expertise in this area sets it apart from other competitors.

This type of model-based automatic annotation tool is emerging as a key workflow for real-world AI. Data classification companies such as Scale, Kognic and Encord are developing AI tools to do this work, while Nvidia has released a set of open source models, Alpamayo, that can be adapted to address the problem.

Varon argues that his company’s tool is more than just a rating tool; It is an “active system of thinking: you describe what it needs and it figures out how to find it,” using multiple models to understand the action that is occurring and put it in context. Nomadic’s backers expect the startup’s focus on this specific infrastructure to win out.

“It’s the same reason Salesforce isn’t building its own cloud and Netflix isn’t building its own cloud [content distribution facilities]“The second a self-driving vehicle company tries to build Nomadic in-house, it gets distracted from what makes it win, which is the robot itself,” Schuster Tanger, a partner at TQ Ventures who led the round, told TechCrunch.

Tangier praised Nomadic’s talent, noting that Krishnan is an international chess master ranked as the 1,549th best player in the world. Meanwhile, Krishnan boasts that all of the company’s dozen or so engineers have published scientific papers.

Now, they’re hard at work developing specific tools, such as one that understands the physics of lane changes from camera footage, or another that derives more precise locations of robot fists in a video. The next challenge, from the perspective of Nomadic and its customers, is to develop similar tools for non-visual data such as lidar sensor readings, or fuse sensor data across multiple modes.

“Taking terabytes of video, comparing it to hundreds of 100 billion-plus parameter models, and then extracting accurate insights from it, is very difficult,” Pal said.