What happens when AI starts building itself?

Richard Sucher has been a major figure in the AI ​​field for some time, best known for founding chatbot startup You.com, and before that for his work on Imagenet. Now he joins the current generation of research-focused AI startups with Recursive Superintelligence, a San Francisco-based startup that emerged from secrecy on Wednesday with $650 million in funding.

Sucher is joined in the new project by a group of leading AI researchers, including Peter Norvig and Christa co-founder Tim Shi. Together they are working to create an iteratively self-improving AI model, one that can autonomously identify its own weaknesses and redesign itself to fix them, without human intervention – the ancient holy grail of contemporary AI research.

I spoke with him on Zoom after the launch, delving into Recursive’s unique technical approach and why he doesn’t think of this new venture as a neolab, an informal term for a new generation of AI startups that prioritize research over building products.

This interview has been edited for length and clarity.

We hear a lot about repetition these days! It seems like a very common target across different laboratories. What do you consider your unique approach?

Our unique approach is to use openness to achieve iterative self-improvement, which no one has yet achieved. It is an unattainable goal for many people. Many people assume that this already happens when you perform an automated search. You know, you can take AI and ask it to improve something else, which could be a machine learning system, or just a message that you write, or, you know, whatever, right? But this is not frequent self-improvement. This is just an improvement.

Our main focus is to build reproducible, self-improving superintelligence at scale, which means the entire process of ideation, implementation, and validation of research ideas will be automated.

Firstly [it would automate] AI research ideas, and eventually any kind of research ideas, even eventually in physical fields. But they are especially powerful when the AI ​​works on itself, developing a new kind of self-awareness of its shortcomings.

You used the term open – does this have a specific technical meaning?

It is. In fact, Tim Rocktachel, one of our founders, led the openness and self-development teams at Google DeepMind and worked specifically on the Genie 3 global model, which is a great example of openness. You can tell it any concept, any world, any agent, and it just creates it, and it’s interactive.

In biological evolution, animals adapt to the environment, and then other animals counter-adapt to those adaptations. It’s just a process that can evolve over billions of years, and interesting things keep happening, right? This is how we developed our eyes [heads].

Another example is Team Rainbow, from another paper from Tim. Have you heard about the red team?

In cybersecurity means—

Therefore, red teaming should also be implemented in the context of LLM. Basically you’re trying to get LLM to tell you how to build a bomb, and you want to make sure it doesn’t.

Now, humans can sit there for a long time and come up with interesting examples of what AI shouldn’t say. But what if you tested this first AI with a second AI, and the second AI now has the task of creating the first AI [try to] Say all the possible bad things. Then they can go back and forth for millions of iterations.

You can actually allow two AI systems to evolve together. One keeps attacking the other, and then comes not just at one angle, but at many different angles, hence the rainbow analogy. Then you can vaccinate your first AI, and become safer and more secure. This was an idea from Tim Rocktachel, and it is now used in all major laboratories.

How do you know when it’s done? I suppose it was never done.

Some of these things will never be done. You can always get smarter. You can always get better at programming, math, etc. There are some limits to intelligence; I’m actually trying to formalize it now, but they are astronomical numbers. We are very far from those limits.

As a new tester, it feels like you’re supposed to do something the bigger labs don’t do. So part of the implication here is that you don’t think the major coefficients are going to hit the RSI [recursive self-improvement] By doing what they do. Is that fair to say?

I can’t really comment on what they do, but I think we approach it differently. We truly embrace the concept of openness, and our entire team is focused on this vision. The team has researched this matter and prepared research papers in this field over the past decade. The team has a proven track record of significantly moving the industry forward and shipping real products. As you know, Tim Shea turned Krista into a unicorn. Josh Tobin was one of the early people at OpenAI, eventually leading the Codex and Deep Search teams.

In fact, I sometimes struggle a little with this Newlab class. I feel like we’re not just a laboratory. I want us to become a truly viable company, to have truly amazing products that people love to use, and that have a positive impact on humanity.

So when do you plan to ship your first product?

I’ve thought about it a lot. The team has made a lot of progress, and we may actually be able to change the timelines from what we initially assumed. But yes, there will be products, and you will have to wait quarters, not years.

One idea about recursive self-improvement is that once we have this kind of system, computation becomes the only important resource. The faster you get the system up and running, the faster it improves, and no outside human activity will make a real difference. So the race becomes: How much processing power can we throw at this? Do you think this is the world we are headed towards?

Arithmetic should not be underestimated. I think in the future, the really important question will be: How much computing does humanity want to spend on solving any problem? This is cancer and this is the virus, which do you want to solve first? How much account do you want to give him? It ultimately becomes an issue of resource allocation. It will be one of the biggest questions in the world.