Sources: SGLang project launches as RadixArk with $400M valuation as inference market explodes

Some members of the team responsible for maintaining SGLang, a popular open source tool used by companies like xAI and Cursor to speed up training of AI models, have moved to the recently launched commercial startup. That company, called RadixArk, was announced last August.

RadixArk, which originated as SGLang in 2023 inside the UC Berkeley lab of Databricks co-founder Ion Stoica, was recently valued at about $400 million in a round led by Accel, according to two people familiar with the matter. TechCrunch was unable to confirm the amount of funding.

The startup has previously raised angel capital from investors, including Intel CEO Lip Bo Tan, the people said.

Ying Sheng, a lead shareholder at SGLang and a former engineer at xAI, has left Elon Musk’s AI startup to become co-founder and CEO of RadixArk, according to a LinkedIn announcement she made last month. Sheng was previously a research scientist at Databricks.

RadixArk’s Ying Sheng, Accel, and Lip-Bu Tan did not respond to a request for comment.

Both SGLang and RadixArk focus on improving inference processing – essentially allowing models to run faster and more efficiently on the same hardware. Along with model training, inference represents a significant portion of the server costs associated with AI services. As a result, tools that improve the process can deliver huge savings almost immediately.

vLLM, a more mature heuristics optimization project, has also been transitioned from an open source project to a startup project. Forbes reported last month that the newly formed company had held talks about raising up to $160 million in funding at a valuation of about $1 billion.

Three people familiar with the deal told TechCrunch that Andreessen Horowitz is leading the investment in vLLM, although the final numbers for that investment remain to be seen. Andreessen Horowitz declined to comment. Simon Moe, co-founder of vLLM, called information about this round “factually inaccurate” in a statement to TechCrunch, though he declined to specify which details were incorrect.

Like SGLang, vLLM is incubated at Ion Stoica’s laboratory at UC Berkeley. Stoica, a professor at UC Berkeley, is the famous co-founder of Databricks as well as a number of other startups.

Many major tech companies already run their inference workloads using vLLM, and SGLang has also gained significant popularity over the past six months, Brittany Walker, general partner at CRV, told TechCrunch. Her company did not support either company.

RadixArk continues to develop SGLang as an open source AI modeling engine. The startup is also building Miles, a specialized framework designed for reinforcement learning, which allows companies to train AI models to become smarter over time.

While most of its tools are still free, RadixArk has begun charging fees for hosting services, a person familiar with the company told TechCrunch.

Startups that provide inference infrastructure to developers have seen a surge in funding in recent months, underscoring the continued importance of the inference layer for AI. Baseten recently received $300 million at a $5 billion valuation, the Wall Street Journal reported Tuesday. This follows a similar move by rival Fireworks AI, which raised $250 million at a $4 billion valuation last October.