Kimi K3 is competitive with Fable; Kimi K3 + Fable is SoTA.

K3 is a frontier quality open model at a fraction of the cost. Even bigger is that it complements Fable predictably, which makes it possible to get the highest quality intelligence by routing tasks.

🧭 tl;dr: We ran Kimi K3 (open) against Fable 5 (closed) on ~1,000 agentic tasks finding:

1. We achieved 93% accuracy with routing between K3 and Fable.
2. Results were up to ~50X more cost effective than Fable alone on long agentic loops, and consistently lower cost across every use case.

How We Measured

We averaged benchmarks, each aimed at a different kind of work, and ran K3 and Fable 5 through the same harness. About 1,030 tasks in all, in real agent loops.

One quick definition before we get into the results. Oracle routing is a method for measuring the best theoretical performance by running the task through each model and then picking the cheapest correct option (the cost/performance ceiling). In a practical router, you don’t get to run your task against multiple models. The router makes a prediction of which model has the best cost and quality trade off, but ultimately it’s a guess.

In this study, oracle routing demonstrated K3 is selected for 72-96% of tasks. This suggests a near-perfect router might be achievable, by learning the difference between day-to-day tasks and the true long tail of frontier work. It will require an order of magnitude more routing data, and real world performance to say definitively.

K3 is a good model.

From a 10,000 foot view, it can be easy to look at both models and call the head-to-head a tie. For example, if you look at SWE, the headline benchmark, K3 gets 92.4%, Fable 92.6%. Across the five types of tasks we benchmarked on, the two models tend to stay within a few points of each other, with Fable pulling slightly ahead on its coding-language breadth (Multi-lang).

It’s easy to stop there and say “they’re roughly even”. The news is that they have discretely better performance across different task types.

Two Models is Better than One

If you take a peek inside a single benchmark, there’s more to see than just a top-line accuracy number. Take SWE, where the two are dead even overall. If you split SWE by problem domain you can see where each model shines. K3 is sharpest on symbolic math and dev tooling; Fable wins on web & data visualization work. The same pattern runs through the multi-language set, where Fable’s breadth carries Java, Python and C++, while K3 draws even on JavaScript and Rust.

For long-horizon work at a terminal, driving a shell and prodding at systems across dozens of turns, K3 showed its true colors. It cleared a batch of tasks Fable never cracked: a 7z hash, FEAL cryptanalysis, leaked secrets, a live vulnerability, runaway async jobs.

K3 can be up to 50x lower cost on Fireworks. 🫳🎤

While quality is a near-tie at a high level, price isn’t close.

K3 is cost optimized on all work types

Put both quality and cost on one plot. K3 in blue lands to the left (the more cost-effective side) of Fable in red in all five task-families. Accuracy trades back and forth: Fable pulls ahead on multi-language, K3 on terminal and legal, the rest roughly level.

Single Models Are Wasteful and No Longer SoTA

Kimi K3 + Fable routed together unlocks their best qualities at the best price.

The single model provider, token maxxing days, are coming to an end. The task-level data says these models are specialists at very different prices. The best AI no longer comes out of a single lab, it’s a mixture of models.

What this means in practice:

1. Open as the default. A 50x lower cost open model like K3 should be your base case, since the oracle sends it most of the traffic anyway.
2. The router is your moat. A router must be tailored to your workload and learning that task/model split continuously is the best chance you’ll have at staying ahead.