SWE-1.7: Frontier Intelligence at a Fraction of the Cost

At Cognition, we have been formulating and refining principles for good agentic software engineering both in evaluation, with FrontierCode^1,2, and now in training, with SWE-1.7. Our model is particularly optimized for longer-horizon asynchronous tasks, an important component of high-quality software engineering.

SWE-1.7 is available today in Devin (Web, Desktop, and CLI) via Cerebras at 1000 TPS. We encourage you to try it for yourself!

The rest of this post covers how we trained SWE-1.7: the infrastructure, algorithms, and data work behind our model. We cover four important components that stand out.

• Preserving entropy and stabilizing training: Long RL runs face two challenging problems: entropy collapse, and instability due to numerical drift between training and inference. We hunted down and addressed causes of each, which enabled training to keep improving well past where earlier runs stalled.

• Multi-cluster training and fault tolerance: RL doesn’t need all of its inference compute in one cluster. We trained on clusters across three continents, shipped weight updates through object storage, and built fault tolerance so that hardware failures never stalled the run.

• Curating high-quality data: We built an extensive data-quality pipeline that runs each task through automated execution tests, filters out tasks with low learning signal, and hardens tasks to prevent reward-hacking.

• Self-compaction for long-horizon tasks: The model learns to summarize its working state and resume from the summary, extending task horizons past the raw context window. We use an alternating length penalty to incentivize concise output without sacrificing correctness.

Finally, we conclude by sharing some observations on interesting behavioral tendencies, such as careful exploration and concise reasoning, that the model acquired as a result of our training setup.

We found training stability to be a key contributor to predictable improvement at scale.

When training with asynchronous RL³, one of the most problematic issues we encountered was the KL divergence mismatch between inference and training⁴, since the trainer policy is usually different from the sampling policy. In the past, to correct for this (albeit at smaller scale), we used importance-sampling⁵ and quantization-aware training for low-precision rollouts in NVFP4 + experts routing replay^6,7.

Here we present additional interventions that become more important at larger scale.

We find that top-p sampling⁸ contributes significantly to staving off entropy collapse^9,10, where a strong model stops exploring and reward plateaus within a few hundred steps.

Very low probability tokens are often part of trajectories that have gone off track or out of distribution. These trajectories are likely to produce low reward, and properties of the softmax function lead to these tokens sharpening the token probability distribution. Indeed, suppose we have three tokens with logits $x_1>x_2\gg x_3$

and the policy gradient update to the logits is $\Delta x_i \propto \hat{A}\ \nabla \log p_3$

In these updates, $x_3$

Policy entropy across training

This entropy-preservation effect makes top-$p$ sampling desirable in our rollouts. But naively implementing top-$p$ clearly increases the training-inference mismatch — the trainer computes probabilities as a selection from all tokens, while rollouts sample from the top-$p$ subset, so the distributions have higher divergence, leading to collapse after a small number of steps. Thus, we implement sampling distribution replay¹¹, where we record a kept-set of tokens available for sampling at rollout time, and renormalize probabilities with those masks in the trainer. With this fix, our run’s entropy stays roughly constant over the course of training and inference-training divergence stays bounded.

Training-inference mismatch across training

Another interesting result of using top-p sampling replay is a targeting of only tokens with $p < \text{top\_p\_threshold}$

We also find benefits from using the Muon optimizer¹² and eliminating non-deterministic operations in the trainer.

The central challenge in this setup is keeping all inference engines up to date with the trainer weights after each optimizer step. We want these weight updates to be fast to reduce staleness of trajectories so we can train with more aggressive learning rates.

Naively broadcasting the full model from one cluster to another would be slow and inefficient. Instead, every K gradient steps, we compute and send a compressed weight delta between the current and previous weights, reducing the size of each transfer by over 99%¹³.

Rather than streaming the weights directly from the trainer to every inference cluster, we use cloud object storage to maintain a single source of truth for weight versions. After the trainer uploads a new weight delta, the inference engines can be updated with almost no inference downtime. Each training run has a weight controller in each involved cluster that manages the run’s weight version lifecycle. The weight controller polls object storage for new manifests, which the trainer writes after each update. When it finds a new delta, it instructs workers to download their shards, which are then replicated across local disks using a tree broadcast. The same object storage also carries routing matrices and top-p masks from the inference engines back to the trainer.

Each inference engine prefetches the delta into CPU memory while continuing to serve trajectories. Only once the delta is fully staged does the engine briefly pause to apply it in-place. Trajectories that are in-flight can simply continue on the new weights with their KV cache intact.

With this approach, cross-continental weight updates for a 1T parameter model complete in 1–2 minutes end-to-end. This happens asynchronously and blocks neither training nor inference beyond 3–4 seconds of inference pause at update.

At large scale, hardware failures occur continuously, and globally restarting on each failure makes long runs infeasible. Our architecture handles this differently depending on where the failure occurs — the inference engines or the trainer.

Failures on the inference side are cheap by construction. Engines are self-contained and hold no state beyond the current weights, so a dead engine costs only its in-flight sessions. We use NVIDIA Dynamo to manage the engine lifecycles and route inference: each agent sandbox has its own proxy that records tokens in and out, so if a replica goes down, we don’t lose the full trajectory, and Dynamo reroutes it to a different worker. When Dynamo reschedules the replica on healthy nodes, our weight controller loads the most recent checkpoint from object storage and replays a series of deltas from the checkpointed version.

The trainer is the one place where a failure is expensive: it’s the single tightly-coupled component, where one dead node stalls the whole cluster. To make recovery fast, each node checkpoints asynchronously to local disk every step and replicates its shards to peers, so a dead node’s state is rebuilt from replicas in seconds. If capacity is still missing, the run shrinks by whole data-parallel replicas and regrows once nodes return. Throughout this process, the rollout pipeline remains warm. After the trainer restarts, a buffer policy selects which accumulated rollouts to use and prevents bias from any imbalance in training-inference throughput during the interruption.

From the start, we built Devin for completing asynchronous, long-running tasks. SWE-1.7 is trained directly in the Devin harness, so naturally we want to train our own model on longer horizon tasks. This introduces two challenges. First, rollouts can extend far beyond the raw context window. Second, as shown by DeepSeek R1¹⁴, RL on reasoning tasks tends to produce progressively longer responses, but we want the model to be efficient in its reasoning and only elaborate on difficult tasks.

We address these issues with training for self-compaction and an alternating length penalty.

• When an agent approaches the context limit, we ask it to summarize its working state, and we resume it from its self-authored summary. During training, the model simultaneously learns (1) to write more informative, succinct summaries, and (2) to better work from and leverage such summaries. We first introduced a version of this approach in Kevin¹⁵, where we explored it for kernel optimization tasks. With self-compaction, our rollouts during the SWE-1.7 training run reach up to six hours in duration.
• Rather than applying a length penalty uniformly throughout training, we use an alternating strategy¹⁶. In unconstrained phases, the model optimizes only for task success. In budget phases, we penalize solutions that exceed a certain budget of our weighted cost function that includes tokens, turns and total time spent in tool calls. With this structure, response length tends to compress on tasks within the model’s ability, while long-horizon behavior on hard tasks is preserved.

Data is the core determining factor in what capabilities and skills our model learns. As such, we ensured that the data we trained on was calibrated and sufficiently difficult, and disincentivized undesirable behaviors to keep the model well-aligned. We focused most on the following aspects:

• Verifier quality: A task’s verifier can be wrong in two directions: accepting incorrect solutions (false positives) or rejecting valid ones (false negatives). We devised extensive quality-assurance pipelines to minimize observations of false positives and false negatives in our training.

• Difficulty: On tasks where the model always solves or always fails, we don’t observe any meaningful learning signal. Instead, we curated training data that the model only solves a low fraction of the time, which generates real learning signal while at the same time pushes model intelligence.

• Cheating detection and prevention: We employed a variety of defenses against different forms of cheating. For instance, we network-restricted our sandboxes and stripped them of git history and reference artifacts. We also isolated the grading path from the agent itself. In addition, we employed programmatic checks to catch known exploit signatures. Finally, to ensure proper incentives, we assigned reward 0 to trajectories with any instance of cheating attempts, regardless of whether they succeeded.

Due to extensive RL, SWE-1.7 exhibits noticeably different behavior from Kimi K2.7 Code, its base model. Firstly, it is significantly more aligned and trustworthy than K2.7 or other frontier open-source models. We expand on this extensively in our companion blog post, Measuring the Trustworthiness of Open-Source-Derived Models.

One behavioral difference we noticed in SWE-1.7 is condensed chain-of-thought. Compared to Kimi-K2.7-Code, SWE-1.7’s first chain-of-thought has a much lower function-word ratio (fraction of words that serve as grammatical “glue”) and nearly half the average number of words per sentence. We think this was influenced directly by the budget phases in our alternating length penalty. We’ve included a couple of examples of condensed chain-of-thought in the dropdown below.

The other major behavioral difference we observed is that SWE-1.7 explores the codebase much more thoroughly before acting, as you can see in the number of tool calls, file reads, and searches the model executes.

This shows up most clearly in bug-fixes. A bug report typically describes one primary symptom, but the underlying issue often affects a larger surface area. SWE-1.7 is much more likely to investigate the root cause of the bug and consider edge cases, hypotheticals, adversarial inputs, and beyond-the-ask requirements than Kimi-K2.7-Code. Through its enhanced codebase exploration, SWE-1.7 also does a much better job understanding the exact design decisions that need to be made. In addition, we’ve observed that SWE-1.7 tends to settle ambiguous semantics by experimenting and probing, for example by writing small Python scripts, rather than guessing.

How often the chain-of-thought probes edge cases and hidden requirements

We believe these behaviors arise directly from the extensive quality-assurance measures we took to strip out false positives and false negatives in our data, forcing our model to come up with more complete, end-to-end solutions. We think that SWE-1.7’s increased due-diligence directly translates to higher performance on various benchmarks. We’ve attached a couple of example trajectories in the dropdown below.

The extra thinking comes at a small cost in increased change scope. As described in FrontierCode¹, a good solution modifies only the minimal set of files needed, without touching unrelated code or introducing unnecessary refactors. Since SWE-1.7 reasons more, it also does more: writing additional test cases and touching more files than the task naively requires. We’ve noticed this trend consistently in models across the industry: as reasoning increases, the scope of files that the model touches also expands. This is an axis we’re excited to improve on.