← All posts

Cyber Agents at Ultra Fast Speed

· 5 min read · Article

Same model, same agent, same target, different inference. We pitted Kimi K3.1 Pro against itself in CyberArena, a live attack-and-defense benchmark. One side ran on Lithos Ultra, about 5× faster than the same model on a major inference provider. The Lithos agent breached first, patched first, and won all three matches 2–0 without losing a flag.

In cybersecurity, speed now decides the outcome. AI agents can find and exploit real vulnerabilities on their own. When an attacker's agent and a defender's agent work on the same system at once, the one that thinks and acts faster wins.

Agent cyber capability is accelerating

The UK AI Security Institute (AISI) estimates that the length of cyber task frontier models can complete now doubles about every 4.7 months, down from 8 months a year earlier (AISI).

The newest models are running ahead of even that trend. They have found thousands of high-severity zero-days across every major operating system and browser (Anthropic). In September, OpenAI rated its Astra model the first to reach its Critical cyber threshold: it can find new zero-days and chain them into working exploits (OpenAI).

Open-weight models are close behind. In July, AISI put GLM-5.2 and DeepSeek V4-Pro 4 to 7 months behind the closed frontier on cyber tasks, down from 6 to 10 months through 2025 (AISI, July 2026). AISI estimated that one DeepSeek V4-Pro run on its cyber range cost about $1, against roughly $85 for a frontier closed model. CyberArena's October results go further. Open-weight GLM-5.3 placed third of 12 agents, ahead of several closed frontier models.

The same skills attack and defend

Attackers and defenders start the same way: find the bug. The attacker then writes an exploit. The defender writes a patch that closes the hole without breaking the service. CyberArena finds the two skills strongly correlated.

The contest is still lopsided. An attacker needs one working route. A defender has to close every route, keep the service up, and do it before the next exploit lands. In CyberArena, defenders often fixed one path and missed another, or declared a fix done based on stale evidence.

Why speed matters

An agent works in a loop of think, act, observe, repeat. Every step waits on the model, so inference sets the pace of the whole loop. Two numbers drive it: time to first token and generation speed in tokens per second.

With the same clock, a faster agent gets more loops. That means more probes, more hypotheses tested, and more patches written and verified. On a static benchmark, speed only shortens the run. Against a live opponent, it decides who breaches first and whether the patch lands in time. CyberArena sees this among frontier models too. The top model's defensive lead came largely from speed, with a median first patch at 2.3 minutes against 3.9 to 12.3 minutes for the rest.

For security leaders, the takeaway is that inference latency is now a security property, not just a statistic that shows up on a dashboard.

From static benchmarks to a live arena

Most cyber benchmarks give an agent a fixed target and no opponent. Frontier models now clear many of them, so they no longer separate the top models. They also never test defense under pressure.

BenchmarkExploitPatchLive opponent
NYU CTF Bench, Cybench, CVE-BenchYesNoNo
CyberGymProof of conceptNoNo
BountyBenchYesSeparate taskNo
CyberArenaYesYes, at the same timeYes

CyberArena was released in October 2026 by 22 researchers from UC Berkeley, UC Santa Barbara, UC Santa Cruz, MIT, Carnegie Mellon, Johns Hopkins, the University of Toronto, UW–Madison, Purdue, Texas A&M and the 2077AI Open Source Foundation.

Each match gives two agents identical copies of a vulnerable service. Each agent attacks the other's copy and defends its own, with no hints. Flags rotate every round and patches carry forward. A health checker penalizes any patch that breaks the service. Challenges come from DEF CON, saarCTF and FAUST attack-defense finals, and from real CVEs in software such as Jenkins and etcd.

Same model, different speed: our results

We ran three CyberArena matches on a web-service challenge (rceaas). Both sides ran Kimi K3.1 Pro in the same agent harness, OpenCode at high reasoning effort. One side was served by Lithos Ultra and the other by a major inference provider.

CyberArena: Kimi K3.1 Pro on Lithos Ultra versus the same model on a major inference provider.
MatchResultFirst breachSpeed (tok/s)Model calls
1Lithos 2–07 s vs none406 vs 82159 vs 25
2Lithos 2–09 s vs none351 vs 71138 vs 22
3Lithos 2–07 s vs none429 vs 73208 vs 36

Bold values are Lithos; the others are the standard agent. Speeds are medians measured inside the match from streamed tokens. First breach is counted from the start of round 1.

What the replay shows (match 1):

  1. Lithos captures the first flag 7 seconds in, while the other agent is still on its first few model calls.
  2. Lithos patches its own service 25 seconds in, then twice more. The standard agent's first patch lands two minutes into the round.
  3. Fresh flags in round 2: Lithos captures again in under 2 seconds.
  4. The standard agent never gets in. All 12 of its attacks failed.

The model, harness and challenge were the same on both sides. Inference speed was the only variable, and it decided every match.

Kimi Ultra is live

Attackers' agents will run as fast as their inference allows, and defenders' agents need to keep up. Kimi K3.1 Pro on Lithos Ultra is live today. It generates about 5× faster than standard serving, with time to first token under half a second, on the same model.

DeepSeek V4.1 Flash and other leading open-weight models are also available on Lithos Ultra.

If you're building or evaluating security agents, we'd like to hear from you.

Sources