Introducing NexBench: MindFort's Internal Model Evaluation

Attackers now find and exploit 0-days faster than defenders can keep up, which is why continuous pen testing against attack surfaces that change often has become table stakes. Better models have made that testing faster, but running the same agents has become token- and cost-inefficient: they run for 8-16 hours at a time, sometimes every day. That leaves a gap between how fast attackers find 0-days and how fast teams can fix them. We built MindFort to make security efficient, performant, and abundant.

This led us to become interested in how efficient agents were when executing offensive security tasks. For us to be able to create these offensive security agents, we needed them to be capable of running for multiple hours at a time, continuously through the year, which led us to create an internal benchmark measuring models' ability to balance token efficiency and performance.

Current cybersecurity evals share one flaw: they are boxed in. Real environments are messy and ambiguous, with layered vulnerabilities that can be chained together in different ways. They come in different deployment and authentication shapes and different sizes, which makes it hard for agents to even work their way in, and that is half the battle. So we built our own, modeled on the thousands of real environments we have watched our agents work through.

Introducing NexBench. NexBench demonstrates the real-world environments that agents find themselves in. By running our agents on NexBench, we aim to understand which models can find the most validated vulnerabilities, have the best price-to-performance ratios, and exhibit long-running coherence and ability.

Our eval provides signals of a model's ability to write, reason and execute real exploits against a target, while measuring their price-to-performance ratio and how well the harness handles the complex environment. In our research, we have found that evaluations of agents in existing benchmarks far outperform the same models/agents in real-world environments.

Across five recorded runs each, GPT-5.6 Sol had the most validated findings with 87. Opus 4.8 followed with 68 (Figs. 1 and 2).

Note: Best score and run time come from one run per model. Findings, cost, and tokens are totals across all recorded runs. The Runs column shows how many each model has. Click any column header to re-sort.

For example, GPT-5.6 Sol's best-scoring run had 32 accepted findings, including 11 high-severity findings, for 61 points. Its totals across five runs were 87 accepted and 40 high-severity findings. Grok 4.6 has one recorded run: 44 accepted findings, including 26 high-severity findings, for 114 points.

Where it starts getting interesting is when we start to implement scoring, assigned based on severity of the finding. We assigned a score of 1 to low findings, 2 for medium findings, and 3 for high/critical findings. In general, critical and high findings were harder to find, since they required chaining multiple vulnerabilities together, which makes this metric a good read on overall performance in offensive security workflows. Each run was scored as shown below. The leaderboard reports each model's highest score across its recorded runs, which range from one to five.

Grok 4.5 scored 52, below GPT-5.6 Sol's 61 and above Opus 4.8's 45. It took the full runtime to reach that score. The later Grok 4.6 and 4.7 runs scored 114 and 70. Kimi K3 reached 42 in 49 minutes (Fig. 3).

GPT-5.6 Sol had the highest cost per accepted finding in our evaluations, followed by Sonnet (Fig. 4). GLM-5.3 Flash, Grok 4.6, and Kimi K2.7 had the lowest costs per accepted finding.

Looking at token and cost efficiency, we can then deduce which models were able to make the most of token and dollar cost. Our x-axis measures validated points per USD (cost efficiency as a ratio), with our y-axis measuring points per million tokens (token efficiency). Kimi K2.7 ended up with one of the best dollar-to-point ratios, but token use inefficiency ended up weighing down overall results (Fig. 5). Among the hosted models, Grok 4.5 performed in the inverse, with strong token efficiency, but was more expensive than Kimi K2.7, along with GLM-5.2. The standout on token efficiency, though, is Qwen3.8 27B, run locally on our own hardware: it validated 45 findings on just 6.85M total tokens. That works out to roughly 11.5 points per million tokens, more than twice the rate of any hosted model, at about $1.14 per validated finding.

Kimi K3 ended up scoring similarly to Grok 4.5 in terms of token efficiency (2.35 points per 1M tokens vs. 2.38 points per 1M tokens) but scored significantly better in terms of dollar efficiency, at roughly double the points per USD, making it a really cost-efficient choice.

Each run appears once by score and standard cost (Fig. 6). The Pareto frontier shows the highest score reached at or below each cost.

The same runs appear by score and total tokens (Fig. 7). This view shows which scores were reached at different token budgets.