Introducing NexBench: MindFort's Internal Model Evaluation

In a time where 0-days can be found and exploited faster by attackers, it's more important than ever to run continuous pen-testing, against all attack surfaces that change often. With rapid advancements in model performance, this process has gotten faster, but running those same agents has also become token and cost inefficient, as they are running for 8-16 hours at a time, sometimes every day. There now becomes a disparity between the security risk of attackers finding these 0-days and the remediation needed to fix them. Efficient, performant, and abundant security became our end goal while creating MindFort.

This led us to become interested in how efficient agents were when executing offensive security tasks. For us to be able to create these offensive security agents, we needed them to be capable of running for multiple hours at a time, continuously through the year, which led us to create an internal benchmark measuring models' ability to balance token efficiency and performance.

Current evals for cybersecurity have a fatal flaw: they are boxed in. Real world environments are messy, ambiguous, and contain layered vulnerabilities that can be chained together in different ways. They also have various deployment and authentication shapes, as well as various sizes which make it challenging for agents to actually navigate through, which is half the battle. That is why we created our own, modeled off the thousands of real world environments we have observed our agents navigate through.

Introducing, NexBench. NexBench demonstrates the real world environments that agents find themselves in. By running our agents on NexBench, we aim to understand which models can find the most validated vulnerabilities, have the best price to performance ratios, and exhibit long running coherence and ability.

Our eval provides signals of a model's ability to write, reason and execute real exploits against a target, while measuring their price to performance ratio and how well the harness handles the complex environment. In our research, we have found that evaluations of agents in existing benchmarks far outperform the same models/agents in real world environments.

On raw performance, GPT-5.6 performed best with 87 validated findings on its best run. Opus 4.8 came in second with 68 total findings (Figs. 1 and 2).

Note: The leaderboards below rank all eleven models by the highlighted column, each shown at its best run across every reasoning effort. No model or effort is dropped from the evaluation. Click any column header to re-sort.

Where it starts getting interesting is when we start to implement scoring, assigned based on severity of the finding. We assigned a score of 1 to low findings, 2 for medium findings, and 3 for high/critical findings. In general, critical and high findings were harder to find, since they required chaining multiple vulnerabilities together, which makes this metric a showcase of overall performance in offensive security workflows. Each score was calculated as shown below and the greatest score out of 3 runs per model was used.

When weighting severity against each finding, Grok 4.5 did impressively well, scoring just below GPT-5.6 Sol, beating out Opus 4.8. It is important to note that Grok did take the full runtime to be able to produce these results, more than double of Opus, Luna, Sol, etc, but given a reasonable amount of time where a pentest should have been completed, was able to outperform most models at a very efficient token rate. Another highly time efficient model was Kimi K3, scoring a 42 in 49 minutes (Fig. 3).

GPT-5.6 Sol also scored as the most expensive model when looking at cost to accepted finding in our evals, with Sonnet coming in second (Fig. 4). Surprisingly, Kimi K2.7, K3, and GLM-5.2 all scored the best in terms of cost per finding, which we will explore later in regards to overall efficiency.

Looking at token and cost efficiency, we can then deduce which models were able to make the most of token and dollar cost. Our x-axis measures validated points per USD, cost efficiency as a ratio, with our y-axis measuring performance per token, token efficiency. Kimi K2.7 ended up having the best dollar to finding ratio, but token use inefficiency ended up weighing down overall results (Fig. 5). Grok 4.5 performed in the inverse, with the best token efficiency, but was more expensive than Kimi K2.7, along with GLM-5.2.

Kimi K3 ended up scoring similarly to Grok 4.5 in terms of token efficiency (1.03 findings per 1M tokens vs 1.02 findings per 1M tokens) but scored significantly better in terms of dollar efficiency, at more than double the findings per USD, making it a really cost efficient choice.

Modeling score to cost in dollars, we can model the pareto frontier of the above models, based on available budgets. Grok on high ended up scoring best, with an additional $300 in spend needed by Sol on high effort to surpass Grok's performance on total points discovered (Fig. 6).

A similar pattern can be observed when modeling the pareto frontier of token cost to findings, with each Grok model appearing along the frontier, no matter the reasoning effort. Additionally, K3 scored on the pareto frontier for both cost and token efficiency (Fig. 7).