GovernSafe beat PentestGPT and Strix in a controlled black-box benchmark

GovernSafe won our controlled black-box test against OWASP Juice Shop 20.1.1. The margin was not close.

Starting from the same clean target and receiving only a base URL, GovernSafe triggered 27 of 113 externally scored challenges. PentestGPT triggered 15 of 113. Strix triggered 5 of 113 observed, although its result did not pass its predeclared rateability rule because one observer poll timed out.

GovernSafe triggered 80% more challenges than PentestGPT and reached 5.4 times the observed Strix count. RidgeGen's separate published benchmark reached 21/110. GovernSafe reached 27/113.

These were not model-written vulnerability lists dressed up as results. A separate observer measured the target state. GovernSafe also accepted 35 evidence-backed findings with zero validator errors.

GovernSafe won by 80%

All three systems were tested against the same frozen OWASP Juice Shop 20.1.1 image. The application was reset to 0/113 before each scored run. Every engine started with the base URL and a generic authorised black-box instruction.

No system received source code, challenge names, known credentials, expected findings, previous run artifacts, or observer output.

PentestGPT had access to GPT-5.6 Sol. GovernSafe used GPT-5.6 Sol as a constrained resolver alongside its own 1.7B planning model. GovernSafe still triggered 12 more challenges.

The model does not explain away that gap. The engine does.

The engine made the difference

GovernSafe is not a chat loop attached to a shell. It is a purpose-built black-box pentest engine that controls the assessment from reconnaissance through validation.

GovernSafe separates the assessment into coverage, execution, validation, and evidence. The planning layer decides where to spend effort. The runtime keeps target state, identities, requests, and candidate attack paths available across the engagement. Validators decide whether an observation is strong enough to publish.

Five design choices mattered in this test:

1. Coverage before guesswork. GovernSafe maps routes, browser flows, forms, APIs, assets, and authenticated surfaces before spending heavily on exploit attempts.
2. Multiple application identities. The engine can hold separate user sessions and compare what one identity can read or change in another identity's context.
3. Typed candidate resolution. GPT-5.6 Sol does not receive unrestricted authority. It orders bounded exploit candidates that the engine can execute and verify.
4. Evidence-gated findings. A model cannot confirm a vulnerability by writing persuasive prose. GovernSafe needs request, response, browser, or scanner evidence tied to a validator.
5. Independent scoring. The benchmark observer sat outside the engine environment. GovernSafe could not read the challenge catalog or its live score.

GPT-5.6 Sol is not the product. It is one bounded component inside GovernSafe. The engine decides what to test, preserves state across the assessment, executes the work, and demands proof before a finding survives.

That orchestration is the novelty, and it showed up on the scoreboard.

GPT-5.6 Sol added the final six

GovernSafe's core engine was already reaching 21/113. GPT-5.6 Sol helped take the result from 21/113 to 27/113 as a tightly constrained candidate resolver.

Its job was specific: help the engine choose and order typed candidates where stronger reasoning could improve the next action. It did not control the whole assessment, confirm findings, or write evidence into existence.

GovernSafe's own 1.7B planning model handles broad orchestration. GPT-5.6 Sol sharpens selected exploit decisions. Deterministic validators retain final authority.

Six extra triggers matter. So do the first 21. GovernSafe uses frontier reasoning where it adds value without handing over the assessment or letting model confidence decide what is real.

Thirty-five accepted findings, not generated claims

GovernSafe produced 35 accepted findings in the winning run. Every accepted item was tied to a typed validator or matcher-backed result, and the run recorded zero validator errors.

The externally observed challenge count stayed separate at 27. That distinction matters because one security weakness can create several pieces of evidence, while a valid finding may not map to a Juice Shop challenge key.

PentestGPT shows why generated report totals are a weaker benchmark. Its vulnerability stage claimed 19 findings, while its final report consolidated them to 10. We did not use either number as an independently accepted finding count. Its 15 externally observed challenge triggers are the comparable measure.

GovernSafe's reporting pipeline starts from evidence and then drafts the explanation. It does not ask the report to decide whether the vulnerability was real.

The black-box controls

The benchmark was built to stop the engine from learning its score or reading the answer set.

The engine, testing environment, and target images stayed pinned throughout the run. The final score came from the external observer, not from GovernSafe's own report.

Where GovernSafe created the strongest coverage

GovernSafe's strongest challenge coverage came from generic behaviours that matter in real web assessments:

The result includes access-control differentials, public-file discovery, configuration exposure, input validation failures, and injection paths. These were not hard-coded Juice Shop challenge handlers. The engine received a URL and used reusable capabilities and validators.

GovernSafe surpassed RidgeGen's published score

Ridge Security published its own RidgeGen vs Shannon vs Strix OWASP Juice Shop showdown in March 2026. Ridge gave each system a fresh target URL, held the model constant at Gemini 3 Flash, and used the target's challenge counter as an exploitation-progress measure.

RidgeGen led that benchmark with 21 solved challenges from a 110-challenge target. It was a strong result. GovernSafe went further, reaching 27 from a 113-challenge target.

On the published challenge-progress measure, GovernSafe recorded six more solved challenges and a 4.8 percentage-point higher raw coverage rate than RidgeGen.

The finding totals use different schemas, and RidgeGen reported 55 evidence-backed findings under its own methodology. We do not pretend unlike finding counts form a clean leaderboard. The direct score is cleaner: GovernSafe 27/113, RidgeGen 21/110.

Ridge also reported that only 10 of Shannon's 27 findings carried exploitation evidence, while Strix produced 6 evidence-backed findings. GovernSafe accepted 35 findings through its own evidence gates.

The wider agentic pentest scoreboard

Public benchmark figures are useful only when the target, access mode, denominator, and validation rule travel with the score. The table below gives market context without pretending unlike tests belong in one ranking.

PentestGPT's public XBOW result should be judged on that suite. Shannon's result demonstrates source-aware white-box testing. Escape, Aikido, and XBOW publish useful finding-based comparisons under their own conditions.

The GovernSafe, PentestGPT, and Strix table at the top is the true head-to-head result because those runs shared our frozen target and external challenge observer.

What GovernSafe customers get

A benchmark is useful when it predicts a product behaviour that customers can inspect.

For GovernSafe, that behaviour is evidence-led autonomous testing. The engine can start from an authorised URL, build the attack surface, test across application identities, execute bounded attacks, retain the request trail, and publish only findings that pass validation.

The 27/113 result is direct evidence that GovernSafe's engine produced more externally observed challenge triggers than PentestGPT and Strix in the controlled same-target test. It also beat RidgeGen's published challenge score under Ridge's separate Juice Shop benchmark.

We are now extending the same engine into deeper authenticated business logic, longer stateful exploit chains, and repeated tests on mutated and unseen applications. The benchmark harness stays outside the runtime so future gains have to be earned the same way.

Bring us an authorised URL

Give GovernSafe an authorised base URL and a clear scope. The engine will map the surface, test it, preserve the evidence, and report only what it can prove.

Talk to GovernSafe about a controlled application-security pilot.

Sources

Public sources were reviewed on 16 August 2026:

• OWASP Juice Shop project
• PentestGPT repository and benchmark history
• PentestGPT USENIX Security 2024 paper
• Ridge Security agentic AI pentest showdown
• Keygraph Shannon Open Source
• Escape AI pentesting benchmark
• Doyensec comparison of Aikido and XBOW
• Strix open-source repository

Detailed benchmark records are retained privately with observer output, proxy counters, target identity, model configuration, and integrity checks.