A checklist of OWASP LLM Top 10 categories is not the same thing as a scan of your agent.
Most tools in this category run the same fixed prompt library against every customer, regardless of what the agent actually does. That's not a small implementation detail — it's the reason the findings below exist and a generic scan never surfaces them.
Same attack category.
Different probe. Different outcome.
This is a real finding from a real customer scan. The generic version on the left is what a fixed-template scanner would have sent — the same category, tested the way most tools test it.
What's actually different
under the hood.
Not every category below is unique to Orithos — some generic scanners cover parts of this. Where that's true, we say so.
| Capability | Orithos | Generic LLM scanners |
|---|---|---|
Probe generation source Where the actual attack text comes from | Generated from your specific system prompt, tool schemas, and RAG sources | Fixed prompt library, identical across every customer |
RAG supply-chain injection Attacks delivered through documents the agent retrieves, not just chat input | Dedicated category, tested against your declared RAG sources specifically | Not modeled — most tools only test the chat input channel |
Tool abuse / parameter limit bypass Whether a declared £5,000 limit is actually enforced | Probes crafted against your actual tool parameter schema and blast radius | PARTIALGeneric tool-calling tests exist, not scoped to your specific parameters |
Judge false-positive rate How a CRITICAL finding is confirmed before it reaches you | 3-tier cascading judge, CRITICAL requires top-capability model confirmation | PARTIALMost use a single classifier pass, higher false-positive risk on severity |
Guardrail-aware bypass testing Testing your declared Nemo/Llama Guard stack specifically | Generates bypass probes targeted at your declared guardrail configuration | Not offered — guardrails aren't a declared input to most scanners at all |
EU AI Act Article 9 evidence export Audit-ready documentation, not just a PDF report | Full Article 9/13/14/15 mapping with control-level evidence citations | Generic compliance badge claims, rarely article-level evidence |
Remediation generation A fix you can actually deploy, not just a description of the problem | Deployable Nemo Colang, Llama Guard prompts, hardened system prompt diffs | Findings report only — remediation is left to your engineering team |
Fix verification loop Confirming a deployed fix actually closed the finding | Re-runs the exact original failing probes against the fix, confirms RESOLVED | Not offered — "re-scan and hope it's fixed" is the default |
CI/CD gate integration Blocking a deployment automatically on a CRITICAL finding | Native GitHub Actions / GitLab gate, sub-5-minute targeted probe subset | PARTIALSome offer a webhook, few offer a fast targeted probe subset for gating specifically |
Open, published probe catalog Can you actually see what's being tested before you connect production? | Full 25-category, 900+ probe catalog published, MIT licensed | Closed-box — probe content undisclosed, "trust our methodology" |
This isn't an abstract
feature comparison.
A generic scan that returns “no findings” doesn't mean your agent is safe. It means the specific attacks in that tool's fixed library didn't work. Here's what that gap actually costs.
Run both scanners on the same agent. Compare the findings yourself.
We're not asking you to take this comparison on faith. Connect your agent, run a free first scan, and put the results next to whatever you're already using.
This comparison describes category-typical generic scanner behaviour, not any single named competitor.