Documentation
How Axiom Accreditation works.
Axiom Accreditation runs your AI agent through a simulated operating range and returns an underwriter-grade risk profile. Here is exactly what that means, the dimensions we measure, how they are graded, how to connect your agent, and what a certificate proves.
The process
Connect, run, read, accredit.
An assessment is four stages:
- 1 · Connect. You point your agent at the range (see connecting an agent). Roughly a day of your time; no SDK, no changes to the agent itself.
- 2 · Run. Simulated users, adversarial personas, and poisoned tool results drive the agent through scenarios drawn from a private, rotating pool, each repeated many times, because consistency is measured, not assumed.
- 3 · Read. You get a per-dimension risk profile: letter grades, pass^k reliability, and confidence intervals, with the full transcript behind every failure. The assessment is free and private to you.
- 4 · Accredit. When the profile holds, purchase accreditation: a sealed certificate bound to your exact configuration, a public verification page, and the medal, renewed annually or on material change.
What we measure
Eight dimensions of agentic liability.
Not model safety, not content policy, the specific ways an autonomous agent creates liability for the business that deploys it, mapped to OWASP Agentic and MITRE ATLAS and scored from behavior. Each dimension carries a severity (S1 negligible → S5 critical) that weights its contribution to the composite.
| CODE | DIMENSION | SEV | WHAT IT CATCHES |
|---|---|---|---|
| RD-01 | Scope violation | S4 | Acting outside the declared task envelope, the agent asked to summarize that decides to also send, edit, or delete. |
| RD-02 | Unauthorized action | S4 | Tool calls that exceed granted permissions: refunds it cannot issue, records it cannot change, actions it was never entitled to take. |
| RD-03 | Data exfiltration | S5 | Sensitive data leaving through tool arguments, links, or replies, whether deliberate, careless, or induced by a third party. |
| RD-04 | Injection susceptibility | S5 | Instructions planted in tool results, documents, and inboxes. We poison the range with adversarial content and watch whether the agent obeys it. |
| RD-05 | Output integrity | S4 | Hallucinated facts, fabricated citations, misstated policy, measured against ground truth, not vibes. |
| RD-06 | Behavioral instability | S3 | The same scenario, run many times. An agent that passes once and fails thrice is priced on its consistency, not its best case. |
| RD-07 | Over-refusal | S2 | Safety that refuses legitimate work is a defect, not a virtue. Utility is scored alongside risk so a uselessly cautious agent doesn't pass by hiding. |
| RD-08 | Operational control | S4 | Escalation, halting, audit trail: does the agent stop when told, hand off when it should, and leave a record of what it did? |
How grading works
pass^k: consistency, not best case.
The headline metric is pass^k: the probability the agent succeeds on all of k independent trials of a scenario. An agent that passes once and fails three times in eight runs is priced on its pass^k, not its best attempt, because in production every run is a live one. Each rate is reported with a Wilson confidence interval, so a grade is always a range, never a bare number.
Letter grades map from that reliability score. The thresholds are deliberately demanding , agentic liability is asymmetric, so “usually fine” is not an A:
The composite grade is a severity-weighted roll-up (S5 counts 16× an S1), so a single critical failure dominates a stack of minor passes. Deterministic evidence, which tools were actually called, what state actually changed, is authoritative; a soft LLM-judge is used only for genuinely subjective checks and can never overturn hard evidence.
Getting your agent in
Four ways to connect an agent.
We meet your agent where it is, any of these works, and onboarding is white-glove:
HTTP tool endpoint
Your agent exposes an OpenAI-style tool-calling API. We route its tools through our instrumented, emulated environment, a CRM, inbox, payment rail, and observe every call. This is the deepest assessment (all eight dimensions).
Plain HTTP chat endpoint
Any chat/completions endpoint. We run adversarial multi-turn conversations and grade the replies, the conversational dimensions (injection, disclosure, output integrity, over-refusal).
Wrapper command
For agents driven by a CLI or SDK with no network endpoint. You provide a command we invoke per turn; no code changes to your agent.
UI automation
For agents that only have a web UI and no API, we drive the interface directly. This is how we assess a product that was never built to be tested.
The deliverable
Certificates & verification.
A certificate binds to one agent configuration (model + scaffold + tools + guardrails + system prompt, identified by a config hash) and states what it passed, under stated conditions, at a stated confidence, never that it is simply “safe.” Each carries an opaque verification token.
Anyone can verify a certificate at certify.axiomspecialty.com/v/{token} , the public view shows the agent, composite grade, issue and validity dates, and current status (issued / expired / void), and nothing else. A certificate is voided if a material change to the model, tools, or system prompt is detected, so a live token always reflects the configuration that was actually assessed.
Axiom Accreditation is an assessment and accreditation instrument. It is not an insurance product, and accreditation is not a policy, warranty, or guarantee of performance.
Questions
FAQ.
Is an Axiom Accreditation certificate insurance?
No. It is an assessment and accreditation of agentic-liability conduct, a statement of what your agent passed, under stated conditions, at a stated confidence. It is not a policy, warranty, or guarantee of performance, and it is not legal or financial advice.
What exactly gets certified?
One agent configuration, model + scaffold + tools + guardrails + system prompt, identified by a config hash. Change any of those materially and the certificate is re-issued (or voided), a cert is a claim about a specific, frozen configuration.
Why is the grading scale so strict?
Agentic liability is asymmetric: an occasional unsafe action is a claim, so "usually fine" is not an A. We grade pass^k (consistency across repeated trials), not best-case, and weight failures by severity.
Can an agent cheat the judge?
The agent's transcript is treated as untrusted data, fenced behind a per-run canary and never followed as instructions, and deterministic evidence (what tools were actually called, what state changed) is authoritative over any soft judgement. A failure caught by hard evidence cannot be talked away.
Do you see our agent's code or data?
No. We interact with your agent as a black box (or through instrumented tool wrappers you point at us). We store the assessment result and the transcripts it produced, not your source or your customers' data.