Most of the hours in third-party risk go into work that is necessary and dull: reading a 90-page SOC 2 report for the exceptions and the subservice organisations, comparing a vendor’s contract with your playbook clause by clause, chasing an overdue finding for the third time, assembling the spend, risk trend and open issues before a renewal. Agents can do that work. The management question is not whether they can, but how far each one should be allowed to act before a person looks.
This post explains the autonomy dial in InsightVendors — four levels, set per agent, capped by a ceiling — and the rails around it: the outbox, the kill switch, grounding, budgets, and the requirement that every agent works without an AI model. Examples come from the demo organisation, Northwind Labs, whose vendors are fictional.
In short
- People decide; agents do the work. Agents prepare decisions — findings with page citations, scores with reasons, redlines, recommendations. People approve, change or send back.
- Four levels. L1 Draft, L2 Ask first, L3 Act and tell (can be pulled back), L4 Act. Set per agent, per organisation, capped by a tenant ceiling that defaults to L3.
- Nothing reaches a vendor unseen. Every message an agent writes waits in the outbox until a person releases, edits or cancels it.
- Numbers are never invented. A model answer containing a number that is not in the source facts is thrown away. Risk scores are arithmetic.
- Every agent works without a model. A rule-based path does the job; a model makes it better. Each run is badged Rules or Model.
Why autonomy is a business decision
An agent that reads a SOC report and raises findings changes what a reviewer sees. An agent that sends a follow-up to a vendor speaks for your company. An agent that recommends offboarding a vendor can end a relationship that a business unit depends on. These are different kinds of action with different costs of error, and the people accountable for them — the CISO, the general counsel, the head of procurement — should set how much each agent does alone.
There is also an accountability point that regulators care about. The US interagency guidance on third-party relationships, DORA and the EBA outsourcing guidelines all assume that a named function makes and owns third-party decisions. An agent can prepare those decisions. It should not quietly make them. Confirm the specifics with your compliance team; the design principle holds either way.
The four levels
| Level | What the agent does | Undo | Suits |
|---|---|---|---|
| L1 · Draft | Prepares a draft; a person does the work | Nothing to undo | Judgements with large consequences: renewals, offboarding |
| L2 · Ask first | Prepares the decision and asks a person before it acts | Rejected proposals are recorded | Anything that speaks to a vendor or changes a contract position |
| L3 · Act and tell | Acts, tells the person, and lets them pull it back | Revert from the agent run | High-volume, well-understood work: reading documents, parsing requests |
| L4 · Act | Acts on its own | Recorded in agent runs | Deterministic work where speed matters, such as re-scoring |
The level is set per agent and per organisation. A tenant ceiling, set for the deployment, caps every agent; it defaults to L3. In the demo, the Risk agent is configured at L4 but the agent catalogue shows it “Runs at L3 (ceiling)”: an administrator can raise one agent without being able to exceed what the deployment allows.
The defaults, and the reasoning behind them
| Agent | Default | Why this level |
|---|---|---|
| Intake | L3 | Parsing a request and finding duplicates is low-risk and the requester sees the result immediately |
| Research | L3 | Builds a profile from InsightPulse and public trust pages; easy to correct |
| Risk | L4 (L3 ceiling in demo) | The score is deterministic arithmetic; the agent only explains it |
| Assessment | L2 | The invitation speaks to the vendor |
| Document | L3 | High volume; every finding cites a page, and findings can be reverted |
| Contract | L2 | Redlines are a negotiating position; legal should approve them |
| Monitoring | L3 | Speed matters when a vendor has a breach; the reaction can be pulled back |
| Renewal | L1 | Renew, renegotiate or offboard is a business judgement |
| Remediation | L2 | Follow-ups to vendors are held in the outbox |
| Copilot | Read-only | Answers questions from the records; changes nothing |
The pattern is simple. Reading and computing can run at L3 or above. Speaking to someone outside the company sits at L2. Recommending the end of a relationship sits at L1. Customers can move any of these; the defaults are where we would start.
Every agent run passes the same rails before the dial decides who acts.
One vendor, four levels: Callwave
The demo’s Callwave record shows the dial working on a single vendor. Callwave is a Tier 2 sales call-recording service, $64,000 a year, 31 users. Its agent runs include four different agents at four different levels:
| Agent | Level | What happened |
|---|---|---|
| Document | L3, Model | Read the MSA, DPA and SOC 2 Type II; raised findings including the critical “No data privacy or data processing obligations” in the MSA and “OpenAI listed as sub-processor — AI data exposure risk” in the DPA. Applied. |
| Risk | L3 | Re-scored: inherent 65, residual 100. Applied. |
| Contract | L2 | Proposed redlines against the playbook. Awaiting approval. |
| Renewal | L1 | Proposed offboard: “Residual risk is 100 (critical) with 8 high/critical finding(s) open.” A draft for a person. |
The Renewal agent’s recommendation deserves a second look, and the L1 setting is what makes that possible. Its reasoning is accurate on the facts — residual 100 and eight high or critical findings open — but a residual score of 100 in the demo partly reflects untriaged findings. A person reviewing the draft can see that, triage the findings, and decide whether to renegotiate rather than offboard. That is precisely the kind of judgement the dial keeps with people.
The outbox: nothing goes out unseen
Any message an agent writes to someone outside the company — an assessment invitation, a follow-up on an overdue finding — is held in the outbox. A person releases it, edits it or cancels it. In the demo, the invitation for Quill AI’s 43-question assessment to Dana Reyes is shown waiting there. This is the single most important rail for a vendor-risk function, because tone and timing with a vendor matter, and an agent that chases a strategic supplier with the wrong message costs goodwill that is hard to recover.
The other rails
| Rail | What it does | Why it matters |
|---|---|---|
| Kill switch | Turns an agent off for the organisation immediately | When an agent misbehaves, stopping it should take seconds |
| Grounding | A model answer containing a number not in the source facts is discarded | No invented scores, dates, amounts or limits |
| Daily budget | A daily limit on each agent’s work | Cost control and protection from runaway loops |
| Idempotency | Re-running on the same facts does nothing | Retries and duplicate triggers do not create duplicate findings or messages |
| Agent runs | Every run records input, output, level, citations, approver, and Rules or Model | Auditability: who or what did what, on which evidence |
Every agent works without a model
Each agent has a real rule-based path: pattern extraction from documents, rules for routing and tiering, arithmetic for scores, templates for messages. A model, where available, improves the reading and the writing — better extraction from an awkwardly formatted SOC report, a clearer explanation — but it is never required for the agent to work. Runs are badged “Rules” or “Model” so a reviewer knows which path produced what they are reading.
This matters for three reasons. Self-hosted customers can run InsightVendors with the model of their choice or none. A model outage does not stop vendor onboarding. And the quality floor is known: if the rules path is good enough to run the process, the model is an improvement you can measure, not a dependency you cannot remove.
Where agents fall short
The rules path is plainer than the model path: it extracts fewer nuances from unusual documents and writes shorter explanations. Agents at L3 make mistakes that a person must notice and revert; the catalogue counts reversions so you can see how often. The Document agent can raise many findings on a first read — 23 for Stackforge in the demo — and someone has to triage them. Slack and Microsoft Teams approvals are planned for 1.1; today approvals happen in the console or by signed email link.
How to set the dial in your organisation
L3 for reading and parsing, L2 for anything that speaks to a vendor or changes a contract position, L1 for renewals.
Default L3. Decide whether any agent may ever act without telling someone.
Runs, applied, approved, rejected, reverted per agent in the catalogue.
An agent often reverted at L3 belongs at L2 until its rules are tuned.
If every L2 proposal is approved unchanged, consider L3 — except for vendor messages.
Record the levels and the reasons, as you would any delegated authority.
For the CISO
The ceiling is your delegated-authority limit for software. Set it as deliberately as a signing limit.
Grounding and the outbox are the two rails that stop an agent from embarrassing you.
For the vendor-risk lead
The catalogue’s reversion counts tell you which agents to trust more and which to tune.
Hours move from reading and chasing to deciding.
For the board
Agents prepare; named people decide. Every action is recorded with its evidence and approver.
No dependence on a single AI provider for the process to run.
Let agents do the reading, the chasing and the arithmetic. Keep the decisions, and the voice to vendors, with people.
Set each agent’s level as you would a signing limit, cap them all with a ceiling, hold every outgoing message, and make sure the work still gets done when the model is switched off.