The PR splits the agent into two roles: an actor LLM proposes a tool call, and a separate guardrail LLM classifies the action before execution. The guardrail sees the current action, recent action history, and optional ToolShield safety experiences.
Nothing becomes the default automatically. Existing agents keep using the current analyzer unless callers opt into this one and wire it to a confirmation policy such as ConfirmRisky.
The current LLMSecurityAnalyzer trusts the same model that wants to run the tool call to self-label its risk.
Reconnaissance, staging, and final execution can each look harmless in isolation.
ToolShield supplies distilled safety experiences from sandbox red-teaming so the guardrail can recognize known bad tool patterns.
The analyzer sits between proposed action and execution. It assigns a risk level; the existing confirmation policy still decides whether to pause.
flowchart TD
User["User task"] --> Actor["Actor LLM proposes tool call"]
Actor --> Event["ActionEvent: tool, summary, thought, arguments"]
Event --> Render["Render current action + recent history"]
Experiences["Optional safety_experiences"] --> Render
Render --> Guard["Separate guardrail LLM"]
Guard --> Risk{"SecurityRisk"}
Risk -->|LOW / MEDIUM| Allow["Default ConfirmRisky allows"]
Risk -->|HIGH| Pause["Pause for confirmation"]
Risk -->|UNKNOWN| Unknown["ConfirmRisky(confirm_unknown=True) pauses"]
Allow --> Tool["Tool executor"]
ToolShield is not embedded into the SDK as a hard dependency. The SDK accepts safety_experiences as a plain string. If the optional extra is installed, helper functions load and format ToolShield's bundled JSON guidelines.
safety_experiences=""The default for the new analyzer. It still separates actor and judge, but no tool-specific experience text is injected.
default_safety_experiences()Loads terminal and filesystem experiences from the optional package. This is explicit opt-in, not construction-time magic.
load_safety_experiences([...])Callers choose the tool experience names, such as postgres-mcp, playwright-mcp, or notion-mcp.
auto_detect_safety_experiences()Uses ToolShield's MCP scanner to inspect localhost ports and load matching bundled experiences for discovered MCP servers.
| Guardrail result | SDK result | Why it matters |
|---|---|---|
RISK: LOW | SecurityRisk.LOW | Normal safe path; no confirmation under default ConfirmRisky. |
RISK: MEDIUM | SecurityRisk.MEDIUM | Allows callers to configure a stricter threshold without forcing all users to pause. |
RISK: HIGH | SecurityRisk.HIGH | Default ConfirmRisky pauses before execution. |
| No parseable label, conflicting labels, or LLM infrastructure error | SecurityRisk.UNKNOWN | Lets ConfirmRisky(confirm_unknown=True) pause without making UNKNOWN dominate ensemble max-risk fusion. |
A sequence such as inspect permissions, stage a helper, then delete protected data may not look like one obvious bad action until the history is evaluated together.
Filesystem, terminal, database, browser, and Notion tools fail differently. ToolShield experiences give the guardrail a compact memory of those failure modes.
The implementation wraps summary, thought, and arguments in structural tags, HTML-escapes field content, and strips untrusted spans before parsing labels.
If the guardrail says both RISK: LOW and RISK: HIGH on standalone lines, the parser returns UNKNOWN instead of letting the last label win.
flowchart LR
subgraph SDK[software-agent-sdk]
A[toolshield_llm_analyzer.py]
H[toolshield_helpers.py]
I[security/__init__.py exports]
P[pyproject optional extra]
T[test_toolshield_llm_analyzer.py]
end
subgraph Package[toolshield optional package]
E[ExperienceStore]
J[experience JSON files]
S[mcp_scan.py]
end
H -->|lazy import| E
H -->|loads| J
H -->|auto-detect| S
H -->|returns prompt string| A
A -->|returns| R[SecurityRisk]
I --> A
I --> H
P --> Package
T --> A
T --> H
Package provenance: Earlier review found ToolShield wheels did not reproduce from source and toolshield auto was broken in PyPI 0.1.2. The PR now pins toolshield>=0.1.3,<0.2. The `.pr/` evidence bundle records PyPI metadata, wheel contents, source-to-package hash matches, and a fresh-venv smoke test.
Prompt/output hardening: The branch went through multiple review rounds: parse failures now return UNKNOWN, untrusted fields are tagged, summary is stripped like thought and arguments, closing-tag injection is neutralized with HTML escaping, and conflicting labels become UNKNOWN.
Edges to watch: The helper that auto-detects MCP tools probes localhost and currently depends on ToolShield's scanner behavior. Reviewers should verify the no-extra fallback contract, stdout/logging behavior, and whether a server process should ever run broad local probing implicitly.