Short version: the PR keeps the existing conversation card chrome and title pipeline, but swaps the expandable details body from markdown text to a React visualizer whenever the event kind is registered. Unknown tools still fall back to the old markdown/JSON rendering.
Before this PR, almost every tool call flowed through two large markdown builders: get-action-content.ts and get-observation-content.ts. That made a bash command, a grep result, a file edit, and an unknown task result look structurally similar, and unsupported observations could degrade into raw JSON.
The new system adds a small visualizer registry under src/components/features/chat/tool-visualizers/. Visualizers are ordinary React components, but they are registered by event kind and type-narrowed by defineVisualizer().
| Part | Role | Why it matters |
|---|---|---|
define.ts | Typed visualizer helper | Narrows Action and Observation unions by kind, so the component body gets the right fields. |
index.ts | Explicit registry | Imports built-ins and indexes them by action/observation kind. No magic auto-discovery in core. |
dispatcher.tsx | Runtime resolver | Looks up the event kind and returns a React node, or null to keep the markdown fallback. |
primitives/* | Shared UI pieces | Reusable CodeBlock, DiffView, OutputPane, KeyValueGrid, and path chip components. |
The key design choice is that the visualizer is not a new card type. It only replaces the card body. Titles, collapse state, success dots, confirmation buttons, grouping, and the ACP path stay where they were.
flowchart TD
A[Agent Server emits event] --> B[REST history / WebSocket stream]
B --> C[Conversation event store]
C --> D[Messages / EventMessage]
D --> E{Event shape?}
E -->|message| M[User or assistant message renderer]
E -->|action| F[GenericEventMessageWrapper]
E -->|observation| G[Find corresponding action by action_id]
G --> F
F --> H[getEventContent]
H --> I[Existing title switch]
H --> J[resolveVisualizerBody]
J --> K{Registered kind?}
K -->|yes| L[Visualizer Body React node]
K -->|no| N[Existing markdown fallback]
L --> O[GenericEventMessage details]
N --> O
I --> O
When an action event arrives first, resolveVisualizerBody(event) checks actionVisualizers.get(event.action.kind). If present, it renders <Body action={event} />. This is why an in-flight task can already show the parent agent’s query before the subagent result exists.
When an observation arrives, EventMessage looks back through the visible events for the action whose id matches event.action_id. The dispatcher then renders <Body action={correspondingAction} observation={event} />. That gives observation cards access to action-side context such as a file editor view range or the original task prompt.
sequenceDiagram
participant AS as Agent Server
participant UI as Event store
participant EM as EventMessage
participant GC as getEventContent
participant VR as visualizer registry
participant Card as GenericEventMessage
AS->>UI: ActionEvent(TaskAction)
UI->>EM: render action
EM->>GC: getEventContent(action)
GC->>VR: lookup TaskAction
VR-->>GC: taskVisualizer.Body
GC-->>Card: title + React details(query)
AS->>UI: ObservationEvent(TaskObservation, action_id)
UI->>EM: render observation
EM->>EM: find action by action_id
EM->>GC: getEventContent(observation, action)
GC->>VR: lookup TaskObservation
VR-->>GC: taskVisualizer.Body
GC-->>Card: title + React details(query + result)
Registered for ExecuteBashAction, TerminalAction, ExecuteBashObservation, and TerminalObservation.
Registered for both the old and new file-editor action/observation names.
view and create.str_replace and insert.Registered for GrepAction, GlobAction, GrepObservation, and GlobObservation.
Registered for TaskAction and TaskObservation.
The fallback remains the safety net. A visualizer can be added for one action/observation pair without migrating every tool. That is important for unsupported or newly introduced SDK tools: the UI may be less pretty, but it still renders the event.
flowchart LR
A[OpenHandsEvent] --> B{Known action/observation kind?}
B -->|Registered visualizer| C[React visualizer body]
B -->|No visualizer| D[Markdown helper]
D --> E{Known markdown switch case?}
E -->|yes| F[Readable markdown]
E -->|no| G[JSON fallback]
C --> H[Same collapsible card]
F --> H
G --> H
Important boundary: InvokeSkillObservation is not a normal visualizer in this PR. It now reuses the existing SkillReadyContentList under an “Invoked Skill Knowledge” heading, so skill knowledge shares the same expandable list UI as the skill-ready event.
Today, defining a different renderer requires editing the Agent Canvas source (or a fork). The in-tree recipe is intentionally small:
src/components/features/chat/tool-visualizers/<name>/<name>.tsx.defineVisualizer({ actionKinds, observationKinds, Body }).tool-visualizers/index.ts and add it to the ALL array.export const myToolVisualizer = defineVisualizer({
actionKinds: ["MyToolAction"],
observationKinds: ["MyToolObservation"],
Body: function MyToolBody({ action, observation }) {
return observation
? <MyResult event={observation} sourceAction={action} />
: <MyInFlight event={action} />;
},
});
Current limitation: this is not yet a user-level extension mechanism. It is a clean internal seam, but a user cannot drop a visualizer into ~/.openhands/agent-canvas and have the installed UI pick it up without a follow-up addon architecture.
Issue #481 proposes trusted local Agent Canvas UI addons. The visualizer registry from this PR is a good first real use case for route-less JavaScript addons: instead of only adding pages or CSS, an addon could register event renderers for the conversation stream.
flowchart TD
A[~/.openhands/agent-canvas/addons/*] --> B[addon manifest]
B --> C[generated addon registry for dev/build]
C --> D[Addon register(api)]
D --> E[api.registerToolVisualizer]
E --> F[Runtime visualizer registry]
F --> G{priority}
G -->|addon override| H[Addon visualizer]
G -->|built-in| I[Core visualizer]
G -->|none| J[Markdown fallback]
The smallest useful follow-up is an addon API method that composes with the existing registry:
export default function register(api: AddonApi) {
api.registerToolVisualizer({
id: "acme.kubernetes.apply",
actionKinds: ["MCPToolAction"],
observationKinds: ["MCPToolObservation"],
priority: 50,
matches({ action, observation }) {
return action?.action.kind === "MCPToolAction"
&& action.action.tool_name === "kubectl_apply";
},
Body({ action, observation }) {
return <KubernetesApplyCard action={action} observation={observation} />;
},
});
}
| Design point | Recommendation |
|---|---|
| Registration | Add api.registerToolVisualizer() and build the active registry as addon visualizers + built-ins. |
| Overrides | Support priority and matches(). This lets a custom MCP tool visualizer handle only one tool name while the generic MCP visualizer handles the rest. |
| Packaging | Start with trusted local source addons compiled by Vite in dev/source mode, matching Devin’s proposal. Later add a compiled-runtime path for installed/static packages. |
| Location | Prefer ~/.openhands/agent-canvas/addons/<id> so user customizations do not dirty the Agent Canvas checkout. |
| Safety | Treat addons as local trusted code, show them in settings, allow disable/remove, wrap each renderer in an error boundary, and fall back to the next renderer or markdown on failure. |
| Stable surface | Export visualizer types and primitives from a public subpath such as @openhands/agent-canvas/visualizers. |
Devin’s issue asks whether route-less JavaScript should be root-mounted React, compiled runtime scripts, explicit shell slots, or deferred. Custom event rendering is narrower than a root-level addon and more concrete than a generic shell slot. It can be modeled as a named slot:
slots: {
"conversation.event.visualizers": [myVisualizer]
}
That aligns with the slot proposal in the issue comments while preserving the visualizer PR’s fallback behavior. Addons can replace or extend rendering for selected event kinds, but the core conversation renderer still owns ordering, grouping, titles, collapse behavior, and the default safety net.
matches() should choose deterministically and make the active renderer inspectable.Proposed next step: land the in-tree visualizer PR as the core seam, then open a focused addon follow-up that exposes only registerToolVisualizer() or a conversation.event.visualizers slot. This gives users a real customization path without needing the full extension platform to be solved first.