Agent activity: messages, events and lifecycle
F06 · 35 mapped behaviors in the Self-Healing Codebase Project. Follow the prerequisites, actions and expected results; expand the original CLI recipe to replay it.
778 behavior IDs · 28 families · 844 named recipes & support steps
16 October 2026 · PR #18334 at 9a3c3a69
Canvas 1.27.1 · Agent Server 1.55.0 · automation 1.20.0
This is the map at the revision above, including its open bug notes. It does not incorporate later changes on main or claim a fresh verification run. Setup & reading guide · Download the scenario inventory
All scenarios & marker key · Open a script to read Given / Do / Expect. Expand the exact recipe for replayable commands.
No scenarios match. Try a different word or reset.
F06 Agent activity: messages, events and lifecycle
35 behaviors · 32 scripts & contracts · Maintained source
Entry points & prerequisites
What this family covers
Inside a conversation the user watches the agent work: their own messages appear at once (optimistically), the agent's replies render as markdown, tool calls are grouped into collapsible "N actions completed" blocks whose cards show commands, outputs, diffs and task lists, and a live chip says what the agent is doing right now. Around the transcript sit lifecycle controls and states: the starting pill, stop and resume, the confirmation prompt when confirmation mode is on, error and reconnect banners, the no-LLM banner and the skill-installed banner. History pages in from the server, survives reloads and backend restarts, and any message can be branched into a new conversation.
How a user gets here
- Open any conversation: a sidebar conversation card, or the direct URL
/conversations/<id>. Leaving one for the home page: the sidebar's New Chat link; coming back: its card again. - Start one from the home composer (New Chat,
/): type and press Send.control-openhands conversation startdrives exactly this path. - An empty conversation: the sidebar's create-thread button (
conversation-panel-new-thread-picker), then No workspace (or a workspace). - Composer status control (right side of the composer): Stop while running, Play while stopped.
- Hover a bubble: Branch from here and Copy to clipboard icons; hover a tool title for its timestamp.
- Confirmation mode and the critic are switched on under Settings → Verification (
/settings/verification; the page itself belongs to the settings families).
Given
Preconditions:
- Baseline state (launched, doctored,
onboard --skipdone) andcontrol-openhands llm preset deepseek(deepseek-flash active). Every prompt below is tiny and stays in the conversation's own workspace. - Conversation ids are printed as
"id"bycontrol-openhands conversation start, or read fromcontrol-openhands browser url(/conversations/<id>).<id>below is always the conversation the step just created or opened. F06.llm-not-configured-bannerneeds a second, freshcontrol-openhands launch --new --build neverwith nollm preset(export its run dir asOH_VERIFY_RUNfor those commands, thencontrol-openhands stopit).F06.critic-resultis blocked: it needs a model and an OpenHands Cloud critic API key (Settings → Verification → Critic API Key); its bullet says what a scored run shows. The automatic-fetch half ofF06.load-older-history(its 16-command conversation) needs a model; the scroll-to-top half is driven without one, on the dummy profile below, with thirty refused user messages.F06.hook-eventsneeds a configured hook; none is set in a fresh run.F06.clock-skew-sendandF06.failed-send-persistsneed no model reply: a conversation that exists is enough (without a key the agent errors at once, and the user rows still land). Without a key, create the family's conversations with a dummy profile (QA_DUMMY_KEY=qa control-openhands llm set --profile qa-zeta --model openai/gpt-4o --api-key-env QA_DUMMY_KEY --no-validate) andcontrol-openhands conversation start --prompt ...without--wait.F06.corrective-nudgeis blocked: the nudge needs a model turn with neither text nor a tool call, which a normal prompt to deepseek-flash does not produce. When one happens,control-openhands browser count 'testid=corrective-nudge-message'is1andbrowser count 'testid=user-message >> has-text=Your last response did not include'is0.- The harness browser grants clipboard access:
control-openhands browser clipboardreads what a Copy button wrote andbrowser clipboard --write qa-emptyresets it first. - A terminal call can come back as "The terminal session was reset..." after this run's agent server restarts; the model reruns it, which turns one action into a group of two (see Gotchas).
One run that exercises the streamF06.markdown-rendering · F06.event-groups · F06.thinking · F06.events-matchKnown issueBlocked
Scenario F06--one-run-that-exercises-the-stream · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoStart a conversation with the prompt “Run the terminal command 'echo qa-f06-hello', then create a file qa_f06.txt containing the word hi. Finally reply with a markdown table with columns Step and Result (two rows) followed by a python code block that prints 1.”.
Expect
agent replies render markdown: tables inside a horizontal scroller with edge fades, fenced code, inline code.
consecutive tool calls collapse into "N actions completed" (or "k/N actions completed" with a spinner while one is pending); expanding lists one titled row per action.
model reasoning appears as a collapsed "Thinking" row that expands.
the stream shows one bubble per message event and one row per action, with no duplicates (compare with conversation events).
Linked known issues
Known issue #18117 (F06.thinking): Inline thinking leaks into the agent chat bubble. Live prerequisite: A live model that emits a leading inline think block on a tool-call turn, plus its finalized ActionEvent.
Known issue #17161 (F06.markdown-rendering): markdown sanitizer lets untrusted content set rel="opener" and any target on links. Live prerequisite: A real reasoning, activated-skill, tool-description or error-detail body rendered without includeStandard. The attempted skills dialog exposed public skills only and did not reach the affected chat renderer.
Known issue #17026 (F06.thinking): agent-canvas: final-reply thinking block missing for text-only turns (reasoning_content not read for MessageEvents). Live prerequisite: A reasoning-capable live model emitting nonempty MessageEvent.llm_message.reasoning_content on a text-only turn, followed by reload.
Known issue #16455 (F06.events-match): Duplicate messages from the agent in 1.12.0. Live prerequisite: The original duplicated event/stream trace or deterministic prompt and timing; the report provides no reproducible trigger.
Known issue OpenHands/software-agent-sdk#5354 (F06.event-groups): A mixed valid/invalid tool batch can be split into separate assistant messages, breaking call/result pairing on the next provider request. Live prerequisite: A live model turn mixing valid, invalid and valid tool calls in one response.
Exact CLI recipe & bug checks
Run control-openhands conversation start --prompt "Run the terminal command 'echo qa-f06-hello', then create a file qa_f06.txt containing the word hi. Finally reply with a markdown table with columns Step and Result (two rows) followed by a python code block that prints 1." --wait --timeout 300, then control-openhands browser testids 'testid=chat-interface' and control-openhands browser snapshot 'testid=agent-message'. The testids include user-message, event-group ("3 actions completed"), agent-message, markdown-table-scroll with markdown-table-scroll-fade-left/-right; the snapshot shows a table with column headers Step and Result and a code block print(1). Compare with control-openhands conversation events <id>: one user and one agent MessageEvent, three ActionEvents (terminal, file_editor, canvas_ui_control), matching one user-message, one group of three rows and one agent-message. The thinking row (collapsible-thinking) lives inside the group, so it is listed only after the next bullet expands it.
Expand a group and its rowsF06.event-groups · F06.thinking · F06.tool-visualizersKnown issueBlocked
Scenario F06--expand-a-group-and-its-rows · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoChoose “Event group toggle”.
- DoChoose “Event group content” / “Expand” / match 1.
- DoChoose “Collapsible thinking toggle” / match 1.
Expect
consecutive tool calls collapse into "N actions completed" (or "k/N actions completed" with a spinner while one is pending); expanding lists one titled row per action.
model reasoning appears as a collapsed "Thinking" row that expands.
expanding a tool row shows a rich body: terminal command and output (non-zero exit badge exit N), file-editor path chip and content.
Linked known issues
Known issue #18117 (F06.thinking): Inline thinking leaks into the agent chat bubble. Live prerequisite: A live model that emits a leading inline think block on a tool-call turn, plus its finalized ActionEvent.
Known issue #17492 (F06.tool-visualizers): Failed browser, glob, grep, and file editor tool calls render as successful in chat. Live prerequisite: A working model emitting failed browser, glob, grep, planning-editor and file-editor observations with is_error=true.
Known issue #17490 (F06.tool-visualizers): Terminal commands that fail with an ordinary non-zero exit render as successful in chat. Live prerequisite: A working model producing a completed terminal command with nonzero exit_code and is_error=false.
Known issue #17026 (F06.thinking): agent-canvas: final-reply thinking block missing for text-only turns (reasoning_content not read for MessageEvents). Live prerequisite: A reasoning-capable live model emitting nonempty MessageEvent.llm_message.reasoning_content on a text-only turn, followed by reload.
Known issue #16341 (F06.tool-visualizers): Code-explorer subagent frequently errors. Live prerequisite: A working model with native code-explorer delegation and the original repository task; no delegated-tool output can be generated without credentials.
Known issue OpenHands/software-agent-sdk#5354 (F06.event-groups): A mixed valid/invalid tool batch can be split into separate assistant messages, breaking call/result pairing on the next provider request. Live prerequisite: A live model turn mixing valid, invalid and valid tool calls in one response.
Known issue OpenHands/software-agent-sdk#5108 (F06.tool-visualizers): The native SDK Gemini editor can report success while corrupting unrelated invalid-UTF8 bytes in the file. Live prerequisite: An Agent Server configured with the native Gemini edit tool set and a real tool edit of an invalid-UTF8 fixture; Gemini ACP uses a different external executor.
Known issue OpenHands/software-agent-sdk#5096 (F06.tool-visualizers): The canonical agent file editor can replace a symlink instead of editing its target; Undo does not restore the link. Live prerequisite: A working model invoking canonical file-editor str_replace/insert through a real symlink, then undo.
Known issue OpenHands/software-agent-sdk#5074 (F06.tool-visualizers): After server restart a native delegated child can remain Running while the parent loses its task mapping/result. Live prerequisite: A native delegated child still running during an owned server crash and parent recovery.
Known issue OpenHands/software-agent-sdk#5038 (F06.tool-visualizers): The agent file editor can refuse valid Chinese UTF8 text as binary when its sample splits a character; Canvas Files uses a separate preview detector. Live prerequisite: A live canonical file-editor view/edit on the specific UTF8 sample alignment; Canvas Files preview uses a separate NUL sniff, not this validator.
Known issue OpenHands/software-agent-sdk#4983 (F06.tool-visualizers): Agent grep can silently miss extended-regex matches when its runtime falls back to system grep. Live prerequisite: A runtime without ripgrep and a live agent grep call using extended regex and escaped literal punctuation.
Known issue OpenHands/software-agent-sdk#4668 (F06.tool-visualizers): A planning editor card can lose old/new content and fail to show the complete edit diff. Live prerequisite: A live planner create/edit yielding PlanningFileEditorObservation with old/new contents for the chat diff.
Known issue OpenHands/software-agent-sdk#4663 (F06.tool-visualizers): Parallel Python-fallback glob calls can change process cwd and return files from the wrong workspace. Live prerequisite: Concurrent real glob/tool calls in distinct workspaces with the Python fallback active.
Known issue OpenHands/software-agent-sdk#4661 (F06.tool-visualizers): Same-name project subagents can resolve to another conversation workspace definition. Live prerequisite: Two native task-enabled conversations with same-name project subagent definitions and a working model for delegation.
Known issue OpenHands/software-agent-sdk#4583 (F06.tool-visualizers): An agent editor insertion at EOF can join onto the last line when the file has no trailing newline. Live prerequisite: A working model invoking file-editor insert at EOF on a file with no trailing newline.
Known issue OpenHands/software-agent-sdk#4540 (F06.tool-visualizers): Synthetic GLM/Kimi tool calls can render as raw tool_call syntax instead of executing. Live prerequisite: Synthetic provider credentials and a GLM/Kimi turn emitting the raw tool_call wrapper.
Exact CLI recipe & bug checks
Run control-openhands browser click 'testid=event-group-toggle' (label becomes Collapse actions), control-openhands browser text 'testid=event-group' (three titles, e.g. "Echo test string in terminal", "Create qa_f06.txt containing hi"; titles are model-written), then control-openhands browser click 'testid=event-group-content >> role=button[name="Expand"][exact] >> nth=0' twice (each click expands the next collapsed row). The group text now contains echo qa-f06-hello and its output qa-f06-hello, the file path chip (testid=file-path-chip) and the content hi. control-openhands browser click 'testid=collapsible-thinking-toggle >> nth=0' reveals collapsible-thinking-content. Take control-openhands browser screenshot 'testid=chat-interface' --feature F06.tool-visualizers --name bash-and-file.
Open files from the streamF06.workspace-path-linksMapped behavior
Scenario F06--open-files-from-the-stream · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoChoose “File quick row close qa f06.txt”.
- DoChoose “Agent message” / “Markdown file path”.
- DoChoose “File path chip”.
Expect
a workspace file path in an agent reply, and the path chip of a file-editor card, open that file in the Files tab.
Exact CLI recipe & bug checks
On the same page the agent may already have opened the file; run control-openhands browser click 'testid=file-quick-row-close-qa_f06.txt' and check control-openhands browser count 'testid=file-quick-row-item-qa_f06.txt' is 0. Run control-openhands browser click 'testid=agent-message >> testid=markdown-file-path-link': the count is 1 and control-openhands browser text 'testid=file-content-viewer-plain' is hi. Close it again and control-openhands browser click 'testid=file-path-chip' (inside the expanded file-editor row): the count is 1 again.
Markdown file previewF06.markdown-file-previewMapped behavior
Scenario F06--markdown-file-preview · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoStart a conversation with the prompt “Use the file_editor tool to create the file qa_notes.md containing a heading '# QA Notes' and one bullet '- first'. Do nothing else and reply DONE.”.
- DoChoose “Markdown file preview view”.
- DoOpen /conversations/<stream.
Expect
when the agent creates a Markdown file, its row stays expanded and ungrouped and shows a rendered, height-limited preview card with the file name and View, which opens the file in the Files tab.
Exact CLI recipe & bug checks
Run control-openhands conversation start --prompt "Use the file_editor tool to create the file qa_notes.md containing a heading '# QA Notes' and one bullet '- first'. Do nothing else and reply DONE." --wait --timeout 150. Without any click, control-openhands browser text 'testid=markdown-file-preview' reads QA Notes first qa_notes.md View (rendered heading and bullet, then the file name) and count 'testid=event-group' is 0. control-openhands browser count 'testid=file-quick-row-item-qa_notes.md' is 0; control-openhands browser click 'testid=markdown-file-preview-view' makes it 1 and control-openhands browser text 'testid=file-content-viewer-markdown' reads QA Notes first. Then control-openhands browser goto /conversations/<stream id> to return to the stream conversation for the next bullets.
TimestampsF06.timestampsMapped behavior
Scenario F06--timestamps · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoUse browser tooltip as specified in the exact recipe.
- DoUse browser tooltip as specified in the exact recipe.
Expect
hovering a bubble or tool title shows the event's local date and time.
Exact CLI recipe & bug checks
Run control-openhands browser tooltip 'testid=user-message' and control-openhands browser tooltip 'testid=event-group-toggle'. Both return the local date and time, e.g. Oct 4, 2026, 11:08 PM.
Copy a message and a code blockF06.message-copy · F06.code-block-copyMapped behavior
Scenario F06--copy-a-message-and-a-code-block · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoHover over “Agent message” / match 1.
- DoChoose “Agent message” / match 1 / “Copy to clipboard” / match 1.
- DoHover over the element identified in the CLI recipe.
- DoChoose the element identified in the CLI recipe.
Expect
hovering a bubble shows Copy to clipboard; it copies the message's markdown source and the label flips to Copied to clipboard for 2 s.
hovering a fenced code block in an agent reply shows its own Copy button, which copies only the code.
Exact CLI recipe & bug checks
On the same page run control-openhands browser clipboard --write qa-empty, control-openhands browser hover 'testid=agent-message >> nth=0' and control-openhands browser click 'testid=agent-message >> nth=0 >> testid=copy-to-clipboard >> nth=0' (the bubble's button; the code block has a second, hidden copy-to-clipboard). control-openhands browser attr 'testid=agent-message >> nth=0 >> testid=copy-to-clipboard >> nth=0' aria-label is Copied to clipboard, and Copy to clipboard again about 2 s later; control-openhands browser clipboard starts with | Step | Result | and ends with the fenced print(1) block. Then control-openhands browser hover 'testid=agent-message >> pre', control-openhands browser click 'testid=agent-message >> pre >> testid=copy-to-clipboard' and control-openhands browser clipboard is exactly print(1).
Long table cellF06.markdown-renderingKnown issueBlocked
Scenario F06--long-table-cell · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoEnter “x” in “Message”.
- DoChoose “Submit”.
- WaitWait for the conversation to finish.
- DoChoose “Right panel toggle”.
Expect
agent replies render markdown: tables inside a horizontal scroller with edge fades, fenced code, inline code.
Linked known issues
Known issue #17161 (F06.markdown-rendering): markdown sanitizer lets untrusted content set rel="opener" and any target on links. Live prerequisite: A real reasoning, activated-skill, tool-description or error-detail body rendered without includeStandard. The attempted skills dialog exposed public skills only and did not reach the affected chat renderer.
Exact CLI recipe & bug checks
On the stream conversation write a prompt file whose first line is Reply with only the word OK. Do not run any tools., then a blank line and the table | Name | Description | Status |, |---|---|---|, | Alpha | This is a really long sentence in a single cell that goes on and on to explain a very involved aspect of the system, which nobody could read on one line without scrolling forever to the right. | ok |, | Beta | brief | fail |. Run control-openhands browser fill 'testid=chat-input' x --value-file <file>, control-openhands browser click 'testid=submit-button' and control-openhands conversation wait <id> --fresh --timeout 120; the user bubble renders the table. Cells wrap once they reach 28rem (max-w-md): control-openhands browser eval "getComputedStyle(document.querySelector('[data-testid=markdown-table-scroll] td')).whiteSpace + ' ' + getComputedStyle(document.querySelector('[data-testid=markdown-table-scroll] td')).maxWidth" is normal 448px. At desktop with the right panel closed, control-openhands browser bbox 'testid=user-message >> nth=-1 >> testid=markdown-table-scroll' has scrollWidth 582 equal to clientWidth and Status is visible (browser screenshot 'testid=chat-interface' --feature F06.markdown-rendering --name long-cell-table). A table still wider than the conversation keeps the horizontal scroll with its edge fade: with the panel open (control-openhands browser click 'testid=right-panel-toggle') clientWidth is 474, and at phone width 326, so Status is scrolled off there while the long cell stays wrapped. An agent reply with such a table behaves the same.
Non-zero exit badgeF06.tool-visualizersKnown issueBlocked
Scenario F06--non-zero-exit-badge · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoEnter “Run exactly this terminal command, verbatim, with nothing appended: ls /qa_missing_dir2 . Then reply OK.” in “Message”.
- DoChoose “Submit”.
- WaitWait for the conversation to finish.
- DoChoose “Chat scroll container” / “Expand” / nth=-1.
- DoChoose “Event group toggle” / nth=-1.
- DoChoose “Event group content” / nth=-1 / “Expand” / nth=-1.
Expect
expanding a tool row shows a rich body: terminal command and output (non-zero exit badge exit N), file-editor path chip and content.
Linked known issues
Known issue #17492 (F06.tool-visualizers): Failed browser, glob, grep, and file editor tool calls render as successful in chat. Live prerequisite: A working model emitting failed browser, glob, grep, planning-editor and file-editor observations with is_error=true.
Known issue #17490 (F06.tool-visualizers): Terminal commands that fail with an ordinary non-zero exit render as successful in chat. Live prerequisite: A working model producing a completed terminal command with nonzero exit_code and is_error=false.
Known issue #16341 (F06.tool-visualizers): Code-explorer subagent frequently errors. Live prerequisite: A working model with native code-explorer delegation and the original repository task; no delegated-tool output can be generated without credentials.
Known issue OpenHands/software-agent-sdk#5108 (F06.tool-visualizers): The native SDK Gemini editor can report success while corrupting unrelated invalid-UTF8 bytes in the file. Live prerequisite: An Agent Server configured with the native Gemini edit tool set and a real tool edit of an invalid-UTF8 fixture; Gemini ACP uses a different external executor.
Known issue OpenHands/software-agent-sdk#5096 (F06.tool-visualizers): The canonical agent file editor can replace a symlink instead of editing its target; Undo does not restore the link. Live prerequisite: A working model invoking canonical file-editor str_replace/insert through a real symlink, then undo.
Known issue OpenHands/software-agent-sdk#5074 (F06.tool-visualizers): After server restart a native delegated child can remain Running while the parent loses its task mapping/result. Live prerequisite: A native delegated child still running during an owned server crash and parent recovery.
Known issue OpenHands/software-agent-sdk#5038 (F06.tool-visualizers): The agent file editor can refuse valid Chinese UTF8 text as binary when its sample splits a character; Canvas Files uses a separate preview detector. Live prerequisite: A live canonical file-editor view/edit on the specific UTF8 sample alignment; Canvas Files preview uses a separate NUL sniff, not this validator.
Known issue OpenHands/software-agent-sdk#4983 (F06.tool-visualizers): Agent grep can silently miss extended-regex matches when its runtime falls back to system grep. Live prerequisite: A runtime without ripgrep and a live agent grep call using extended regex and escaped literal punctuation.
Known issue OpenHands/software-agent-sdk#4668 (F06.tool-visualizers): A planning editor card can lose old/new content and fail to show the complete edit diff. Live prerequisite: A live planner create/edit yielding PlanningFileEditorObservation with old/new contents for the chat diff.
Known issue OpenHands/software-agent-sdk#4663 (F06.tool-visualizers): Parallel Python-fallback glob calls can change process cwd and return files from the wrong workspace. Live prerequisite: Concurrent real glob/tool calls in distinct workspaces with the Python fallback active.
Known issue OpenHands/software-agent-sdk#4661 (F06.tool-visualizers): Same-name project subagents can resolve to another conversation workspace definition. Live prerequisite: Two native task-enabled conversations with same-name project subagent definitions and a working model for delegation.
Known issue OpenHands/software-agent-sdk#4583 (F06.tool-visualizers): An agent editor insertion at EOF can join onto the last line when the file has no trailing newline. Live prerequisite: A working model invoking file-editor insert at EOF on a file with no trailing newline.
Known issue OpenHands/software-agent-sdk#4540 (F06.tool-visualizers): Synthetic GLM/Kimi tool calls can render as raw tool_call syntax instead of executing. Live prerequisite: Synthetic provider credentials and a GLM/Kimi turn emitting the raw tool_call wrapper.
Exact CLI recipe & bug checks
In an open conversation run control-openhands browser fill 'testid=chat-input' "Run exactly this terminal command, verbatim, with nothing appended: ls /qa_missing_dir2 . Then reply OK.", control-openhands browser click 'testid=submit-button' and control-openhands conversation wait <id> --fresh --timeout 150 (without --fresh it returns the previous run's finished at once). If the reply's actions render as a single row, run control-openhands browser click 'testid=chat-scroll-container >> role=button[name="Expand"][exact] >> nth=-1'; if they render as a group (browser count 'testid=event-group' grew, e.g. 2 actions completed after a terminal reset and rerun), run control-openhands browser click 'testid=event-group-toggle >> nth=-1' and control-openhands browser click 'testid=event-group-content >> nth=-1 >> role=button[name="Expand"][exact] >> nth=-1'. Then control-openhands browser count 'testid=chat-scroll-container >> text="exit 2"' is 1; the badge sits above the No such file or directory output (browser screenshot 'testid=event-group >> nth=-1' --feature F06.tool-visualizers --name exit-code). Repeating the check in the same conversation needs a new directory name and a count that grows, because an earlier expanded badge stays on the page.
Task listF06.task-list · F06.status-indicatorMapped behavior
Scenario F06--task-list · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoOpen /.
- DoEnter “Use your task_tracker tool to plan exactly two tasks titled 'qa step one' and 'qa step two'; mark 'qa step one' done and 'qa step two' todo. Do nothing else and reply DONE.” in “Message”.
- DoChoose “Submit”.
- WaitWait for the conversation to finish.
Expect
task-tracker calls render a "Tasks" card with done / todo icons.
while a new conversation starts, a "Starting" pill shows above the composer.
Exact CLI recipe & bug checks
From / run control-openhands browser goto /, control-openhands browser fill 'testid=chat-input' "Use your task_tracker tool to plan exactly two tasks titled 'qa step one' and 'qa step two'; mark 'qa step one' done and 'qa step two' todo. Do nothing else and reply DONE." and control-openhands browser click 'testid=submit-button' --expect-url '/conversations/' --observe '[data-testid=chat-status-indicator]' --observe-ms 8000. observed contains Starting for a moment (the pill). Then control-openhands conversation wait <id> --timeout 150 and control-openhands browser text 'testid=chat-scroll-container': it contains Tasks, qa step one, qa step two; the screenshot shows a check icon and muted text for the done task, an empty circle for the todo one.
Empty stateF06.empty-state-suggestionsMapped behavior
Scenario F06--empty-state · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoChoose “Conversation panel new thread picker”.
- DoChoose “Launch no workspace”.
- DoChoose “Chat suggestions” / “Fix README”.
Expect
an empty conversation shows "Let's start building!" with four suggestion chips; a chip fills the composer with its prompt.
Exact CLI recipe & bug checks
Run control-openhands browser click 'testid=conversation-panel-new-thread-picker', control-openhands browser click 'testid=launch-no-workspace' --expect-url '/conversations/', then control-openhands browser text 'testid=chat-suggestions': Let's start building! plus Increase test coverage, Auto-merge PRs, Fix README, Clean dependencies. Run control-openhands browser click 'testid=chat-suggestions >> text=Fix README' and control-openhands browser text 'testid=chat-input': the composer holds the README-improvement prompt.
Pending bubble and long messagesF06.pending-messages · F06.user-and-agent-messagesMapped behavior
Scenario F06--pending-bubble-and-long-messages · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoEnter “x” in “Message”.
- DoChoose “Submit”.
- WaitWait for the conversation to finish.
- DoChoose “Chat message expand”.
Expect
a sent message shows "Sending..." until the server echoes it; a failed send shows Failed to send with Retry and Dismiss; Retry delivers it exactly once.
user and agent bubbles; a long user message is clipped with a gradient and expands on click ("View More").
Exact CLI recipe & bug checks
Write a 25-line message to a file (first line Reply with only the word OK. Ignore the filler lines below., then filler line 1…filler line 24), run control-openhands browser fill 'testid=chat-input' x --value-file <file> and control-openhands browser click 'testid=submit-button' --observe '[data-testid=chat-message-sending]' --observe-ms 3000. observed shows Sending... then <absent> once the server echoes it. After control-openhands conversation wait <id> --timeout 120, control-openhands browser testids 'testid=chat-scroll-container' lists chat-message-truncation-gradient and chat-message-expand ("View More"); control-openhands browser click 'testid=chat-message-expand' removes both and the whole text shows. testid=agent-message reads OK.
Browser clock aheadF06.clock-skew-sendMapped behavior
Scenario F06--browser-clock-ahead · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoUse browser clock as specified in the exact recipe.
- DoOpen /conversations/<id>.
- WaitWait for “Message” to appear.
- DoEnter “Reply with only: clock-ok” in “Message”.
- DoChoose “Submit”.
- DoUse browser clock as specified in the exact recipe.
- DoReload the page.
- WaitWait for the conversation to finish.
Expect
a message sent while the browser clock runs ahead of the server is still confirmed by the server echo: no lingering "Sending..." and no Retry.
Exact CLI recipe & bug checks
On the same conversation, skew the page's clock before loading it: control-openhands browser clock --offset-ms 300000, control-openhands browser goto /conversations/<id>, control-openhands browser wait 'testid=chat-input'; control-openhands browser eval "new Date().toISOString()" is about five minutes ahead of date -u (read-only shell check). Run control-openhands browser fill 'testid=chat-input' 'Reply with only: clock-ok' and control-openhands browser click 'testid=submit-button' --observe '[data-testid=chat-message-sending],[data-testid=chat-message-error]' --observe-ms 6000: observed shows Sending... and then <absent> (the server echo confirmed it), never Failed to send. control-openhands browser count 'testid=chat-message-sending' and control-openhands browser count 'testid=chat-message-retry' are 0, and control-openhands conversation events <id> --kinds MessageEvent gains one user row Reply with only: clock-ok whose ts is the server's time (close to date -u), not the browser's. Undo the skew before moving on: control-openhands browser clock --offset-ms 0 and control-openhands browser reload (with a model, control-openhands conversation wait <id> --fresh --timeout 120 first).
Server lost: banner, failed send, reconnectF06.error-banner · F06.pending-messages · F06.reconnectKnown failureKnown issueBlocked
Scenario F06--server-lost-banner-failed-send-reconnect · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- SetupChange the run-owned service state: stop agent-server.
- WaitWait for “Error message banner” to appear.
- DoEnter “Reply with only the word PONG.” in “Message”.
- DoChoose “Submit”.
- DoChoose “Chat message retry”.
- DoChoose “Chat message dismiss”.
- DoChoose “Error message banner dismiss”.
- DoRestart the run-owned stack as specified.
- DoChoose “Chat message retry”.
- WaitWait for the conversation to finish.
Expect
losing the server shows "Unable to connect to server" with Retry, Copy and Close; Retry reconnects once the server is back.
a sent message shows "Sending..." until the server echoes it; a failed send shows Failed to send with Retry and Dismiss; Retry delivers it exactly once.
when the Agent Server goes away the composer shows Disconnected; after it returns the page reconnects without a reload.
Linked known issues
Known issue #16842 (F06.reconnect): A late close from a replaced WebSocket marks the active connection disconnected. Live prerequisite: A transport fixture delaying an old socket close until its replacement is open; ordinary server restart cannot select that ordering.
Known issue #15527 (F06.reconnect): Canvas briefly shows "disconnected" when coming back to an idle tab. Live prerequisite: The original custom-domain deployment, idle duration and tab-suspension/network transition; those details are missing.
Known failure OpenHands/software-agent-sdk#5635 (F06.reconnect), reproduced 2026-10-12 at fa1f724ab573 with SDK 1.54.0 and Chromium 153.0.8010.12: sandbox pause returned 204 but an already communicating session socket remained open for more than 10 seconds. A new Canvas tab received a subsequent run:false marker while the original tab did not. DELETE returned 200 and GET returned 404, yet both held sockets remained open for more than 10 seconds; Canvas later redirected by metadata polling. Recipe: open an idle dummy-profile conversation and confirm its marker/socket; POST /api/conversations/prepare-for-sandbox-pause, retain the original tab for 10 seconds, open a second tab, POST a new unique user event with run:false, then compare the two transcripts. DELETE the owned conversation and read lifecycle metadata for another 10 seconds. Use --run <run> and browser events --kinds websocket-created,websocket-open,websocket-close --last 100; no frame bodies or auth data are recorded. Evidence: backfill-18309-surface-rechecks/F06.reconnect/pause-delete-sockets-5635.json, pause-new-tab-5635.png, pause-stale-tab-5635.png. The expected socket shutdown/reconnect remains unchanged; polling recovery must not be mistaken for socket closure.
Known issue #18445 (F06.load-older-history, F06.reconnect): a failed initial Cloud history search can be swallowed into an empty page while the runtime count succeeds, advancing after_seq past history that was never displayed. Live prerequisite: a Cloud account and owned conversation with a genuinely failing App API events/search and a successful runtime events/count. The Local failure path throws instead and is not an equivalent check.
Exact CLI recipe & bug checks
With a conversation open run control-openhands service stop agent-server and control-openhands browser wait 'testid=error-message-banner' --timeout 20000: the banner reads Unable to connect to server with error-message-banner-retry, -copy and -dismiss; the composer shows Disconnected. Run control-openhands browser fill 'testid=chat-input' 'Reply with only the word PONG.' and control-openhands browser click 'testid=submit-button' --observe '[data-testid=chat-message-sending],[data-testid=chat-message-error]' --observe-ms 8000: Sending... turns into Failed to send Retry Dismiss. control-openhands browser click 'testid=chat-message-retry' while down fails again; control-openhands browser click 'testid=chat-message-dismiss' removes the bubble (count 'testid=chat-message-error' is 0). Send the same message again so one failed bubble remains, control-openhands browser click 'testid=error-message-banner-dismiss' (banner count 0), then control-openhands restart. Poll control-openhands browser text 'testid=interactive-chat-box' every 10 s: within about a minute after restart returns it no longer contains Disconnected (reconnected without reload). control-openhands browser click 'testid=chat-message-retry', control-openhands conversation wait <id> --timeout 120, then control-openhands conversation events <id> --kinds MessageEvent: Reply with only the word PONG. appears once and the last agent-message is PONG.
Banner RetryF06.error-bannerMapped behavior
Scenario F06--banner-retry · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- SetupChange the run-owned service state: stop agent-server.
- DoRestart the run-owned stack as specified.
- DoChoose “Error message banner retry”.
Expect
losing the server shows "Unable to connect to server" with Retry, Copy and Close; Retry reconnects once the server is back.
Exact CLI recipe & bug checks
Run control-openhands service stop agent-server, wait for testid=error-message-banner as above, control-openhands restart, then control-openhands browser click 'testid=error-message-banner-retry'. control-openhands browser count 'testid=error-message-banner' is 0 and the composer is no longer Disconnected. Take browser screenshot 'testid=chat-interface' --feature F06.error-banner --name disconnected while the banner is up.
Failed send survives leavingF06.failed-send-persistsMapped behavior
Scenario F06--failed-send-survives-leaving · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- SetupChange the run-owned service state: stop agent-server.
- WaitWait for “Error message banner” to appear.
- DoEnter “Reply with only: retry-ok” in “Message”.
- DoChoose “Submit”.
- DoRestart the run-owned stack as specified.
- WaitWait for “Error message banner” to disappear.
- DoChoose “New Chat”.
- WaitWait for “Home chat launcher” to appear.
- DoChoose the element identified in the CLI recipe.
- WaitWait for “User message” to appear.
- DoChoose “Chat message retry”.
- WaitWait for “Chat message retry” to disappear.
- WaitWait for the conversation to finish.
Expect
a failed send keeps its Retry row after the backend returns and after leaving and reopening the conversation from the sidebar; sending the same text again succeeds without clearing it, and Retry then delivers that attempt exactly once.
Exact CLI recipe & bug checks
With /conversations/<id> open and healthy, run control-openhands service stop agent-server, control-openhands browser wait 'testid=error-message-banner' --timeout 20000, control-openhands browser fill 'testid=chat-input' 'Reply with only: retry-ok' and control-openhands browser click 'testid=submit-button' --observe '[data-testid=chat-message-sending],[data-testid=chat-message-error]' --observe-ms 8000: Sending... turns into Failed to send Retry Dismiss and control-openhands browser count 'testid=chat-message-retry' is 1 (control-openhands browser screenshot 'testid=chat-interface' --feature F06.failed-send-persists --name failed-before-leaving). Run control-openhands restart and control-openhands browser wait 'testid=error-message-banner' --state hidden --timeout 90000 (the page reconnects by itself; testid=error-message-banner-retry hurries it while the banner is still up): the Retry row is still there (count 1). Leave through the sidebar and come back: control-openhands browser click 'role=link[name="New Chat"]', control-openhands browser wait 'testid=home-chat-launcher', control-openhands browser click 'a[href*="/conversations/<id>"] >> nth=0' --expect-url '/conversations/<id>', control-openhands browser wait 'testid=user-message'; control-openhands browser count 'testid=chat-message-retry' is still 1 after the history reloaded (control-openhands browser screenshot 'testid=chat-interface' --feature F06.failed-send-persists --name retry-after-reopen). Send the same text again (fill Reply with only: retry-ok, click testid=submit-button with the same --observe): the new bubble goes Sending... and settles while Failed to send Retry Dismiss stays; control-openhands browser count 'testid=chat-message-sending' is 0, control-openhands browser count 'testid=chat-message-retry' is 1, and the rows of control-openhands conversation events <id> --kinds MessageEvent --grep 'retry-ok' whose source is user number exactly one, Reply with only: retry-ok (in a model-free run count is 1; with a model the agent's retry-ok reply matches the grep too, so count the user rows, not count). Then control-openhands browser click 'testid=chat-message-retry' and control-openhands browser wait 'testid=chat-message-retry' --state detached --timeout 15000: control-openhands browser count 'testid=chat-message-sending' is 0 and the same --grep now lists exactly two user rows Reply with only: retry-ok (one per attempt, none duplicated). With a model, control-openhands conversation wait <id> --fresh --timeout 120 before the next bullet.
Live chip, stop and resumeF06.live-activity · F06.stop-resumeKnown issueBlocked
Scenario F06--live-chip-stop-and-resume · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoStart a conversation with the prompt “Run exactly one terminal command: sleep 25 && echo qa-slept. Then reply with only its output.”.
- WaitWait for “Live activity chip” to appear.
- DoChoose “Stop”.
- DoChoose “Play”.
- WaitWait for the conversation to finish.
Expect
while the agent runs, a chip above the composer names the current action ("Thinking", then the action title) and the composer status reads Running.
Stop in the composer interrupts the agent, including a running tool call (status Stopped, play button, an Agent error row for the interrupted call); Play resumes it to completion.
Linked known issues
Known issue #16719 (F06.live-activity): Agent status always says "thinking" even when nothing is happening. Live prerequisite: The event/status trace and deterministic stalled or errored state from the original report; its trigger is unspecified.
Known issue #16582 (F06.stop-resume): After reconnect, a maximum-iteration error can fail to pause the agent through the legacy event handler. Live prerequisite: A maximum-iteration legacy error delivered after socket replacement, with a real model run reaching that limit.
Known issue OpenHands/software-agent-sdk#4990 (F06.stop-resume): An ACP turn can keep executing tool calls while the UI already reports Paused. Live prerequisite: An authenticated Claude Code ACP turn executing separate marker-writing tools while Pause is pressed.
Known issue OpenHands/software-agent-sdk#4777 (F06.stop-resume): Stop can still start a queued tool after it finishes waiting on a resource lock. Live prerequisite: A live batch with a second real tool waiting on a declared-resource lock when Stop is pressed.
Known issue OpenHands/software-agent-sdk#5969 (F06.stop-resume): a conversation can report finished but reject Play with already-running, or delay a follow-up, while a slow event webhook is still pending. Live prerequisite: a genuinely successful model run on an owned backend with a bounded slow webhook receiver, then Play or send during that completion window.
Exact CLI recipe & bug checks
Run control-openhands conversation start --prompt "Run exactly one terminal command: sleep 25 && echo qa-slept. Then reply with only its output." (no --wait), control-openhands browser wait 'testid=live-activity-chip' --timeout 30000 and control-openhands browser text 'testid=live-activity-chip': first Thinking, then the action title (e.g. Run sleep 25 then echo qa-slept); browser text 'testid=interactive-chat-box' contains Running. Run control-openhands browser click 'testid=stop-button': control-openhands conversation status <id> is paused, the composer reads Stopped, count 'testid=play-button' is 1 and the chip is gone. Stop interrupts the running sleep too: control-openhands conversation events <id> shows an AgentErrorEvent (Tool call interrupted before completion. The conversation was paused.) followed by an InterruptEvent, and browser text 'testid=chat-scroll-container' shows an Agent error row under the action title. Run control-openhands browser click 'testid=play-button': status running, then control-openhands conversation wait <id> --timeout 120 ends finished with a new agent MessageEvent (the model reports that the command was interrupted; qa-slept never prints).
Reload mid-runF06.reload-mid-runKnown issueBlocked
Scenario F06--reload-mid-run · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoStart a conversation with the prompt “Run exactly one terminal command: sleep 15 && echo qa-reloaded. Then reply with only its output.”.
- DoReload the page.
- WaitWait for the conversation to finish.
Expect
reloading while the agent runs reattaches to the live run; it finishes without duplicated events.
Linked known issues
Known issue #15606 (F06.reload-mid-run): When re-loading ACP conversations, messages disappear or move. Live prerequisite: An authenticated Codex ACP session emitting intermediate messages while another window reloads history.
Known issue OpenHands/software-agent-sdk#4077 (F06.reload-mid-run): Retried or late streaming deltas can duplicate live text; a stalled subscriber can delay Stop and finalization. Live prerequisite: A working stream interrupted/retried or delivered to a controlled stalled subscriber, with delta/state-order capture.
Exact CLI recipe & bug checks
Run control-openhands conversation start --prompt "Run exactly one terminal command: sleep 15 && echo qa-reloaded. Then reply with only its output.", wait for testid=live-activity-chip, then control-openhands browser reload. The composer still reads Running and the chip is back. After control-openhands conversation wait <id> --timeout 120: testid=agent-message is qa-reloaded, count 'testid=user-message' is 1, control-openhands browser eval "document.querySelectorAll('[data-testid=chat-scroll-container] [data-testid=generic-event-message-title]').length" is 1.
Older historyF06.load-older-historyKnown issueBlocked
Scenario F06--older-history · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoStart a conversation with the prompt “Run these 16 terminal commands strictly one per tool call, sequentially (never combine them): echo qa1, echo qa2, echo qa3, echo qa4, echo qa5, echo qa6, echo qa7, echo qa8, echo qa9, echo qa10, echo qa11, echo qa12, echo qa13, echo qa14, echo qa15, echo qa16. Then reply DONE.”.
- DoReload the page.
- SetupConfigure or inspect the model profiles specified in the recipe.
- SetupConfigure or inspect the model profiles specified in the recipe.
- SetupArrange the prerequisite backend state through the API.
- DoStart a conversation with the prompt “qa older 1”.
- DoReload the page.
- WaitWait for “Message” to appear.
- DoScroll “Chat scroll container” as specified in the recipe.
- DoScroll “Chat scroll container” as specified in the recipe.
- DoOpen /conversations/<history-id>.
- WaitWait for “Message” to appear.
Expect
only the newest 50 events load first; the previous 50 are fetched when the user scrolls the transcript to its top (a "Fetching older messages…" row shows until they arrive) and automatically when the content is too short to scroll, so the first prompt and all actions appear.
Linked known issues
Known issue #17559 (F06.load-older-history): History pagination can show both an action and its observation (ready-for-dev companion to #16947). Live prerequisite: A real persisted history page boundary splitting an action/observation or ACP started/completed pair; dummy-key errors do not produce these tool pairs.
Known issue #16947 (F06.load-older-history): Loading older history can leave both the action and its newer observation visible because UI projection depends on arrival order. Live prerequisite: A real long conversation whose REST boundary splits an action/observation pair; keep companion #17559 linked separately.
Known issue #18445 (F06.load-older-history, F06.reconnect): a failed initial Cloud history search can be swallowed into an empty page while the runtime count succeeds, advancing after_seq past history that was never displayed. Live prerequisite: a Cloud account and owned conversation with a genuinely failing App API events/search and a successful runtime events/count. The Local failure path throws instead and is not an equivalent check.
Exact CLI recipe & bug checks
Run control-openhands conversation start --prompt "Run these 16 terminal commands strictly one per tool call, sequentially (never combine them): echo qa1, echo qa2, echo qa3, echo qa4, echo qa5, echo qa6, echo qa7, echo qa8, echo qa9, echo qa10, echo qa11, echo qa12, echo qa13, echo qa14, echo qa15, echo qa16. Then reply DONE." --wait --timeout 400 (note <history-id>); control-openhands conversation events <history-id> --last 500 reports a count above 50. Run control-openhands browser network --clear, control-openhands browser reload, then control-openhands browser network --last 300: two GET /api/conversations/<history-id>/events/search requests (newest page, then the older page). control-openhands browser text 'testid=chat-scroll-container' starts with the prompt and shows 16 actions completed and DONE. The other trigger is the user scrolling to the top, driven without a model on the dummy profile from Preconditions (qa-zeta active; in a keyed run create and activate it with that same llm set command for this bullet, then re-run control-openhands llm preset deepseek and remove it with control-openhands api DELETE /api/profiles/qa-zeta --write, so later families still count two profiles; arrange, not proof). Run control-openhands conversation start --prompt "qa older 1" (no --wait; note <older-id>), then for N in $(seq 2 30); do control-openhands conversation send <older-id> --prompt "qa older $N"; done: every send is accepted while the agent sits in error, about 1.5 s apiece (42 s for the 29 here), and each refused message adds five events (a MessageEvent, three ConversationStateUpdateEvents, a ConversationErrorEvent), so the conversation ends with about 150 events (152 here). control-openhands conversation events <older-id> --last 500 reports "total": 152, "pages": 2, "more": false (both server pages read, see Gotchas); its first row is the SystemPromptEvent and the first MessageEvent is qa older 1, while --last 25 reports "pages": 1, "more": true and starts at qa older 26. Run control-openhands browser network --clear, control-openhands browser reload, control-openhands browser wait 'testid=chat-input', then control-openhands browser network --filter 'events/search' --last 10: two requests, ?limit=50&sort_order=TIMESTAMP_DESC and one with timestamp__lt=<ts of qa older 21>, because the newest 50 events hold only ten bubbles, shorter than the viewport, so the automatic fetch ran once and then stopped. control-openhands browser count 'testid=user-message' is 20, control-openhands browser text 'testid=chat-scroll-container' starts qa older 11 and holds none of qa older 1…qa older 10, and control-openhands browser bbox 'testid=chat-scroll-container' has scrollHeight 1576 against clientHeight 814: taller than the viewport, so the third page waits for the user. Run control-openhands browser scroll 'testid=chat-scroll-container' --by -100000: control-openhands browser network --filter 'events/search' --last 5 gains a request with timestamp__lt=<ts of qa older 11>, control-openhands browser count 'testid=user-message' is 29 and the text starts qa older 2 (that page held messages 2–10 plus the tail of the first message's events; not seen in two runs: if the first refusal ever logs six events instead of seven, the arithmetic gives count 30, text qa older 1 and a last page of only the SystemPromptEvent), and the view stays on qa older 11: control-openhands browser eval "document.querySelector('[data-testid=chat-scroll-container]').scrollTop" is 684, the height of the prepended rows. Scroll to the top again the same way: a fourth request (timestamp__lt= the oldest loaded event), the count is 30, the text starts qa older 1 and scrollTop is 76 (control-openhands browser screenshot 'testid=chat-interface' --feature F06.load-older-history --name scroll-top-loaded-qa-older-1). A third scroll adds no request: the short page ended the paging. control-openhands browser count 'testid=loading-older-events' is 0 after each scroll because the Fetching older messages… row lasts a few tens of milliseconds on a local stack; to see it, scroll with control-openhands browser scroll 'testid=chat-scroll-container' --by -100000 --observe 'testid=loading-older-events' --observe-ms 1500 instead (both scrolls were driven this way here, with the same counts), whose observed goes <absent> at 0 ms, Fetching older messages… at about 20 ms and <absent> again within about 40 to 120 ms (116 ms on the first scroll, 39 on the second; observedBy mutation); the timestamp__lt= network rows remain the durable proof. Return to the 16-command conversation for the next bullet: control-openhands browser goto /conversations/<history-id> and control-openhands browser wait 'testid=chat-input'.
Scroll to bottomF06.scroll-to-bottomMapped behavior
Scenario F06--scroll-to-bottom · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoChoose “Event group toggle”.
- DoScroll “Chat scroll container” as specified in the recipe.
- DoScroll “Chat scroll container” as specified in the recipe.
- DoChoose “Scroll to bottom”.
Expect
scrolling up shows a scroll-to-bottom button; clicking it returns to the bottom and hides it.
Exact CLI recipe & bug checks
On the 16-command conversation (<history-id>; in a model-free run this bullet is blocked with the automatic-fetch half, prerequisite DEEPSEEK_API_KEY), control-openhands browser click 'testid=event-group-toggle' makes the transcript taller than the viewport while the view stays where it was. Run control-openhands browser scroll 'testid=chat-scroll-container' --by 2000 first (the button only reacts to scroll events, so a page that is already at the top never shows it): count 'testid=scroll-to-bottom' is 0. Run control-openhands browser scroll 'testid=chat-scroll-container' --by -2000; the count is 1. control-openhands browser click 'testid=scroll-to-bottom', then the count is 0 and control-openhands browser eval "(()=>{const e=document.querySelector('[data-testid=chat-scroll-container]');return Math.round(e.scrollHeight-e.scrollTop-e.clientHeight)})()" is 0.
Confirmation mode onF06.confirmation-modeKnown issueBlocked
Scenario F06--confirmation-mode-on · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoOpen /settings/verification.
- DoChoose “Verification settings screen” / “Confirmation Mode”.
- DoChoose “Sdk section all toggle”.
- DoChoose “Verification settings screen” / “Show suggestions”.
- DoChoose the listbox / “None”.
- DoChoose “Verification settings screen” / “Save”.
- DoStart a conversation with the prompt “Run the terminal command 'echo qa-confirm-ok' and reply with its output.”.
- DoUse browser tooltip as specified in the exact recipe.
- DoHover over “Message”.
- DoChoose “Action reject”.
- WaitWait for “Action confirm” to appear.
- DoChoose “Action confirm”.
- WaitWait for “Action confirm” to appear.
- DoPress Control+Enter.
- WaitWait for “Action confirm” to appear.
- DoPress Control+Shift+Backspace.
- DoReload the page.
- DoChoose the element identified in the CLI recipe.
- SetupUse the phone viewport.
Expect
with confirmation mode on, a lock chip sits above the composer and each action waits for "Do you want to continue with this action?" with Cancel and Continue (hints ⇧⌘⌫ / ⌘↩ on macOS, Ctrl+⇧+⌫ / Ctrl+↩ elsewhere, hidden below 640 px); a cancelled action is marked Rejected and counts as resolved in its group.
Linked known issues
Known issue OpenHands/software-agent-sdk#5092 (F06.confirmation-mode): A revocation message sent while waiting for confirmation can implicitly approve the pending action. Live prerequisite: A working model proposing a real confirmation-gated marker action, then a revocation user message.
Known issue OpenHands/software-agent-sdk#4532 (F06.confirmation-mode): After restarting while awaiting confirmation, approval can execute the tool but the next model request omits its assistant call. Live prerequisite: A real confirmation-paused action persisted across backend restart, then approval and a working provider call.
Exact CLI recipe & bug checks
Arrange through the settings UI, in this order (the Security Analyzer field appears only once Confirmation Mode is on): control-openhands browser goto /settings/verification, control-openhands browser click 'testid=verification-settings-screen >> text=Confirmation Mode', control-openhands browser click 'testid=sdk-section-all-toggle', control-openhands browser click 'testid=verification-settings-screen >> role=button[name="Show suggestions"]', control-openhands browser click 'role=listbox >> role=option[name="None"]', control-openhands browser value 'testid=verification-settings-screen >> role=combobox[name="Security Analyzer"]' (None), control-openhands browser click 'testid=verification-settings-screen >> testid=save-button'; control-openhands api GET /api/settings shows "confirmation_mode": true and "security_analyzer": "none" (None means every action is confirmed; the default LLM analyzer only stops HIGH-risk actions, so echo would run unprompted). Run control-openhands conversation start --prompt "Run the terminal command 'echo qa-confirm-ok' and reply with its output." --wait --timeout 180: it returns "status": "waiting_for_confirmation". control-openhands browser text 'testid=chat-scroll-container' ends Do you want to continue with this action? Cancel Ctrl+⇧+⌫ Continue Ctrl+↩ on Linux (Cancel ⇧⌘⌫ Continue ⌘↩ on macOS, #18127), control-openhands browser tooltip 'testid=action-confirm-button' is Confirm the requested action, and a lock icon sits above the composer (screenshot). The tooltip stays open over the Cancel button, so move the pointer first: control-openhands browser hover 'testid=chat-input'. Run control-openhands browser click 'testid=action-reject-button': status idle, the buttons are gone, conversation events <id> --last 6 ends with UserRejectObservation, and the action's row reads Rejected (control-openhands browser count 'testid=rejected-indicator' is 1) with no spinner. Send Run the terminal command 'echo qa-confirm-two' and reply with its output. through testid=chat-input / testid=submit-button, control-openhands browser wait 'testid=action-confirm-button' --timeout 20000, control-openhands browser click 'testid=action-confirm-button'; about 15 s later the status is finished and the events contain the observation qa-confirm-two. A rejection inside a group resolves it too: send Run the terminal command 'echo qa-g1'. After it finishes, run the terminal command 'echo qa-g2' as a separate tool call. Then reply DONE., continue the first action (browser wait 'testid=action-confirm-button', then control-openhands browser press Control+Enter, which is also the next bullet's shortcut) and cancel the second (browser wait 'testid=action-confirm-button' --timeout 60000, control-openhands browser press Control+Shift+Backspace): status idle, control-openhands browser text 'testid=event-group >> nth=-1' ends 2 actions completed, control-openhands browser count 'testid=event-group >> nth=-1 >> .animate-spin' is 0, the same after browser reload, and the expanded group (control-openhands browser click 'testid=event-group >> nth=-1 >> button >> nth=0') shows Rejected on the qa-g2 row (browser screenshot 'testid=chat-interface' --feature F06.confirmation-mode --name group-after-reload). Each button keeps its label and hint on one line: while the prompt is waiting, control-openhands browser eval "[...document.querySelectorAll('[data-testid=action-reject-button],[data-testid=action-confirm-button]')].map(b=>b.offsetHeight+'/'+b.scrollHeight).join(' ')" is 26/26 26/26 at desktop, also with the side panel open (testid=right-panel-toggle). At phone width (browser viewport phone) the shortcut hints are hidden (no hardware keyboard), the buttons read Cancel and Continue, and browser text 'testid=chat-scroll-container' ends Do you want to continue with this action? Cancel Continue (screenshot the awaiting prompt at phone width).
Confirmation shortcutsF06.confirmation-shortcutsMapped behavior
Scenario F06--confirmation-shortcuts · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- WaitWait for “Action confirm” to appear.
- DoPress Meta+Enter.
- WaitWait for the conversation to finish.
- DoOpen /settings/verification.
Expect
Cmd+Enter or Ctrl+Enter continues and Shift+Cmd+Backspace or Shift+Ctrl+Backspace cancels a pending action.
Exact CLI recipe & bug checks
The previous bullet resolved its action, so arrange a new one in the same conversation: send Run the terminal command 'echo qa-confirm-three' and reply with its output. through testid=chat-input / testid=submit-button and control-openhands browser wait 'testid=action-confirm-button' --timeout 20000. control-openhands browser text 'testid=action-confirm-button' is Continue Ctrl+↩ off macOS. Ctrl works off macOS (#17928, fixed by #18127): the previous bullet's Control+Enter continued and Control+Shift+Backspace cancelled. control-openhands browser press Meta+Enter continues too: control-openhands conversation wait <id> --fresh --timeout 120 ends finished and control-openhands conversation events <id> --kinds ObservationEvent contains qa-confirm-three. Restore afterwards: browser goto /settings/verification, click testid=sdk-section-all-toggle, Show suggestions, role=listbox >> role=option[name="LLM"], then testid=verification-settings-screen >> text=Confirmation Mode, then save-button; api GET /api/settings shows "confirmation_mode": false, "security_analyzer": "llm".
Branch from hereF06.branch-from-hereKnown issueBlocked
Scenario F06--branch-from-here · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoOpen /conversations/<id>.
- DoHover over “Agent message” / match 1.
- DoChoose “Agent message” / match 1 / “Branch from here”.
- WaitWait for “User message” to appear.
- DoOpen /conversations/<id>.
- DoHover over “User message” / match 2.
- DoChoose “User message” / match 2 / “Branch from here”.
- DoPress Enter then --selector then testid=chat-input.
- WaitWait for the conversation to finish.
Expect
hovering a message shows Branch from here; on an agent message it forks inclusively; on a user message it forks without that message and pre-fills its text in the new composer, ready to send. The branch is titled "<title> (branch)" and keeps working.
Linked known issues
Known issue OpenHands/software-agent-sdk#5465 (F06.branch-from-here): Branch from here can lose encrypted LLM credentials, so the fork fails its first turn with LLMAuthenticationError. Live prerequisite: A working cipher-backed model conversation, then fork and a successful turn on the fork.
Known issue OpenHands/software-agent-sdk#5806 (F06.branch-from-here): editing a displayed user message first reads its parent event; if that event has become unreadable/missing on the server, the action can fail with 500 instead of a missing-event response. Live prerequisite: an owned persisted conversation with a still-displayed user message whose event storage is missing, followed by the actual Edit action. A random unknown-ID API call proves neither this UI path nor the batch-get contract, which Canvas does not call.
Exact CLI recipe & bug checks
Open a conversation with at least two exchanges (the OK/PONG one above): control-openhands browser goto /conversations/<id> with its id (the previous bullet left /settings/verification). Agent message: control-openhands browser hover 'testid=agent-message >> nth=0', control-openhands browser click 'testid=agent-message >> nth=0 >> role=button[name="Branch from here"]' --expect-url '/conversations/(?!<id>)'; after control-openhands browser wait 'testid=user-message' (the history mounts a moment after the URL changes, so an immediate count reads 0) the new conversation has count 'testid=user-message' 1, the agent message OK, and browser eval "document.title" contains (branch). User message (edit): back on the OK/PONG conversation (control-openhands browser goto /conversations/<id>; the branch above has one user message), control-openhands browser hover 'testid=user-message >> nth=1' and control-openhands browser click 'testid=user-message >> nth=1 >> role=button[name="Branch from here"]' --observe '[data-testid=chat-input]' --observe-ms 4000: observed shows the composer change from empty to Reply with only the word PONG. and stay. The new conversation (<branch-id> from browser url) omits that message, control-openhands browser text 'testid=chat-input' is Reply with only the word PONG., and nothing was sent: control-openhands conversation events <branch-id> --kinds MessageEvent lists only the copied first exchange (count 2: the long message that starts Reply with only the word OK., then OK; no PONG). control-openhands browser enabled 'testid=submit-button' should be true with the prefilled text; today it is false until the text is edited or the page is reloaded (see Gotchas). Send it with control-openhands browser press Enter --selector 'testid=chat-input' (Enter sends whatever the Send button's state is) and control-openhands conversation wait <branch-id> --fresh --timeout 120: expected finished with a reply; today it ends error (Agent Server bug, see Gotchas).
ImagesF06.image-attachmentsMapped behavior
Scenario F06--images · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- SetupCreate the disposable image fixture specified in the recipe.
- DoUse browser upload as specified in the exact recipe.
- DoHover over “Image preview”.
- DoChoose “Expand image”.
- DoPress Escape.
- DoChoose “Image lightbox close”.
Expect
images sent with a message show as thumbnails in the bubble; expand opens a lightbox that closes with X or Escape.
Exact CLI recipe & bug checks
Run control-openhands fixture image --name qa-f06-img (prints the PNG path), control-openhands browser upload 'testid=upload-image-input' <png path> (a thumbnail appears in the composer), fill testid=chat-input with Reply with only the word SEEN. and click testid=submit-button. The user bubble contains image-carousel / image-preview. Run control-openhands browser hover 'testid=image-preview', control-openhands browser click 'testid=expand-image-button' (count 'testid=image-lightbox' is 1), control-openhands browser press Escape (0), click expand-image-button again and control-openhands browser click 'testid=image-lightbox-close' (0).
Error eventsF06.error-eventsKnown issueBlocked
Scenario F06--error-events · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- WaitWait for the conversation to finish.
Expect
a conversation error (ConversationErrorEvent) surfaces as a warning banner and the composer status Error.
Linked known issues
Known issue #16686 (F06.error-events): LLM errors not propagated to the frontend. Live prerequisite: A Nebius credential and its rejected DeepSeek-V3 request, or the original complete emitted error fixture.
Known issue #15626 (F06.error-events): Conversation can fail with LiteLLM auth error when database server at 127.0.0.1:5432 is unreachable. Live prerequisite: The original LiteLLM proxy/database-failure event or a disposable proxy with its PostgreSQL dependency unavailable.
Known issue OpenHands/software-agent-sdk#5447 (F06.error-events): An Anthropic Claude Opus5.5 conversation can fail with400 when its next request ends in an assistant prefill. Live prerequisite: Anthropic Claude Opus5.5 credentials and a real trailing-assistant-prefill request.
Known issue OpenHands/software-agent-sdk#5225 (F06.error-events): A provider-rejected image tool result can leave the conversation permanently failing on every resume. Live prerequisite: A real provider-rejected poisoned image tool_result in a disposable persisted history, followed by retry.
Known issue OpenHands/software-agent-sdk#4965 (F06.error-events): A strict OpenAI-compatible provider can reject blank content in the saved history, preventing later turns. Live prerequisite: A strict OpenAI-compatible provider and a real history containing blank text or empty content arrays.
Known issue OpenHands/software-agent-sdk#4558 (F06.error-events): A saved encrypted fallback profile can send ciphertext as its key and fail authentication, leaving only the primary error visible. Live prerequisite: A working encrypted saved fallback profile and real primary-model failure that triggers fallback resolution.
Known issue OpenHands/software-agent-sdk#5740 (F06.error-events): a Responses display delta without item_id can terminate the turn with an AttributeError. Live prerequisite: the genuine provider/transformer streaming path producing that incomplete event; the report did not retain its original raw SSE. A credential-free parser diagnostic proves the SDK exception, not a Canvas model turn.
Exact CLI recipe & bug checks
Any run that ends in a ConversationErrorEvent (today: any message sent in a branched conversation) leaves conversation wait <id> at "status": "error"; control-openhands browser testids 'testid=chat-interface' lists error-message-banner with warning-message-banner-icon and the error's text, and the composer status reads Error. Read the code with control-openhands api GET "/api/conversations/<id>/events/search?limit=3&sort_order=TIMESTAMP_DESC".
Skill installed bannerF06.skill-install-bannerMapped behavior
Scenario F06--skill-installed-banner · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoStart a conversation with the prompt “$(cat <prompt file>)”.
- DoChoose “Skill install restart action”.
- DoOpen /conversations/<id>.
- DoChoose “Skill install restart dismiss”.
- DoReload the page.
Expect
after the agent installs a skill into the workspace, a banner offers "Start new conversation with this skill" (same workspace) and Close.
Exact CLI recipe & bug checks
The banner keys on the installer's success line in terminal output. Put this one line in a prompt file: Run exactly this one terminal command, verbatim, then reply DONE: mkdir -p .agents/skills/qa-skill && printf '# qa-skill\n' > .agents/skills/qa-skill/SKILL.md && echo "✅ Successfully installed 'qa-skill' to $PWD/.agents/skills/qa-skill", then run control-openhands conversation start --prompt "$(cat <prompt file>)" --wait --timeout 150. control-openhands browser text 'testid=skill-install-restart-banner' reads Installed to this workspace: qa-skill. Skills load when a conversation starts, so this conversation can't use them yet. Run control-openhands browser click 'testid=skill-install-restart-action' --expect-url '/conversations/(?!<id>)'; control-openhands api GET /api/conversations/<new id> has the same workspace.working_dir as the source. Back on the source (control-openhands browser goto /conversations/<id>), control-openhands browser click 'testid=skill-install-restart-dismiss' hides it (count 0); after browser reload it is back (dismissal is session-only by design).
No LLMF06.llm-not-configured-bannerMapped behavior
Scenario F06--no-llm · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoFollow the onboarding setup in the recipe.
- DoChoose “Conversation panel new thread picker”.
- DoChoose “Launch no workspace”.
- DoEnter “the value specified in the recipe” in the current page.
- DoChoose “Home llm not configured action”.
Expect
with no usable LLM, a conversation shows "Your LLM isn't set up yet..." with Set up LLM; suggestions are hidden and sending is disabled.
Exact CLI recipe & bug checks
On the fresh no-LLM run: control-openhands onboard --skip, control-openhands browser click 'testid=conversation-panel-new-thread-picker', control-openhands browser click 'testid=launch-no-workspace' --expect-url '/conversations/'. control-openhands browser text 'testid=home-llm-not-configured-banner' reads Your LLM isn't set up yet, so conversations won't run. Finish setup to get started. Set up LLM; count 'testid=chat-suggestions' is 0; the composer cannot take text (control-openhands browser attr 'testid=chat-input' contenteditable is false, so browser fill fails) and control-openhands browser enabled 'testid=submit-button' is false. control-openhands browser click 'testid=home-llm-not-configured-action' then browser url ends in /settings/llm.
Critic (`F06.critic-result`, blocked)F06.critic-resultBlocked
Scenario F06--critic-f06-critic-result-blocked · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoOpen /settings/verification.
- DoChoose the element identified in the CLI recipe.
- DoStart a conversation with the prompt “Create qa_critic.txt containing hi, then finish.”.
- DoChoose “Expand details” / match 1.
Expect
with the critic enabled, agent messages and the finishing action each carry a "Critic: agent success likelihood" block (stars labelled Score: <N.N>%, one decimal, with the percentage beside them); Expand details lists the issue categories (Potential Issues, Infrastructure, Likely Follow-up).
Exact CLI recipe & bug checks
Toggle the hidden switch through its label on /settings/verification: control-openhands browser goto /settings/verification, control-openhands browser click 'label:has([data-testid="sdk-settings-verification.critic_enabled"])', save, api GET /api/settings shows "critic_enabled": true. Without a Critic API key a Reply with only the word OK. conversation renders no critic block. With a key and a model, run control-openhands conversation start --prompt "Create qa_critic.txt containing hi, then finish." --wait --timeout 180: every scored event carries a block, so control-openhands browser count 'text=Critic: agent success likelihood' is at least 1 and a finishing action (finish tool row) shows its own block next to the agent's message; the stars carry the score as their label, which an ARIA snapshot does not list: control-openhands browser count '[aria-label^="Score: "]' is at least 2 and control-openhands browser attr '[aria-label^="Score: "] >> nth=0' aria-label reads Score: <N.N>% with one decimal (Score: 82.0%), the same number as the (82.0%) text beside the stars. control-openhands browser click 'role=button[name="Expand details"] >> nth=0' opens the categories Potential Issues:, Infrastructure: and Likely Follow-up: with their named issues in control-openhands browser text 'testid=chat-scroll-container', and the button's label flips to Collapse details. Blocked here: DEEPSEEK_API_KEY and a Cloud critic key. Turn it off again the same way.
PhoneF06.phoneMapped behavior
Scenario F06--phone · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- SetupUse the phone viewport.
- DoOpen /conversations/<stream.
- SetupUse the desktop viewport.
Expect
at 390 px the transcript fits the viewport; wide tables scroll inside their own container.
Exact CLI recipe & bug checks
Run control-openhands browser viewport phone, open the table conversation (control-openhands browser goto /conversations/<stream id>), control-openhands browser bbox 'testid=chat-interface' (insideViewport true, pageHorizontalOverflow false) and control-openhands browser bbox 'testid=markdown-table-scroll' (scrollWidth larger than clientWidth: the table scrolls inside itself). control-openhands browser screenshot --feature F06.phone --name conversation, then control-openhands browser viewport desktop.
After each groupF06 · setup / cleanupSupport step
Scenario F06--after-each-group · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- SetupChange the run-owned service state: stop.
Expect
Complete this support step before continuing to the next scenario.
Exact CLI recipe & bug checks
After each group run control-openhands browser errors --app-only. Outages driven with service stop log connection-refused console errors by design; pageErrors must stay 0.
Planner discovery with many childrenF06.tool-visualizersKnown failureKnown issueBlocked
Scenario F06--planner-discovery-with-many-children · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoExpected: reloading a parent still discovers its
plannerparent-tagged planner when it has100 child conversations. Known failure (reproduced 2026-10-16, SDK 1.55.0, OpenHands/software-agent-sdk#5790): the unchunked100-ID request returns500, showsInternal Server Errorand omits the planner session connection. A99-child control returns200 and opens both planner and parent sockets. These are genuine idle API-created children, without a model run or browser-storage changes; a new parent ID has no UI-created planner hint.
Expect
expanding a tool row shows a rich body: terminal command and output (non-zero exit badge exit N), file-editor path chip and content.
Known bug observations
Known failure (reproduced 2026-10-16, SDK 1.55.0, OpenHands/software-agent-sdk#5790): the unchunked100-ID request returns500, shows Internal Server Error and omits the planner session connection. A99-child control returns200 and opens both planner and parent sockets. These are genuine idle API-created children, without a model run or browser-storage changes; a new parent ID has no UI-created planner hint.
Linked known issues
Known issue #17492 (F06.tool-visualizers): Failed browser, glob, grep, and file editor tool calls render as successful in chat. Live prerequisite: A working model emitting failed browser, glob, grep, planning-editor and file-editor observations with is_error=true.
Known issue #17490 (F06.tool-visualizers): Terminal commands that fail with an ordinary non-zero exit render as successful in chat. Live prerequisite: A working model producing a completed terminal command with nonzero exit_code and is_error=false.
Known issue #16341 (F06.tool-visualizers): Code-explorer subagent frequently errors. Live prerequisite: A working model with native code-explorer delegation and the original repository task; no delegated-tool output can be generated without credentials.
Known issue OpenHands/software-agent-sdk#5108 (F06.tool-visualizers): The native SDK Gemini editor can report success while corrupting unrelated invalid-UTF8 bytes in the file. Live prerequisite: An Agent Server configured with the native Gemini edit tool set and a real tool edit of an invalid-UTF8 fixture; Gemini ACP uses a different external executor.
Known issue OpenHands/software-agent-sdk#5096 (F06.tool-visualizers): The canonical agent file editor can replace a symlink instead of editing its target; Undo does not restore the link. Live prerequisite: A working model invoking canonical file-editor str_replace/insert through a real symlink, then undo.
Known issue OpenHands/software-agent-sdk#5074 (F06.tool-visualizers): After server restart a native delegated child can remain Running while the parent loses its task mapping/result. Live prerequisite: A native delegated child still running during an owned server crash and parent recovery.
Known issue OpenHands/software-agent-sdk#5038 (F06.tool-visualizers): The agent file editor can refuse valid Chinese UTF8 text as binary when its sample splits a character; Canvas Files uses a separate preview detector. Live prerequisite: A live canonical file-editor view/edit on the specific UTF8 sample alignment; Canvas Files preview uses a separate NUL sniff, not this validator.
Known issue OpenHands/software-agent-sdk#4983 (F06.tool-visualizers): Agent grep can silently miss extended-regex matches when its runtime falls back to system grep. Live prerequisite: A runtime without ripgrep and a live agent grep call using extended regex and escaped literal punctuation.
Known issue OpenHands/software-agent-sdk#4668 (F06.tool-visualizers): A planning editor card can lose old/new content and fail to show the complete edit diff. Live prerequisite: A live planner create/edit yielding PlanningFileEditorObservation with old/new contents for the chat diff.
Known issue OpenHands/software-agent-sdk#4663 (F06.tool-visualizers): Parallel Python-fallback glob calls can change process cwd and return files from the wrong workspace. Live prerequisite: Concurrent real glob/tool calls in distinct workspaces with the Python fallback active.
Known issue OpenHands/software-agent-sdk#4661 (F06.tool-visualizers): Same-name project subagents can resolve to another conversation workspace definition. Live prerequisite: Two native task-enabled conversations with same-name project subagent definitions and a working model for delegation.
Known issue OpenHands/software-agent-sdk#4583 (F06.tool-visualizers): An agent editor insertion at EOF can join onto the last line when the file has no trailing newline. Live prerequisite: A working model invoking file-editor insert at EOF on a file with no trailing newline.
Known issue OpenHands/software-agent-sdk#4540 (F06.tool-visualizers): Synthetic GLM/Kimi tool calls can render as raw tool_call syntax instead of executing. Live prerequisite: Synthetic provider credentials and a GLM/Kimi turn emitting the raw tool_call wrapper.
Exact CLI recipe & bug checks
Expected: reloading a parent still discovers its plannerparent-tagged planner when it has100 child conversations. Known failure (reproduced 2026-10-16, SDK 1.55.0, OpenHands/software-agent-sdk#5790): the unchunked100-ID request returns500, shows Internal Server Error and omits the planner session connection. A99-child control returns200 and opens both planner and parent sockets. These are genuine idle API-created children, without a model run or browser-storage changes; a new parent ID has no UI-created planner hint.
sh known-bug F06.tool-visualizers planner-batch-100
control-openhands browser viewport desktop
QA_ID="$(python3 - <<'PY'
import json, os, pathlib, subprocess, tempfile
base = pathlib.Path(os.environ["OH_VERIFY_RUN"]) / "workspace"
base.mkdir(exist_ok=True)
fixture = pathlib.Path(tempfile.mkdtemp(prefix="qa-surface-", dir=base))
llm = {"model":"openai/gpt-4o", "base_url":"http://127.0.0.1:1/v1", "api_key":"qa-not-real", "num_retries":0, "timeout":2}
def create(workspace, parent=None, tags=None):
payload = {"agent":{"kind":"Agent", "llm":llm, "tools":[]}, "workspace":{"working_dir":str(workspace)}, "autotitle":False}
if parent: payload["parent_conversation_id"] = parent
if tags: payload["tags"] = tags
request = fixture / "request.json"
request.write_text(json.dumps(payload))
result = subprocess.run(["control-openhands", "api", "POST", "/api/conversations", "--write", "--data", "@"+str(request), "--pick", "id"], check=True, capture_output=True, text=True)
return json.loads(result.stdout)["value"]
def git(directory, *args):
subprocess.run(["git", "-C", str(directory), *args], check=True, capture_output=True)
workspace = fixture / "workspace"
workspace.mkdir()
parent = create(workspace)
for index in range(100):
create(workspace, parent, {"plannerparent":parent} if index == 0 else None)
print(parent)
PY
)"
control-openhands assert --field value --equals 100 -- api GET "/api/conversations/$QA_ID" --pick sub_conversation_ids.length
QA_PLANNER="$(control-openhands api GET "/api/conversations/$QA_ID" --pick sub_conversation_ids.0 | python3 -c 'import json,sys; print(json.load(sys.stdin)["value"])')"
control-openhands browser network --clear
control-openhands browser goto "/conversations/$QA_ID"
control-openhands browser wait 'testid=chat-input'
control-openhands browser screenshot --feature F06.tool-visualizers --name planner-batch-100
control-openhands assert --field recent.0.status --equals 200 --bug-shows 500 -- browser network --filter "ids=$QA_PLANNER" # bug
Hook eventsF06.hook-eventsBehavior contract
Scenario F06--contract-f06-hook-events · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do
This behavior has no separately named recipe in this snapshot. Use the family’s entry points and the contract below; the page does not invent a replay.
Expect
hook executions render as their own rows in the stream.
Corrective nudgeF06.corrective-nudgeBehavior contract
Scenario F06--contract-f06-corrective-nudge · Permalink · Source
Given
Use F06’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do
This behavior has no separately named recipe in this snapshot. Use the family’s entry points and the contract below; the page does not invent a replay.
Expect
when the model answers with neither a message nor a tool call, the SDK's nudge (Your last response did not include a function call or a message. …) shows as a muted, italic role=note line with an info icon (corrective-nudge-message), not as a user bubble (#17864).
Family notes, prerequisites for known issues & limits Known failureKnown issueBlocked
control-openhands conversation events <id>returns the last 25 events unless you pass--last N; it reads the server's 100-event pages untilNrows are in hand (pagessays how many it read,more: truethat older pages remain), so--last 500covers a 150-event conversation in two pages (total152,pages2,morefalseon the Older history conversation;--last 25there reads one page,moretrue) and--from-startreads from the oldest.--kindsand--grepfilter before--lastcounts, and pages are read until--lastmatching rows are in hand:--kinds MessageEvent --last 3is the three newest messages,--kinds MessageEvent --last 500all thirty, and--grep 'qa older 1'reachesqa older 1on the oldest page with the default--last(count11,pages2). Event texts are cut at about 160 characters.--observetakes the same selector syntax as the verbs:testid=and plain CSS segments (joined with>>) are watched by an in-page MutationObserver (observedBymutation; the Older history bullet's--observe 'testid=loading-older-events'is one), while other engines (role=,text=,nth=) are polled every 20 ms instead (not driven in this family).role=button[name="Expand"]also matches "Expand thinking" and "Expand actions"; use[exact]to target tool rows. Each click flips the row to Collapse, sonth=0walks to the next collapsed row.- Tool rows show no success icon; the only indicators are the
exit Nbadge for non-zero exits and a clock icon (status-icon) for timeouts. A single action is not grouped: it renders as a bare row withoutevent-group. - The model writes the tool titles and may append to commands (it turned
ls /missingintols ...; echo "EXIT:$?", exit 0). Ask for the command "verbatim, with nothing appended" when the exit code matters. - The agent itself may open files with
canvas_ui_control(a third action in the echo/file run). Close the file tab before proving that a path link opens it. - On Settings → Verification the Security Analyzer field (All tab) is shown only while Confirmation Mode is on: switch Confirmation Mode on first, then pick the analyzer. Check
api GET /api/settingsafter saving. - Confirmation mode with the default Security Analyzer
LLMuses ConfirmRisky(HIGH): harmless commands run without a prompt. Set the analyzer toNoneto confirm every action, and restoreLLMafterwards. The composer stays editable while waiting (status "User needed"); the buttons read Cancel / Continue. - The confirm and cancel shortcuts accept Meta or Ctrl and ignore a held key's auto-repeat; once a response is sent the listener is detached, so a second press sends nothing (#18127). Hints read
Ctrl+↩/Ctrl+⇧+⌫off Apple platforms. - Known failure, Canvas UI (#18202, reproduced 2026-10-09; draft fix PR #18203): after a user-message branch the composer holds the prefilled text but Send stays disabled (
browser enabled 'testid=submit-button'isfalse, and a click on it times out withelement is not enabled). The prefill is a saved draft thatsrc/hooks/chat/use-draft-persistence.tswrites into the field without an input event, and the composer is reused across the client-side switch, so its Send state keeps the source conversation's empty field. Editing the text, a reload, or Enter in the field (browser press Enter --selector 'testid=chat-input') all work. A draft restored by a sidebar switch behaves the same (F05 Gotchas). - Known failure, likely Agent Server: every message in a branched conversation ends with
ConversationErrorEventLLMAuthenticationError("Your LLM API key appears to be invalid or has expired."), while the source conversation keeps working (OpenHands/software-agent-sdk#5465). Do not use branches as fixtures for other checks. - "The terminal session was reset because the underlying tmux server/session disappeared" (clock
status-iconon the row) appears after aservice stop agent-server/restartcycle. Sincelaunchgives each run its own shortTMUX_TMPDIR, other runs no longer cause it. The underlying product bug is still open (OpenHands/OpenHands#17946) for plainagent-canvasinstances:bin/agent-canvas.mjs→scripts/dev-with-automation.mjspassesTMUX_TMPDIR=<state>/tmuxto the Agent Server butensureDirectoriesnever creates it (dev-safe.mjsdoes), so tmux 3.4 falls back to the shared/tmp/tmux-<uid>/openhandssocket and another instance's restart or stop kills this run's windows. The model usually reruns the command; expect an extra action. - The
ConversationErrorEventbanner is not restored after navigating away and back (or a reload); read the error from the events API instead. - After
restart, reconnection takes about 20 s without a click; clickerror-message-banner-retryto reconnect at once. The error banner does not come back after a page reload once the stack is healthy. - "Sending..." lasts about 100 to 650 ms on a local stack, too short to hover and click
chat-message-stop; that stop path is not driven here. TheF06.pending-messagesfailure path needsservice stop agent-server. browser clockinstalls a fake clock that stays with the page across reloads and client-side navigation. Install it before thegotowhose page should see it, and put it back withbrowser clock --offset-ms 0plusbrowser reload; comparebrowser eval "new Date().toISOString()"withdate -uwhen in doubt.- The pending-message queue lives in page memory, so a failed bubble survives sidebar navigation and a backend
restartbut not abrowser reloadorbrowser gotoof the conversation (src/stores/optimistic-user-message-store.ts). Leave and return through the sidebar when provingF06.failed-send-persists. The sidebar card's title is the first message or model-written: click its link by id,'a[href*="/conversations/<id>"] >> nth=0'. - Both the pending bubble and the echoed message carry
testid=user-message, socount 'testid=user-message >> has-text=...'is one too high while a failed bubble is still shown; count attempts withconversation events <id> --kinds MessageEvent(user rows:"source": "user") and the bubbles withchat-message-retry. - Local
Stopcalls the Agent Server's interrupt (Cloud pauses the sandbox instead, seepauseConversation): a command already running in the terminal is cut off, recorded asAgentErrorEvent+InterruptEvent, and after Play the model only reports the interruption. - The no-LLM banner reuses the home banner's test ids (
home-llm-not-configured-banner). It exists only in a run where no LLM was ever configured: use a separatelaunch --new, never remove the profile from the shared run. - Read the clipboard with
control-openhands browser clipboard, neverbrowser eval "navigator.clipboard.readText()". An agent bubble that contains a code block holds twocopy-to-clipboardbuttons (the code block's is hidden until hovered): scope the bubble's with>> nth=0. - Plan-mode previews (
plan-preview-*) and the BTW side messages render in this stream but are reached from the composer's mode controls; they are not mapped here. - Known issue #18117 (
F06.thinking): Inline thinking leaks into the agent chat bubble. Live prerequisite: A live model that emits a leading inline think block on a tool-call turn, plus its finalized ActionEvent. - Known issue #17559 (
F06.load-older-history): History pagination can show both an action and its observation (ready-for-dev companion to #16947). Live prerequisite: A real persisted history page boundary splitting an action/observation or ACP started/completed pair; dummy-key errors do not produce these tool pairs. - Known issue #17492 (
F06.tool-visualizers): Failed browser, glob, grep, and file editor tool calls render as successful in chat. Live prerequisite: A working model emitting failed browser, glob, grep, planning-editor and file-editor observations with is_error=true. - Known issue #17490 (
F06.tool-visualizers): Terminal commands that fail with an ordinary non-zero exit render as successful in chat. Live prerequisite: A working model producing a completed terminal command with nonzero exit_code and is_error=false. - Known issue #17161 (
F06.markdown-rendering): markdown sanitizer lets untrusted content set rel="opener" and any target on links. Live prerequisite: A real reasoning, activated-skill, tool-description or error-detail body rendered without includeStandard. The attempted skills dialog exposed public skills only and did not reach the affected chat renderer. - Known issue #17026 (
F06.thinking): agent-canvas: final-reply thinking block missing for text-only turns (reasoning_content not read for MessageEvents). Live prerequisite: A reasoning-capable live model emitting nonempty MessageEvent.llm_message.reasoning_content on a text-only turn, followed by reload. - Known issue #16842 (
F06.reconnect): A late close from a replaced WebSocket marks the active connection disconnected. Live prerequisite: A transport fixture delaying an old socket close until its replacement is open; ordinary server restart cannot select that ordering. - Known issue #16719 (
F06.live-activity): Agent status always says "thinking" even when nothing is happening. Live prerequisite: The event/status trace and deterministic stalled or errored state from the original report; its trigger is unspecified. - Known issue #16686 (
F06.error-events): LLM errors not propagated to the frontend. Live prerequisite: A Nebius credential and its rejected DeepSeek-V3 request, or the original complete emitted error fixture. - Known issue #16582 (
F06.stop-resume): After reconnect, a maximum-iteration error can fail to pause the agent through the legacy event handler. Live prerequisite: A maximum-iteration legacy error delivered after socket replacement, with a real model run reaching that limit. - Known issue #16455 (
F06.events-match): Duplicate messages from the agent in 1.12.0. Live prerequisite: The original duplicated event/stream trace or deterministic prompt and timing; the report provides no reproducible trigger. - Known issue #16341 (
F06.tool-visualizers): Code-explorer subagent frequently errors. Live prerequisite: A working model with native code-explorer delegation and the original repository task; no delegated-tool output can be generated without credentials. - Known issue #15626 (
F06.error-events): Conversation can fail with LiteLLM auth error when database server at 127.0.0.1:5432 is unreachable. Live prerequisite: The original LiteLLM proxy/database-failure event or a disposable proxy with its PostgreSQL dependency unavailable. - Known issue #15606 (
F06.reload-mid-run): When re-loading ACP conversations, messages disappear or move. Live prerequisite: An authenticated Codex ACP session emitting intermediate messages while another window reloads history. - Known issue #15527 (
F06.reconnect): Canvas briefly shows "disconnected" when coming back to an idle tab. Live prerequisite: The original custom-domain deployment, idle duration and tab-suspension/network transition; those details are missing. - Known failure OpenHands/software-agent-sdk#5635 (
F06.reconnect), reproduced 2026-10-12 atfa1f724ab573with SDK 1.54.0 and Chromium 153.0.8010.12: sandbox pause returned 204 but an already communicating session socket remained open for more than 10 seconds. A new Canvas tab received a subsequentrun:falsemarker while the original tab did not. DELETE returned 200 and GET returned 404, yet both held sockets remained open for more than 10 seconds; Canvas later redirected by metadata polling. Recipe: open an idle dummy-profile conversation and confirm its marker/socket; POST/api/conversations/prepare-for-sandbox-pause, retain the original tab for 10 seconds, open a second tab, POST a new unique user event withrun:false, then compare the two transcripts. DELETE the owned conversation and read lifecycle metadata for another 10 seconds. Use--run <run>andbrowser events --kinds websocket-created,websocket-open,websocket-close --last 100; no frame bodies or auth data are recorded. Evidence:backfill-18309-surface-rechecks/F06.reconnect/pause-delete-sockets-5635.json,pause-new-tab-5635.png,pause-stale-tab-5635.png. The expected socket shutdown/reconnect remains unchanged; polling recovery must not be mistaken for socket closure. - Known issue OpenHands/software-agent-sdk#5465 (
F06.branch-from-here): Branch from here can lose encrypted LLM credentials, so the fork fails its first turn with LLMAuthenticationError. Live prerequisite: A working cipher-backed model conversation, then fork and a successful turn on the fork. - Known issue OpenHands/software-agent-sdk#5447 (
F06.error-events): An Anthropic Claude Opus5.5 conversation can fail with400 when its next request ends in an assistant prefill. Live prerequisite: Anthropic Claude Opus5.5 credentials and a real trailing-assistant-prefill request. - Known issue OpenHands/software-agent-sdk#5354 (
F06.event-groups): A mixed valid/invalid tool batch can be split into separate assistant messages, breaking call/result pairing on the next provider request. Live prerequisite: A live model turn mixing valid, invalid and valid tool calls in one response. - Known issue OpenHands/software-agent-sdk#5225 (
F06.error-events): A provider-rejected image tool result can leave the conversation permanently failing on every resume. Live prerequisite: A real provider-rejected poisoned image tool_result in a disposable persisted history, followed by retry. - Known issue OpenHands/software-agent-sdk#5108 (
F06.tool-visualizers): The native SDK Gemini editor can report success while corrupting unrelated invalid-UTF8 bytes in the file. Live prerequisite: An Agent Server configured with the native Gemini edit tool set and a real tool edit of an invalid-UTF8 fixture; Gemini ACP uses a different external executor. - Known issue OpenHands/software-agent-sdk#5096 (
F06.tool-visualizers): The canonical agent file editor can replace a symlink instead of editing its target; Undo does not restore the link. Live prerequisite: A working model invoking canonical file-editor str_replace/insert through a real symlink, then undo. - Known issue OpenHands/software-agent-sdk#5092 (
F06.confirmation-mode): A revocation message sent while waiting for confirmation can implicitly approve the pending action. Live prerequisite: A working model proposing a real confirmation-gated marker action, then a revocation user message. - Known issue OpenHands/software-agent-sdk#5074 (
F06.tool-visualizers): After server restart a native delegated child can remain Running while the parent loses its task mapping/result. Live prerequisite: A native delegated child still running during an owned server crash and parent recovery. - Known issue OpenHands/software-agent-sdk#5038 (
F06.tool-visualizers): The agent file editor can refuse valid Chinese UTF8 text as binary when its sample splits a character; Canvas Files uses a separate preview detector. Live prerequisite: A live canonical file-editor view/edit on the specific UTF8 sample alignment; Canvas Files preview uses a separate NUL sniff, not this validator. - Known issue OpenHands/software-agent-sdk#4990 (
F06.stop-resume): An ACP turn can keep executing tool calls while the UI already reports Paused. Live prerequisite: An authenticated Claude Code ACP turn executing separate marker-writing tools while Pause is pressed. - Known issue OpenHands/software-agent-sdk#4983 (
F06.tool-visualizers): Agent grep can silently miss extended-regex matches when its runtime falls back to system grep. Live prerequisite: A runtime without ripgrep and a live agent grep call using extended regex and escaped literal punctuation. - Known issue OpenHands/software-agent-sdk#4965 (
F06.error-events): A strict OpenAI-compatible provider can reject blank content in the saved history, preventing later turns. Live prerequisite: A strict OpenAI-compatible provider and a real history containing blank text or empty content arrays. - Known issue OpenHands/software-agent-sdk#4777 (
F06.stop-resume): Stop can still start a queued tool after it finishes waiting on a resource lock. Live prerequisite: A live batch with a second real tool waiting on a declared-resource lock when Stop is pressed. - Known issue OpenHands/software-agent-sdk#4668 (
F06.tool-visualizers): A planning editor card can lose old/new content and fail to show the complete edit diff. Live prerequisite: A live planner create/edit yielding PlanningFileEditorObservation with old/new contents for the chat diff. - Known issue OpenHands/software-agent-sdk#4663 (
F06.tool-visualizers): Parallel Python-fallback glob calls can change process cwd and return files from the wrong workspace. Live prerequisite: Concurrent real glob/tool calls in distinct workspaces with the Python fallback active. - Known issue OpenHands/software-agent-sdk#4661 (
F06.tool-visualizers): Same-name project subagents can resolve to another conversation workspace definition. Live prerequisite: Two native task-enabled conversations with same-name project subagent definitions and a working model for delegation. - Known issue OpenHands/software-agent-sdk#4583 (
F06.tool-visualizers): An agent editor insertion at EOF can join onto the last line when the file has no trailing newline. Live prerequisite: A working model invoking file-editor insert at EOF on a file with no trailing newline. - Known issue OpenHands/software-agent-sdk#4558 (
F06.error-events): A saved encrypted fallback profile can send ciphertext as its key and fail authentication, leaving only the primary error visible. Live prerequisite: A working encrypted saved fallback profile and real primary-model failure that triggers fallback resolution. - Known issue OpenHands/software-agent-sdk#4540 (
F06.tool-visualizers): Synthetic GLM/Kimi tool calls can render as raw tool_call syntax instead of executing. Live prerequisite: Synthetic provider credentials and a GLM/Kimi turn emitting the raw tool_call wrapper. - Known issue OpenHands/software-agent-sdk#4532 (
F06.confirmation-mode): After restarting while awaiting confirmation, approval can execute the tool but the next model request omits its assistant call. Live prerequisite: A real confirmation-paused action persisted across backend restart, then approval and a working provider call. - Known issue OpenHands/software-agent-sdk#4077 (
F06.reload-mid-run): Retried or late streaming deltas can duplicate live text; a stalled subscriber can delay Stop and finalization. Live prerequisite: A working stream interrupted/retried or delivered to a controlled stalled subscriber, with delta/state-order capture. - Known issue #16947 (
F06.load-older-history): Loading older history can leave both the action and its newer observation visible because UI projection depends on arrival order. Live prerequisite: A real long conversation whose REST boundary splits an action/observation pair; keep companion #17559 linked separately. - Known issue #18445 (
F06.load-older-history,F06.reconnect): a failed initial Cloud history search can be swallowed into an empty page while the runtime count succeeds, advancingafter_seqpast history that was never displayed. Live prerequisite: a Cloud account and owned conversation with a genuinely failing App APIevents/searchand a successful runtimeevents/count. The Local failure path throws instead and is not an equivalent check. - Known issue OpenHands/software-agent-sdk#5969 (
F06.stop-resume): a conversation can report finished but reject Play with already-running, or delay a follow-up, while a slow event webhook is still pending. Live prerequisite: a genuinely successful model run on an owned backend with a bounded slow webhook receiver, then Play or send during that completion window. - Known issue OpenHands/software-agent-sdk#5740 (
F06.error-events): a Responses display delta withoutitem_idcan terminate the turn with an AttributeError. Live prerequisite: the genuine provider/transformer streaming path producing that incomplete event; the report did not retain its original raw SSE. A credential-free parser diagnostic proves the SDK exception, not a Canvas model turn. - Known issue OpenHands/software-agent-sdk#5806 (
F06.branch-from-here): editing a displayed user message first reads its parent event; if that event has become unreadable/missing on the server, the action can fail with 500 instead of a missing-event response. Live prerequisite: an owned persisted conversation with a still-displayed user message whose event storage is missing, followed by the actual Edit action. A random unknown-ID API call proves neither this UI path nor the batch-get contract, which Canvas does not call. - Session reconnects use the latest received
after_seqcursor (F06.reconnect). For a local metadata check, retain the open page, send genuinerun:falseuser events, record/events/count, restart the owned backend and inspect finite WebSocket lifecycle metadata: the reconnect cursor is the previous count minus one and a subsequent recovery marker appears once. This does not measure frame-body replay or Cloud behavior. - Stop persists
pausedbefore an in-flight model request completes (F06.stop-resume); the persisted state and reloaded Stopped UI survive an Agent Server restart. A bounded credential-free check can use the tracked accept-only endpoint and an exact listener PID proven to descend from the owned launcher; never kill by pattern or resume the paused conversation.
Before you start
Use an OpenHands checkout at the source revision, with Node 24+, installed npm dependencies, uv / uvx, and a supported Chromium browser. The verification skill is the complete setup contract.
export PATH="$PWD/.agents/skills/verify-openhands/scripts:$PATH"
export OH_VERIFY_RUN=$(control-openhands launch --new --print-run)
control-openhands doctor
control-openhands onboard --skip
# Configure a model only for a scenario that needs one:
control-openhands llm preset deepseek
# When the run is complete:
control-openhands stop
F01 walks onboarding instead of skipping it. No-model checks run before model configuration. Keep OH_VERIFY_RUN set so commands address your own run. Desktop is 1440 × 1000; phone is 390 × 844. Model-backed checks need the configured provider key and budget.
Recipes run in source order within each family and can share fixtures. The page preserves that order; “as above” is a dependency on an earlier case. Setup commands and API writes arrange state. They do not prove a UI path. Follow the family cleanup steps.
A known failure preserves the intended expectation beside the observed defect. Executable known-bug blocks use xfail for a reproduced symptom and xpass when it stops reproducing. Cloud accounts, enterprise capabilities and other unavailable prerequisites remain explicit. “Not run” is different from “blocked”.
The maintained map describes application behavior at its pinned revision. Once a fix is verified in the application version under test, remove the obsolete failure note and keep the normal expectation. Dated evidence belongs in the backfill ledger.
Read the full map index and coverage boundaries, the CLI guide, or the earlier illustrated edition. The verification approach adapts Lauren Tan’s (@poteto) pstack verification-skill generators; attribution and adaptation.