First run, onboarding and sign-in

F01 · 22 mapped behaviors in the Self-Healing Codebase Project. Follow the prerequisites, actions and expected results; expand the original CLI recipe to replay it.

778 behavior IDs · 28 families · 844 named recipes & support steps
16 October 2026 · PR #18334 at 9a3c3a69
Canvas 1.27.1 · Agent Server 1.55.0 · automation 1.20.0

This is the map at the revision above, including its open bug notes. It does not incorporate later changes on main or claim a fresh verification run. Setup & reading guide · Download the scenario inventory

All scenarios & marker key · Open a script to read Given / Do / Expect. Expand the exact recipe for replayable commands.

F01 First run, onboarding and sign-in

22 behaviors · 27 scripts & contracts · Maintained source

Entry points & prerequisites

What this family covers

What a user sees before the normal app shell. A browser that has never finished or skipped onboarding gets a full-screen onboarding modal on every route: choose an agent, set up the LLM (or ACP credentials), then say hello, which starts the first conversation. In public mode the flow starts with an "Add a backend" step that asks for the server's API key, and a browser with no stored key gets a full-screen API-key screen instead of the app. A one-time telemetry consent modal appears for each local backend. Unknown routes render an in-app "Page not found" page inside the app shell, and the tab title follows the open conversation.

How a user gets here

First run: open any URL of the app in a browser profile that has never onboarded. Every control-openhands launch --new starts with a fresh browser profile, and control-openhands browser reset gives the same run a fresh one again. Backend-side state (consent answer, LLM profile, agent kind) stays.
Public mode: control-openhands launch --new --public serves the UI without the injected key, so step 0 is "Add a backend". The API-key screen appears after skipping onboarding, after removing the last backend under backend selector → Manage backends (see F25), or when the server's key changes under a stored one.
Telemetry consent: appears by itself over onboarding or the app. It is changed later in Settings → Application (see F16).
Onboarding preview: direct URL with ?previewOnboardingStep=N, for example /?previewOnboardingStep=3 or /settings/app?previewOnboardingStep=3.
Last onboarding step: Close, a recommended-automation card, or the "Skip Getting Started checklist" box below the modal.
Not-found page: any unknown direct URL, such as /this-route-does-not-exist. Its Home button returns to /.
There is no command-menu entry or keyboard shortcut for onboarding.

Given

Preconditions:

A fresh run you launched yourself: control-openhands launch --new (local) or control-openhands launch --new --public, then export OH_VERIFY_RUN=<run from the launch JSON> and control-openhands doctor. Do not run control-openhands onboard --skip: it consumes the first-run state this family tests.
The model-backed say-hello check needs a DeepSeek key in DEEPSEEK_API_KEY. It is typed into the onboarding form with --value-env, which replaces llm preset deepseek here.
Each fresh browser profile walks first run once; control-openhands browser reset starts another first run on the same stack. Local and public mode still need separate runs (launch --new and launch --new --public): the local walk, public run A (backend step walked) and public run B (onboarding skipped). Stop each one with control-openhands stop when done.
Desktop viewport unless stated.

Local run (launch --new):

First-run gateF01.first-run-gateMapped behavior

Scenario F01--first-run-gate · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Open /conversations.

Expect

with no openhands-onboarded flag in localStorage, any URL (including /conversations) shows only first-run-onboarding-screen with the onboarding modal. There is no sidebar. Finishing or skipping sets the flag, and the app stays revealed after a reload.

Exact CLI recipe & bug checks

Run control-openhands browser goto /conversations and control-openhands browser testids. The list starts with first-run-onboarding-screen, telemetry-consent-form and onboarding-modal; root-layout is absent. Run control-openhands browser screenshot --feature F01.first-run-gate --name fresh. The screenshot shows the consent modal over "Choose your agent".

Choose agentF01.onboarding-choose-agentKnown issueBlocked

Scenario F01--choose-agent · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Onboarding agent option codex”.
Do
Choose “Onboarding agent option openhands”.
Do
Choose “Onboarding agent next”.
Wait
Wait for the element identified in the CLI recipe to appear.
Do
Choose “Onboarding llm back”.

Expect

radio tiles for OpenHands (preselected), Claude Code, Codex and Gemini CLI, plus Pi and OpenCode on a local backend (#17758; a Cloud backend offers only the first four, and a hidden choice falls back to OpenHands). Next saves the agent kind with a "Settings saved" toast.

Linked known issues

Known issue #16526 (F01.onboarding-choose-agent): 403 error when selecting an agent in the onboarding agent selection modal. Live prerequisite: An authenticated hosted Cloud account with the reported organization-membership mismatch during onboarding.

Exact CLI recipe & bug checks

Run control-openhands browser attr 'testid=onboarding-agent-option-openhands' aria-checked (true) and control-openhands browser count 'testid=onboarding-agent-back' (0). Then run control-openhands browser click 'testid=onboarding-agent-option-codex' and control-openhands browser attr 'testid=onboarding-agent-option-codex' aria-checked (true), and switch back with control-openhands browser click 'testid=onboarding-agent-option-openhands'. control-openhands browser snapshot 'testid=onboarding-step-choose-agent' lists six radios: OpenHands, Claude Code, Codex, Gemini CLI, Pi The open source Pi coding agent. and OpenCode The open source OpenCode coding agent. (onboarding-agent-option-pi, -opencode). Run control-openhands browser click 'testid=onboarding-agent-next' and control-openhands browser wait-text 'Settings saved'; data-current-step becomes 1. Run control-openhands browser click 'testid=onboarding-llm-back'; data-current-step is 0. Click testid=onboarding-agent-next again.

LLM stepF01.onboarding-setup-llmMapped behavior

Scenario F01--llm-step · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Onboarding step setup llm” / “Sdk section advanced toggle”.
Do
Enter “deepseek/deepseek-flash” in “Onboarding step setup llm” / “Llm custom model”.
Do
Enter “the value specified in the recipe” in “Onboarding step setup llm” / “Llm api key”.
Do
Choose “Onboarding llm next”.
Wait
Wait for the element identified in the CLI recipe to appear.
Setup
Configure or inspect the model profiles specified in the recipe.

Expect

on a first run the embedded LLM form is prefilled with openai/gpt-5.6-sol. Typing a custom model and key then Next saves them. On a local backend Next saves the form as an LLM profile named after the model (deepseek/deepseek-flash → deepseek-flash) and activates it, without writing the raw LLM settings.

Exact CLI recipe & bug checks

Run control-openhands browser snapshot 'testid=onboarding-step-setup-llm'. It shows Set up your LLM and Basic tab comboboxes OpenAI / gpt-5.6-sol. Run control-openhands browser click 'testid=onboarding-step-setup-llm >> testid=sdk-section-advanced-toggle' and control-openhands browser value 'testid=onboarding-step-setup-llm >> testid=llm-custom-model-input' (openai/gpt-5.6-sol). Then run control-openhands browser fill 'testid=onboarding-step-setup-llm >> testid=llm-custom-model-input' deepseek/deepseek-flash, control-openhands browser fill 'testid=onboarding-step-setup-llm >> testid=llm-api-key-input' --value-env DEEPSEEK_API_KEY and control-openhands browser click 'testid=onboarding-llm-next'. Wait with control-openhands browser wait '[data-testid=onboarding-modal][data-current-step="2"]'. Second view: control-openhands llm show lists profile deepseek-flash (deepseek/deepseek-flash, api_key_set: true) as active_profile.

Say helloF01.onboarding-say-helloMapped behavior

Scenario F01--say-hello · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Enter “” in “Onboarding hello”.
Do
Enter “Reply with only the word hello. Do not run any tools.” in “Onboarding hello”.
Do
Press Enter then --selector then testid=onboarding-hello-input.
Wait
Wait for the element identified in the CLI recipe to appear.
Wait
Wait for the conversation to finish.
Do
Reload the page.
Do
Open /.

Expect

the last step has a prefilled message. An empty message disables send. Enter or the send button creates a conversation, navigates to /conversations/<id> and completes onboarding, and the agent replies. A recommended-automations list sits under an "Or" separator, and a "Skip Getting Started checklist" box sits below the modal.

Exact CLI recipe & bug checks

Run control-openhands browser count 'testid=onboarding-skip' (0 on the last step), control-openhands browser fill 'testid=onboarding-hello-input' '' and control-openhands browser enabled 'testid=onboarding-hello-input-form >> testid=submit-button' (false). Then run control-openhands browser fill 'testid=onboarding-hello-input' 'Reply with only the word hello. Do not run any tools.' (enabled turns true), control-openhands browser press Enter --selector 'testid=onboarding-hello-input' and control-openhands browser wait-url '/conversations/[0-9a-f-]+'. Take <id> from control-openhands browser url. Run control-openhands conversation wait <id> --timeout 180 and control-openhands conversation events <id> --kinds MessageEvent. The agent message is hello, and control-openhands browser count 'testid=onboarding-modal' is 0. control-openhands browser eval "localStorage.getItem('openhands-onboarded')" returns "1". After control-openhands browser reload and control-openhands browser goto /, testid=first-run-onboarding-screen and testid=onboarding-modal both count 0.

Tab titleF01.document-titleMapped behavior

Scenario F01--tab-title · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
On / run control-openhands browser url; title is OpenHands. On /conversations/<id> (after a reload, once the title is generated) the title is <emoji> <conversation title> | OpenHands, for example ✅ … | OpenHands for a finished conversation.

Expect

the tab title is OpenHands. On a conversation it is <status emoji> <conversation title> | OpenHands.

Exact CLI recipe & bug checks

On / run control-openhands browser url; title is OpenHands. On /conversations/<id> (after a reload, once the title is generated) the title is <emoji> <conversation title> | OpenHands, for example ✅ … | OpenHands for a finished conversation.

PreviewF01.onboarding-previewMapped behavior

Scenario F01--preview · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Open /?previewOnboardingStep=3.
Do
Open /settings/app?previewOnboardingStep=3.
Do
Open /?previewOnboardingStep=1.
Do
Choose “Onboarding skip”.
Do
Open /.

Expect

?previewOnboardingStep=0..3 on any route opens the modal (data-preview="true") on that phase, even after onboarding. Values outside 0-3 are ignored. Skip and Close do nothing, so leave by navigating to a URL without the parameter.

Exact CLI recipe & bug checks

Run control-openhands browser goto '/?previewOnboardingStep=3', then control-openhands browser attr 'testid=onboarding-modal' data-preview (true) and control-openhands browser count 'testid=onboarding-step-say-hello' (1). Step 2 shows the LLM slide. Steps 0 and 1 both show "Choose your agent" when the backend is healthy. /?previewOnboardingStep=9 renders no modal. control-openhands browser goto '/settings/app?previewOnboardingStep=3' works too. Run control-openhands browser goto '/?previewOnboardingStep=1' and control-openhands browser click 'testid=onboarding-skip'; the modal stays (count 1). control-openhands browser goto / shows no modal.

Skip checklist boxF01.onboarding-skip-checklistMapped behavior

Scenario F01--skip-checklist-box · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Open /conversations.
Do
Open /?previewOnboardingStep=3.
Do
Choose “Skip Getting Started checklist”.
Do
Open /conversations.
Do
Reload the page.

Expect

the "Skip Getting Started checklist" box under the last step hides the sidebar Getting started card (F02) once the app is revealed. It is the same per-browser setting as the Settings → Application switch, and it is saved as soon as you tick it, preview mode included.

Exact CLI recipe & bug checks

Run control-openhands browser goto /conversations and control-openhands browser count 'testid=sidebar-onboarding-checklist' (1). Run control-openhands browser goto '/?previewOnboardingStep=3', then control-openhands browser click 'text=Skip Getting Started checklist'. control-openhands browser eval "document.querySelector('[data-testid=onboarding-skip-getting-started-checklist]').checked" is true. Run control-openhands browser goto /conversations and control-openhands browser reload; the checklist count is 0. On /settings/app, control-openhands browser eval "document.querySelector('[data-testid=show-getting-started-checklist-switch]').checked" is false. Restore it: go back to /?previewOnboardingStep=3, where the box now reads true, click the same text again, and the checklist count on /conversations is 1.

PhoneF01.onboarding-phoneMapped behavior

Scenario F01--phone · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Setup
Use the phone viewport.
Do
Open /?previewOnboardingStep=N.
Setup
Use the desktop viewport.

Expect

every onboarding step fits a 390 px viewport without horizontal overflow.

Exact CLI recipe & bug checks

Run control-openhands browser viewport phone. Then for N in 1, 2 and 3 run control-openhands browser goto '/?previewOnboardingStep=N', control-openhands browser bbox 'testid=onboarding-modal' and control-openhands browser screenshot --feature F01.onboarding-phone --name step-N. The modal is 351 px wide with insideViewport true and pageHorizontalOverflow false, and Back/Next or Back/Close are visible. Return with control-openhands browser viewport desktop.

Error pageF01.route-error-boundaryMapped behavior

Scenario F01--error-page · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Open /this-route-does-not-exist.
Do
Choose “Not found home”.

Expect

an unknown URL renders a "Page not found" page inside the app shell, with the sidebar and a Home link back to /, and logs no page errors.

Exact CLI recipe & bug checks

Run control-openhands browser errors --clear, control-openhands browser goto /this-route-does-not-exist and control-openhands browser snapshot 'testid=not-found-screen'. It shows heading "Page not found", the paragraph This address does not match any page. Check the URL, or go back to the home page. and link "Home" (/url: /). control-openhands browser count 'aside[data-collapsed]' is 1: the sidebar stays. control-openhands browser screenshot --feature F01.route-error-boundary --name not-found shows the message and the Home button centered beside the sidebar. control-openhands browser errors --app-only reports pageErrors 0 and appErrors 0. Run control-openhands browser click 'testid=not-found-home-link' --expect-url '/$'; control-openhands browser count 'testid=home-screen' is 1.

Onboarding again with the saved endpointF01.onboarding-repeat-endpointKnown failure

Scenario F01--onboarding-again-with-the-saved-endpoint · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Start with a fresh browser profile.
Do
Open /.
Do
Choose “Onboarding agent next”.
Wait
Wait for the element identified in the CLI recipe to appear.
Do
Choose “Onboarding step setup llm” / “Sdk section all toggle”.
Do
Enter “openai/deepseek-chat” in “Onboarding step setup llm” / “Llm custom model”.
Do
Enter “https://api.deepseek.com/v1” in “Onboarding step setup llm” / “Base url”.
Do
Enter “the value specified in the recipe” in “Onboarding step setup llm” / “Llm api key”.
Do
Choose “Onboarding llm next”.
Wait
Wait for the element identified in the CLI recipe to appear.
Setup
Configure or inspect the model profiles specified in the recipe.
Do
Enter “Reply with only the word hello. Do not run any tools.” in “Onboarding hello”.
Do
Press Enter then --selector then testid=onboarding-hello-input.
Wait
Wait for the element identified in the CLI recipe to appear.
Wait
Wait for the conversation to finish.
Setup
Configure or inspect the model profiles specified in the recipe.
Setup
Arrange the prerequisite backend state through the API.
Setup
Configure or inspect the model profiles specified in the recipe.
Setup
Configure or inspect the model profiles specified in the recipe.
Setup
Configure or inspect the model profiles specified in the recipe.
Setup
Arrange the prerequisite backend state through the API.

Expect

onboarding again (a new browser, the same backend) fills the form from the active LLM profile, or from the one the default agent profile points at, model included. The All view prefills its Base URL and key even when the raw LLM settings hold another Base URL. Next without changes keeps that profile and saves no copy. Changing only the model creates a profile that keeps that Base URL and key, and the say-hello conversation reaches that endpoint.

Known bug observations

Known failure (reproduced 2026-10-12 and 2026-10-13, #18302): Next replaces a profile that already has the model-derived name. Arrange one before pass 2 with control-openhands llm set --profile deepseek-v4-flash --model openai/deepseek-v4-flash --base-url http://127.0.0.1:9/other-endpoint --api-key-env DEEPSEEK_API_KEY --no-activate --no-validate. After pass 2's Next, browser network --filter '/api/profiles' shows POST /api/profiles/deepseek-v4-flash, and control-openhands llm show lists deepseek-v4-flash with base_url https://api.deepseek.com/v1: its own Base URL is gone, and nothing warned. Expected: the existing profile is kept, under a name that does not collide or after a confirmation. Restore: run control-openhands llm preset deepseek. It also points the default agent profile back at deepseek-flash: its output has repointed from deepseek-v4-flash to deepseek-flash. Then control-openhands api DELETE /api/profiles/deepseek-v4-flash --write and .../deepseek-chat --write answer 200. Delete the two say-hello conversations with Clean up's steps.

Exact CLI recipe & bug checks

Two more first runs on this stack, both through the All view, which shows the Base URL field. Pass 1 saves a custom endpoint: control-openhands browser reset, control-openhands browser goto /, control-openhands browser click 'testid=onboarding-agent-next', control-openhands browser wait '[data-testid=onboarding-modal][data-current-step="1"]', control-openhands browser click 'testid=onboarding-step-setup-llm >> testid=sdk-section-all-toggle' and control-openhands browser value 'testid=onboarding-step-setup-llm >> testid=base-url-input' (empty). Run control-openhands browser fill 'testid=onboarding-step-setup-llm >> testid=llm-custom-model-input' openai/deepseek-chat, control-openhands browser fill 'testid=onboarding-step-setup-llm >> testid=base-url-input' https://api.deepseek.com/v1, control-openhands browser fill 'testid=onboarding-step-setup-llm >> testid=llm-api-key-input' --value-env DEEPSEEK_API_KEY, control-openhands browser click 'testid=onboarding-llm-next' and control-openhands browser wait '[data-testid=onboarding-modal][data-current-step="2"]'. control-openhands llm show lists deepseek-chat (openai/deepseek-chat, base_url https://api.deepseek.com/v1) as active_profile. Say hello as above (browser fill 'testid=onboarding-hello-input' 'Reply with only the word hello. Do not run any tools.', browser press Enter --selector 'testid=onboarding-hello-input', browser wait-url '/conversations/[0-9a-f-]+'), then control-openhands conversation wait <id> --timeout 180 is finished and the agent MessageEvent is hello. Pass 2 types the same endpoint again: repeat the steps up to the All view. browser value 'testid=onboarding-step-setup-llm >> testid=base-url-input' is now https://api.deepseek.com/v1, the saved value. Fill the model openai/deepseek-v4-flash (a DeepSeek alias, so the new profile gets its own name), the same Base URL and the key, then Next and wait for step 2 (control-openhands browser screenshot 'testid=onboarding-step-setup-llm' --feature F01.onboarding-repeat-endpoint --name second-pass-form before Next). Expected: control-openhands llm show lists deepseek-v4-flash with base_url https://api.deepseek.com/v1, and the say-hello conversation finishes with hello. Raw LLM settings that disagree with the active profile do not reach the form. To check it, arrange them before pass 2 with control-openhands api PATCH /api/settings --write --data '{"agent_settings_diff":{"llm":{"base_url":"http://127.0.0.1:9/stale-raw-settings"}}}'. browser value 'testid=onboarding-step-setup-llm >> testid=base-url-input' still reads https://api.deepseek.com/v1. Fill only the model, run browser network --clear before Next and browser network --filter '/api/(settings|profiles)' after it: the rows are the profile POST and its /activate, with no PATCH /api/settings. deepseek-v4-flash keeps the DeepSeek Base URL, and the say-hello conversation finishes with hello (verified 2026-10-09). Known failure (reproduced 2026-10-12 and 2026-10-13, #18302): Next replaces a profile that already has the model-derived name. Arrange one before pass 2 with control-openhands llm set --profile deepseek-v4-flash --model openai/deepseek-v4-flash --base-url http://127.0.0.1:9/other-endpoint --api-key-env DEEPSEEK_API_KEY --no-activate --no-validate. After pass 2's Next, browser network --filter '/api/profiles' shows POST /api/profiles/deepseek-v4-flash, and control-openhands llm show lists deepseek-v4-flash with base_url https://api.deepseek.com/v1: its own Base URL is gone, and nothing warned. Expected: the existing profile is kept, under a name that does not collide or after a confirmation. Restore: run control-openhands llm preset deepseek. It also points the default agent profile back at deepseek-flash: its output has repointed from deepseek-v4-flash to deepseek-flash. Then control-openhands api DELETE /api/profiles/deepseek-v4-flash --write and .../deepseek-chat --write answer 200. Delete the two say-hello conversations with Clean up's steps.

Clean upF01 · setup / cleanupSupport step

Scenario F01--clean-up · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Open /conversations/<id>.
Do
Choose “Chat pane header” / “More options”.
Do
Choose “Conversation name context menu” / “Delete”.
Do
Choose “Confirm Delete”.

Expect

Complete this support step before continuing to the next scenario.

Exact CLI recipe & bug checks

Delete the hello conversation from its header menu: control-openhands browser goto /conversations/<id> (the hello conversation's <id> from Say hello; the Error page bullet left /), control-openhands browser click 'testid=chat-pane-header >> testid=ellipsis-button', control-openhands browser click 'testid=conversation-name-context-menu >> testid=delete-button', control-openhands browser click 'role=button[name="Confirm Delete"]'. control-openhands conversation list then shows count 0.

Public run A (launch --new --public), backend step walked:

Backend stepF01.onboarding-backend-stepMapped behavior

Scenario F01--backend-step · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Open /.
Do
Enter “qa-wrong-key” in “Onboarding backend api key”.
Do
Choose “Onboarding backend next”.
Do
Enter “the value specified in the recipe” in “Onboarding backend api key”.
Do
Choose “Onboarding backend next”.
Wait
Wait for “Telemetry consent form” to appear.
Do
Choose “Confirm telemetry preferences”.

Expect

when no reachable backend exists (public mode), step 0 "Add a backend" prefills the name Local and the page origin as host. Next stays disabled until an API key is typed. A wrong key shows "Could not connect to <host>" / "Invalid API key" inline. The right key saves the backend, and the flow collapses to 3 steps on "Choose your agent" with no Back button. The first paint must not raise an error toast.

Exact CLI recipe & bug checks

Run control-openhands browser goto /, sleep 12 (gives a late toast time to appear), control-openhands browser toasts --history, control-openhands browser screenshot --feature F01.onboarding-backend-step --name fresh and control-openhands browser snapshot 'testid=onboarding-step-check-backend'. The toast history is [], and the screenshot shows no toast. The snapshot shows Add a backend, name Local, host http://127.0.0.1:<port>, Type Local checked, button "Next" [disabled], and an "OpenHands Cloud" column with "Connect to OpenHands". Run control-openhands browser fill 'testid=onboarding-backend-api-key' qa-wrong-key, control-openhands browser click 'testid=onboarding-backend-next' and control-openhands browser text 'testid=onboarding-backend-error'. It reads Could not connect to http://127.0.0.1:<port> / Invalid API key, and data-current-step stays 0. Run control-openhands browser fill 'testid=onboarding-backend-api-key' --value-file "$OH_VERIFY_RUN/private/session-key", control-openhands browser click 'testid=onboarding-backend-next' and control-openhands browser wait 'testid=telemetry-consent-form'. The modal is on "Choose your agent" with data-current-step 0, testid=onboarding-progress-step-3 and testid=onboarding-agent-back both count 0, and the consent form names “Local”. Answer it with control-openhands browser click 'testid=confirm-telemetry-preferences'.

ACP credentialsF01.onboarding-acp-secretsMapped behavior

Scenario F01--acp-credentials · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Onboarding agent option claude code”.
Do
Choose “Onboarding agent next”.
Wait
Wait for the element identified in the CLI recipe to appear.
Do
Choose “Onboarding acp secrets back”.
Do
Choose “Onboarding agent option openhands”.
Do
Choose “Onboarding agent next”.

Expect

choosing an ACP agent swaps the LLM step for "Add your API keys" with that provider's fields. When the host CLI is already logged in, a green "already signed in" banner makes them optional. OpenCode's fields (OPENCODE_AUTH_CONTENT, OPENCODE_API_KEY) are always optional; Pi's two fields (PI_AUTH_JSON, ANTHROPIC_API_KEY) are each labelled Optional, but without a host login at least one of them is needed: Next stays disabled under a warning, readable in light themes, until either is filled. Back returns to the agent step.

Exact CLI recipe & bug checks

Run control-openhands browser click 'testid=onboarding-agent-option-claude-code', control-openhands browser click 'testid=onboarding-agent-next' and control-openhands browser wait-text 'Settings saved'. Then run control-openhands browser snapshot 'testid=onboarding-step-setup-acp-secrets'. It shows Add your API keys with CLAUDE_CODE_OAUTH_TOKEN, ANTHROPIC_API_KEY and ANTHROPIC_BASE_URL, and control-openhands api GET /api/settings shows "agent_kind": "acp". Run control-openhands browser click 'testid=onboarding-acp-secrets-back', control-openhands browser click 'testid=onboarding-agent-option-openhands' and control-openhands browser click 'testid=onboarding-agent-next'. testid=onboarding-step-setup-llm is shown and agent_kind is back to openhands.

Pi and OpenCode credentialsF01.onboarding-acp-secretsMapped behavior

Scenario F01--pi-and-opencode-credentials · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Onboarding llm back”.
Do
Choose “Onboarding agent option opencode”.
Do
Choose “Onboarding agent next”.
Wait
Wait for the element identified in the CLI recipe to appear.
Do
Reload the page.

Expect

Exact CLI recipe & bug checks

The previous bullet ends on the LLM step: run control-openhands browser click 'testid=onboarding-llm-back' to return to the agent step, then control-openhands browser click 'testid=onboarding-agent-option-opencode', control-openhands browser click 'testid=onboarding-agent-next', browser wait-text 'Settings saved' and browser snapshot 'testid=onboarding-step-setup-acp-secrets': OPENCODE_AUTH_CONTENT Optional and OPENCODE_API_KEY Optional, and control-openhands browser enabled 'testid=onboarding-acp-secrets-next' is true. Back, then the same with onboarding-agent-option-pi: This backend has no host login, so credentials are required to run Pi., PI_AUTH_JSON Optional and ANTHROPIC_API_KEY Optional (either one is enough), Next false while both are empty, and browser text 'testid=onboarding-acp-secrets-blocked' reads Add the credentials above to continue, or sign in on the backend. Its colour, browser eval "getComputedStyle(document.querySelector('[data-testid=onboarding-acp-secrets-blocked]')).color", is rgb(201, 185, 116) in OpenHands-Neutral and rgb(125, 93, 8) in Light+ (arrange with browser eval "localStorage.setItem('openhands-color-theme','light-plus')" and browser reload, which returns to the agent step with OpenHands selected again, so click onboarding-agent-option-pi and onboarding-agent-next once more before reading the colour; #18333). Restore the theme with localStorage.removeItem, go Back, choose OpenHands and Next (agent_kind back to openhands).

Skip into the appF01.onboarding-skipMapped behavior

Scenario F01--skip-into-the-app · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Onboarding skip”.
Wait
Wait for “Root layout” to appear.
Do
Reload the page.

Expect

"Skip for now" closes the modal and sets the flag. In public mode, skipping before a key is stored lands on the API-key screen; skipping after the backend step lands in the app.

Exact CLI recipe & bug checks

Run control-openhands browser click 'testid=onboarding-skip', control-openhands browser wait 'testid=root-layout', control-openhands browser reload, control-openhands browser count 'testid=onboarding-modal' (0) and control-openhands browser count 'testid=api-key-entry-screen' (0).

Last backend removedF01.api-key-entryMapped behavior

Scenario F01--last-backend-removed · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Backend selector”.
Do
Choose “Manage backends menu item”.
Do
Choose “Manage backends remove local”.
Do
Choose “Confirmation dialog” / “Confirm”.
Wait
Wait for “Api key entry screen” to appear.

Expect

in public mode with no usable key (onboarding skipped, the last backend removed, or a stored key the server now rejects), a full-screen "Add a backend" card appears. The host is read-only and set to the origin, and Connect stays disabled until the name and key are filled. A wrong key shows "Invalid API key. Please check the key and try again." The right key opens the app shell, which survives a reload, and the consent modal then names the new backend.

Exact CLI recipe & bug checks

Run control-openhands browser click 'testid=backend-selector', control-openhands browser click 'testid=manage-backends-menu-item', control-openhands browser click 'testid=manage-backends-remove-Local' and control-openhands browser click 'testid=confirmation-modal >> testid=confirm-button'. control-openhands browser wait 'testid=api-key-entry-screen' succeeds. Restore with the Connect recipe below.

Public run B (launch --new --public), onboarding skipped:

Skip to API-key screenF01.onboarding-skip · F01.api-key-entryMapped behavior

Scenario F01--skip-to-api-key-screen · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Open /.
Do
Choose “Onboarding skip”.
Wait
Wait for “Api key entry screen” to appear.
Do
Reload the page.
Wait
Wait for “Api key entry screen” to appear.
Do
Enter “qa-wrong-key” in “Api key entry api key”.
Do
Enter “QA Public” in “Api key entry name”.
Do
Choose “Api key entry submit”.
Setup
Use the phone viewport.
Setup
Use the desktop viewport.

Expect

"Skip for now" closes the modal and sets the flag. In public mode, skipping before a key is stored lands on the API-key screen; skipping after the backend step lands in the app.

Exact CLI recipe & bug checks

Run control-openhands browser goto /, control-openhands browser click 'testid=onboarding-skip' and control-openhands browser wait 'testid=api-key-entry-screen'. After sleep 12, control-openhands browser toasts --history is []: the screen raises no error toast. Run control-openhands browser reload, control-openhands browser wait 'testid=api-key-entry-screen', sleep 12 and control-openhands browser toasts --history again: still [] when the API-key screen is the first paint (the toast these checks guard against reads No backend is configured.; control-openhands browser count 'text=No backend is configured' is 0). Run control-openhands browser value 'testid=api-key-entry-host' (the origin; the field is disabled) and control-openhands browser enabled 'testid=api-key-entry-submit' (false). Fill only the key with control-openhands browser fill 'testid=api-key-entry-api-key' qa-wrong-key; enabled is still false because Host Name is required. Run control-openhands browser fill 'testid=api-key-entry-name' 'QA Public' (enabled turns true), control-openhands browser click 'testid=api-key-entry-submit' and control-openhands browser text 'testid=api-key-entry-status'. It reads Invalid API key. Please check the key and try again., the screen stays, and control-openhands browser screenshot --feature F01.api-key-entry --name wrong-key shows the red line above Connect. While the screen is still up, run control-openhands browser viewport phone and control-openhands browser bbox 'testid=api-key-entry-form'. It gives insideViewport true and pageHorizontalOverflow false. Return with control-openhands browser viewport desktop.

ConnectF01.api-key-entryMapped behavior

Scenario F01--connect · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Enter “QA Public” in “Api key entry name”.
Do
Enter “the value specified in the recipe” in “Api key entry api key”.
Do
Choose “Api key entry submit”.
Wait
Wait for “Api key entry screen” to disappear.
Do
Reload the page.

Expect

Exact CLI recipe & bug checks

Run control-openhands browser fill 'testid=api-key-entry-name' 'QA Public', control-openhands browser fill 'testid=api-key-entry-api-key' --value-file "$OH_VERIFY_RUN/private/session-key", control-openhands browser click 'testid=api-key-entry-submit' and control-openhands browser wait 'testid=api-key-entry-screen' --state detached. testid=root-layout is shown, the consent form names “QA Public”, and control-openhands browser snapshot 'testid=backend-selector' shows combobox "QA Public". After control-openhands browser reload, testid=api-key-entry-screen counts 0 and testid=root-layout counts 1.

Stale key re-promptF01.api-key-entryMapped behavior

Scenario F01--stale-key-re-prompt · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Confirm telemetry preferences”.
Do
Restart the run-owned stack as specified.
Do
Reload the page.
Wait
Wait for “Api key entry screen” to appear.

Expect

Exact CLI recipe & bug checks

Answer consent if it is open (control-openhands browser click 'testid=confirm-telemetry-preferences'). Run control-openhands restart --rotate-key (rotatedKey true), then control-openhands browser reload and control-openhands browser wait 'testid=api-key-entry-screen'; testid=root-layout counts 0. Connect again as above with the new $OH_VERIFY_RUN/private/session-key. After a reload, testid=root-layout counts 1, and control-openhands browser eval "JSON.parse(localStorage.getItem('openhands-backends')).length" is 1, so no duplicate backend is added.

Close on the last stepF01.onboarding-hello-closeMapped behavior

Scenario F01--close-on-the-last-step · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Start with a fresh browser profile.
Do
Open /.
Do
Enter “the value specified in the recipe” in “Onboarding backend api key”.
Do
Choose “Onboarding backend next”.
Wait
Wait for “Onboarding step choose agent” to appear.
Do
Choose “Onboarding agent next”.
Wait
Wait for the element identified in the CLI recipe to appear.
Do
Enter “the value specified in the recipe” in “Onboarding step setup llm” / “Llm api key”.
Do
Choose “Onboarding llm next”.
Wait
Wait for the element identified in the CLI recipe to appear.
Do
Choose “Onboarding hello close”.
Wait
Wait for “Onboarding modal” to disappear.
Do
Reload the page.

Expect

the last step has Back and Close instead of "Skip for now". Close dismisses the modal without creating a conversation, sets the openhands-onboarded flag and leaves the app shell on /.

Exact CLI recipe & bug checks

Run control-openhands browser reset and control-openhands browser goto /. Walk to the last step: control-openhands browser fill 'testid=onboarding-backend-api-key' --value-file "$OH_VERIFY_RUN/private/session-key", control-openhands browser click 'testid=onboarding-backend-next', control-openhands browser wait 'testid=onboarding-step-choose-agent' (consent does not return: the backend already has an answer), control-openhands browser click 'testid=onboarding-agent-next' and control-openhands browser wait '[data-testid=onboarding-modal][data-current-step="1"]'. Then fill the key with control-openhands browser fill 'testid=onboarding-step-setup-llm >> testid=llm-api-key-input' --value-env DEEPSEEK_API_KEY, open testid=sdk-section-advanced-toggle and fill llm-custom-model-input with deepseek/deepseek-flash as in the LLM step. Run control-openhands browser click 'testid=onboarding-llm-next' and control-openhands browser wait '[data-testid=onboarding-modal][data-current-step="2"]'. Check control-openhands conversation list (count 0), then run control-openhands browser click 'testid=onboarding-hello-close' and control-openhands browser wait 'testid=onboarding-modal' --state detached. testid=root-layout counts 1, the URL is / and localStorage.getItem('openhands-onboarded') is "1". conversation list still shows count 0, and after browser reload the modal count is 0.

Recommended automationF01.onboarding-recommended-automationMapped behavior

Scenario F01--recommended-automation · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Start with a fresh browser profile.
Do
Open /.
Do
Choose “Recommended automation card news digest”.
Do
Choose “Setup dialog close”.
Wait
Wait for “Setup dialog” to disappear.

Expect

picking a card under "Start from a proven workflow" on the last step completes onboarding too. A card with its own setup form (for example "Daily news digest") navigates to /automations/new/<id> and opens that form. Nothing is created until the form is submitted.

Exact CLI recipe & bug checks

Run control-openhands browser reset, control-openhands browser goto /, and walk to the last step as above (the LLM step's Next works without retyping the key once the profile is saved). Run control-openhands browser click 'testid=recommended-automation-card-news-digest' --expect-url '/automations/new/news-digest(\?|$)'. testid=onboarding-modal counts 0, openhands-onboarded is "1", and the "Daily news digest" setup form (testid=setup-dialog) is open. Leave without creating anything: control-openhands browser click 'testid=setup-dialog-close' and control-openhands browser wait 'testid=setup-dialog' --state detached (the URL becomes /). control-openhands api GET /api/automation/v1 shows "total": 0.

Cloud login columnF01.onboarding-cloud-loginBlocked

Scenario F01--cloud-login-column · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Start with a fresh browser profile.
Do
Open /.
Do
Choose “Onboarding backend advanced toggle”.

Expect

the backend step's "OpenHands Cloud" column has "Connect to OpenHands" (device-flow login) and an "Advanced" toggle. The toggle reveals a Cloud Host field (placeholder https://app.all-hands.dev) for self-hosted Cloud deployments. The login itself is blocked: it needs an OpenHands Cloud account.

Exact CLI recipe & bug checks

Run control-openhands browser reset, control-openhands browser goto / and control-openhands browser click 'testid=onboarding-backend-advanced-toggle'. browser attr 'testid=onboarding-backend-advanced-toggle' aria-expanded is true, and browser attr 'testid=onboarding-backend-cloud-host' placeholder is https://app.all-hands.dev. browser text 'testid=onboarding-backend-advanced-panel' mentions self-hosted Cloud deployments. Do not click testid=onboarding-backend-login-button: the device-flow login is blocked without an OpenHands Cloud account.

Not reachable locally:

Locked Cloud first runF01.locked-cloud-first-runBlocked

Scenario F01--locked-cloud-first-run · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Blocked: needs a build locked to a Cloud host (VITE_LOCK_TO_CLOUD or static-server --lock-to-cloud) and an OpenHands Cloud account. launch has no such flag.

Expect

on a canvas locked to an OpenHands Cloud host, first run shows only the Cloud login (Add backend) without a progress bar or close button (blocked).

Exact CLI recipe & bug checks

Blocked: needs a build locked to a Cloud host (VITE_LOCK_TO_CLOUD or static-server --lock-to-cloud) and an OpenHands Cloud account. launch has no such flag.

Bootstrap spinnerF01.bootstrap-loadingNot run

Scenario F01--bootstrap-spinner · Permalink · Source

Given

Use F01’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Blocked on the 2026-10-09 run: the agent attempted a fresh first paint, but the spinner lasted under a second on the healthy stack; slowing /server_info would require request interception rather than the real service response.

Expect

a centered spinner card shows while /server_info loads (transient, not-run).

Exact CLI recipe & bug checks

Blocked on the 2026-10-09 run: the agent attempted a fresh first paint, but the spinner lasted under a second on the healthy stack; slowing /server_info would require request interception rather than the real service response.

Family notes, prerequisites for known issues & limits Known issueBlocked

Use a fresh run or browser reset for every first-run check, and never onboard --skip first. The browser profile lives in <run>/private/browser-profile. browser reset clears only the browser, so the server keeps the consent answer (the consent modal does not return), the LLM profile and the agent kind.
control-openhands login passes both key prompts: the onboarding backend step, and the API-key screen, where it names the backend Local when Host Name is empty (after Skip and after restart --rotate-key, with no duplicate backend). Use it when the screen is only in the way. To check the screen itself (the disabled Connect, the wrong-key line, a chosen name), use the explicit fills above.
Inactive onboarding slides stay mounted and are translated off-screen. Plain browser testids lists only the active slide, but count and wait still find handles on the others. Assert the active step with data-current-step, for example browser wait '[data-testid=onboarding-modal][data-current-step="2"]'. The testid= selector cannot carry attribute filters.
Slide indices renumber. After the public backend step succeeds, "Choose your agent" becomes index 0 and has no Back. previewOnboardingStep counts phases with the backend step included, so 0 and 1 look identical on a healthy backend.
The consent modal (z-70) sits over the onboarding modal, and clicks at the screen center land on agent tiles once it closes. A forced click on first-run-onboarding-screen selected Codex. Check backdrop non-dismissal with browser mouse-click 40 40, a corner outside the modal.
The say-hello default message ("Create a basic webpage…") makes the agent work for a while. Replace it with a tiny prompt to keep model cost down.
On a local backend, Next on the LLM step also points the default agent profile at the profile it just created. Activating another LLM profile does not move that pointer, and while it stays, deleting the onboarding profile answers 409 LLM profile is referenced by 1 agent profile(s): default. control-openhands llm preset deepseek moves it to deepseek-flash and reports repointed; llm set does not. In the UI, edit default in Settings → Agents and pick the profile in testid=agent-profile-llm-selector.
On a local backend the LLM step edits an LLM profile and never reads or writes the raw LLM settings (#18218). It takes the active profile, else the one the default agent profile points at (the Agent Server's backfill of an older install's raw settings), else a fresh one with the default model. It reads the profile only once the backend's health check reports it connected, so a step mounted while the Agent Server was down fills in after it comes back (verified 2026-10-12 with control-openhands service stop agent-server, then restart). If the profiles cannot be read it shows Failed to load existing profile. Please try again. (testid=onboarding-llm-load-error) with Retry (testid=onboarding-llm-load-retry) and Next disabled. Next without changes only sends POST /api/profiles/<name>/activate.
The tab title prefixes a status emoji to the stored title. A generated title that already starts with an emoji therefore shows twice (✅ ✅ Reply hello without tools | OpenHands). Right after launch the title is Conversation <id prefix> until a reload picks up the generated one.
Right after Next, the ACP step reads "Checking for an existing Claude Code login…" with Next disabled. Wait for testid=onboarding-acp-auth-detected before reading the banner. The "already signed in" banner depends on the host's CLI login. On a clean CI machine the fields become required, and Next is blocked until they are filled.
Choosing an ACP agent saves agent_kind: acp immediately. Re-choose OpenHands before leaving, or later conversations in the run use the ACP agent.
In preview mode, Skip and Close are inert. The LLM slide shows gpt-5.6-sol even when another model is saved, so do not read it as data loss.
browser toasts --history lists every status or alert text (toasts and alert banners) seen since the page loaded, including success toasts and progress lines such as Checking your backend connection…. Read it on a page that has just loaded (or reloaded) to check that no error toast appeared. A page with a standing banner never reads []: the app shell of a backend with no LLM lists the Your LLM isn't set up yet alert.
Known issue #16601 (F01.telemetry-consent): Analytics consent reappears sporadically. Live prerequisite: The multi-day trigger and backend consent response when the modal returns; opted-in localStorage alone does not reproduce it.
Known issue #16526 (F01.onboarding-choose-agent): 403 error when selecting an agent in the onboarding agent selection modal. Live prerequisite: An authenticated hosted Cloud account with the reported organization-membership mismatch during onboarding.

Before you start

Use an OpenHands checkout at the source revision, with Node 24+, installed npm dependencies, uv / uvx, and a supported Chromium browser. The verification skill is the complete setup contract.

export PATH="$PWD/.agents/skills/verify-openhands/scripts:$PATH"
export OH_VERIFY_RUN=$(control-openhands launch --new --print-run)
control-openhands doctor
control-openhands onboard --skip
# Configure a model only for a scenario that needs one:
control-openhands llm preset deepseek
# When the run is complete:
control-openhands stop

F01 walks onboarding instead of skipping it. No-model checks run before model configuration. Keep OH_VERIFY_RUN set so commands address your own run. Desktop is 1440 × 1000; phone is 390 × 844. Model-backed checks need the configured provider key and budget.

Recipes run in source order within each family and can share fixtures. The page preserves that order; “as above” is a dependency on an earlier case. Setup commands and API writes arrange state. They do not prove a UI path. Follow the family cleanup steps.

A known failure preserves the intended expectation beside the observed defect. Executable known-bug blocks use xfail for a reproduced symptom and xpass when it stops reproducing. Cloud accounts, enterprise capabilities and other unavailable prerequisites remain explicit. “Not run” is different from “blocked”.

The maintained map describes application behavior at its pinned revision. Once a fix is verified in the application version under test, remove the obsolete failure note and keep the normal expectation. Dated evidence belongs in the backfill ledger.

Read the full map index and coverage boundaries, the CLI guide, or the earlier illustrated edition. The verification approach adapts Lauren Tan’s (@poteto) pstack verification-skill generators; attribution and adaptation.