Engel’s notebook

Settings shell and navigation

F09 · 26 mapped behaviors in the Self-Healing Codebase Project. Follow the prerequisites, actions and expected results; expand the original CLI recipe to replay it.

778 behavior IDs · 28 families · 844 named recipes & support steps
16 October 2026 · PR #18334 at 9a3c3a69
Canvas 1.27.1 · Agent Server 1.55.0 · automation 1.20.0

This is the map at the revision above, including its open bug notes. It does not incorporate later changes on main or claim a fresh verification run. Setup & reading guide · Download the scenario inventory

All scenarios & marker key · Open a script to read Given / Do / Expect. Expand the exact recipe for replayable commands.

F09 Settings shell and navigation

26 behaviors · 24 scripts & contracts · Maintained source

Entry points & prerequisites

What this family covers

Settings is one shell around eight pages (Agent, LLM, Model Router, Condenser, Agent Context, Verification, Application, Secrets). On desktop a pinned left column lists the pages, highlights the current one and ends with an "Agent Canvas" version card and a "synced from … backend" note; each page gets a title and a grey subline. Below 1024 px (phones and tablets) /settings is a hub list instead, at tablet widths (768–1023 px) each subpage has a compact Settings selector, and at phone width every page shows a Back chevron to the hub. Users reach Settings from the sidebar gear, the collapsed rail, the command menu and several deep links (LLM banner, getting started checklist, composer pickers). The individual pages are other families (F10–F16; Secrets is F14).

How a user gets here

  • Sidebar footer gear next to the backend selector (backend-selector-settings-link), on desktop or inside the phone drawer (sidebar-mobile-menu-toggle, then sidebar-mobile-drawer >> backend-selector-settings-link).
  • Collapsed sidebar: the gear icon at the bottom of the rail (collapsed-settings-link).
  • Command menu (Control+k/Meta+k or command-menu-trigger): group Settings with Settings, Agent settings, LLM profiles, Model Router, Condenser settings, Agent Context, Verification settings, Application settings, Secrets settings.
  • Deep links: Home banner Set up LLM (home-llm-not-configured-action, only while no LLM key is configured); getting-started checklist Add LLM API key and Customize your agent; composer LLM pill (chat-input-llm-profile) → LLM Profiles; composer + (chat-plus-button) → Switch agent profile → Manage agent profiles. Application settings also links "Manage LLM profiles" (F15's page).
  • Direct URLs: /settings, /settings/agents, /settings/llm, /settings/meta-llm, /settings/condenser, /settings/agent-context, /settings/verification, /settings/app, /settings/secrets, and the legacy /settings/agent.
  • Inside Settings: the left nav (sidebar-settings-/settings/<page>) on desktop; the hub list (settings-mobile-hub) plus the top-bar Back chevron (sidebar-mobile-back-button) on phones.

Given

Preconditions:

  • Baseline state (launched, doctored, onboard --skip done), desktop viewport.
  • Run the banner and checklist deep links (F09.entry-deep-links, first bullet) before configuring a model: the banner only exists while no LLM key is set.
  • The composer picker deep links need a configured profile: control-openhands llm preset deepseek (no model call is made).
  • F09.cloud-links positive case and F09.locked-cloud-landing need a Cloud backend/account (blocked in a local run). F09.hidden-llm-redirect is blocked: the frontend hardcodes hide_llm_settings: false. The update states depend on the browser reaching registry.npmjs.org: with access and a current client you get F09.update-up-to-date; F09.update-available and the version tile also need an npm latest newer than the client (blocked while the client is the latest release); the failure text of F09.update-modal needs the registry to be unreachable (no mocks).
Gear entry and index redirectF09.entry-gear · F09.index-redirectMapped behavior

Scenario F09--gear-entry-and-index-redirect · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    Choose “Settings”.
  2. Wait
    Wait for the element identified in the CLI recipe to appear.
  3. Do
    Go back in browser history.
  4. Do
    Open /settings.

Expect

the sidebar gear opens Settings on the Agent page; the gear is marked active on every settings URL; at phone width the drawer's gear opens the hub and closes the drawer.

on desktop, /settings replaces itself with /settings/agents (Back skips /settings).

Exact CLI recipe & bug checks

From /, run control-openhands browser click 'testid=backend-selector-settings-link' then control-openhands browser wait-url '/settings/agents$'. Then control-openhands browser attr 'testid=sidebar-settings-/settings/agents' aria-current is page and control-openhands browser attr 'testid=backend-selector-settings-link' data-active is true (it is false again on /). Run control-openhands browser back then control-openhands browser url: the URL is /, not /settings, which proves the redirect replaced history. Direct entry: control-openhands browser goto /settings returns "url": ".../settings/agents".

Collapsed railF09.entry-collapsed-railMapped behavior

Scenario F09--collapsed-rail · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    Open /.
  2. Do
    Choose “Sidebar collapse toggle”.
  3. Do
    Choose “Collapsed settings”.
  4. Wait
    Wait for the element identified in the CLI recipe to appear.
  5. Do
    Choose “Sidebar collapse toggle”.

Expect

with the sidebar collapsed, the rail's Settings icon opens the Agent page.

Exact CLI recipe & bug checks

From / (control-openhands browser goto /; the previous bullet ends on /settings/agents, where the URL wait would pass at once) run control-openhands browser click 'testid=sidebar-collapse-toggle', control-openhands browser click 'testid=collapsed-settings-link', control-openhands browser wait-url '/settings/agents$' and control-openhands browser screenshot --feature F09.entry-collapsed-rail --name rail. The screenshot shows the icon rail, the settings nav with Agent highlighted. Restore with control-openhands browser click 'testid=sidebar-collapse-toggle' (the label flips back to "Collapse sidebar").

Command menuF09.entry-command-menuMapped behavior

Scenario F09--command-menu · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    Hover over “Search commands”.
  2. Do
    Open /.
  3. Do
    Press Control+k.
  4. Do
    Enter “LLM profiles” in “Command menu” / the combobox.
  5. Do
    Press Enter.
  6. Wait
    Wait for the element identified in the CLI recipe to appear.

Expect

the command menu's Settings group (Settings, Agent settings, LLM profiles, Model Router, Condenser, Agent Context, Verification, Application, Secrets settings) opens the matching page.

Exact CLI recipe & bug checks

First rest the pointer off the list with control-openhands browser hover 'testid=command-menu-trigger': a hover can move the menu's selection, and Enter runs the selected option (F02 Gotchas). For each title run control-openhands browser goto /, control-openhands browser press Control+k, control-openhands browser type 'testid=command-menu >> role=combobox' 'LLM profiles', control-openhands browser press Enter, control-openhands browser wait-url '/settings/llm$'. Expected targets: Settings → /settings/agents, Agent settings → /settings/agents, LLM profiles → /settings/llm, Model Router → /settings/meta-llm, Condenser settings → /settings/condenser, Agent Context → /settings/agent-context, Verification settings → /settings/verification, Application settings → /settings/app, Secrets settings → /settings/secrets. control-openhands browser snapshot 'testid=command-menu >> role=listbox' after typing LLM profiles shows two options under the "Settings" group text: LLM profiles [selected] and Model Router, whose subline mentions LLM profiles. Enter runs the selected one.

Command menu coverageF09.command-menu-coverageMapped behavior

Scenario F09--command-menu-coverage · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    Open /.
  2. Do
    Press Control+k.
  3. Do
    Enter “Model Router” in “Command menu” / the combobox.
  4. Do
    Press Enter.
  5. Wait
    Wait for the element identified in the CLI recipe to appear.

Expect

every page in the settings nav, including Model Router and Agent Context, can be found in the command menu and opens from it.

Exact CLI recipe & bug checks

Run control-openhands browser goto /, control-openhands browser press Control+k, control-openhands browser type 'testid=command-menu >> role=combobox' 'Model Router' and control-openhands browser snapshot 'testid=command-menu >> role=listbox'. It shows one option under the "Settings" group text: Model Router Meta-profiles that route each task to the right LLM profile. Go [selected]. Run control-openhands browser press Enter and control-openhands browser wait-url '/settings/meta-llm$'; control-openhands browser count 'testid=command-menu' is 0. Repeat from / with Agent Context: the option reads Agent Context Let the agent keep notes and recall them in new conversations. Go and the URL becomes /settings/agent-context. With the previous bullet, every page in the settings nav has opened from the menu.

Deep links without a modelF09.entry-deep-linksMapped behavior

Scenario F09--deep-links-without-a-model · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    Open /.
  2. Do
    Choose “Home llm not configured action”.
  3. Wait
    Wait for the element identified in the CLI recipe to appear.
  4. Do
    Open /.
  5. Do
    Choose “Sidebar onboarding checklist item configure llm”.
  6. Wait
    Wait for the element identified in the CLI recipe to appear.
  7. Do
    Open /.
  8. Do
    Choose “Sidebar onboarding checklist item customize agent”.
  9. Wait
    Wait for the element identified in the CLI recipe to appear.

Expect

the Home "Set up LLM" banner, the checklist items "Add LLM API key" and "Customize your agent", the composer LLM picker's "LLM Profiles" link and the + menu's "Manage agent profiles" link each land on /settings/llm or /settings/agents.

Exact CLI recipe & bug checks

On / with no LLM key (control-openhands browser goto /): control-openhands browser click 'testid=home-llm-not-configured-action' then control-openhands browser wait-url '/settings/' → /settings/llm. control-openhands browser goto /, control-openhands browser click 'testid=sidebar-onboarding-checklist-item-configure-llm', control-openhands browser wait-url '/settings/' → /settings/llm. control-openhands browser goto /, control-openhands browser click 'testid=sidebar-onboarding-checklist-item-customize-agent', control-openhands browser wait-url '/settings/' → /settings/agents.

Deep links from the composerF09.entry-deep-linksMapped behavior

Scenario F09--deep-links-from-the-composer · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Setup
    Configure or inspect the model profiles specified in the recipe.
  2. Do
    Open /.
  3. Do
    Choose “Chat input llm profile”.
  4. Do
    Choose “Chat input llm profile popover” / “LLM Profiles”.
  5. Wait
    Wait for the element identified in the CLI recipe to appear.
  6. Do
    Open /.
  7. Do
    Choose “Chat plus”.
  8. Do
    Hover over “Switch agent profile”.
  9. Do
    Choose “Agent profile submenu” / “Manage agent profiles”.
  10. Wait
    Wait for the element identified in the CLI recipe to appear.

Expect

the Home "Set up LLM" banner, the checklist items "Add LLM API key" and "Customize your agent", the composer LLM picker's "LLM Profiles" link and the + menu's "Manage agent profiles" link each land on /settings/llm or /settings/agents.

Exact CLI recipe & bug checks

After control-openhands llm preset deepseek, run control-openhands browser goto /, control-openhands browser click 'testid=chat-input-llm-profile', control-openhands browser click 'testid=chat-input-llm-profile-popover >> role=link[name="LLM Profiles"]', control-openhands browser wait-url '/settings/llm$'; control-openhands browser count 'testid=chat-input-llm-profile-popover' is 0. Then control-openhands browser goto /, control-openhands browser click 'testid=chat-plus-button', control-openhands browser hover 'testid=switch-agent-profile-button', control-openhands browser click 'testid=agent-profile-submenu >> role=link[name="Manage agent profiles"]', control-openhands browser wait-url '/settings/agents$'; control-openhands browser count 'testid=tools-context-menu' is 0.

Legacy URLF09.legacy-agent-redirectMapped behavior

Scenario F09--legacy-url · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    Open /.
  2. Do
    Open /settings/agent.
  3. Do
    Go back in browser history.

Expect

the retired /settings/agent URL replaces itself with /settings/agents.

Exact CLI recipe & bug checks

Run control-openhands browser goto /, then control-openhands browser goto /settings/agent; the result already reads .../settings/agents. control-openhands browser back returns to /.

Desktop navF09.desktop-navMapped behavior

Scenario F09--desktop-nav · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    Open /settings/agents.
  2. Do
    Choose “Meta llm settings”.
  3. Wait
    Wait for the element identified in the CLI recipe to appear.
  4. Do
    Open /settings/app.
  5. Do
    Hover over “Settings page subtitle”.
  6. Do
    Scroll the element identified in the CLI recipe as specified in the recipe.

Expect

the desktop nav lists the eight pages in order, exactly one link is aria-current="page", and the nav stays pinned while the page content scrolls.

Exact CLI recipe & bug checks

On any settings page (control-openhands browser goto /settings/agents; Legacy URL ends on /) run control-openhands browser snapshot 'testid=settings-navbar-desktop': links Agent, LLM, Model Router, Condenser, Agent Context, Verification, Application, Secrets (with /urls), then button "Agent Canvas Version …" and the synced text. Click each one, e.g. control-openhands browser click 'testid=sidebar-settings-/settings/meta-llm', control-openhands browser wait-url '/settings/meta-llm$', control-openhands browser attr 'testid=sidebar-settings-/settings/meta-llm' aria-current (page), and control-openhands browser count 'testid=settings-navbar-desktop >> [aria-current="page"]' (1). Pinned: control-openhands browser goto /settings/app (the only page taller than the viewport), control-openhands browser bbox 'testid=settings-navbar-desktop' (y 32), control-openhands browser hover 'testid=settings-page-subtitle', control-openhands browser scroll --by 1000, then control-openhands browser bbox 'testid=settings-page-subtitle' (negative y, content scrolled) and control-openhands browser bbox 'testid=settings-navbar-desktop' (still y 32); control-openhands browser screenshot --feature F09.desktop-nav --name scrolled shows Git Settings on the right and the full nav on the left.

Page headerF09.page-headerMapped behavior

Scenario F09--page-header · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    Open /settings/agents.
  2. Do
    Choose “Add agent profile”.
  3. Do
    Choose “Back to agent profiles”.

Expect

each page shows its nav title as an H2 with a grey subline; Secrets draws its own header instead, and the Agent profile editor hides the shell header until you leave it.

Exact CLI recipe & bug checks

After each nav click, control-openhands browser text 'testid=settings-screen >> header' reads title + subline: Agent "Create and manage reusable agent setups.", LLM "Model, API key, and options for the agent.", Model Router "Meta-profiles that route each task to the right LLM profile.", Condenser "Summarize long chats to stay within context limits.", Agent Context "Let the agent keep notes and recall them in new conversations.", Verification "Confirmation prompts and security checks on actions.", Application "Language, theme, notifications, and Git identity.". On Secrets, control-openhands browser count 'testid=settings-screen >> header >> h2' is 0 (the page draws its own "Secrets" heading). Editor: on /settings/agents (control-openhands browser goto /settings/agents) run control-openhands browser click 'testid=add-agent-profile'; control-openhands browser count 'testid=settings-page-subtitle' is 0 and control-openhands browser text 'testid=settings-screen >> h2' is Add agent profile. control-openhands browser click 'testid=back-to-agent-profiles' brings the subtitle back (count 1); opening the editor again and clicking testid=sidebar-settings-/settings/condenser also shows the Condenser subline.

LandmarksF09.landmarksMapped behavior

Scenario F09--landmarks · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    Open /settings/agents.
  2. Do
    Open /settings/app.
  3. Setup
    Use the phone viewport.
  4. Do
    Open /settings.
  5. Setup
    Use the desktop viewport.

Expect

the settings screen exposes a single main landmark, on desktop pages and on the phone hub.

Exact CLI recipe & bug checks

Run control-openhands browser goto /settings/agents, control-openhands browser count 'role=main' (1) and control-openhands browser eval "document.querySelector('[data-testid=settings-screen]').tagName" (DIV: the shell wrapper is not a landmark; the layout's own <main> is). control-openhands browser goto /settings/app and the same count is 1. At phone width (control-openhands browser viewport phone, control-openhands browser goto /settings) the hub also counts 1; return with control-openhands browser viewport desktop.

Phone hub and BackF09.phone-hub · F09.phone-backMapped behavior

Scenario F09--phone-hub-and-back · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Setup
    Use the phone viewport.
  2. Do
    Open /settings.
  3. Do
    Choose “Settings mobile hub” / “Condenser settings”.
  4. Wait
    Wait for the element identified in the CLI recipe to appear.
  5. Do
    Choose “Sidebar mobile back”.
  6. Wait
    Wait for the element identified in the CLI recipe to appear.
  7. Do
    Open /.
  8. Do
    Choose “Sidebar mobile menu toggle”.
  9. Do
    Choose “Sidebar mobile drawer” / “Settings”.
  10. Wait
    Wait for the element identified in the CLI recipe to appear.
  11. Wait
    Wait for “Sidebar mobile drawer” to disappear.
  12. Setup
    Use the desktop viewport.

Expect

below 1024 px (phones and tablets), /settings stays put and shows a "Settings" heading, the eight links, the version card and the synced note, with no page header and no horizontal overflow.

at phone width each settings page shows a Back chevron (label "Settings") in the top bar that returns to the hub.

Exact CLI recipe & bug checks

Run control-openhands browser viewport phone, control-openhands browser goto /settings (URL stays /settings), control-openhands browser snapshot 'testid=settings-mobile-hub' (heading "Settings", the eight links, the version button, the synced text), control-openhands browser count 'testid=settings-page-subtitle' (0), control-openhands browser bbox 'testid=settings-mobile-hub' (pageHorizontalOverflow false) and control-openhands browser screenshot --feature F09.phone-hub --name hub. Then control-openhands browser click 'testid=settings-mobile-hub >> testid=sidebar-settings-/settings/condenser', control-openhands browser wait-url '/settings/condenser$', control-openhands browser attr 'testid=sidebar-mobile-back-button' aria-label (Settings), control-openhands browser click 'testid=sidebar-mobile-back-button', control-openhands browser wait-url '/settings$' and control-openhands browser visible 'testid=settings-mobile-hub' (true). Drawer entry: control-openhands browser goto /, control-openhands browser click 'testid=sidebar-mobile-menu-toggle', control-openhands browser click 'testid=sidebar-mobile-drawer >> testid=backend-selector-settings-link', control-openhands browser wait-url '/settings$' and control-openhands browser wait 'testid=sidebar-mobile-drawer' --state detached --timeout 5000 pass (the drawer closes as it animates out; an immediate visible can still read true). Return with control-openhands browser viewport desktop.

Resize from the hubF09.hub-resize-redirectMapped behavior

Scenario F09--resize-from-the-hub · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    Open /.
  2. Setup
    Use the phone viewport.
  3. Do
    Open /settings.
  4. Setup
    Use the desktop viewport.
  5. Wait
    Wait for the element identified in the CLI recipe to appear.
  6. Do
    Go back in browser history.
  7. Do
    Open /settings/condenser.
  8. Setup
    Use the phone viewport.
  9. Setup
    Use the tablet viewport.
  10. Do
    Open /settings.
  11. Setup
    Use the 1023x900 viewport.
  12. Do
    Open /settings.
  13. Setup
    Use the 1024x900 viewport.
  14. Wait
    Wait for the element identified in the CLI recipe to appear.
  15. Setup
    Use the desktop viewport.

Expect

widening the window to ≥1024 px while the phone hub is open replaces /settings with /settings/agents; narrowing on a sub-page keeps its URL and adds the Back chevron.

Exact CLI recipe & bug checks

Run control-openhands browser goto /, control-openhands browser viewport phone, control-openhands browser goto /settings and control-openhands browser visible 'testid=settings-mobile-hub' (true). Then control-openhands browser viewport desktop and control-openhands browser wait-url '/settings/agents$': the hub is gone (count 'testid=settings-mobile-hub' is 0) and control-openhands browser back lands on / (the redirect replaced /settings). Reverse: control-openhands browser goto /settings/condenser, control-openhands browser viewport phone, control-openhands browser url (still /settings/condenser) and control-openhands browser visible 'testid=sidebar-mobile-back-button' (true). Tablets get the hub too, and the boundary is 1024 px: run control-openhands browser viewport tablet (820 px), control-openhands browser goto /settings (URL stays /settings) and control-openhands browser visible 'testid=settings-mobile-hub' (true); then control-openhands browser viewport 1023x900, control-openhands browser goto /settings and the same visible (true); then control-openhands browser viewport 1024x900 and control-openhands browser wait-url '/settings/agents$' (browser visible 'testid=settings-navbar-desktop' is true). Return with control-openhands browser viewport desktop.

Widths around the breakpointF09.breakpoint-sweepMapped behavior

Scenario F09--widths-around-the-breakpoint · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Setup
    Use the Wx1024 viewport.
  2. Do
    Open /settings.
  3. Wait
    Wait for the element identified in the CLI recipe to appear.
  4. Do
    Open /settings/app.
  5. Setup
    Use the desktop viewport.

Expect

at 767, 768, 820, 1023, 1024 and 1440 px, /settings is the hub below 1024 px and redirects to the desktop nav from 1024 px up, and the Application page's main is at least 340 px wide with no horizontal overflow. Below 1024 px the settings <main> fills the column down to the viewport bottom and scrolls itself, so a popover inside it (the tablet selector) is not clipped on a short page (#18256).

Exact CLI recipe & bug checks

For each width W in 767, 768, 820, 1023, 1024, 1440 run control-openhands browser viewport Wx1024 and control-openhands browser goto /settings. Below 1024 px the URL stays /settings, control-openhands browser visible 'testid=settings-mobile-hub' is true and control-openhands browser bbox 'testid=settings-mobile-hub' has pageHorizontalOverflow false (725, 426, 478 and 681 px wide); from 1024 px up control-openhands browser wait-url '/settings/agents$' passes and control-openhands browser visible 'testid=settings-navbar-desktop' is true. Then control-openhands browser goto /settings/app and control-openhands browser bbox 'role=main': width is at least 340 (767, 468, 520, 723, 424 and 840 today: full width at 767 px, W − 300 beside the rail up to 1023 px, W − 600 beside the rail and the settings nav from 1024 px), scrollWidth equals clientWidth and pageHorizontalOverflow is false (insideViewport is false only because the page is taller than 1024 px). Take control-openhands browser screenshot --feature F09.breakpoint-sweep --name app-767 and --name app-1024. Height: at 390x844, 767x1024, 820x1180 and 1023x900, on /settings/secrets (a short page) control-openhands browser eval "Math.round(document.querySelector('[data-testid=settings-screen] main').getBoundingClientRect().bottom)+'/'+innerHeight" reads the viewport height twice (1180/1180 at 820x1180), and on /settings/app control-openhands browser eval "(()=>{const m=document.querySelector('[data-testid=settings-screen] main');return m.scrollHeight>m.clientHeight})()" is true (the <main> scrolls, not an outer column). At 1440x1000 the desktop layout is unchanged (the Secrets <main> ends at its content). Finish with control-openhands browser viewport desktop.

Tablet selectorF09.tablet-selectorMapped behavior

Scenario F09--tablet-selector · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Setup
    Use the tablet viewport.
  2. Do
    Open /settings/app.
  3. Do
    Choose “Settings compact navigation” / the button.
  4. Do
    Choose “Settings compact navigation” / “Secrets settings”.
  5. Do
    Press Tab.
  6. Do
    Press Escape.

Expect

at 768–1023 px, a Settings selector above the subpage opens the same visible section links as the desktop sidebar, including Cloud links when available. Selecting a sibling navigates directly and closes it; Tab reaches links, Escape closes and returns focus, and outside clicks dismiss it. The current section uses a slash-boundary match, so a trailing slash does not hide the selector or its selected link. It is absent on the hub, phones and desktops.

Exact CLI recipe & bug checks

Run control-openhands browser viewport tablet, control-openhands browser goto /settings/app, then control-openhands browser click 'testid=settings-compact-navigation >> role=button'. control-openhands browser snapshot 'testid=settings-compact-navigation >> role=navigation' lists every visible settings destination with Application current. Select control-openhands browser click 'testid=settings-compact-navigation >> testid=sidebar-settings-/settings/secrets' --expect-url '/settings/secrets$'; the selector now shows Secrets and its button has aria-expanded=false. Open again, browser press Tab, browser press Escape: browser eval "document.activeElement.getAttribute('aria-expanded')" is false (focus returned to the trigger). Capture browser screenshot --feature F09.tablet-selector --name settings. Repeat at 768 and 1023 px; at 767 and 1024 px browser count 'testid=settings-compact-navigation' is 0. On /settings below 1024 px the hub remains the list. At tablet width, visit /settings/llm/: the trigger reads Settings: LLM, the page heading is LLM, and opening the selector marks LLM current. A nested URL such as /settings/llm/example is not a registered page and shows the global Not Found screen (F09.unknown-subpath).

Synced badgeF09.synced-badgeMapped behavior

Scenario F09--synced-badge · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    Run control-openhands browser text 'testid=backend-synced-settings-badge' and control-openhands browser eval 'location.origin'. The text is These settings are synced from Local backend (<origin>) with the same origin.

Expect

the nav footer reads "These settings are synced from Local backend (<origin>)" for the active backend.

Exact CLI recipe & bug checks

Run control-openhands browser text 'testid=backend-synced-settings-badge' and control-openhands browser eval 'location.origin'. The text is These settings are synced from Local backend (<origin>) with the same origin.

Cloud linksF09.cloud-linksBlocked

Scenario F09--cloud-links · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    On a local backend control-openhands browser count 'testid=settings-integrations-link' and control-openhands browser count 'testid=settings-cloud-link' are both 0. Positive case (blocked without a Cloud account): with a Cloud backend active both exist; "All Cloud Settings" points at <cloud host>/settings?org=<org id> and opens in a new tab unless the canvas is locked to Cloud.

Expect

on a Cloud backend only, the nav adds "Integrations" and "All Cloud Settings" external links; on a local backend neither exists.

Exact CLI recipe & bug checks

On a local backend control-openhands browser count 'testid=settings-integrations-link' and control-openhands browser count 'testid=settings-cloud-link' are both 0. Positive case (blocked without a Cloud account): with a Cloud backend active both exist; "All Cloud Settings" points at <cloud host>/settings?org=<org id> and opens in a new tab unless the canvas is locked to Cloud.

Update card and dialogF09.update-card · F09.update-modal · F09.update-up-to-dateMapped behavior

Scenario F09--update-card-and-dialog · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    Choose “Agent canvas update toggle”.
  2. Wait
    Wait for “Agent canvas update modal” to appear.
  3. Do
    Choose “Agent canvas update check”.
  4. Do
    Choose “Close agent canvas update modal”.
  5. Do
    Press Escape.

Expect

the "Agent Canvas / Version x.y.z" card opens a dialog; its "Up to date" / "Update" badge appears only once the npm check succeeded.

the dialog shows the version, a "Check for updates" button that re-runs the check, and an inline status; a failed check reads "Couldn't check for updates. Try again later."; Close and Escape dismiss it.

when the npm check succeeds and the client is current, the card badge reads "Up to date", the dialog status reads "You're running the latest version." with a "What's new" link to the GitHub releases page (new tab), and "Check for updates" briefly shows "Checking for updates…".

Exact CLI recipe & bug checks

On a settings page run control-openhands browser text 'testid=agent-canvas-update-toggle' (Agent Canvas, Version, the client version, plus the badge text when present), control-openhands browser count 'testid=agent-canvas-update-badge' (0 while the npm check fails; 1 with "Up to date" or "Update" when it succeeds), then control-openhands browser click 'testid=agent-canvas-update-toggle', control-openhands browser wait 'testid=agent-canvas-update-modal' and control-openhands browser text 'testid=agent-canvas-update-status'. With registry access and a current client the status is You're running the latest version. What's new, and control-openhands browser attr 'testid=agent-canvas-update-release-notes' href is https://github.com/OpenHands/OpenHands/releases (target _blank). Without registry access the status is Couldn't check for updates. Try again later. Re-check: run control-openhands browser network --clear, then control-openhands browser click 'testid=agent-canvas-update-check-button' --observe 'testid=agent-canvas-update-modal' --observe-ms 2000 (the observed states include Checking for updates… before the final status), then control-openhands browser network: exactly one new GET https://registry.npmjs.org/@openhands/agent-canvas/latest proves the button re-checked (browser errors does not list successful requests). control-openhands browser enabled 'testid=agent-canvas-update-check-button' is true again. Close with control-openhands browser click 'testid=close-agent-canvas-update-modal' (modal count 0); reopen and control-openhands browser press Escape also closes it.

Update availableF09.update-availableKnown issueBlocked

Scenario F09--update-available · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    Blocked unless the browser reaches the registry and npm latest is newer than the client (a checkout at the latest release shows F09.update-up-to-date instead). With both, and a newer npm latest than the client, the status shows Version <x> is available. with testid=agent-canvas-update-release-notes ("What's new"), the npm tab testid=agent-canvas-update-command-npm reads npm install -g @openhands/agent-canvas@latest, the Docker tab reads docker pull ghcr.io/openhands/agent-canvas:latest, and testid=copy-to-clipboard turns into a check mark for 2 s.

Expect

when npm has a newer version, the dialog shows "Version … is available.", a "What's new" link, and npm/Docker command tabs with a copy button.

Linked known issues

Known issue #17508 (F09.update-available): After installing a newer package, an older running bundle can keep showing its own version and an update banner. Live prerequisite: Isolated old/new package installs with the old process still serving, or the original stale bundle/cache capture after update.

Exact CLI recipe & bug checks

Blocked unless the browser reaches the registry and npm latest is newer than the client (a checkout at the latest release shows F09.update-up-to-date instead). With both, and a newer npm latest than the client, the status shows Version <x> is available. with testid=agent-canvas-update-release-notes ("What's new"), the npm tab testid=agent-canvas-update-command-npm reads npm install -g @openhands/agent-canvas@latest, the Docker tab reads docker pull ghcr.io/openhands/agent-canvas:latest, and testid=copy-to-clipboard turns into a check mark for 2 s.

Update dialog at phone widthF09.update-modal-phoneMapped behavior

Scenario F09--update-dialog-at-phone-width · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Setup
    Use the phone viewport.
  2. Do
    Open /settings.
  3. Do
    Choose “Settings mobile hub” / “Agent canvas update toggle”.
  4. Wait
    Wait for “Agent canvas update modal” to appear.
  5. Do
    Choose “Close agent canvas update modal”.
  6. Setup
    Use the narrow viewport.
  7. Do
    Press Escape.
  8. Setup
    Use the desktop viewport.

Expect

the update dialog fits a 390 px (and a 320 px) viewport with its title and Close button on screen, and Close dismisses it.

Exact CLI recipe & bug checks

Run control-openhands browser viewport phone, control-openhands browser goto /settings, control-openhands browser click 'testid=settings-mobile-hub >> testid=agent-canvas-update-toggle', control-openhands browser wait 'testid=agent-canvas-update-modal', control-openhands browser bbox 'testid=agent-canvas-update-modal' and control-openhands browser bbox 'testid=close-agent-canvas-update-modal', then control-openhands browser screenshot --feature F09.update-modal-phone --name dialog. Both boxes have insideViewport true and pageHorizontalOverflow false: the dialog is 351 px wide at x 19.5 and Close is at x 329.5; the screenshot shows the whole "Agent Canvas" title and the X. control-openhands browser click 'testid=close-agent-canvas-update-modal' closes it (control-openhands browser count 'testid=agent-canvas-update-modal' is 0). Run control-openhands browser viewport narrow (320×700) and repeat the goto, click, wait and both bbox commands: the dialog is 288 px wide at x 16 and Close is at x 263, both inside the viewport; control-openhands browser press Escape closes it. Then control-openhands browser viewport desktop.

Sidebar version tileF09.sidebar-version-tileBlocked

Scenario F09--sidebar-version-tile · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    On / run control-openhands browser count 'testid=agent-canvas-version-tile'. It is 0 unless the npm check found a newer version, also when the check succeeded with "Up to date" (positive case blocked without registry access or while the client is the latest release).

Expect

the main sidebar shows an "Agent Canvas" version tile only when an update is available.

Exact CLI recipe & bug checks

On / run control-openhands browser count 'testid=agent-canvas-version-tile'. It is 0 unless the npm check found a newer version, also when the check succeeded with "Up to date" (positive case blocked without registry access or while the client is the latest release).

Unknown sub-pageF09.unknown-subpathMapped behavior

Scenario F09--unknown-sub-page · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    Open /settings/does-not-exist.
  2. Do
    Choose “Not found home”.

Expect

an unknown /settings/<x> URL shows the app's "Page not found" page inside the app shell (sidebar and a Home link), without the settings shell.

Exact CLI recipe & bug checks

Run control-openhands browser goto /settings/does-not-exist and control-openhands browser snapshot 'testid=not-found-screen': heading "Page not found", the paragraph This address does not match any page. Check the URL, or go back to the home page. and link "Home". The settings shell is absent (control-openhands browser count 'testid=settings-screen' and control-openhands browser count 'testid=settings-navbar-desktop' are 0), while the app sidebar stays (control-openhands browser count 'aside[data-collapsed]' is 1). control-openhands browser screenshot --feature F09.unknown-subpath --name not-found shows the message beside the sidebar. Leave with control-openhands browser click 'testid=not-found-home-link' --expect-url '/$'.

After the familyF09 · setup / cleanupSupport step

Scenario F09--after-the-family · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

  1. Do
    Run control-openhands browser errors --clear before the family and control-openhands browser errors --app-only after it. Expected: pageErrors 0 and appErrors 0; the unknown sub-page adds no errors. The Agent profile editor logs [i18n] Missing translation for key "SCHEMA$TOOL_CONCURRENCY_LIMIT$…" console warnings, counted under warnings, not as errors (the Agent page's family). A registry request failure, when the sandbox blocks npm, is external.

Expect

Complete this support step before continuing to the next scenario.

Exact CLI recipe & bug checks

Run control-openhands browser errors --clear before the family and control-openhands browser errors --app-only after it. Expected: pageErrors 0 and appErrors 0; the unknown sub-page adds no errors. The Agent profile editor logs [i18n] Missing translation for key "SCHEMA$TOOL_CONCURRENCY_LIMIT$…" console warnings, counted under warnings, not as errors (the Agent page's family). A registry request failure, when the sandbox blocks npm, is external.

Locked cloud landingF09.locked-cloud-landingBehavior contract

Scenario F09--contract-f09-locked-cloud-landing · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do

This behavior has no separately named recipe in this snapshot. Use the family’s entry points and the contract below; the page does not invent a replay.

Expect

on a canvas locked to a Cloud host, /settings lands on /settings/app instead.

Hidden llm redirectF09.hidden-llm-redirectBehavior contract

Scenario F09--contract-f09-hidden-llm-redirect · Permalink · Source

Given

Use F09’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do

This behavior has no separately named recipe in this snapshot. Use the family’s entry points and the contract below; the page does not invent a replay.

Expect

with the hide_llm_settings feature flag, the LLM link disappears and /settings/llm redirects to /settings/agents.

Family notes, prerequisites for known issues & limits Known issueBlocked
  • browser click returns the URL before client-side navigation and redirects settle (it prints / or /settings). Always follow a navigating click with browser wait-url '<regex>$' before asserting.
  • At phone width there are two backend-selector-settings-link elements (hidden desktop sidebar plus the drawer); scope with testid=sidebar-mobile-drawer >> ... or the click fails in strict mode. The hub's nav links share testids with the desktop nav, so scope them with testid=settings-mobile-hub >> ....
  • The page does not scroll inside the settings <main>; the app shell column does. browser scroll <selector> --by N only scrolls the element into view and ignores --by; hover the content first, then browser scroll --by N (mouse wheel at the pointer).
  • Only /settings/app is taller than a 1000 px viewport in a fresh run; use it for the pinned-nav check.
  • Visiting /settings/agents ticks the checklist item "Customize your agent" for good (struck through, still a working link, so the bullet order above is fine), and configuring a model hides the Home banner. Run the banner deep link before llm preset on a fresh run.
  • Registry reachability depends on the sandbox and changes between runs. When the browser cannot reach registry.npmjs.org (seen as ERR_CERT_AUTHORITY_INVALID through a proxy), the version badge and sidebar tile stay hidden and the dialog shows the failure state; that is environment, not product. When it can, a checkout at the latest release shows "Up to date" and the failure text cannot be produced. Check browser network for the registry request before deciding which update rows are blocked.
  • The update dialog's role="dialog" wrapper has no accessible name, so role=dialog[name=...] does not match it; use testid=agent-canvas-update-modal.
  • The /settings → /settings/agents redirect only happens at ≥1024 px. Below that (phones and tablets) /settings is the hub; do not expect a redirect there.
  • From 768 to 1023 px the sidebar rail stays and the phone top bar is gone; the Settings selector above each subpage provides sibling navigation. The rail's gear (backend-selector-settings-link) still returns to the hub.
  • The phone hub is indented 32 px while sub-pages start at 16 px (hub padding inside the page padding); that is current layout, not overflow.
  • The "LLM Profiles" rename in useSettingsNavItems targets a /settings nav item that does not exist, so it never applies; the LLM nav item always reads "LLM".
  • Known issue #17508 (F09.update-available): After installing a newer package, an older running bundle can keep showing its own version and an update banner. Live prerequisite: Isolated old/new package installs with the old process still serving, or the original stale bundle/cache capture after update.
  • The public SettingsNavigation export renders the legacy SettingsMobileDrawer with a half-opacity black scrim (bg-black/50). Check its modern and fallback compiled CSS as a build contract. The routed phone hub uses SettingsMobileHub, so its navigation recipe does not exercise the exported drawer; visual verification of that export needs an embedding host.

Before you start

Use an OpenHands checkout at the source revision, with Node 24+, installed npm dependencies, uv / uvx, and a supported Chromium browser. The verification skill is the complete setup contract.

export PATH="$PWD/.agents/skills/verify-openhands/scripts:$PATH"
export OH_VERIFY_RUN=$(control-openhands launch --new --print-run)
control-openhands doctor
control-openhands onboard --skip
# Configure a model only for a scenario that needs one:
control-openhands llm preset deepseek
# When the run is complete:
control-openhands stop

F01 walks onboarding instead of skipping it. No-model checks run before model configuration. Keep OH_VERIFY_RUN set so commands address your own run. Desktop is 1440 × 1000; phone is 390 × 844. Model-backed checks need the configured provider key and budget.

Recipes run in source order within each family and can share fixtures. The page preserves that order; “as above” is a dependency on an earlier case. Setup commands and API writes arrange state. They do not prove a UI path. Follow the family cleanup steps.

A known failure preserves the intended expectation beside the observed defect. Executable known-bug blocks use xfail for a reproduced symptom and xpass when it stops reproducing. Cloud accounts, enterprise capabilities and other unavailable prerequisites remain explicit. “Not run” is different from “blocked”.

The maintained map describes application behavior at its pinned revision. Once a fix is verified in the application version under test, remove the obsolete failure note and keep the normal expectation. Dated evidence belongs in the backfill ledger.

Read the full map index and coverage boundaries, the CLI guide, or the earlier illustrated edition. The verification approach adapts Lauren Tan’s (@poteto) pstack verification-skill generators; attribution and adaptation.