Customize hub and MCP servers

F17 · 31 mapped behaviors in the Self-Healing Codebase Project. Follow the prerequisites, actions and expected results; expand the original CLI recipe to replay it.

778 behavior IDs · 28 families · 844 named recipes & support steps
16 October 2026 · PR #18334 at 9a3c3a69
Canvas 1.27.1 · Agent Server 1.55.0 · automation 1.20.0

This is the map at the revision above, including its open bug notes. It does not incorporate later changes on main or claim a fresh verification run. Setup & reading guide · Download the scenario inventory

All scenarios & marker key · Open a script to read Given / Do / Expect. Expand the exact recipe for replayable commands.

F17 Customize hub and MCP servers

31 behaviors · 37 scripts & contracts · Maintained source

Entry points & prerequisites

What this family covers

Customize groups MCP Servers, Skills, Plugins and Apps behind one sidebar entry. From 1024 px up /customize goes straight to /mcp; below that (phones and tablets) it is a list hub. At tablet widths (768–1023 px), each Customize subpage also has a compact selector for switching directly between siblings. The MCP page lets a user browse a library of catalog integrations, install one through a modal (credentials masked, optionally also saved as secrets), hand-author custom SSE, Streamable HTTP or STDIO servers, test connections, check each installed server's health, enable or disable, edit and delete servers. Enabled servers are offered to every new conversation's agent.

How a user gets here

Sidebar Customize (sidebar-skills-link); on desktop it lands on /mcp. On a phone, open the drawer (sidebar-mobile-menu-toggle) first; it lands on the hub.
Direct URLs /customize and /mcp.
Command menu (Control+k/Meta+k): Customize (to /customize, then /mcp) and MCP servers (to /mcp). The menu itself is F02.
Customize sub-navigation row MCP Servers (sidebar-extensions-/mcp), desktop aside or phone hub. Pinning Customize as home is F02.
The onboarding checklist item Connect an MCP integration links to /mcp and is ticked once a server is configured, before any conversation uses it (F02).
On /mcp: a library card or its + toggle (install modal), Add custom server (mcp-add-custom-server), an installed card (editor) and its health-row buttons.

Given

Preconditions:

Baseline state (launched, doctored, onboard --skip done) on a fresh run, so nothing is installed yet (control-openhands api GET /api/settings --pick agent_settings.mcp_config is {}).
uvx and npx on the machine. F17.agent-uses-server, F17.custom-add-remote's agent check and the disabled-server check need an active LLM profile (control-openhands llm preset deepseek).
TLS-intercepting proxy (this sandbox): MCP stdio servers start with a minimal environment (HOME, PATH, no proxy or CA variables), so without trust settings uvx/npx cannot download packages and catalog STDIO installs fail. launch now seeds the run's private HOME when SSL_CERT_FILE or NODE_EXTRA_CA_CERTS points at a CA bundle: check with cat "$OH_VERIFY_RUN/private/home/.config/uv/uv.toml" (native-tls = true) and cat "$OH_VERIFY_RUN/private/home/.npmrc" (cafile=<bundle>). Only if they are missing, write them once: mkdir -p "$OH_VERIFY_RUN/private/home/.config/uv", printf 'native-tls = true\n' > "$OH_VERIFY_RUN/private/home/.config/uv/uv.toml" and printf 'cafile=/root/.ccr/ca-bundle.crt\n' > "$OH_VERIFY_RUN/private/home/.npmrc" (use your proxy's CA bundle path). F17.install-error moves them aside for one step. Custom servers can also pass UV_NATIVE_TLS=true in their env. Skip all of this on a machine with direct internet.
F17.custom-add-remote needs a local Streamable HTTP MCP server (arrange, not proof; no fixture verb exists for it yet): from a shell, cd "$OH_VERIFY_RUN/workspace" && (PORT=38417 nohup npx -y @modelcontextprotocol/server-everything streamableHttp > "$OH_VERIFY_RUN/private/qa-shttp.log" 2>&1 &), pick any free port, and wait until the log says MCP Streamable HTTP Server listening on port 38417. Stop it in Cleanup by its own PID (pgrep -f 'server-everything streamableHttp' lists it; never pkill -f).
F17.cloud-nav and F17.native-git-tabs need an OpenHands Cloud backend (with git providers configured); not available here.
F17.stored-secret-reuse and F17.single-request-mutations use control-openhands fixture mcp-server --name qa-vault; it prints the <node> binary and <server.mjs> path the recipes substitute.
F17.credential-probe sends a dummy token to GitHub's hosted MCP endpoint and, through the catalog's npx stdio server, to Slack; both need the network. Behind the TLS-intercepting proxy the stdio server's own HTTPS call fails unless it gets the proxy CA (NODE_EXTRA_CA_CERTS), which only the custom editor can pass.
Never type real credentials: every key below is a dummy.

Desktop entry pointsF17.customize-redirectMapped behavior

Scenario F17--desktop-entry-points · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Open /conversations.
Do
Choose “Sidebar skills”.
Do
Open /customize.
Wait
Wait for the element identified in the CLI recipe to appear.
Do
Open /conversations.
Do
Press Control+k.
Do
Enter “MCP” in “Command menu” / the combobox.
Do
Press Enter.
Wait
Wait for the element identified in the CLI recipe to appear.

Expect

on desktop, the sidebar Customize link, the URL /customize and the command-menu items Customize and MCP servers all land on /mcp.

Exact CLI recipe & bug checks

Run control-openhands browser goto /conversations, control-openhands browser click 'testid=sidebar-skills-link' --expect-url '/mcp(\?|$)' and control-openhands browser count 'testid=mcp-page' (1). Run control-openhands browser goto /customize and control-openhands browser wait-url '/mcp(\?|$)'. Command menu: run control-openhands browser goto /conversations (the browser is on /mcp, where the URL wait would pass at once), control-openhands browser press Control+k, control-openhands browser type 'testid=command-menu >> role=combobox' MCP, control-openhands browser press Enter and control-openhands browser wait-url '/mcp(\?|$)'; repeat with the term Customize (its only option reads Customize Browse skills, plugins, and integrations.). Every path ends on /mcp.

Desktop sub-navigationF17.desktop-subnavMapped behavior

Scenario F17--desktop-sub-navigation · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Sidebar extensions /skills”.
Do
Choose “Sidebar extensions /mcp”.

Expect

from 1024 px up, Customize pages show a left aside headed Customize with MCP Servers, Skills, Plugins and Apps, the active row marked, and the "synced from" backend badge.

Exact CLI recipe & bug checks

On /mcp run control-openhands browser text 'testid=extensions-navbar-desktop': Customize, MCP Servers, Skills, Plugins, Apps, then These settings are synced from Local backend (<base url>). control-openhands browser attr 'testid=sidebar-extensions-/mcp' aria-current is page. Run control-openhands browser click 'testid=sidebar-extensions-/skills' --expect-url '/skills(\?|$)'; that row's aria-current becomes page. Do the same for /plugins and /apps, then return with control-openhands browser click 'testid=sidebar-extensions-/mcp' --expect-url '/mcp(\?|$)'. control-openhands browser screenshot 'testid=extensions-navbar-desktop' --feature F17.desktop-subnav --name subnav shows the MCP Servers row highlighted.

Tablet selectorF17.tablet-selectorMapped behavior

Scenario F17--tablet-selector · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Setup
Use the tablet viewport.
Do
Open /mcp.
Do
Choose “Extensions compact navigation” / the button.
Do
Choose “Extensions compact navigation” / “Sidebar extensions /skills”.

Expect

at 768–1023 px, the compact Customize selector exposes the same MCP, Skills, Plugins and Apps links as the desktop nav; it follows Cloud visibility/external-link rules, dismisses on selection/outside click/Escape, and supports Tab navigation. It is absent on phones, desktops and the hub.

Exact CLI recipe & bug checks

Run control-openhands browser viewport tablet, control-openhands browser goto /mcp, and control-openhands browser click 'testid=extensions-compact-navigation >> role=button'. control-openhands browser snapshot 'testid=extensions-compact-navigation >> role=navigation' lists MCP Servers (current), Skills, Plugins and Apps. control-openhands browser click 'testid=extensions-compact-navigation >> testid=sidebar-extensions-/skills' --expect-url '/skills$' navigates directly and closes the list. Repeat from Skills to Plugins, Apps and MCP; the selector's current label follows each route. Capture browser screenshot --feature F17.tablet-selector --name customize. Check 768 and 1023 px, then 767 and 1024 px where browser count 'testid=extensions-compact-navigation' is 0. Escape after keyboard opening returns focus to the trigger. The same links have sidebar-extensions-* IDs in the hidden desktop nav, so scope every tablet action to extensions-compact-navigation.

Phone hubF17.mobile-hub · F17.phoneMapped behavior

Scenario F17--phone-hub · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Setup
Use the phone viewport.
Do
Open /customize.
Do
Choose “Extensions mobile hub” / “Sidebar extensions /mcp”.
Do
Choose “Sidebar mobile back”.
Do
Open /conversations.
Do
Choose “Sidebar mobile menu toggle”.
Do
Choose “Sidebar mobile drawer” / “Sidebar skills”.
Setup
Use the tablet viewport.
Do
Choose “Extensions mobile hub” / “Sidebar extensions /mcp”.
Do
Choose “Sidebar skills”.
Setup
Use the desktop viewport.
Wait
Wait for the element identified in the CLI recipe to appear.

Expect

below 1024 px (phones and tablets), /customize is a hub with the same four rows and badge; at phone width detail pages show a Back chevron to the hub; widening the window to 1024 px or more on /customize redirects to /mcp.

the MCP page, the install modal and the custom editor fit a 390 px viewport.

Exact CLI recipe & bug checks

Run control-openhands browser viewport phone, control-openhands browser goto /customize and control-openhands browser text 'testid=extensions-mobile-hub' (the same five lines as the desktop aside); control-openhands browser screenshot --feature F17.mobile-hub --name hub. Run control-openhands browser click 'testid=extensions-mobile-hub >> testid=sidebar-extensions-/mcp' --expect-url '/mcp(\?|$)'; control-openhands browser attr 'testid=sidebar-mobile-back-button' aria-label is Customize, and control-openhands browser count 'testid=extensions-navbar-desktop >> visible' is 0. control-openhands browser bbox 'testid=mcp-page' has insideViewport true and pageHorizontalOverflow false; control-openhands browser screenshot --feature F17.phone --name mcp-page shows the header, toolbar and one column of cards. Run control-openhands browser click 'testid=sidebar-mobile-back-button' --expect-url '/customize(\?|$)'; control-openhands browser count 'testid=extensions-mobile-hub' is 1. Drawer entry: control-openhands browser goto /conversations, control-openhands browser click 'testid=sidebar-mobile-menu-toggle' and control-openhands browser click 'testid=sidebar-mobile-drawer >> testid=sidebar-skills-link' --expect-url '/customize(\?|$)'; the hub is shown. Tablets get the hub too: control-openhands browser viewport tablet (820 px) keeps it on /customize; run control-openhands browser click 'testid=extensions-mobile-hub >> testid=sidebar-extensions-/mcp' --expect-url '/mcp(\?|$)', then control-openhands browser visible 'testid=extensions-navbar-desktop' and control-openhands browser visible 'testid=sidebar-mobile-back-button' (both false), control-openhands browser bbox 'role=main' (520 px wide, pageHorizontalOverflow false) and control-openhands browser screenshot --feature F17.mobile-hub --name tablet-mcp; the rail's control-openhands browser click 'testid=sidebar-skills-link' --expect-url '/customize(\?|$)' brings the hub back. Then control-openhands browser viewport desktop and control-openhands browser wait-url '/mcp(\?|$)': widening on /customize redirects to /mcp.

Widths around the breakpointF17.breakpoint-sweepMapped behavior

Scenario F17--widths-around-the-breakpoint · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Setup
Use the Wx1024 viewport.
Do
Open /customize.
Do
Choose “Extensions mobile hub” / “Sidebar extensions /mcp”.
Wait
Wait for the element identified in the CLI recipe to appear.
Do
Enter “GitHub” in “Mcp search”.
Do
Choose “Mcp search clear”.
Setup
Use the desktop viewport.

Expect

at 767, 768, 820, 1023, 1024 and 1440 px, /customize is the hub below 1024 px and the desktop aside from 1024 px up, and the MCP page keeps Add custom server, a search box wider than 150 px and no horizontal overflow.

Exact CLI recipe & bug checks

For each width W in 767, 768, 820, 1023, 1024, 1440 run control-openhands browser viewport Wx1024 and control-openhands browser goto /customize. Below 1024 px control-openhands browser visible 'testid=extensions-mobile-hub' is true and control-openhands browser click 'testid=extensions-mobile-hub >> testid=sidebar-extensions-/mcp' --expect-url '/mcp(\?|$)' opens the MCP page; from 1024 px up control-openhands browser wait-url '/mcp(\?|$)' passes at once and control-openhands browser count 'testid=extensions-mobile-hub' is 0. On /mcp, control-openhands browser visible 'testid=extensions-navbar-desktop' is false below 1024 px and true from 1024 px (the switch is exactly 1023 → 1024), control-openhands browser visible 'testid=mcp-add-custom-server' is true, control-openhands browser bbox 'testid=mcp-search-input' is wider than 150 px (622, 323, 375, 578, 257 and 673 today) and control-openhands browser bbox 'role=main' has pageHorizontalOverflow false with equal scrollWidth and clientWidth (main is full width at 767 px, W − 300 beside the rail from 768 to 1023 px, W − 640 beside the rail and the aside from 1024 px). control-openhands browser fill 'testid=mcp-search-input' GitHub keeps control-openhands browser count 'testid=mcp-marketplace-card-github' at 1; control-openhands browser click 'testid=mcp-search-clear'. At the boundary take control-openhands browser screenshot --feature F17.breakpoint-sweep --name mcp-1023 and --name mcp-1024. Finish with control-openhands browser viewport desktop.

Page and empty stateF17.page-statesMapped behavior

Scenario F17--page-and-empty-state · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Open /mcp.

Expect

/mcp shows the Model Context Protocol (MCP) header, Add custom server, the toolbar, an Installed section (empty-state copy when nothing is installed) and the Library grid, without page errors.

Exact CLI recipe & bug checks

On a fresh run run control-openhands browser goto /mcp, control-openhands browser text 'testid=mcp-installed-empty' (No MCP servers installed yet. and Pick one from the marketplace below to get started.) and control-openhands browser count '[data-testid^="mcp-marketplace-card-"]' (56 with @openhands/extensions 0.30.0, #18336, which added an installable Bitbucket card; 55 before; the catalog ships with @openhands/extensions). control-openhands browser eval "[...document.querySelectorAll('main h2')].map(h=>h.textContent)" is ["Model Context Protocol (MCP)","Installed","Library"]. Take control-openhands browser screenshot --feature F17.page-states --name empty-desktop. After the family, control-openhands browser errors --app-only shows pageErrors 0.

SearchF17.searchMapped behavior

Scenario F17--search · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Enter “time” in “Mcp search”.
Do
Choose “Mcp search clear”.

Expect

one search box filters the library and the installed list (catalog name, description and args count); no match shows No matches for your search. in each section; the X clears it.

Exact CLI recipe & bug checks

Run control-openhands browser fill 'testid=mcp-search-input' time and control-openhands browser count '[data-testid^="mcp-marketplace-card-"]' (1, mcp-marketplace-card-time). Fill qa-zzz-nothing; control-openhands browser text 'testid=mcp-marketplace-empty' is No matches for your search. (the installed list's search needs installed cards: it is driven at the end of F17.install-stdio). Run control-openhands browser click 'testid=mcp-search-clear'; control-openhands browser value 'testid=mcp-search-input' is "" and the card count is back to 56.

Section filterF17.section-filterMapped behavior

Scenario F17--section-filter · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Mcp section filter” / “Dropdown trigger”.
Do
Choose “Mcp section filter library”.

Expect

the All / Installed / Library dropdown shows only the chosen sections.

Exact CLI recipe & bug checks

Run control-openhands browser click 'testid=mcp-section-filter >> testid=dropdown-trigger', control-openhands browser snapshot 'testid=mcp-section-filter' (button "Filter MCP servers" [expanded]: All and menu "Filter MCP servers" with All [checked], Installed, Library) and control-openhands browser click 'testid=mcp-section-filter-library'; the main h2 list above loses Installed. Open the dropdown again and click testid=mcp-section-filter-installed; control-openhands browser count 'testid=mcp-marketplace-section' is 0 and control-openhands browser text 'testid=mcp-section-filter >> testid=dropdown-trigger' is Installed. Restore with testid=mcp-section-filter-all.

Remote entry, masked keyF17.install-remoteMapped behavior

Scenario F17--remote-entry-masked-key · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Mcp marketplace card stripe”.
Do
Enter “rk_test_qa_dummy_000” in “Mcp install modal” / “Mcp install field api key”.

Expect

a remote (HTTP) entry shows its fixed URL read-only and a type="password" credential field that is required.

Exact CLI recipe & bug checks

Run control-openhands browser click 'testid=mcp-marketplace-card-stripe'; control-openhands browser value 'testid=mcp-install-modal >> testid=mcp-install-field-url' is https://mcp.stripe.com/ and browser enabled on it is false. Click testid=mcp-install-modal >> testid=mcp-install-submit with the key empty: control-openhands browser eval "document.querySelector('[data-testid=mcp-install-field-api_key]').validationMessage" is Please fill out this field. and no request is sent. Run control-openhands browser fill 'testid=mcp-install-modal >> testid=mcp-install-field-api_key' rk_test_qa_dummy_000; control-openhands browser attr 'testid=mcp-install-modal >> testid=mcp-install-field-api_key' type is password and control-openhands browser screenshot 'testid=mcp-install-modal' --feature F17.install-remote --name stripe-masked shows dots. Cancel; never click Install with a dummy key on a hosted server.

OAuth entryF17.install-oauthKnown issue

Scenario F17--oauth-entry · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Mcp marketplace card granola”.

Expect

an entry whose server runs OAuth itself shows the OAuth info panel instead of a key field.

Linked known issues

Known issue #15430: An OAuth MCP callback in Docker can redirect to an unexposed dynamic localhost port. Completing the callback needs a Docker stack and an approved OAuth MCP provider/account (the report uses GitLab); a local unauthenticated Test connection cannot reach the redirect. (F17.install-oauth).

Known issue OpenHands/software-agent-sdk#5375: A successful GitLab OAuth login can be followed by an invalid localhost redirect URI on the first ticket-to-code-change MCP action. Reproduction needs the authorized self-hosted GitLab MCP endpoint/account and the ticket-to-code-change plugin/workflow; the reported endpoint is not a public fixture. (F17.install-oauth, F17.agent-uses-server).

Exact CLI recipe & bug checks

Run control-openhands browser click 'testid=mcp-marketplace-card-granola' and control-openhands browser text 'testid=mcp-install-modal >> testid=mcp-install-oauth-info': This server uses OAuth for authentication. Click Install to connect — you will be redirected to authorize access. and No API key needed. The server handles the OAuth flow automatically. The URL field reads https://mcp.granola.ai/mcp and there is no mcp-install-field-api_key. Cancel; Install would start a real OAuth flow.

Entry that cannot be installedF17.install-unsupportedMapped behavior

Scenario F17--entry-that-cannot-be-installed · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Mcp marketplace card sentry”.
Do
Choose “Mcp install modal” / “Mcp install submit”.

Expect

entries the local backend cannot install (provider-OAuth only, 12 today such as Sentry) should be hidden or explain themselves; today their modal is empty and Install does nothing (fail).

Exact CLI recipe & bug checks

Run control-openhands browser click 'testid=mcp-marketplace-card-sentry', control-openhands browser network --clear, control-openhands browser click 'testid=mcp-install-modal >> testid=mcp-install-submit', then control-openhands browser network --last 5, control-openhands browser count 'testid=mcp-install-modal' and control-openhands browser toasts. Expected: such entries are hidden from the library, or the modal explains that the local backend cannot install them. Today the modal shows only the title, description and docs link, Install sends no request, shows nothing and the modal stays open (fail; cloudflare-bindings behaves the same). Cancel.

Install a STDIO entryF17.install-stdioMapped behavior

Scenario F17--install-a-stdio-entry · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Mcp marketplace card time”.
Do
Choose “Mcp install modal” / “Mcp install submit”.
Do
Reload the page.
Do
Reload the page.
Do
Enter “Timezone” in “Mcp search”.
Do
Enter “qa-zzz-nothing” in “Mcp search”.
Do
Choose “Mcp search clear”.

Expect

a STDIO entry shows its read-only command; Install shows Verifying…, Saving..., toasts MCP server saved. and adds a card with the catalog name and description.

Exact CLI recipe & bug checks

With the TLS precondition, run control-openhands browser click 'testid=mcp-marketplace-card-time' and control-openhands browser click 'testid=mcp-install-modal >> testid=mcp-install-submit' --observe 'testid=mcp-install-modal >> testid=mcp-install-submit' --observe-ms 20000: Install, Verifying…, Saving..., then <absent> (closed). After control-openhands browser reload, control-openhands browser text 'testid=mcp-server-item >> has-text=Timezone-aware' is Time, STDIO, Timezone-aware current time, conversions, and timestamp formatting., uvx mcp-server-time, Not checked yet, Test connection; its data-server-id is time. Installing it again (same two commands, then browser reload) adds a second card (time_1): the library is add-only and its + toggle never shows installed (aria-checked stays false). The copy keeps the Time catalog identity (#18034): control-openhands browser text '[data-server-id=time_1]' is time_1, STDIO, Timezone-aware current time, conversions, and timestamp formatting., uvx mcp-server-time, Not checked yet, Test connection (the title is the settings key because it differs from the catalog id), and control-openhands browser eval "[...document.querySelectorAll('[data-testid=mcp-server-item]')].map(c=>c.dataset.serverId+':'+c.querySelector('span[aria-hidden=true][title]')?.title)" is ["time:Time","time_1:Time"] (both badges come from the Time entry). Time has no logo, so both installed cards show the generic puzzle glyph where its library card shows a robot glyph. Search the installed list: control-openhands browser fill 'testid=mcp-search-input' Timezone keeps both cards (browser eval "[...document.querySelectorAll('[data-testid=mcp-server-item]')].map(c=>c.dataset.serverId)" is ["time","time_1"], matched by the catalog description) and one library card; control-openhands browser fill 'testid=mcp-search-input' qa-zzz-nothing makes control-openhands browser text 'testid=mcp-installed-empty-search' No matches for your search.; clear with control-openhands browser click 'testid=mcp-search-clear'. control-openhands browser screenshot --feature F17.install-stdio --name time-and-time_1 shows both cards with the description.

Argument fieldsF17.install-argsMapped behavior

Scenario F17--argument-fields · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Mcp marketplace card filesystem”.
Do
Enter “$OH_VERIFY_RUN/workspace/qa-fs-a $OH_VERIFY_RUN/workspace/qa-fs-b” in “Mcp install modal” / “Mcp install field paths”.
Do
Choose “Mcp install modal” / “Mcp install submit”.
Do
Reload the page.

Expect

an entry with argument fields (Filesystem's Paths (space separated)) requires them and appends each whitespace-separated token to the stored command's args.

Exact CLI recipe & bug checks

Run control-openhands browser click 'testid=mcp-marketplace-card-filesystem'; control-openhands browser snapshot 'testid=mcp-install-modal' shows the read-only Command npx -y @modelcontextprotocol/server-filesystem, a Paths (space separated) field and Each whitespace-separated token is appended as its own argument.; control-openhands browser attr 'testid=mcp-install-modal >> role=link' target is _blank (View documentation →, rel noreferrer). Click testid=mcp-install-submit with the field empty: control-openhands browser eval "document.querySelector('[data-testid=mcp-install-field-paths]').validationMessage" is Please fill out this field. and browser network records no request. Shell arrange mkdir -p "$OH_VERIFY_RUN/workspace/qa-fs-a" "$OH_VERIFY_RUN/workspace/qa-fs-b", then control-openhands browser fill 'testid=mcp-install-modal >> testid=mcp-install-field-paths' "$OH_VERIFY_RUN/workspace/qa-fs-a $OH_VERIFY_RUN/workspace/qa-fs-b" and control-openhands browser click 'testid=mcp-install-modal >> testid=mcp-install-submit' --observe 'testid=mcp-install-modal >> testid=mcp-install-submit' --observe-ms 90000 (Install, Verifying…, Saving..., <absent>). After browser reload, control-openhands browser text '[data-server-id=filesystem]' ends the command line with both paths, and control-openhands api GET /api/settings --pick agent_settings.mcp_config.filesystem has args ["-y","@modelcontextprotocol/server-filesystem","<…>/qa-fs-a","<…>/qa-fs-b"].

Secret selection before installF17.save-as-secretMapped behavior

Scenario F17--secret-selection-before-install · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Open /settings/app.
Do
Choose “Color theme”.
Do
Choose “<theme>”.
Setup
Use the <viewport> viewport.
Do
Open /mcp.
Do
Choose “Mcp marketplace card github”.
Do
Use browser record as specified in the exact recipe.
Do
Use browser focus as specified in the exact recipe.
Do
Press Tab.
Do
Press Space.
Do
Use browser focus as specified in the exact recipe.
Do
Press Enter.
Do
Choose “Mcp install save secret github personal access token”.
Do
Use browser record as specified in the exact recipe.
Do
Choose “Mcp install cancel”.

Expect

password credential fields carry Also save as secret (checked by default, with a tooltip); label clicks and Space change the native checkbox, while its help button leaves the choice unchanged. The selection follows the colour theme and long secret names fit on phones. After install the value is listed in Settings → Secrets.

Exact CLI recipe & bug checks

No provider key or install is needed. For each theme OpenHands-Neutral and Light+, run control-openhands browser goto /settings/app, control-openhands browser click 'testid=color-theme-input', and control-openhands browser click 'role=option[name="<theme>"]'. For each viewport desktop and phone, run control-openhands browser viewport <viewport>, control-openhands browser goto /mcp, and control-openhands browser click 'testid=mcp-marketplace-card-github'. control-openhands browser bbox 'testid=mcp-install-save-secret-GITHUB_PERSONAL_ACCESS_TOKEN' shows insideViewport: true and pageHorizontalOverflow: false; the full secret name and help fit inside the row. Start control-openhands browser record start --feature F17.save-as-secret --name <theme>-<viewport>-selection. Run control-openhands browser focus 'testid=mcp-install-modal >> testid=mcp-install-field-api_key' and control-openhands browser press Tab twice (the first Tab visits GitHub settings, the second the native checkbox), take control-openhands browser screenshot --feature F17.save-as-secret --name <theme>-<viewport>-focus, then control-openhands browser press Space. control-openhands browser eval "document.querySelector('[data-testid=mcp-install-save-secret-GITHUB_PERSONAL_ACCESS_TOKEN] input').checked" is false, and the focus outline, label and secret name remain readable. Run control-openhands browser focus 'testid=mcp-install-save-secret-GITHUB_PERSONAL_ACCESS_TOKEN >> role=button', control-openhands browser text 'role=tooltip', and control-openhands browser press Enter; the tooltip explains sharing with automations and the same .checked read stays false. Run control-openhands browser click 'testid=mcp-install-save-secret-GITHUB_PERSONAL_ACCESS_TOKEN'; .checked becomes true. Stop with control-openhands browser record stop, then close with control-openhands browser click 'testid=mcp-install-cancel' before the next viewport/theme. Restore desktop and OpenHands-Neutral through the same theme picker before continuing. This is selection proof only; the next bullet proves install persistence.

Save as secretF17.save-as-secretMapped behavior

Scenario F17--save-as-secret · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Mcp marketplace card tavily”.
Do
Use browser tooltip as specified in the exact recipe.
Do
Enter “tvly-qa-dummy-000” in “Mcp install modal” / “Mcp install field tavily api key”.
Do
Choose “Mcp install modal” / “Mcp install submit”.
Do
Open /settings/secrets.

Expect

Exact CLI recipe & bug checks

With the TLS precondition and no TAVILY_API_KEY secret (control-openhands api GET /api/settings/secrets), run control-openhands browser click 'testid=mcp-marketplace-card-tavily'. control-openhands browser attr 'testid=mcp-install-modal >> testid=mcp-install-field-TAVILY_API_KEY' type is password, control-openhands browser eval "document.querySelector('[data-testid=mcp-install-save-secret-TAVILY_API_KEY] input').checked" is true, and control-openhands browser tooltip 'testid=mcp-install-modal >> role=button[name=/secret/i]' is MCP credentials aren't shared with automations. Save as a secret to make this value available to automations. Run control-openhands browser fill 'testid=mcp-install-modal >> testid=mcp-install-field-TAVILY_API_KEY' tvly-qa-dummy-000 and control-openhands browser click 'testid=mcp-install-modal >> testid=mcp-install-submit' --observe 'role=status' --observe-ms 60000; the observed toasts are MCP server saved. and Saved to secrets: TAVILY_API_KEY. Then control-openhands browser goto /settings/secrets and control-openhands browser count 'testid=secret-item >> has-text=TAVILY_API_KEY' is 1. The GitHub card's hosted token has the same toggle (mcp-install-save-secret-GITHUB_PERSONAL_ACCESS_TOKEN, checked). Clean up by deleting the tavily server (Delete below) and the secret through F14's Delete recipe.

Editor validationF17.custom-validationMapped behavior

Scenario F17--editor-validation · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Open /mcp.
Do
Choose “Mcp add custom server”.
Do
Enter “not a url” in “Add mcp server form” / “Url”.
Do
Choose “Add mcp server form” / “Auth mode dropdown”.
Do
Choose “Header”.
Do
Choose “Add mcp server form” / “Server type dropdown”.
Do
Choose “SHTTP”.
Do
Press Escape.
Do
Enter “NOEQUALS” in “Add mcp server form” / “Headers”.
Do
Press Escape.
Setup
Use the phone viewport.
Do
Choose “Mcp add custom server”.
Setup
Use the desktop viewport.

Expect

the custom editor refuses bad input with an inline message (URL required/invalid/wrong protocol, name invalid or duplicate, command required or with spaces, env/header not KEY=value, header required, OAuth secret without client ID, timeout out of range). Malformed header lines use a header-specific message, distinct from environment-variable errors.

Exact CLI recipe & bug checks

Run control-openhands browser goto /mcp (Save as secret ends on /settings/secrets), control-openhands browser click 'testid=mcp-add-custom-server'. With the default SSE type, click testid=add-mcp-server-form >> testid=submit-button after each change and read control-openhands browser text 'testid=add-mcp-server-form >> css=p.text-feedback-error >> nth=0': empty URL → URL is required; control-openhands browser fill 'testid=add-mcp-server-form >> testid=url-input' 'not a url' → Invalid URL format; ftp://127.0.0.1/mcp → URL must use http:// or https://; URL http://127.0.0.1:9/sse plus testid=server-name-input 'bad name' → Name can only contain letters, numbers, hyphens, and underscores; name qa_sse, then control-openhands browser click 'testid=add-mcp-server-form >> testid=auth-mode-dropdown' and control-openhands browser click 'role=option[name="Header"]' → Header authentication requires a header; auth OAuth with testid=oauth-client-secret-input filled dummy-secret → OAuth client secret requires a client ID. Switch type with control-openhands browser click 'testid=add-mcp-server-form >> testid=server-type-dropdown' and control-openhands browser click 'role=option[name="SHTTP"]', set auth back to None; testid=timeout-input 0 → Timeout must be positive, 5000 → Timeout cannot exceed 3600 seconds (1 hour). Switch to STDIO: empty → Name is required; testid=name-input 'bad name!' → the invalid-name message; name qa_time → Command is required; testid=command-input 'uvx mcp-server-time' → Command cannot contain spaces; command uvx plus testid=env-input NOEQUALS → Environment variables must follow KEY=value format. Once a STDIO qa_time exists (or any installed STDIO id such as time), the same name gives A STDIO server with this name already exists. control-openhands browser press Escape closes the editor. Header format: reopen the editor, pick SHTTP, fill url-input http://127.0.0.1:9/mcp, auth Header, control-openhands browser fill 'testid=add-mcp-server-form >> testid=headers-input' NOEQUALS and submit. control-openhands browser text 'testid=add-mcp-server-form >> css=p.text-feedback-error >> nth=0' is Headers must follow KEY=value format. Take control-openhands browser screenshot --feature F17.custom-validation --name header-format, Close the editor with control-openhands browser press Escape, run control-openhands browser viewport phone, reopen with control-openhands browser click 'testid=mcp-add-custom-server' and repeat the SHTTP header-format commands. Restore control-openhands browser viewport desktop, leaving that SHTTP editor open for Test connection.

Test connectionF17.test-connectionMapped behavior

Scenario F17--test-connection · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Add mcp server form” / “Auth mode dropdown”.
Do
Choose “None”.
Do
Enter “10” in “Add mcp server form” / “Timeout”.
Wait
Wait for “Add mcp server form” / “Mcp test message” to appear.
Do
Choose “Add mcp server form” / “Server type dropdown”.
Do
Choose “STDIO”.
Do
Enter “mcp-server-time” in “Add mcp server form” / “Args”.
Do
Enter “UV_NATIVE_TLS=true” in “Add mcp server form” / “Env”.
Do
Choose “Add mcp server form” / “Mcp test connection”.

Expect

Test connection in the editor shows Verifying…, then a green Connected — N tool(s) available or a red categorized error; Add/Save tests first and stays open on failure.

Exact CLI recipe & bug checks

Failure: in the SHTTP form left open by validation, run control-openhands browser click 'testid=add-mcp-server-form >> testid=auth-mode-dropdown', control-openhands browser click 'role=option[name="None"]' and control-openhands browser fill 'testid=add-mcp-server-form >> testid=timeout-input' 10. With URL http://127.0.0.1:9/mcp, click testid=add-mcp-server-form >> testid=mcp-test-connection, control-openhands browser wait 'testid=add-mcp-server-form >> testid=mcp-test-message' --timeout 60000; the text is Could not reach the server (check the URL and server type): Client failed to connect: All connection attempts failed (the remote wording followed by the Agent Server's detail) and its class contains text-feedback-error. Clicking Add Server leaves the editor open (browser count 'testid=mcp-custom-editor' is 1). Success: run control-openhands browser click 'testid=add-mcp-server-form >> testid=server-type-dropdown' and control-openhands browser click 'role=option[name="STDIO"]', then fill testid=name-input qa_time, testid=command-input uvx, control-openhands browser fill 'testid=add-mcp-server-form >> testid=args-input' mcp-server-time and control-openhands browser fill 'testid=add-mcp-server-form >> testid=env-input' UV_NATIVE_TLS=true, then control-openhands browser click 'testid=add-mcp-server-form >> testid=mcp-test-connection' --observe 'testid=add-mcp-server-form >> testid=mcp-test-connection' --observe-ms 20000 (Test connection, Verifying…, Test connection), the same wait and control-openhands browser text 'testid=add-mcp-server-form >> testid=mcp-test-message': Connected — 2 tool(s) available, class text-feedback-success. control-openhands browser screenshot 'testid=mcp-custom-editor' --feature F17.test-connection --name stdio-success.

Add the serverF17.custom-addMapped behavior

Scenario F17--add-the-server · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Add mcp server form” / “Submit”.
Wait
Wait for “Mcp custom editor” to disappear.
Do
Reload the page.

Expect

Add Server saves and closes; the card is listed after a reload.

Exact CLI recipe & bug checks

In the successful STDIO form run control-openhands browser click 'testid=add-mcp-server-form >> testid=submit-button' and control-openhands browser wait 'testid=mcp-custom-editor' --state detached --timeout 60000, then control-openhands browser reload and control-openhands browser text 'testid=mcp-server-item >> has-text=qa_time': qa_time, STDIO, Timezone-aware current time, conversions, and timestamp formatting., uvx mcp-server-time, Not checked yet, Test connection (it runs the Time entry's command and args, so it shows that entry's description and badge under its own name, #18034). control-openhands browser attr 'testid=mcp-server-item >> has-text=qa_time' data-server-id is qa_time (the id in the per-card test ids below) and its aria-label is Edit qa_time.

Custom remote serverF17.custom-add-remoteMapped behavior

Scenario F17--custom-remote-server · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Mcp add custom server”.
Do
Reload the page.
Do
Choose “Mcp server detail qa shttp”.
Do
Start a conversation with the prompt “Call the qa_shttp MCP server's echo tool with message qa-ping. Do not use the terminal. Reply with only the tool's output.”.
Do
Open /mcp.

Expect

a custom Streamable HTTP (SHTTP) server with a Bearer token connects, saves as an HTTP card showing its URL, reopens with the token masked, and its tools reach the agent as <server>_<tool>.

Exact CLI recipe & bug checks

With the Streamable HTTP fixture from Preconditions running, run control-openhands browser click 'testid=mcp-add-custom-server', pick SHTTP (server-type-dropdown, then role=option[name="SHTTP"]), fill testid=add-mcp-server-form >> testid=server-name-input qa_shttp and url-input http://127.0.0.1:38417/mcp, pick auth Bearer token (auth-mode-dropdown, then role=option[name="Bearer token"]) and fill testid=add-mcp-server-form >> testid=api-key-input qa-dummy-token (browser attr ... type is password). Click testid=add-mcp-server-form >> testid=mcp-test-connection; control-openhands browser text 'testid=add-mcp-server-form >> testid=mcp-test-message' --timeout 60000 is Connected — 13 tool(s) available. Click testid=add-mcp-server-form >> testid=submit-button, wait for testid=mcp-custom-editor to detach and browser reload: control-openhands browser text '[data-server-id=qa_shttp]' is qa_shttp, HTTP, http://127.0.0.1:38417/mcp, Not checked yet, Test connection; control-openhands api GET /api/settings --pick agent_settings.mcp_config.qa_shttp has transport http and auth {"strategy":"bearer","value":"**********"}. Its health probe reads Reachable — credentials not verified. Reopen it with control-openhands browser click 'testid=mcp-server-detail-qa_shttp': browser snapshot 'testid=mcp-custom-editor' shows Edit MCP server, combobox Authentication Bearer token, and control-openhands browser value 'testid=edit-mcp-server-form >> testid=api-key-input' is **********; Escape. Agent side: control-openhands conversation start --prompt "Call the qa_shttp MCP server's echo tool with message qa-ping. Do not use the terminal. Reply with only the tool's output." --wait --timeout 240, then control-openhands conversation events <id> --kinds ActionEvent,ObservationEvent,MessageEvent: tool qa_shttp_echo, observation Echo: qa-ping. Run control-openhands browser goto /mcp afterwards.

Card healthF17.server-healthMapped behavior

Scenario F17--card-health · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Mcp health probe qa time”.
Setup
Arrange the prerequisite backend state through the API.
Do
Reload the page.

Expect

each card's health row runs Test connection (Checking connection…), then shows a verdict with a coloured dot; a failure is red, worded for the transport (a STDIO command that cannot start, a remote server that cannot be reached) with the Agent Server's detail, and offers Retry.

Exact CLI recipe & bug checks

Run control-openhands browser click 'testid=mcp-health-probe-qa_time' --observe 'testid=mcp-health-label-qa_time' --observe-ms 20000: Not checked yet, Checking connection…, Reachable — credentials not verified. control-openhands browser text 'testid=mcp-server-health-qa_time' adds the hint This check only proves the server is reachable and lists its tools; it does not verify your credentials., control-openhands browser attr 'testid=mcp-server-health-qa_time >> testid=mcp-health-dot' data-status is healthy-connectivity, and the click did not open the editor (browser count 'testid=mcp-custom-editor' is 0). Failure: the editor refuses to save a broken server, so arrange one with control-openhands api POST /api/settings/mcp/qa_broken --data '{"transport":"stdio","command":"qa-no-such-command","enabled":false}' --write, control-openhands browser reload, then click testid=mcp-health-probe-qa_broken: the label is Could not start the server command: Client failed to connect: [Errno 2] No such file or directory: 'qa-no-such-command' in red (STDIO wording, no URL hint; the card clamps it to two lines, browser text 'testid=mcp-server-health-qa_broken' reads it whole), the dot's data-status is failed and the button reads Retry; control-openhands browser screenshot 'testid=mcp-server-item >> has-text=qa_broken' --feature F17.server-health --name failed.

DeleteF17.delete-serverMapped behavior

Scenario F17--delete · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Mcp server detail qa broken”.
Do
Choose “Mcp custom editor delete”.
Do
Choose “Confirmation dialog” / “Cancel”.
Do
Choose “Confirmation dialog” / “Confirm”.
Do
Reload the page.

Expect

Delete in the editor asks for confirmation; Cancel keeps the server, Confirm removes it with MCP server removed.

Exact CLI recipe & bug checks

Run control-openhands browser click 'testid=mcp-server-detail-qa_broken' (a click on the card's centre lands on its health row and opens nothing: see Gotchas), control-openhands browser click 'testid=mcp-custom-editor-delete' and control-openhands browser text 'testid=confirmation-modal' (Are you sure you want to delete this server?, Cancel, Confirm). Run control-openhands browser click 'testid=confirmation-modal >> testid=cancel-button'; the editor is still open. Click Delete again, then control-openhands browser click 'testid=confirmation-modal >> testid=confirm-button' and control-openhands browser toasts (MCP server removed.). After control-openhands browser reload, control-openhands browser count 'testid=mcp-server-item >> has-text=qa_broken' is 0, and control-openhands api GET /api/settings has no qa_broken.

EditF17.custom-editMapped behavior

Scenario F17--edit · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Mcp server detail qa time”.
Do
Use browser focus as specified in the exact recipe.
Do
Press Enter.
Do
Enter “$mcp-server-time\n--local-timezone\nUTC” in “Edit mcp server form” / “Args”.
Do
Choose “Edit mcp server form” / “Submit”.
Do
Reload the page.

Expect

clicking a card opens Edit MCP server prefilled (secrets shown as **********); Save tests and persists the change.

Exact CLI recipe & bug checks

Run control-openhands browser click 'testid=mcp-server-detail-qa_time' (the detail line opens the editor on every card; a plain click on the card's centre happens to open it on qa_time, whose Time description fills the centre, but not on a card without a description; keyboard: browser focus '[data-server-id=qa_time]' then browser press Enter) and control-openhands browser snapshot 'testid=mcp-custom-editor': heading Edit MCP server, no Server Type field, a Delete button; control-openhands browser value 'testid=edit-mcp-server-form >> testid=env-input' is UV_NATIVE_TLS=**********. Run control-openhands browser fill 'testid=edit-mcp-server-form >> testid=args-input' $'mcp-server-time\n--local-timezone\nUTC' (bash ANSI-C quoting for the newlines), click testid=edit-mcp-server-form >> testid=mcp-test-connection (still Connected — 2 tool(s) available: the stored env value is reused), then control-openhands browser click 'testid=edit-mcp-server-form >> testid=submit-button' and wait for testid=mcp-custom-editor to detach. The card shows a verdict at once (seeded from the save's test). After control-openhands browser reload, control-openhands browser text 'testid=mcp-server-detail-qa_time' is uvx mcp-server-time --local-timezone UTC.

Stored secrets are reused, not their placeholderF17.stored-secret-reuseMapped behavior

Scenario F17--stored-secrets-are-reused-not-their-placeholder · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Setup
Create the disposable mcp server fixture specified in the recipe.
Do
Choose “Mcp add custom server”.
Do
Enter “$-c\ntest "$QA_VAULT_TOKEN" = qa-vault-secret-000 && exec "$QA_NODE" "$QA_SERVER"” in “Add mcp server form” / “Args”.
Do
Enter “$QA_VAULT_TOKEN=qa-vault-secret-000\nQA_NODE=<node>\nQA_SERVER=<server.mjs>” in “Add mcp server form” / “Env”.
Wait
Wait for “Mcp custom editor” to disappear.
Do
Reload the page.
Do
Choose “Mcp server detail qa vault”.
Do
Enter “$QA_VAULT_TOKEN=qa-wrong\nQA_NODE=**********\nQA_SERVER=**********” in “Edit mcp server form” / “Env”.
Wait
Wait for “Edit mcp server form” / “Mcp test message” / “Connection” to appear.
Do
Choose “Mcp custom editor close”.
Do
Press Escape.
Do
Choose “Mcp server detail qa vault”.
Do
Enter “$-c\ntest "$QA_VAULT_TOKEN" = qa-vault-secret-000 && exec "$QA_NODE" "$QA_SERVER"\nqa-v2” in “Edit mcp server form” / “Args”.
Do
Choose “Edit mcp server form” / “Submit”.
Wait
Wait for “Mcp custom editor” to disappear.
Do
Reload the page.
Do
Choose “Mcp server detail qa vault”.
Do
Press Escape.

Expect

a saved server's env values and tokens come back from the settings API and in the editor as **********; the editor's Test connection swaps each placeholder for the stored encrypted value (never the placeholder text, never the plaintext) and Save leaves untouched secrets out of its sparse patch, so a Save from the edit form that keeps them changes only the edited fields.

Exact CLI recipe & bug checks

Make a STDIO server that only starts with the right secret: control-openhands fixture mcp-server --name qa-vault prints <node> and <server.mjs>. Run control-openhands browser click 'testid=mcp-add-custom-server', pick STDIO, fill testid=add-mcp-server-form >> testid=name-input qa_vault and testid=add-mcp-server-form >> testid=command-input /bin/sh, then control-openhands browser fill 'testid=add-mcp-server-form >> testid=args-input' $'-c\ntest "$QA_VAULT_TOKEN" = qa-vault-secret-000 && exec "$QA_NODE" "$QA_SERVER"' and control-openhands browser fill 'testid=add-mcp-server-form >> testid=env-input' $'QA_VAULT_TOKEN=qa-vault-secret-000\nQA_NODE=<node>\nQA_SERVER=<server.mjs>'. Run control-openhands browser network --clear, click testid=add-mcp-server-form >> testid=mcp-test-connection; control-openhands browser text 'testid=add-mcp-server-form >> testid=mcp-test-message' --timeout 60000 is Connected — 1 tool(s) available. Direct proof of what the page sent: control-openhands browser network --bodies --filter 'api/mcp/test' --last 1 lists one POST /api/mcp/test (200) whose body is {"server":{"type":"stdio","command":"/bin/sh","args":["-c","test \"$QA_VAULT_TOKEN\" = qa-vault-secret-000 && exec \"$QA_NODE\" \"$QA_SERVER\""],"env":{"QA_VAULT_TOKEN":"<redacted 19 chars>","QA_NODE":"<redacted 40 chars>","QA_SERVER":"<redacted 142 chars>"}},"name":"qa_vault"}: the command and args in clear, each env value replaced by the CLI with the length of the real value (19 for qa-vault-secret-000; 40 and 142 are <node> and <server.mjs> on this machine, ${#var} in a shell). Click testid=add-mcp-server-form >> testid=submit-button, control-openhands browser wait 'testid=mcp-custom-editor' --state detached --timeout 60000 and control-openhands browser reload. control-openhands api GET /api/settings --pick agent_settings.mcp_config.qa_vault has env {"QA_VAULT_TOKEN":"**********","QA_NODE":"**********","QA_SERVER":"**********"} (every env value is redacted; the page reads this same endpoint, there is no GET /api/settings/mcp). Run control-openhands browser click 'testid=mcp-server-detail-qa_vault'; control-openhands browser value 'testid=edit-mcp-server-form >> testid=env-input' is the three lines QA_VAULT_TOKEN=**********, QA_NODE=**********, QA_SERVER=**********. Run control-openhands browser network --clear and click testid=edit-mcp-server-form >> testid=mcp-test-connection: control-openhands browser text 'testid=edit-mcp-server-form >> testid=mcp-test-message' --timeout 60000 is Connected — 1 tool(s) available, which the shell gate only allows when the stored token, binary and path were sent (control-openhands browser screenshot 'testid=mcp-custom-editor' --feature F17.stored-secret-reuse --name masked-test-ok). The same control-openhands browser network --bodies --filter 'api/mcp/test' --last 1 row now carries "env":{"QA_VAULT_TOKEN":"<redacted 120 chars>","QA_NODE":"<redacted 140 chars>","QA_SERVER":"<redacted 268 chars>"}: neither the ********** placeholder (which --bodies would keep as is) nor the typed lengths. control-openhands browser network --last 5 shows why: a GET /api/settings (and /server_info) right before the POST. The page fetches the settings in encrypted mode and swaps each placeholder for the stored encrypted value (src/api/mcp-service/mcp-redacted-credentials.ts), which the Agent Server decrypts, so the browser never holds the plaintext and the lengths are those of Fernet tokens of the real values (19 → 120, 40 → 140, 142 → 268). Counter-check that the gate bites: control-openhands browser fill 'testid=edit-mcp-server-form >> testid=env-input' $'QA_VAULT_TOKEN=qa-wrong\nQA_NODE=**********\nQA_SERVER=**********', control-openhands browser network --clear, click Test connection again and control-openhands browser wait 'testid=edit-mcp-server-form >> testid=mcp-test-message >> text=Connection' --timeout 90000; the text is red and reads Connection failed: McpError: Connection closed or, when the client only notices at its deadline, Connection timed out. Check the URL and try again. (the gate exits before the MCP handshake; see Gotchas; --name wrong-token), and the --bodies row shows "QA_VAULT_TOKEN":"<redacted 8 chars>" next to the two kept placeholders at 140 and 268 chars: only the retyped value replaced its stored one. control-openhands browser click 'testid=mcp-custom-editor-close' discards it (count 'testid=mcp-custom-editor' 0); reopen with testid=mcp-server-detail-qa_vault: the env is the three placeholders again and Test connection is Connected — 1 tool(s) available. control-openhands browser press Escape. Save with the placeholders in place: reopen with control-openhands browser click 'testid=mcp-server-detail-qa_vault', append one harmless argument (it only becomes the shell script's $0) with control-openhands browser fill 'testid=edit-mcp-server-form >> testid=args-input' $'-c\ntest "$QA_VAULT_TOKEN" = qa-vault-secret-000 && exec "$QA_NODE" "$QA_SERVER"\nqa-v2', leave env-input as the three placeholders, run control-openhands browser network --clear, then control-openhands browser click 'testid=edit-mcp-server-form >> testid=submit-button' --observe 'testid=edit-mcp-server-form >> testid=submit-button' --observe-ms 6000 (Save Server, Save Server [disabled], <absent>) and control-openhands browser wait 'testid=mcp-custom-editor' --state detached --timeout 60000; control-openhands browser network --bodies --filter 'api/settings/mcp' --last 3 lists one PATCH /api/settings/mcp/qa_vault (200) whose body is {"transport":"stdio","command":"/bin/sh","args":["-c","test \"$QA_VAULT_TOKEN\" = qa-vault-secret-000 && exec \"$QA_NODE\" \"$QA_SERVER\"","qa-v2"]}: a sparse patch with no env key at all (untouched secrets are left out, so the server keeps its stored values; neither the placeholder nor an encrypted value travels). The same control-openhands browser network --filter 'api/settings/mcp' --last 3 without --bodies lists that row without a body field. The card's health label reads Reachable — credentials not verified. After control-openhands browser reload, control-openhands api GET /api/settings --pick agent_settings.mcp_config.qa_vault.env is still {"QA_VAULT_TOKEN":"**********","QA_NODE":"**********","QA_SERVER":"**********"}, control-openhands api GET /api/settings --pick agent_settings.mcp_config.qa_vault.args ends with qa-v2, and control-openhands browser text 'testid=mcp-server-detail-qa_vault' ends with "$QA_SERVER" qa-v2. Reopen with control-openhands browser click 'testid=mcp-server-detail-qa_vault' (the env is the three placeholders), run control-openhands browser network --clear and click testid=edit-mcp-server-form >> testid=mcp-test-connection: control-openhands browser text 'testid=edit-mcp-server-form >> testid=mcp-test-message' --timeout 60000 is Connected — 1 tool(s) available again, which the gate only allows if the Save kept the real token, binary and path rather than storing the placeholder text (--name saved-with-placeholders), and the --bodies row carries qa-v2 in args with the three encrypted lengths (120, 140, 268) in env. control-openhands browser press Escape.

One request per mutationF17.single-request-mutationsMapped behavior

Scenario F17--one-request-per-mutation · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Mcp add custom server”.
Wait
Wait for “Mcp custom editor” to disappear.
Do
Choose “Mcp server detail qa docs”.
Do
Enter “$<server.mjs>\n--qa-v2” in “Edit mcp server form” / “Args”.
Do
Choose “Mcp server detail qa docs”.
Do
Choose “Mcp custom editor delete”.
Do
Choose “Confirmation dialog” / “Confirm”.
Wait
Wait for the element identified in the CLI recipe to disappear.
Do
Choose “Mcp server detail qa vault”.
Do
Press Escape.

Expect

adding, editing or deleting one server sends exactly one POST, PATCH or DELETE to /api/settings/mcp/<name> and leaves sibling servers and their stored credentials untouched.

Exact CLI recipe & bug checks

With qa_vault as the sibling, run control-openhands browser network --clear, control-openhands browser click 'testid=mcp-add-custom-server', pick STDIO, fill testid=add-mcp-server-form >> testid=name-input qa_docs, testid=add-mcp-server-form >> testid=command-input <node> and testid=add-mcp-server-form >> testid=args-input <server.mjs>, click testid=add-mcp-server-form >> testid=submit-button and control-openhands browser wait 'testid=mcp-custom-editor' --state detached --timeout 60000. control-openhands browser network --filter '/api/settings/mcp' has total 1, a POST /api/settings/mcp/qa_docs with status 201, and control-openhands browser network --filter qa_vault has total 0. Edit: control-openhands browser network --clear, control-openhands browser click 'testid=mcp-server-detail-qa_docs', control-openhands browser fill 'testid=edit-mcp-server-form >> testid=args-input' $'<server.mjs>\n--qa-v2', click testid=edit-mcp-server-form >> testid=submit-button and the same wait; the filter now lists one PATCH /api/settings/mcp/qa_docs (200) and control-openhands api GET /api/settings --pick agent_settings.mcp_config.qa_docs.args ends with --qa-v2. Delete: control-openhands browser network --clear, control-openhands browser click 'testid=mcp-server-detail-qa_docs', control-openhands browser click 'testid=mcp-custom-editor-delete', control-openhands browser click 'testid=confirmation-modal >> testid=confirm-button' and control-openhands browser wait '[data-server-id=qa_docs]' --state detached --timeout 10000; the filter lists one DELETE /api/settings/mcp/qa_docs (200). Sibling: control-openhands api GET /api/settings --pick agent_settings.mcp_config.qa_vault is unchanged from the end of the previous bullet (command /bin/sh, args still ending with qa-v2, the three ********** env values, enabled true), and after control-openhands browser click 'testid=mcp-server-detail-qa_vault' its Test connection still reads Connected — 1 tool(s) available: the stored secrets survived the sibling's writes. control-openhands browser press Escape.

Credential probeF17.credential-probeKnown failure

Scenario F17--credential-probe · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Mcp marketplace card github”.
Do
Enter “ghp_qaInvalid000” in “Mcp install modal” / “Mcp install field api key”.
Do
Choose “Mcp install modal” / “Mcp install submit”.
Do
Choose “Mcp install cancel”.
Do
Choose “Mcp marketplace card slack”.
Do
Enter “T_QA_INVALID” in “Mcp install modal” / “Mcp install field slack team id”.
Do
Enter “xoxb-qa-invalid-000” in “Mcp install modal” / “Mcp install field slack bot token”.
Do
Choose “Mcp install modal” / “Mcp install submit”.
Wait
Wait for “Mcp install modal error” to appear.
Do
Choose “Mcp install cancel”.
Do
Choose “Mcp add custom server”.
Do
Enter “$-y\n@zencoderai/slack-mcp-server” in “Add mcp server form” / “Args”.
Do
Enter “$SLACK_TEAM_ID=T_QA_INVALID\nSLACK_BOT_TOKEN=xoxb-qa-invalid-000\nNODE_EXTRA_CA_CERTS=/root/.ccr/ca-bundle.crt” in “Add mcp server form” / “Env”.
Do
Choose “Add mcp server form” / “Mcp test connection”.
Do
Choose “Add mcp server form” / “Submit”.
Do
Choose “Mcp custom editor close”.
Do
Choose “Mcp health probe qa vault”.

Expect

a catalog entry whose server lists its tools with any credentials (Slack) also gets a read-only credential probe during Install and Test connection: a failed probe reads Credential check failed: <provider error> and saves nothing (

Known bug observations

Known failure, reproduced 2026-10-08: any errored probe call, a network fetch failed included, is worded as a credential failure; #18161). A hosted server that rejects the token at connect (GitHub) never reaches a probe and reads Connection failed: … 401 Unauthorized …, also saving nothing (Known failure, reproduced 2026-10-08: the Agent Server's POST /api/mcp/test returns the 401 as error_kind unknown, so the modal cannot call it a credential failure; OpenHands/software-agent-sdk#5607); a server the catalog does not know gets no probe.

Exact CLI recipe & bug checks

GitHub (hosted): run control-openhands browser click 'testid=mcp-marketplace-card-github', control-openhands browser fill 'testid=mcp-install-modal >> testid=mcp-install-field-api_key' ghp_qaInvalid000, control-openhands browser network --clear, then control-openhands browser click 'testid=mcp-install-modal >> testid=mcp-install-submit' --observe 'testid=mcp-install-modal >> testid=mcp-install-submit' --observe-ms 20000 (Install, Verifying…, Install) and control-openhands browser text 'testid=mcp-install-modal-error'. The server refuses the token before any tool is listed, so the probe never runs and the text is Connection failed: HTTPStatusError: Client error '401 Unauthorized' for url 'https://api.githubcopilot.com/mcp/' …; the modal stays open (control-openhands browser count 'testid=mcp-install-modal' is 1), control-openhands browser network --filter 'api/(mcp|settings)' shows one POST /api/mcp/test and no /api/settings/mcp write, control-openhands api GET /api/settings --pick agent_settings.mcp_config.github has no value and control-openhands api GET /api/settings/secrets lists no GITHUB_PERSONAL_ACCESS_TOKEN (control-openhands browser screenshot 'testid=mcp-install-modal' --feature F17.credential-probe --name github-invalid); control-openhands browser click 'testid=mcp-install-cancel'. Slack (stdio; it lists 8 tools with any token, so only the probe, a channel listing, can tell a bad token): control-openhands browser click 'testid=mcp-marketplace-card-slack', control-openhands browser fill 'testid=mcp-install-modal >> testid=mcp-install-field-SLACK_TEAM_ID' T_QA_INVALID, control-openhands browser fill 'testid=mcp-install-modal >> testid=mcp-install-field-SLACK_BOT_TOKEN' xoxb-qa-invalid-000 (type password), control-openhands browser click 'testid=mcp-install-modal >> testid=mcp-install-submit' and control-openhands browser wait 'testid=mcp-install-modal-error' --timeout 150000 (the first run downloads the package through npx, about a minute: a plain click, then one long wait, rather than --observe-ms, which records a few transient states). control-openhands browser text 'testid=mcp-install-modal-error' is Credential check failed: invalid_auth with direct internet; behind the TLS-intercepting proxy the probe's own HTTPS call fails first and it reads Credential check failed: fetch failed. Either way the modal stays open and control-openhands api GET /api/settings --pick agent_settings.mcp_config.slack has no value (--name slack-invalid); control-openhands browser click 'testid=mcp-install-cancel'. For Slack's own verdict behind the proxy, run the same command as a custom server, which keeps the entry's probe (#18034): control-openhands browser click 'testid=mcp-add-custom-server', pick STDIO, fill testid=add-mcp-server-form >> testid=name-input qa_slack, testid=add-mcp-server-form >> testid=command-input npx, control-openhands browser fill 'testid=add-mcp-server-form >> testid=args-input' $'-y\n@zencoderai/slack-mcp-server' and control-openhands browser fill 'testid=add-mcp-server-form >> testid=env-input' $'SLACK_TEAM_ID=T_QA_INVALID\nSLACK_BOT_TOKEN=xoxb-qa-invalid-000\nNODE_EXTRA_CA_CERTS=/root/.ccr/ca-bundle.crt' (your proxy's CA bundle), then control-openhands browser click 'testid=add-mcp-server-form >> testid=mcp-test-connection' and control-openhands browser text 'testid=add-mcp-server-form >> testid=mcp-test-message' --timeout 90000: Credential check failed: invalid_auth, class text-red-500 (control-openhands browser screenshot 'testid=mcp-custom-editor' --feature F17.credential-probe --name slack-custom-invalid-auth). control-openhands browser click 'testid=add-mcp-server-form >> testid=submit-button' --observe 'testid=add-mcp-server-form >> testid=submit-button' --observe-ms 20000 (Add Server, Add Server [disabled], Add Server: Save tests first) keeps the editor open (control-openhands browser count 'testid=mcp-custom-editor' is 1) with the same red message, and control-openhands api GET /api/settings --pick agent_settings.mcp_config.qa_slack has no value; close it with control-openhands browser click 'testid=mcp-custom-editor-close' (an Escape pressed while the Save's test still runs is ignored). A server the catalog does not know gets no probe: control-openhands browser click 'testid=mcp-health-probe-qa_vault' --observe 'testid=mcp-health-label-qa_vault' --observe-ms 20000 ends on Reachable — credentials not verified (the Card health wording).

One error per stale mutationF17.delete-server · F17.enable-disableKnown issue

Scenario F17--one-error-per-stale-mutation · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Setup
Use the <viewport> viewport.
Setup
Create the disposable mcp server fixture specified in the recipe.
Setup
Arrange the prerequisite backend state through the API.
Do
Open /mcp.
Do
Reload the page.
Do
Choose “Edit qa_toasts”.
Setup
Arrange the prerequisite backend state through the API.
Do
Use browser record as specified in the exact recipe.
Do
Choose “Mcp custom editor delete”.
Do
Choose “Confirmation dialog” / “Confirm”.
Do
Use browser record as specified in the exact recipe.
Do
Choose “Mcp installed toggle qa toasts”.

Expect

Delete in the editor asks for confirmation; Cancel keeps the server, Confirm removes it with MCP server removed.

the card toggle disables a server without removing its configuration; the state persists and a disabled server is withheld from new conversations.

Linked known issues

Known issue #17941: turning off an enabled card toggle can drop the click when the pointer moves in and presses at once (F17.enable-disable).

Exact CLI recipe & bug checks

For each viewport desktop and phone, run control-openhands browser viewport <viewport>. Arrange a disposable fixture with control-openhands fixture mcp-server --name qa-toasts; substitute its printed <node> and <server.mjs> in control-openhands api POST /api/settings/mcp/qa_toasts --write --data '{"command":"<node>","args":["<server.mjs>"],"enabled":true}' --pick agent_settings.mcp_config. Run control-openhands browser goto /mcp, control-openhands browser reload, and control-openhands browser click 'role=button[name="Edit qa_toasts"]'. Remove it behind the loaded editor with control-openhands api DELETE /api/settings/mcp/qa_toasts --write --pick agent_settings.mcp_config (arrange a stale view, not UI deletion proof). Start control-openhands browser record start --feature F17.delete-server --name <viewport>-stale-delete, then control-openhands browser click 'testid=mcp-custom-editor-delete' and control-openhands browser click 'testid=confirmation-modal >> testid=confirm-button' --observe 'role=status' --observe-ms 3000. control-openhands browser toasts --history contains exactly one error MCP server 'qa_toasts' was not found; the editor stays open. Capture control-openhands browser screenshot --feature F17.delete-server --name <viewport>-stale-delete, stop recording, close testid=mcp-custom-editor-close, and reload. Recreate the fixture with the same POST, reload, then repeat the API DELETE while the card is still loaded. Start control-openhands browser record start --feature F17.enable-disable --name <viewport>-stale-toggle, run control-openhands browser click 'testid=mcp-installed-toggle-qa_toasts' --hover-first --observe 'role=status' --observe-ms 3000, and read control-openhands browser toasts --history: again exactly one error with that text. Capture under F17.enable-disable, stop recording, and reload. control-openhands api GET /api/settings --pick agent_settings.mcp_config.qa_toasts has no value; restore desktop after the phone pass. The Agent Server's 404s and browser HTTP-error diagnostics are expected in these checks.

The agent uses itF17.agent-uses-serverKnown issue

Scenario F17--the-agent-uses-it · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Start a conversation with the prompt “Use the qa_time MCP server's get_current_time tool with timezone UTC. Do not use the terminal. Reply with only the tool's datetime value.”.
Do
Start a conversation as specified in the recipe.
Do
Open /mcp.

Expect

a conversation started after the save can call the server's tools.

Linked known issues

Known issue #15421: A configured Linear MCP can be ignored while an explicitly disabled Linear skill is used instead. The new-chat and automation paths need a working authorized Linear MCP connection, a concrete issue ID and an approved model; the report's automation ZIP is also needed to replay its exact automation settings. (F17.agent-uses-server).

Exact CLI recipe & bug checks

With qa_time enabled run control-openhands conversation start --prompt "Use the qa_time MCP server's get_current_time tool with timezone UTC. Do not use the terminal. Reply with only the tool's datetime value." --wait --timeout 240, then control-openhands conversation events <id> --kinds ActionEvent,ObservationEvent,MessageEvent with the id it printed. There is an ActionEvent with tool qa_time_get_current_time (MCP tools reach the agent as <server id>_<tool>), its observation reads [Tool 'qa_time_get_current_time' executed.] { "timezone": "UTC", "datetime": … }, and the agent's reply is that datetime. control-openhands conversation events <id> --grep qa_time_get_current_time --from-start also matches the SystemPromptEvent (the tool was offered). control-openhands browser screenshot --feature F17.agent-uses-server --name conversation shows the tool step (its title is model-written, e.g. Get current time in UTC via qa_time MCP) and the datetime reply. conversation start leaves the browser on the conversation: control-openhands browser goto /mcp next.

Enable and disableF17.enable-disableKnown issue

Scenario F17--enable-and-disable · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Mcp installed toggle qa time”.
Do
Reload the page.
Do
Start a conversation with the prompt “Without running any command: is a tool named get_current_time available to you right now? Reply with only YES or NO.”.
Do
Open /mcp.

Expect

the card toggle disables a server without removing its configuration; the state persists and a disabled server is withheld from new conversations.

Linked known issues

Known issue #17941: turning off an enabled card toggle can drop the click when the pointer moves in and presses at once (F17.enable-disable).

Exact CLI recipe & bug checks

On /mcp run control-openhands browser attr 'testid=mcp-installed-toggle-qa_time' aria-checked (true; aria-label Disable server), control-openhands browser click 'testid=mcp-installed-toggle-qa_time', control-openhands browser reload and the same attr (false; label Enable server). control-openhands api GET /api/settings shows agent_settings.mcp_config.qa_time.enabled false with command, args and env intact; control-openhands browser screenshot 'testid=mcp-server-item >> has-text=qa_time' --feature F17.enable-disable --name disabled shows the + icon. With no other server exposing that tool, run control-openhands conversation start --prompt "Without running any command: is a tool named get_current_time available to you right now? Reply with only YES or NO." --wait --timeout 240; the reply is NO, control-openhands api GET /api/conversations/<id> --pick agent.mcp_config.qa_time.enabled is false, and control-openhands conversation events <id> --grep qa_time_get_current_time --from-start has count 0 (no SystemPromptEvent lists it). Then control-openhands browser goto /mcp. Click the toggle again and reload; it is true.

Phone modalsF17.phoneMapped behavior

Scenario F17--phone-modals · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Setup
Use the phone viewport.
Do
Choose “Mcp add custom server”.
Setup
Use the desktop viewport.

Expect

the MCP page, the install modal and the custom editor fit a 390 px viewport.

Exact CLI recipe & bug checks

At browser viewport phone on /mcp run control-openhands browser click 'testid=mcp-add-custom-server', control-openhands browser bbox 'testid=mcp-custom-editor' (insideViewport true, pageHorizontalOverflow false) and control-openhands browser screenshot --feature F17.phone --name custom-editor; close with testid=mcp-custom-editor-close. Do the same for testid=mcp-marketplace-card-stripe with testid=mcp-install-modal (--name install-modal), then control-openhands browser viewport desktop.

Cloud backendF17.cloud-nav · F17.native-git-tabsBlocked

Scenario F17--cloud-backend · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Choose “Mcp marketplace card github”.

Expect

with a Cloud backend active, Plugins and Apps are hidden and Skills becomes an external link to <cloud host>/settings/skills (blocked here).

on a Cloud backend, GitHub/GitLab/Bitbucket offer a recommended native-integration tab next to the MCP tab (blocked here).

Exact CLI recipe & bug checks

Blocked without an OpenHands Cloud backend. On the local backend control-openhands browser click 'testid=mcp-marketplace-card-github' shows only the MCP form (browser count 'testid=mcp-install-tab-native' is 0). With Cloud active, browser text 'testid=extensions-navbar-desktop' should list only MCP Servers and the cloud Skills link (browser attr 'testid=sidebar-extensions-/skills' href ends in /settings/skills), and the GitHub modal should show mcp-install-tab-native (Recommended) and mcp-install-tab-mcp.

CleanupF17 · setup / cleanupSupport step

Scenario F17--cleanup · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Start a conversation as specified in the recipe.
Do
Use browser focus as specified in the exact recipe.
Do
Press Enter.
Do
Choose “Mcp custom editor delete”.
Do
Choose “Confirmation dialog” / “Confirm”.
Wait
Wait for the element identified in the CLI recipe to disappear.
Do
Reload the page.

Expect

Complete this support step before continuing to the next scenario.

Exact CLI recipe & bug checks

On /mcp (not the conversation page conversation start left open), delete qa_time, qa_shttp, qa_vault and any time, time_1, tavily, filesystem cards through the editor's Delete: for each id, browser focus '[data-server-id=<id>]', browser press Enter, browser click 'testid=mcp-custom-editor-delete', browser click 'testid=confirmation-modal >> testid=confirm-button', browser wait '[data-server-id=<id>]' --state detached --timeout 10000. After browser reload, browser count 'testid=mcp-installed-empty' is 1 and api GET /api/settings --pick agent_settings.mcp_config is {}. Delete the TAVILY_API_KEY secret on /settings/secrets (F14: testid=secret-item >> has-text=TAVILY_API_KEY >> testid=delete-secret-button, then testid=confirmation-modal >> testid=confirm-button), stop the Streamable HTTP fixture by its PID, and restore deepseek-flash as the active profile if you changed it.

Health belongs to a backendF17.server-healthKnown failure

Scenario F17--health-belongs-to-a-backend · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Setup
Create the disposable mcp server fixture specified in the recipe.
Do
Open the URL in the recipe.

Expect

Known bug observations

Known failure, reproduced twice 2026-10-12 (#16954): the new identity shows the first identity's Reachable verdict. Return to Local, reload only before the next independent trial, probe and switch again. This proves the identity collision; different credentials, delayed probes and revision rotation on independent servers are additional branches. Restore Local and remove the alias afterwards.

Exact CLI recipe & bug checks

Arrange a real fixture mcp-server echo entry. Register a second disposable backend identity against the own stack's direct Agent Server port (the first uses its ingress); this exercises separate registry identities with one physical stack. On Local open /mcp, probe the echo server and wait for Reachable — credentials not verified. Switch through backend-selector → role=listbox >> text="QA_SCOPE" and read the same card without browser goto or reload (both clear the global store). Expected: the unprobed identity is unchecked. Known failure, reproduced twice 2026-10-12 (#16954): the new identity shows the first identity's Reachable verdict. Return to Local, reload only before the next independent trial, probe and switch again. This proves the identity collision; different credentials, delayed probes and revision rotation on independent servers are additional branches. Restore Local and remove the alias afterwards.

Incomplete sibling must not hide working toolsF17.custom-add · F17.agent-uses-serverKnown failureKnown issue

Scenario F17--incomplete-sibling-must-not-hide-working-tools · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Setup
Create the disposable mcp server fixture specified in the recipe.
Setup
Arrange the prerequisite backend state through the API.

Expect

Add Server saves and closes; the card is listed after a reload.

a conversation started after the save can call the server's tools.

Known bug observations

Known failure, reproduced twice 2026-10-12 (OpenHands/software-agent-sdk#5643): 201 stores {"enabled":true}, the entry has no card in the UI, and a real Home-composer start initializes without the valid echo tool. Delete qa_empty and recheck the good-only baseline. No successful model reply is required to inspect the initialized tool list.

Linked known issues

Exact CLI recipe & bug checks

Use fixture mcp-server --name qa-settings-echo and the real settings API to store its printed stdio command/args under qa_settings_echo. Probe it through the MCP card; a UI-started dummy-loopback conversation's SystemPromptEvent includes qa_echo. Arrange api POST /api/settings/mcp/qa_empty --write --data '{}'. Expected: 422 and no entry; valid tools remain available. Known failure, reproduced twice 2026-10-12 (OpenHands/software-agent-sdk#5643): 201 stores {"enabled":true}, the entry has no card in the UI, and a real Home-composer start initializes without the valid echo tool. Delete qa_empty and recheck the good-only baseline. No successful model reply is required to inspect the initialized tool list.

Unreachable MCP degrades initializationF17.agent-uses-serverKnown issue

Scenario F17--unreachable-mcp-degrades-initialization · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
Beside the actual echo fixture arrange first a missing stdio executable, then a streamable-http server at http://127.0.0.1:1/mcp; for each, start a chat through Home with the dummy loopback model and inspect conversation events <id> --kinds SystemPromptEvent --from-start --last 1. Expected: initialization completes and retains qa_settings_echo_qa_echo, then reaches the refused LLM endpoint. This checks connection-failure degradation before a successful reply. Structural invalid config has the separate OpenHands/software-agent-sdk#5643 failure above. Remove only the own bad entry.

Expect

a conversation started after the save can call the server's tools.

Linked known issues

Exact CLI recipe & bug checks

Beside the actual echo fixture arrange first a missing stdio executable, then a streamable-http server at http://127.0.0.1:1/mcp; for each, start a chat through Home with the dummy loopback model and inspect conversation events <id> --kinds SystemPromptEvent --from-start --last 1. Expected: initialization completes and retains qa_settings_echo_qa_echo, then reaches the refused LLM endpoint. This checks connection-failure degradation before a successful reply. Structural invalid config has the separate OpenHands/software-agent-sdk#5643 failure above. Remove only the own bad entry.

MCP timeout leaves initializationF17.agent-uses-serverKnown issue

Scenario F17--mcp-timeout-leaves-initialization · Permalink · Source

Given

Use F17’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.

Do — reading outline

Do
An own stdio process that keeps stdin open for 90 seconds without answering the MCP handshake is a real unresponsive-server precondition. Configure it beside the echo fixture, then start a chat from Home using the dummy loopback model. Expected: after the bounded initialization timeout, a SystemPromptEvent and user message persist and the run reaches the refused LLM endpoint. Inspect the initialized tools separately: proceeding past the timeout alone does not establish healthy-sibling retention or a successful model reply. Remove the own slow entry and doctor.

Expect

a conversation started after the save can call the server's tools.

Linked known issues

Exact CLI recipe & bug checks

An own stdio process that keeps stdin open for 90 seconds without answering the MCP handshake is a real unresponsive-server precondition. Configure it beside the echo fixture, then start a chat from Home using the dummy loopback model. Expected: after the bounded initialization timeout, a SystemPromptEvent and user message persist and the run reaches the refused LLM endpoint. Inspect the initialized tools separately: proceeding past the timeout alone does not establish healthy-sibling retention or a successful model reply. Remove the own slow entry and doctor.

Family notes, prerequisites for known issues & limits Known issue

Server errors on MCP mutations show one toast with the server's message, or An error occurred when the body has none. A custom server's Save Server or Add Server action whose pre-save connection test fails keeps the editor open. To check backend loss, stop the owned Agent Server with the editor open, click Save, then restore it with control-openhands restart; background settings polls can add separate error toasts, so count only the toast caused by the click. A failed add, edit or delete is retried three times (withRetry in settings-service.api.ts, 0.5 s and 1 s apart) before its single toast; that retry behavior is open #17932.
Desktop /customize is a client-side redirect: browser url may still read /customize for a moment; use wait-url '/mcp(\?|$)'.
From 768 to 1023 px the sidebar rail stays and the phone top bar is gone. The compact Customize selector above each detail page provides sibling navigation; the rail's Customize link (sidebar-skills-link, unscoped at this width) still returns to the hub.
On a phone the drawer and the hidden desktop rail both render sidebar-skills-link; scope it with testid=sidebar-mobile-drawer >> ... or the click fails with a strict-mode error, even though browser testids lists it once.
Editor dropdowns are comboboxes: click server-type-dropdown or auth-mode-dropdown, then role=option[name="..."]. Server types read SSE, STDIO and SHTTP (not "Streamable HTTP"); auth modes read None, Bearer token, Header and OAuth. The type dropdown exists only when adding.
The timeout field is type="number": browser fill refuses letters, so Timeout must be a valid number cannot be reached from the UI.
A red test message stays on screen when a later validation error appears, so css=p.text-feedback-error can match two paragraphs; take >> nth=0 (the validation error comes first).
Install-modal required fields use native browser validation (a tooltip, no paragraph): assert validationMessage.
The installed and library toggles are role=switch buttons: read aria-checked, not aria-pressed.
MCP stdio servers start with a minimal environment. Behind a TLS-intercepting proxy, without trust settings in the run's private HOME, uvx catalog installs fail with Connection closed (<run>/private/stack.log shows UnknownIssuer) and npx ones with Connection timed out. That is the sandbox, not the product. launch now seeds the trust files from SSL_CERT_FILE/NODE_EXTRA_CA_CERTS, so a fresh run installs fine and F17.install-error must move them aside; once a package is in the run's uv cache it installs even without them.
An installed card's health row swallows clicks (so the probe button does not open the editor). On a card without a catalog description (qa_broken) that row is the card's centre, so a plain card click opens nothing; a catalog description moves it down. Open the editor by the card's name or mcp-server-detail-<id> line, or focus the card and press Enter.
conversation start leaves the browser on the conversation page; every MCP-page command after it needs browser goto /mcp first, or the per-card selectors silently wait on a page that has none.
@modelcontextprotocol/server-everything sse printed Server is running here but never answered on its port; use the streamableHttp mode for the remote-server fixture.
A server without a catalog credential probe reads Reachable — credentials not verified, never Connected — credentials verified (N tool(s)), even without credentials. Health after a save is seeded from the pre-save test and is lost on reload (Not checked yet).
Adding a custom server shows no toast; only catalog installs and deletes toast. Assert the card after reload.
Catalog installs are add-only: a second Time install becomes time_1. STDIO servers match the catalog by command plus leading args rather than name, so time_1 and a custom server running a catalog command under its own name (qa_time) retain the description, badge and search keywords. A server that only borrows a catalog name does not (F17.install-stdio, F17.custom-add). Twelve provider-OAuth-only library entries, such as Sentry, HubSpot and Cloudflare OAuth, open an empty modal whose Install does nothing (F17.install-unsupported, #17922).
The section filter's trigger and menu are named Filter MCP servers.
Header and environment-variable format errors have distinct copy. F17.custom-validation expects Headers must follow KEY=value format for headers and Environment variables must follow KEY=value format for env lines. Feedback uses readable theme classes text-feedback-error and text-feedback-success.
A catalog stdio server runs with the Agent Server's minimal environment: behind a TLS-intercepting proxy its own HTTPS calls fail (fetch failed) even when npx could download it, and the install modal has no field for NODE_EXTRA_CA_CERTS; use the custom editor with the same command to see the provider's real answer (F17.credential-probe). The Credential check failed: fetch failed wording itself is #18161: the Slack interpreter turns any is_error tool result into a credential failure, though the server connected and listed its tools.
GitHub's hosted MCP endpoint answers an invalid token with HTTP 401 at connect, before tools are listed, so its get_me probe never runs and the modal shows the connection wording (Connection failed: HTTPStatusError … 401 Unauthorized), not Credential check failed; filed as OpenHands/software-agent-sdk#5607, since the test endpoint reports the 401 as error_kind unknown.
The shell-gated qa_vault fixture exits before the MCP handshake, which the Agent Server reports either at once as Connection failed: McpError: Connection closed or only at its deadline as Connection timed out. Check the URL and try again.; neither is the Could not start the server command wording, which needs a command that cannot be spawned at all. Wait on text=Connection, which matches both and not Connected — ….
browser network --bodies shows what the page sent, never a secret: a value under a key that names a credential (key, token, secret, auth, pass, session, sig, credential) and every value of an env or headers map becomes <redacted N chars> (N the real length), and a URL in any other string (a server's url) keeps its host and path but loses its userinfo and the values of its secret-named query parameters (https://<redacted 2 chars>:<redacted 8 chars>@host/mcp?api_key=<redacted 17 chars>), while the settings API's ********** placeholder is kept as is, so a row tells "placeholder sent" from "a value sent". The MCP editor sends neither for an untouched secret: Test connection swaps the placeholder for the stored encrypted value (a GET /api/settings in encrypted mode first; Fernet tokens, so 120 chars for a 19-char token, 140 for 40, 268 for 142) and Save leaves env out of its sparse PATCH altogether (F17.stored-secret-reuse). Without --bodies no row carries a body field, and only app-origin writes keep one.
A browser verb's client call waits 120 s, or 10 s longer than the verb's own --timeout or --observe-ms when that is longer, so browser wait 'testid=mcp-install-modal-error' --timeout 150000 runs its full deadline and a miss ends with Playwright's timeout and a failure screenshot (before this, every call was aborted at 120 s with TimeoutError: The operation was aborted due to timeout while the page kept going). For a first Slack install use a plain click, then that wait; --observe-ms is for a few transient states, not a minute-long outcome.
Known issue #17941: turning off an enabled card toggle can drop the click when the pointer moves in and presses at once (F17.enable-disable).
Known issue #15421: A configured Linear MCP can be ignored while an explicitly disabled Linear skill is used instead. The new-chat and automation paths need a working authorized Linear MCP connection, a concrete issue ID and an approved model; the report's automation ZIP is also needed to replay its exact automation settings. (F17.agent-uses-server).
Known issue #15430: An OAuth MCP callback in Docker can redirect to an unexposed dynamic localhost port. Completing the callback needs a Docker stack and an approved OAuth MCP provider/account (the report uses GitLab); a local unauthenticated Test connection cannot reach the redirect. (F17.install-oauth).
Known issue OpenHands/software-agent-sdk#5375: A successful GitLab OAuth login can be followed by an invalid localhost redirect URI on the first ticket-to-code-change MCP action. Reproduction needs the authorized self-hosted GitLab MCP endpoint/account and the ticket-to-code-change plugin/workflow; the reported endpoint is not a public fixture. (F17.install-oauth, F17.agent-uses-server).

Before you start

Use an OpenHands checkout at the source revision, with Node 24+, installed npm dependencies, uv / uvx, and a supported Chromium browser. The verification skill is the complete setup contract.

export PATH="$PWD/.agents/skills/verify-openhands/scripts:$PATH"
export OH_VERIFY_RUN=$(control-openhands launch --new --print-run)
control-openhands doctor
control-openhands onboard --skip
# Configure a model only for a scenario that needs one:
control-openhands llm preset deepseek
# When the run is complete:
control-openhands stop

F01 walks onboarding instead of skipping it. No-model checks run before model configuration. Keep OH_VERIFY_RUN set so commands address your own run. Desktop is 1440 × 1000; phone is 390 × 844. Model-backed checks need the configured provider key and budget.

Recipes run in source order within each family and can share fixtures. The page preserves that order; “as above” is a dependency on an earlier case. Setup commands and API writes arrange state. They do not prove a UI path. Follow the family cleanup steps.

A known failure preserves the intended expectation beside the observed defect. Executable known-bug blocks use xfail for a reproduced symptom and xpass when it stops reproducing. Cloud accounts, enterprise capabilities and other unavailable prerequisites remain explicit. “Not run” is different from “blocked”.

The maintained map describes application behavior at its pinned revision. Once a fix is verified in the application version under test, remove the obsolete failure note and keep the normal expectation. Dated evidence belongs in the backfill ledger.

Read the full map index and coverage boundaries, the CLI guide, or the earlier illustrated edition. The verification approach adapts Lauren Tan’s (@poteto) pstack verification-skill generators; attribution and adaptation.