Skills catalog
F18 · 22 mapped behaviors in the Self-Healing Codebase Project. Follow the prerequisites, actions and expected results; expand the original CLI recipe to replay it.
778 behavior IDs · 28 families · 844 named recipes & support steps
16 October 2026 · PR #18334 at 9a3c3a69
Canvas 1.27.1 · Agent Server 1.55.0 · automation 1.20.0
This is the map at the revision above, including its open bug notes. It does not incorporate later changes on main or claim a fresh verification run. Setup & reading guide · Download the scenario inventory
All scenarios & marker key · Open a script to read Given / Do / Expect. Expand the exact recipe for replayable commands.
No scenarios match. Try a different word or reset.
F18 Skills catalog
22 behaviors · 32 scripts & contracts · Maintained source
Entry points & prerequisites
What this family covers
The Skills page (Customize → Skills, /skills) lists every skill the backend can load: the built-in catalog bundled with the frontend plus personal and project skills reported by the Agent Server. A user searches and filters the cards (filters are mirrored into the URL), opens a detail modal, switches skills on or off (saved to settings straight away, effective for new conversations only), launches a skill into chat, and reads how to add one with /add-skill; after the agent installs a skill in a conversation, a banner offers a fresh conversation that loads it.
How a user gets here
- Sidebar Customize (
sidebar-skills-link, goes to/customize, which redirects to/mcpon desktop), then Skills in the Customize navigation (sidebar-extensions-/skills). - Direct URL
/skills, with optional filters:/skills?q=docker&state=enabled&recommendation=recommended&source=project&category=agent-authoring&type=knowledge. - Command menu (
Control+k/Meta+k): searching "Skills" lists only Customize ("Browse skills, plugins, and integrations."), which lands on/mcp; then click Skills. There is no direct Skills command. - Phone:
/customizeshows the Customize hub (extensions-mobile-hub) with a Skills row; on/skillsthe header back button (sidebar-mobile-back-button) returns to the hub. - Install banner: inside a conversation, after the agent ran
/add-skill <GitHub URL>(local backend only). - Project skills: a
SKILL.mdcommitted under<workspace>/.agents/skills/<name>/in the folder picked with Open Workspace on Home (Local Repo or New Worktree mode); the conversation's chat shows Skill Ready when a message triggers one. - Cloud: Customize → Skills on a Cloud backend (external link).
Given
Preconditions:
- Baseline state (launched, doctored,
onboard --skipdone); desktop viewport unless a bullet says otherwise. - A fresh run lists only the bundled catalog,
<N>skills, all with sourcepublic.<N>is the catalog length the checkout bundles: runnode --input-type=module -e "import {SKILLS_CATALOG} from '@openhands/extensions/skills'; console.log(SKILLS_CATALOG.length)"in the checkout (68 with@openhands/extensions0.29.0, #17998). The Source and Type facet groups stay hidden until a personal/project skill exists.add-javadocandflarglebargleare off;dockeris on (recommended). F18.skill-in-chat,F18.toggle-local,F18.copy-sourceandF18.install-bannerneed an active LLM profile (control-openhands llm preset deepseek).F18.install-banneralso needs the agent to reach github.com.- Personal skill fixture (arranged through the agent, not proof): run
control-openhands conversation start --prompt 'Use one shell command to create the file ~/.agents/skills/qa-hello/SKILL.md (create the folders) with exactly these 7 lines: "---", "name: qa-hello", "description: QA test skill that answers qa-ping with QA-PONG-7731.", "triggers:", "- qa-ping", "---", "When the user says qa-ping, reply with exactly QA-PONG-7731 and nothing else." Then reply done.' --wait --timeout 240. Afterwards/skillsshows<N+1> result(s)and aqa-hellocard (type pillAuto-discovery), and the Enabled facet count goes from 12 to 13. - Project skill fixture for
F18.project-skill(arranged, not proof; no model needed), in this family's own repo so the commit never lands in F08'sqa-repo:control-openhands fixture git-repo --name qa-f18-repo, thencontrol-openhands fixture skill --repo qa-f18-repo --name qa-f18-skill --trigger qa-f18-ping --commit: it writes<run>/workspace/qa-f18-repo/.agents/skills/qa-f18-skill/SKILL.md(scopeproject) and commits it in the fixture repo (committed.shaandcommitted.messageAdd qa-f18-skill skill;git -C "$OH_VERIFY_RUN/workspace/qa-f18-repo" log --onelinelistsAdd qa-f18-skill skillaboveInitial fixture commit). A re-run reports the samecommitted.shawithcommitted.unchangedtrue. Without--committhe file stays uncommitted, and a New Worktree checkout holds only committed files. A fixture skill's name must not be a substring of its trigger (see Gotchas). F18.skill-deletedcreates its own personal fixture right before its bullet (control-openhands fixture skill --name qa-gone --trigger qa-farewell, written into the run's private HOME) and deletes it inside the bullet, so the counts in the other bullets are unaffected.F18.cloud-linkneeds an OpenHands Cloud backend and account (blocked in a sandbox).
Open from the sidebarF18.pageKnown issue
Scenario F18--open-from-the-sidebar · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoChoose “Sidebar skills”.
- DoChoose “Sidebar extensions /skills”.
Expect
the page shows the title, description, the notice "Skill changes apply to new conversations only.", an Add skill button, a result count, the facet rail and the card grid. (Empty state "No skills found." and loading skeletons are not reachable on a local backend.)
Linked known issues
Known issue #15458: The global Skills page can list AGENTS.md files from cloned automation-run worktrees. The report says the discovery trigger is unknown; replay needs the affected automation workspace/discovery-root layout or captured request parameters, rather than only a normal project-skill fixture. (F18.page, F18.project-skill).
Exact CLI recipe & bug checks
From / run control-openhands browser click 'testid=sidebar-skills-link' --expect-url '/mcp(\?|$)', control-openhands browser click 'testid=sidebar-extensions-/skills' --expect-url '/skills(\?|$)', then control-openhands browser text 'testid=skills-settings-description' (Discover skills to add to your workspace. Search above the cards or filter by state, recommendation, and category, then open a card to see its details and use the skill. Enable or disable default skills. Disabled skills will not be loaded into agent context., describing this layout since #18035), control-openhands browser text 'testid=skills-new-conversation-notice' (Skill changes apply to new conversations only.) and control-openhands browser text 'testid=skills-result-summary' (<N> result(s) on a fresh run). control-openhands browser screenshot --feature F18.page --name desktop shows the Customize nav, the facet rail and the card grid.
SearchF18.searchMapped behavior
Scenario F18--search · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoEnter “docker” in “Skills search”.
- DoEnter “zzqa-nomatch” in “Skills search”.
- DoChoose “Skills toolbar” / “Clear search”.
Expect
typing filters cards by name, description, content or trigger; after 300 ms the query is mirrored to ?q= without a new history entry; the X clears it; no match shows "No skills match your search."
Exact CLI recipe & bug checks
On /skills run control-openhands browser fill 'testid=skills-search-input' docker, control-openhands browser text 'testid=skills-result-summary' (6 result(s) on a fresh run: content matches count too) and, a second later, control-openhands browser url (ends in /skills?q=docker); control-openhands browser eval "history.length" is the same before and after typing. Then control-openhands browser fill 'testid=skills-search-input' zzqa-nomatch and control-openhands browser text 'testid=skills-no-match': No skills match your search. with 0 result(s). Clear with control-openhands browser click 'testid=skills-toolbar >> role=button[name="Clear search"]'; control-openhands browser value 'testid=skills-search-input' is empty, the full count is back and ?q= leaves the URL.
Back and forwardF18.search-historyMapped behavior
Scenario F18--back-and-forward · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoOpen /skills?q=docker.
- DoChoose “Skill facet state enabled”.
- DoEnter “github” in “Skills search”.
- DoGo back in browser history.
- DoGo forward in browser history.
Expect
browser Back/Forward put the matching query back into the search box.
Exact CLI recipe & bug checks
Run control-openhands browser goto '/skills?q=docker', control-openhands browser click 'testid=skill-facet-state-enabled' (pushes ?q=docker&state=enabled), control-openhands browser fill 'testid=skills-search-input' github, wait a second, then control-openhands browser back and control-openhands browser value 'testid=skills-search-input': docker, URL /skills?q=docker. control-openhands browser forward gives github and /skills?q=github&state=enabled.
FacetsF18.facetsMapped behavior
Scenario F18--facets · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoChoose “Skill facet recommendation recommended”.
- DoChoose “Skill facet state enabled”.
- DoChoose “Skill facet category agent authoring”.
- DoChoose “Skills clear filters”.
Expect
the desktop facet rail (State, Recommendation, Source, Category, Type) filters with per-row counts, writes canonical URL params, disables zero-count rows and offers Clear filters; deep links apply and unknown values are dropped.
Exact CLI recipe & bug checks
On /skills run control-openhands browser click 'testid=skill-facet-recommendation-recommended' (URL ?recommendation=recommended, 12 result(s)), control-openhands browser click 'testid=skill-facet-state-enabled' (URL ?state=enabled&recommendation=recommended: canonical order, not click order), control-openhands browser attr 'testid=skill-facet-state-enabled' aria-checked (true), control-openhands browser enabled 'testid=skill-facet-state-disabled' (false, its count reads 0), control-openhands browser click 'testid=skill-facet-category-agent-authoring' (8 result(s)), then control-openhands browser click 'testid=skills-clear-filters': URL /skills, all results, and control-openhands browser count 'testid=skills-clear-filters' is 0.
Source and Type facetsF18.facetsMapped behavior
Scenario F18--source-and-type-facets · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoChoose “Skill facet source project”.
- DoChoose “Skill facet type agentskills”.
Expect
the desktop facet rail (State, Recommendation, Source, Category, Type) filters with per-row counts, writes canonical URL params, disables zero-count rows and offers Clear filters; deep links apply and unknown values are dropped.
Exact CLI recipe & bug checks
With the qa-hello fixture, control-openhands browser text 'testid=skill-facet-group-source' lists This project 1 and Built in <N>, and testid=skill-facet-group-type lists Auto-discovery 1 and Trigger-based <N>. control-openhands browser click 'testid=skill-facet-source-project' gives ?source=project and 1 result(s); after Clear filters, control-openhands browser click 'testid=skill-facet-type-agentskills' gives ?type=agentskills and only qa-hello (control-openhands browser eval "[...document.querySelectorAll('[data-testid^=skill-name-]')].map(e=>e.textContent).join(',')"). Clear filters again.
Deep linkF18.facetsMapped behavior
Scenario F18--deep-link · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoOpen /skills?category=bogus&type=knowledge&source=project.
Expect
the desktop facet rail (State, Recommendation, Source, Category, Type) filters with per-row counts, writes canonical URL params, disables zero-count rows and offers Clear filters; deep links apply and unknown values are dropped.
Exact CLI recipe & bug checks
Run control-openhands browser goto '/skills?category=bogus&type=knowledge&source=project' on a fresh run, control-openhands browser text 'testid=skills-result-summary' (0 result(s)) and control-openhands browser text 'testid=skill-facet-group-source': SOURCE, This project 0, Built in <N>. A URL selection keeps its group visible even when it matches nothing, and bogus is ignored. Clear with testid=skills-clear-filters.
Card and pill overflowF18.cardMapped behavior
Scenario F18--card-and-pill-overflow · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoChoose “Skill triggers add javadoc overflow”.
- DoPress Escape.
Expect
each card shows icon, name, source, a two-line description and pills; extra pills collapse into a +N popover; click, Enter or Space opens the detail modal.
Exact CLI recipe & bug checks
Run control-openhands browser testids 'testid=skill-card-add-javadoc' (name, skill-source-add-javadoc = public, skill-toggle-add-javadoc, description, pills, skill-triggers-add-javadoc-overflow labelled Show 2 more), control-openhands browser click 'testid=skill-triggers-add-javadoc-overflow' and control-openhands browser text 'testid=skill-triggers-add-javadoc-overflow-popover' (java documentation, document java); control-openhands browser count 'testid=skill-detail-modal' stays 0. control-openhands browser screenshot --feature F18.card --name overflow-popover, then control-openhands browser press Escape.
Keyboard openF18.cardMapped behavior
Scenario F18--keyboard-open · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoUse browser focus as specified in the exact recipe.
- DoPress Enter.
- DoPress Escape.
- DoPress Space.
Expect
each card shows icon, name, source, a two-line description and pills; extra pills collapse into a +N popover; click, Enter or Space opens the detail modal.
Exact CLI recipe & bug checks
Run control-openhands browser focus 'testid=skill-card-docker', control-openhands browser press Enter and control-openhands browser attr 'testid=skill-detail-modal' data-skill-name (docker); control-openhands browser press Escape, focus again, control-openhands browser press Space and control-openhands browser count 'testid=skill-detail-modal' (1). Escape closes it.
Copy the source pathF18.copy-sourceMapped behavior
Scenario F18--copy-the-source-path · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoChoose “Skill copy source qa hello”.
- DoChoose “Skill card qa hello”.
- DoChoose “Skill modal copy source qa hello”.
Expect
personal and project skills (path sources) have a copy-path button on the card and in the modal that flips to "Copied to clipboard" for 2 s; built-in skills (source public) have none.
Exact CLI recipe & bug checks
With the qa-hello fixture, control-openhands browser attr 'testid=skill-source-qa-hello' title is the full …/.agents/skills/qa-hello/SKILL.md path. Run control-openhands browser click 'testid=skill-copy-source-qa-hello', control-openhands browser attr 'testid=skill-copy-source-qa-hello' aria-label (Copied to clipboard; count 'testid=skill-detail-modal' stays 0), control-openhands browser clipboard (text is the same full SKILL.md path) and, after 2 s, the same attr (Copy source path). In the modal (control-openhands browser click 'testid=skill-card-qa-hello') control-openhands browser click 'testid=skill-modal-copy-source-qa-hello' turns its label to Copied to clipboard and the modal stays open. count 'testid=skill-copy-source-add-javadoc' is 0.
Toggle a built-in skillF18.toggleMapped behavior
Scenario F18--toggle-a-built-in-skill · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoChoose “Skill toggle add javadoc”.
- DoReload the page.
- DoHover over “Skill toggle add javadoc”.
Expect
the card's plus/check toggle switches a built-in skill on or off; the choice survives a reload (enabled_skills allow-list on a local backend).
Exact CLI recipe & bug checks
Run control-openhands browser attr 'testid=skill-toggle-add-javadoc' aria-checked (false; aria-label is Enable skill), control-openhands browser click 'testid=skill-toggle-add-javadoc' (no modal opens; aria-checked turns true and the Enabled facet count goes up by one), wait a second, control-openhands browser reload and read aria-checked again: true. control-openhands api GET /api/settings --pick misc_settings.app_preferences.enabled_skills lists add-javadoc (--pick paths start inside body). To switch an enabled skill off from its card, control-openhands browser hover 'testid=skill-toggle-add-javadoc' first (attr ... data-showing-remove is true), then click (see Gotchas).
Toggle a personal skillF18.toggle-localMapped behavior
Scenario F18--toggle-a-personal-skill · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoHover over “Skill toggle qa hello”.
- DoChoose “Skill toggle qa hello”.
- DoReload the page.
- DoStart a conversation with the prompt “qa-ping”.
- DoStart a conversation as specified in the recipe.
- DoOpen /skills.
- DoStart a conversation as specified in the recipe.
Expect
switching a personal/project skill off writes disabled_skills; new conversations then do not load it, and do again once it is back on.
Exact CLI recipe & bug checks
Run control-openhands browser hover 'testid=skill-toggle-qa-hello', control-openhands browser click 'testid=skill-toggle-qa-hello', wait a second, control-openhands browser reload and control-openhands browser attr 'testid=skill-toggle-qa-hello' aria-checked (false). control-openhands api GET /api/settings shows disabled_skills: ["qa-hello"] under body.misc_settings.app_preferences and enabled_skills without it. control-openhands conversation start --prompt "qa-ping" --wait --timeout 240, then control-openhands api GET '/api/conversations/<id>/events/search?limit=5' --max-bytes 2000000 (id from conversation start): the user MessageEvent has "activated_skills": []. Back on /skills (control-openhands browser goto /skills; conversation start left the browser on the conversation), click the toggle on again (no hover needed), start another qa-ping conversation: "activated_skills": ["qa-hello"] and control-openhands browser text 'testid=agent-message >> nth=-1' is QA-PONG-7731; disabled_skills is [] again.
Skill reaches chatF18.skill-in-chatKnown issue
Scenario F18--skill-reaches-chat · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoOpen /skills.
- DoChoose “Skill toggle flarglebargle”.
- DoStart a conversation with the prompt “flarglebargle”.
- DoOpen /skills.
Expect
an enabled skill is loaded into new conversations and fires on its trigger; a disabled one is not.
Linked known issues
Known issue #16298: A Slack-monitor setup can leave the agent without the expected skill. The report omits the concrete monitor setup and loaded skill name/version; reproducing the runtime also needs an approved model and a configured Slack integration. Listing the bundled catalog alone cannot settle it. (F18.skill-in-chat).
Exact CLI recipe & bug checks
Run control-openhands browser goto /skills (the previous bullet ends on a conversation), control-openhands browser click 'testid=skill-toggle-flarglebargle' (aria-checked true), then control-openhands conversation start --prompt "flarglebargle" --wait --timeout 240, control-openhands conversation events <id> --kinds MessageEvent --from-start (the user row has skills: ["flarglebargle"]; the events/search?limit=5 read this bullet used before can miss the user row) and control-openhands browser text 'testid=agent-message >> nth=-1' (a reply praising how smart the user is). control-openhands browser screenshot --feature F18.skill-in-chat --name enabled. Switch it off again on /skills (control-openhands browser goto /skills, hover, click; enabled_skills no longer lists it) and start another conversation with --prompt "flarglebargle. Do not run any tools; reply in one line.": the user row has no skills, and the agent does not praise the user (2026-10-08: Flarglebargle to you too — what would you like help with?).
Project skill in a workspaceF18.project-skillKnown issueBlocked
Scenario F18--project-skill-in-a-workspace · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoOpen /skills.
- DoReload the page.
- DoStart a conversation with the prompt “qa-f18-ping”.
- DoChoose “Expand”.
- DoOpen the fixture workspace.
- DoStart a conversation with the prompt “qa-f18-ping”.
- DoOpen the fixture workspace.
Expect
a skill committed under <workspace>/.agents/skills/<name>/SKILL.md is loaded into every conversation started in that workspace, in Local Repo and New Worktree mode alike, and its trigger word activates it (the chat shows a Skill Ready row naming it); workspace skills are not listed on the Skills page.
Linked known issues
Known issue #15458: The global Skills page can list AGENTS.md files from cloned automation-run worktrees. The report says the discovery trigger is unknown; replay needs the affected automation workspace/discovery-root layout or captured request parameters, rather than only a normal project-skill fixture. (F18.page, F18.project-skill).
Exact CLI recipe & bug checks
With the committed qa-f18-skill fixture, run control-openhands browser goto /skills, control-openhands browser reload and control-openhands browser count 'testid=skill-card-qa-f18-skill': 0 (the catalog lists the Agent Server's own skills, not a workspace's). Then control-openhands conversation start --workspace qa-f18-repo --prompt "qa-f18-ping" --wait --timeout 240 (note <repo-skill-id>; its workspace is the fixture's path) and, in the chat it leaves open, control-openhands browser text 'testid=generic-event-message-title' (Skill Ready; with a model a second title, Invoked skill qa-f18-skill, can follow, because the agent may also call invoke_skill), control-openhands browser click 'role=button[name="Expand"]' and control-openhands browser count 'role=button[name="qa-f18-skill"]' (1, under Triggered Skill Knowledge:); control-openhands browser screenshot --feature F18.project-skill --name skill-ready. With an LLM the agent also obeys the skill, and this is the moment to read it, while the chat is still open: control-openhands browser text 'testid=agent-message >> nth=-1' ends with QA_SKILL_OK; without a key the conversation ends in error right after the events below and that reply check is blocked (prerequisite: DEEPSEEK_API_KEY). Agent side, control-openhands conversation events <repo-skill-id> --grep qa-f18-skill --from-start matches the SystemPromptEvent (its <available_skills> lists <name>qa-f18-skill</name>) and the user MessageEvent (skills: ["qa-f18-skill"], with excerpts showing "activated_skills":["qa-f18-skill"] and the …/qa-f18-repo/.agents/skills/qa-f18-skill/SKILL.md path). New Worktree: run control-openhands workspace open qa-f18-repo --mode new_worktree (it picks the mode in the preview bar's selector and prints mode new_worktree and preview qa-f18-repo / New Worktree; control-openhands browser text 'testid=home-git-control-bar-preview' reads the same two lines), then control-openhands conversation start --stay --prompt "qa-f18-ping" --wait --timeout 240 (note <worktree-id>): its workspace is /tmp/conversation-worktrees/<worktree-id>/qa-f18-repo, a worktree of the fixture on branch openhands/<worktree-id> (read-only check: git -C /tmp/conversation-worktrees/<worktree-id>/qa-f18-repo log --oneline lists Add qa-f18-skill skill above Initial fixture commit), control-openhands browser text 'testid=generic-event-message-title' is Skill Ready again (and the same reply check applies here, with a key), and control-openhands conversation events <worktree-id> --grep qa-f18-skill --from-start matches the same events with the worktree's SKILL.md path (a ConversationStateUpdateEvent carrying the agent context matches in both conversations too). Restore the mode: control-openhands workspace open qa-f18-repo --mode local_repo (mode local_repo, preview qa-f18-repo / Local Repo).
Deleted personal skillF18.skill-deletedMapped behavior
Scenario F18--deleted-personal-skill · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- SetupCreate the disposable skill fixture specified in the recipe.
- DoOpen /skills.
- DoReload the page.
- DoStart a conversation with the prompt “qa-farewell”.
- DoOpen /skills.
- DoReload the page.
- DoStart a conversation with the prompt “qa-farewell. Do not run any tools; reply in one line.”.
Expect
a personal skill whose folder was deleted disappears from the Skills page after a reload and is not loaded into new conversations: its trigger word activates nothing.
Exact CLI recipe & bug checks
Run control-openhands fixture skill --name qa-gone --trigger qa-farewell, control-openhands browser goto /skills, control-openhands browser reload, control-openhands browser count 'testid=skill-card-qa-gone' (1; control-openhands browser attr 'testid=skill-source-qa-gone' title is the run's …/private/home/.agents/skills/qa-gone/SKILL.md path and control-openhands browser text 'testid=skills-result-summary' counts one result more than before) and control-openhands browser screenshot --feature F18.skill-deleted --name present. Prove it loads: control-openhands conversation start --prompt "qa-farewell" --wait --timeout 240 (note <gone-a>), control-openhands browser text 'testid=generic-event-message-title' (Skill Ready) and control-openhands conversation events <gone-a> --grep qa-gone --from-start: the SystemPromptEvent and the user MessageEvent match, the latter with skills: ["qa-gone"]. Delete the folder (outside the UI; it is inside the run): rm -rf "$OH_VERIFY_RUN/private/home/.agents/skills/qa-gone". Then control-openhands browser goto /skills, control-openhands browser reload, control-openhands browser count 'testid=skill-card-qa-gone' (0; the summary is back to its earlier count), control-openhands conversation start --prompt "qa-farewell. Do not run any tools; reply in one line." --wait --timeout 240 (note <gone-b>; with a model, a bare qa-farewell sent deepseek-flash searching the disk for the word, and it quoted the run's stack.log line Skill 'qa-gone' triggered by keyword 'qa-farewell', so --grep qa-gone matched 9 tool rows although the skill was not loaded), control-openhands browser count 'text=Skill Ready' (0; control-openhands browser screenshot --feature F18.skill-deleted --name absent) and control-openhands conversation events <gone-b> --grep qa-gone --from-start: count is 0, and control-openhands conversation events <gone-b> --kinds MessageEvent --from-start shows the user message without a skills row. No model is needed: the skill set is fixed before the first model call.
Both personal skill foldersF18.personal-skill-dirsMapped behavior
Scenario F18--both-personal-skill-folders · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- SetupCreate the disposable skill fixture specified in the recipe.
- DoOpen /skills.
- DoReload the page.
- DoStart a conversation with the prompt “qa-legacy-ping. Do not run any tools; reply in one line.”.
- DoOpen /skills.
- DoReload the page.
Expect
personal skills load from both user folders: ~/.agents/skills/<name>/SKILL.md and the persistence folder's skills/<name>/SKILL.md (~/.openhands/skills, which a Canvas stack moves to $OH_PERSISTENCE_DIR/skills). Each shows on the Skills page with its path and fires on its trigger in new conversations.
Exact CLI recipe & bug checks
The qa-gone bullet above used ~/.agents/skills. For the other folder, write a fixture into the stack's persistence folder (outside the UI, inside the run; fixture skill writes only ~/.agents/skills): mkdir -p "$OH_VERIFY_RUN/private/skills/qa-legacy" and a SKILL.md there with the front matter name: qa-legacy, description: QA fixture skill in the persistence-dir skills folder., triggers: [qa-legacy-ping] and the body When this skill is active, end your reply with the word QA_LEGACY_OK. ($OH_VERIFY_RUN/private is the Agent Server's OH_PERSISTENCE_DIR, which replaces ~/.openhands). Run control-openhands browser goto /skills, control-openhands browser reload, control-openhands browser count 'testid=skill-card-qa-legacy' (1) and control-openhands browser attr 'testid=skill-source-qa-legacy' title (…/private/skills/qa-legacy/SKILL.md). Then control-openhands conversation start --prompt "qa-legacy-ping. Do not run any tools; reply in one line." --wait --timeout 240 (note <legacy-id>): control-openhands browser text 'testid=generic-event-message-title' is Skill Ready, control-openhands conversation events <legacy-id> --kinds MessageEvent --from-start shows the user row with skills: ["qa-legacy"], and with a model control-openhands browser text 'testid=agent-message >> nth=-1' ends with QA_LEGACY_OK (control-openhands browser screenshot --feature F18.personal-skill-dirs --name persistence-dir-skill). Clean up with rm -rf "$OH_VERIFY_RUN/private/skills/qa-legacy", browser goto /skills, browser reload; the card count is 0.
Save failureF18.toggle-errorMapped behavior
Scenario F18--save-failure · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoOpen /skills.
- SetupChange the run-owned service state: stop agent-server.
- DoChoose “Skill toggle add javadoc”.
- DoRestart the run-owned stack as specified.
- DoOpen /skills.
Expect
a failed save shows an error toast and nothing is persisted.
Exact CLI recipe & bug checks
Run control-openhands browser goto /skills, wait for testid=skill-toggle-add-javadoc and note its aria-checked (true when this follows the toggle bullet, which left add-javadoc on). Then control-openhands service stop agent-server, control-openhands browser click 'testid=skill-toggle-add-javadoc' --timeout 5000 (the card flips at once), wait about 2 s (the PATCH /api/settings is tried three times before the error shows; browser network lists three 502s) and control-openhands browser toasts --history: HTTP request failed (502 Bad Gateway): "Bad Gateway: connect ECONNREFUSED 127.0.0.1:<agent-server port>". control-openhands browser screenshot --feature F18.toggle-error --name toast. Run control-openhands restart --timeout 240, control-openhands browser goto /skills: aria-checked on testid=skill-toggle-add-javadoc is back to the value noted before the click and enabled_skills is unchanged. Clear the expected 502 noise with control-openhands browser errors --clear.
Detail modal and switchF18.detail-modalMapped behavior
Scenario F18--detail-modal-and-switch · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoChoose “Skill card add javadoc”.
- DoChoose “Skill modal enable row add javadoc” / “Enabled”.
Expect
the detail modal shows source, an Enabled/Disabled switch, description, pills and the read-only content; switching off disables Use skill.
Exact CLI recipe & bug checks
With add-javadoc on, run control-openhands browser click 'testid=skill-card-add-javadoc', control-openhands browser testids 'testid=skill-detail-modal' (name, source, skill-modal-enable-row-add-javadoc, description, pills incl. license MIT, skill-modal-field-content-add-javadoc, skill-detail-close, skill-detail-use-skill-add-javadoc) and control-openhands browser enabled 'testid=skill-detail-use-skill-add-javadoc' (true). Switch off with control-openhands browser click 'testid=skill-modal-enable-row-add-javadoc >> text=Enabled'; control-openhands browser eval "document.querySelector('[data-testid=skill-modal-toggle-add-javadoc]').checked" is false, the row reads Disabled, enabled on Use skill is false and the card toggle behind reads aria-checked false. control-openhands browser screenshot --feature F18.detail-modal --name disabled. Close, reload: the card stays off and enabled_skills no longer lists add-javadoc.
Close the modalF18.detail-closeMapped behavior
Scenario F18--close-the-modal · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoChoose “Skill detail modal close”.
- DoChoose “Skill detail close”.
- DoPress Escape.
- DoUse browser mouse click as specified in the exact recipe.
Expect
the X, Close, Escape and a backdrop click close the detail modal.
Exact CLI recipe & bug checks
With testid=skill-card-docker opened each time, each of control-openhands browser click 'testid=skill-detail-modal-close', control-openhands browser click 'testid=skill-detail-close', control-openhands browser press Escape and control-openhands browser mouse-click 100 500 (backdrop) leaves control-openhands browser count 'testid=skill-detail-modal' at 0.
Pill overflow in the modalF18.detail-pillsMapped behavior
Scenario F18--pill-overflow-in-the-modal · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoChoose “Skill card add javadoc”.
- DoChoose “Skill modal pills add javadoc overflow”.
- DoChoose “Skill modal description add javadoc”.
- DoChoose “Skill modal pills add javadoc overflow”.
- DoUse browser record as specified in the exact recipe.
- DoPress Escape.
- WaitWait for “Skill modal pills add javadoc overflow popover” to disappear.
- WaitWait for “Skill detail modal” to disappear.
- DoUse browser record as specified in the exact recipe.
- SetupUse the phone viewport.
- SetupUse the desktop viewport.
Expect
the modal shows the full pill set (type, category, recommended, license, compatibility, version, triggers); pills that do not fit collapse into a +N button whose popover lists the rest without closing the modal; Escape closes the popover first and returns focus to its trigger, and a second Escape closes the modal.
Exact CLI recipe & bug checks
Run control-openhands browser click 'testid=skill-card-add-javadoc', control-openhands browser text 'testid=skill-modal-pills-add-javadoc' (at desktop: Trigger-based, Code quality & review, MIT, +4; license and compatibility appear only in the modal), control-openhands browser attr 'testid=skill-modal-pills-add-javadoc-overflow' aria-label (Show 4 more at desktop; Show 5 more at phone width), control-openhands browser click 'testid=skill-modal-pills-add-javadoc-overflow' and control-openhands browser text 'testid=skill-modal-pills-add-javadoc-overflow-popover' (Requires Java source files, javadoc, java documentation, document java at desktop; MIT joins them at phone width); count 'testid=skill-detail-modal' stays 1. control-openhands browser screenshot --feature F18.detail-pills --name popover. Close the popover with control-openhands browser click 'testid=skill-modal-description-add-javadoc'; control-openhands browser count 'testid=skill-modal-pills-add-javadoc-overflow-popover' is 0 while the detail modal stays open. Reopen it with control-openhands browser click 'testid=skill-modal-pills-add-javadoc-overflow'. With focus on the overflow trigger, run control-openhands browser record start --feature F18.detail-pills --name escape, control-openhands browser press Escape and control-openhands browser wait 'testid=skill-modal-pills-add-javadoc-overflow-popover' --state detached. control-openhands browser count 'testid=skill-detail-modal' stays 1; focus returns to the overflow trigger. Press Escape again and control-openhands browser wait 'testid=skill-detail-modal' --state detached, then control-openhands browser record stop. Repeat this modal/pill sequence at browser viewport phone, then restore browser viewport desktop.
Use skillF18.use-skillMapped behavior
Scenario F18--use-skill · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoChoose “Skill card docker”.
- DoChoose “Skill detail use skill docker”.
- DoChoose “Sidebar automations”.
- DoChoose “Sidebar conversations”.
- DoOpen /.
Expect
Use skill closes the modal, opens /conversations and pre-fills the composer with /<skill-name> .
Exact CLI recipe & bug checks
Run control-openhands browser click 'testid=skill-card-docker', control-openhands browser click 'testid=skill-detail-use-skill-docker' --expect-url '/conversations(\?|$)', then control-openhands browser eval "document.querySelector('[data-testid=chat-input]').innerText": /docker (the skill name as a slash command), with the modal gone (control-openhands browser count 'testid=skill-detail-modal' is 0) and the composer focused (control-openhands browser eval "document.activeElement.getAttribute('data-testid')" is chat-input). control-openhands browser screenshot --feature F18.use-skill --name prefilled shows /docker in the Home composer. Nothing is sent, and the text is applied once: leave and come back without a page load (control-openhands browser click 'testid=sidebar-automations-link' --expect-url '/automations$', then control-openhands browser click 'testid=sidebar-conversations-link' --expect-url '/conversations(\?|$)') and the same chat-input eval returns an empty string. A browser goto / would prove nothing here: a page load empties the composer either way.
Add skill instructionsF18.add-skill-modalMapped behavior
Scenario F18--add-skill-instructions · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoOpen /skills.
- DoChoose “Skills add skill”.
- DoChoose “Add skill modal example copy”.
- DoPress Escape.
- DoOpen /skills?q=code-review.
Expect
Add skill opens an instructions-only modal with a copyable /add-skill example (which must point at an existing skill), steps, URL formats, storage notes and a docs link; Close, X and Escape close it.
Exact CLI recipe & bug checks
Run control-openhands browser goto /skills (Use skill ends on /conversations), control-openhands browser click 'testid=skills-add-skill-button', control-openhands browser text 'testid=add-skill-modal' (title Add a skill, the example /add-skill https://github.com/OpenHands/extensions/tree/main/skills/code-review, five steps, "Supported URL formats" with the short form OpenHands/extensions/skills/code-review, "Where skills are stored", the GITHUB_TOKEN note) and control-openhands browser attr 'testid=add-skill-modal-docs-link' href (https://docs.openhands.dev/overview/skills/adding#adding-new-skills; target is _blank). Copy with control-openhands browser click 'testid=add-skill-modal-example-copy': its aria-label becomes Copied to clipboard, enabled is false and control-openhands browser clipboard returns the example command; after 2 s it reads Copy to clipboard again. Close with testid=add-skill-modal-dismiss, testid=add-skill-modal-close or control-openhands browser press Escape; count 'testid=add-skill-modal' is 0 each time. The example points at an existing skill (#18035): control-openhands browser goto '/skills?q=code-review' and control-openhands browser text 'testid=skill-source-code-review' (public) show it in the bundled catalog, which F18.install-banner installs from that same URL.
Install bannerF18.install-bannerMapped behavior
Scenario F18--install-banner · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoStart a conversation with the prompt “/add-skill https://github.com/OpenHands/extensions/tree/main/skills/code-review”.
- DoChoose “Skill install restart action”.
- DoStart a conversation as specified in the recipe.
- DoStart a conversation as specified in the recipe.
- DoOpen /conversations/<id>.
- DoChoose “Skill install restart dismiss”.
- DoReload the page.
Expect
after the agent installs a skill with /add-skill, the conversation shows "Installed to this workspace: …" with Start new conversation with this skill (same workspace) and a dismiss X (session-only).
Exact CLI recipe & bug checks
Run control-openhands conversation start --prompt "/add-skill https://github.com/OpenHands/extensions/tree/main/skills/code-review" --wait --timeout 300 and control-openhands browser text 'testid=skill-install-restart-banner': Installed to this workspace: code-review. Skills load when a conversation starts, so this conversation can't use them yet. plus Start new conversation with this skill. control-openhands browser screenshot --feature F18.install-banner --name banner. Run control-openhands browser click 'testid=skill-install-restart-action' --expect-url '/conversations/(?!<id>)[0-9a-f-]+(\?|$)' (<id> from conversation start; the lookahead skips the current URL), then control-openhands conversation status <new id> (the id in the returned URL): its workspace equals the install conversation's workspace from conversation start. Back on the first conversation (control-openhands browser goto /conversations/<id>), control-openhands browser click 'testid=skill-install-restart-dismiss' makes count 'testid=skill-install-restart-banner' 0; after control-openhands browser reload it is 1 again (dismissal is session-only by design).
Same-name source filteringF18.facetsKnown failure
Scenario F18--same-name-source-filtering · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- SetupUse the desktop viewport.
- DoOpen /skills.
- DoEnter “skill-creator” in “Skills search”.
- WaitWait for the element identified in the CLI recipe to appear.
- SetupCreate the disposable skill fixture specified in the recipe.
- DoUse browser record as specified in the exact recipe.
- DoChoose “Skill facet source project”.
- DoChoose “Skills clear filters”.
- DoChoose “Skill facet source public”.
- SetupUse the phone viewport.
- DoOpen /skills?q=skill-creator.
Expect
the desktop facet rail (State, Recommendation, Source, Category, Type) filters with per-row counts, writes canonical URL params, disables zero-count rows and offers Clear filters; deep links apply and unknown values are dropped.
Known bug observations
Known failure (reproduced 2026-10-12, #18287): the public-filter step keeps an extra local card; returning to This project** shows two cards with the identical local source path on Local macOS browser, desktop and phone. Origin unconfirmed; this does not establish a causal PR or verify the original Windows installer environment. Cleanup: after checking that the fixture path is inside this run and its file contains QA_F18_LOCAL_COLLISION, remove only $OH_VERIFY_RUN/private/home/.agents/skills/skill-creator; navigate to /skills?q=skill-creator, reload and count (1, public only). Restore desktop before continuing. No profile, model call or real key is needed.
Exact CLI recipe & bug checks
Expected: a personal and a built-in definition with the same name appear once each with their own source; This project excludes the built-in card, and Built in excludes the personal card. Start with no personal skill-creator fixture. Run control-openhands browser viewport desktop, control-openhands browser goto /skills, control-openhands browser fill 'testid=skills-search-input' skill-creator, and control-openhands browser wait-url '[?&]q=skill-creator'; control-openhands browser count 'testid=skill-card-skill-creator' is 1. Arrange control-openhands fixture skill --name skill-creator --body 'QA_F18_LOCAL_COLLISION. This local fixture is never loaded into an agent.'; note its run-owned path for cleanup. Reload and count: expected 2 distinct sources. Read them with control-openhands browser eval "[...document.querySelectorAll('[data-testid=skill-card-skill-creator]')].map(c=>c.querySelector('[data-testid=skill-source-skill-creator]')?.title)". Start control-openhands browser record start --feature F18.facets --name desktop-same-name. Run control-openhands browser click 'testid=skill-facet-source-project' (expected one local card), control-openhands browser click 'testid=skills-clear-filters', control-openhands browser click 'testid=skill-facet-source-public' (expected one public card), Clear filters, then This project again. Read the count and source array after each filter. Capture control-openhands browser screenshot --feature F18.facets --name desktop-repeated-project and stop recording. Repeat on phone after control-openhands browser viewport phone and control-openhands browser goto '/skills?q=skill-creator': before each filter open testid=skills-filters-button, scope the facet click to testid=skill-filters-modal >> testid=skill-facet-source-project or ...-public, and close with testid=skill-filters-modal-done. Clear through testid=skill-filters-modal-clear while the modal is open. Record and capture under F18.facets with phone names. **Known failure (reproduced 2026-10-12, #18287): the public-filter step keeps an extra local card; returning to This project** shows two cards with the identical local source path on Local macOS browser, desktop and phone. Origin unconfirmed; this does not establish a causal PR or verify the original Windows installer environment. Cleanup: after checking that the fixture path is inside this run and its file contains QA_F18_LOCAL_COLLISION, remove only $OH_VERIFY_RUN/private/home/.agents/skills/skill-creator; navigate to /skills?q=skill-creator, reload and count (1, public only). Restore desktop before continuing. No profile, model call or real key is needed.
Filters modal on a phoneF18.filters-modalMapped behavior
Scenario F18--filters-modal-on-a-phone · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- SetupUse the phone viewport.
- DoOpen /skills.
- DoChoose “Skills filters”.
- DoChoose “Skill filters modal” / “Skill facet state enabled”.
- DoChoose “Skill filters modal done”.
- DoChoose “Skill filters modal clear”.
Expect
below 768 px the rail is replaced by a Filters button with an active-count badge that opens the same facets in a modal with Clear filters and Close.
Exact CLI recipe & bug checks
Run control-openhands browser viewport phone, control-openhands browser goto /skills, control-openhands browser visible 'testid=skill-facet-rail' (false), control-openhands browser click 'testid=skills-filters-button', control-openhands browser count 'testid=skill-filters-modal-clear' (0), control-openhands browser click 'testid=skill-filters-modal >> testid=skill-facet-state-enabled' (URL ?state=enabled, Clear filters appears), control-openhands browser screenshot --feature F18.filters-modal --name phone, control-openhands browser click 'testid=skill-filters-modal-done', then control-openhands browser text 'testid=skills-filters-button' (Filters and badge 1) and the summary (12 result(s) on a fresh run, 13 result(s) with the qa-hello fixture). Reopen and control-openhands browser click 'testid=skill-filters-modal-clear': URL /skills and the Clear button disappears; Escape closes the modal.
Phone layout and hubF18.phoneMapped behavior
Scenario F18--phone-layout-and-hub · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoChoose “Skills add skill”.
- DoOpen /customize.
- DoChoose “Extensions mobile hub” / “Sidebar extensions /skills”.
- DoChoose “Sidebar mobile back”.
- SetupUse the desktop viewport.
Expect
at 390 px the page, cards and both modals fit without horizontal overflow; /customize shows a hub that links to Skills and the header back button returns to it.
Exact CLI recipe & bug checks
At the phone viewport run control-openhands browser bbox 'testid=skills-page' (pageHorizontalOverflow false), control-openhands browser screenshot --feature F18.phone --name list, open testid=skill-card-docker and control-openhands browser bbox 'testid=skill-detail-modal' (insideViewport true), Escape, control-openhands browser click 'testid=skills-add-skill-button' and control-openhands browser bbox 'testid=add-skill-modal' (insideViewport true), Escape. Then control-openhands browser goto /customize, control-openhands browser text 'testid=extensions-mobile-hub' (Customize, MCP Servers, Skills, Plugins, Apps), control-openhands browser click 'testid=extensions-mobile-hub >> testid=sidebar-extensions-/skills' --expect-url '/skills(\?|$)' and control-openhands browser click 'testid=sidebar-mobile-back-button' --expect-url '/customize(\?|$)'. Return with control-openhands browser viewport desktop.
Cloud linkF18.cloud-linkBlocked
Scenario F18--cloud-link · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoBlocked: needs a Cloud backend. Expected:
control-openhands browser attr 'testid=sidebar-extensions-/skills' hrefis<cloud host>/settings/skillswithtarget_blank, and the Plugins and Apps items are hidden.
Expect
with a Cloud backend the Customize Skills item opens the Cloud skills page in a new tab instead of /skills (blocked here).
Exact CLI recipe & bug checks
Blocked: needs a Cloud backend. Expected: control-openhands browser attr 'testid=sidebar-extensions-/skills' href is <cloud host>/settings/skills with target _blank, and the Plugins and Apps items are hidden.
RestoreF18 · setup / cleanupSupport step
Scenario F18--restore · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoLeave
add-javadocandflarglebargleoff andqa-helloon; the fixture skill and the conversations vanish with the run's private state.
Expect
Complete this support step before continuing to the next scenario.
Exact CLI recipe & bug checks
Leave add-javadoc and flarglebargle off and qa-hello on; the fixture skill and the conversations vanish with the run's private state.
Colliding skill names do not leave ghost cardsF18.card · F18.facetsKnown failure
Scenario F18--colliding-skill-names-do-not-leave-ghost-cards · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- SetupCreate the disposable skill fixture specified in the recipe.
Expect
each card shows icon, name, source, a two-line description and pills; extra pills collapse into a +N popover; click, Enter or Space opens the detail modal.
the desktop facet rail (State, Recommendation, Source, Category, Type) filters with per-row counts, writes canonical URL params, disables zero-count rows and offers Clear filters; deep links apply and unknown values are dropped.
Known bug observations
Known failure, reproduced 2026-10-12 (#18287): skills-result-summary says 1 result(s) while two then three stale same-source skill-creator cards remain. Read their source titles and screenshot the mismatched list. Remove only the own colliding fixture and reload to clear ghosts.
Exact CLI recipe & bug checks
Arrange fixture skill --name skill-creator --trigger qa-collision-ping in the run's isolated personal folder beside the bundled same-name skill, plus qa-toggle. Reload /skills, search skill-creator, then search qa-toggle; repeat the two searches. Expected: only the one matching toggle card and no skill-card-skill-creator. Known failure, reproduced 2026-10-12 (#18287): skills-result-summary says 1 result(s) while two then three stale same-source skill-creator cards remain. Read their source titles and screenshot the mismatched list. Remove only the own colliding fixture and reload to clear ghosts.
Skill state survives a delayed sampleF18.toggleMapped behavior
Scenario F18--skill-state-survives-a-delayed-sample · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoWith an own enabled
qa-togglefixture, use its real switch with--hover-first 250when removing; drive disabled → enabled → disabled, observe for eight seconds and readaria-checked, settingsdisabled_skillsand a reload. Expected: the switch stays false and the server keepsqa-toggledisabled across reload. Repeat the cycle, then restore enabled; a bounded sample does not establish every possible delay.
Expect
the card's plus/check toggle switches a built-in skill on or off; the choice survives a reload (enabled_skills allow-list on a local backend).
Exact CLI recipe & bug checks
With an own enabled qa-toggle fixture, use its real switch with --hover-first 250 when removing; drive disabled → enabled → disabled, observe for eight seconds and read aria-checked, settings disabled_skills and a reload. Expected: the switch stays false and the server keeps qa-toggle disabled across reload. Repeat the cycle, then restore enabled; a bounded sample does not establish every possible delay.
Backend switch refreshes skill dataF18.pageKnown failureKnown issue
Scenario F18--backend-switch-refreshes-skill-data · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoUse Local and the disposable
QA_SCOPEidentity of the own stack described in F17. Load/reload the Local Skills page once, then arrange a new personal fixtureqa-scope-new-3; clear the skills network log. Switch to QA_SCOPE through the actual selector and inspect the still-mounted Skills page, with nogotoor reload between the switch and observation. Expected: a skills request for the selected backend and the new card. Known failure, reproduced twice 2026-10-12 (#16893): zero/api/skillsrequests and card count 0; a subsequent reload requests actual data and shows the card (1). Repeat in the other direction with a new markerqa-scope-new-4. Two registry identities / real fixture changes share one physical run here; separate Cloud data and conversation-scoped caches are further branches. Restore Local and remove only the fixtures/alias.
Expect
the page shows the title, description, the notice "Skill changes apply to new conversations only.", an Add skill button, a result count, the facet rail and the card grid. (Empty state "No skills found." and loading skeletons are not reachable on a local backend.)
Known bug observations
Known failure, reproduced twice 2026-10-12 (#16893): zero /api/skills requests and card count 0; a subsequent reload requests actual data and shows the card (1). Repeat in the other direction with a new marker qa-scope-new-4. Two registry identities / real fixture changes share one physical run here; separate Cloud data and conversation-scoped caches are further branches. Restore Local and remove only the fixtures/alias.
Linked known issues
Known issue #15458: The global Skills page can list AGENTS.md files from cloned automation-run worktrees. The report says the discovery trigger is unknown; replay needs the affected automation workspace/discovery-root layout or captured request parameters, rather than only a normal project-skill fixture. (F18.page, F18.project-skill).
Exact CLI recipe & bug checks
Use Local and the disposable QA_SCOPE identity of the own stack described in F17. Load/reload the Local Skills page once, then arrange a new personal fixture qa-scope-new-3; clear the skills network log. Switch to QA_SCOPE through the actual selector and inspect the still-mounted Skills page, with no goto or reload between the switch and observation. Expected: a skills request for the selected backend and the new card. Known failure, reproduced twice 2026-10-12 (#16893): zero /api/skills requests and card count 0; a subsequent reload requests actual data and shows the card (1). Repeat in the other direction with a new marker qa-scope-new-4. Two registry identities / real fixture changes share one physical run here; separate Cloud data and conversation-scoped caches are further branches. Restore Local and remove only the fixtures/alias.
Persistence folder skill loadsF18.personal-skill-dirsMapped behavior
Scenario F18--persistence-folder-skill-loads · Permalink · Source
Given
Use F18’s prerequisites and the common setup. Follow the family’s order when a script uses an earlier fixture or says “as above”.
Do — reading outline
- DoArrange a valid
qa-legacyfixture underprivate/skills/qa-legacy/SKILL.md(move the CLI-created personal fixture there;privateis this run's persistence directory). Reload Skills, read the card/source path and start a chat through Home withqa-legacy-ping. Do not run tools; reply in one line.using the dummy loopback profile. Expected: the card appears and the initialized SystemPromptEvent/user message includesqa-legacyin available/activated skills. This checks discovery and activation before the model call. Remove the own skill afterwards.
Expect
personal skills load from both user folders: ~/.agents/skills/<name>/SKILL.md and the persistence folder's skills/<name>/SKILL.md (~/.openhands/skills, which a Canvas stack moves to $OH_PERSISTENCE_DIR/skills). Each shows on the Skills page with its path and fires on its trigger in new conversations.
Exact CLI recipe & bug checks
Arrange a valid qa-legacy fixture under private/skills/qa-legacy/SKILL.md (move the CLI-created personal fixture there; private is this run's persistence directory). Reload Skills, read the card/source path and start a chat through Home with qa-legacy-ping. Do not run tools; reply in one line. using the dummy loopback profile. Expected: the card appears and the initialized SystemPromptEvent/user message includes qa-legacy in available/activated skills. This checks discovery and activation before the model call. Remove the own skill afterwards.
Family notes, prerequisites for known issues & limits Known issue
sidebar-skills-linkis labelled Customize and lands on/mcpon desktop (/customizeredirects); Skills is one more click. Test ids embed the route: quote'testid=sidebar-extensions-/skills'.- A
browser clickon an enabled card toggle is lost when the pointer was not already over it:pointerenterswaps the checkmark icon for the X icon between mousedown and mouseup, so Chrome fires no click and noPATCH /api/settingsgoes out (#17941).browser hoverthe toggle first (or click twice). Turning a skill on is not affected. Recorded asfailonF18.toggle. - The card toggle and the facet rows are
role=switch/role=checkboxbuttons: readaria-checked. The modal switch is ansr-onlycheckbox (keyboard reachable): click the row text and read.checkedwithbrowser eval. - Typing writes
?q=withreplaceafter a 300 ms debounce, so readbrowser urla second later; facet clicks push a history entry immediately. - Facet counts are disjunctive: a row counts skills matching the other groups' selections, so Disabled reads
0(and is disabled) while Recommended is checked. - Groups with fewer than two values are hidden (Source and Type on a fresh run) unless the URL selects one of their values.
- The run's
HOMElives under/tmp, so a skill in~/.agents/skillsis filed under Source This project (source=project), not Personal; with a real/home/<user>or/Users/<user>home it would be Personal. conversation eventsrows show a userMessageEvent's activated skills asskills: [...](since #18162).api GET '/api/conversations/<id>/events/search?limit=5' --max-bytes 2000000shows the rawactivated_skills, but only when the user row is among those five events. On 2026-10-08 it was not: the first five were the system prompt and state updates.- Skill lists are cached for 10 minutes:
browser reload(orgoto) after the agent creates a skill file, or the new card does not appear. navigator.clipboard.readText()throughbrowser evalhangs (no clipboard permission in the page); read what a copy button wrote withcontrol-openhands browser clipboardand assert the button'saria-labeltoo. The app runs on127.0.0.1(a secure context), so the unguardedclipboard.writeTextcalls cannot be tested for insecure-origin failures here.- With the Agent Server stopped, reloading
/skillsshows the "Manage backends / Disconnected" screen instead of the page: load the page first, then stop the service. After the failed save the card keeps showing the new state until a reload (#17941). - Escape from the modal's focused
+Noverflow trigger closes only the popover and restores trigger focus; the next Escape closes the detail modal. The card popover uses the same Escape behavior. - The Add skill example, its Copy button and the add-skill Use skill message share the
.../skills/code-reviewpath. - Skill changes apply to new conversations only; an open conversation keeps the skills it started with.
conversation events --grepsearches whole events, the user's message text included: a trigger that contains the skill name (qa-gone-pingforqa-gone) matches the userMessageEventeven when the skill is not loaded. Give fixture skills a trigger that does not contain their name, or grep for<name>qa-gone</name>.fixture skill --repowrites the projectSKILL.mdand records it in the repo only with--commit(committed.sha; a re-run reportsunchanged); a New Worktree conversation checks out the repo's commits, so pass--commit. Workspace skills are loaded per conversation and never appear on/skills.- Without an LLM key the skill set is still observable: the
SystemPromptEvent(available skills), the userMessageEvent(activated_skills) and the chat's Skill Ready row (generic-event-message-title; Expand lists the skills underTriggered Skill Knowledge:) all exist before the first model call, after which the conversation ends inerror. - Known issue #16298: A Slack-monitor setup can leave the agent without the expected skill. The report omits the concrete monitor setup and loaded skill name/version; reproducing the runtime also needs an approved model and a configured Slack integration. Listing the bundled catalog alone cannot settle it. (
F18.skill-in-chat). - Known issue #15458: The global Skills page can list AGENTS.md files from cloned automation-run worktrees. The report says the discovery trigger is unknown; replay needs the affected automation workspace/discovery-root layout or captured request parameters, rather than only a normal project-skill fixture. (
F18.page,F18.project-skill).
Before you start
Use an OpenHands checkout at the source revision, with Node 24+, installed npm dependencies, uv / uvx, and a supported Chromium browser. The verification skill is the complete setup contract.
export PATH="$PWD/.agents/skills/verify-openhands/scripts:$PATH"
export OH_VERIFY_RUN=$(control-openhands launch --new --print-run)
control-openhands doctor
control-openhands onboard --skip
# Configure a model only for a scenario that needs one:
control-openhands llm preset deepseek
# When the run is complete:
control-openhands stop
F01 walks onboarding instead of skipping it. No-model checks run before model configuration. Keep OH_VERIFY_RUN set so commands address your own run. Desktop is 1440 × 1000; phone is 390 × 844. Model-backed checks need the configured provider key and budget.
Recipes run in source order within each family and can share fixtures. The page preserves that order; “as above” is a dependency on an earlier case. Setup commands and API writes arrange state. They do not prove a UI path. Follow the family cleanup steps.
A known failure preserves the intended expectation beside the observed defect. Executable known-bug blocks use xfail for a reproduced symptom and xpass when it stops reproducing. Cloud accounts, enterprise capabilities and other unavailable prerequisites remain explicit. “Not run” is different from “blocked”.
The maintained map describes application behavior at its pinned revision. Once a fix is verified in the application version under test, remove the obsolete failure note and keep the normal expectation. Dated evidence belongs in the backfill ledger.
Read the full map index and coverage boundaries, the CLI guide, or the earlier illustrated edition. The verification approach adapts Lauren Tan’s (@poteto) pstack verification-skill generators; attribution and adaptation.