SmolPaws · design

The Secretary

Fuse the insider SmolPaws cat with a board that manages the user's conversations — dispatch, track, act — talk to it in realtime voice, and ship the whole thing as an Agent Canvas Skin. The cat stops being a corner mascot and becomes the face of a secretary that runs your other agents for you.

STATUS: DESIGN   epic smolpaws-08f (insider)   epic smolpaws-3e1 (voice)

1The idea

The insider cat today is a presence: a corner avatar, a "call the cat" button, an identity that knows it runs on a local agent-server. Useful, but small. The next step gives it a job: be the user's secretary — the one agent whose whole purpose is to manage the user's other agents and conversations, so the human stops babysitting a dozen chats.

Naming: we call ours the Secretary. The upstream lineage this builds on is Robert Brennan's "manager" (rbren/vibe-manager, the productized OpenHands/app-acm); those keep their name. Ours is the secretary — a named, avatar'd cat, closer in spirit to AmpCode's "Puck" than to a kanban tool.

Three things combine, and each already half-exists:

PieceTodayBecomes
The catCorner avatar + call-the-cat, insider identity shippedThe face and voice of the secretary.
The viewNothing (a launcher)A board of the user's conversations it manages.
The brainFixed toolbox in a spikeThe real OpenHands agent via /v1/chat/completions.

2Lineage: what the "manager" already proved

rbren/vibe-manager is the reference. A kanban SPA plus a per-workspace Manager automation (a 1-minute cron) that dispatches and tracks worker agent conversations. Its own words, from the Slack thread that kicked this off:

"I'll have 17 ideas for fixes, but managing 17 agents is too much… So I built a Manager! …a 1m cron automation that checks the board and kicks off conversations as necessary. It funnels similar tickets into the same convo, and knows to hold off on tickets that will conflict with WIP. It also knows to keep at most N conversations going at once."

The mechanics worth stealing (all proven in that repo):

  • A board per workspace — tickets flow pending → in_progress → needs_input / finished / verified.
  • A manager is a conversation that spawns worker conversations — the deterministic cron only wakes the manager agent when the board's fingerprint changes, with loop guards so it can't churn.
  • Live action summaries — the manager watches each worker's websocket and shows its latest tool action on the card.
  • Mechanical transitions — PR merged → finished, push verified → finished — done in code, not by the LLM.
  • It only manages what you put on the board; conversations created elsewhere don't appear. Good boundary.

This got productized as OpenHands/app-acm ("Agent Canvas Manager") and made the default Skin. So the pattern is blessed; our job is to give it a cat's face, a voice, and the SmolPaws shared memory — and to scope it to what one person actually needs.

3The agent as a brain — /v1/chat/completions

The unlock that makes the voice (and the secretary) real: the agent-server already exposes an OpenAI-compatible chat endpoint that runs the whole agent as an LLM.

POST /v1/chat/completions      (X-Session-API-Key)
{ "model": "openhands_deepseek-v4-flash",
  "messages": [{"role":"user","content":"Run: echo hi. Tell me the output."}],
  "stream": true }

Verified live: it doesn't just chat — it runs the agent's tools inside the turn. Asked to run a shell command, it ran it; asked "how many conversations do I have", it did the work and answered with the real number. Prompt tokens ~12–24k per call (that's the full agent system prompt). Streaming works.

This is the "replace its brain" Engel wanted: any front-end that speaks OpenAI chat can borrow the OpenHands agent as its model. The realtime voice model calls one tool — ask_the_agent(request) — and the agent does the rest.

Related and useful: the same server has native ChatGPT-subscription auth (/api/llm/subscription/openai/* device flow, /api/llm/subscription/openai/models) — the secretary can run on a subscription, not just an API key.

4Voice: what the spike already showed

The realtime-voice spike (smolpaws-3e1.1, odie branch spike/realtime-voice) is the secretary's mouth and ears. Path A — an OpenAI Realtime model as the conversational brain, function-calling into our agent — was live-tested and feels good. What's proven:

  • Full-duplex WebRTC voice with barge-in, in-browser mic/speaker.
  • Both-side transcripts captured (user + cat) → the written chat record.
  • ask_the_agent → /v1/chat/completions → real agent action, spoken back.
  • Auth on Engel's ChatGPT subscription (Codex token mints the Realtime ephemeral key), API key only as fallback.
Open latency item. A full agent turn is ~12 s — too long to sit through silently. Fixes, both cheap: stream /v1/chat/completions so the cat starts talking as tokens arrive, and/or a spoken "let me check…" filler the instant a tool call fires. Trivial lookups (count/list) keep a fast direct-REST path.

5The secretary board

The cat's new home view. It is the manager board, re-skinned as SmolPaws and scoped to one person. The good news: we already have a working skeleton — Purr Projects (my current Canvas skin) is a read-only board over the agent-server's conversation list. It becomes the secretary board by adding tickets and actions.

Secretary board — columns are conversation state Pending In progress Needs input Done 🐾 Fix login queued 🐾 Transpile ● running: editing file… sonnet · conv 81c4… 🐾 Deploy ? push to main? 🐾 Docs ✓ PR #16240 merged Each card = a ticket bound to a conversation. The cat dispatches, watches WS status, and moves cards.
The board is the view; the cat is the agent behind it.

What Purr Projects already gives us (~/workspace/skin): a Node app with npm run start, a skin.yaml with the SmolPaws theme, and handlers that read /api/conversations/search + per-conversation events/search with the injected X-Session-API-Key, never exposing it to the browser. What it lacks: a ticket store, dispatch, and the state machine. Those are the manager's contribution.

Engel's steer: our Purr Projects board "is not pretty" — so the secretary board gets a fixed, deliberate layout, not the current auto-grouped grid. Design the columns and card first; wire second.

6Ship it as a Skin

The delivery vehicle is the Skins mechanism from OpenHands/OpenHands#16232 (open, all-hands-bot). A Skin is a git-backed per-instance UI: a repo with skin.yaml + a web app started with npm run start on OPENHANDS_SKIN_PORT, surfaced as the top sidebar item → an iframe tab, and it's the default landing tab. One skin per instance. The static server mounts a /skin-api management surface and reverse-proxies /skin-app (HTTP + WS).

That's exactly the shape Purr Projects already targets, so the secretary is "just" a richer skin. Engel's constraint on adopting the PR:

Pick up the PR as a single squashed commit; do not modify the PR itself. We vendor the Skins runtime as one commit onto our branch and build on top, leaving #16232 untouched upstream. If it changes upstream, we re-squash.

7The endpoint is a conversation, not a completion

Corrected (verified in the SDK + live). An earlier draft here framed /v1/chat/completions as a "stateless, ephemeral completion" that can't show on the board. That was wrong. In agent-sdk (openhands-agent-server/.../openai/service.py) the endpoint calls start_conversation() and returns get_agent_final_response(): it runs the whole OpenHands agent — tools and all — as one turn, and the turn is a real, persisted conversation. It returns an X-OpenHands-ServerConversation-ID header, and you can reuse it on the next call to continue the same conversation.

So the two "wirings" aren't stateless vs stateful — both are agent conversations. The real difference is only how you drive and observe them:

(a) /v1/chat/completions(b) conversation API + event stream
What it isThe full agent, one synchronous turn per call. Runs tools internally; returns only the final assistant text.The same agent, driven message-by-message with the live event stream (every action/observation).
StateA real conversation — persisted, retrievable, reusable via the returned server-conversation id.A real conversation — you hold the id from the start and watch it live.
Tool calls visible?No — they happen, but you get the final text only (that's the point: clean answer for voice).Yes — you see each tool call/result as events.
Shows on board?Yes, if we want it to — it's a conversation; tag it on create and it gets a card like any other.Yes — it's the same kind of conversation.
Best forThe voice brain: ask once, get the final answer back to speak. Simplest possible call.The board's live on-card action summary, and dispatched workers you watch progress on.
What this unlocks. The voice cat can use (a) as its brain — one call, final answer, spoken back — and that answer already lives in a real conversation the board can surface. The spoken reply being a shorter summary of the written answer isn't a bug: voice is realtime and rephrases; the full agent answer is the written record. We just have to make sure the final answer reaches the voice model (it does — the voice model function-calls the brain and receives the result to speak).

8Build plan

  1. Board view, fixed layout — design the columns + card (state, live action, model chip, links) as a deliberate layout; prototype against real conversations via the Purr Projects skeleton.
  2. Vendor Skins — squash #16232 onto our branch as one commit; confirm install/iframe/default-tab works locally.
  3. Ticket store + state machine — add tickets bound to conversations; the pending→in_progress→needs_input→done transitions, mechanical where possible.
  4. Dispatch — the secretary (a smolpaws Agent Profile conversation) creates worker conversations from tickets; caps concurrency; funnels/holds conflicts.
  5. Live status — reuse Purr Projects' events read + a WS watch for the on-card action summary.
  6. Voice on top — graft the realtime spike in: the cat answers and dispatches by voice, streaming + filler for latency.
  7. Shared memory — the secretary reads/writes the SmolPaws store (smolpaws-08f.2) so it remembers across sessions and faces.

9Risks & open questions

latency Full agent turn ~12 sStream /v1/chat/completions; speak a filler on tool-call start. Fast REST path for trivial lookups.
scope One board or per-workspace?rbren's insight was one board per workspace. For a single user the secretary may want one board across workspaces, tagged. Decide before the ticket store.
Skins PR Not merged yetVendor as a squashed commit; don't touch #16232. Re-squash if it moves. Watch canvas_version range.
loop guard Cron churnrbren hit a 50-run overnight loop; his fixes (fingerprint + capped retries + absorbing manager's own entries) are the template — don't reinvent, port the guard.
dispatch safety Voice can spawn workDispatch creates real conversations that run tools. Keep a confirm step for anything beyond read-only until trust is high; the spike's tools are read-only on purpose.
pretty Purr Projects layoutFixed, designed layout — not the current auto-grouped grid.
Bottom line. Every hard part is already proven somewhere: the manager pattern (rbren), the Skin delivery (#16232), the agent-as-brain (/v1/chat/completions), and the voice (our spike). The secretary is the assembly — a cat that runs your other agents, that you can talk to.