OpenHands · software-agent-sdk · PR #4465

One TaskOutcome, two sources

Rohit's PR attaches a structured task outcome to the finish tool. Engel's question: does it merge the two categories of outcome — agent-reported vs system/harness failure — and how? Short answer: yes, into a single type discriminated by a source field.

fix: attach task outcome to finish tool · base main ← head task-outcome-reporting · 28 files, +1502 / −49 · head be17960
github.com/OpenHands/software-agent-sdk/pull/4465 · links automation#323 · 🐾 smolpaws for Engel

1The answer, in one screen

They are unified, not kept separate. There is one model — TaskOutcome — with a field source: Literal["agent","system"]. Both an agent's self-assessment and a harness/LLM failure land in the same shape; only source (plus terminal_reason) records which produced it. task_outcome.py:37

source = "agent" The agent decided it was done (or stuck)

Produced when the agent calls finish. The tool now carries a structured task_outcome response schema, so the model reports status, blockers, needs_user_action, confidence alongside its user-facing message. terminal_reason="finish_action".

source = "system" The run died before the agent could finish

Produced by the runtime when a harness/LLM failure ends the run (auth, quota, rate-limit, config, transient, exception, timeout, max-iterations, stuck). Synthesized from ErrorClassification, not the model. Fallback so a conversation always has an outcome.

How the "merge" actually works: the two are merged at the type level (one schema, one storage slot on the conversation), and kept distinguishable at the data level (source + terminal_reason). Downstream consumers — the automation service especially — read one field and branch on source/status if they care. There is no separate "system outcome" object and no separate reporting tool (the old standalone report_task_outcome tool is removed).

2One type, two sources

The whole contract is one small file. The status vocabulary and the discriminator are the parts that answer Engel's question:

TaskOutcomeStatus = Literal["success", "partial_success", "blocked", "failed", "unknown"]
TaskOutcomeSource = Literal["agent", "system"]   # ← the discriminator

class TaskOutcome(BaseModel):
    status: TaskOutcomeStatus              # semantic assessment
    summary: str
    blockers: list[TaskOutcomeBlocker] = []
    confidence: float | None = None
    needs_user_action: bool = False
    source: TaskOutcomeSource = "agent"    # agent | system
    reported_at: datetime | None = None
    terminal_reason: str | None = None     # finish_action | exception | timeout | max_iterations | stuck | <code>

A TaskOutcomeBlocker is {type, message, recoverable} — the same blocker shape is used whether the blocker was named by the agent or derived from a failure classification. task_outcome.py:22

3Before → after: who writes the outcome

Before this PR there was no unified outcome on the conversation. A short-lived standalone report_task_outcome tool existed (opt-in, agent-only) and harness failures had no structured semantic outcome at all. After: both roads lead to one field.

BEFORE agent calls finish message only report_task_outcome standalone tool, opt-in harness failure no semantic outcome no single owner ·consumers guess AFTER FinishTool + task_outcome schema runtime error path ErrorClassification TaskOutcome source: agent|system on ConversationState from_finish() from_error()
The delta (orange/teal): two producers now converge on one TaskOutcome stored on the conversation. The old opt-in tool is gone; harness failures gain a first-class outcome.

4The agent path — source="agent"

The finish tool used to just take a free-text message. Now it attaches a structured response schema, so the model fills in the semantic fields as part of finishing — no extra tool, no extra LLM call.

# finish.py — the tool now carries a structured response schema
return [tool.set_response_schema(FinishTaskOutcomeResponse)]

# task_outcome.py — normalize what the agent reported
def task_outcome_from_finish(outcome: TaskOutcome) -> TaskOutcome:
    return outcome.model_copy(update={
        "source": "agent",
        "reported_at": utc_now(),
        "terminal_reason": "finish_action",
    })

The tool description now tells the model: use message for the user-facing summary, put status/blockers/confidence in task_outcome. finish.py:50 · task_outcome.py:88

5The system path — source="system"

When the run ends without the agent finishing, the runtime synthesizes an outcome from the failure. This is the part that makes the abstraction "unified": a harness failure is expressed in the same vocabulary the agent uses.

def task_outcome_from_error(*, code, detail, classification=None, terminal_reason=None):
    status = "failed"
    if classification is not None:
        recoverable = classification.retryable or classification.user_action != "none"
        needs_user_action = classification.user_action == "settings"
        if classification.kind in {AUTH, QUOTA, CONFIG, RATE_LIMIT, TRANSIENT}:
            status = "blocked"          # recoverable → "blocked", not "failed"
    return TaskOutcome(status=status, summary=summary,
        blockers=[TaskOutcomeBlocker(type=blocker_type, message=summary, recoverable=recoverable)],
        needs_user_action=needs_user_action, source="system",
        reported_at=utc_now(), terminal_reason=terminal_reason or code)

The mapping decision worth noting for review: recoverable harness failures (auth / quota / config / rate-limit / transient) become status="blocked", everything else "failed". So blocked spans both "agent is blocked and needs help" and "harness is blocked but retryable" — the reader tells them apart by source. task_outcome.py:99

6Where it lives & who reads it

  • Stored once, on ConversationState.task_outcome — persisted in base_state.json like the rest of runtime state. state.py:222
  • Recorded via a callback wrapper — LocalConversation wraps its event callback with _task_outcome_recording(...), so finishing (or failing) updates the field automatically. local_conversation.py:413
  • Exposed on the API — agent-server ConversationInfo gains a task_outcome field, and BaseConversation a task_outcome property, so SDK / remote / server consumers read it uniformly. models.py:206
  • Forwarded to automations, opt-in — BaseWorkspace can attach the outcome as agent_outcome on the completion callback, but only when the automation service advertises support via AUTOMATION_CALLBACK_SUPPORTS_AGENT_OUTCOME (older services reject unknown fields). workspace/base.py:86

This is why the PR message frames it around automations: the whole point is that a background/automation run always ends with a machine-parsable outcome — whether the agent declared it or the harness died — so downstream services (e.g. "email the user that conversation X is blocked because Y") can act without per-automation configuration.

7Changed files

28 files; the ones that carry the design:

FileRole
sdk/task_outcome.pynewThe unified model + both constructors (from_finish, from_error).
sdk/tool/builtins/finish.py✏️Attaches FinishTaskOutcomeResponse schema; reworded description.
sdk/conversation/state.py✏️New task_outcome field on persisted state.
sdk/conversation/impl/local_conversation.py✏️_task_outcome_recording callback; agent-finish + error → outcome.
sdk/conversation/base.py / remote_conversation.py✏️task_outcome property across conversation impls.
agent_server/models.py✏️task_outcome on ConversationInfo (API surface).
sdk/workspace/base.py✏️Opt-in agent_outcome on automation completion callback.
examples/…/57_task_outcome_missing_mcp.py · tests/sdk/tool/test_finish_task_outcome.pynewWorked example (blocked-on-missing-MCP) + 395-line test.
Show the core diff — finish.py (tool gains the schema)
-        return [
-            cls(action_type=FinishAction, observation_type=FinishObservation,
-                description=TOOL_DESCRIPTION, executor=FinishExecutor(), …)
-        ]
+        tool = cls(action_type=FinishAction, observation_type=FinishObservation,
+            description=TOOL_DESCRIPTION, executor=FinishExecutor(), …)
+        return [tool.set_response_schema(FinishTaskOutcomeResponse)]

8Open questions for review

Reading the diff, the discussion points that map to the thread (Xingyao: "not sure we need that / prefer to delete"; Engel: "we did have finish-tool trouble in v0"):

  • Overloading finish. Engel's v0 memory — some LLMs treated finish as mandatory-after-every-step. Adding a required-looking structured schema to finish could nudge that behavior back on weaker models. Worth an eval on a cheaper model before merge.
  • One status vocabulary, two meanings for blocked. Agent-blocked and harness-blocked share the enum value and separate only by source. Fine if every consumer reads source; a foot-gun if any consumer keys on status alone.
  • Additive surface. New field on persisted ConversationState + ConversationInfo + a workspace env flag. Xingyao's "hard to remove once added" caution applies; the opt-in env gate for automations is the mitigation.

Net: the answer to the thread question is unambiguous in the code — the two outcome categories are unified into one TaskOutcome type, produced by two constructors, discriminated by source. The design debate is not "are they merged" but "should finish be the carrier, and is one status enum across both origins the right granularity."

Built with the show-me skill · grounded to head be17960b7846fdfc4bd66ce2fb6dd40bd890f6b3 · from a code read of PR #4465, not a live run.
🐾 smolpaws · for Engel · PR #4465