FinishTool: before → after
Just the agent-facing change: what the finish tool's prompt and definition looked like before, and after this PR. Everything else from #4465 (system outcomes, storage, automations) is left out on purpose.
1What changed, exactly
Three edits, one behavior change. (1) The tool description (the prompt the LLM reads) is reworded. (2) create() attaches a structured FinishTaskOutcomeResponse via set_response_schema(...). (3) FinishAction is unchanged — still just message: str. The structured task_outcome rides on a response schema, not on the action.
| Part | Before | After |
|---|---|---|
FinishAction | message: str | unchanged — message: str |
| Tool description / prompt | "the message should include: summary, next steps, explanation, follow-ups" | "use message for the summary; put status/blockers/user-action/confidence in task_outcome" |
| Response schema | none | set_response_schema(FinishTaskOutcomeResponse) |
| Import | — | from …task_outcome import FinishTaskOutcomeResponse |
2The prompt — TOOL_DESCRIPTION
This is the text the model reads to decide how to call finish. The opening ("Signals the completion… use when…") is identical; only the trailing instruction block changed.
Signals the completion of the current task
or conversation.
Use this tool when:
- You have successfully completed the
user's requested task
- You cannot proceed further due to
technical limitations or missing info
The message should include:
- A clear summary of actions taken and
their results
- Any next steps for the user
- Explanation if you're unable to complete
the task
- Any follow-up questions if more
information is needed
Signals the completion of the current task
or conversation.
Use this tool when:
- You have successfully completed the
user's requested task
- You cannot proceed further due to
technical limitations or missing info
Use `message` for a concise user-facing
summary of the result. Put structured
status, blockers, user-action
requirements, and confidence in the
`task_outcome` fields.
message; machine-readable judgement moves to task_outcome. finish.py:483The definition — action unchanged, schema added
return [
cls(
action_type=FinishAction,
observation_type=FinishObservation,
description=TOOL_DESCRIPTION,
executor=FinishExecutor(),
annotations=ToolAnnotations(
title="finish", …
),
)
]
# no structured output
tool = cls(
action_type=FinishAction,
observation_type=FinishObservation,
description=TOOL_DESCRIPTION,
executor=FinishExecutor(),
annotations=ToolAnnotations(
title="finish", …
),
)
return [tool.set_response_schema(
FinishTaskOutcomeResponse)]
The new schema is one field wrapping the outcome model:
class FinishTaskOutcomeResponse(BaseModel):
task_outcome: TaskOutcome # status, summary, blockers[], confidence, needs_user_action
set_response_schema merges the schema's fields into the tool's parameter schema (and rejects any name that collides with message or reserved fields). So the model emits message and task_outcome in a single finish call — no second tool, no extra round-trip. tool.py: set_response_schema
4What the model now sees when calling finish
{
"message": "Done — added the endpoint
and tests pass."
}
{
"message": "Done — added the endpoint
and tests pass.",
"task_outcome": {
"status": "success",
"summary": "Endpoint added; 12 tests
green.",
"blockers": [],
"needs_user_action": false,
"confidence": 0.9
}
}
On the way in, task_outcome_from_finish() stamps source="agent", reported_at=now, terminal_reason="finish_action" and stores it on the conversation. task_outcome.py:88
5The v0 risk worth an eval
Engel's point on the thread: in v0, some LLMs treated finish as required after every step — they'd call it to ask a question or reply, not only to end. Making finish the carrier of a rich structured schema makes it a more "attractive" tool to call. Worth an eval on a cheaper/weaker model to confirm this reword doesn't nudge premature or over-eager finish calls back. The mitigation in-PR is that the schema is additive and message stays the primary field, but the prompt no longer lists "follow-up questions" as a finish use — which is arguably good (discourages finish-to-ask) but changes behavior.
1fae5eb1fa583d23fe63b1317c9b7fac4921e40d · after be17960b7846fdfc4bd66ce2fb6dd40bd890f6b3 · code read, not a live run.🐾 smolpaws · for Engel · PR #4465 · full-PR page