Programmatic Tool Calling — and the item the parser drops
GPT-5.6 can write JavaScript that runs in a hosted sandbox and orchestrates tool calls before returning one compact result. This page shows what the feature needs, what the OpenHands agent-sdk does today, and the precise two-sided gap that stops it working.
1What the feature is
In ordinary tool calling, every tool result flows back through the model: fetch ten things, and the model reads ten results. Programmatic Tool Calling (PTC) flips that. The model writes a small JavaScript program that runs in a hosted, isolated V8 runtime; the program calls tools (in parallel, in loops), reduces the intermediate data, and returns a single small result. Good for “fetch many, keep the two that matter.”
The runtime has top-level await but no Node, no network, no filesystem, no persistent state — it only reaches the outside world through the tools you enabled, and emits output via text(...) / image(...).
2What OpenAI needs in the request
Three additions to a normal Responses request:
tools = [
{"type":"function","name":"get_inventory",
"output_schema": {...}, # so generated JS can trust the shape
"allowed_callers": ["programmatic"]}, # direct | programmatic | both
{"type": "programmatic_tool_calling"} # the hosted tool
]And three new output item types come back in response.output:
| Item | Meaning |
|---|---|
program | The generated JS + a call_id + an opaque fingerprint used to resume. |
function_call with caller.caller_id | A tool call made by the program; caller_id points at the program's call_id. |
program_output | The program's final result + status. |
function_call, then return a function_call_output that echoes the original caller unchanged. That's how the hosted runtime resumes the right program.3What the SDK does today
The OpenHands Responses path is a straight pipe bracketed by two methods. Here's the outbound one:
# tool.py:497 — to_responses_tool()
return {
"type": "function",
"name": self.name,
"description": self.description,
"parameters": self._get_tool_schema(...),
"strict": False,
}No output_schema, no allowed_callers, and nothing injects the programmatic_tool_calling hosted tool. Notably, we already compute output_schema for to_mcp_tool() a few lines up (tool.py:407) — the data exists, it just isn't threaded to the Responses serializer.
The inbound method only understands three nouns:
# message.py:541 — from_llm_responses_output()
for item in output:
t = _get(item, "type")
if t == "message": ... # collect text
elif t == "function_call": ... # → MessageToolCall (caller NOT captured)
elif t == "reasoning": ... # → ReasoningItemModel
# no else — unknown items vanish4The two-sided gap
Here's the same PTC response flowing through today's parser, with real item types:
{caller:{caller_id}}
drops caller
program/program_output and never captures caller. Without echoing caller back on function_call_output, the hosted runtime can't resume the program — so a PTC loop can look half-working while being fundamentally unable to finish.The fix is therefore two-sided: thread output_schema/allowed_callers out and inject the hosted tool (responses_options.py), and teach the parser to preserve program, program_output, and caller end-to-end. Tracked in #4082.
4Quiz
Five questions to check you actually understood it — not trivia. Click an answer to see whether it's right and why.