A second opinion, without the agent switching its own model
PR #3673 adds a built-in ask_oracle tool. When the agent is stuck or wants a stronger reviewer, it consults a saved LLM profile named oracle for a one-shot answer — stateless: no conversation history, no tools, and the agent's own active model is never changed.
1What it adds: a callable "ask a smarter model" button
ask_oracle gives the agent a way to borrow a stronger model for a single question and then keep going on its own model. The Oracle is just a saved LLM profile named oracle — no new settings field, no wiring; save the profile and the tool works.The agent adds it like any other tool — Tool(name="ask_oracle") — and calls it with a question and optional context. The tool answers from the oracle profile and hands the text back as an observation, so the agent can fold that advice into its next reply.
Definition: AskOracleAction — definition.py:28-53 · AskOracleTool.create — definition.py:87-115
2The model you'd expect — and the one that's true
The natural first reaction is "isn't this just switching the model, or spawning a subagent?" The SDK already has both of those, and ask_oracle is deliberately neither. The whole design is in the difference.
ask_oracle both leaves the active model untouched and avoids spinning up a full agent. It borrows the stronger model for exactly one answer.3How one consult flows, step by step
The executor is small and does exactly four things: resolve the profile, build two messages, call the model once, return the text. Everything that makes it "stateless" is a thing it deliberately does not pass along.
| Step | What happens | Source |
|---|---|---|
| 1 | Resolve the Oracle: load the profile named oracle, decrypting stored secrets with the conversation's cipher. | impl.py:49-51 |
| 2 | Build exactly two messages: the fixed Oracle system prompt + the agent's question (and optional context). | impl.py:71-86 |
| 3 | One completion — make_llm_completion(oracle_llm, messages), no tools, no prior turns. | impl.py:88-89 |
| 4 | Return the text as an observation; on any failure return an error observation instead of raising. | impl.py:99-113 |
oracle profile is missing, it returns a friendly error observation (impl.py:52-59) — it does not crash the turn.4The surface: five small files, all additive
This is a pure addition — no existing behavior is rewritten, so there is no "before" to redraw. The table is the whole feature.
| File | Role | Link |
|---|---|---|
🟢 ask_oracle/definition.py | Action, observation, tool schema, read-only annotation, and self-registration. | definition.py |
🟢 ask_oracle/impl.py | The executor — the four-step flow in §3. | impl.py |
🟢 ask_oracle/__init__.py | Public exports for the sub-package. | __init__.py |
✏️ tools/__init__.py | Re-export AskOracleTool at the top level. | __init__.py:21 |
🟢 example 56_ask_oracle_tool/main.py | End-to-end: save two profiles, add the tool, drive a real consult. | main.py |
5"Oracle" is a convention, not configuration
An earlier revision of this PR added an oracle_llm_profile settings field; it was dropped in favor of a single fixed name. The tool looks up the profile literally called oracle — ORACLE_PROFILE_NAME = "oracle" · definition.py:25. To enable the Oracle, you just save that profile:
store = LLMProfileStore()
store.save("oracle", LLM(model="a-stronger-model", api_key=...))
agent = Agent(llm=..., tools=[Tool(name="ask_oracle")])That keeps the agent schema untouched (no new settings to migrate) and makes the Oracle swappable at rest — point the oracle profile at any model without changing agent config. The example wires exactly this: main.py:37-61.
conversation._cipher rather than reading plaintext secrets itself (impl.py:49-51).6Main-merge note: what changed to land on current main
The branch was ~190 commits behind. Merging main touched the feature in three small, mechanical ways — the tool logic itself was unaffected.
| Merge issue | Resolution | Where |
|---|---|---|
Conflict in settings/model.py create_agent() | Took main's refactor (new default_tool_specs defaulting, self.mcp_config passthrough); the branch's incidental edits here were superseded. | model.py:1309-1348 |
Example number collided with main's 55_persistent_memory | Renumbered the example 55 → 56; the duplicate-example check now passes. | 56_ask_oracle_tool/ |
| Test drift from main's API changes | Assert on the live conversation.agent.tools_map (plugin load now replaces the agent with a copy), and add the new call_context param to the test LLM's completion override. | test_ask_oracle.py |
7Proof and current status
| Check | Result |
|---|---|
pytest tests/tools/ask_oracle/test_ask_oracle.py | 6 passed |
pytest test_examples.py::test_directory_example_is_discovered | passed (example 56 discovered) |
python .github/scripts/check_duplicate_example_numbers.py | no duplicates |
pytest test_openapi_contract.py test_openapi_discriminator.py | 13 passed (AskOracleAction in the Action union) |
pyright (ask_oracle pkg + example + model.py) · pre-commit hooks | clean |
- test_ask_oracle.py:114-162 — a captured test-LLM proves the Oracle sees only the system prompt + question, no history.
- test_ask_oracle.py:164-207 — missing profile and empty response both return error observations, not exceptions.
The PR is mergeable after the merge; remaining CI/approval gating is normal branch protection.
Feature diff (five files, additive)
examples/01_standalone_sdk/56_ask_oracle_tool/main.py | +77 openhands-tools/openhands/tools/ask_oracle/__init__.py | +36 openhands-tools/openhands/tools/ask_oracle/definition.py| +119 openhands-tools/openhands/tools/ask_oracle/impl.py | +113 openhands-tools/openhands/tools/__init__.py | +2 tests/tools/ask_oracle/test_ask_oracle.py | +208