Two files, one truth.

1The short version

2The two files, and who writes them

Every persisted conversation directory contains both files. They are written by different layers, on different schedules, with different serializers.

The problem is the overlap. meta.json has a legitimate job — id, title, created_at/updated_at, fork provenance, the launched agent-profile snapshot (models.py:78, StoredConversation). But it inherits agent: AgentBase from its base class StartConversationRequest, so the full agent blob rides along and becomes a second, independently-written copy of what base_state.json already owns.

3Where the copy is actually made

The meta.json agent originates from the request that started the conversation and is then re-serialized on every save_meta(). On start, the server even rebuilds the live agent from the stored copy:

# event_service.py — reconstruct the runtime agent from the STORED (meta) copy
agent_cls = type(self.stored.agent)
agent = agent_cls.model_validate(
    self.stored.agent.model_dump(context={"expose_secrets": True}),
)
# … then hand that agent to the SDK, which ALSO persists it into base_state.json
conversation = LocalConversation(agent=agent, …)

event_service.py:974–976 · L1037 LocalConversation(...)

Now watch the two writers diverge. A runtime LLM switch mutates the SDK-side agent — which auto-persists to base_state.json — but the server-side StoredConversation.agent in meta.json is only re-written on its own save_meta() triggers:

# local_conversation.py — switch_llm mutates the SDK agent (→ base_state.json)
def switch_llm(self, llm: LLM) -> None:
    …
    self.agent = self.agent.model_copy(update=update)  # new llm + condenser
    self._state.agent = self.agent                     # autosave → base_state.json

local_conversation.py:1522 switch_llm · endpoint conversation_router.py:492

4How it broke

This is the PR #4028 / issue #4032 report, reduced to the mechanism. A user set an LLM profile timeout, and after an agent-server restart the conversation ran with a different timeout — one that wasn't in any profile.

1. Start with a profileThe agent's LLM has timeout=600. Both files agree.
2. Switch the LLM while the conversation is runningswitch_llm installs a new LLM and saves it to base_state.json. It does not re-save meta.json. Now the two files disagree — base_state.json has the new timeout, meta.json still has the old one.
3. Restart, then touch the conversationHere's the subtle part. Right after a restart, reading the conversation is answered from base_state.json — the correct value, so it looks fine. But the first time anything actually uses the conversation, the server rebuilds the agent from meta.json (the stale copy) and overwrites base_state.json with it.
4. The timeout revertsThe agent now runs with a timeout the user never chose — silently, because the stale copy won at the moment of use.

The report that looked contradictory

The reporter said the timeout is lost on plain restart, with no switch — while the maintainer could only reproduce it with a switch. Both are right, and here's the reconciliation:

• Editing a profile does not reach a running conversation. The LLM config is frozen into the agent when the conversation is created. So to push a new timeout into an existing conversation, the reporter toggled profiles A→B→A — which he thought of as “fixing the setting,” but mechanically that toggle is the switch.
• That switch plants the base_state.json ⁄ meta.json divergence, and the restart then reverts it. Same bug, described from two ends.

5Why it's like this (verified from git)

Engel's instinct in-thread — “I don't know why we are doing that or since when” — is right to be surprised, because it was never a deliberate decision in this repo.

6The secret-leak connection

The same duplication doubled the blast radius of a separate security bug. The GET /api/file/download-trajectory/{id} endpoint zips the raw conversation directory. Because the full agent — including the condenser's own LLM — is serialized into both base_state.json and meta.json, a downloaded trajectory carried the LLM API key and the condenser LLM API key, in two files, as plaintext (no OH_SECRET_KEY) or recoverable Fernet ciphertext (with one).

That was fixed at the export boundary in software-agent-sdk #4217 (redact secrets while building the zip) — but that is a bandaid over the duplication. If the agent config lived in one place and, better still, held credential references only (OpenHands #15298), there would be nothing to redact and nothing to keep in sync.

7Two ways to fix it

Both fixes reach the same goal — one place that owns the agent — and both keep the SDK working the same way when you run it on its own, without the agent-server. They differ in how many files end up on disk.

Option A — slim down meta.json

Make base_state.json the only owner of the agent, and cut meta.json down to the few things the server genuinely needs to answer “what conversations exist?” without opening the big state file.

{
  "id": "…", "title": "…",
  "created_at": "…","updated_at":"…",
  "forked_from_…": "…",
  "launched_agent_profile": {…},
  // everything below is a
  // copy of base_state.json:
  "agent": {
    "llm": { "model":"…",
             "api_key":"…", … },
    "condenser": { "llm": {
             "api_key":"…", … } },
    "tools": [ … ],
    "agent_context": { … }
  }
}

{
  "id": "…", "title": "…",
  "created_at": "…","updated_at":"…",
  "forked_from_conversation_id": "…",
  "forked_from_event_id": "…",
  "launched_agent_profile": {…},
  "metrics": {…}
  // the agent now lives only
  // in base_state.json.
  // the conversation list
  // reads these few fields.
}

The steps:

1. Don't write the agent into meta.jsonLeave it out of the saved StoredConversation, the same way agent_profile_id is already marked exclude=True at models.py:85.
2. On reopen, read the agent from base_state.jsonUse the SDK's copy instead of self.stored.agent. A brand-new conversation still gets its agent from the start request — it's just saved to one file, not two.
3. Keep the list view fastListing conversations needs id, title, timestamps, metrics — all still in meta.json. It never needed the agent.
4. Be kind to old filesExisting meta.json files still carry an agent; readers should just ignore it and trust base_state.json. Rewrite drops it on the next reopen. Never write it to both again.

8The single-file idea

Engel's question: could we drop meta.json entirely and keep one file — where the SDK writes the normal conversation state, and the agent-server just adds a few of its own fields on top? So running through the SDK alone gives you the common file, and running through the agent-server gives you the same file plus a small server section.

Short answer: yes, it can work — and it's a nice mental model — but only if you get one detail right (who is allowed to write the file) and accept one trade-off (you may still want a tiny separate index for speed). Here's the reasoning, plainly.

The shape that works

Give the agent-server its own clearly-labelled corner of the SDK file — not fields sprinkled next to the SDK's own. Something like:

{
  // ── common: written by the SDK, same as running standalone ──
  "agent": { "llm": {…}, "condenser": {…}, "tools": [ … ] },
  "events HEAD": "…", "stats": {…}, "tags": {…},

  // ── one labelled corner the SDK carries but never reads ──
  "agent_server": {
    "title": "…",
    "created_at": "…", "updated_at": "…",
    "forked_from_conversation_id": "…",
    "launched_agent_profile": {…}
  }
}

Why a single labelled corner, and not loose fields? Because the SDK validates this file when it loads. Pydantic (what the SDK uses) ignores keys it doesn't know when reading — so extra server fields won't crash a standalone SDK load. Good. But it drops those unknown keys when it writes the file back. So if the SDK is the writer and doesn't know about the server's fields, it will quietly erase them on the next save. That's the whole trick, and it points to one firm rule:

Why this doesn't fall out for free today

1. The SDK would need a “carry this, don't touch it” slotRight now the SDK's state model has no place to stash foreign fields across a save. It would need one explicit opaque field (e.g. extensions) that survives read → write untouched. Without it, Pydantic drops unknown keys and the server's data vanishes on the next save.
2. The server writes its fields through the SDK, not around itUpdating the title or updated_at would have to go through the SDK's save path (hand the SDK the new block), instead of the server writing its own file. That's a real change to save_meta.
3. The fast conversation list gets more expensiveThis is the real trade-off. Today the list reader opens the small meta.json to get id/title/time (conversation_service.py:589). With one file, listing 500 conversations means opening 500 large state files (they can be hundreds of KB) just to read a title. You'd want a small separate index or a DB row — which is, quietly, a second file again.

So which is it?

9The two bandaids shipped

Two PRs now fix the symptom (#4032). Neither removes the duplication — both keep the agent in both files and just fight the divergence. Worth being honest about that, and about a consistency cost one of them introduces.

Both are fine as stop-the-bleeding patches. But every mirror or per-kind special case is a tax the duplication keeps charging. The real fix deletes the tax.

10Doing it right: two objects, not two files' worth of overlap

Step back from the field lists and there are really only two things worth persisting, plus a little server bookkeeping:

So the target is simple to state: base_state.json is the one persisted Conversation (Agent included). meta.json holds only what the server genuinely adds, plus whatever create-time inputs the SDK consumes at construction but does not round-trip into state.

What meta.json is today (measured, not eyeballed)

StoredConversation is literally the server's create-request re-persisted: it subclasses StartConversationRequest and adds a few fields. Comparing the classes directly gives an exact partition — no hand-labeling.

Bucket 1 — Duplicates the Conversation (the bug surface): 8 fields

In both StoredConversation and ConversationState today. base_state.json already owns these; meta re-storing them is the duplication.

Bucket 2 — Create-request inputs the SDK consumes but doesn't round-trip into state

These are inherited from StartConversationRequest and are not ConversationState fields. The SDK transforms them at construction (plugins → merged into the agent; secrets → secret_registry; observability → telemetry) and doesn't keep the raw input in state. So the server legitimately keeps them to reconstruct the conversation on restart.

Bucket 3 — Genuinely added by the agent-server: 9 fields

These are exactly StoredConversation − StartConversationRequest — the only fields the server truly owns.

The concrete moves

1. Delete bucket 1 from meta.jsonStop persisting agent, workspace, max_iterations, stuck_detection, confirmation_policy, security_analyzer, hook_config, tags into meta. base_state.json already owns them; on resume, read them from there.
2. Add a tiny agent discriminator to metaThe one thing the server needs cold (before loading state) is the agent kind — today it reaches for the whole stored.agent (e.g. _is_codex_agent). Replace with agent_kind ("Agent"|"ACPAgent") and, for ACP, acp_server. Small, stable, not the agent.
3. Reconcile the two secret shapesmeta's secrets (sources) vs base_state's secret_registry (resolved). Describe a credential in exactly one place — the direction #15298 wants anyway.
4. Keep listing cheapThe list view reads bucket 3 (+ maybe bucket 2) from the small meta. If you ever fold meta into base_state (the single-file option), keep a rebuildable index instead — a derived cache, not a second source of truth.

11What to watch

The compact test for any real fix: after an LLM switch and after an ACP model switch, a cold resume must observe the switched config — with no field living authoritatively in two files.

Two open questions, now resolved

12The build plan: burn the overlap down under a guard test

Turning "one source of truth" from a claim in a PR description into a property CI enforces. The order matters: land the scoreboard first, then let each phase shrink it.

The test that defines "done"

Two tests. The first proves the behaviour; the second holds the line so no future PR can quietly re-add a duplicate.

(a) Round-trip, parametrized over every mutable field — create a conversation, mutate the field at runtime through its real endpoint, tear the service down, start a fresh one over the same directory, hydrate through the real resume path, and assert the live conversation observes the mutated value.

(b) The structural guard — the rule as an assertion over the type system's own metadata, so it needs no bug report to fire:

# fails the moment anyone re-adds a duplicated field
stored_persisted = StoredConversation.model_fields.keys() - EXCLUDED
overlap = stored_persisted & ConversationState.model_fields.keys()
assert overlap == {"id"}, f"meta.json re-persists ConversationState fields: {overlap - {'id'}}"

Land (b) first, xfail-ed with the current 9-field overlap listed, and let each phase shrink the expected set. Note EXCLUDED must subtract exclude=True fields — exclude=True suppresses serialization, not validation, so an excluded field is still readable in memory during the migration window and would otherwise let the guard pass while the field is still live.

Sequencing

13Sources

• state.py:421 — _save_base_statebase_state.json is the SDK's single source of truth; persists ConversationState.agent.
• event_service.py:169 — save_metaWrites the full StoredConversation (incl. agent) into meta.json.
• models.py:78 — StoredConversationExtends StartConversationRequest; inherits agent: AgentBase (L324).
• event_service.py:974 — agent rebuilt from stored.agentStart path reconstructs the runtime agent from the meta copy.
• local_conversation.py:1522 — switch_llmMutates SDK agent → base_state.json only; does not save_meta.
• issue #4032The bug report: LLM profile timeout reset after agent-server restart.
• PR #4028Bandaid 1 — mirror the switched llm/condenser into meta.json on change.
• PR #4219Bandaid 2 — read llm/condenser from base_state.json on resume (regular agents only).
• PR #4201perf — index conversation execution status for search/count; replaces a full-state load with a signature-gated raw enum read. The concrete cost of loading state for listing.
• PR #4100 / PR #4202Lazy hydration + idle eviction — the codebase actively investing in less state loading, which is why Option B's index cost is real.
• PR #4217 / issue #4216Trajectory secret-leak fix (export-boundary redaction) — a related bandaid.
• OpenHands #15298Reference-only credentials — the endgame that makes even one copy safe.