One source of truth.

1The short version

The study proved the mechanism. What was missing was the part that makes a fix stick: an exact account of which file should own each field, a rule that a reviewer can't accidentally violate, and a sequence that never regresses the three perf PRs (#4201/#4100/#4202) the codebase just landed to make listing cheap. That's this page.

2Four things that move the difficulty

Re-reading upstream/main surfaced four facts the study's framing didn't yet carry. Two make the fix easier than it looks; two make it harder. All four change where the real work is.

2.1 The HTTP API is already base_state-authoritative — easier

_compose_conversation_info() builds every conversation response from ConversationState, not from the meta snapshot:

return ConversationInfo(
    **state.model_dump(mode="json"),   # <- agent comes from base_state.json
    title=stored.title, metrics=stored.metrics, created_at=stored.created_at, ...
)

So ConversationInfo.agent is fed by base_state.json. Removing agent from meta.json changes zero bytes of any API response. agent-canvas reads info.agent.kind / .acp_server / .acp_model / .llm.model (agent-server-adapter.ts:305,314,340-343) — all still served, unchanged. This kills the biggest perceived blocker: the client contract is not in the way.

2.2 The cheap-listing argument is alive — and was just reinforced

#4201 landed 2025-07-23 — days before the study's rewrite — and it is a deliberate investment in exactly the direction I'd called dead: it replaced a per-conversation full ConversationState load with a signature-gated raw read of one enum. Its docstring names the cost I was waving away:

With #4100 (lazy hydration) and #4202 (idle eviction) beside it, the trajectory is unambiguous: less state loading, not more. What actually survives of my point, stated narrowly:

So Phase 1 is free on listing performance and the catalog must stay answerable without loading state. Compatible. The error was generalising the second row into a verdict on the first — and it flips Option B from "tidier end state" to actively regressive: folding meta into base_state would delete the index #4201 just built.

2.3 The two agent blobs are not copies — they're two things wearing one type

This is the finding that reframes the whole problem.

• meta.json's agent is the declarative launch spec: what the caller asked for, pre-merge.
• base_state.json's agent is the resolved runtime agent: post-merge. _ensure_plugins_loaded() merges plugin skills, expands MCP config, and writes the merged agent into state (which autosaves).

They were never two copies of one value that drifted — they are two different values that were never given different names. That is exactly why neither shipped bandaid can be right: #4028 copies resolved → declarative, #4219 reads declarative ← resolved, and both had to hand-scope to llm+condenser so they wouldn't move the merged form. That coincidence is the type confusion showing through. Every future field would need the same manual adjudication.

2.4 The resolved agent has secret values baked in — harder

_ensure_plugins_loaded calls expand_mcp_variables(..., expand_defaults=True) and persists the result. mcp_config is a plain dict — no SecretStr, so no redaction on dump. So base_state.json's agent carries resolved secret values that meta.json's does not. Two consequences, both real:

• base_state.json is the more dangerous file for the trajectory-export leak (#4216/#4217), not the safer one.
• A naive "resume entirely from base_state" would freeze rotated secrets forever — the baked-in values have no ${...} left to re-expand.

This is the code-level reason the declarative copy has value, and the concrete link to #15298. The endgame isn't "one agent blob" — it's one resolved agent that is safe to persist because it holds references, plus one declarative launch spec that is the only thing ever re-resolved.

3Three rules, none written down

Restated: the system has two files and at least three different, undocumented, per-field rules about which one wins on resume. agent is just the instance that got reported. Verified on upstream/main:

4The measured partition

The destination comes from set arithmetic over three Pydantic models, not hand-labeling. meta.json is StoredConversation(StartConversationRequest) — the server's create-request re-persisted. (StartConversationRequest lives in the SDK only so client and server share one schema; the SDK's own constructors take plain args and never consume a request object.) So: meta = persisted request, base_state = persisted Conversation. Diffing the classes gives an exact three-way split — which I re-derived independently of the study, and which reproduces it exactly:

Two details the arithmetic settles cleanly: agent_profile_id is the only exclude=True field in the whole pair — so agent_settings is genuinely persisted (bucket 2), and the study was right to correct agent_settings/parent_conversation_id from "server-only" to SDK request fields. And bucket 1 is 9 including id; the eight that actually matter are the rest.

5The rule, and its one exception

Stated once, mechanically checkable — with the exception named so it stays honest:

The second clause is why §2.2 matters: the catalog genuinely must answer sort/filter/count without loading state. And the exception is not a loophole I'm inventing to dodge the hard cases — it already exists and is already implemented correctly. execution_status is a ConversationState-only field the server caches in _ConversationRecord and refreshes through _refresh_execution_statuses(): gated on an os.stat signature, invalidated to None the moment the conversation goes live ("Autosave will invalidate any signature we hold"), and explicitly memory-wins on the race ("Went live while we were reading disk; memory wins").

That is the shape any derived index copy must take — and the reason the rule stays enforceable. A derived copy lives in _ConversationRecord, not in StoredConversation, so the guard test (§7), which checks StoredConversation against ConversationState, still fires on any real duplication.

The two cold-read wrinkles, re-judged

• tags — clean, delete it. I'd assumed the catalog filters on tags; it doesn't. _search_conversations takes only page_id, limit, execution_status, sort_order — there is no tag filter in the router today. So tags leaves meta.json with no index consequence at all. If a filter is ever added, it takes the execution_status treatment, not a resurrected authoritative copy.
• workspace — genuine, and my first answer was wrong. The server needs it before LocalConversation exists: start() reads stored.workspace, mkdirs the working dir and calls _ensure_workspace_is_git_repo(). I first wrote "the resume path can read base_state.json first" — which is precisely the reasoning #4201 spent a PR removing. Better: leave workspace in meta as launch-spec input and delete it from ConversationState instead. create() already overwrites it from the caller every time, so base_state's copy has never once been authoritative. It is the one bucket-1 field where meta is the more defensible owner. Decide it explicitly in Phase 2.

6The phases

Each phase is independently shippable and revertable. The full sequencing table and the compact guard live in the study's §12; below is what a phase actually touches.

Phase 0 — Unblock #4032 now

kripper is blocked and has confirmed #4028 fixes it in his environment. Ship this week; don't hold the bug hostage to the redesign. Recommendation: land #4219's read-side approach, extended to ACP — take llm+condenser from base_state on resume, and delete the ACP mirror rather than adding a sibling to it (Q1 in §8 shows that's safe). #4028 adds a second hand-rolled mirror that Phase 1 then removes; #4219 leans toward the real fix. Either is acceptable — but do not merge both. Two overlapping fixes in one file pair is how the next inconsistency is born.

Phase 1 — agent out of meta.json

The whole point, and the blast radius is small: only six production sites read stored.agent, and §2.1 means the API layer isn't one of them.

1. Add a discriminator to StoredConversationagent_kind: str + acp_server: str | None, populated at creation — the one thing the server needs to know about the agent before state loads.
2. Replace the cold codex checks_is_codex_agent(stored.agent) at three sites → read the discriminator.
3. Resume reads the agent from base_state.jsonDrop the type(stored.agent) / model_validate rebuild at event_service.py:974-976. A brand-new conversation still gets its agent from the start request — written once, to one file.
4. ForkAlready copies fork_conv.agent (the live agent, in base_state); just drop "agent" from fork_overrides.
5. Stop persisting itMark agent exclude=True on StoredConversation, mirroring the agent_profile_id precedent. Old files keep an agent; readers ignore it, the next save_meta() drops it. Never write it again.

Phase 2 — the rest of bucket 1 (8 of 9)

Where the arithmetic changed the shape of the work — no longer a tidy-up, the actual fix.

• confirmation_policy + security_analyzer — two live bugs, not hygiene. Remove the re-assertion at event_service.py:1059-1060 on the resume path (keep it for fresh creation). These are §3's unreported #4032 instances; ship them with a regression test each and say they were reproducible before the change. That turns "refactor" into "bug fix" for a reviewer.
• max_iterations, workspace — via the generalised create(agent=None, ...) change; workspace carries the §5 wrinkle.
• stuck_detection, hook_config, tags — stop passing them from stored into the resume-path ctor; let loaded state supply them.
• id stays in both — shared key, not a duplicated value. The guard test whitelists exactly this one field and no more.
• Write the §5 rule into a docstring on StoredConversation and ConversationState. The rule never being written down is why this recurred.

Phase 2b — reconcile the two secret shapes

meta's secrets/secrets_encrypted (sources, bucket 2) vs base_state's secret_registry (resolved). Related, not identical, so the arithmetic doesn't flag them — but they are two descriptions of one credential, the same disease in a different shape. Keep it separate from Phase 2 so the mechanical deletions land without waiting on the credential-model decision (#15298).

Phase 3 — make the single copy safe

Phase 1 makes base_state.json the sole home of the agent; §2.4 says that file bakes resolved secret values into mcp_config. Persist mcp_config in its unexpanded form (${...} intact) and expand at use time only. Then a rotated secret is picked up on the next resume instead of frozen, and the trajectory export has nothing to redact — the #4217 bandaid becomes unnecessary rather than load-bearing. This is #15298's invariant applied to one concrete field, and the step that earns "once and for all": Phases 1–2 make the config correct; Phase 3 makes the single copy defensible.

Phase 4 — Option B / single file

Cut. #4201 already built the derived, signature-gated, rebuildable index that Option B would need — and better than a relabel would have. A literal single file is off the table per §5. The only survivor worth keeping is a docstring on _ConversationRecord: this is a derived cache, never a source of truth — folded into Phase 2.

7The test that defines "done"

Two tests. The first proves the behaviour; the second holds the line so no future PR can quietly re-add a duplicate.

(a) Round-trip, parametrized over every mutable field — create a conversation, mutate the field at runtime through its real endpoint, tear the service down, start a fresh one over the same directory, hydrate through the real resume path (get_event_service(), not the _event_services dict — #4100's lazy hydration means the direct read skips the resume path; this is the trap #4028 hit on rebase), and assert the live conversation observes the mutated value.

(b) The structural guard — the §5 rule as an assertion over the type system's own metadata, so it needs no repro to fire:

# fails the moment anyone re-adds a duplicated field
stored_persisted = StoredConversation.model_fields.keys() - EXCLUDED
overlap = stored_persisted & ConversationState.model_fields.keys()
assert overlap == {"id"}, f"meta.json re-persists ConversationState fields: {overlap - {'id'}}"

Land (b) first, xfail-ed with today's 9-field overlap listed, and let each phase shrink the expected set. Note EXCLUDED must subtract exclude=True fields — exclude=True suppresses serialization, not validation, so an excluded field is still readable in memory during the migration window and would otherwise let the guard pass while the field is still live. This is the whole point of switching to set arithmetic: "one source of truth" becomes a property CI enforces.

8Open questions, resolved

Two of the three that gated Phase 1 are settled; the study's §11 carries the full Q2 treatment. Summarised here so this plan stands on its own:

9Sources

• Companion study — Two files, one truthWhere the copy is made, how it broke (#4032), the field inventory, and the compact §12 build plan this page expands.
• issue #4032The bug report: LLM profile timeout reset after agent-server restart.
• PR #4028 / PR #4219The two bandaids — mirror-on-write vs read-from-base_state. Both hand-scope to llm+condenser: the type confusion of §2.3 showing through.
• PR #4201Indexes execution status for search/count — replaces a full-state load with a signature-gated raw enum read. The concrete cost of loading state for listing (§2.2).
• PR #4100 / PR #4202Lazy hydration + idle eviction — the codebase investing in less state loading, which is why Option B is cut.
• PR #4217 / issue #4216Trajectory secret-leak fix (export-boundary redaction) — a bandaid Phase 3 makes unnecessary.
• OpenHands #15298Reference-only credentials — the endgame that makes even one copy safe (Phase 2b / Phase 3).