Short answer: deferred init wants a standardized runtime boundary. Register routes once, but make every request and WebSocket resolve a typed AgentServerRuntime from app.state. Build the ready runtime off to the side, enter its services, then atomically swap it in. Avoid per-route config closures, module-level services, and conditional auth registration.
PR #3287 added a dormant warm-pool mode. The server boots, starts stateless side services, mounts all routers, and waits for POST /api/init. That init call merges a narrow runtime payload into the dormant Config, starts ConversationService, stores the new config and service on app.state, then flips InitService.state to ready.
Auth was the first concrete failure. Dependencies captured startup config, so a key delivered by /api/init was not honored. Also, if there were no session keys at startup, the auth dependency was never registered. PR #3844 fixed this by making auth dependencies read request.app.state.config on each request and by registering them unconditionally.
The current design separates route shape from runtime readiness. Routes exist immediately, but most /api/* routes include require_initialized, which returns 503 while the init service is dormant or initializing. The init endpoint uses X-Init-API-Key and the dormant server's bootstrap secret_key; the session key delivered later becomes the normal API credential.
sequenceDiagram
participant Pod as Warm pod process
participant App as FastAPI app
participant Init as InitService
participant Orchestrator
participant Runtime as ConversationService
participant Client
Pod->>App: create_app(dormant Config)
App->>App: mount routers and middleware
App->>Init: attach only when deferred_init=True
App-->>Orchestrator: /ready is ready, /api/* mostly 503
Orchestrator->>Init: POST /api/init + X-Init-API-Key
Init->>Init: merge InitRequest into base Config
Init->>Runtime: build and enter ConversationService(new_config)
Init->>App: app.state.config = new_config
Init->>App: app.state.conversation_service = service
Init-->>Orchestrator: state=ready
Client->>App: /api/* + X-Session-API-Key
App->>App: auth reads app.state.config at request time
App->>Runtime: handle conversation/tool/workspace request
One FastAPI app is created from a dormant config. Its root path, static files, middleware, route dependencies, and some module globals are established at import/startup time.
InitService merges the runtime payload, mutates process env, starts the conversation service, updates app.state, and marks the server ready.
HTTP auth now checks the latest config. Handlers use a mixture of app.state, default singleton accessors, and module-level service references.
PR #3844 fixed the known session-key bug, but the shape that produced it is still easy to repeat. The risks are less about one bad dependency and more about the lack of one canonical place where request-time runtime state is resolved.
Any dependency factory or closure that receives Config during route registration can silently ignore the runtime config delivered by /api/init.
If routes decide at startup whether auth or guards are needed, a dormant server that starts without keys may never protect the routes after keys arrive.
The effective runtime can live in base_config, app.state.config, conversation-service singletons, route-module globals, and middleware copies.
HTTP dependencies can be corrected while WebSocket modules still use import-time service objects or default config helpers unless they share the same resolver.
Values like root_path, static file mounts, and CORS middleware are established when the app is created. If they are allowed in InitRequest, they need dynamic handling or they are only partially effective.
InitRequest.env updates os.environ. That is acceptable for one runtime per process, but it is not scoped, reversible, or naturally test-isolated.
PR #400 was not the deferred-init PR. Its goal was to move agent-server toward a FastAPI factory pattern: create the app from an explicit Config, store config and services on app.state, and make routers/dependencies resolve services from that app state instead of relying on imported globals. That is the right direction for deferred init because it makes the app instance the ownership boundary.
flowchart TD
subgraph Factory[App factory]
C["Config input"] --> A["create_app(config)"]
A --> R["register routers once"]
A --> S["app.state.runtime = DormantRuntime"]
end
subgraph Provider[Request-time provider]
G["get_runtime(request or websocket)"]
G --> Auth["auth policy"]
G --> Ready["require_ready_runtime"]
G --> Services["conversation, bash, workspace services"]
end
subgraph Init[Deferred initialization]
I["POST /api/init"] --> B["build ReadyRuntime off to the side"]
B --> E["enter services"]
E --> Swap["atomic app.state.runtime swap"]
end
R --> G
S --> G
Swap --> G
A deferred-init version of that pattern should go one step further than PR #400's original sketch: the thing stored on app.state should not be just config plus several independent service attributes. It should be a typed runtime object with a single state transition.
@dataclass(frozen=True)
class DormantRuntime:
bootstrap_config: Config
state: Literal["dormant", "initializing"]
@dataclass(frozen=True)
class ReadyRuntime:
config: Config
conversation_service: ConversationService
bash_event_service: BashEventService
state: Literal["ready"] = "ready"
AgentServerRuntime = DormantRuntime | ReadyRuntime
App shape is static; runtime is explicit. The factory owns route registration and static server-shape decisions. A runtime provider owns all request-time config, auth, and services. Deferred init replaces the runtime object only after the new runtime is fully built and entered.
| Concern | Current merged flow | Runtime-context flow |
|---|---|---|
| Route registration | Routes are registered once. PR #3844 removed session-key conditionals for HTTP auth. | Routes are registered once, always with the same typed dependencies. No route decides auth based on startup keys. |
| Config lookup | Some auth paths read app.state.config; other code may still use default accessors or import-time objects. | Every route, WebSocket, and service dependency uses get_runtime(...). Fallbacks to process defaults are not part of request handling. |
| Readiness | InitService.state gates most /api/* routes. Other surfaces need to remember to participate. | require_ready_runtime returns a ReadyRuntime or 503. Any handler needing services must request this dependency. |
| Initialization | Init mutates env, resets a singleton, enters service, then updates multiple app.state fields. | Init constructs and enters a complete ready runtime locally, then swaps one pointer. Failure cleans up the local candidate and leaves the old runtime intact. |
| Testing | Tests can miss stale closures unless they mutate app.state.config after route registration. | Unit tests target the provider: dormant returns 503, ready returns typed services, swap changes auth and services for HTTP and WebSocket together. |
| Design pressure | Each new router must remember which global helpers are unsafe in deferred mode. | The safe path is the default. A router cannot get runtime services without going through the provider. |
The deepest simplification is to stop treating all config fields as equally mutable. Some fields define the ASGI app's shape; others define the user runtime. Mixing them is how partial application bugs appear.
flowchart LR Static[Server-shape config
root_path, static files, middleware strategy, enabled routers] --> Factory[create_app] Runtime[Runtime config
session keys, secret, persistence paths, webhooks, user env] --> Init[POST /api/init] Factory --> App[FastAPI app] Init --> Context[AgentServerRuntime] Context --> Req[HTTP + WebSocket dependencies] App --> Req
If changing a value would require rebuilding middleware, remounting routes, or changing root path behavior, prefer setting it before create_app. If it must vary per user, implement it as dynamic middleware that reads the runtime provider.
If a value affects auth, storage locations, user webhooks, secret encryption, or tools used after init, it belongs in the ready runtime and should be visible through one provider.
Use a candidate runtime: build, enter, validate, then swap. Do not publish partially initialized pieces across several app.state attributes.
get_runtime, get_ready_runtime, HTTP auth, WebSocket auth, and service accessors. Make it the only approved request-time path to config-backed services.InitRequest fields are truly runtime mutable. For web_url and CORS, either make the middleware/root-path behavior dynamic or remove them from deferred-init payload expectations.Design bias: make deferred init boring. The warm-pool controller should deliver one payload; the server should either remain dormant or publish one ready runtime. Every handler should ask the same question: “what runtime owns this request?” If the answer is standardized, stale closures and missing guards become much harder to write.
Config.deferred_init, InitService, /api/init, and the dormant-to-ready transition.app.state.config at request time and registering auth dependencies unconditionally.