Lifecycle and ownership¶
Chassis owns runtime composition and lifecycle. This document describes the guarantees that make it safe to change an agent's environment while it is running.
Ownership¶
Every harness-managed effect has exactly one owning Scope. A scope owns:
- cleanup callbacks (the inverse of an operation, registered by a registry);
- entered (async) context managers;
- child scopes;
- background tasks;
- a diagnostic record for each effect it currently owns.
from chassis import Scope
scope = Scope("cache")
client = scope.enter_context(open_client()) # closed when the scope closes
scope.cleanup("close client", client.close)
scope.child("indexer") # closed before the parent's earlier effects
scope.create_task(warm_cache(), name="warmer") # cancelled and awaited on close
Plugins never register cleanups directly. They call ctx.tools.register(...),
ctx.hooks.register(...), ctx.capabilities.provide(...), or
ctx.create_task(...), and the harness registers the inverse operation on the
plugin's scope. Closing the scope reverses every registration.
If the runtime cannot answer "which scope owns this resource?", the resource is not safely managed, and it should not be created through the harness.
Teardown¶
Closing a scope is deterministic:
- the scope stops accepting new work;
- owned tasks are cancelled and awaited, under a configurable timeout;
- effects are unwound in reverse registration order;
- failures are aggregated into a single
EffectCleanupError.
A failing disposer never aborts the unwind: each failure is recorded and the remaining disposers still run. A task that refuses to stop is reported rather than silently leaking, and a task that dies with an exception is reported rather than having its failure disappear.
aclose() is idempotent and safe under concurrent callers: the first caller
creates the close task and every caller awaits it. Cancelling a caller does not
abort an in-progress close.
Plugin lifecycle¶
PENDING ──▶ LOADING ──▶ ACTIVE ──▶ UNLOADING ──▶ DISPOSED
│
└──▶ FAILED
Lifecycle describes ownership state. Health describes operational quality:
ACTIVE + HEALTHY, ACTIVE + DEGRADED, ACTIVE + UNHEALTHY. A transient upstream
outage degrades a plugin; it does not destroy and recreate it.
Illegal transitions raise HarnessStateError rather than corrupting state.
Setup rollback¶
If setup raises, the partially constructed scope is closed and the instance is
marked FAILED. No registration, task, or resource created during the failed setup
survives, and no partially configured instance becomes visible in a generation.
If a candidate mount fails while composing a generation, everything that attempt mounted is rolled back and the previously published generation remains current, so no run can observe a partial composition.
Capabilities and reactivity¶
Plugins declare what they provide and require:
@plugin(name="memory", version="1.0.0", provides={"memory": "1.0.0"}, requires={"database": ">=1,<2"})
async def memory(ctx: PluginContext) -> None:
ctx.capabilities.provide(MEMORY, Store(ctx.require(DATABASE)))
Resolution is a greatest fixpoint over the desired plugins and the providers that are actually available. Two rules make it well-defined:
- a plugin cannot satisfy its own requirement;
- active instances contribute what they registered, not what their manifest claims.
Removing a provider therefore removes its consumers too, in dependency-safe order, and restoring it reactivates the same plugins. Nothing in configuration decides this order; declared capabilities do.
Ambiguity is diagnosed, not resolved arbitrarily: two providers satisfying one
requirement make the consumer PENDING until a preference is set with
prefer_provider or provider_preferences in configuration.
Generations¶
A RuntimeGeneration is an immutable view of composition, published atomically.
Runs acquire a generation and keep it:
async with harness.acquire() as generation:
provider = generation.snapshot.require(DATABASE)
- The snapshot is deeply immutable and is never mutated.
- Acquisition, publication, and release are synchronous sequences with no
awaitbetween read and write, so a run can never observe a half-published generation nor lease one that is retiring. - The data plane takes no lock. Composition is serialized on the control plane, and a run acquires while a mount is still in progress.
Logical unload is not physical disposal¶
Removing a plugin affects the next generation. It does not destroy the instance:
generation 17: model, search, postgres run A acquires 17
↓ publish
generation 18: model, postgres run B acquires 18
run A keeps using 17
generation 17 → DRAINING
↓ run A finishes, leases reach 0
generation 17 → RETIRED
↓ search is now unreachable
search.scope.aclose()
The invariant is:
A plugin scope may be physically disposed only when no live runtime generation can reach it.
Shared instances are covered by the same rule: a plugin referenced by several live
generations survives until all of them retire. Liveness is tracked independently of
the bounded generation history kept for diagnostics
(Harness(generation_history_limit=...)): that buffer only evicts retired
generations, so a run holding an old generation keeps its resources alive however
many newer generations are published.
Disposal order is derived from the providers each instance actually resolved, so consumers are disposed before the providers they still reach.
Incremental reuse¶
A reconcile does not rebuild the whole composition. Each eligible entry's semantic
identity is recomputed and compared with the identity its instance was mounted
with; when they are equal, the exact same instance is carried into the new
generation. A change rebuilds only the nodes whose semantic inputs changed, plus the
consumers that reach them. See
incremental-composition.md for the model and
harness.diagnostics.analyze_impact(old, new) for the analysis.
Reuse is sharing, never shared mutability: a reused instance is never reconfigured in place, so a published generation still never observes a composition change after publication. A change that would require in-place mutation rebuilds the node.
Generation pressure¶
Draining for a long time is correct but not free: while a run holds a lease, the generation keeps its plugin instances alive and the resources behind them. Chassis does not guess whether that is too long — it reports it.
report = harness.diagnostics.generation_pressure()
print(report.to_text())
current_generation: gen_00a2
live_generations: 4
draining_generations: 3
oldest_lease_age_seconds: 862
gen_009f
state: draining
age_seconds: 912
leases: 1
oldest_lease_age_seconds: 862
retained_plugins:
- postgres-primary (postgres)
GenerationPressureReport is structured data; to_dict() gives a JSON-compatible
form and metrics() gives vendor-neutral gauges for a telemetry backend:
report.live_generations # how many generations are still live
report.draining_generations # how many are waiting for their last lease
report.oldest_lease_age_seconds # age of the oldest outstanding lease, if any
report.generations # per-generation age, state, leases, retained plugins
report.instance_generations # instance id -> live generations that reach it
report.resources # per-resource reachability and why it is retained
harness.diagnostics.instance_generations(instance.instance_id) # newest first
report.resources answers "who keeps this resource alive?" for each reachable
instance, including resources shared by several generations after an incremental
reconcile:
resource = next(item for item in report.resources if item.entry_id == "postgres")
resource.generations # ("gen_0044", "gen_0043") — newest first
resource.retained_by # ("lease", "sharing")
lease means a run still holds a generation that reaches the resource; sharing
means more than one live generation reaches it, so no single generation's
retirement would release it. A resource with no reaching generation is absent
because it is about to be disposed, not because its reachability is unknown.
Four different things are easy to conflate, and the report keeps them apart:
| Concept | Meaning | Where it lives |
|---|---|---|
| Diagnostic history | Retired generations kept for diagnostics only, bounded by generation_history_limit |
GenerationManager.history |
| Liveness | Generations a run can still reach: current + draining | GenerationManager.live() |
| Lease | One run's hold on one generation, with a start time | GenerationAccounting, GenerationLease |
| Physical disposal | Closing the scope of an instance no live generation reaches | Harness._reclaim |
The bounded history never decides liveness: it only evicts retired generations, so
a leased generation survives any number of newer publications and its plugin
instances stay alive. report.history_limit, report.history_retained, and
report.history_evicted make the difference visible.
Pressure is observability, not enforcement. Chassis adds no default limit and reclaims nothing because it is old: a loitering generation is reported so the operator can find the run keeping it alive (see troubleshooting.md).
Reconciliation is transactional¶
reconcile() computes a plan, mounts or reuses instances, publishes a generation,
and only then disposes what left the composition. A composition-identical reconcile
reuses the current generation instead of churning one, so repeated reconciliation is
a no-op.
Programmatic install/provide/uninstall are synchronous desired-state changes;
ensure_ready() applies them at asynchronous entry points such as agent invocation.
Shutdown¶
stop() is graceful and idempotent:
- stop accepting new runs;
- mark the current generation draining;
- wait for active runs, bounded by
shutdown_grace_seconds; - retire generations that are still busy, reporting the timeout as a failure;
- dispose every remaining instance, consumers first;
- close the harness scope;
- raise one aggregated
EffectCleanupErrorif anything failed.
Diagnostics¶
harness.diagnostics.plugins() # state, health, eligibility, per-requirement resolution, owned effects
harness.diagnostics.capabilities() # registered providers
harness.diagnostics.dependencies() # edges, activation order, pending, cycles
harness.diagnostics.generations() # state, leases, instances, plugins
harness.diagnostics.generation_pressure() # liveness, lease age, retained work
harness.diagnostics.instance_generations(id) # live generations reaching one instance
harness.diagnostics.budgets() # default limits and their enforcement modes
harness.diagnostics.tools() # owner, policy
harness.diagnostics.hooks() # owner, mode, ordering
harness.diagnostics.agents() # registered runtimes
harness.diagnostics.desired_state() # drift against the applied configuration
harness.diagnostics.status() # summary
harness.diagnostics.explain("memory") # why one plugin is active, pending, or excluded
harness.diagnostics.scopes() # the resolved scope tree of the current generation
harness.diagnostics.explain_requirement("agent", "database") # provenance of one requirement
harness.diagnostics.explain_scope("/research") # visibility and ownership of one scope
harness.diagnostics.diff_generations("gen_0004", "gen_0005") # semantic composition diff
harness.diagnostics.analyze_impact("gen_0004", "gen_0005") # reuse/rebuild analysis
harness.diagnostics.explain_reuse("gen_0004", "gen_0005", "search") # why one node was reused or rebuilt
harness.diagnostics.explain_agent("finance", revision="17") # composition of one agent revision
harness.diagnostics.diff_agents("finance", "17", "18") # what changed between revisions
Scoped composition is described in scopes.md: hierarchy, inheritance,
capability narrowing, provenance, and the explain and diff APIs.
Agent composition is described in agent-composition.md:
AgentSpec, immutable revisions, materialization, pinning, and retirement.
Diagnostics are generated from authoritative state, never scraped from logs, and they describe configuration by key rather than by value. Effect descriptions are redacted before they are reported, so diagnostics never become somewhere a secret accumulates.