Safe Metalab Contribution Workflow

Context

Metalab mode lets an agent modify autograph itself from inside an autographed project. The metalab agent edits engine sources and core packages directly through L1 volume mounts while the operator handles Git operations, lifecycle commands, and container orchestration from the host.

An incident exposed three failures in the current workflow:

1. local autograph changes were destroyed by an update operation because the system did not distinguish dirty sources from clean ones;
2. a lifecycle command reported success while the requested agent was not actually usable;
3. a fresh-install test could not consume the exact local package state because there was no way to capture it without committing first.

The existing ADRs provide the building blocks but do not define the contribution lifecycle that ties them together:

• lifecycle-tasks.md - task namespaces and symmetric destructive counterparts;
• meta-mode-architecture.md - dev/metalab volume inversion, shared image;
• security-model-3-layers.md - L1/L2/L3 access boundaries;
• dependency-management.md - engine submodule, West package resolution;
• package-env-aggregation.md - generated config and environment state;
• forge-workflow.md - the operator journey from review to merge.

This ADR defines the entity state model, the safe contribution lifecycle, the operation guards, and the resilience model. The forge workflow itself is documented in forge-workflow.md; this ADR induces an update of that document to align with the guard and observation rules defined here.

Entities

The contribution workflow involves five entity types. Each has its own state model because each is managed differently.

Entity	Location	Management	Reference state
Root project	Repository root	Git (operator)	On the operator's reference branch, clean
Engine submodule	engine/	Git submodule	At submodule-recorded commit
Core West packages	engine/deps/modules/*	West (detached HEAD on manifest-rev)	At manifest-rev, clean
Project West packages	deps/modules/*	West (same as core)	At manifest-rev, clean
Agent runtime	env/<ENV>/services/<agent>/	Filesystem (per agent, per env)	Directories populated, agent healthy

Entity state model

Source entities (engine, West packages)

A source entity is a Git repository that the metalab agent edits and the operator forges. The engine submodule and each West package share the same state model, differing only in what "reference" means.

State	HEAD position	Working tree	Description
aligned	At reference	Clean	Nominal post-install/update state. No local work.
aligned-dirty	At reference, detached	Dirty	Metalab edits without branching first. Danger zone: changes are orphaned on west update or git submodule update.
branched	On local branch	Clean	Contribution branch exists. All edits committed. Safe development state.
branched-dirty	On local branch	Dirty	Active editing on a contribution branch. Uncommitted changes present.
diverged	At reference but reference is outdated	Any	Remote has newer commits not yet reflected in the local reference.
indeterminate	Cannot determine	Cannot determine	.git missing, unreadable, or metadata inaccessible from the container.

"Reference" means:

• For the engine: the commit recorded in the root project's submodule pointer.
• For West packages: the manifest-rev branch set by west update from engine/deps/manifest/west.yml.

Indeterminate is not clean. An entity whose state cannot be determined must be treated as if it has unprotected local work. This is a lifecycle control concern (not a security concern): the system cannot guarantee that an operation will not destroy work if it cannot observe the current state.

Root project

The root project is a standard Git repository. Its contribution state is simpler because it is not affected by west update or submodule resolution directly. However, it must track the correct engine submodule version. If autograph introduces interface-breaking changes, a migration mechanism may be needed (out of scope for this ADR).

State	Description
ref-clean	On the operator's reference branch, clean working tree. Nominal state.
ref-dirty	On the reference branch, dirty (typically pending submodule ref changes).
branched	On a contribution branch.
branched-dirty	On a contribution branch, dirty.

Agent runtime

State	Description
active	Container running, service directories populated, healthcheck passing.
stopped	Container stopped, state directories preserved on disk.
absent	Service directories do not exist (pre-setup or post-teardown).
degraded	Container running but healthcheck failing, or state directories incomplete.

Dev-agent runtime lives under env/<ENV>/services/dev-agent/. Metalab-agent runtime lives under env/<ENV>/services/metalab-agent/. The two are fully independent: they never share writable state.

Contribution lifecycle

The contribution lifecycle applies per source entity (engine or West package). It runs from "I start working on a change" to "I have merged my change and synced the ecosystem."

The lifecycle is not a linear sequence. Testing happens continuously through reactive environment principles. Branching happens when the operator is ready, but must happen before commit. The phases below describe what must be true at each stage, not a rigid order.

A single contribution can span multiple entities simultaneously (e.g. a package change that requires an engine compose update). In that case, the lifecycle applies to each entity in parallel, with commit ordering constraints at forge time (see forge-workflow.md: core packages first, engine second, root project last). Project West packages have no ordering constraint with the core/engine chain.

The companion diagram (safe-metalab-contribution-workflow.drawio, tab "Contribution Lifecycle") shows the full per-entity lifecycle with phase columns, ordering constraints, and guards.

Observe

Before starting work or running any guarded operation, verify the ecosystem state.

Scope	What to observe
Per source entity	Current state (aligned/branched/dirty/...), HEAD position, reference SHA, ahead/behind count, staged/unstaged/untracked counts, local-only branches, stash count
Root project	Branch, dirty state, pending submodule ref changes
Agent runtime	Container health per agent (running/stopped/absent), healthcheck status, service directory existence
Generated config	Freshness of .env.project relative to env.d/ sources
Exchange zone	Pending proposals in exchange.lab.tmp/ or exchange.dev.tmp/

Observation blocks guarded operations when any entity is aligned-dirty or indeterminate (see "Blocking conditions" below).

Decision: The observation tool is an extension of augr:diagnos. It must produce a structured report readable by both the operator and the agent. The report must never contain secret values.

Develop (edit + test continuously)

The metalab agent edits source files. The operator can test changes at any point through the reactive environment - testing is continuous, not a distinct phase after committing.

What "testing" means depends on the type of change:

Change type	How it takes effect	Test method
Skills, prompts, documentation, AGENTS.md	Read directly by the running agent on next session or skill load.	Start a new chat session in the affected agent.
Entrypoint, agent config template	Requires container restart.	task augr:reup
Dockerfile, base image dependencies	Requires image rebuild. See resilience model for staged rebuild rule.	task augr:pkg:agent:rebuild
Compose files (volumes, networks, ports)	Requires container recreation.	task augr:reup
Taskfiles	Takes effect immediately for host-side task commands.	Run the affected task from host.
Engine scripts	Takes effect immediately or on next agent session.	Run from host or agent.
env.template files	Requires augr:install (env-collect) + augr:build (aggregate).	Re-run the install and build lifecycle.

Gap: safe env.template refresh. Currently, testing an env.template change requires augr:uninstall + augr:install + augr:build, which is destructive (removes resolved deps). A lighter command is missing -- one that re-collects env templates and rebuilds .env.project without wiping engine/deps/ or losing runtime state. This gap is likely what led to the misuse of admin:teardown as a validation shortcut. It should be addressed as a required implementation item.

Changes that only affect skills, prompts, or documentation do not require any infrastructure restart - they are live immediately.

Commit readiness

When the operator is ready to commit, a contribution branch must exist on each modified entity. If it does not exist yet, the operator must create it before committing. This is a hard requirement because West packages in aligned-dirty state (edited on detached HEAD) lose their changes on west update.

For West packages, the operator should also verify the remote configuration and ensure the branch is based on the current reference before committing. A convenience command to reset all deps to the correct remote and latest reference version would be valuable (deferred to implementation).

Commits follow dependency order when the contribution spans multiple entities: core packages first, engine second, root project last. Project West packages can be committed independently. This ordering is documented in forge-workflow.md.

Validate infrastructure changes

If the contribution modifies infrastructure (Dockerfile, compose, Taskfile, env.template, entrypoint), a full install test may be warranted before forging. This means verifying that the lifecycle sequence (admin:setup through augr:up) still works correctly with the modified sources.

Risk mitigation for infrastructure validation:

1. Checkpoint runtime state before attempting any destructive lifecycle step (T03 implementation scope). This ensures sessions, auth, and agent state can be restored if the test fails.
2. Staged agent rebuild. When the change affects the Docker image, rebuild and restart one agent only. Verify it is healthy before touching the other. This preserves the resilience invariant (see "Shared image risk" below).
3. Use augr:build + augr:reup for compose and Taskfile changes -- these do not destroy resolved deps or runtime state.
4. Use the safe env.template refresh (once implemented) for env.template changes -- re-collect templates and rebuild .env.project without wiping deps.
5. Reserve augr:uninstall / admin:teardown for full lifecycle validation only, after all less-destructive tests pass and runtime state is checkpointed.
6. Rollback path: if the test fails, restore the runtime checkpoint, revert the source changes, and augr:reup. The operator should always know the recovery command before running a destructive test.

If the contribution only modifies documentation, skills, prompts, or application code within a package, a full install test is not necessary - the reactive environment testing during development is sufficient.

Forge and sync

The forge workflow (push, PR, merge) is defined in forge-workflow.md. After merge, the operator syncs references:

• For West packages: update manifest revision in west.yml if needed, then task augr:update.
• For the engine: update the submodule reference in the root project.

The entity returns to aligned at the new reference. These sync steps and their ordering constraints are the subject of the companion lifecycle diagram (safe-metalab-contribution-workflow.drawio).

Operation guards

Guards apply to task commands only. Direct git and west commands are the operator's responsibility - using them to bypass guards voids the safety guarantees of this workflow. The observation tool should warn about resulting state inconsistencies but cannot prevent direct command usage.

Guards are general rules that apply at any point in the lifecycle, not at specific steps. Any guarded operation must check source entity states before proceeding.

Guard table

Operation	Precondition	Violation consequence
augr:update	All West packages are aligned or branched (clean). No entity is aligned-dirty or indeterminate.	west update resets packages to detached HEAD on manifest-rev. Uncommitted changes on detached HEAD are lost. Committed branches are preserved but no longer checked out.
augr:upgrade	Engine is aligned or branched (clean). All packages satisfy augr:update precondition.	git submodule update --remote changes engine HEAD. Same risks as augr:update for packages.
augr:engine:reset	All packages are aligned or have work committed and pushed.	Removes .west/ and modules/ entirely. All local-only content is destroyed.
augr:uninstall	Same as augr:engine:reset.	Backs up env.d/ but removes resolved deps. Same destruction risk.
augr:pkg:agent:rebuild	Warning if metalab-agent uses dev-agent image (which it currently does).	augr:pkg:agent:rebuild rebuilds the shared image + recreates dev-agent only; metalab adopts it on its next augr:pkg:metalab:reup. A broken image downs dev-agent now, metalab on its next reup. See resilience model.
admin:restore	Backup exists. Warn if source entities are branched-dirty.	Replaces current env.d/ with backup. Generated config becomes stale, must be rebuilt.
admin:teardown	All source entities have work committed and pushed. Operator explicitly confirms.	Removes env/ contents, deinitializes engine submodule. All local-only work and runtime state unreachable. Nuclear.
admin:env:purge	Operator explicitly confirms.	Permanently deletes environment backups. No recovery.

Force override: Each guard must support an explicit operator override (e.g. --force, -f or confirmation prompt) for cases where the operator intentionally wants to proceed despite the warning. The override must log what was bypassed. A generic force flag that silently skips all checks is not acceptable.

Nuclear operations

The following must not be used as routine workflow steps - reserved for explicit reset or incident recovery:

• admin:teardown
• admin:env:purge
• host:clean
• host:purge

Blocking conditions (summary)

Condition	Effect
Any source entity is indeterminate	Block guarded operations. Operator must resolve manually.
Any source entity is aligned-dirty	Block guarded operations. Operator must branch or discard.
Any source entity is branched-dirty	Warn on revision-changing operations. Block destructive operations.
All entities are aligned or branched (clean)	Allow all operations.

Diverged detection: An entity is diverged when its local reference is behind the remote. The observation tool must detect this when network access allows and report it as informational (not blocking). The operator decides when to sync.

Resilience model

Two agents, mutual fallback

The system runs two agents:

• dev-agent (augr_dev_agent): works on the project (L1 = workspace).
• metalab-agent (augr_metalab_agent): works on autograph (L1 = engine).

They run on independent networks (AUGR_NETWORK_AGENT, AUGR_NETWORK_METALAB), use independent runtime state directories, and do not share writable files. If one breaks, the operator can use the other to inspect the system and prepare a recovery.

Decision: At least one of these two agents must remain functional at all times during metalab contribution. This is the resilience invariant.

Enforcement: The observation tool must check agent health (container running + healthcheck passing) as part of its report. Before any operation that can affect agent availability (rebuild, reup, teardown), the observation tool must verify that the other agent is healthy. If both agents are degraded or stopped, the observation tool must flag this as a critical warning with recovery instructions.

Shared image risk

Metalab-agent uses the dev-agent Docker image (image: augr-pkg-agent-dev-agent in autograph.pkg.metalab/infra/compose.yaml). This means augr:pkg:agent:rebuild rebuilds this shared image and recreates dev-agent, while metalab keeps running on the previous image until augr:pkg:metalab:reup. A broken image therefore downs dev-agent immediately and metalab only on its next reup -- the staged rollout (owner rebuild, then consumer reup) is what preserves the resilience invariant.

Decision: When testing changes that affect the Docker image (Dockerfile, base image, entrypoint), the operator must roll one agent first (augr:pkg:agent:rebuild rebuilds the shared image and recreates dev-agent), verify it is healthy, then roll the other (augr:pkg:metalab:reup). The package-level verbs enforce this staging by construction; the observation tool must still warn that the image is shared with metalab.

Best practice: Keep image-affecting commits small and isolated. This reduces the risk surface and simplifies diagnosis when a rebuild breaks an agent.

Deferred: Separate images or tagged image versions for each agent would eliminate this risk entirely.

Recovery paths

Failure	Recovery
Metalab-agent broken (bad config, skill, prompt)	Use dev-agent to inspect engine sources. Fix from host or dev-agent shell. task augr:reup to restart metalab.
Dev-agent broken	Use metalab-agent to inspect package sources. Fix and restart from host.
Both agents broken (bad image)	From host: copy the broken files for analysis, revert the image-affecting changes in the source, then task augr:pkg:agent:rebuild (then augr:pkg:metalab:reup to roll metalab onto the fixed image). Diagnose the issue afterward from the restored agent.
Runtime state corrupted	Restore from checkpoint (see runtime state preservation).
admin:teardown ran prematurely	Re-run admin:setup, augr:install, augr:build. Local-only source changes are lost unless they were on branches.

Runtime state independence

Each agent has its own runtime state under env/<ENV>/services/:

Agent	Data	State	Agent config
dev-agent	services/dev-agent/data/	services/dev-agent/state/	(workspace agent dir)
metalab-agent	services/metalab-agent/data/	services/metalab-agent/state/	services/metalab-agent/agent/

Phoenix data is shared (read by both agents, written by the phoenix container) but this is read-heavy and not a corruption risk.

Decision: Runtime state must be checkpointable and restorable per agent. The checkpoint mechanism and its integration with guarded operations are implementation concerns.

Observation requirements

The observation tool (augr:diagnos extended or a new dedicated command) must produce the following report.

Per source entity

Field	Description
Entity name	e.g. engine, autograph.pkg.agent
Entity type	submodule, west-package
State	One of: aligned, aligned-dirty, branched, branched-dirty, diverged, indeterminate
HEAD	Commit SHA (short), branch name if on branch, "detached" if detached
Reference	manifest-rev SHA or submodule ref SHA
Ahead/behind	Number of commits ahead of / behind reference
Staged	Count of staged changes
Unstaged	Count of unstaged changes
Untracked	Count of untracked files
Stashes	Count of stash entries
Local-only branches	Branch names that have no upstream tracking

System level

Field	Description
Root project	Branch, dirty state, pending submodule changes
Exchange zone	Pending proposals in exchange.lab.tmp/ or exchange.dev.tmp/
Container health	Per agent: running/stopped/absent, healthcheck status
Config freshness	.env.project age relative to env.d/ sources
Service directories	Per agent: exist/absent

Consequences

Positive

• Each source entity has a concrete, observable state that determines what operations are allowed.
• The aligned-dirty danger zone (editing on detached HEAD) is identified and guarded.
• The resilience invariant (one agent always available) is formalized with enforcement via observation.
• The shared image risk is documented with a staged rebuild rule and small-commit best practice.
• Guards are scoped to task commands with explicit force overrides - they protect without constraining the operator.

Negative / Costs

• Observation tooling must be built or augr:diagnos significantly extended. A shell script is a pragmatic starting point but may lack portability for other platforms.
• augr:update, augr:upgrade, and destructive commands gain preconditions that add friction.
• The staged rebuild rule adds manual steps when testing Dockerfile changes.

Required implementation

• Extend augr:diagnos (or create a new command) to produce the observation report defined above.
• Add precondition checks with force overrides to guarded task commands.
• Implement runtime state checkpoint and restore per agent.
• Implement staged agent rebuild (one at a time, verify health before the other).
• Update forge-workflow.md to reference the guard and observation rules from this ADR.
• Add a convenience command to reset West package remotes and verify reference alignment before development.
• Add a safe env.template refresh command that re-collects templates and rebuilds .env.project without destroying resolved deps or runtime state.

Deferred explorations

AUGR_ENV multiplexing for parallel environments

The current architecture uses a single AUGR_ENV value (typically dev). An interesting approach for isolated testing would be to create additional environment profiles (e.g. AUGR_ENV=lab-test-1) with their own service directories and generated config, allowing a second set of containers from modified sources without shutting down the current environment.

This requires: parameterized container names (currently hardcoded), parameterized ports (currently fixed), and network isolation between environments. Deferred. The current resilience model (dev-agent and metalab-agent as mutual fallbacks) provides the minimum safety.

Separate agent images

Eliminating the shared image risk by building separate images for dev-agent and metalab-agent, allowing independent rebuild without the staged constraint. Deferred until the shared image becomes a practical problem.

Non-goals

This ADR does not define:

• a full release platform or CI/CD pipeline;
• automated commits, pushes, or PR creation;
• multi-instance runtime orchestration (see E02);
• the forge workflow itself (see forge-workflow.md);
• fresh-install sandbox validation mechanics (deferred until entity state model and observation tooling are in place);
• engine version migration mechanisms for breaking interface changes.