Skip to content

ADR: dry_run is not a scope lane — it's a pipeline-stage decision

  • Date: 2026-04-15
  • Status: Accepted
  • Iteration: v0.3 Iter 11

Context

The v0.2 architecture docs (ARCHITECTURE.md §6.1) showed a YAML scope example that included a dry_run: "*" lane alongside read and write:

scopes:
  read:  { allow: [...] }
  write: { allow: [...], deny: [...] }
  dry_run: "*"

The synthesized v0.3 plan (docs/plans/v0.3.md) flagged this as a documentation lie: backend/src/gaby/safety/scopes.py only models two lanes (Scope = Literal["read", "write"]), the authz evaluator only consults those two, and the 5 hypothesis property tests in tests/property/test_safety_invariants.py only exercise those two. Shipping a write-capable connector (Stripe, from Iter 15) before closing the drift would leave a claim in the docs that reviewers and downstream authors would trust — and which the implementation wouldn't honor.

Two remediation options were on the table:

  • (a) Add the dry_run lane to scopes.py, extend the authz matrix, extend the property tests.
  • (b) Remove dry_run from the scope DSL and document it as what it actually is — a pipeline-stage decision made at step 3 of the safety pipeline.

Decision

Option (b). Dry-run is not a lane. It is a runtime decision made at step 3 of the safety pipeline in ARCHITECTURE.md §6 and implemented in the connector.

  • The scope DSL (safety/scopes.py) continues to have exactly two lanes: read and write.
  • The pipeline already makes the dry-run decision at step 3: 
    dry_run = (autonomy ≠ act) OR (tool.dangerous AND not approved)
  • The tool manifest declares whether dry-run is supported via a supports_dry_run: bool field on ToolSpec. Connectors whose manifest says supports_dry_run=true must implement a simulate() method the host calls in lieu of the real tool when the pipeline decides to dry-run.
  • The Allow.dry_run field on the authz decision (introduced in v0.1) is the transport for that decision from the safety pipeline down to the MCP host.

Consequences

  • No change to safety/scopes.py, safety/authz.py, or the existing property tests. The current 100% line + branch coverage floor on safety/ is preserved.
  • ARCHITECTURE.md §6.1 loses the dry_run: "*" line from its illustrative YAML example and gains a cross-link to this ADR.
  • Connector authors writing a new write-capable connector (Stripe in Iter 15 is the first) follow the existing supports_dry_run + simulate() pattern documented in connectors/_contract/. The 8-test contract already covers dry-run support (test_dry_run_supported in ARCHITECTURE.md §12).
  • Future authors reading scope examples in a tenant's connectors.scopes blob will never see a dry_run key — and if they try to set one, the _lane_from_json parser ignores it (unknown top-level keys are dropped, per scopes.config_from_json).

Not chosen: (a) add the lane

Rejected because:

  1. It would add a third axis to a safety surface we hold to 100% coverage. New tests, new matrix rows, new authz branches — all of it at the highest test-bar in the codebase.
  2. It doesn't model anything the pipeline doesn't already know. The autonomy × tool.dangerous × approval-status cross product already determines whether to dry-run; a third scopes.dry_run lane would duplicate that decision in a second place.
  3. It encourages mis-configuration. A tenant that writes dry_run: { allow: ["users/*/delete"] } would believe they've restricted dry-run to user-deletion, when really the setting is a no-op and every tool still dry-runs on the pipeline decision.

Verification

  • ARCHITECTURE.md §6.1 updated in this commit; no dry_run string appears in the scope-YAML example anywhere in the repo.
  • scopes.py is unchanged; coverage gate still green.
  • Future connector tests exercise dry-run via the existing test_dry_run_supported contract row.