# Actionable implementation plan

Implementation status (2026-07-23): phases 0–7 are implemented in the proto,
control, and client worktrees. Phase 8 has a passing local 2×2 happy path,
deterministic selective/recovery matrix, and adversarial v2/fault matrix:
complementary partial placements merge correctly, eviction uses archive
coverage unions, queued and active cancellation preserve their safety
boundaries, restarts replay durable work, and hostile preconditions fail
without committing. Pure-v2/hybrid round trips, recoverable stalls,
capacity-guard failure, and duplicate/reordered WebSocket events also pass from
a clean topology. The remaining fixture and fault categories listed below are
still release gates; phase 9 remains intentionally deferred until the complete
release matrix passes.

No application implementation begins until Phase 0 is reviewed. Within every
later phase, tests and documentation are part of the deliverable rather than a
follow-up task.

## Phase 0 — Freeze the contract

### Actions

1. Convert the decision ledger into the system, workflow, safety, service API,
   deployment, Telegram, and test documents in this directory.
2. Create the independent `archive-control-proto` Git repository.
3. Define all v1 resource, selection, route, inventory, job, progress,
   command/event, reconciliation, and envelope messages.
4. Add canonical protobuf-JSON examples for registration, inventory queries,
   route creation, transfer start/progress/commit, cancellation, eviction, and
   recovery snapshots.
5. Run Dockerized Buf format/lint and protoc descriptor compilation.
6. Configure the Gitea remote `cabbage/archive-control-proto` with its dedicated
   SSH key. Tag the reviewed baseline `v0.1.0` only after review.

### Acceptance gate

- No unresolved `TODO` changes wire meaning or destructive behavior.
- Every workflow transition maps to a protobuf command/event.
- JSON examples parse through generated descriptors.
- Documentation and protobuf enums/state machines agree.

## Phase 1 — Build the control core inside mogic-bot

### Actions

1. Add `archive_control/` parallel to `remote_control/`, with no imports from
   Telegram in its core service, scheduler, protocol, persistence, or
   orchestration layers.
2. Add strict configuration parsing to the existing bot configuration,
   including shared token, WebSocket bind/path, database/backup paths,
   heartbeat, scheduling, route policy, retention, and Telegram refresh knobs.
3. Add SQLite schema migrations for clients, connections, routes, jobs,
   commands, job events, reservations, placements, UI sessions, and
   tombstones. Enable foreign keys, WAL, busy timeout, and transactional schema
   migration.
4. Implement online backups, integrity verification, retention, and offline
   verify/restore commands.
5. Implement WebSocket registration-first authentication, duplicate-client
   replacement, heartbeats, bounded send queues, envelope validation, and
   protocol version negotiation.
6. Implement durable command outbox/inbox semantics, acknowledgements,
   per-job event sequencing, snapshots, and reconnect reconciliation.
7. Implement the global resource reservation and FIFO per-client/per-route
   scheduler.
8. Add the thin test-only HTTP adapter over `ArchiveControlService`.

### Acceptance gate

- Bot startup/shutdown cleanly owns Archive Control tasks in its existing
  asyncio lifecycle.
- Existing bot tests and workflows are unchanged.
- Restart, duplicate registration, lost acknowledgement, event duplication,
  sequence gap, and DB backup/restore tests pass.
- The test HTTP adapter can create/query/cancel synthetic jobs without Telegram.

## Phase 2 — Build the archive-client foundation

### Actions

1. Scaffold `archive-clients` with a `src/` layout, Python 3.11, strict TOML
   parsing, environment/file secrets, CLI overrides, structured redacted logs,
   and console entry points.
2. Implement `--mode archive|cache`, stable client ID, state/backup DBs, and
   Docker-friendly shutdown handling.
3. Implement capped exponential reconnect with jitter, registration,
   heartbeat acknowledgements, one-writer WebSocket output, durable command
   deduplication, and active-job snapshots.
4. Implement API-visible-to-local root mapping and path-confinement helpers.
5. Probe qBittorrent, Syncthing, filesystem/link/reflink, sparse, and permission
   capabilities at startup; advertise a precise health/capability report.
6. Add multi-stage Dockerfile and Compose example. Run unprivileged and mount
   only config, secrets, DB/backups, qB data, and sync data.
7. Build amd64 first; keep the Dockerfile architecture-neutral for the release
   buildx phase.

### Acceptance gate

- Invalid/unknown configuration fails before connecting.
- Secrets never appear in logs or SQLite.
- Restart and duplicate-command tests prove idempotent client behavior.
- Both daemon roles start from the same image and differ only by config/mode.

## Phase 3 — Service adapters and on-demand discovery

### Actions

1. Implement qBittorrent cookie authentication and capability adapters for
   supported 4.5–4.6 and 5.x APIs.
2. Implement on-demand torrent summaries, scoped hash lookup, lazy complete
   content trees, v1/v2 identity validation, torrent export, stopped add,
   selection application, recheck monitoring, and entry-only deletion.
3. Implement Syncthing API-key authentication, version/config/status/events,
   folder/device discovery, per-file progress, need/file completion checks, and
   rescan support.
4. Discover only healthy pairwise `sendreceive` routes under the configured
   roots.
5. Implement idempotent device pairing and folder creation using advertised
   addresses, plus bidirectional nonce verification and 30-minute default
   timeout.
6. Implement scoped, chunked, revisioned inventory and tree queries; do not add
   background cluster inventory polling.

### Acceptance gate

- Contract fixtures cover API version differences and permanent/transient
  errors.
- A pair of isolated client/Syncthing stacks can auto-create and verify a route.
- Large synthetic inventories stream in bounded chunks and interrupted results
  are not committed by Titan.

## Phase 4 — Safe file operation and manifest layer

### Actions

1. Define the on-disk job staging layout and atomically written protobuf-JSON
   transfer manifest/ready marker.
2. Implement safe traversal using normalized torrent paths, no-follow checks,
   root confinement, regular-file-only validation, and duplicate/case-collision
   detection.
3. Implement hardlink → reflink → safe-copy fallback with per-file journaling,
   progress, cancellation points, and restart reconciliation.
4. Implement exact logical/allocated size accounting, free-space/reserve checks,
   and startup sparse capability enforcement.
5. Implement target never-overwrite behavior and provenance for created versus
   pre-existing reused files.
6. Implement tested qBittorrent/libtorrent partfile adapters, compatibility
   gates, baseline backups, and safe merge/rollback behavior.

### Acceptance gate

- Tests cover cross-filesystem fallback, insufficient space, permission errors,
  symlink races, special files, path escape, pre-existing collisions, sparse
  artifacts, and supported/unsupported partfiles.
- Cancelling/restarting every file-operation phase leaves a deterministic
  journal and no uncontrolled deletion.

## Phase 5 — Transfer orchestration

### Actions

1. Implement fresh source/target tree preflight, requested-selection expansion,
   immutable file-index snapshots, exact delta calculation, and no-op rejection.
2. Implement source staging, per-job Syncthing progress, destination completion
   proof, and target materialization.
3. Implement new-target stopped/manual add and existing-target baseline/delta
   transaction.
4. Apply selected/skipped state only, recheck the union while stopped, monitor
   for any download attempt, and restore/start the correct run state.
5. Commit placement generation only after verification; treat staging removal
   as post-commit cleanup with retryable `CLEANUP_REQUIRED` state.
6. Implement cooperative cancellation/compensation for every pre-commit phase
   and non-destructive post-commit cleanup behavior.

### Acceptance gate

- Full, selective, subset, and incremental merge scenarios pass in both
  directions.
- A target never downloads data during verification.
- Failure injection at every transition preserves source and target baselines.
- Archive jobs retain their source cache placements.

## Phase 6 — Eviction and placement reconciliation

### Actions

1. Implement on-demand cache placement listing and fresh archive coverage sets.
2. Require complete per-file coverage across one or more online archive nodes;
   expose disabled reasons and provide no force override.
3. Implement eviction jobs: coverage recheck, qB entry-only removal, remaining
   torrent exact-path overlap snapshot, safe file unlink, partfile ownership
   check, and non-recursive bottom-up directory removal.
4. Reconcile observed placements with successful job generations and external
   qBittorrent changes without background polling.

### Acceptance gate

- Eviction cannot proceed with any uncovered selected file.
- Shared exact paths, shared directories, unknown files, and unrelated
  partfiles survive.
- Retrying after interruption is idempotent and an archive placement is never
  deleted.

## Phase 7 — Telegram integration

### Actions

1. Add `ArchiveControlHandler` without changing existing handler behavior.
2. Implement Archive, Unarchive, Evict Cache, and Job Status flows with one
   edited message, pagers, chained literal filters, cancel/back behavior, and
   single-use confirmation nonces.
3. Implement normal-all and one-entry tree selection; keep protobuf support for
   multiple selections.
4. Persist UI sessions and restore live status subscriptions after bot restart.
5. Inherit private-chat username authorization and optionally pin configured
   numeric IDs. Keep jobs shared and ownerless.
6. Render five-step transfer and three-step eviction progress, including stale,
   waiting, stalled percentage, cleanup-required, failure reason, paging,
   Clear Finished, and Remove One.
7. Coalesce routine edits to the configurable five-second default and handle
   Telegram rate limits.

### Acceptance gate

- Telegram and test HTTP adapters produce identical service-layer outcomes.
- Stale/double callbacks are harmless.
- Bot restart preserves active UI/status behavior.
- Existing mogic-bot regression tests pass.

## Phase 8 — Cross-project E2E and chaos testing

### Actions

1. Create five Compose projects under this repository's `e2e/`: control,
   cache-1, cache-2, archive-1, and archive-2.
2. Give every data node isolated qBittorrent, Syncthing, archive-client, config,
   state, backup, and data volumes on one dedicated test network.
3. Configure static advertised Syncthing service addresses while testing
   automatic device/folder pairing.
4. Generate small v1/v2/hybrid torrents and fixtures for complementary and
   overlapping selections, partfiles, sparse files, path collisions, shared
   directories, and permission errors.
5. Drive scenarios with curl against the test HTTP adapter.
6. Inject control/client/service restarts, network loss, duplicate commands,
   stale snapshots, stalls, disk exhaustion, cancellation, and DB restore.
7. Tear down only test resources identified by unique project names/labels.

### Acceptance gate

- The complete acceptance matrix in `testing.md` passes repeatedly without
  live infrastructure.
- No test uses credentials or paths from x1, x2, lithium, or Titan production.
- Test teardown is scope-validated and leaves unrelated Docker resources alone.

## Phase 9 — Release and staged deployment

### Actions

1. Run Buf breaking checks against the last proto release and tag compatible
   repositories.
2. Build `sodium/archive-clients` for linux/amd64 and linux/arm64 with OCI
   source/protocol labels; publish an immutable semantic-version tag and the
   stable `latest` alias.
3. Record image digest and provide pinned Compose examples.
4. Back up real qBittorrent/Syncthing and Archive Control state before a
   separately approved deployment.
5. Deploy one cache/archive pair in observation mode, validate on-demand
   inventory and route discovery, then run a small non-critical canary transfer.
6. Exercise recovery and eviction only after canary verification and explicit
   approval, then expand node-by-node.

### Acceptance gate

- Multi-platform manifest resolves to tested images and the runtime image is
  reasonably small.
- Rollback instructions and prior images/config/DB backups are verified.
- No production qBittorrent/Syncthing global setting is modified.