archive-clients/docs/system-design.md

System design

Scope and invariants

Archive Control moves already-complete BitTorrent file selections between hot cache nodes and durable archive nodes. It does not download missing content, change global qBittorrent or Syncthing settings, or decide retention on archive nodes.

The non-negotiable invariants are:

1. A committed archive copy is never deleted by a transfer rollback or cleanup.
2. A cache placement is evicted only by an explicit eviction job after fresh proof that every selected file is covered by online archive placements.
3. A target is committed only after qBittorrent verifies the selected union in stopped state without attempting a download.
4. One control daemon is the only cross-node orchestrator. Clients execute one commanded local step at a time.
5. Every destructive transition is durable, attributable, retryable, and guarded by an expected revision or external-state fingerprint.
6. File operations stay beneath configured roots, never follow symlinks, and never recursively delete a content tree.

Components

Archive control core

The existing mogic-bot process hosts a new archive_control/ package parallel to remote_control/. Its core has no Telegram imports and exposes an ArchiveControlService facade used by:

• the production Telegram handler;
• a test-only HTTP adapter used by local E2E tests;
• the WebSocket connection manager for clients.

Internally the core contains configuration, SQLite persistence, connection registry, durable command outbox, protocol codec, scheduler, route coordinator, inventory coordinator, job state machines, backup manager, and event/view projection. The existing bot lifecycle starts and stops these tasks; no second bot or independent control process is introduced.

Archive client

One Python image and codebase runs with role archive or cache. Each instance contains:

• a reconnecting control connection and durable command inbox;
• a local SQLite journal and backup manager;
• qBittorrent and Syncthing adapters;
• safe path and file-operation modules;
• route discovery/provisioning;
• inventory and content-tree responders;
• transfer and eviction step executors.

The role controls policy, not the available transport mechanics. Cache clients may source archive jobs, receive unarchive jobs, and perform eviction. Archive clients may receive archive jobs and source unarchive jobs.

External local services

Clients talk to their locally configured qBittorrent and Syncthing APIs. API paths may differ from paths inside the client container, so each root is configured as an API-visible path plus its daemon-local equivalent. The mapping must be one-to-one and is validated at startup.

Logical data flow

Telegram handler ─┐
                  ├─ ArchiveControlService ─ SQLite/control scheduler
test HTTP adapter ┘             │
                                │ versioned protobuf JSON over WebSocket
               ┌────────────────┴─────────────────┐
        cache client                         archive client
        ├─ qB adapter                        ├─ qB adapter
        ├─ Syncthing adapter                 ├─ Syncthing adapter
        ├─ SQLite journal                    ├─ SQLite journal
        └─ local file ops                    └─ local file ops
               └──────── pairwise Syncthing route ────────┘

The control service persists intent and the job's current event-sequence lease before emitting a command. The client persists command acceptance before acknowledging it. Completion is a later event. This separates delivery from execution and makes reconnect replay safe.

Persistent model

Control database

The control database contains at least:

• clients and last known capabilities/health;
• connection generations and heartbeat timestamps;
• discovered/provisioned routes and verification state;
• immutable job definitions and mutable job projections;
• per-job ordered events;
• durable commands, delivery attempts, and acknowledgements;
• resource reservations;
• cache/archive placements and generations;
• bounded inventory snapshots and revisions;
• persisted Telegram sessions and confirmation nonces;
• hidden-record tombstones and backup metadata.

Client database

Each client stores:

• accepted command IDs and their stable acknowledgements;
• job definitions, local step journals, revisions, and emitted events;
• file-operation provenance and compensation state;
• staging manifests and ready-marker observations;
• route-creation ownership and verification state;
• last control connection/session cursors;
• backup metadata.

Credentials, shared tokens, qBittorrent passwords, Syncthing API keys, and Telegram secrets are never stored in either database.

Placements are not job history

A placement is current, verified residency of a resource selection on one node. It is keyed by canonical resource identity and client ID and includes a generation plus verified file-index set. Jobs mutate placements, but deleting or hiding a job record never changes placement data. A resource may have many archive and cache placements.

Scheduling and ownership

Jobs enter a durable FIFO queue while holding a global reservation on every known v1/v2 alias of their resource identity. Only one transfer or eviction for a resource may be active or queued at a time. This prevents incompatible baselines even when jobs would use different nodes or observe different sides of a hybrid identity.

Defaults allow one active data-moving job per client and one per route. A job must acquire its source client, target client, route, and resource reservation atomically. Disjoint node pairs may run concurrently. Route setup is a preflight activity and does not permit a data step to bypass these leases.

Offline nodes do not prevent unrelated jobs from being listed or run. A job requiring an offline node remains waiting indefinitely; it does not consume an active data slot until work can proceed.

State and recovery

Control state is authoritative for orchestration; neither client independently advances to the next step. On reconnect:

1. The client registers with active-job cursors.
2. Control compares revisions and per-job event sequences.
3. Missing events are replayed when available; gaps or disagreement trigger a full client/job snapshot.
4. Control observes relevant qBittorrent, Syncthing, staging, and manifest state before selecting retry, resume, compensation, cleanup, or manual intervention.
5. A command is reissued with its original ID when the acceptance result is uncertain.

A client never guesses that a job committed. If a database is lost, evidence from a verified target, immutable manifest, ready marker, and file-operation journal may reconstruct state only when it proves one unique outcome. Otherwise the resource remains reserved and the job fails closed for manual reconciliation.

Routine no-progress and connectivity conditions become WAITING or STALLED, not failures. Permanent protocol, validation, integrity, unsafe-path, unsupported-partfile, or permission conditions fail with a stable code and a clear operator-facing reason.