170 lines
7.2 KiB
Markdown
170 lines
7.2 KiB
Markdown
# System design
|
|
|
|
## Scope and invariants
|
|
|
|
Archive Control moves already-complete BitTorrent file selections between hot
|
|
cache nodes and durable archive nodes. It does not download missing content,
|
|
change global qBittorrent or Syncthing settings, or decide retention on archive
|
|
nodes.
|
|
|
|
The non-negotiable invariants are:
|
|
|
|
1. A committed archive copy is never deleted by a transfer rollback or cleanup.
|
|
2. A cache placement is evicted only by an explicit eviction job after fresh
|
|
proof that every selected file is covered by online archive placements.
|
|
3. A target is committed only after qBittorrent verifies the selected union in
|
|
stopped state without attempting a download.
|
|
4. One control daemon is the only cross-node orchestrator. Clients execute one
|
|
commanded local step at a time.
|
|
5. Every destructive transition is durable, attributable, retryable, and
|
|
guarded by an expected revision or external-state fingerprint.
|
|
6. File operations stay beneath configured roots, never follow symlinks, and
|
|
never recursively delete a content tree.
|
|
|
|
## Components
|
|
|
|
### Archive control core
|
|
|
|
The existing mogic-bot process hosts a new `archive_control/` package parallel
|
|
to `remote_control/`. Its core has no Telegram imports and exposes an
|
|
`ArchiveControlService` facade used by:
|
|
|
|
- the production Telegram handler;
|
|
- a test-only HTTP adapter used by local E2E tests;
|
|
- the WebSocket connection manager for clients.
|
|
|
|
Internally the core contains configuration, SQLite persistence, connection
|
|
registry, durable command outbox, protocol codec, scheduler, route coordinator,
|
|
inventory coordinator, job state machines, backup manager, and event/view
|
|
projection. The existing bot lifecycle starts and stops these tasks; no second
|
|
bot or independent control process is introduced.
|
|
|
|
### Archive client
|
|
|
|
One Python image and codebase runs with role `archive` or `cache`. Each instance
|
|
contains:
|
|
|
|
- a reconnecting control connection and durable command inbox;
|
|
- a local SQLite journal and backup manager;
|
|
- qBittorrent and Syncthing adapters;
|
|
- safe path and file-operation modules;
|
|
- route discovery/provisioning;
|
|
- inventory and content-tree responders;
|
|
- transfer and eviction step executors.
|
|
|
|
The role controls policy, not the available transport mechanics. Cache clients
|
|
may source archive jobs, receive unarchive jobs, and perform eviction. Archive
|
|
clients may receive archive jobs and source unarchive jobs.
|
|
|
|
### External local services
|
|
|
|
Clients talk to their locally configured qBittorrent and Syncthing APIs. API
|
|
paths may differ from paths inside the client container, so each root is
|
|
configured as an API-visible path plus its daemon-local equivalent. The
|
|
mapping must be one-to-one and is validated at startup.
|
|
|
|
## Logical data flow
|
|
|
|
```text
|
|
Telegram handler ─┐
|
|
├─ ArchiveControlService ─ SQLite/control scheduler
|
|
test HTTP adapter ┘ │
|
|
│ versioned protobuf JSON over WebSocket
|
|
┌────────────────┴─────────────────┐
|
|
cache client archive client
|
|
├─ qB adapter ├─ qB adapter
|
|
├─ Syncthing adapter ├─ Syncthing adapter
|
|
├─ SQLite journal ├─ SQLite journal
|
|
└─ local file ops └─ local file ops
|
|
└──────── pairwise Syncthing route ────────┘
|
|
```
|
|
|
|
The control service persists intent and the job's current event-sequence lease
|
|
before emitting a command. The client
|
|
persists command acceptance before acknowledging it. Completion is a later
|
|
event. This separates delivery from execution and makes reconnect replay safe.
|
|
|
|
## Persistent model
|
|
|
|
### Control database
|
|
|
|
The control database contains at least:
|
|
|
|
- `clients` and last known capabilities/health;
|
|
- connection generations and heartbeat timestamps;
|
|
- discovered/provisioned routes and verification state;
|
|
- immutable job definitions and mutable job projections;
|
|
- per-job ordered events;
|
|
- durable commands, delivery attempts, and acknowledgements;
|
|
- resource reservations;
|
|
- cache/archive placements and generations;
|
|
- bounded inventory snapshots and revisions;
|
|
- persisted Telegram sessions and confirmation nonces;
|
|
- hidden-record tombstones and backup metadata.
|
|
|
|
### Client database
|
|
|
|
Each client stores:
|
|
|
|
- accepted command IDs and their stable acknowledgements;
|
|
- job definitions, local step journals, revisions, and emitted events;
|
|
- file-operation provenance and compensation state;
|
|
- staging manifests and ready-marker observations;
|
|
- route-creation ownership and verification state;
|
|
- last control connection/session cursors;
|
|
- backup metadata.
|
|
|
|
Credentials, shared tokens, qBittorrent passwords, Syncthing API keys, and
|
|
Telegram secrets are never stored in either database.
|
|
|
|
### Placements are not job history
|
|
|
|
A placement is current, verified residency of a resource selection on one
|
|
node. It is keyed by canonical resource identity and client ID and includes a
|
|
generation plus verified file-index set. Jobs mutate placements, but deleting
|
|
or hiding a job record never changes placement data. A resource may have many
|
|
archive and cache placements.
|
|
|
|
## Scheduling and ownership
|
|
|
|
Jobs enter a durable FIFO queue while holding a global reservation on every
|
|
known v1/v2 alias of their resource identity. Only one transfer or eviction for
|
|
a resource may be active or queued at a time. This prevents incompatible
|
|
baselines even when jobs would use different nodes or observe different sides
|
|
of a hybrid identity.
|
|
|
|
Defaults allow one active data-moving job per client and one per route. A job
|
|
must acquire its source client, target client, route, and resource reservation
|
|
atomically. Disjoint node pairs may run concurrently. Route setup is a
|
|
preflight activity and does not permit a data step to bypass these leases.
|
|
|
|
Offline nodes do not prevent unrelated jobs from being listed or run. A job
|
|
requiring an offline node remains waiting indefinitely; it does not consume an
|
|
active data slot until work can proceed.
|
|
|
|
## State and recovery
|
|
|
|
Control state is authoritative for orchestration; neither client independently
|
|
advances to the next step. On reconnect:
|
|
|
|
1. The client registers with active-job cursors.
|
|
2. Control compares revisions and per-job event sequences.
|
|
3. Missing events are replayed when available; gaps or disagreement trigger a
|
|
full client/job snapshot.
|
|
4. Control observes relevant qBittorrent, Syncthing, staging, and manifest
|
|
state before selecting retry, resume, compensation, cleanup, or manual
|
|
intervention.
|
|
5. A command is reissued with its original ID when the acceptance result is
|
|
uncertain.
|
|
|
|
A client never guesses that a job committed. If a database is lost, evidence
|
|
from a verified target, immutable manifest, ready marker, and file-operation
|
|
journal may reconstruct state only when it proves one unique outcome. Otherwise
|
|
the resource remains reserved and the job fails closed for manual
|
|
reconciliation.
|
|
|
|
Routine no-progress and connectivity conditions become `WAITING` or `STALLED`,
|
|
not failures. Permanent protocol, validation, integrity, unsafe-path,
|
|
unsupported-partfile, or permission conditions fail with a stable code and a
|
|
clear operator-facing reason.
|