85 lines
4.7 KiB
Markdown
85 lines
4.7 KiB
Markdown
# Archive Clients
|
||
|
||
One Python daemon runs as either an `archive` or `cache` node for Archive
|
||
Control. Both roles use the same code and `sodium/archive-clients` image.
|
||
|
||
The current foundation provides strict TOML configuration, file-backed secrets,
|
||
API/local root mapping, fail-fast permission and sparse-file probes, a durable
|
||
SQLite command inbox, bounded one-writer WebSocket output, registration-first
|
||
authentication, heartbeat handling, duplicate-command acknowledgements, and
|
||
indefinite capped exponential reconnect with optional jitter. The state DB has
|
||
an exclusive process lease plus checksummed, integrity-verified online backups
|
||
with recent/daily/weekly retention.
|
||
Startup probes report qBittorrent/Web API/libtorrent versions, Syncthing
|
||
version/device identity, service health, and per-root hardlink/reflink/sparse
|
||
support without exposing local paths to control.
|
||
|
||
The first read-only discovery layer strictly decodes v1/v2/hybrid metainfo,
|
||
derives canonical resource identities and selection ranges, builds lazy content
|
||
trees, and reports only safe Syncthing folders below the configured root.
|
||
Renamed/noncanonical torrent paths remain visible but are marked noncanonical
|
||
so later job preflight can reject them without hiding the resource.
|
||
The qBittorrent adapter uses cookie authentication, bounded responses, one
|
||
reauthentication attempt on session expiry, and hash-scoped file/metainfo
|
||
fetches. Its mutation boundary adds targets stopped, applies only selected vs
|
||
skipped state, supports qBittorrent 4 pause and qBittorrent 5 stop semantics,
|
||
performs guarded full rechecks, and removes entries without requesting data
|
||
deletion. It never logs credentials, cookies, response bodies, or endpoints.
|
||
Durable inventory commands now stream bounded atomic summary, lookup, and
|
||
content-tree chunks. Page tokens are guarded by timestamp-independent snapshot
|
||
revisions, stale trees fail explicitly, and slow scans run outside the socket
|
||
reader so heartbeat acknowledgements remain responsive.
|
||
Durable `EnsureRoute` commands use scoped Syncthing device/folder REST updates,
|
||
safe API/local path mapping, exact pairwise configuration read-back, and
|
||
bidirectional nonce/acknowledgement files. Route attempt nonces, ownership, and
|
||
ordered updates survive restart; accepted attempts resume on reconnect without
|
||
concurrent duplicate execution. Conflicting existing folders and foreign data
|
||
are reported without being overwritten.
|
||
The transfer storage foundation now publishes isolated per-job namespaces with
|
||
canonical protobuf-JSON manifests and digest-bound ready markers. Selected
|
||
regular files and metainfo artifacts are materialized with hardlink, reflink,
|
||
then sparse-aware copy fallback; every intent/result is journaled in SQLite.
|
||
Receiver materialization never overwrites a path, can reuse a same-size regular
|
||
file for later qBittorrent verification, and replays completed operations
|
||
idempotently after a database reopen.
|
||
The Syncthing transfer observer requests scoped rescans and combines verified
|
||
local manifest payloads, folder completion, and outstanding job-prefix needs;
|
||
an advisory percentage alone is never treated as transfer completion.
|
||
|
||
```bash
|
||
archive-client --config /etc/archive-control/client.toml --check-config
|
||
archive-client --config /etc/archive-control/client.toml --mode archive
|
||
archive-client-backup --database /var/lib/archive-control/client.db \
|
||
--backup-dir /var/backups/archive-control list
|
||
```
|
||
|
||
`--mode` accepts only `archive` or `cache` and overrides the configured role.
|
||
Secrets must be regular files without group/world permissions. The daemon never
|
||
stores them in SQLite or sends the shared token after registration.
|
||
|
||
The daemon executes heartbeat, inventory, state-snapshot, route-provisioning,
|
||
archive/unarchive assignment and step commands, compensation, staging cleanup,
|
||
and safe cache eviction. Durable journals and job cursors make interrupted
|
||
work replayable; unsupported or invalid assignments are rejected explicitly
|
||
and never falsely acknowledged as accepted.
|
||
|
||
Run tests and build using containers:
|
||
|
||
```bash
|
||
docker build --target test -t archive-clients-test .
|
||
docker run --rm archive-clients-test
|
||
docker build -t sodium/archive-clients:dev .
|
||
```
|
||
|
||
The local cross-project route test is under `e2e/`. It brings up isolated
|
||
2×2 cache/archive node stacks plus the standalone control core and verifies all
|
||
four eager-mesh routes, transfer/eviction/unarchive workflows, and the
|
||
adversarial matrix without live infrastructure or Telegram. See
|
||
`e2e/README.md`.
|
||
|
||
The complete cross-project design, protocol, workflow, safety, deployment,
|
||
Telegram UX, and testing documentation is published under [`docs/`](docs/).
|
||
|
||
Generated bindings are pinned to archive-control-proto commit
|
||
`4ec852014dad74606d4078b3ae1aa208c814b033`.
|