75 lines
4.0 KiB
Markdown
75 lines
4.0 KiB
Markdown
# Archive Clients
|
||
|
||
One Python daemon runs as either an `archive` or `cache` node for Archive
|
||
Control. Both roles use the same code and `sodium/archive-clients` image.
|
||
|
||
The current foundation provides strict TOML configuration, file-backed secrets,
|
||
API/local root mapping, fail-fast permission and sparse-file probes, a durable
|
||
SQLite command inbox, bounded one-writer WebSocket output, registration-first
|
||
authentication, heartbeat handling, duplicate-command acknowledgements, and
|
||
indefinite capped exponential reconnect with optional jitter. The state DB has
|
||
an exclusive process lease plus checksummed, integrity-verified online backups
|
||
with recent/daily/weekly retention.
|
||
Startup probes report qBittorrent/Web API/libtorrent versions, Syncthing
|
||
version/device identity, service health, and per-root hardlink/reflink/sparse
|
||
support without exposing local paths to control.
|
||
|
||
The first read-only discovery layer strictly decodes v1/v2/hybrid metainfo,
|
||
derives canonical resource identities and selection ranges, builds lazy content
|
||
trees, and reports only safe Syncthing folders below the configured root.
|
||
Renamed/noncanonical torrent paths remain visible but are marked noncanonical
|
||
so later job preflight can reject them without hiding the resource.
|
||
The qBittorrent read adapter uses cookie authentication, bounded responses,
|
||
one reauthentication attempt on session expiry, hash-scoped file/metainfo
|
||
fetches, and never logs credentials, cookies, response bodies, or endpoints.
|
||
Durable inventory commands now stream bounded atomic summary, lookup, and
|
||
content-tree chunks. Page tokens are guarded by timestamp-independent snapshot
|
||
revisions, stale trees fail explicitly, and slow scans run outside the socket
|
||
reader so heartbeat acknowledgements remain responsive.
|
||
Durable `EnsureRoute` commands use scoped Syncthing device/folder REST updates,
|
||
safe API/local path mapping, exact pairwise configuration read-back, and
|
||
bidirectional nonce/acknowledgement files. Route attempt nonces, ownership, and
|
||
ordered updates survive restart; accepted attempts resume on reconnect without
|
||
concurrent duplicate execution. Conflicting existing folders and foreign data
|
||
are reported without being overwritten.
|
||
The transfer storage foundation now publishes isolated per-job namespaces with
|
||
canonical protobuf-JSON manifests and digest-bound ready markers. Selected
|
||
regular files and metainfo artifacts are materialized with hardlink, reflink,
|
||
then sparse-aware copy fallback; every intent/result is journaled in SQLite.
|
||
Receiver materialization never overwrites a path, can reuse a same-size regular
|
||
file for later qBittorrent verification, and replays completed operations
|
||
idempotently after a database reopen.
|
||
|
||
```bash
|
||
archive-client --config /etc/archive-control/client.toml --check-config
|
||
archive-client --config /etc/archive-control/client.toml --mode archive
|
||
archive-client-backup --database /var/lib/archive-control/client.db \
|
||
--backup-dir /var/backups/archive-control list
|
||
```
|
||
|
||
`--mode` accepts only `archive` or `cache` and overrides the configured role.
|
||
Secrets must be regular files without group/world permissions. The daemon never
|
||
stores them in SQLite or sends the shared token after registration.
|
||
|
||
The daemon currently executes heartbeat, inventory, state-snapshot, and
|
||
route-provisioning commands. Transfer storage primitives are implemented and
|
||
tested but are not yet wired to assignment/step commands; those and eviction
|
||
commands remain durably rejected as unsupported until their orchestration and
|
||
qBittorrent executors are added. They are never falsely acknowledged as
|
||
accepted.
|
||
|
||
Run tests and build using containers:
|
||
|
||
```bash
|
||
docker build --target test -t archive-clients-test .
|
||
docker run --rm archive-clients-test
|
||
docker build -t sodium/archive-clients:dev .
|
||
```
|
||
|
||
The local cross-project route test is under `e2e/`. It brings up isolated
|
||
2×2 cache/archive node stacks plus the standalone control core and verifies all
|
||
four eager-mesh routes without Telegram. See `e2e/README.md`.
|
||
|
||
Generated bindings are pinned to archive-control-proto commit
|
||
`4ec852014dad74606d4078b3ae1aa208c814b033`.
|