273 lines
9.2 KiB
Markdown
273 lines
9.2 KiB
Markdown
# Configuration, deployment, and usage
|
|
|
|
## Configuration rules
|
|
|
|
Both daemons parse strict TOML: unknown keys, duplicate keys, invalid types,
|
|
unsafe relative paths, and missing required secrets fail startup. Effective
|
|
precedence is command line, environment, TOML, then documented default. Secret
|
|
fields support a direct environment value or a `*_file` path, never both.
|
|
Environment interpolation is explicit `${NAME}` syntax; an unset value fails.
|
|
|
|
`--print-effective-config` emits a validated, fully resolved configuration with
|
|
all secrets redacted. Durations accept a documented unit string. Paths shown to
|
|
the APIs and paths visible inside the daemon are configured as pairs and tested
|
|
for consistent mapping before registration.
|
|
|
|
Important defaults:
|
|
|
|
- registration deadline `10s`;
|
|
- heartbeat interval `15s`, offline timeout `45s`;
|
|
- unacknowledged command retry timeout `15s`, three total attempts;
|
|
- reconnect `1s` initial, `60s` cap, jitter enabled;
|
|
- route policy `on_demand`, setup timeout `30m`;
|
|
- stall warning `30m`, with no automatic failure timeout;
|
|
- one active data job per client and route;
|
|
- Telegram coalesce window `5s`, UI session TTL `24h`;
|
|
- tombstone retention `30d`;
|
|
- database backup interval `6h`, retention `12/14/8`.
|
|
|
|
## Control configuration example
|
|
|
|
```toml
|
|
[archive_control]
|
|
enabled = true
|
|
listen = "127.0.0.1:8765"
|
|
websocket_path = "/archive_control"
|
|
shared_token_file = "/run/secrets/archive_control_token"
|
|
database = "/var/lib/mogic/archive-control.db"
|
|
backup_dir = "/var/backups/mogic/archive-control"
|
|
registration_timeout = "10s"
|
|
heartbeat_interval = "15s"
|
|
offline_timeout = "45s"
|
|
command_ack_timeout = "15s" # Delay before resending the same command ID.
|
|
command_max_attempts = 3 # Total sends, including the initial attempt.
|
|
stall_after = "30m" # Warning state only; jobs continue waiting.
|
|
route_policy = "on_demand" # Alternative: eager_mesh.
|
|
route_setup_timeout = "30m"
|
|
max_active_per_client = 1
|
|
max_active_per_route = 1
|
|
max_envelope_bytes = 1048576
|
|
inventory_chunk_bytes = 524288
|
|
|
|
[archive_control.backup]
|
|
interval = "6h"
|
|
recent = 12
|
|
daily = 14
|
|
weekly = 8
|
|
|
|
[archive_control.telegram]
|
|
progress_coalesce = "5s"
|
|
session_ttl = "24h"
|
|
tombstone_retention = "30d"
|
|
# Existing username allowlist remains authoritative when no ID is pinned.
|
|
admin_user_ids = { "@alice" = 123456789 }
|
|
```
|
|
|
|
The test-only HTTP adapter has a separate `test_http.enabled` flag that is
|
|
false by default and omitted from production Compose configuration.
|
|
|
|
```toml
|
|
[archive_control.test_http]
|
|
enabled = false # Enable only in the isolated local E2E control stack.
|
|
listen = "127.0.0.1:0" # Non-loopback binds are rejected.
|
|
max_body_bytes = 1048576
|
|
```
|
|
|
|
## Client configuration examples
|
|
|
|
All nodes use the same image. The examples show two active cache and two active
|
|
archive nodes. Values are illustrative; credentials are mounted secrets.
|
|
|
|
```toml
|
|
# cache-1.toml
|
|
client_id = "cache-1"
|
|
display_name = "Cache 1"
|
|
role = "cache"
|
|
control_endpoint = "ws://control:8765/archive_control"
|
|
shared_token_file = "/run/secrets/archive_control_token"
|
|
state_db = "/var/lib/archive-control/client.db"
|
|
backup_dir = "/var/backups/archive-control"
|
|
|
|
[connection]
|
|
registration_timeout = "10s" # First response must arrive within this window.
|
|
heartbeat_interval = "15s" # Server may negotiate a different effective value.
|
|
offline_timeout = "45s"
|
|
reconnect_initial = "1s"
|
|
reconnect_max = "60s" # Retry forever, never wait longer than this.
|
|
reconnect_reset_after = "60s"
|
|
reconnect_jitter = true
|
|
|
|
[jobs]
|
|
stall_after = "30m" # Warning only; no automatic job failure.
|
|
verification_timeout = "30m" # Stopped qB full-recheck deadline.
|
|
poll_interval = "1s" # Active qB/Syncthing observation interval.
|
|
# Worst-case copy fallback must leave this many bytes free.
|
|
free_space_reserve_bytes = 33554432 # 32 MiB; payload bytes are added for copy paths.
|
|
|
|
[backup]
|
|
interval = "6h"
|
|
recent = 12
|
|
daily = 14
|
|
weekly = 8
|
|
|
|
[qbittorrent]
|
|
endpoint = "http://qbittorrent:8080"
|
|
username = "${QB_USER}"
|
|
password_file = "/run/secrets/qb_password"
|
|
api_root = "/downloads"
|
|
local_root = "/data/qb"
|
|
|
|
[syncthing]
|
|
endpoint = "http://syncthing:8384"
|
|
api_key_file = "/run/secrets/syncthing_api_key"
|
|
api_root = "/sync"
|
|
local_root = "/data/sync"
|
|
advertised_addresses = ["dynamic"]
|
|
```
|
|
|
|
When an existing Syncthing folder is physically nested in the qB data root,
|
|
use a `local_path_overrides` entry to map that exact Syncthing API path through
|
|
the same client bind mount. This enables hardlinks without creating two Docker
|
|
mount boundaries for the same host files.
|
|
|
|
The remaining node examples omit optional `[connection]`, `[jobs]`, and
|
|
`[backup]` tables and therefore use these same defaults; deployments may
|
|
override them per node.
|
|
|
|
```toml
|
|
# cache-2.toml: same service layout, distinct stable identity/state volumes.
|
|
client_id = "cache-2"
|
|
display_name = "Cache 2"
|
|
role = "cache"
|
|
control_endpoint = "ws://control:8765/archive_control"
|
|
shared_token_file = "/run/secrets/archive_control_token"
|
|
state_db = "/var/lib/archive-control/client.db"
|
|
backup_dir = "/var/backups/archive-control"
|
|
|
|
[qbittorrent]
|
|
endpoint = "http://qbittorrent:8080"
|
|
username = "${QB_USER}"
|
|
password_file = "/run/secrets/qb_password"
|
|
api_root = "/downloads"
|
|
local_root = "/data/qb"
|
|
|
|
[syncthing]
|
|
endpoint = "http://syncthing:8384"
|
|
api_key_file = "/run/secrets/syncthing_api_key"
|
|
api_root = "/sync"
|
|
local_root = "/data/sync"
|
|
advertised_addresses = ["dynamic"]
|
|
```
|
|
|
|
```toml
|
|
# archive-1.toml
|
|
client_id = "archive-1"
|
|
display_name = "Archive 1"
|
|
role = "archive"
|
|
control_endpoint = "ws://control:8765/archive_control"
|
|
shared_token_file = "/run/secrets/archive_control_token"
|
|
state_db = "/var/lib/archive-control/client.db"
|
|
backup_dir = "/var/backups/archive-control"
|
|
|
|
[qbittorrent]
|
|
endpoint = "http://qbittorrent:8080"
|
|
username = "${QB_USER}"
|
|
password_file = "/run/secrets/qb_password"
|
|
api_root = "/archive"
|
|
local_root = "/data/archive"
|
|
|
|
[syncthing]
|
|
endpoint = "http://syncthing:8384"
|
|
api_key_file = "/run/secrets/syncthing_api_key"
|
|
api_root = "/sync"
|
|
local_root = "/data/sync"
|
|
advertised_addresses = ["dynamic"]
|
|
```
|
|
|
|
```toml
|
|
# archive-2.toml
|
|
client_id = "archive-2"
|
|
display_name = "Archive 2"
|
|
role = "archive"
|
|
control_endpoint = "ws://control:8765/archive_control"
|
|
shared_token_file = "/run/secrets/archive_control_token"
|
|
state_db = "/var/lib/archive-control/client.db"
|
|
backup_dir = "/var/backups/archive-control"
|
|
|
|
[qbittorrent]
|
|
endpoint = "http://qbittorrent:8080"
|
|
username = "${QB_USER}"
|
|
password_file = "/run/secrets/qb_password"
|
|
api_root = "/archive"
|
|
local_root = "/data/archive"
|
|
|
|
[syncthing]
|
|
endpoint = "http://syncthing:8384"
|
|
api_key_file = "/run/secrets/syncthing_api_key"
|
|
api_root = "/sync"
|
|
local_root = "/data/sync"
|
|
advertised_addresses = ["dynamic"]
|
|
```
|
|
|
|
No explicit route matrix is configured. Clients discover existing compatible
|
|
folders, report their Syncthing device ID, and control creates missing
|
|
cache/archive routes according to policy.
|
|
|
|
## Client Compose example
|
|
|
|
```yaml
|
|
services:
|
|
archive-client:
|
|
image: sodium/archive-clients:v0.1.8
|
|
user: "1001:1001"
|
|
restart: unless-stopped
|
|
command: ["archive-client", "--config", "/etc/archive-control/client.toml"]
|
|
environment:
|
|
QB_USER: admin
|
|
volumes:
|
|
- ./client.toml:/etc/archive-control/client.toml:ro
|
|
- ./secrets:/run/secrets:ro
|
|
- ./state:/var/lib/archive-control
|
|
- ./backups:/var/backups/archive-control
|
|
- ./qb-data:/data/qb
|
|
- ./sync-data:/data/sync
|
|
```
|
|
|
|
Archive nodes mount `/data/archive` instead of `/data/qb`. Mounts must line up
|
|
with the configured local roots and the service containers' API-visible roots.
|
|
The daemon runs unprivileged; directory ownership is prepared by the operator,
|
|
and mismatch causes fail-fast startup.
|
|
|
|
## Image and release policy
|
|
|
|
The production client image contains application source and dependencies; it
|
|
does not bind-mount source code. A multi-stage Dockerfile uses a slim Python
|
|
base and removes build-only packages/caches. Buildx publishes
|
|
`sodium/archive-clients:<semver>` for `linux/amd64` and `linux/arm64`, records
|
|
the multi-platform digest, and may update `latest`. Deployments pin the
|
|
immutable semantic tag or digest.
|
|
|
|
Generated protobuf Python bindings are committed in each consumer with the
|
|
exact `archive-control-proto` tag/commit recorded. Release order is proto,
|
|
control consumer, client consumer, E2E, then image publication.
|
|
|
|
## Operator usage
|
|
|
|
1. Prepare local qB, sync, state, backup, config, and secret mounts with the
|
|
container UID/GID (default example `1001:1001`).
|
|
2. Run a config validation/effective-config command before starting.
|
|
3. Start clients. Confirm service health, sparse/root probes, and registration.
|
|
4. Allow route discovery/provisioning to complete when first needed.
|
|
5. Use Telegram Archive/Unarchive; review the exact source, target, and
|
|
selection before confirming.
|
|
6. Use Evict Cache separately only when its fresh coverage check is enabled.
|
|
7. Treat orange Cleanup Required as a committed transfer needing staging
|
|
cleanup; do not manually remove the target placement.
|
|
8. Verify database backups periodically with the offline tooling.
|
|
|
|
The relevant runtime must be stopped before restore. The control maintenance
|
|
entry point and the client's `archive-client-backup` command support `list`,
|
|
`verify <backup>`, and `restore <backup>`; restore verifies the copy, preserves
|
|
the replaced database and SQLite sidecars with a timestamped `suspect` name,
|
|
and installs the selected database atomically.
|