archive-clients/docs/decisions.md

# Agreed design decisions

This is the concise decision ledger from the design interview. Later documents
expand these rules but must not contradict them.

## Roles and authority

- Client roles are `archive` and `cache`. `archive` and `unarchive` are job
  operations, not daemon roles.
- Titan's control daemon is the sole cross-node orchestrator. Clients never
  command one another and do not advance to the next workflow step without a
  persisted Titan command.
- Archive nodes retain committed resources. Cache eviction is a distinct,
  explicit operation.
- Transfer and eviction jobs are durable, idempotent, recoverable, and
  serialized globally per torrent resource.

## Connection and protocol

- Every WebSocket connection starts with a registration containing client ID,
  role, protocol/capabilities, and the single pre-shared token. The token occurs
  in no later message. Invalid registration closes the connection.
- A new valid registration for a duplicate client ID disconnects the old
  connection.
- Clients reconnect forever with configurable capped exponential backoff.
- Commands are at-least-once, durably deduplicated, and separate acceptance
  acknowledgements from completion events.
- Protobuf JSON uses a versioned `archive_control.v1` envelope. Per-job event
  sequences support gap detection and snapshot reconciliation.

## State, recovery, and scheduling

- Titan and every client have SQLite state. They reconcile active jobs on every
  reconnect and fail closed when lost state cannot be proven from journals,
  manifests, and observed external state.
- SQLite backups use the online backup API. Defaults are every six hours, 12
  recent, 14 daily, and eight weekly copies, plus pre/post-migration backups.
- Default concurrency is unrestricted: every eligible queued job is admitted
  in one scheduler pass and independent jobs execute concurrently on a client.
  Physical qBittorrent, Syncthing, disk, and network capacity are therefore
  the natural limit. `enforce_concurrency_limits=true` restores the optional
  legacy per-client/per-route gates. Commands for one job remain serialized.
- A queued job owns a per-resource reservation. Cancelling it removes only the
  queued record/reservation and never sends cleanup commands.
- Connectivity or transfer stalls wait indefinitely. A configurable 30-minute
  no-progress threshold marks the job `STALLED` without losing its percentage.

## Resources and selections

- Resource identity supports BitTorrent v1, v2, and hybrid info hashes. A
  qBittorrent-local hash is a separate adapter identifier. Hybrid hash aliases
  share reservations and are joined only when no supplied hash conflicts.
- Only selected versus skipped matters; qBittorrent priority levels are not
  preserved.
- Placements are keyed by resource and node and contain a verified set of
  selected torrent file indices. Multiple cache and archive placements are
  allowed.
- Transfer jobs have one source and one target. A requested selection is
  expanded to immutable file indices. Targets merge the missing delta into
  their existing selection transactionally.
- Initial Telegram selection supports either all source-selected complete
  entries or one file/directory. The protocol supports multiple entries from
  v1.
- Full source and target file inventories are fetched immediately before
  staging, even for the normal `Confirm` path. The previewed baseline and delta
  are persisted immutably and must still match at execution.

## Transfer safety

- Each transfer uses `<sync-folder>/.archive-control/jobs/<job-id>/` with a
  payload, exported torrent, protobuf-JSON manifest, auxiliary artifacts, and a
  ready marker.
- File materialization tries hard link, then reflink, then safe copy. It never
  follows symlinks or special files and never escapes configured roots.
- Pre-existing target paths are never overwritten. Same-size files may be
  reused only after qBittorrent's stopped recheck proves them valid.
- New target torrents are explicitly placed under the configured qB root,
  added stopped with manual torrent management, assigned selected/skipped
  states, fully rechecked, and started only after successful verification.
- Existing target merges preserve a durable baseline. Failure restores the
  baseline selection/state and removes only attributable delta data.
- qBittorrent/libtorrent partfiles are transferred or merged only through a
  tested version-aware adapter. An unsafe format fails job preflight with a
  clear reason.
- Sparse support is probed once at client startup for configured roots. Jobs
  involving sparse files require semantic sparse support on every leg; exact
  physical extent layouts are not required.
- Permission handling is fail-fast. The daemon never chmods/chowns source data.

## Commit, cleanup, and deletion

- A transfer commits after the target's selected union passes a stopped full
  recheck without downloading. After commit, the new placement is retained.
- Staging cleanup is post-commit. Its failure is `CLEANUP_REQUIRED`, not a
  failed transfer, and cannot cause committed archive data to be deleted.
- Archive jobs no longer delete their source cache placement.
- Eviction is a durable job. It requires fresh, complete archive coverage for
  every selected cache file; coverage may be the union of multiple online
  archive nodes. There is no force override initially.
- Eviction removes the qBittorrent entry without data deletion, checks exact
  path overlap with remaining torrents, unlinks eligible known files, and uses
  non-recursive bottom-up directory removal. It never blindly removes a tree.

## Syncthing routes

- Client config supplies Syncthing endpoint/API key and separate API-visible
  and daemon-local sync roots. Compatible `sendreceive` folders beneath the
  root are discovered automatically.
- Syncthing's native folder ID is the route ID. Routes are dedicated to exactly
  two devices: one archive and one cache client.
- Missing routes can be created `on_demand` (default) or by `eager_mesh`.
  Clients automatically pair missing Syncthing devices and create/verify
  folders. Optional advertised addresses default to `dynamic`.
- Route setup timeout defaults to 30 minutes. Archive Control never removes a
  verified route merely because a node is absent.

## User interface and retention

- Main actions are Archive, Unarchive, Evict Cache, and Job Status.
- Telegram state is shared across authorized admins. Jobs/events store no
  Telegram user owner.
- Existing private-chat `@username` authorization remains compatible; optional
  numeric user-ID pins harden configured entries when available.
- Archive Control's Telegram view/filter/confirmation sessions persist in
  SQLite. Destructive confirmations use single-use nonces.
- Telegram progress edits are coalesced to at most once every five seconds by
  default, with immediate important transitions.
- Clear/Remove hides completed records without touching placements. Minimal
  idempotency tombstones remain for 30 days by default.

## Delivery and testing

- `archive-control-proto` is drafted and reviewed before application code.
- Host tooling stays light. Buf, protoc, tests, and builds run in Docker and
  preserve host UID/GID `1001:1001`.
- The client image is `sodium/archive-clients`, published for linux/amd64 and
  linux/arm64 with immutable release tags.
- Cross-project E2E lives in this coordination repository and simulates two
  cache nodes, two archive nodes, and a thin control node. It never mutates the
  live x1/x2/lithium services.