160 lines
7.4 KiB
Markdown
160 lines
7.4 KiB
Markdown
# Storage safety, rollback, and recovery
|
|
|
|
## Path trust boundary
|
|
|
|
qBittorrent metadata, Syncthing configuration, transfer manifests, and
|
|
filesystem directory entries are all untrusted inputs. Before any access, the
|
|
client:
|
|
|
|
1. validates the canonical torrent-relative path;
|
|
2. rejects absolute paths, empty/`.`/`..` components, NULs, platform separators,
|
|
duplicate normalized names, and configured case-fold collisions;
|
|
3. resolves the API-visible path through one validated root mapping;
|
|
4. traverses without following symlinks and checks the resulting descriptor is
|
|
a regular file or an explicitly created directory;
|
|
5. verifies every operation remains beneath the local configured root.
|
|
|
|
For qBittorrent content, the root is resolved per resource: qB's authoritative
|
|
API-visible `save_path` is mapped beneath `qbittorrent.api_root` (or a more
|
|
specific configured local override) into the client namespace. This permits
|
|
existing nested save paths without moving data, but does not permit paths
|
|
outside the configured boundary. The resolved path remains local client state
|
|
and is never sent to the control daemon.
|
|
|
|
Sockets, devices, FIFOs, symlinks, and other special entries fail preflight.
|
|
Permission or ownership mismatch is fail-fast. Archive Control never changes
|
|
source ownership or mode to make a job pass.
|
|
|
|
## Isolated staging
|
|
|
|
Every route stores transient jobs under:
|
|
|
|
```text
|
|
<sync-root>/.archive-control/jobs/<job-id>/
|
|
├── payload/<original torrent hierarchy>
|
|
├── metainfo/source.torrent
|
|
├── artifacts/...
|
|
├── manifest.json
|
|
└── ready.json
|
|
```
|
|
|
|
Temporary writes use an attempt-specific name, fsync where durability matters,
|
|
and atomic rename. The manifest records every expected file and method. The
|
|
ready marker is the final publish action. Job IDs and known subpaths are used
|
|
directly; cleanup never expands a glob or recursively targets the sync root.
|
|
|
|
## Materialization fallback
|
|
|
|
For each file, source staging and target materialization try:
|
|
|
|
1. hard link when the same filesystem and permissions allow it;
|
|
2. reflink/clone when supported;
|
|
3. bounded buffered copy into a temporary file followed by atomic placement.
|
|
|
|
The journal records intent before mutation and completion afterward, including
|
|
source identity, destination, method, logical/allocated sizes, and whether the
|
|
target pre-existed. Restarts reconcile the journal against file descriptors and
|
|
the manifest instead of blindly repeating operations.
|
|
|
|
Free-space checks account for methods that allocate data plus a configurable
|
|
reserve. A mid-copy space failure leaves only an attributable temporary file.
|
|
No operation overwrites an existing target. A same-type, same-size regular file
|
|
may be provisionally reused, but qBittorrent's full recheck is the integrity
|
|
authority; any other collision fails before mutation.
|
|
|
|
## Sparse files
|
|
|
|
At startup, each client probes semantic sparse-file behavior once for every
|
|
configured qB and Syncthing local root. Capability is advertised with the
|
|
connection. A job with any sparse source file requires sparse support for every
|
|
staging and destination root involved. If a target leg lacks it, the job fails
|
|
clearly before transfer.
|
|
|
|
Sparse files are retained as sparse, without attempting to reproduce an exact
|
|
physical hole map. Hardlink/reflink naturally preserve semantics; the copy
|
|
implementation uses a sparse-aware strategy and verifies logical length and
|
|
allocation behavior. Exact allocated-block counts need not match between
|
|
filesystems.
|
|
|
|
## qBittorrent transaction boundary
|
|
|
|
For a new target, the client exports source metainfo, places candidate data,
|
|
adds the torrent stopped with manual management and explicit save path, applies
|
|
the exact union selection, and starts a full recheck. It monitors both state and
|
|
transfer counters. Any download transition or received payload byte is a hard
|
|
failure and the torrent is stopped immediately.
|
|
|
|
For an existing target, the durable baseline includes its selected indices,
|
|
run state, local identifier, save-path fingerprint, content revision, and
|
|
partfile state. The job adds only the delta. Before commit, rollback restores
|
|
that baseline and removes only current-job-created, unshared files. Source
|
|
torrent state is never changed by transfer logic.
|
|
|
|
The target placement commits only after the selected union verifies completely
|
|
while stopped. The placement generation and commit event are persisted in the
|
|
same control transaction. Once committed, target content is outside rollback
|
|
scope.
|
|
|
|
## Partfiles
|
|
|
|
Libtorrent partfiles may contain data for skipped or non-materialized torrent
|
|
files and their layout varies by version. Archive Control therefore uses a
|
|
version-aware adapter keyed by observed qBittorrent/libtorrent version and a
|
|
tested format signature. The adapter must support export/import and, for an
|
|
existing target, merge with a recoverable baseline.
|
|
|
|
Unknown, ambiguous, or incompatible partfile state fails the job at live
|
|
preflight with `UNSUPPORTED` or `PRECONDITION_FAILED` and a precise reason. It
|
|
is never copied over an existing partfile blindly. Rollback restores the
|
|
baseline. Eviction removes a partfile only when its ownership by the removed
|
|
torrent is unambiguous.
|
|
|
|
## Safe eviction deletion
|
|
|
|
Before unlinking, the client persists:
|
|
|
|
- the confirmed cache placement generation and file set;
|
|
- fresh archive coverage proofs;
|
|
- the exact torrent manifest and local paths;
|
|
- qBittorrent entry-removal result;
|
|
- exact paths referenced by all remaining torrents;
|
|
- per-file unlink and per-directory `rmdir` results.
|
|
|
|
The qBittorrent entry is removed with `deleteFiles=false`. For each owned file,
|
|
the client refuses removal if another remaining torrent references the exact
|
|
path. Eligible known files are unlinked individually. It then attempts
|
|
non-recursive `rmdir` deepest-first for known owned directories. `ENOTEMPTY`
|
|
means retain and continue; unknown files or directories are never traversed or
|
|
deleted. There is no recursive delete fallback.
|
|
|
|
## Compensation by phase
|
|
|
|
| Point | Safe outcome |
|
|
| --- | --- |
|
|
| Queued, no command | Delete queue row/reservation only |
|
|
| Source staging | Stop; unlink only journaled staging artifacts |
|
|
| Syncthing transfer | Stop publishing; clean only isolated job namespace |
|
|
| Target materialization | Restore target baseline; remove attributable delta |
|
|
| qB recheck before commit | Stop/remove newly added entry without data deletion, then compensate delta |
|
|
| After commit | Keep placement; retry staging cleanup only |
|
|
| Eviction after qB removal | Resume safe per-file unlink; never re-add or delete archives automatically |
|
|
|
|
If attribution is uncertain, retain data and require manual intervention.
|
|
|
|
## SQLite durability and backup
|
|
|
|
Both daemons enable foreign keys, WAL, a bounded busy timeout, and explicit
|
|
transactions. Schema migrations create an online backup before and after the
|
|
migration and fail atomically.
|
|
|
|
The default recurring policy runs SQLite's online backup API every six hours,
|
|
verifies checksum and SQLite integrity, writes via temporary file plus atomic
|
|
rename, and retains 12 recent, 14 daily, and eight weekly copies. Backups live
|
|
outside qBittorrent and Syncthing roots. Configuration warns if the backup and
|
|
database share a filesystem, but may permit it for small installations.
|
|
|
|
Offline tooling provides list, verify, and restore. Restore requires stopped
|
|
daemon access, verifies the chosen backup, preserves the suspect database under
|
|
a timestamped name, installs the replacement atomically, and runs integrity and
|
|
schema checks before normal startup. Backups contain no configured secrets.
|