181 lines
8.8 KiB
Markdown
181 lines
8.8 KiB
Markdown
# Resource and workflow model
|
|
|
|
## Resource identity and file selection
|
|
|
|
A resource is identified by its BitTorrent metainfo identity:
|
|
|
|
- v1: lowercase 40-character SHA-1 info hash;
|
|
- v2: lowercase 64-character SHA-256 info hash;
|
|
- hybrid: both values.
|
|
|
|
The qBittorrent-local torrent hash is retained only as an adapter identifier.
|
|
Identity matching never relies on display name or filesystem path. Control
|
|
maintains aliases for both hashes of a hybrid torrent: one matching hash joins
|
|
records only when any other supplied hash does not conflict, and the normalized
|
|
identity is then enriched with both values. Reservations cover every known
|
|
alias so v1-only and hybrid observations cannot schedule concurrently.
|
|
|
|
A selection is a normalized set of torrent file indices. Ranges are sorted,
|
|
coalesced, and immutable after job creation. Only selected/skipped state is
|
|
portable; qBittorrent priority levels are deliberately ignored. A directory
|
|
choice expands to all currently available selected-complete descendant file
|
|
indices before confirmation.
|
|
|
|
For every transfer, control fetches complete source and target content trees
|
|
immediately before staging. It validates the source fingerprint, expands the
|
|
requested selection, reads the target's verified baseline, and computes:
|
|
|
|
```text
|
|
transfer delta = requested source files - verified target files
|
|
target union = verified target files + transfer delta
|
|
```
|
|
|
|
An empty delta is a no-op and is rejected rather than recorded as a successful
|
|
transfer. Separate one-source/one-target jobs can merge complementary
|
|
selections into the same placement over time.
|
|
|
|
## Job creation
|
|
|
|
All creation paths follow the same service-layer transaction:
|
|
|
|
1. Resolve a fresh source summary and suitable online targets.
|
|
2. Optionally resolve one file or directory; otherwise use all
|
|
selected-complete source files.
|
|
3. Fetch full source and target trees and compute the exact delta.
|
|
4. Validate role, route feasibility, canonical paths, generation, completion,
|
|
disk capacity, and advertised capabilities. Partfile compatibility is not an
|
|
eligibility filter at this stage.
|
|
5. Show a final confirmation containing operation, resource, source, target,
|
|
requested size/count, and whether this extends an existing placement.
|
|
6. Consume a single-use confirmation nonce and atomically insert the immutable
|
|
job, resource reservation, and initial event.
|
|
|
|
The previewed requested selection, verified target baseline, exact transfer
|
|
delta, and their logical byte counts become immutable fields in the job
|
|
definition. Execution recomputes and compares them rather than silently
|
|
adapting to external changes.
|
|
|
|
Partfile handling is deliberately checked again immediately before the first
|
|
data step. A resource is not hidden as ineligible merely because its partfile
|
|
format may be unsafe; the accepted job fails with an explicit reason when the
|
|
live preflight proves it cannot be handled.
|
|
|
|
## Route readiness
|
|
|
|
Each data route is one `sendreceive` Syncthing folder shared by exactly two
|
|
devices: one cache client and one archive client. The Syncthing folder ID is the
|
|
route ID.
|
|
|
|
At registration, clients report compatible folders below their configured sync
|
|
root. Control pairs matching route IDs. It ignores and never modifies
|
|
multi-device or wrong-type folders.
|
|
|
|
If a needed pair has no route, the default `on_demand` policy performs:
|
|
|
|
1. Generate a route/folder ID and deterministic relative folder path.
|
|
2. Send each client the peer device ID and advertised addresses.
|
|
3. Have each client idempotently add the peer device and its side of the same
|
|
folder.
|
|
4. Wait for both Syncthing instances to report a healthy shared folder.
|
|
5. Exchange unique nonce files in both directions and require both observations.
|
|
6. Mark the route ready and refresh registrations/routes.
|
|
|
|
The default timeout is 30 minutes and is configurable. A timeout leaves a
|
|
diagnosable provisioning state. Rollback may remove only configuration created
|
|
by that attempt and only while ownership is unambiguous. A previously verified
|
|
route is never automatically deleted. `eager_mesh` runs the same process for
|
|
all missing online cache/archive pairs.
|
|
|
|
## Archive and unarchive transfers
|
|
|
|
Archive and unarchive share one transfer state machine; only source/target roles
|
|
and placement policy differ.
|
|
|
|
### Preflight
|
|
|
|
- Revalidate both resource fingerprints and placement generations.
|
|
- Require source-selected files to be complete and regular, canonical entries.
|
|
- Reject noncanonical qBittorrent file/folder renames in v1.
|
|
- Confirm target path types/sizes do not conflict.
|
|
- For an existing target, require every currently selected baseline file to be
|
|
complete and verified. Unexpected selected-incomplete files fail as stale
|
|
external state rather than being downloaded or silently deselected.
|
|
- Confirm free space/reserve, permissions, sparse capability, supported
|
|
partfile format, route health, and negotiated protocol features.
|
|
- Export the source `.torrent` metainfo.
|
|
- Persist the exact source, target baseline, requested selection, and delta.
|
|
|
|
### Five visible steps
|
|
|
|
1. **Source Stage/HardLink.** Create the isolated job directory and materialize
|
|
only delta files plus required metainfo/auxiliary data. Try hard link,
|
|
reflink, then copy, journaling every result. Write the manifest atomically
|
|
and the ready marker last.
|
|
2. **Syncthing.** Wait for the destination to receive the complete job prefix.
|
|
Report job-scoped percentage and approximate speed, but do not infer success
|
|
from a progress percentage alone.
|
|
3. **Target Merge/Copy.** Materialize the delta beneath the configured qB root.
|
|
Reuse same-size pre-existing regular files without overwriting them; record
|
|
provenance so verification or rollback can distinguish them.
|
|
4. **QB Verification.** Add a missing torrent stopped with manual management
|
|
and explicit save path, or apply a transactional delta to an existing
|
|
torrent. Set selected/skipped state, run a full recheck of the target union,
|
|
and fail immediately if qBittorrent attempts to download. Successful recheck
|
|
atomically advances the placement generation and is the commit point.
|
|
5. **Staging Cleanup.** Remove only job-owned staging artifacts from both
|
|
endpoints. A post-commit cleanup failure produces `CLEANUP_REQUIRED`; it
|
|
never rolls back or deletes the committed placement.
|
|
|
|
Archive keeps the cache source. Unarchive always keeps the archive source. A
|
|
new target is started only after successful verification so it can seed; an
|
|
existing target returns to its captured stopped/running state. Archive Control
|
|
does not copy tags, categories, rate limits, queue position, or other
|
|
source-local metadata.
|
|
|
|
### Existing target merge
|
|
|
|
Before changing an existing target, the client durably records its selection,
|
|
run state, qB identifier, content revision, file provenance, and partfile
|
|
baseline. Failure before commit restores that baseline and removes only files
|
|
created by the current job that remain unreferenced. Pre-existing paths are
|
|
never removed. Success stores the verified union, allowing later jobs from
|
|
other sources to add complementary file indices.
|
|
|
|
## Evict cache
|
|
|
|
Eviction is independent of transfer history and operates on one whole cache
|
|
placement in v1.
|
|
|
|
1. Query the cache placement and all relevant online archive placements fresh.
|
|
2. Build per-file archive coverage; the union may span multiple archive nodes.
|
|
3. Disable confirmation and explain missing files if coverage is incomplete.
|
|
4. On confirmation, reserve the resource and persist the exact cache
|
|
fingerprint/generation plus archive proofs.
|
|
5. Revalidate those proofs immediately before deletion.
|
|
6. Ask qBittorrent to remove the cache entry with data deletion disabled.
|
|
7. Snapshot exact paths referenced by all remaining local torrents.
|
|
8. Unlink only manifest-owned files not exactly referenced by another torrent.
|
|
9. Remove a partfile only when ownership by the evicted torrent is
|
|
unambiguous.
|
|
10. Attempt non-recursive `rmdir` from deepest known owned directory upward;
|
|
retain directories containing unknown files, subdirectories, or other
|
|
application data.
|
|
|
|
There is no force override in v1. Retrying any step observes prior effects and
|
|
continues safely. Eviction never changes archive placements.
|
|
|
|
## Cancellation, removal, and display retention
|
|
|
|
- Cancelling a queued job deletes the queue record and releases its reservation
|
|
only; no client command or file operation occurs.
|
|
- Cancelling an active pre-commit job requests cooperative stop and runs
|
|
attributable compensation. The source placement remains intact.
|
|
- After commit, cancellation can only drive staging cleanup. It cannot undo the
|
|
committed target.
|
|
- A failed or cancelling job stays visible until its cleanup/compensation state
|
|
is known.
|
|
- Clear Finished and removal of a successful record affect only UI visibility.
|
|
They never touch qBittorrent, files, routes, or placements.
|
|
- Hidden jobs retain a minimal idempotency tombstone for 30 days by default;
|
|
detailed events may be purged sooner under configured retention.
|