docs: publish archive control design set
This commit is contained in:
+245
@@ -0,0 +1,245 @@
|
||||
# Testing strategy
|
||||
|
||||
Tests run locally in containers and never inspect or mutate live x1, x2,
|
||||
lithium, or Titan services. Host tooling follows `playground/ENV.md`: Docker and
|
||||
Compose provide runtimes/build tools and write generated files as UID/GID
|
||||
`1001:1001`.
|
||||
|
||||
## Current happy-path coverage
|
||||
|
||||
The initial happy-path pass is tracked by executable capability rather than by
|
||||
declaring an unfinished phase complete:
|
||||
|
||||
As of 2026-07-23, the full control/bot suite passes 102 tests, the full client
|
||||
suite passes 60 tests, and a clean five-project Compose run passes automatic
|
||||
2×2 route creation plus archive → covered eviction → unarchive with SHA-256
|
||||
verification at both materialized targets.
|
||||
|
||||
| Capability | Unit/contract | Component/E2E |
|
||||
| --- | --- | --- |
|
||||
| Registration, heartbeat, replay, snapshots | Covered | Control/client integration covered |
|
||||
| On-demand inventory and content trees | Covered | WebSocket integration covered |
|
||||
| Automatic pairwise route provisioning | Covered | Local 2×2 topology covered |
|
||||
| Manifest/ready-marker publication | Covered | Receiver filesystem round trip covered |
|
||||
| Hardlink, reflink, sparse-copy fallback | Covered | Local filesystem round trip covered |
|
||||
| Existing same-size target reuse | Covered | Local filesystem round trip covered |
|
||||
| qB stopped add, selection, guarded recheck, start/resume, entry-only delete | Covered for 4.x/5.x semantics | Current pinned service covered |
|
||||
| Syncthing scoped rescan and completion proof | Covered | Real route transfer covered |
|
||||
| Transfer command state machine and qB verification | Covered | Archive and unarchive covered |
|
||||
| Archive/unarchive placement commit | Covered | Both directions covered |
|
||||
| Cache eviction | Coverage union, inode/path safety, idempotency covered | Real retained-source eviction covered |
|
||||
| Telegram archive-control flows | Persistence, nonce, stale callback, stalled rendering, dummy controls/trace covered | Real handler with isolated dummy backend covered; live Telegram remains release smoke |
|
||||
|
||||
Cancellation, rollback provenance, shared/unknown eviction paths,
|
||||
case-collisions, and stale preview guards now have focused tests. The broader
|
||||
chaos matrix remains a release gate rather than part of the initial happy-path
|
||||
run.
|
||||
|
||||
The executable complex E2E matrix additionally covers complementary partial
|
||||
archive placements, selective cache-placement merging, repeated placement
|
||||
generation after eviction, two-archive coverage union,
|
||||
uncovered/no-op/stale-preview rejection, exact-path sharing and unknown file
|
||||
retention, queued record-only cancellation, precommit rollback after target
|
||||
materialization, control restart after source staging, durable command replay
|
||||
after a client disconnect, and a no-downloaded-bytes assertion at the target.
|
||||
It also mutates the source selection and creates an absent target after job
|
||||
confirmation; both hostile races must fail before commit with the precise
|
||||
precondition reason while retaining the source. A wrong-size pre-existing
|
||||
target and an unwritable target root likewise fail without commit; the latter
|
||||
is wrapped in fixture cleanup that restores the original directory mode.
|
||||
|
||||
The second adversarial matrix also passes from clean state. It covers pure-v2
|
||||
and hybrid archive/evict/unarchive round trips, including qBittorrent's
|
||||
truncated-v2 primary key and hybrid v1/v2 aliases; a disconnected target that
|
||||
becomes `STALLED` at 20% and resumes successfully; bounded disk exhaustion with
|
||||
an exact precommit failure and intact source; and duplicate/reordered protocol
|
||||
delivery from a separate WebSocket probe process.
|
||||
|
||||
The remaining Compose expansion includes supported
|
||||
partfiles, sparse-capability mismatch, case/type collisions, broader
|
||||
service-level network partitions, and database backup/restore during active
|
||||
recovery. Focused unit/component tests already cover several of these
|
||||
invariants, but they remain unchecked in the full five-project topology.
|
||||
|
||||
## Test layers
|
||||
|
||||
### Proto contract
|
||||
|
||||
- Buf format, lint, descriptor build, and breaking checks.
|
||||
- Decode every canonical protobuf-JSON fixture through the descriptor.
|
||||
- Golden round trips for envelopes and on-disk manifests.
|
||||
- Reject duplicate JSON keys, unknown major versions, invalid hashes/ranges,
|
||||
malformed UUIDs/timestamps, over-limit frames, token outside registration,
|
||||
and unknown required commands.
|
||||
- Consumer tests assert the generated binding header records the exact proto
|
||||
release/commit.
|
||||
|
||||
### Unit and property tests
|
||||
|
||||
Control tests cover state transitions, hybrid-identity alias reservations,
|
||||
FIFO scheduling,
|
||||
placement unions, coverage-set calculation, command retries, event gap
|
||||
reconciliation, UI nonce idempotency, retention, and backup selection.
|
||||
|
||||
Client tests cover path normalization/confinement, file-index range algebra,
|
||||
tree expansion, root mapping, capability probes, version adapters, manifest
|
||||
digesting, journal replay, progress aggregation, and safe directory removal.
|
||||
Property/fuzz inputs emphasize path traversal, Unicode/case collisions,
|
||||
overlapping selections, reordered/duplicated events, and partial journals.
|
||||
|
||||
### Adapter contract tests
|
||||
|
||||
Pinned qBittorrent and Syncthing container versions cover every supported API
|
||||
family. Assertions include stopped add, exact selection, export, full recheck,
|
||||
download-attempt detection, entry-only delete, folder/device idempotency,
|
||||
events fallback, need/completion proof, and redacted errors. Partfile adapters
|
||||
are tested only against explicitly supported qBittorrent/libtorrent fixtures.
|
||||
|
||||
### Component integration tests
|
||||
|
||||
- Control plus fake clients: reconnect, duplicate ID replacement, lost ack,
|
||||
replay, sequence gaps, old connection fencing, queue leases, and restart.
|
||||
- One client plus qB/Syncthing: inventory, staging, sparse/copy fallbacks,
|
||||
service restart, permission failure, and database recovery.
|
||||
- Two clients plus two Syncthing services: automatic device/folder creation,
|
||||
bidirectional nonce verification, timeout, and existing-route discovery.
|
||||
|
||||
## Cross-project E2E topology
|
||||
|
||||
The coordination repository owns uniquely named Compose projects:
|
||||
|
||||
```text
|
||||
e2e/
|
||||
├── control/ # mogic archive-control core + test HTTP adapter
|
||||
├── cache-1/ # qBittorrent + Syncthing + cache client
|
||||
├── cache-2/ # qBittorrent + Syncthing + cache client
|
||||
├── archive-1/ # qBittorrent + Syncthing + archive client
|
||||
├── archive-2/ # qBittorrent + Syncthing + archive client
|
||||
├── fixtures/
|
||||
├── scenarios/
|
||||
└── scripts/
|
||||
```
|
||||
|
||||
Each data-node project has isolated config, state, backup, qB data, sync data,
|
||||
and service volumes. All join one dedicated, uniquely labeled test network.
|
||||
Syncthing advertises static Compose service addresses in E2E; production
|
||||
defaults remain `dynamic`. The topology deliberately starts without explicit
|
||||
route folders so tests exercise automatic 2×2 pairing.
|
||||
|
||||
Teardown resolves exact project names/labels and stops only those resources.
|
||||
Volume deletion is a separate explicit test action, never an unscoped prune.
|
||||
|
||||
## Thin test HTTP adapter
|
||||
|
||||
The adapter is enabled only in the control E2E configuration and delegates to
|
||||
`ArchiveControlService`; it contains no orchestration logic. Initial endpoints:
|
||||
|
||||
| Method/path | Purpose |
|
||||
| --- | --- |
|
||||
| `GET /test/v1/clients` | connection/health/capability view |
|
||||
| `GET /test/v1/routes` | route discovery/provisioning state |
|
||||
| `GET /test/v1/resources?operation=...` | on-demand eligible source listing |
|
||||
| `GET /test/v1/resources/{id}/tree?...` | scoped complete tree |
|
||||
| `POST /test/v1/jobs/preview` | validate and calculate exact delta/coverage |
|
||||
| `POST /test/v1/jobs` | create using preview revision and idempotency key |
|
||||
| `GET /test/v1/jobs/{id}` | projected state and progress |
|
||||
| `POST /test/v1/jobs/{id}/cancel` | queued removal or active compensation |
|
||||
| `POST /test/v1/jobs/{id}/hide` | UI/history-only removal |
|
||||
| `POST /test/v1/scheduler/pause` | stop automatic advancement at a durable boundary |
|
||||
| `POST /test/v1/scheduler/advance` | dispatch exactly the next eligible command |
|
||||
| `POST /test/v1/scheduler/resume` | restore automatic advancement |
|
||||
| `POST /test/v1/protocol/order-probe` | create and lease one synthetic ordering-test job |
|
||||
| `POST /test/v1/maintenance/backup` | request and observe a test backup |
|
||||
|
||||
JSON objects are adapter DTOs, not a second daemon protocol. Test scripts use
|
||||
curl, poll bounded state conditions, and dump control/client/service logs on
|
||||
failure. Scheduler gating is available only through the loopback, opt-in test
|
||||
adapter and is absent from normal daemon and Telegram workflows.
|
||||
|
||||
## Interactive Telegram UX harness
|
||||
|
||||
The integrated bot has an opt-in `Archive Control (Test)` entry backed by a
|
||||
scripted in-process service. It runs the production conversation handler and
|
||||
planner, but never starts a control listener and never contacts a client
|
||||
daemon, qBittorrent, or Syncthing. Its UI sessions, scenario state, synthetic
|
||||
jobs, and append-only trace share one dedicated SQLite test database that
|
||||
configuration validation forbids from being the production control database.
|
||||
|
||||
The `comprehensive` scenario provides enough resources, targets, and retained
|
||||
jobs to exercise resource/job pagination, filtering, Unicode and long names,
|
||||
full and partial placements, selective entry previews, archive, unarchive,
|
||||
covered eviction, job removal, every significant displayed job state, and
|
||||
single-use confirmations. `empty` covers empty lists. `route_slow` keeps a
|
||||
newly requested route pending until an administrator explicitly marks it ready.
|
||||
|
||||
The existing Telegram manager allowlist and optional numeric-ID pin apply.
|
||||
Private-chat controls are:
|
||||
|
||||
```text
|
||||
/actest status
|
||||
/actest reset [comprehensive|empty|route_slow]
|
||||
/actest advance <job-id> <state>
|
||||
/actest fail-next <backend-method> [message]
|
||||
/actest route-ready
|
||||
/actest trace [limit]
|
||||
/actest export
|
||||
```
|
||||
|
||||
`advance` accepts queued, preparing, running, waiting, stalled, cancelling,
|
||||
cleanup_required, succeeded, failed, and cancelled. `fail-next` injects one
|
||||
deterministic planner-style backend failure. Reset clears only the test
|
||||
sessions, jobs, scenario state, and trace. State otherwise survives bot
|
||||
restarts.
|
||||
|
||||
The SQLite trace records callback receipt and validated callback state before
|
||||
and after handling, text filters, complete rendered text and inline keyboards,
|
||||
backend calls/results/errors, test-control commands, and job transitions.
|
||||
`trace` shows a bounded recent view in chat; `export` sends the complete trace
|
||||
as JSON Lines for offline diagnosis.
|
||||
|
||||
## Fixture matrix
|
||||
|
||||
Fixtures include small deterministic v1, v2, and hybrid torrents with:
|
||||
|
||||
- full, complementary, overlapping, and one-directory selections;
|
||||
- same-size corrupt pre-existing targets and type/size collisions;
|
||||
- nested shared directories and exact file paths shared by multiple torrents;
|
||||
- unknown files/directories inside a resource-owned directory;
|
||||
- supported, unsupported, and ambiguous partfiles;
|
||||
- sparse files plus a simulated non-sparse target capability;
|
||||
- canonical and renamed/noncanonical qBittorrent paths;
|
||||
- hardlink, reflink, and copy-only filesystem conditions;
|
||||
- permissions and space-reserve failures.
|
||||
|
||||
## Acceptance scenarios
|
||||
|
||||
| Area | Required scenarios |
|
||||
| --- | --- |
|
||||
| Archive | new target, selective target, source retained, second archive replica |
|
||||
| Merge | complementary sources build union, overlap transfers delta only, no-op rejected, selected-incomplete target rejected |
|
||||
| Unarchive | new cache, merge existing cache, archive always retained, repeat after prior eviction |
|
||||
| Eviction | one-archive coverage, union coverage, missing coverage disabled, shared/unknown content retained |
|
||||
| Routes | on-demand 2×2 creation, eager mesh, existing match, slow success, timeout, restart during provisioning |
|
||||
| Recovery | restart every step on source/target/control, lost DB with proof, ambiguous loss fails closed |
|
||||
| Messaging | duplicate command/event, lost ack, sequence gap, stale revision, duplicate client ID |
|
||||
| Safety | attempted download, corrupt same-size file, symlink/path escape, special file, partfile mismatch |
|
||||
| Operations | stall retains percent, offline wait, cancellation each phase, postcommit cleanup retry |
|
||||
| Database | online backup under load, pre/post migration, retention, corrupt backup rejection, offline restore |
|
||||
| UI | pagination/filter chain, stale/double callback, persisted session, clear vs evict separation |
|
||||
|
||||
Every scenario asserts final qB selections/states, placement generations,
|
||||
manifests/journals, filesystem ownership/provenance, job event order, absence of
|
||||
unexpected received bytes, and preservation of unrelated files.
|
||||
|
||||
## Fault injection and pass criteria
|
||||
|
||||
Scenario hooks pause or restart each container, disconnect network edges,
|
||||
duplicate/reorder protocol frames, fill a bounded test filesystem, change an
|
||||
external qB selection, and kill processes between intent and completion journal
|
||||
writes. Tests use eventual assertions with explicit deadlines; long production
|
||||
waits are shortened by test-only configuration without altering state logic.
|
||||
|
||||
A release candidate passes only when the whole matrix succeeds repeatedly from
|
||||
clean volumes and from retained/restarted volumes, existing mogic-bot regression
|
||||
tests remain green, Buf reports no unintended break, and both amd64 and arm64
|
||||
client images start and pass smoke/config checks.
|
||||
Reference in New Issue
Block a user