docs: expand practical test coverage matrix

This commit is contained in:
2026-08-29 07:50:42 +00:00
parent c03a3edef2
commit a110e1df57
+431 -7
View File
@@ -189,7 +189,7 @@ developers call rather than duplicating orchestration in workflow YAML:
scripts/test-unit [--package PATTERN] [--run REGEXP] [--race] scripts/test-unit [--package PATTERN] [--run REGEXP] [--race]
scripts/test-integration [--suite NAME|all] [--run-id ID] [--resume] scripts/test-integration [--suite NAME|all] [--run-id ID] [--resume]
scripts/test-e2e [--scenario NAME|all] [--run-id ID] [--resume] scripts/test-e2e [--scenario NAME|all] [--run-id ID] [--resume]
scripts/test-env doctor|status|logs|collect|recover|reuse|stop|reset|purge|gc ... scripts/test-env doctor|coverage|status|logs|collect|recover|reuse|stop|reset|purge|gc ...
scripts/windows/test-host.ps1 Prepare|Status|Run|Collect|Stop|Reset scripts/windows/test-host.ps1 Prepare|Status|Run|Collect|Stop|Reset
``` ```
@@ -286,6 +286,8 @@ and preserves only the manifest plus state required for `status`, `logs`, or
- `doctor [--e2e]` is read-only and validates local/container prerequisites and, - `doctor [--e2e]` is read-only and validates local/container prerequisites and,
when requested, the native Windows runner before allocating a run; when requested, the native Windows runner before allocating a run;
- `coverage` is read-only and validates the coverage inventory against listed
unit/integration/E2E cases and their required host/layer metadata;
- `status RUN_ID` is read-only and reports steps, health, disk usage, owned - `status RUN_ID` is read-only and reports steps, health, disk usage, owned
Compose/Windows resources, and whether resume is valid; Compose/Windows resources, and whether resume is valid;
- `logs RUN_ID [COMPONENT]` reads bounded/tail output; `collect RUN_ID` writes a - `logs RUN_ID [COMPONENT]` reads bounded/tail output; `collect RUN_ID` writes a
@@ -344,6 +346,425 @@ stable nonzero exit codes, and a final summary giving the run ID, seed, report
path, resume command when applicable, and exact cleanup command. Document the path, resume command when applicable, and exact cleanup command. Document the
same workflow in `docs/testing.md` during Phase 0. same workflow in `docs/testing.md` during Phase 0.
### 2.5 Traceable practical-scenario coverage catalogue
Create `test/coverage.toml` as the executable coverage inventory. Every concrete
case has a stable ID, class (`happy`, `hostile-recovery`, or `race`), owning
requirement/plan section, layer, suite, supported platforms, privilege/session
fixture, speed class, implementation phase, and case name. The ranges below are
shorthand; the inventory expands them into one row per case. `scripts/test-env
coverage` rejects duplicate IDs, missing case implementations, cases that cannot
be listed by their suite, and normative requirements with no test. Test code
references the ID in its name or metadata, and failures print it.
Use line coverage as a warning signal, not as a substitute for scenarios.
Nevertheless, pure policy/domain/protocol/config packages must maintain at least
90% statement coverage and core store/session/runtime packages at least 80%.
Platform syscall wrappers report coverage separately and are judged primarily by
native integration cases. Any excluded unreachable/generated/error branch needs
a reviewed explanation in the inventory. CI reports coverage changes by package;
a new critical branch may not lower coverage merely because the repository-wide
percentage remains above threshold.
For every numeric/time/size limit, generate `minimum`, `minimum+1`, normal,
`maximum-1`, `maximum`, and `maximum+1` cases where meaningful, plus zero,
negative decoded values where the source type permits them, integer overflow,
and unit-conversion boundaries. For every state machine, exhaustively generate
all state/event pairs and then add the concurrent interleavings listed below.
Pairwise generation covers independent dimensions such as shell, source type,
foreground/background, privilege intent, online/offline state, and output kind;
use a full Cartesian product only where interactions affect an invariant.
#### Happy-path cases
- `HP-CFG-01..12` — Parse every annotated server/client/Windows TOML; apply
compiled defaults and explicit flag precedence; normalize Unicode/space-
containing absolute paths; validate Unix and Windows shell paths; advertise
only current-platform shells; render redacted effective configuration; run
`--check-config` without opening listeners/stores; create the first Windows
template; start service live-but-not-ready for placeholder routing; restart
into ready state after valid configuration; preserve config across upgrade.
- `HP-PROTO-01..10` — Negotiate exact and overlapping protocol ranges; ignore
unknown optional fields; round-trip every envelope and control message;
transfer maximum legal binary output/script chunks; Zstandard round-trip
empty/compressible/incompressible data; accept Unicode command/output bytes;
preserve client-observed and server-receipt timestamps; encode/decode every
structured error and optional field without losing presence; store/forward the
single `elevated` request bit unchanged while accepting Windows attempted/
effective contexts only as client-originated result metadata.
- `HP-CTL-01..18` — List/get clients and commands; run foreground/background
command text and scripts; print UUID before following; resume follow; paginate
commands and byte slices; select stdout/stderr; append text/raw/file stdin;
close stdin; send TERM/KILL successfully to Windows and parse every portable
signal name for later Unix delivery; inspect Windows identity,
expiry, truncation, incomplete output, incidents, and takeover; repair and
acknowledge incidents; exercise matching gRPC and JSON-RPC unary behavior;
accept optional global `--request-id` and ignore it for reads.
- `HP-IDEM-01..10` — Generate UUIDv7 at the correct owner; accept externally
supplied UUIDv7; return the same result for identical run/stdin/close/signal/
incident/takeover retries; map run request ID directly to `issue_uuid`; retain
one mutation result through transport reconnect; suppress a matching command
tombstone replay; preserve UUIDv7 FIFO order for equal-millisecond issuance.
- `HP-SES-01..14` — Register a new client; reconnect the same durable instance;
fence its old connection; accept a different instance when none is live;
authorize and consume an exact live-instance takeover; expire an unused grant;
advertise/reconcile capacity; dispatch into running and queued slots; requeue
a transient rejection; resume event and script delivery; reconcile active and
terminal-unacknowledged records; discard a server-confirmed terminal client
record; keep multiple clients fair.
- `HP-CLIENT-01..14` — Create and retain one durable client-instance ID; start
with an empty spool; durably admit and acknowledge commands; advertise queue/
running capacity; dequeue fairly; assign and replay event sequences; compact
acknowledged events; retain terminal-unacknowledged records; discard them only
after reconciliation/acknowledgement; enforce local command/aggregate quotas;
restart with clean recovery; reconnect with saved backoff/session inputs; stop
all supervised Jobs on orderly service shutdown.
- `HP-CMD-01..18` — Execute `cmd` and PowerShell command text and uploaded
scripts; use foreground/background modes; apply requested/omitted CWD;
materialize identity-scoped default CWD; overlay an empty/small/large legal
environment; handle empty/multiline text, shell metacharacters, PowerShell
Unicode, and space/non-ASCII executable/CWD/wrapper paths; return exit codes 0
and nonzero; record every public lifecycle; run 1 and configured maximum
concurrent commands; queue then start work; execute short and descendant-
producing commands; collect resource snapshots; apply every supported
individual/composed resource profile before release.
- `HP-WINCTX-01..14` — With one active standard user run normal as
`ACTIVE_USER`; with a split-token administrator run normal filtered and
elevated through the linked full token; accept an already-full admin token;
build/verify a restricted medium token when UAC is off; run elevated as
`ACTIVE_SYSTEM` after linked-token unavailability; use `LOCAL_SYSTEM` as final
elevated fallback; with no active session run normal as `LOCAL_SERVICE` and
elevated as `LOCAL_SYSTEM`; persist attempts/effective SIDs/session; omit
session fields for Session 0; load/unload active profile; build the effective
environment; access ACL-permitted drive, localhost, and network resources.
- `HP-LAUNCH-01..16` — Authenticate the per-command launcher pipe; create the
launcher and shell suspended; assign the Job before release; persist/flush
prepared then authorized state; resume exactly once; capture both streams;
write and close stdin; keep descendants in the Job; observe root and complete-
tree exit; drain pipes to EOF; deliver CTRL_BREAK through the verified helper;
escalate TERM after grace; terminate immediately on KILL; enforce Job limits;
remove wrappers/scripts after terminal acknowledgement; preserve effective
identity on later lifecycle events.
- `HP-OUT-01..14` — Preserve binary/invalid-UTF-8 bytes, empty writes, partial
lines, no-newline output, long lines, interleaved stdout/stderr, exact chunk
boundaries, cumulative event acknowledgements, reconnect replay, byte-offset
pagination within an event, live follow after historical output, normal drain,
explicit incomplete marker, and normal client/server compression reuse.
- `HP-STORE-01..20` — Initialize/migrate SQLite; append inline and segmented
payloads; group commit; reopen cleanly; query stable cursors; charge command/
client/server bytes; rotate output window; evict whole oldest terminal UUID;
protect active commands; reserve closeout; reclaim terminal age; create and
cap tombstones; rotate audit by bytes/age; maintain incident history; run safe
repair; acknowledge loss; derive/clear dirty state; checkpoint WAL; backup and
restore a quiesced store.
- `HP-FLOW-01..12` — Send heartbeat while idle; reset activity on any valid
frame/Pong; reset reconnect backoff after stability; keep Ping/Pong/Close and
acknowledgements ahead of saturated data; apply fair writer/persistence
scheduling; cross high then low watermarks; retain assigned events; summarize
unsequenced client loss; use closeout reserve; enforce per-command/session send
windows; reconnect with deterministic full jitter.
- `HP-SVC-01..16` — Idempotently install/start/stop/restart/uninstall the SCM
service; boot before login; preserve data on uninstall; change Automatic/
Manual in SCM; start one tray per session from the Run key; reconnect tray
after Explorer restart; exit tray without stopping service; read status/log;
perform an elevated configuration action; rotate service logs; operate without
tray; run Session 0 on Server Core; attach console for human CLI modes; display
native error/dialog when no parent console exists.
- `HP-OPS-01..12` — Expose liveness during recovery and readiness afterward;
keep healthy scopes usable; emit bounded-cardinality metrics; rotate/redact
logs and audits; report storage and cache usage; perform orderly shutdown;
resume after service/server restart; validate deployment examples; upgrade
with config check/snapshot/migration; restore prior version/config on an
intentionally failed pre-migration check; generate checksums/package metadata.
- `HP-HARNESS-01..14` — Doctor a cold and cached environment; create/list a run;
execute one case/scenario; collect JSON/JUnit/report; stop and resume; recover
an interrupted controller; reuse immutable definitions into fresh state;
reset; purge one run; dry-run and execute age GC; purge all confirmed RVBox
runs; retain caches by default; explicitly clear caches/images; preserve file
ownership and unrelated labelled fixtures.
#### Bad, hostile, and abnormal-recovery cases
- `BH-CFG-01..24` — Reject missing/unknown/duplicate keys, wrong TOML types,
invalid UTF-8, bare/overflow/negative durations, illegal zeros, limit and
watermark inversions, incompatible profiles, relative paths, missing/wrong-
type shell files, changed shell identity before launch, disallowed CWD,
insecure state/config/work ownership or ACL, symlink/reparse-point escape,
reserved device/ADS/NT-object path misuse, disallowed or inaccessible UNC
paths, bad URL/TLS name, listener collision,
malformed Windows quoting, and non-loopback JSON-RPC without explicit enable;
prove no listener, store mutation, or child starts after static failure.
- `BH-PROTO-01..28` — Reject text WebSocket messages, invalid fragmentation,
empty/unknown payload, incompatible major/minor, absent/wrong fencing fields,
oversize envelope/spec/control/HTTP body/chunk/script, protobuf recursion/
length abuse, integer overflow, invalid enum/oneof combinations, malformed
UUID/client ID, invalid declared compressed sizes, truncated/corrupt Zstandard,
compression bombs, impossible event sequence/revision, changed duplicate
payload, out-of-order/overlap/hole script chunks, post-commit chunks, SHA/size
mismatch, invalid stdin sequence, and unsupported signal; close/fence only the
offending peer and keep daemons alive.
- `BH-TLSNET-01..18` — Reject wrong CA/name/expired/not-yet-valid certificate,
plaintext endpoint, wrong nginx path/upgrade, and unreachable proxy; recover
from DNS refusal, TCP reset, TLS failure, proxy restart, half-open connection,
packet loss/duplication/reordering, extreme latency, slow reads/writes,
permanently blocked writer, write deadline, missing Pong, clock jump, and
reconnect storm without goroutine/file-descriptor growth or data loss.
- `BH-CTL-01..24` — Reject malformed/empty/batch JSON-RPC where unsupported,
unknown method, invalid params/base64/timestamp, oversized body, tampered or
filter-mismatched cursor, page-size overflow, unknown client/command/incident,
illegal lifecycle action, stdin after close/terminal, signal before eligible
state, takeover for wrong/stale instance, repair of irreparable evidence,
acknowledgement without/beyond note bound, reused request ID with changed
method/target/content, read-only request ID leakage into domain, and client
disconnect during follow; never cancel work merely because the caller exits.
- `BH-SES-01..24` — Reject stale/unknown generation, stale output after fencing,
live different-instance collision without exact grant, mismatched/expired/
consumed grant, capacity lies/overflow, dispatch to wrong generation,
permanent client rejection, reconnect snapshot missing accepted/running state,
matching tombstone against stale server state, hash/revision/terminal
contradiction, incomplete reconciliation snapshot, client dirty-store claim,
lost/repeated result, offline target, queue capacity exhaustion, and one noisy/
corrupt client; isolate the scope and keep unrelated clients/control responsive.
- `BH-CLIENT-01..28` — Handle missing/corrupt/permission-denied client-instance
identity, duplicate daemon lock, spool schema/checksum/segment corruption,
uncommitted spool tail, missing acknowledged bytes, stale accepted record,
incomplete prepared/authorized process metadata, unknown live process, local
queue full, per-command/aggregate spool full, client disk floor, script temp
orphan, tombstone overflow, event-sequence counter drift, impossible server Ack,
dirty local storage, server rejecting registration/version, repeated connect
refusal, shutdown deadline, service kill, and config/shell change across
restart. Do not claim a complete reconciliation snapshot or accept work until
essential local state is trustworthy or explicitly resolved.
- `BH-IDEM-01..16` — Reject noncanonical/non-v7 IDs, all-zero/wrong-length binary
forms, same UUID with changed immutable execution spec/script bytes, same
request ID across mutation kinds/targets, duplicate sequence with changed
bytes, and replay older than full history but still in tombstones; define and
test best-effort behavior after tombstone horizon expiry; never create a second
process or mutation result on any retained duplicate.
- `BH-SCRIPT-01..18` — Treat filename as display only; reject path separators,
traversal, reserved names, NUL/control abuse, excessive metadata, changed
replay bytes, sparse/overlapping writes, premature commit, digest mismatch,
quota exhaustion, disk-full, permission/share violation, wrapper replacement,
symlink/reparse race, shell executable replacement, and cancellation during
upload; clean only owned temporary files and preserve durable rejection detail.
- `BH-WINCTX-01..26` — Handle no linked token, standard user elevation request,
Administrator Protection/approval-only policy, disabled/removed account,
locked/disconnected/logging-off session, `WTSQueryUserToken` failure, multiple
ambiguous active sessions, console-session change, session-ID reuse with a new
logon SID, token SID/session/integrity/type mismatch, restricted-token creation
failure, profile load/unload failure, environment block failure, LocalService
logon failure, `TokenSessionId` failure, active-SYSTEM failure, final LocalSystem
failure, explicit CWD denial, identity-work-root tampering, absent mapped drive/
HKCU/network credentials, and Session 0 interactive API failure. Follow only
the allowed pre-launch chain and emit a bounded decision record with no
effective context if exhausted.
- `BH-LAUNCH-01..34` — Reject spoofed/remote/early launcher or signal-helper pipe
clients, wrong PID/creation time/SID/session/generation/checksum/length,
inherited unrelated handles, launcher outside Job, breakaway attempt, shell/
wrapper/CWD/profile validation failure, invalid or case-colliding Windows
environment keys, oversized environment block, unencodable wrapper content,
process creation/assignment/resume failure, helper attach/delivery refusal,
PID reuse, root disappearance, child
holding pipes forever, descendant escape attempt, output read/write error,
stdin broken pipe, Job accounting failure, unsupported Job limit, service death
before/after preparation/authorization, launcher death at every handshake, and
incomplete EOF. Before authorization reject/cancel safely; after durable
authorization terminate/interrupt and never redispatch.
- `BH-OUTFLOW-01..24` — Survive output flood from one/many commands, tiny writes,
incompressible data, invalid bytes, compressor error, client raw backlog,
offline spool full, pinned send window full, server ingress full, command/
client/global quota full, physical disk floor, loss-marker reservation
pressure, missing EOF, corrupt spool record, repeated Ack, Ack beyond sent
sequence, server retention removing resume point, and slow follower. Preserve
lifecycle/control progress and expose every loss/incomplete range honestly.
- `BH-STORE-01..36` — Recover uncommitted tails and marked eviction; detect short
committed files, checksum/frame corruption, missing segment, wrong file type,
path replacement, SQLite corruption/lock/busy/readonly, WAL/shm loss, schema or
migration checksum mismatch, partial migration, fsync/rename/directory-sync
failure, ENOSPC at every append/closeout/audit phase, counter/reservation drift,
orphan temp/tombstone files, interrupted audit/age rotation, tombstone cap,
filesystem free-space breach, permission change, backup inconsistency, and
restore mismatch. Gate only affected scopes, expose incidents promptly, and
require acknowledgement before clearing known loss.
- `BH-RET-01..20` — Reject new command when its own cap cannot fit; reject one
client/global admission when active/non-evictable data alone fills the tier;
evict terminal commands with no output; charge scripts/stdin/metadata/history/
idempotency data; never partly evict a command; preserve active commands,
tombstones, audit, unresolved incidents, assigned client events, and closeout
reserve; handle no eligible victim; retain truncation metadata; apply 30-day
and independent audit age/byte rotation exactly once.
- `BH-SVC-01..28` — Reject non-admin install/config/uninstall, untrusted binary/
config path, direct internal service/launcher/helper mode, wrong SCM launch,
duplicate/corrupt service registration, insecure ProgramData ACL, malicious
tray pipe client, spoofed admin claim, cross-session tray action, oversized/
stalled IPC, second tray mutex, config/log path substitution, Run-key quoting
attack, startup config invalidity, network unavailable at boot, tray absent/
crashed, Explorer absent, service start timeout, forced stop with active Jobs,
uninstall interruption, log disk-full/rotation sharing violation, and Server
Core without GUI. Keep service live/not-ready where designed and never let tray
failure own daemon state.
- `BH-SEC-01..22` — Exercise command/script path traversal, SQL metacharacters,
log/terminal escape injection, Unicode confusables in IDs, environment-value
redaction, secret-looking payloads in metrics/logs/audits, malicious PATH/
COMSPEC/PATHEXT/file association, unsafe executable replacement, named-pipe ACL
bypass, remote pipe access, inherited-handle leakage, decompression allocation
abuse, cursor forgery, JSON-RPC bind warning, unauthenticated JSON-RPC authority
as the explicit v1 behavior, and least-access state/work/log ACLs. Tests assert
the documented insecure boundary rather than pretending authentication exists.
- `BH-OPS-01..20` — Handle asynchronous recovery timeout, one dirty scope, metrics
scrape during churn, log sink failure, full audit budget, invalid health path,
shutdown deadline, kill during shutdown, restart loop, incompatible/failed
upgrade, downgrade attempt, backup destination full, restore with wrong config,
clock rollback/forward/DST, hostname/client-ID change, and missing OS diagnostic
data. Liveness stays meaningful, readiness stays conservative, and absence is
never fabricated as zero/healthy data.
- `BH-HARNESS-01..26` — Reject invalid/traversal run ID, corrupt/truncated/foreign
manifest, commit/image/config/VM mismatch, live lock stealing, stale lock with
live owner, unlabelled or wrong-repository Docker resource, symlinked run root,
path outside `.test-runs`, Windows resource outside test root, secret in report,
oversized artifact, no disk, unavailable image/runner, port collision, lost
runner, interrupted setup/collect/cleanup, failed Docker/SCM removal, unsafe
resume, reuse attempting mutable-state copy, GC of resumable run, purge without
confirmation, and `--all` with unrelated resources. Fail closed and print an
exact recovery/cleanup instruction without invoking global prune.
#### Obscure race, crash-window, and interleaving cases
All race cases run under deterministic schedule hooks first, then selected cases
run repeatedly with the Go race detector or native process concurrency. A case
must assert durable state, process count/identity, quota counters, emitted event
order, and leaked goroutine/handle/file counts—not only the CLI exit status.
- `RC-DOM-01..18` — Concurrent identical/conflicting command creation; mutation
retry while original commits; cancel versus queue dispatch, acceptance,
preparation, authorization, running, and terminal commit; signal versus cancel/
terminal; stdin write versus close/terminal; late resource snapshot versus
terminal; duplicate terminal events; command revision increments from multiple
controllers; UUIDv7 generation in one millisecond and across clock rollback.
- `RC-SES-01..20` — Same-instance reconnects cross; old read/write loops race
fencing; different-instance reconnect races grant creation/expiry/consumption;
session closes while dispatch reserves capacity; capacity update crosses
acceptance/rejection; reconnect crosses queue expiry; stale output arrives
before/after new generation commit; reconciliation result is lost while new
dispatch wakes; two clients contend for global fairness; heartbeat timeout
crosses Pong/read activity/write deadline/server shutdown.
- `RC-CLIENT-01..22` — Duplicate dispatch crosses durable local admission;
acceptance acknowledgement crosses service crash; queue dequeue crosses cancel/
shutdown; output/event sequence assignment crosses Ack/reconnect/compaction;
terminal acknowledgement crosses local cleanup/tombstone insertion; server
discard result crosses retransmission; local quota reservation crosses script/
stdin/output writes; reconnect crosses config restart; identity/spool recovery
crosses connection startup; two commands become runnable as capacity changes;
stop-all crosses a new prepared Job. Preserve at-most-once execution, monotonic
event order, and accurate advertised capacity.
- `RC-STORE-01..28` — Event append races Ack/query/retention; terminal commit
races output drain and whole-command eviction; eviction races pagination,
follow, idempotency retry, and age rotation; tombstone insertion races full-
record deletion/replay; audit rotation races incident resolution; repair races
startup recovery/retention/backup; command/client/global reservations cross
exact caps concurrently; closeout consumes its reserve while disk floor trips;
group commit crosses process kill; WAL checkpoint crosses readers/shutdown;
two failures create incidents for one scope without reopening resolved history.
- `RC-OUT-01..20` — stdout/stderr readers race root exit/descendant exit; child
inherits pipe while grace expires; raw backlog crosses high/low repeatedly;
compressor completion reorders across commands but not within one command;
Ack arrives during reconnect/spool compaction; output assignment races client
loss conversion; server retention marker races live follow; terminal event
races final output/incomplete marker; slow follower disconnects while page read
crosses segment eviction.
- `RC-SCRIPT-01..12` — Duplicate chunks arrive concurrently; commit crosses last
fsync, cancellation, quota eviction pressure, client restart, digest failure,
and launch eligibility; cleanup crosses terminal acknowledgement/service death;
wrapper identity is swapped between validation and creation/revalidation;
identical upload resumes from competing old/new sessions.
- `RC-WINCTX-01..18` — User logs on/off, locks/unlocks, disconnects/reconnects,
or changes console session during enumeration/token construction/revalidation;
session ID is reused with a different logon SID; multiple RDP sessions become
active/ambiguous; linked-token policy changes; profile unload races Job exit;
active-user CWD disappears or ACL changes before release. Re-enumerate at most
once as specified, never retarget silently, and never fall back after
`launch_prepared`.
- `RC-LAUNCH-01..30` — Cancellation/service stop/client crash at every instruction
boundary among durable acceptance, token selection, Job/pipe creation,
suspended launcher creation, Job assignment, pipe authentication, suspended
shell report, prepared fsync, authorized fsync, release send, resume, running
acknowledgement, root exit, drain, terminal fsync, EventAck, cleanup, and
tombstone insertion. Duplicate dispatch after each restart must yield zero or
one process, never two.
- `RC-SIGNAL-01..16` — TERM/KILL/cancel collide with launcher connection, shell
resume, root exit, descendant-only Job, helper pipe authentication,
`AttachConsole`, CTRL_BREAK delivery, grace timeout, Job termination, service
shutdown, PID reuse, and repeated signal revision. Record the actual attempted/
escalated result and never signal an unrelated PID/console.
- `RC-SVC-01..20` — SCM start/stop/restart/uninstall cross recovery, command
admission, prepared launch, and active Jobs; two installers/configurators race;
tray startup crosses logon/logoff/Explorer restart/service restart; multiple
trays contend for the per-session mutex; pipe disconnect crosses a mutating
helper result; log rotation crosses tray open/read; config edit crosses explicit
restart; Automatic/Manual change crosses SCM query.
- `RC-OPS-01..14` — Health/metrics/read RPCs cross recovery state transitions,
dirty resolution, retention, and shutdown; log/audit rotation crosses process
crash; backup crosses group commit and dispatch pause; upgrade crosses queued/
active commands and rollback; wall-clock changes cross queue TTL/retention while
monotonic heartbeat remains correct.
- `RC-HARNESS-01..18` — Controller dies before/after manifest fsync, resource
creation/label recording, Windows lease, fault checkpoint, result write,
artifact collection, stop/reset/purge, and lock release; two cleanup commands
race; status/collect runs during cleanup; resume crosses a late old controller;
GC crosses a newly resumed run. Recovery must converge idempotently and never
delete an unrelated resource.
For each crash-window family, implement a loop that enumerates named hooks rather
than hand-selecting a few attractive points. Persist the hook name before
triggering death, restart from the real store/spool, replay the same external
request, and check the invariant matrix. A newly added durable write or external
side effect must add a hook and coverage row before merging.
### 2.6 Native Windows test-host requirements
Native Windows is mandatory for the Phase 4/5 gates. Development can begin with
unit tests and cross-compilation before a host is connected, but the Windows
supervisor/service/tray implementation cannot be called complete without it.
Prefer disposable or snapshot-resettable VMs over a personal workstation.
Provision at least one primary interactive host with:
- current Windows 11 Pro/Enterprise, Desktop Experience, Explorer, `cmd.exe`,
Windows PowerShell 5.1, UAC enabled, and all current updates captured in a
named clean snapshot;
- at least 4 vCPU, 8 GiB RAM, and 60 GiB free disk recommended (2 vCPU, 4 GiB,
and 30 GiB free is the minimum smoke lane), with reboot and snapshot-revert
authority;
- one local standard account and one traditional split-token local administrator,
plus a separate automation principal able to install/control the test service;
- a preconfigured CI runner, OpenSSH, or WinRM management channel reachable only
from the test controller; credentials live outside the repository and run
manifest;
- bidirectional reachability to the Linux nginx test endpoint, stable DNS or a
supplied address, time synchronization, and permission to transfer the
CI-built binary/config/CA into a dedicated test root;
- no valuable user data, credentials, mapped drives, or production services,
because tests deliberately change SCM/Run-key state, ACLs, UAC fixtures, logon
sessions, kill processes, fill bounded scratch storage, reboot, and revert.
Maintain snapshot/policy variants for logged-out, standard-user active,
split-token-admin active, UAC disabled/already-full admin, and Administrator
Protection when supported. The harness must restore the baseline after any
variant that changes machine policy. A Windows Server Desktop Experience VM with
multiple simultaneous `WTSActive` sessions is required for the final native
ambiguous-session case; until available, keep its exhaustive selector unit test
mandatory and mark only that native case blocked. Add a Server Core VM for the
headless Session 0 release smoke. Before publishing, also exercise the oldest
supported Windows 10 or Server 2016 baseline; it need not be the everyday runner.
A physical Windows machine is optional. It is useful for an additional real
display/audio/DDC command smoke, but RVBox only guarantees correct token/session/
process execution—not success of arbitrary vendor hardware APIs—so physical
hardware is not a release blocker. `test-host.ps1 Prepare` must inventory the
host against this checklist and refuse destructive suites unless the machine is
explicitly marked disposable/resettable and the clean snapshot identity is
recorded.
## 3. Repository and build bootstrap (Phase 0) ## 3. Repository and build bootstrap (Phase 0)
### 3.1 Establish the repository layout ### 3.1 Establish the repository layout
@@ -384,6 +805,7 @@ scripts/
test-env # doctor/recover/reuse/cleanup by exact run ID test-env # doctor/recover/reuse/cleanup by exact run ID
windows/test-host.ps1 # native Windows host lifecycle adapter windows/test-host.ps1 # native Windows host lifecycle adapter
test/ test/
coverage.toml # requirement-to-case inventory with stable IDs
harness/ # shared manifest, journal, orchestration, and reporting harness/ # shared manifest, journal, orchestration, and reporting
defaults.toml # local resource/time/artifact budgets defaults.toml # local resource/time/artifact budgets
integration/ # cross-package/real-resource suite definitions integration/ # cross-package/real-resource suite definitions
@@ -425,8 +847,8 @@ Add:
resource limits, run-ID labels, ephemeral PKI, fault proxy, and artifact/state resource limits, run-ID labels, ephemeral PKI, fault proxy, and artifact/state
mounts. It accepts only values produced by the validated harness manifest and mounts. It accepts only values produced by the validated harness manifest and
never uses an implicit default project name. never uses an implicit default project name.
- `Makefile`: `generate`, `fmt`, `lint`, `test`, `test-race`, `test-integration`, - `Makefile`: `generate`, `fmt`, `lint`, `test`, `test-race`, `test-coverage`,
`test-e2e`, `test-status`, `build`, and `verify`. `test` aliases the unit `test-integration`, `test-e2e`, `test-status`, `build`, and `verify`. `test` aliases the unit
layer; integration/E2E targets delegate to the checked-in scripts and print layer; integration/E2E targets delegate to the checked-in scripts and print
their run ID. Each target invokes the container workflow and must not silently their run ID. Each target invokes the container workflow and must not silently
fall back to host tools. fall back to host tools.
@@ -451,11 +873,13 @@ Add CI (or a repository script ready for CI) that runs, in order:
v1 compatibility baseline; enable `buf breaking` against that baseline v1 compatibility baseline; enable `buf breaking` against that baseline
immediately after it merges, not against the obsolete draft. immediately after it merges, not against the obsolete draft.
2. Generation freshness check. 2. Generation freshness check.
3. `go fmt`, `go vet`, static analysis, and unit tests. 3. Coverage-inventory validation: every required stable ID maps to a listed
4. Race tests for server/client concurrency packages. case and every normative critical invariant has at least one case.
5. Linux server/storage/session integration tests in Compose; no Linux client 4. `go fmt`, `go vet`, static analysis, and unit tests with per-package coverage.
5. Race tests for server/client concurrency packages.
6. Linux server/storage/session integration tests in Compose; no Linux client
supervisor is required at this stage. supervisor is required at this stage.
6. Cross-compilation checks for Windows packages in the container plus native 7. Cross-compilation checks for Windows packages in the container plus native
Windows build/runtime smoke tests on a Windows runner. A Windows runner is a Windows build/runtime smoke tests on a Windows runner. A Windows runner is a
Phase 0 prerequisite, not a later optional enhancement. Phase 0 prerequisite, not a later optional enhancement.