Files
rvbox/docs/testing.md
T

24 KiB

RVBox implementation test workflow

All Go, protobuf, and harness work runs in the pinned toolchain container. No host Go installation is used.

Run the normal pre-commit gate with:

make verify

Run focused unit tests with:

scripts/test-unit --package ./internal/domain --run UUIDv7 --race

Run one bounded hostile-input fuzz target in the same pinned container with:

scripts/test-fuzz --package ./internal/agentproto --name FuzzDecodeEnvelopeBounded_SEC_PROTO_01 --time 30s
scripts/test-fuzz --package ./internal/server/control --name FuzzDecodeJSONRPCRequestBounded_SEC_CTL_01 --time 30s

The fuzz command disables ordinary tests for that invocation and runs exactly one target. A crash stores Go's minimized reproducer in the affected package's fuzz corpus, so the normal test gate exercises it as a deterministic seed after it is reviewed and committed.

Build all supported binaries without installing make or Go on the host:

scripts/build build

scripts/build doctor reports the exact owned builder image, Go cache volumes, and ignored bin/ artifact directory. To reclaim space, cleanup is dry-run by default and never performs a global Docker prune:

scripts/build clean --all
scripts/build clean --all --execute --yes
scripts/build recover

The second command removes only bin/, rvbox-dev-toolchain:latest, and the two named RVBox Go-cache volumes. recover rebuilds the pinned toolchain and all binaries from source. It intentionally does not remove .test-runs/, which may contain resumable environments; use the exact-run scripts/test-env purge --run-id ID --execute --yes workflow for those.

For a native Windows run, create the immutable per-run test bundle with the pinned toolchain. Supply a test-specific config whose endpoint and CA path are valid for that run; the command refuses to replace an existing bundle.

scripts/windows/build-test-bundle \
  --run-id windows-smoke \
  --config .test-runs/windows-smoke/client.toml \
  --ca .test-runs/windows-smoke/ca.pem
scripts/windows/test-host prepare --run-id windows-smoke
scripts/windows/test-host stage --run-id windows-smoke \
  --bundle .test-runs/windows-smoke/windows-bundle
scripts/windows/test-host install --run-id windows-smoke

The bundle manifest records the source commit and SHA-256 of every bundled file. It is a test artifact, not a signed release package; release signing, version resources, and publication are Phase 8 gates.

The preferred complete native lane is now one command:

scripts/windows/native-test run --run-id windows-hierarchy

It builds a disposable fixture-tagged Windows binary in the pinned toolchain, starts the real Linux server and nginx TLS proxy in Docker Compose under test/linux-server on the current controller, and connects the Helium-hosted NAT guest to the current controller's explicit endpoint (x1.xcel.me by default). Helium hosts the VM only; it does not host any RVBox server containers. The self-signed server certificate is intentionally accepted by the v1 client without a test CA. It then drives the installed SCM service through the server's real Unix control socket and verifies every Windows execution context, ordered stdin close, TERM delivery, and a server-process restart while a command is running. The restart check preserves the server state volume, waits for a new reconciled WSS session, then proves that the same command can receive its terminal signal; it covers reconnect without treating the old session as valid. The tagged binary's controlled pre-launch failures are limited to the test fixture; a release binary rejects that switch.

Successful runs collect bounded artifacts, remove only their labeled Compose project, and restore the exact clean snapshot. A failed or --keep run stays recoverable:

scripts/windows/native-test recover --run-id windows-hierarchy
scripts/windows/native-test clean --run-id windows-hierarchy
scripts/windows/native-test clean --run-id windows-hierarchy --purge --yes

clean retains local artifacts by default. The explicit purge form removes only the exact local run root after the local stack is down and the VM snapshot has been restored.

The integration harness provides the Phase 0 sample suite, the incremental Phase 2 store suite, Phase 3 server-session, and control and client-agent protocol suites. The deterministic E2E lane provides smoke, interactive, idempotency, reconnect, retention, expiry-and-incidents, script, and recovery. Native Windows execution contexts remain the separately leased scripts/windows/native-test lane. The resumable E2E harness adds smoke, script, recovery, and all scenarios. Each run writes its manifest and run ID before starting work. The storage suite uses a real temporary SQLite database in WAL mode and a real segment/audit filesystem. The session suite uses a real HTTP/WebSocket listener, binary protobuf frames, SQLite fencing, and the race detector; neither mocks its respective durability or wire boundary:

scripts/test-env doctor
scripts/test-integration --suite sample --run-id my-sample
scripts/test-integration --list
scripts/test-integration --suite all --run-id integration-all
scripts/test-integration --suite store --case '^TestStore' --run-id store-one-case
scripts/test-env status --run-id my-sample
scripts/test-env collect --run-id my-sample
scripts/test-env reset --run-id my-sample
scripts/test-env reuse --run-id my-sample --new-run-id my-sample-retry
scripts/test-integration --suite sample --run-id my-sample-retry --resume
scripts/test-env reset --run-id my-sample-retry
scripts/test-env purge --run-id my-sample --execute --yes
scripts/test-env purge --run-id my-sample-retry --execute --yes

scripts/test-integration --suite store --run-id store-smoke
scripts/test-env logs --run-id store-smoke
scripts/test-env collect --run-id store-smoke
scripts/test-env reset --run-id store-smoke
scripts/test-env purge --run-id store-smoke --execute --yes

scripts/test-integration --suite server-session --run-id session-smoke
scripts/test-env logs --run-id session-smoke
scripts/test-env collect --run-id session-smoke
scripts/test-env reset --run-id session-smoke
scripts/test-env purge --run-id session-smoke --execute --yes

scripts/test-e2e --scenario smoke --run-id e2e-smoke
scripts/test-e2e --list
scripts/test-e2e --scenario smoke --case '^Test' --run-id e2e-one-case
scripts/test-env status --run-id e2e-smoke
scripts/test-env recover --run-id e2e-smoke
scripts/test-e2e --scenario smoke --run-id e2e-smoke --resume
scripts/test-env reset --run-id e2e-smoke
scripts/test-env purge --run-id e2e-smoke --execute --yes

The client runtime unit lane also exercises a real child process through the portable supervisor adapter. internal/client/agent/executor_test.go verifies that command text is accepted once, output is journaled, lifecycle/terminal events are durable, and a script cannot launch before its contiguous upload is committed. The Windows build uses the same executor contract with the platform-native adapter: a verified token is selected, the child is created suspended, assigned to a kill-on-close Job, and held behind an authenticated per-command launcher pipe. The daemon records launch_prepared, then the durable launch_authorized transition sends the launcher's release frame; the launcher resumes the shell only after that acknowledgement. The durable launch_phase barrier is recovered as interrupted after a daemon restart, so an uncertain release is never redispatched.

The Windows artifact is linked with the GUI subsystem (-H=windowsgui) so service, launcher, and tray startup do not flash a console. Human-facing modes still attach to a parent console explicitly when one exists.

Persisted output-loss and reconnect regressions

BH-RUNTIME-05 through 08 cover a capture-reader failure producing a valid, normally ordered incomplete-output event before terminal state; replay of an early v1 event with missing stream scope after reopening its original spool; deterministic replay digests without rewriting stored evidence; continued rejection of invalid/duplicate explicit streams; and reserved closeout under normal command-quota exhaustion. The known legacy repair conservatively names stdout and stderr. It does not weaken the protocol validator or delete history.

REC-RUNTIME-01 through 03 cover replay from the post-reconciliation snapshot, stale internal wakeups after terminal discard, and acknowledgement credit draining a durable tail without a new producer notification. REC-RUNTIME-04 uses a real WebSocket server and durable client/server stores: deliver a legacy loss marker plus terminal result, restart the same client spool, discard the server-confirmed terminal row, and complete a fresh command. It also requires prompt server-session cleanup rather than waiting for a heartbeat tick.

docker compose -f deploy/compose.yaml run --rm toolchain \
  go test -race ./internal/client/agent ./test/integration/clientagent

The runtime unit cases are platform-neutral and can also be compiled as a Windows console test executable. Inspect the persisted typed loss event itself when checking evidence: the existing server's GetOutput summary does not yet derive its incomplete flag/stream details from every incoming peer loss marker.

Focused native tray and installer regression checks

internal/client/windowstray/native_windows_test.go contains real Win32 tests, not Linux stubs: enabled/deny-only primary-token peer checks, one-shot status, rejection before reading a request, malformed responses, stalled-peer deadlines, eight idle peers plus a bounded ninth request, shutdown cancellation, native NOTIFYICONDATAW layout, and actual icon creation/message-loop/close under GC. Build its executable in Docker:

docker compose -f deploy/compose.yaml run --rm toolchain \
  env CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go test -c \
  -o .test-runs/RUN/windowstray.test.exe ./internal/client/windowstray

Use the checksum-verified fixture transfer route to place it at a distinct test-only path, never the installed SCM service executable. On a prepared interactive fixture, run it through the server control socket:

rvc run --shell cmd CLIENT \
  '@C:\Users\Public\RVBoxTrayTests.exe -test.v -test.run TestNativeTray -test.timeout 45s'

Require PASS and the durable command lifecycle COMMAND_SUCCEEDED; the presence of a process or a successful RPC alone is not test success. The UI case briefly creates/removes its own test icon. Session 0 runs verify rejection/transport but skip the actual desktop-window case, so they do not substitute for the interactive run.

For checks against the installed tray itself, run the probe/test runner from Explorer's desktop (for example, the VM's Run dialog), not merely a process with an active-user token. Matching WTS session IDs alone does not prove that the probe shares Explorer's window station/desktop; a service-created command can otherwise report a false-negative window lookup. Transfer the console test executables separately from the installed service image and write bounded logs to the run's owned test-artifact directory.

The status/menu tests additionally cover connected versus disconnected versus stopped, missing/stalled IPC, pending SCM states, a disabled status banner, exactly one Auto start checkbox, no Restart menu item, native disabled/checked flags, menu refresh after a state change, and normal-user read-only SCM access. Installer tests verify the original pre-UAC user SID survives elevation, invalid SID strings cannot enter SDDL, config ACL repair preserves contents and grants that user read/write without an Everyone/Users grant, and status reporting does not depend on the optional HTTP observability endpoint. Acceptance checks must also prove config can be opened/saved unelevated, spool ACLs remain private, Auto start reflects actual SCM state after successful or cancelled UAC, and a live server disconnect/reconnect changes the banner without stopping the daemon.

cmd/rvbox/installer_flow_test.go checks install-before-tray ordering, cancelled/ failed elevation, and explicit partial success when tray startup fails. The native installer structure-layout test is in installer_windows_test.go. For the real double-click acceptance route, preserve UAC policy: the unelevated parent must wait for its elevated child to exit successfully, start the installed tray with the original user's token, and confirm a registered icon before its single success dialog. Its entire message is Installed and is running on server <server_url>, using the installed configuration's URL (including a preserved URL on upgrade). Verify both service and tray identities, current-session startup without a reboot, Run-key sign-in startup, duplicate prevention, and continued service operation when only the tray exits. The tray must remain visible/responsive when its service is absent or its IPC peer stalls. Windows may put a new application icon under its hidden-icons arrow; do not rewrite the user's global notification-area preferences to force pinning.

Installer lifecycle regressions HP-WINCLI-10/11 and BH-WINCLI-03/04 run on Linux: fresh/upgrade ordering, healthy same-version no-op, stopped/paused/pending SCM states, missing tray, cancellation/inspection failure without subsequent mutations, and version-string/hash identity. Native HP-WINCLI-12..15 cover SCM path retention, packaged PE version strings, concurrent installer exclusion, and real Win32 executable replacement while retaining adjacent config contents. Build ./cmd/rvbox with GOOS=windows GOARCH=amd64 go test -c as a distinct console test executable. To exercise the actual release resource, set RVBOX_NATIVE_RELEASE_EXE to the package path and RVBOX_NATIVE_RELEASE_VERSION to its full release label before running it; otherwise that one case explicitly skips.

On an interactive physical machine or disposable Windows fixture, test:

  1. Upgrade an older installed package. Confirm UAC, exactly one service and tray, the new executable version, unchanged custom config/client identity/history and Automatic/Manual setting, and the minimal success dialog.
  2. Run that same package again with both components running. Require no UAC or dialog and unchanged service/tray PIDs. It must not reinstall or rewrite files.
  3. Exit only the tray and relaunch setup. Require a new tray with unchanged service PID and no UAC. Stop the service and repeat with/without the tray; require service start (UAC when necessary) and only a missing tray to start.
  4. Cancel the service-start/upgrade UAC prompt. Require no replacement/new tray. Launch two setups during UAC; the second must exit quietly. On a disposable fixture, reject invalid existing TOML before stopping the working service and exercise replacement/startup failure plus best-effort prior-binary recovery.

These native lifecycle checks are pending for the new version-aware installer while Helium is offline; passing portable tests and Windows cross-compilation does not prove them. Retain the previous installer for recovery during physical testing, and collect service-startup.log if service start fails.

Regression BH-WINTRAY-06 checks successful HRESULTs (S_OK and S_FALSE) versus actual failures. On the Windows 10 fixture, the installed, responsive tray returned S_FALSE (1) with valid icon coordinates; a readiness check that accepts only zero falsely times out. Native BH-WINTRAY-07 calls the installer's readiness check against the already-installed icon; require a non-skipped pass on an installed interactive fixture. It skips if that desktop has no RVBox tray. The native window-lifecycle check also exercises readiness for its test icon. Preserve the user's icon visibility preferences during verification.

Suite output is capped by the checked-in harness policy (10 MiB by default) and stored as artifacts/suite.log. A failed run remains inspectable and can be moved back to ready with recover, then resumed with the same run ID and deterministic shuffle seed only while its source definitions and working tree still match its manifest. reuse is intentionally different: after a reset or completed run, it creates a fresh manifest/run ID with the same layer, suite, and case selection, leaving the original evidence unchanged. reset refuses a running run or one with recorded owned runtime resources, removes only its ephemeral runtime, pki, and scratch directories, and retains the manifest, journal, reports, and artifacts. Test-run cleanup never removes the shared Go module or build-cache volumes.

Each run owns only .test-runs/<run-id> and resources explicitly recorded in that run's versioned manifest. The journal is append-only and fsynced. purge first prints the exact validated target and is dry-run by default. It requires --execute --yes, refuses symlink targets or manifests that still list runtime resources, and then removes only one completed/reset run (or eligible runs selected by --all). gc --older-than DURATION uses the same explicit execution confirmation. Purged artifacts are not recoverable. Dependency cache volumes are never part of run cleanup.

test/coverage.toml is the incremental requirement-to-test inventory. The make verify lint stage checks unique stable IDs and verifies every implemented test reference against source. A resettable Windows smoke VM is now available. The authoritative fixture record is testing-vm.md: it lists the VM/host UUIDs, Windows build, hardware and device profile, NAT and VRDE endpoints, snapshot UUIDs, credential-file contract, and the required reset sequence. The reset target is baseline-clean-python-usb-ready, which includes all-users Python 3.13.16 on machine PATH, pip and venv, plus standard USB-tablet input without the problematic VBoxMouse class filter, but no installed RVBox client. Preparation verifies that mouse path; recovery and rollback are documented with the RDP helper. The guest address 10.0.2.15 is DHCP state only; use SSH plus VirtualBox Guest Control rather than treating it as a stable endpoint. VRDE is enabled only on Helium loopback at 127.0.0.1:3389 for diagnostics, while native Windows RDP is disabled in the baseline.

Direct LAN RDP access can be enabled with test/rdp-access/rdp-access lan-up: the small Docker relay runs on Helium, binds its private LAN IP, restricts peers to that subnet, and preserves the existing private VRDE backend and Windows session. The independent lan-status/lan-down/lan-clean lifecycle is documented in the RDP helper. Disconnect the Guacamole desktop before switching to an RDP app; VRDE still has one console slot and this does not enable Windows TermService.

Interactive browser access is a deliberately temporary recovery path only. See test/rdp-access for the Docker-only, self-signed HTTPS Guacamole lifecycle; it must be started only after the native fixture controller has prepared and leased the VM, and stopped before reset. For a public hostname with an AAAA record, its up --bind 0.0.0.0 mode also tracks an IPv6-to-IPv4 forward; verify ipv6_forward=active before debugging a browser stuck at “Waiting for response”. If several already-running VMs need browser access at once, use the documented RDP_ACCESS_PROFILE and matching host/VRDE/HTTP/tunnel-port overrides in the helper README.

The Linux production Compose asset has a separate, loopback-only smoke lane. It uses a disposable self-signed key only under the ignored .test-runs tree, starts the real non-root server and nginx services, probes TLS liveness and readiness, and invokes rvc through the private socket. It leaves compact logs and output for inspection, while removing its exact Compose project by default:

docker build -f deploy/Dockerfile.runtime -t rvbox-server:test .
scripts/test-production-compose run --run-id production-smoke
scripts/test-production-compose clean --run-id production-smoke --purge --yes

The canonical headless VirtualBox/Guest Control adapter is scripts/windows/test-host. It is a POSIX controller script because the fixture's VirtualBox host is Arch Linux and has no PowerShell runtime. The controller connects to Helium over SSH; VBoxManage and the host-only password file never need to exist on the Linux development controller. It takes the VM identity, baseline snapshot, guest identity, and password-file only from host environment variables, acquires an exclusive remote lease, and never writes secrets to the repository, run manifest, or command line. Set RVBOX_TEST_GUEST_PASSWORD_FILE to the mode-600 host-side file; the adapter passes it only as VirtualBox --passwordfile. The provisioned fixture's VM identity, fixed test-only account names, and password-file path are safe defaults in that script and may be overridden for another documented fixture. The fixed disposable-VM password remains only in that mode-600 file; the controller never puts it on a command line, manifest, log, or artifact.

The native lifecycle is status, prepare, stage, install, run, collect, logs, stop, and reset. prepare verifies the VM and snapshot UUIDs, restores the clean baseline, starts headless, waits for Guest Additions, and proves that RVBoxClient is absent. stage copies a versioned non-secret test bundle through a run-specific host directory to a run-specific guest directory. install uses the fixture-only high-integrity automation principal to invoke the real rvbox.exe --install-service path and proves completion through SCM. run is for reconfiguration/restart scenarios after that first installation. Neither action invokes the GUI-subsystem executable directly with the normal Guest Control account. collect obtains only bounded/redacted artifacts. logs is the narrow read-only service-startup/client-diagnostics action for a retained prepared run. The service diagnostic file grants access to SYSTEM and local Administrators only; it contains no command spool data. reset restores the exact clean baseline and leaves the VM powered off. It first permits a bounded ACPI shutdown; if that hangs, it force-powers off only the exact leased disposable VM before snapshot restoration. That intentional state loss is confined to the test isolation boundary.

This service-driven protocol is required because VirtualBox Guest Control 7.2.16 does not reliably complete a direct GUI-subsystem rvbox.exe run; wrapping it in cmd.exe can leave the Guest Control wrapper waiting. Guest Control is therefore limited to console-safe setup tools (sc.exe, whoami, query, bounded file operations) and artifact collection. A native E2E run also performs a bounded guest-to-nginx HTTPS/TCP readiness probe before it starts the service. Wine and protocol stubs are not equivalent Windows coverage.

For the hierarchy fixture only, run --fail-contexts ACTIVE_USER_ELEVATED[,ACTIVE_SYSTEM] restarts the separately tagged test service and forces the named token preparation step to fail before process creation. This proves the real daemon's fallback order without changing the wire protocol, normal client TOML, or release binary. The logoff action ends the sole active fixture session so the LocalService and no-user LocalSystem rows can be tested with the same running service.

The clean baseline intentionally contains no RVBox service, tray registration, or RVBox state. Guest Control supplies rvboxtest with a filtered medium UAC token, so it cannot safely perform the first machine-wide install. The fixture therefore uses its separately enabled built-in Administrator account as a test-only full-token automation principal. Its FilterAdministratorToken must remain 0, preserving UAC for rvboxtest rather than disabling it machine-wide. Its username and mode-600 host-side password-file are provided only as RVBOX_TEST_PROVISIONER_USER and RVBOX_TEST_PROVISIONER_PASSWORD_FILE for the install/machine-mutation actions. It is not an RVBox process, service, broker, or Task Scheduler dependency, and it is never used to choose a command execution context. The normal rvboxtest console session remains the subject of active-user and elevation tests. See testing-vm.md for the exact fixture contract.

The VM is the current Windows 10 v1 smoke lane. Native multi-session/ambiguous-session, Server Core, and older-build entries are explicitly deferred compatibility work until their own fixtures exist; they do not block the Windows 10 v1 baseline. Their pure selector tests remain mandatory.