Files
rvbox/docs/platform-and-operations.md
T

362 lines
22 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# RVBox v1 platform and operations contract
Daemon configuration uses strict TOML as specified in
[`configuration.md`](configuration.md); the annotated examples contain every
v1 knob and default.
The v1 implementation scope is the Linux server plus Windows client. A
Unix-like client is post-v1 work and must not be implemented as part of this
milestone. Its section below is retained only as future design material; the
document order does not authorize or reprioritize that work.
## Unix-like clients
The client starts `sh` or `bash` in a new session/process group. Unix signals
address that group, so normally created descendants receive the signal too. On
orderly shutdown, or recovery after an unclean daemon failure, managed command
groups are terminated and marked interrupted because pipe capture cannot be
safely resumed.
Both command text and uploaded scripts execute from generated private files
beneath the effective CWD using exactly the selected executable (`sh FILE` or
`bash FILE`). No user-supplied filename becomes a filesystem path. The wrapper
file is removed during terminal cleanup.
Root-process exit begins a configurable 5-second drain grace period. RVBox waits
for the supervised tree and capture pipes, then terminates residual group/cgroup
members, drains to EOF, and only afterward emits the terminal lifecycle event.
If capture still cannot reach EOF, it closes the handles and emits explicit
incomplete-output metadata first. Shell-level detachment is not a supported way
to leave descendants running; callers use RVBox background mode instead.
Launch uses an internal blocked launcher rather than starting requested command
code directly. The launcher establishes its session/process group, reports its
identity, and waits on a private release/watchdog channel. The client durably
records `launch_prepared`, then durably records `launch_authorized`, and only
then sends the release token. The launcher creates the requested shell inside
that group and remains as a non-user-code watchdog until the tree exits. The
daemon keeps the channel open for that lifetime: EOF before authorization exits
without execution, while EOF after release terminates the group. Once
`launch_authorized` is durable, recovery never retries that UUID; an uncertain
launch is marked interrupted.
On Linux, create a per-command cgroup v2 for supervision even when no resource
profile was requested, whenever the daemon has a delegated writable cgroup.
Put the blocked launcher into that cgroup before release; use
`clone3(CLONE_INTO_CGROUP | CLONE_PIDFD)` where available, otherwise migrate the
still-blocked launcher through `cgroup.procs`. Persist the cgroup path, PID,
process group, `/proc/<pid>/stat` start time, and launch generation. A live
daemon uses the pidfd where available. Recovery uses `cgroup.kill` as the primary
tree-cleanup operation and verifies the recorded birth identity before any
PID/process-group fallback. Without cgroup delegation it uses the generic
watchdog/process-group fallback unless a requested profile requires cgroup
controls, in which case acceptance fails as unsupported. It never signals a
process based only on a persisted numeric PID or PGID.
Other Unix-like systems use the same launch barrier plus a watchdog control
channel whose EOF triggers process-group termination. Recovery validates the
platform's process-birth identity before signaling. Descendants that deliberately
create a new session may escape this generic fallback, so complete tree cleanup
outside Linux cgroup supervision is best-effort; the at-most-once launch
guarantee still applies.
Linux diagnostics sample `/proc/<pid>` and relevant children for state, CPU,
resident memory, I/O counters, CWD, and wait-channel information when readable.
These values may be unavailable due to permissions, kernel configuration, or a
short-lived process; absence is represented explicitly rather than fabricated.
Cgroup v2 profile limits are applied only when requested; a no-profile
supervisory cgroup imposes no resource limit.
## Windows clients
The minimum supported v1 Windows versions are Windows 10 and Windows Server
2016. Desktop Experience is required only for the tray and active-session
execution contexts; the service and Session 0 contexts may run headless on
Server Core. Installation is an explicit UAC-elevated operation that registers
one machine-wide Automatic SCM service. The service runs as `LocalSystem` before
login and alone owns configuration, WSS, durable state, logs, admission,
history, process handles, and Job Objects. Normal service startup never shows
UAC and is not blocked by tray or interactive-user availability.
The same signed `rvbox.exe` has explicit `service`, `tray`, `install-service`,
`uninstall-service`, `start-service`, `stop-service`, `restart-service`,
`configure-service`, per-command launcher, and signal-
helper modes. Internal modes require SCM state or a service-created launch
proof. Task Scheduler is not used. The installer registers an unelevated per-
user tray launch through the machine-wide `Run` key. One tray may run in each
logged-in session; its exit, failure, disablement, or Explorer restart does not
stop the service or commands.
The executable uses the Windows GUI subsystem so service, tray, and internal
helpers do not flash consoles. Human-invoked console modes such as
`--check-config`, install, uninstall, and service configuration first call
`AttachConsole(ATTACH_PARENT_PROCESS)`, rebuild the standard handles, and emit
normal UTF-8 diagnostics when a parent console exists; otherwise they use a
native dialog or documented exit code. Internal launcher arguments are never
printed. This preserves usable terminal help/errors without adding a second
persistent executable or process.
The tray is a thin frontend over a local-only named pipe. It never opens the
SQLite store/spool or owns a server connection. The pipe rejects remote clients,
has explicit ACLs, and the service impersonates each caller for authorization.
Read-only health/status is available to an interactive local user. Service
start/stop/restart, Automatic/Manual startup changes, config editing, incident
resolution, and other machine-wide mutations require a locally elevated
administrator helper. `Exit` closes only that tray. Server Core and logged-out
machines simply have no tray.
When no config path was installed, resolve `FOLDERID_ProgramData` with the
Windows Known Folder API and use `RVBox\client.toml`, `RVBox\state`,
`RVBox\work`, and `RVBox\logs\rvbox.log` beneath it. Do not expand
`%ProgramData%` text from TOML. State and the work root are writable only by
SYSTEM and Administrators. When `cwd` is omitted, the service creates or opens
an identity-scoped child beneath the work root: a user-SID directory for either
active-user context, a LocalService directory, or a SYSTEM-only directory for
each SYSTEM context. Its non-inherited ACL grants only SYSTEM and the effective
non-SYSTEM SID the required access. The service validates the existing owner,
ACL, and reparse-point state before reuse, so one user cannot pre-create or
modify another identity's work area. Config/log read access needed by the tray
is separate from edit access. First installation writes the annotated template
and lets the service remain live but not ready until routing validates.
Configuration is restart-only. File opening uses exact resolved regular paths,
never a constructed shell command.
### Windows execution-context hierarchy
`ExecutionSpec.elevated` is the only caller-facing privilege choice. Windows
combines it with the presence of one usable active interactive session to select
an effective context. The ordered rules are:
| Login state | `elevated` | Ordered pre-launch contexts |
| --- | --- | --- |
| usable active user | false | `active_user` only |
| usable active user | true | `active_user_elevated` -> `active_system` -> `local_system` |
| no usable active user | false | `local_service` only |
| no usable active user | true | `local_system` only |
Here `active_user` is the selected user's deliberately non-elevated token;
`active_user_elevated` is that user's traditional full administrator token;
`active_system` is SYSTEM placed in the selected session; and the two `local_*`
contexts run in Session 0. “Usable” means the deterministic WTS selection and
token validation below succeeded, not merely that some disconnected session
record exists.
Fallback is allowed only during token/session selection before
`launch_prepared`. Failure to create a launcher/shell, an uncertain launch, CWD
or executable rejection, or failure after authorization never tries another
identity. The persisted attempt list and selection detail state why an elevated
request reached SYSTEM. If a formerly active user logs out during selection,
the service re-enumerates once and applies the no-user row; it never retargets a
different user silently.
The LocalSystem service obtains `local_service` with passwordless
`LogonUserW("LocalService", "NT AUTHORITY", NULL, LOGON32_LOGON_SERVICE, ...)`.
`local_system` duplicates the service token. To find an active user, enumerate
WTS sessions and obtain the selected token with `WTSQueryUserToken`. Prefer a
valid active physical-console session; if none exists, accept exactly one
`WTSActive` interactive session. Multiple remaining candidates are ambiguous,
so there is no usable active user rather than a nondeterministic choice.
Immediately before release, require the same session ID, logon SID, and user SID.
For `active_user_elevated`, inspect `TokenElevationType`. A traditional limited
administrator token must expose a linked full token; an already-full
administrator token is usable as-is. A standard user returns
a `CODE_ELEVATION_UNAVAILABLE` attempt reason. Windows Administrator Protection, or
another policy requiring interactive approval rather than exposing a reusable
full token, records a bounded policy-specific attempt reason. V1 never waits
for a UAC/Hello prompt; an elevated request proceeds to `active_system`.
That context duplicates the service token and sets its `TokenSessionId`
explicitly. If session-scoped SYSTEM construction also fails before preparation,
the final context is Session 0 `local_system`. This fallback intentionally
bypasses user-scoped approval because the installed service already holds
SYSTEM; it must be conspicuous in status/audit history. The final
`local_system` fallback supplies elevation but not access to the interactive
desktop: a display, audio, or other session-scoped command may therefore fail
normally in Session 0. Selection fallback is not a promise that the requested
operation is meaningful in the resulting context.
For normal `active_user`, use the filtered token when Windows supplies one. If
an administrator is logged in with only a full token (for example traditional
UAC is disabled), create and verify a LUA-style restricted medium-integrity
token with administrator SIDs deny-only and unnecessary privileges removed.
Never let `elevated=false` inherit a full administrator token merely because of
machine policy; if a verified non-elevated token cannot be built, reject rather
than use SYSTEM or LocalService while that active session remains selected.
Build the base environment from the effective token and then apply persisted
overrides. Session 0 contexts do not inherit a person's profile, mapped drives,
user certificates, or per-user proxy settings. LocalService may use ordinary
DNS/TCP/HTTP(S), localhost sockets, and ACL-permitted local files, but presents
anonymous credentials to remote Windows resources. Active-user contexts load
that user's profile/environment as needed and unload only after the whole Job
exits. `active_system` remains SYSTEM: putting it in the user's session does not
give it that user's HKCU, profile, mapped drives, or network credentials.
### Windows process launch and supervision
For every command, create a non-inheritable Job Object, set
`JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE`, and do not enable breakaway. Use a short-
lived stateless per-command launcher mode of the same executable for every
context. This uniform path is required because Windows does not permit normal
inherited handles across Terminal Services sessions. The launcher owns no
durable state or policy and remains inside the command Job only to bridge
standard I/O and watch its shell.
The service creates local named pipes for launcher control and stdin/stdout/
stderr using unpredictable per-command names, `PIPE_REJECT_REMOTE_CLIENTS`, and
ACLs limited to SYSTEM plus the selected token SID. It creates the launcher
suspended with `CreateProcessAsUser`, no inherited handles, the selected token/
session, and an opaque channel identifier. Before resuming it, the service
assigns the launcher to the empty Job. On connection, verify the named-pipe
client PID, its `GetProcessTimes` creation `FILETIME`, expected token SID,
session ID, and launch generation; a same-user process racing for a pipe cannot
satisfy that complete identity.
The launcher opens only those pipes, creates the selected shell suspended in
its own dedicated console with `CREATE_NEW_CONSOLE`, `CREATE_SUSPENDED`,
`CREATE_UNICODE_ENVIRONMENT`, and `EXTENDED_STARTUPINFO_PRESENT`, and uses
`STARTF_USESHOWWINDOW`/`SW_HIDE`. It inherits only the explicit standard-I/O
handles through `PROC_THREAD_ATTRIBUTE_HANDLE_LIST`. Non-breakaway Job
membership propagates from the launcher. It reports the suspended shell PID and
creation `FILETIME` and waits for the service's release/watchdog channel.
Invoke exactly the selected shell against the generated wrapper:
`cmd.exe /D /S /C` for a `.cmd` wrapper, or `powershell.exe` with `-NoLogo`,
`-NoProfile`, `-NonInteractive`, and `-File` for a `.ps1` wrapper. Application
paths and argument quoting are constructed by one reviewed Windows routine,
never by concatenating an untrusted command line. There is no fallback.
Persist and flush `launch_prepared` with requested elevation, attempted/effective
context, selection detail, effective token SID, selected session/user SID where
applicable, launcher and shell PID/creation times, and launch generation while
the shell remains suspended.
Persist and flush `launch_authorized` before sending the release token that
makes the launcher call `ResumeThread`. Control-channel EOF before authorization
exits without running user code; EOF afterward terminates the Job. A failure
after authorization is interrupted and never redispatched. The service retains
the sole Job handle, so an unclean service exit terminates launcher, suspended
or running shell, and descendants. Recovery never kills by persisted PID alone.
Job Object limits enforce requested profiles and `KILL_ON_JOB_CLOSE` protects
against lost supervision.
Do not request `CREATE_NEW_PROCESS_GROUP` with `CREATE_NEW_CONSOLE`; Windows
ignores that combination. TERM instead launches a short-lived private mode of
the same `rvbox.exe` under the command's effective token and session. It uses a
separate PID-verified local named-pipe handshake, not inherited cross-session
handles, to receive the target PID/creation-time/generation. It attaches to the
command's dedicated console, installs a handler that consumes its own
CTRL_BREAK, and calls
`GenerateConsoleCtrlEvent(CTRL_BREAK_EVENT, 0)`. Prefer the live root PID; after
root exit, select and birth-verify a live PID from the Job list. Because each
command owns its console, this does not address an unrelated command. The
helper detaches and reports delivery; it receives neither the Job handle nor
command stdio.
Root-process exit begins the same drain grace period. Completion waits for the
Job Object to reach zero active processes; after the grace period RVBox
terminates the Job, drains its capture handles, records any incomplete-output
marker, and emits the terminal lifecycle event last.
Only `TERM`/`SIGTERM` and `KILL`/`SIGKILL` are accepted. `TERM` attempts the
console-helper `CTRL_BREAK_EVENT` and waits 10 seconds, then calls Job Object
termination if the job persists;
`SIGKILL` calls Job Object termination immediately. A console signal is
best-effort, so callers receive an explicit escalation result. Windows status
uses process and Job Object accounting APIs; it does not claim Linux-only
diagnostics such as an I/O wait channel.
Display topology APIs that require the console desktop, DDC/CI monitor
enumeration, and active-user audio policy normally require an active context;
SYSTEM privilege in Session 0 does not substitute for session visibility.
Localhost TCP is machine-wide and normally works in every context, subject to
the listener's own authentication. Fixed local-drive access follows NTFS ACLs;
the client never broadens a requested directory automatically. Every command
record and `stat` view includes requested elevation, attempted/effective
contexts, selection detail, effective token SID, and target session ID/user SID
when present.
## Storage and recovery
SQLite runs in WAL mode. Every append-only segment has a SQLite-owned
`committed_end_offset`. The writer validates and appends records, syncs the file
(grouped by the configured durability interval), and only then commits event
metadata plus the new offset in SQLite. An acknowledgement waits for both
steps. Therefore a crash can leave an uncommitted file tail, but cannot validly
acknowledge metadata whose bytes were not durable.
Startup acquires the instance lock and binds liveness/diagnostic endpoints, then
runs integrity and segment recovery asynchronously. A file longer than its
committed offset is safely truncated to that offset. A file shorter than the
offset, a checksum failure inside the committed range, or corrupt essential
metadata creates a durable scoped storage incident; affected output is marked
truncated/incomplete and affected active commands are interrupted when their
essential state cannot be trusted. Healthy scopes remain usable. Readiness is
false and mutations requiring an unrecovered or dirty scope return `UNAVAILABLE`,
but process startup, liveness, incident inspection, and unaffected work do not
wait for a full-store scan.
Safe repairs are attempted automatically and can also be requested online with
`rvc storage repair`. Irrecoverable loss stays dirty until explicitly accepted
with `rvc storage acknowledge`; an offline server has equivalent
`rvbox-server repair --data-dir ...` repair/list/acknowledge operations. Client
spool recovery follows the same committed-offset rule and exposes equivalent
offline `rvbox repair --state-dir ...` operations and local health diagnostics.
Resolving an incident clears derived dirty health but never erases the incident
or audit history as part of resolution. Unresolved compact incident records are
non-evictable; resolved incident/audit history follows the separate 100 MiB
rotation. Segment compression is Zstandard;
limits measure stored compressed bytes, while raw byte counts are reported
separately.
The system must reserve headroom before writes and use transactional metadata
updates. Storage-full, permission, and corruption failures are surfaced as
structured server/client health states and audit events. They must isolate the
affected command/session, reject work when needed, and keep the daemon's
heartbeat/control loops alive.
V1 state is plaintext at rest, including command text, scripts, environment
override values, stdin, and output. Private directory/file modes and dedicated
daemon accounts are deployment hygiene, not an application-level encryption
guarantee. Backups copy the same plaintext sensitivity. Encryption and external
key management are future-version work.
## Metrics, logging, and safe defaults
Both daemons should emit structured logs and metrics for session transitions,
heartbeat timeout, reconnect backoff, command state transitions, queue depth,
spool bytes, segment rotation/eviction, output loss markers, storage errors,
and protocol violations. Never emit stdin or raw output in normal daemon logs.
Recommended configuration defaults are: 10-second heartbeat idle period,
30-second liveness timeout, 1–60-second full-jitter reconnect backoff,
60-second stable-session reset, 5-minute one-shot live-conflict takeover grant,
16 running/100 queued commands per client, 1,000 server-queued commands per
target client and 10,000 server-wide,
15-minute queue TTL, 10 MiB per-command output window, 32 MiB total per command,
256 MiB per client/client daemon, 4 GiB server-wide command storage, 30-day
terminal retention, 100 MiB audit storage with quota-only rotation by default,
one million compact command tombstones, 64 KiB uncompressed stream chunk, 1 MiB
decoded agent envelope,
1 MiB/256 KiB per-command raw-output high/low watermarks, 8 MiB/4 MiB per-client
watermarks, 64 MiB/32 MiB server-wide watermarks, 1 MiB per-command and 8 MiB
per-session unacknowledged send windows, and 10 MiB raw script maximum. Control
gRPC accepts at most 16 MiB decoded requests; the JSON-RPC adapter accepts at
most 24 MiB HTTP bodies to allow protobuf JSON's base64 expansion while
retaining the same decoded field limits. Each active command reserves 64 KiB
within its quota for terminal/loss closeout metadata; protocol detail/reason and
incident-note text fields are individually limited to 4 KiB. Default emergency
filesystem free-space floors are 256 MiB on the server and 64 MiB on a client;
crossing one rejects new unreserved allocations even if the logical quota has
headroom. Already-reserved terminal/loss closeout remains writable while bytes
physically remain.
The Windows service installs with Automatic startup. Its bounded file logging
defaults to a 10 MiB current file and five retained sealed files under the
resolved ProgramData log directory; tray lifecycle does not affect log output.
These bounds protect RVBox's own loops; they cannot make arbitrary child
commands harmless when no resource profile is requested. Operators should
enable resource profiles for untrusted or expensive workloads and keep nginx,
Unix-socket permissions, filesystem capacity, and service supervision correctly
configured.