docs: add initial RVBox design and protocol contracts
This commit is contained in:
@@ -0,0 +1,18 @@
|
||||
# RVBox v1 design set
|
||||
|
||||
This design set turns the initial proposal into an implementation contract.
|
||||
Read the documents in this order:
|
||||
|
||||
1. [Architecture](architecture.md) — scope, trust model, state machine,
|
||||
execution, storage, and retention decisions.
|
||||
2. [Agent protocol](protocol.md) — WSS framing, sessions, replay, ordering,
|
||||
script transfer, heartbeat, and failure containment.
|
||||
3. [Control plane](control-plane.md) — `rvc`, Unix-socket gRPC, JSON-RPC, and
|
||||
query/foreground semantics.
|
||||
4. [Platform and operations](platform-and-operations.md) — Unix/Windows
|
||||
contracts, recovery, storage safety, telemetry, and defaults.
|
||||
|
||||
The wire authority is in [`../protos/rvbox/v1`](../protos/rvbox/v1):
|
||||
`common.proto` contains shared data types, `agent.proto` contains the
|
||||
client/server WebSocket envelopes, and `control.proto` defines the local gRPC
|
||||
service that the HTTP JSON-RPC adapter mirrors.
|
||||
@@ -0,0 +1,183 @@
|
||||
# RVBox v1 architecture
|
||||
|
||||
## Purpose and scope
|
||||
|
||||
RVBox is a reverse-connection remote command system. A client daemon (`rvbox`)
|
||||
maintains a WebSocket connection to `rvbox-server`; the server persists command
|
||||
state and exposes a local control plane to `rvc` and, optionally, JSON-RPC
|
||||
callers. This design is the v1 contract for Go implementations and a later Rust
|
||||
client implementation.
|
||||
|
||||
RVBox deliberately executes arbitrary commands with the identity, permissions,
|
||||
and base environment of the client daemon. It is therefore an administrative
|
||||
tool, not a multi-tenant remote-execution service.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Mutual TLS, client certificates, enrollment tokens, and a client-ID allowlist
|
||||
are not part of v1. TLS is terminated by the deployment's nginx instance.
|
||||
- RVBox does not promise a definitive cross-platform “hung” determination.
|
||||
- V1 does not provide arbitrary file transfer; `--script` transfers only the
|
||||
temporary script required to execute that request.
|
||||
- Resource limits are supported only when explicitly requested by an execution
|
||||
profile; they are not imposed by default.
|
||||
|
||||
## Trust boundary
|
||||
|
||||
The server accepts a self-reported hostname as `client_id`; it is an opaque
|
||||
1–128 ASCII-character routing/display key. Unknown IDs are accepted. A newer
|
||||
registration for an ID replaces its prior live session. Consequently, a peer
|
||||
able to reach nginx can impersonate or take over a client ID. This is an
|
||||
accepted v1 limitation and deployments must restrict the endpoint to a trusted
|
||||
network.
|
||||
|
||||
The Unix control socket is local-only and mode `0600`, owned by the server
|
||||
account. The optional HTTP JSON-RPC endpoint is intentionally unauthenticated;
|
||||
it defaults to loopback but can be bound elsewhere by configuration. Exposing
|
||||
it to a network exposes full remote-command authority and is unsafe without an
|
||||
external access-control layer.
|
||||
|
||||
## Components
|
||||
|
||||
```text
|
||||
rvc -- gRPC/Unix socket -- rvbox-server -- WSS/nginx -- rvbox client -- shell
|
||||
| |
|
||||
SQLite/WAL +-- optional HTTP JSON-RPC (debug/batch)
|
||||
|
|
||||
compressed output segment files + audit log
|
||||
```
|
||||
|
||||
`rvc` is a thin control-plane client. The server owns durable command history,
|
||||
dispatch, queueing, pagination, audit records, and session fencing. A client
|
||||
owns active process supervision, unacknowledged output spooling, and safe
|
||||
reconnection. Neither daemon lets a slow peer, command, or output stream block
|
||||
its dispatch loops.
|
||||
|
||||
## Identity, sessions, and lifecycle
|
||||
|
||||
1. The client connects over WSS and sends `ClientHello` with its client ID,
|
||||
protocol capability, OS/architecture, daemon version, current daemon CWD,
|
||||
supported shells, and a fresh reconnect UUID.
|
||||
2. The server accepts the current compatible protocol version, fences the
|
||||
previous connection for that ID, and returns a fresh server-issued
|
||||
`session_id` and monotonic `session_generation`.
|
||||
3. Every client-to-server envelope and server dispatch is bound to that token.
|
||||
The server discards traffic from superseded sessions, including late output.
|
||||
4. The server queues work while a client is offline and dispatches it only when
|
||||
the active session advertises capacity. A replacement session immediately
|
||||
resumes non-terminal reconciliation.
|
||||
|
||||
## Command model
|
||||
|
||||
Each user request has a server-generated UUID (`issue_uuid`) and a durable
|
||||
request record: target client, request/issue timestamps, shell type, command
|
||||
text or script descriptor, CWD, environment overrides, resource-profile flags,
|
||||
and lifecycle state. The UUID is the end-to-end idempotency key. Transport is
|
||||
at-least-once, but the client durably remembers accepted UUIDs and never starts
|
||||
the same request twice.
|
||||
|
||||
The server states are:
|
||||
|
||||
```text
|
||||
queued -> dispatched -> accepted -> running -> succeeded | failed | terminated
|
||||
\-------------------------------> cancelled
|
||||
```
|
||||
|
||||
`accepted` means the client has durably accepted the request; `running` means
|
||||
the process has been launched. `cancelled` is used when it is stopped before
|
||||
launch. A kill racing launch is resolved by command revision: the client either
|
||||
acknowledges cancellation before launch or launches then immediately applies
|
||||
the requested signal, recording the race.
|
||||
|
||||
The client permits 16 concurrent processes and 100 pending commands by default.
|
||||
Those values are configurable and advertised to the server. A full queue causes
|
||||
a structured capacity error rather than creating unbounded work.
|
||||
|
||||
### Execution contract
|
||||
|
||||
- Shell selection is explicit: Unix-like clients support `sh` and `bash`;
|
||||
Windows supports `cmd` and `powershell`. Defaults are `sh` and `powershell`.
|
||||
Unsupported shells are rejected; no fallback occurs.
|
||||
- A command receives the daemon account's permissions and startup environment,
|
||||
overlaid with the persisted `env_overrides` map. The effective CWD is the
|
||||
requested existing directory or the registered daemon CWD when omitted.
|
||||
- Each command is isolated into a process tree: a Unix session/process group or
|
||||
a Windows Job Object. A daemon that cannot supervise its children terminates
|
||||
them and reports interruption rather than claiming recovery it cannot make.
|
||||
- Resource profiles are composable flags (`LIGHT`, `CPU_MEDIUM`, `CPU_HEAVY`,
|
||||
`MEM_MEDIUM`, `MEM_HEAVY`, `DISK_MEDIUM`, `DISK_HEAVY`). Profiles are opt-in;
|
||||
the concrete administrator-configured limits are applied with cgroup v2 when
|
||||
available on Linux and Job Object limits on Windows. Unsupported requested
|
||||
controls are reported, never silently ignored.
|
||||
|
||||
### Script execution
|
||||
|
||||
Scripts are content-addressed uploads, not shell-escaped command strings. The
|
||||
server sends a descriptor containing SHA-256 and then ordered chunks (default
|
||||
maximum: 10 MiB). The client verifies the digest, writes an owner-only temporary
|
||||
file beneath the effective CWD, executes it with the selected shell, and removes
|
||||
it after the command reaches a terminal state. Script content is not copied into
|
||||
the audit log; its digest and metadata are.
|
||||
|
||||
## Process control and diagnostics
|
||||
|
||||
On Unix, `kill` addresses the command's process group and accepts normal signal
|
||||
names/numbers supported by that client. On Windows, only `SIGTERM` and `SIGKILL`
|
||||
are valid. `SIGTERM` makes a best-effort `CTRL_BREAK_EVENT` delivery to the
|
||||
dedicated console group, waits 10 seconds, then terminates the Job Object if
|
||||
needed. `SIGKILL` immediately terminates the Job Object. The response reports
|
||||
the actual escalation outcome.
|
||||
|
||||
Lifecycle state never asserts `hung`. A separate `suspected_hung` diagnostic is
|
||||
emitted after the configurable default of 10 minutes without observable
|
||||
progress. Linux enriches this with `/proc` state, CPU, memory, and I/O data;
|
||||
Windows uses process and Job Object APIs where available. It is explicitly a
|
||||
heuristic, not proof of an I/O stall.
|
||||
|
||||
## Durability, ordering, and retention
|
||||
|
||||
The server uses SQLite in WAL mode for metadata/indexes and append-only Zstandard
|
||||
compressed segment files for output. It recovers non-terminal commands after a
|
||||
restart and reconciles only these; server-confirmed historical terminal commands
|
||||
are not re-reconciled. The client stores only active command state and output
|
||||
not acknowledged by the server.
|
||||
|
||||
Every execution-originated event has a strictly increasing `event_seq` scoped to
|
||||
one command. This includes lifecycle transitions, stdout/stderr chunks, stdin
|
||||
acknowledgments, resource snapshots, signals, and terminal events. The server
|
||||
preserves the sequence and additionally records receipt time. This is the
|
||||
canonical reconstruction order across interleaved streams and retries.
|
||||
Truncation is separate range metadata so it can truthfully describe missing
|
||||
event sequences without consuming one itself.
|
||||
|
||||
Output chunks are Zstandard-compressed before persistent quota accounting.
|
||||
Per-command history is a rolling compressed window (10 MiB default), so the
|
||||
oldest output segments for that command are removed first and a sequence-range
|
||||
truncation marker remains. This applies to active and terminal commands. When
|
||||
connected, the client first removes server-acknowledged segments. While offline,
|
||||
it must still honor both hard caps: it retains the newest tail, removes oldest
|
||||
unacknowledged compressed chunks when necessary, and records their exact missing
|
||||
ranges for durable reporting on reconnect.
|
||||
|
||||
Each client also has a 50 MiB aggregate compressed spool cap for active,
|
||||
unacknowledged work. The server's matching per-client compressed-history cap is
|
||||
50 MiB; it evicts that client's oldest terminal command records as needed. The
|
||||
server-wide cap is 1 GiB; it evicts whole oldest terminal command records
|
||||
(metadata and output), never arbitrary stdout/stderr rows. Active commands are
|
||||
protected. If active commands alone consume a per-client budget, their oldest
|
||||
acknowledged output rotates by the per-command rule; pipes continue draining so
|
||||
a child cannot deadlock on output.
|
||||
|
||||
## Audit and timestamps
|
||||
|
||||
The server writes a separate durable audit trail for control actions and session
|
||||
events: source/transport identity where available, target client, UUID, action,
|
||||
request time, result, and error. It records command text, environment-override
|
||||
names (not values), and script metadata/digest, but not duplicated
|
||||
stdin/stdout/stderr payloads. The persisted execution request necessarily keeps
|
||||
override values for dispatch/retry and must be access-controlled as sensitive
|
||||
data. Audit retention is configured independently of output retention.
|
||||
|
||||
All protocol timestamps are UTC `google.protobuf.Timestamp` values. Client
|
||||
observed timestamps and server receipt timestamps are distinct; the latter is
|
||||
authoritative for server records, while `event_seq` is authoritative for order.
|
||||
@@ -0,0 +1,58 @@
|
||||
# RVBox v1 control plane
|
||||
|
||||
[`../protos/rvbox/v1/control.proto`](../protos/rvbox/v1/control.proto) defines
|
||||
the canonical control API. `rvc` uses the `Control` gRPC service over the local
|
||||
Unix-domain socket. The server also exposes the same unary operations through
|
||||
an optional JSON-RPC 2.0 HTTP adapter for local debugging and batch automation.
|
||||
|
||||
## Endpoints
|
||||
|
||||
- Unix socket: enabled by default, mode `0600`, owned by the server account.
|
||||
- HTTP JSON-RPC: disabled unless enabled; default bind `127.0.0.1:6900`.
|
||||
It has no authentication by design. Binding it beyond loopback is an explicit
|
||||
deployment choice and requires external protection.
|
||||
|
||||
gRPC can stream `RunCommandAndFollow` and `FollowCommand`. JSON-RPC remains
|
||||
simple: callers issue work, query command state, poll event/output pages after
|
||||
an event sequence, append stdin, close stdin, or signal a command. It does not
|
||||
invent a separate event-stream protocol.
|
||||
|
||||
The JSON-RPC method names are the lower-camel protobuf operation names:
|
||||
`listClients`, `getClient`, `listCommands`, `getCommand`, `runCommand`,
|
||||
`appendStdin`, `closeStdin`, `signalCommand`, and `getOutput`. Parameters and
|
||||
results use protobuf JSON mapping (including base64 strings for `bytes` and UTC
|
||||
RFC 3339 strings for timestamps); JSON-RPC errors carry the corresponding
|
||||
`ControlError` code/data. `getOutput` and `getCommand` are the polling path for
|
||||
what gRPC exposes as follow streams.
|
||||
|
||||
## CLI semantics
|
||||
|
||||
`rvc stat` maps to `ListClients`, `GetClient`, `ListCommands`, and `GetCommand`.
|
||||
History pages default to 20 commands and may request at most 100. Output pages
|
||||
default to 100 lines; a line is a display operation over ordered chunks, not a
|
||||
protocol boundary. Output can be filtered by stream and timestamped with the
|
||||
server's recorded client-observed timestamp plus stream name.
|
||||
|
||||
`rvc run` creates a command. Foreground mode runs `RunCommandAndFollow`, which
|
||||
streams output and stops on a terminal event. `--background` uses `RunCommand`
|
||||
and returns the UUID immediately. Interrupting the CLI, timing out its local
|
||||
wait, or losing the local control connection never cancels remote work. The
|
||||
explicit `rvc kill` operation is the only termination path.
|
||||
|
||||
`rvc append` turns a string into `StdinWrite` with `append_newline=true` unless
|
||||
the caller selects raw mode; `--file` supplies raw bytes; `--attach` streams
|
||||
local standard input. `CloseStdin` is available separately. All stdin actions
|
||||
are acknowledged and idempotent.
|
||||
|
||||
## Query behavior
|
||||
|
||||
Command details expose persisted request fields, effective execution metadata,
|
||||
lifecycle, terminal result, output availability/truncation, current resource
|
||||
snapshot, and `suspected_hung` diagnostics. The latter is informational and
|
||||
must not be shown as a terminal state. Pagination response cursors are stable
|
||||
within their declared ordering (newest issue time for command lists; increasing
|
||||
`event_seq` for events/output).
|
||||
|
||||
The service returns `ControlError` codes for not found, offline, capacity,
|
||||
invalid request, unsupported platform feature, conflict, truncation, and
|
||||
internal/transient errors. It never encodes errors only as CLI text.
|
||||
@@ -0,0 +1,64 @@
|
||||
# RVBox v1 platform and operations contract
|
||||
|
||||
## Unix-like clients
|
||||
|
||||
The client starts `sh` or `bash` in a new session/process group. Unix signals
|
||||
address that group, so normally created descendants receive the signal too. On
|
||||
orderly shutdown, or recovery after an unclean daemon failure, managed command
|
||||
groups are terminated and marked interrupted because pipe capture cannot be
|
||||
safely resumed.
|
||||
|
||||
Linux diagnostics sample `/proc/<pid>` and relevant children for state, CPU,
|
||||
resident memory, I/O counters, CWD, and wait-channel information when readable.
|
||||
These values may be unavailable due to permissions, kernel configuration, or a
|
||||
short-lived process; absence is represented explicitly rather than fabricated.
|
||||
Cgroup v2 is used for requested resource profiles only when available.
|
||||
|
||||
## Windows clients
|
||||
|
||||
The client launches `cmd` or `powershell` in an appropriate dedicated console
|
||||
process group and assigns the root process to a per-command Job Object. Child
|
||||
processes normally join the Job Object. Job Object limits enforce requested
|
||||
profiles and `KILL_ON_JOB_CLOSE` protects against lost supervision.
|
||||
|
||||
Only `SIGTERM` and `SIGKILL` are accepted. `SIGTERM` attempts `CTRL_BREAK_EVENT`
|
||||
and waits 10 seconds, then calls Job Object termination if the job persists;
|
||||
`SIGKILL` calls Job Object termination immediately. A console signal is
|
||||
best-effort, so callers receive an explicit escalation result. Windows status
|
||||
uses process and Job Object accounting APIs; it does not claim Linux-only
|
||||
diagnostics such as an I/O wait channel.
|
||||
|
||||
## Storage and recovery
|
||||
|
||||
SQLite runs in WAL mode with integrity checking on startup. Output segments are
|
||||
written atomically, fsynced according to the configured durability interval, and
|
||||
indexed only after successful durable append. Startup scans/repairs incomplete
|
||||
tail records before accepting control requests. Segment compression is Zstandard;
|
||||
limits always measure stored compressed bytes, while clients expose raw byte
|
||||
counts separately.
|
||||
|
||||
The system must reserve headroom before writes and use transactional metadata
|
||||
updates. Storage-full, permission, and corruption failures are surfaced as
|
||||
structured server/client health states and audit events. They must isolate the
|
||||
affected command/session, reject work when needed, and keep the daemon's
|
||||
heartbeat/control loops alive.
|
||||
|
||||
## Metrics, logging, and safe defaults
|
||||
|
||||
Both daemons should emit structured logs and metrics for session transitions,
|
||||
heartbeat timeout, reconnect backoff, command state transitions, queue depth,
|
||||
spool bytes, segment rotation/eviction, output loss markers, storage errors,
|
||||
and protocol violations. Never emit stdin or raw output in normal daemon logs.
|
||||
|
||||
Recommended configuration defaults are: 10-second heartbeat idle period,
|
||||
30-second liveness timeout, 1–60-second full-jitter reconnect backoff,
|
||||
60-second stable-session reset, 16 running/100 queued commands per client,
|
||||
10 MiB per-command compressed window, 50 MiB per-client active spool and server
|
||||
history, 1 GiB server history, 64 KiB uncompressed stream chunk, 1 MiB decoded
|
||||
envelope, and 10 MiB script maximum.
|
||||
|
||||
These bounds protect RVBox's own loops; they cannot make arbitrary child
|
||||
commands harmless when no resource profile is requested. Operators should
|
||||
enable resource profiles for untrusted or expensive workloads and keep nginx,
|
||||
Unix-socket permissions, filesystem capacity, and service supervision correctly
|
||||
configured.
|
||||
@@ -0,0 +1,99 @@
|
||||
# RVBox v1 agent protocol
|
||||
|
||||
The authoritative schemas are [`../protos/rvbox/v1/common.proto`](../protos/rvbox/v1/common.proto)
|
||||
and [`../protos/rvbox/v1/agent.proto`](../protos/rvbox/v1/agent.proto). This
|
||||
document specifies their use over WSS.
|
||||
|
||||
## Transport and compatibility
|
||||
|
||||
The nginx-terminated `wss://` connection carries exactly one serialized
|
||||
`rvbox.v1.AgentEnvelope` in each binary WebSocket message. There is no extra
|
||||
length prefix. A decoded envelope may not exceed 1 MiB. A chunk's uncompressed
|
||||
payload may not exceed 64 KiB. Both peers validate declared and actual expanded
|
||||
sizes before allocation/decompression.
|
||||
|
||||
The protobuf package is `rvbox.v1`. Registration negotiates a major/minor
|
||||
protocol range: incompatible majors are rejected; the highest shared minor is
|
||||
chosen. New fields are append-only. A peer ignores unknown optional fields but
|
||||
must respond with `PROTOCOL_ERROR` to an unknown required envelope feature.
|
||||
|
||||
## Heartbeat and reconnect
|
||||
|
||||
Both endpoints use the same algorithm. Any received valid WebSocket frame is
|
||||
inbound activity. After 10 seconds without inbound activity, send a WebSocket
|
||||
Ping. After 30 seconds without inbound activity, close the session and treat it
|
||||
as dead. Pong processing is normal WebSocket behavior; it is not an application
|
||||
message and never queues behind command traffic.
|
||||
|
||||
The client reconnects with full-jitter exponential backoff (initial 1 second,
|
||||
cap 60 seconds). A session stable for 60 seconds resets the backoff. The server
|
||||
does not reconnect; it waits for clients. A reconnect always registers again,
|
||||
receives a new fencing generation, resends unacknowledged delivery/output, and
|
||||
reconciles only commands the server still considers non-terminal.
|
||||
|
||||
## Session fencing
|
||||
|
||||
`ClientHello` starts registration. `ServerWelcome` gives the selected version,
|
||||
random `session_id`, and `session_generation`. Except `ClientHello`, all
|
||||
envelopes carry those values. A new accepted registration fences and disconnects
|
||||
the prior one for the same client ID. The server accepts messages only from the
|
||||
current generation; command dispatches also identify their intended generation.
|
||||
|
||||
## Reliable work flows
|
||||
|
||||
### Dispatch and reconciliation
|
||||
|
||||
The server persistently creates a command before dispatching `CommandDispatch`.
|
||||
It redelivers until it receives `CommandAccepted`. Clients durably deduplicate
|
||||
on `issue_uuid`. A client whose queue is full sends a capacity rejection.
|
||||
|
||||
Following registration the server sends `ReconcileRequest` only for its
|
||||
non-terminal commands for that client. The client replies with a
|
||||
`ReconcileSnapshot` per requested known command, or `unknown_to_client`. It
|
||||
continues normal event retransmission from the server's last acknowledged event
|
||||
sequence. Terminal history already confirmed by the server is deliberately
|
||||
excluded.
|
||||
|
||||
### Output and events
|
||||
|
||||
Client execution events use an increasing `event_seq`; retries reuse the same
|
||||
sequence and content. The server durably writes an event before sending
|
||||
`EventAck`. `EventAck` is cumulative through a sequence number. Output is
|
||||
Zstandard compressed with an explicit original-size field. Server storage can
|
||||
reuse the validated compressed bytes.
|
||||
|
||||
When rolling output removes old retained segments, the server writes
|
||||
`OutputTruncation` metadata containing the removed event/byte ranges. Queries
|
||||
must show that marker rather than silently presenting an apparently complete
|
||||
stream. An offline client that reaches a cap sends `ClientOutputTruncated`
|
||||
before replaying its retained tail on reconnect. The server records it, permits
|
||||
the named event-sequence gap, and exposes it in output queries. Truncation
|
||||
metadata is not an execution event and does not consume an `event_seq`.
|
||||
|
||||
### Stdin and signals
|
||||
|
||||
`StdinWrite` is binary-safe and ordered by `write_seq`; the client durably
|
||||
deduplicates it and returns `StdinAck`. `append_newline` is true by default in
|
||||
the CLI but explicit on the wire. `CloseStdin` is a separate idempotent action.
|
||||
Signal and cancellation requests carry a command revision to settle start/kill
|
||||
races.
|
||||
|
||||
### Scripts
|
||||
|
||||
For a script command, dispatch first contains a `ScriptDescriptor`; then the
|
||||
server sends `ScriptChunk` messages and a commit. The client checks offset,
|
||||
chunk order, full length, and SHA-256 before it reports upload complete or
|
||||
launches the process. A retransmitted chunk is idempotent by offset/content.
|
||||
|
||||
## Flow control and failure containment
|
||||
|
||||
No receive loop runs an executor, database write, decompressor, or slow socket
|
||||
operation inline. Each side has bounded staging queues. Durable spools are the
|
||||
source of truth and are charged to the 10 MiB/50 MiB/1 GiB compressed retention
|
||||
budgets described in the architecture document. A full staging queue pauses the
|
||||
related read/dispatch path and drains from disk; it never grows without bound.
|
||||
|
||||
Malformed protobuf, over-size payload, invalid compressed data, impossible
|
||||
sequence, bad session token, or protocol-version violation yields a structured
|
||||
error where safe and closes that WebSocket session. It does not crash either
|
||||
daemon. Network loss is normal and is handled through idempotent resend.
|
||||
Reference in New Issue
Block a user