feat(transport): negotiate bounded automatic message chunking
This commit is contained in:
1 parent
474afb3484
commit
8acffac985
20 files changed
+1332
-38
No files matched your search
@@ -1,5 +1,13 @@
|
|||||||
# Pending issues
|
# Pending issues
|
||||||
|
|
||||||
|
## Auto-chunking rollout is explicitly held
|
||||||
|
|
||||||
|
The user requested on 2026-11-09 that auto-chunking **not be brought online**
|
||||||
|
while the existing RVBox deployment is in use. Implement and verify it only in
|
||||||
|
the repository and isolated fixtures. Do not restart/upgrade live services or
|
||||||
|
replace the published installer for this feature without a later explicit
|
||||||
|
instruction lifting that hold. See [the design](docs/transport-chunking.md).
|
||||||
|
|
||||||
## RVBox Windows commands do not reliably show GUI prompts
|
## RVBox Windows commands do not reliably show GUI prompts
|
||||||
|
|
||||||
When a GUI uninstaller is launched through `rvc run --shell cmd`, its dialog may
|
When a GUI uninstaller is launched through `rvc run --shell cmd`, its dialog may
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ All versions are exact in `go.mod`, generated code, or the toolchain image.
|
|||||||
|
|
||||||
| Dependency | Purpose | Decision |
|
| Dependency | Purpose | Decision |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| coder/websocket 1.8.15 | Context-aware WebSocket server/client transport | Maintained Go WebSocket implementation with bounded message reads and explicit Ping support; RVBox keeps protobuf validation, session fencing, origin policy, and backpressure in its own code. |
|
| coder/websocket 1.8.15 | Context-aware WebSocket server/client transport | Maintained Go WebSocket implementation with bounded message reads, subprotocol negotiation, and explicit Ping support; RVBox keeps protobuf validation, session fencing, origin policy, backpressure, and negotiated `internal/agentwire` chunking in its own code. |
|
||||||
| Buf 1.72.0 | Protobuf formatting, linting, and deterministic generation | Current stable release when the v1 implementation began; installed only in the toolchain image. |
|
| Buf 1.72.0 | Protobuf formatting, linting, and deterministic generation | Current stable release when the v1 implementation began; installed only in the toolchain image. |
|
||||||
| protoc 35.0 | Protobuf compiler and well-known includes | Pinned archive with a checked SHA-256; included even though normal generation is driven by Buf. |
|
| protoc 35.0 | Protobuf compiler and well-known includes | Pinned archive with a checked SHA-256; included even though normal generation is driven by Buf. |
|
||||||
| UPX 5.2.1 | Optional compact Windows release packaging | Official AMD64/ARM64 Linux archives pinned by SHA-256 in the Docker toolchain. Opt-in LZMA packing preserves resources/relocations, retains an unpacked fallback, and runs before signing. Native runtime acceptance is still required. |
|
| UPX 5.2.1 | Optional compact Windows release packaging | Official AMD64/ARM64 Linux archives pinned by SHA-256 in the Docker toolchain. Opt-in LZMA packing preserves resources/relocations, retains an unpacked fallback, and runs before signing. Native runtime acceptance is still required. |
|
||||||
|
|||||||
@@ -1659,8 +1659,13 @@ terminal-age, audit, and tombstone rules match the documented semantics.
|
|||||||
|
|
||||||
Expose an HTTP endpoint suitable for nginx WebSocket proxying. TLS may terminate
|
Expose an HTTP endpoint suitable for nginx WebSocket proxying. TLS may terminate
|
||||||
at nginx, but the server must verify WebSocket upgrade origin/path/configuration
|
at nginx, but the server must verify WebSocket upgrade origin/path/configuration
|
||||||
and impose read limits itself. It accepts one binary protobuf envelope per
|
and impose read limits itself. An AgentEnvelope is one logical message. Without
|
||||||
message and refuses text messages, fragmentation misuse, oversized frames,
|
transport negotiation it is one binary WebSocket message; when both peers select
|
||||||
|
`rvbox.agent.v1.chunked`, the shared `internal/agentwire` layer splits it into
|
||||||
|
16 KiB-or-smaller packets and reassembles it before protobuf validation. See
|
||||||
|
[the transport specification](transport-chunking.md) for the exact record format,
|
||||||
|
bounded independent control/data assemblies, reconnect rules, and rollout hold.
|
||||||
|
It refuses text messages, fragmentation misuse, oversized frames,
|
||||||
invalid compressed payloads, and malformed/unknown mandatory messages.
|
invalid compressed payloads, and malformed/unknown mandatory messages.
|
||||||
|
|
||||||
Use four independently bounded activities per session:
|
Use four independently bounded activities per session:
|
||||||
@@ -1690,6 +1695,20 @@ and split data below the frame ceiling. A full control lane is a session-fatal
|
|||||||
internal invariant; a full data lane leaves work durable and wakes the writer
|
internal invariant; a full data lane leaves work durable and wakes the writer
|
||||||
later rather than allocating another goroutine.
|
later rather than allocating another goroutine.
|
||||||
|
|
||||||
|
For negotiated chunking, apply the write deadline per packet and check the
|
||||||
|
monotonic heartbeat between packets, including while output stays continuously
|
||||||
|
queued. Between data packets, drain at most the configured control burst. Large
|
||||||
|
control messages use their independent assembly lane; do not interleave two
|
||||||
|
fragmented messages within one lane. `Written`/`OnWritten` fire only after the
|
||||||
|
whole logical message has crossed the writer, preserving dispatch and
|
||||||
|
reconciliation barriers. One bounded network read pump services Ping/Pong while
|
||||||
|
the application writes a large reconciliation snapshot; validated packet
|
||||||
|
activity advances liveness before full reassembly. Close cancels this pump,
|
||||||
|
including when its single delivery slot is full. A disconnected assembly is
|
||||||
|
never persisted or acknowledged: the existing durable sender replays the logical
|
||||||
|
operation after reconnect. Logical envelope/spec/chunk/script limits remain in
|
||||||
|
force; transport fragmentation cannot bypass those limits or quota accounting.
|
||||||
|
|
||||||
The reader checks WebSocket type/size before protobuf decode, validates envelope
|
The reader checks WebSocket type/size before protobuf decode, validates envelope
|
||||||
session fields before payload allocation, and routes only lightweight immutable
|
session fields before payload allocation, and routes only lightweight immutable
|
||||||
work descriptors. Persistence workers preserve per-command ordering while a fair
|
work descriptors. Persistence workers preserve per-command ordering while a fair
|
||||||
|
|||||||
@@ -126,6 +126,11 @@ exact ignored `dist/` directory after preserving any evidence.
|
|||||||
|
|
||||||
## Common incidents
|
## Common incidents
|
||||||
|
|
||||||
|
- **Auto-chunking rollout hold:** the negotiated transport implementation is
|
||||||
|
intentionally offline at the user's request. Do not upgrade the running
|
||||||
|
deployment or replace published installers to enable it yet. See the
|
||||||
|
[wire format and isolated checks](transport-chunking.md). It requires both
|
||||||
|
endpoints to negotiate support; it does not change logical admission limits.
|
||||||
- **Reconnect loop after a command produces substantial output:** inspect
|
- **Reconnect loop after a command produces substantial output:** inspect
|
||||||
`C:\ProgramData\RVBox\service-startup.log` on Windows and the server log for
|
`C:\ProgramData\RVBox\service-startup.log` on Windows and the server log for
|
||||||
`StatusMessageTooBig` / `read limited at 32769 bytes`. Early v1 transports
|
`StatusMessageTooBig` / `read limited at 32769 bytes`. Early v1 transports
|
||||||
|
|||||||
@@ -15,6 +15,13 @@ Run focused unit tests with:
|
|||||||
scripts/test-unit --package ./internal/domain --run UUIDv7 --race
|
scripts/test-unit --package ./internal/domain --run UUIDv7 --race
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Automatic Agent transport chunking is **offline only** under the recorded
|
||||||
|
user rollout hold. Its unit, hostile-input, recovery, concurrency, and local TLS
|
||||||
|
integration fixtures need no VM or live deployment. See the
|
||||||
|
[wire design and focused test command](transport-chunking.md). Do not run a
|
||||||
|
native install/upgrade or publish an installer to enable it yet. The ordinary
|
||||||
|
verification gate only tests/builds it locally; it does not deploy it.
|
||||||
|
|
||||||
Run one bounded hostile-input fuzz target in the same pinned container with:
|
Run one bounded hostile-input fuzz target in the same pinned container with:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
|
|||||||
@@ -0,0 +1,111 @@
|
|||||||
|
# Automatic Agent transport chunking
|
||||||
|
|
||||||
|
## Status and scope
|
||||||
|
|
||||||
|
**Offline only.** The user explicitly held deployment on 2026-11-09. Development,
|
||||||
|
isolated tests, and ordinary source commits are authorized; live service updates
|
||||||
|
and replacement of published installers for this feature are not.
|
||||||
|
|
||||||
|
The shared `internal/agentwire` package splits a logical protobuf message into
|
||||||
|
bounded packets and assembles it before the existing Agent handler sees it.
|
||||||
|
Scripts, output, large inline commands, stdin, and reconciliation all use the
|
||||||
|
same transport path. Business-level script/output chunks remain unchanged.
|
||||||
|
|
||||||
|
This handles messages larger than a wire packet, not requests exceeding the
|
||||||
|
configured logical limits. The normal 1 MiB envelope, 768 KiB execution spec,
|
||||||
|
64 KiB raw chunk and 10 MiB script limits, existing quotas, and Control API
|
||||||
|
admission rules still apply. A request beyond its logical limit is rejected;
|
||||||
|
fragmentation does not raise limits or make unbounded snapshots valid.
|
||||||
|
|
||||||
|
## Negotiation and compatibility
|
||||||
|
|
||||||
|
Both WebSocket endpoints advertise `rvbox.agent.v1.chunked` using the standard
|
||||||
|
`Sec-WebSocket-Protocol` handshake. Framing is enabled only if that exact
|
||||||
|
subprotocol is selected. Otherwise the existing one-binary-message-per-envelope
|
||||||
|
format remains in use. No client-version guessing, protobuf capability fields,
|
||||||
|
database migration, or operator configuration knob is required.
|
||||||
|
|
||||||
|
New client/old server and old client/new server retain the raw format. Old
|
||||||
|
clients do not gain a reassembler from a server upgrade; their existing 32 KiB
|
||||||
|
reader can still reject large inline commands, stdin, or reconciliation. The
|
||||||
|
server's existing 16 KiB script chunks continue to support pending uploads to
|
||||||
|
those clients. Proxy configurations must preserve the negotiated subprotocol
|
||||||
|
header in both directions.
|
||||||
|
|
||||||
|
## Packet format
|
||||||
|
|
||||||
|
Each negotiated packet is one binary WebSocket message of at most 16,384 bytes.
|
||||||
|
The WebSocket library enforces that aggregate message bound even when a peer
|
||||||
|
fragments it into native WebSocket continuation frames. Compression remains
|
||||||
|
disabled at the WebSocket layer.
|
||||||
|
|
||||||
|
| First byte | Remaining bytes | Meaning |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `0` | Serialized logical message | Complete message; payload is 1–16,383 bytes. |
|
||||||
|
| `1` | uint32 total length, uint32 offset, data | Fragment of a data-lane message. |
|
||||||
|
| `2` | uint32 total length, uint32 offset, data | Fragment of a control-lane message. |
|
||||||
|
|
||||||
|
Integers use big-endian order. Fragment headers occupy nine bytes including the
|
||||||
|
kind; data is nonempty and at most 16,375 bytes. Fragmented messages have total
|
||||||
|
length at least 16,384 bytes. The declared length is checked against the local
|
||||||
|
logical envelope ceiling before allocation. Offset starts at zero and must
|
||||||
|
equal the assembled prefix length; total length must stay constant. Gaps,
|
||||||
|
overlaps, duplicate prefixes, changed totals, unknown kinds, empty packets,
|
||||||
|
overflowing ranges, and oversized messages are fatal framing errors. There is
|
||||||
|
no recursive interpretation of an assembled payload as another transport packet.
|
||||||
|
|
||||||
|
The final fragment is identified by offset plus data length equalling total
|
||||||
|
length. The next logical message starts again at offset zero. Ordered WebSocket
|
||||||
|
delivery and one in-flight message per lane remove the need for fragment UUIDs,
|
||||||
|
extra event sequences, final flags, or transport acknowledgements. Existing
|
||||||
|
payload/request hashes remain validated after assembly; no additional transport
|
||||||
|
checksum or durable fragment ledger is introduced.
|
||||||
|
|
||||||
|
## Ordering, liveness, and bounds
|
||||||
|
|
||||||
|
One connection-scoped decoder owns two bounded partial buffers, one per lane.
|
||||||
|
Complete small messages can overtake either partial message. A large control
|
||||||
|
message can complete while a data message is paused. The server's sole writer
|
||||||
|
keeps lane order, checks heartbeat state between packets, and drains a bounded
|
||||||
|
control burst between data packets so data also progresses. Write deadlines
|
||||||
|
apply to each packet; a slow complete logical message can take longer than one
|
||||||
|
packet deadline while continuing to make progress.
|
||||||
|
Parent operation deadlines still apply.
|
||||||
|
|
||||||
|
`Written` and `OnWritten` remain whole-message barriers. A partially transmitted
|
||||||
|
dispatch cannot release dependent stdin or signals, and a partial reconciliation
|
||||||
|
result cannot enable new work. The decoder delivers no partial envelope to
|
||||||
|
protobuf validation, storage, execution, or acknowledgement handlers.
|
||||||
|
|
||||||
|
One read pump per connection handles control frames even when the application
|
||||||
|
is sending a large message. It has one complete-message delivery slot. Partial
|
||||||
|
buffers are individually limited by the logical ceiling; the queued/in-flight
|
||||||
|
complete messages are also bounded. Valid packet traffic updates server
|
||||||
|
liveness before assembly finishes. Socket close cancels the pump, including a
|
||||||
|
blocked delivery. There is no goroutine, queue, or persistence record per packet.
|
||||||
|
|
||||||
|
Client concurrent logical writes are serialized. The server uses its existing
|
||||||
|
single writer, with bounded control preemption and separate fragment lanes.
|
||||||
|
On disconnect, partial buffers are discarded with the connection. A new
|
||||||
|
connection cannot finish an old tail. Existing durable commands/events/stdin
|
||||||
|
and reconciliation supply replay; IDs, event digests, sequences, newline
|
||||||
|
behavior, and idempotency are unchanged.
|
||||||
|
|
||||||
|
## Offline verification
|
||||||
|
|
||||||
|
Run `scripts/build verify` in the established Docker toolchain. Focused checks:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker compose -f deploy/compose.yaml run --rm toolchain \
|
||||||
|
go test -race ./internal/agentwire ./internal/client/agent \
|
||||||
|
./internal/server/session ./test/integration/clientagent
|
||||||
|
```
|
||||||
|
|
||||||
|
The tests cover exact packet/logical boundaries, malformed declarations and
|
||||||
|
ordering, bounded allocation, two-lane preemption, whole-message write barriers,
|
||||||
|
heartbeat/Pong progress, concurrent writes, partial-disconnect replay, TLS,
|
||||||
|
old/new fallback in both directions, large output/dispatch/reconciliation, and
|
||||||
|
large stdin preserving one write sequence/newline/idempotency result. Fuzz
|
||||||
|
seeds exercise the decoder during normal tests; run a longer fuzz campaign in
|
||||||
|
the same isolated toolchain when needed. None of these commands deploys the
|
||||||
|
feature or contacts the live workstation.
|
||||||
@@ -22,7 +22,7 @@ const (
|
|||||||
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
|
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
|
||||||
)
|
)
|
||||||
|
|
||||||
// One AgentEnvelope is carried in one binary WebSocket message.
|
// Logical Agent message; negotiated transport chunking may span WebSocket messages.
|
||||||
type AgentEnvelope struct {
|
type AgentEnvelope struct {
|
||||||
state protoimpl.MessageState `protogen:"open.v1"`
|
state protoimpl.MessageState `protogen:"open.v1"`
|
||||||
// Empty only for ClientHello. All other envelopes are fenced to a session.
|
// Empty only for ClientHello. All other envelopes are fenced to a session.
|
||||||
|
|||||||
@@ -0,0 +1,136 @@
|
|||||||
|
package agentwire
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"math"
|
||||||
|
"sync"
|
||||||
|
|
||||||
|
"github.com/coder/websocket"
|
||||||
|
)
|
||||||
|
|
||||||
|
type incoming struct {
|
||||||
|
kind websocket.MessageType
|
||||||
|
payload []byte
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
|
||||||
|
// Conn owns one bounded read pump so WebSocket Ping/Pong is processed even
|
||||||
|
// while a caller is writing a large hello/reconciliation message. Application
|
||||||
|
// readers see only complete logical messages. Legacy peers retain raw protobuf.
|
||||||
|
type Conn struct {
|
||||||
|
*websocket.Conn
|
||||||
|
maximum uint64
|
||||||
|
chunked bool
|
||||||
|
cancel context.CancelFunc
|
||||||
|
read chan incoming
|
||||||
|
writeMu sync.Mutex
|
||||||
|
}
|
||||||
|
|
||||||
|
func Wrap(raw *websocket.Conn, maximum uint64, activity func()) (*Conn, error) {
|
||||||
|
if raw == nil || maximum == 0 || maximum > math.MaxInt64-1 {
|
||||||
|
return nil, ErrMessageTooBig
|
||||||
|
}
|
||||||
|
chunked := raw.Subprotocol() == Subprotocol
|
||||||
|
if chunked {
|
||||||
|
raw.SetReadLimit(PacketBytes)
|
||||||
|
} else {
|
||||||
|
raw.SetReadLimit(int64(maximum))
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
connection := &Conn{Conn: raw, maximum: maximum, chunked: chunked, cancel: cancel, read: make(chan incoming, 1)}
|
||||||
|
go connection.readLoop(ctx, activity)
|
||||||
|
return connection, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (connection *Conn) readLoop(ctx context.Context, activity func()) {
|
||||||
|
defer close(connection.read)
|
||||||
|
decoder := Decoder{Maximum: connection.maximum}
|
||||||
|
for {
|
||||||
|
kind, payload, err := connection.Conn.Read(ctx)
|
||||||
|
if err == nil && kind == websocket.MessageBinary && connection.chunked {
|
||||||
|
payload, err = decoder.Push(payload)
|
||||||
|
if err != nil {
|
||||||
|
code := websocket.StatusPolicyViolation
|
||||||
|
if errors.Is(err, ErrMessageTooBig) {
|
||||||
|
code = websocket.StatusMessageTooBig
|
||||||
|
}
|
||||||
|
_ = connection.Conn.Close(code, err.Error())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
_ = connection.Conn.CloseNow()
|
||||||
|
select {
|
||||||
|
case connection.read <- incoming{err: err}:
|
||||||
|
case <-ctx.Done():
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if activity != nil {
|
||||||
|
activity()
|
||||||
|
}
|
||||||
|
if connection.chunked && kind == websocket.MessageBinary && payload == nil { // Incomplete fragment.
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case connection.read <- incoming{kind: kind, payload: payload}:
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (connection *Conn) Read(ctx context.Context) (websocket.MessageType, []byte, error) {
|
||||||
|
select {
|
||||||
|
case frame, ok := <-connection.read:
|
||||||
|
if !ok {
|
||||||
|
return 0, nil, io.EOF
|
||||||
|
}
|
||||||
|
return frame.kind, frame.payload, frame.err
|
||||||
|
case <-ctx.Done():
|
||||||
|
return 0, nil, ctx.Err()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Encode is used by the server's sole queue writer so it can yield to control
|
||||||
|
// messages and check heartbeats between data packets. Each lane stays ordered.
|
||||||
|
func (connection *Conn) Encode(payload []byte, priority bool) (*Encoder, error) {
|
||||||
|
return NewEncoder(payload, connection.maximum, connection.chunked, priority)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (connection *Conn) WritePacket(ctx context.Context, packet []byte) error {
|
||||||
|
return connection.Conn.Write(ctx, websocket.MessageBinary, packet)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (connection *Conn) Write(ctx context.Context, kind websocket.MessageType, payload []byte) error {
|
||||||
|
if kind != websocket.MessageBinary {
|
||||||
|
return errors.New("agent transport writes must be binary")
|
||||||
|
}
|
||||||
|
encoder, err := connection.Encode(payload, false)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
connection.writeMu.Lock()
|
||||||
|
defer connection.writeMu.Unlock()
|
||||||
|
for {
|
||||||
|
packet, last := encoder.Next()
|
||||||
|
if err := connection.WritePacket(ctx, packet); err != nil {
|
||||||
|
_ = connection.CloseNow()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if last {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (connection *Conn) Close(code websocket.StatusCode, reason string) error {
|
||||||
|
defer connection.cancel()
|
||||||
|
return connection.Conn.Close(code, reason)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (connection *Conn) CloseNow() error {
|
||||||
|
connection.cancel()
|
||||||
|
return connection.Conn.CloseNow()
|
||||||
|
}
|
||||||
@@ -0,0 +1,216 @@
|
|||||||
|
package agentwire
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"sync/atomic"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/coder/websocket"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestNegotiationFallbackAndBackgroundPong_HP_CHUNK_02(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
for _, scenario := range []struct {
|
||||||
|
name string
|
||||||
|
server, client bool
|
||||||
|
}{
|
||||||
|
{"both", true, true}, {"old server", false, true}, {"old client", true, false},
|
||||||
|
} {
|
||||||
|
t.Run(scenario.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
payload := bytes.Repeat([]byte{0x53}, 64<<10)
|
||||||
|
pong := make(chan error, 1)
|
||||||
|
var activity atomic.Uint32
|
||||||
|
server := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
options := &websocket.AcceptOptions{}
|
||||||
|
if scenario.server {
|
||||||
|
options.Subprotocols = []string{Subprotocol}
|
||||||
|
}
|
||||||
|
raw, err := websocket.Accept(w, r, options)
|
||||||
|
if err != nil {
|
||||||
|
pong <- err
|
||||||
|
return
|
||||||
|
}
|
||||||
|
connection, err := Wrap(raw, 1<<20, func() { activity.Add(1) })
|
||||||
|
if err != nil {
|
||||||
|
_ = raw.CloseNow()
|
||||||
|
pong <- err
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer connection.CloseNow()
|
||||||
|
_, got, err := connection.Read(ctx)
|
||||||
|
if err == nil && !bytes.Equal(got, payload) {
|
||||||
|
err = errors.New("payload changed")
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
err = connection.Write(ctx, websocket.MessageBinary, got)
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
err = connection.Ping(ctx)
|
||||||
|
}
|
||||||
|
pong <- err
|
||||||
|
_, _, _ = connection.Read(ctx)
|
||||||
|
}))
|
||||||
|
defer server.Close()
|
||||||
|
options := &websocket.DialOptions{HTTPClient: server.Client()}
|
||||||
|
if scenario.client {
|
||||||
|
options.Subprotocols = []string{Subprotocol}
|
||||||
|
}
|
||||||
|
raw, _, err := websocket.Dial(ctx, "wss"+strings.TrimPrefix(server.URL, "https"), options)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
connection, err := Wrap(raw, 1<<20, nil)
|
||||||
|
if err != nil {
|
||||||
|
_ = raw.CloseNow()
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer connection.Close(websocket.StatusNormalClosure, "")
|
||||||
|
if connection.chunked != (scenario.client && scenario.server) {
|
||||||
|
t.Fatal("wrong negotiated mode")
|
||||||
|
}
|
||||||
|
if err := connection.Write(ctx, websocket.MessageBinary, payload); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// Do not call Read yet: the read pump must still service Ping while
|
||||||
|
// the echoed complete message waits for the application.
|
||||||
|
select {
|
||||||
|
case err := <-pong:
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal("Ping blocked behind application read")
|
||||||
|
}
|
||||||
|
_, got, err := connection.Read(ctx)
|
||||||
|
if err != nil || !bytes.Equal(got, payload) {
|
||||||
|
t.Fatalf("echo bytes=%d error=%v", len(got), err)
|
||||||
|
}
|
||||||
|
if connection.chunked && activity.Load() <= 1 {
|
||||||
|
t.Fatal("large message was not chunked")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestConcurrentLogicalWrites_RACE_CHUNK_01(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 8*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
result := make(chan error, 1)
|
||||||
|
server := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
raw, err := websocket.Accept(w, r, &websocket.AcceptOptions{Subprotocols: []string{Subprotocol}})
|
||||||
|
if err != nil {
|
||||||
|
result <- err
|
||||||
|
return
|
||||||
|
}
|
||||||
|
connection, err := Wrap(raw, 1<<20, nil)
|
||||||
|
if err != nil {
|
||||||
|
_ = raw.CloseNow()
|
||||||
|
result <- err
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer connection.CloseNow()
|
||||||
|
seen := map[byte]bool{}
|
||||||
|
for len(seen) < 8 {
|
||||||
|
_, payload, err := connection.Read(ctx)
|
||||||
|
if err != nil {
|
||||||
|
result <- err
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if len(payload) != 128<<10 || seen[payload[0]] || !bytes.Equal(payload, bytes.Repeat(payload[:1], len(payload))) {
|
||||||
|
result <- errors.New("concurrent messages interleaved or duplicated")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
seen[payload[0]] = true
|
||||||
|
}
|
||||||
|
result <- nil
|
||||||
|
}))
|
||||||
|
defer server.Close()
|
||||||
|
raw, _, err := websocket.Dial(ctx, "wss"+strings.TrimPrefix(server.URL, "https"), &websocket.DialOptions{HTTPClient: server.Client(), Subprotocols: []string{Subprotocol}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
connection, err := Wrap(raw, 1<<20, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer connection.CloseNow()
|
||||||
|
var workers sync.WaitGroup
|
||||||
|
for i := byte(0); i < 8; i++ {
|
||||||
|
workers.Go(func() {
|
||||||
|
if err := connection.Write(ctx, websocket.MessageBinary, bytes.Repeat([]byte{i}, 128<<10)); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
workers.Wait()
|
||||||
|
select {
|
||||||
|
case err := <-result:
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal("concurrent write test stalled")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInterruptedPacketNeverDelivered_REC_CHUNK_02(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
result := make(chan incoming, 2)
|
||||||
|
server := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
raw, err := websocket.Accept(w, r, &websocket.AcceptOptions{Subprotocols: []string{Subprotocol}})
|
||||||
|
if err != nil {
|
||||||
|
result <- incoming{err: err}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
connection, err := Wrap(raw, 1<<20, nil)
|
||||||
|
if err != nil {
|
||||||
|
_ = raw.CloseNow()
|
||||||
|
result <- incoming{err: err}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer connection.CloseNow()
|
||||||
|
_, payload, err := connection.Read(ctx)
|
||||||
|
result <- incoming{payload: payload, err: err}
|
||||||
|
}))
|
||||||
|
defer server.Close()
|
||||||
|
connect := func() *websocket.Conn {
|
||||||
|
t.Helper()
|
||||||
|
connection, _, err := websocket.Dial(ctx, "wss"+strings.TrimPrefix(server.URL, "https"), &websocket.DialOptions{HTTPClient: server.Client(), Subprotocols: []string{Subprotocol}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return connection
|
||||||
|
}
|
||||||
|
payload := bytes.Repeat([]byte{0x55}, 64<<10)
|
||||||
|
first := connect()
|
||||||
|
if err := first.Write(ctx, websocket.MessageBinary, packets(t, payload, false)[0]); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
_ = first.CloseNow()
|
||||||
|
if got := <-result; got.err == nil || got.payload != nil {
|
||||||
|
t.Fatal("partial message delivered")
|
||||||
|
}
|
||||||
|
second := connect()
|
||||||
|
defer second.CloseNow()
|
||||||
|
for _, packet := range packets(t, payload, false) {
|
||||||
|
if err := second.Write(ctx, websocket.MessageBinary, packet); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if got := <-result; got.err != nil || !bytes.Equal(got.payload, payload) {
|
||||||
|
t.Fatalf("replay: %v", got.err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,177 @@
|
|||||||
|
package agentwire
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/coder/websocket"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestLogicalSendCeilingBeforeAnyPacket_BH_CHUNK_02(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
result := make(chan incoming, 1)
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
raw, err := websocket.Accept(w, r, &websocket.AcceptOptions{Subprotocols: []string{Subprotocol}})
|
||||||
|
if err != nil {
|
||||||
|
result <- incoming{err: err}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
connection, err := Wrap(raw, 2048, nil)
|
||||||
|
if err != nil {
|
||||||
|
_ = raw.CloseNow()
|
||||||
|
result <- incoming{err: err}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer connection.CloseNow()
|
||||||
|
_, payload, err := connection.Read(ctx)
|
||||||
|
result <- incoming{payload: payload, err: err}
|
||||||
|
}))
|
||||||
|
defer server.Close()
|
||||||
|
raw, _, err := websocket.Dial(ctx, "ws"+strings.TrimPrefix(server.URL, "http"), &websocket.DialOptions{Subprotocols: []string{Subprotocol}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
connection, err := Wrap(raw, 2048, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer connection.CloseNow()
|
||||||
|
if err := connection.Write(ctx, websocket.MessageBinary, make([]byte, 2049)); !errors.Is(err, ErrMessageTooBig) {
|
||||||
|
t.Fatal("logical send limit bypassed")
|
||||||
|
}
|
||||||
|
if err := connection.Write(ctx, websocket.MessageBinary, []byte("still usable")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case frame := <-result:
|
||||||
|
if frame.err != nil || !bytes.Equal(frame.payload, []byte("still usable")) {
|
||||||
|
t.Fatal("oversized prefix was sent or connection was poisoned")
|
||||||
|
}
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal("receiver did not receive next valid message")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCloseStopsBlockedReadPump_RACE_CHUNK_03(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
sent := make(chan struct{})
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
connection, err := websocket.Accept(w, r, &websocket.AcceptOptions{Subprotocols: []string{Subprotocol}})
|
||||||
|
if err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer connection.CloseNow()
|
||||||
|
for i := 0; i < 3; i++ {
|
||||||
|
if err := connection.Write(ctx, websocket.MessageBinary, []byte{0, 0x51}); err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
close(sent)
|
||||||
|
_, _, _ = connection.Read(ctx)
|
||||||
|
}))
|
||||||
|
defer server.Close()
|
||||||
|
raw, _, err := websocket.Dial(ctx, "ws"+strings.TrimPrefix(server.URL, "http"), &websocket.DialOptions{Subprotocols: []string{Subprotocol}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
connection, err := Wrap(raw, 1<<20, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer connection.CloseNow()
|
||||||
|
select {
|
||||||
|
case <-sent:
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal("peer did not fill delivery queue")
|
||||||
|
}
|
||||||
|
for len(connection.read) == 0 {
|
||||||
|
select {
|
||||||
|
case <-time.After(time.Millisecond):
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal("delivery queue never filled")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := connection.Close(websocket.StatusNormalClosure, ""); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case _, ok := <-connection.read:
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal("read pump leaked after close with stopped consumer")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMalformedWireClosesWithoutDelivery_BH_CHUNK_03(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
for _, scenario := range []struct {
|
||||||
|
name string
|
||||||
|
packet []byte
|
||||||
|
want error
|
||||||
|
}{
|
||||||
|
{"invalid kind", []byte{0xff, 1}, ErrPacket},
|
||||||
|
{"oversized declaration", []byte{1, 0xff, 0xff, 0xff, 0xff, 0, 0, 0, 0, 1}, ErrMessageTooBig},
|
||||||
|
{"oversized wire packet", make([]byte, PacketBytes+1), websocket.ErrMessageTooBig},
|
||||||
|
} {
|
||||||
|
t.Run(scenario.name, func(t *testing.T) {
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
result := make(chan incoming, 1)
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
raw, err := websocket.Accept(w, r, &websocket.AcceptOptions{Subprotocols: []string{Subprotocol}})
|
||||||
|
if err != nil {
|
||||||
|
result <- incoming{err: err}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
connection, err := Wrap(raw, 1<<20, nil)
|
||||||
|
if err != nil {
|
||||||
|
_ = raw.CloseNow()
|
||||||
|
result <- incoming{err: err}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer connection.CloseNow()
|
||||||
|
_, payload, err := connection.Read(ctx)
|
||||||
|
result <- incoming{payload: payload, err: err}
|
||||||
|
}))
|
||||||
|
defer server.Close()
|
||||||
|
raw, _, err := websocket.Dial(ctx, "ws"+strings.TrimPrefix(server.URL, "http"), &websocket.DialOptions{Subprotocols: []string{Subprotocol}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer raw.CloseNow()
|
||||||
|
if err := raw.Write(ctx, websocket.MessageBinary, scenario.packet); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
_, _, closed := raw.Read(ctx)
|
||||||
|
wantCode := websocket.StatusPolicyViolation
|
||||||
|
if !errors.Is(scenario.want, ErrPacket) {
|
||||||
|
wantCode = websocket.StatusMessageTooBig
|
||||||
|
}
|
||||||
|
if websocket.CloseStatus(closed) != wantCode {
|
||||||
|
t.Fatalf("close code = %v, want %v", closed, wantCode)
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case frame := <-result:
|
||||||
|
if frame.payload != nil || !errors.Is(frame.err, scenario.want) {
|
||||||
|
t.Fatalf("malformed frame delivered or wrong error: %v", frame.err)
|
||||||
|
}
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal("malformed packet did not close reader")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,130 @@
|
|||||||
|
// Package agentwire carries bounded logical Agent messages over WebSocket.
|
||||||
|
// It does not assign command IDs, event sequences, or durable acknowledgements.
|
||||||
|
package agentwire
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/binary"
|
||||||
|
"errors"
|
||||||
|
"math"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
Subprotocol = "rvbox.agent.v1.chunked"
|
||||||
|
PacketBytes = 16 << 10
|
||||||
|
headerBytes = 9
|
||||||
|
complete = byte(0)
|
||||||
|
data = byte(1)
|
||||||
|
control = byte(2)
|
||||||
|
)
|
||||||
|
|
||||||
|
var (
|
||||||
|
ErrPacket = errors.New("invalid agent transport packet")
|
||||||
|
ErrMessageTooBig = errors.New("agent transport message exceeds logical limit")
|
||||||
|
)
|
||||||
|
|
||||||
|
// Encoder retains one immutable logical message and allocates one wire packet
|
||||||
|
// at a time. Two fragment lanes reuse the server's control/data distinction;
|
||||||
|
// no fragment IDs are needed on an ordered, connection-scoped stream.
|
||||||
|
type Encoder struct {
|
||||||
|
payload []byte
|
||||||
|
offset int
|
||||||
|
kind byte
|
||||||
|
chunked bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewEncoder(payload []byte, maximum uint64, chunked, priority bool) (*Encoder, error) {
|
||||||
|
if len(payload) == 0 {
|
||||||
|
return nil, ErrPacket
|
||||||
|
}
|
||||||
|
if uint64(len(payload)) > maximum || uint64(len(payload)) > math.MaxUint32 {
|
||||||
|
return nil, ErrMessageTooBig
|
||||||
|
}
|
||||||
|
kind := data
|
||||||
|
if priority {
|
||||||
|
kind = control
|
||||||
|
}
|
||||||
|
return &Encoder{payload: payload, chunked: chunked, kind: kind}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Next returns a packet and whether it finishes this logical message.
|
||||||
|
func (encoder *Encoder) Next() ([]byte, bool) {
|
||||||
|
if encoder.offset == len(encoder.payload) {
|
||||||
|
return nil, true
|
||||||
|
}
|
||||||
|
if !encoder.chunked {
|
||||||
|
encoder.offset = len(encoder.payload)
|
||||||
|
return encoder.payload, true
|
||||||
|
}
|
||||||
|
if len(encoder.payload) < PacketBytes {
|
||||||
|
packet := make([]byte, len(encoder.payload)+1)
|
||||||
|
copy(packet[1:], encoder.payload)
|
||||||
|
encoder.offset = len(encoder.payload)
|
||||||
|
return packet, true
|
||||||
|
}
|
||||||
|
end := min(len(encoder.payload), encoder.offset+PacketBytes-headerBytes)
|
||||||
|
packet := make([]byte, headerBytes+end-encoder.offset)
|
||||||
|
packet[0] = encoder.kind
|
||||||
|
binary.BigEndian.PutUint32(packet[1:5], uint32(len(encoder.payload)))
|
||||||
|
binary.BigEndian.PutUint32(packet[5:9], uint32(encoder.offset))
|
||||||
|
copy(packet[headerBytes:], encoder.payload[encoder.offset:end])
|
||||||
|
encoder.offset = end
|
||||||
|
return packet, end == len(encoder.payload)
|
||||||
|
}
|
||||||
|
|
||||||
|
type partial struct {
|
||||||
|
total uint32
|
||||||
|
bytes []byte
|
||||||
|
}
|
||||||
|
|
||||||
|
// Decoder is owned by one network reader. At most one incomplete message per
|
||||||
|
// lane is retained, each bounded by Maximum before allocation. Complete small
|
||||||
|
// messages can bypass either partial lane. Errors are connection-fatal.
|
||||||
|
type Decoder struct {
|
||||||
|
Maximum uint64
|
||||||
|
lanes [2]partial
|
||||||
|
}
|
||||||
|
|
||||||
|
func (decoder *Decoder) Push(packet []byte) ([]byte, error) {
|
||||||
|
if len(packet) == 0 || len(packet) > PacketBytes {
|
||||||
|
return nil, ErrPacket
|
||||||
|
}
|
||||||
|
if packet[0] == complete {
|
||||||
|
if len(packet) == 1 {
|
||||||
|
return nil, ErrPacket
|
||||||
|
}
|
||||||
|
if uint64(len(packet)-1) > decoder.Maximum {
|
||||||
|
return nil, ErrMessageTooBig
|
||||||
|
}
|
||||||
|
return packet[1:], nil
|
||||||
|
}
|
||||||
|
if (packet[0] != data && packet[0] != control) || len(packet) <= headerBytes {
|
||||||
|
return nil, ErrPacket
|
||||||
|
}
|
||||||
|
total := binary.BigEndian.Uint32(packet[1:5])
|
||||||
|
offset := binary.BigEndian.Uint32(packet[5:9])
|
||||||
|
if uint64(total) > decoder.Maximum {
|
||||||
|
return nil, ErrMessageTooBig
|
||||||
|
}
|
||||||
|
length := uint32(len(packet) - headerBytes)
|
||||||
|
if total < PacketBytes || offset > total || length > total-offset {
|
||||||
|
return nil, ErrPacket
|
||||||
|
}
|
||||||
|
lane := &decoder.lanes[packet[0]-1]
|
||||||
|
if lane.bytes == nil {
|
||||||
|
if offset != 0 {
|
||||||
|
return nil, ErrPacket
|
||||||
|
}
|
||||||
|
lane.total = total
|
||||||
|
lane.bytes = make([]byte, 0, int(total))
|
||||||
|
}
|
||||||
|
if lane.total != total || uint64(offset) != uint64(len(lane.bytes)) {
|
||||||
|
return nil, ErrPacket
|
||||||
|
}
|
||||||
|
lane.bytes = append(lane.bytes, packet[headerBytes:]...)
|
||||||
|
if uint32(len(lane.bytes)) != total {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
result := lane.bytes
|
||||||
|
*lane = partial{}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,169 @@
|
|||||||
|
package agentwire
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/binary"
|
||||||
|
"errors"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func packets(t *testing.T, payload []byte, priority bool) [][]byte {
|
||||||
|
t.Helper()
|
||||||
|
encoder, err := NewEncoder(payload, 1<<20, true, priority)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var result [][]byte
|
||||||
|
for {
|
||||||
|
packet, last := encoder.Next()
|
||||||
|
if len(packet) > PacketBytes {
|
||||||
|
t.Fatal("wire packet exceeds ceiling")
|
||||||
|
}
|
||||||
|
result = append(result, packet)
|
||||||
|
if last {
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMessageRoundTripAndBoundaries_HP_CHUNK_01(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
for _, size := range []int{1, PacketBytes - 1, PacketBytes, PacketBytes + 1, 64 << 10, 1 << 20} {
|
||||||
|
payload := bytes.Repeat([]byte{0x53}, size)
|
||||||
|
decoder := Decoder{Maximum: 1 << 20}
|
||||||
|
encoded := packets(t, payload, false)
|
||||||
|
for index, packet := range encoded {
|
||||||
|
result, err := decoder.Push(packet)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if index != len(encoded)-1 && result != nil {
|
||||||
|
t.Fatal("partial logical message delivered")
|
||||||
|
}
|
||||||
|
if index == len(encoded)-1 && !bytes.Equal(result, payload) {
|
||||||
|
t.Fatalf("message size %d changed", size)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := NewEncoder(make([]byte, (1<<20)+1), 1<<20, true, false); !errors.Is(err, ErrMessageTooBig) {
|
||||||
|
t.Fatalf("logical ceiling bypassed: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := NewEncoder(nil, 1<<20, true, false); !errors.Is(err, ErrPacket) {
|
||||||
|
t.Fatal("empty logical message accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestIndependentControlLaneAndDisconnect_REC_CHUNK_01(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
large := bytes.Repeat([]byte{0x51}, 200<<10)
|
||||||
|
priority := bytes.Repeat([]byte{0x52}, 48<<10)
|
||||||
|
dataPackets := packets(t, large, false)
|
||||||
|
controlPackets := packets(t, priority, true)
|
||||||
|
decoder := Decoder{Maximum: 1 << 20}
|
||||||
|
if value, err := decoder.Push(dataPackets[0]); err != nil || value != nil {
|
||||||
|
t.Fatalf("first partial packet: %v", err)
|
||||||
|
}
|
||||||
|
for index, packet := range controlPackets {
|
||||||
|
value, err := decoder.Push(packet)
|
||||||
|
if err != nil || index == len(controlPackets)-1 && !bytes.Equal(value, priority) {
|
||||||
|
t.Fatalf("control lane did not overtake partial data: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ack := []byte("ack")
|
||||||
|
if value, err := decoder.Push(packets(t, ack, false)[0]); err != nil || !bytes.Equal(value, ack) {
|
||||||
|
t.Fatal("small message blocked behind partial data")
|
||||||
|
}
|
||||||
|
for index, packet := range dataPackets[1:] {
|
||||||
|
value, err := decoder.Push(packet)
|
||||||
|
if err != nil || index == len(dataPackets)-2 && !bytes.Equal(value, large) {
|
||||||
|
t.Fatalf("preemption changed data: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// A new connection has no partial state. A tail without its prefix fails;
|
||||||
|
// a complete durable replay from offset zero succeeds.
|
||||||
|
restarted := Decoder{Maximum: 1 << 20}
|
||||||
|
if _, err := restarted.Push(dataPackets[1]); !errors.Is(err, ErrPacket) {
|
||||||
|
t.Fatal("new connection accepted an old tail")
|
||||||
|
}
|
||||||
|
restarted = Decoder{Maximum: 1 << 20}
|
||||||
|
var result []byte
|
||||||
|
for _, packet := range dataPackets {
|
||||||
|
var err error
|
||||||
|
result, err = restarted.Push(packet)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !bytes.Equal(result, large) {
|
||||||
|
t.Fatal("durable replay changed message")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRejectMalformedPacketsWithoutAllocation_BH_CHUNK_01(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
base := packets(t, bytes.Repeat([]byte{0x53}, 64<<10), false)
|
||||||
|
changed := func(packet []byte, position int, value uint32) []byte {
|
||||||
|
result := bytes.Clone(packet)
|
||||||
|
binary.BigEndian.PutUint32(result[position:position+4], value)
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
for _, scenario := range []struct {
|
||||||
|
name string
|
||||||
|
input [][]byte
|
||||||
|
want error
|
||||||
|
max uint64
|
||||||
|
}{
|
||||||
|
{name: "empty packet", input: [][]byte{nil}, want: ErrPacket},
|
||||||
|
{name: "empty complete", input: [][]byte{{0}}, want: ErrPacket},
|
||||||
|
{name: "unknown kind", input: [][]byte{{3, 1}}, want: ErrPacket},
|
||||||
|
{name: "short header", input: [][]byte{{1, 0}}, want: ErrPacket},
|
||||||
|
{name: "oversized packet", input: [][]byte{make([]byte, PacketBytes+1)}, want: ErrPacket},
|
||||||
|
{name: "oversized declaration", input: [][]byte{changed(base[0], 1, 0xffffffff)}, want: ErrMessageTooBig},
|
||||||
|
{name: "small fragmented declaration", input: [][]byte{changed(base[0], 1, 1)}, want: ErrPacket},
|
||||||
|
{name: "tail first", input: [][]byte{base[1]}, want: ErrPacket},
|
||||||
|
{name: "duplicate prefix", input: [][]byte{base[0], base[0]}, want: ErrPacket},
|
||||||
|
{name: "gap", input: [][]byte{base[0], base[2]}, want: ErrPacket},
|
||||||
|
{name: "changed total", input: [][]byte{base[0], changed(base[1], 1, 65537)}, want: ErrPacket},
|
||||||
|
{name: "offset overflow", input: [][]byte{changed(base[0], 5, 0xffffffff)}, want: ErrPacket},
|
||||||
|
{name: "configured complete limit", input: [][]byte{{0, 1, 2}}, max: 1, want: ErrMessageTooBig},
|
||||||
|
} {
|
||||||
|
t.Run(scenario.name, func(t *testing.T) {
|
||||||
|
maximum := scenario.max
|
||||||
|
if maximum == 0 {
|
||||||
|
maximum = 1 << 20
|
||||||
|
}
|
||||||
|
decoder := Decoder{Maximum: maximum}
|
||||||
|
var err error
|
||||||
|
for _, packet := range scenario.input {
|
||||||
|
_, err = decoder.Push(packet)
|
||||||
|
if err != nil {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !errors.Is(err, scenario.want) {
|
||||||
|
t.Fatalf("error = %v, want %v", err, scenario.want)
|
||||||
|
}
|
||||||
|
if scenario.name == "oversized declaration" && decoder.lanes[0].bytes != nil {
|
||||||
|
t.Fatal("allocated an oversized declared message")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func FuzzDecoder(f *testing.F) {
|
||||||
|
f.Add([]byte{0, 1}, []byte{1})
|
||||||
|
f.Add([]byte{1, 0, 1, 0, 0, 0, 0, 0, 0, 1}, []byte{0, 1})
|
||||||
|
f.Fuzz(func(t *testing.T, first, second []byte) {
|
||||||
|
decoder := Decoder{Maximum: 64 << 10}
|
||||||
|
for _, packet := range [][]byte{first, second} {
|
||||||
|
if _, err := decoder.Push(packet); err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for _, lane := range decoder.lanes {
|
||||||
|
if cap(lane.bytes) > 64<<10 {
|
||||||
|
t.Fatal("reassembly exceeded configured bound")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
@@ -9,11 +9,12 @@ import (
|
|||||||
|
|
||||||
"github.com/coder/websocket"
|
"github.com/coder/websocket"
|
||||||
"github.com/rvbox/rvbox/internal/agentproto"
|
"github.com/rvbox/rvbox/internal/agentproto"
|
||||||
|
"github.com/rvbox/rvbox/internal/agentwire"
|
||||||
)
|
)
|
||||||
|
|
||||||
var ErrUnexpectedMessage = errors.New("agent transport received a non-binary WebSocket message")
|
var ErrUnexpectedMessage = errors.New("agent transport received a non-binary WebSocket message")
|
||||||
|
|
||||||
type WebSocketTransport struct{ connection *websocket.Conn }
|
type WebSocketTransport struct{ connection *agentwire.Conn }
|
||||||
|
|
||||||
func DialWebSocket(ctx context.Context, address string, client *http.Client) (*WebSocketTransport, error) {
|
func DialWebSocket(ctx context.Context, address string, client *http.Client) (*WebSocketTransport, error) {
|
||||||
return DialWebSocketWithLimits(ctx, address, client, agentproto.DefaultLimits())
|
return DialWebSocketWithLimits(ctx, address, client, agentproto.DefaultLimits())
|
||||||
@@ -30,11 +31,17 @@ func DialWebSocketWithLimits(ctx context.Context, address string, client *http.C
|
|||||||
if readLimit > math.MaxInt64-1 {
|
if readLimit > math.MaxInt64-1 {
|
||||||
return nil, fmt.Errorf("invalid WebSocket envelope limit: %d", readLimit)
|
return nil, fmt.Errorf("invalid WebSocket envelope limit: %d", readLimit)
|
||||||
}
|
}
|
||||||
connection, _, err := websocket.Dial(ctx, address, &websocket.DialOptions{HTTPClient: client, CompressionMode: websocket.CompressionDisabled})
|
raw, _, err := websocket.Dial(ctx, address, &websocket.DialOptions{
|
||||||
|
HTTPClient: client, CompressionMode: websocket.CompressionDisabled, Subprotocols: []string{agentwire.Subprotocol},
|
||||||
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
connection.SetReadLimit(int64(readLimit))
|
connection, err := agentwire.Wrap(raw, readLimit, nil)
|
||||||
|
if err != nil {
|
||||||
|
_ = raw.CloseNow()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
return &WebSocketTransport{connection: connection}, nil
|
return &WebSocketTransport{connection: connection}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -16,6 +16,7 @@ import (
|
|||||||
"github.com/coder/websocket"
|
"github.com/coder/websocket"
|
||||||
rvboxv1 "github.com/rvbox/rvbox/gen/go/rvbox/v1"
|
rvboxv1 "github.com/rvbox/rvbox/gen/go/rvbox/v1"
|
||||||
"github.com/rvbox/rvbox/internal/agentproto"
|
"github.com/rvbox/rvbox/internal/agentproto"
|
||||||
|
"github.com/rvbox/rvbox/internal/agentwire"
|
||||||
"github.com/rvbox/rvbox/internal/domain"
|
"github.com/rvbox/rvbox/internal/domain"
|
||||||
"github.com/rvbox/rvbox/internal/observability"
|
"github.com/rvbox/rvbox/internal/observability"
|
||||||
"github.com/rvbox/rvbox/internal/server/store"
|
"github.com/rvbox/rvbox/internal/server/store"
|
||||||
@@ -78,8 +79,9 @@ func (server *AgentServer) ServeHTTP(response http.ResponseWriter, request *http
|
|||||||
}
|
}
|
||||||
started := time.Now()
|
started := time.Now()
|
||||||
heartbeat := newSynchronizedHeartbeat(server.heartbeatIdle(), server.livenessTimeout(), 0)
|
heartbeat := newSynchronizedHeartbeat(server.heartbeatIdle(), server.livenessTimeout(), 0)
|
||||||
connection, err := websocket.Accept(response, request, &websocket.AcceptOptions{
|
raw, err := websocket.Accept(response, request, &websocket.AcceptOptions{
|
||||||
CompressionMode: websocket.CompressionDisabled,
|
CompressionMode: websocket.CompressionDisabled,
|
||||||
|
Subprotocols: []string{agentwire.Subprotocol},
|
||||||
// coder/websocket consumes control frames inside Read. Count ping and
|
// coder/websocket consumes control frames inside Read. Count ping and
|
||||||
// pong callbacks as inbound activity so a healthy, otherwise idle agent
|
// pong callbacks as inbound activity so a healthy, otherwise idle agent
|
||||||
// is not mistaken for a dead peer; application Read results are observed
|
// is not mistaken for a dead peer; application Read results are observed
|
||||||
@@ -95,16 +97,20 @@ func (server *AgentServer) ServeHTTP(response http.ResponseWriter, request *http
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
connection.SetReadLimit(int64(server.limits().MaxEnvelopeBytes))
|
connection, err := agentwire.Wrap(raw, server.limits().MaxEnvelopeBytes, func() { heartbeat.Observe(time.Since(started)) })
|
||||||
|
if err != nil {
|
||||||
|
_ = raw.CloseNow()
|
||||||
|
return
|
||||||
|
}
|
||||||
server.incMetric("agent_connection")
|
server.incMetric("agent_connection")
|
||||||
defer connection.CloseNow()
|
defer connection.CloseNow()
|
||||||
server.serveConnection(request.Context(), connection, heartbeat, started)
|
server.serveConnection(request.Context(), connection, heartbeat, started)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (server *AgentServer) serveConnection(parent context.Context, connection *websocket.Conn, heartbeat *synchronizedHeartbeat, started time.Time) {
|
func (server *AgentServer) serveConnection(parent context.Context, connection *agentwire.Conn, heartbeat *synchronizedHeartbeat, started time.Time) {
|
||||||
messageType, payload, err := connection.Read(parent)
|
messageType, payload, err := connection.Read(parent)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if errors.Is(err, websocket.ErrMessageTooBig) {
|
if errors.Is(err, websocket.ErrMessageTooBig) || errors.Is(err, agentwire.ErrMessageTooBig) {
|
||||||
server.incMetric("agent_message_too_big")
|
server.incMetric("agent_message_too_big")
|
||||||
}
|
}
|
||||||
server.logSessionError("", 0, "hello-read", err)
|
server.logSessionError("", 0, "hello-read", err)
|
||||||
@@ -255,7 +261,7 @@ func (server *AgentServer) serveConnection(parent context.Context, connection *w
|
|||||||
for {
|
for {
|
||||||
messageType, payload, err = connection.Read(sessionContext)
|
messageType, payload, err = connection.Read(sessionContext)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if errors.Is(err, websocket.ErrMessageTooBig) {
|
if errors.Is(err, websocket.ErrMessageTooBig) || errors.Is(err, agentwire.ErrMessageTooBig) {
|
||||||
server.incMetric("agent_message_too_big")
|
server.incMetric("agent_message_too_big")
|
||||||
}
|
}
|
||||||
server.logSessionError(hello.GetClientId(), registration.Generation, "read", err)
|
server.logSessionError(hello.GetClientId(), registration.Generation, "read", err)
|
||||||
@@ -784,39 +790,81 @@ func (server *AgentServer) closeForDispatchFailure(cancel context.CancelFunc) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (server *AgentServer) writeLoop(ctx context.Context, connection *websocket.Conn, queue *WriterQueue, heartbeat *synchronizedHeartbeat, started time.Time) {
|
type packetWriter interface {
|
||||||
|
Encode([]byte, bool) (*agentwire.Encoder, error)
|
||||||
|
WritePacket(context.Context, []byte) error
|
||||||
|
Ping(context.Context) error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (server *AgentServer) writeLoop(ctx context.Context, connection packetWriter, queue *WriterQueue, heartbeat *synchronizedHeartbeat, started time.Time) {
|
||||||
|
checkHeartbeat := func() error {
|
||||||
|
switch heartbeat.Check(time.Since(started)) {
|
||||||
|
case HeartbeatPing:
|
||||||
|
pingContext, cancel := context.WithTimeout(ctx, server.writeDeadline())
|
||||||
|
defer cancel()
|
||||||
|
return connection.Ping(pingContext)
|
||||||
|
case HeartbeatClose:
|
||||||
|
server.incMetric("heartbeat_timeout")
|
||||||
|
return ErrSessionClosed
|
||||||
|
}
|
||||||
|
return ctx.Err()
|
||||||
|
}
|
||||||
|
var writeFrame func(Frame) error
|
||||||
|
writeFrame = func(frame Frame) error {
|
||||||
|
encoder, err := connection.Encode(frame.Payload, frame.Kind == FrameControl)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for {
|
||||||
|
packet, last := encoder.Next()
|
||||||
|
writeContext, cancel := context.WithTimeout(ctx, server.writeDeadline())
|
||||||
|
err := connection.WritePacket(writeContext, packet)
|
||||||
|
cancel()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := checkHeartbeat(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if last {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if frame.Kind == FrameData {
|
||||||
|
// A bounded control burst may overtake a partial data message.
|
||||||
|
// Its independent fragment lane cannot corrupt data reassembly.
|
||||||
|
for burst := uint32(0); burst < queue.controlBurst; burst++ {
|
||||||
|
priority, ok := queue.TryControl()
|
||||||
|
if !ok {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if err := writeFrame(priority); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if frame.Written != nil {
|
||||||
|
close(frame.Written)
|
||||||
|
}
|
||||||
|
if frame.OnWritten != nil {
|
||||||
|
frame.OnWritten()
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
for {
|
for {
|
||||||
frameContext, cancel := context.WithTimeout(ctx, server.heartbeatPollInterval())
|
frameContext, cancel := context.WithTimeout(ctx, server.heartbeatPollInterval())
|
||||||
frame, err := queue.Next(frameContext)
|
frame, err := queue.Next(frameContext)
|
||||||
cancel()
|
cancel()
|
||||||
if err == nil {
|
if err == nil {
|
||||||
writeContext, writeCancel := context.WithTimeout(ctx, server.writeDeadline())
|
if err := writeFrame(frame); err != nil {
|
||||||
err = connection.Write(writeContext, websocket.MessageBinary, frame.Payload)
|
|
||||||
writeCancel()
|
|
||||||
if err != nil {
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if frame.Written != nil {
|
|
||||||
close(frame.Written)
|
|
||||||
}
|
|
||||||
if frame.OnWritten != nil {
|
|
||||||
frame.OnWritten()
|
|
||||||
}
|
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if !errors.Is(err, context.DeadlineExceeded) {
|
if !errors.Is(err, context.DeadlineExceeded) {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
switch heartbeat.Check(time.Since(started)) {
|
if err := checkHeartbeat(); err != nil {
|
||||||
case HeartbeatPing:
|
|
||||||
pingContext, pingCancel := context.WithTimeout(ctx, server.writeDeadline())
|
|
||||||
err = connection.Ping(pingContext)
|
|
||||||
pingCancel()
|
|
||||||
if err != nil {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
case HeartbeatClose:
|
|
||||||
server.incMetric("heartbeat_timeout")
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -834,7 +882,7 @@ func (server *AgentServer) observeMetric(name string, duration time.Duration) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (server *AgentServer) close(connection *websocket.Conn, status websocket.StatusCode, reason string) {
|
func (server *AgentServer) close(connection *agentwire.Conn, status websocket.StatusCode, reason string) {
|
||||||
_ = connection.Close(status, reason)
|
_ = connection.Close(status, reason)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,116 @@
|
|||||||
|
package session
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"sync/atomic"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/rvbox/rvbox/internal/agentwire"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestChunkedWriterPreemptionAndBarrier_RACE_CHUNK_02(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
queue := NewWriterQueue(8, 2, 4)
|
||||||
|
defer queue.Close()
|
||||||
|
data := bytes.Repeat([]byte{0x51}, 128<<10)
|
||||||
|
priority := bytes.Repeat([]byte{0x52}, 48<<10)
|
||||||
|
dataWritten, controlWritten := make(chan struct{}), make(chan struct{})
|
||||||
|
queue.EnqueueData(Frame{Kind: FrameData, Payload: data, Written: dataWritten})
|
||||||
|
writer := &blockedPacketWriter{packets: make(chan []byte, 32), gate: make(chan struct{})}
|
||||||
|
done := make(chan struct{})
|
||||||
|
started := time.Now()
|
||||||
|
go func() {
|
||||||
|
defer close(done)
|
||||||
|
(&AgentServer{}).writeLoop(ctx, writer, queue, newSynchronizedHeartbeat(time.Millisecond, time.Second, 0), started)
|
||||||
|
}()
|
||||||
|
var first []byte
|
||||||
|
select {
|
||||||
|
case first = <-writer.packets:
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal("first packet not written")
|
||||||
|
}
|
||||||
|
if err := queue.EnqueueControl(Frame{Kind: FrameControl, Payload: []byte("ack")}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := queue.EnqueueControl(Frame{Kind: FrameControl, Payload: priority, Written: controlWritten}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-dataWritten:
|
||||||
|
t.Fatal("dispatch barrier opened on a partial message")
|
||||||
|
default:
|
||||||
|
}
|
||||||
|
time.Sleep(2 * time.Millisecond) // Cross the heartbeat idle boundary.
|
||||||
|
close(writer.gate)
|
||||||
|
decoder := agentwire.Decoder{Maximum: 1 << 20}
|
||||||
|
if value, err := decoder.Push(first); err != nil || value != nil {
|
||||||
|
t.Fatal("invalid first partial packet")
|
||||||
|
}
|
||||||
|
var received [][]byte
|
||||||
|
for len(received) < 3 {
|
||||||
|
select {
|
||||||
|
case packet := <-writer.packets:
|
||||||
|
value, err := decoder.Push(packet)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if value != nil {
|
||||||
|
received = append(received, value)
|
||||||
|
}
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal("preempted writer stalled")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !bytes.Equal(received[0], []byte("ack")) || !bytes.Equal(received[1], priority) || !bytes.Equal(received[2], data) {
|
||||||
|
t.Fatal("control did not preempt partial data intact")
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-dataWritten:
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal("completed data barrier never opened")
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-controlWritten:
|
||||||
|
default:
|
||||||
|
t.Fatal("completed control barrier never opened")
|
||||||
|
}
|
||||||
|
cancel()
|
||||||
|
<-done
|
||||||
|
if writer.pings.Load() == 0 {
|
||||||
|
t.Fatal("heartbeat skipped during chunked transfer")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type blockedPacketWriter struct {
|
||||||
|
packets chan []byte
|
||||||
|
gate chan struct{}
|
||||||
|
writes int
|
||||||
|
pings atomic.Uint32
|
||||||
|
}
|
||||||
|
|
||||||
|
func (writer *blockedPacketWriter) Encode(payload []byte, priority bool) (*agentwire.Encoder, error) {
|
||||||
|
return agentwire.NewEncoder(payload, 1<<20, true, priority)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (writer *blockedPacketWriter) WritePacket(ctx context.Context, packet []byte) error {
|
||||||
|
select {
|
||||||
|
case writer.packets <- bytes.Clone(packet):
|
||||||
|
case <-ctx.Done():
|
||||||
|
return ctx.Err()
|
||||||
|
}
|
||||||
|
writer.writes++
|
||||||
|
if writer.writes == 1 {
|
||||||
|
select {
|
||||||
|
case <-writer.gate:
|
||||||
|
case <-ctx.Done():
|
||||||
|
return ctx.Err()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (writer *blockedPacketWriter) Ping(context.Context) error { writer.pings.Add(1); return nil }
|
||||||
@@ -126,6 +126,17 @@ func (queue *WriterQueue) Next(ctx context.Context) (Frame, error) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TryControl is used only by the sole writer between packets of a data frame.
|
||||||
|
func (queue *WriterQueue) TryControl() (Frame, bool) {
|
||||||
|
select {
|
||||||
|
case frame := <-queue.control:
|
||||||
|
queue.controlRun++
|
||||||
|
return frame, true
|
||||||
|
default:
|
||||||
|
return Frame{}, false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func (queue *WriterQueue) Close() { queue.closeOnce.Do(func() { close(queue.closed) }) }
|
func (queue *WriterQueue) Close() { queue.closeOnce.Do(func() { close(queue.closed) }) }
|
||||||
|
|
||||||
type HeartbeatAction uint8
|
type HeartbeatAction uint8
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ import "rvbox/v1/common.proto";
|
|||||||
|
|
||||||
option go_package = "github.com/rvbox/rvbox/gen/go/rvbox/v1;rvboxv1";
|
option go_package = "github.com/rvbox/rvbox/gen/go/rvbox/v1;rvboxv1";
|
||||||
|
|
||||||
// One AgentEnvelope is carried in one binary WebSocket message.
|
// Logical Agent message; negotiated transport chunking may span WebSocket messages.
|
||||||
message AgentEnvelope {
|
message AgentEnvelope {
|
||||||
// Empty only for ClientHello. All other envelopes are fenced to a session.
|
// Empty only for ClientHello. All other envelopes are fenced to a session.
|
||||||
string session_id = 1;
|
string session_id = 1;
|
||||||
|
|||||||
@@ -56,6 +56,15 @@ servers send scripts in 16 KiB chunks, allowing pending uploads to reach older
|
|||||||
clients without lowering inbound validation limits. Large inline commands,
|
clients without lowering inbound validation limits. Large inline commands,
|
||||||
stdin, and reconciliation responses still need the updated client reader.
|
stdin, and reconciliation responses still need the updated client reader.
|
||||||
|
|
||||||
|
Development builds may negotiate `rvbox.agent.v1.chunked` in the WebSocket
|
||||||
|
handshake. When both endpoints support it, messages automatically use bounded
|
||||||
|
16 KiB packets and are reassembled before protocol handling. Old peers retain
|
||||||
|
the raw format. Logical envelope/spec/chunk limits still apply; chunking does
|
||||||
|
not authorize larger requests or change stdin/event sequence numbers. Framing
|
||||||
|
errors close the connection and discard incomplete assemblies; durable replay
|
||||||
|
uses the existing operation IDs. Check actual deployed capabilities and proxy
|
||||||
|
subprotocol headers before attributing behavior to this development feature.
|
||||||
|
|
||||||
## Windows identity, GUI, and shells
|
## Windows identity, GUI, and shells
|
||||||
|
|
||||||
`--elevated` is the only execution-context request toggle. Selection and
|
`--elevated` is the only execution-context request toggle. Selection and
|
||||||
|
|||||||
@@ -1157,6 +1157,72 @@ layer = "integration"
|
|||||||
status = "implemented"
|
status = "implemented"
|
||||||
tests = ["test/integration/store/store_integration_test.go:TestIncidentDirtyResolutionIdempotencyAndRecurrence_HP_STORE_11"]
|
tests = ["test/integration/store/store_integration_test.go:TestIncidentDirtyResolutionIdempotencyAndRecurrence_HP_STORE_11"]
|
||||||
|
|
||||||
|
[[requirements]]
|
||||||
|
id = "HP-CHUNK-01"
|
||||||
|
layer = "unit"
|
||||||
|
status = "implemented"
|
||||||
|
tests = ["internal/agentwire/packet_test.go:TestMessageRoundTripAndBoundaries_HP_CHUNK_01"]
|
||||||
|
|
||||||
|
[[requirements]]
|
||||||
|
id = "HP-CHUNK-02"
|
||||||
|
layer = "unit"
|
||||||
|
status = "implemented"
|
||||||
|
tests = ["internal/agentwire/connection_test.go:TestNegotiationFallbackAndBackgroundPong_HP_CHUNK_02"]
|
||||||
|
|
||||||
|
[[requirements]]
|
||||||
|
id = "HP-CHUNK-03"
|
||||||
|
layer = "integration"
|
||||||
|
status = "implemented"
|
||||||
|
tests = ["test/integration/clientagent/websocket_limits_integration_test.go:TestChunkedStdinKeepsWriteSequence_HP_CHUNK_03"]
|
||||||
|
|
||||||
|
[[requirements]]
|
||||||
|
id = "BH-CHUNK-01"
|
||||||
|
layer = "unit"
|
||||||
|
status = "implemented"
|
||||||
|
tests = ["internal/agentwire/packet_test.go:TestRejectMalformedPacketsWithoutAllocation_BH_CHUNK_01"]
|
||||||
|
|
||||||
|
[[requirements]]
|
||||||
|
id = "BH-CHUNK-02"
|
||||||
|
layer = "unit"
|
||||||
|
status = "implemented"
|
||||||
|
tests = ["internal/agentwire/limits_test.go:TestLogicalSendCeilingBeforeAnyPacket_BH_CHUNK_02"]
|
||||||
|
|
||||||
|
[[requirements]]
|
||||||
|
id = "BH-CHUNK-03"
|
||||||
|
layer = "unit"
|
||||||
|
status = "implemented"
|
||||||
|
tests = ["internal/agentwire/limits_test.go:TestMalformedWireClosesWithoutDelivery_BH_CHUNK_03"]
|
||||||
|
|
||||||
|
[[requirements]]
|
||||||
|
id = "REC-CHUNK-01"
|
||||||
|
layer = "unit"
|
||||||
|
status = "implemented"
|
||||||
|
tests = ["internal/agentwire/packet_test.go:TestIndependentControlLaneAndDisconnect_REC_CHUNK_01"]
|
||||||
|
|
||||||
|
[[requirements]]
|
||||||
|
id = "REC-CHUNK-02"
|
||||||
|
layer = "unit"
|
||||||
|
status = "implemented"
|
||||||
|
tests = ["internal/agentwire/connection_test.go:TestInterruptedPacketNeverDelivered_REC_CHUNK_02"]
|
||||||
|
|
||||||
|
[[requirements]]
|
||||||
|
id = "RACE-CHUNK-01"
|
||||||
|
layer = "unit"
|
||||||
|
status = "implemented"
|
||||||
|
tests = ["internal/agentwire/connection_test.go:TestConcurrentLogicalWrites_RACE_CHUNK_01"]
|
||||||
|
|
||||||
|
[[requirements]]
|
||||||
|
id = "RACE-CHUNK-02"
|
||||||
|
layer = "unit"
|
||||||
|
status = "implemented"
|
||||||
|
tests = ["internal/server/session/chunk_writer_test.go:TestChunkedWriterPreemptionAndBarrier_RACE_CHUNK_02"]
|
||||||
|
|
||||||
|
[[requirements]]
|
||||||
|
id = "RACE-CHUNK-03"
|
||||||
|
layer = "unit"
|
||||||
|
status = "implemented"
|
||||||
|
tests = ["internal/agentwire/limits_test.go:TestCloseStopsBlockedReadPump_RACE_CHUNK_03"]
|
||||||
|
|
||||||
[[requirements]]
|
[[requirements]]
|
||||||
id = "REC-WS-01"
|
id = "REC-WS-01"
|
||||||
layer = "integration"
|
layer = "integration"
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"github.com/coder/websocket"
|
||||||
rvboxv1 "github.com/rvbox/rvbox/gen/go/rvbox/v1"
|
rvboxv1 "github.com/rvbox/rvbox/gen/go/rvbox/v1"
|
||||||
"github.com/rvbox/rvbox/internal/agentproto"
|
"github.com/rvbox/rvbox/internal/agentproto"
|
||||||
"github.com/rvbox/rvbox/internal/client/agent"
|
"github.com/rvbox/rvbox/internal/client/agent"
|
||||||
@@ -99,10 +100,12 @@ func TestLargeScriptFitsLegacyReader_REC_WS_02(t *testing.T) {
|
|||||||
fixture := newFrameFixture(t)
|
fixture := newFrameFixture(t)
|
||||||
legacy := agentproto.DefaultLimits()
|
legacy := agentproto.DefaultLimits()
|
||||||
legacy.MaxEnvelopeBytes = 32 << 10
|
legacy.MaxEnvelopeBytes = 32 << 10
|
||||||
transport, err := agent.DialWebSocketWithLimits(ctx, "wss"+strings.TrimPrefix(fixture.server.URL, "https")+"/v1/agent", fixture.server.Client(), legacy)
|
raw, _, err := websocket.Dial(ctx, "wss"+strings.TrimPrefix(fixture.server.URL, "https")+"/v1/agent", &websocket.DialOptions{HTTPClient: fixture.server.Client()})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
|
raw.SetReadLimit(32 << 10)
|
||||||
|
transport := legacyAgentTransport{raw}
|
||||||
defer transport.Close()
|
defer transport.Close()
|
||||||
accepted, err := agent.Handshake(ctx, transport, fixture.hello, legacy)
|
accepted, err := agent.Handshake(ctx, transport, fixture.hello, legacy)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -170,6 +173,46 @@ func bytesToSHA(value []byte) [sha256.Size]byte {
|
|||||||
return result
|
return result
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestChunkedStdinKeepsWriteSequence_HP_CHUNK_03(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
fixture := newFrameFixture(t)
|
||||||
|
transport, accepted := fixture.connect(t, ctx, &rvboxv1.ReconcileSnapshot{})
|
||||||
|
dispatch := fixture.dispatch(t, ctx, transport, "more")
|
||||||
|
data := bytes.Repeat([]byte("stdin"), 12000)
|
||||||
|
request, err := domain.NewUUIDv7()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
input := &rvboxv1.AppendStdinRequest{ClientId: fixture.hello.ClientId, IssueUuid: dispatch.IssueUuid,
|
||||||
|
RequestId: request.String(), Data: data, AppendNewline: true}
|
||||||
|
first, err := fixture.control.AppendStdin(ctx, input)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
duplicate, err := fixture.control.AppendStdin(ctx, input)
|
||||||
|
if err != nil || duplicate.WriteSeq != first.WriteSeq {
|
||||||
|
t.Fatal("chunking changed stdin idempotency")
|
||||||
|
}
|
||||||
|
write := readFrame(t, ctx, transport).GetStdinWrite()
|
||||||
|
if write == nil || write.WriteSeq != first.WriteSeq || !write.AppendNewline || !bytes.Equal(write.Data, data) {
|
||||||
|
t.Fatal("chunking split a logical stdin write or changed its bytes")
|
||||||
|
}
|
||||||
|
if err := agent.SendCommandEvent(ctx, transport, accepted, &rvboxv1.CommandEvent{
|
||||||
|
IssueUuid: dispatch.IssueUuid, EventSeq: 1, ObservedAt: timestamppb.Now(),
|
||||||
|
Payload: &rvboxv1.CommandEvent_StdinAck{StdinAck: &rvboxv1.StdinAcknowledgement{WriteSeq: first.WriteSeq}},
|
||||||
|
}, agentproto.DefaultLimits()); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if ack := readFrame(t, ctx, transport).GetEventAck(); ack == nil || ack.ThroughEventSeq != 1 {
|
||||||
|
t.Fatal("logical stdin acknowledgement changed")
|
||||||
|
}
|
||||||
|
if pending, err := fixture.persistence.PendingStdin(ctx, fixture.hello.ClientId); err != nil || len(pending) != 0 {
|
||||||
|
t.Fatalf("stdin intent did not drain: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
type frameFixture struct {
|
type frameFixture struct {
|
||||||
persistence *store.Store
|
persistence *store.Store
|
||||||
control *control.Service
|
control *control.Service
|
||||||
@@ -219,7 +262,7 @@ func (fixture *frameFixture) connect(t *testing.T, ctx context.Context, snapshot
|
|||||||
return transport, accepted
|
return transport, accepted
|
||||||
}
|
}
|
||||||
|
|
||||||
func (fixture *frameFixture) dispatch(t *testing.T, ctx context.Context, transport *agent.WebSocketTransport, text string) *rvboxv1.CommandDispatch {
|
func (fixture *frameFixture) dispatch(t *testing.T, ctx context.Context, transport agent.Transport, text string) *rvboxv1.CommandDispatch {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
if _, err := fixture.control.RunCommand(ctx, &rvboxv1.RunCommandRequest{TargetClientId: fixture.hello.ClientId,
|
if _, err := fixture.control.RunCommand(ctx, &rvboxv1.RunCommandRequest{TargetClientId: fixture.hello.ClientId,
|
||||||
Spec: &rvboxv1.ExecutionSpec{ShellType: rvboxv1.ShellType_SHELL_CMD, Source: &rvboxv1.ExecutionSpec_CommandText{CommandText: text}},
|
Spec: &rvboxv1.ExecutionSpec{ShellType: rvboxv1.ShellType_SHELL_CMD, Source: &rvboxv1.ExecutionSpec_CommandText{CommandText: text}},
|
||||||
@@ -244,7 +287,7 @@ func (fixture *frameFixture) dispatch(t *testing.T, ctx context.Context, transpo
|
|||||||
return dispatch
|
return dispatch
|
||||||
}
|
}
|
||||||
|
|
||||||
func readFrame(t *testing.T, ctx context.Context, transport *agent.WebSocketTransport) *rvboxv1.AgentEnvelope {
|
func readFrame(t *testing.T, ctx context.Context, transport agent.Transport) *rvboxv1.AgentEnvelope {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
encoded, err := transport.Read(ctx)
|
encoded, err := transport.Read(ctx)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -256,3 +299,19 @@ func readFrame(t *testing.T, ctx context.Context, transport *agent.WebSocketTran
|
|||||||
}
|
}
|
||||||
return envelope
|
return envelope
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Deliberately omits subprotocol negotiation and the new wire wrapper.
|
||||||
|
type legacyAgentTransport struct{ connection *websocket.Conn }
|
||||||
|
|
||||||
|
func (transport legacyAgentTransport) Read(ctx context.Context) ([]byte, error) {
|
||||||
|
_, payload, err := transport.connection.Read(ctx)
|
||||||
|
return payload, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (transport legacyAgentTransport) Write(ctx context.Context, payload []byte) error {
|
||||||
|
return transport.connection.Write(ctx, websocket.MessageBinary, payload)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (transport legacyAgentTransport) Close() error {
|
||||||
|
return transport.connection.Close(websocket.StatusNormalClosure, "")
|
||||||
|
}
|
||||||
Reference in new issue
Block a user