fix: bind client job events to command leases

This commit is contained in:
2026-08-13 07:09:03 +00:00
parent 62a0f24437
commit 0311f6f084
29 changed files with 288 additions and 93 deletions
+15 -12
View File
@@ -45,8 +45,8 @@ seconds. A connection stable for 60 seconds resets the backoff.
## Version and feature negotiation
The envelope and registration both state the sender version. Major `1` accepts
only major `1`; a different major is rejected. The negotiated minor is the
The envelope and registration both state the sender version. Major `2` accepts
only major `2`; a different major is rejected. The negotiated minor is the
highest mutually supported minor no greater than either endpoint's advertised
minor.
@@ -96,19 +96,22 @@ client state snapshot before a route can become ready.
Every mutating command includes an expected job revision or immutable job
definition plus the expected last global event sequence. A stale revision or
sequence is rejected without side effects. The control daemon sends only one
active step command for a job and waits for its persisted completion event
before commanding the next participant. This single-writer lease lets whichever
client owns the current step allocate the next global per-job event sequence;
the following command starts from the sequence control has durably accepted.
sequence is rejected without side effects. Assignment is represented solely by
the durable assignment `CommandAck`; it produces no `JobEvent`. The control
daemon sends only one active step command for a job and waits for its persisted
completion event before commanding the next participant.
## Events, progress, and reconciliation
Each client-originated job event has a unique ID, monotonically increasing
global per-job `sequence`, and resulting job revision. Duplicate IDs/sequences
are idempotent only when their complete content matches. Control never grants
concurrent event-writer leases for one job. A gap or conflicting duplicate
pauses destructive orchestration and requests snapshots.
global per-job `sequence`, resulting job revision, and the `command_id` of its
owning `ExecuteStepCommand` or `CancelJobCommand`. Duplicate IDs/sequences are
idempotent only when their complete content matches. Control verifies the
client, acknowledged command lease, expected cursor, and active step before
accepting it. A gap, conflict, or stale lease triggers a durable authoritative
`ReconcileJobCommand`: the client retires only the named stale command leases
and restores that job cursor, without deleting resource data or unrelated job
state.
`fraction_complete` is current-step progress and
`overall_fraction_complete` is the weighted five- or three-step job progress;
@@ -120,7 +123,7 @@ On registration, active-job cursors provide the client's revision, last event
sequence, state, and commit flag. Reconciliation applies these rules:
1. Equal cursors resume normal delivery.
2. A client behind receives safe replay/snapshot commands.
2. A client behind receives an authoritative reconciliation command.
3. Control behind requests and validates the client's full job snapshot.
4. Conflicting commit evidence reserves the resource and requires manual
reconciliation; neither side performs cleanup.