fix: harden sharkd replay sessions and serve the uncapped frame list

Review-driven session/transport fixes (each reproduced live against
sharkd 4.6.7 before fixing):

- raise the RPC stream limit to 16 MB: a full 1000-row frames page
  measures ~190 KB against the 64 KB StreamReader default, which failed
  the request with a 500 and desynchronized the resident session; a
  line-over-limit ValueError is now treated as a transport failure
- verify JSON-RPC reply ids: a timed-out request's late reply was
  served as the next request's answer; timeouts, dead pipes, malformed
  and stale replies now kill the session for good instead
- make check-spawn atomic under one manager lock: concurrent requests
  for the same pcap double-spawned sharkd and leaked the loser (process
  plus /tmp scratch copy) forever
- refcount sessions and evict idle only (LRU, cap raised 8 -> 16): a
  tag with more sources than the cap respawned every source on every
  request, and concurrent requests could get their session killed
  mid-RPC (spurious 502)
- map FilterError to sharkd's filter rejection (-13002) only; other
  engine failures with a filter set are 502, not a client 400
- detail: accept an optional frame_number to disambiguate
  same-microsecond frames (ts is not unique within a pcap); drop the
  -8003 -> 404 mapping (the range is validated locally, engine errors
  are real faults); a failed hex read is a 404 instead of "hex": null
- a pcap deleted mid-request is a 404, not a 500; the pcap-sized
  scratch copy runs off the event loop; server shutdown kills every
  resident session and drops its scratch directory
- pin the packet-list layout through scratch-HOME Wireshark
  preferences: the column indexes are a contract the server owns
  (protocol-level column negotiation is rejected by sharkd 4.6.x)

Range contract change (WebUI moved to an always-flat list): the merged
frame list is returned in full, deliberately uncapped - truncated and
per-second buckets are removed, frame_count always equals
len(frames), and rendering cost is the client's concern (the window
endpoint remains the incremental path).

docs/features/marker-tag-replay.md

@@ -41,7 +41,7 @@ graph TB
     TMP["/tmp scratch copies<br/>(hardened-profile workaround)"]
     SK["sharkd -<br/>(resident JSON-RPC on stdio)"]
 
-    UI -->|"GET range / frames [?filter=]"| GATE
+    UI -->|"GET range / frames [?filter=&link=]"| GATE
     GATE --> SCAN
     SCAN -->|"columns / filter matches"| SESS
     SESS --> TMP --> SK
@@ -60,11 +60,44 @@ Backbone vs engine, deliberately separated:
 
 Session lifecycle: each session is validated per request against the source pcap's
 `(mtime, size)` — a mismatch (e.g. the capture node restarted and uBridge truncated the
-pcap while paused) kills and respawns it. Sessions are LRU-bounded (8), each RPC has a
-timeout and is serialized by a per-session lock (sharkd serves one request at a time).
-sharkd reads a `/tmp` scratch copy of the pcap with a scratch `HOME` — hardened profiles
-(AppArmor &c.) deny it the project directory and the user's home even though the server
-process can read both.
+pcap while paused) kills and respawns it. Sessions are refcounted while a request holds
+them and the LRU bound (16) only ever evicts **idle** sessions, so neither a request's
+own walk over a tag with more sources than the bound nor a concurrent request can have
+its session killed mid-RPC. A single manager lock makes check-spawn atomic (concurrent
+requests share one spawn instead of double-spawning a process nobody reaps). Each RPC
+is serialized by a per-session lock (sharkd serves one request at a time), carries a
+timeout and an **id check** — any transport failure (timeout, dead pipe, malformed or
+stale reply, oversized line) kills the session for good, because a desynchronized
+session would serve shifted results. sharkd reads a `/tmp` scratch **directory**
+containing a copy of the pcap plus pinned Wireshark preferences (the column layout the
+frames RPC is parsed against is a contract the server owns, immune to system or user
+column customization) — hardened profiles (AppArmor &c.) deny it the project directory
+and the user's home even though the server process can read both. The pcap copy runs
+off the event loop, and server shutdown kills every session so no scratch copy leaks
+into `/tmp` across restarts.
+
+### Resource bounds
+
+Three independent bounds, easily confused:
+
+| Bound | Value | Governs |
+|-------|-------|---------|
+| `_STREAM_LIMIT_BYTES` | 16 MB | Max **single JSON-RPC reply line** read from sharkd. A full 1000-row page measures ~190 KB against asyncio's 64 KB default — the trigger is the frame count in one pcap (≥ 1000), never the number of pcaps |
+| `_FRAMES_PAGE` | 1000 | Rows fetched per `frames` RPC while draining one source's columns |
+| `SESSION_MAX` | 16 | Resident **idle** sharkd sessions (LRU eviction) |
+
+The `range` frame list itself is uncapped by decision — the client owns the
+rendering cost of a huge list.
+
+Process count follows usage, not the pcap inventory. sharkd loads one file per
+process, so a session exists per source pcap **actually consulted** — a replay
+request only touches its own tag's sources, and a session is either in use
+(refcounted, held for a whole pagination drain rather than per RPC) or idle
+(no holder — the only state the LRU cap ever evicts). At any moment the pool
+holds at most `SESSION_MAX` idle sessions plus one per source being drained by
+an in-flight request: a project with a thousand marker pcaps still runs at
+most `16 + Σ(in-flight sources)` sharkd processes. Each process holds its pcap
+in memory (~file size), which is the quantity the session cap primarily bounds.
 
 ## Business Process
 
@@ -79,12 +112,12 @@ sequenceDiagram
     Note over UI: ③ pause every marker under the tag
 
     Note over UI,SK: ④ replay
-    UI->>C: GET /markers/tags/666/replay/range[?filter=…]
+    UI->>C: GET /markers/tags/666/replay/range[?filter=…&link=…]
     C->>C: gate → scan record headers → merge order
     C->>SK: frames {filter, skip, limit} → columns + matches
     C-->>UI: {start, end, sources, frames[] with src/dst/proto/info/bg/fg}
     UI->>C: GET frames?ts=T&window_ms=W  (paging — {"frames": []} on a miss)
-    UI->>C: GET frame/detail?ts=…&node_id=…&link_id=…&marker=…
+    UI->>C: GET frame/detail?ts=…&node_id=…&link_id=…&marker=…[&frame_number=…]
     C->>C: hex straight from the pcap
     C->>SK: frame {frame: N, proto: true}
     SK-->>C: tree (keys renamed to the REST contract)
@@ -126,7 +159,7 @@ without it.
 |--------|------|-------------|
 | GET | `/v3/projects/{pid}/markers/tags/{tag}/replay/range[?filter=&link=]` | Timeline metadata + full merged frame list with packet-list columns |
 | GET | `/v3/projects/{pid}/markers/tags/{tag}/replay/frames?ts=&window_ms=&limit=[&filter=&link=]` | Frames with ts in `[T, T+window]`, merged across sources |
-| GET | `/v3/projects/{pid}/markers/tags/{tag}/replay/frame/detail?ts=&node_id=&link_id=&marker=` | Single frame: protocol tree + raw hex (lazy — one call per frame the user opens) |
+| GET | `/v3/projects/{pid}/markers/tags/{tag}/replay/frame/detail?ts=&node_id=&link_id=&marker=[&frame_number=]` | Single frame: protocol tree + raw hex (lazy — one call per frame the user opens) |
 
 ### `range` — the timeline
 
@@ -136,7 +169,6 @@ without it.
   "start": "1788369209.406812",
   "end": "1788369219.249085",
   "frame_count": 29,
-  "truncated": false,
   "sources": [
     { "node_id": "47703cad…", "link_id": "2697a7c6…", "marker": "global-ospf",
       "data_link_type": "DLT_EN10MB", "count": 4 }
@@ -152,9 +184,10 @@ without it.
 }
 ```
 
-- `frames` is the **full, merged, time-ordered list** (cap 5000) — one request lays out
-  the whole timeline. Over the cap, `frames` is omitted and per-second `buckets` are
-  returned instead with `truncated: true` (check with `'frames' in response`, not null).
+- `frames` is the **full, merged, time-ordered list** — one request lays out the whole
+  timeline, **deliberately uncapped**: rendering a huge list is the client's concern,
+  and the window endpoint exists for incremental views. There is no `truncated` flag
+  and no bucket histogram; `frame_count` always equals `len(frames)`.
 - Every frame entry carries the Wireshark packet-list columns — `src` / `dst` / `proto`
   / `info` — plus the **coloring-rule hints `bg` / `fg`** (Wireshark's own palette
   decisions, so the UI can color rows exactly like Wireshark without shipping the
@@ -166,12 +199,14 @@ without it.
 ### Display filter
 
 `?filter=<expression>` on both `range` and `frames` is a Wireshark display filter,
-applied **before** counting and slicing — `start` / `end` / `frame_count` /
-`frames` | `buckets` are all computed on the matching frames only. Filtered frames keep
+applied **before** counting and slicing — `start` / `end` / `frame_count` / `frames`
+are all computed on the matching frames only. Filtered frames keep
 their original pcap frame numbers. The filter travels as one argv-style element (never
 through a shell) and is capped at 2000 characters. An invalid expression is a **400**
 whose message carries sharkd's original error text — suitable for inline display in the
-filter bar, and distinct from the 409 gate / 404 unknown-tag semantics.
+filter bar, and distinct from the 409 gate / 404 unknown-tag semantics. Only sharkd's
+filter-rejection error maps to 400; any other engine failure while a filter is set is a
+502 (the filter was fine — the engine was not).
 
 ### Capture-source selection
 
@@ -185,8 +220,7 @@ design:
   listed, with engine-free **total** counts, unaffected by `link` / `filter` — a
   source dropdown must not shrink when the view narrows.
 - **An unknown `link_id` matches nothing**: `frame_count: 0`, `start: null`, empty
-  `frames` / `buckets` — the same shape as a zero-match display filter, deliberately
-  not a 404.
+  `frames` — the same shape as a zero-match display filter, deliberately not a 404.
 
 ### `frames` — point / window query (paging)
 
@@ -209,7 +243,10 @@ Invoked only when the user opens a frame. The `ts` must be the **exact string re
 in the timeline/frame list** (round-tripped verbatim — never re-serialized through a
 float); `node_id + link_id + marker` identify the pcap. The server re-resolves the ts
 against the file, guarding against a capture rebuilt between the timeline view and this
-click.
+click. Since ts is **not unique within one pcap** (same-microsecond frames are kept
+deliberately), the optional `frame_number` — carried by every frame list entry —
+disambiguates them: it must still land on the exact ts, and without it the first ts
+match decodes.
 
 ```json
 {
@@ -268,12 +305,12 @@ All error bodies are `{"message": "…"}` (the app's unified format).
 
 | Status | Description |
 |--------|-------------|
-| 400 | Invalid display filter (message carries sharkd's original text) or filter longer than 2000 chars |
+| 400 | Invalid display filter (sharkd's filter rejection only; message carries its original text) or filter longer than 2000 chars |
 | 401 | Not authenticated |
-| 404 | Tag has no markers in the project; detail source unknown, or ts does not match the file (the capture may have been rebuilt) |
+| 404 | Tag has no markers in the project; detail source unknown; `ts`/`frame_number` do not match the file (the capture may have been rebuilt); or the pcap vanished mid-request |
 | 409 | Tag gate: a marker under the tag is still `enabled: true` (the response lists them) |
 | 501 | sharkd not installed / unavailable — replay is unavailable, no degraded mode |
-| 502 | sharkd failed or timed out (10 s per RPC) |
+| 502 | sharkd failed, timed out (10 s per RPC), or answered out of sync — the session is killed and re-spawned on the next request |
 
 ## Notes
 
@@ -293,7 +330,16 @@ All error bodies are `{"message": "…"}` (the app's unified format).
   natural follow-up if list latency ever matters at many-source scale.
 - **Session invalidation is cheap and total.** Every request stats the source pcap; a
   rewritten file (mtime/size change) respawns the session — a paused-but-restarted
-  capture can never serve stale dissect state.
+  capture can never serve stale dissect state. Transport failures are equally total: a
+  timeout, dead pipe, malformed or stale reply (id mismatch) kills the session instead
+  of leaving a poisoned one resident, and concurrent requests for the same pcap share a
+  single spawn.
+- **The packet-list layout is pinned, not assumed.** sharkd runs with a scratch `HOME`
+  whose Wireshark preferences fix `gui.column.format` to exactly the four columns the
+  frame entries carry (personal config overrides any `/etc/wireshark` customization), so
+  the column indexes the server parses are a contract it owns.
+- **Server shutdown kills every resident session** and drops its `/tmp` scratch
+  directory — nothing accumulates across restarts.
 - **Tag type.** REST and the `marker.match` WS event both carry `tag` as `int` (the
   listener normalizes); replay keys on that int value.
 - **Follow-ups.** Remote-compute support via the existing capture-file proxy pattern;