feat: drive marker replay with resident sharkd sessions

sharkd (the Wireshark daemon) is now the single decode engine — the
tshark/PDML path is gone, and without sharkd every replay endpoint
returns 501 (no degraded mode: one engine, one rendering shape for the
Web UI).

- Frame entries gain packet-list columns from sharkd's frames RPC:
  src/dst/proto/info plus the Wireshark coloring hints bg/fg
- range and frames accept ?filter=<display filter>, applied before
  counting and slicing; invalid expressions are 400 carrying sharkd's
  original text; filters travel as single argv-style elements, capped
  at 2000 chars; filtered frames keep their original pcap frame numbers
- frame detail returns sharkd's protocol tree with keys renamed into
  the REST contract (element/label/name/filter_expr/pos+size/expert/
  generated/children): a census-verified closed key set, values
  untouched, unknown keys passed through verbatim, Wireshark-internal
  hf ids dropped. filter_expr gives the UI click-to-filter; pos/size
  drives hex highlighting (hex still read straight from the pcap)
- one resident 'sharkd -' session per source pcap: lazy spawn, /tmp
  scratch copy + scratch HOME (hardened profiles), per-request
  (mtime,size) validation with respawn, LRU bound, per-session lock,
  per-RPC timeout, bounded close

Timeline backbone (gate, record-header scan, merge ordering, canonical
ts strings, hex reads) stays plain Python — identity and ordering never
depend on the engine.

docs/features/marker-tag-replay.md

@@ -12,14 +12,18 @@ See LICENSE file for licensing information.
 Replays traffic captured by [markers](marker-traffic-insight.md) **across links**, keyed by
 `tag`. Markers on different links that share a tag form one *distributed capture session*;
 once every marker under the tag is paused, their per-marker pcaps are merged into a single
-timestamp-ordered timeline. The Web UI browses that timeline and fetches individual frames
-on demand — each fetch decodes exactly one frame via `tshark` into a self-describing JSON
-protocol tree.
+timestamp-ordered timeline. The Web UI browses that timeline (with Wireshark-style packet
+list columns) and fetches individual frames on demand — each fetch decodes exactly one
+frame via the resident **sharkd** daemon into a self-describing JSON protocol tree.
 
 The unique observable: the delta between the same packet hitting two consecutive links
 measures the **intermediate node's forwarding latency** (host view) — something a
 single-link capture can never show.
 
+**sharkd is a hard requirement** (part of the Wireshark package). Without it every replay
+endpoint returns 501 — there is deliberately no degraded mode; one engine, one rendering
+shape for the Web UI.
+
 ## Architecture
 
 ```mermaid
@@ -28,35 +32,38 @@ graph TB
 
     subgraph Controller["Controller (replay endpoints)"]
         GATE["Tag gate<br/>(all markers under tag paused?)"]
-        SCAN["Timeline scan<br/>(pcap record headers)"]
-        MAP["PDML → JSON<br/>isomorphic mapper"]
+        SCAN["Timeline backbone<br/>(pcap record-header scan,<br/>merge ordering, hex reads)"]
+        SESS["sharkd sessions<br/>(one per source pcap)"]
     end
 
     FS[("markers dir<br/>{node}_{link}_{marker}.pcap")]
-    TS["tshark -T pdml<br/>(one frame at a time)"]
-    TMP["/tmp scratch copy<br/>(hardened-profile workaround)"]
+    TMP["/tmp scratch copies<br/>(hardened-profile workaround)"]
+    SK["sharkd -<br/>(resident JSON-RPC on stdio)"]
 
-    UI -->|"GET range / frames"| GATE
+    UI -->|"GET range / frames [?filter=]"| GATE
     GATE --> SCAN
-    SCAN --> FS
-    UI -->|"GET frame detail (lazy)"| MAP
-    MAP --> TMP
-    TMP --> TS
-    MAP -->|"hex: raw bytes"| FS
+    SCAN -->|"columns / filter matches"| SESS
+    SESS --> TMP --> SK
+    SCAN -->|"hex: raw bytes"| FS
+    UI -->|"GET frame detail (lazy)"| SESS
 ```
 
-Two deliberately separated performance regimes:
+Backbone vs engine, deliberately separated:
 
-| Path | Work | tshark |
-|------|------|--------|
-| Timeline | 16-byte record-header scan per frame, cross-file merge sort | **never invoked** — browsing works without tshark |
-| Frame detail | locate frame → `tshark -T pdml -Y "frame.number == N"` → map to JSON | forked once per frame the user opens |
+- **Timeline backbone** (plain Python): the tag gate, 16-byte-per-frame pcap record-header
+  scan, cross-source merge ordering, canonical ts strings, and raw-bytes reads for the hex
+  view. Identity and ordering never depend on the engine.
+- **Engine layer (sharkd)**: packet-list columns, display filters, and per-frame protocol
+  trees. One resident `sharkd -` process per source pcap (sharkd loads one file at a
+  time), spawned lazily on first use, addressed with one-line JSON-RPC on stdio.
 
-tshark call count equals user clicks — no caching or rate limiting needed, and a tshark
-failure (501/502) affects only that one frame, never timeline browsing. pcap files are
-compute-side (`<project>/project-files/markers/`); the initial scope is single-server
-deployments (controller and compute in one process, direct file access) — remote computes
-will reuse the existing capture-file proxy pattern.
+Session lifecycle: each session is validated per request against the source pcap's
+`(mtime, size)` — a mismatch (e.g. the capture node restarted and uBridge truncated the
+pcap while paused) kills and respawns it. Sessions are LRU-bounded (8), each RPC has a
+timeout and is serialized by a per-session lock (sharkd serves one request at a time).
+sharkd reads a `/tmp` scratch copy of the pcap with a scratch `HOME` — hardened profiles
+(AppArmor &c.) deny it the project directory and the user's home even though the server
+process can read both.
 
 ## Business Process
 
@@ -64,29 +71,22 @@ will reuse the existing capture-file proxy pattern.
 sequenceDiagram
     participant UI as Web UI
     participant C as Controller
-    participant PC as markers dir (pcaps)
-    participant TS as tshark
+    participant SK as sharkd session
 
-    Note over UI,PC: ① configure — same tag on every link's marker
-    UI->>C: POST markers (bpf, tag=666) on each link
+    Note over UI: ① configure — same tag on every link's marker
+    Note over C: ② capture — uBridge appends matches, replay forbidden (409)
+    Note over UI: ③ pause every marker under the tag
 
-    Note over PC: ② capture — uBridge appends matches (flushed per packet), replay forbidden
-    UI->>C: GET range
-    C--xC: 409 (a marker under tag 666 is still enabled)
-
-    Note over UI: ③ pause — every marker under the tag
-    UI->>C: PUT markers {"enabled": false} × each
-
-    Note over UI,TS: ④ replay
-    UI->>C: GET /markers/tags/666/replay/range
-    C->>PC: scan record headers, merge sort
-    C-->>UI: {start, end, sources, frames}
-    UI->>C: GET frames?ts=T&window_ms=W
-    C-->>UI: frames in [T, T+W] — or {"frames": []}
+    Note over UI,SK: ④ replay
+    UI->>C: GET /markers/tags/666/replay/range[?filter=…]
+    C->>C: gate → scan record headers → merge order
+    C->>SK: frames {filter, skip, limit} → columns + matches
+    C-->>UI: {start, end, sources, frames[] with src/dst/proto/info/bg/fg}
+    UI->>C: GET frames?ts=T&window_ms=W  (paging — {"frames": []} on a miss)
     UI->>C: GET frame/detail?ts=…&node_id=…&link_id=…&marker=…
-    C->>PC: locate frame, read raw bytes (hex)
-    C->>TS: -T pdml (reads the /tmp copy)
-    TS-->>C: PDML
+    C->>C: hex straight from the pcap
+    C->>SK: frame {frame: N, proto: true}
+    SK-->>C: tree (keys renamed to the REST contract)
     C-->>UI: protocol tree + hex
 ```
 
@@ -110,25 +110,22 @@ markers at all → 404.
 - **Pause → resume → pause is fine.** The pcap accumulates the full history; replay covers
   everything up to the current pause point.
 - **The replay window ends when nodes restart.** A pcap's lifetime equals its uBridge's
-  lifetime: a fresh uBridge reinstalls every desired marker — paused ones too (install
-  first, then turn the filter off) — and uBridge opens the pcap with truncate semantics
-  (`pcap_dump_open`, not `_append`). Server restart + project reopen **without starting
-  nodes** is safe: nothing touches the files until a uBridge comes up (verified live).
-  Docker nodes effectively restart on server restart as well (stale-container cleanup),
-  so their window is shorter still.
-- uBridge flushes every matched packet to the pcap immediately (`pcap_dump_flush` per
-  packet under a mutex — verified in the uBridge source), so a pause boundary never loses
-  tail frames.
+  lifetime: a fresh uBridge reinstalls every desired marker — paused ones too — and
+  uBridge opens the pcap with truncate semantics (`pcap_dump_open`, not `_append`). Server
+  restart + project reopen **without starting nodes** is safe: nothing touches the files
+  until a uBridge comes up. Docker nodes effectively restart on server restart as well
+  (stale-container cleanup), so their window is shorter still.
 
 ## API Endpoints
 
-All read-only; JWT bearer token, privilege `Project.Audit`.
+All read-only; JWT bearer token, privilege `Project.Audit`. All require sharkd — 501
+without it.
 
 | Method | Path | Description |
 |--------|------|-------------|
-| GET | `/v3/projects/{pid}/markers/tags/{tag}/replay/range` | Timeline metadata + full frame list for the tag |
-| GET | `/v3/projects/{pid}/markers/tags/{tag}/replay/frames?ts=&window_ms=&limit=` | Frames with ts in `[T, T+window]`, merged across sources |
-| GET | `/v3/projects/{pid}/markers/tags/{tag}/replay/frame/detail?ts=&node_id=&link_id=&marker=` | Single frame: tshark protocol tree + raw hex |
+| GET | `/v3/projects/{pid}/markers/tags/{tag}/replay/range[?filter=]` | Timeline metadata + full merged frame list with packet-list columns |
+| GET | `/v3/projects/{pid}/markers/tags/{tag}/replay/frames?ts=&window_ms=&limit=[&filter=]` | Frames with ts in `[T, T+window]`, merged across sources |
+| GET | `/v3/projects/{pid}/markers/tags/{tag}/replay/frame/detail?ts=&node_id=&link_id=&marker=` | Single frame: protocol tree + raw hex (lazy — one call per frame the user opens) |
 
 ### `range` — the timeline
 
@@ -144,62 +141,95 @@ All read-only; JWT bearer token, privilege `Project.Audit`.
       "data_link_type": "DLT_EN10MB", "count": 10 }
   ],
   "frames": [
-    { "ts": "1788196663.226372", "len": 98, "node_id": "b764c434…",
-      "link_id": "316ef8fd…", "marker": "global-def-…", "frame_number": 1 }
+    { "ts": "1788196663.226372", "len": 98,
+      "node_id": "b764c434…", "link_id": "316ef8fd…",
+      "marker": "global-def-…", "frame_number": 1,
+      "src": "10.1.10.101", "dst": "203.0.113.1",
+      "proto": "ICMP", "info": "Echo (ping) request id=0x6ed5, seq=1/0, ttl=64",
+      "bg": "ffffff", "fg": "000000" }
   ]
 }
 ```
 
-- `frames` is the **full, merged, time-ordered list** (cap 5000) — the Web UI lays out the
-  whole timeline from one request. Over the cap, `frames` is omitted and per-second
-  `buckets` are returned instead with `truncated: true`.
+- `frames` is the **full, merged, time-ordered list** (cap 5000) — one request lays out
+  the whole timeline. Over the cap, `frames` is omitted and per-second `buckets` are
+  returned instead with `truncated: true` (check with `'frames' in response`, not null).
+- Every frame entry carries the Wireshark packet-list columns — `src` / `dst` / `proto`
+  / `info` — plus the **coloring-rule hints `bg` / `fg`** (Wireshark's own palette
+  decisions, so the UI can color rows exactly like Wireshark without shipping the
+  colorization engine). Columns are `null` only for a frame the engine could not describe.
 - Each frame entry carries `(node_id, link_id, marker, frame_number)` — the locating
-  tuple for the detail request.
+  tuple for the detail request, and the link association for timeline/topology rendering
+  (`link_id` joins the Web UI's own link objects).
 
-### `frames` — point / window query
+### Display filter
 
-A time with no frames is a normal, successful answer — an empty array, no sentinel strings:
+`?filter=<expression>` on both `range` and `frames` is a Wireshark display filter,
+applied **before** counting and slicing — `start` / `end` / `frame_count` /
+`frames` | `buckets` are all computed on the matching frames only. Filtered frames keep
+their original pcap frame numbers. The filter travels as one argv-style element (never
+through a shell) and is capped at 2000 characters. An invalid expression is a **400**
+whose message carries sharkd's original error text — suitable for inline display in the
+filter bar, and distinct from the 409 gate / 404 unknown-tag semantics.
+
+### `frames` — point / window query (paging)
+
+A time with no frames is a normal, successful answer — an empty array, no sentinel
+strings:
 
 ```json
 GET …/replay/frames?ts=1788196700.000&window_ms=500
 → { "frames": [] }
 ```
 
+Paging is deliberately **ts + window_ms only** (no offset/limit over the filtered set):
+the merge spans multiple pcaps, so slicing happens server-side on the merged stream
+either way, windows align with timeline semantics, and the gate freezes the data (the
+window answer is deterministic).
+
 ### `frame/detail` — lazy single-frame decode
 
-Invoked only when the user opens a frame. The `ts` must be the **exact string received in
-the timeline/frame list** (round-tripped verbatim — never re-serialized through a float);
-`node_id + link_id + marker` identify the pcap. The server re-resolves the ts against the
-file, guarding against a capture rebuilt between the timeline view and this click.
+Invoked only when the user opens a frame. The `ts` must be the **exact string received
+in the timeline/frame list** (round-tripped verbatim — never re-serialized through a
+float); `node_id + link_id + marker` identify the pcap. The server re-resolves the ts
+against the file, guarding against a capture rebuilt between the timeline view and this
+click.
 
 ```json
 {
   "ts": "1788196663.226372",
   "source": { "node_id": "b764c434…", "link_id": "316ef8fd…",
               "marker": "global-def-…", "frame_number": 1 },
-  "tshark_version": "TShark (Wireshark) 4.6.7 …",
   "field_count": 85,
-  "hex": "45000062…",
+  "hex": "00005e00010a…",
   "tree": [
-    { "element": "proto", "name": "ip",
-      "showname": "Internet Protocol Version 4, Src: 10.1.10.101, Dst: 203.0.113.1",
-      "children": [
-        { "element": "field", "name": "ip.ttl", "show": "64",
-          "showname": "Time to Live: 64", "value": "40", "size": "1",
-          "pos": "22", "children": [] }
+    { "element": "proto", "label": "Internet Protocol Version 4, …", "children": [
+        { "element": "field", "name": "ip.ttl", "label": "Time to Live: 64",
+          "filter_expr": "ip.ttl == 64", "pos": 22, "size": 1, "children": [] }
       ] }
   ]
 }
 ```
 
-- `tree` mirrors PDML **isomorphically**: every `<proto>`/`<field>` becomes a node, every
-  XML attribute (`name`, `show`, `showname`, `value`, `size`, `pos`, `hide`, `mask`,
-  `unmaskedvalue`) becomes a JSON key, plus one structural key `element` (proto/field).
-  Nothing is selected out, nothing interpreted, and **all values stay strings** — numeric
-  conversion is the client's business.
-- `hex` is the raw frame bytes read straight from the pcap (not via tshark); keeping
-  `pos`/`size` on every field enables Wireshark-style *click field → highlight bytes*.
-- `field_count` is the mapped node count — a client-side sanity check.
+The tree is sharkd's protocol tree with **keys renamed into the REST contract** — a
+closed, protocol-independent key set (census-verified across ICMP / TCP / VLAN+OSPF
+trees and pinned by a test), with values untouched:
+
+| sharkd | Contract key | Meaning |
+|--------|--------------|---------|
+| `t` | `element` | node type (`proto`, …) |
+| `l` | `label` | display text |
+| `fn` | `name` | field name (`ip.ttl`) |
+| `f` | `filter_expr` | **ready-made display filter with the value baked in** — click-to-filter |
+| `h` | `pos` + `size` | byte range — click field → highlight hex bytes |
+| `s` | `expert` | expert severity name (`Chat` / `Warn` / …) |
+| `g` | `generated` | generated-by-Wireshark flag |
+| `n` | `children` | nested fields |
+| `e` | *(dropped)* | Wireshark-internal hf id, unstable across versions |
+
+Unknown keys from a newer Wireshark pass through verbatim (never silently dropped); a
+census test flags new keys for naming. `hex` is the raw frame bytes read straight from
+the pcap; `field_count` is the mapped node count (client-side sanity check).
 
 ## Ordering and timestamps
 
@@ -209,45 +239,33 @@ file, guarding against a capture rebuilt between the timeline view and this clic
   environment.
 - The sort key is `(ts, source file, frame_number)` — ts alone is **not** unique (two
   links can hit the same microsecond); the tiebreaker yields a stable, determined order
-  instead of a fictional one. Index structures must never use ts as a dict key, or
-  same-microsecond frames silently overwrite each other.
+  instead of a fictional one.
 - The cross-link delta is the intermediate node's end-to-end forwarding latency
   (veth/TAP → guest protocol stack → back to host), typically hundreds of microseconds to
   milliseconds. UI labels should read *node forwarding latency (host view)*, not link
   propagation delay. A live capture pair confirmed it end-to-end: same `ip.id`,
   TTL 64→63, 509 µs between two links.
 
-## Fidelity guarantee (PDML → JSON)
-
-The conversion is an isomorphic structure map, not a semantic transform, with two rules:
-**map every attribute** and **keep values as strings**. A round-trip test enforces both:
-PDML element count equals JSON node count, and every XML attribute survives with an
-identical JSON value (`tests/controller/test_marker_replay.py`). The raw frame bytes —
-the one thing PDML genuinely does not contain — are covered by `hex` read directly from
-the pcap.
-
 ## Error Responses
 
 | Status | Description |
 |--------|-------------|
+| 400 | Invalid display filter (message carries sharkd's original text) or filter longer than 2000 chars |
 | 401 | Not authenticated |
 | 404 | Tag has no markers in the project; detail source unknown, or ts does not match the file (the capture may have been rebuilt) |
 | 409 | Tag gate: a marker under the tag is still `enabled: true` (the response lists them) |
-| 501 | tshark not installed / unavailable — affects detail only; the timeline never needs tshark |
-| 502 | tshark failed or timed out (10 s); truncated output never reaches the mapper |
+| 501 | sharkd not installed / unavailable — replay is unavailable, no degraded mode |
+| 502 | sharkd failed or timed out (10 s per RPC) |
 
 ## Notes
 
 - **Heterogeneous link types coexist.** Frames are never merged into a single pcap
   (mergecap is deliberately not used) — each frame carries its source and is decoded
   individually, so Ethernet and serial (cHDLC/PPP) markers can share one timeline.
-  Malformed packets are tshark's problem: it emits `[Malformed Packet]` as regular PDML
-  and carries on.
-- **Hardened tshark profiles.** openSUSE-style profiles (AppArmor &c.) can deny tshark
-  access to the project directory and the user's home even though the server process can
-  read both. The detail path therefore copies the pcap to a real file under `/tmp`
-  (not a symlink — the profile resolves real paths) and gives tshark a scratch `HOME`;
-  the copy is unlinked afterwards. The hex view still reads the original file.
+  Malformed packets are dissected like any other; sharkd marks them in the tree.
+- **Session invalidation is cheap and total.** Every request stats the source pcap; a
+  rewritten file (mtime/size change) respawns the session — a paused-but-restarted
+  capture can never serve stale dissect state.
 - **Tag type.** REST and the `marker.match` WS event both carry `tag` as `int` (the
   listener normalizes); replay keys on that int value.
 - **Follow-ups.** Remote-compute support via the existing capture-file proxy pattern;