fix: harden sharkd replay sessions and serve the uncapped frame list

Review-driven session/transport fixes (each reproduced live against
sharkd 4.6.7 before fixing):

- raise the RPC stream limit to 16 MB: a full 1000-row frames page
  measures ~190 KB against the 64 KB StreamReader default, which failed
  the request with a 500 and desynchronized the resident session; a
  line-over-limit ValueError is now treated as a transport failure
- verify JSON-RPC reply ids: a timed-out request's late reply was
  served as the next request's answer; timeouts, dead pipes, malformed
  and stale replies now kill the session for good instead
- make check-spawn atomic under one manager lock: concurrent requests
  for the same pcap double-spawned sharkd and leaked the loser (process
  plus /tmp scratch copy) forever
- refcount sessions and evict idle only (LRU, cap raised 8 -> 16): a
  tag with more sources than the cap respawned every source on every
  request, and concurrent requests could get their session killed
  mid-RPC (spurious 502)
- map FilterError to sharkd's filter rejection (-13002) only; other
  engine failures with a filter set are 502, not a client 400
- detail: accept an optional frame_number to disambiguate
  same-microsecond frames (ts is not unique within a pcap); drop the
  -8003 -> 404 mapping (the range is validated locally, engine errors
  are real faults); a failed hex read is a 404 instead of "hex": null
- a pcap deleted mid-request is a 404, not a 500; the pcap-sized
  scratch copy runs off the event loop; server shutdown kills every
  resident session and drops its scratch directory
- pin the packet-list layout through scratch-HOME Wireshark
  preferences: the column indexes are a contract the server owns
  (protocol-level column negotiation is rejected by sharkd 4.6.x)

Range contract change (WebUI moved to an always-flat list): the merged
frame list is returned in full, deliberately uncapped - truncated and
per-second buckets are removed, frame_count always equals
len(frames), and rendering cost is the client's concern (the window
endpoint remains the incremental path).

gns3server/controller/marker_replay.py

@@ -31,13 +31,18 @@ Python, but it is an implementation detail, not an availability promise.
 
 Session model — sharkd loads one file at a time, so there is one resident
 session **per source pcap**: spawned lazily on first use, fed a ``/tmp``
-scratch copy (hardened profiles deny sharkd the project directory; a real
-copy, not a symlink, since the profile resolves real paths) with a scratch
-``HOME``, validated per request against the original's ``(mtime, size)`` —
+scratch directory (hardened profiles deny sharkd the project directory; a
+real copy, not a symlink, since the profile resolves real paths) with a
+scratch ``HOME`` whose Wireshark preferences pin the packet-list column
+layout, validated per request against the original's ``(mtime, size)`` —
 a mismatch (e.g. the capture node restarted and uBridge truncated the pcap
-while paused) kills and respawns the session. A per-pcap asyncio lock
-serializes RPCs (sharkd serves one request at a time), each with a timeout.
-An LRU cap bounds concurrent sessions.
+while paused) kills and respawns the session. A per-session asyncio lock
+serializes RPCs (sharkd serves one request at a time), each with a timeout
+and an id check; any transport failure (timeout, dead pipe, malformed or
+stale reply) kills the session for good — a desynced session must never
+serve shifted results. A single manager lock makes check-spawn atomic (no
+double spawn under concurrent requests) and sessions are refcounted while
+in use, so the LRU cap only ever evicts idle ones.
 
 Timestamps are uBridge's userspace ``gettimeofday`` at match time (µs, a
 value measured after the packet has crossed the kernel twice — the last
@@ -56,21 +61,20 @@ import shutil
 import struct
 import tempfile
 import time
+from contextlib import asynccontextmanager
 
 from .controller_error import ControllerError, ControllerNotFoundError, ControllerBadRequestError
 
 log = logging.getLogger(__name__)
 
-# Full frame list is embedded in the range response while under this cap;
-# above it the response degrades to start/end + per-second buckets so the
-# client is never flooded by a high-traffic BPF.
-FRAME_LIST_CAP = 5000
-
 # One JSON-RPC per sharkd session, serialized by a per-session lock.
 RPC_TIMEOUT_SECONDS = 10.0
 
-# Resident sharkd sessions are bounded; least-recently-used evicted first.
-SESSION_MAX = 8
+# Resident sharkd sessions are bounded; only IDLE sessions are ever evicted
+# (least-recently-used first), so a request's own walk over a tag with more
+# sources than the cap never kills a session out from under it — the
+# population is trimmed back as uses drop instead.
+SESSION_MAX = 16
 
 # Display filters travel as one argv element (never through a shell) and are
 # capped to keep absurd expressions off the command line.
@@ -79,6 +83,16 @@ FILTER_MAX_LENGTH = 2000
 # Batch size when draining sharkd's `frames` RPC (columns / filter matches).
 _FRAMES_PAGE = 1000
 
+# A full `frames` page (1000 rows of pinned columns) measures ~190 KB against
+# the 64 KB default StreamReader limit — over it, readline() raises and clears
+# the stream, failing the request AND desyncing the resident session. Raise
+# the ceiling well above any page (or single-frame tree) we can produce.
+_STREAM_LIMIT_BYTES = 16 * 1024 * 1024
+
+# sharkd's JSON-RPC error code for a rejected display filter — the only error
+# that belongs to the client (400); everything else is an engine fault (502).
+_ERR_INVALID_FILTER = -13002
+
 
 class SharkdMissingError(ControllerError):
     """sharkd is not installed (or not on PATH) — replay is unavailable (501)."""
@@ -194,26 +208,47 @@ def read_frame_bytes(path, frame_number):
 # sharkd: process environment and scratch copies
 # ---------------------------------------------------------------------------
 
-def _scratch_copy(pcap):
+# The packet-list layout is pinned through Wireshark preferences in the
+# scratch HOME: personal config overrides any system-wide customization
+# (/etc/wireshark &c.), so the column indexes in _columns_for are a contract
+# we own rather than an environment default. Exactly the four columns the
+# frame entries consume — nothing else rides along in every page.
+_PINNED_COLUMNS = (
+    'gui.column.format: "Source", "%s", "Destination", "%d", '
+    '"Protocol", "%p", "Info", "%i"\n'
+)
+
+
+async def _prepare_scratch(pcap):
     """
-    Copy the pcap to a scratch file under the system temp dir for sharkd to
-    load. Hardened profiles (AppArmor &c.) can deny it access to the project
-    directory / the user's home while still allowing /tmp — a real copy,
-    deliberately not a symlink, since the profile resolves real paths.
-    Caller must unlink the returned path.
+    Build a scratch directory under the system temp dir holding the pcap copy
+    for sharkd to load plus the pinned column preferences. Hardened profiles
+    (AppArmor &c.) can deny sharkd the project directory / the user's home
+    while still allowing /tmp — a real copy, deliberately not a symlink,
+    since the profile resolves real paths. Caller must rmtree the directory.
+
+    :returns: ``(scratch_dir, scratch_pcap_path)``
     """
 
-    fd, scratch = tempfile.mkstemp(suffix=".pcap", prefix="gns3-replay-")
-    os.close(fd)
-    shutil.copyfile(pcap, scratch)
-    return scratch
+    scratch_dir = tempfile.mkdtemp(prefix="gns3-replay-")
+    scratch = os.path.join(scratch_dir, "capture.pcap")
+    # A pcap-sized copy has no business stalling the event loop — a 1 GB
+    # capture must not freeze every other request for the duration.
+    await asyncio.to_thread(shutil.copyfile, pcap, scratch)
+    prefs_dir = os.path.join(scratch_dir, ".config", "wireshark")
+    os.makedirs(prefs_dir, exist_ok=True)
+    with open(os.path.join(prefs_dir, "preferences"), "w") as f:
+        f.write(_PINNED_COLUMNS)
+    return scratch_dir, scratch
 
 
-def _engine_env():
-    """Scratch HOME so sharkd never even tries to read the user's home."""
+def _engine_env(scratch_dir):
+    """Scratch HOME (and XDG config) so sharkd never even tries to read the
+    user's home — and always reads OUR pinned preferences instead."""
 
     env = dict(os.environ)
-    env["HOME"] = tempfile.gettempdir()
+    env["HOME"] = scratch_dir
+    env["XDG_CONFIG_HOME"] = os.path.join(scratch_dir, ".config")
     return env
 
 
@@ -276,10 +311,13 @@ def _count_tree_nodes(value):
 class _SharkdSession:
     """A resident `sharkd -` process with one pcap loaded, addressed through
     line-oriented JSON-RPC. Serialized by an asyncio lock (sharkd serves one
-    request at a time)."""
+    request at a time). Any transport-level failure aborts the session for
+    good (see _abort) — a timed-out or desynced session must never answer
+    later requests with shifted results."""
 
-    def __init__(self, pcap, scratch, proc, stat):
+    def __init__(self, pcap, scratch_dir, scratch, proc, stat):
         self.pcap = pcap
+        self.scratch_dir = scratch_dir
         self.scratch = scratch
         self.proc = proc
         self.mtime_ns = stat.st_mtime_ns
@@ -287,6 +325,12 @@ class _SharkdSession:
         self.last_used = time.monotonic()
         self.lock = asyncio.Lock()
         self._next_id = 0
+        # Refcount of in-use holders (the manager's `session` context); the
+        # LRU cap only ever evicts sessions with zero uses, and a detached
+        # session (replaced in the manager, e.g. its pcap was rebuilt) closes
+        # itself when its last holder releases it.
+        self._uses = 0
+        self._detached = False
 
     def matches(self, stat):
         """True while the source pcap is byte-identical to what was loaded —
@@ -303,26 +347,41 @@ class _SharkdSession:
     async def rpc(self, method, params):
         async with self.lock:
             self._next_id += 1
-            request = {"jsonrpc": "2.0", "id": self._next_id, "method": method, "params": params}
+            request_id = self._next_id
+            request = {"jsonrpc": "2.0", "id": request_id, "method": method, "params": params}
             try:
                 self.proc.stdin.write((json.dumps(request) + "\n").encode())
                 await self.proc.stdin.drain()
                 raw = await asyncio.wait_for(self.proc.stdout.readline(), RPC_TIMEOUT_SECONDS)
             except asyncio.TimeoutError:
-                raise SharkdError(f"sharkd timed out after {RPC_TIMEOUT_SECONDS:.0f}s on {method!r}")
-            except OSError as e:
-                raise SharkdError(f"sharkd session died on {method!r}: {e}")
+                await self._abort(f"sharkd timed out after {RPC_TIMEOUT_SECONDS:.0f}s on {method!r}")
+            except (OSError, ValueError) as e:
+                # ValueError: a reply line over the StreamReader limit — the
+                # buffer is cleared, so the session is desynced either way.
+                await self._abort(f"sharkd session died on {method!r}: {e}")
             if not raw:
-                raise SharkdError(f"sharkd closed the session during {method!r}")
+                await self._abort(f"sharkd closed the session during {method!r}")
             try:
                 response = json.loads(raw)
             except ValueError as e:
-                raise SharkdError(f"Malformed sharkd response: {e}")
+                await self._abort(f"Malformed sharkd response: {e}")
+            # The echoed id proves this reply belongs to THIS request: a stale
+            # reply left over from a timed-out predecessor (or any desync)
+            # must never be served as fresh data.
+            if response.get("id") != request_id:
+                await self._abort(f"sharkd reply id mismatch on {method!r} (session desynchronized)")
             if "error" in response:
                 error = response["error"]
                 raise _SharkdRpcError(error.get("code"), str(error.get("message", "")))
             return response.get("result")
 
+    async def _abort(self, message):
+        """Kill the session for good (process + scratch copy) and raise — the
+        manager replaces it on the next acquire. Idempotent with close()."""
+
+        await self.close()
+        raise SharkdError(message)
+
     async def close(self):
         try:
             if self.proc.returncode is None:
@@ -336,56 +395,112 @@ class _SharkdSession:
                     log.warning("sharkd session for %s did not exit after kill", self.pcap)
         except ProcessLookupError:
             pass
-        try:
-            os.unlink(self.scratch)
-        except OSError:
-            pass
+        shutil.rmtree(self.scratch_dir, ignore_errors=True)
 
 
 class _SharkdManager:
-    """Resident sharkd sessions keyed by source pcap path, LRU-bounded."""
+    """Resident sharkd sessions keyed by source pcap path.
+
+    One asyncio lock covers check-spawn-insert-reap: the check-then-spawn
+    window spans the pcap copy, the fork and the load RPC, so without it two
+    concurrent requests for the same pcap double-spawn and the loser leaks
+    (process + scratch copy) forever. Sessions are refcounted while in use
+    (the `session` context manager) and the LRU cap only evicts IDLE
+    sessions — a tag with more sources than the cap, or a concurrent request,
+    can never have its session killed mid-RPC; the population is trimmed
+    back to the cap as uses drop (temporary overshoot is allowed)."""
 
     def __init__(self):
         self._sessions = {}
+        self._mu = asyncio.Lock()
 
-    async def session_for(self, pcap):
+    @asynccontextmanager
+    async def session(self, pcap):
+        session = await self._acquire(pcap)
+        try:
+            yield session
+        finally:
+            await self._release(session)
+
+    async def _acquire(self, pcap):
         if shutil.which("sharkd") is None:
             raise SharkdMissingError(
                 "sharkd is not available on this server — marker replay requires sharkd "
                 "(part of the Wireshark package)"
             )
-        stat = os.stat(pcap)
-        session = self._sessions.get(pcap)
-        if session is not None and session.matches(stat) and session.alive():
-            session.touch()
-            return session
-        if session is not None:
-            await session.close()
-            del self._sessions[pcap]
-        # Bound resident sessions: evict the least recently used.
-        while len(self._sessions) >= SESSION_MAX:
-            victim = min(self._sessions.values(), key=lambda s: s.last_used)
+        try:
+            stat = os.stat(pcap)
+        except FileNotFoundError:
+            # Deleted between the directory listing and here (marker removed,
+            # project cleaned up) — not a server fault.
+            raise ControllerNotFoundError(f"Capture file {os.path.basename(pcap)} no longer exists")
+        async with self._mu:
+            session = self._sessions.get(pcap)
+            if session is not None and session.matches(stat) and session.alive():
+                session._uses += 1
+                session.touch()
+                return session
+            if session is not None:
+                # Rebuilt/truncated source or a dead session: replace it. An
+                # in-use one closes itself when its last holder releases it.
+                del self._sessions[pcap]
+                if session._uses == 0:
+                    await session.close()
+                else:
+                    session._detached = True
+            spawned = await self._spawn(pcap, stat)
+            spawned._uses += 1
+            self._sessions[pcap] = spawned
+            victims = self._evict_locked()
+        for victim in victims:
+            await victim.close()
+        return spawned
+
+    def _evict_locked(self):
+        """Caller holds ``_mu``. Pop (not close — that can block on the reap)
+        least-recently-used IDLE sessions down to the cap; if everything is
+        in use, allow the overshoot rather than killing a live request."""
+
+        victims = []
+        while len(self._sessions) > SESSION_MAX:
+            idle = [s for s in self._sessions.values() if s._uses == 0]
+            if not idle:
+                break
+            victim = min(idle, key=lambda s: s.last_used)
+            del self._sessions[victim.pcap]
+            victims.append(victim)
+        return victims
+
+    async def _release(self, session):
+        async with self._mu:
+            session._uses -= 1
+            victims = []
+            if session._uses == 0:
+                if session._detached:
+                    # Replaced in the manager while still held — now orphaned.
+                    victims.append(session)
+                elif not session.alive():
+                    # Failed mid-use (its RPC aborted it) — drop the corpse.
+                    if self._sessions.get(session.pcap) is session:
+                        del self._sessions[session.pcap]
+                else:
+                    victims = self._evict_locked()
+        for victim in victims:
             await victim.close()
-            self._sessions.pop(victim.pcap, None)
-        session = await self._spawn(pcap, stat)
-        self._sessions[pcap] = session
-        return session
 
     async def _spawn(self, pcap, stat):
-        scratch = _scratch_copy(pcap)
+        scratch_dir, scratch = await _prepare_scratch(pcap)
         try:
             proc = await asyncio.create_subprocess_exec(
                 "sharkd", "-",
                 stdin=asyncio.subprocess.PIPE, stdout=asyncio.subprocess.PIPE,
-                stderr=asyncio.subprocess.DEVNULL, env=_engine_env(),
+                stderr=asyncio.subprocess.DEVNULL, env=_engine_env(scratch_dir),
+                limit=_STREAM_LIMIT_BYTES,
             )
         except OSError as e:
-            try:
-                os.unlink(scratch)
-            except OSError:
-                pass
+            shutil.rmtree(scratch_dir, ignore_errors=True)
             raise SharkdError(f"Could not run sharkd: {e}")
-        session = _SharkdSession(pcap, scratch, proc, stat)
+        session = _SharkdSession(pcap, scratch_dir, scratch, proc, stat)
         try:
             await session.rpc("load", {"file": scratch})
         except Exception as e:
@@ -394,9 +509,11 @@ class _SharkdManager:
         return session
 
     async def close_all(self):
-        for session in list(self._sessions.values()):
+        async with self._mu:
+            sessions = list(self._sessions.values())
+            self._sessions.clear()
+        for session in sessions:
             await session.close()
-        self._sessions.clear()
 
 
 _manager = None
@@ -409,6 +526,17 @@ def _get_manager():
     return _manager
 
 
+async def close_sessions():
+    """Server shutdown hook: kill every resident session and drop its scratch
+    copy, so nothing leaks into /tmp across restarts."""
+
+    global _manager
+    manager = _manager
+    if manager is not None:
+        await manager.close_all()
+    _manager = None
+
+
 # ---------------------------------------------------------------------------
 # Columns + display filter (sharkd `frames` RPC)
 # ---------------------------------------------------------------------------
@@ -419,42 +547,49 @@ async def _columns_for(pcap, filter_expr):
     ``{frame_number: {src, dst, proto, info, bg, fg}}``; with a
     ``filter_expr`` the keys are exactly the matching frame numbers, so one
     pass both filters and enriches the merge.
+
+    The column layout is pinned by the scratch-HOME preferences (see
+    _PINNED_COLUMNS): ``c[0]``=src, ``c[1]``=dst, ``c[2]``=proto, ``c[3]``=info
+    is a layout we own, not an environment default. The whole pagination
+    loop holds the session so the LRU cap cannot evict it between pages.
     """
 
-    session = await _get_manager().session_for(pcap)
-    columns = {}
-    skip = 0
-    while True:
-        # sharkd rejects skip=0 ("must be a positive integer") — only send it
-        # once there is actually something to skip.
-        params = {"limit": _FRAMES_PAGE}
-        if skip:
-            params["skip"] = skip
-        if filter_expr is not None:
-            params["filter"] = filter_expr
-        try:
-            rows = await session.rpc("frames", params)
-        except _SharkdRpcError as e:
+    manager = _get_manager()
+    async with manager.session(pcap) as session:
+        columns = {}
+        skip = 0
+        while True:
+            # sharkd rejects skip=0 ("must be a positive integer") — only send
+            # it once there is actually something to skip.
+            params = {"limit": _FRAMES_PAGE}
+            if skip:
+                params["skip"] = skip
             if filter_expr is not None:
-                raise FilterError(f"Invalid display filter: {e.message}")
-            raise SharkdError(f"sharkd frames failed on {os.path.basename(pcap)}: {e.message}")
-        for row in rows:
+                params["filter"] = filter_expr
             try:
-                frame_number = int(row.get("num"))
-            except (TypeError, ValueError):
-                continue
-            cols = row.get("c") or []
-            columns[frame_number] = {
-                "src": cols[2] or None if len(cols) > 2 else None,
-                "dst": cols[3] or None if len(cols) > 3 else None,
-                "proto": cols[4] or None if len(cols) > 4 else None,
-                "info": cols[6] or None if len(cols) > 6 else None,
-                "bg": row.get("bg"),
-                "fg": row.get("fg"),
-            }
-        if len(rows) < _FRAMES_PAGE:
-            return columns
-        skip += len(rows)
+                rows = await session.rpc("frames", params)
+            except _SharkdRpcError as e:
+                if filter_expr is not None and e.code == _ERR_INVALID_FILTER:
+                    raise FilterError(f"Invalid display filter: {e.message}")
+                # Anything else is an engine fault, not the client's filter.
+                raise SharkdError(f"sharkd frames failed on {os.path.basename(pcap)}: {e.message}")
+            for row in rows or []:
+                try:
+                    frame_number = int(row.get("num"))
+                except (TypeError, ValueError):
+                    continue
+                cols = row.get("c") or []
+                columns[frame_number] = {
+                    "src": cols[0] or None if len(cols) > 0 else None,
+                    "dst": cols[1] or None if len(cols) > 1 else None,
+                    "proto": cols[2] or None if len(cols) > 2 else None,
+                    "info": cols[3] or None if len(cols) > 3 else None,
+                    "bg": row.get("bg"),
+                    "fg": row.get("fg"),
+                }
+            if len(rows or []) < _FRAMES_PAGE:
+                return columns
+            skip += len(rows or [])
 
 
 # ---------------------------------------------------------------------------
@@ -572,39 +707,28 @@ def _validate_filter(filter_expr):
         )
 
 
-async def build_timeline(project, tag, frame_cap=FRAME_LIST_CAP, filter_expr=None, link_id=None):
+async def build_timeline(project, tag, filter_expr=None, link_id=None):
     """
-    The ``range`` response: timeline bounds, per-source stats, and (under
-    ``frame_cap``) the full merged frame list for one-request timeline
-    layout. Over the cap the list is replaced by per-second buckets. With a
-    ``filter_expr`` and/or a ``link_id`` every figure is computed on the
-    matching frames only (``sources`` stays the full tag inventory).
+    The ``range`` response: timeline bounds, per-source stats, and the full
+    merged frame list — deliberately uncapped (a flat list is the whole
+    contract; rendering a huge list is the client's concern, and the window
+    endpoint exists for incremental views). With a ``filter_expr`` and/or a
+    ``link_id`` every figure is computed on the matching frames only
+    (``sources`` stays the full tag inventory).
     """
 
     _validate_filter(filter_expr)
     entries = gate_tag(project, tag)
     frames, sources = await _merged_frames(project, entries, filter_expr, link_id=link_id)
 
-    response = {
+    return {
         "tag": tag,
         "start": frames[0]["ts"] if frames else None,
         "end": frames[-1]["ts"] if frames else None,
         "frame_count": len(frames),
-        "truncated": len(frames) > frame_cap,
         "sources": sources,
+        "frames": frames,
     }
-    if len(frames) <= frame_cap:
-        response["frames"] = frames
-    else:
-        buckets = {}
-        for frame in frames:
-            second = _parse_ts(frame["ts"]) // 1_000_000
-            buckets[second] = buckets.get(second, 0) + 1
-        response["buckets"] = [
-            {"ts": _format_ts(second, 0), "count": count}
-            for second, count in sorted(buckets.items())
-        ]
-    return response
 
 
 async def query_frames(project, tag, ts, window_ms=100, limit=1000, filter_expr=None, link_id=None):
@@ -629,13 +753,19 @@ async def query_frames(project, tag, ts, window_ms=100, limit=1000, filter_expr=
 # Frame detail (lazy — one frame per call, via the resident session)
 # ---------------------------------------------------------------------------
 
-async def decode_frame(project, tag, ts, node_id, link_id, marker):
+async def decode_frame(project, tag, ts, node_id, link_id, marker, frame_number=None):
     """
     Decode exactly one frame: locate its pcap by source identity, verify the
     round-tripped ts still matches the file (guards a rebuild between the
     timeline view and this click), read the raw bytes for the hex view
     straight from the pcap, and rename the sharkd protocol tree into the
     REST contract (closed key set, values untouched).
+
+    ``ts`` is not unique within one pcap (same-microsecond frames are kept
+    deliberately); an explicit ``frame_number`` from the frame list entry
+    disambiguates them and must still land on the exact ts. Without one the
+    first ts match decodes — fine unless two frames share a microsecond on
+    the same link.
     """
 
     entries = gate_tag(project, tag)
@@ -654,28 +784,38 @@ async def decode_frame(project, tag, ts, node_id, link_id, marker):
         raise ControllerNotFoundError(f"No capture file for marker '{marker}' (nothing ever matched)")
 
     frames = scan_pcap_frames(pcap)
-    # The ts must be the exact string the timeline returned; find the frame
-    # it identifies rather than trusting any position hint from the client.
-    frame_number = next(
-        (i for i, (sec, usec, _len) in enumerate(frames, start=1)
-         if _format_ts(sec, usec) == ts),
-        None,
+    rebuilt_message = (
+        f"No frame at ts {ts} in marker '{marker}' (the capture may have been rebuilt)"
     )
     if frame_number is None:
-        raise ControllerNotFoundError(
-            f"No frame at ts {ts} in marker '{marker}' (the capture may have been rebuilt)"
+        # The ts must be the exact string the timeline returned; find the frame
+        # it identifies rather than trusting any position hint from the client.
+        frame_number = next(
+            (i for i, (sec, usec, _len) in enumerate(frames, start=1)
+             if _format_ts(sec, usec) == ts),
+            None,
         )
+        if frame_number is None:
+            raise ControllerNotFoundError(rebuilt_message)
+    else:
+        if not 1 <= frame_number <= len(frames):
+            raise ControllerNotFoundError(rebuilt_message)
+        sec, usec, _len = frames[frame_number - 1]
+        if _format_ts(sec, usec) != ts:
+            raise ControllerNotFoundError(rebuilt_message)
 
     raw_hex = read_frame_bytes(pcap, frame_number)
-    session = await _get_manager().session_for(pcap)
-    try:
-        result = await session.rpc("frame", {"frame": frame_number, "proto": True})
-    except _SharkdRpcError as e:
-        if e.code == -8003:  # frame number out of range — file changed under us
-            raise ControllerNotFoundError(
-                f"No frame at ts {ts} in marker '{marker}' (the capture may have been rebuilt)"
-            )
-        raise SharkdError(f"sharkd frame failed: {e.message}")
+    if raw_hex is None:
+        # The header scan listed it but the bytes are gone — mid-write tail.
+        raise ControllerNotFoundError(rebuilt_message)
+
+    async with _get_manager().session(pcap) as session:
+        try:
+            result = await session.rpc("frame", {"frame": frame_number, "proto": True})
+        except _SharkdRpcError as e:
+            # The frame range was validated against the file above, so an
+            # engine error here is a real fault (502), not a client 404.
+            raise SharkdError(f"sharkd frame failed: {e.message}")
 
     tree = _rename_value(result.get("tree", []))
     return {