Preserve image reconciliation, locking, and regression coverage while integrating upstream release, dependency, typing, and formatting updates. Align feature code with the new Ruff checks.
When a Docker node is created on a remote compute whose Docker daemon
does not have the image, the compute now raises ImageMissingError
instead of blindly pulling from the Docker repository. The controller
exports the image from the Docker daemon on its host (docker save
stream) and streams it to the compute which loads it, so locally built
or docker-loaded images work across computes. When the image is not
available on the controller host either, the compute is asked to pull
it from the Docker repository as a fallback.
- add a POST /docker/images/load compute endpoint that streams a
docker save tar into the Docker daemon
- let Docker.http_query pass raw (non-dict) request bodies through so
the tar can be streamed to the daemon
- drop the inline pull from DockerVM.create() and the now unused
DockerVM.pull_image wrapper
Review-driven session/transport fixes (each reproduced live against
sharkd 4.6.7 before fixing):
- raise the RPC stream limit to 16 MB: a full 1000-row frames page
measures ~190 KB against the 64 KB StreamReader default, which failed
the request with a 500 and desynchronized the resident session; a
line-over-limit ValueError is now treated as a transport failure
- verify JSON-RPC reply ids: a timed-out request's late reply was
served as the next request's answer; timeouts, dead pipes, malformed
and stale replies now kill the session for good instead
- make check-spawn atomic under one manager lock: concurrent requests
for the same pcap double-spawned sharkd and leaked the loser (process
plus /tmp scratch copy) forever
- refcount sessions and evict idle only (LRU, cap raised 8 -> 16): a
tag with more sources than the cap respawned every source on every
request, and concurrent requests could get their session killed
mid-RPC (spurious 502)
- map FilterError to sharkd's filter rejection (-13002) only; other
engine failures with a filter set are 502, not a client 400
- detail: accept an optional frame_number to disambiguate
same-microsecond frames (ts is not unique within a pcap); drop the
-8003 -> 404 mapping (the range is validated locally, engine errors
are real faults); a failed hex read is a 404 instead of "hex": null
- a pcap deleted mid-request is a 404, not a 500; the pcap-sized
scratch copy runs off the event loop; server shutdown kills every
resident session and drops its scratch directory
- pin the packet-list layout through scratch-HOME Wireshark
preferences: the column indexes are a contract the server owns
(protocol-level column negotiation is rejected by sharkd 4.6.x)
Range contract change (WebUI moved to an always-flat list): the merged
frame list is returned in full, deliberately uncapped - truncated and
per-second buckets are removed, frame_count always equals
len(frames), and rendering cost is the client's concern (the window
endpoint remains the incremental path).
Narrows the merged frame stream to one capture source BEFORE counting,
slicing and bucketing (frame_count / frames | buckets all recomputed on
the narrowed set), AND-composing with the display filter. A pure
identity filter applied before any engine work — only the selected
link's pcap gets a sharkd pass, so link+filter is cheaper than filter
alone.
Two boundaries by contract with the Web UI:
- sources[] stays the tag's stable inventory: every capture source
listed with engine-free TOTAL counts, unaffected by link/filter — a
source dropdown must not shrink when the view narrows (this also
settles sources[].count on total counts rather than post-filter
matches, which no spec ever required)
- an unknown link_id matches nothing: frame_count 0, start null, empty
frames/buckets — the same shape as a zero-match display filter,
deliberately not a 404; an empty link param is treated as absent
sharkd (the Wireshark daemon) is now the single decode engine — the
tshark/PDML path is gone, and without sharkd every replay endpoint
returns 501 (no degraded mode: one engine, one rendering shape for the
Web UI).
- Frame entries gain packet-list columns from sharkd's frames RPC:
src/dst/proto/info plus the Wireshark coloring hints bg/fg
- range and frames accept ?filter=<display filter>, applied before
counting and slicing; invalid expressions are 400 carrying sharkd's
original text; filters travel as single argv-style elements, capped
at 2000 chars; filtered frames keep their original pcap frame numbers
- frame detail returns sharkd's protocol tree with keys renamed into
the REST contract (element/label/name/filter_expr/pos+size/expert/
generated/children): a census-verified closed key set, values
untouched, unknown keys passed through verbatim, Wireshark-internal
hf ids dropped. filter_expr gives the UI click-to-filter; pos/size
drives hex highlighting (hex still read straight from the pcap)
- one resident 'sharkd -' session per source pcap: lazy spawn, /tmp
scratch copy + scratch HOME (hardened profiles), per-request
(mtime,size) validation with respawn, LRU bound, per-session lock,
per-RPC timeout, bounded close
Timeline backbone (gate, record-header scan, merge ordering, canonical
ts strings, hex reads) stays plain Python — identity and ordering never
depend on the engine.
The bare 'os.kill = MagicMock()' was never restored, so every later test
in the same process ran with a no-op kill. Any test that kills a child
process and waits for it then hangs forever (resident sharkd sessions
waiting on an immortal process). Use monkeypatch so the patch is undone.
The project-open bulk path (_add_nio_binding / _get_existing_nio /
_update_nio_binding in routes/compute/projects.py) dropped port_number
for Docker nodes, unlike the per-node routes and the IOU branch. With
multi-port docker adapters (iol-runner nodes model 4 ports per adapter,
0ca9ccc63) every batched NIO landed on port 0 where Adapter.add_nio()
silently overwrites — the last entry per node won — so reopening a
project clobbered the port-0 links with the port-1 NIOs: links ended up
cross-wired between the wrong node pairs and the real links died
(IOL direct-link ping failures after close/reopen, EXCESSCOLL storms
from the phantom loops).
Templates reference a config file with the GNS3_IOL_STARTUP_CONFIG
environment knob; the controller materializes the file content into
startup_config_content on node creation (sent once, knob consumed —
the same pattern as the IOU startup_config mapping). The compute builds
the content into the node's nvram_<app id> at the next start using the
IOU nvram_import utility (IOL and IOU share the nvram container format,
verified against iol-xe 17.18.02): valid config at boot, no setup
dialog, %h hostname substitution, hostname rewrite on rename.
Semantics verified against the runner: IOL boots from NVRAM whenever it
holds a config, so a plain stop/start never re-applies the startup
config and 'write memory' survives restarts; an explicit content edit
(PUT) is re-applied on the next start and wins over the saved config,
like IOU.
Markers on different links sharing a tag form one distributed capture
session. Once every marker under the tag is paused (409 otherwise), three
read-only endpoints replay it:
- GET .../markers/tags/{tag}/replay/range merges the per-marker pcaps by
scanning 16-byte record headers only (no tshark) into a timestamp-
ordered frame list (per-second buckets above a 5000-frame cap)
- GET .../replay/frames?ts=&window_ms= returns frames in [T, T+window];
an empty window is a normal empty array
- GET .../replay/frame/detail lazily decodes one frame the user opened:
raw bytes for the hex view read straight from the pcap, protocol tree
from 'tshark -T pdml' mapped isomorphically to JSON (every attribute
survives, values stay strings). tshark reads a /tmp scratch copy with a
scratch HOME — hardened profiles deny it the project directory.
Sort key is (ts, source file, frame number): ts is not unique across a
merge. The ts parameter round-trips as the exact string from the frame
list. Round-trip tests pin the PDML→JSON fidelity (element count and
attribute coverage).
Closing a web console while the node kept streaming output crashed the
ws_console/vnc_console handlers with an uncaught WebSocketDisconnect from
the compute-to-client send path (only the opposite direction was guarded),
producing a full ASGI traceback on every console close.
Restructure both endpoints as symmetric forwarding tasks managed with
asyncio.wait(FIRST_COMPLETED): exceptions from either direction are
collected as task exceptions, the peer task is cancelled, the compute
WebSocket is closed, and the client is notified. The receive loops now
close and log on every exit path instead of only in the exception branch.
Tests patch the in-memory ASGI transport to deliver a conformant
websocket.disconnect on client close (it sends a non-conformant
websocket.close that starlette receive() rejects).
Deleting a template with prune_images, deleting an image, or pruning
orphan images only checked template references — a project node still
pointing at the image (e.g. via hda_disk_image_backing_file) was left
with a dangling reference and the project could no longer be opened.
- Add controller helpers scanning every known project (opened projects
via in-memory nodes, closed projects via their .gns3 file) for
template and image usage; unreadable topologies are skipped
- Guard DELETE /templates/{id}, DELETE /templates/{id}?prune_images,
DELETE /images/{path} and /images/prune with a 409 listing the
project names
- Run all template-delete checks before any mutation so a refused
deletion cannot leave the template gone while its images survive
link_capture_download and get_symbol embedded a 10-min JWT in the
Authorization header of the curl command they return; LLM clients
retyping that command corrupted the long token — the same failure
class as the console WebSocket URLs fixed in the previous commit.
Generalize the ticket store (console_tickets.py -> access_tickets.py,
ConsoleTicketService -> AccessTicketService): a ticket now binds to
either a node's console endpoints (WebSocket, matched against route
path params) or one exact REST resource path (capture file, symbol
image). The two binding modes are isolated — a node-bound ticket
cannot authenticate a REST resource and vice versa.
get_user_from_token redeems path-bound tickets through the existing
token parameter / Bearer header, matched exactly against
request.url.path, then reuses the shared user lookup and token_version
revocation checks. Download URLs embed ?token=<ticket> and the curl
commands no longer carry a Bearer header.
LLM clients transcribing the console WebSocket URL into shell commands
reliably corrupted the ~200-char JWT embedded in it (dropped header
segment -> "MissingAlgorithmError: Missing 'alg' value in header" on
every connection attempt). The node_console tool now mints a short
random ticket ("gns3t_" + 16 urlsafe chars, 10 min TTL, multi-use)
stored server-side and bound to the node's console endpoints:
- new ConsoleTicketService (gns3server/services/console_tickets.py),
in-memory store with lazy expiry sweeps
- get_current_active_user_from_websocket redeems tickets through the
existing "token" query parameter, gated on websocket.path_params so a
ticket only authenticates the console/ws and console/vnc routes of
the node it was minted for; the JWT path is unchanged
- redemption reuses the existing user lookup, token_version revocation
and is_active checks, so logging out invalidates outstanding tickets
- vnc_url no longer embeds the full session JWT
- the tool docstring now tells clients to run the returned command
verbatim instead of reconstructing the URL
The compute-to-client forwarding loops in ws_console and vnc_console had
no WebSocketDisconnect handling: when a client (WebUI, or an MCP-driven
websocat session killed by timeout) disconnected while the compute was
still streaming console output, the next send raised WebSocketDisconnect
that leaked all the way up to uvicorn as an ERROR-level ASGI traceback.
Catch it and log at info level, symmetric with the receive-side handlers.
