Review-driven session/transport fixes (each reproduced live against
sharkd 4.6.7 before fixing):
- raise the RPC stream limit to 16 MB: a full 1000-row frames page
measures ~190 KB against the 64 KB StreamReader default, which failed
the request with a 500 and desynchronized the resident session; a
line-over-limit ValueError is now treated as a transport failure
- verify JSON-RPC reply ids: a timed-out request's late reply was
served as the next request's answer; timeouts, dead pipes, malformed
and stale replies now kill the session for good instead
- make check-spawn atomic under one manager lock: concurrent requests
for the same pcap double-spawned sharkd and leaked the loser (process
plus /tmp scratch copy) forever
- refcount sessions and evict idle only (LRU, cap raised 8 -> 16): a
tag with more sources than the cap respawned every source on every
request, and concurrent requests could get their session killed
mid-RPC (spurious 502)
- map FilterError to sharkd's filter rejection (-13002) only; other
engine failures with a filter set are 502, not a client 400
- detail: accept an optional frame_number to disambiguate
same-microsecond frames (ts is not unique within a pcap); drop the
-8003 -> 404 mapping (the range is validated locally, engine errors
are real faults); a failed hex read is a 404 instead of "hex": null
- a pcap deleted mid-request is a 404, not a 500; the pcap-sized
scratch copy runs off the event loop; server shutdown kills every
resident session and drops its scratch directory
- pin the packet-list layout through scratch-HOME Wireshark
preferences: the column indexes are a contract the server owns
(protocol-level column negotiation is rejected by sharkd 4.6.x)
Range contract change (WebUI moved to an always-flat list): the merged
frame list is returned in full, deliberately uncapped - truncated and
per-second buckets are removed, frame_count always equals
len(frames), and rendering cost is the client's concern (the window
endpoint remains the incremental path).
Narrows the merged frame stream to one capture source BEFORE counting,
slicing and bucketing (frame_count / frames | buckets all recomputed on
the narrowed set), AND-composing with the display filter. A pure
identity filter applied before any engine work — only the selected
link's pcap gets a sharkd pass, so link+filter is cheaper than filter
alone.
Two boundaries by contract with the Web UI:
- sources[] stays the tag's stable inventory: every capture source
listed with engine-free TOTAL counts, unaffected by link/filter — a
source dropdown must not shrink when the view narrows (this also
settles sources[].count on total counts rather than post-filter
matches, which no spec ever required)
- an unknown link_id matches nothing: frame_count 0, start null, empty
frames/buckets — the same shape as a zero-match display filter,
deliberately not a 404; an empty link param is treated as absent
sharkd (the Wireshark daemon) is now the single decode engine — the
tshark/PDML path is gone, and without sharkd every replay endpoint
returns 501 (no degraded mode: one engine, one rendering shape for the
Web UI).
- Frame entries gain packet-list columns from sharkd's frames RPC:
src/dst/proto/info plus the Wireshark coloring hints bg/fg
- range and frames accept ?filter=<display filter>, applied before
counting and slicing; invalid expressions are 400 carrying sharkd's
original text; filters travel as single argv-style elements, capped
at 2000 chars; filtered frames keep their original pcap frame numbers
- frame detail returns sharkd's protocol tree with keys renamed into
the REST contract (element/label/name/filter_expr/pos+size/expert/
generated/children): a census-verified closed key set, values
untouched, unknown keys passed through verbatim, Wireshark-internal
hf ids dropped. filter_expr gives the UI click-to-filter; pos/size
drives hex highlighting (hex still read straight from the pcap)
- one resident 'sharkd -' session per source pcap: lazy spawn, /tmp
scratch copy + scratch HOME (hardened profiles), per-request
(mtime,size) validation with respawn, LRU bound, per-session lock,
per-RPC timeout, bounded close
Timeline backbone (gate, record-header scan, merge ordering, canonical
ts strings, hex reads) stays plain Python — identity and ordering never
depend on the engine.
The bare 'os.kill = MagicMock()' was never restored, so every later test
in the same process ran with a no-op kill. Any test that kills a child
process and waits for it then hangs forever (resident sharkd sessions
waiting on an immortal process). Use monkeypatch so the patch is undone.
Markers on different links sharing a tag form one distributed capture
session. Once every marker under the tag is paused (409 otherwise), three
read-only endpoints replay it:
- GET .../markers/tags/{tag}/replay/range merges the per-marker pcaps by
scanning 16-byte record headers only (no tshark) into a timestamp-
ordered frame list (per-second buckets above a 5000-frame cap)
- GET .../replay/frames?ts=&window_ms= returns frames in [T, T+window];
an empty window is a normal empty array
- GET .../replay/frame/detail lazily decodes one frame the user opened:
raw bytes for the hex view read straight from the pcap, protocol tree
from 'tshark -T pdml' mapped isomorphically to JSON (every attribute
survives, values stay strings). tshark reads a /tmp scratch copy with a
scratch HOME — hardened profiles deny it the project directory.
Sort key is (ts, source file, frame number): ts is not unique across a
merge. The ts parameter round-trips as the exact string from the frame
list. Round-trip tests pin the PDML→JSON fidelity (element count and
attribute coverage).
Closing a web console while the node kept streaming output crashed the
ws_console/vnc_console handlers with an uncaught WebSocketDisconnect from
the compute-to-client send path (only the opposite direction was guarded),
producing a full ASGI traceback on every console close.
Restructure both endpoints as symmetric forwarding tasks managed with
asyncio.wait(FIRST_COMPLETED): exceptions from either direction are
collected as task exceptions, the peer task is cancelled, the compute
WebSocket is closed, and the client is notified. The receive loops now
close and log on every exit path instead of only in the exception branch.
Tests patch the in-memory ASGI transport to deliver a conformant
websocket.disconnect on client close (it sends a non-conformant
websocket.close that starlette receive() rejects).
Deleting a template with prune_images, deleting an image, or pruning
orphan images only checked template references — a project node still
pointing at the image (e.g. via hda_disk_image_backing_file) was left
with a dangling reference and the project could no longer be opened.
- Add controller helpers scanning every known project (opened projects
via in-memory nodes, closed projects via their .gns3 file) for
template and image usage; unreadable topologies are skipped
- Guard DELETE /templates/{id}, DELETE /templates/{id}?prune_images,
DELETE /images/{path} and /images/prune with a 409 listing the
project names
- Run all template-delete checks before any mutation so a refused
deletion cannot leave the template gone while its images survive
link_capture_download and get_symbol embedded a 10-min JWT in the
Authorization header of the curl command they return; LLM clients
retyping that command corrupted the long token — the same failure
class as the console WebSocket URLs fixed in the previous commit.
Generalize the ticket store (console_tickets.py -> access_tickets.py,
ConsoleTicketService -> AccessTicketService): a ticket now binds to
either a node's console endpoints (WebSocket, matched against route
path params) or one exact REST resource path (capture file, symbol
image). The two binding modes are isolated — a node-bound ticket
cannot authenticate a REST resource and vice versa.
get_user_from_token redeems path-bound tickets through the existing
token parameter / Bearer header, matched exactly against
request.url.path, then reuses the shared user lookup and token_version
revocation checks. Download URLs embed ?token=<ticket> and the curl
commands no longer carry a Bearer header.
LLM clients transcribing the console WebSocket URL into shell commands
reliably corrupted the ~200-char JWT embedded in it (dropped header
segment -> "MissingAlgorithmError: Missing 'alg' value in header" on
every connection attempt). The node_console tool now mints a short
random ticket ("gns3t_" + 16 urlsafe chars, 10 min TTL, multi-use)
stored server-side and bound to the node's console endpoints:
- new ConsoleTicketService (gns3server/services/console_tickets.py),
in-memory store with lazy expiry sweeps
- get_current_active_user_from_websocket redeems tickets through the
existing "token" query parameter, gated on websocket.path_params so a
ticket only authenticates the console/ws and console/vnc routes of
the node it was minted for; the JWT path is unchanged
- redemption reuses the existing user lookup, token_version revocation
and is_active checks, so logging out invalidates outstanding tickets
- vnc_url no longer embeds the full session JWT
- the tool docstring now tells clients to run the returned command
verbatim instead of reconstructing the URL
The compute-to-client forwarding loops in ws_console and vnc_console had
no WebSocketDisconnect handling: when a client (WebUI, or an MCP-driven
websocat session killed by timeout) disconnected while the compute was
still streaming console output, the next send raised WebSocketDisconnect
that leaked all the way up to uvicorn as an ERROR-level ASGI traceback.
Catch it and log at info level, symmetric with the receive-side handlers.
The compute_get/compute_images MCP tools typed compute_id as a UUID, so
passing 'local' (the actual id of the built-in compute, which the
compute_images description itself pointed to) was rejected by schema
validation. Both tools now take a string defaulting to 'local', and the
compute_get REST route resolves 'local' through the controller since the
local compute has no database entry.
install_appliances_from_image relied on the name+version pair check in
TemplatesService, so the same appliance reached through a second image
(the CSR1000v case) created a second template sharing the name. The auto
path now skips when any template with the same name exists, whatever the
version, and returns a manifest of created and skipped candidates;
POST /images/install replies 200 with that manifest instead of an empty
204, and the image_install MCP tool surfaces it.
POST /appliances/{id}/install replied 204 with an empty body, so the MCP
appliance_install tool crashed with 'Expecting value: line 1 column 1'
while the template had actually been created. The route now returns the
created template (201, response_model=schemas.Template), _create_template
propagates it, and the MCP handler parses the body defensively so an
empty reply degrades to a plain success message.
Suspending a single VPCS/IOU node returned a fake 204: the controller
route swallowed the compute 405 that the node types honestly raise, so
callers saw success while the node stayed started. Surface the 405
instead. The best-effort swallow on suspend_all is kept (and now covered
by a test) since mixed projects legitimately contain always-running node
types.
GET /projects/{id}/locked returned True for a project with no drawings
or nodes: both loops ran zero times and the fallback return won. Locking
and unlocking such a project always succeeded while the state stayed
"locked", so it could never be unlocked.
Report a project with nothing to lock as not locked, and re-check the
state after unlock in the route tests.
Loading a .gns3 placed directly in the projects root registered the
shared projects directory as the project path (load_project derives
the path from the file's parent directory). Deleting such an entry ran
rmtree on the projects directory itself, wiping every project until a
root-owned file stopped it, and left a zombie entry in the controller.
Three layers of protection:
- Controller.load_project() refuses a .gns3 whose parent directory is
the projects directory; the normal subdirectory layout is unaffected
- the Project.path setter rejects the projects directory itself and
its ancestors, closing the same hole for POST/PUT with an explicit
path
- Project.delete() uses realpath + commonpath instead of commonprefix:
entries whose path is the projects root are refused, and sibling
directories sharing a string prefix (/srv/projects-evil vs
/srv/projects) are no longer treated as inside the projects dir
Also removes the project_load MCP tool: loading by raw server
filesystem path is a footgun for automated clients; projects can still
be opened by project_id via the remaining tools.
GET /v3/projects/{project_id}/gns3file returns the raw .gns3 file as
application/json so the WebUI can render topology thumbnails without
opening the project. Adds a public Project.topology_file property.
GET returns all settings sections except the deprecated
VirtualBox/VMware ones (Server.Audit privilege); secrets are
masked and Controller.jwt_secret_key is excluded entirely.
PUT applies a partial update (Server.Modify privilege): masked
or empty secrets mean unchanged, null removes the option, the
response carries the new values plus restart_required, and a
settings.updated notification is emitted. New
Server.Audit/Server.Modify privileges are seeded into the
Administrator role at table creation.
New GET /v3/netmiko/device_types endpoint returns the device types
supported by the netmiko library installed on the server, including
the gns3-copilot custom drivers, so the web UI can populate the
netmiko_device_type dropdown on templates and nodes.
The list is read at runtime from netmiko's ssh_dispatcher.CLASS_MAPPER
registry (the same table ConnectHandler dispatches on), filtered to
drop the '_ssh' aliases and the 'autodetect' pseudo type, and cached
for the process lifetime. Returns 501 when netmiko is not installed
(ai-features extra).
Appliance fields that describe the appliance (vendor information, default
credentials, installation instructions...) were dropped when installing a
template. Keep them in a new appliance_metadata JSON column on the
templates table, filled by the appliance-to-template conversion for both
registry v1-6 and v8 (version level values override the appliance level
ones). The nested schema allows extra fields so future registry fields
persist without a migration.
- install: resolve the image directory from the version's settings type and
skip image handling for docker appliances; guard appliance.images
- appliance schema: validate template_properties against template_type,
align cpu_throttling with the qemu template, add kvm and version idlepc
- conversion: map IOU image to path, kvm disable to accel=tcg, inherit only
same-type default settings, symbol fallback from the effective category,
template_properties cannot override structural fields
- allow clearing netmiko_device_type with an empty string
- download the template symbol regardless of the level it is defined at and
give qemu guests a default symbol
Common template field (schema + templates table column + Alembic
migration) holding the Netmiko device type (e.g. 'cisco_xr', 'nokia_srl')
so Netmiko/Nornir based tooling can look up how to reach a node's CLI
without hard-coded vendor mappings. Free-form lowercase string on
purpose: Netmiko's platform list evolves independently of GNS3.
128 characters was far beyond any realistic marker label (icmp, arp, tcp-syn)
and would have collided with the pcap filename budget once a tag prefix is
added later. Cap the user-facing name at 32 in both MarkerCreate and
MarkerDefinitionCreate; the compute-side name guard now also rejects names
longer than 48, which covers the `global-{def_name}` inherited form (≤ 39).
19 controller-layer tests covering start/stop/update_marker (storage,
inheritance guards + bypasses, partial-update preservation of render
hints), project-def CRUD (fan-out, sync, delete-cleanup regression),
apply_defs_to_new_link, persist_markers/asdict, and aggregation.
15 API-route tests covering per-link create/update/delete (201/200/204),
global-prefix rejection on create (409 regression), bad-format rejection
(422), PUT/DELETE-on-inherited guard (409 regression), project-def CRUD
endpoints, and the aggregation view.
All 34 tests pass when run as part of the full suite.
- New config: Controller.jwt_refresh_token_expire_minutes (default 30 days)
- New endpoint: POST /v3/access/users/refresh (public, unauthenticated)
- Login/authenticate responses now include refresh_token
- AuthService: _create_token helper, create_refresh_token, get_token_data
now parses type claim (token_use) for token classification
- Security: refresh tokens rejected on HTTP + WebSocket access paths;
/refresh strictly requires type=='refresh'
- Logout works for free via existing token_version mechanism
- Tests: 9 new TestRefreshToken cases, all passing; 34 existing tests
still pass (no regressions)
- Stream file GET/POST through controller without buffering in memory
- Add recursive and subdirectory filtering to node file listing
- Replace file extension with magic-based file type detection
- Add DELETE endpoint for node and project files
- Include directories in listing response
- Add params and stream support to http_query
- Fix lambda closures, streamer exception scope, and delete error codes
The LLMConfig.Audit and LLMConfig.Modify privileges added to the User
role increased the default privilege count from 25 to 27. Update test
assertions to match the new counts.
