Add an `extra_configs` field (list of {target, content}) to the docker
node/template/appliance schemas. For each entry GNS3 writes `content` to a
file in the node working directory and bind-mounts it read-only at `target`
inside the container.
This lets a NOS appliance seed its startup config without rebuilding the
image: XRd points XR_FIRST_BOOT_CONFIG at an injected /firstboot.cfg, FRR at
/etc/frr/frr.conf, etc. The bind is a single-file mount applied at create
time, so it works for both the generic init.sh path and vendor nodes that
skip init.sh (console_type=docker_exec). Entries are only injected when
present, so ordinary nodes are unaffected.
The content can't go through `environment` (it is line-delimited, one var per
line), hence a dedicated field -- the same plumbing shape as extra_volumes.
create_docker_node() passes console, aux etc. to create_node() via
.get() so those keys remain in node_data. The setattr fallback loop
then re-applies them — if reserve_tcp_port returned a different port
in __init__, the setter fires an INFO log and performs a wasted
release→reserve round-trip.
Pop the 15 keys already consumed by create_node() before the loop so
it only handles truly extra keys.
create_batch_nios bound NIOs in a serial for-loop, so during project open
every builtin L2 node (ethernet_switch/hub/cloud/nat) started its uBridge
one at a time (~0.5s each for fork + AF_UNIX connect). Group entries by
node and bind in parallel with asyncio.gather — mirroring update_batch_nios
— so independent uBridge processes start concurrently. Within a node,
entries stay serial to respect the per-node uBridge command lock.
The print(node_data.chassis, platform in DEFAULT_CHASSIS) at
dynamips_nodes.py:67 (added in 8ad7b3f6, 2022) emitted "None False"
to stdout on every Dynamips router creation. Pure debug leftover;
the very next line tests the same condition.
High-frequency marker.matches shared the single project notification queue with topology events (node.*/link.*), causing head-of-line blocking. Add a separate marker channel: Notification.project_marker_queue/marker_emit, dispatch routes marker.* off the main project queue, plus a new WS /{project_id}/notifications/markers/ws endpoint. Fully migrated (the main project WS no longer carries marker.match); marker listeners are independent of project auto_close. Compute side unchanged.
The update_batch_nios handler looped serially across all entries (5010
for a 2505-link topology). On started nodes each entry does uBridge I/O,
so the serial loop added orders of magnitude to the fan-out wall time.
Group entries by node_id before dispatching. Different nodes talk to
their own uBridge process (AF_UNIX socket) and are fully independent, so
their updates run in parallel via asyncio.gather. Per-node entries are
still serial (respecting the per-node uBridge command lock).
When a marker definition is created (or re-applied on data_link_type
change), the fan-out used to call inherit_marker -> update() on every
link, issuing one PUT /nio per link end (5000+ round-trips on a 2500-link
project). On started nodes each round-trip also reconfigured uBridge.
Two-phase fan-out:
- inherit_marker/start_marker gain memory_only: writes the marker into
link._markers and refreshes _link_data without any HTTP/emit/dump.
- _apply_def_to_all_links applies memory-only to every link, then
_batch_update_link_nios groups the updated NIO specs by compute and
sends a single PUT /projects/{id}/nios/batch per compute.
compute: new PUT /projects/{id}/nios/batch endpoint with _get_existing_nio
+ _update_nio_binding dispatch (mirrors create_batch_nios), re-applies
filters+markers to uBridge on started nodes.
Precise per-marker operations (update_marker bpf change, stop_marker on
def delete) are untouched — they deliberately avoid a full reapply to
preserve sibling marker pcaps.
Dynamips.create_nio is async def while BaseManager.create_nio is a sync
def. The previous fix only added the extra 'node' argument but did not
await the resulting coroutine, causing 'was never awaited' warnings and
passing a coroutine object instead of an NIO instance to the binding
dispatch. Add 'await' on the Dynamips branch. Test updated to verify
both the async nature and the parameter count.
The inspect check tested unbound function signatures (3 params unbound vs
2 unbound) but node.manager.create_nio is a bound method — inspect
excludes 'self'. Dynamips bound = 2 (node + nio_settings), standard
bound = 1 (nio_settings). The old '== 3' never matched, so the extra
'node' arg was never passed. Switch to '>= 2' and rewrite the test to
exercise the actual bound-method scenario.
Dynamips.create_nio requires the node as first positional argument
(unlike every other manager which takes only nio_settings). The batch
handler now detects this via parameter-count inspection (3 vs 2) and
passes node when needed.
Also add Dynamips to _add_nio_binding dispatch: routers use
slot_add_nio_binding(slot, port, nio), switches/hubs fall back to
add_nio(nio, port_number).
Add tests covering Dynamips router dispatch, switch dispatch, and the
create_nio signature detection to prevent regression.
Project open used to create each link by issuing two NIO POSTs from the
controller to the compute — ~5000 HTTP round-trips for a 2500-link
topology, all funnelling through the single shared controller/compute
event loop and capping throughput near 12 links/s.
Replace it with a bulk path:
- UDPLink split into _prepare() (local: ports, peer addrs, link_data)
and _commit_nios() (dispatch). create() = prepare + commit (interactive).
- Link.add_node gains batch=True: attach both nodes without triggering
per-link NIO HTTP.
- compute: new POST /projects/{id}/nios/batch endpoint with a unified
_add_nio_binding dispatch across node types (docker/qemu/iou/vpcs/
builtin differ in signature).
- project.open: prepare all links locally, group NIO entries by compute,
send each compute a single /nios/batch, then finalise (wire node/port
refs, mark created, notify, apply marker defs) in parallel.
Cuts controller->compute HTTP from O(links) to O(computes). Test added
for the batch endpoint.
The per-link create_marker route (links.py) called start_marker without the data_link_type from the body, so it always defaulted to DLT_EN10MB and a serial encapsulation chosen by the caller was silently dropped. Same oversight the IOU capture and definition routes had; one-arg fix.
Markers now work on serial links (Cisco HDLC / PPP / Frame Relay / ATM), not just Ethernet. A marker carries a data_link_type (default DLT_EN10MB); at the uBridge boundary it becomes the 'mark ... linktype <dlt>' keyword so the BPF compiles and the pcap is written with the matching link-layer.
- MarkerCreate / MarkerDefinitionCreate gain data_link_type (default DLT_EN10MB). Per-link it is create-only; definitions are updatable (a change re-fans-out).
- base_node._marker_linktype() normalizes the GNS3 DLT name (strip DLT_, uppercase, None for EN10MB). Single source is SerialPort.data_link_types, so Cisco PPP -> PPP_SERIAL (50), matching the capture path -- no second mapping table.
- _ubridge_add_marker_filter (generic) and the IOU marker loop append 'linktype <dlt>'.
- Definition fan-out branches on link_type in inherit_marker: Ethernet is always EN10MB; a serial link uses the definition's WAN encapsulation, or is SKIPPED when none was chosen (an EN10MB pcap on serial is undecodable). One definition covers a mixed topology.
- MCP marker_definition exposes data_link_type (None = not forwarded).
- No uBridge rebuild on a data_link_type change -- only that one marker's filter is swapped (delete + re-add), mirroring a BPF change; reset_packet_filters preserves sibling mark filters.
Requires the uBridge build with 'mark ... linktype' support.
The IOU compute capture/start route received data_link_type in the NodeCapture body but dropped it when calling node.start_capture(), so iou_vm.start_capture always defaulted to DLT_EN10MB -- every IOU serial capture (Cisco HDLC / PPP / Frame Relay) was written as an Ethernet pcap. Dynamips, cloud and the L2 switches already forward this value; IOU was the only serial-capable node that omitted it. One-argument fix: the node method already accepts and forwards data_link_type to 'iol_bridge start_capture', so only the route was missing it.
Verified: IOU serial captures now produce the expected linktype (C_HDLC / PPP_SERIAL / FRELAY) instead of Ethernet.
A marker definition fans out to every link and auto-selects its capture node
on each, so tx/rx is relative to a node that varies per link — the controller
already rejects it (409). Exposing direction on the MCP definition tool let an
agent ask for something that could only fail. Remove the parameter and the
handler's direction handling; the docstring now points to encoding direction in
the BPF (e.g. 'icmp and icmp[icmptype]==8'). Per-link link_marker keeps
direction, where the capture node is fixed.
Deleting a marker while its node was stopped, then starting the node, recreated
an empty pcap. Root cause: delete_marker_capture removed the uBridge filter and
the pcap file but not the marker spec cached on the port NIO (nio.markers) —
the data source _ubridge_apply_markers reads on node start. The stale spec
reinstalled the marker when uBridge came up.
This was a regression from switching stop_marker off update() (which re-sent
the NIO and implicitly refreshed nio.markers) to the fine-grained
node.delete(/markers/{name}) path.
Fix: make the delete port-aware so the compute can locate the NIO. The DELETE
marker route becomes /adapters/{a}/ports/{p}/markers/{name} across all six
node types; the handler resolves the NIO via get_nio and passes it to
delete_marker_capture, which now pops the marker from nio.markers. get_nio
works regardless of uBridge state, so the stopped-node case is covered. The
controller's stop_marker targets the capture side's adapter/port.
Deleting or updating a marker no longer triggers a full NIO reapply
(reset_packet_filters + re-add), which closed/reopened every sibling
marker's pcap via uBridge. Instead operate on single filters:
- stop_marker: bridge delete_packet_filter + unlink the pcap (works with
the node stopped; filter removal is skipped, the file is still deleted).
- update_marker: bpf/tag/direction → rebuild just that filter (delete + add);
enabled → instant toggle; color/highlight_duration → stored only.
- compute delete_marker_capture / rebuild_marker_filter + per-node routes
(DELETE /markers/{name}, PUT /markers/{name}/rebuild) + MarkerRebuild schema.
IOU overrides _ubridge_delete_marker_filter for iol_bridge; rebuild reuses
the already-overridden add/delete/set, so IOU needs no rebuild override.
The _marker_filter_bridges dict was keyed by marker name alone, so when one
node hosted the same filter name on several links (IUOL-BRIDGE per node with
many bays/units, or a multi-interface router), successive apply calls
overwrote earlier entries. pause_marker_definition then toggled only the last
recorded bridge/location — other copies stayed active and kept emitting.
Key by (name, link_id) so each copy is independent, and iterate all matching
entries in _ubridge_set_marker_filter_state (both generic bridge and IOU
iol_bridge override). Toggle route existence checks also iterate matching
names. Tests updated.
Toggle routes silently no-opped when _marker_filter_bridges lacked the filter
name, so update_marker's enabled-only short-circuit succeeded without toggling
uBridge — the controller-layer enabled was set but the uBridge filter stayed on
and kept emitting signals. Now the toggle routes raise HTTPException 404
(FastAPI handles it directly, no ERROR log); the controller's except catches it
and falls back to self.update() (NIO rebuild, which applies the marker + off).
The project-wide mute (POST /markers/pause|resume + _markers_paused) paused
every marker with one button. The actual need is per-rule control: pause one
definition and toggle only its inherited global-{name} copies across all links.
- Drop project-level: _markers_paused (init/asdict/load/start_all), the
pause_all/resume_all_markers methods, and the /markers/pause|resume routes.
- Add per-definition: a persisted `paused` flag on each definition;
pause/resume_marker_definition fan out update_marker(enabled) to every
global-{name} copy — instant, via the existing enable_packet_filter toggle
(no NIO rebuild, pcap/emitted preserved). New links inherit a paused
definition already off (inherit_marker passes enabled=not paused).
- start_marker takes an enabled kwarg; update_marker's enabled-only short-circuit
now also covers inherited copies so def pause/resume is instant.
- Routes: POST /marker-definitions/{name}/pause|resume.
- Docs + tests updated.
update_marker reused MarkerCreate, whose bpf is required, so a partial PUT like
{"enabled": false} was rejected with 422 "bpf field required". Add a MarkerUpdate
schema with every field optional (bpf included; capture_node_id and name are
create-only/path-driven and omitted) and use it for the PUT route — partial
updates now validate cleanly.
Wire gns3-server to uBridge's real-time marker controls (contract
../ubridge/doc/gns3server-integration.md §3.2), in three layers:
A. enabled reaches uBridge — _markers_for_node no longer drops disabled
markers controller-side; it carries `enabled` in the NIO spec. apply
installs every marker then issues `enable_packet_filter … off` for the
disabled ones (base_node bridge / iou iol_bridge; old-ubridge errors
downgrade to a warning so a toggle can't break link create).
B. instant per-filter toggle — apply records name→bridge so a new
`_ubridge_set_marker_filter_state` can flip a running filter with
`enable_packet_filter on|off` (iou overrides for iol_bridge + bay/unit).
Each marker-capable node type gains PUT /markers/{name}; update_marker
short-circuits to it when only `enabled` changes (no NIO rebuild, no pcap
flush), falling back to reset+reapply if the route is unavailable.
C. global pause/resume — `_ubridge_marker_pause/resume` send `marker pause`
/ `marker resume` direct to the hypervisor (pause stops signal+pcap,
resume instant, sink retained). Six node-type routes add POST
/markers/pause|resume; project.pause_all/resume_all_markers fan out to
each capture node (deduped, best-effort); REST exposes
POST /projects/{id}/markers/pause|resume.
Toggling enabled and pause/resume are now both instant — only marker create
or a bpf change still go through reset+reapply.
direction was settable but not clearable: once a marker or project-level
definition had direction=tx/rx, no update path could return it to "both
directions", and the definition-sync fan-out silently kept stale values on
every inherited copy.
Introduce a _UNSET sentinel (link.py) distinct from None so updaters can
tell "caller omitted direction" (preserve) from "caller passed None"
(clear). Thread it through UDPLink.update_marker and
Project.update_marker_definition; the two REST routes use Pydantic v2
model_fields_set to translate an explicit JSON null into the sentinel.
MCP follows with a "both" token: link_marker / marker_definition handlers
map direction="both" to a null in the REST body (tri-state: omit=preserve,
tx/rx=set, both=clear), and the tool descriptions/docstrings document it.
Backward compatible: omitting direction or passing tx/rx behaves exactly
as before; only an explicit null / "both" clears.
A marker is single-sided — only the chosen capture node's uBridge installs the
mark filter — and dir=tx|rx is interpreted from that node's perspective. Until
now the observer was always auto-picked (_choose_marker_side), so dir=tx meant
"the auto-chosen endpoint is sending", which is unpredictable and makes the
direction filter hard to render meaningfully in the Web UI.
Add an optional create-only capture_node_id to MarkerCreate: when set, the
marker is pinned to that endpoint's uBridge (validated as a link endpoint and a
marker-capable type); when omitted, behavior is unchanged (auto-pick). The
chosen id is already echoed back as capture_node_id and in MARK signals, so the
UI can always render the observer regardless of who picked it.
capture_node_id is create-only (changing it would silently flip the meaning of
stored direction; recreate instead) and is not accepted on project-level
definitions — they are link-agnostic and have no endpoints, so inherited
markers keep auto-picking per link.
Plumbed through REST create_marker, the MCP link_marker tool, and base
Link.start_marker. update_marker does not forward it.
Direction field in marker.match events
=======================================
Read ubridge dir=<tx|rx> from MARK signal datagrams and forward it
as a "dir" key in the marker.match notification event. The field is
additive -- older ubridge builds omit it and the parser leaves it null,
so consumers fall back to undirected rendering with no version coupling.
Semantics are relative to the capture node (the signal's node=<id>):
tx = capture node is sending (ingressed device-side NIO), rx = it is
receiving (ingressed link-side NIO).
Per-marker direction filter (opt-in, server-side pipeline)
==========================================================
Add a direction field to MarkerCreate and MarkerDefinitionCreate
schemas ("tx" | "rx" | null). Plumb it through the full pipeline:
Schema -> controller (start_marker/update_marker, marker_entry,
_markers_for_node, update_marker_definition sync)
-> REST/MCP handlers -> compute _ubridge_add_marker_filter +
IOU _ubridge_apply_markers -> bridge add_packet_filter dir <tx|rx>
When set, ubridge only fires the mark handler (signal + pcap) for
packets matching the chosen direction. null (default/legacy) = both
directions -- zero behavioural change for existing markers.
Docs and tests
==============
- docs/features/marker-traffic-insight.md: signal format updated,
new Direction section with NIO mapping, arrow mapping, and additive
compatibility note.
- tests/compute/marker/test_marker_manager.py: 3 new parser tests
(dir tx/rx/absent) plus existing test extended to assert dir=None.
13 files, +123/-22, 72 tests pass (zero breakage)
Cloud and NAT nodes need live host network interface data. Previously,
GET /projects/{project_id}/nodes/{node_id} returned cached properties
from creation time, so newly added host interfaces (e.g. kernel bridges
created by EthernetSwitch nodes) were invisible until the node was
deleted and recreated.
Now the controller fetches fresh data from the compute node before
returning the response, so host interface changes are reflected
immediately. Falls back to cached data if compute is unreachable.
Replace the builtin EthernetSwitch stub with a Linux kernel bridge backed
by uBridge's brctl module. Each switch node creates one kernel bridge
(gns3br{N}) with VLAN filtering; each port is a persistent TAP enslaved to
the bridge, relayed by a per-port uBridge bridge (nio_tap <-> nio_udp).
- Access/dot1q/qinq port modes translated to brctl vlan primitives
- Compute router repointed from Dynamips to Builtin manager
- Tests updated: 21 router-level tests + 215 surrounding tests pass
- Real-kernel e2e verified: access 100 PVID untagged, dot1q trunk
VIDs 1-4094 + native 1, qinq 802.1ad proto + 200 PVID
Ethertype 0x9100/0x9200 handling and port-count guard relaxation are
deferred pending resolution.
For Docker nodes, reload bottoms out as a raw POST /containers/{id}/restart
to the Docker daemon, bypassing GNS3's start/stop lifecycle (uBridge
re-attach, console servers, NIC setup). The container restarts at the
Docker level but GNS3's plumbing goes out of sync, and the daemon call can
block up to the controller's 240s timeout — manifesting as "reload hangs /
no response". stop+start runs the full lifecycle and is reliable.
Remove the node_reload and node_reload_all MCP tools (tool functions,
handlers, NODE_TOOLS entry, tests, docs). The underlying REST endpoints
(POST /nodes/{id}/reload, POST /nodes/reload) are kept for native API
users. MCP callers should use node_stop + node_start (partial) or
close/open project (full restart) instead.
A heredoc (<<<) closes stdin at once, so websocat dropped the WebSocket
before the device's reply arrived. Add --no-close to the node_console usage
examples and the returned command field so the connection stays open while
output is read.
get_current_active_user_from_websocket returns None after closing the socket
on an auth failure (revoked token, bad credentials, inactive user).
has_privilege_on_websocket dereferenced current_user.is_superadmin without a
None check, so any websocket auth failure surfaced as an AttributeError
traceback instead of a clean close. Bail out early when current_user is None,
mirroring the guard already present in ws_console.
Token revocation is a strict version check (token_data.token_version !=
user.token_version). create_access_token defaults token_version to 0, so the
short-lived JWTs minted for the console WebSocket URL (nodes) and the download
URLs (symbols, links) carried ver=0. Any user who had logged out at least once
(token_version >= 1, e.g. the default admin) got tokens rejected as "revoked"
on first use.
e433991cf fixed this in _resolve_token's API-key branch but missed these three
independent minting sites. Now resolve token_version during _resolve_token (the
JWT branch decodes it, the API-key branch reads user.token_version), carry it
through gns3_ctx, and pass it at every minting call.
_resolve_token generated a temp JWT with a hardcoded ver=0 after
validating the API key. Users who had logged out at least once
(token_version >= 1) would hit "Token has been revoked" 401 on
every MCP tool call, because the REST auth chain rejects ver=0
when the user's token_version no longer matches.
Fix: pass the user's actual token_version to create_access_token
so the temp JWT carries the correct ver claim.
The PUT /nio (update) endpoints for iou, dynamips and cloud copied nio.filters from the request body but never copied nio.markers. So adding a marker (e.g. a project-level global definition) to an already-created link of these node types left nio.markers empty: no mark filter was installed on uBridge and no marker.match signal was emitted. vpcs/qemu/docker already set it; mirror them (getattr for the Union NIO types).
The global-prefix reservation belongs on the create path, where it keeps
user-chosen names disjoint from inherited global-{name} markers. It was
implemented as a field_validator on MarkerCreate / MarkerDefinitionCreate,
which also back the PUT update bodies — so updating an inherited marker
while echoing its name in the body tripped the validator with a generic
422 before the controller could return the actionable 409 ("inherited,
use the marker-definitions API").
Drop the schema validators and enforce the prefix in the two create routes
(ControllerError -> 409). PUT no longer validates the body name (ignored
anyway — the target is the {name} path param), so editing an inherited
marker now reaches the controller's inheritance guard and returns the
clear 409. The name-format regex stays on the schema (still 422).
Per-marker UI hint (milliseconds, ge=1) for how long the Web UI keeps a
marker highlighted after a match. Mirrors color: stored on the link,
persisted in the topology, inherited via project-level definitions, and
never sent to uBridge. Omitted = null = frontend uses its own default.
The default is intentionally NOT set in the schema: MarkerCreate also
backs PUT updates, so a schema default would make every partial update
silently reset the value. None means "not provided" (keep existing on
update / frontend decides on create).
Project-level marker definitions fan out to every link (existing and new).
A definition is stored once on Project._marker_definitions; when applied to
a link the marker is named "global-{def_name}" — the "global" prefix was
pre-reserved in the schema, so inherited and per-link markers can never
collide, nor will their registry keys.
Key behavior:
- POST /projects/{pid}/marker-definitions → fan out to all existing links
- PUT /projects/{pid}/marker-definitions/{name} → sync all inherited copies
- DELETE → remove every inherited copy from every link
- New links auto-inherit all active defs (hook in UDPLink.create)
- Per-link DELETE/PUT of a "global-*" marker is rejected (409)
- Inherited markers are NOT persisted in the topology; they are re-created
from _marker_definitions on project load
- Compute side is untouched — the marker reaches uBridge via the existing
start_marker→update→NIO→ubridge pipeline
Files:
- controller/project.py — _marker_definitions + CRUD + fanout + topology load
- controller/link.py — Link.inherit_marker() + asdict() filter
- controller/udp_link.py — guards on stop/update + create() inheritance hook
- controller/topology.py — persist marker_definitions in project topology
- schemas/controller/links.py — MarkerDefinitionCreate schema
- api/routes/controller/projects.py — REST endpoints (marker-definitions)
Add a name pattern constraint to MarkerCreate schema (alphanumeric +
_.-) to prevent injection into uBridge commands, with a matching
defense-in-depth check in _ubridge_add_marker_filter for hand-edited
topology files. Also add the missing "enabled" field to MarkerCreate
and pass it through the PUT route so the documented toggle actually
works.
Auto-generating `marker-{link.id[:8]}` collided on the second anonymous
marker on the same link (start_marker rejects duplicate names). Append a
short uuid hex suffix so multiple anonymous markers coexist.
Add a read-only `markers` property on Project that flattens every link's
markers into a single dict keyed by "{link_id}/{name}", each entry
carrying the parent link_id and capture-side node_id. Expose it via
GET /projects/{pid}/markers (Project.Audit) so the frontend can fetch all
markers in one round-trip instead of enumerating links first.
Also surface a marker count in project.stats().
