Deleting or updating a marker no longer triggers a full NIO reapply
(reset_packet_filters + re-add), which closed/reopened every sibling
marker's pcap via uBridge. Instead operate on single filters:
- stop_marker: bridge delete_packet_filter + unlink the pcap (works with
the node stopped; filter removal is skipped, the file is still deleted).
- update_marker: bpf/tag/direction → rebuild just that filter (delete + add);
enabled → instant toggle; color/highlight_duration → stored only.
- compute delete_marker_capture / rebuild_marker_filter + per-node routes
(DELETE /markers/{name}, PUT /markers/{name}/rebuild) + MarkerRebuild schema.
IOU overrides _ubridge_delete_marker_filter for iol_bridge; rebuild reuses
the already-overridden add/delete/set, so IOU needs no rebuild override.
Restore a private marker's direction (and highlight_duration) when a
project is reopened — _create_link_from_topology_data previously dropped
them, silently reverting rx/tx markers to "both".
Reject tx/rx direction on marker definitions with HTTP 409: a definition
auto-selects its capture node per link and direction is relative to that
node, so a fixed tx/rx has no stable project-wide meaning. Per-link
markers still support tx/rx; only the project-wide definition is restricted
to "both" (the default).
The _marker_filter_bridges dict was keyed by marker name alone, so when one
node hosted the same filter name on several links (IUOL-BRIDGE per node with
many bays/units, or a multi-interface router), successive apply calls
overwrote earlier entries. pause_marker_definition then toggled only the last
recorded bridge/location — other copies stayed active and kept emitting.
Key by (name, link_id) so each copy is independent, and iterate all matching
entries in _ubridge_set_marker_filter_state (both generic bridge and IOU
iol_bridge override). Toggle route existence checks also iterate matching
names. Tests updated.
Toggle routes silently no-opped when _marker_filter_bridges lacked the filter
name, so update_marker's enabled-only short-circuit succeeded without toggling
uBridge — the controller-layer enabled was set but the uBridge filter stayed on
and kept emitting signals. Now the toggle routes raise HTTPException 404
(FastAPI handles it directly, no ERROR log); the controller's except catches it
and falls back to self.update() (NIO rebuild, which applies the marker + off).
_ubridge_set_marker_filter_state raised "Marker X is not installed on this
node" when the capture node wasn't running — the name->bridge map is only
populated during _ubridge_apply_markers, which runs when the node is up. The
error was noise: the controller's enabled-only short-circuit catches it and
falls back, and the controller-layer enabled is authoritative (honoured when
the node starts and applies the marker). Treat a missing entry as a no-op
instead of raising.
The project-wide mute (POST /markers/pause|resume + _markers_paused) paused
every marker with one button. The actual need is per-rule control: pause one
definition and toggle only its inherited global-{name} copies across all links.
- Drop project-level: _markers_paused (init/asdict/load/start_all), the
pause_all/resume_all_markers methods, and the /markers/pause|resume routes.
- Add per-definition: a persisted `paused` flag on each definition;
pause/resume_marker_definition fan out update_marker(enabled) to every
global-{name} copy — instant, via the existing enable_packet_filter toggle
(no NIO rebuild, pcap/emitted preserved). New links inherit a paused
definition already off (inherit_marker passes enabled=not paused).
- start_marker takes an enabled kwarg; update_marker's enabled-only short-circuit
now also covers inherited copies so def pause/resume is instant.
- Routes: POST /marker-definitions/{name}/pause|resume.
- Docs + tests updated.
update_marker reused MarkerCreate, whose bpf is required, so a partial PUT like
{"enabled": false} was rejected with 422 "bpf field required". Add a MarkerUpdate
schema with every field optional (bpf included; capture_node_id and name are
create-only/path-driven and omitted) and use it for the PUT route — partial
updates now validate cleanly.
pause/resume was fire-and-forget: the controller sent marker pause/resume but
stored nothing, so the Web UI could only keep a local optimistic flag that was
lost on panel reopen. Treat the mute as a project-level config (like per-marker
enabled): record _markers_paused, persist it in the topology (asdict + load),
and echo it on the project object so the UI renders from server truth.
Because marker pause is a uBridge runtime flag that resets on node restart,
start_all re-applies the mute to freshly started uBridges after a project
reopen — a paused project stays paused across close/reopen.
Wire gns3-server to uBridge's real-time marker controls (contract
../ubridge/doc/gns3server-integration.md §3.2), in three layers:
A. enabled reaches uBridge — _markers_for_node no longer drops disabled
markers controller-side; it carries `enabled` in the NIO spec. apply
installs every marker then issues `enable_packet_filter … off` for the
disabled ones (base_node bridge / iou iol_bridge; old-ubridge errors
downgrade to a warning so a toggle can't break link create).
B. instant per-filter toggle — apply records name→bridge so a new
`_ubridge_set_marker_filter_state` can flip a running filter with
`enable_packet_filter on|off` (iou overrides for iol_bridge + bay/unit).
Each marker-capable node type gains PUT /markers/{name}; update_marker
short-circuits to it when only `enabled` changes (no NIO rebuild, no pcap
flush), falling back to reset+reapply if the route is unavailable.
C. global pause/resume — `_ubridge_marker_pause/resume` send `marker pause`
/ `marker resume` direct to the hypervisor (pause stops signal+pcap,
resume instant, sink retained). Six node-type routes add POST
/markers/pause|resume; project.pause_all/resume_all_markers fan out to
each capture node (deduped, best-effort); REST exposes
POST /projects/{id}/markers/pause|resume.
Toggling enabled and pause/resume are now both instant — only marker create
or a bpf change still go through reset+reapply.
direction was settable but not clearable: once a marker or project-level
definition had direction=tx/rx, no update path could return it to "both
directions", and the definition-sync fan-out silently kept stale values on
every inherited copy.
Introduce a _UNSET sentinel (link.py) distinct from None so updaters can
tell "caller omitted direction" (preserve) from "caller passed None"
(clear). Thread it through UDPLink.update_marker and
Project.update_marker_definition; the two REST routes use Pydantic v2
model_fields_set to translate an explicit JSON null into the sentinel.
MCP follows with a "both" token: link_marker / marker_definition handlers
map direction="both" to a null in the REST body (tri-state: omit=preserve,
tx/rx=set, both=clear), and the tool descriptions/docstrings document it.
Backward compatible: omitting direction or passing tx/rx behaves exactly
as before; only an explicit null / "both" clears.
A marker is single-sided — only the chosen capture node's uBridge installs the
mark filter — and dir=tx|rx is interpreted from that node's perspective. Until
now the observer was always auto-picked (_choose_marker_side), so dir=tx meant
"the auto-chosen endpoint is sending", which is unpredictable and makes the
direction filter hard to render meaningfully in the Web UI.
Add an optional create-only capture_node_id to MarkerCreate: when set, the
marker is pinned to that endpoint's uBridge (validated as a link endpoint and a
marker-capable type); when omitted, behavior is unchanged (auto-pick). The
chosen id is already echoed back as capture_node_id and in MARK signals, so the
UI can always render the observer regardless of who picked it.
capture_node_id is create-only (changing it would silently flip the meaning of
stored direction; recreate instead) and is not accepted on project-level
definitions — they are link-agnostic and have no endpoints, so inherited
markers keep auto-picking per link.
Plumbed through REST create_marker, the MCP link_marker tool, and base
Link.start_marker. update_marker does not forward it.
Direction field in marker.match events
=======================================
Read ubridge dir=<tx|rx> from MARK signal datagrams and forward it
as a "dir" key in the marker.match notification event. The field is
additive -- older ubridge builds omit it and the parser leaves it null,
so consumers fall back to undirected rendering with no version coupling.
Semantics are relative to the capture node (the signal's node=<id>):
tx = capture node is sending (ingressed device-side NIO), rx = it is
receiving (ingressed link-side NIO).
Per-marker direction filter (opt-in, server-side pipeline)
==========================================================
Add a direction field to MarkerCreate and MarkerDefinitionCreate
schemas ("tx" | "rx" | null). Plumb it through the full pipeline:
Schema -> controller (start_marker/update_marker, marker_entry,
_markers_for_node, update_marker_definition sync)
-> REST/MCP handlers -> compute _ubridge_add_marker_filter +
IOU _ubridge_apply_markers -> bridge add_packet_filter dir <tx|rx>
When set, ubridge only fires the mark handler (signal + pcap) for
packets matching the chosen direction. null (default/legacy) = both
directions -- zero behavioural change for existing markers.
Docs and tests
==============
- docs/features/marker-traffic-insight.md: signal format updated,
new Direction section with NIO mapping, arrow mapping, and additive
compatibility note.
- tests/compute/marker/test_marker_manager.py: 3 new parser tests
(dir tx/rx/absent) plus existing test extended to assert dir=None.
13 files, +123/-22, 72 tests pass (zero breakage)
Use the node id (UUID) instead of an incrementing counter for the unix
control socket name, so each socket identifies its owning node at a glance
(one ubridge per node => node_id is unique). Falls back to the counter only
when no node id is supplied. A single UUID fits sun_path's 107-byte cap
(~69 bytes), so no project_id is needed.
start() now checks whether ubridge exited immediately after launch. An
old ubridge build that does not understand -U exits with a non-zero code
right away; surface the reason from ubridge.log instead of waiting for
connect() to time out 10s later with a confusing "couldn't connect" error.
Add ubridge_control_transport to [Server] config (default "tcp", fully
backward compatible). Selecting "unix" switches the uBridge hypervisor
control channel from the unauthenticated TCP listener (-H) to an AF_UNIX
socket (-U) authenticated in-kernel via SO_PEERCRED.
- UBridgeHypervisor: supports both socket_path (AF_UNIX) and host/port
(TCP); a new `endpoint` property unifies log/error strings for both
- Hypervisor: unix mode allocates a short socket path under a 0700
private runtime dir and unlinks it on stop; tcp mode restores the
original getaddrinfo ephemeral-port allocation
- base_node: reads the transport from config and passes host through
- schemas/config.py + config_samples/gns3_server.conf: new option
Default deployments are unchanged. "unix" requires a ubridge build that
understands -U.
Filter out host interfaces whose name starts with 'gns3' (e.g.
EthernetSwitch kernel bridges) from Cloud.asdict() interfaces list.
Other special interfaces (virbr0, docker0, etc.) remain visible.
NAT now includes an 'interfaces' field in its response, filtered to
contain only the mapped NAT interface (virbr0 on Linux, vmnet8 on
macOS/Windows). This lets connected nodes discover the NAT subnet
and gateway address without needing to list all host interfaces.
The field format mirrors Cloud.asdict(): name, type, special, and
ip_addresses list (both IPv4 and IPv6).
Cloud and NAT nodes need live host network interface data. Previously,
GET /projects/{project_id}/nodes/{node_id} returned cached properties
from creation time, so newly added host interfaces (e.g. kernel bridges
created by EthernetSwitch nodes) were invisible until the node was
deleted and recreated.
Now the controller fetches fresh data from the compute node before
returning the response, so host interface changes are reflected
immediately. Falls back to cached data if compute is unreachable.
Replace psutil.net_if_addrs() scan with a deterministic name derived from
the switch's node UUID: gns3 + first 6 hex chars (10 chars, fits kernel
IFNAMSIZ limit of 15). Taps: <bridge>-<port> (12-13 chars).
Crash recovery: brctl delete the bridge first (best-effort), then create
fresh. Stale bridges from abnormal gns3server shutdown are automatically
reclaimed on the next start — no EEXIST or leaked interfaces.
Remove _free_iface helper and psutil import (no longer needed).
brctl create leaves the bridge administratively DOWN. Add link set up
so the bridge actually forwards frames between enslaved ports.
Also record Docker iptables FORWARD DROP pitfall in project memory.
Replace the builtin EthernetSwitch stub with a Linux kernel bridge backed
by uBridge's brctl module. Each switch node creates one kernel bridge
(gns3br{N}) with VLAN filtering; each port is a persistent TAP enslaved to
the bridge, relayed by a per-port uBridge bridge (nio_tap <-> nio_udp).
- Access/dot1q/qinq port modes translated to brctl vlan primitives
- Compute router repointed from Dynamips to Builtin manager
- Tests updated: 21 router-level tests + 215 surrounding tests pass
- Real-kernel e2e verified: access 100 PVID untagged, dot1q trunk
VIDs 1-4094 + native 1, qinq 802.1ad proto + 200 PVID
Ethertype 0x9100/0x9200 handling and port-count guard relaxation are
deferred pending resolution.
For Docker nodes, reload bottoms out as a raw POST /containers/{id}/restart
to the Docker daemon, bypassing GNS3's start/stop lifecycle (uBridge
re-attach, console servers, NIC setup). The container restarts at the
Docker level but GNS3's plumbing goes out of sync, and the daemon call can
block up to the controller's 240s timeout — manifesting as "reload hangs /
no response". stop+start runs the full lifecycle and is reliable.
Remove the node_reload and node_reload_all MCP tools (tool functions,
handlers, NODE_TOOLS entry, tests, docs). The underlying REST endpoints
(POST /nodes/{id}/reload, POST /nodes/reload) are kept for native API
users. MCP callers should use node_stop + node_start (partial) or
close/open project (full restart) instead.
A heredoc (<<<) closes stdin at once, so websocat dropped the WebSocket
before the device's reply arrived. Add --no-close to the node_console usage
examples and the returned command field so the connection stays open while
output is read.
get_current_active_user_from_websocket returns None after closing the socket
on an auth failure (revoked token, bad credentials, inactive user).
has_privilege_on_websocket dereferenced current_user.is_superadmin without a
None check, so any websocket auth failure surfaced as an AttributeError
traceback instead of a clean close. Bail out early when current_user is None,
mirroring the guard already present in ws_console.
Token revocation is a strict version check (token_data.token_version !=
user.token_version). create_access_token defaults token_version to 0, so the
short-lived JWTs minted for the console WebSocket URL (nodes) and the download
URLs (symbols, links) carried ver=0. Any user who had logged out at least once
(token_version >= 1, e.g. the default admin) got tokens rejected as "revoked"
on first use.
e433991cf fixed this in _resolve_token's API-key branch but missed these three
independent minting sites. Now resolve token_version during _resolve_token (the
JWT branch decodes it, the API-key branch reads user.token_version), carry it
through gns3_ctx, and pass it at every minting call.
_resolve_token generated a temp JWT with a hardcoded ver=0 after
validating the API key. Users who had logged out at least once
(token_version >= 1) would hit "Token has been revoked" 401 on
every MCP tool call, because the REST auth chain rejects ver=0
when the user's token_version no longer matches.
Fix: pass the user's actual token_version to create_access_token
so the temp JWT carries the correct ver claim.
_start_ubridge configures the marker sink via _ubridge_configure_marker_sink, which used _ubridge_send. But _start_ubridge itself is reached THROUGH _ubridge_send when uBridge is started lazily — e.g. linking a stopped node. _ubridge_send's @locking lock (___ubridge_send_lock) is non-reentrant, so the nested _ubridge_send('marker sink') deadlocked forever, making the NIO create POST time out after 120s. Started nodes never hit this (uBridge already running, no _start_ubridge), so only stopped-node links (notably dynamips) hung.
_ubridge_configure_marker_sink now talks to self._ubridge_hypervisor.send directly: it runs inside _start_ubridge right after connect(), so uBridge is already up and the raw send is safe, with no reentrant lock acquisition.
A uBridge MARK signal carries only node= and filter= (no bridge/link field), so when one node is the capture side for several links that share a marker name (always the case for global-{name} definitions on a multi-interface node) the signals were indistinguishable and the (node, filter) registry collapsed them to a single link.
The mark filter is now stamped with its link id (mark <bpf> ... link <link_id>); uBridge echoes it verbatim (link=<link_id>) and the listener uses the signal's link= as the authoritative link_id of the marker.match event, falling back to the registry only for legacy signals without it. base_node and iou apply paths pass link_id; covered by two new listener tests.
The PUT /nio (update) endpoints for iou, dynamips and cloud copied nio.filters from the request body but never copied nio.markers. So adding a marker (e.g. a project-level global definition) to an already-created link of these node types left nio.markers empty: no mark filter was installed on uBridge and no marker.match signal was emitted. vpcs/qemu/docker already set it; mirror them (getattr for the Union NIO types).
The global-prefix reservation belongs on the create path, where it keeps
user-chosen names disjoint from inherited global-{name} markers. It was
implemented as a field_validator on MarkerCreate / MarkerDefinitionCreate,
which also back the PUT update bodies — so updating an inherited marker
while echoing its name in the body tripped the validator with a generic
422 before the controller could return the actionable 409 ("inherited,
use the marker-definitions API").
Drop the schema validators and enforce the prefix in the two create routes
(ControllerError -> 409). PUT no longer validates the body name (ignored
anyway — the target is the {name} path param), so editing an inherited
marker now reaches the controller's inheritance guard and returns the
clear 409. The name-format regex stays on the schema (still 422).
delete_marker_definition removed the def but left the inherited global-*
copies on every link: it called stop_marker(), which rejects inherited
markers (409), and the ControllerError was swallowed as a warning. The
orphaned copies were in-memory only (_persist_markers filters inherited
markers) so a restart hid the symptom, but during a running session they
were undeletable via either the per-link or project API.
Add the same inherited=True bypass that update_marker already has, and
pass it from the def-delete fan-out so the copies are removed for real.
Per-marker UI hint (milliseconds, ge=1) for how long the Web UI keeps a
marker highlighted after a match. Mirrors color: stored on the link,
persisted in the topology, inherited via project-level definitions, and
never sent to uBridge. Omitted = null = frontend uses its own default.
The default is intentionally NOT set in the schema: MarkerCreate also
backs PUT updates, so a schema default would make every partial update
silently reset the value. None means "not provided" (keep existing on
update / frontend decides on create).
Bugs found via end-to-end testing of project-level marker definitions:
1. New links didn't inherit — apply_defs_to_new_link is async but was
called without await in UDPLink.create().
2. Project load crashed — load_project passed marker_definitions to
Project.__init__. Now popped in load_project and restored separately
in Project.open() (it backs a read-only property).
3. PUT on a definition didn't sync to links — update_marker's guard
rejected even the project-layer sync call. Added an `inherited`
bypass flag used by update_marker_definition.
4. _ubridge_add_marker_filter raised re.PatternError — the name regex
used (?i)(?!global) look-around, invalid in Python's re module.
Dropped the prefix check there: "global-*" names are legitimate at
the uBridge boundary (inherited definitions); forbidden only at the
user-facing schema.
5. GET /links hid inherited markers — asdict()'s runtime branch used
_persist_markers() (which filters inherited markers). Restored
self._markers for the runtime branch; only the topology_dump branch
filters (inherited markers are rebuilt from definitions on load).
6. Duplicate "already exists" warnings on project open — open() fanned
out definitions to all links, but UDPLink.create() had already done
so via its inheritance hook. Removed the redundant fan-out in open().
pydantic_core (Rust engine) does not support regex look-around.
Keep the character-class pattern for basic safety and enforce the
"global" prefix reservation via a field_validator instead.
Project-level marker definitions fan out to every link (existing and new).
A definition is stored once on Project._marker_definitions; when applied to
a link the marker is named "global-{def_name}" — the "global" prefix was
pre-reserved in the schema, so inherited and per-link markers can never
collide, nor will their registry keys.
Key behavior:
- POST /projects/{pid}/marker-definitions → fan out to all existing links
- PUT /projects/{pid}/marker-definitions/{name} → sync all inherited copies
- DELETE → remove every inherited copy from every link
- New links auto-inherit all active defs (hook in UDPLink.create)
- Per-link DELETE/PUT of a "global-*" marker is rejected (409)
- Inherited markers are NOT persisted in the topology; they are re-created
from _marker_definitions on project load
- Compute side is untouched — the marker reaches uBridge via the existing
start_marker→update→NIO→ubridge pipeline
Files:
- controller/project.py — _marker_definitions + CRUD + fanout + topology load
- controller/link.py — Link.inherit_marker() + asdict() filter
- controller/udp_link.py — guards on stop/update + create() inheritance hook
- controller/topology.py — persist marker_definitions in project topology
- schemas/controller/links.py — MarkerDefinitionCreate schema
- api/routes/controller/projects.py — REST endpoints (marker-definitions)
Dynamips needed its own NIO base class extended with a `_markers`
attribute and `markers` property (mirroring the existing `filters`
pattern), plus marker propagation from the controller-supplied NIO
settings through NIOUDP.create/update to the common destination
NIO that base_node._ubridge_apply_markers reads.
Cloud simply needed the missing _ubridge_apply_markers calls after
_ubridge_apply_filters in its two ubridge connection methods, same
pattern as the earlier docker fix.
Both types are now added back to _MARKER_CAPABLE_TYPES.
