Compute:
- Track the anchor that owns uBridge's single AF_PACKET capture slot: a
second kernel port's capture is refused with a clear error (its flag
rolled back), a failed start no longer wedges the port, and only the
owning port's stop issues the process-wide `capture stop_kernel`
(docker/qemu/iou/dynamips/switch, via the mixin's slot helpers).
- Unwind failed starts instead of leaking uBridge and the anchor TAPs:
QEMU's stop body runs under try/finally, QEMU and IOU start the datapath
inside the launch try, Dynamips undoes a half-attached port (the retry
used to be short-circuited into a silent black hole), and Docker deletes
the veth pair when the namespace move fails.
- Restore the relay filter guard on IOU (kernel-only filters raise
NodeError, an uncompilable BPF line degrades to a warning), validate the
eBPF/quota/window/bpf filter shapes before touching the bridge (short or
non-numeric values surfaced as 500s after partial state), and refuse a
second NIO on an occupied Docker/QEMU port (the guard IOU and Dynamips
already had).
- Probe fixes: `_probe_bridge_tap` survives a failing uBridge start, and
the probe caches key on ctime too, so a chmod/setcap remedy re-probes.
Controller:
- Parse GNS3_UNIX_SOCKET_NIO like the compute does (line-anchored, truthy
value) and mirror the compute's class selection in kernel_anchor_type.
- Read partial capability reports defensively; make BridgeNIO.bridge
required-but-nullable so an omitted key is a 422 rather than a KeyError.
- Strip the packet_loss correlation slot even when it is zero: the relay
filter takes exactly one argument and rejected `packet_loss 10 0`.
Ethernet switch: skip ports whose absorbed anchor does not exist yet (the
join is deferred by design) and roll the ports_mapping back when the VLAN
reconcile fails, so a retry still diffs against the old mapping.
Agent: reject device config entries without a device_name instead of
silently dropping them (a malformed batch used to execute partially while
reporting success).
Also split the netem builder to clear C901 and drop the two per-file ruff
ignores the PR had added, keeping the remove-only policy.
The rebase onto upstream 3.1 (mypy + ruff I001/F401 waves) left the tree
short of the newly enforced baselines in three ways:
* contextlib / UbridgeError imports dropped as false duplicates during
conflict resolution left NameErrors in iou_vm and qemu_vm (F821);
the Dynamips router grabbed the compute-level NIOTAP instead of the
hypervisor one, breaking TAP anchoring (test failure).
* unused imports (KERNEL_UNSUPPORTED_FILTERS, UbridgeError, List) and
unsorted blocks left behind where the stack's later commits moved the
last usage elsewhere — ruff I001/F401/RUF022 now enforce these.
* mypy, now baseline-free: cache dict annotations in tc_probe, the
locators candidate list, the resolved-params None branch in the MCP
runner, and validate_bpf_syntax keeps the upstream Dict[str, Any]
return (the strict Optional[str] was wrong for the "valid" bool entry).
The e2e raw-frame probe resolves its compiler via shutil.which (S607)
and zips with strict= (B905); the remaining subprocess call is S603 by
nature and gets a per-file ignore like other test files.
ruff check/format clean, mypy at upstream parity (no new errors),
full unit suite 2167 passed.
_is_unix_socket_docker no longer excludes the IOL runner marker: those
containers anchor on persistent TAPs now (compute side), so they
graduate through _kernel_endpoint_ready's gate instead — both
ubridge_bridge_tap (the swappable leg) and ubridge_tap (the tap module)
strict True, an unreported capability keeping the link on the relay
where it always works. The generic GNS3_UNIX_SOCKET_NIO exclusion keeps
its shape (an IOL runner environment wins the compute's class
selection, so the exclusion must not fire when both markers are set).
The switch fast path names an IOL runner peer's anchor through
kernel_anchor_type — the iol_docker key, never the gv veth namespace.
The 307d676cb regression tests move with the semantics: the capability
matrix replaces the unconditional relay, and the switch-path test also
asserts the gx naming. Its original concern stands — a compute that
cannot serve the anchor lifecycle must never be handed a kernel link
(the switch's deferred join would make it a silent dead cable).
Switch-to-switch links were the last shape left on the relay: the switch
fast path works by absorbing the *peer's* anchor, and a switch has none —
its ports are its own bridge, and a kernel interface belongs to exactly
one bridge, so two bridges cannot be joined by enslaving anything.
Join them with a veth pair instead: one end per bridge, each side
enslaving its own end and applying its own port mode to it — an access
port on one side and a trunk on the other behaves like the real cable it
models. uBridge's docker create_veth/delete_veth (the primitive every
Docker adapter is already born from) creates and destroys the pair; the
names derive from the link id (utils.kernel_anchor.kernel_cascade_names),
so each side computes its own end and the names survive a restart.
The pair belongs to the link, and the two ends are wired by two
independent uBridge processes — the project-open batch drives different
nodes concurrently — so the protocol is order-free: both sides create the
pair when their own end is missing (one create wins; the loser's failure
is re-checked against the kernel and "the pair is there" is not an
error), and either side's teardown destroys it (deleting one veth end
takes the other with it, so both converge on the same gone pair and the
second delif answers ENOENT). A pre-existing end — this switch restarted,
or a leftover from a previous life of the link id — is reused rather than
recreated, because the other end may still be a live port of the peer
switch; a stale tc qdisc is all a reused end can carry.
No new wire format was needed: a cascade is an anchor NIO on both ends,
the absorbed-anchor case plus the peer end's name. That also corrects the
filter routing — the single-sided rule exists because an absorbed anchor
is one interface, while a cascade has one per switch, so impairments go
to both ends like any other kernel link.
Tests: the wire format, the eligibility flip and the both-ends re-push on
node start (controller); the naming contract (utils); the switch-side
lifecycle — create, reuse, lost create race, delete destroys the pair
(compute routes).
Drive-by: the relay filter test's compute mock carried no awaitable
`put`, so tests/controller/test_kernel_datapath_link.py failed when run
alone while an argument-less full run passed; callers now get one.
The unix-socket container gate matched GNS3_UNIX_SOCKET_NIO in the node
environment only, but the IOL runner reaches the same wiring through the
GNS3_IOL_RUNNER marker: the compute selects IOLDockerVM on it, and that
class forces the AF_UNIX socket pairs whatever the generic knob says —
and the documented environment for those nodes carries only the marker.
Such a container was therefore handed a kernel link its compute cannot
serve. The plain case failed at link creation (no veth end to enslave);
on the switch fast path it was worse — the switch defers a join against a
missing anchor by design (a link to a stopped node must not fail), so the
cable was silently dead instead of merely slow.
Match both markers, through the shared is_iol_runner_environment helper so
the controller and the compute select the same way. Verified live on a
real server: docker↔IOL-XE and switch↔IOL-XE links stay relay while the
docker↔switch control link still takes the kernel datapath.
The Ethernet switch's kernel fast path: instead of a link crossing the
switch's per-port uBridge relay, the switch absorbs the peer's anchor into
its own kernel bridge. The peer's interface (Docker veth host end, QEMU
TAP, IOU port TAP, Dynamips port TAP) becomes the switch port, and frames
cross anchor -> switch bridge -> switch ports entirely in the kernel.
Wire format: the switch end receives nio_anchor ('join this interface to
my own bridge' — the mirror of nio_bridge), the peer end nio_bridge with a
null bridge ('your anchor is bridged elsewhere': carrier, markers and
impairments only — never bridge membership, and never a bridge delete, on
an interface this node does not own). Anchor names come from the shared
naming contract (utils.kernel_anchor), a pure function of node type, id,
adapter and port — the controller computes the peer's anchor while the
node may be stopped, which is what makes links to stopped nodes wireable.
Stopped peers defer the join; after any node of a link starts
(UDPLink.node_started) the controller re-pushes the switch NIO, and the
switch completes — or repairs, when a peer restart replaced its interface
under the same name — the join against the kernel's actual membership
instead of assuming it.
VLAN changes now edit membership in place: only the ports whose mode/VLAN
actually changed are touched, each losing exactly the previous mode's
entries the new mode does not want (the dot1q admits-all entry is one
range operation in both directions), and no port ever leaves the bridge to
change its VLANs. This replaces the old 'reset by re-enslaving' shortcut
(delif+addif on every wired port, added when the ports were the switch's
own TAPs): on absorbed anchors, re-enslaving an interface owned by another
node is where live testing watched links die — anchors occasionally lost
their bridge membership right after a ports_mapping update (later updates
then failed with delif EINVAL), reproducibly with qemu, IOU and Dynamips
peers and never with Docker's veths.
Eligibility: the switch side needs no capability of its own (its bridge
exists by construction); the peer must be able to anchor. Switch-to-switch
cascades stay on the relay — a kernel interface belongs to one bridge, and
joining two bridges needs a veth pair, deferred.
Verified live on a real server: qemu peers through four VLAN cycles with
both anchors enslaved throughout; a mixed switch (2 IOU + 2 Dynamips, four
kernel ports); Docker peers with real ICMP isolating and recovering (with
both anchors still enslaved); VPCS relay ports likewise in place.
A Dynamips Ethernet port can anchor a kernel link whenever its compute's
uBridge has the tap module — the anchors are uBridge-created TAPs the
hypervisor merely opens, the same capability QEMU gates on, so no
Dynamips-specific capability is needed. Serial, ATM and POS ports are
excluded per port by the existing link-type check (the controller's port
matrix decides), keeping them on the relay whatever the capabilities.
Eligibility tests cover the mixed pairs (docker/qemu/iou endpoints on the
same compute), the strict tap-capability requirement, and the serial
exclusion.
Eligibility gains an IOU branch gated on the compute's ubridge_iol_tap
(strict True: an unreported capability keeps the link on the relay), and
a port-level rule: a kernel link is an Ethernet segment, so any endpoint
whose port is not Ethernet (IOU serial) disqualifies the link before it
is dispatched. Without that, a serial link between two capable IOU nodes
would be sent as nio_bridge and fail at the compute.
The IOU NIO routes accept nio_bridge alongside the UDP/TAP/Ethernet NIOs
— without the widening the controller's POST is 422'd before the node
ever sees it, the same trap the QEMU routes hit.
Kernel eligibility becomes "both endpoints on one compute, each able to
anchor": Docker always (adapters are born as veth pairs), QEMU when that
compute's uBridge can create the persistent TAPs its adapters anchor on. The
question is asked per compute, because the link needs both ends anchored --
so a mixed or half-upgraded deployment degrades to the relay instead of
failing an attach.
probe_tap_support() creates and deletes one throwaway persistent TAP through a
scratch uBridge, cached by binary identity like the tc report; /capabilities
exposes `ubridge_tap` (absent = unknown, which keeps links on the relay).
The compute route for a QEMU adapter NIO took UDPNIO only, so a kernel link
was rejected with 422: its body and response now accept
Union[UDPNIO, BridgeNIO] exactly like Docker's.
Stale "Docker-to-Docker" wording in the config, link and MCP descriptions now
talks about anchors, and docs/features/qemu-kernel-datapath.md documents the
datapath -- verified end-to-end on a real instance: 32/32 kernel-datapath
checks and 29/29 on the relay variant.
Consume the eBPF impairment classifier delivered in uBridge
feature/tc-precision: frequency_drop becomes the exact every-Nth mode
(tc nth_drop; -1 = drop everything maps to every 1st) and the new
kernel-only quota type is the byte-cap mode (tc quota_drop, 100 = hard
cutoff). KERNEL_UNSUPPORTED_FILTERS is gone entirely — kernel/relay
eligibility is now purely topological. Both modes gate on ebpf=1 from
the per-process tc capabilities probe (error names the setcap fix); the
cleanup path (modes off) only talks to the eBPF commands when the probe
reported them. Reconcile is a plain re-set: the netem reset-before-set
already detaches the program and clears uBridge's per-interface
registry, so a mode set after it is a fresh load+attach.
Documented semantic difference: the relay counts both directions
through one bridge counter, the kernel counts per direction — N=3
round-trip loss is 1-(2/3)^2 = 55.6% on kernel vs ~1/N on relay
(measured 57%). window_drop is NOT exposed: the delivered
implementation advances the window with no gap, making 'outside the
window packets pass' unreachable (needs outage/period fields —
recorded in the spec); flow_drop awaits a parameter-shape decision.
Also fixes a pre-existing restart bug the e2e exposed: _bridges (relay
bridge names) was never cleared on stop, so restarting a node with a
relay link skipped 'bridge create' and failed on add_nio_udp.
e2e (isolated instance): 13/13 — pref-1 tc_impair classifier on both
veths, 57% round-trip for N=3, -1 = exact 100%, quota hard cutoff,
delay coexistence, clean teardown, suspend/resume, node restart,
relay-leg frequency_drop regression through an Ethernet switch. Full
suite: 1925 passed.
Surface the netem extensions delivered in uBridge feature/tc-netem-ext
as GNS3 filter types: rate (tc-style bandwidth string), reorder (needs
delay), gemodel (Gilbert-Elliot bursty loss, exclusive with
packet_loss), duplicate, seed and limit, plus the delay jitter
distribution and the loss/dup correlation parameters. All are
kernel-datapath only — the relay's packet-filter registry has no
equivalent, so available_filters hides them on relay links, setting one
on a created relay link returns 409, and the project-load path drops
them with a warning (update_filters runs before the datapath is
decided, so it cannot reject there). Validation mirrors the tc grammar:
reorder requires delay, gemodel and packet_loss are mutually exclusive,
a distribution requires jitter > 0, rate is integer+unit capped at
100gbit.
The translation in DockerVM._netem_command_parts emits the frozen
grammar order and gates extension keywords on the 'tc capabilities'
netem token list (probed once per uBridge process; a plain
delay/loss/corrupt/dup filter never probes, keeping old uBridge builds
untouched; the correl suffix has no token and gates on 'rate' as the
extension-build marker).
Reconcile became reset-before-set: the kernel's netem change merges
optional attributes (rate, correlation, reorder, corrupt, gemodel,
distribution, seed keep their previous value when the new message omits
them), so removing a parameter needs an explicit 'tc reset' — the
bpf_drop filters are re-added right after in the same flow, so the
clsact churn is invisible. The capability check runs before the reset
so a failed update leaves the previous qdisc in place.
e2e (isolated instance, kernel link): 24/24 — rate 512kbit adds exactly
2x the per-packet serialization to the RTT (45.5 ms for 1400-byte
pings), paretonormal jitter + rate-cleared leak check, reorder gap 5,
gemodel 50% bursty loss, duplicate, loss 30% correl 50%, seed 42 +
limit 5000 visible in the qdisc dump, suspend/resume and node restart
keep the filters, relay link rejects rate with 409, teardown leaves no
netem. Full unit suite: 1916 passed.
Impairment filters with a netem equivalent (delay, packet_loss, corrupt)
now run on the kernel datapath as one tc netem qdisc per veth host end,
pushed through uBridge's netlink tc module (tc netem set / tc reset).
This removes the relay delay filter's nanosleep bottleneck (upstream
Both link endpoints carry the filters: each end's qdisc egress covers
the traffic entering that container, so every direction is impaired
exactly once -- the same net effect as the relay, where both directions
cross the single filtered bridge.
- controller: eligibility keys on relay-only types (frequency_drop, bpf)
instead of any filter; kernel links push the real filters (the
synthetic suspend frequency_drop no longer flips a suspended link to
relay on reopen/reset); update_filters 409s only relay-only types;
available_filters hides them on kernel links
- compute: create_nio whitelists netem-capable filter types; the NIO
update path keeps its own guard (PUT mutates the NIO in place);
_ubridge_apply_netem translates the filter dict into one atomic
netem-set (full re-apply IS the reconcile); node restart restores the
qdisc, link deletion detaches it (the veth survives the link)
- e2e (27/27): delay 100 -> 200.2 ms RTT, atomic replace 50 -> 100.2 ms,
clear -> baseline, packet_loss 30 -> 48 % round-trip (expected 51 %),
suspend/resume and node restart keep the qdisc, link delete leaves no
residual impairment
Every Docker adapter is now born as a veth pair at container start
(unified interface) regardless of NIO type: guest end in the container
namespace, host end in the root namespace. Kernel links enslave the host
end into the per-link Linux bridge; relay links attach it to the uBridge
relay via AF_PACKET (bridge add_nio_ethernet) — the TAP path is gone.
This makes the datapath a runtime decision:
- links can be created on running containers (kernel or relay) without
touching container interfaces — the controller stopped-node
eligibility check is removed
- project reopen re-prepares relay links and upgrades eligible ones to
the kernel datapath
- packet capture start/stop now key on the NIO type (not veth presence)
so relay-over-veth captures use the relay capture, kernel ones the
AF_PACKET module
- vendor unix-socket containers keep their wiring: no veth is required
on the relay path
Markers no longer disqualify the kernel datapath. They ride the NIO like
on the relay path and attach to the veth host end through uBridge's
marker add_kernel/delete_kernel/enable_kernel commands (AF_PACKET taps,
same BPF/direction/signal semantics as the relay mark filter).
- base_node: marker anchors are now polymorphic (relay bridge name or
kernel veth host interface); the reconcile loop is datapath-agnostic
and DockerVM overrides the add/delete/enable primitives to translate
the commands. Marker-name validation extracted for both add paths.
- docker_vm: kernel NIO connect/update apply markers on the veth anchor;
link deletion tears markers down explicitly (the veth survives the
link, unlike a relay bridge whose death drops its filters).
- controller: eligibility and start_marker no longer exclude markers or
project-level marker definitions; the kernel NIO carries the capture
node's marker specs.
- NIO creation accepts markers on kernel NIOs (filters still rejected).
Replace the capture rejection with wiring to uBridge's capture module:
an AF_PACKET socket bound to the veth host end writes into the same pcap
file the relay path uses (capture start_kernel <if> "<pcap>" /
capture stop_kernel), so the pcap streaming to clients is unchanged.
- capture survives node restarts: _connect_nio re-arms it when the NIO
is already capturing, mirroring the relay path
- uBridge's kernel capture is a singleton per process: a concurrent
second capture on the same node surfaces uBridge's EALREADY error
- update the datapath docs: filters and markers remain relay-only,
capture is now served on both datapaths
update_suspend() piggybacks on the filter system: get_active_filters()
returns {"frequency_drop": [-1]} while a link is suspended so relay-based
node types drop all traffic at the uBridge filter. Kernel-datapath links
implement suspend natively via interface carrier, and pushing the
synthetic filter tripped the compute-side guard, making suspend fail
with 409 on every kernel link (and leaving controller/kernel state torn).
_get_node_filters() now returns empty filters for kernel links.
Eligible links (both endpoints Docker, same compute, no filters/markers,
stopped nodes, no unix-socket NIO containers) are wired as veth pairs
enslaved into a per-link Linux kernel bridge (gns3 + 11 hex of the link
id, filling IFNAMSIZ) instead of the uBridge UDP relay, so frames flow
entirely in the kernel.
- NIOBridge NIO type: the controller decides the datapath in
UDPLink._prepare and both computes enslave their deterministic gv*/gc*
veth pair via uBridge's docker/brctl modules
- carrier/suspend = host-end admin state; both endpoints race-safely
create/delete the bridge (EEXIST/EBUSY/ENOENT tolerated)
- veth adapters can still attach relay (UDP) links on a running node:
lazy relay bridge over AF_PACKET on the host end (add_nio_ethernet)
- filters/markers/capture are rejected with explicit errors on this
datapath (they live in the uBridge relay); vendor unix-socket
containers are excluded; Server.enable_kernel_datapath is the
kill-switch (default on)
- host-end veths survive container death and are removed on stop plus
swept by name on the next start (crash/stale recovery)
The brctl Ethernet switch runs a per-port uBridge relay, so it can host
the mark filter and packet filters like any uBridge-backed node. Add
ethernet_switch to _MARKER_CAPABLE_TYPES and _get_filter_node, narrow the
UDPLink.update() NIO-PUT skip down to the Dynamips-hosted ethernet_hub,
and expose the matching compute endpoints: PUT nio (filter/marker
reapply) plus the per-marker toggle/pause/resume/delete/rebuild routes.
The ethernet_hub keeps its exclusion: its routes still wire into the
Dynamips hub, which has no uBridge of its own.
PortManager.get_free_udp_port had an unguarded find-then-add sequence.
A link allocates both ends concurrently (asyncio.gather in
UDPLink._prepare -> two POST /ports/udp) and FastAPI runs the sync
route handler in a threadpool, so both threads could probe the same
'free' port before either recorded it — handing lport == rport to both
ends. uBridge sets SO_REUSEADDR on UDP NIO sockets, so the double bind
succeeds silently and the kernel delivers everything to the last-bound
socket: one node starves, the other echoes to itself.
Make every TCP/UDP allocate/reserve/release path atomic with an RLock,
and rebuild _link_data in UDPLink._prepare so reset() commits the fresh
port pair instead of re-sending the stale, already-released one.
Regression tests: threaded barrier allocation never returns duplicates
(red on the old code, UDP and TCP); reset() leaves exactly one mirrored
NIO pair per side with lport != rport (red on the old code).
per-def operations are project-level — create, update AND delete all
modify the marker policy on every link — so all three must batch, not
just create.
Extend memory_only to update_marker and stop_marker (merge/delete into
_markers + refresh _link_data, no per-link HTTP), and route
update_marker_definition and delete_marker_definition through the same
two-phase path as create: memory-only per link, then one
PUT /projects/{id}/nios/batch per compute.
Trade-off: a full reapply resets every marker's pcap on the link (the
old precise update_marker/stop_marker preserved sibling pcaps). For
project-level policy changes this is acceptable — real-time insight
matters more than pcap continuity, and batching turns 5000+ round-trips
into one per compute.
When a marker definition is created (or re-applied on data_link_type
change), the fan-out used to call inherit_marker -> update() on every
link, issuing one PUT /nio per link end (5000+ round-trips on a 2500-link
project). On started nodes each round-trip also reconfigured uBridge.
Two-phase fan-out:
- inherit_marker/start_marker gain memory_only: writes the marker into
link._markers and refreshes _link_data without any HTTP/emit/dump.
- _apply_def_to_all_links applies memory-only to every link, then
_batch_update_link_nios groups the updated NIO specs by compute and
sends a single PUT /projects/{id}/nios/batch per compute.
compute: new PUT /projects/{id}/nios/batch endpoint with _get_existing_nio
+ _update_nio_binding dispatch (mirrors create_batch_nios), re-applies
filters+markers to uBridge on started nodes.
Precise per-marker operations (update_marker bpf change, stop_marker on
def delete) are untouched — they deliberately avoid a full reapply to
preserve sibling marker pcaps.
Project open used to create each link by issuing two NIO POSTs from the
controller to the compute — ~5000 HTTP round-trips for a 2500-link
topology, all funnelling through the single shared controller/compute
event loop and capping throughput near 12 links/s.
Replace it with a bulk path:
- UDPLink split into _prepare() (local: ports, peer addrs, link_data)
and _commit_nios() (dispatch). create() = prepare + commit (interactive).
- Link.add_node gains batch=True: attach both nodes without triggering
per-link NIO HTTP.
- compute: new POST /projects/{id}/nios/batch endpoint with a unified
_add_nio_binding dispatch across node types (docker/qemu/iou/vpcs/
builtin differ in signature).
- project.open: prepare all links locally, group NIO entries by compute,
send each compute a single /nios/batch, then finalise (wire node/port
refs, mark created, notify, apply marker defs) in parallel.
Cuts controller->compute HTTP from O(links) to O(computes). Test added
for the batch endpoint.
host_ip resolved socket.gethostbyname on every access with no cache.
get_ip_on_same_subnet touches host_ip 2-4 times per link, so opening a
2500-link project issued thousands of blocking DNS calls inline on the
event loop — freezing all concurrent link coroutines each time. This is
the most likely cause of the 12-link/s throughput (1000x below what
Pool(concurrency=100) should deliver) and the burst+pause pattern.
- compute.host_ip: cache the resolution in _host_ip_cache, invalidate
on host setter change
- UDPLink.create: timing log splitting get_ip / ports / nio so the next
project-open confirms where time actually goes
Previously the two sides' UDP port reservations and NIO tunnel
POSTs were issued sequentially, even though they are independent
once the peer addresses are known — each node talks to its own
compute/uBridge with no shared lock, so the HTTP round-trips can
overlap.
- Port allocation: gather _allocate_port for both computes
- NIO creation: gather both node.post calls together
- Error handling: if either side fails, roll back whichever side
succeeded (previously only node2-fails→cleanup-node1) before
re-raising the first error
Batch loading (open/import project) benefits most because links
are created at high concurrency and the per-link wall-clock
time is dominated by the sum of its two sequential NIO POSTs.
Every per-link marker operation (start_marker / stop_marker / update_marker) called Project.dump() -- a full topology serialization + file write. A definition fan-out over 500 links therefore wrote the whole topology 500+ times (each blocking the event loop), which dominated the observed ~1 minute; the NIO round-trips themselves were negligible.
Add a dump: bool = True parameter to the three per-link operations and inherit_marker (matching the existing dump param on Link.add_node). The bulk paths -- definition create fan-out, definition-update sync and re-fan-out, definition-delete cleanup, pause/resume, new-link inheritance -- pass dump=False and their caller dumps once after. apply_defs_to_new_link suppresses per-def dumps too: link create / project open dump once after, so opening a 500-link project with N definitions no longer does 500xN topology writes.
Markers now work on serial links (Cisco HDLC / PPP / Frame Relay / ATM), not just Ethernet. A marker carries a data_link_type (default DLT_EN10MB); at the uBridge boundary it becomes the 'mark ... linktype <dlt>' keyword so the BPF compiles and the pcap is written with the matching link-layer.
- MarkerCreate / MarkerDefinitionCreate gain data_link_type (default DLT_EN10MB). Per-link it is create-only; definitions are updatable (a change re-fans-out).
- base_node._marker_linktype() normalizes the GNS3 DLT name (strip DLT_, uppercase, None for EN10MB). Single source is SerialPort.data_link_types, so Cisco PPP -> PPP_SERIAL (50), matching the capture path -- no second mapping table.
- _ubridge_add_marker_filter (generic) and the IOU marker loop append 'linktype <dlt>'.
- Definition fan-out branches on link_type in inherit_marker: Ethernet is always EN10MB; a serial link uses the definition's WAN encapsulation, or is SKIPPED when none was chosen (an EN10MB pcap on serial is undecodable). One definition covers a mixed topology.
- MCP marker_definition exposes data_link_type (None = not forwarded).
- No uBridge rebuild on a data_link_type change -- only that one marker's filter is swapped (delete + re-add), mirroring a BPF change; reset_packet_filters preserves sibling mark filters.
Requires the uBridge build with 'mark ... linktype' support.
Deleting a marker while its node was stopped, then starting the node, recreated
an empty pcap. Root cause: delete_marker_capture removed the uBridge filter and
the pcap file but not the marker spec cached on the port NIO (nio.markers) —
the data source _ubridge_apply_markers reads on node start. The stale spec
reinstalled the marker when uBridge came up.
This was a regression from switching stop_marker off update() (which re-sent
the NIO and implicitly refreshed nio.markers) to the fine-grained
node.delete(/markers/{name}) path.
Fix: make the delete port-aware so the compute can locate the NIO. The DELETE
marker route becomes /adapters/{a}/ports/{p}/markers/{name} across all six
node types; the handler resolves the NIO via get_nio and passes it to
delete_marker_capture, which now pops the marker from nio.markers. get_nio
works regardless of uBridge state, so the stopped-node case is covered. The
controller's stop_marker targets the capture side's adapter/port.
A marker definition validated its BPF N times — once per link in the fan-out
(start_marker runs validate_bpf_syntax on every copy), spawning one tcpdump -d
subprocess per link for the same expression. Definitions did not validate BPF
at all; only direction was checked.
Move the single validation point to the definition layer (create/update), and
validate each definition's BPF on project load (dropping any that have gone
invalid, like private markers). The inherited fan-out (start_marker) and def
sync (update_marker) now skip validate_bpf_syntax for inherited copies, since
the BPF comes from an already-validated definition. Private per-link markers
still validate inline as before. uBridge still runs pcap_compile at install, so
an invalid expression can never slip through.
Creating a definition over N links now runs one tcpdump instead of N.
Deleting or updating a marker no longer triggers a full NIO reapply
(reset_packet_filters + re-add), which closed/reopened every sibling
marker's pcap via uBridge. Instead operate on single filters:
- stop_marker: bridge delete_packet_filter + unlink the pcap (works with
the node stopped; filter removal is skipped, the file is still deleted).
- update_marker: bpf/tag/direction → rebuild just that filter (delete + add);
enabled → instant toggle; color/highlight_duration → stored only.
- compute delete_marker_capture / rebuild_marker_filter + per-node routes
(DELETE /markers/{name}, PUT /markers/{name}/rebuild) + MarkerRebuild schema.
IOU overrides _ubridge_delete_marker_filter for iol_bridge; rebuild reuses
the already-overridden add/delete/set, so IOU needs no rebuild override.
The project-wide mute (POST /markers/pause|resume + _markers_paused) paused
every marker with one button. The actual need is per-rule control: pause one
definition and toggle only its inherited global-{name} copies across all links.
- Drop project-level: _markers_paused (init/asdict/load/start_all), the
pause_all/resume_all_markers methods, and the /markers/pause|resume routes.
- Add per-definition: a persisted `paused` flag on each definition;
pause/resume_marker_definition fan out update_marker(enabled) to every
global-{name} copy — instant, via the existing enable_packet_filter toggle
(no NIO rebuild, pcap/emitted preserved). New links inherit a paused
definition already off (inherit_marker passes enabled=not paused).
- start_marker takes an enabled kwarg; update_marker's enabled-only short-circuit
now also covers inherited copies so def pause/resume is instant.
- Routes: POST /marker-definitions/{name}/pause|resume.
- Docs + tests updated.
Wire gns3-server to uBridge's real-time marker controls (contract
../ubridge/doc/gns3server-integration.md §3.2), in three layers:
A. enabled reaches uBridge — _markers_for_node no longer drops disabled
markers controller-side; it carries `enabled` in the NIO spec. apply
installs every marker then issues `enable_packet_filter … off` for the
disabled ones (base_node bridge / iou iol_bridge; old-ubridge errors
downgrade to a warning so a toggle can't break link create).
B. instant per-filter toggle — apply records name→bridge so a new
`_ubridge_set_marker_filter_state` can flip a running filter with
`enable_packet_filter on|off` (iou overrides for iol_bridge + bay/unit).
Each marker-capable node type gains PUT /markers/{name}; update_marker
short-circuits to it when only `enabled` changes (no NIO rebuild, no pcap
flush), falling back to reset+reapply if the route is unavailable.
C. global pause/resume — `_ubridge_marker_pause/resume` send `marker pause`
/ `marker resume` direct to the hypervisor (pause stops signal+pcap,
resume instant, sink retained). Six node-type routes add POST
/markers/pause|resume; project.pause_all/resume_all_markers fan out to
each capture node (deduped, best-effort); REST exposes
POST /projects/{id}/markers/pause|resume.
Toggling enabled and pause/resume are now both instant — only marker create
or a bpf change still go through reset+reapply.
direction was settable but not clearable: once a marker or project-level
definition had direction=tx/rx, no update path could return it to "both
directions", and the definition-sync fan-out silently kept stale values on
every inherited copy.
Introduce a _UNSET sentinel (link.py) distinct from None so updaters can
tell "caller omitted direction" (preserve) from "caller passed None"
(clear). Thread it through UDPLink.update_marker and
Project.update_marker_definition; the two REST routes use Pydantic v2
model_fields_set to translate an explicit JSON null into the sentinel.
MCP follows with a "both" token: link_marker / marker_definition handlers
map direction="both" to a null in the REST body (tri-state: omit=preserve,
tx/rx=set, both=clear), and the tool descriptions/docstrings document it.
Backward compatible: omitting direction or passing tx/rx behaves exactly
as before; only an explicit null / "both" clears.
A marker is single-sided — only the chosen capture node's uBridge installs the
mark filter — and dir=tx|rx is interpreted from that node's perspective. Until
now the observer was always auto-picked (_choose_marker_side), so dir=tx meant
"the auto-chosen endpoint is sending", which is unpredictable and makes the
direction filter hard to render meaningfully in the Web UI.
Add an optional create-only capture_node_id to MarkerCreate: when set, the
marker is pinned to that endpoint's uBridge (validated as a link endpoint and a
marker-capable type); when omitted, behavior is unchanged (auto-pick). The
chosen id is already echoed back as capture_node_id and in MARK signals, so the
UI can always render the observer regardless of who picked it.
capture_node_id is create-only (changing it would silently flip the meaning of
stored direction; recreate instead) and is not accepted on project-level
definitions — they are link-agnostic and have no endpoints, so inherited
markers keep auto-picking per link.
Plumbed through REST create_marker, the MCP link_marker tool, and base
Link.start_marker. update_marker does not forward it.
Direction field in marker.match events
=======================================
Read ubridge dir=<tx|rx> from MARK signal datagrams and forward it
as a "dir" key in the marker.match notification event. The field is
additive -- older ubridge builds omit it and the parser leaves it null,
so consumers fall back to undirected rendering with no version coupling.
Semantics are relative to the capture node (the signal's node=<id>):
tx = capture node is sending (ingressed device-side NIO), rx = it is
receiving (ingressed link-side NIO).
Per-marker direction filter (opt-in, server-side pipeline)
==========================================================
Add a direction field to MarkerCreate and MarkerDefinitionCreate
schemas ("tx" | "rx" | null). Plumb it through the full pipeline:
Schema -> controller (start_marker/update_marker, marker_entry,
_markers_for_node, update_marker_definition sync)
-> REST/MCP handlers -> compute _ubridge_add_marker_filter +
IOU _ubridge_apply_markers -> bridge add_packet_filter dir <tx|rx>
When set, ubridge only fires the mark handler (signal + pcap) for
packets matching the chosen direction. null (default/legacy) = both
directions -- zero behavioural change for existing markers.
Docs and tests
==============
- docs/features/marker-traffic-insight.md: signal format updated,
new Direction section with NIO mapping, arrow mapping, and additive
compatibility note.
- tests/compute/marker/test_marker_manager.py: 3 new parser tests
(dir tx/rx/absent) plus existing test extended to assert dir=None.
13 files, +123/-22, 72 tests pass (zero breakage)
delete_marker_definition removed the def but left the inherited global-*
copies on every link: it called stop_marker(), which rejects inherited
markers (409), and the ControllerError was swallowed as a warning. The
orphaned copies were in-memory only (_persist_markers filters inherited
markers) so a restart hid the symptom, but during a running session they
were undeletable via either the per-link or project API.
Add the same inherited=True bypass that update_marker already has, and
pass it from the def-delete fan-out so the copies are removed for real.
Per-marker UI hint (milliseconds, ge=1) for how long the Web UI keeps a
marker highlighted after a match. Mirrors color: stored on the link,
persisted in the topology, inherited via project-level definitions, and
never sent to uBridge. Omitted = null = frontend uses its own default.
The default is intentionally NOT set in the schema: MarkerCreate also
backs PUT updates, so a schema default would make every partial update
silently reset the value. None means "not provided" (keep existing on
update / frontend decides on create).
Bugs found via end-to-end testing of project-level marker definitions:
1. New links didn't inherit — apply_defs_to_new_link is async but was
called without await in UDPLink.create().
2. Project load crashed — load_project passed marker_definitions to
Project.__init__. Now popped in load_project and restored separately
in Project.open() (it backs a read-only property).
3. PUT on a definition didn't sync to links — update_marker's guard
rejected even the project-layer sync call. Added an `inherited`
bypass flag used by update_marker_definition.
4. _ubridge_add_marker_filter raised re.PatternError — the name regex
used (?i)(?!global) look-around, invalid in Python's re module.
Dropped the prefix check there: "global-*" names are legitimate at
the uBridge boundary (inherited definitions); forbidden only at the
user-facing schema.
5. GET /links hid inherited markers — asdict()'s runtime branch used
_persist_markers() (which filters inherited markers). Restored
self._markers for the runtime branch; only the topology_dump branch
filters (inherited markers are rebuilt from definitions on load).
6. Duplicate "already exists" warnings on project open — open() fanned
out definitions to all links, but UDPLink.create() had already done
so via its inheritance hook. Removed the redundant fan-out in open().
Project-level marker definitions fan out to every link (existing and new).
A definition is stored once on Project._marker_definitions; when applied to
a link the marker is named "global-{def_name}" — the "global" prefix was
pre-reserved in the schema, so inherited and per-link markers can never
collide, nor will their registry keys.
Key behavior:
- POST /projects/{pid}/marker-definitions → fan out to all existing links
- PUT /projects/{pid}/marker-definitions/{name} → sync all inherited copies
- DELETE → remove every inherited copy from every link
- New links auto-inherit all active defs (hook in UDPLink.create)
- Per-link DELETE/PUT of a "global-*" marker is rejected (409)
- Inherited markers are NOT persisted in the topology; they are re-created
from _marker_definitions on project load
- Compute side is untouched — the marker reaches uBridge via the existing
start_marker→update→NIO→ubridge pipeline
Files:
- controller/project.py — _marker_definitions + CRUD + fanout + topology load
- controller/link.py — Link.inherit_marker() + asdict() filter
- controller/udp_link.py — guards on stop/update + create() inheritance hook
- controller/topology.py — persist marker_definitions in project topology
- schemas/controller/links.py — MarkerDefinitionCreate schema
- api/routes/controller/projects.py — REST endpoints (marker-definitions)
Dynamips needed its own NIO base class extended with a `_markers`
attribute and `markers` property (mirroring the existing `filters`
pattern), plus marker propagation from the controller-supplied NIO
settings through NIOUDP.create/update to the common destination
NIO that base_node._ubridge_apply_markers reads.
Cloud simply needed the missing _ubridge_apply_markers calls after
_ubridge_apply_filters in its two ubridge connection methods, same
pattern as the earlier docker fix.
Both types are now added back to _MARKER_CAPABLE_TYPES.
Docker's _connect_nio and adapter_update_nio_binding applied packet
filters but never called _ubridge_apply_markers, so markers silently
did nothing on Docker links despite docker being in the allowlist.
Add the missing calls (same pattern as the IOU fix).
Also narrow _MARKER_CAPABLE_TYPES to the four types that actually
implement marker support — vpcs, qemu, docker, iou — removing
dynamips, virtualbox, vmware, and cloud which have no marker pathway
and would silently fail when selected as the capture side.
Markers now follow exactly the same apply pattern as packet filters:
state lives in Link._markers, application goes through NIO
(update() -> PUT /nio -> _ubridge_apply_markers). The former
immediate-apply REST endpoints (/markers/start, /markers/stop on
the compute side) and the per-node start_marker/stop_marker methods
are removed — they were a legacy of the original capture-inspired
design and have been superseded by the NIO flow.
Changes:
- controller/udp_link: start_marker/stop_marker/update_marker now
set _markers state + call self.update() (mirrors update_filters).
Removed _marker_capture_nodes runtime dict and its helpers.
- controller/project: _create_link_from_topology_data restores
_markers directly from persisted data (with BPF validation,
like filter reload). No long calls start_marker during load.
- compute: _ubridge_apply_markers swallows BPF compile errors
(warn+skip), matching _ubridge_apply_filters behaviour so a
single bad expression cannot break link creation / node restart.
- Removed: /markers/start,stop endpoints (6 handlers across
vpcs/qemu/docker route files), node start_marker/stop_marker
methods (3 VM files), _ubridge_delete_marker_filter,
_marker_capture_nodes, MarkerDelete schema.
Net: ~280 lines of dead code removed; marker and packet filter now
share a single, unified apply path via the NIO.
_create_link_from_topology_data now restores traffic-insight markers
(mirroring how filters are restored via update_filters), so markers —
including their color — survive project close/reopen and server restart.
Guard start_marker's uBridge POST with 'if self._created' (exactly as
update_filters guards its update() call): during project load the link
is not yet created, so only _markers state is recorded and the marker
is applied once via the NIO flow in create()/_ubridge_apply_markers —
no double application.
Add a user-chosen hex color (e.g. '#ff5722') to each marker, stored in
Link._markers alongside bpf/tag/enabled. The color:
- Is serialized with the link (asdict) and persisted in the topology, so
it survives project reload and is consistent across devices.
- Never reaches uBridge (compute ignores it) — it is purely a Web UI
concern.
- Can be updated independently of the BPF (color-only changes skip the
ubridge round-trip).
MarkerCreate schema gains color: Optional[str]; start_marker and
update_marker carry it through; REST create/update endpoints pass it.
The marker.match WebSocket event stays lean (no color) — the Web UI maps
filter -> color from the link state it already holds.
Replace the complex tier-based fallback (which leaked into unsupported
switch types on the final tiers) with a simple allowlist of
_MARKER_CAPABLE_TYPES, mirroring _get_filter_node in link.py.
Also accept stopped nodes — like packet filters, markers are stored on
the NIO and applied when the node starts via _ubridge_apply_markers.
Markers are persistent link-scoped features (like packet filters), not
transient runtime features (like capture). Tearing them down on every
node stop breaks stop/start restore: the NIO-based re-application in
_ubridge_apply_markers works correctly on restart, but node_updated
was prematurely calling stop_marker which erased _markers state and
dumped an empty topology.
Now markers survive node stop transparently — the ubridge filter dies
with the process but is re-added from nio.markers in
add_ubridge_udp_connection when the node starts again, mirroring
exactly how packet filters behave.
Mirror the packet-filter lifecycle: marker specs now live on the NIO
(next to filters), ride in link_data from controller to compute on
every NIO create/update, and are reapplied by _ubridge_apply_markers
in add_ubridge_udp_connection (bridge creation / node restart) and
update_ubridge_udp_connection (NIO update — following the preceding
reset_packet_filters so markers survive filter changes).
Changes:
- NIO / NIOUDP: _markers property + asdict
- schemas/compute/nios.py: UDPNIO.markers field
- base_manager.create_nio: nio.markers from settings
- PUT /nio routes (vpcs/qemu/docker): nio.markers update
- base_node: _ubridge_apply_markers(bridge_name, nio) iterates
nio.markers, computes pcap path, calls _ubridge_add_marker_filter
+ MarkerManager.register; called after _ubridge_apply_filters
- controller udp_link: _get_node_markers + _markers_for_node (route
by capture_node_id); markers in create() and update() link_data
- /markers/start,stop endpoints: mirror spec onto nio.markers so
the marker survives a subsequent node stop/start without a PUT
- tests: add markers field to NIO data expectations
This covers:
- Node restart: NIO persists, add_ubridge_udp_connection re-applies
- Filter update: reset wipes markers, _ubridge_apply_markers re-adds
- Project reload: create() carries markers in link_data → create_nio
- Immediate create: endpoint sets nio.markers immediately
