Regression coverage added with the fixes:
- capture-slot semantics: second-port refusal, failed-start rollback, and
a non-owner port's stop not reaching `capture stop_kernel`;
- QEMU stop/start teardown on a mid-body or pre-launch failure, IOU start
cleanup and filter guards, Dynamips half-attach undo plus retry;
- switch VLAN reconcile skipping a deferred anchor and rolling the
ports_mapping back on failure;
- Docker/QEMU occupied-port refusal, probe ctime invalidation, probe
survival of a failing uBridge start;
- env-marker dialects (mid-line mentions, falsy values) and the agent
merge rejecting entries without a device_name.
Test-body imports of product modules move to the shared header.
IOL runner containers (IOLDockerVM, GNS3_IOL_RUNNER) had no kernel
datapath: their guest leg is the runner's AF_UNIX socket pairs, so every
link rode the uBridge relay. Their Ethernet bays now own persistent TAP
anchors (gx names, the iol_docker key of the shared naming contract —
born at node start like IOU's, before any link, so the switch fast
path's deferred join can never miss them), and the per-port bridge's
topology leg swaps between UDP (relay) and the anchor (kernel link)
through the frozen stop -> delete -> add -> start contract — the unix
binding never re-binds across a swap.
- iol_docker_vm: KernelDatapathMixin hooks, the anchor lifecycle
(_prepare_tap_datapath at _start_ubridge: probe ubridge_bridge_tap +
ubridge_tap, sweep leftover, tap create born down), the kernel attach
(ensure-then-add — tap create is strictly create-only, IFF_TUN_EXCL,
so the ensure checks /sys/class/net first: an unconditional create
answers EBUSY on the device the start loop just made), the release
(port bridge stop + delete_nio_tap before the shared teardown) and the
stop order the fd holders dictate (port bridges first, then tap
delete, then the per-link bridges — all before the hypervisor stops).
- vendor_docker_vm: the port-bridge creation factored into
_ensure_unix_port_bridge (idempotent on the registered name), keeping
the NIOBridge rejection for the generic unix-socket containers.
- docker_vm: the three _kernel_veths lookups go through the
_kernel_host_ifc hook — behavior-identical for DockerVM, and it is
what lets the subclass route without overriding the binding methods.
- kernel_anchor: the iol_docker key (gx) and kernel_anchor_type, the
controller-side mapping for naming an IOL runner's anchor on the
switch fast path (the marker check stays inline: this module sits
below everything and must not import the controller package).
Switch-to-switch links were the last shape left on the relay: the switch
fast path works by absorbing the *peer's* anchor, and a switch has none —
its ports are its own bridge, and a kernel interface belongs to exactly
one bridge, so two bridges cannot be joined by enslaving anything.
Join them with a veth pair instead: one end per bridge, each side
enslaving its own end and applying its own port mode to it — an access
port on one side and a trunk on the other behaves like the real cable it
models. uBridge's docker create_veth/delete_veth (the primitive every
Docker adapter is already born from) creates and destroys the pair; the
names derive from the link id (utils.kernel_anchor.kernel_cascade_names),
so each side computes its own end and the names survive a restart.
The pair belongs to the link, and the two ends are wired by two
independent uBridge processes — the project-open batch drives different
nodes concurrently — so the protocol is order-free: both sides create the
pair when their own end is missing (one create wins; the loser's failure
is re-checked against the kernel and "the pair is there" is not an
error), and either side's teardown destroys it (deleting one veth end
takes the other with it, so both converge on the same gone pair and the
second delif answers ENOENT). A pre-existing end — this switch restarted,
or a leftover from a previous life of the link id — is reused rather than
recreated, because the other end may still be a live port of the peer
switch; a stale tc qdisc is all a reused end can carry.
No new wire format was needed: a cascade is an anchor NIO on both ends,
the absorbed-anchor case plus the peer end's name. That also corrects the
filter routing — the single-sided rule exists because an absorbed anchor
is one interface, while a cascade has one per switch, so impairments go
to both ends like any other kernel link.
Tests: the wire format, the eligibility flip and the both-ends re-push on
node start (controller); the naming contract (utils); the switch-side
lifecycle — create, reuse, lost create race, delete destroys the pair
(compute routes).
Drive-by: the relay filter test's compute mock carried no awaitable
`put`, so tests/controller/test_kernel_datapath_link.py failed when run
alone while an argument-less full run passed; callers now get one.
The Ethernet switch's kernel fast path: instead of a link crossing the
switch's per-port uBridge relay, the switch absorbs the peer's anchor into
its own kernel bridge. The peer's interface (Docker veth host end, QEMU
TAP, IOU port TAP, Dynamips port TAP) becomes the switch port, and frames
cross anchor -> switch bridge -> switch ports entirely in the kernel.
Wire format: the switch end receives nio_anchor ('join this interface to
my own bridge' — the mirror of nio_bridge), the peer end nio_bridge with a
null bridge ('your anchor is bridged elsewhere': carrier, markers and
impairments only — never bridge membership, and never a bridge delete, on
an interface this node does not own). Anchor names come from the shared
naming contract (utils.kernel_anchor), a pure function of node type, id,
adapter and port — the controller computes the peer's anchor while the
node may be stopped, which is what makes links to stopped nodes wireable.
Stopped peers defer the join; after any node of a link starts
(UDPLink.node_started) the controller re-pushes the switch NIO, and the
switch completes — or repairs, when a peer restart replaced its interface
under the same name — the join against the kernel's actual membership
instead of assuming it.
VLAN changes now edit membership in place: only the ports whose mode/VLAN
actually changed are touched, each losing exactly the previous mode's
entries the new mode does not want (the dot1q admits-all entry is one
range operation in both directions), and no port ever leaves the bridge to
change its VLANs. This replaces the old 'reset by re-enslaving' shortcut
(delif+addif on every wired port, added when the ports were the switch's
own TAPs): on absorbed anchors, re-enslaving an interface owned by another
node is where live testing watched links die — anchors occasionally lost
their bridge membership right after a ports_mapping update (later updates
then failed with delif EINVAL), reproducibly with qemu, IOU and Dynamips
peers and never with Docker's veths.
Eligibility: the switch side needs no capability of its own (its bridge
exists by construction); the peer must be able to anchor. Switch-to-switch
cascades stay on the relay — a kernel interface belongs to one bridge, and
joining two bridges needs a veth pair, deferred.
Verified live on a real server: qemu peers through four VLAN cycles with
both anchors enslaved throughout; a mixed switch (2 IOU + 2 Dynamips, four
kernel ports); Docker peers with real ICMP isolating and recovering (with
both anchors still enslaved); VPCS relay ports likewise in place.
Three fixes from live-calibrating the kernel impairment filters on an
isolated instance (Docker-to-Docker veth link, per-sequence evidence):
- update_filters skipped an unchanged filters dict as a no-op, so a
one-shot window_drop outage that had already passed could never be
re-armed: re-applying the same values produced no datapath command at
all. A PUT still carrying window_drop now reconciles, restarting the
schedule from the apply as documented. Measured: identical re-apply
moves the outage window from seq 50-70 to 70-91 (arm at +8s of the
new apply).
- gemodel parameter semantics were documented backwards in the
validation rules, available_filters, the MCP/copilot descriptions and
the feature docs. Measured against the kernel (per-direction counters,
offered = sent + dropped): p is the good-to-bad transition chance, r
the bad-to-good transition, 1-h the loss chance in the bad state (the
good state loses nothing) - mean one-way loss p/(p+r)*(1-h), e.g.
[10,30,100] -> 24.8%/25.7% measured vs 25% predicted, [5,45,100] ->
10.8% both ways vs 10%.
- kernel_datapath was silently stripped from every REST/MCP link payload
by the response model (LinkBase carried no such field), so clients
could not tell a kernel link from a relay one.
Also verified on the same setup: the 3-argument window_drop form works
exactly as specified ([3000,5000,100] -> a single seq 31-55 outage,
[20000,6000,100] -> seq 115-144), and both generations of the command
builder emit the parameters unpadded.
Adds an API test for the field and controller tests for the no-op skip
and the window_drop re-arm exemption.
Consume uBridge's ebpf_modes contract (feature/tc-window 40c36ae): a
stateful mode is usable iff ebpf=1 and its token is listed; builds
predating the field keep their shipped modes (nth/quota/flow), so the
pre-correction window semantics can never be installed silently.
- gns3server/utils/tc_capabilities.py: shared reply parser, mode
policy and filter-type mappings (compute and controller gate on the
same rules)
- gns3server/compute/ubridge/tc_probe.py: standalone probe spawning a
throwaway uBridge as the server's own user, cached by binary
identity (path/mtime/size - the version string cannot distinguish
rebuilds); failures cache as unknown
- GET /v3/compute/capabilities gains ubridge_tc {netem, ebpf,
ebpf_modes, cbpf}; the controller forwards it (Compute.capabilities)
and available_filters hides kernel-datapath filter types an endpoint
compute cannot run (both ends apply their own filters, so one
incapable compute hides the type); computes without a report keep
the full list and apply-time 409 stays the guard
- docker: _ubridge_apply_ebpf_drops gates per mode token; the cleanup
path only sends "off" for modes the build declares
- split DockerVM's kernel-datapath machinery (veth lifecycle, carrier,
marker primitives, tc filter translation, kernel NIO teardown) into
compute/docker/docker_kernel_datapath.py (DockerKernelDatapathMixin)
uBridge's window mode gained the semantics the B.2 spec asked for —
`tc window_drop <if> <start> <outage> <pct> [<period> [<jitter>]]`: a
single outage that traffic passes before *and* after, recurring flaps
with period >= outage, and a per-cycle jitter — so the kernel-only
`window_drop` filter type is now exposed:
- validation mirrors the tc grammar (3-5 params, ranges, period >=
outage, jitter cap) and adds window_drop to KERNEL_ONLY_FILTERS;
filter_inactive_filters keeps start=0 — unlike every other filter a
zero first parameter means "the outage starts now", not "disabled".
- `_ubridge_apply_ebpf_drops` sends `tc window_drop` (positional
pass-through) and turns the mode off explicitly when absent, under the
same ebpf=1 gate.
- FILTERS entry (Start/Outage/Chance/Period/Jitter) + available_filters.
The schedule is relative to the apply: reconcile re-sets the classifier
on every apply, so any filter change, node restart or link reset restarts
the window.
Verified on a two-container kernel link, 15/15: a single outage
[1000, 1500, 100] arrives per-seq as [0..4, 13..19] (passing before the
window, dropped inside, passing after); recurring 1000/2000 ms = 52 %
round trip (duty cycle); period == outage = permanent outage; period <
outage = 409 before uBridge; jittered flap keeps the ~50 % duty cycle;
30 % chance on a permanent window = 53 % round trip against the
per-direction theory 1-0.7^2 = 51 %; coexist with netem; node restart
restores; available_filters kernel vs relay (409 + hidden); teardown
clean. P6b re-run as a regression on the same uBridge build: 13/13.
Worth recording for the next rebuild: the first delivered window program
loaded as root but reported ebpf=0 through gns3-server — the verifier's
non-root path loses the loop counter's bound, treats the catch-up loop as
unbounded and rejects the *whole* program (E2BIG, 8193 pending states),
which also took frequency_drop and quota down, with ebpf=0 as the only
symptom. Smaller trip caps did not help; the loop had to go (uBridge
572ca0c), on top of a Makefile header-dependency fix (53e34bb) that had
been shipping a stale program object.
Consume the eBPF impairment classifier delivered in uBridge
feature/tc-precision: frequency_drop becomes the exact every-Nth mode
(tc nth_drop; -1 = drop everything maps to every 1st) and the new
kernel-only quota type is the byte-cap mode (tc quota_drop, 100 = hard
cutoff). KERNEL_UNSUPPORTED_FILTERS is gone entirely — kernel/relay
eligibility is now purely topological. Both modes gate on ebpf=1 from
the per-process tc capabilities probe (error names the setcap fix); the
cleanup path (modes off) only talks to the eBPF commands when the probe
reported them. Reconcile is a plain re-set: the netem reset-before-set
already detaches the program and clears uBridge's per-interface
registry, so a mode set after it is a fresh load+attach.
Documented semantic difference: the relay counts both directions
through one bridge counter, the kernel counts per direction — N=3
round-trip loss is 1-(2/3)^2 = 55.6% on kernel vs ~1/N on relay
(measured 57%). window_drop is NOT exposed: the delivered
implementation advances the window with no gap, making 'outside the
window packets pass' unreachable (needs outage/period fields —
recorded in the spec); flow_drop awaits a parameter-shape decision.
Also fixes a pre-existing restart bug the e2e exposed: _bridges (relay
bridge names) was never cleared on stop, so restarting a node with a
relay link skipped 'bridge create' and failed on add_nio_udp.
e2e (isolated instance): 13/13 — pref-1 tc_impair classifier on both
veths, 57% round-trip for N=3, -1 = exact 100%, quota hard cutoff,
delay coexistence, clean teardown, suspend/resume, node restart,
relay-leg frequency_drop regression through an Ethernet switch. Full
suite: 1925 passed.
Surface the netem extensions delivered in uBridge feature/tc-netem-ext
as GNS3 filter types: rate (tc-style bandwidth string), reorder (needs
delay), gemodel (Gilbert-Elliot bursty loss, exclusive with
packet_loss), duplicate, seed and limit, plus the delay jitter
distribution and the loss/dup correlation parameters. All are
kernel-datapath only — the relay's packet-filter registry has no
equivalent, so available_filters hides them on relay links, setting one
on a created relay link returns 409, and the project-load path drops
them with a warning (update_filters runs before the datapath is
decided, so it cannot reject there). Validation mirrors the tc grammar:
reorder requires delay, gemodel and packet_loss are mutually exclusive,
a distribution requires jitter > 0, rate is integer+unit capped at
100gbit.
The translation in DockerVM._netem_command_parts emits the frozen
grammar order and gates extension keywords on the 'tc capabilities'
netem token list (probed once per uBridge process; a plain
delay/loss/corrupt/dup filter never probes, keeping old uBridge builds
untouched; the correl suffix has no token and gates on 'rate' as the
extension-build marker).
Reconcile became reset-before-set: the kernel's netem change merges
optional attributes (rate, correlation, reorder, corrupt, gemodel,
distribution, seed keep their previous value when the new message omits
them), so removing a parameter needs an explicit 'tc reset' — the
bpf_drop filters are re-added right after in the same flow, so the
clsact churn is invisible. The capability check runs before the reset
so a failed update leaves the previous qdisc in place.
e2e (isolated instance, kernel link): 24/24 — rate 512kbit adds exactly
2x the per-packet serialization to the RTT (45.5 ms for 1400-byte
pings), paretonormal jitter + rate-cleared leak check, reorder gap 5,
gemodel 50% bursty loss, duplicate, loss 30% correl 50%, seed 42 +
limit 5000 visible in the qdisc dump, suspend/resume and node restart
keep the filters, relay link rejects rate with 409, teardown leaves no
netem. Full unit suite: 1916 passed.
Config.update_config() applies submitted options to the main
configuration file via configparser read-modify-write: unknown
options are preserved, null removes an option, the merged view
of all files is validated as ServerConfig before anything is
written (a bad file would kill the FileWatcher polling loop),
and the write is atomic (.tmp + os.replace, mode 0600). Options
whose effective value is owned by a later configuration file
raise ConfigConflictError instead of writing a no-op. The
reload logic is factored into reload_and_notify() and the file
watcher callback is exception-guarded so polling never dies.
NotificationQueue.get only generated a synthetic ping when the queue
was idle for the full timeout. Under sustained event load (e.g. a
project with markers matching at 15-260 events/s) the queue never
idled, so compute notification streams never carried a ping and the
controller stopped emitting compute.updated: clients lost compute
statistics until the event flow paused or the server restarted.
A ping is now guaranteed at least every timeout seconds regardless of
event flow: when the ping deadline is reached the next get() returns a
ping ahead of queued events (pings only carry statistics, so skipping
ahead of real events is harmless). Both the compute stream (compute
CPU/memory/disk stats -> compute.updated) and the controller stream
(idle keepalive) benefit.
Each host interface surfaced by the cloud node now reports:
- ip_addresses: every IPv4 and IPv6 address (previously only a single
IPv4 was collected internally and then dropped before the response)
- status / speed / mtu / flags: operational state and link attributes
sourced from psutil.net_if_stats(), with flags normalized to a list
The legacy ip_address / netmask / mac_address fields are preserved so
existing callers (compute link detection, GNS3 VM, VMware, has_netmask)
keep working. The new fields travel through the existing interfaces
payload that the controller forwards verbatim, so no controller-side
change is required and the PUT / ports_mapping flow is unaffected.
This fix addresses the issue where delay: [0, X] configurations were being
silently dropped instead of returning validation errors.
Changes:
- Created new utility function filter_inactive_filters() in packet_filter_validation.py
- Implemented smart filtering logic for delay filter that checks both latency and jitter:
* delay: [0, 0] → User wants to disable delay, filter out silently
* delay: [0, X] where X > 0 → Invalid config, keep for validation error
* delay: [X, X] where X > 0 → Normal configuration, validate normally
- Simplified link.py update_filters() method to use the new utility function
- Added comprehensive tests for the new filtering logic
Before this fix:
- delay: [0, 100] would be silently dropped with no error message
- Users wouldn't know their configuration was invalid
After this fix:
- delay: [0, 100] returns proper error: "delay parameter Latency must be between 1 and 32767 ms, got: 0"
- delay: [0, 0] is correctly handled as intentional disable
- Normal delay configurations continue to work as expected
Align validation rules with ubridge source: delay latency must be > 0
(packet_filter.c delay_setup line 182). Update FILTERS definition in
link.py and test cases accordingly.
Changes:
- Replace tshark BPF validation with tcpdump -d (calls pcap_compile
internally like ubridge, returns instantly without waiting for traffic)
- Support multi-line BPF expressions: split on newlines and validate
each line individually
- Always validate, never save invalid filters on error
- Drop invalid filters during project load with warning (prevents
old topologies with bad filters from failing to open)
- Simplify test cases (no longer depend on tshark availability)
