The live suite asserted the datapath over IPv4 only. Give every guest a
ULA IPv6 address next to its v4 one and twin every v4 assertion point,
sharing the impairment window (one filter application, both families
asserted) so the runtime grows by minutes, not double.
- harness: configure_ios(eth_ipv6=), ping() dispatches to `ping ipv6`,
docker_bring_up_ipv6 (the Docker daemon leaves eth0 with
disable_ipv6=1 on v6-less bridge networks, so the interfaces-file
route cannot work), family-aware docker_wait_address, pcap_records
scans every record (a leading DAD/MLD frame must not mask the
sampled ethertype), and pcap_icmp_rtt measures the wire round trip
from capture timestamps;
- scenarios: connectivity/delay/suspend/isolation/churn checks in both
families, captures assert 86dd next to 0800, the libpcap bpf drops
are asserted family-selective (icmp vs icmp6, each killing its own
family and leaving the other alive), the eBPF classifier modes on
both families, marker v6 twins with an echo-specific expression
(bare icmp6 also matches DAD/MLD and would break the exact counts),
the v6 sharkd replay, and the link-local matrix gains the 86dd
all-nodes row; serial (HDLC) legs stay v4-only;
- the dynamips-emulated classic IOS under-reports its `ping ipv6` RTT
by ~40 % of a delayed round trip (pcap-verified: the wire carries the
v6 echo exchange in the same ~230 ms as v4 while the console reports
~130 ms), so the dynamips-family scenarios assert the v6 delay from
the capture's own timestamps — recorded in tests/e2e/README.md.
Full suite: 17 passed in 58m50s.
The 'impairment cleared' asserts used avg < 50 on three-packet IOS pings;
the unimpaired baseline is 20-45 ms with 4 ms quantization, and host
scheduling jitter pushed averages to 54/56 in two full-suite runs
(esw, dynamips) — a margin, not an impairment (the qdisc asserts pass,
success is 100 %). The contrasted delayed regime is >= 150 ms, so a
100 ms bound still separates the two cleanly. Docker's sub-millisecond
veth asserts keep their 50 ms bound (three orders of magnitude of
headroom, never observed to flake).
The exec stream can truncate after busybox's 4294967 ms RTT timer-wrap
artifact, dropping the summary line — docker_ping then reported a
phantom 100 % loss and made the corrupt-filter scenario fail spuriously.
Factor the parsing into _busybox_ping_parse: the summary stays
authoritative when present; otherwise the loss is derived from the
seq= replies actually seen (no replies at all still reports 100 %).
The esw kernel fast-path scenario deletes and re-creates a link, which
re-enslaves the peer anchor — and on NetworkManager hosts NM's netdev
state machine can release an NM-managed TAP from its bridge some 60-70 ms
after enslavement, ~50% of re-enslavements, while the tap is in NM's
not-yet-assumed state. The freed port stays administratively up and no
GNS3 process logs anything: the membership assert sees the join, then the
ping measures a silent blackhole (this was the line-277 e2e failure).
Root-caused with a server-free harness (one uBridge, one fresh tap, one
bridge, an addif/delif loop): the same tap frees 20/40 while NM-managed,
0/40 after `nmcli device set <tap> managed no`, 20/40 again when
re-managed. Docker's veth host ends are not NM-managed, which is why only
TAP anchors ever showed it — including the "anchor lost its bridge
membership after a ports_mapping update (TAP peers only)" incident from
the switch's development, whose conclusion is corrected in the docs.
Unmanage the anchors once the routers have booted, before the scenario
re-enslaves them; document the failure signature and the host-side remedy
(per-device nmcli call, or NM unmanaged-devices configuration) in the
L2-anchor spec and the switch feature doc.
The rebase onto upstream 3.1 (mypy + ruff I001/F401 waves) left the tree
short of the newly enforced baselines in three ways:
* contextlib / UbridgeError imports dropped as false duplicates during
conflict resolution left NameErrors in iou_vm and qemu_vm (F821);
the Dynamips router grabbed the compute-level NIOTAP instead of the
hypervisor one, breaking TAP anchoring (test failure).
* unused imports (KERNEL_UNSUPPORTED_FILTERS, UbridgeError, List) and
unsorted blocks left behind where the stack's later commits moved the
last usage elsewhere — ruff I001/F401/RUF022 now enforce these.
* mypy, now baseline-free: cache dict annotations in tc_probe, the
locators candidate list, the resolved-params None branch in the MCP
runner, and validate_bpf_syntax keeps the upstream Dict[str, Any]
return (the strict Optional[str] was wrong for the "valid" bool entry).
The e2e raw-frame probe resolves its compiler via shutil.which (S607)
and zips with strict= (B905); the remaining subprocess call is S603 by
nature and gets a per-file ignore like other test files.
ruff check/format clean, mypy at upstream parity (no new errors),
full unit suite 2167 passed.
Two Alpine containers on a kernel link, a static raw-Ethernet probe built
at run time and docker-cp'd in (the image has no raw-frame tool; DockerDaemon
gains the cp), injecting per-destination-MAC frames from guest 1 while
guest 2 counts arrivals: ordinary multicast, STP, LACP, 802.1X and LLDP must
cross, PAUSE must not (the documented kernel limit). Asserts the anchors'
sysfs brport group_fwd_mask reads 0xfffd through the server's own wiring
(int-parsed: newer kernels print it hex), that the bridge level stays 0, and
that a container stop/start replays the mask with LACP surviving. Negative
control is the uBridge suite's setportgroupfwd 0 case.
Two Alpine containers on a kernel-only link with traffic-insight markers
attached to a veth anchor (marker add_kernel), observed on the dedicated
marker WebSocket via a new harness piece (WebSocketCollector, a sync
facade over websockets' sync client). Real ICMP proves: the marker.match
identity is complete (filter/tag/link_id/capture node/ts/len and the
tx/rx direction set of a ping), the main project channel carries no
matches, the BPF and direction are enforced (a non-matching sibling and
a tx-only marker each see exactly their traffic), the host pcap holds
exactly the matched frames, pausing silences one marker while a sibling
keeps counting (silence is never a dead channel), tag replay serves the
paused tag's timeline with display-filter/link/window narrowing and
refuses with 409 while capturing, and deleting a marker takes its pcap
and signals with it.
Two switches cascaded through the link-owned veth pair, two c7200
routers per switch, real ICMP crossing both bridges: the pair is born
UP with exactly one end per switch bridge, a dot1q trunk cascade
carries two VLANs at once, the symmetric access/trunk mode flip on the
cascade ends isolates and revives a VLAN in place (asymmetric flips are
semantically broken — the access side egresses untagged into the far
trunk's native PVID), a delay filter lands as netem on both ends (one
per direction, RTT ~= 2x delay), suspend admin-downs both ends, and a
link delete/re-create retires the whole pair and mints a new one under
the new link id — zero host residue after the project goes.
Two real IOU routers on a real L3 image (the image's own IOL fabric, the
real IOS CLI on the server's telnet console), driving the anchor TAPs the
kernel datapath builds on: anchors born with the node (one per Ethernet
bay/unit), real ICMP through the [fabric <-> TAP] port bridge, netem
delay, suspend, capture, link delete/re-create, node stop/start rewiring,
the serial link staying on the relay (serial bays never anchor) and a
relay negative control on an isolated instance. Every scenario ends with
no anchors, bridges or tmpdirs left behind.
harness: create_iou_node/iou_images/iou_anchor_name, and the IOS console
helper is now ios_console (dynamips, IOU and the IOL containers share it).
Two real Alpine containers on an isolated instance, addressed the way a
GNS3 user does it (the node's persistent /etc/network/interfaces, applied
by init.sh) and driven through the Docker daemon socket the server itself
uses. Four scenarios:
* fast path: the per-link kernel bridge enslaves exactly the two veth host
ends, real ICMP crosses, the L2-anchor spec E.1/E.2 assertions hold
(idle silence measured with the guests' links down -- a container's own
boot-time IGMP report re-fires ~10 s later, so silencing is not just
IPv6), netem delay, suspend/resume, capture, link delete/re-create,
container stop/start (the veth host ends die with the adapter and are
rebuilt from the NIO), zero residue on project delete;
* filter matrix, one type at a time, each asserted through real traffic
and its tc state: netem core (delay, jitter+distribution, loss, corrupt,
duplicate), the extensions (reorder+correl+gap, seed+limit, gemodel,
rate), cls_bpf match-drop including a byte-size discriminator, the eBPF
classifier (frequency_drop every-Nth and -1, quota byte-cap cutoff,
window_drop single outage with per-seq pass/drop/pass evidence), plus
available_filters offering the kernel-only types;
* relay control: no per-link bridge, the kernel-only types hidden;
* relay->kernel reopen upgrade: the datapath choice is flipped across a
server restart and the project reopened -- the link is rebuilt on the
kernel datapath and forwards.
The harness gains a Docker guest driver (a stdlib Engine API client over
the daemon socket: exec with stream demux, container id by deterministic
name -- the controller's node payload does not carry it, image inspect),
a digest-pinned image helper (cache hit = no network, miss = pull through
the server's own route, neither = skip) and Server.restart() for the
reopen scenario (which also needs the local compute's capabilities to be
up before a project open is accepted).
Live: 4/4 scenarios, 142 s for the file; full e2e suite 10/10 (1503 s);
unit suite 2149 passed; ruff clean. The matrix measures loss with 20
packets at 0.2 s spacing (143 s -> 58 s) and the jitter RTT assertion
keeps statistical margin (per-direction N(50,20); 10 packets, >= 65 ms).
Without one every c7200 busy-waits a full CPU core for the whole scenario
-- the tests create raw nodes and no template supplies a value (measured:
constant ~100 % per router, up to ~6 minutes per dynamips scenario).
harness.dynamips_idlepc() detects a value through the controller's
auto_idlepc endpoint (the GUI idle-PC finder path: a throwaway project
boots the image, candidate values are validated by measuring actual CPU
usage) and caches it in ~/.cache/gns3-e2e/idlepc.json keyed by the image
checksum, so the cost (~100 s here) is paid once per image, not per run.
GNS3_E2E_IDLEPC overrides detection; any failure returns None and the
scenarios run without one rather than fail on a CPU optimisation.
All four scenarios that boot c7200 routers (dynamips kernel/relay,
ethernet switch kernel/relay) pass the value at creation. Live: detected
0x60189234 for the M11 image (238 s first run including detection); the
routers drop from ~100 % during boot to 3-8 % steady (sampled per
process), and a re-run reports the cached value with no detection.
The spec's gns3-server alignment section promised three assertions the
suites never carried; they land in harness.py and run in every kernel
scenario (plus the relay controls' anchor checks):
- E.1 assert_pure_l2: anchors and bridges carry no L3 identity (addrgenmode
none, no address in /proc/net/if_inet6, no ip -o -4 addr) -- the kernel
scenarios' anchors and per-link/switch bridges, relay-mode anchors and
the switch's relay TAPs alike (hardening is a creation-time property,
not a datapath one);
- E.2 assert_idle_silence (Dynamips suite): guest interfaces shut to
silence IOS's own CDP/keepalives, settle 2.5 s, then a 5 s window of
zero traffic over both anchors and the per-link bridge;
- assert_forwarding: every attached port reports brport/state == 3 -- a
bridge left DOWN keeps its ports disabled and forwards nothing, with no
error anywhere; the assertion is the only signal.
harness.l2only_supported() probes a throwaway uBridge once (link l2only
against a nonexistent device: 208 = command present, 202-Unknown command
= old build); without the command the assertions skip -- the spec's
degradation contract, mirroring the tc-capability skip.
The first live run caught a fifth-creator gap: the switch's relay port
TAPs (minted implicitly by bridge add_nio_tap) carried a live fe80::.
Delivered uBridge-side per the spec (fb72758: the transient TAP is
hardened at creation, attaches untouched); the server needed no change,
and the relay-control E.1 check is that creator's end-to-end coverage.
Two real iol-xe containers (Cisco CML iol-runner images, real IOS-XE on
the telnet console) against an isolated instance, the real uBridge with
the swappable bridge TAP leg:
* kernel link reports kernel_datapath; anchors exist from node start
and the per-link bridge enslaves exactly the two of them;
* real ICMP crosses the [unix <-> tap] port-bridge relay; delay 100
lands as netem on both anchors with the measured RTT growing >= 150 ms
and returning < 50 ms when cleared;
* suspend admin-downs the anchor and kills the traffic; resume
restores; AF_PACKET capture writes a real pcap;
* link delete/re-create swaps the port leg out and back in with the
ping returning; node stop leaves no gx devices and a start recreates
the whole wiring from the NIO; project delete cleans everything;
* the relay control on a relay-configured instance keeps the link off
the kernel and still pings (anchors may exist idle — the same
semantics as QEMU's and Dynamips' anchors: the server flag decides
attachment, not anchor birth).
Harness: create_iol_router (the appliance's template properties),
docker_images, iol_anchor_name, stage_iol_base_config (the shipped
iol-xe-base.txt copied into the isolated instance's configs dir — the
GNS3_IOL_STARTUP_CONFIG lookup path), boot_wait(straight_prompt=True)
for images that boot from NVRAM straight into the CLI with no 'Press
RETURN' sentinel, and GNS3_E2E_DEBUG=1 to run the isolated instance
with debug logging (the uBridge command stream lands in server.log —
how the ensure-anchor EBUSY was pinned down).
The Ethernet switch's kernel fast path: instead of a link crossing the
switch's per-port uBridge relay, the switch absorbs the peer's anchor into
its own kernel bridge. The peer's interface (Docker veth host end, QEMU
TAP, IOU port TAP, Dynamips port TAP) becomes the switch port, and frames
cross anchor -> switch bridge -> switch ports entirely in the kernel.
Wire format: the switch end receives nio_anchor ('join this interface to
my own bridge' — the mirror of nio_bridge), the peer end nio_bridge with a
null bridge ('your anchor is bridged elsewhere': carrier, markers and
impairments only — never bridge membership, and never a bridge delete, on
an interface this node does not own). Anchor names come from the shared
naming contract (utils.kernel_anchor), a pure function of node type, id,
adapter and port — the controller computes the peer's anchor while the
node may be stopped, which is what makes links to stopped nodes wireable.
Stopped peers defer the join; after any node of a link starts
(UDPLink.node_started) the controller re-pushes the switch NIO, and the
switch completes — or repairs, when a peer restart replaced its interface
under the same name — the join against the kernel's actual membership
instead of assuming it.
VLAN changes now edit membership in place: only the ports whose mode/VLAN
actually changed are touched, each losing exactly the previous mode's
entries the new mode does not want (the dot1q admits-all entry is one
range operation in both directions), and no port ever leaves the bridge to
change its VLANs. This replaces the old 'reset by re-enslaving' shortcut
(delif+addif on every wired port, added when the ports were the switch's
own TAPs): on absorbed anchors, re-enslaving an interface owned by another
node is where live testing watched links die — anchors occasionally lost
their bridge membership right after a ports_mapping update (later updates
then failed with delif EINVAL), reproducibly with qemu, IOU and Dynamips
peers and never with Docker's veths.
Eligibility: the switch side needs no capability of its own (its bridge
exists by construction); the peer must be able to anchor. Switch-to-switch
cascades stay on the relay — a kernel interface belongs to one bridge, and
joining two bridges needs a veth pair, deferred.
Verified live on a real server: qemu peers through four VLAN cycles with
both anchors enslaved throughout; a mixed switch (2 IOU + 2 Dynamips, four
kernel ports); Docker peers with real ICMP isolating and recovering (with
both anchors still enslaved); VPCS relay ports likewise in place.
Live e2e verification moves into the repository instead of being rewritten
as a throwaway script every time: tests/e2e/ carries a shared harness (REST
client, isolated-instance lifecycle, a deliberately dumb IOS console
driver, host-side kernel inspection: bridge membership, tap admin state,
tc qdiscs) and each node type adds only its scenario module. The e2e marker
plus 'addopts = -m "not e2e"' keeps these privileged, image-dependent
tests out of normal runs; they are opt-in: pytest -m e2e tests/e2e -s.
The harness targets an existing server via GNS3_E2E_URL or starts an
isolated instance (own config dir — DB and paths follow it — on a free
port) and deletes its project afterwards (GNS3_E2E_KEEP=1 keeps it on
failure).
The first scenario drives two real c7200 routers on a real IOS image
through the REST API and the consoles: the kernel link is created while the
routers are stopped (deferred wiring), the per-link kernel bridge enslaves
exactly the two anchors, real ICMP crosses it, delay 100 shows netem on
both anchors and grows the RTT (~200 ms), suspend kills the traffic on an
admin-down anchor, capture writes a pcap of ICMP records, a serial link
between the same routers stays on the relay and still pings, delete/
re-create and stop/start leave the wiring consistent, and deleting the
project removes every anchor and bridge. A negative control runs the same
topology on an isolated relay-configured instance (no per-link bridge,
nothing enslaved, still pings).
