Adds what running the suite on a real machine actually requires (uBridge
with caps, tc for the filter assertions, a Docker daemon, cc -static for
the raw-frame probe), the full image-selection matrix (preferred
filenames, fallback patterns, per-folder layout, the alpine digest
ladder; missing images skip, never false-fail), the NetworkManager host
hazard and its record, timing/load sensitivity notes, the complete
GNS3_E2E_* environment-variable list — and the link-local scenario,
which the scenario list was missing.
The relay engines' observable paths had no live coverage: every
capture/start call in the suite sat on a kernel anchor (AF_PACKET
capture start_kernel) and markers were only asserted on marker
add_kernel. The two relay engines' own command shapes were unit-tested
only:
* bridge engine - bridge start_capture / bridge add_packet_filter ... mark
* iol_bridge engine - iol_bridge start_capture / iol_bridge add_packet_filter ... mark
Docker relay control (bridge engine, 1 passed 33s): the capture start
REST call writes the ICMP exchange to a pcap off the relay bridge; a
mark filter attached with capture_node_id n1 signals exactly 10
marker.match events over the dedicated marker WS for 5 pings (5
requests + 5 replies, both directions), carrying node_id/link_id/tag and
the busybox frame length, writes exactly those 10 frames to the marker
pcap, and deleting the marker takes the pcap with it.
IOU relay control (iol_bridge engine, 1 passed 173s): the same two
checks on iol_bridge start_capture and the port's mark filter, driven
by the real IOS ping through the fabric socket relay.
Not duplicated in the ESW/VPCS/cloud relay controls: they issue the same
uBridge bridge commands through their own call sites, which stay covered
by unit tests; the IOL container relay goes through the same iol_bridge
engine already exercised here.
The L2-anchor spec's behavioral guarantee — an idle link stays silent for
5 s once the enslavement burst settles — only ran in the Dynamips and
Docker suites. It now also runs in QEMU, IOU, the IOL container, the
switch fast path and the switch cascade, which covers every host-device
creation path:
* tap create (Dynamips/QEMU/IOU anchors), docker create_veth (Docker host
ends; the cascade's race-loser end is hardened after create by the
server), bridge add_nio_tap (the IOL container anchor, the fifth
creator), and the brctl switch bridges, whose multicast snooping is off
precisely so the bridge role can reach silence;
* each scenario silences its guests first (IOS interfaces shut through
the console; Docker guests' links down) and re-verifies the ping after;
* the cascade measures 8 devices at once (four absorbed router anchors,
both cascade ends, both switch bridges).
IOL 158 s, cascade 293 s, ESW fast path 378 s, QEMU 428 s, IOU 308 s —
all green.
The filter matrix (cls_bpf, the eBPF classifier) only ran on Docker veth
host ends; the IOU and IOL-container scenarios now spot-check it on their
TAP anchors and prove a filter survives a node restart:
* bpf "icmp" drops everything with clsact on the anchor, both ends, and
clears back to an empty qdisc;
* frequency_drop 3 (the eBPF every-nth mode) measured 56 % round-trip
loss (1 - (2/3)^2 = 55.6 % expected) in both scenarios;
* the IOU test applies delay 100 before a router stop/start: the fresh
anchor gets netem back (tc level) and the restarted router's pings
measure RTT >= 150 ms until the filter is cleared (a re-attach that
dropped the filter would show ~100 ms).
IOU file 492 s, IOL file 182 s, both green.
Extend the docker and IOU relay negative controls with the filters the
relay itself supports (delay, frequency_drop, bpf), asserted through real
ICMP — the first live coverage of the relay filter path:
* bridge engine (docker): delay 100 grows the RTT by ~200 ms and comes
back on clear; frequency_drop 2 kills the round trips (the filter counts
every packet crossing the bridge, both directions sharing one counter,
so on an alternating ping stream one whole direction dies; verified
against uBridge directly), frequency_drop -1 blackholes, and bpf drops
by libpcap expression ('icmp', then 'greater 150' discriminating by
frame size).
* iol_bridge engine (IOU): the same delay and frequency_drop effects
through the port's iol_bridge filter list, plus the available_filters
spot check.
Docker 33 s, IOU 168 s, both green.
Two Alpine containers on a kernel-only link with traffic-insight markers
attached to a veth anchor (marker add_kernel), observed on the dedicated
marker WebSocket via a new harness piece (WebSocketCollector, a sync
facade over websockets' sync client). Real ICMP proves: the marker.match
identity is complete (filter/tag/link_id/capture node/ts/len and the
tx/rx direction set of a ping), the main project channel carries no
matches, the BPF and direction are enforced (a non-matching sibling and
a tx-only marker each see exactly their traffic), the host pcap holds
exactly the matched frames, pausing silences one marker while a sibling
keeps counting (silence is never a dead channel), tag replay serves the
paused tag's timeline with display-filter/link/window narrowing and
refuses with 409 while capturing, and deleting a marker takes its pcap
and signals with it.
Two switches cascaded through the link-owned veth pair, two c7200
routers per switch, real ICMP crossing both bridges: the pair is born
UP with exactly one end per switch bridge, a dot1q trunk cascade
carries two VLANs at once, the symmetric access/trunk mode flip on the
cascade ends isolates and revives a VLAN in place (asymmetric flips are
semantically broken — the access side egresses untagged into the far
trunk's native PVID), a delay filter lands as netem on both ends (one
per direction, RTT ~= 2x delay), suspend admin-downs both ends, and a
link delete/re-create retires the whole pair and mints a new one under
the new link id — zero host residue after the project goes.
Two real IOU routers on a real L3 image (the image's own IOL fabric, the
real IOS CLI on the server's telnet console), driving the anchor TAPs the
kernel datapath builds on: anchors born with the node (one per Ethernet
bay/unit), real ICMP through the [fabric <-> TAP] port bridge, netem
delay, suspend, capture, link delete/re-create, node stop/start rewiring,
the serial link staying on the relay (serial bays never anchor) and a
relay negative control on an isolated instance. Every scenario ends with
no anchors, bridges or tmpdirs left behind.
harness: create_iou_node/iou_images/iou_anchor_name, and the IOS console
helper is now ios_console (dynamips, IOU and the IOL containers share it).
Two real Alpine containers on an isolated instance, addressed the way a
GNS3 user does it (the node's persistent /etc/network/interfaces, applied
by init.sh) and driven through the Docker daemon socket the server itself
uses. Four scenarios:
* fast path: the per-link kernel bridge enslaves exactly the two veth host
ends, real ICMP crosses, the L2-anchor spec E.1/E.2 assertions hold
(idle silence measured with the guests' links down -- a container's own
boot-time IGMP report re-fires ~10 s later, so silencing is not just
IPv6), netem delay, suspend/resume, capture, link delete/re-create,
container stop/start (the veth host ends die with the adapter and are
rebuilt from the NIO), zero residue on project delete;
* filter matrix, one type at a time, each asserted through real traffic
and its tc state: netem core (delay, jitter+distribution, loss, corrupt,
duplicate), the extensions (reorder+correl+gap, seed+limit, gemodel,
rate), cls_bpf match-drop including a byte-size discriminator, the eBPF
classifier (frequency_drop every-Nth and -1, quota byte-cap cutoff,
window_drop single outage with per-seq pass/drop/pass evidence), plus
available_filters offering the kernel-only types;
* relay control: no per-link bridge, the kernel-only types hidden;
* relay->kernel reopen upgrade: the datapath choice is flipped across a
server restart and the project reopened -- the link is rebuilt on the
kernel datapath and forwards.
The harness gains a Docker guest driver (a stdlib Engine API client over
the daemon socket: exec with stream demux, container id by deterministic
name -- the controller's node payload does not carry it, image inspect),
a digest-pinned image helper (cache hit = no network, miss = pull through
the server's own route, neither = skip) and Server.restart() for the
reopen scenario (which also needs the local compute's capabilities to be
up before a project open is accepted).
Live: 4/4 scenarios, 142 s for the file; full e2e suite 10/10 (1503 s);
unit suite 2149 passed; ruff clean. The matrix measures loss with 20
packets at 0.2 s spacing (143 s -> 58 s) and the jitter RTT assertion
keeps statistical margin (per-direction N(50,20); 10 packets, >= 65 ms).
Without one every c7200 busy-waits a full CPU core for the whole scenario
-- the tests create raw nodes and no template supplies a value (measured:
constant ~100 % per router, up to ~6 minutes per dynamips scenario).
harness.dynamips_idlepc() detects a value through the controller's
auto_idlepc endpoint (the GUI idle-PC finder path: a throwaway project
boots the image, candidate values are validated by measuring actual CPU
usage) and caches it in ~/.cache/gns3-e2e/idlepc.json keyed by the image
checksum, so the cost (~100 s here) is paid once per image, not per run.
GNS3_E2E_IDLEPC overrides detection; any failure returns None and the
scenarios run without one rather than fail on a CPU optimisation.
All four scenarios that boot c7200 routers (dynamips kernel/relay,
ethernet switch kernel/relay) pass the value at creation. Live: detected
0x60189234 for the M11 image (238 s first run including detection); the
routers drop from ~100 % during boot to 3-8 % steady (sampled per
process), and a re-run reports the cached value with no detection.
The spec's gns3-server alignment section promised three assertions the
suites never carried; they land in harness.py and run in every kernel
scenario (plus the relay controls' anchor checks):
- E.1 assert_pure_l2: anchors and bridges carry no L3 identity (addrgenmode
none, no address in /proc/net/if_inet6, no ip -o -4 addr) -- the kernel
scenarios' anchors and per-link/switch bridges, relay-mode anchors and
the switch's relay TAPs alike (hardening is a creation-time property,
not a datapath one);
- E.2 assert_idle_silence (Dynamips suite): guest interfaces shut to
silence IOS's own CDP/keepalives, settle 2.5 s, then a 5 s window of
zero traffic over both anchors and the per-link bridge;
- assert_forwarding: every attached port reports brport/state == 3 -- a
bridge left DOWN keeps its ports disabled and forwards nothing, with no
error anywhere; the assertion is the only signal.
harness.l2only_supported() probes a throwaway uBridge once (link l2only
against a nonexistent device: 208 = command present, 202-Unknown command
= old build); without the command the assertions skip -- the spec's
degradation contract, mirroring the tc-capability skip.
The first live run caught a fifth-creator gap: the switch's relay port
TAPs (minted implicitly by bridge add_nio_tap) carried a live fe80::.
Delivered uBridge-side per the spec (fb72758: the transient TAP is
hardened at creation, attaches untouched); the server needed no change,
and the relay-control E.1 check is that creator's end-to-end coverage.
Two real iol-xe containers (Cisco CML iol-runner images, real IOS-XE on
the telnet console) against an isolated instance, the real uBridge with
the swappable bridge TAP leg:
* kernel link reports kernel_datapath; anchors exist from node start
and the per-link bridge enslaves exactly the two of them;
* real ICMP crosses the [unix <-> tap] port-bridge relay; delay 100
lands as netem on both anchors with the measured RTT growing >= 150 ms
and returning < 50 ms when cleared;
* suspend admin-downs the anchor and kills the traffic; resume
restores; AF_PACKET capture writes a real pcap;
* link delete/re-create swaps the port leg out and back in with the
ping returning; node stop leaves no gx devices and a start recreates
the whole wiring from the NIO; project delete cleans everything;
* the relay control on a relay-configured instance keeps the link off
the kernel and still pings (anchors may exist idle — the same
semantics as QEMU's and Dynamips' anchors: the server flag decides
attachment, not anchor birth).
Harness: create_iol_router (the appliance's template properties),
docker_images, iol_anchor_name, stage_iol_base_config (the shipped
iol-xe-base.txt copied into the isolated instance's configs dir — the
GNS3_IOL_STARTUP_CONFIG lookup path), boot_wait(straight_prompt=True)
for images that boot from NVRAM straight into the CLI with no 'Press
RETURN' sentinel, and GNS3_E2E_DEBUG=1 to run the isolated instance
with debug logging (the uBridge command stream lands in server.log —
how the ensure-anchor EBUSY was pinned down).
Live e2e verification moves into the repository instead of being rewritten
as a throwaway script every time: tests/e2e/ carries a shared harness (REST
client, isolated-instance lifecycle, a deliberately dumb IOS console
driver, host-side kernel inspection: bridge membership, tap admin state,
tc qdiscs) and each node type adds only its scenario module. The e2e marker
plus 'addopts = -m "not e2e"' keeps these privileged, image-dependent
tests out of normal runs; they are opt-in: pytest -m e2e tests/e2e -s.
The harness targets an existing server via GNS3_E2E_URL or starts an
isolated instance (own config dir — DB and paths follow it — on a free
port) and deletes its project afterwards (GNS3_E2E_KEEP=1 keeps it on
failure).
The first scenario drives two real c7200 routers on a real IOS image
through the REST API and the consoles: the kernel link is created while the
routers are stopped (deferred wiring), the per-link kernel bridge enslaves
exactly the two anchors, real ICMP crosses it, delay 100 shows netem on
both anchors and grows the RTT (~200 ms), suspend kills the traffic on an
admin-down anchor, capture writes a pcap of ICMP records, a serial link
between the same routers stays on the relay and still pings, delete/
re-create and stop/start leave the wiring consistent, and deleting the
project removes every anchor and bridge. A negative control runs the same
topology on an isolated relay-configured instance (no per-link bridge,
nothing enslaved, still pings).
