The persistent-TAP lifecycle — probe, sweep, create, register, delete —
was implemented four times, once per node type, and the copies had drifted
(probe mechanism, _tap_datapath reset, keep-vs-recreate policy). The
mechanical half moves into KernelDatapathMixin:
- _probe_persistent_taps(probe_name): the scratch-TAP probe, returning the
refusal for the caller's own warning (QEMU, Dynamips);
- _create_anchor_taps(ports, set_owner, keep_existing): sweep + create +
optional ownership handover + carrier down + registry, with the ports
enumerated by the node type and named through the new _tap_name hook;
- _delete_anchor_taps(): the suppressed tap sweep + per-link bridge
cleanup + registry clear.
Each node type keeps what its fd holder dictates: the release step before
the tap sweep (the QEMU process, the IOL bridge, the hypervisor's NIO
bindings, the container port bridges), its probe policy, and its registry
lifetime. QEMU's now-unconditional registry clear aligns it with the other
three; everything else is behavior-preserving.
Compute:
- Track the anchor that owns uBridge's single AF_PACKET capture slot: a
second kernel port's capture is refused with a clear error (its flag
rolled back), a failed start no longer wedges the port, and only the
owning port's stop issues the process-wide `capture stop_kernel`
(docker/qemu/iou/dynamips/switch, via the mixin's slot helpers).
- Unwind failed starts instead of leaking uBridge and the anchor TAPs:
QEMU's stop body runs under try/finally, QEMU and IOU start the datapath
inside the launch try, Dynamips undoes a half-attached port (the retry
used to be short-circuited into a silent black hole), and Docker deletes
the veth pair when the namespace move fails.
- Restore the relay filter guard on IOU (kernel-only filters raise
NodeError, an uncompilable BPF line degrades to a warning), validate the
eBPF/quota/window/bpf filter shapes before touching the bridge (short or
non-numeric values surfaced as 500s after partial state), and refuse a
second NIO on an occupied Docker/QEMU port (the guard IOU and Dynamips
already had).
- Probe fixes: `_probe_bridge_tap` survives a failing uBridge start, and
the probe caches key on ctime too, so a chmod/setcap remedy re-probes.
Controller:
- Parse GNS3_UNIX_SOCKET_NIO like the compute does (line-anchored, truthy
value) and mirror the compute's class selection in kernel_anchor_type.
- Read partial capability reports defensively; make BridgeNIO.bridge
required-but-nullable so an omitted key is a 422 rather than a KeyError.
- Strip the packet_loss correlation slot even when it is zero: the relay
filter takes exactly one argument and rejected `packet_loss 10 0`.
Ethernet switch: skip ports whose absorbed anchor does not exist yet (the
join is deferred by design) and roll the ports_mapping back when the VLAN
reconcile fails, so a retry still diffs against the old mapping.
Agent: reject device config entries without a device_name instead of
silently dropping them (a malformed batch used to execute partially while
reporting success).
Also split the netem builder to clear C901 and drop the two per-file ruff
ignores the PR had added, keeping the remove-only policy.
The rebase onto 65c858d45 brought upstream's UP032 enforcement; the
conflict resolution kept this branch's semantics on the overlapping
lines, so their .format() calls needed the conversion afterwards
(ruff check --fix, 41 sites) plus the formatter's rewrap (7 files).
Replicates the CI recipe locally: pip check, ruff check/format, mypy
2.3.1 (clean, 434 files), unit suite 2216 passed.
The rebase onto upstream 3.1 (mypy + ruff I001/F401 waves) left the tree
short of the newly enforced baselines in three ways:
* contextlib / UbridgeError imports dropped as false duplicates during
conflict resolution left NameErrors in iou_vm and qemu_vm (F821);
the Dynamips router grabbed the compute-level NIOTAP instead of the
hypervisor one, breaking TAP anchoring (test failure).
* unused imports (KERNEL_UNSUPPORTED_FILTERS, UbridgeError, List) and
unsorted blocks left behind where the stack's later commits moved the
last usage elsewhere — ruff I001/F401/RUF022 now enforce these.
* mypy, now baseline-free: cache dict annotations in tc_probe, the
locators candidate list, the resolved-params None branch in the MCP
runner, and validate_bpf_syntax keeps the upstream Dict[str, Any]
return (the strict Optional[str] was wrong for the "valid" bool entry).
The e2e raw-frame probe resolves its compiler via shutil.which (S607)
and zips with strict= (B905); the remaining subprocess call is S603 by
nature and gets a per-file ignore like other test files.
ruff check/format clean, mypy at upstream parity (no new errors),
full unit suite 2167 passed.
The peer end of an Ethernet switch fast-path link is a NIOBridge with no
bridge of its own (the switch absorbed its anchor into the switch bridge).
Its update reconcile still ran the impairment pass with the empty filter
set the controller routes to that end — and an empty set is a plain
`tc reset` on the shared anchor, silently wiping the netem/cls_bpf/eBPF
the switch had just applied. Any link update did it (filters, markers,
suspend, capture start/stop), and the outcome depended on the link's
endpoint order: UdpLink.update() PUTs node1 then node2, so with the
switch first its apply ran first and the peer's reset ran last (API
green, data plane unimpaired), while with the peer first the reset ran
first and the filter survived. That order dependence is also why the
shipped e2e never caught it — it only created peer-first links.
Ownership is now explicit: only the end that owns the anchor's tc state
applies the impairment reconcile (_owns_anchor_impairments — a NIOBridge
with `bridge is None` is a passive carrier, the same idiom the
bridge-delete path already used). Markers, capture and the suspend
carrier still apply on both ends: markers reconcile per process, so
neither end can strip the other's, and the carrier converges by
agreement.
Red/green: two new docker unit tests fail on the unfixed tree with
`tc reset "<anchor>"` and pass after; the switch e2e also creates one
link switch-first and asserts the netem qdisc and the RTT hold under
that order (failed with the wipe, passes with the fix). Full unit suite
2110 passed; the switch e2e file passed end to end (378s).
A correctness read-through of the five delivered datapath documents, plus
the two code comments that carried the same stale vocabulary.
Datapath selection: the VendorDockerVM selector is `console_type=docker_exec`
alone (`GNS3_SKIP_INIT` and friends are behavior knobs of the chosen class);
the cross-compute and anchor-less fallbacks are named for what they are; the
Ethernet-switch endpoint is the documented exception to the anchoring rule,
and the IOL runner container is called out as the Docker node that does not
anchor on a veth.
Filter vocabulary: the stateful eBPF program evaluates four modes
(nth → quota → window → flow), GNS3 exposes three of them, and window_drop
joins the kernel-only list; the correl suffixes gate on the `rate` token as
the extension-build marker.
group_fwd_mask: the 0xfffd port mask is applied inside uBridge's `brctl addif`
itself, not as a separate command — the link-creation sequences and the
operation tables no longer show an arrow the server never sends. Same class
of fix for `link set up` on the IOL anchor (addif already brought it up; the
command only pushes a suspended anchor back down) and for the QEMU carrier
replication (QMP `set_link`, gated by `replicate_network_connection_state`).
Lifecycles and stop orders re-worded around the invariant rather than the
node type: the fd holder must release before the devices are deleted, which
is QEMU's process-first order, IOU's uBridge-first order and Dynamips'
hypervisor-NIO-first order. Deferred wiring for a stopped endpoint is
documented as the common path it is, and the Dynamips "stop is not a close"
note now says what recovery from a dead hypervisor actually requires.
Dynamips specifics: a c7200 needs an adapter in slot 1 (slot 0 is the fixed
I/O slot), the tap-fd direction is stated correctly (writes inject into the
kernel, reads return what the kernel transmits), and the anchor registry is
name-keyed against the controller's port matrix by convention, not derivation.
IOL port bridge naming: `bridge{bay}` on unit 0, `bridge{bay}_{unit}`
elsewhere; the leg swap happens on link delete/re-create or a datapath
switch, while suspend is an in-place NIO update and a project reopen is a
fresh build.
Verified and Roadmap sections: the in-repo e2e suite is distinguished from
the isolated-instance development harness (IOU), assertions are stated with
their bounds and capability gates, and the roadmap entries collapse into the
one remaining project — VXLAN/GENEVE for cross-compute links.
Code comments: `flow` added to the documented mode order in the eBPF apply
path, and the `nio_bridge` module docstring corrected on which filter types
run on the kernel datapath (bpf via cls_bpf, frequency_drop/quota/window_drop
via the eBPF classifier — neither is relay-only any more).
The Ethernet switch's kernel fast path: instead of a link crossing the
switch's per-port uBridge relay, the switch absorbs the peer's anchor into
its own kernel bridge. The peer's interface (Docker veth host end, QEMU
TAP, IOU port TAP, Dynamips port TAP) becomes the switch port, and frames
cross anchor -> switch bridge -> switch ports entirely in the kernel.
Wire format: the switch end receives nio_anchor ('join this interface to
my own bridge' — the mirror of nio_bridge), the peer end nio_bridge with a
null bridge ('your anchor is bridged elsewhere': carrier, markers and
impairments only — never bridge membership, and never a bridge delete, on
an interface this node does not own). Anchor names come from the shared
naming contract (utils.kernel_anchor), a pure function of node type, id,
adapter and port — the controller computes the peer's anchor while the
node may be stopped, which is what makes links to stopped nodes wireable.
Stopped peers defer the join; after any node of a link starts
(UDPLink.node_started) the controller re-pushes the switch NIO, and the
switch completes — or repairs, when a peer restart replaced its interface
under the same name — the join against the kernel's actual membership
instead of assuming it.
VLAN changes now edit membership in place: only the ports whose mode/VLAN
actually changed are touched, each losing exactly the previous mode's
entries the new mode does not want (the dot1q admits-all entry is one
range operation in both directions), and no port ever leaves the bridge to
change its VLANs. This replaces the old 'reset by re-enslaving' shortcut
(delif+addif on every wired port, added when the ports were the switch's
own TAPs): on absorbed anchors, re-enslaving an interface owned by another
node is where live testing watched links die — anchors occasionally lost
their bridge membership right after a ports_mapping update (later updates
then failed with delif EINVAL), reproducibly with qemu, IOU and Dynamips
peers and never with Docker's veths.
Eligibility: the switch side needs no capability of its own (its bridge
exists by construction); the peer must be able to anchor. Switch-to-switch
cascades stay on the relay — a kernel interface belongs to one bridge, and
joining two bridges needs a veth pair, deferred.
Verified live on a real server: qemu peers through four VLAN cycles with
both anchors enslaved throughout; a mixed switch (2 IOU + 2 Dynamips, four
kernel ports); Docker peers with real ICMP isolating and recovering (with
both anchors still enslaved); VPCS relay ports likewise in place.
QEMU had no host-side interface at all: every adapter was a local UDP tunnel
into uBridge (-netdev socket plus two UDP ports per adapter), so a link could
only ever be a userspace relay. Each adapter now owns a persistent TAP
(gq{node_id[:8]}e{adapter}p{port}), created by uBridge before QEMU is launched
and handed to this user, and the netdev is that TAP -- the anchor role Docker's
veth host end plays. Links attach to it:
* kernel link (NIOBridge): brctl create/up/addif enslaves the TAP into the
per-link bridge and capture, markers and tc filters are installed on it,
verbatim through the shared KernelDatapathMixin -- a TAP and a veth host end
are interchangeable there (an egress qdisc on either impairs traffic entering
the node, and PACKET_OUTGOING on either is the node receiving);
* relay link: bridge add_nio_ethernet plus add_nio_udp -- AF_PACKET, never the
TAP NIO, which would take the fd QEMU itself holds.
The netdev type is fixed at launch, so this is all-or-nothing per VM: on the
TAP datapath every link type attaches to (or detaches from) a running VM,
including switching a link between the two datapaths. Stop un-persists the
TAPs (QEMU's open fd keeps each device alive until it exits) and removes the
per-link bridges; start sweeps leftovers first. A uBridge without the tap
module keeps the previous socket-netdev datapath, so an unupgraded install is
unaffected -- such a node simply cannot carry kernel links.
Move the datapath-agnostic half of DockerKernelDatapathMixin into a new
KernelDatapathMixin (compute/kernel_datapath.py) so a second node type can
reuse it verbatim: the three marker primitives, the tc netem / cls_bpf / eBPF
appliers, the uBridge tc-capability probe, kernel marker teardown, link
detach, the per-link bridge cleanup and the adapter carrier -- plus
_kernel_attach(), lifted from DockerVM._connect_nio's kernel branch.
The mixin talks about "the anchor" (the host-side interface an adapter owns in
the root namespace) instead of "the veth host end": a node type provides
_kernel_host_ifc(), _kernel_anchors() and, when needed, _relay_carrier_bridge()
and _kernel_error(). Docker keeps only the veth lifecycle and those hooks;
behaviour is unchanged.
The C901 baseline entry moves with _netem_command_parts (the docker module is
no longer over the limit) and the bpf-compile-failure test patches the logger
of the module that now owns the code -- net zero new ignores.
