diff --git a/docs/design/ubridge-l2-anchor-spec.md b/docs/design/ubridge-l2-anchor-spec.md index 168386d4c..2594de893 100644 --- a/docs/design/ubridge-l2-anchor-spec.md +++ b/docs/design/ubridge-l2-anchor-spec.md @@ -86,6 +86,12 @@ acceptance test in §E.2 therefore measures after a settle window. Suppressing it (e.g. `disable_ipv6` on the bridge before enslaving, for the MLD half) would be a new, separate requirement and has not been asked for. +### NetworkManager releases managed TAPs from bridges (measured) + +On hosts where NetworkManager manages the data-plane TAPs — it claims uBridge-created tun/tap devices on many desktops (`nmcli device` shows them as `disconnected` before NM's state machine has assumed them, `connected (externally)` after) — NM can **release a tap from its bridge** some 60-70 ms after enslavement, while the port is in that not-yet-assumed state. Measured with a minimal harness (one uBridge, one fresh tap, one bridge, an `addif`/`delif` loop, no server): ~50 % of re-enslavements lose the port, the freed device stays administratively **up** with the bridge's promiscuity/allmulti cleared, and no GNS3 process sends anything — gns3-server and uBridge log nothing, and the server's attachment bookkeeping stays intact. The *first* enslavement of a newly created tap was stable in every run; the hazard is a **re-enslavement** (link delete + recreate, a port re-join after a peer restart) before NM assumes the device — on the same tap, `nmcli device set managed no` flips the failure rate from 20/40 to 0/40, and re-managing it restores 20/40. + +Consequence for an affected link: the port silently leaves the bridge (frames drop) while the API stays green — the next NIO update re-heals it, because the switch's `_kernel_update` re-checks `brif` membership, but nothing forces one. Docker's veth host ends are not NM-managed and never showed the failure; every TAP anchor family is exposed (Dynamips `gd`, QEMU `gq`, IOU `gi`, IOL `gx`, and switch-absorbed anchors). Host-side remedy: `nmcli device set managed no` (runtime, per device), or NM's `unmanaged-devices` configuration keyed on the anchor name prefixes; the e2e harness takes the anchors it re-enslaves out of NM's hands (`harness.unmanage_from_networkmanager`). + ### Goal Every host-side device uBridge creates for the data plane is **pure L2**: no diff --git a/docs/features/ethernet-switch-kernel-datapath.md b/docs/features/ethernet-switch-kernel-datapath.md index 322e06d43..eb1ed607a 100644 --- a/docs/features/ethernet-switch-kernel-datapath.md +++ b/docs/features/ethernet-switch-kernel-datapath.md @@ -113,10 +113,16 @@ on **absorbed anchors** re-enslaving an interface owned by another node proved to be where links died: live testing showed an anchor occasionally losing its bridge membership right after a `ports_mapping` update (later updates then failed `delif` with EINVAL), reproducibly with qemu, IOU and -Dynamips peers, never with Docker's veths. Replaying the exact command -sequence by hand against uBridge (held TAPs, no server involved) did not -reproduce it, and the in-place reconcile removes the operation entirely — -the failure mode no longer exists by construction. +Dynamips peers, never with Docker's veths. That failure was root-caused +later: it is NetworkManager releasing NM-managed TAPs from their bridge +tens of milliseconds after (re)enslavement — veths are not NM-managed, +which is exactly why only TAP peers showed it (the mechanics and the +`nmcli device set managed no` remedy are in the L2-anchor spec). The +in-place reconcile removes this particular re-enslavement, but link delete ++ recreate and peer restarts still re-enslave the anchor, so the hazard +remains on NM hosts wherever NM manages the taps; the kernel e2e scenarios +take the anchors out of NM's hands +(`harness.unmanage_from_networkmanager`) before they exercise those paths. ## Link-local frames (LACP, LLDP, 802.1X, STP) diff --git a/tests/e2e/harness.py b/tests/e2e/harness.py index 9ebb0e351..852367709 100644 --- a/tests/e2e/harness.py +++ b/tests/e2e/harness.py @@ -1053,6 +1053,32 @@ def bridge_members(bridge): return sorted(os.listdir(f"/sys/class/net/{bridge}/brif")) +def unmanage_from_networkmanager(*names): + """Best-effort ``nmcli device set managed no`` for the given + interfaces. + + NetworkManager (desktop hosts) manages the TAP devices uBridge creates. + While a tap is in NM's not-yet-assumed state, NM's netdev-event state + machine can *release* it from its bridge some 60-70 ms after + enslavement (measured: ~50% of re-enslavements on an NM host): the port + is left administratively up, out of the bridge, with no server-side log + or error — a silent blackhole. Scenarios that detach and re-enslave an + anchor (link delete + recreate, node restart) must call this once their + nodes have booted, or they are nondeterministically flaky on such + hosts. Silently a no-op where nmcli is unavailable. + """ + nmcli = shutil.which("nmcli") + if not nmcli: + return + for name in names: + # S603: resolved binary, fixed argv, no shell + subprocess.run( # noqa: S603 + [nmcli, "device", "set", name, "managed", "no"], + capture_output=True, + check=False, + ) + + # --------------------------------------------------------------------------- # L2-anchor spec assertions (docs/design/ubridge-l2-anchor-spec.md §E) # --------------------------------------------------------------------------- diff --git a/tests/e2e/test_ethernet_switch_kernel_datapath.py b/tests/e2e/test_ethernet_switch_kernel_datapath.py index 6dc85fe2c..7cd91f48f 100644 --- a/tests/e2e/test_ethernet_switch_kernel_datapath.py +++ b/tests/e2e/test_ethernet_switch_kernel_datapath.py @@ -146,6 +146,15 @@ def test_ethernet_switch_kernel_fast_path(): c1 = _boot(server, pid, r1_id) c2 = _boot(server, pid, r2_id) + # Keep NetworkManager's hands off the anchors: on NM hosts its + # netdev state machine can release a freshly re-enslaved tap from + # the bridge ~60 ms after the join (silently — the port stays up, + # the server keeps believing it is attached), which the link + # delete/recreate below triggers nondeterministically. The first + # join at node start is unaffected; this covers everything after + # it. + harness.unmanage_from_networkmanager(a1, a2) + # Deferred joins completed by the node-start re-push: the anchors # ARE the switch ports now. assert harness.wait_until(lambda: harness.bridge_members(sw_bridge) == sorted([a1, a2]), timeout=15), (