mirror of
https://github.com/GNS3/gns3-server.git
synced 2026-10-12 21:45:41 +03:00
test(e2e): take the switch anchors out of NetworkManager's reach
The esw kernel fast-path scenario deletes and re-creates a link, which re-enslaves the peer anchor — and on NetworkManager hosts NM's netdev state machine can release an NM-managed TAP from its bridge some 60-70 ms after enslavement, ~50% of re-enslavements, while the tap is in NM's not-yet-assumed state. The freed port stays administratively up and no GNS3 process logs anything: the membership assert sees the join, then the ping measures a silent blackhole (this was the line-277 e2e failure). Root-caused with a server-free harness (one uBridge, one fresh tap, one bridge, an addif/delif loop): the same tap frees 20/40 while NM-managed, 0/40 after `nmcli device set <tap> managed no`, 20/40 again when re-managed. Docker's veth host ends are not NM-managed, which is why only TAP anchors ever showed it — including the "anchor lost its bridge membership after a ports_mapping update (TAP peers only)" incident from the switch's development, whose conclusion is corrected in the docs. Unmanage the anchors once the routers have booted, before the scenario re-enslaves them; document the failure signature and the host-side remedy (per-device nmcli call, or NM unmanaged-devices configuration) in the L2-anchor spec and the switch feature doc.
This commit is contained in:
parent
dc5e2d90ba
commit
1c85befbab
@ -86,6 +86,12 @@ acceptance test in §E.2 therefore measures after a settle window. Suppressing
|
||||
it (e.g. `disable_ipv6` on the bridge before enslaving, for the MLD half) would
|
||||
be a new, separate requirement and has not been asked for.
|
||||
|
||||
### NetworkManager releases managed TAPs from bridges (measured)
|
||||
|
||||
On hosts where NetworkManager manages the data-plane TAPs — it claims uBridge-created tun/tap devices on many desktops (`nmcli device` shows them as `disconnected` before NM's state machine has assumed them, `connected (externally)` after) — NM can **release a tap from its bridge** some 60-70 ms after enslavement, while the port is in that not-yet-assumed state. Measured with a minimal harness (one uBridge, one fresh tap, one bridge, an `addif`/`delif` loop, no server): ~50 % of re-enslavements lose the port, the freed device stays administratively **up** with the bridge's promiscuity/allmulti cleared, and no GNS3 process sends anything — gns3-server and uBridge log nothing, and the server's attachment bookkeeping stays intact. The *first* enslavement of a newly created tap was stable in every run; the hazard is a **re-enslavement** (link delete + recreate, a port re-join after a peer restart) before NM assumes the device — on the same tap, `nmcli device set <tap> managed no` flips the failure rate from 20/40 to 0/40, and re-managing it restores 20/40.
|
||||
|
||||
Consequence for an affected link: the port silently leaves the bridge (frames drop) while the API stays green — the next NIO update re-heals it, because the switch's `_kernel_update` re-checks `brif` membership, but nothing forces one. Docker's veth host ends are not NM-managed and never showed the failure; every TAP anchor family is exposed (Dynamips `gd`, QEMU `gq`, IOU `gi`, IOL `gx`, and switch-absorbed anchors). Host-side remedy: `nmcli device set <tap> managed no` (runtime, per device), or NM's `unmanaged-devices` configuration keyed on the anchor name prefixes; the e2e harness takes the anchors it re-enslaves out of NM's hands (`harness.unmanage_from_networkmanager`).
|
||||
|
||||
### Goal
|
||||
|
||||
Every host-side device uBridge creates for the data plane is **pure L2**: no
|
||||
|
||||
@ -113,10 +113,16 @@ on **absorbed anchors** re-enslaving an interface owned by another node
|
||||
proved to be where links died: live testing showed an anchor occasionally
|
||||
losing its bridge membership right after a `ports_mapping` update (later
|
||||
updates then failed `delif` with EINVAL), reproducibly with qemu, IOU and
|
||||
Dynamips peers, never with Docker's veths. Replaying the exact command
|
||||
sequence by hand against uBridge (held TAPs, no server involved) did not
|
||||
reproduce it, and the in-place reconcile removes the operation entirely —
|
||||
the failure mode no longer exists by construction.
|
||||
Dynamips peers, never with Docker's veths. That failure was root-caused
|
||||
later: it is NetworkManager releasing NM-managed TAPs from their bridge
|
||||
tens of milliseconds after (re)enslavement — veths are not NM-managed,
|
||||
which is exactly why only TAP peers showed it (the mechanics and the
|
||||
`nmcli device set <tap> managed no` remedy are in the L2-anchor spec). The
|
||||
in-place reconcile removes this particular re-enslavement, but link delete
|
||||
+ recreate and peer restarts still re-enslave the anchor, so the hazard
|
||||
remains on NM hosts wherever NM manages the taps; the kernel e2e scenarios
|
||||
take the anchors out of NM's hands
|
||||
(`harness.unmanage_from_networkmanager`) before they exercise those paths.
|
||||
|
||||
## Link-local frames (LACP, LLDP, 802.1X, STP)
|
||||
|
||||
|
||||
@ -1053,6 +1053,32 @@ def bridge_members(bridge):
|
||||
return sorted(os.listdir(f"/sys/class/net/{bridge}/brif"))
|
||||
|
||||
|
||||
def unmanage_from_networkmanager(*names):
|
||||
"""Best-effort ``nmcli device set <name> managed no`` for the given
|
||||
interfaces.
|
||||
|
||||
NetworkManager (desktop hosts) manages the TAP devices uBridge creates.
|
||||
While a tap is in NM's not-yet-assumed state, NM's netdev-event state
|
||||
machine can *release* it from its bridge some 60-70 ms after
|
||||
enslavement (measured: ~50% of re-enslavements on an NM host): the port
|
||||
is left administratively up, out of the bridge, with no server-side log
|
||||
or error — a silent blackhole. Scenarios that detach and re-enslave an
|
||||
anchor (link delete + recreate, node restart) must call this once their
|
||||
nodes have booted, or they are nondeterministically flaky on such
|
||||
hosts. Silently a no-op where nmcli is unavailable.
|
||||
"""
|
||||
nmcli = shutil.which("nmcli")
|
||||
if not nmcli:
|
||||
return
|
||||
for name in names:
|
||||
# S603: resolved binary, fixed argv, no shell
|
||||
subprocess.run( # noqa: S603
|
||||
[nmcli, "device", "set", name, "managed", "no"],
|
||||
capture_output=True,
|
||||
check=False,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# L2-anchor spec assertions (docs/design/ubridge-l2-anchor-spec.md §E)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@ -146,6 +146,15 @@ def test_ethernet_switch_kernel_fast_path():
|
||||
c1 = _boot(server, pid, r1_id)
|
||||
c2 = _boot(server, pid, r2_id)
|
||||
|
||||
# Keep NetworkManager's hands off the anchors: on NM hosts its
|
||||
# netdev state machine can release a freshly re-enslaved tap from
|
||||
# the bridge ~60 ms after the join (silently — the port stays up,
|
||||
# the server keeps believing it is attached), which the link
|
||||
# delete/recreate below triggers nondeterministically. The first
|
||||
# join at node start is unaffected; this covers everything after
|
||||
# it.
|
||||
harness.unmanage_from_networkmanager(a1, a2)
|
||||
|
||||
# Deferred joins completed by the node-start re-push: the anchors
|
||||
# ARE the switch ports now.
|
||||
assert harness.wait_until(lambda: harness.bridge_members(sw_bridge) == sorted([a1, a2]), timeout=15), (
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user