gns3-server/docs/design/ubridge-l2-anchor-spec.md

Frozen requirements spec for the uBridge project. Status: delivered (uBridge e2e3155 — link l2only plus the creators that apply it; fb72758 — the fifth creator: the TAP bridge add_nio_tap creates for a free name) and verified from the server side: anchors and per-link bridges report addrgenmode none, ip -4/-6 addr are empty, and an idle anchor is silent once the one-shot membership burst of §0 has settled.

Residual, accepted (measured after delivery): enslaving a port still makes the bridge device announce its multicast memberships — one IGMPv3 plus one or two MLDv2 reports in the first second — which floods into the emulated segment. It is outside this command's reach by design (no addresses are involved, and IPv4 has no per-device IGMP off switch); see §0.

Scope guard: no wire format, no command grammar change to existing commands — this adds one command and one internal step in four creators.

L2-only host anchor devices (link l2only)

0. Background & scope

What an anchor is

The host-side device that carries a node's adapter in the root namespace: the veth host end of a Docker adapter (gv…, created by docker create_veth), the persistent TAP of a QEMU adapter (gq…, created by tap create), and the per-link kernel bridge those anchors are enslaved to (brctl create). Links attach to these devices (kernel bridge port or uBridge relay endpoint); the devices themselves are the only thing uBridge creates on the host's behalf.

The problem (measured)

A host-side anchor that is UP gets an IPv6 link-local address from the kernel automatically — no user-space actor involved — and with it the kernel emits its own traffic on that device:

• a freshly created persistent TAP, brought UP: 6 frames in 2 s of MLD reports (33:33:00:00:00:16, 33:33:00:00:00:fb) and DAD neighbor solicitations (33:33:ff:…, ICMPv6 next-header 0/17/58), all sourced from the device's own link-local address;
• production: the Docker veth host end gv6ee8d537e1p0 carries fe80::5c30:e8ff:fe00:5d8d/64 (disable_ipv6=0), and the per-link bridge gns30dbb78c5718 carries fe80::f0d9:94ff:feac:e42b/64.

Consequences inside an emulated topology:

• on a kernel-datapath link the bridge floods those frames into the emulated segment; on a relay link uBridge relays them to the peer compute;
• they land in captures and marker pcaps, so "idle link" is not quiet;
• the host answers neighbor solicitations for its own link-local address, so an emulated IPv6 router can form a real adjacency with the host — a phantom neighbour inside the emulated network.

IPv4 needs no equivalent knob: nothing self-provisions an IPv4 address, and the invariant we keep is no addresses at all (measured: ip -4 addr is empty on every anchor and per-link bridge on the deployment above, with NetworkManager reporting them unmanaged). IPv6 is the only protocol that lights itself up.

The residual this command does not remove (measured after delivery)

With l2only in place (anchors and bridges report addrgenmode none, no addresses), enslaving a port still makes the bridge device speak once — the kernel announces the multicast memberships the enslavement brings to the bridge:

+0.00 s   90 B   IPv6   MLDv2 report   dst 33:33:00:00:00:16  (ICMPv6 143)
+0.09 s   54 B   IPv4   IGMPv3 report  dst 01:00:5e:00:00:16  0.0.0.0 → 224.0.0.22
+0.73 s   90 B   IPv6   MLDv2 report   dst 33:33:00:00:00:16

(Scratch tap pair + per-link bridge on the test host, unprivileged; source MAC = the bridge's. A 10 s window contains exactly those three frames, then silence. The same burst appears through the server: one 90 B IPv6 frame inside a 2 s window right after a link is created.)

It is outside l2only's reach by design: no addresses are involved (the report sources are :: and 0.0.0.0), and IPv4 has no per-device "no IGMP" knob to set. Decision: accepted and documented, not suppressed — it is standard L2 control traffic, one-shot per link creation or port change, and it stops. The acceptance test in §E.2 therefore measures after a settle window. Suppressing it (e.g. disable_ipv6 on the bridge before enslaving, for the MLD half) would be a new, separate requirement and has not been asked for.

NetworkManager releases managed TAPs from bridges (measured)

On hosts where NetworkManager manages the data-plane TAPs — it claims uBridge-created tun/tap devices on many desktops (nmcli device shows them as disconnected before NM's state machine has assumed them, connected (externally) after) — NM can release a tap from its bridge some 60-70 ms after enslavement, while the port is in that not-yet-assumed state. Measured with a minimal harness (one uBridge, one fresh tap, one bridge, an addif/delif loop, no server): ~50 % of re-enslavements lose the port, the freed device stays administratively up with the bridge's promiscuity/allmulti cleared, and no GNS3 process sends anything — gns3-server and uBridge log nothing, and the server's attachment bookkeeping stays intact. The first enslavement of a newly created tap was stable in every run; the hazard is a re-enslavement (link delete + recreate, a port re-join after a peer restart) before NM assumes the device — on the same tap, nmcli device set <tap> managed no flips the failure rate from 20/40 to 0/40, and re-managing it restores 20/40.

Consequence for an affected link: the port silently leaves the bridge (frames drop) while the API stays green — the next NIO update re-heals it, because the switch's _kernel_update re-checks brif membership, but nothing forces one. Docker's veth host ends are not NM-managed and never showed the failure; every TAP anchor family is exposed (Dynamips gd, QEMU gq, IOU gi, IOL gx, and switch-absorbed anchors). Host-side remedy: nmcli device set <tap> managed no (runtime, per device), or NM's unmanaged-devices configuration keyed on the anchor name prefixes; the e2e harness takes the anchors it re-enslaves out of NM's hands (harness.unmanage_from_networkmanager). Full root-cause record with the minimal reproducer, the NetworkManager source-level mechanism and the verified fix: docs/bugs/networkmanager-tap-release.md.

Goal

Every host-side device uBridge creates for the data plane is pure L2: no IPv4 address, no IPv6 address, and no IPv6 stack activity on that device.

Explicitly not in scope:

• the guest/container side (the peer veth end inside a container netns keeps its normal IPv6 behaviour);
• global sysctls and any change outside the named interface;
• brctl addip-style L3 use of a bridge (cloud / Ethernet-switch paths) — those devices are meant to have addresses and are not data-plane anchors.

A. New command: link l2only <iface> [on|off]

Belongs to the link module (per-interface operations: link set, link veth, link addr, link delete).

Arg	Description
<iface>	An existing device. A missing name must fail (208/ENODEV), never create a transient device — mirror the tap module's tap_require_existing reasoning.
on / off	Default on. on = suppress IPv6 address generation and stack activity on the device; off = restore the kernel default.

Semantics of on: after the call the kernel has assigned no IPv6 link-local address to the device, and the device's addrgenmode reports none (IFLA_INET6_ADDR_GEN_MODE = IN6_ADDR_GEN_MODE_NONE), so no DAD, MLD or RS is generated from it. If an address already exists it is removed as part of the call (an anchor that was created and brought up before this command must end up clean, not just "no new addresses from now on").

Replies:

link l2only gq1234abcd e0
100-L2-only set on gq1234abcd e0
link l2only gq1234abcd e0 on
100-L2-only set on gq1234abcd e0          # idempotent, same reply
link l2only gq1234abcd e0 off
100-L2-only cleared on gq1234abcd e0

The success reply is emitted only after a read-back (RTM_GETLINK + IFLA_AF_SPEC → IFLA_INET6_ADDR_GEN_MODE) confirms the device state, so a caller may treat 100 as verified rather than requested.

B. Where it must be applied

The creators apply it themselves — gns3-server never issues link l2only directly, and a caller cannot forget it:

Creator	Device hardened	Notes
tap create <name>	the persistent TAP	before the success reply; the device already exists (TUNSETPERSIST)
docker create_veth <host> <guest>	the host end only	the guest end moves into a container netns and is deliberately untouched
link veth <name> <peer>	both ends	both ends are host-side
brctl create <bridge>	the bridge device itself	the fabric the anchors are enslaved to; the bridge's own link-local floods to every port
bridge add_nio_tap <br> <name>	the TAP it creates (name free)	by-name TUNSETIFF creates a transient device when the name is free — cloud's bridge interfaces, and the relay swap's create-if-missing path. Hardened at creation; an attach to an existing device is left untouched (the caller may name a user-owned TAP that has addresses).

Rule for future creators: any command that creates a host-side device for the data plane hardens it, and the acceptance checks in §E apply to it.

An attach is not a creation: bridge add_nio_tap (and every other open of a TAP that already exists) must leave that device's addresses alone. The transient TAP bridge add_nio_tap creates for a free name is a creation and is hardened like the rest — best-effort, before the NIO is handed back.

C. Implementation notes (netlink, not /proc)

• Use RTM_SETLINK with IFLA_AF_SPEC{ AF_INET6, IFLA_INET6_ADDR_GEN_MODE = IN6_ADDR_GEN_MODE_NONE } — this is what ip link set dev X addrgenmode none does. Removing an already-assigned link-local is RTM_DELADDR on that address (or letting the kernel drop it once generation is off; verify the end state either way).
• Optionally also set the nested IFLA_INET6_CONF attributes (DEVCONF_DISABLE_IPV6 = 1, DEVCONF_ACCEPT_RA = 0) when the running kernel accepts writes for them. The addr-gen-mode attribute alone already removes the address, and without an address there is no DAD/MLD/RS.
• Do not write /proc/sys/net/ipv6/conf/<if>/disable_ipv6. The sysctl files are root-owned mode 0644, so a setcap'd, non-root uBridge can be refused by the DAC check despite holding CAP_NET_ADMIN. The netlink route has no such problem.
• Failure handling: EOPNOTSUPP / EINVAL from an old kernel without the attribute must be non-fatal (log, continue) so device creation never fails because of this hardening; ENODEV and any other error are real and reported.

D. Reply contract & error codes

Code	Meaning
100	OK — device verified L2-only (or verified restored for off)
203	Bad number of parameters
204	Invalid parameter (on/off expected)
208	Object not found (ENODEV): no such device
206	Unable to set (netlink error other than EOPNOTSUPP/EINVAL)

E. Test requirements

1. No addresses. After each creator: ip -6 addr show dev <dev> is empty (in particular no fe80::) and ip -4 addr show dev <dev> is empty; the same for the bridge created by brctl create.
2. Idle silence, after the settle window. Let the last port enslavement settle (≥ 2 s — that burst is expected, see §0), then with the device UP and in its normal role (TAP with an open fd / veth with its peer up / bridge with two attached ports) nothing appears on it for 5 s — measured from the peer end, or via capture start_kernel <dev> <pcap> while idle with the pcap asserted empty. Baseline before this work: 6 frames / 2 s, continuous.
3. Idempotency & revert. on twice → 100 both times, state unchanged; off restores the kernel default (link-local returns after the device is cycled down/up).
4. No transient device. link l2only nosuchif0 → 208, and no interface is created (check ip -o link).
5. Far side untouched. Inside the container / VM the normal IPv6 link-local still exists — nothing in this change reaches the peer.
6. Non-fatal on old kernels. Simulated by testing on a kernel without the attribute: the creators still succeed, and §E.1 is then expected to fail with the device left in the default state.

gns3-server alignment (informational — not uBridge scope)

• Call sites: the creators harden the devices; the server issues no link l2only itself, and no NIO/JSON schema, filter or MCP tool description changes. This is host-side hygiene only.
• The fifth creator was found by these servers' e2e assertions: on a build predating fb72758, the Ethernet switch's relay port TAPs (gns3{id}-N, created implicitly by bridge add_nio_tap) carried a live fe80:: — the §E.1 assertion failed on exactly that device, live. The close is uBridge-side (§B table row 5): the transient TAP is hardened at creation, attaches stay untouched. The server side needed no change for it — the assertion alone caught it, and it now guards the fixed build.
• e2e assertions (landed in tests/e2e/harness.py, asserted by the kernel-datapath suites alongside the existing capture/marker/filter checks):
  • anchors and per-link bridges have no IPv4/IPv6 addresses, addrgenmode none (assert_pure_l2, §E.1) — on the kernel scenarios' anchors and bridges, and on relay-mode anchors/switch TAPs alike (the hardening is a creation-time property, not a datapath one). The switch relay-TAP assertion is what exercises the fifth creator end-to-end (the same uBridge creation path cloud's transient TAPs take);
  • an idle link stays silent for 5 s after the settle window (assert_idle_silence, §E.2) — measured with the guests silenced (the IOS consoles' interfaces shut, the Docker guests' links down, all four cascade routers shut) over the anchors, the per-link bridge and the switch bridges, in every kernel scenario: Dynamips, Docker, QEMU, IOU, the IOL container, the switch fast path and the switch cascade. That covers each creation path — tap create, docker create_veth (plus the server-side harden-after-create on the cascade's race-loser end), bridge add_nio_tap and the brctl switch bridges (whose multicast snooping is off precisely so the bridge role can reach silence);
  • a silent-failure guard found while validating this spec: after attach, every bridge port reports brport/state == 3 (forwarding) (assert_forwarding). A bridge device left DOWN keeps its ports DISABLED (state == 0) and forwards nothing — with no error anywhere, so the assertion is the only signal.
• Rollout: with a uBridge that lacks the command the anchors keep today's behaviour (noise present). The e2e assertions skip when link l2only answers 202-Unknown command (harness.l2only_supported() probes a throwaway uBridge once), mirroring the existing tc-capability degradation. A build that has the command but predates fb72758 fails the switch relay-TAP assertion instead of skipping: the gap is exactly what it exists to catch.