feat(qemu): anchor QEMU adapters on a persistent TAP

QEMU had no host-side interface at all: every adapter was a local UDP tunnel
into uBridge (-netdev socket plus two UDP ports per adapter), so a link could
only ever be a userspace relay. Each adapter now owns a persistent TAP
(gq{node_id[:8]}e{adapter}p{port}), created by uBridge before QEMU is launched
and handed to this user, and the netdev is that TAP -- the anchor role Docker's
veth host end plays. Links attach to it:

* kernel link (NIOBridge): brctl create/up/addif enslaves the TAP into the
  per-link bridge and capture, markers and tc filters are installed on it,
  verbatim through the shared KernelDatapathMixin -- a TAP and a veth host end
  are interchangeable there (an egress qdisc on either impairs traffic entering
  the node, and PACKET_OUTGOING on either is the node receiving);
* relay link: bridge add_nio_ethernet plus add_nio_udp -- AF_PACKET, never the
  TAP NIO, which would take the fd QEMU itself holds.

The netdev type is fixed at launch, so this is all-or-nothing per VM: on the
TAP datapath every link type attaches to (or detaches from) a running VM,
including switching a link between the two datapaths. Stop un-persists the
TAPs (QEMU's open fd keeps each device alive until it exits) and removes the
per-link bridges; start sweeps leftovers first. A uBridge without the tap
module keeps the previous socket-netdev datapath, so an unupgraded install is
unaffected -- such a node simply cannot carry kernel links.
This commit is contained in:
YueGuobin committed 2026-10-06 00:21:08 +08:00
1 parent 12d8abce45
commit 7d2d8c6747
4 files changed
+650 -53

No files matched your search

+58
View File
@@ -171,6 +171,64 @@ class KernelDatapathMixin:
await self._ubridge_apply_bpf_drops(anchor, nio.filters)
await self._ubridge_apply_ebpf_drops(anchor, nio.filters)
async def _relay_attach(self, anchor, bridge_name, nio):
"""
Attach an anchor to a uBridge relay bridge (NIOUDP): the bridge turns
the anchor into an AF_PACKET endpoint and relays it to the link's UDP
peer. Called on link creation and on node start, and after a link was
re-created — the relay analogue of _kernel_attach.
"""
await self._ubridge_send(f"bridge create {bridge_name}")
# libpcap cannot open a packet socket on an admin-down interface (its
# netlink promiscuous-mode transaction returns ENOENT), and an anchor
# is born down (carrier off until a link attaches) — bring it up for
# the relay attach. The carrier pass in the caller refines the state
# afterwards (a suspended NIO sets it back down).
await self._ubridge_send(f'link set "{anchor}" up')
await self._ubridge_send(f'bridge add_nio_ethernet {bridge_name} "{anchor}"')
await self._ubridge_send(
"bridge add_nio_udp {bridge_name} {lport} {rhost} {rport}".format(
bridge_name=bridge_name, lport=nio.lport, rhost=nio.rhost, rport=nio.rport
)
)
if nio.capturing:
await self._ubridge_send(
'bridge start_capture {bridge_name} "{pcap_file}"'.format(
bridge_name=bridge_name, pcap_file=nio.pcap_output_file
)
)
await self._ubridge_send(f"bridge start {bridge_name}")
await self._ubridge_apply_filters(bridge_name, nio.filters)
await self._ubridge_apply_markers(bridge_name, nio)
async def _relay_detach(self, bridge_name):
"""
Drop a relay bridge: its AF_PACKET endpoint on the anchor goes with
it, which is what makes switching an adapter from the relay datapath
to a kernel link safe (otherwise the anchor would both be a bridge
port and a relay endpoint, duplicating one direction).
"""
if self.ubridge:
with contextlib.suppress(UbridgeError):
await self._ubridge_send(f"bridge delete {bridge_name}")
async def _kernel_update(self, anchor, nio):
"""
Re-apply everything a kernel link carries on an already-attached
anchor (filters and markers changed): no re-enslaving, no bridge
create.
"""
await self._ubridge_apply_markers(anchor, nio)
# The netem apply resets the interface first (the kernel merges
# optional netem attrs on replace), so everything anchored on clsact
# must re-apply after it: bpf drops flush + re-add, eBPF modes re-set.
await self._ubridge_apply_netem(anchor, nio.filters)
await self._ubridge_apply_bpf_drops(anchor, nio.filters)
await self._ubridge_apply_ebpf_drops(anchor, nio.filters)
async def _remove_kernel_nio(self, nio, adapter_number, port_number=0):
"""
Detach an anchor from its per-link kernel bridge. Both endpoints run