feat: cross-compute VXLAN kernel datapath

Cross-compute links can ride kernel VXLAN tunnels instead of the uBridge
UDP relay: each endpoint creates one VXLAN device (uBridge's rtnetlink
vxlan module) enslaved into the same per-link Linux bridge the same-host
datapath uses, so filters, markers, capture and suspend keep running on
the anchor unchanged. Opt-in (Server.enable_vxlan_cross_compute, default
off) because the tunnel caps the usable MTU at the underlay minus the
outer headers; the relay stays the default and the fallback.

The controller derives what the pair must agree on: one VNI per link
(upper VNI space, per-link random salt) and deterministic device names
(gvl{link}{node}) that make crashed-run leftovers claimable
(show -> delete -> create). Same-host compute pairs, switch endpoints
and serial/unix-socket containers stay on the relay.

A VNI the kernel refuses self-heals on every path it can surface:
interactive create retries a fresh salt then degrades to the relay;
project open re-dispatches refused batch entries the same way; node
start recognises the refused device, stops the half-started node,
re-wires both ends on a fresh VNI (or degrades the link) and retries
the start. The device MTU is derived from the lower device (dev=) and
the per-link bridge inherits it - no configuration. The capability
probe (ubridge_vxlan in /capabilities) runs one scratch create/delete
on a scratch port that never equals the configured dstport.

The e2e scenario runs two computes on one machine (a network namespace
joined by a veth pair as the fabric) and needs root.
This commit is contained in:
YueGuobin committed 2026-10-17 14:11:00 +08:00
1 parent 5514b2e929
commit 21b866aeed
24 files changed
+2107 -59

No files matched your search

+25
View File
@@ -73,6 +73,26 @@ class BridgeNIOType(str, Enum):
bridge = "nio_bridge"
class BridgeVxlan(BaseModel):
"""
VXLAN tunnel spec carried by a cross-compute BridgeNIO. Each endpoint
creates one VXLAN device (uBridge ``vxlan create``) and enslaves it into
the link's per-link kernel bridge next to the node's anchor; the kernel
encapsulates/decapsulates the frames. The controller derives the name
and VNI (utils.vxlan) and resolves the peer address, so both endpoints
agree on the VNI without ever talking to each other.
The device's MTU is derived by the kernel from the underlay interface
the route to ``remote`` picks (MTU - 50 for the outer headers), which is
what the link's bridge then reports as "what this fabric can carry".
"""
name: str = Field(..., max_length=15, description="VXLAN device name on this endpoint's host (IFNAMSIZ-1)")
vni: int = Field(..., ge=1, le=16777215, description="VXLAN network identifier, identical on both endpoints")
remote: str = Field(..., description="Peer compute's address (unicast tunnel endpoint)")
dstport: int = Field(..., ge=1, le=65535, description="UDP destination port, identical on both endpoints")
class BridgeNIO(BaseModel):
"""
Kernel-datapath bridge Network Input/Output properties. The NIO tells the
@@ -99,6 +119,11 @@ class BridgeNIO(BaseModel):
description="Kernel bridge name e.g. gns3a1b2c3d4e5; None when an Ethernet switch "
"owns the anchor's bridge membership (absorbed link)",
)
vxlan: Optional[BridgeVxlan] = Field(
None,
description="Cross-compute tunnel: the VXLAN device this end creates and enslaves into the bridge "
"next to the anchor; absent for same-host kernel links",
)
suspend: Optional[bool] = Field(None, description="Suspend the NIO")
filters: Optional[dict] = Field(
None,
+18
View File
@@ -192,6 +192,24 @@ class ServerSettings(BaseModel):
"or QEMU when the compute's uBridge supports persistent TAPs) through kernel interfaces instead "
"of the uBridge UDP relay",
)
enable_vxlan_cross_compute: bool = Field(
False,
description="Wire eligible cross-compute links through kernel VXLAN tunnels (one device per endpoint, "
"uBridge's vxlan module) instead of the uBridge UDP relay. The kernel encapsulates/decapsulates, so "
"no user-space forwarding is involved; the price is MTU: a tunnel can only carry the underlay MTU "
"minus 50 bytes of outer headers, so enable this where the fabric between computes carries jumbo "
"frames. The relay stays the default and the fallback when a tunnel cannot be wired",
)
vxlan_dstport: int = Field(
14789,
ge=1,
le=65535,
description="UDP destination port every cross-compute VXLAN tunnel listens on. The kernel shares one "
"socket per port across all VXLAN devices, so all links of a deployment ride this one port — chosen "
"off the IANA 4789 and the legacy 8472 (other VXLAN software's ports: sharing a socket would merge "
"their demux table with ours) and above the ephemeral range. Change it only when it collides with "
"a service already listening on the computes",
)
auto_discover_images: bool = Field(True, description="Automatically discover images in the images directory")
image_sync_interval: int = Field(900, ge=10, description="Seconds between automatic image inventory scans")
report_errors: bool = Field(True, description="Automatically send crash reports to the GNS3 team")
@@ -73,3 +73,8 @@ class Capabilities(BaseModel):
"keeps the guest from auto-configuring the host side of the anchor TAP); absent when the "
"probe failed",
)
ubridge_vxlan: Optional[bool] = Field(
None,
description="Whether uBridge can create VXLAN devices (the vxlan module, kernel support included) — "
"the tunnel a cross-compute kernel link rides; absent when the probe failed",
)