NgTech-LTD/gns3-server
2
0
Fork 0
mirror of https://github.com/GNS3/gns3-server.git synced 2026-10-18 00:50:09 +03:00
Code Issues Packages Projects Releases Wiki Activity

test: drop the cross-compute VXLAN e2e scenario

Browse Source 
The scenario needed root for its network namespace and its fabric subnet
turned out host-specific: policy routing (a proxy's table consulted
before main) sends any non-RFC1918 destination out the default route, so
the peer compute was unreachable behind the veth — unfixable by picking
another subnet, and the failure paths leaked namespaces and processes.
Too environment-coupled to keep in the suite.

Unit coverage is unchanged (selection matrix, tunnel spec derivation,
VNI retry/degrade/self-heal, device lifecycle, capability probe); the
feature doc's Verification section now carries the manual two-compute
checklist instead of the e2e reference.
This commit is contained in:
YueGuobin 2026-10-11 22:56:42 +08:00
parent 21b866aeed
commit ca31031fdc
No known key found for this signature in database
4 changed files with 17 additions and 566 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/features/vxlan-cross-compute.md
View File

@ -112,7 +112,7 @@ The `vxlan show` step is the idempotency contract: a device found under the link
## Verification
* Unit: `tests/controller/test_kernel_datapath_link.py` (selection matrix, tunnel spec derivation, VNI retry, downgrade), `tests/utils/test_vxlan.py`, `tests/compute/docker/test_docker_kernel_datapath.py` (device lifecycle, leftover replacement, teardown ordering), `tests/api/routes/compute/test_capabilities.py`.
* Live: `tests/e2e/test_vxlan_cross_compute.py` — two computes on one machine (a network namespace joined to the host by a veth pair as the second "machine"), real containers per compute: VNI/port agreement, bridge membership, the derived MTU and its hard egress drop, dual-stack traffic, a delay filter, leftover replacement, a VNI squatted at node start (both ends re-wired on a fresh VNI, the start succeeding), residue-free delete and a rebuild through the project-open batch path.
* Live: no automated scenario — a second network stack on one machine means a network namespace, which needs root and host-specific routing and proved too environment-coupled for the suite. Validate on a real two-compute deployment (two machines, or a namespace-isolated second server): both ends agree on the VNI and dstport, each per-link bridge enslaves exactly `[anchor, tunnel]`, the derived MTU crosses while one past it is dropped at the bridge egress, traffic and filters behave as on the same-host datapath, and link delete plus project close/reopen leave no devices behind.
## Roadmap

3
tests/e2e/README.md
View File

@ -27,7 +27,6 @@ Beyond a Linux kernel with the datapath features (bridges, veth/TAP devices, tc
* **`tc` (iproute2)** for the filter scenarios: their qdisc and classifier assertions read it from `PATH`. Without it those assertions cannot run.
* **A Docker daemon** for the Docker scenarios (the harness's own guest driver talks to the same daemon as the server; socket override: `GNS3_E2E_DOCKER_SOCKET`).
* **A C compiler that can link statically** (`cc -static`) for the raw-frame probe of the link-local scenario; without it that probe skips.
* **Root (or passwordless sudo) for the cross-compute scenario** — a second network stack on one machine is a network namespace, and Linux confines creating and entering one to CAP_SYS_ADMIN (uBridge's file capabilities do not help there). The scenario skips with the exact run command otherwise; it also writes the second compute's node files, which the harness's passwordless-sudo path cannot do as the test user.
* **Emulator images** — see the matrix below.
## Test images
@ -94,5 +93,3 @@ Every kernel scenario also asserts the L2-anchor spec's server-side guarantees (
* `GNS3_E2E_DOCKER_SOCKET` — the Docker daemon socket the harness's guest driver talks to (default `/var/run/docker.sock` — the same daemon the server uses).
* `GNS3_E2E_IDLEPC` — skip idle-PC detection for the Dynamips scenarios and use the given value.
* `GNS3_E2E_DEBUG=1` — run the isolated instance with debug logging (the uBridge command stream lands in its `server.log`).
* `test_vxlan_cross_compute.py` — the cross-compute VXLAN datapath, with two computes on one machine: the isolated instance on the host and a second isolated instance inside a network namespace joined to the host by a veth pair (`harness.cross_compute_fabric` — two computes cannot tunnel from the host namespace, since the kernel keeps one VXLAN socket per (namespace, dstport) and the pair must share one VNI on it). One real Alpine container per compute, linked across the tunnel: both ends agree on the VNI and the shared dstport, each per-link bridge enslaves exactly `[anchor, tunnel]`, the tunnel device's MTU is the underlay minus 50 with the bridges inheriting it (a frame at it crosses, one past it is dropped at the bridge egress), dual-stack ICMP crosses, a `delay 100` filter shapes both directions, a stale device squatting on the link's deterministic name is replaced on node restart, a device squatting the link's VNI at node start self-heals (both ends re-wired on a fresh VNI, the start succeeding), link delete leaves nothing behind on either machine, and a project close/reopen rebuilds the tunnel through the batch path. **Needs root** (see host requirements) — run it with `sudo env GNS3_E2E_IMAGES=$HOME/GNS3/images pytest -m e2e tests/e2e/test_vxlan_cross_compute.py -s`.

273
tests/e2e/harness.py
View File

@ -67,7 +67,6 @@ import time
import urllib.error
import urllib.parse
import urllib.request
import uuid
import pytest
from websockets.sync.client import connect as ws_connect
@ -146,15 +145,14 @@ class Compute:
{"compute_id": "local", "name": name, "node_type": "dynamips", "properties": properties},
)
def create_docker_node(self, project_id, name, image, adapters=1, compute_id="local"):
def create_docker_node(self, project_id, name, image, adapters=1):
"""A standard Docker node (the veth datapath): the scenario addresses
its eth0 from the host and pings through it. *compute_id* places it
on a remote compute (the cross-compute VXLAN scenario's peer)."""
its eth0 from the host and pings through it."""
return self.call(
"POST",
f"/projects/{project_id}/nodes",
{
"compute_id": compute_id,
"compute_id": "local",
"name": name,
"node_type": "docker",
"properties": {"image": image, "adapters": adapters, "console_type": "telnet"},
@ -297,11 +295,10 @@ default_admin_username = admin
default_admin_password = admin
[Server]
host = {host}
host = 127.0.0.1
port = {port}
ubridge_path = {ubridge}
enable_kernel_datapath = {kernel}
enable_vxlan_cross_compute = {vxlan}
report_errors = False
projects_path = {tmp}/projects
configs_path = {tmp}/configs
@ -333,14 +330,10 @@ class Server:
instance (own config dir — the controller DB and every path follow it —
under a temporary directory removed on stop)."""
def __init__(self, compute, process=None, tmpdir=None, privilege=()):
def __init__(self, compute, process=None, tmpdir=None):
self.compute = compute
self.process = process
self.tmpdir = tmpdir
# Command prefix for cleaning up the temp directory when the process
# ran privileged (a compute spawned inside a netns as root writes
# root-owned files the test user cannot remove itself).
self.privilege = privilege
@property
def isolated(self):
@ -368,11 +361,6 @@ class Server:
self._stop_process()
if self.tmpdir:
shutil.rmtree(self.tmpdir, ignore_errors=True)
if self.privilege and os.path.exists(self.tmpdir):
# S603: fixed argv (the privilege prefix is sudo -n or empty)
subprocess.run( # noqa: S603
[*self.privilege, "rm", "-rf", self.tmpdir], capture_output=True, check=False
)
def restart(self, kernel=True):
"""Restart an isolated instance in place: stop the process — the
@ -388,15 +376,10 @@ class Server:
return self
def _spawn_isolated(tmpdir, kernel, host="127.0.0.1", vxlan=False, netns=None, privilege=()):
def _spawn_isolated(tmpdir, kernel):
"""Start one isolated instance inside *tmpdir* (its config is written
there; a fresh free port each boot) and return ``(process, compute)``,
logged in. Shared by the first boot and by ``Server.restart``.
With *netns* the instance is spawned inside that network namespace
(through *privilege*, e.g. ``sudo -n``) and binds *host* — the pattern
the cross-compute VXLAN scenario uses for its second "machine".
"""
logged in. Shared by the first boot and by ``Server.restart``."""
ubridge = shutil.which("ubridge")
dynamips = shutil.which("dynamips")
if not ubridge:
@ -419,15 +402,7 @@ def _spawn_isolated(tmpdir, kernel, host="127.0.0.1", vxlan=False, netns=None, p
fork_build = os.path.expanduser("~/myCode/GNS3/vpcs/src/vpcs")
vpcs = fork_build if os.path.isfile(fork_build) else (shutil.which("vpcs") or "vpcs")
config = ISOLATED_CONFIG.format(
port=port,
ubridge=ubridge,
dynamips=dynamips,
images=images,
tmp=tmpdir,
kernel=kernel,
vpcs=vpcs,
vxlan=vxlan,
host=host,
port=port, ubridge=ubridge, dynamips=dynamips, images=images, tmp=tmpdir, kernel=kernel, vpcs=vpcs
)
config_path = os.path.join(tmpdir, "gns3_server.conf")
with open(config_path, "w") as f:
@ -435,15 +410,7 @@ def _spawn_isolated(tmpdir, kernel, host="127.0.0.1", vxlan=False, netns=None, p
log = open(os.path.join(tmpdir, "server.log"), "a")
# S603: fixed argv, this interpreter, no shell involvement
argv = [
*privilege,
*(["ip", "netns", "exec", netns] if netns else []),
sys.executable,
"-m",
"gns3server",
"--config",
config_path,
]
argv = [sys.executable, "-m", "gns3server", "--config", config_path]
if os.environ.get("GNS3_E2E_DEBUG"):
argv.append("-d") # debug logging — the uBridge command stream lands in server.log
process = subprocess.Popen( # noqa: S603
@ -453,13 +420,13 @@ def _spawn_isolated(tmpdir, kernel, host="127.0.0.1", vxlan=False, netns=None, p
stderr=subprocess.STDOUT,
)
process.log = log # keep the file object alive with the process handle
compute = Compute(f"http://{host}:{port}/v3")
compute = Compute(f"http://127.0.0.1:{port}/v3")
deadline = time.time() + 60
while time.time() < deadline:
try:
# S310: the address is ours (loopback or the scenario's veth fabric)
with urllib.request.urlopen(f"http://{host}:{port}/v3/version", timeout=2):
# S310: the port is ours, the host loopback
with urllib.request.urlopen(f"http://127.0.0.1:{port}/v3/version", timeout=2):
break
except Exception:
if process.poll() is not None:
@ -479,13 +446,13 @@ def _spawn_isolated(tmpdir, kernel, host="127.0.0.1", vxlan=False, netns=None, p
return process, compute
def _start_isolated(kernel, vxlan=False):
def _start_isolated(kernel):
tmpdir = tempfile.mkdtemp(prefix="gns3-e2e-")
process, compute = _spawn_isolated(tmpdir, kernel, vxlan=vxlan)
process, compute = _spawn_isolated(tmpdir, kernel)
return Server(compute, process, tmpdir)
def live_server(kernel=True, vxlan=False):
def live_server(kernel=True):
"""The server under test.
With ``GNS3_E2E_URL`` set, that server is used for kernel mode (it must
@ -508,7 +475,7 @@ def live_server(kernel=True, vxlan=False):
).login()
server = Server(compute)
else:
server = _start_isolated(kernel, vxlan=vxlan)
server = _start_isolated(kernel)
if not wait_until(lambda: (server.compute.capabilities() or {}).get("version"), timeout=45):
raise AssertionError(
f"{server.compute.url}: the controller's local compute did not report capabilities within 45s"
@ -1538,211 +1505,3 @@ def pcap_icmp_rtt(path, ethertype):
deltas.append((ts - pending) * 1000)
pending = None
return sum(deltas) / len(deltas) if deltas else None
# ---------------------------------------------------------------------------
# Cross-compute fabric: a second "machine" for VXLAN links
# ---------------------------------------------------------------------------
# The /24 both ends of the veth "fabric" are addressed in. RFC 2544's
# benchmarking range: never routed on a real network, so it cannot collide
# with anything the host legitimately talks to.
FABRIC_SUBNET_PREFIX = "198.18.7"
def fabric_privilege():
"""The command prefix for network-namespace operations: nothing when the
suite runs as root, ``sudo -n`` when passwordless sudo is configured,
None otherwise (the caller skips with the explanation).
A second network stack — the thing a cross-compute VXLAN link needs on
a single machine — is a network namespace, and Linux confines both
creating one and entering one to CAP_SYS_ADMIN of its owning user
namespace. uBridge's file capabilities do not help here (they are caps
for netlink *operations*, not for joining a namespace), so either the
whole run is root or sudo carries the two calls.
"""
if os.geteuid() == 0:
return []
sudo = shutil.which("sudo")
# S603: resolved binary, fixed argv, no shell
if sudo and subprocess.run([sudo, "-n", "true"], capture_output=True, check=False).returncode == 0: # noqa: S603
return [sudo, "-n"]
return None
class CrossComputeFabric:
"""A second "machine" on one host: a network namespace joined to the
host by a veth pair, both ends addressed inside ``198.18.7.0/24``.
Two computes on one machine cannot tunnel to each other from the host
namespace: the kernel keeps one VXLAN socket per (namespace, dstport)
and both ends must register the same VNI on it — the pair collides by
design. Distinct network namespaces give the pair its two sockets, and
the veth pair is the L2 fabric the tunnels ride. uBridge's file
capabilities make the *devices* work inside the namespace; only the
namespace itself needs privilege (see ``fabric_privilege``).
"""
def __init__(self, netns, host_ifc, peer_ifc, host_ip, peer_ip, prefix):
self.netns = netns
self.host_ifc = host_ifc
self.peer_ifc = peer_ifc
self.host_ip = host_ip
self.peer_ip = peer_ip
self.prefix = prefix
def netns_run(self, *argv, **kwargs):
"""Run a command inside the namespace (privileged, like every netns
access)."""
kwargs.setdefault("capture_output", True)
kwargs.setdefault("text", True)
# S603: fixed argv (the caller builds the netns-side command)
return subprocess.run( # noqa: S603
[*self.prefix, "ip", "netns", "exec", self.netns, *argv], check=False, **kwargs
)
def iface_mtu(self, name):
"""The MTU of an interface inside the namespace (None when absent)."""
out = self.netns_run("ip", "-o", "link", "show", name).stdout
match = re.search(r"\bmtu (\d+)", out)
return int(match.group(1)) if match else None
def iface_exists(self, name):
return self.netns_run("ip", "link", "show", name).returncode == 0
def vxlan_show(self, name):
"""``ip -d link show`` details of an interface inside the namespace,
as one string (None when absent) — carries the ``vxlan id <vni>
... dstport <port>`` attributes."""
result = self.netns_run("ip", "-d", "link", "show", name)
return result.stdout if result.returncode == 0 and name in result.stdout else None
def bridge_members(self, bridge):
"""Interfaces enslaved to a bridge inside the namespace, or None
when the bridge does not exist."""
result = self.netns_run("ls", f"/sys/class/net/{bridge}/brif")
if result.returncode != 0:
return None
return sorted(result.stdout.split())
def brport_state(self, name):
"""An interface's bridge-port STP state inside the namespace (None
when it is not enslaved; the kernel reports 3 for forwarding)."""
result = self.netns_run("cat", f"/sys/class/net/{name}/brport/state")
if result.returncode != 0:
return None
return result.stdout.strip() or None
def teardown(self):
"""Delete the namespace (all its devices go with it) and any host
end a half-created pair left behind."""
# S603: fixed argv, resolved ip binary, no shell
subprocess.run( # noqa: S603
[*self.prefix, "ip", "netns", "del", self.netns], capture_output=True, check=False
)
subprocess.run( # noqa: S603
[*self.prefix, "ip", "link", "del", self.host_ifc], capture_output=True, check=False
)
def cross_compute_fabric():
"""Create the fabric, or skip the calling test when this host cannot.
The names carry a random suffix so concurrent runs (and leftovers of a
crashed one) cannot collide; the caller must call ``teardown()`` in a
finally."""
prefix = fabric_privilege()
if prefix is None:
pytest.skip(
"cross-compute VXLAN needs a second network namespace: run the suite as root, e.g. "
"`sudo env GNS3_E2E_IMAGES=$HOME/GNS3/images pytest -m e2e "
"tests/e2e/test_vxlan_cross_compute.py -s`, or configure passwordless sudo"
)
suffix = uuid.uuid4().hex[:6]
fabric = CrossComputeFabric(
netns=f"gns3vx{suffix}",
host_ifc=f"vxh{suffix}",
peer_ifc=f"vxp{suffix}",
host_ip=f"{FABRIC_SUBNET_PREFIX}.1",
peer_ip=f"{FABRIC_SUBNET_PREFIX}.2",
prefix=prefix,
)
try:
# S603: fixed argv, resolved ip binary, no shell
subprocess.run( # noqa: S603
[*prefix, "ip", "netns", "add", fabric.netns], check=True, capture_output=True
)
subprocess.run( # noqa: S603
[*prefix, "ip", "link", "add", fabric.host_ifc, "type", "veth", "peer", "name", fabric.peer_ifc],
check=True,
capture_output=True,
)
subprocess.run( # noqa: S603
[*prefix, "ip", "link", "set", fabric.peer_ifc, "netns", fabric.netns], check=True, capture_output=True
)
subprocess.run( # noqa: S603
[*prefix, "ip", "addr", "add", f"{fabric.host_ip}/24", "dev", fabric.host_ifc],
check=True,
capture_output=True,
)
subprocess.run( # noqa: S603
[*prefix, "ip", "link", "set", fabric.host_ifc, "up"], check=True, capture_output=True
)
for argv in (
("ip", "addr", "add", f"{fabric.peer_ip}/24", "dev", fabric.peer_ifc),
("ip", "link", "set", fabric.peer_ifc, "up"),
("ip", "link", "set", "lo", "up"),
):
fabric.netns_run(*argv, check=True)
except subprocess.CalledProcessError:
fabric.teardown()
raise
return fabric
def start_isolated_peer(fabric, kernel=True):
"""An isolated instance on the other side of the fabric: spawned inside
the namespace and bound to its address, so the first server can register
it as a remote compute whose network stack is the second "machine"."""
tmpdir = tempfile.mkdtemp(prefix="gns3-e2e-peer-")
process, compute = _spawn_isolated(tmpdir, kernel, host=fabric.peer_ip, netns=fabric.netns, privilege=fabric.prefix)
return Server(compute, process, tmpdir, privilege=fabric.prefix)
def register_remote_compute(server, peer, name="vxlan-peer"):
"""Register the peer's server on the first server's controller, so links
between nodes on the two computes are cross-compute links. Waits until
the controller connected it (capabilities reported — the vxlan probe in
particular: the controller gates tunnel eligibility on it)."""
compute = server.compute.call(
"POST",
"/computes",
{
"name": name,
"protocol": "http",
"host": peer.compute.url.split("://", 1)[1].split(":", 1)[0],
"port": int(peer.compute.url.rsplit(":", 1)[1].split("/", 1)[0]),
"user": peer.compute.username,
"password": peer.compute.password,
},
)
compute_id = compute["compute_id"]
if not wait_until(lambda: (server.compute.call("GET", f"/computes/{compute_id}") or {}).get("version"), timeout=60):
raise AssertionError(f"controller did not connect the remote compute {compute_id} within 60s")
if not wait_until(
lambda: (
((server.compute.call("GET", f"/computes/{compute_id}") or {}).get("capabilities") or {}).get(
"ubridge_vxlan"
)
is True
),
timeout=60,
):
raise AssertionError(f"remote compute {compute_id} never reported ubridge_vxlan (its netns probe failed)")
return compute_id

305
tests/e2e/test_vxlan_cross_compute.py
View File

@ -1,305 +0,0 @@
#
# Copyright (C) 2026 GNS3 Technologies Inc.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""
Live end-to-end test for cross-compute VXLAN links (pytest marker ``e2e``):
two computes on one machine — the harness's isolated instance on the host and
a second isolated instance inside a network namespace joined to the host by a
veth pair (``harness.cross_compute_fabric``) — with one real Alpine container
per compute, linked across the tunnel.
Two computes on one machine cannot tunnel from the host namespace: the kernel
keeps one VXLAN socket per (namespace, dstport), and both ends of a link must
register the same VNI on it, so the pair collides by design (the controller
falls back to the relay for same-host pairs). Separate network namespaces
give the pair its two sockets, and the veth pair is the L2 fabric the tunnels
ride — the closest a single machine gets to two hosts. The same-host
fallback itself is asserted by the controller unit tests; this scenario is
the real-kernel proof of the tunnel.
Needs root: a network namespace is privileged to create and to enter, and the
second compute's node files (its project directories) are written by the
test. Run it as::
sudo env GNS3_E2E_IMAGES=$HOME/GNS3/images pytest -m e2e \\
tests/e2e/test_vxlan_cross_compute.py -s
What it asserts:
* both ends report the same VNI on the shared dstport, each pointing at the
other machine's address, and each per-link bridge enslaves exactly
``[anchor, tunnel]``;
* the tunnel device's MTU is the underlay MTU minus 50 (the kernel derives
it from the lower device the peer route resolves to), the bridges inherit
it, and a frame past it is dropped at the bridge egress while one at it
crosses — the reason the datapath is opt-in on non-jumbo fabrics;
* real dual-stack ICMP crosses, a ``delay 100`` filter shapes it (one end
per direction), link delete leaves no device or bridge on either machine;
* a stale tunnel device squatting on the link's deterministic name is
replaced by the attach path, and the tunnel recovers;
* a device squatting the link's VNI at node start self-heals: the start
re-wires the link on a fresh VNI — the peer re-wired too — and succeeds.
"""
import os
import re
import subprocess
import time
import pytest
from gns3server.utils.vxlan import VXLAN_VNI_MAX, VXLAN_VNI_MIN, vxlan_device_name
from tests.e2e import harness
pytestmark = pytest.mark.e2e
# every standard Docker adapter has a single port
ETH = (0, 0)
G1_IP = "10.66.1.1"
G2_IP = "10.66.1.2"
# the dual-stack half: a ULA mirroring the v4 addressing
G1_IP6 = "fd00:66:1::1"
G2_IP6 = "fd00:66:1::2"
# ICMP payloads against the derived tunnel MTU (underlay 1500 -> tunnel
# 1450): 1400 + 42 bytes of headers is a 1442-byte frame, under the bridge
# egress gate; 1500 + 42 is a 1542-byte frame, over it — and over the
# guest's own 1500 MTU too, so the IP stack may fragment it; either way the
# pieces cannot cross the 1450-wide egress and the ping must fail silently.
FIT_PAYLOAD = 1400
TOO_BIG_PAYLOAD = 1500
def _ip(*argv):
"""Run an iproute2 command on the host (the scenario runs as root)."""
ip = harness.IP
assert ip, "ip binary not found"
# S603: resolved binary, fixed argv from the test itself
return subprocess.run([ip, *argv], capture_output=True, text=True, check=True) # noqa: S603
def _host_vxlan_attrs(name):
"""(vni, dstport, remote) of a host-side tunnel device, from
``ip -d link show``."""
details = _ip("-d", "link", "show", name).stdout
vni = re.search(r"\bvxlan id (\d+)", details)
dstport = re.search(r"\bdstport (\d+)", details)
remote = re.search(r"\bremote (\S+)", details)
assert vni and dstport and remote, details
return int(vni.group(1)), int(dstport.group(1)), remote.group(1)
def _peer_vxlan_attrs(fabric, name):
details = fabric.vxlan_show(name)
assert details, f"tunnel device {name} missing inside {fabric.netns}"
vni = re.search(r"\bvxlan id (\d+)", details)
dstport = re.search(r"\bdstport (\d+)", details)
remote = re.search(r"\bremote (\S+)", details)
assert vni and dstport and remote, details
return int(vni.group(1)), int(dstport.group(1)), remote.group(1)
def _container_id(daemon, node):
container_id = daemon.container_id(f"GNS3.{node['name']}.{node['project_id']}")
assert container_id, f"container for {node['name']} not found on the Docker daemon"
node["container_id"] = container_id
return container_id
def _start_pair(compute, pid, daemon, n1, n2):
"""Start both containers and wait for their userspace to be reachable:
the v4 address from the interfaces file (init.sh), the v6 twin from the
harness (the daemon's disable_ipv6 on bridge networks makes the
interfaces-file route fail). The container ids are re-resolved after the
starts: a project reopen can replace a stale container with a fresh one
under the same deterministic name (a new id)."""
for node in (n1, n2):
compute.call("POST", f"/projects/{pid}/nodes/{node['node_id']}/start")
_container_id(daemon, node)
for node, address in ((n1, G1_IP), (n2, G2_IP)):
harness.docker_wait_address(daemon, node["container_id"], address)
for node, address6 in ((n1, G1_IP6), (n2, G2_IP6)):
harness.docker_bring_up_ipv6(daemon, node["container_id"], address6)
def test_vxlan_cross_compute_link():
if os.geteuid() != 0:
pytest.skip(
"cross-compute VXLAN needs a second network stack (a netns) and writes the second "
"compute's node files: run as root, e.g. `sudo env GNS3_E2E_IMAGES=$HOME/GNS3/images "
"pytest -m e2e tests/e2e/test_vxlan_cross_compute.py -s`"
)
fabric = harness.cross_compute_fabric()
server = harness.live_server(kernel=True, vxlan=True)
peer = harness.start_isolated_peer(fabric)
compute = server.compute
daemon = harness.DockerDaemon()
failed = True
pid = None
try:
peer_id = harness.register_remote_compute(server, peer)
image = harness.ensure_docker_image(server)
project = compute.create_project("vxlan-e2e")
pid = project["project_id"]
print(".. creating one container per compute")
n1 = compute.create_docker_node(pid, "VXE1", image)
harness.configure_docker_interfaces(n1, G1_IP)
_container_id(daemon, n1)
n2 = compute.create_docker_node(pid, "VXE2", image, compute_id=peer_id)
harness.configure_docker_interfaces(n2, G2_IP)
_container_id(daemon, n2)
assert n1["compute_id"] == "local", n1["compute_id"]
assert n2["compute_id"] == peer_id, n2["compute_id"]
print(".. linking them across the computes")
link = compute.create_link(pid, (n1["node_id"], *ETH), (n2["node_id"], *ETH))
lid = link["link_id"]
assert link["kernel_datapath"] is True, link
bridge = harness.link_bridge_name(lid)
a1 = harness.docker_anchor_name(n1["node_id"], *ETH)
a2 = harness.docker_anchor_name(n2["node_id"], *ETH)
vx1 = vxlan_device_name(lid, n1["node_id"])
vx2 = vxlan_device_name(lid, n2["node_id"])
assert not harness.tap_exists(vx1), "the tunnel lands with node start"
print(".. starting the containers (the wiring lands with node start)")
_start_pair(compute, pid, daemon, n1, n2)
print(".. asserting the tunnel on both machines")
assert harness.tap_exists(vx1), f"tunnel device {vx1} missing on the host"
assert harness.bridge_members(bridge) == sorted([a1, vx1]), harness.bridge_members(bridge)
assert fabric.bridge_members(bridge) == sorted([a2, vx2]), fabric.bridge_members(bridge)
vni1, port1, remote1 = _host_vxlan_attrs(vx1)
vni2, port2, remote2 = _peer_vxlan_attrs(fabric, vx2)
assert vni1 == vni2, (vni1, vni2)
assert VXLAN_VNI_MIN <= vni1 <= VXLAN_VNI_MAX, vni1
assert port1 == port2, (port1, port2)
assert remote1 == fabric.peer_ip, (remote1, fabric.peer_ip)
assert remote2 == fabric.host_ip, (remote2, fabric.host_ip)
# MTU: the kernel derives the tunnel device's MTU from the lower
# device the peer route resolves to (underlay - 50 for the outer
# headers), and the per-link bridge — no MTU of its own — reports
# the minimum of its ports, i.e. the honest "what this fabric can
# carry".
underlay_mtu = int(harness.sysfs(f"/sys/class/net/{fabric.host_ifc}/mtu"))
assert underlay_mtu == 1500, underlay_mtu # the veth default; the derived MTU follows from it
assert int(harness.sysfs(f"/sys/class/net/{vx1}/mtu")) == underlay_mtu - 50
assert int(harness.sysfs(f"/sys/class/net/{bridge}/mtu")) == underlay_mtu - 50
assert fabric.iface_mtu(vx2) == underlay_mtu - 50
assert fabric.iface_mtu(bridge) == underlay_mtu - 50
assert fabric.iface_mtu(fabric.peer_ifc) == underlay_mtu
# The tunnel device is an L2 anchor like any other: hardened
# (lenient — the shared with plain `ip` in the doc's example) and
# FORWARDING on both bridges.
if harness.l2only_supported():
harness.assert_pure_l2(vx1)
harness.assert_forwarding(vx1)
assert fabric.brport_state(vx2) == "3", fabric.brport_state(vx2)
print(".. pinging across the tunnel")
ping = harness.docker_wait_ping(daemon, n1["container_id"], G2_IP)
assert ping["loss"] == 0, ping["raw"]
assert ping["avg"] is not None and ping["avg"] < 50, ping
ping6 = harness.docker_wait_ping(daemon, n1["container_id"], G2_IP6)
assert ping6["loss"] == 0, ping6["raw"]
# The derived MTU is the hard limit: a frame at it crosses, one past
# it is dropped at the bridge egress — silently, which is why the
# deployment has to size the underlay for the frames its guests send.
fit = harness.docker_ping(daemon, n1["container_id"], G2_IP, size=FIT_PAYLOAD)
assert fit["loss"] == 0, fit["raw"]
too_big = harness.docker_ping(daemon, n1["container_id"], G2_IP, count=2, size=TOO_BIG_PAYLOAD)
assert too_big["loss"] > 0 or too_big["exit"] != 0, too_big["raw"]
print(".. delay 100 shapes the tunnel on both ends (one direction each)")
compute.call("PUT", f"/projects/{pid}/links/{lid}", {"filters": {"delay": [100]}})
time.sleep(1)
delayed = harness.docker_ping(daemon, n1["container_id"], G2_IP, count=5, interval=None, timeout=2)
assert delayed["loss"] == 0, delayed["raw"]
assert 150 <= (delayed["avg"] or 0) <= 450, delayed
compute.call("PUT", f"/projects/{pid}/links/{lid}", {"filters": {}})
print(".. a stale device on the link's name is replaced on node restart")
compute.call("POST", f"/projects/{pid}/nodes/{n1['node_id']}/stop")
_ip("link", "del", vx1)
# A stale tunnel: same deterministic name, some other VNI/port —
# what a crashed run or a hand-made device would leave behind.
_ip("link", "add", vx1, "type", "vxlan", "id", "4242", "dstport", "47999", "remote", "127.0.0.1")
compute.call("POST", f"/projects/{pid}/nodes/{n1['node_id']}/start")
harness.docker_wait_address(daemon, n1["container_id"], G1_IP)
harness.docker_bring_up_ipv6(daemon, n1["container_id"], G1_IP6)
stale_vni, stale_port, stale_remote = _host_vxlan_attrs(vx1)
assert (stale_vni, stale_port, stale_remote) == (vni1, port1, remote1), (stale_vni, stale_port, stale_remote)
assert harness.bridge_members(bridge) == sorted([a1, vx1]), harness.bridge_members(bridge)
assert int(harness.sysfs(f"/sys/class/net/{bridge}/mtu")) == underlay_mtu - 50
assert harness.docker_wait_ping(daemon, n1["container_id"], G2_IP, attempts=5)["loss"] == 0
print(".. a VNI squatted at node start self-heals on a fresh VNI")
compute.call("POST", f"/projects/{pid}/nodes/{n1['node_id']}/stop")
# A device holding the link's exact VNI on the shared dstport socket
# — what a colliding link (or an operator) would have registered.
# The node's start replays the stored NIO, the kernel refuses the
# VNI, and the controller must re-wire the link on a fresh one and
# retry the start instead of failing it.
_ip("link", "add", "gvxsquat", "type", "vxlan", "id", str(vni1), "dstport", str(port1), "remote", "127.0.0.1")
try:
compute.call("POST", f"/projects/{pid}/nodes/{n1['node_id']}/start")
harness.docker_wait_address(daemon, n1["container_id"], G1_IP)
harness.docker_bring_up_ipv6(daemon, n1["container_id"], G1_IP6)
healed_vni, healed_port, healed_remote = _host_vxlan_attrs(vx1)
assert healed_vni != vni1, (healed_vni, vni1)
assert VXLAN_VNI_MIN <= healed_vni <= VXLAN_VNI_MAX, healed_vni
assert healed_port == port1, (healed_port, port1)
assert healed_remote == remote1, (healed_remote, remote1)
# the peer was re-wired to the fresh VNI too — the pair must agree
peer_vni, _, _ = _peer_vxlan_attrs(fabric, vx2)
assert peer_vni == healed_vni, (peer_vni, healed_vni)
assert harness.docker_wait_ping(daemon, n1["container_id"], G2_IP, attempts=5)["loss"] == 0
finally:
_ip("link", "del", "gvxsquat")
print(".. deleting the link leaves no tunnel or bridge behind on either machine")
compute.call("DELETE", f"/projects/{pid}/links/{lid}")
assert not harness.tap_exists(vx1)
assert not harness.tap_exists(bridge)
assert not fabric.iface_exists(vx2), f"tunnel device {vx2} survived on the peer"
assert fabric.bridge_members(bridge) is None
print(".. closing and reopening the project rebuilds the tunnel through the batch path")
compute.call("POST", f"/projects/{pid}/close")
compute.call("POST", f"/projects/{pid}/open")
_start_pair(compute, pid, daemon, n1, n2)
assert harness.tap_exists(vx1), f"tunnel device {vx1} not rebuilt on reopen"
assert fabric.iface_exists(vx2), f"tunnel device {vx2} not rebuilt on reopen"
assert harness.docker_wait_ping(daemon, n1["container_id"], G2_IP, attempts=5)["loss"] == 0
failed = False
finally:
keep = failed and harness.keep_on_failure()
if not keep:
peer.stop()
if pid is not None:
harness.release(server, pid, failed=failed)
else:
server.stop()
if keep:
print(f"\n!! kept the fabric {fabric.netns} (and its peer compute) for inspection")
else:
fabric.teardown()