mirror of
https://github.com/GNS3/gns3-server.git
synced 2026-10-13 05:55:35 +03:00
Merge pull request #2979 from yueguobin/feat/vpcs-kernel-datapath
feat: VPCS kernel datapath
This commit is contained in:
commit
6d838c890d
@ -97,7 +97,7 @@ Everything in the root namespace is built the same way: gns3-server creates the
|
||||
|
||||
uBridge sits off the forwarding path — the dashed edges are control and observation, never data flow. The `qdisc install` edge is uBridge configuring the kernel over netlink; the kernel then runs those qdiscs in-flight on the host end's egress (they are stages of the transmit path, not packet consumers). The `frame copies` edge is the AF_PACKET tap: bound to the host end, the kernel clones every frame the interface sends or receives to uBridge's packet socket (the tcpdump mechanism) — read-only copies; the originals never leave the kernel.
|
||||
|
||||
Relay link on the same unified veth and the same single compute as above — what the link becomes when `enable_kernel_datapath` is off (cross-compute wiring and an endpoint that cannot anchor — VPCS, a cloud node, … — fall back too; across computes the UDP hop rides the real network instead of loopback, the copy count unchanged):
|
||||
Relay link on the same unified veth and the same single compute as above — what the link becomes when `enable_kernel_datapath` is off (cross-compute wiring and an endpoint that cannot anchor — a cloud node, … — fall back too; across computes the UDP hop rides the real network instead of loopback, the copy count unchanged):
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
|
||||
@ -238,4 +238,4 @@ enable_kernel_datapath = True
|
||||
|
||||
## Roadmap
|
||||
|
||||
The Ethernet switch has landed (see `ethernet-switch-kernel-datapath.md` — it absorbs the peer's anchor into its own kernel bridge, and switch-to-switch cascades with it), and so has the IOL runner container (`iol-docker-kernel-datapath.md`). Every router and switch node type now anchors; VPCS, the hub, the cloud and the NAT node stay on the relay by design. Cross-compute kernel links need VXLAN/GENEVE encapsulation — a separate, still-deferred project.
|
||||
The Ethernet switch has landed (see `ethernet-switch-kernel-datapath.md` — it absorbs the peer's anchor into its own kernel bridge, and switch-to-switch cascades with it), and so has the IOL runner container (`iol-docker-kernel-datapath.md`) and VPCS (`vpcs-kernel-datapath.md` — it attaches to a pre-created anchor with `-e -d`). Every router and switch node type now anchors, and so does VPCS; the hub, the cloud and the NAT node stay on the relay by design. Cross-compute kernel links need VXLAN/GENEVE encapsulation — a separate, still-deferred project.
|
||||
|
||||
@ -37,8 +37,9 @@ on the relay:
|
||||
* **switch ↔ switch** cascades ride the kernel too — through a veth pair
|
||||
joining the two bridges (one end per switch, each side applying its own
|
||||
port mode to its own end); see the wire format below;
|
||||
* anchor-less peers (VPCS, cloud, the Dynamips-hosted hub, generic
|
||||
`GNS3_UNIX_SOCKET_NIO` containers);
|
||||
* anchor-less peers (cloud, the Dynamips-hosted hub, generic
|
||||
`GNS3_UNIX_SOCKET_NIO` containers, a VPCS binary without `-n` — see
|
||||
`vpcs-kernel-datapath.md`);
|
||||
* cross-compute links, non-Ethernet ports: the common rules.
|
||||
|
||||
**Relay port TAPs.** The switch's own relay-path port TAPs (the
|
||||
|
||||
@ -274,4 +274,4 @@ The anchor lifecycle needs no configuration of its own — the capability probes
|
||||
|
||||
## Roadmap
|
||||
|
||||
Cross-compute kernel links need VXLAN/GENEVE encapsulation — deferred until every node type is kernelized; with the IOL runner container anchored, every node type in scope can now anchor (hub, cloud, NAT and VPCS stay on the relay by decision), so the encapsulation work is all that remains.
|
||||
Cross-compute kernel links need VXLAN/GENEVE encapsulation — deferred until every node type is kernelized; with the IOL runner container anchored, every node type in scope can now anchor (hub, cloud and NAT stay on the relay by decision; VPCS anchors since `vpcs-kernel-datapath.md`), so the encapsulation work is all that remains.
|
||||
|
||||
55
docs/features/vpcs-kernel-datapath.md
Normal file
55
docs/features/vpcs-kernel-datapath.md
Normal file
@ -0,0 +1,55 @@
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
> This documentation is AI-generated with reference to actual code and verified against real-kernel testing. AI can make mistakes — please verify against the source code when in doubt.
|
||||
|
||||
# VPCS kernel datapath
|
||||
|
||||
A VPCS node's single adapter anchors on a **persistent TAP** that the server creates and the vpcs process *attaches to* (`vpcs -e -d <anchor> -n`), the same server-creates/guest-attaches shape as QEMU's netdev. Links between a VPCS node and any other anchoring node type on the same compute are wired in the kernel (per-link Linux bridge) instead of the uBridge UDP relay; the compute side is the shared `KernelDatapathMixin`, which is anchor-agnostic.
|
||||
|
||||
## Architecture
|
||||
|
||||
The anchor follows the shared naming contract of `gns3server/utils/kernel_anchor.py`: `gp{node_id[:8]}e0p0` (14 chars, within the kernel's 16-byte `IFNAMSIZ`). uBridge creates it at node start (`tap create`, `tap set_owner <uid>`, `link set down`) before the vpcs process is launched; vpcs opens it by name with `-e -d` and speaks its user-space IP stack through the tap fd. The per-link bridge, filters, capture, markers and suspend all ride the same mixin paths as every other anchor-backed node type — including the Ethernet switch fast path (a switch link absorbs the `gp` anchor into the switch's own bridge).
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph VA["vpcs process"]
|
||||
ip["user-space IP stack"]
|
||||
end
|
||||
subgraph RootNS["root namespace"]
|
||||
gpA["gp anchor TAP"]
|
||||
BR["per-link bridge"]
|
||||
end
|
||||
ip <-->|"tap fd (-e -d)"| gpA
|
||||
gpA <-->|"bridge port"| BR
|
||||
```
|
||||
|
||||
## The three vpcs-side requirements
|
||||
|
||||
The stock vpcs build cannot carry this datapath; three things have to be true of the binary, all verified against [the vpcs fork](https://github.com/GNS3/vpcs):
|
||||
|
||||
* **`-n` (no host auto-configuration)** — without it, every guest `ip` command runs `ifconfig <tap> <addr> netmask <mask> up` on the *anchor itself*: the server owns the anchor's addresses and bridge membership, so the guest reconfiguring it ranges from noise (a failed `ifconfig` on a name that cannot resolve) to sabotage (an address on an L2-only anchor). `-n` also needs the `ifconfig` to follow the `-d` name when it does run — the stock build hardcodes `tap0`.
|
||||
* **`-d` only in single-VPC mode** — `-i 1` is how the server always launches vpcs, so this is free.
|
||||
* **`IFNAMSIZ`, not `IFNAMESIZ`** — upstream vpcs has a one-letter bug: `globle.h` defines its own `IFNAMESIZ 12`, `open_tap()` sizes its name buffer with it (`char dev[IFNAMESIZ]`) while the length guard checks the kernel's `IFNAMSIZ` (16). A 14-char anchor name passes the guard, overflows the 12-byte buffer, and reaches `TUNSETIFF` truncated to 12 chars — a nonexistent device, which the kernel tries to *create*, which needs `CAP_NET_ADMIN`, which the unprivileged vpcs does not have: `EPERM`, every time. The fork uses `IFNAMSIZ` consistently (worth an upstream PR; names up to 12 chars never trip it, which is why four years of `-d tap0` never noticed).
|
||||
|
||||
## Gating and fallback
|
||||
|
||||
A compute reports two capabilities for this (`/capabilities`, probed like every other kernel-datapath capability): `ubridge_tap` (uBridge's persistent-TAP module) and `vpcs_tap` (the binary answers `-n` in its usage — probed once per binary identity). A kernel link needs both strict-True on the compute; anything else keeps the node on the relay, where it always works. The node also decides its own datapath at start: without the probes it keeps the legacy local-UDP-tunnel shape (`-s/-c/-t` to the uBridge relay), with them it anchors and a relay link rides the anchor as an AF_PACKET endpoint — the relay-mode negative control covers both shapes.
|
||||
|
||||
## Stop order and the delete race
|
||||
|
||||
The vpcs process holds the anchor's fd, so stop is the QEMU order: terminate the process first, then `tap delete` — a held device refuses deletion and the suppressed delete would leak it. vpcs's daemon parent additionally kills its console child and exits *before* the child's fds are actually released, so the server can reap the tracked parent while the tap is still held: the shared `_delete_anchor_taps` now verifies each tap is gone from `/sys/class/net` and retries a bounded three times with a grace period (this race is not VPCS-specific — any holder that releases its fd a beat after its process was reaped hits it).
|
||||
|
||||
## Verified (real kernel, real vpcs)
|
||||
|
||||
The e2e scenario (`tests/e2e/test_vpcs_kernel_datapath.py`) drives two real VPCS nodes through the controller API and their TCP consoles:
|
||||
|
||||
* kernel link reports `kernel_datapath`, anchors exist from node start, the per-link bridge enslaves exactly the two of them; real ICMP crosses the tap-fd leg both ways;
|
||||
* the L2-anchor spec §E.1 (the anchor stays address-free — the assertion `-n` exists for) and §E.2 (a VPCS sends nothing spontaneously, so the silence window needs no guest shutdown);
|
||||
* `delay 100` lands as netem on both anchors (RTT grows ~200 ms); suspend admin-downs the anchor and kills the traffic; a capture on the link writes a real pcap;
|
||||
* link delete releases the anchor but it survives (the running vpcs holds it); re-creating the link re-enslaves it;
|
||||
* node stop terminates vpcs, deletes the anchor and the orphan bridges; a restart recreates the TAP, rewires the link from the NIO and the reconfigured VPCS pings again;
|
||||
* deleting the project leaves no anchors and no bridges behind;
|
||||
* the relay negative control (isolated relay-configured instance) wires nothing into the kernel and still pings, with the relay filters served by the uBridge bridge engine — verified with both the fork binary (relay on the anchor) and a stock binary (the legacy UDP tunnel).
|
||||
@ -31,6 +31,7 @@ from gns3server.compute.ubridge.tc_probe import (
|
||||
probe_tap_support,
|
||||
probe_tc_capabilities,
|
||||
)
|
||||
from gns3server.compute.vpcs.vpcs_probe import probe_vpcs_tap
|
||||
from gns3server.utils.path import get_default_project_directory
|
||||
from gns3server.utils.tc_capabilities import usable_ebpf_modes
|
||||
from gns3server.version import __version__
|
||||
@ -70,6 +71,10 @@ async def get_capabilities(request: Request) -> dict:
|
||||
# anchor. None = unknown (probe failed or uBridge too old).
|
||||
ubridge_bridge_tap = await probe_bridge_tap_support()
|
||||
|
||||
# VPCS tap-mode anchoring (-e -d with -n): what lets a VPCS node carry a
|
||||
# kernel link. None = unknown (probe failed or binary missing).
|
||||
vpcs_tap = await probe_vpcs_tap()
|
||||
|
||||
# record the controller hostname or IP address
|
||||
if request.client:
|
||||
request.app.state.controller_host = request.client.host
|
||||
@ -85,4 +90,5 @@ async def get_capabilities(request: Request) -> dict:
|
||||
"ubridge_tap": ubridge_tap,
|
||||
"ubridge_iol_tap": ubridge_iol_tap,
|
||||
"ubridge_bridge_tap": ubridge_bridge_tap,
|
||||
"vpcs_tap": vpcs_tap,
|
||||
}
|
||||
|
||||
@ -175,16 +175,16 @@ async def reload_vpcs_node(node: VPCSVM = Depends(dep_node)) -> None:
|
||||
@router.post(
|
||||
"/{node_id}/adapters/{adapter_number}/ports/{port_number}/nio",
|
||||
status_code=status.HTTP_201_CREATED,
|
||||
response_model=schemas.UDPNIO,
|
||||
response_model=Union[schemas.UDPNIO, schemas.BridgeNIO],
|
||||
dependencies=[Depends(compute_authentication)],
|
||||
)
|
||||
async def create_vpcs_node_nio(
|
||||
*,
|
||||
adapter_number: int = Path(..., ge=0, le=0),
|
||||
port_number: int,
|
||||
nio_data: schemas.UDPNIO,
|
||||
nio_data: Union[schemas.UDPNIO, schemas.BridgeNIO],
|
||||
node: VPCSVM = Depends(dep_node),
|
||||
) -> schemas.UDPNIO:
|
||||
) -> Union[schemas.UDPNIO, schemas.BridgeNIO]:
|
||||
"""
|
||||
Add a NIO (Network Input/Output) to the node.
|
||||
The adapter number on the VPCS node is always 0.
|
||||
@ -198,16 +198,16 @@ async def create_vpcs_node_nio(
|
||||
@router.put(
|
||||
"/{node_id}/adapters/{adapter_number}/ports/{port_number}/nio",
|
||||
status_code=status.HTTP_201_CREATED,
|
||||
response_model=schemas.UDPNIO,
|
||||
response_model=Union[schemas.UDPNIO, schemas.BridgeNIO],
|
||||
dependencies=[Depends(compute_authentication)],
|
||||
)
|
||||
async def update_vpcs_node_nio(
|
||||
*,
|
||||
adapter_number: int = Path(..., ge=0, le=0),
|
||||
port_number: int,
|
||||
nio_data: schemas.UDPNIO,
|
||||
nio_data: Union[schemas.UDPNIO, schemas.BridgeNIO],
|
||||
node: VPCSVM = Depends(dep_node),
|
||||
) -> schemas.UDPNIO:
|
||||
) -> Union[schemas.UDPNIO, schemas.BridgeNIO]:
|
||||
"""
|
||||
Update a NIO (Network Input/Output) on the node.
|
||||
The adapter number on the VPCS node is always 0.
|
||||
@ -217,6 +217,7 @@ async def update_vpcs_node_nio(
|
||||
nio.filters.clear()
|
||||
if nio_data.filters:
|
||||
nio.filters = nio_data.filters
|
||||
nio.suspend = nio_data.suspend
|
||||
nio.markers = nio_data.markers or {}
|
||||
await node.port_update_nio_binding(port_number, nio)
|
||||
return nio.asdict()
|
||||
|
||||
@ -44,6 +44,7 @@ A node type using this mixin provides:
|
||||
failures surface as ``DockerError`` / ``QemuError`` rather than ``NodeError``.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import contextlib
|
||||
import logging
|
||||
import os
|
||||
@ -213,12 +214,34 @@ class KernelDatapathMixin:
|
||||
a held device answers EBADFD and the best-effort delete below would
|
||||
leak it. The sweep is suppressed per TAP — an anchor already gone
|
||||
is not an error — and the registry is cleared either way.
|
||||
|
||||
A holder's fd can be released a moment after its process was
|
||||
reaped (VPCS's daemon parent kills its child then exits — the
|
||||
server reaps the parent before the child's fds are actually gone),
|
||||
so the first delete races EBUSY. Every tap that is still on the
|
||||
host afterwards (same-host sysfs check, zero cost on the happy
|
||||
path) gets a small grace and up to three retries; a survivor is
|
||||
warned about, not fatal.
|
||||
"""
|
||||
|
||||
if self.ubridge:
|
||||
for tap in self._kernel_taps.values():
|
||||
with contextlib.suppress(UbridgeError):
|
||||
await self._ubridge_send(f'tap delete "{tap}"')
|
||||
for attempt in range(3):
|
||||
if not os.path.exists(f"/sys/class/net/{tap}"):
|
||||
break
|
||||
await asyncio.sleep(0.3)
|
||||
with contextlib.suppress(UbridgeError):
|
||||
await self._ubridge_send(f'tap delete "{tap}"')
|
||||
else:
|
||||
if os.path.exists(f"/sys/class/net/{tap}"):
|
||||
log.warning(
|
||||
"Node '%s' [%s]: anchor TAP %s survived its delete — a holder still has it open",
|
||||
self._name,
|
||||
self._id,
|
||||
tap,
|
||||
)
|
||||
await self._remove_kernel_bridges()
|
||||
self._kernel_taps.clear()
|
||||
|
||||
|
||||
92
gns3server/compute/vpcs/vpcs_probe.py
Normal file
92
gns3server/compute/vpcs/vpcs_probe.py
Normal file
@ -0,0 +1,92 @@
|
||||
#
|
||||
# Copyright (C) 2026 GNS3 Technologies Inc.
|
||||
#
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU General Public License as published by
|
||||
# the Free Software Foundation, either version 3 of the License, or
|
||||
# (at your option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
#
|
||||
|
||||
"""
|
||||
Standalone VPCS tap-mode probe.
|
||||
|
||||
A VPCS node anchors a kernel-datapath link on a persistent TAP the vpcs
|
||||
process attaches to by name (``-e -d <anchor>``). That shape needs a vpcs
|
||||
build that also refuses to auto-configure the host side of the TAP (``-n``):
|
||||
without the flag every ``ip`` command inside the guest runs
|
||||
``ifconfig <tap> … up`` on the anchor — the server owns the anchor's
|
||||
addresses and bridge membership, so a vpcs touching them ranges from noise
|
||||
(an ifconfig on a name that cannot resolve) to sabotage (reconfiguring an
|
||||
interface that does exist).
|
||||
|
||||
The probe asks the configured binary for its usage (``vpcs -h``) and looks
|
||||
for the ``-n`` line, caching the answer by binary identity (path, mtime,
|
||||
size, ctime) exactly like the uBridge probes — the same binary never
|
||||
answers differently, but replacing it must re-probe. This feeds the compute
|
||||
``/capabilities`` payload (``vpcs_tap``) so the controller can gate VPCS
|
||||
kernel links on it. Never raises: any failure (missing binary, timeout)
|
||||
means unknown, reported as None and cached as such — the safe answer keeps
|
||||
the node on the relay datapath.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
|
||||
from gns3server.config import Config
|
||||
from gns3server.utils.asyncio import subprocess_check_output
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
# binary identity (path, mtime, size, ctime) -> whether -n is available
|
||||
# (None = probed and unknown)
|
||||
_cache: dict[tuple[str, int, int, int], bool | None] = {}
|
||||
_lock = asyncio.Lock()
|
||||
|
||||
# The stable tail of the -n usage line (the flag itself is wrapped in ANSI
|
||||
# escapes, the description is not).
|
||||
_NO_HOSTCONF_NEEDLE = "do not auto-configure"
|
||||
|
||||
|
||||
async def probe_vpcs_tap(timeout: float = 5.0) -> bool | None:
|
||||
"""
|
||||
Whether this host's VPCS binary can anchor a kernel link (tap mode with
|
||||
``-n``), probed once per binary identity.
|
||||
|
||||
:returns: True when the usage advertises ``-n``, False when it does not
|
||||
(an upstream build), None when the probe failed (unknown binary).
|
||||
"""
|
||||
|
||||
config = Config.instance()
|
||||
vpcs_path = config.settings.VPCS.vpcs_path
|
||||
if not os.path.isabs(vpcs_path):
|
||||
vpcs_path = shutil.which(vpcs_path)
|
||||
if not vpcs_path or not os.path.isfile(vpcs_path):
|
||||
return None
|
||||
try:
|
||||
stat = os.stat(vpcs_path)
|
||||
key = (vpcs_path, stat.st_mtime_ns, stat.st_size, stat.st_ctime_ns)
|
||||
except OSError:
|
||||
return None
|
||||
|
||||
async with _lock:
|
||||
if key in _cache:
|
||||
return _cache[key]
|
||||
try:
|
||||
output = await asyncio.wait_for(subprocess_check_output(vpcs_path, "-h"), timeout=timeout)
|
||||
except (OSError, asyncio.TimeoutError, ValueError) as e:
|
||||
log.debug("VPCS tap-mode probe failed: %s", e)
|
||||
supported = None
|
||||
else:
|
||||
supported = _NO_HOSTCONF_NEEDLE in output
|
||||
_cache[key] = supported
|
||||
return supported
|
||||
@ -31,16 +31,20 @@ import sys
|
||||
|
||||
from gns3server.utils import parse_version
|
||||
from gns3server.utils.asyncio import monitor_process, subprocess_check_output, wait_for_process_termination
|
||||
from gns3server.utils.kernel_anchor import kernel_anchor_name
|
||||
|
||||
from ..adapters.ethernet_adapter import EthernetAdapter
|
||||
from ..base_node import BaseNode
|
||||
from ..kernel_datapath import KernelDatapathMixin
|
||||
from ..nios.nio_bridge import NIOBridge
|
||||
from ..nios.nio_udp import NIOUDP
|
||||
from .vpcs_error import VPCSError
|
||||
from .vpcs_probe import probe_vpcs_tap
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class VPCSVM(BaseNode):
|
||||
class VPCSVM(KernelDatapathMixin, BaseNode):
|
||||
module_name = "vpcs"
|
||||
|
||||
"""
|
||||
@ -62,6 +66,13 @@ class VPCSVM(BaseNode):
|
||||
self._vpcs_version = None
|
||||
self._started = False
|
||||
self._local_udp_tunnel = None
|
||||
self._execute_lock = asyncio.Lock()
|
||||
# Kernel-datapath state (the anchor is a persistent TAP uBridge
|
||||
# creates and the vpcs process attaches to — see the datapath
|
||||
# section below).
|
||||
self._tap_datapath = False
|
||||
self._kernel_taps = {}
|
||||
self._ubridge_tc_caps = None
|
||||
|
||||
# VPCS settings
|
||||
if startup_script is not None and not self.script_file: # We disallow override at startup
|
||||
@ -72,6 +83,15 @@ class VPCSVM(BaseNode):
|
||||
def ethernet_adapter(self):
|
||||
return self._ethernet_adapter
|
||||
|
||||
@property
|
||||
def _ethernet_adapters(self):
|
||||
"""
|
||||
The adapter list the kernel-datapath mixin walks (the per-link
|
||||
bridge sweep): VPCS owns exactly one single-port adapter.
|
||||
"""
|
||||
|
||||
return [self._ethernet_adapter]
|
||||
|
||||
async def close(self):
|
||||
"""
|
||||
Closes this VPCS VM.
|
||||
@ -89,10 +109,14 @@ class VPCSVM(BaseNode):
|
||||
self.manager.port_manager.release_udp_port(self._local_udp_tunnel[1].lport, self._project)
|
||||
self._local_udp_tunnel = None
|
||||
|
||||
await self._stop_ubridge()
|
||||
|
||||
# stop() tears the datapath down in the right order (the vpcs
|
||||
# process holds the anchor TAPs open — a held device refuses
|
||||
# deletion and would leak); the else branch covers a node that is
|
||||
# not running but may still hold a uBridge with anchors.
|
||||
if self.is_running():
|
||||
self._terminate_process()
|
||||
await self.stop()
|
||||
else:
|
||||
await self._stop_ubridge()
|
||||
|
||||
return True
|
||||
|
||||
@ -225,34 +249,51 @@ class VPCSVM(BaseNode):
|
||||
await self._check_requirements()
|
||||
if not self.is_running():
|
||||
nio = self._ethernet_adapter.get_nio(0)
|
||||
command = self._build_command()
|
||||
# uBridge owns the anchor TAP and must be up before it is
|
||||
# created — and before vpcs attaches to it. The whole setup runs
|
||||
# inside the try so a failure at any step (probe refused, tap
|
||||
# create, launch) takes the datapath down again: a node that
|
||||
# reports stopped must not keep a uBridge process and a
|
||||
# root-namespace TAP alive.
|
||||
self._ubridge_tc_caps = None
|
||||
try:
|
||||
log.info(f"Starting VPCS: {command}")
|
||||
self._vpcs_stdout_file = os.path.join(self.working_dir, "vpcs.log")
|
||||
log.info(f"Logging to {self._vpcs_stdout_file}")
|
||||
flags = 0
|
||||
if sys.platform.startswith("win32"):
|
||||
flags = subprocess.CREATE_NEW_PROCESS_GROUP
|
||||
with open(self._vpcs_stdout_file, "w", encoding="utf-8") as fd:
|
||||
self.command_line = " ".join(command)
|
||||
self._process = await asyncio.create_subprocess_exec(
|
||||
*command, stdout=fd, stderr=subprocess.STDOUT, cwd=self.working_dir, creationflags=flags
|
||||
)
|
||||
monitor_process(self._process, self._termination_callback)
|
||||
|
||||
await self._start_ubridge()
|
||||
if nio:
|
||||
await self.add_ubridge_udp_connection(f"VPCS-{self._id}", self._local_udp_tunnel[1], nio)
|
||||
await self._prepare_tap_datapath()
|
||||
|
||||
await self.start_wrap_console()
|
||||
command = self._build_command()
|
||||
try:
|
||||
log.info(f"Starting VPCS: {command}")
|
||||
self._vpcs_stdout_file = os.path.join(self.working_dir, "vpcs.log")
|
||||
log.info(f"Logging to {self._vpcs_stdout_file}")
|
||||
flags = 0
|
||||
if sys.platform.startswith("win32"):
|
||||
flags = subprocess.CREATE_NEW_PROCESS_GROUP
|
||||
with open(self._vpcs_stdout_file, "w", encoding="utf-8") as fd:
|
||||
self.command_line = " ".join(command)
|
||||
self._process = await asyncio.create_subprocess_exec(
|
||||
*command, stdout=fd, stderr=subprocess.STDOUT, cwd=self.working_dir, creationflags=flags
|
||||
)
|
||||
monitor_process(self._process, self._termination_callback)
|
||||
|
||||
log.info(f"VPCS instance {self.name} started PID={self._process.pid}")
|
||||
self._started = True
|
||||
self.status = "started"
|
||||
except (OSError, subprocess.SubprocessError) as e:
|
||||
vpcs_stdout = self.read_vpcs_stdout()
|
||||
log.error(f"Could not start VPCS {self._vpcs_path()}: {e}\n{vpcs_stdout}")
|
||||
raise VPCSError(f"Could not start VPCS {self._vpcs_path()}: {e}\n{vpcs_stdout}")
|
||||
if nio:
|
||||
# a link that outlived a stop is re-attached from its NIO
|
||||
await self._connect_nio(0, nio)
|
||||
await self._set_adapter_carrier(0, not nio.suspend)
|
||||
|
||||
await self.start_wrap_console()
|
||||
|
||||
log.info(f"VPCS instance {self.name} started PID={self._process.pid}")
|
||||
self._started = True
|
||||
self.status = "started"
|
||||
except (OSError, subprocess.SubprocessError) as e:
|
||||
vpcs_stdout = self.read_vpcs_stdout()
|
||||
log.error(f"Could not start VPCS {self._vpcs_path()}: {e}\n{vpcs_stdout}")
|
||||
raise VPCSError(f"Could not start VPCS {self._vpcs_path()}: {e}\n{vpcs_stdout}")
|
||||
except Exception:
|
||||
# uBridge was started (and the anchor TAP created) before the
|
||||
# launch: don't leave either behind on a failed start.
|
||||
await self._stop_ubridge()
|
||||
raise
|
||||
|
||||
async def _termination_callback(self, returncode):
|
||||
"""
|
||||
@ -263,11 +304,9 @@ class VPCSVM(BaseNode):
|
||||
|
||||
if self._started:
|
||||
log.info("VPCS process has stopped, return code: %d", returncode)
|
||||
self._started = False
|
||||
self.status = "stopped"
|
||||
self._process = None
|
||||
await self._stop_ubridge()
|
||||
await super().stop()
|
||||
# stop() tears the datapath down under the execute lock (an API
|
||||
# stop may be running concurrently) and sets the status.
|
||||
await self.stop()
|
||||
if returncode != 0:
|
||||
self.project.emit(
|
||||
"log.error",
|
||||
@ -279,24 +318,156 @@ class VPCSVM(BaseNode):
|
||||
Stops the VPCS process.
|
||||
"""
|
||||
|
||||
await self._stop_ubridge()
|
||||
if self.is_running():
|
||||
self._terminate_process()
|
||||
if self._process.returncode is None:
|
||||
try:
|
||||
await wait_for_process_termination(self._process, timeout=3)
|
||||
except asyncio.TimeoutError:
|
||||
if self._process.returncode is None:
|
||||
try:
|
||||
self._process.kill()
|
||||
except OSError as e:
|
||||
log.error(f"Cannot stop the VPCS process: {e}")
|
||||
async with self._execute_lock:
|
||||
# Release the datapath once vpcs is gone: the process holds the
|
||||
# anchor TAP open (it is the guest side), and uBridge's tap
|
||||
# delete refuses a held device ("Device or resource busy") —
|
||||
# deleting it while vpcs still ran leaked the persistent TAP,
|
||||
# because that best-effort delete is suppressed. The process
|
||||
# monitor calls stop() on its own when vpcs dies, so an API stop
|
||||
# and a process-death stop run concurrently in the normal case;
|
||||
# the lock serializes their TAP sweeps (the second one finds
|
||||
# uBridge already gone and sweeps nothing).
|
||||
if self.is_running():
|
||||
self._terminate_process()
|
||||
if self._process.returncode is None:
|
||||
try:
|
||||
await wait_for_process_termination(self._process, timeout=3)
|
||||
except asyncio.TimeoutError:
|
||||
if self._process.returncode is None:
|
||||
log.warning(f'VPCS VM "{self._name}" with PID={self._process.pid} is still running')
|
||||
try:
|
||||
self._process.kill()
|
||||
except OSError as e:
|
||||
log.error(f"Cannot stop the VPCS process: {e}")
|
||||
if self._process.returncode is None:
|
||||
log.warning(f'VPCS VM "{self._name}" with PID={self._process.pid} is still running')
|
||||
|
||||
self._process = None
|
||||
self._started = False
|
||||
await super().stop()
|
||||
self._process = None
|
||||
self._started = False
|
||||
await self._stop_ubridge()
|
||||
await super().stop()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Kernel datapath: the adapter anchor is a persistent TAP
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _tap_name(self, adapter_number, port_number=0):
|
||||
"""
|
||||
Deterministic anchor TAP name for the adapter port (the shared
|
||||
utils.kernel_anchor naming contract — the controller names a peer's
|
||||
anchor with the same function when an Ethernet switch absorbs it).
|
||||
"""
|
||||
|
||||
return kernel_anchor_name("vpcs", self._id, adapter_number, port_number)
|
||||
|
||||
def _kernel_host_ifc(self, adapter_number, port_number=0):
|
||||
"""
|
||||
The persistent TAP this adapter owns, or None before the node
|
||||
started (or on the legacy UDP relay datapath).
|
||||
"""
|
||||
|
||||
return self._kernel_taps.get((adapter_number, port_number))
|
||||
|
||||
def _kernel_anchors(self):
|
||||
return set(self._kernel_taps.values())
|
||||
|
||||
def _kernel_error(self, message):
|
||||
return VPCSError(message)
|
||||
|
||||
async def _prepare_tap_datapath(self):
|
||||
"""
|
||||
Decide this start's datapath: a vpcs build with ``-n`` (no host
|
||||
auto-configuration) and a uBridge with the persistent-TAP module
|
||||
anchor the adapter on a persistent TAP the vpcs process attaches to
|
||||
(``-e -d``). Anything else (upstream vpcs, old uBridge, no
|
||||
CAP_NET_ADMIN) keeps the legacy UDP relay datapath — kernel links
|
||||
are refused on it.
|
||||
"""
|
||||
|
||||
self._tap_datapath = False
|
||||
self._kernel_taps.clear()
|
||||
if not self.ubridge:
|
||||
return
|
||||
|
||||
vpcs_tap = await probe_vpcs_tap()
|
||||
if vpcs_tap is not True:
|
||||
# Without -n every guest `ip` command would ifconfig the anchor
|
||||
# on the host — the server owns the anchor's addresses and
|
||||
# bridge membership, so the relay datapath is the only safe
|
||||
# shape. False (a build definitively without -n) is actionable
|
||||
# for the user; None (probe failed) stays server-log-only.
|
||||
log.warning(
|
||||
"VPCS '%s': vpcs '%s' cannot anchor a kernel link (no -n support)", self._name, self._vpcs_path()
|
||||
)
|
||||
if vpcs_tap is False:
|
||||
self.project.emit(
|
||||
"log.warning",
|
||||
{
|
||||
"message": f"VPCS '{self._name}': the vpcs binary does not support tap mode without host "
|
||||
"auto-configuration (-n); this node runs on the legacy relay datapath and cannot carry "
|
||||
"kernel links"
|
||||
},
|
||||
)
|
||||
return
|
||||
|
||||
probe_error = await self._probe_persistent_taps(f"gp{self._id.replace('-', '')[:8]}prob")
|
||||
if probe_error is not None:
|
||||
log.warning("VPCS '%s': uBridge cannot create persistent TAPs (%s)", self._name, probe_error)
|
||||
self.project.emit(
|
||||
"log.warning",
|
||||
{
|
||||
"message": f"VPCS '{self._name}': uBridge does not support persistent TAPs ({probe_error}); "
|
||||
"this node runs on the legacy relay datapath and cannot carry kernel links"
|
||||
},
|
||||
)
|
||||
return
|
||||
|
||||
self._tap_datapath = True
|
||||
# The shared lifecycle sweeps leftovers first, creates the TAP and
|
||||
# hands it to this user (the unprivileged vpcs process opens it
|
||||
# itself), leaving it down until a link attaches.
|
||||
await self._create_anchor_taps([(0, 0)], set_owner=True)
|
||||
|
||||
async def _stop_ubridge(self):
|
||||
"""
|
||||
Stops uBridge, removing the anchor TAP first: it lives in the root
|
||||
namespace and outlives the process (the tap module needs the control
|
||||
channel to delete it), and vpcs holding the other end keeps the
|
||||
device alive until it exits — the caller must terminate vpcs first,
|
||||
or the suppressed tap delete leaks the anchor. The next start
|
||||
spawns a fresh uBridge, so the tc capabilities must be probed again.
|
||||
"""
|
||||
|
||||
await self._delete_anchor_taps()
|
||||
self._ubridge_tc_caps = None
|
||||
await super()._stop_ubridge()
|
||||
|
||||
async def _connect_nio(self, port_number, nio):
|
||||
"""
|
||||
Attach a link's NIO to the adapter's TAP anchor: a kernel link
|
||||
enslaves the TAP into the per-link kernel bridge, a relay link
|
||||
attaches it to the uBridge relay bridge as an AF_PACKET endpoint.
|
||||
The legacy datapath (no anchor) bridges the local UDP tunnel
|
||||
instead.
|
||||
"""
|
||||
|
||||
bridge_name = f"VPCS-{self._id}"
|
||||
if not self._tap_datapath:
|
||||
if isinstance(nio, NIOBridge):
|
||||
raise VPCSError(
|
||||
f"VPCS '{self._name}' runs on the legacy relay datapath (no anchor TAP) "
|
||||
"and cannot carry a kernel link"
|
||||
)
|
||||
await self.add_ubridge_udp_connection(bridge_name, self._local_udp_tunnel[1], nio)
|
||||
return
|
||||
|
||||
anchor = self._kernel_host_ifc(0, port_number)
|
||||
if anchor is None:
|
||||
raise VPCSError(f"VPCS '{self._name}' has no anchor TAP interface; restart the node to attach a link to it")
|
||||
if isinstance(nio, NIOBridge):
|
||||
await self._kernel_attach(anchor, nio)
|
||||
return
|
||||
await self._relay_attach(anchor, bridge_name, nio)
|
||||
|
||||
async def reload(self):
|
||||
"""
|
||||
@ -386,8 +557,15 @@ class VPCSVM(BaseNode):
|
||||
if not self._ethernet_adapter.port_exists(port_number):
|
||||
raise VPCSError(f"Port {port_number} doesn't exist on adapter {self._ethernet_adapter}")
|
||||
|
||||
# The compute-side backstop of the controller's duplicate-port guard:
|
||||
# a port carries at most one NIO, and overwriting the one a link
|
||||
# still owns orphans that link's teardown (its host state leaks).
|
||||
if self._ethernet_adapter.get_nio(port_number) is not None:
|
||||
raise VPCSError(f"Port {port_number} of VPCS '{self._name}' already has a link")
|
||||
|
||||
if self.is_running():
|
||||
await self.add_ubridge_udp_connection(f"VPCS-{self._id}", self._local_udp_tunnel[1], nio)
|
||||
await self._connect_nio(port_number, nio)
|
||||
await self._set_adapter_carrier(0, not nio.suspend)
|
||||
|
||||
self._ethernet_adapter.add_nio(port_number, nio)
|
||||
log.info(f'VPCS "{self._name}" [{self.id}]: {nio} added to port {port_number}')
|
||||
@ -405,7 +583,19 @@ class VPCSVM(BaseNode):
|
||||
if not self._ethernet_adapter.port_exists(port_number):
|
||||
raise VPCSError(f"Port {port_number} doesn't exist on adapter {self._ethernet_adapter}")
|
||||
if self.is_running():
|
||||
await self.update_ubridge_udp_connection(f"VPCS-{self._id}", self._local_udp_tunnel[1], nio)
|
||||
if self._tap_datapath and isinstance(nio, NIOBridge):
|
||||
# Filters or markers changed on an attached kernel link:
|
||||
# re-apply on the anchor, no re-enslaving.
|
||||
anchor = self._kernel_host_ifc(0, port_number)
|
||||
if anchor is not None:
|
||||
await self._kernel_update(anchor, nio)
|
||||
elif self._tap_datapath:
|
||||
# Relay link on the TAP anchor: re-apply on its bridge.
|
||||
await self._ubridge_apply_filters(f"VPCS-{self._id}", nio.filters)
|
||||
await self._ubridge_apply_markers(f"VPCS-{self._id}", nio)
|
||||
else:
|
||||
await self.update_ubridge_udp_connection(f"VPCS-{self._id}", self._local_udp_tunnel[1], nio)
|
||||
await self._set_adapter_carrier(0, not nio.suspend)
|
||||
|
||||
async def port_remove_nio_binding(self, port_number):
|
||||
"""
|
||||
@ -419,11 +609,16 @@ class VPCSVM(BaseNode):
|
||||
if not self._ethernet_adapter.port_exists(port_number):
|
||||
raise VPCSError(f"Port {port_number} doesn't exist on adapter {self._ethernet_adapter}")
|
||||
|
||||
nio = self._ethernet_adapter.get_nio(port_number)
|
||||
await self.stop_capture(port_number)
|
||||
if self.is_running():
|
||||
await self._ubridge_send("bridge delete {name}".format(name=f"VPCS-{self._id}"))
|
||||
if self._tap_datapath and isinstance(nio, NIOBridge):
|
||||
await self._remove_kernel_nio(nio, 0, port_number)
|
||||
elif self._tap_datapath:
|
||||
await self._relay_detach(f"VPCS-{self._id}")
|
||||
else:
|
||||
await self._ubridge_send("bridge delete {name}".format(name=f"VPCS-{self._id}"))
|
||||
|
||||
nio = self._ethernet_adapter.get_nio(port_number)
|
||||
if isinstance(nio, NIOUDP):
|
||||
self.manager.port_manager.release_udp_port(nio.lport, self._project)
|
||||
self._ethernet_adapter.remove_nio(port_number)
|
||||
@ -461,9 +656,25 @@ class VPCSVM(BaseNode):
|
||||
|
||||
nio.start_packet_capture(output_file)
|
||||
if self.ubridge:
|
||||
await self._ubridge_send(
|
||||
'bridge start_capture {name} "{output_file}"'.format(name=f"VPCS-{self._id}", output_file=output_file)
|
||||
)
|
||||
anchor = self._kernel_host_ifc(0, port_number)
|
||||
if isinstance(nio, NIOBridge) and anchor is not None:
|
||||
# Kernel link: capture on the adapter's TAP anchor (AF_PACKET,
|
||||
# one capture per uBridge process — see _reserve_kernel_capture;
|
||||
# a failed start rolls the port's flag back so the port cannot
|
||||
# later stop the winner's capture).
|
||||
try:
|
||||
self._reserve_kernel_capture(anchor)
|
||||
await self._ubridge_send(f'capture start_kernel {anchor} "{output_file}"')
|
||||
except Exception:
|
||||
self._release_kernel_capture(anchor)
|
||||
nio.stop_packet_capture()
|
||||
raise
|
||||
else:
|
||||
await self._ubridge_send(
|
||||
'bridge start_capture {name} "{output_file}"'.format(
|
||||
name=f"VPCS-{self._id}", output_file=output_file
|
||||
)
|
||||
)
|
||||
|
||||
log.info(f"VPCS '{self.name}' [{self.id}]: starting packet capture on port {port_number}")
|
||||
|
||||
@ -480,7 +691,16 @@ class VPCSVM(BaseNode):
|
||||
|
||||
nio.stop_packet_capture()
|
||||
if self.ubridge:
|
||||
await self._ubridge_send("bridge stop_capture {name}".format(name=f"VPCS-{self._id}"))
|
||||
anchor = self._kernel_host_ifc(0, port_number)
|
||||
if isinstance(nio, NIOBridge) and anchor is not None:
|
||||
# Process-wide and argument-less: only the port owning the
|
||||
# slot may stop it (a second port's stop would kill this
|
||||
# capture while its own flag still says capturing).
|
||||
if self._kernel_capture_owned_by(anchor):
|
||||
await self._ubridge_send("capture stop_kernel")
|
||||
self._release_kernel_capture(anchor)
|
||||
else:
|
||||
await self._ubridge_send("bridge stop_capture {name}".format(name=f"VPCS-{self._id}"))
|
||||
|
||||
log.info(f"VPCS '{self.name}' [{self.id}]: stopping packet capture on port {port_number}")
|
||||
|
||||
@ -511,6 +731,7 @@ class VPCSVM(BaseNode):
|
||||
|
||||
tap mode options:
|
||||
-d vm device name, works only when -i is set to 1
|
||||
-n do not auto-configure the host tap interface
|
||||
|
||||
hypervisor mode option:
|
||||
-H port run as the hypervisor listening on the tcp 'port'
|
||||
@ -532,20 +753,28 @@ class VPCSVM(BaseNode):
|
||||
else:
|
||||
log.warning("The VPCS relay feature could not be disabled because the VPCS version is below 0.8b")
|
||||
|
||||
# use the local UDP tunnel to uBridge instead
|
||||
if not self._local_udp_tunnel:
|
||||
self._local_udp_tunnel = self._create_local_udp_tunnel()
|
||||
nio = self._local_udp_tunnel[0]
|
||||
if nio and isinstance(nio, NIOUDP):
|
||||
# UDP tunnel
|
||||
command.extend(["-s", str(nio.lport)]) # source UDP port
|
||||
command.extend(["-c", str(nio.rport)]) # destination UDP port
|
||||
try:
|
||||
command.extend(
|
||||
["-t", socket.gethostbyname(nio.rhost)]
|
||||
) # destination host, we need to resolve the hostname because VPCS doesn't support it
|
||||
except socket.gaierror as e:
|
||||
raise VPCSError(f"Can't resolve hostname {nio.rhost}")
|
||||
if self._tap_datapath:
|
||||
# Kernel-datapath anchor: uBridge created the persistent TAP
|
||||
# before the launch, vpcs attaches to it by name and must leave
|
||||
# its host-side configuration alone (-n).
|
||||
command.extend(["-e"]) # tap mode
|
||||
command.extend(["-d", self._tap_name(0, 0)]) # the anchor TAP
|
||||
command.extend(["-n"]) # no host auto-configuration on the anchor
|
||||
else:
|
||||
# use the local UDP tunnel to uBridge instead
|
||||
if not self._local_udp_tunnel:
|
||||
self._local_udp_tunnel = self._create_local_udp_tunnel()
|
||||
nio = self._local_udp_tunnel[0]
|
||||
if nio and isinstance(nio, NIOUDP):
|
||||
# UDP tunnel
|
||||
command.extend(["-s", str(nio.lport)]) # source UDP port
|
||||
command.extend(["-c", str(nio.rport)]) # destination UDP port
|
||||
try:
|
||||
command.extend(
|
||||
["-t", socket.gethostbyname(nio.rhost)]
|
||||
) # destination host, we need to resolve the hostname because VPCS doesn't support it
|
||||
except socket.gaierror as e:
|
||||
raise VPCSError(f"Can't resolve hostname {nio.rhost}")
|
||||
|
||||
if self.script_file:
|
||||
command.extend([os.path.basename(self.script_file)])
|
||||
|
||||
@ -221,7 +221,11 @@ class UDPLink(Link):
|
||||
be present or absent independently of the tap module. Dynamips
|
||||
Ethernet slot ports anchor on persistent TAPs the hypervisor opens
|
||||
(the tap module again — the Dynamips binary itself creates nothing),
|
||||
so it gates like QEMU. A kernel link is an Ethernet segment, so
|
||||
so it gates like QEMU. A VPCS node's single adapter anchors on a
|
||||
persistent TAP the vpcs process attaches to by name (``-e -d``),
|
||||
which needs the tap module plus a vpcs build that keeps its hands
|
||||
off the host side of the TAP (``-n``) — its own capability on top
|
||||
of the tap one. A kernel link is an Ethernet segment, so
|
||||
non-Ethernet ports (IOU serial, Dynamips serial/ATM/POS adapters)
|
||||
stay on the relay whatever the node's capabilities.
|
||||
"""
|
||||
@ -291,6 +295,14 @@ class UDPLink(Link):
|
||||
# merely opens (nio create_tap) — the same tap-module capability
|
||||
# QEMU needs, not a Dynamips-specific one.
|
||||
return capabilities.get("ubridge_tap") is True
|
||||
if node.node_type == "vpcs":
|
||||
# The anchor is a uBridge-created TAP the vpcs process attaches
|
||||
# to (-e -d): the tap module must exist, and the binary must
|
||||
# support -n — without it every guest `ip` command would
|
||||
# ifconfig the anchor on the host. Both strict True: an
|
||||
# unreported one keeps the link on the relay, where it always
|
||||
# works.
|
||||
return capabilities.get("ubridge_tap") is True and capabilities.get("vpcs_tap") is True
|
||||
return False
|
||||
|
||||
async def _prepare(self):
|
||||
|
||||
@ -67,3 +67,9 @@ class Capabilities(BaseModel):
|
||||
"(bridge delete_nio_tap, the swappable TAP leg an IOL runner container's per-port bridge "
|
||||
"needs to wire its links on the kernel datapath); absent when the probe failed",
|
||||
)
|
||||
vpcs_tap: Optional[bool] = Field(
|
||||
None,
|
||||
description="Whether the VPCS binary can anchor a kernel link (tap mode with -n, which "
|
||||
"keeps the guest from auto-configuring the host side of the anchor TAP); absent when the "
|
||||
"probe failed",
|
||||
)
|
||||
|
||||
@ -22,10 +22,11 @@ deterministically: ``g<type>{node_id[:8]}e{adapter}p{port}`` — ``gv`` for a
|
||||
Docker veth host end, ``gq`` for a QEMU TAP, ``gi`` for an IOU port TAP,
|
||||
``gd`` for a Dynamips port TAP, ``gx`` for an IOL runner container's port
|
||||
TAP (a Docker node whose guest leg is unix sockets, so its anchor is a TAP
|
||||
like IOU's, not a veth). The prefixes keep anchors out of the ``gns3``
|
||||
bridge/TAP name space and name-collision-free on a host; the 8 hex chars of
|
||||
the node id plus adapter/port keep every name unique and within IFNAMSIZ
|
||||
(15).
|
||||
like IOU's, not a veth), ``gp`` for a VPCS TAP (the vpcs process attaches to
|
||||
the pre-created anchor with ``-e -d`` instead of speaking UDP). The
|
||||
prefixes keep anchors out of the ``gns3`` bridge/TAP name space and
|
||||
name-collision-free on a host; the 8 hex chars of the node id plus
|
||||
adapter/port keep every name unique and within IFNAMSIZ (15).
|
||||
|
||||
The compute classes create their anchors with these helpers (single source
|
||||
of truth), and so does the controller when a switch link has to name the
|
||||
@ -46,6 +47,7 @@ ANCHOR_PREFIX = {
|
||||
"qemu": "gq",
|
||||
"iou": "gi",
|
||||
"dynamips": "gd",
|
||||
"vpcs": "gp",
|
||||
}
|
||||
|
||||
|
||||
|
||||
@ -44,6 +44,7 @@ class TestCapabilitiesRoutes:
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_tap_support", _no_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_iol_tap_support", _no_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_bridge_tap_support", _no_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_vpcs_tap", _no_tap_probe)
|
||||
response = await compute_client.get(app.url_path_for("compute:get_capabilities"))
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.json() == {
|
||||
@ -71,6 +72,7 @@ class TestCapabilitiesRoutes:
|
||||
"ubridge_tap": None,
|
||||
"ubridge_iol_tap": None,
|
||||
"ubridge_bridge_tap": None,
|
||||
"vpcs_tap": None,
|
||||
}
|
||||
|
||||
async def test_get_on_gns3vm(self, app: FastAPI, compute_client: AsyncClient, on_gns3vm, monkeypatch) -> None:
|
||||
@ -82,6 +84,7 @@ class TestCapabilitiesRoutes:
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_tap_support", _no_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_iol_tap_support", _no_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_bridge_tap_support", _no_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_vpcs_tap", _no_tap_probe)
|
||||
response = await compute_client.get(app.url_path_for("compute:get_capabilities"))
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.json() == {
|
||||
@ -109,6 +112,7 @@ class TestCapabilitiesRoutes:
|
||||
"ubridge_tap": None,
|
||||
"ubridge_iol_tap": None,
|
||||
"ubridge_bridge_tap": None,
|
||||
"vpcs_tap": None,
|
||||
}
|
||||
|
||||
async def test_get_reports_ubridge_tc(self, app: FastAPI, compute_client: AsyncClient, monkeypatch) -> None:
|
||||
@ -120,6 +124,7 @@ class TestCapabilitiesRoutes:
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_tap_support", _no_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_iol_tap_support", _no_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_bridge_tap_support", _no_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_vpcs_tap", _no_tap_probe)
|
||||
response = await compute_client.get(app.url_path_for("compute:get_capabilities"))
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.json()["ubridge_tc"] == {
|
||||
@ -141,6 +146,7 @@ class TestCapabilitiesRoutes:
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_tap_support", fake_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_iol_tap_support", _no_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_bridge_tap_support", _no_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_vpcs_tap", _no_tap_probe)
|
||||
response = await compute_client.get(app.url_path_for("compute:get_capabilities"))
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.json()["ubridge_tap"] is True
|
||||
@ -175,6 +181,24 @@ class TestCapabilitiesRoutes:
|
||||
)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_tap_support", _no_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_iol_tap_support", _no_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_vpcs_tap", _no_tap_probe)
|
||||
response = await compute_client.get(app.url_path_for("compute:get_capabilities"))
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.json()["ubridge_bridge_tap"] is True
|
||||
|
||||
async def test_get_reports_vpcs_tap(self, app: FastAPI, compute_client: AsyncClient, monkeypatch) -> None:
|
||||
"""
|
||||
The VPCS tap-mode capability is what makes a VPCS node
|
||||
kernel-eligible, so it travels with the other compute capabilities.
|
||||
"""
|
||||
|
||||
async def fake_vpcs_tap_probe():
|
||||
return True
|
||||
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_vpcs_tap", fake_vpcs_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_tap_support", _no_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_iol_tap_support", _no_tap_probe)
|
||||
monkeypatch.setattr("gns3server.api.routes.compute.capabilities.probe_bridge_tap_support", _no_tap_probe)
|
||||
response = await compute_client.get(app.url_path_for("compute:get_capabilities"))
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.json()["vpcs_tap"] is True
|
||||
|
||||
78
tests/compute/vpcs/test_vpcs_probe.py
Normal file
78
tests/compute/vpcs/test_vpcs_probe.py
Normal file
@ -0,0 +1,78 @@
|
||||
#
|
||||
# Copyright (C) 2026 GNS3 Technologies Inc.
|
||||
#
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU General Public License as published by
|
||||
# the Free Software Foundation, either version 3 of the License, or
|
||||
# (at your option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
#
|
||||
|
||||
import os
|
||||
import stat
|
||||
|
||||
import pytest
|
||||
|
||||
from gns3server.compute.vpcs.vpcs_probe import probe_vpcs_tap
|
||||
|
||||
pytestmark = pytest.mark.asyncio
|
||||
|
||||
|
||||
def _fake_vpcs(path, usage):
|
||||
path.write_text(f"#!/bin/sh\ncat <<'EOF'\n{usage}\nEOF\n")
|
||||
mode = path.stat().st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH
|
||||
os.chmod(path, mode)
|
||||
|
||||
|
||||
async def test_probe_reports_dash_n_build(config, tmp_path):
|
||||
|
||||
binary = tmp_path / "vpcs"
|
||||
_fake_vpcs(
|
||||
binary,
|
||||
" -d device device name, works only when -i is set to 1\n"
|
||||
" -n do not auto-configure the host tap interface\n",
|
||||
)
|
||||
config.settings.VPCS.vpcs_path = str(binary)
|
||||
assert await probe_vpcs_tap() is True
|
||||
|
||||
|
||||
async def test_probe_reports_upstream_build(config, tmp_path):
|
||||
|
||||
binary = tmp_path / "vpcs"
|
||||
_fake_vpcs(
|
||||
binary,
|
||||
" -d device device name, works only when -i is set to 1\n",
|
||||
)
|
||||
config.settings.VPCS.vpcs_path = str(binary)
|
||||
assert await probe_vpcs_tap() is False
|
||||
|
||||
|
||||
async def test_probe_reports_missing_binary(config, tmp_path):
|
||||
|
||||
config.settings.VPCS.vpcs_path = str(tmp_path / "does-not-exist")
|
||||
assert await probe_vpcs_tap() is None
|
||||
|
||||
|
||||
async def test_probe_caches_by_binary_identity(config, tmp_path):
|
||||
"""
|
||||
The same binary answers once; replacing it (a different mtime/size)
|
||||
re-probes.
|
||||
"""
|
||||
|
||||
binary = tmp_path / "vpcs"
|
||||
_fake_vpcs(binary, " -n do not auto-configure\n")
|
||||
config.settings.VPCS.vpcs_path = str(binary)
|
||||
assert await probe_vpcs_tap() is True
|
||||
|
||||
# replace with a build without the flag: the binary identity changed,
|
||||
# so the cached True must not answer
|
||||
_fake_vpcs(binary, " -d device device name\n")
|
||||
os.utime(binary, (0, 0))
|
||||
assert await probe_vpcs_tap() is False
|
||||
@ -45,7 +45,12 @@ async def vm(compute_project, manager, tmpdir, ubridge_path):
|
||||
vm._start_ubridge = AsyncioMagicMock()
|
||||
vm._ubridge_hypervisor = MagicMock()
|
||||
vm._ubridge_hypervisor.is_running.return_value = True
|
||||
return vm
|
||||
# The legacy-datapath tests run with an unknown probe answer (None):
|
||||
# the tap datapath stays off without a project warning, exactly like a
|
||||
# host whose probe failed. The tap-datapath tests patch the probe to
|
||||
# True (or False, when they assert the fallback) on top.
|
||||
with patch("gns3server.compute.vpcs.vpcs_vm.probe_vpcs_tap", new=AsyncioMagicMock(return_value=None)):
|
||||
yield vm
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@ -350,3 +355,175 @@ async def test_close(vm):
|
||||
await vm.start()
|
||||
await vm.close()
|
||||
assert vm.is_running() is False
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Kernel datapath (the adapter anchor is a persistent TAP the vpcs process
|
||||
# attaches to with -e -d, kept away from host auto-configuration with -n)
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
async def _start_on_tap_datapath(vm):
|
||||
"""
|
||||
Start the node with both probes answering (a -n-capable vpcs build, a
|
||||
uBridge with the tap module), returning the recorded uBridge commands.
|
||||
"""
|
||||
|
||||
# a startup.vpc left in the shared working directory by an earlier test
|
||||
# would append its name to the command line (the script tests run in any
|
||||
# order) — start from a clean one.
|
||||
try:
|
||||
os.remove(os.path.join(vm.working_dir, "startup.vpc"))
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
|
||||
commands = []
|
||||
|
||||
async def record_send(command):
|
||||
commands.append(command)
|
||||
|
||||
vm._ubridge_send = AsyncioMagicMock(side_effect=record_send)
|
||||
process = MagicMock()
|
||||
process.returncode = None
|
||||
with patch("gns3server.compute.vpcs.vpcs_vm.probe_vpcs_tap", new=AsyncioMagicMock(return_value=True)):
|
||||
with asyncio_patch("gns3server.compute.vpcs.vpcs_vm.VPCSVM._check_requirements", return_value=True):
|
||||
with asyncio_patch("asyncio.create_subprocess_exec", return_value=process) as mock_exec:
|
||||
with asyncio_patch("gns3server.compute.vpcs.vpcs_vm.VPCSVM.start_wrap_console"):
|
||||
await vm.start()
|
||||
return mock_exec, commands
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_start_tap_datapath(vm):
|
||||
"""
|
||||
A -n-capable vpcs build on a tap-capable uBridge anchors the adapter on
|
||||
a persistent TAP and speaks tap mode: -e -d <anchor> -n, no UDP tunnel.
|
||||
"""
|
||||
|
||||
mock_exec, commands = await _start_on_tap_datapath(vm)
|
||||
assert vm._tap_datapath is True
|
||||
assert vm._kernel_taps == {(0, 0): "gp00010203e0p0"}
|
||||
assert mock_exec.call_args[0] == (
|
||||
vm._vpcs_path(),
|
||||
"-p",
|
||||
str(vm._internal_console_port),
|
||||
"-m",
|
||||
"1",
|
||||
"-i",
|
||||
"1",
|
||||
"-F",
|
||||
"-R",
|
||||
"-e",
|
||||
"-d",
|
||||
"gp00010203e0p0",
|
||||
"-n",
|
||||
)
|
||||
# the anchor was created through uBridge's tap module, owned by this user
|
||||
assert 'tap create "gp00010203e0p0"' in commands
|
||||
assert f"tap set_owner gp00010203e0p0 {os.getuid()}" in commands
|
||||
# no local UDP tunnel on this datapath
|
||||
assert vm._local_udp_tunnel is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_start_legacy_datapath_when_binary_lacks_dash_n(vm):
|
||||
"""
|
||||
A vpcs build without -n cannot be kept away from the host side of the
|
||||
anchor, so the node keeps the legacy UDP relay datapath.
|
||||
"""
|
||||
|
||||
process = MagicMock()
|
||||
process.returncode = None
|
||||
with patch("gns3server.compute.vpcs.vpcs_vm.probe_vpcs_tap", new=AsyncioMagicMock(return_value=False)):
|
||||
with asyncio_patch("gns3server.compute.vpcs.vpcs_vm.VPCSVM._check_requirements", return_value=True):
|
||||
with asyncio_patch("asyncio.create_subprocess_exec", return_value=process) as mock_exec:
|
||||
with asyncio_patch("gns3server.compute.vpcs.vpcs_vm.VPCSVM.start_wrap_console"):
|
||||
await vm.start()
|
||||
|
||||
assert vm._tap_datapath is False
|
||||
assert vm._kernel_taps == {}
|
||||
command = mock_exec.call_args[0]
|
||||
assert "-e" not in command
|
||||
assert "-n" not in command
|
||||
assert "-s" in command and "-c" in command and "-t" in command
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_stop_terminates_process_before_deleting_taps(vm):
|
||||
"""
|
||||
The vpcs process holds the anchor TAP open: a held device refuses
|
||||
deletion, so the TAP delete must follow the process termination (the
|
||||
suppressed delete of the old order leaked the anchor).
|
||||
"""
|
||||
|
||||
calls = []
|
||||
process = MagicMock()
|
||||
process.returncode = None
|
||||
process.terminate.side_effect = lambda: calls.append("terminate")
|
||||
|
||||
async def record_send(command):
|
||||
commands.append(command)
|
||||
|
||||
commands = []
|
||||
vm._ubridge_send = AsyncioMagicMock(side_effect=record_send)
|
||||
with patch("gns3server.compute.vpcs.vpcs_vm.probe_vpcs_tap", new=AsyncioMagicMock(return_value=True)):
|
||||
with asyncio_patch("gns3server.compute.vpcs.vpcs_vm.VPCSVM._check_requirements", return_value=True):
|
||||
with asyncio_patch("asyncio.create_subprocess_exec", return_value=process):
|
||||
with asyncio_patch("gns3server.compute.vpcs.vpcs_vm.VPCSVM.start_wrap_console"):
|
||||
await vm.start()
|
||||
|
||||
with asyncio_patch("gns3server.utils.asyncio.wait_for_process_termination"):
|
||||
await vm.stop()
|
||||
|
||||
assert calls == ["terminate"]
|
||||
# every tap delete (the start-time sweep and the stop-time teardown)
|
||||
# happened after the process termination
|
||||
assert all(calls.index("terminate") < commands.index(c) for c in commands if c.startswith("tap delete"))
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_nio_binding_duplicate_port_rejected(vm):
|
||||
"""
|
||||
A port carries at most one NIO: overwriting the one a link still owns
|
||||
orphans that link's teardown.
|
||||
"""
|
||||
|
||||
nio = VPCS.instance().create_nio(
|
||||
{"type": "nio_udp", "lport": 4242, "rport": 4243, "rhost": "127.0.0.1", "filters": {}}
|
||||
)
|
||||
await vm.port_add_nio_binding(0, nio)
|
||||
with pytest.raises(VPCSError, match="already has a link"):
|
||||
await vm.port_add_nio_binding(0, nio)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_kernel_nio_on_legacy_datapath_rejected(vm):
|
||||
"""
|
||||
A kernel link to a node running on the legacy relay datapath (no anchor)
|
||||
is refused with a clear error instead of a late uBridge failure.
|
||||
"""
|
||||
|
||||
vm._process = MagicMock()
|
||||
vm._process.returncode = None
|
||||
vm._tap_datapath = False
|
||||
nio = VPCS.instance().create_nio({"type": "nio_bridge", "bridge": "gns3testbridge", "filters": {}})
|
||||
with pytest.raises(VPCSError, match="legacy relay datapath"):
|
||||
await vm.port_add_nio_binding(0, nio)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_kernel_nio_attach_enslaves_the_anchor(vm):
|
||||
"""
|
||||
A kernel link enslaves the anchor into the per-link bridge and installs
|
||||
the impairment filters the NIO carries — the same _kernel_attach flow
|
||||
every anchor-backed node type uses.
|
||||
"""
|
||||
|
||||
await _start_on_tap_datapath(vm)
|
||||
nio = VPCS.instance().create_nio({"type": "nio_bridge", "bridge": "gns3testbridge", "filters": {"delay": [100]}})
|
||||
await vm.port_add_nio_binding(0, nio)
|
||||
|
||||
sent = [call[0][0] for call in vm._ubridge_send.call_args_list]
|
||||
assert 'brctl create "gns3testbridge"' in sent
|
||||
assert 'brctl addif "gns3testbridge" "gp00010203e0p0"' in sent
|
||||
assert any(c.startswith('tc netem set "gp00010203e0p0" delay 100') for c in sent)
|
||||
|
||||
@ -123,10 +123,11 @@ async def _link(project, node1, node2):
|
||||
@pytest.mark.asyncio
|
||||
async def test_kernel_datapath_not_eligible_for_other_node_types(project):
|
||||
"""
|
||||
A node type with no anchor of its own (VPCS, cloud, ...) keeps the relay
|
||||
however capable the compute is — and a QEMU node on a compute whose
|
||||
uBridge cannot create persistent TAPs does too, because it runs on the
|
||||
legacy relay datapath there.
|
||||
A node type with no anchor of its own (cloud, ...) keeps the relay
|
||||
however capable the compute is — a QEMU node on a compute whose uBridge
|
||||
cannot create persistent TAPs does too (it runs on the legacy relay
|
||||
datapath there), and so does a VPCS node whose binary cannot anchor
|
||||
(no -n support, so no vpcs_tap capability).
|
||||
"""
|
||||
|
||||
compute = _tap_capable_compute(tap_support=None)
|
||||
@ -227,6 +228,77 @@ async def test_dynamips_eligibility_follows_the_tap_capability(project):
|
||||
assert link._kernel_datapath_eligible(node1, node2) is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_kernel_datapath_eligible_for_vpcs_with_tap_support(project):
|
||||
"""
|
||||
A VPCS node anchors on a persistent TAP the vpcs process attaches to
|
||||
(-e -d), which needs the tap module plus a vpcs build with -n — both
|
||||
capabilities reported, and mixable with docker/qemu/iou/dynamips
|
||||
endpoints on the same compute.
|
||||
"""
|
||||
|
||||
compute = MagicMock()
|
||||
compute.id = "compute-1"
|
||||
compute.capabilities = {"ubridge_tap": True, "vpcs_tap": True, "ubridge_iol_tap": True}
|
||||
node1 = _node(project, compute, "pc1", node_type="vpcs")
|
||||
node2 = _node(project, compute, "pc2", node_type="vpcs")
|
||||
link = await _link(project, node1, node2)
|
||||
assert link._kernel_datapath_eligible(node1, node2) is True
|
||||
|
||||
for peer in (
|
||||
_node(project, compute, "docker1"),
|
||||
_node(project, compute, "qemu1", node_type="qemu"),
|
||||
_node(project, compute, "iou1", node_type="iou"),
|
||||
_node(project, compute, "r1", node_type="dynamips"),
|
||||
):
|
||||
link = await _link(project, node1, peer)
|
||||
assert link._kernel_datapath_eligible(node1, peer) is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_vpcs_eligible_for_the_switch_fast_path(project):
|
||||
"""
|
||||
A switch link absorbs the peer's anchor into the switch's own bridge,
|
||||
so only the peer must be able to anchor — a capable VPCS does (the
|
||||
controller names its gp anchor with the shared contract), an
|
||||
incapable one stays on the switch's relay port.
|
||||
"""
|
||||
|
||||
switch = _node(project, _tap_capable_compute(), "sw1", node_type="ethernet_switch")
|
||||
|
||||
compute = MagicMock()
|
||||
compute.id = "compute-1"
|
||||
compute.capabilities = {"ubridge_tap": True, "vpcs_tap": True}
|
||||
pc = _node(project, compute, "pc1", node_type="vpcs")
|
||||
link = await _link(project, switch, pc)
|
||||
assert link._kernel_datapath_eligible(switch, pc) is True
|
||||
|
||||
incapable = MagicMock()
|
||||
incapable.id = "compute-1"
|
||||
incapable.capabilities = {"ubridge_tap": True}
|
||||
pc = _node(project, incapable, "pc1", node_type="vpcs")
|
||||
link = await _link(project, switch, pc)
|
||||
assert link._kernel_datapath_eligible(switch, pc) is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_vpcs_eligibility_needs_both_capabilities(project):
|
||||
"""
|
||||
The tap module and the vpcs -n build land independently: a compute
|
||||
reporting only one of them keeps VPCS on the relay, and so does an
|
||||
unreported/failed probe (strict True on both).
|
||||
"""
|
||||
|
||||
node2 = _node(project, _tap_capable_compute(), "docker1")
|
||||
for capabilities in ({"vpcs_tap": True}, {"ubridge_tap": True, "vpcs_tap": None}, {"ubridge_tap": True}):
|
||||
compute = MagicMock()
|
||||
compute.id = "compute-1"
|
||||
compute.capabilities = capabilities
|
||||
node1 = _node(project, compute, "pc1", node_type="vpcs")
|
||||
link = await _link(project, node1, node2)
|
||||
assert link._kernel_datapath_eligible(node1, node2) is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_kernel_datapath_not_eligible_for_dynamips_serial_ports(project):
|
||||
"""
|
||||
|
||||
@ -235,6 +235,21 @@ class Compute:
|
||||
},
|
||||
)
|
||||
|
||||
def create_vpcs_node(self, project_id, name, startup_script=None):
|
||||
"""A VPCS node: the real vpcs binary on this host behind the server's
|
||||
TCP console (the driver sends ``ip``/``ping`` like a user would). A
|
||||
kernel-datapath scenario additionally needs a build with ``-n``
|
||||
(no host auto-configuration) — see GNS3_E2E_VPCS and the ``vpcs_tap``
|
||||
capability."""
|
||||
properties = {"console_type": "telnet"}
|
||||
if startup_script is not None:
|
||||
properties["startup_script"] = startup_script
|
||||
return self.call(
|
||||
"POST",
|
||||
f"/projects/{project_id}/nodes",
|
||||
{"compute_id": "local", "name": name, "node_type": "vpcs", "properties": properties},
|
||||
)
|
||||
|
||||
def create_link(self, project_id, a, b):
|
||||
return self.call(
|
||||
"POST",
|
||||
@ -298,6 +313,9 @@ allocate_aux_console_ports = False
|
||||
mmap_support = False
|
||||
sparse_memory_support = False
|
||||
ghost_ios_support = False
|
||||
|
||||
[VPCS]
|
||||
vpcs_path = {vpcs}
|
||||
"""
|
||||
|
||||
|
||||
@ -374,8 +392,17 @@ def _spawn_isolated(tmpdir, kernel):
|
||||
|
||||
os.makedirs(os.path.join(tmpdir, "projects"), exist_ok=True)
|
||||
port = _free_port()
|
||||
# GNS3_E2E_VPCS points at a vpcs build with -n (tap mode without host
|
||||
# auto-configuration) — what the kernel-datapath scenario needs. The
|
||||
# local fork build (where this feature is developed) is the default when
|
||||
# present; otherwise the stock binary in PATH keeps the relay scenarios
|
||||
# running (the kernel scenario skips itself on the vpcs_tap capability).
|
||||
vpcs = os.environ.get("GNS3_E2E_VPCS")
|
||||
if not vpcs:
|
||||
fork_build = os.path.expanduser("~/myCode/GNS3/vpcs/src/vpcs")
|
||||
vpcs = fork_build if os.path.isfile(fork_build) else (shutil.which("vpcs") or "vpcs")
|
||||
config = ISOLATED_CONFIG.format(
|
||||
port=port, ubridge=ubridge, dynamips=dynamips, images=images, tmp=tmpdir, kernel=kernel
|
||||
port=port, ubridge=ubridge, dynamips=dynamips, images=images, tmp=tmpdir, kernel=kernel, vpcs=vpcs
|
||||
)
|
||||
config_path = os.path.join(tmpdir, "gns3_server.conf")
|
||||
with open(config_path, "w") as f:
|
||||
@ -702,6 +729,13 @@ def ios_console(server, project_id, node_id):
|
||||
return IOSConsole(host, node["console"])
|
||||
|
||||
|
||||
def vpcs_console(server, project_id, node_id):
|
||||
"""The console of a VPCS node: the vpcs binary's own TCP daemon behind
|
||||
the server-side console — same dumb driver (its prompt ``PC1>`` matches
|
||||
the shared prompt regex)."""
|
||||
return ios_console(server, project_id, node_id)
|
||||
|
||||
|
||||
def configure_ios(
|
||||
console, hostname, eth_ip=None, eth_ipv6=None, serial_ip=None, clock=False, eth_if="f1/0", serial_if="s2/0"
|
||||
):
|
||||
@ -769,6 +803,34 @@ def wait_ping(console, target, attempts=4, repeat=3, timeout=1):
|
||||
return result
|
||||
|
||||
|
||||
def vpcs_ping(console, target, count=5, timeout=1):
|
||||
"""Run a VPCS ping; returns {success, avg, raw} (success in percent).
|
||||
The reply lines are ``<host> icmp_seq=<n> ttl=<t> time=<ms> ms`` in both
|
||||
families (VPCS prints the same shape for v4 and v6 targets)."""
|
||||
|
||||
out = console.run(f"ping {target} -c {count} -w {int(timeout * 1000)}", timeout=count * timeout + 25)
|
||||
replies = re.findall(r"icmp_seq=\d+ ttl=\d+ time=([\d.]+) ms", out)
|
||||
return {
|
||||
"success": int(100 * len(replies) / count),
|
||||
"avg": round(sum(float(rtt) for rtt in replies) / len(replies), 1) if replies else None,
|
||||
"raw": out,
|
||||
}
|
||||
|
||||
|
||||
def wait_vpcs_ping(console, target, attempts=4, count=5, timeout=1):
|
||||
"""The VPCS twin of wait_ping: the first exchange of a fresh adjacency
|
||||
is ARP (a lost first ping is legitimate), so retry until it fully
|
||||
succeeds; returns the last result either way."""
|
||||
|
||||
result = None
|
||||
for _ in range(attempts):
|
||||
result = vpcs_ping(console, target, count=count, timeout=timeout)
|
||||
if result["success"] == 100:
|
||||
return result
|
||||
time.sleep(1.5)
|
||||
return result
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Docker guest driver (Docker Engine API over the daemon socket)
|
||||
# ---------------------------------------------------------------------------
|
||||
@ -1330,6 +1392,13 @@ def qemu_anchor_name(node_id, adapter, port=0):
|
||||
return "gq" + node_id.replace("-", "")[:8] + f"e{adapter}p{port}"
|
||||
|
||||
|
||||
def vpcs_anchor_name(node_id, adapter=0, port=0):
|
||||
"""The VPCS node's anchor TAP name: one persistent TAP for the single
|
||||
adapter, attached to by name through -e -d (see VPCSVM._tap_name and
|
||||
utils.kernel_anchor)."""
|
||||
return "gp" + node_id.replace("-", "")[:8] + f"e{adapter}p{port}"
|
||||
|
||||
|
||||
def stage_iol_base_config(server):
|
||||
"""
|
||||
Copy the shipped iol-xe base config into an isolated instance's configs
|
||||
|
||||
260
tests/e2e/test_vpcs_kernel_datapath.py
Normal file
260
tests/e2e/test_vpcs_kernel_datapath.py
Normal file
@ -0,0 +1,260 @@
|
||||
#
|
||||
# Copyright (C) 2026 GNS3 Technologies Inc.
|
||||
# This program is free software and is redistributed under the same terms as gns3-server.
|
||||
# See the LICENSE file for licensing information.
|
||||
#
|
||||
|
||||
"""
|
||||
Live end-to-end test for the VPCS node's kernel datapath (pytest marker
|
||||
``e2e``): two real VPCS nodes (the local vpcs binary, its own TCP console
|
||||
driven like a user would), linked through the controller REST API. VPCS is
|
||||
the node type whose guest leg attaches to a *pre-created* anchor: uBridge
|
||||
creates the persistent TAP and hands it to this user, and the vpcs process
|
||||
opens it by name (``-e -d``) while ``-n`` keeps its hands off the host side
|
||||
of the anchor — the same server-creates/guest-attaches shape as QEMU.
|
||||
|
||||
* two VPCS nodes on one compute report ``kernel_datapath`` when the
|
||||
compute's uBridge can create persistent TAPs (``ubridge_tap``) and the
|
||||
vpcs build has ``-n`` (``vpcs_tap``); the anchor (``gp`` TAP) exists from
|
||||
node start, before any link, and the per-link bridge enslaves exactly the
|
||||
two of them;
|
||||
* real ICMP crosses the [vpcs tun fd ↔ anchor TAP] leg;
|
||||
* the L2-anchor spec §E.1: with ``-n`` the anchor carries no address — vpcs
|
||||
never ran its ``ifconfig`` on it; §E.2 comes free: a VPCS sends nothing
|
||||
spontaneously (no CDP, no keepalive), so the silence window needs no
|
||||
guest shutdown;
|
||||
* a ``delay 100`` filter lands as netem on both anchors, RTT grows ~200 ms;
|
||||
* suspend admin-downs the anchor and kills the traffic; resume restores;
|
||||
* a capture on the kernel link writes a real pcap;
|
||||
* link delete releases the anchor from the bridge and it survives (the
|
||||
running vpcs holds it open) — re-creating the link re-enslaves it;
|
||||
* node stop terminates vpcs first, then deletes the anchor (a held TAP
|
||||
refuses deletion — the order is the point); a restart recreates the TAP
|
||||
and rewires the link from the NIO;
|
||||
* deleting the project leaves no anchors and no bridges behind.
|
||||
|
||||
``test_vpcs_relay_control`` is the negative control on an isolated
|
||||
relay-configured instance: nothing is enslaved, no per-link bridge exists,
|
||||
and the relay still carries the pings — with the legacy UDP tunnel on a
|
||||
stock vpcs, or as an AF_PACKET endpoint on the anchor with a ``-n`` build
|
||||
(the datapath a link rides is a server configuration, not a node property).
|
||||
"""
|
||||
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
from tests.e2e import harness
|
||||
|
||||
pytestmark = pytest.mark.e2e
|
||||
|
||||
# VPCS has one adapter with one port; the kernel scenario addresses it.
|
||||
ETH = (0, 0)
|
||||
|
||||
PC1_IP = "10.1.1.1"
|
||||
PC2_IP = "10.1.1.2"
|
||||
|
||||
|
||||
def _configure(console, ip):
|
||||
console.run("set pcname E2E-PC")
|
||||
out = console.run(f"ip {ip}/24")
|
||||
assert ip in out, out
|
||||
|
||||
|
||||
def test_vpcs_kernel_datapath():
|
||||
server = harness.live_server(kernel=True)
|
||||
compute = server.compute
|
||||
caps = compute.capabilities()
|
||||
if caps.get("ubridge_tap") is not True:
|
||||
pytest.skip(f"this compute's uBridge cannot create persistent TAPs (ubridge_tap={caps.get('ubridge_tap')})")
|
||||
if caps.get("vpcs_tap") is not True:
|
||||
pytest.skip(
|
||||
f"this compute's vpcs binary cannot anchor a kernel link (vpcs_tap={caps.get('vpcs_tap')}); "
|
||||
"point GNS3_E2E_VPCS (or the server's VPCS.vpcs_path) at a vpcs build with -n"
|
||||
)
|
||||
|
||||
project = compute.create_project("vpcs-e2e")
|
||||
pid = project["project_id"]
|
||||
a1 = a2 = None
|
||||
try:
|
||||
n1 = compute.create_vpcs_node(pid, "E2E-PC1")
|
||||
n2 = compute.create_vpcs_node(pid, "E2E-PC2")
|
||||
n1_id, n2_id = n1["node_id"], n2["node_id"]
|
||||
a1 = harness.vpcs_anchor_name(n1_id, *ETH)
|
||||
a2 = harness.vpcs_anchor_name(n2_id, *ETH)
|
||||
|
||||
# The link is created while both nodes are stopped: the kernel NIO
|
||||
# is bound with no anchors in existence yet and wired by node start
|
||||
# (the deferred-wiring path, same as every anchor-backed node).
|
||||
link = compute.create_link(pid, (n1_id, *ETH), (n2_id, *ETH))
|
||||
lid = link["link_id"]
|
||||
bridge = harness.link_bridge_name(lid)
|
||||
assert link["kernel_datapath"] is True, link
|
||||
assert not harness.tap_exists(a1), "anchors must not exist before start"
|
||||
|
||||
compute.call("POST", f"/projects/{pid}/nodes/{n1_id}/start")
|
||||
compute.call("POST", f"/projects/{pid}/nodes/{n2_id}/start")
|
||||
assert harness.wait_until(lambda: harness.tap_exists(a1) and harness.tap_exists(a2), timeout=15), (a1, a2)
|
||||
c1 = harness.vpcs_console(server, pid, n1_id)
|
||||
c2 = harness.vpcs_console(server, pid, n2_id)
|
||||
|
||||
# NetworkManager can release an NM-managed TAP from its bridge after
|
||||
# (re-)enslavement (docs/bugs/networkmanager-tap-release.md); take
|
||||
# the anchors out of its reach for the rest of the run (the restart
|
||||
# below re-attaches them under a fresh device).
|
||||
harness.unmanage_from_networkmanager(a1, a2)
|
||||
|
||||
# Kernel objects: one persistent TAP per node, and the per-link
|
||||
# bridge has exactly the two anchors as members.
|
||||
assert harness.bridge_members(bridge) == sorted([a1, a2]), harness.bridge_members(bridge)
|
||||
|
||||
# L2-anchor spec §E.1: with -n the anchor stays address-free — this
|
||||
# is the assertion the -n flag exists for (an upstream vpcs would
|
||||
# have ifconfig'd the anchor on the guest's `ip` command below).
|
||||
if harness.l2only_supported():
|
||||
harness.assert_pure_l2(a1)
|
||||
harness.assert_pure_l2(a2)
|
||||
harness.assert_pure_l2(bridge)
|
||||
else:
|
||||
print(".. uBridge without link l2only: skipping the §E.1 assertions")
|
||||
harness.assert_forwarding(a1)
|
||||
harness.assert_forwarding(a2)
|
||||
|
||||
_configure(c1, PC1_IP)
|
||||
_configure(c2, PC2_IP)
|
||||
baseline = harness.wait_vpcs_ping(c1, PC2_IP)
|
||||
assert baseline["success"] == 100, baseline["raw"]
|
||||
reverse = harness.wait_vpcs_ping(c2, PC1_IP)
|
||||
assert reverse["success"] == 100, reverse["raw"]
|
||||
|
||||
# L2-anchor spec §E.2: a VPCS speaks only when spoken to (no
|
||||
# platform chatter), so the silence window needs no guest shutdown —
|
||||
# the two anchors and the bridge must be quiet 2.5 s after the pings.
|
||||
if harness.l2only_supported():
|
||||
harness.assert_idle_silence(a1, a2, bridge)
|
||||
else:
|
||||
print(".. uBridge without link l2only: skipping the §E.2 assertion")
|
||||
|
||||
# delay 100: netem on both anchors (each impairs one direction),
|
||||
# one-way ~100 ms => RTT grows by ~200 ms
|
||||
compute.call("PUT", f"/projects/{pid}/links/{lid}", {"filters": {"delay": [100]}})
|
||||
assert "netem" in harness.qdiscs(a1) and "netem" in harness.qdiscs(a2), (
|
||||
harness.qdiscs(a1),
|
||||
harness.qdiscs(a2),
|
||||
)
|
||||
delayed = harness.wait_vpcs_ping(c1, PC2_IP)
|
||||
assert delayed["success"] == 100, delayed["raw"]
|
||||
assert delayed["avg"] >= 150, (baseline, delayed)
|
||||
compute.call("PUT", f"/projects/{pid}/links/{lid}", {"filters": {}})
|
||||
assert "netem" not in harness.qdiscs(a1), harness.qdiscs(a1)
|
||||
fast = harness.wait_vpcs_ping(c1, PC2_IP)
|
||||
assert fast["success"] == 100 and fast["avg"] < harness.FAST_RTT_MS, fast
|
||||
|
||||
# suspend: anchor admin-down — the tun fd writes go nowhere and
|
||||
# nothing crosses; resume restores
|
||||
compute.call("PUT", f"/projects/{pid}/links/{lid}", {"suspend": True})
|
||||
assert not harness.tap_up(a1)
|
||||
dead = harness.vpcs_ping(c1, PC2_IP, count=2, timeout=0.5)
|
||||
assert dead["success"] == 0, dead["raw"]
|
||||
compute.call("PUT", f"/projects/{pid}/links/{lid}", {"suspend": False})
|
||||
assert harness.tap_up(a1)
|
||||
assert harness.wait_vpcs_ping(c1, PC2_IP, attempts=5)["success"] == 100
|
||||
|
||||
# capture: on the anchor, a real pcap
|
||||
capture = compute.call("POST", f"/projects/{pid}/links/{lid}/capture/start", {"data_link_type": "DLT_EN10MB"})
|
||||
harness.vpcs_ping(c1, PC2_IP, count=3)
|
||||
time.sleep(1)
|
||||
compute.call("POST", f"/projects/{pid}/links/{lid}/capture/stop")
|
||||
count, ethertypes = harness.pcap_records(capture["capture_file_path"])
|
||||
assert count >= 4, f"{capture['capture_file_path']}: {count} records"
|
||||
assert "0800" in ethertypes, ethertypes
|
||||
|
||||
# delete / re-create the kernel link: the anchor leaves the bridge
|
||||
# but survives (the running vpcs holds it open), and re-creating
|
||||
# re-enslaves it without touching the process.
|
||||
compute.call("DELETE", f"/projects/{pid}/links/{lid}")
|
||||
assert harness.bridge_members(bridge) is None
|
||||
assert harness.tap_exists(a1) and harness.tap_exists(a2)
|
||||
link = compute.create_link(pid, (n1_id, *ETH), (n2_id, *ETH))
|
||||
lid = link["link_id"]
|
||||
bridge = harness.link_bridge_name(lid)
|
||||
assert link["kernel_datapath"] is True, link
|
||||
assert harness.wait_vpcs_ping(c1, PC2_IP, attempts=5)["success"] == 100
|
||||
|
||||
# node stop: vpcs is terminated first and only then the anchor is
|
||||
# deleted (the process holds the TAP open — a held device refuses
|
||||
# deletion and would leak); a restart recreates the TAP, rewires the
|
||||
# link from the NIO that outlived it, and the reconfigured VPCS
|
||||
# pings again.
|
||||
print(".. stopping and restarting PC1")
|
||||
compute.call("POST", f"/projects/{pid}/nodes/{n1_id}/stop")
|
||||
assert harness.wait_until(lambda: not harness.tap_exists(a1), timeout=15), a1
|
||||
compute.call("POST", f"/projects/{pid}/nodes/{n1_id}/start")
|
||||
assert harness.wait_until(lambda: harness.tap_exists(a1), timeout=15), a1
|
||||
assert harness.bridge_members(bridge) == sorted([a1, a2]), harness.bridge_members(bridge)
|
||||
c1.close()
|
||||
c1 = harness.vpcs_console(server, pid, n1_id)
|
||||
_configure(c1, PC1_IP)
|
||||
assert harness.wait_vpcs_ping(c1, PC2_IP, attempts=5)["success"] == 100
|
||||
|
||||
c1.close()
|
||||
c2.close()
|
||||
except BaseException:
|
||||
harness.release(server, pid, failed=True)
|
||||
raise
|
||||
harness.release(server, pid, failed=False)
|
||||
|
||||
# deleting the project closed the nodes: anchors and bridges go with them
|
||||
assert harness.wait_until(lambda: not harness.tap_exists(a1) and not harness.tap_exists(a2), timeout=15), (
|
||||
a1,
|
||||
a2,
|
||||
)
|
||||
|
||||
|
||||
def test_vpcs_relay_control():
|
||||
"""
|
||||
Negative control: the same topology with ``enable_kernel_datapath =
|
||||
false`` (an isolated instance) wires nothing into the kernel — no
|
||||
per-link bridge, no enslavement — and still pings, with the relay
|
||||
filters served by the uBridge bridge engine.
|
||||
"""
|
||||
|
||||
server = harness.live_server(kernel=False)
|
||||
compute = server.compute
|
||||
|
||||
project = compute.create_project("vpcs-relay-e2e")
|
||||
pid = project["project_id"]
|
||||
try:
|
||||
n1 = compute.create_vpcs_node(pid, "E2E-PC1")
|
||||
n2 = compute.create_vpcs_node(pid, "E2E-PC2")
|
||||
n1_id, n2_id = n1["node_id"], n2["node_id"]
|
||||
|
||||
link = compute.create_link(pid, (n1_id, *ETH), (n2_id, *ETH))
|
||||
lid = link["link_id"]
|
||||
assert link["kernel_datapath"] is False, link
|
||||
|
||||
compute.call("POST", f"/projects/{pid}/nodes/{n1_id}/start")
|
||||
compute.call("POST", f"/projects/{pid}/nodes/{n2_id}/start")
|
||||
c1 = harness.vpcs_console(server, pid, n1_id)
|
||||
c2 = harness.vpcs_console(server, pid, n2_id)
|
||||
_configure(c1, PC1_IP)
|
||||
_configure(c2, PC2_IP)
|
||||
|
||||
# no kernel wiring: the per-link bridge does not exist
|
||||
assert harness.bridge_members(harness.link_bridge_name(lid)) is None
|
||||
relay_ping = harness.wait_vpcs_ping(c1, PC2_IP)
|
||||
assert relay_ping["success"] == 100, relay_ping["raw"]
|
||||
|
||||
# the relay filters keep working: one uBridge bridge filters both
|
||||
# directions, delay 100 => RTT >= 150 ms
|
||||
compute.call("PUT", f"/projects/{pid}/links/{lid}", {"filters": {"delay": [100]}})
|
||||
delayed = harness.wait_vpcs_ping(c1, PC2_IP)
|
||||
assert delayed["success"] == 100, delayed["raw"]
|
||||
assert delayed["avg"] >= 150, delayed
|
||||
|
||||
c1.close()
|
||||
c2.close()
|
||||
except BaseException:
|
||||
harness.release(server, pid, failed=True)
|
||||
raise
|
||||
harness.release(server, pid, failed=False)
|
||||
@ -33,6 +33,7 @@ def test_every_node_type_has_its_own_prefix():
|
||||
assert kernel_anchor_name("iou", NODE_ID, 0, 3) == "gi00010203e0p3"
|
||||
assert kernel_anchor_name("dynamips", NODE_ID, 6, 48) == "gd00010203e6p48"
|
||||
assert kernel_anchor_name("iol_docker", NODE_ID, 0, 0) == "gx00010203e0p0"
|
||||
assert kernel_anchor_name("vpcs", NODE_ID, 0, 0) == "gp00010203e0p0"
|
||||
|
||||
|
||||
def test_anchor_type_keys_iol_runner_containers_apart():
|
||||
@ -55,7 +56,7 @@ def test_anchor_type_keys_iol_runner_containers_apart():
|
||||
|
||||
def test_unknown_node_types_have_no_anchor():
|
||||
assert kernel_anchor_name("ethernet_switch", NODE_ID, 0, 0) is None
|
||||
assert kernel_anchor_name("vpcs", NODE_ID, 0, 0) is None
|
||||
assert kernel_anchor_name("cloud", NODE_ID, 0, 0) is None
|
||||
|
||||
|
||||
def test_names_fit_ifnamsiz():
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user