Review feedback: 3.1 is still in alpha and the GNS3 VM ships a matching
uBridge, so a hard minimum is simpler than a version-based degrade path,
and it also covers Docker endpoints and the builtin Ethernet switch,
which never went through the anchor probes. Raise minimum_required_version
to 1.3.1 and drop supports_kernel_datapath and its uses; the relay
fallback for capability problems (no CAP_NET_ADMIN) is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The TAP netdev kept QEMU's default vnet_hdr, so virtio-net, e1000e, igb
and vmxnet3 guests offload TCP/UDP checksums (and TSO) to the TAP: their
frames carry only the pseudo-header sum in the checksum field. A kernel
bridge link finishes them, but the uBridge relay (cloud, NAT, hub, ...)
reads the TAP with libpcap and forwards the frames as they are, so DNS
and TCP through a cloud fail while ICMP and DHCP work.
With vnet_hdr=off QEMU stops offering the offloads and the guest (or the
emulated NIC) sends finished frames, as with the 2.x socket netdev.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A server killed before close() leaves the switch's kernel bridge behind
with its ports still enslaved: a peer's persistent anchor TAP and cascade
veth ends outlive uBridge. uBridge's brctl delete refuses a bridge that
still has ports (EBUSY), so the best-effort recovery delete in
_ensure_bridge() failed silently and brctl create hit EEXIST: the switch,
and the project holding it, could no longer be opened (HTTP 409 "Could
not create bridge gns3xxxxxx: File exists").
List the leftover bridge's ports from /sys/class/net/<bridge>/brif and
release each with brctl delif before the delete. uBridge's brctl show
reports no ports, and the switch already reads sysfs on the same host.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
uBridge 1.2.3 already has the tap module, so the persistent-TAP probes
(_probe_persistent_taps on the node, probe_tap_support and siblings for
/capabilities) pass on it. QEMU, Dynamips and VPCS nodes then anchor on
TAPs and the controller builds kernel links that need 1.3.x commands:
kernel markers and the eBPF tc modes (1.3.0) and per-interface kernel
captures (`capture stop_kernel <ifc>`, 1.3.1). On 1.2.3 a MikroTik CHR
node sent frames but never got DHCP past DISCOVER/OFFER, and stopping
a capture failed with "Bad number of parameters".
Add Hypervisor.supports_kernel_datapath (version >= 1.3.1) and refuse
older builds in the node-side probe and the four anchoring capability
probes (tap, iol tap, bridge tap, vxlan). The existing fallback then
applies: nodes keep the legacy relay datapath with the usual warning,
which now names the installed version, and links stay on the relay.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When only a port is renamed in a template's custom adapters, the adapter is
saved with "adapter_type": null. The emulators read it with
custom_adapter.get("adapter_type", self._adapter_type), which returns None
for a key set to null: QEMU was then started with "-device None,..." and
died ("'None' is not a valid device model name"), GNS3 only reporting
"Could not start Telnet QEMU console". VirtualBox and VMware read the
adapter type the same way.
Drop the null settings in _get_custom_adapter_settings so the node's own
adapter type (and MAC address) applies.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the compute is itself a Hyper-V guest without an invariant TSC (e.g.
the GNS3 VM on hosts whose CPU lacks TSC scaling), Linux falls back to the
hyperv_clocksource_tsc_page clocksource and the guest timekeeping KVM
derives from the TSC is broken with recent kernels:
- kvmclock runs thousands of times too fast: Linux guests (MikroTik
RouterOS, ...) panic with "watchdog: BUG: soft lockup - CPU#0 stuck for
29604s" and reboot in a loop right after boot;
- TSC deadline timers fire far too late: guests with -cpu host hang.
The only workaround so far was -machine accel=tcg, i.e. no KVM at all.
Add a kvmclock setting to the [Qemu] section (auto, enabled, disabled).
In auto mode, KVM guests started on a host whose clocksource is a Hyper-V
one get kvmclock and tsc-deadline turned off: on the -cpu given in the
additional options (unless it sets these features itself), otherwise on
QEMU's default CPU model.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Add per-drive IDE/SATA selection with backward-compatible IDE defaults
- Persist interfaces in nodes, templates, and appliance imports
- Add database migration support for versioned and unversioned databases
- Prevent partial updates when rejecting live interface changes
- Validate interfaces and support SATA media changes through the monitor
Cross-compute links can ride kernel VXLAN tunnels instead of the uBridge
UDP relay: each endpoint creates one VXLAN device (uBridge's rtnetlink
vxlan module) enslaved into the same per-link Linux bridge the same-host
datapath uses, so filters, markers, capture and suspend keep running on
the anchor unchanged. Opt-in (Server.enable_vxlan_cross_compute, default
off) because the tunnel caps the usable MTU at the underlay minus the
outer headers; the relay stays the default and the fallback.
The controller derives what the pair must agree on: one VNI per link
(upper VNI space, per-link random salt) and deterministic device names
(gvl{link}{node}) that make crashed-run leftovers claimable
(show -> delete -> create). Same-host compute pairs, switch endpoints
and serial/unix-socket containers stay on the relay.
A VNI the kernel refuses self-heals on every path it can surface:
interactive create retries a fresh salt then degrades to the relay;
project open re-dispatches refused batch entries the same way; node
start recognises the refused device, stops the half-started node,
re-wires both ends on a fresh VNI (or degrades the link) and retries
the start. The device MTU is derived from the lower device (dev=) and
the per-link bridge inherits it - no configuration. The capability
probe (ubridge_vxlan in /capabilities) runs one scratch create/delete
on a scratch port that never equals the configured dstport.
The e2e scenario runs two computes on one machine (a network namespace
joined by a veth pair as the fabric) and needs root.
Updating any Docker node property destroys and recreates the container
(the route calls DockerVM.update() as soon as the property diff is
non-empty), so it discards everything written inside it. On a running
node that happened silently, as a side effect of an unrelated edit: the
user only saw the node restart.
Refuse the update while the container is up and say what stopping it is
for, like QEMU, IOU, Dynamips, VPCS, VirtualBox and VMware already do
for their own restart-requiring settings. The check runs before anything
is applied — a refused update leaves the node untouched — and only when
the PUT really changes a property, since Web clients send the whole node
back on every edit.
is_running() is not reusable for the question: it reports a paused
container as not running and stops crashed containers as a side effect,
so DockerVM gains is_up() for "running or paused".
When a project is opened, Docker nodes are created in parallel. The free
display was found by checking whether /tmp/.X11-unix/X<n> exists, but that
socket only appears once Xvnc has started, so several nodes got the same
display. Only the first Xvnc started; the others exited with "Server is
already active for display N" ("The vnc process has stopped with return
code 1"), and their containers drew on the screen of another node.
- reserve the displays given to the nodes of this server, and also skip a
display whose lock file belongs to a live process
- restart a crashed Xvnc on the display the container was created with,
after removing its stale socket, instead of picking a new display the
container cannot reach (black screen)
- on close, only remove the X socket of our own Xvnc, never the socket of
the node that owns the display
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
uBridge's capture module now serves one AF_PACKET capture per interface,
any number per process, and `capture stop_kernel` takes the interface it
stops — this rides that (breaking, paired-binary) protocol change.
The server-side single-slot tracker (_kernel_capture_ifc with its
reserve/release/owned_by helpers) is gone: same-port duplicates were
already refused by the NIO capturing flag, uBridge answers EALREADY on a
same-interface duplicate, and per-interface stops removed the process-
wide stop the ownership gating guarded against.
The docker e2e drives ten concurrent captures on one node's uBridge and
asserts a per-interface stop freezes exactly the stopped half while the
live half keeps recording.
The cloud node offered the kernel datapath's host-side interfaces —
node anchors (gv/gq/gi/gd/gx/gp), switch cascade veth ends (gs) and
cross-compute VXLAN devices (gvl) — in its interface list and even
pre-filled them as ports on a freshly created cloud: none of them is
flagged special and only "gns3*" names were filtered. These interfaces
belong to the link or node that created them for the project's whole
lifetime, so bridging a cloud to one would race its owner.
Add is_kernel_datapath_interface() next to the naming contract in
utils/kernel_anchor.py: exact-shape matching (prefix plus its
fixed-length id-derived tail), so a user interface that merely starts
with "gv" is never swallowed. The cloud keeps managed names out of its
host-interface list and default ports mapping, while saved mappings
keep working (the existence checks still see the full interface list).
Docker ids are not comparable across image stores: for the same content a
classic store reports the config digest while a containerd store (or an
image loaded from an OCI archive) reports the manifest digest. The
controller pins the id of its own daemon, so a compute whose store reports
another id for that content answered ImageMissingError forever: every node
creation streamed the image again (save/load round trip) without ever
converging, then failed with "The image 'x' is missing" after the six
retries. Docker 29 makes this easy to hit, since the id a daemon reports
now depends on how the image entered it.
The load endpoint now takes the tag and the digest pinned by the
controller, and the compute records how its own daemon identifies the
image it just received. Node creation accepts the pin through that pair,
and only while the tag still resolves to the synced content: a tag moved
to some other image, or an image the controller never pushed here, still
counts as missing. The pair lives in the compute process, so a restarted
compute syncs once more.
Computes that predate the new endpoint parameters keep the previous
behaviour, as do controllers that do not send them.
A VPCS node's single adapter anchors on a persistent TAP the server
creates (tap create + set_owner via uBridge) and the vpcs process
attaches to by name: -e -d <gp{id8}e0p0> -n, the same
server-creates/guest-attaches shape as QEMU's netdev. -n (a vpcs fork
feature) keeps the guest's ip command from ifconfig'ing the anchor on
the host; the vpcs build is probed once per binary identity
(vpcs_tap capability, usage scan) and gates kernel links together
with ubridge_tap — anything else keeps the node on the relay (the
legacy local UDP tunnel on a stock binary, an AF_PACKET endpoint on
the anchor with a capable one).
VPCSVM takes KernelDatapathMixin: _connect_nio dispatches kernel
(enslave into the per-link bridge) / relay-on-anchor / legacy,
captures pick capture start_kernel vs the bridge engine, stop runs
under an execute lock and terminates the process before deleting the
anchor TAPs (a held device refuses deletion), and the duplicate-port
guard backstops the controller's. The NIO routes accept nio_bridge
and write suspend back on update.
_delete_anchor_taps verifies each tap is gone from sysfs and retries
a bounded three times: a holder can release its fd a moment after its
process was reaped (vpcs's daemon parent kills its console child and
exits first), so the unconditional delete raced EBUSY and leaked the
anchor intermittently.
The vpcs side needs IFNAMSIZ (16) instead of its own IFNAMESIZ (12)
for the -d name buffer — anchor names are 14 chars and truncated to a
nonexistent device, which TUNSETIFF then tries to create without
CAP_NET_ADMIN (fixed in the vpcs fork).
e2e (tests/e2e/test_vpcs_kernel_datapath.py): kernel and relay
scenarios green against the real binary — real ICMP both ways, §E.1/§
E.2 on the anchors, delay/suspend/capture, link delete/re-create,
node stop/restart self-heal, zero leftovers; the relay control passes
with both the fork binary and a stock one.
Resolve conflicts against the upstream wave (PRs #2952-#2965, incl. the
typed-response/schema hardening and the drop of Python 3.10):
- controller/node.py: keep upstream's _transitioning/_confirm_status
lifecycle guard; the kernel-anchor link wiring (node_started) runs
after the transition completes.
- controller/link.py: keep upstream's add_node rollback (release_ports
on create failure) and delete via release_ports; release_ports now
resolves ports live (same staleness reason as _bind_members) and
discards the node link unconditionally.
- schemas/controller/links.py: extend LinkFilterType with the eight
kernel-datapath filter types and LinkFilterParameter.type with "str"
so the newly typed /available_filters response validates; Link.filters
stays a free-form dict — the vocabulary is capability-gated per link
and validated by the controller (409), not by a strict request schema.
- schemas/controller/topology.py: drop the type: ignore made redundant
by the dict filters field.
- tests: route-level filter validation now expects the controller's 409
(unknown types, out-of-range values); packet_loss correlation is a
valid kernel parameter (positive test added); OpenAPI assertions match
the free-form filters property.
The persistent-TAP lifecycle — probe, sweep, create, register, delete —
was implemented four times, once per node type, and the copies had drifted
(probe mechanism, _tap_datapath reset, keep-vs-recreate policy). The
mechanical half moves into KernelDatapathMixin:
- _probe_persistent_taps(probe_name): the scratch-TAP probe, returning the
refusal for the caller's own warning (QEMU, Dynamips);
- _create_anchor_taps(ports, set_owner, keep_existing): sweep + create +
optional ownership handover + carrier down + registry, with the ports
enumerated by the node type and named through the new _tap_name hook;
- _delete_anchor_taps(): the suppressed tap sweep + per-link bridge
cleanup + registry clear.
Each node type keeps what its fd holder dictates: the release step before
the tap sweep (the QEMU process, the IOL bridge, the hypervisor's NIO
bindings, the container port bridges), its probe policy, and its registry
lifetime. QEMU's now-unconditional registry clear aligns it with the other
three; everything else is behavior-preserving.
Compute:
- Track the anchor that owns uBridge's single AF_PACKET capture slot: a
second kernel port's capture is refused with a clear error (its flag
rolled back), a failed start no longer wedges the port, and only the
owning port's stop issues the process-wide `capture stop_kernel`
(docker/qemu/iou/dynamips/switch, via the mixin's slot helpers).
- Unwind failed starts instead of leaking uBridge and the anchor TAPs:
QEMU's stop body runs under try/finally, QEMU and IOU start the datapath
inside the launch try, Dynamips undoes a half-attached port (the retry
used to be short-circuited into a silent black hole), and Docker deletes
the veth pair when the namespace move fails.
- Restore the relay filter guard on IOU (kernel-only filters raise
NodeError, an uncompilable BPF line degrades to a warning), validate the
eBPF/quota/window/bpf filter shapes before touching the bridge (short or
non-numeric values surfaced as 500s after partial state), and refuse a
second NIO on an occupied Docker/QEMU port (the guard IOU and Dynamips
already had).
- Probe fixes: `_probe_bridge_tap` survives a failing uBridge start, and
the probe caches key on ctime too, so a chmod/setcap remedy re-probes.
Controller:
- Parse GNS3_UNIX_SOCKET_NIO like the compute does (line-anchored, truthy
value) and mirror the compute's class selection in kernel_anchor_type.
- Read partial capability reports defensively; make BridgeNIO.bridge
required-but-nullable so an omitted key is a 422 rather than a KeyError.
- Strip the packet_loss correlation slot even when it is zero: the relay
filter takes exactly one argument and rejected `packet_loss 10 0`.
Ethernet switch: skip ports whose absorbed anchor does not exist yet (the
join is deferred by design) and roll the ports_mapping back when the VLAN
reconcile fails, so a retry still diffs against the old mapping.
Agent: reject device config entries without a device_name instead of
silently dropping them (a malformed batch used to execute partially while
reporting success).
Also split the netem builder to clear C901 and drop the two per-file ruff
ignores the PR had added, keeping the remove-only policy.
The rebase onto 65c858d45 brought upstream's UP032 enforcement; the
conflict resolution kept this branch's semantics on the overlapping
lines, so their .format() calls needed the conversion afterwards
(ruff check --fix, 41 sites) plus the formatter's rewrap (7 files).
Replicates the CI recipe locally: pip check, ruff check/format, mypy
2.3.1 (clean, 434 files), unit suite 2216 passed.
The rebase onto upstream 3.1 (mypy + ruff I001/F401 waves) left the tree
short of the newly enforced baselines in three ways:
* contextlib / UbridgeError imports dropped as false duplicates during
conflict resolution left NameErrors in iou_vm and qemu_vm (F821);
the Dynamips router grabbed the compute-level NIOTAP instead of the
hypervisor one, breaking TAP anchoring (test failure).
* unused imports (KERNEL_UNSUPPORTED_FILTERS, UbridgeError, List) and
unsorted blocks left behind where the stack's later commits moved the
last usage elsewhere — ruff I001/F401/RUF022 now enforce these.
* mypy, now baseline-free: cache dict annotations in tc_probe, the
locators candidate list, the resolved-params None branch in the MCP
runner, and validate_bpf_syntax keeps the upstream Dict[str, Any]
return (the strict Optional[str] was wrong for the "valid" bool entry).
The e2e raw-frame probe resolves its compiler via shutil.which (S607)
and zips with strict= (B905); the remaining subprocess call is S603 by
nature and gets a per-file ignore like other test files.
ruff check/format clean, mypy at upstream parity (no new errors),
full unit suite 2167 passed.
The peer end of an Ethernet switch fast-path link is a NIOBridge with no
bridge of its own (the switch absorbed its anchor into the switch bridge).
Its update reconcile still ran the impairment pass with the empty filter
set the controller routes to that end — and an empty set is a plain
`tc reset` on the shared anchor, silently wiping the netem/cls_bpf/eBPF
the switch had just applied. Any link update did it (filters, markers,
suspend, capture start/stop), and the outcome depended on the link's
endpoint order: UdpLink.update() PUTs node1 then node2, so with the
switch first its apply ran first and the peer's reset ran last (API
green, data plane unimpaired), while with the peer first the reset ran
first and the filter survived. That order dependence is also why the
shipped e2e never caught it — it only created peer-first links.
Ownership is now explicit: only the end that owns the anchor's tc state
applies the impairment reconcile (_owns_anchor_impairments — a NIOBridge
with `bridge is None` is a passive carrier, the same idiom the
bridge-delete path already used). Markers, capture and the suspend
carrier still apply on both ends: markers reconcile per process, so
neither end can strip the other's, and the carrier converges by
agreement.
Red/green: two new docker unit tests fail on the unfixed tree with
`tc reset "<anchor>"` and pass after; the switch e2e also creates one
link switch-first and asserts the netem qdisc and the RTT hold under
that order (failed with the wipe, passes with the fix). Full unit suite
2110 passed; the switch e2e file passed end to end (378s).
A correctness read-through of the five delivered datapath documents, plus
the two code comments that carried the same stale vocabulary.
Datapath selection: the VendorDockerVM selector is `console_type=docker_exec`
alone (`GNS3_SKIP_INIT` and friends are behavior knobs of the chosen class);
the cross-compute and anchor-less fallbacks are named for what they are; the
Ethernet-switch endpoint is the documented exception to the anchoring rule,
and the IOL runner container is called out as the Docker node that does not
anchor on a veth.
Filter vocabulary: the stateful eBPF program evaluates four modes
(nth → quota → window → flow), GNS3 exposes three of them, and window_drop
joins the kernel-only list; the correl suffixes gate on the `rate` token as
the extension-build marker.
group_fwd_mask: the 0xfffd port mask is applied inside uBridge's `brctl addif`
itself, not as a separate command — the link-creation sequences and the
operation tables no longer show an arrow the server never sends. Same class
of fix for `link set up` on the IOL anchor (addif already brought it up; the
command only pushes a suspended anchor back down) and for the QEMU carrier
replication (QMP `set_link`, gated by `replicate_network_connection_state`).
Lifecycles and stop orders re-worded around the invariant rather than the
node type: the fd holder must release before the devices are deleted, which
is QEMU's process-first order, IOU's uBridge-first order and Dynamips'
hypervisor-NIO-first order. Deferred wiring for a stopped endpoint is
documented as the common path it is, and the Dynamips "stop is not a close"
note now says what recovery from a dead hypervisor actually requires.
Dynamips specifics: a c7200 needs an adapter in slot 1 (slot 0 is the fixed
I/O slot), the tap-fd direction is stated correctly (writes inject into the
kernel, reads return what the kernel transmits), and the anchor registry is
name-keyed against the controller's port matrix by convention, not derivation.
IOL port bridge naming: `bridge{bay}` on unit 0, `bridge{bay}_{unit}`
elsewhere; the leg swap happens on link delete/re-create or a datapath
switch, while suspend is an in-place NIO update and a project reopen is a
fresh build.
Verified and Roadmap sections: the in-repo e2e suite is distinguished from
the isolated-instance development harness (IOU), assertions are stated with
their bounds and capability gates, and the roadmap entries collapse into the
one remaining project — VXLAN/GENEVE for cross-compute links.
Code comments: `flow` added to the documented mode order in the eBPF apply
path, and the `nio_bridge` module docstring corrected on which filter types
run on the kernel datapath (bpf via cls_bpf, frequency_drop/quota/window_drop
via the eBPF classifier — neither is relay-only any more).
docker create_veth hardens only its FIRST end (the Docker shape: host
anchor plus a container leg that keeps its L3 life). A cascade end is
that second, unhardened end whenever the peer switch's create won the
race, and the kernel gives it an IPv6 link-local — a silent L3 identity
on the switch fabric (ubridge-l2-anchor-spec §E). Each switch now
hardens its OWN end with 'link l2only <end> on' once the pair exists,
whatever branch brought it there (fresh create, lost race, reuse);
idempotent, and best-effort with a warning on a uBridge without the
command.
The compute-side backstop of the controller's duplicate-port guard
(keep-port-links + live-port bind made the controller guard reliable;
this is the layer under it). IOU and Dynamips bound the adapter NIO
before the uBridge/hypervisor attach: a failed attach left the port's
binding replaced — the link that still owned the port lost its handle
and its host state leaked (the IOU EBUSY incident's 8 orphan TAPs).
Dynamips was worse: its wiring idempotence check silently skipped the
attach on a second binding and rebound only the bookkeeping. A port
that already carries an NIO is now refused, and both types bind the
NIO only after the wiring succeeded.
The spec's B table grows its fifth row: bridge add_nio_tap mints a
transient TAP when the name is free and hardens it at creation, while an
attach to an existing device (possibly a user-owned TAP with addresses)
is left untouched. The rule paragraph spells out attach-is-not-a-creation.
Delivery: uBridge fb72758, closing the gap the new e2e assertions found
(live: the switch's gns3*-0 relay TAP carried fe80::). The server issues
no link l2only of its own -- the switch call site now documents that the
TAP is minted-and-hardened by add_nio_tap, and the ESW feature doc
describes the uBridge-side close instead of a server workaround.