Dynamips links can now ride the kernel datapath like Docker, QEMU and IOU:
each Ethernet slot/port gets a persistent TAP anchor (gd{id}e{slot}p{port}),
created through uBridge's tap module, and the Dynamips hypervisor opens it
with nio create_tap — the hypervisor holds the fd, exactly QEMU's shape.
Links attach the anchor to their per-link kernel bridge; the legacy UDP
tunnel remains the relay datapath for every other port.
Anchor lifecycle: born at node start (tap module probed with one
create/delete cycle, sweep of stale devices, ownership handed to the server
user, born down); a kernel link opens the anchor in the hypervisor, binds
the slot/port and enslaves it; removing the link releases the fd again
(nio delete) and detaches. A link bound while the router is stopped is
wired by the next start (deferred wiring). Node close follows QEMU's stop
order — the hypervisor's TAP NIOs go before tap delete, since it is
Dynamips, not uBridge, that holds the fds.
Stop is not close: vm stop only halts the emulated router, so the
hypervisor, the TAP fds, the bindings and the per-link bridges all survive
a stop/start untouched (attach skips ports that still hold their TAP NIO).
Port selection follows the adapter registry: ETHERNET_ADAPTERS and
ETHERNET_WICS name the models whose ports carry Ethernet frames (WIC-1ENET
included, at its Dynamips port number); serial, ATM and POS ports never
anchor, and a kernel link there errors with the relay-datapath message
instead of half-wiring. Adapters hot-added to a running router (OIR) get
their anchors at once.
The compute NIO routes accept nio_bridge alongside nio_udp (the same
widening QEMU needed), carry the endpoint's suspend flag on update (it is
the anchor's admin state on a kernel link), skip the hypervisor delete for
a kernel NIO (its TAP NIO was already released by the slot removal), and
the close path leaves NIOBridge instances to _stop_ubridge.
The IOU NIO update route never copied the body's suspend flag onto the
NIO; the Docker and QEMU routes always did. Harmless on the relay, where
suspend is emulated by the synthetic frequency_drop filter the controller
injects, but fatal on a kernel link, whose suspend *is* the anchor's
admin state: a suspended link kept forwarding traffic, and the resume
looked like it worked because nothing had ever stopped.
Eligibility gains an IOU branch gated on the compute's ubridge_iol_tap
(strict True: an unreported capability keeps the link on the relay), and
a port-level rule: a kernel link is an Ethernet segment, so any endpoint
whose port is not Ethernet (IOU serial) disqualifies the link before it
is dispatched. Without that, a serial link between two capable IOU nodes
would be sent as nio_bridge and fail at the compute.
The IOU NIO routes accept nio_bridge alongside the UDP/TAP/Ethernet NIOs
— without the widening the controller's POST is 422'd before the node
ever sees it, the same trap the QEMU routes hit.
Whether an IOU port can terminate on a persistent TAP (iol_bridge
add_nio_tap) is a capability of its own: the tap module and that command
land independently, so a build may have one without the other. Probed on
a scratch IOL bridge with a scratch TAP — one add/delete cycle, no
iol_bridge start needed (attaching a NIO works on a stopped bridge) —
cached by binary identity, with the scratch bridge id above the per-node
id space the fabric locks (513..1024) so a probe cannot collide with a
running node. Any failure, including an old build's "Unknown command",
means unknown: IOU stays on the relay datapath.
Reported through /capabilities as ubridge_iol_tap, next to ubridge_tap.
Kernel eligibility becomes "both endpoints on one compute, each able to
anchor": Docker always (adapters are born as veth pairs), QEMU when that
compute's uBridge can create the persistent TAPs its adapters anchor on. The
question is asked per compute, because the link needs both ends anchored --
so a mixed or half-upgraded deployment degrades to the relay instead of
failing an attach.
probe_tap_support() creates and deletes one throwaway persistent TAP through a
scratch uBridge, cached by binary identity like the tc report; /capabilities
exposes `ubridge_tap` (absent = unknown, which keeps links on the relay).
The compute route for a QEMU adapter NIO took UDPNIO only, so a kernel link
was rejected with 422: its body and response now accept
Union[UDPNIO, BridgeNIO] exactly like Docker's.
Stale "Docker-to-Docker" wording in the config, link and MCP descriptions now
talks about anchors, and docs/features/qemu-kernel-datapath.md documents the
datapath -- verified end-to-end on a real instance: 32/32 kernel-datapath
checks and 29/29 on the relay variant.
Consume uBridge's ebpf_modes contract (feature/tc-window 40c36ae): a
stateful mode is usable iff ebpf=1 and its token is listed; builds
predating the field keep their shipped modes (nth/quota/flow), so the
pre-correction window semantics can never be installed silently.
- gns3server/utils/tc_capabilities.py: shared reply parser, mode
policy and filter-type mappings (compute and controller gate on the
same rules)
- gns3server/compute/ubridge/tc_probe.py: standalone probe spawning a
throwaway uBridge as the server's own user, cached by binary
identity (path/mtime/size - the version string cannot distinguish
rebuilds); failures cache as unknown
- GET /v3/compute/capabilities gains ubridge_tc {netem, ebpf,
ebpf_modes, cbpf}; the controller forwards it (Compute.capabilities)
and available_filters hides kernel-datapath filter types an endpoint
compute cannot run (both ends apply their own filters, so one
incapable compute hides the type); computes without a report keep
the full list and apply-time 409 stays the guard
- docker: _ubridge_apply_ebpf_drops gates per mode token; the cleanup
path only sends "off" for modes the build declares
- split DockerVM's kernel-datapath machinery (veth lifecycle, carrier,
marker primitives, tc filter translation, kernel NIO teardown) into
compute/docker/docker_kernel_datapath.py (DockerKernelDatapathMixin)
Eligible links (both endpoints Docker, same compute, no filters/markers,
stopped nodes, no unix-socket NIO containers) are wired as veth pairs
enslaved into a per-link Linux kernel bridge (gns3 + 11 hex of the link
id, filling IFNAMSIZ) instead of the uBridge UDP relay, so frames flow
entirely in the kernel.
- NIOBridge NIO type: the controller decides the datapath in
UDPLink._prepare and both computes enslave their deterministic gv*/gc*
veth pair via uBridge's docker/brctl modules
- carrier/suspend = host-end admin state; both endpoints race-safely
create/delete the bridge (EEXIST/EBUSY/ENOENT tolerated)
- veth adapters can still attach relay (UDP) links on a running node:
lazy relay bridge over AF_PACKET on the host end (add_nio_ethernet)
- filters/markers/capture are rejected with explicit errors on this
datapath (they live in the uBridge relay); vendor unix-socket
containers are excluded; Server.enable_kernel_datapath is the
kill-switch (default on)
- host-end veths survive container death and are removed on stop plus
swept by name on the next start (crash/stale recovery)
Only relying on the image name lets a moved tag (e.g. a newer :latest)
silently serve stale content from a compute that already has an image
under the same name. When creating a Docker node, the controller now
pins the image id (Id from the Docker daemon on the controller host)
into the create payload. A compute holding a different image under the
same tag reports the image as missing, which routes it through the
image sync added by the previous commit and re-aligns the tag.
No new template fields or database changes: the controller host daemon
remains the source of truth and the pin is resolved per creation. When
the image is not available on the controller host the pin is omitted
and behavior is unchanged (the compute pulls from the repository).
When a Docker node is created on a remote compute whose Docker daemon
does not have the image, the compute now raises ImageMissingError
instead of blindly pulling from the Docker repository. The controller
exports the image from the Docker daemon on its host (docker save
stream) and streams it to the compute which loads it, so locally built
or docker-loaded images work across computes. When the image is not
available on the controller host either, the compute is asked to pull
it from the Docker repository as a fallback.
- add a POST /docker/images/load compute endpoint that streams a
docker save tar into the Docker daemon
- let Docker.http_query pass raw (non-dict) request bodies through so
the tar can be streamed to the daemon
- drop the inline pull from DockerVM.create() and the now unused
DockerVM.pull_image wrapper
The project-open bulk path (_add_nio_binding / _get_existing_nio /
_update_nio_binding in routes/compute/projects.py) dropped port_number
for Docker nodes, unlike the per-node routes and the IOU branch. With
multi-port docker adapters (iol-runner nodes model 4 ports per adapter,
0ca9ccc63) every batched NIO landed on port 0 where Adapter.add_nio()
silently overwrites — the last entry per node won — so reopening a
project clobbered the port-0 links with the port-1 NIOs: links ended up
cross-wired between the wrong node pairs and the real links died
(IOL direct-link ping failures after close/reopen, EXCESSCOLL storms
from the phantom loops).
Templates reference a config file with the GNS3_IOL_STARTUP_CONFIG
environment knob; the controller materializes the file content into
startup_config_content on node creation (sent once, knob consumed —
the same pattern as the IOU startup_config mapping). The compute builds
the content into the node's nvram_<app id> at the next start using the
IOU nvram_import utility (IOL and IOU share the nvram container format,
verified against iol-xe 17.18.02): valid config at boot, no setup
dialog, %h hostname substitution, hostname rewrite on rename.
Semantics verified against the runner: IOL boots from NVRAM whenever it
holds a config, so a plain stop/start never re-applies the startup
config and 'write memory' survives restarts; an explicit content edit
(PUT) is re-applied on the next start and wins over the saved config,
like IOU.
IOL interfaces come in 4-port units (Ethernet0/0-3, Ethernet1/0-3, ...),
addressed like IOU as (adapter_number, port_number 0-3):
- IOLDockerVM builds EthernetAdapter(interfaces=4) per adapter and asks
the runner for adapters x 4 interfaces (num-eth).
- DockerVM threads port_number through the NIO/capture API (the compute
routes parsed it from the URL but dropped it); single-port adapters
keep the historical bridge{N} names and command sequence, multi-port
adapters get one bridge per port (bridge{a}_{p}).
- The unix-socket NIO wiring addresses sockets flat across adapters:
adapter x ports-per-adapter + port, so single-port images keep their
exact socket layout.
- The controller port list for GNS3_IOL_RUNNER docker nodes is generated
by StandardPortFactory with the IOU naming (Ethernet{segment0}/{port0},
segment size 4); plain docker nodes keep eth{N}.
The brctl Ethernet switch runs a per-port uBridge relay, so it can host
the mark filter and packet filters like any uBridge-backed node. Add
ethernet_switch to _MARKER_CAPABLE_TYPES and _get_filter_node, narrow the
UDPLink.update() NIO-PUT skip down to the Dynamips-hosted ethernet_hub,
and expose the matching compute endpoints: PUT nio (filter/marker
reapply) plus the per-marker toggle/pause/resume/delete/rebuild routes.
The ethernet_hub keeps its exclusion: its routes still wire into the
Dynamips hub, which has no uBridge of its own.
Nine fixes from a review of the docker-shm-devices diff:
* GNS3_STOP_TIMEOUT >300 s aborted at the manager's default HTTP timeout
before Docker finished the stop — the stop query now gets a timeout
with a margin over the grace period.
* Overlapping bind targets (GNS3_MASK_UDEV + GNS3_MASK_SYSTEMD on the
same unit, a unit named twice, an extra_configs target equal to a
masked unit) made Docker reject the create with 'Duplicate mount
point' — Mounts are deduplicated by target.
* ExtraConfig.target now carries a pydantic validator (absolute file
path, no '..'), so bad targets 422 at template-save time instead of
failing at node-create time after a multi-GB image pull; directory
forms ('/', '/etc/') are also rejected by the runtime guard instead
of raising IsADirectoryError (raw 500).
* _check_host_readiness skipped every remaining check when one
/proc/sys key was unreadable (mid-loop return) — now continues.
* The base-class GNS3_* env parser strips trailing commas like the
vendor parser, so 'GNS3_MASK_UDEV=1,' composed from a list still
activates.
* Vendor env knobs are re-parsed on every create(), so a PUT to the
node's environment takes effect on the next (re)create.
* The graceful SIGTERM stop is now limited to the explicit user stop
route; delete/update/close/crash-cleanup keep the immediate kill
(those paths force-delete or recreate the container right after).
* An extra_configs target beneath a persisted volume is shadowed by the
volume bind — warn at create time.
Add an `extra_configs` field (list of {target, content}) to the docker
node/template/appliance schemas. For each entry GNS3 writes `content` to a
file in the node working directory and bind-mounts it read-only at `target`
inside the container.
This lets a NOS appliance seed its startup config without rebuilding the
image: XRd points XR_FIRST_BOOT_CONFIG at an injected /firstboot.cfg, FRR at
/etc/frr/frr.conf, etc. The bind is a single-file mount applied at create
time, so it works for both the generic init.sh path and vendor nodes that
skip init.sh (console_type=docker_exec). Entries are only injected when
present, so ordinary nodes are unaffected.
The content can't go through `environment` (it is line-delimited, one var per
line), hence a dedicated field -- the same plumbing shape as extra_volumes.
create_docker_node() passes console, aux etc. to create_node() via
.get() so those keys remain in node_data. The setattr fallback loop
then re-applies them — if reserve_tcp_port returned a different port
in __init__, the setter fires an INFO log and performs a wasted
release→reserve round-trip.
Pop the 15 keys already consumed by create_node() before the loop so
it only handles truly extra keys.
create_batch_nios bound NIOs in a serial for-loop, so during project open
every builtin L2 node (ethernet_switch/hub/cloud/nat) started its uBridge
one at a time (~0.5s each for fork + AF_UNIX connect). Group entries by
node and bind in parallel with asyncio.gather — mirroring update_batch_nios
— so independent uBridge processes start concurrently. Within a node,
entries stay serial to respect the per-node uBridge command lock.
