Web clients serialize empty form fields as "" while unset values are
stored as None on the node. The bare != diff in the update handler then
sees a phantom change on every full PUT and recreates the container for
nothing -- even when the user only changed a controller-only field such
as netmiko_device_type.
Normalize at the schema boundary ("" -> None for start_command,
environment and extra_hosts; "" -> "/" for console_http_path), make
the setters apply the same canonicalization, and create nodes through
the setters instead of bypassing them in __init__ so both paths store
identical values.
Nine fixes from a review of the docker-shm-devices diff:
* GNS3_STOP_TIMEOUT >300 s aborted at the manager's default HTTP timeout
before Docker finished the stop — the stop query now gets a timeout
with a margin over the grace period.
* Overlapping bind targets (GNS3_MASK_UDEV + GNS3_MASK_SYSTEMD on the
same unit, a unit named twice, an extra_configs target equal to a
masked unit) made Docker reject the create with 'Duplicate mount
point' — Mounts are deduplicated by target.
* ExtraConfig.target now carries a pydantic validator (absolute file
path, no '..'), so bad targets 422 at template-save time instead of
failing at node-create time after a multi-GB image pull; directory
forms ('/', '/etc/') are also rejected by the runtime guard instead
of raising IsADirectoryError (raw 500).
* _check_host_readiness skipped every remaining check when one
/proc/sys key was unreadable (mid-loop return) — now continues.
* The base-class GNS3_* env parser strips trailing commas like the
vendor parser, so 'GNS3_MASK_UDEV=1,' composed from a list still
activates.
* Vendor env knobs are re-parsed on every create(), so a PUT to the
node's environment takes effect on the next (re)create.
* The graceful SIGTERM stop is now limited to the explicit user stop
route; delete/update/close/crash-cleanup keep the immediate kill
(those paths force-delete or recreate the container right after).
* An extra_configs target beneath a persisted volume is shadowed by the
volume bind — warn at create time.
DockerVM.stop() terminated containers with an immediate SIGKILL — fine
for init.sh-based containers whose state is persisted beforehand, but a
systemd NOS (Cisco XRd, SR Linux) needs a graceful shutdown and treats
the abrupt kill as an unclean shutdown (exit 137 on every stop).
Extract the final termination into _terminate_container() and override
it in VendorDockerVM: POST /containers/{id}/stop?t=60 sends SIGTERM and
waits for systemd to stop services; Docker itself SIGKILLs the
container once the grace period expires, so no fallback is needed.
Docker's 304 (already stopped) is swallowed.
Masking the udev systemd units stopped the daemon's coldplug (host audio
resets), but host USB devices still reconnected on every XRd start. A/B
testing with plain `docker run` isolated the trigger: XRd's own
xr_startup.sh calls udevadm directly (USB license-dongle probing, e.g.
`udevadm trigger --action=add --parent-match=<usb device>`), which
synthesizes uevents into the host kernel from the privileged container --
no udevd required.
GNS3_MASK_UDEV=1 now also binds /dev/null over the udevadm binary
(/bin, /sbin, /usr/bin). Verified with a plain-run experiment: with the
bind, host udev monitor shows zero usb/input/hid/sound events during XRd
boot (only normal docker veth traffic), and XRd itself boots to running
state -- it does not need udevadm under GNS3 (interfaces are pre-created
veths).
A privileged systemd-based NOS container (Cisco XRd boots /usr/sbin/init)
runs systemd-udevd, which on startup coldplugs every device it can reach.
In privileged mode that includes the HOST's USB/input/audio/disk devices,
so every XRd start reconnects USB, mutes audio, and disrupts the host
journal -- highly disruptive on Linux desktops (caught in the act: the
container's udevd was even rescanning the host BTRFS root device).
XRd doesn't need udev (its interfaces are pre-created by GNS3 veth and
mapped via XR_INTERFACES). Add two opt-in env vars, consumed host-side at
container create time in the inherited DockerVM.create (so VendorDockerVM
nodes get it too):
GNS3_MASK_UDEV=1 -> bind /dev/null over the udevd unit, its two
activation sockets, and the coldplug/settle
trigger services
GNS3_MASK_SYSTEMD=u1,u2 -> bind /dev/null over arbitrary units in
/etc/systemd/system/ (comma/semicolon list)
Only injected when set, so ordinary nodes are unaffected.
Add an `extra_configs` field (list of {target, content}) to the docker
node/template/appliance schemas. For each entry GNS3 writes `content` to a
file in the node working directory and bind-mounts it read-only at `target`
inside the container.
This lets a NOS appliance seed its startup config without rebuilding the
image: XRd points XR_FIRST_BOOT_CONFIG at an injected /firstboot.cfg, FRR at
/etc/frr/frr.conf, etc. The bind is a single-file mount applied at create
time, so it works for both the generic init.sh path and vendor nodes that
skip init.sh (console_type=docker_exec). Entries are only injected when
present, so ordinary nodes are unaffected.
The content can't go through `environment` (it is line-delimited, one var per
line), hence a dedicated field -- the same plumbing shape as extra_volumes.
Heavy NOS containers (e.g. Cisco XRd) need /dev/shm larger than Docker's
64 MB default and host device nodes such as /dev/fuse. Add two opt-in
environment variables, consumed host-side and applied as native Docker
HostConfig keys at create time:
GNS3_SHM_SIZE (MB) -> HostConfig.ShmSize (bytes)
GNS3_DEVICES -> HostConfig.Devices in `docker run --device` syntax
(host[:container[:perm]]; Docker resolves major/minor
from the host node itself)
Native HostConfig (rather than remount/mknod inside init.sh) is used so this
works for vendor NOS nodes that skip init.sh (console_type=docker_exec) --
the path XRd must take, since GNS3's init.sh wrapper crashes XRd's glibc
loader. It applies whether or not init.sh runs, needs no schema/API/UI
change (reuses the `environment` field), and only takes effect when the vars
are set, so ordinary nodes keep default Docker behaviour.
GNS3_-prefixed user env vars stay dropped from the container environment
(only consumed here host-side), keeping GNS3-injected vars safe.
Extract the docker_exec console and GNS3_* prototype knobs (SKIP_INIT,
INTERFACE_NAMES, CONSOLE_CMD) from DockerVM into a VendorDockerVM subclass.
DockerVM is restored to its 3.1 baseline plus four small extension hooks
(_prepare_init_and_interface_env, _start_console_server,
_get_container_ifname, _cleanup_console_resources) that are pure
refactorings with zero behaviour change for existing nodes.
VendorDockerVM additionally replicates init.sh's volume persistence
(bind-mount /gns3volumes over the in-container path) via docker exec for
containers that skip init.sh, so vendor NOS config (e.g. /etc/opt/srlinux)
survives node stop/start.
The Docker manager selects VendorDockerVM when console_type == docker_exec;
all other nodes keep using DockerVM unchanged.
The _connect_nio thread-pool optimisation (send_batch_sync) targeted
node-start performance, but start_all already runs at concurrency=3
(by design, to avoid overwhelming the host). It also introduced a
Python 3.13 incompatibility (trsock.setblocking forbidden) that
prevented docker nodes from starting. Since node-start is not the
target of this branch (project-open link creation is), revert to the
simple per-command async _ubridge_send.
The project-open batch NIO dispatch (create_batch_nios) is unaffected —
it never called _connect_nio (nodes aren't started during open).
At 1000+ nodes the per-node INFO lines flood the log during open /
start-all / stop-all: MAC changed, adapters changed, created, started,
console listen, fix ownership, stopped, paused, removed, adapter created,
NIO removed, capture start/stop, CPU/memory limits, mount resources.
Demote all of these routine per-node/per-adapter lines to DEBUG. Keep
INFO only for genuinely rare/important events: image pull (missing image)
and stale-container cleanup. Warnings unchanged.
- Drop the diagnostic stage-timing logs added during link-create perf
work (nodes / preallocate / prepare / dispatch) now that bottlenecks
are resolved and verified.
- Lower the per-NIO 'added to adapter' log in docker_vm from INFO to
DEBUG — at 5000+ NIOs per project open it floods the log at INFO.
Replace the default asyncio executor (capped at ~32 threads) with a
dedicated ThreadPoolExecutor sized for large-topology parallelism.
When 500 nodes each call _connect_nio, up to 500 OS threads can now
send blocking ubridge commands in parallel — no longer serialised by
either the event loop or a small thread pool.
- ubridge_hypervisor: module-level _ubridge_sync_pool (max_workers=500)
- docker_vm._connect_nio: dispatches to the dedicated pool instead of
the default executor
Replace the 3-5 sequential await _ubridge_send calls in _connect_nio
with a single run_in_executor batch. The batch holds the node-level
asyncio Lock to prevent interleaving with async sends, then uses the
hypervisor's new send_batch_sync method which does blocking socket
sendall/recv inside the thread pool. Different nodes' batches now
run in true OS-thread parallelism rather than serialising through
the asyncio event loop between every command.
- ubridge_hypervisor.send_batch_sync: blocking batch send using
the underlying socket from the asyncio transport, protected by
threading.Lock.
- _connect_nio: builds command list (add_nio_udp, start_capture,
bridge start, reset_packet_filters, add_packet_filter) and
dispatches to the default executor.
Temporary diagnostic instrumentation to measure the wall-clock time of
each ubridge command during NIO addition (add_nio_udp, bridge start,
filters, markers). The logs will reveal whether the 12-NIO/s
throughput stems from ubridge command latency itself or from lock
contention / HTTP overhead outside _connect_nio.
Docker node stop took ~5s every time. The stop API grace period
(params t=5, unchanged since 2015) was always exhausted: the business
process (often an interactive shell) ignores SIGTERM, and GNS3 doesn't
depend on graceful shutdown — _fix_permissions and /gns3volumes already
persist container state before stop() is called.
Use POST /containers/{id}/kill (SIGKILL, zero delay) instead of stop.
The 409 (container already stopped) replaces the previous 304 handling
for the race where the container exits between the state check and the call.
t=5 traced to commit 33edbefa3 (2015-10-14) "Docker cleanup and
improvements" — introduced with no recorded rationale.
The condition 'state != "stopped" or state != "exited"' is a tautology,
so the state check was a no-op and a stop request was sent even for a
container that had already exited.
_get_container_state() never returns "stopped" (only "running",
"paused" or "exited"), so the intended negation of the condition used in
_fix_permissions() requires 'and', not 'or' (De Morgan's law).
Added a regression test asserting no stop query is issued for an
already-exited container.
Docker's _connect_nio and adapter_update_nio_binding applied packet
filters but never called _ubridge_apply_markers, so markers silently
did nothing on Docker links despite docker being in the allowlist.
Add the missing calls (same pattern as the IOU fix).
Also narrow _MARKER_CAPABLE_TYPES to the four types that actually
implement marker support — vpcs, qemu, docker, iou — removing
dynamips, virtualbox, vmware, and cloud which have no marker pathway
and would silently fail when selected as the capture side.
Markers now follow exactly the same apply pattern as packet filters:
state lives in Link._markers, application goes through NIO
(update() -> PUT /nio -> _ubridge_apply_markers). The former
immediate-apply REST endpoints (/markers/start, /markers/stop on
the compute side) and the per-node start_marker/stop_marker methods
are removed — they were a legacy of the original capture-inspired
design and have been superseded by the NIO flow.
Changes:
- controller/udp_link: start_marker/stop_marker/update_marker now
set _markers state + call self.update() (mirrors update_filters).
Removed _marker_capture_nodes runtime dict and its helpers.
- controller/project: _create_link_from_topology_data restores
_markers directly from persisted data (with BPF validation,
like filter reload). No long calls start_marker during load.
- compute: _ubridge_apply_markers swallows BPF compile errors
(warn+skip), matching _ubridge_apply_filters behaviour so a
single bad expression cannot break link creation / node restart.
- Removed: /markers/start,stop endpoints (6 handlers across
vpcs/qemu/docker route files), node start_marker/stop_marker
methods (3 VM files), _ubridge_delete_marker_filter,
_marker_capture_nodes, MarkerDelete schema.
Net: ~280 lines of dead code removed; marker and packet filter now
share a single, unified apply path via the NIO.
When a Docker container with the same name already exists (e.g., from a
previous crashed GNS3 session), Docker returns a 409 Conflict error
when trying to create a new container with that name. This causes the
project open operation to fail.
This fix adds automatic cleanup of stale containers when encountering
a name conflict:
- Added DockerHttp409Error exception class
- Updated http_query to detect 409 status codes
- Modified create() to remove conflicting containers and retry
Fixes the issue where opening a project fails with:
"Docker has returned an error: 409 Conflict. The container name
'/GNS3.xxx' is already in use by container 'xxx'"
Rootful Docker recreates volume mount points as root on start,
preventing the GNS3 process from writing files into node directories
while the container is running. self._fix_permissions() would resolve
this but is currently only called at container stop time.
- _fix_permissions: capture stderr, check returncode, only set
_permissions_fixed on success instead of silently marking as fixed
- list_node_files: wrap os.scandir in try-except to handle
PermissionError gracefully
When updating project variables while Docker containers are running, the
system now properly handles both dictionary-format variables and Pydantic
Variable objects. This prevents AttributeError when containers are recreated
after variable updates.
Changes:
- Modified DockerVM.create() to detect and handle Pydantic Variable objects
- Updated _format_env() method to support both variable formats
- Maintains backward compatibility with existing dictionary format
Fixes error: AttributeError: 'Variable' object has no attribute 'get'
When closing a Docker node, if container deletion fails, the error
is silently ignored. This can lead to stale containers remaining on
the system and causing 409 conflicts when reopening projects.
Changes:
- Distinguish between 404 (container already removed, normal) and
other DockerError (deletion failed, needs attention)
- Log warning when deletion fails with error details
- Add comment explaining stale containers will be cleaned up on
project open (via automatic 409 conflict resolution)
This improves observability without blocking project close operations.
The root cause of stale containers can now be diagnosed from logs.
Fixes#2708
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a hostname validation fails, the error message now includes
the allowed character set to help users provide valid names.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a Docker container with the same name already exists (e.g., from a
previous crashed GNS3 session), Docker returns a 409 Conflict error
when trying to create a new container with that name. This causes the
project open operation to fail.
This fix adds automatic cleanup of stale containers when encountering
a name conflict:
- Added DockerHttp409Error exception class
- Updated http_query to detect 409 status codes
- Modified create() to remove conflicting containers and retry
Fixes the issue where opening a project fails with:
"Docker has returned an error: 409 Conflict. The container name
'/GNS3.xxx' is already in use by container 'xxx'"
Most VNC clients use the Desktop name in their window title.
Currently this defaults to user@host which means all docker vnc
connections have identical window names of e.g.
SSVNC: gns3@gns3server
This uses the node name instead.
