The _connect_nio thread-pool optimisation (send_batch_sync) targeted
node-start performance, but start_all already runs at concurrency=3
(by design, to avoid overwhelming the host). It also introduced a
Python 3.13 incompatibility (trsock.setblocking forbidden) that
prevented docker nodes from starting. Since node-start is not the
target of this branch (project-open link creation is), revert to the
simple per-command async _ubridge_send.
The project-open batch NIO dispatch (create_batch_nios) is unaffected —
it never called _connect_nio (nodes aren't started during open).
At 1000+ nodes the per-node INFO lines flood the log during open /
start-all / stop-all: MAC changed, adapters changed, created, started,
console listen, fix ownership, stopped, paused, removed, adapter created,
NIO removed, capture start/stop, CPU/memory limits, mount resources.
Demote all of these routine per-node/per-adapter lines to DEBUG. Keep
INFO only for genuinely rare/important events: image pull (missing image)
and stale-container cleanup. Warnings unchanged.
- Drop the diagnostic stage-timing logs added during link-create perf
work (nodes / preallocate / prepare / dispatch) now that bottlenecks
are resolved and verified.
- Lower the per-NIO 'added to adapter' log in docker_vm from INFO to
DEBUG — at 5000+ NIOs per project open it floods the log at INFO.
Replace the default asyncio executor (capped at ~32 threads) with a
dedicated ThreadPoolExecutor sized for large-topology parallelism.
When 500 nodes each call _connect_nio, up to 500 OS threads can now
send blocking ubridge commands in parallel — no longer serialised by
either the event loop or a small thread pool.
- ubridge_hypervisor: module-level _ubridge_sync_pool (max_workers=500)
- docker_vm._connect_nio: dispatches to the dedicated pool instead of
the default executor
Replace the 3-5 sequential await _ubridge_send calls in _connect_nio
with a single run_in_executor batch. The batch holds the node-level
asyncio Lock to prevent interleaving with async sends, then uses the
hypervisor's new send_batch_sync method which does blocking socket
sendall/recv inside the thread pool. Different nodes' batches now
run in true OS-thread parallelism rather than serialising through
the asyncio event loop between every command.
- ubridge_hypervisor.send_batch_sync: blocking batch send using
the underlying socket from the asyncio transport, protected by
threading.Lock.
- _connect_nio: builds command list (add_nio_udp, start_capture,
bridge start, reset_packet_filters, add_packet_filter) and
dispatches to the default executor.
Temporary diagnostic instrumentation to measure the wall-clock time of
each ubridge command during NIO addition (add_nio_udp, bridge start,
filters, markers). The logs will reveal whether the 12-NIO/s
throughput stems from ubridge command latency itself or from lock
contention / HTTP overhead outside _connect_nio.
Docker node stop took ~5s every time. The stop API grace period
(params t=5, unchanged since 2015) was always exhausted: the business
process (often an interactive shell) ignores SIGTERM, and GNS3 doesn't
depend on graceful shutdown — _fix_permissions and /gns3volumes already
persist container state before stop() is called.
Use POST /containers/{id}/kill (SIGKILL, zero delay) instead of stop.
The 409 (container already stopped) replaces the previous 304 handling
for the race where the container exits between the state check and the call.
t=5 traced to commit 33edbefa3 (2015-10-14) "Docker cleanup and
improvements" — introduced with no recorded rationale.
The condition 'state != "stopped" or state != "exited"' is a tautology,
so the state check was a no-op and a stop request was sent even for a
container that had already exited.
_get_container_state() never returns "stopped" (only "running",
"paused" or "exited"), so the intended negation of the condition used in
_fix_permissions() requires 'and', not 'or' (De Morgan's law).
Added a regression test asserting no stop query is issued for an
already-exited container.
Docker's _connect_nio and adapter_update_nio_binding applied packet
filters but never called _ubridge_apply_markers, so markers silently
did nothing on Docker links despite docker being in the allowlist.
Add the missing calls (same pattern as the IOU fix).
Also narrow _MARKER_CAPABLE_TYPES to the four types that actually
implement marker support — vpcs, qemu, docker, iou — removing
dynamips, virtualbox, vmware, and cloud which have no marker pathway
and would silently fail when selected as the capture side.
Markers now follow exactly the same apply pattern as packet filters:
state lives in Link._markers, application goes through NIO
(update() -> PUT /nio -> _ubridge_apply_markers). The former
immediate-apply REST endpoints (/markers/start, /markers/stop on
the compute side) and the per-node start_marker/stop_marker methods
are removed — they were a legacy of the original capture-inspired
design and have been superseded by the NIO flow.
Changes:
- controller/udp_link: start_marker/stop_marker/update_marker now
set _markers state + call self.update() (mirrors update_filters).
Removed _marker_capture_nodes runtime dict and its helpers.
- controller/project: _create_link_from_topology_data restores
_markers directly from persisted data (with BPF validation,
like filter reload). No long calls start_marker during load.
- compute: _ubridge_apply_markers swallows BPF compile errors
(warn+skip), matching _ubridge_apply_filters behaviour so a
single bad expression cannot break link creation / node restart.
- Removed: /markers/start,stop endpoints (6 handlers across
vpcs/qemu/docker route files), node start_marker/stop_marker
methods (3 VM files), _ubridge_delete_marker_filter,
_marker_capture_nodes, MarkerDelete schema.
Net: ~280 lines of dead code removed; marker and packet filter now
share a single, unified apply path via the NIO.
When a Docker container with the same name already exists (e.g., from a
previous crashed GNS3 session), Docker returns a 409 Conflict error
when trying to create a new container with that name. This causes the
project open operation to fail.
This fix adds automatic cleanup of stale containers when encountering
a name conflict:
- Added DockerHttp409Error exception class
- Updated http_query to detect 409 status codes
- Modified create() to remove conflicting containers and retry
Fixes the issue where opening a project fails with:
"Docker has returned an error: 409 Conflict. The container name
'/GNS3.xxx' is already in use by container 'xxx'"
Rootful Docker recreates volume mount points as root on start,
preventing the GNS3 process from writing files into node directories
while the container is running. self._fix_permissions() would resolve
this but is currently only called at container stop time.
- _fix_permissions: capture stderr, check returncode, only set
_permissions_fixed on success instead of silently marking as fixed
- list_node_files: wrap os.scandir in try-except to handle
PermissionError gracefully
When updating project variables while Docker containers are running, the
system now properly handles both dictionary-format variables and Pydantic
Variable objects. This prevents AttributeError when containers are recreated
after variable updates.
Changes:
- Modified DockerVM.create() to detect and handle Pydantic Variable objects
- Updated _format_env() method to support both variable formats
- Maintains backward compatibility with existing dictionary format
Fixes error: AttributeError: 'Variable' object has no attribute 'get'
When closing a Docker node, if container deletion fails, the error
is silently ignored. This can lead to stale containers remaining on
the system and causing 409 conflicts when reopening projects.
Changes:
- Distinguish between 404 (container already removed, normal) and
other DockerError (deletion failed, needs attention)
- Log warning when deletion fails with error details
- Add comment explaining stale containers will be cleaned up on
project open (via automatic 409 conflict resolution)
This improves observability without blocking project close operations.
The root cause of stale containers can now be diagnosed from logs.
Fixes#2708
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a hostname validation fails, the error message now includes
the allowed character set to help users provide valid names.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a Docker container with the same name already exists (e.g., from a
previous crashed GNS3 session), Docker returns a 409 Conflict error
when trying to create a new container with that name. This causes the
project open operation to fail.
This fix adds automatic cleanup of stale containers when encountering
a name conflict:
- Added DockerHttp409Error exception class
- Updated http_query to detect 409 status codes
- Modified create() to remove conflicting containers and retry
Fixes the issue where opening a project fails with:
"Docker has returned an error: 409 Conflict. The container name
'/GNS3.xxx' is already in use by container 'xxx'"
Most VNC clients use the Desktop name in their window title.
Currently this defaults to user@host which means all docker vnc
connections have identical window names of e.g.
SSVNC: gns3@gns3server
This uses the node name instead.
