Replace the builtin EthernetSwitch stub with a Linux kernel bridge backed
by uBridge's brctl module. Each switch node creates one kernel bridge
(gns3br{N}) with VLAN filtering; each port is a persistent TAP enslaved to
the bridge, relayed by a per-port uBridge bridge (nio_tap <-> nio_udp).
- Access/dot1q/qinq port modes translated to brctl vlan primitives
- Compute router repointed from Dynamips to Builtin manager
- Tests updated: 21 router-level tests + 215 surrounding tests pass
- Real-kernel e2e verified: access 100 PVID untagged, dot1q trunk
VIDs 1-4094 + native 1, qinq 802.1ad proto + 200 PVID
Ethertype 0x9100/0x9200 handling and port-count guard relaxation are
deferred pending resolution.
_start_ubridge configures the marker sink via _ubridge_configure_marker_sink, which used _ubridge_send. But _start_ubridge itself is reached THROUGH _ubridge_send when uBridge is started lazily — e.g. linking a stopped node. _ubridge_send's @locking lock (___ubridge_send_lock) is non-reentrant, so the nested _ubridge_send('marker sink') deadlocked forever, making the NIO create POST time out after 120s. Started nodes never hit this (uBridge already running, no _start_ubridge), so only stopped-node links (notably dynamips) hung.
_ubridge_configure_marker_sink now talks to self._ubridge_hypervisor.send directly: it runs inside _start_ubridge right after connect(), so uBridge is already up and the raw send is safe, with no reentrant lock acquisition.
A uBridge MARK signal carries only node= and filter= (no bridge/link field), so when one node is the capture side for several links that share a marker name (always the case for global-{name} definitions on a multi-interface node) the signals were indistinguishable and the (node, filter) registry collapsed them to a single link.
The mark filter is now stamped with its link id (mark <bpf> ... link <link_id>); uBridge echoes it verbatim (link=<link_id>) and the listener uses the signal's link= as the authoritative link_id of the marker.match event, falling back to the registry only for legacy signals without it. base_node and iou apply paths pass link_id; covered by two new listener tests.
Bugs found via end-to-end testing of project-level marker definitions:
1. New links didn't inherit — apply_defs_to_new_link is async but was
called without await in UDPLink.create().
2. Project load crashed — load_project passed marker_definitions to
Project.__init__. Now popped in load_project and restored separately
in Project.open() (it backs a read-only property).
3. PUT on a definition didn't sync to links — update_marker's guard
rejected even the project-layer sync call. Added an `inherited`
bypass flag used by update_marker_definition.
4. _ubridge_add_marker_filter raised re.PatternError — the name regex
used (?i)(?!global) look-around, invalid in Python's re module.
Dropped the prefix check there: "global-*" names are legitimate at
the uBridge boundary (inherited definitions); forbidden only at the
user-facing schema.
5. GET /links hid inherited markers — asdict()'s runtime branch used
_persist_markers() (which filters inherited markers). Restored
self._markers for the runtime branch; only the topology_dump branch
filters (inherited markers are rebuilt from definitions on load).
6. Duplicate "already exists" warnings on project open — open() fanned
out definitions to all links, but UDPLink.create() had already done
so via its inheritance hook. Removed the redundant fan-out in open().
Dynamips needed its own NIO base class extended with a `_markers`
attribute and `markers` property (mirroring the existing `filters`
pattern), plus marker propagation from the controller-supplied NIO
settings through NIOUDP.create/update to the common destination
NIO that base_node._ubridge_apply_markers reads.
Cloud simply needed the missing _ubridge_apply_markers calls after
_ubridge_apply_filters in its two ubridge connection methods, same
pattern as the earlier docker fix.
Both types are now added back to _MARKER_CAPABLE_TYPES.
Docker's _connect_nio and adapter_update_nio_binding applied packet
filters but never called _ubridge_apply_markers, so markers silently
did nothing on Docker links despite docker being in the allowlist.
Add the missing calls (same pattern as the IOU fix).
Also narrow _MARKER_CAPABLE_TYPES to the four types that actually
implement marker support — vpcs, qemu, docker, iou — removing
dynamips, virtualbox, vmware, and cloud which have no marker pathway
and would silently fail when selected as the capture side.
When the configured marker_listen_port (default 3070) is already in
use, fall back to an OS-assigned port instead of crashing the server.
If even port 0 fails, log the error and continue without marker
signals — traffic insight degrades rather than taking down startup.
Add a name pattern constraint to MarkerCreate schema (alphanumeric +
_.-) to prevent injection into uBridge commands, with a matching
defense-in-depth check in _ubridge_add_marker_filter for hand-edited
topology files. Also add the missing "enabled" field to MarkerCreate
and pass it through the PUT route so the documented toggle actually
works.
IOU uses iol_bridge (not bridge) with extra bay/unit positional
arguments in add_packet_filter. Add _ubridge_apply_markers override
that mirrors the existing _ubridge_apply_filters pattern, called at
the same three sites: _networking, adapter_add_nio_binding, and
adapter_update_nio_binding.
Markers now follow exactly the same apply pattern as packet filters:
state lives in Link._markers, application goes through NIO
(update() -> PUT /nio -> _ubridge_apply_markers). The former
immediate-apply REST endpoints (/markers/start, /markers/stop on
the compute side) and the per-node start_marker/stop_marker methods
are removed — they were a legacy of the original capture-inspired
design and have been superseded by the NIO flow.
Changes:
- controller/udp_link: start_marker/stop_marker/update_marker now
set _markers state + call self.update() (mirrors update_filters).
Removed _marker_capture_nodes runtime dict and its helpers.
- controller/project: _create_link_from_topology_data restores
_markers directly from persisted data (with BPF validation,
like filter reload). No long calls start_marker during load.
- compute: _ubridge_apply_markers swallows BPF compile errors
(warn+skip), matching _ubridge_apply_filters behaviour so a
single bad expression cannot break link creation / node restart.
- Removed: /markers/start,stop endpoints (6 handlers across
vpcs/qemu/docker route files), node start_marker/stop_marker
methods (3 VM files), _ubridge_delete_marker_filter,
_marker_capture_nodes, MarkerDelete schema.
Net: ~280 lines of dead code removed; marker and packet filter now
share a single, unified apply path via the NIO.
Mirror the packet-filter lifecycle: marker specs now live on the NIO
(next to filters), ride in link_data from controller to compute on
every NIO create/update, and are reapplied by _ubridge_apply_markers
in add_ubridge_udp_connection (bridge creation / node restart) and
update_ubridge_udp_connection (NIO update — following the preceding
reset_packet_filters so markers survive filter changes).
Changes:
- NIO / NIOUDP: _markers property + asdict
- schemas/compute/nios.py: UDPNIO.markers field
- base_manager.create_nio: nio.markers from settings
- PUT /nio routes (vpcs/qemu/docker): nio.markers update
- base_node: _ubridge_apply_markers(bridge_name, nio) iterates
nio.markers, computes pcap path, calls _ubridge_add_marker_filter
+ MarkerManager.register; called after _ubridge_apply_filters
- controller udp_link: _get_node_markers + _markers_for_node (route
by capture_node_id); markers in create() and update() link_data
- /markers/start,stop endpoints: mirror spec onto nio.markers so
the marker survives a subsequent node stop/start without a PUT
- tests: add markers field to NIO data expectations
This covers:
- Node restart: NIO persists, add_ubridge_udp_connection re-applies
- Filter update: reset wipes markers, _ubridge_apply_markers re-adds
- Project reload: create() carries markers in link_data → create_nio
- Immediate create: endpoint sets nio.markers immediately
Each host interface surfaced by the cloud node now reports:
- ip_addresses: every IPv4 and IPv6 address (previously only a single
IPv4 was collected internally and then dropped before the response)
- status / speed / mtu / flags: operational state and link attributes
sourced from psutil.net_if_stats(), with flags normalized to a list
The legacy ip_address / netmask / mac_address fields are preserved so
existing callers (compute link detection, GNS3 VM, VMware, has_netmask)
keep working. The new fields travel through the existing interfaces
payload that the controller forwards verbatim, so no controller-side
change is required and the PUT / ports_mapping flow is unaffected.
- Add class-level caches for loader and default RAM/NVRAM values
- When multiple IOU nodes use the same image, only the first one runs
the ld-linux --verify and iou-image -h subprocesses
- Subsequent nodes reuse cached results, saving ~2 subprocess calls per node
- Cache is populated only on successful subprocess execution to avoid
overwriting explicitly set ram/nvram values with stale cached defaults
Rootful Docker recreates volume mount points as root on start,
preventing the GNS3 process from writing files into node directories
while the container is running. self._fix_permissions() would resolve
this but is currently only called at container stop time.
- _fix_permissions: capture stderr, check returncode, only set
_permissions_fixed on success instead of silently marking as fixed
- list_node_files: wrap os.scandir in try-except to handle
PermissionError gracefully
- Stream file GET/POST through controller without buffering in memory
- Add recursive and subdirectory filtering to node file listing
- Replace file extension with magic-based file type detection
- Add DELETE endpoint for node and project files
- Include directories in listing response
- Add params and stream support to http_query
- Fix lambda closures, streamer exception scope, and delete error codes
Performance improvement for project variable updates when multiple containers
are present. Previously, nodes were updated serially in a for loop, causing:
- 5 containers: ~35 seconds (7s per container)
- 10 containers: ~70 seconds
- 20 containers: ~140 seconds (2min 20sec)
Changed to parallel processing using asyncio.gather(), reducing total time
to the duration of the slowest single node update (~7 seconds regardless
of container count).
The change maintains error handling with return_exceptions=True to ensure
one node's update failure doesn't prevent others from completing.
This is particularly important for users with large topologies containing
many Docker containers that need to be recreated when project variables change.
Related to issue #2755 ghost node timeout fix.
When updating project variables while Docker containers are running, the
system now properly handles both dictionary-format variables and Pydantic
Variable objects. This prevents AttributeError when containers are recreated
after variable updates.
Changes:
- Modified DockerVM.create() to detect and handle Pydantic Variable objects
- Updated _format_env() method to support both variable formats
- Maintains backward compatibility with existing dictionary format
Fixes error: AttributeError: 'Variable' object has no attribute 'get'
Fix F821 undefined name 'status' error by importing the status
module from fastapi. This resolves build errors when using
status.HTTP_403_FORBIDDEN and status.HTTP_404_NOT_FOUND.
Add exception handling in stop_wrap_console to gracefully handle
ConnectionResetError, BrokenPipeError, and OSError when waiting
for console writer to close.
This prevents 500 errors when stopping QEMU nodes if the console
connection is reset before the writer finishes closing.
Fixes race condition where QEMU process exits and closes connections
before the console writer cleanup completes.
Remove MD5 checksum calculation from the node files API since
disk image files are dynamic and change frequently. MD5 calculation
was also causing significant performance overhead.
Changes:
- Remove md5sum field from NodeFile schema
- Remove MD5 calculation from list_node_files method
- Improve error handling for timestamp conversion
- Simplify code by removing lambda functions
Performance improvement:
- Response time reduced from ~0.5-1s to ~0.017s (30-60x faster)
- Especially beneficial for large files and multiple files
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Enhance the node files API to include comprehensive file metadata:
- File size in bytes
- File creation time (ISO 8601 format)
- File modification time (ISO 8601 format)
- File extension
Create new NodeFile schema to support these additional fields
while keeping the existing ProjectFile schema for backward compatibility.
This provides users with better information to manage and identify
files in the Web UI.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implement issue #2719 - Add API endpoint to list project files
- Add GET /v3/projects/{project_id}/nodes/{node_id}/files endpoint
- Add list_node_files() method to Project class
- Add security checks to prevent path traversal
- Filter out .ghost temporary files
- Return file paths with MD5 checksums
- Require Node.Audit privilege
This allows users to discover dynamically created files
such as QEMU disk images created via the disk image API.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When closing a Docker node, if container deletion fails, the error
is silently ignored. This can lead to stale containers remaining on
the system and causing 409 conflicts when reopening projects.
Changes:
- Distinguish between 404 (container already removed, normal) and
other DockerError (deletion failed, needs attention)
- Log warning when deletion fails with error details
- Add comment explaining stale containers will be cleaned up on
project open (via automatic 409 conflict resolution)
This improves observability without blocking project close operations.
The root cause of stale containers can now be diagnosed from logs.
Fixes#2708
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a hostname validation fails, the error message now includes
the allowed character set to help users provide valid names.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove all SPICE WebSocket-related code due to frontend dependency issues
with spice-html5 library (missing RSAKey/BigInteger implementations).
Changes:
- Remove start_spice_websocket_console() from BaseNode
- Remove SPICE WebSocket endpoints from QEMU and Docker compute APIs
- Remove SPICE WebSocket proxy endpoint from controller API
- Remove WebSocket subprotocol handling from authentication layers
- Remove SPICE documentation
The SPICE console type remains functional for direct connections,
but WebSocket proxy support has been removed.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add WebSocket-based console support for SPICE protocol, enabling
browser-based graphical console access with enhanced features
like clipboard sharing, USB redirection, and audio streaming.
Changes:
- Add start_spice_websocket_console() method in BaseNode for SPICE
WebSocket to TCP bridging with bidirectional binary forwarding
- Add /console/spice WebSocket endpoints in QEMU and Docker compute APIs
- Add /console/spice WebSocket proxy endpoint in controller API
- Add comprehensive API documentation in docs/features/
Supported console types:
- spice: Basic SPICE protocol support
- spice+agent: SPICE with spice-vdagent for enhanced features
Architecture:
- Browser WebSocket → Controller (JWT + RBAC) → Compute (Basic Auth)
- No external websockify processes required
- Consistent with existing VNC WebSocket implementation
Add VNC console WebSocket endpoints for Docker and QEMU nodes:
- Add /console/vnc WebSocket endpoint to compute API (Docker & QEMU)
- Add /console/vnc WebSocket endpoint to controller API
- Implement start_vnc_websocket_console() in BaseNode
- Forward VNC WebSocket traffic between controller and compute layers
The implementation provides bidirectional WebSocket to TCP forwarding
for VNC protocol (RFB) connections, allowing browser-based VNC console
access to containers and VMs.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
