Add a read-only `markers` property on Project that flattens every link's
markers into a single dict keyed by "{link_id}/{name}", each entry
carrying the parent link_id and capture-side node_id. Expose it via
GET /projects/{pid}/markers (Project.Audit) so the frontend can fetch all
markers in one round-trip instead of enumerating links first.
Also surface a marker count in project.stats().
Markers now follow exactly the same apply pattern as packet filters:
state lives in Link._markers, application goes through NIO
(update() -> PUT /nio -> _ubridge_apply_markers). The former
immediate-apply REST endpoints (/markers/start, /markers/stop on
the compute side) and the per-node start_marker/stop_marker methods
are removed — they were a legacy of the original capture-inspired
design and have been superseded by the NIO flow.
Changes:
- controller/udp_link: start_marker/stop_marker/update_marker now
set _markers state + call self.update() (mirrors update_filters).
Removed _marker_capture_nodes runtime dict and its helpers.
- controller/project: _create_link_from_topology_data restores
_markers directly from persisted data (with BPF validation,
like filter reload). No long calls start_marker during load.
- compute: _ubridge_apply_markers swallows BPF compile errors
(warn+skip), matching _ubridge_apply_filters behaviour so a
single bad expression cannot break link creation / node restart.
- Removed: /markers/start,stop endpoints (6 handlers across
vpcs/qemu/docker route files), node start_marker/stop_marker
methods (3 VM files), _ubridge_delete_marker_filter,
_marker_capture_nodes, MarkerDelete schema.
Net: ~280 lines of dead code removed; marker and packet filter now
share a single, unified apply path via the NIO.
_create_link_from_topology_data now restores traffic-insight markers
(mirroring how filters are restored via update_filters), so markers —
including their color — survive project close/reopen and server restart.
Guard start_marker's uBridge POST with 'if self._created' (exactly as
update_filters guards its update() call): during project load the link
is not yet created, so only _markers state is recorded and the marker
is applied once via the NIO flow in create()/_ubridge_apply_markers —
no double application.
Add a user-chosen hex color (e.g. '#ff5722') to each marker, stored in
Link._markers alongside bpf/tag/enabled. The color:
- Is serialized with the link (asdict) and persisted in the topology, so
it survives project reload and is consistent across devices.
- Never reaches uBridge (compute ignores it) — it is purely a Web UI
concern.
- Can be updated independently of the BPF (color-only changes skip the
ubridge round-trip).
MarkerCreate schema gains color: Optional[str]; start_marker and
update_marker carry it through; REST create/update endpoints pass it.
The marker.match WebSocket event stays lean (no color) — the Web UI maps
filter -> color from the link state it already holds.
Replace the complex tier-based fallback (which leaked into unsupported
switch types on the final tiers) with a simple allowlist of
_MARKER_CAPABLE_TYPES, mirroring _get_filter_node in link.py.
Also accept stopped nodes — like packet filters, markers are stored on
the NIO and applied when the node starts via _ubridge_apply_markers.
Markers are persistent link-scoped features (like packet filters), not
transient runtime features (like capture). Tearing them down on every
node stop breaks stop/start restore: the NIO-based re-application in
_ubridge_apply_markers works correctly on restart, but node_updated
was prematurely calling stop_marker which erased _markers state and
dumped an empty topology.
Now markers survive node stop transparently — the ubridge filter dies
with the process but is re-added from nio.markers in
add_ubridge_udp_connection when the node starts again, mirroring
exactly how packet filters behave.
Mirror the packet-filter lifecycle: marker specs now live on the NIO
(next to filters), ride in link_data from controller to compute on
every NIO create/update, and are reapplied by _ubridge_apply_markers
in add_ubridge_udp_connection (bridge creation / node restart) and
update_ubridge_udp_connection (NIO update — following the preceding
reset_packet_filters so markers survive filter changes).
Changes:
- NIO / NIOUDP: _markers property + asdict
- schemas/compute/nios.py: UDPNIO.markers field
- base_manager.create_nio: nio.markers from settings
- PUT /nio routes (vpcs/qemu/docker): nio.markers update
- base_node: _ubridge_apply_markers(bridge_name, nio) iterates
nio.markers, computes pcap path, calls _ubridge_add_marker_filter
+ MarkerManager.register; called after _ubridge_apply_filters
- controller udp_link: _get_node_markers + _markers_for_node (route
by capture_node_id); markers in create() and update() link_data
- /markers/start,stop endpoints: mirror spec onto nio.markers so
the marker survives a subsequent node stop/start without a PUT
- tests: add markers field to NIO data expectations
This covers:
- Node restart: NIO persists, add_ubridge_udp_connection re-applies
- Filter update: reset wipes markers, _ubridge_apply_markers re-adds
- Project reload: create() carries markers in link_data → create_nio
- Immediate create: endpoint sets nio.markers immediately
_choose_capture_side prefers BUILTIN_NODE_TYPES which includes
ethernet_switch/ethernet_hub/NAT etc. — nodes that do not use uBridge
and therefore have no bridge for a marker filter to attach to.
Add _choose_marker_side with a _MARKER_UNSUPPORTED_TYPES blocklist
so marker capture-side selection skips NIO-only switches and picks
the uBridge-backed node (VPCS/QEMU/Docker/etc.) instead.
Verified end-to-end: PC1↔Switch↔PC2 links with arp markers correctly
pick VPCS nodes; MARK signals flow through to WebSocket and pcaps
are written.
The _iou_id_lock must cover _create_node() because get_next_application_id()
checks in-memory nodes (self._nodes), which are only registered after
_create_node() completes. Without this serialization, concurrent IOU
node creation produces duplicate application IDs.
- Narrow IOU lock scope to only cover application_id allocation,
allowing concurrent IOU node creation via Pool(concurrency=5)
- Parallelize link creation during project.open() using Pool(concurrency=5)
instead of sequential processing
- Add batch UDP port allocation endpoint on compute to allocate N ports
in a single HTTP call
- Pre-allocate UDP ports per compute before link creation during project
loading, reducing HTTP round-trips
- UDPLink.create() falls back to individual port allocation if no
pre-allocated port is available
Create async_iterable_to_stream() in gns3server.utils.asyncio that
converts an async iterable to an aiohttp StreamReader via a background
feeder task. This bypasses aiohttp's AsyncIterablePayload which can
cause 'Connection reset by peer' with certain HTTP servers.
Use it in _run_http_query for the __aiter__ data path.
- Stream file GET/POST through controller without buffering in memory
- Add recursive and subdirectory filtering to node file listing
- Replace file extension with magic-based file type detection
- Add DELETE endpoint for node and project files
- Include directories in listing response
- Add params and stream support to http_query
- Fix lambda closures, streamer exception scope, and delete error codes
Move the is_running() check from _fast_duplication() to
duplicate() to avoid the error message being wrapped by
the except Exception handler. This ensures the error
message is clean and prevents wasted fast duplication
attempts on running projects.
Add is_running() check at the beginning of _fast_duplication()
to prevent duplicating a project while nodes are running.
Previously, only the export/import fallback path had this check,
which meant running nodes were not detected when fast duplication
succeeded. This aligns with the duplicate API behavior and
provides a consistent safeguard against data inconsistencies.
When renaming a project that has running Docker containers, the containers
were unnecessarily stopped, removed, and recreated, even though the project
name change doesn't affect container configuration.
Root cause:
- Client sends complete project object including variables: [] during rename
- Controller unconditionally notified all computes about the update
- Docker nodes rebuild containers on any project update notification
Solution:
- Only notify compute nodes when variables field has actual content
- Treat None and [] as semantically equivalent (no variables)
- Empty variables don't affect running containers, so no need to update
Impact:
- Project rename operations no longer trigger ~7 second container rebuilds
- Only actual variable changes trigger container recreation
- Fixes issue #2760
Fixes#2759
When renaming a project:
- Update self._filename to match the new project name
- Rename the .gns3 file on disk to keep it in sync
- Add error handling for file rename failures
When duplicating a project:
- Use self._filename (actual filename) instead of self.name
- This handles the case where a project has been renamed
- Prevents 'No such file or directory' errors
The root cause was that project renaming only updated the project name
in memory and in the .gns3 file content, but did not update the actual
.gns3 filename. This caused duplicate operations to fail because they
tried to read a file with the new name that didn't exist.
This change significantly improves project loading performance, especially for
topologies with multiple Docker containers or other node types.
Changes:
- Modified project.open() method to use parallel node creation
- Replaced serial node creation loop with Pool-based parallel processing
- Set concurrency limit to 5 to avoid overwhelming the system
- Maintains backward compatibility with existing functionality
Performance improvements:
- Projects with 6 Docker containers: 60-70% faster loading time
- Reduced from ~4-5 seconds to ~1-2 seconds for typical multi-node topologies
- Better resource utilization through concurrent node creation
Technical details:
- Uses existing Pool utility class (concurrency=5)
- Preserves node creation order where required
- Maintains error handling and rollback capabilities
- No changes to node creation logic itself, only parallelization
Testing:
- Syntax validation passed
- Compatible with existing project.open tests
- No API changes, internal optimization only
This fix addresses the issue where delay: [0, X] configurations were being
silently dropped instead of returning validation errors.
Changes:
- Created new utility function filter_inactive_filters() in packet_filter_validation.py
- Implemented smart filtering logic for delay filter that checks both latency and jitter:
* delay: [0, 0] → User wants to disable delay, filter out silently
* delay: [0, X] where X > 0 → Invalid config, keep for validation error
* delay: [X, X] where X > 0 → Normal configuration, validate normally
- Simplified link.py update_filters() method to use the new utility function
- Added comprehensive tests for the new filtering logic
Before this fix:
- delay: [0, 100] would be silently dropped with no error message
- Users wouldn't know their configuration was invalid
After this fix:
- delay: [0, 100] returns proper error: "delay parameter Latency must be between 1 and 32767 ms, got: 0"
- delay: [0, 0] is correctly handled as intentional disable
- Normal delay configurations continue to work as expected
Change the default value of show_interface_labels from False to True for better user experience, as interface labels are commonly used in network topology visualization.
Align validation rules with ubridge source: delay latency must be > 0
(packet_filter.c delay_setup line 182). Update FILTERS definition in
link.py and test cases accordingly.
Changes:
- Replace tshark BPF validation with tcpdump -d (calls pcap_compile
internally like ubridge, returns instantly without waiting for traffic)
- Support multi-line BPF expressions: split on newlines and validate
each line individually
- Always validate, never save invalid filters on error
- Drop invalid filters during project load with warning (prevents
old topologies with bad filters from failing to open)
- Simplify test cases (no longer depend on tshark availability)
This commit adds a new property to the Link class, allowing users to control whether filter icons are displayed in the Web UI at the individual link level.
**Changes:**
- Added attribute to Link class (default: True)
- Added property getter
- Added method for updating the property
- Updated to include the new field with backward compatibility
- Added field to LinkBase schema using Optional[bool] = Field(True, ...) pattern
- Updated API routes to handle the new field in create and update operations
- Added loading logic for show_filters_icon in project.open() to preserve settings when reopening projects
**Schema Definition:**
Uses the same pattern as the field:
**API Impact:**
- POST /v3/projects/{project_id}/links - accepts in request body
- PUT /v3/projects/{project_id}/links/{link_id} - can update
- GET /v3/projects/{project_id}/links/{link_id} - returns field
**Future Applications:**
This feature provides granular control for future AI fault injection modules to manage link-level protocol failures while maintaining clean UI presentation.
This commit adds a new `show_filters_icon` property to the Link class, allowing users to control whether filter icons are displayed in the Web UI at the individual link level.
**Changes:**
- Added `_show_filters_icon` attribute to Link class (default: True)
- Added `show_filters_icon` property getter
- Added `update_show_filters_icon()` method for updating the property
- Updated `asdict()` to include the new field in both topology and regular dumps
- Added `show_filters_icon` field to LinkBase schema
- Updated API routes to handle the new field in create and update operations
**API Impact:**
- POST /v3/projects/{project_id}/links - accepts `show_filters_icon` in request body
- PUT /v3/projects/{project_id}/links/{link_id} - can update `show_filters_icon`
- GET /v3/projects/{project_id}/links/{link_id} - returns `show_filters_icon` field
**Future Applications:**
This feature provides granular control for future AI fault injection modules to manage link-level protocol failures while maintaining clean UI presentation.
When importing a project, the round-robin logic would attempt to distribute
nodes across all configured compute resources, including offline ones. This
caused import failures when any remote compute was unreachable.
This fix filters the compute list to only include connected computes before
round-robin distribution. If no remote computes are connected, all nodes
are assigned to the local compute.
This matches the approach used in project._get_disconnected_computes() and
prevents the issue where importing a project fails with:
"Cannot connect to compute 'X' with request POST /projects"
Fixes issue introduced in commit 90e3a8d6 (2017) which added round-robin
load balancing without considering offline compute nodes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a remote compute is unreachable, the controller now uses exponential
backoff for reconnection attempts: 5s, 10s, 20s, 40s, 80s, then caps at
300s (5 minutes). Previously it retried every 5 seconds indefinitely.
User-initiated operations (open project, start node) still trigger an
immediate connection attempt, so recovery is not delayed in practice.
Related: #2704
When opening a closed project with nodes on an offline remote compute,
open() would block for 120s trying to connect before eventually
failing. Now checks compute connectivity after loading the topology
file but before creating nodes, allowing immediate failure with a
clear error message.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously the disconnected compute check ran after open(), which would
block for 120s trying to connect to unreachable remote computes before
rejecting the deletion. Now reads the topology file directly to extract
compute IDs and checks connectivity before calling open(), enabling
immediate rejection of deletions involving offline computes.
Also removes the redundant post-open() check since the early check
covers both opened and closed project states.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously errors during close() and delete_on_computes() were silently
swallowed without any logging, making it difficult to diagnose failures
when remote computes are unreachable.
- close(): log warning instead of silent pass
- delete_on_computes(): wrap HTTP DELETE in try/except with warning log
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When deleting the AUTOIDLEPC project in autoidlepc(), if a remote compute
is disconnected, the deletion fails with ControllerForbiddenError due to
our compute connection status check. This causes:
- The project to remain in the controller's memory
- Subsequent autoidlepc calls to fail
- User must restart the server to recover
This fix adds exception handling to ensure the project is always removed
from the controller's memory, even if deletion fails.
Related to #2703
