1634 Commits

Author SHA1 Message Date
YueGuobin
38742ad4b6
tests: sign class-scoped client tokens with the default JWT secret
Since aef337e86 the config loader generates a random JWT secret even
without a main config file, so the class-scoped client/authorized_client
fixtures signed their tokens with a key that the autouse
run_around_tests fixture would immediately replace with the default
one for every test function. Every controller API test using those
fixtures was failing with 401 (BadSignature).

Sign both fixtures explicitly with DEFAULT_JWT_SECRET_KEY to match
the secret enforced at request time.
2026-08-21 21:41:42 +08:00
YueGuobin
7e8c515a3c
api: expose installed netmiko device types for the web UI
New GET /v3/netmiko/device_types endpoint returns the device types
supported by the netmiko library installed on the server, including
the gns3-copilot custom drivers, so the web UI can populate the
netmiko_device_type dropdown on templates and nodes.

The list is read at runtime from netmiko's ssh_dispatcher.CLASS_MAPPER
registry (the same table ConnectHandler dispatches on), filtered to
drop the '_ssh' aliases and the 'autodetect' pseudo type, and cached
for the process lifetime. Returns 501 when netmiko is not installed
(ai-features extra).
2026-08-21 21:41:42 +08:00
YueGuobin
92d4e3d378
nodes: per-node default credentials seeded from the template
Add default_username/default_password as controller-only node properties
(the netmiko_device_type pattern): they are not sent to the compute,
persist with the project topology and can be updated or cleared per
node. Creating a node from a template seeds them from the template
appliance metadata, and the metadata itself is dropped there so it
never leaks into the node properties.
2026-08-21 21:41:42 +08:00
YueGuobin
300c53e6fb
templates: persist appliance metadata on install
Appliance fields that describe the appliance (vendor information, default
credentials, installation instructions...) were dropped when installing a
template. Keep them in a new appliance_metadata JSON column on the
templates table, filled by the appliance-to-template conversion for both
registry v1-6 and v8 (version level values override the appliance level
ones). The nested schema allows extra fields so future registry fields
persist without a migration.
2026-08-21 21:41:42 +08:00
YueGuobin
1c68a52856
fix: address appliance v8 install review findings
- install: resolve the image directory from the version's settings type and
  skip image handling for docker appliances; guard appliance.images
- appliance schema: validate template_properties against template_type,
  align cpu_throttling with the qemu template, add kvm and version idlepc
- conversion: map IOU image to path, kvm disable to accel=tcg, inherit only
  same-type default settings, symbol fallback from the effective category,
  template_properties cannot override structural fields
- allow clearing netmiko_device_type with an empty string
- download the template symbol regardless of the level it is defined at and
  give qemu guests a default symbol
2026-08-21 21:41:42 +08:00
YueGuobin
435aa4c257
copilot: prefer netmiko_device_type over the device_type:<type> tag
The vendored gns3fy Node model and its nodes_inventory() now carry the
node's netmiko_device_type field, and get_device_ports_from_topology()
resolves the Netmiko device type from it first, falling back to the
device_type:<type> tag. Nodes created from a template inherit the value
from the template automatically, so automation tooling gets the correct
Netmiko driver without tags.
2026-08-21 21:41:35 +08:00
YueGuobin
f524e9a713
appliance: seed netmiko_device_type from the appliance file
Both the v1-6 and v8 appliance models accept an optional top-level
netmiko_device_type, and ApplianceToTemplate copies it into the created
template so installed appliances carry the automation hint end to end.
2026-08-21 21:41:35 +08:00
YueGuobin
d64f47afa4
nodes: per-node netmiko_device_type (controller-only)
netmiko_device_type follows the CONTROLLER_ONLY_PROPERTIES pattern
(like console_auto_start): a node created from a template inherits the
template value, PUT /nodes can override it inside a topology, updates
never round-trip to the compute, and the value persists in the project
topology file.
2026-08-21 21:41:35 +08:00
YueGuobin
9260108f53
templates: add netmiko_device_type for automation tools
Common template field (schema + templates table column + Alembic
migration) holding the Netmiko device type (e.g. 'cisco_xr', 'nokia_srl')
so Netmiko/Nornir based tooling can look up how to reach a node's CLI
without hard-coded vendor mappings. Free-form lowercase string on
purpose: Netmiko's platform list evolves independently of GNS3.
2026-08-21 21:41:35 +08:00
YueGuobin
254721dbda
appliance: close the v8 gaps for Docker vendor appliances
DockerPropertiesV8 now accepts custom_adapters (already available to
v1-6 top-level appliances and to Qemu v8 properties), so port-named
Docker appliances (XRd, SR Linux) can move to the v8 format without
losing their interface naming.

Appliance.type resolves the node type from the v8 settings template_type
(default set first) instead of misclassifying every v8 appliance as
qemu, and _get_default_symbol applies the docker guest symbol to v8
Docker guest appliances.
2026-08-21 21:41:35 +08:00
YueGuobin
57fe549773
appliance: implement install support for registry version 8
new_template() now converts the v8 settings[] format per the spec in
gns3-registry#734: settings selection (version name reference, then the
default set, then a single set), inherit_default_properties merging, and
template_properties expansion with category/usage/symbol resolved from
template_properties > version > appliance levels. Undefined properties
are left out so controller template defaults apply.

Registry versions 1-6 keep the existing top-level emulator block path.
2026-08-21 21:41:35 +08:00
YueGuobin
f43a717b20
copilot: sync CONSOLE_TYPES with the server ConsoleType enum
The vendored gns3fy copy keeps its type lists as literals (the module is
shared with the standalone MCP service and cannot import server enums),
and CONSOLE_TYPES had drifted: 'ssh' and 'docker_exec' were missing while
both are valid server-side. Impact: the copilot topology reader validates
the whole node list in one pydantic pass, so a single vendor NOS node
(console_type 'docker_exec') made it drop the entire project and return
zero devices to every copilot device tool.

Add the missing values plus drift tests asserting the vendored lists
cover the server enums (skipped when ai-features extras are absent).
2026-08-15 23:32:24 +08:00
Guobin Yue
74a51a1f9f
Merge branch '3.1' into code-review-fixes 2026-08-15 22:39:42 +08:00
Cristi
a35df76384 feat: Enhance information displayed on the system status dashboard 2026-08-14 23:36:03 +03:00
YueGuobin
110e041e56
docker: cap GNS3_STOP_TIMEOUT at 210 s (controller stop budget)
The 600 s clamp was unreachable in practice: the controller's stop
request times out at 240 s (controller/node.py) and the Docker stop
query gets the value +30 s as its HTTP timeout, so anything above 210
would abort upstream first and surface an error while the stop keeps
running server-side. Cap at the derived ceiling and document the chain
in the clamp and the docstring.
2026-08-15 01:25:50 +08:00
YueGuobin
9604c85fda
docker: harden the shm/devices/extra_configs/masking work (code review)
Nine fixes from a review of the docker-shm-devices diff:

* GNS3_STOP_TIMEOUT >300 s aborted at the manager's default HTTP timeout
  before Docker finished the stop — the stop query now gets a timeout
  with a margin over the grace period.
* Overlapping bind targets (GNS3_MASK_UDEV + GNS3_MASK_SYSTEMD on the
  same unit, a unit named twice, an extra_configs target equal to a
  masked unit) made Docker reject the create with 'Duplicate mount
  point' — Mounts are deduplicated by target.
* ExtraConfig.target now carries a pydantic validator (absolute file
  path, no '..'), so bad targets 422 at template-save time instead of
  failing at node-create time after a multi-GB image pull; directory
  forms ('/', '/etc/') are also rejected by the runtime guard instead
  of raising IsADirectoryError (raw 500).
* _check_host_readiness skipped every remaining check when one
  /proc/sys key was unreadable (mid-loop return) — now continues.
* The base-class GNS3_* env parser strips trailing commas like the
  vendor parser, so 'GNS3_MASK_UDEV=1,' composed from a list still
  activates.
* Vendor env knobs are re-parsed on every create(), so a PUT to the
  node's environment takes effect on the next (re)create.
* The graceful SIGTERM stop is now limited to the explicit user stop
  route; delete/update/close/crash-cleanup keep the immediate kill
  (those paths force-delete or recreate the container right after).
* An extra_configs target beneath a persisted volume is shadowed by the
  volume bind — warn at create time.
2026-08-15 00:52:55 +08:00
YueGuobin
62076c1727
docker: make the vendor graceful-stop grace period configurable (GNS3_STOP_TIMEOUT)
The 60 s SIGTERM grace was hardcoded, unlike every other vendor knob
(GNS3_SHM_SIZE, GNS3_DEVICES, GNS3_MASK_UDEV, ...) which rides the
environment line. Parse GNS3_STOP_TIMEOUT=<seconds> (default 60,
clamped to 1-600, invalid values keep the default) and use it in
VendorDockerVM._terminate_container().
2026-08-14 22:11:34 +08:00
YueGuobin
e9339faaa7
docker: graceful stop for vendor NOS containers (SIGTERM + 60s grace)
DockerVM.stop() terminated containers with an immediate SIGKILL — fine
for init.sh-based containers whose state is persisted beforehand, but a
systemd NOS (Cisco XRd, SR Linux) needs a graceful shutdown and treats
the abrupt kill as an unclean shutdown (exit 137 on every stop).

Extract the final termination into _terminate_container() and override
it in VendorDockerVM: POST /containers/{id}/stop?t=60 sends SIGTERM and
waits for systemd to stop services; Docker itself SIGKILLs the
container once the grace period expires, so no fallback is needed.
Docker's 304 (already stopped) is swallowed.
2026-08-14 22:02:58 +08:00
YueGuobin
350f2b24e7
fix: UDP port allocation race causing link self-loop (one-way links)
PortManager.get_free_udp_port had an unguarded find-then-add sequence.
A link allocates both ends concurrently (asyncio.gather in
UDPLink._prepare -> two POST /ports/udp) and FastAPI runs the sync
route handler in a threadpool, so both threads could probe the same
'free' port before either recorded it — handing lport == rport to both
ends. uBridge sets SO_REUSEADDR on UDP NIO sockets, so the double bind
succeeds silently and the kernel delivers everything to the last-bound
socket: one node starves, the other echoes to itself.

Make every TCP/UDP allocate/reserve/release path atomic with an RLock,
and rebuild _link_data in UDPLink._prepare so reset() commits the fresh
port pair instead of re-sending the stale, already-released one.

Regression tests: threaded barrier allocation never returns duplicates
(red on the old code, UDP and TCP); reset() leaves exactly one mirrored
NIO pair per side with lport != rport (red on the old code).
2026-08-14 21:31:58 +08:00
YueGuobin
e8b51aa12c
docker: also null-bind udevadm in GNS3_MASK_UDEV
Masking the udev systemd units stopped the daemon's coldplug (host audio
resets), but host USB devices still reconnected on every XRd start. A/B
testing with plain `docker run` isolated the trigger: XRd's own
xr_startup.sh calls udevadm directly (USB license-dongle probing, e.g.
`udevadm trigger --action=add --parent-match=<usb device>`), which
synthesizes uevents into the host kernel from the privileged container --
no udevd required.

GNS3_MASK_UDEV=1 now also binds /dev/null over the udevadm binary
(/bin, /sbin, /usr/bin). Verified with a plain-run experiment: with the
bind, host udev monitor shows zero usb/input/hid/sound events during XRd
boot (only normal docker veth traffic), and XRd itself boots to running
state -- it does not need udevadm under GNS3 (interfaces are pre-created
veths).
2026-08-14 19:09:48 +08:00
YueGuobin
347537f1b3
docker: mask systemd-udevd in privileged containers (GNS3_MASK_UDEV)
A privileged systemd-based NOS container (Cisco XRd boots /usr/sbin/init)
runs systemd-udevd, which on startup coldplugs every device it can reach.
In privileged mode that includes the HOST's USB/input/audio/disk devices,
so every XRd start reconnects USB, mutes audio, and disrupts the host
journal -- highly disruptive on Linux desktops (caught in the act: the
container's udevd was even rescanning the host BTRFS root device).

XRd doesn't need udev (its interfaces are pre-created by GNS3 veth and
mapped via XR_INTERFACES). Add two opt-in env vars, consumed host-side at
container create time in the inherited DockerVM.create (so VendorDockerVM
nodes get it too):

  GNS3_MASK_UDEV=1        -> bind /dev/null over the udevd unit, its two
                             activation sockets, and the coldplug/settle
                             trigger services
  GNS3_MASK_SYSTEMD=u1,u2 -> bind /dev/null over arbitrary units in
                             /etc/systemd/system/ (comma/semicolon list)

Only injected when set, so ordinary nodes are unaffected.
2026-08-14 01:12:21 +08:00
YueGuobin
08e37a4509
docker: inject config files into containers via extra_configs
Add an `extra_configs` field (list of {target, content}) to the docker
node/template/appliance schemas. For each entry GNS3 writes `content` to a
file in the node working directory and bind-mounts it read-only at `target`
inside the container.

This lets a NOS appliance seed its startup config without rebuilding the
image: XRd points XR_FIRST_BOOT_CONFIG at an injected /firstboot.cfg, FRR at
/etc/frr/frr.conf, etc. The bind is a single-file mount applied at create
time, so it works for both the generic init.sh path and vendor nodes that
skip init.sh (console_type=docker_exec). Entries are only injected when
present, so ordinary nodes are unaffected.

The content can't go through `environment` (it is line-delimited, one var per
line), hence a dedicated field -- the same plumbing shape as extra_volumes.
2026-08-13 23:33:34 +08:00
YueGuobin
2bee34031c
docker: warn about low inotify/file-max and missing FUSE at connect
Add a read-only _check_host_readiness() that runs once after the Docker
daemon connection is established. It reads /proc/sys inotify/file-max
limits and /proc/filesystems (for FUSE), and logs a warning with the exact
commands to fix when they are too low for heavy containers -- XRd wants
~4000 inotify instances per node against a stock default of 128.

The server runs unprivileged (only the setuid ubridge helper has root), so
it can only check, not set; the warning tells the admin exactly what to
raise once. Stays silent when the limits are already sufficient.
2026-08-13 22:39:48 +08:00
YueGuobin
86d30f34b4
docker: inject /dev/shm size and host devices via HostConfig from env
Heavy NOS containers (e.g. Cisco XRd) need /dev/shm larger than Docker's
64 MB default and host device nodes such as /dev/fuse. Add two opt-in
environment variables, consumed host-side and applied as native Docker
HostConfig keys at create time:

  GNS3_SHM_SIZE (MB)  -> HostConfig.ShmSize (bytes)
  GNS3_DEVICES        -> HostConfig.Devices in `docker run --device` syntax
                         (host[:container[:perm]]; Docker resolves major/minor
                         from the host node itself)

Native HostConfig (rather than remount/mknod inside init.sh) is used so this
works for vendor NOS nodes that skip init.sh (console_type=docker_exec) --
the path XRd must take, since GNS3's init.sh wrapper crashes XRd's glibc
loader. It applies whether or not init.sh runs, needs no schema/API/UI
change (reuses the `environment` field), and only takes effect when the vars
are set, so ordinary nodes keep default Docker behaviour.

GNS3_-prefixed user env vars stay dropped from the container environment
(only consumed here host-side), keeping GNS3-injected vars safe.
2026-08-13 22:39:48 +08:00
YueGuobin
8889cea38b
fix: recreate docker_exec console on reconnect after CLI exit
The reconnect-blank-screen bug: when sr_cli exited (quit / idle timeout /
crash) the while-true wrapper restarted it mid-session with no client
attached, so its startup CPR probe (\e[6n) went unanswered and the TUI
degraded/blocked. On reconnect lazy_started=True skipped recreation, so the
client saw a blank screen.

Fix: drop the while-true wrapper. Now when the CLI exits, the exec pty
closes (EOF), the broadcast task ends, and the next client connection
detects the dead upstream via _upstream_alive() and recreates the exec —
with a terminal attached, so CPR is answered. A live exec is reused
(just a Ctrl-L redraw).

_LazyExecTelnetServer is extracted from a closure to module level so the
reconnect/recreate logic is unit-testable. Add 9 tests covering
_upstream_alive states and the recreate-on-death / reuse-if-live /
close-half-dead-writer / no-while-true behaviors.

Full Docker suite (120) passes.
2026-08-13 02:05:17 +08:00
YueGuobin
1e154ac85f
test: add tests for VendorDockerVM and the docker_exec factory
25 tests covering:
- Docker.create_node factory: selects VendorDockerVM iff console_type ==
  docker_exec, DockerVM otherwise (including telnet/ssh/vnc/http/none/spice)
- GNS3_* env parsing: SKIP_INIT, INTERFACE_NAMES, CONSOLE_CMD (single and
  multiline), defaults
- create(): init.sh skipped under GNS3_SKIP_INIT, prepended otherwise;
  GNS3_MAX_ETHERNET follows the interface rename; /etc/network mount dropped
  under SKIP_INIT (and host skeleton dir removed) but kept without it
- _add_ubridge_connection: move_to_ns targets the renamed interface
  (mgmt0) or falls back to eth{N}
- start(): docker_exec console dispatch + SKIP_INIT volume bridge + permission
  fix; without SKIP_INIT the vendor passes are skipped
- _fix_permissions: skips dead/missing containers (no restart), targets
  /gns3volumes bind-mount paths
- _setup_skip_init_volumes: runs the docker exec bridge script
- _cleanup_console_resources: closes the exec pty writer

Full Docker suite (111) and compute suite (395) pass — the four hook
extractions in DockerVM introduce no regressions.
2026-08-13 02:05:17 +08:00
Jeremy Grossmann
913ff3823a
Merge branch '3.1' into update-dependencies 2026-08-11 18:48:54 +02:00
grossmj
0219914103
fix: breaking change with dependency FastAPI v0.137.0
https://fastapi.tiangolo.com/release-notes/#specific-breaking-changes
2026-08-11 18:47:02 +02:00
YueGuobin
958aa59d7e
fix: scope marker reconcile to the current bridge/NIO
The reconcile pass in _ubridge_apply_markers walked the node-wide
_marker_filter_bridges map but compared against `desired`, which only
carries the markers of the NIO being updated. Updating any one link
therefore deleted every other link's markers (and their pcaps) on that
node — a regression from the add-only→reconcile switch. IOU's override
had the same flaw across its ports.

Guard the delete pass with the current bridge (base_node) / IOL location
(IOU) so only markers on the NIO being reconciled can be removed. Added
a regression test that fails without the guard.
2026-08-11 22:11:44 +08:00
YueGuobin
958c45b6fc
fix: reconcile marker filters in _ubridge_apply_markers (delete/update)
The batch marker-def fan-out (PR #2848) routed create/update/delete
marker_definition through memory_only + a batch PUT /nios/batch that
re-applies markers via _ubridge_apply_markers. But _ubridge_apply_markers
was strictly add-only: it skipped any (name, link_id) already in
_marker_filter_bridges, and reset_packet_filters preserves mark filters
(contract). So:

  * delete_marker_definition left the deleted marker's filter alive in
    uBridge (still matching / signalling / writing pcap) until node restart.
  * update_marker_definition (bpf/tag/direction change) never reached
    uBridge — the live filter kept the old expression until node restart.

Make _ubridge_apply_markers a real reconcile against the desired
nio.markers:
  - installed but no longer desired  → delete_packet_filter + unlink pcap
                                       + unregister
  - desired with changed filter field → rebuild (delete + re-add)
  - desired with only enabled changed  → instant toggle (pcap preserved)
  - desired and unchanged              → skip
  - desired and new                    → add

Track installed specs in a parallel _marker_specs dict so changes can be
detected. Both base_node and the IOU iol_bridge override are updated.
Added tests for the delete-removed and rebuild-changed-bpf paths.
2026-08-11 21:00:03 +08:00
YueGuobin
2ac0bb7d25
marker: route marker.match to a dedicated project WS channel
High-frequency marker.matches shared the single project notification queue with topology events (node.*/link.*), causing head-of-line blocking. Add a separate marker channel: Notification.project_marker_queue/marker_emit, dispatch routes marker.* off the main project queue, plus a new WS /{project_id}/notifications/markers/ws endpoint. Fully migrated (the main project WS no longer carries marker.match); marker listeners are independent of project auto_close. Compute side unchanged.
2026-08-11 11:19:56 +08:00
YueGuobin
2f70bd6daa
revert: drop _connect_nio thread-pool executor, restore async _ubridge_send
The _connect_nio thread-pool optimisation (send_batch_sync) targeted
node-start performance, but start_all already runs at concurrency=3
(by design, to avoid overwhelming the host).  It also introduced a
Python 3.13 incompatibility (trsock.setblocking forbidden) that
prevented docker nodes from starting.  Since node-start is not the
target of this branch (project-open link creation is), revert to the
simple per-command async _ubridge_send.

The project-open batch NIO dispatch (create_batch_nios) is unaffected —
it never called _connect_nio (nodes aren't started during open).
2026-08-11 01:16:13 +08:00
YueGuobin
4e30b6905a
fix: await Dynamips.create_nio (it is async, unlike sync base)
Dynamips.create_nio is async def while BaseManager.create_nio is a sync
def.  The previous fix only added the extra 'node' argument but did not
await the resulting coroutine, causing 'was never awaited' warnings and
passing a coroutine object instead of an NIO instance to the binding
dispatch.  Add 'await' on the Dynamips branch.  Test updated to verify
both the async nature and the parameter count.
2026-08-11 01:00:01 +08:00
YueGuobin
09a9555d02
fix: use bound-method param count for Dynamips create_nio detection
The inspect check tested unbound function signatures (3 params unbound vs
2 unbound) but node.manager.create_nio is a bound method — inspect
excludes 'self'.  Dynamips bound = 2 (node + nio_settings), standard
bound = 1 (nio_settings).  The old '== 3' never matched, so the extra
'node' arg was never passed.  Switch to '>= 2' and rewrite the test to
exercise the actual bound-method scenario.
2026-08-11 00:58:08 +08:00
YueGuobin
e56488c2f7
test: add _add_nio_binding dispatch tests for Qemu, IOU, VPCS, Builtin
Cover every dispatch branch in the batch NIO endpoint so that future
additions of node types with unusual NIO-binding signatures are caught
at test time.
2026-08-11 00:55:54 +08:00
YueGuobin
ed0f1bb4de
fix: Dynamips create_nio(node, nio_settings) takes extra arg + add tests
Dynamips.create_nio requires the node as first positional argument
(unlike every other manager which takes only nio_settings). The batch
handler now detects this via parameter-count inspection (3 vs 2) and
passes node when needed.

Also add Dynamips to _add_nio_binding dispatch: routers use
slot_add_nio_binding(slot, port, nio), switches/hubs fall back to
add_nio(nio, port_number).

Add tests covering Dynamips router dispatch, switch dispatch, and the
create_nio signature detection to prevent regression.
2026-08-11 00:54:53 +08:00
YueGuobin
38c49a655c
perf: batch NIO dispatch on project open (one HTTP per compute)
Project open used to create each link by issuing two NIO POSTs from the
controller to the compute — ~5000 HTTP round-trips for a 2500-link
topology, all funnelling through the single shared controller/compute
event loop and capping throughput near 12 links/s.

Replace it with a bulk path:
- UDPLink split into _prepare() (local: ports, peer addrs, link_data)
  and _commit_nios() (dispatch). create() = prepare + commit (interactive).
- Link.add_node gains batch=True: attach both nodes without triggering
  per-link NIO HTTP.
- compute: new POST /projects/{id}/nios/batch endpoint with a unified
  _add_nio_binding dispatch across node types (docker/qemu/iou/vpcs/
  builtin differ in signature).
- project.open: prepare all links locally, group NIO entries by compute,
  send each compute a single /nios/batch, then finalise (wire node/port
  refs, mark created, notify, apply marker defs) in parallel.

Cuts controller->compute HTTP from O(links) to O(computes). Test added
for the batch endpoint.
2026-08-10 23:42:50 +08:00
YueGuobin
05934fa8e8
perf: offload per-NIO ubridge commands to thread-pool executor
Replace the 3-5 sequential await _ubridge_send calls in _connect_nio
with a single run_in_executor batch.  The batch holds the node-level
asyncio Lock to prevent interleaving with async sends, then uses the
hypervisor's new send_batch_sync method which does blocking socket
sendall/recv inside the thread pool.  Different nodes' batches now
run in true OS-thread parallelism rather than serialising through
the asyncio event loop between every command.

- ubridge_hypervisor.send_batch_sync: blocking batch send using
  the underlying socket from the asyncio transport, protected by
  threading.Lock.
- _connect_nio: builds command list (add_nio_udp, start_capture,
  bridge start, reset_packet_filters, add_packet_filter) and
  dispatches to the default executor.
2026-08-10 23:06:54 +08:00
YueGuobin
e7dfe9c7c4
fix: SIGKILL docker container on stop instead of 5s grace period
Docker node stop took ~5s every time. The stop API grace period
(params t=5, unchanged since 2015) was always exhausted: the business
process (often an interactive shell) ignores SIGTERM, and GNS3 doesn't
depend on graceful shutdown — _fix_permissions and /gns3volumes already
persist container state before stop() is called.

Use POST /containers/{id}/kill (SIGKILL, zero delay) instead of stop.
The 409 (container already stopped) replaces the previous 304 handling
for the race where the container exits between the state check and the call.

t=5 traced to commit 33edbefa3 (2015-10-14) "Docker cleanup and
improvements" — introduced with no recorded rationale.
2026-08-10 21:48:26 +08:00
Jeremy Grossmann
909ccf8fcd
Merge branch '3.1' into base-configs-3.0 2026-08-08 22:11:39 +02:00
grossmj
544d15c87e
fix: tests after merging 2026-08-08 16:50:19 +02:00
grossmj
dff79f65d3
Merge branch '2.2' into 3.1
# Conflicts:
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/controller/import_project.py
#	gns3server/controller/project.py
#	gns3server/crash_report.py
#	gns3server/version.py
#	tests/compute/docker/test_docker_vm.py
#	tests/controller/test_import_project.py
#	tests/controller/test_project.py
2026-08-08 16:27:14 +02:00
YueGuobin
078cf92aef
marker: concurrent (bounded) definition fan-out
The per-definition fan-out applied markers to links in a serial loop -- one compute round-trip per link. On a 1000-link project that serializes N HTTP round-trips (minutes on remote computes). Fan out with asyncio.gather + Semaphore(32): links are independent (own _markers/_link_data), per-link ControllerError stays isolated, and Project.dump is synchronous + atomic (tmp + rename) so concurrent dumps cannot corrupt the topology file.

Converts the definition-create fan-out, the definition-update sync and re-fan-out loops, and the definition-delete cleanup to the shared _marker_apply_concurrently helper. apply_defs_to_new_link stays serial deliberately: all definitions share one link and each push carries the link's full marker set, so concurrent pushes would race and lose markers.
2026-08-07 23:50:41 +08:00
YueGuobin
3322233658
marker: serial-link (WAN) support via data_link_type -> uBridge linktype
Markers now work on serial links (Cisco HDLC / PPP / Frame Relay / ATM), not just Ethernet. A marker carries a data_link_type (default DLT_EN10MB); at the uBridge boundary it becomes the 'mark ... linktype <dlt>' keyword so the BPF compiles and the pcap is written with the matching link-layer.

- MarkerCreate / MarkerDefinitionCreate gain data_link_type (default DLT_EN10MB). Per-link it is create-only; definitions are updatable (a change re-fans-out).
- base_node._marker_linktype() normalizes the GNS3 DLT name (strip DLT_, uppercase, None for EN10MB). Single source is SerialPort.data_link_types, so Cisco PPP -> PPP_SERIAL (50), matching the capture path -- no second mapping table.
- _ubridge_add_marker_filter (generic) and the IOU marker loop append 'linktype <dlt>'.
- Definition fan-out branches on link_type in inherit_marker: Ethernet is always EN10MB; a serial link uses the definition's WAN encapsulation, or is SKIPPED when none was chosen (an EN10MB pcap on serial is undecodable). One definition covers a mixed topology.
- MCP marker_definition exposes data_link_type (None = not forwarded).
- No uBridge rebuild on a data_link_type change -- only that one marker's filter is swapped (delete + re-add), mirroring a BPF change; reset_packet_filters preserves sibling mark filters.

Requires the uBridge build with 'mark ... linktype' support.
2026-08-07 01:53:33 +08:00
YueGuobin
7ed4eeab29
mcp: drop direction from marker_definition tool
A marker definition fans out to every link and auto-selects its capture node
on each, so tx/rx is relative to a node that varies per link — the controller
already rejects it (409). Exposing direction on the MCP definition tool let an
agent ask for something that could only fail. Remove the parameter and the
handler's direction handling; the docstring now points to encoding direction in
the BPF (e.g. 'icmp and icmp[icmptype]==8'). Per-link link_marker keeps
direction, where the capture node is fixed.
2026-08-05 01:40:22 +08:00
YueGuobin
f7b19dae99
marker: tighten marker name max length from 128 to 32
128 characters was far beyond any realistic marker label (icmp, arp, tcp-syn)
and would have collided with the pcap filename budget once a tag prefix is
added later. Cap the user-facing name at 32 in both MarkerCreate and
MarkerDefinitionCreate; the compute-side name guard now also rejects names
longer than 48, which covers the `global-{def_name}` inherited form (≤ 39).
2026-08-05 00:49:37 +08:00
YueGuobin
75f278228b
marker: drop deleted marker from port NIO cache to stop empty pcap on restart
Deleting a marker while its node was stopped, then starting the node, recreated
an empty pcap. Root cause: delete_marker_capture removed the uBridge filter and
the pcap file but not the marker spec cached on the port NIO (nio.markers) —
the data source _ubridge_apply_markers reads on node start. The stale spec
reinstalled the marker when uBridge came up.

This was a regression from switching stop_marker off update() (which re-sent
the NIO and implicitly refreshed nio.markers) to the fine-grained
node.delete(/markers/{name}) path.

Fix: make the delete port-aware so the compute can locate the NIO. The DELETE
marker route becomes /adapters/{a}/ports/{p}/markers/{name} across all six
node types; the handler resolves the NIO via get_nio and passes it to
delete_marker_capture, which now pops the marker from nio.markers. get_nio
works regardless of uBridge state, so the stopped-node case is covered. The
controller's stop_marker targets the capture side's adapter/port.
2026-08-05 00:08:12 +08:00
YueGuobin
1a0ce51f38
marker: validate def BPF once, skip re-validation on inherited fan-out
A marker definition validated its BPF N times — once per link in the fan-out
(start_marker runs validate_bpf_syntax on every copy), spawning one tcpdump -d
subprocess per link for the same expression. Definitions did not validate BPF
at all; only direction was checked.

Move the single validation point to the definition layer (create/update), and
validate each definition's BPF on project load (dropping any that have gone
invalid, like private markers). The inherited fan-out (start_marker) and def
sync (update_marker) now skip validate_bpf_syntax for inherited copies, since
the BPF comes from an already-validated definition. Private per-link markers
still validate inline as before. uBridge still runs pcap_compile at install, so
an invalid expression can never slip through.

Creating a definition over N links now runs one tcpdump instead of N.
2026-08-04 23:20:58 +08:00
YueGuobin
95824b1861
marker: incremental apply, clear bridge map on uBridge stop
_ubridge_apply_markers now installs only markers not already on the bridge
(uBridge's reset_packet_filters preserves mark filters), so an NIO update no
longer re-adds — and reopens — sibling markers' pcaps. _stop_ubridge clears
_marker_filter_bridges so a node restart re-installs everything (the map would
otherwise keep stale entries pointing at a fresh, empty uBridge).
2026-08-04 22:19:43 +08:00
YueGuobin
caec71aa71
marker: fine-grained filter ops, clean pcap on remove
Deleting or updating a marker no longer triggers a full NIO reapply
(reset_packet_filters + re-add), which closed/reopened every sibling
marker's pcap via uBridge. Instead operate on single filters:

- stop_marker: bridge delete_packet_filter + unlink the pcap (works with
  the node stopped; filter removal is skipped, the file is still deleted).
- update_marker: bpf/tag/direction → rebuild just that filter (delete + add);
  enabled → instant toggle; color/highlight_duration → stored only.
- compute delete_marker_capture / rebuild_marker_filter + per-node routes
  (DELETE /markers/{name}, PUT /markers/{name}/rebuild) + MarkerRebuild schema.

IOU overrides _ubridge_delete_marker_filter for iol_bridge; rebuild reuses
the already-overridden add/delete/set, so IOU needs no rebuild override.
2026-08-04 21:35:31 +08:00