Compare commits

...
1280 Commits
Author SHA1 Message Date
grossmj 582412bac3 Release v3.1.0a5 2026-08-19 18:06:12 +02:00
grossmj 51c6440403 Bundle web-ui v3.1.0a5 2026-08-19 17:57:42 +02:00
grossmj 08d7ba6bd0 chore: upgrade dependencies 2026-08-19 17:44:38 +02:00
grossmj 52de01523f Merge branch '2.2' into 3.1
# Conflicts:
#	gns3server/compute/builtin/nodes/cloud.py
#	gns3server/utils/interfaces.py
2026-08-18 18:05:56 +02:00
grossmj 9db55cafec Merge branch 'master' into 2.2 2026-08-18 18:03:56 +02:00
grossmj c85551ec09 Sync appliances 2026-08-18 18:02:53 +02:00
Jeremy Grossmann a3e85b7041 Merge pull request #2853 from hjicks/master
add support basic support for OpenBSD.
2026-08-18 18:01:15 +02:00
Jeremy Grossmann 80a69814b3 Merge pull request #2854 from yueguobin/code-review-fixes
docker: vendor NOS containers (XRd, SR Linux) as first-class nodes; UDP port race fix
2026-08-15 18:24:13 +02:00
YueGuobin f43a717b20 copilot: sync CONSOLE_TYPES with the server ConsoleType enum
The vendored gns3fy copy keeps its type lists as literals (the module is
shared with the standalone MCP service and cannot import server enums),
and CONSOLE_TYPES had drifted: 'ssh' and 'docker_exec' were missing while
both are valid server-side. Impact: the copilot topology reader validates
the whole node list in one pydantic pass, so a single vendor NOS node
(console_type 'docker_exec') made it drop the entire project and return
zero devices to every copilot device tool.

Add the missing values plus drift tests asserting the vendored lists
cover the server enums (skipped when ai-features extras are absent).
2026-08-15 23:32:24 +08:00
Guobin Yue 74a51a1f9f Merge branch '3.1' into code-review-fixes 2026-08-15 22:39:42 +08:00
Jeremy Grossmann 74191e7a8c Merge pull request #2855 from cristian-ciobanu/3.1
feat: Enhance information displayed on the system status dashboard
2026-08-14 23:33:47 +02:00
Cristi a35df76384 feat: Enhance information displayed on the system status dashboard 2026-08-14 23:36:03 +03:00
YueGuobin 110e041e56 docker: cap GNS3_STOP_TIMEOUT at 210 s (controller stop budget)
The 600 s clamp was unreachable in practice: the controller's stop
request times out at 240 s (controller/node.py) and the Docker stop
query gets the value +30 s as its HTTP timeout, so anything above 210
would abort upstream first and surface an error while the stop keeps
running server-side. Cap at the derived ceiling and document the chain
in the clamp and the docstring.
2026-08-15 01:25:50 +08:00
YueGuobin 9604c85fda docker: harden the shm/devices/extra_configs/masking work (code review)
Nine fixes from a review of the docker-shm-devices diff:

* GNS3_STOP_TIMEOUT >300 s aborted at the manager's default HTTP timeout
  before Docker finished the stop — the stop query now gets a timeout
  with a margin over the grace period.
* Overlapping bind targets (GNS3_MASK_UDEV + GNS3_MASK_SYSTEMD on the
  same unit, a unit named twice, an extra_configs target equal to a
  masked unit) made Docker reject the create with 'Duplicate mount
  point' — Mounts are deduplicated by target.
* ExtraConfig.target now carries a pydantic validator (absolute file
  path, no '..'), so bad targets 422 at template-save time instead of
  failing at node-create time after a multi-GB image pull; directory
  forms ('/', '/etc/') are also rejected by the runtime guard instead
  of raising IsADirectoryError (raw 500).
* _check_host_readiness skipped every remaining check when one
  /proc/sys key was unreadable (mid-loop return) — now continues.
* The base-class GNS3_* env parser strips trailing commas like the
  vendor parser, so 'GNS3_MASK_UDEV=1,' composed from a list still
  activates.
* Vendor env knobs are re-parsed on every create(), so a PUT to the
  node's environment takes effect on the next (re)create.
* The graceful SIGTERM stop is now limited to the explicit user stop
  route; delete/update/close/crash-cleanup keep the immediate kill
  (those paths force-delete or recreate the container right after).
* An extra_configs target beneath a persisted volume is shadowed by the
  volume bind — warn at create time.
2026-08-15 00:52:55 +08:00
YueGuobin 1124d7a539 docs: XRd appliance tunes GNS3_STOP_TIMEOUT to 40 s; note version-agnostic template image 2026-08-15 00:09:53 +08:00
YueGuobin 62076c1727 docker: make the vendor graceful-stop grace period configurable (GNS3_STOP_TIMEOUT)
The 60 s SIGTERM grace was hardcoded, unlike every other vendor knob
(GNS3_SHM_SIZE, GNS3_DEVICES, GNS3_MASK_UDEV, ...) which rides the
environment line. Parse GNS3_STOP_TIMEOUT=<seconds> (default 60,
clamped to 1-600, invalid values keep the default) and use it in
VendorDockerVM._terminate_container().
2026-08-14 22:11:34 +08:00
YueGuobin e9339faaa7 docker: graceful stop for vendor NOS containers (SIGTERM + 60s grace)
DockerVM.stop() terminated containers with an immediate SIGKILL — fine
for init.sh-based containers whose state is persisted beforehand, but a
systemd NOS (Cisco XRd, SR Linux) needs a graceful shutdown and treats
the abrupt kill as an unclean shutdown (exit 137 on every stop).

Extract the final termination into _terminate_container() and override
it in VendorDockerVM: POST /containers/{id}/stop?t=60 sends SIGTERM and
waits for systemd to stop services; Docker itself SIGKILLs the
container once the grace period expires, so no fallback is needed.
Docker's 304 (already stopped) is swallowed.
2026-08-14 22:02:58 +08:00
YueGuobin 350f2b24e7 fix: UDP port allocation race causing link self-loop (one-way links)
PortManager.get_free_udp_port had an unguarded find-then-add sequence.
A link allocates both ends concurrently (asyncio.gather in
UDPLink._prepare -> two POST /ports/udp) and FastAPI runs the sync
route handler in a threadpool, so both threads could probe the same
'free' port before either recorded it — handing lport == rport to both
ends. uBridge sets SO_REUSEADDR on UDP NIO sockets, so the double bind
succeeds silently and the kernel delivers everything to the last-bound
socket: one node starves, the other echoes to itself.

Make every TCP/UDP allocate/reserve/release path atomic with an RLock,
and rebuild _link_data in UDPLink._prepare so reset() commits the fresh
port pair instead of re-sending the stale, already-released one.

Regression tests: threaded barrier allocation never returns duplicates
(red on the old code, UDP and TCP); reset() leaves exactly one mirrored
NIO pair per side with lport != rport (red on the old code).
2026-08-14 21:31:58 +08:00
YueGuobin 1cddeb8c3f docs: record the link UDP self-loop bug; mark XRd datapath validated
Add bugs/link-udp-self-loop.md: the intermittent one-way docker link
observed during XRd validation (one end's nio_udp rport pointing at its
own lport after a uBridge restart), with the evidence table, the uBridge
console capture diagnostics, the delete/recreate workaround, and the
narrowed suspects (batch port preallocation / link re-creation race).
Also note that a docker node's in-container ethN is a TAP device held by
uBridge (no veth host end exists).

Update the XRd feature doc: datapath validated end-to-end (XRd brings
its own interfaces up, ARP/ICMP bidirectional), add a troubleshooting
row pointing at the bug doc. Index the bug in docs/README.md.
2026-08-14 19:50:58 +08:00
YueGuobin 0f12786885 docs: document the XRd control-plane adaptation
New feature doc covering why XRd takes the vendor docker_exec/SKIP_INIT
path (init.sh wrapper crashes its glibc loader), the four generic
mechanisms added for it (GNS3_SHM_SIZE/GNS3_DEVICES HostConfig injection,
extra_configs file injection, GNS3_MASK_UDEV + udevadm null-binding,
host-readiness check), the three host-disturbance root causes isolated by
plain docker-run A/B/C testing, the appliance recipe with XRd-specific
gotchas (Mg0/RP0/CPU0/0, /xr-storage-shadow persistence, first-boot
semantics), and troubleshooting. Indexed in docs/README.md alongside the
docker-exec-console base doc.
2026-08-14 19:14:59 +08:00
YueGuobin e8b51aa12c docker: also null-bind udevadm in GNS3_MASK_UDEV
Masking the udev systemd units stopped the daemon's coldplug (host audio
resets), but host USB devices still reconnected on every XRd start. A/B
testing with plain `docker run` isolated the trigger: XRd's own
xr_startup.sh calls udevadm directly (USB license-dongle probing, e.g.
`udevadm trigger --action=add --parent-match=<usb device>`), which
synthesizes uevents into the host kernel from the privileged container --
no udevd required.

GNS3_MASK_UDEV=1 now also binds /dev/null over the udevadm binary
(/bin, /sbin, /usr/bin). Verified with a plain-run experiment: with the
bind, host udev monitor shows zero usb/input/hid/sound events during XRd
boot (only normal docker veth traffic), and XRd itself boots to running
state -- it does not need udevadm under GNS3 (interfaces are pre-created
veths).
2026-08-14 19:09:48 +08:00
chara 80da7da515 add support basic support for OpenBSD.
there remains some work on ubridge, but this seems to work so far.
2026-08-13 22:24:58 +03:30
YueGuobin b928a2f48a docker: use the container's chown (not busybox) in vendor volume/perms path
The vendor skip-init path (_setup_skip_init_volumes, _fix_permissions) runs
`/gns3/bin/busybox chown` inside the container via docker exec. busybox is
statically linked, and its chown dlopens NSS modules (libnss_*) from the
container; on NOS images whose glibc differs from the host's (e.g. Cisco
XRd) that mismatches and aborts with the glibc assertion
`_dl_call_libc_early_init: sym != NULL` (SIGABRT). The per-file chown loop
then crash-loops, and the resulting core-dump storm -- processed by the
host's systemd-coredump -- cascades into host device rescans, reconnecting
USB / resetting audio / corrupting the journal on every XRd start.

cp/chmod/find/stat don't touch NSS and work fine on busybox, so only chown
is affected. Prefer the container's own coreutils chown
(`command -v chown && chown ...`), falling back to busybox chown only when
the container ships no chown (minimal images, where the glibc matches and
busybox is safe).
2026-08-14 01:27:18 +08:00
YueGuobin 347537f1b3 docker: mask systemd-udevd in privileged containers (GNS3_MASK_UDEV)
A privileged systemd-based NOS container (Cisco XRd boots /usr/sbin/init)
runs systemd-udevd, which on startup coldplugs every device it can reach.
In privileged mode that includes the HOST's USB/input/audio/disk devices,
so every XRd start reconnects USB, mutes audio, and disrupts the host
journal -- highly disruptive on Linux desktops (caught in the act: the
container's udevd was even rescanning the host BTRFS root device).

XRd doesn't need udev (its interfaces are pre-created by GNS3 veth and
mapped via XR_INTERFACES). Add two opt-in env vars, consumed host-side at
container create time in the inherited DockerVM.create (so VendorDockerVM
nodes get it too):

  GNS3_MASK_UDEV=1        -> bind /dev/null over the udevd unit, its two
                             activation sockets, and the coldplug/settle
                             trigger services
  GNS3_MASK_SYSTEMD=u1,u2 -> bind /dev/null over arbitrary units in
                             /etc/systemd/system/ (comma/semicolon list)

Only injected when set, so ordinary nodes are unaffected.
2026-08-14 01:12:21 +08:00
YueGuobin 088f1da77a docker: persist extra_configs in the template DB table
The extra_configs schema field needs a column on the docker_templates table
to actually round-trip through the controller DB (the schema alone is
accepted but dropped by the SQLAlchemy model mapping). Add the JSON column
and an Alembic migration so existing databases get it on upgrade.
2026-08-14 00:20:08 +08:00
YueGuobin 08e37a4509 docker: inject config files into containers via extra_configs
Add an `extra_configs` field (list of {target, content}) to the docker
node/template/appliance schemas. For each entry GNS3 writes `content` to a
file in the node working directory and bind-mounts it read-only at `target`
inside the container.

This lets a NOS appliance seed its startup config without rebuilding the
image: XRd points XR_FIRST_BOOT_CONFIG at an injected /firstboot.cfg, FRR at
/etc/frr/frr.conf, etc. The bind is a single-file mount applied at create
time, so it works for both the generic init.sh path and vendor nodes that
skip init.sh (console_type=docker_exec). Entries are only injected when
present, so ordinary nodes are unaffected.

The content can't go through `environment` (it is line-delimited, one var per
line), hence a dedicated field -- the same plumbing shape as extra_volumes.
2026-08-13 23:33:34 +08:00
YueGuobin 2bee34031c docker: warn about low inotify/file-max and missing FUSE at connect
Add a read-only _check_host_readiness() that runs once after the Docker
daemon connection is established. It reads /proc/sys inotify/file-max
limits and /proc/filesystems (for FUSE), and logs a warning with the exact
commands to fix when they are too low for heavy containers -- XRd wants
~4000 inotify instances per node against a stock default of 128.

The server runs unprivileged (only the setuid ubridge helper has root), so
it can only check, not set; the warning tells the admin exactly what to
raise once. Stays silent when the limits are already sufficient.
2026-08-13 22:39:48 +08:00
YueGuobin 86d30f34b4 docker: inject /dev/shm size and host devices via HostConfig from env
Heavy NOS containers (e.g. Cisco XRd) need /dev/shm larger than Docker's
64 MB default and host device nodes such as /dev/fuse. Add two opt-in
environment variables, consumed host-side and applied as native Docker
HostConfig keys at create time:

  GNS3_SHM_SIZE (MB)  -> HostConfig.ShmSize (bytes)
  GNS3_DEVICES        -> HostConfig.Devices in `docker run --device` syntax
                         (host[:container[:perm]]; Docker resolves major/minor
                         from the host node itself)

Native HostConfig (rather than remount/mknod inside init.sh) is used so this
works for vendor NOS nodes that skip init.sh (console_type=docker_exec) --
the path XRd must take, since GNS3's init.sh wrapper crashes XRd's glibc
loader. It applies whether or not init.sh runs, needs no schema/API/UI
change (reuses the `environment` field), and only takes effect when the vars
are set, so ordinary nodes keep default Docker behaviour.

GNS3_-prefixed user env vars stay dropped from the container environment
(only consumed here host-side), keeping GNS3-injected vars safe.
2026-08-13 22:39:48 +08:00
Jeremy Grossmann f65d9b17c6 Merge pull request #2851 from yueguobin/fix/docker-node-create-setattr-noise
fix: suppress redundant console port setter log during Docker node create
2026-08-12 23:02:49 +02:00
Jeremy Grossmann c588652768 Merge pull request #2852 from yueguobin/docker-srlinux-support
Support vendor NOS Docker containers: docker_exec console + SR Linux integration
2026-08-12 22:49:05 +02:00
YueGuobin 64657918b5 docs: appliance packaging section + symbol theme caveat
Document the SR Linux gns3a (35-adapter full chassis, matching
GNS3_INTERFACE_NAMES + custom_adapters), the three server-side schema fixes
needed for it to load (DockerConsoleType.docker_exec,
ApplianceV1_6.custom_adapters, extra_volumes docker-block passthrough), and
the symbol-theme behaviour that rewrites any :/symbols/-prefixed symbol to
the category default at load time (so router_cloud.svg cannot be used from
an appliance; use a custom symbol under symbols_path instead).
2026-08-13 02:05:17 +08:00
YueGuobin 9e830f7883 appliance: expose custom_adapters on the v1-6 appliance model
The top-level ApplianceV1_6 model declared first_port_name /
port_name_format / port_segment_size but not custom_adapters, so the
GET /appliances endpoint (response_model=schemas.Appliance) stripped
custom_adapters from the response — the frontend never saw per-adapter
port names even though the appliance file and the server-side template
conversion (appliance_to_template reads it from the raw dict) handled it.

Add custom_adapters: Optional[List[CustomAdapterItem]] to ApplianceV1_6
so the field round-trips through the API. (ApplianceV8 already models it
inside its TemplateSetting.)
2026-08-13 02:05:17 +08:00
YueGuobin 141b3d8701 appliance: accept docker_exec console type in Docker appliances
The Docker appliance Pydantic model (DockerConsoleType) rejected
console_type='docker_exec', so an appliance file using the vendor NOS
docker_exec console could not be loaded (validation error at import).
Add docker_exec to the enum — it is already a valid ConsoleType
(schemas/common.py) and is handled by VendorDockerVM.
2026-08-13 02:05:17 +08:00
YueGuobin 8889cea38b fix: recreate docker_exec console on reconnect after CLI exit
The reconnect-blank-screen bug: when sr_cli exited (quit / idle timeout /
crash) the while-true wrapper restarted it mid-session with no client
attached, so its startup CPR probe (\e[6n) went unanswered and the TUI
degraded/blocked. On reconnect lazy_started=True skipped recreation, so the
client saw a blank screen.

Fix: drop the while-true wrapper. Now when the CLI exits, the exec pty
closes (EOF), the broadcast task ends, and the next client connection
detects the dead upstream via _upstream_alive() and recreates the exec —
with a terminal attached, so CPR is answered. A live exec is reused
(just a Ctrl-L redraw).

_LazyExecTelnetServer is extracted from a closure to module level so the
reconnect/recreate logic is unit-testable. Add 9 tests covering
_upstream_alive states and the recreate-on-death / reuse-if-live /
close-half-dead-writer / no-while-true behaviors.

Full Docker suite (120) passes.
2026-08-13 02:05:17 +08:00
YueGuobin 1e154ac85f test: add tests for VendorDockerVM and the docker_exec factory
25 tests covering:
- Docker.create_node factory: selects VendorDockerVM iff console_type ==
  docker_exec, DockerVM otherwise (including telnet/ssh/vnc/http/none/spice)
- GNS3_* env parsing: SKIP_INIT, INTERFACE_NAMES, CONSOLE_CMD (single and
  multiline), defaults
- create(): init.sh skipped under GNS3_SKIP_INIT, prepended otherwise;
  GNS3_MAX_ETHERNET follows the interface rename; /etc/network mount dropped
  under SKIP_INIT (and host skeleton dir removed) but kept without it
- _add_ubridge_connection: move_to_ns targets the renamed interface
  (mgmt0) or falls back to eth{N}
- start(): docker_exec console dispatch + SKIP_INIT volume bridge + permission
  fix; without SKIP_INIT the vendor passes are skipped
- _fix_permissions: skips dead/missing containers (no restart), targets
  /gns3volumes bind-mount paths
- _setup_skip_init_volumes: runs the docker exec bridge script
- _cleanup_console_resources: closes the exec pty writer

Full Docker suite (111) and compute suite (395) pass — the four hook
extractions in DockerVM introduce no regressions.
2026-08-13 02:05:17 +08:00
YueGuobin f5554d816a vendor: drop the hardcoded /etc/network mount for SKIP_INIT containers
Override _mount_binds in VendorDockerVM: for GNS3_SKIP_INIT containers the
/etc/network volume (GNS3's own network config consumed by init.sh's ifup)
is dead weight — init.sh never runs and the NOS manages its own
interfaces. The override removes the bind, filters /etc/network out of
self._volumes (keeping GNS3_VOLUMES and the bridge/fix passes consistent)
and deletes the host-side skeleton directory created by the base class.
Without GNS3_SKIP_INIT the mount is kept, matching base behaviour.
_persistent_volumes() is removed — the mount override is now the single
filter point.
2026-08-13 02:05:17 +08:00
YueGuobin 46e863b975 vendor: skip /etc/network in volume bridge and permission passes
GNS3_SKIP_INIT containers never run init.sh, so /etc/network (GNS3's own
network config consumed by init.sh's ifup) has no consumer — the NOS
manages its own interfaces. VendorDockerVM._persistent_volumes() filters it
out for both _setup_skip_init_volumes and _fix_permissions, saving one
docker exec per pass. The shared _mount_binds is untouched, and without
GNS3_SKIP_INIT the full volume list is returned so behaviour matches the
base class.
2026-08-13 02:05:17 +08:00
YueGuobin 486c5cc1cb docs: runtime ownership safety and boot-ordering caveat
Document why host-user ownership of volume files during runtime is
harmless for SR Linux (root processes, self-healing daemons like aaamgr
rewriting its managed files, ACL-based access) and the deviation from the
standard init.sh model, with the escape hatch of dropping the start-time
fix pass for strict-ownership NOS images.

Also document the boot-ordering caveat: the volume bridge comes up after
the NOS boots, so early boot reads overlay defaults — verify the
save/stop/start closed loop, and add troubleshooting entry #10 for config
present on the host but not applied after restart.
2026-08-13 02:05:17 +08:00
YueGuobin 6f852c83fd docs: explain mid-run Permission denied warning for boot-written NOS files
Document the aaamgr_local_user.json case: SR Linux's aaamgr daemon rewrites
the file during boot as the image's srlinux user (uid 1002) after the
start-time permission pass, leaving it unreadable until the stop-time pass.
Trace the warning to the file-browser API chain (Show in file manager ->
list_node_files -> magic.from_file) and note the impact is limited to the
file_type field.
2026-08-13 02:05:17 +08:00
YueGuobin 3455da7da3 fix: run _fix_permissions container-side on /gns3volumes mount targets
The host-side pass could not work for unprivileged GNS3 processes: the
.gns3_perms marker is created root-owned by the container-side touch, and
chowning root-owned files from the host requires root.

Rewrite VendorDockerVM._fix_permissions to run the busybox
record/chmod/chown script inside the container (as root) on the
/gns3volumes bind-mount targets — they exist for the container's whole
lifetime and do not depend on the mount --bind bridge, so a container
restart can no longer make the fix hit the overlay copy. A
stopped/exited container is skipped (logged) instead of restarted; the
next start's pass fixes ownership.
2026-08-13 02:05:17 +08:00
YueGuobin 2f36471a55 fix: host-side permission fix + SKIP_INIT volume persistence docs
Replace the container-side _fix_permissions for vendor NOS containers with a
host-side pass that walks the node's project directories directly (they are
the Docker bind-mount sources): records mode:uid:gid into .gns3_perms and
chowns to the GNS3 user. No docker exec, no container restart — the base
implementation restarts an exited container just to chown, and after the
restart the mount --bind bridge is gone so it would fix the overlay copy
instead of the host files.

The pass runs at start (after _setup_skip_init_volumes seeds and bridges the
volumes) so the controller can read project files while the node runs, and
again at stop for files written during runtime.

Update docker-exec-console.md: VendorDockerVM architecture, hook points,
class-selection factory, volume-persistence lifecycle, and new
troubleshooting entries.
2026-08-13 02:05:17 +08:00
YueGuobin 5388fd3796 refactor: move vendor NOS Docker support into VendorDockerVM subclass
Extract the docker_exec console and GNS3_* prototype knobs (SKIP_INIT,
INTERFACE_NAMES, CONSOLE_CMD) from DockerVM into a VendorDockerVM subclass.
DockerVM is restored to its 3.1 baseline plus four small extension hooks
(_prepare_init_and_interface_env, _start_console_server,
_get_container_ifname, _cleanup_console_resources) that are pure
refactorings with zero behaviour change for existing nodes.

VendorDockerVM additionally replicates init.sh's volume persistence
(bind-mount /gns3volumes over the in-container path) via docker exec for
containers that skip init.sh, so vendor NOS config (e.g. /etc/opt/srlinux)
survives node stop/start.

The Docker manager selects VendorDockerVM when console_type == docker_exec;
all other nodes keep using DockerVM unchanged.
2026-08-13 02:05:17 +08:00
YueGuobin 511c52330b docker_exec console: wrap exec command in while-true loop so sr_cli restarts on quit 2026-08-13 02:05:17 +08:00
YueGuobin 486178d05c docs: document the docker_exec console and vendor NOS Docker knobs 2026-08-13 02:05:17 +08:00
YueGuobin 474dc1db64 Prototype: vendor NOS Docker node support (SR Linux etc.) — docker_exec console via Docker exec API (pty + hijacked HTTP + NAWS resize) 2026-08-13 02:05:17 +08:00
Guobin Yue abaddec6df Merge branch '3.1' into fix/docker-node-create-setattr-noise 2026-08-12 12:44:39 +08:00
grossmj 1988768d32 fix: update CI configuration to use a single OS and remove Windows-specific dependencies 2026-08-11 19:23:18 +02:00
Jeremy Grossmann 0763ec3bd4 Merge pull request #2804 from GNS3/update-dependencies
Update dependencies
2026-08-11 19:03:41 +02:00
grossmj 78f175d827 Merge remote-tracking branch 'origin/update-dependencies' into update-dependencies 2026-08-11 18:52:20 +02:00
grossmj 386c88f230 Downgrade websockets version 2026-08-11 18:52:09 +02:00
Jeremy Grossmann 913ff3823a Merge branch '3.1' into update-dependencies 2026-08-11 18:48:54 +02:00
grossmj 9e1c533847 Finish to upgrade other dependencies 2026-08-11 18:47:35 +02:00
grossmj 0219914103 fix: breaking change with dependency FastAPI v0.137.0
https://fastapi.tiangolo.com/release-notes/#specific-breaking-changes
2026-08-11 18:47:02 +02:00
YueGuobin 72917596ab fix: suppress redundant console port setter log during Docker node create
create_docker_node() passes console, aux etc. to create_node() via
.get() so those keys remain in node_data.  The setattr fallback loop
then re-applies them — if reserve_tcp_port returned a different port
in __init__, the setter fires an INFO log and performs a wasted
release→reserve round-trip.

Pop the 15 keys already consumed by create_node() before the loop so
it only handles truly extra keys.
2026-08-12 00:10:01 +08:00
Jeremy Grossmann f69098f441 Merge pull request #2850 from yueguobin/fix/marker-batch-reconcile
fix: marker reconcile scope, batch NIO parallelism, aiohttp WS upgrade
2026-08-11 18:04:45 +02:00
YueGuobin fdcc7058cb revert: restore force_close=True in compute._session()
The per-commit force_close=not _local optimisation reused TCP connections
for the loopback compute, but _session() is also used for the controller's
WebSocket heartbeat connection (_connect_notification -> ws_connect).
The different connector behaviour prevented the compute from receiving
pings, so no compute.updated events reached the WebUI and the compute
cache stayed empty.
2026-08-11 23:27:30 +08:00
YueGuobin 1dd334caff perf: parallelize batch NIO creation across nodes
create_batch_nios bound NIOs in a serial for-loop, so during project open
every builtin L2 node (ethernet_switch/hub/cloud/nat) started its uBridge
one at a time (~0.5s each for fork + AF_UNIX connect). Group entries by
node and bind in parallel with asyncio.gather — mirroring update_batch_nios
— so independent uBridge processes start concurrently. Within a node,
entries stay serial to respect the per-node uBridge command lock.
2026-08-11 22:11:44 +08:00
YueGuobin 958aa59d7e fix: scope marker reconcile to the current bridge/NIO
The reconcile pass in _ubridge_apply_markers walked the node-wide
_marker_filter_bridges map but compared against `desired`, which only
carries the markers of the NIO being updated. Updating any one link
therefore deleted every other link's markers (and their pcaps) on that
node — a regression from the add-only→reconcile switch. IOU's override
had the same flaw across its ports.

Guard the delete pass with the current bridge (base_node) / IOL location
(IOU) so only markers on the NIO being reconciled can be removed. Added
a regression test that fails without the guard.
2026-08-11 22:11:44 +08:00
YueGuobin 5de902c8b1 cleanup: remove stray debug print in Dynamips node creation
The print(node_data.chassis, platform in DEFAULT_CHASSIS) at
dynamips_nodes.py:67 (added in 8ad7b3f6, 2022) emitted "None False"
to stdout on every Dynamips router creation. Pure debug leftover;
the very next line tests the same condition.
2026-08-11 21:46:38 +08:00
YueGuobin 958c45b6fc fix: reconcile marker filters in _ubridge_apply_markers (delete/update)
The batch marker-def fan-out (PR #2848) routed create/update/delete
marker_definition through memory_only + a batch PUT /nios/batch that
re-applies markers via _ubridge_apply_markers. But _ubridge_apply_markers
was strictly add-only: it skipped any (name, link_id) already in
_marker_filter_bridges, and reset_packet_filters preserves mark filters
(contract). So:

  * delete_marker_definition left the deleted marker's filter alive in
    uBridge (still matching / signalling / writing pcap) until node restart.
  * update_marker_definition (bpf/tag/direction change) never reached
    uBridge — the live filter kept the old expression until node restart.

Make _ubridge_apply_markers a real reconcile against the desired
nio.markers:
  - installed but no longer desired  → delete_packet_filter + unlink pcap
                                       + unregister
  - desired with changed filter field → rebuild (delete + re-add)
  - desired with only enabled changed  → instant toggle (pcap preserved)
  - desired and unchanged              → skip
  - desired and new                    → add

Track installed specs in a parallel _marker_specs dict so changes can be
detected. Both base_node and the IOU iol_bridge override are updated.
Added tests for the delete-removed and rebuild-changed-bpf paths.
2026-08-11 21:00:03 +08:00
Jeremy Grossmann f045cde0da Merge pull request #2848 from yueguobin/perf/link-create-parallel
perf: link creation & marker fan-out batch optimization
2026-08-11 09:35:08 +02:00
YueGuobin 04592a1107 log: suppress marker sink stats when no matches received 2026-08-11 11:43:43 +08:00
YueGuobin 11461982b6 log: add periodic marker.match throughput statistics
MarkerManager now logs every 10s how many marker datagrams the UDP sink
processed and the current throughput rate (match/s), so operators can
tell at a glance whether the single sink keeps up with the aggregated
uBridge traffic. Error count is also logged.
2026-08-11 11:40:22 +08:00
YueGuobin 2ac0bb7d25 marker: route marker.match to a dedicated project WS channel
High-frequency marker.matches shared the single project notification queue with topology events (node.*/link.*), causing head-of-line blocking. Add a separate marker channel: Notification.project_marker_queue/marker_emit, dispatch routes marker.* off the main project queue, plus a new WS /{project_id}/notifications/markers/ws endpoint. Fully migrated (the main project WS no longer carries marker.match); marker listeners are independent of project auto_close. Compute side unchanged.
2026-08-11 11:19:56 +08:00
YueGuobin a699383933 perf: raise marker UDP receive buffer from ~208KB to 8MB
1000+ uBridge processes share a single UDP marker.sink endpoint. The
default kernel receive buffer (~208 KB) holds ~1000 datagrams — a
traffic burst can overflow it before the event loop drains them. Grow
it to 8 MB via setsockopt(SO_RCVBUF) so the kernel absorbs bursts
without silent packet loss.  UDP is unordered — buffer size does not
affect per-datagram latency, only burst-loss resilience.
2026-08-11 10:44:16 +08:00
YueGuobin 6c0ec30d5d perf: move project-open marker inheritance to prepare phase
Previously apply_defs_to_new_link ran during finalize (after
link._created=True), issuing one PUT /nio round-trip per link end for
each inherited marker — 5000+ HTTP round-trips even for a single def.
Move it into _prepare_link_from_topology (memory_only) so the inherited
markers are already in _link_data when _prepare() constructs the NIO
specs, and create_batch_nios carries them in the single batch dispatch.

Finalize no longer calls apply_defs_to_new_link.  Interactive link
creation (dragging a cable in the UI) still goes through the per-link
create() → apply_defs_to_new_link path — a single link is fast.
2026-08-11 09:13:14 +08:00
YueGuobin e4d4282026 perf: parallelise compute-side batch NIO update per node
The update_batch_nios handler looped serially across all entries (5010
for a 2505-link topology). On started nodes each entry does uBridge I/O,
so the serial loop added orders of magnitude to the fan-out wall time.

Group entries by node_id before dispatching.  Different nodes talk to
their own uBridge process (AF_UNIX socket) and are fully independent, so
their updates run in parallel via asyncio.gather.  Per-node entries are
still serial (respecting the per-node uBridge command lock).
2026-08-11 09:02:46 +08:00
YueGuobin 57558508bf perf: batch update/delete marker-def fan-out too (full project-level)
per-def operations are project-level — create, update AND delete all
modify the marker policy on every link — so all three must batch, not
just create.

Extend memory_only to update_marker and stop_marker (merge/delete into
_markers + refresh _link_data, no per-link HTTP), and route
update_marker_definition and delete_marker_definition through the same
two-phase path as create: memory-only per link, then one
PUT /projects/{id}/nios/batch per compute.

Trade-off: a full reapply resets every marker's pcap on the link (the
old precise update_marker/stop_marker preserved sibling pcaps). For
project-level policy changes this is acceptable — real-time insight
matters more than pcap continuity, and batching turns 5000+ round-trips
into one per compute.
2026-08-11 08:54:16 +08:00
YueGuobin 48991f2d29 perf: batch marker-def fan-out to one PUT /nios/batch per compute
When a marker definition is created (or re-applied on data_link_type
change), the fan-out used to call inherit_marker -> update() on every
link, issuing one PUT /nio per link end (5000+ round-trips on a 2500-link
project). On started nodes each round-trip also reconfigured uBridge.

Two-phase fan-out:
- inherit_marker/start_marker gain memory_only: writes the marker into
  link._markers and refreshes _link_data without any HTTP/emit/dump.
- _apply_def_to_all_links applies memory-only to every link, then
  _batch_update_link_nios groups the updated NIO specs by compute and
  sends a single PUT /projects/{id}/nios/batch per compute.

compute: new PUT /projects/{id}/nios/batch endpoint with _get_existing_nio
+ _update_nio_binding dispatch (mirrors create_batch_nios), re-applies
filters+markers to uBridge on started nodes.

Precise per-marker operations (update_marker bpf change, stop_marker on
def delete) are untouched — they deliberately avoid a full reapply to
preserve sibling marker pcaps.
2026-08-11 08:44:57 +08:00
YueGuobin c877ed87a6 log: add marker fan-out timing for project-open and interactive def-change
Distinguish the two marker-policy fan-out paths in the log so we can tell
which is slow:
- apply_defs_to_new_link (project-open finalize): logs def count + link
  count + elapsed, only when marker_definitions is non-empty.
- _marker_apply_concurrently (interactive create/update/delete def): logs
  link count + elapsed per fan-out.
2026-08-11 08:29:39 +08:00
YueGuobin d84e756228 tune: raise start_all concurrency from 3 to 10
3 is too conservative for modern hardware; 10 provides a moderate boost
without the risk of overwhelming the host (start is the heaviest
operation: ubridge process + docker start + network config for each
node).
2026-08-11 01:20:14 +08:00
YueGuobin 2f70bd6daa revert: drop _connect_nio thread-pool executor, restore async _ubridge_send
The _connect_nio thread-pool optimisation (send_batch_sync) targeted
node-start performance, but start_all already runs at concurrency=3
(by design, to avoid overwhelming the host).  It also introduced a
Python 3.13 incompatibility (trsock.setblocking forbidden) that
prevented docker nodes from starting.  Since node-start is not the
target of this branch (project-open link creation is), revert to the
simple per-command async _ubridge_send.

The project-open batch NIO dispatch (create_batch_nios) is unaffected —
it never called _connect_nio (nodes aren't started during open).
2026-08-11 01:16:13 +08:00
YueGuobin 46bad0b639 fix: use socket.fromfd in send_batch_sync for Python 3.13 compat
Python 3.13's asyncio TransportSocket wrapper rejects setblocking(). Dup
the underlying fd via socket.fromfd() into a plain socket that the
executor thread can drive in blocking mode, then detach() after the
batch to avoid closing the transport's fd.
2026-08-11 01:10:18 +08:00
YueGuobin f9f1a4d4d8 log: demote all per-node lifecycle INFO logs to DEBUG
Extend the docker_vm / base_node demotion to the remaining node types
and supporting layers:

  qemu_vm: MAC, adapters, disk image, RAM, priority, NIO added, created
  iou_vm:  application ID, adapters, serial, image, RAM, NIO added
  dynamips router: created, adapter, RAM, NVRAM, IOS, idle-PC, disk,
    MAC, NIO bound; hypervisor create/start/connect; nio_udp created
  builtin: ethernet_switch/hub, cloud, nat — created, NIO bound
  ubridge: hypervisor start/connect

At multi-node scale these per-node lines flood the log. Only the
project-open progress summary (loaded N nodes / creating N links)
now remains at INFO alongside genuinely exceptional events.
2026-08-11 01:05:53 +08:00
YueGuobin 4e30b6905a fix: await Dynamips.create_nio (it is async, unlike sync base)
Dynamips.create_nio is async def while BaseManager.create_nio is a sync
def.  The previous fix only added the extra 'node' argument but did not
await the resulting coroutine, causing 'was never awaited' warnings and
passing a coroutine object instead of an NIO instance to the binding
dispatch.  Add 'await' on the Dynamips branch.  Test updated to verify
both the async nature and the parameter count.
2026-08-11 01:00:01 +08:00
YueGuobin 09a9555d02 fix: use bound-method param count for Dynamips create_nio detection
The inspect check tested unbound function signatures (3 params unbound vs
2 unbound) but node.manager.create_nio is a bound method — inspect
excludes 'self'.  Dynamips bound = 2 (node + nio_settings), standard
bound = 1 (nio_settings).  The old '== 3' never matched, so the extra
'node' arg was never passed.  Switch to '>= 2' and rewrite the test to
exercise the actual bound-method scenario.
2026-08-11 00:58:08 +08:00
YueGuobin e56488c2f7 test: add _add_nio_binding dispatch tests for Qemu, IOU, VPCS, Builtin
Cover every dispatch branch in the batch NIO endpoint so that future
additions of node types with unusual NIO-binding signatures are caught
at test time.
2026-08-11 00:55:54 +08:00
YueGuobin ed0f1bb4de fix: Dynamips create_nio(node, nio_settings) takes extra arg + add tests
Dynamips.create_nio requires the node as first positional argument
(unlike every other manager which takes only nio_settings). The batch
handler now detects this via parameter-count inspection (3 vs 2) and
passes node when needed.

Also add Dynamips to _add_nio_binding dispatch: routers use
slot_add_nio_binding(slot, port, nio), switches/hubs fall back to
add_nio(nio, port_number).

Add tests covering Dynamips router dispatch, switch dispatch, and the
create_nio signature detection to prevent regression.
2026-08-11 00:54:53 +08:00
YueGuobin c366f20340 log: add start/stop/close progress lines; revert start concurrency to 3
- start_all, stop_all: emit 'starting/stopping N nodes...' and complete
  lines with project name+id, matching the open-node/link log style.
- close: emit 'project closing.../closed' bracketing the whole teardown.
- Revert start_all concurrency 20->3 and reset_console 20->3: start
  (ubridge process + docker start + network) is heavy enough that high
  concurrency risks overwhelming the server. stop remains 100 (kill is
  light) and suspend remains 50 (pause is light).
2026-08-11 00:48:13 +08:00
YueGuobin ec2a0ffff3 log: lower per-node lifecycle logs in base_node to DEBUG
The per-node created / is-closing / Starting-uBridge / Stopping-uBridge
INFO lines flood the log at 1000+ node scale during project open, close,
start-all and stop-all. Demote them to DEBUG (consistent with the
docker_vm lifecycle-log demotion).
2026-08-11 00:39:49 +08:00
YueGuobin 3aa39a2da3 perf: raise start/stop/suspend/reset-console concurrency from 3
The historic concurrency=3 was far too conservative for 1000+ node
topologies. Raise them based on operation weight:

  stop_all:      3 -> 100  (docker kill + permissions cleanup, light)
  suspend_all:   3 ->  50  (docker pause/unpause, light)
  start_all:     3 ->  20  (ubridge startup + docker start + network, heavy)
  reset_console: 3 ->  20  (terminal reset, light)

Node creation (open) already uses concurrency=100 and approaches the
container-create daemon limit (~20/s); stop/kill is substantially faster
than create, so 100 is safe.
2026-08-11 00:33:49 +08:00
YueGuobin 7dc66d4836 fix: race in concurrent node creation sending duplicate POST /projects
When opening a project, nodes are created with Pool(concurrency=100).
Each _create_node did an unlocked check-then-act: 'if compute not in
_project_created_on_compute' -> await POST /projects -> add(compute).
The await let dozens of concurrent node creations race past the check
before any registered, each firing a redundant POST /projects at the
same compute. The compute-side sync handler then ran in a thread pool
and instantiated the Project N times (the repeated 'Project ... created'
INFO logs, ~16x).

Guard the check+POST+register with a project-level asyncio.Lock. The
first creation holds it for one POST; the rest acquire, see the compute
already registered, and return immediately — negligible serialization.

Also annotate the node/link progress logs with project name+id and add
completion lines, so the log clearly shows which project is loading and
when each phase finishes.
2026-08-11 00:30:33 +08:00
YueGuobin df3ca6e26b log: lower per-node docker lifecycle logs to DEBUG
At 1000+ nodes the per-node INFO lines flood the log during open /
start-all / stop-all: MAC changed, adapters changed, created, started,
console listen, fix ownership, stopped, paused, removed, adapter created,
NIO removed, capture start/stop, CPU/memory limits, mount resources.

Demote all of these routine per-node/per-adapter lines to DEBUG. Keep
INFO only for genuinely rare/important events: image pull (missing image)
and stale-container cleanup. Warnings unchanged.
2026-08-11 00:22:26 +08:00
YueGuobin 4c6ef7752a log: emit progress lines for node and link loading on project open
A large topology takes ~50s to open (dominated by docker daemon node
creation). Without any start marker the user sees no feedback that work
is in progress. Add two INFO lines: 'Loading N nodes...' before the node
pool and 'Creating N links...' before the bulk link prepare/dispatch.
2026-08-11 00:16:32 +08:00
YueGuobin e2fd922922 cleanup: remove project-open stage timing logs, lower NIO-added log to debug
- Drop the diagnostic stage-timing logs added during link-create perf
  work (nodes / preallocate / prepare / dispatch) now that bottlenecks
  are resolved and verified.
- Lower the per-NIO 'added to adapter' log in docker_vm from INFO to
  DEBUG — at 5000+ NIOs per project open it floods the log at INFO.
2026-08-11 00:14:58 +08:00
YueGuobin c9a93c0aeb perf: skip per-link topology dump during project-open prepare
Stage timing showed prepare taking 96s for 1275 links (vs 0.31s for the
batch dispatch itself). Root cause: _prepare_link_from_topology called
add_link (dump=True default) and update_link_style/update_show_filters_icon
(each unconditionally dump the full topology). 1275 links x serialize-
and-write-the-whole-topology = the entire 96s.

- add_link(..., dump=False): the project is dumped once at the end of open
- set link._link_style / _show_filters_icon directly instead of the
  update_* helpers, which also avoids spurious 'link.updated' notifications
  before the link is finalised

The final self.dump() at the end of project.open already persists everything.
2026-08-10 23:54:26 +08:00
YueGuobin 38c49a655c perf: batch NIO dispatch on project open (one HTTP per compute)
Project open used to create each link by issuing two NIO POSTs from the
controller to the compute — ~5000 HTTP round-trips for a 2500-link
topology, all funnelling through the single shared controller/compute
event loop and capping throughput near 12 links/s.

Replace it with a bulk path:
- UDPLink split into _prepare() (local: ports, peer addrs, link_data)
  and _commit_nios() (dispatch). create() = prepare + commit (interactive).
- Link.add_node gains batch=True: attach both nodes without triggering
  per-link NIO HTTP.
- compute: new POST /projects/{id}/nios/batch endpoint with a unified
  _add_nio_binding dispatch across node types (docker/qemu/iou/vpcs/
  builtin differ in signature).
- project.open: prepare all links locally, group NIO entries by compute,
  send each compute a single /nios/batch, then finalise (wire node/port
  refs, mark created, notify, apply marker defs) in parallel.

Cuts controller->compute HTTP from O(links) to O(computes). Test added
for the batch endpoint.
2026-08-10 23:42:50 +08:00
YueGuobin 96d4d82716 perf: cache compute.host_ip + add UDPLink.create timing log
host_ip resolved socket.gethostbyname on every access with no cache.
get_ip_on_same_subnet touches host_ip 2-4 times per link, so opening a
2500-link project issued thousands of blocking DNS calls inline on the
event loop — freezing all concurrent link coroutines each time. This is
the most likely cause of the 12-link/s throughput (1000x below what
Pool(concurrency=100) should deliver) and the burst+pause pattern.

- compute.host_ip: cache the resolution in _host_ip_cache, invalidate
  on host setter change
- UDPLink.create: timing log splitting get_ip / ports / nio so the next
  project-open confirms where time actually goes
2026-08-10 23:20:40 +08:00
YueGuobin b155980402 perf: dedicated 500-worker thread pool for ubridge batch I/O
Replace the default asyncio executor (capped at ~32 threads) with a
dedicated ThreadPoolExecutor sized for large-topology parallelism.
When 500 nodes each call _connect_nio, up to 500 OS threads can now
send blocking ubridge commands in parallel — no longer serialised by
either the event loop or a small thread pool.

- ubridge_hypervisor: module-level _ubridge_sync_pool (max_workers=500)
- docker_vm._connect_nio: dispatches to the dedicated pool instead of
  the default executor
2026-08-10 23:11:23 +08:00
YueGuobin 05934fa8e8 perf: offload per-NIO ubridge commands to thread-pool executor
Replace the 3-5 sequential await _ubridge_send calls in _connect_nio
with a single run_in_executor batch.  The batch holds the node-level
asyncio Lock to prevent interleaving with async sends, then uses the
hypervisor's new send_batch_sync method which does blocking socket
sendall/recv inside the thread pool.  Different nodes' batches now
run in true OS-thread parallelism rather than serialising through
the asyncio event loop between every command.

- ubridge_hypervisor.send_batch_sync: blocking batch send using
  the underlying socket from the asyncio transport, protected by
  threading.Lock.
- _connect_nio: builds command list (add_nio_udp, start_capture,
  bridge start, reset_packet_filters, add_packet_filter) and
  dispatches to the default executor.
2026-08-10 23:06:54 +08:00
YueGuobin 506b0b9f4a perf: enable TCP keep-alive for local compute HTTP requests
When controller and compute share the same process, every HTTP request
to localhost was paying a full TCP handshake (force_close=True forced
connection teardown after each request).  With 2500+ links each sending
two NIO POSTs, that's 5000 SYN->SYN-ACK->ACK cycles even for sub-ms
in-memory handlers.  Switch to keep-alive for loopback compute
(127.0.0.1 / ::1 / localhost) while keeping force_close for remote
computes that may sit behind NAT/firewalls that drop idle connections.
2026-08-10 23:00:59 +08:00
YueGuobin 82fc7f2bd1 debug: add per-command timing to _connect_nio ubridge calls
Temporary diagnostic instrumentation to measure the wall-clock time of
each ubridge command during NIO addition (add_nio_udp, bridge start,
filters, markers).  The logs will reveal whether the 12-NIO/s
throughput stems from ubridge command latency itself or from lock
contention / HTTP overhead outside _connect_nio.
2026-08-10 22:45:32 +08:00
YueGuobin acc9e670af perf: parallelize UDP port allocation and NIO creation in UDPLink.create
Previously the two sides' UDP port reservations and NIO tunnel
POSTs were issued sequentially, even though they are independent
once the peer addresses are known — each node talks to its own
compute/uBridge with no shared lock, so the HTTP round-trips can
overlap.

- Port allocation: gather _allocate_port for both computes
- NIO creation: gather both node.post calls together
- Error handling: if either side fails, roll back whichever side
  succeeded (previously only node2-fails→cleanup-node1) before
  re-raising the first error

Batch loading (open/import project) benefits most because links
are created at high concurrency and the per-link wall-clock
time is dominated by the sum of its two sequential NIO POSTs.
2026-08-10 22:36:41 +08:00
Jeremy Grossmann 372874c7d5 Merge pull request #2847 from yueguobin/fix/docker-pid1-signal-chain
fix: SIGKILL docker container on stop instead of 5s grace period
2026-08-10 16:03:22 +02:00
YueGuobin e7dfe9c7c4 fix: SIGKILL docker container on stop instead of 5s grace period
Docker node stop took ~5s every time. The stop API grace period
(params t=5, unchanged since 2015) was always exhausted: the business
process (often an interactive shell) ignores SIGTERM, and GNS3 doesn't
depend on graceful shutdown — _fix_permissions and /gns3volumes already
persist container state before stop() is called.

Use POST /containers/{id}/kill (SIGKILL, zero delay) instead of stop.
The 409 (container already stopped) replaces the previous 304 handling
for the race where the container exits between the state check and the call.

t=5 traced to commit 33edbefa3 (2015-10-14) "Docker cleanup and
improvements" — introduced with no recorded rationale.
2026-08-10 21:48:26 +08:00
Jeremy Grossmann 3689e2bb87 Merge branch '3.1' into update-dependencies 2026-08-08 22:18:17 +02:00
Jeremy Grossmann cf072c7922 Merge pull request #2752 from Volo6uev/base-configs-3.0
Api endpoints to manage base configuration files for templates
2026-08-08 22:16:36 +02:00
Jeremy Grossmann 909ccf8fcd Merge branch '3.1' into base-configs-3.0 2026-08-08 22:11:39 +02:00
Jeremy Grossmann 1d6218a3f1 Merge pull request #2846 from GNS3/bugfix/2773
Handle manual project deletion
2026-08-08 22:10:46 +02:00
grossmj bc8e43dfca fix: add event handling for deleted projects 2026-08-08 17:27:52 +02:00
grossmj 544d15c87e fix: tests after merging 2026-08-08 16:50:19 +02:00
grossmj dff79f65d3 Merge branch '2.2' into 3.1
# Conflicts:
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/controller/import_project.py
#	gns3server/controller/project.py
#	gns3server/crash_report.py
#	gns3server/version.py
#	tests/compute/docker/test_docker_vm.py
#	tests/controller/test_import_project.py
#	tests/controller/test_project.py
2026-08-08 16:27:14 +02:00
Jeremy Grossmann 955d545cf2 Merge pull request #2844 from yueguobin/marker-enabled-pause-resume
marker: direction, real-time control, serial-link (WAN) support; uBridge AF_UNIX control channel
2026-08-07 22:00:13 +02:00
Guobin Yue 183739e630 Merge branch '3.1' into marker-enabled-pause-resume 2026-08-08 01:18:39 +08:00
YueGuobin 82945cd81e server: raise the open-files (RLIMIT_NOFILE) limit at startup
Every started node holds ~3 file descriptors in the server's table (pidfd + stdout/stderr pipes per child process), so a few hundred started nodes exhaust the default 1024 soft limit and the uBridge version-check subprocess fails with EMFILE ('Too many open files: /dev/null').

At startup, best-effort raise RLIMIT_NOFILE to 65535 (capped by the hard limit); failures are logged, never fatal. Runs before daemonize() so the daemon inherits the raised limit. Linux-only, matching the platform target.
2026-08-08 00:26:00 +08:00
YueGuobin ad8bac8328 marker: batch topology dumps in bulk fan-out (per-def on 500+ links was ~1 minute)
Every per-link marker operation (start_marker / stop_marker / update_marker) called Project.dump() -- a full topology serialization + file write. A definition fan-out over 500 links therefore wrote the whole topology 500+ times (each blocking the event loop), which dominated the observed ~1 minute; the NIO round-trips themselves were negligible.

Add a dump: bool = True parameter to the three per-link operations and inherit_marker (matching the existing dump param on Link.add_node). The bulk paths -- definition create fan-out, definition-update sync and re-fan-out, definition-delete cleanup, pause/resume, new-link inheritance -- pass dump=False and their caller dumps once after. apply_defs_to_new_link suppresses per-def dumps too: link create / project open dump once after, so opening a 500-link project with N definitions no longer does 500xN topology writes.
2026-08-08 00:03:30 +08:00
YueGuobin 078cf92aef marker: concurrent (bounded) definition fan-out
The per-definition fan-out applied markers to links in a serial loop -- one compute round-trip per link. On a 1000-link project that serializes N HTTP round-trips (minutes on remote computes). Fan out with asyncio.gather + Semaphore(32): links are independent (own _markers/_link_data), per-link ControllerError stays isolated, and Project.dump is synchronous + atomic (tmp + rename) so concurrent dumps cannot corrupt the topology file.

Converts the definition-create fan-out, the definition-update sync and re-fan-out loops, and the definition-delete cleanup to the shared _marker_apply_concurrently helper. apply_defs_to_new_link stays serial deliberately: all definitions share one link and each push carries the link's full marker set, so concurrent pushes would race and lose markers.
2026-08-07 23:50:41 +08:00
YueGuobin 0f86dabfa3 marker: forward data_link_type on per-link marker create
The per-link create_marker route (links.py) called start_marker without the data_link_type from the body, so it always defaulted to DLT_EN10MB and a serial encapsulation chosen by the caller was silently dropped. Same oversight the IOU capture and definition routes had; one-arg fix.
2026-08-07 13:38:33 +08:00
YueGuobin ba318150fa iou: skip already-installed markers in _ubridge_apply_markers (NIO update idempotency)
The IOU override of _ubridge_apply_markers lacked the incremental guard the generic path has, so on an NIO update it re-added every marker the port already carried. uBridge's add_packet_filter rejects a duplicate filter name (packet_filter.c find_packet_filter), so adding a private marker to an IOU link that already hosted an inherited global-* copy failed with 'Failed to add filter global-...' -- the NIO update re-sends ALL markers on the port (inherited + new private), and the pre-existing one collided.

Add the same (name, link_id) in self._marker_filter_bridges skip as the generic base_node path, so an update only installs markers not already on the port. Mirrors how Dynamips/vpcs/etc. stay idempotent across add + update.
2026-08-07 13:30:12 +08:00
YueGuobin 3322233658 marker: serial-link (WAN) support via data_link_type -> uBridge linktype
Markers now work on serial links (Cisco HDLC / PPP / Frame Relay / ATM), not just Ethernet. A marker carries a data_link_type (default DLT_EN10MB); at the uBridge boundary it becomes the 'mark ... linktype <dlt>' keyword so the BPF compiles and the pcap is written with the matching link-layer.

- MarkerCreate / MarkerDefinitionCreate gain data_link_type (default DLT_EN10MB). Per-link it is create-only; definitions are updatable (a change re-fans-out).
- base_node._marker_linktype() normalizes the GNS3 DLT name (strip DLT_, uppercase, None for EN10MB). Single source is SerialPort.data_link_types, so Cisco PPP -> PPP_SERIAL (50), matching the capture path -- no second mapping table.
- _ubridge_add_marker_filter (generic) and the IOU marker loop append 'linktype <dlt>'.
- Definition fan-out branches on link_type in inherit_marker: Ethernet is always EN10MB; a serial link uses the definition's WAN encapsulation, or is SKIPPED when none was chosen (an EN10MB pcap on serial is undecodable). One definition covers a mixed topology.
- MCP marker_definition exposes data_link_type (None = not forwarded).
- No uBridge rebuild on a data_link_type change -- only that one marker's filter is swapped (delete + re-add), mirroring a BPF change; reset_packet_filters preserves sibling mark filters.

Requires the uBridge build with 'mark ... linktype' support.
2026-08-07 01:53:33 +08:00
Jeremy Grossmann 555c3ecab5 Merge pull request #2845 from yueguobin/fix/iou-serial-capture-linktype
Fix IOU serial-link captures always written as Ethernet (DLT_EN10MB)
2026-08-06 19:07:22 +02:00
YueGuobin 117f580cfd Merge fix/iou-serial-capture-linktype: IOU serial capture data_link_type 2026-08-07 00:36:25 +08:00
YueGuobin 17020b7f7b iou: forward data_link_type on capture start so serial pcaps use the correct linktype
The IOU compute capture/start route received data_link_type in the NodeCapture body but dropped it when calling node.start_capture(), so iou_vm.start_capture always defaulted to DLT_EN10MB -- every IOU serial capture (Cisco HDLC / PPP / Frame Relay) was written as an Ethernet pcap. Dynamips, cloud and the L2 switches already forward this value; IOU was the only serial-capable node that omitted it. One-argument fix: the node method already accepts and forwards data_link_type to 'iol_bridge start_capture', so only the route was missing it.

Verified: IOU serial captures now produce the expected linktype (C_HDLC / PPP_SERIAL / FRELAY) instead of Ethernet.
2026-08-07 00:35:39 +08:00
YueGuobin 5f4ac38434 ubridge: default the control transport to unix
Switch ubridge_control_transport from tcp (-H) to unix (-U), the
AF_UNIX + SO_PEERCRED channel recommended on Linux for kernel-level
peer authentication. tcp is retained as an opt-in for backward
compatibility. Existing deployments that set the key explicitly are
unaffected; only fresh installs / unset keys pick up the new default.
2026-08-06 21:35:09 +08:00
YueGuobin b8ac76f047 ubridge: require version >= 1.2.0
Drop the platform-specific floor (0.9.12 darwin / 0.9.14 others) in
favour of a single 1.2.0 minimum. This server now relies on features
only present in recent uBridge builds: the AF_UNIX control channel
(-U / SO_PEERCRED), the marker (mark) filter, and the brctl-backed
builtin Ethernet Switch. Removes the now-unused sys import left behind
by the dropped darwin branch (target is Linux-only).
2026-08-06 21:35:09 +08:00
grossmj 29c1b04078 fix: add missing HTTPException import 2026-08-05 21:55:23 +02:00
YueGuobin 7ed4eeab29 mcp: drop direction from marker_definition tool
A marker definition fans out to every link and auto-selects its capture node
on each, so tx/rx is relative to a node that varies per link — the controller
already rejects it (409). Exposing direction on the MCP definition tool let an
agent ask for something that could only fail. Remove the parameter and the
handler's direction handling; the docstring now points to encoding direction in
the BPF (e.g. 'icmp and icmp[icmptype]==8'). Per-link link_marker keeps
direction, where the capture node is fixed.
2026-08-05 01:40:22 +08:00
YueGuobin f7b19dae99 marker: tighten marker name max length from 128 to 32
128 characters was far beyond any realistic marker label (icmp, arp, tcp-syn)
and would have collided with the pcap filename budget once a tag prefix is
added later. Cap the user-facing name at 32 in both MarkerCreate and
MarkerDefinitionCreate; the compute-side name guard now also rejects names
longer than 48, which covers the `global-{def_name}` inherited form (≤ 39).
2026-08-05 00:49:37 +08:00
YueGuobin 75f278228b marker: drop deleted marker from port NIO cache to stop empty pcap on restart
Deleting a marker while its node was stopped, then starting the node, recreated
an empty pcap. Root cause: delete_marker_capture removed the uBridge filter and
the pcap file but not the marker spec cached on the port NIO (nio.markers) —
the data source _ubridge_apply_markers reads on node start. The stale spec
reinstalled the marker when uBridge came up.

This was a regression from switching stop_marker off update() (which re-sent
the NIO and implicitly refreshed nio.markers) to the fine-grained
node.delete(/markers/{name}) path.

Fix: make the delete port-aware so the compute can locate the NIO. The DELETE
marker route becomes /adapters/{a}/ports/{p}/markers/{name} across all six
node types; the handler resolves the NIO via get_nio and passes it to
delete_marker_capture, which now pops the marker from nio.markers. get_nio
works regardless of uBridge state, so the stopped-node case is covered. The
controller's stop_marker targets the capture side's adapter/port.
2026-08-05 00:08:12 +08:00
YueGuobin 1a0ce51f38 marker: validate def BPF once, skip re-validation on inherited fan-out
A marker definition validated its BPF N times — once per link in the fan-out
(start_marker runs validate_bpf_syntax on every copy), spawning one tcpdump -d
subprocess per link for the same expression. Definitions did not validate BPF
at all; only direction was checked.

Move the single validation point to the definition layer (create/update), and
validate each definition's BPF on project load (dropping any that have gone
invalid, like private markers). The inherited fan-out (start_marker) and def
sync (update_marker) now skip validate_bpf_syntax for inherited copies, since
the BPF comes from an already-validated definition. Private per-link markers
still validate inline as before. uBridge still runs pcap_compile at install, so
an invalid expression can never slip through.

Creating a definition over N links now runs one tcpdump instead of N.
2026-08-04 23:20:58 +08:00
YueGuobin 60e2bbbbbb marker: point directional defs at BPF, refresh implementation doc
The 409 on a tx/rx definition now recommends encoding direction in the BPF
(e.g. icmp[icmptype]==8) as the primary fix, with per-link markers as the
single-link fallback. Doc updated: per-def rejects tx/rx (why + BPF), the
pause section no longer claims bpf changes reset+reapply (they rebuild one
filter), and a new Capture files section covers pcap cleanup + reset-preserves-mark.
2026-08-04 22:59:12 +08:00
YueGuobin 95824b1861 marker: incremental apply, clear bridge map on uBridge stop
_ubridge_apply_markers now installs only markers not already on the bridge
(uBridge's reset_packet_filters preserves mark filters), so an NIO update no
longer re-adds — and reopens — sibling markers' pcaps. _stop_ubridge clears
_marker_filter_bridges so a node restart re-installs everything (the map would
otherwise keep stale entries pointing at a fresh, empty uBridge).
2026-08-04 22:19:43 +08:00
YueGuobin caec71aa71 marker: fine-grained filter ops, clean pcap on remove
Deleting or updating a marker no longer triggers a full NIO reapply
(reset_packet_filters + re-add), which closed/reopened every sibling
marker's pcap via uBridge. Instead operate on single filters:

- stop_marker: bridge delete_packet_filter + unlink the pcap (works with
  the node stopped; filter removal is skipped, the file is still deleted).
- update_marker: bpf/tag/direction → rebuild just that filter (delete + add);
  enabled → instant toggle; color/highlight_duration → stored only.
- compute delete_marker_capture / rebuild_marker_filter + per-node routes
  (DELETE /markers/{name}, PUT /markers/{name}/rebuild) + MarkerRebuild schema.

IOU overrides _ubridge_delete_marker_filter for iol_bridge; rebuild reuses
the already-overridden add/delete/set, so IOU needs no rebuild override.
2026-08-04 21:35:31 +08:00
YueGuobin 19815f7a37 marker: restore direction on project load, reject tx/rx on definitions
Restore a private marker's direction (and highlight_duration) when a
project is reopened — _create_link_from_topology_data previously dropped
them, silently reverting rx/tx markers to "both".

Reject tx/rx direction on marker definitions with HTTP 409: a definition
auto-selects its capture node per link and direction is relative to that
node, so a fixed tx/rx has no stable project-wide meaning. Per-link
markers still support tx/rx; only the project-wide definition is restricted
to "both" (the default).
2026-08-04 10:14:02 +08:00
YueGuobin 9323ea4cec schema: accept direction='both' in marker schemas, normalize to None 2026-08-03 00:31:45 +08:00
YueGuobin b6959f8214 qemu: demote QEMU monitor connect and set_link logs to DEBUG 2026-08-03 00:17:34 +08:00
YueGuobin e3ce234a09 docs: add log-interpretation note across node types for marker operations 2026-08-03 00:05:08 +08:00
YueGuobin ff907da5f6 marker: key _marker_filter_bridges by (name, link_id) so multi-link nodes toggle every copy
The _marker_filter_bridges dict was keyed by marker name alone, so when one
node hosted the same filter name on several links (IUOL-BRIDGE per node with
many bays/units, or a multi-interface router), successive apply calls
overwrote earlier entries. pause_marker_definition then toggled only the last
recorded bridge/location — other copies stayed active and kept emitting.

Key by (name, link_id) so each copy is independent, and iterate all matching
entries in _ubridge_set_marker_filter_state (both generic bridge and IOU
iol_bridge override). Toggle route existence checks also iterate matching
names. Tests updated.
2026-08-02 23:48:24 +08:00
YueGuobin 34b644a548 marker: make per-filter toggle fall back to NIO rebuild when the marker isn't installed
Toggle routes silently no-opped when _marker_filter_bridges lacked the filter
name, so update_marker's enabled-only short-circuit succeeded without toggling
uBridge — the controller-layer enabled was set but the uBridge filter stayed on
and kept emitting signals. Now the toggle routes raise HTTPException 404
(FastAPI handles it directly, no ERROR log); the controller's except catches it
and falls back to self.update() (NIO rebuild, which applies the marker + off).
2026-08-02 23:41:43 +08:00
YueGuobin 0afbb897a5 marker: make per-filter toggle a no-op when the marker isn't installed
_ubridge_set_marker_filter_state raised "Marker X is not installed on this
node" when the capture node wasn't running — the name->bridge map is only
populated during _ubridge_apply_markers, which runs when the node is up. The
error was noise: the controller's enabled-only short-circuit catches it and
falls back, and the controller-layer enabled is authoritative (honoured when
the node starts and applies the marker). Treat a missing entry as a no-op
instead of raising.
2026-08-02 23:23:53 +08:00
YueGuobin 1eeee024bd marker: rework pause/resume from project-wide to per-definition
The project-wide mute (POST /markers/pause|resume + _markers_paused) paused
every marker with one button. The actual need is per-rule control: pause one
definition and toggle only its inherited global-{name} copies across all links.

- Drop project-level: _markers_paused (init/asdict/load/start_all), the
  pause_all/resume_all_markers methods, and the /markers/pause|resume routes.
- Add per-definition: a persisted `paused` flag on each definition;
  pause/resume_marker_definition fan out update_marker(enabled) to every
  global-{name} copy — instant, via the existing enable_packet_filter toggle
  (no NIO rebuild, pcap/emitted preserved). New links inherit a paused
  definition already off (inherit_marker passes enabled=not paused).
- start_marker takes an enabled kwarg; update_marker's enabled-only short-circuit
  now also covers inherited copies so def pause/resume is instant.
- Routes: POST /marker-definitions/{name}/pause|resume.
- Docs + tests updated.
2026-08-02 22:46:33 +08:00
YueGuobin e97df86d96 marker: fix PUT marker with an enabled-only body (bpf no longer required)
update_marker reused MarkerCreate, whose bpf is required, so a partial PUT like
{"enabled": false} was rejected with 422 "bpf field required". Add a MarkerUpdate
schema with every field optional (bpf included; capture_node_id and name are
create-only/path-driven and omitted) and use it for the PUT route — partial
updates now validate cleanly.
2026-08-02 22:32:04 +08:00
YueGuobin f7d7ba165a marker: persist project-wide markers_paused to the .gns3 file
pause/resume was fire-and-forget: the controller sent marker pause/resume but
stored nothing, so the Web UI could only keep a local optimistic flag that was
lost on panel reopen. Treat the mute as a project-level config (like per-marker
enabled): record _markers_paused, persist it in the topology (asdict + load),
and echo it on the project object so the UI renders from server truth.

Because marker pause is a uBridge runtime flag that resets on node restart,
start_all re-applies the mute to freshly started uBridges after a project
reopen — a paused project stays paused across close/reopen.
2026-08-02 22:21:05 +08:00
YueGuobin 8ba950fa29 marker: document pause/resume and instant enabled toggle 2026-08-02 16:22:31 +08:00
YueGuobin 84179c239c marker: test enabled/pause/resume and instant toggle
- compute (test_base_node.py): set_marker_filter_state on/off command,
  marker pause/resume command, and apply issues enable_packet_filter off
  for a disabled marker (+ records the name->bridge map).
- controller (test_marker.py): _markers_for_node keeps disabled markers
  and carries enabled; update_marker enabled-only hits the toggle route
  (not NIO rebuild) while a bpf change still rebuilds; pause/resume fan
  out to capture nodes.
2026-08-02 16:19:20 +08:00
YueGuobin 3d06c4e22f marker: drive uBridge enabled/pause/resume in real time
Wire gns3-server to uBridge's real-time marker controls (contract
../ubridge/doc/gns3server-integration.md §3.2), in three layers:

A. enabled reaches uBridge — _markers_for_node no longer drops disabled
   markers controller-side; it carries `enabled` in the NIO spec. apply
   installs every marker then issues `enable_packet_filter … off` for the
   disabled ones (base_node bridge / iou iol_bridge; old-ubridge errors
   downgrade to a warning so a toggle can't break link create).

B. instant per-filter toggle — apply records name→bridge so a new
   `_ubridge_set_marker_filter_state` can flip a running filter with
   `enable_packet_filter on|off` (iou overrides for iol_bridge + bay/unit).
   Each marker-capable node type gains PUT /markers/{name}; update_marker
   short-circuits to it when only `enabled` changes (no NIO rebuild, no pcap
   flush), falling back to reset+reapply if the route is unavailable.

C. global pause/resume — `_ubridge_marker_pause/resume` send `marker pause`
   / `marker resume` direct to the hypervisor (pause stops signal+pcap,
   resume instant, sink retained). Six node-type routes add POST
   /markers/pause|resume; project.pause_all/resume_all_markers fan out to
   each capture node (deduped, best-effort); REST exposes
   POST /projects/{id}/markers/pause|resume.

Toggling enabled and pause/resume are now both instant — only marker create
or a bpf change still go through reset+reapply.
2026-08-02 16:19:20 +08:00
YueGuobin 7368e4a65b marker: add direction-clear, routing, and ubridge transport tests
Close the test gaps surfaced in review:

- controller (test_marker.py): direction clear vs preserve, definition-sync
  clear propagation, pinned-marker NIO routing, _markers_for_node carrying
  direction, and non-capable capture-node rejection.
- ubridge (new tests/compute/ubridge/test_hypervisor.py): unix/tcp __init__
  branches + socket-dir perms, _build_command -U/-H/debug, endpoint, stop
  socket unlink, and start() immediate-exit fail-fast.
- mcp (test_handlers.py): TestLinkMarker / TestMarkerDefinition covering
  direction tri-state (both/tx/omitted) for create and update.
2026-08-02 11:37:49 +08:00
YueGuobin 37cb9f0a9c marker: support clearing direction via explicit null / "both"
direction was settable but not clearable: once a marker or project-level
definition had direction=tx/rx, no update path could return it to "both
directions", and the definition-sync fan-out silently kept stale values on
every inherited copy.

Introduce a _UNSET sentinel (link.py) distinct from None so updaters can
tell "caller omitted direction" (preserve) from "caller passed None"
(clear). Thread it through UDPLink.update_marker and
Project.update_marker_definition; the two REST routes use Pydantic v2
model_fields_set to translate an explicit JSON null into the sentinel.

MCP follows with a "both" token: link_marker / marker_definition handlers
map direction="both" to a null in the REST body (tri-state: omit=preserve,
tx/rx=set, both=clear), and the tool descriptions/docstrings document it.

Backward compatible: omitting direction or passing tx/rx behaves exactly
as before; only an explicit null / "both" clears.
2026-08-02 11:37:49 +08:00
YueGuobin 71fa778d50 marker: let callers pin the capture node via capture_node_id
A marker is single-sided — only the chosen capture node's uBridge installs the
mark filter — and dir=tx|rx is interpreted from that node's perspective. Until
now the observer was always auto-picked (_choose_marker_side), so dir=tx meant
"the auto-chosen endpoint is sending", which is unpredictable and makes the
direction filter hard to render meaningfully in the Web UI.

Add an optional create-only capture_node_id to MarkerCreate: when set, the
marker is pinned to that endpoint's uBridge (validated as a link endpoint and a
marker-capable type); when omitted, behavior is unchanged (auto-pick). The
chosen id is already echoed back as capture_node_id and in MARK signals, so the
UI can always render the observer regardless of who picked it.

capture_node_id is create-only (changing it would silently flip the meaning of
stored direction; recreate instead) and is not accepted on project-level
definitions — they are link-agnostic and have no endpoints, so inherited
markers keep auto-picking per link.

Plumbed through REST create_marker, the MCP link_marker tool, and base
Link.start_marker. update_marker does not forward it.
2026-08-01 22:24:47 +08:00
YueGuobin 6749b872fa marker: forward dir= from ubridge and add per-marker direction filter
Direction field in marker.match events
=======================================

Read ubridge dir=<tx|rx> from MARK signal datagrams and forward it
as a "dir" key in the marker.match notification event.  The field is
additive -- older ubridge builds omit it and the parser leaves it null,
so consumers fall back to undirected rendering with no version coupling.
Semantics are relative to the capture node (the signal's node=<id>):
tx = capture node is sending (ingressed device-side NIO), rx = it is
receiving (ingressed link-side NIO).

Per-marker direction filter (opt-in, server-side pipeline)
==========================================================

Add a direction field to MarkerCreate and MarkerDefinitionCreate
schemas ("tx" | "rx" | null).  Plumb it through the full pipeline:

  Schema -> controller (start_marker/update_marker, marker_entry,
  _markers_for_node, update_marker_definition sync)
  -> REST/MCP handlers -> compute _ubridge_add_marker_filter +
  IOU _ubridge_apply_markers -> bridge add_packet_filter dir <tx|rx>

When set, ubridge only fires the mark handler (signal + pcap) for
packets matching the chosen direction.  null (default/legacy) = both
directions -- zero behavioural change for existing markers.

Docs and tests
==============

- docs/features/marker-traffic-insight.md: signal format updated,
  new Direction section with NIO mapping, arrow mapping, and additive
  compatibility note.
- tests/compute/marker/test_marker_manager.py: 3 new parser tests
  (dir tx/rx/absent) plus existing test extended to assert dir=None.

13 files, +123/-22, 72 tests pass (zero breakage)
2026-08-01 14:52:07 +08:00
YueGuobin d729f76856 Name AF_UNIX sockets by node id for self-describing debuggability
Use the node id (UUID) instead of an incrementing counter for the unix
control socket name, so each socket identifies its owning node at a glance
(one ubridge per node => node_id is unique). Falls back to the counter only
when no node id is supplied. A single UUID fits sun_path's 107-byte cap
(~69 bytes), so no project_id is needed.
2026-07-31 23:48:07 +08:00
YueGuobin 2488c42cd4 Detect unsupported -U flag and fail fast with a clear error
start() now checks whether ubridge exited immediately after launch. An
old ubridge build that does not understand -U exits with a non-zero code
right away; surface the reason from ubridge.log instead of waiting for
connect() to time out 10s later with a confusing "couldn't connect" error.
2026-07-31 22:40:38 +08:00
YueGuobin 3d3e20f69e Add configurable uBridge control channel transport (tcp/unix)
Add ubridge_control_transport to [Server] config (default "tcp", fully
backward compatible). Selecting "unix" switches the uBridge hypervisor
control channel from the unauthenticated TCP listener (-H) to an AF_UNIX
socket (-U) authenticated in-kernel via SO_PEERCRED.

- UBridgeHypervisor: supports both socket_path (AF_UNIX) and host/port
  (TCP); a new `endpoint` property unifies log/error strings for both
- Hypervisor: unix mode allocates a short socket path under a 0700
  private runtime dir and unlinks it on stop; tcp mode restores the
  original getaddrinfo ephemeral-port allocation
- base_node: reads the transport from config and passes host through
- schemas/config.py + config_samples/gns3_server.conf: new option

Default deployments are unchanged. "unix" requires a ubridge build that
understands -U.
2026-07-31 22:27:16 +08:00
Jeremy Grossmann d2f73cae28 Merge pull request #2842 from GNS3/release/v2.2.61
Release v2.2.61
2026-07-30 19:15:25 +02:00
grossmj 8c0150cd62 Development on 2.2.62.dev1 2026-07-30 19:11:11 +02:00
grossmj da2942c139 Release v2.2.61 2026-07-30 12:53:08 +02:00
grossmj 5485eef3ad Sync appliances 2026-07-29 18:52:23 +02:00
Jeremy Grossmann ccf4b3f9db Merge pull request #2840 from GNS3/fix-unvalidated-symlink
Fix unvalidated symlink creation in import_project
2026-07-29 18:51:21 +02:00
grossmj b797981a65 test: add tests for unvalidated symlink creation in import_project 2026-07-29 18:48:31 +02:00
grossmj a225622a6e fix(import): unvalidated symlink creation in import_project 2026-07-29 18:10:52 +02:00
Jeremy Grossmann 3f19ede8d2 Merge pull request #2839 from GNS3/bugfix/2838
Fix addition of QEMU RNG device in 2.2.60 causes interface names to change
2026-07-29 18:00:16 +02:00
Jeremy Grossmann 668169d1cc Merge branch '2.2' into bugfix/2838 2026-07-29 17:55:53 +02:00
grossmj 66b39f0616 Merge remote-tracking branch 'origin/master' into 2.2 2026-07-29 17:54:18 +02:00
Jeremy Grossmann 7efef28d4f Merge branch '2.2' into bugfix/2838 2026-07-29 17:50:01 +02:00
grossmj d9140d2f8c fix(qemu): move the bios options after the network ones 2026-07-29 17:47:03 +02:00
grossmj e2657febf0 fix(qemu): fix addition of QEMU RNG device causes interface names to change 2026-07-29 17:23:28 +02:00
Jeremy Grossmann d161f1997d Merge pull request #2837 from my-code-is-grey/ghcr_lowercase
fix: lowercase GHCR image name in docker-build workflow
2026-07-28 17:28:41 +02:00
my-code-is-grey 37490f2c74 bugfix: docker workflow needs lowercase name 2026-07-27 22:27:48 -04:00
Jeremy Grossmann 9b1d7d44bf Merge pull request #2836 from iskanred/bugfix/2835
fix(qemu): remove trailing space from RNG object argument
2026-07-27 20:42:29 +02:00
Iskander Nafikov 9562c9d828 fix(qemu): remove trailing space from RNG object argument 2026-07-28 00:27:55 +08:00
Jeremy Grossmann deba9f8409 Merge pull request #2834 from yueguobin/mcp-tool-descriptions-enhancement
Improve MCP tool descriptions and add traffic-insight marker tools
2026-07-26 19:09:05 +02:00
Jeremy Grossmann 6c2a980afc Merge pull request #2833 from Sanjays2402/fix/delete-project-does-not-start-nodes
Do not start nodes when deleting a project
2026-07-26 09:23:12 +02:00
YueGuobin 309d388b0b Add MCP tools for traffic-insight marker feature
Add 2 new MCP tools to expose the marker (traffic-insight) REST API:

- link_marker: per-link marker CRUD (create/update/delete)
  POST/PUT/DELETE /projects/{pid}/links/{lid}/markers
- marker_definition: project-level marker definition CRUD (create/update/delete/list)
  POST/PUT/DELETE/GET /projects/{pid}/marker-definitions
  Create auto-fans out global-{name} to every link

Read operations use existing link_get (returns markers dict).
2026-07-26 13:01:06 +08:00
YueGuobin a8c6546c44 Improve MCP tool descriptions for link_update, device_show_run, and link_reset
- link_update: add bidirectional filter effect note, packet loss formula
  (packet_loss [50] ≈ 75% observed), filter clearing syntax (filters: {}),
  and ARP-also-filtered warning with static ARP recommendation
- device_show_run: add prerequisite section — device_type:<type> tag
  required, Docker/Linux nodes unsupported (use node_console)
- link_reset: clarify that filter state machines (e.g. frequency_drop
  counters) restart while filter configuration is preserved
2026-07-26 12:50:14 +08:00
Sanjay Santhanam 00ac2c19bd Do not start nodes when deleting a project
Project.delete() calls open() to rebuild the internal data structures
needed for cleanup. open() schedules start_all() when the project has
auto_start enabled, so deleting an auto-start project actually launched
every node process, allocating ports and consuming resources, only for
close() to kill them moments later.

open() now takes an auto_start argument (default True, so normal opens
are unchanged) and delete() passes auto_start=False.

Fixes #2784
2026-07-25 20:00:43 -07:00
Jeremy Grossmann 6742d09a70 Merge pull request #2831 from Sanjays2402/fix/docker-stop-state-check
fix: correct always-true state check in DockerVM.stop()
2026-07-25 19:59:28 +02:00
Sanjay Santhanam 41e8777609 fix: correct always-true state check in DockerVM.stop()
The condition 'state != "stopped" or state != "exited"' is a tautology,
so the state check was a no-op and a stop request was sent even for a
container that had already exited.

_get_container_state() never returns "stopped" (only "running",
"paused" or "exited"), so the intended negation of the condition used in
_fix_permissions() requires 'and', not 'or' (De Morgan's law).

Added a regression test asserting no stop query is issued for an
already-exited container.
2026-07-25 10:30:16 -07:00
Jeremy Grossmann 20868aa233 Merge pull request #2830 from rafael-33/master
docs: add structured bug-report template
2026-07-24 22:55:36 +02:00
rafael-33 27c6535ad0 add bug report issue template 2026-07-24 22:51:44 +02:00
Jeremy Grossmann f1be5eae9b Merge pull request #2829 from cristian-ciobanu/docker-pull-image
Add controller and compute API support for explicitly pulling or updating Docker images
2026-07-24 20:14:55 +02:00
Cristi 2abc7e09a3 Adds controller and compute API support for explicitly pulling or updating Docker images. 2026-07-24 16:40:20 +03:00
Jeremy Grossmann 28f883b46a Merge pull request #2828 from GNS3/set-ulimit-pytest
feat(tests): increase maximum open file descriptors for test runs on Unix
2026-07-23 18:05:19 +02:00
grossmj c010be1a02 feat(tests): increase maximum open file descriptors for test runs on Unix 2026-07-23 18:00:11 +02:00
Jeremy Grossmann 2b949b1a77 Merge pull request #2826 from cristian-ciobanu/iol-l1-keepalive
(feat): Implement functional Layer 1 keepalive support for IOU/IOL nodes
2026-07-22 18:10:42 +02:00
Cristian Ciobanu ac003c0a71 Merge branch 'GNS3:master' into iol-l1-keepalive 2026-07-22 11:50:09 +03:00
Cristi dd497e4308 (feat): Implement functional Layer 1 keepalive support for IOU/IOL nodes 2026-07-22 10:47:09 +03:00
grossmj 7bc9ae4aa8 Fix typo with ovmf_firmware_dir 2026-07-21 17:34:05 +02:00
grossmj 1f6e00d29f Fix issues after merging 2026-07-21 17:20:46 +02:00
grossmj 0091f2c64a Merge remote-tracking branch 'origin/3.1' into 3.1 2026-07-21 17:04:35 +02:00
grossmj 7487ec14e0 Merge branch '2.2' into 3.1
# Conflicts:
#	CHANGELOG
#	conf/gns3_server.conf
#	gns3server/compute/docker/__init__.py
#	gns3server/compute/docker/docker_vm.py
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/controller/import_project.py
#	gns3server/crash_report.py
#	gns3server/version.py
2026-07-21 17:03:00 +02:00
Jeremy Grossmann e90f250294 Merge pull request #2823 from yueguobin/feature/cloud-node-interface-refresh
fix(controller): refresh cloud/nat node interfaces from compute on GET, plus NAT interface info
2026-07-19 12:49:07 +02:00
YueGuobin 977d9628e5 fix(builtin): hide GNS3 internal bridge interfaces from Cloud node
Filter out host interfaces whose name starts with 'gns3' (e.g.
EthernetSwitch kernel bridges) from Cloud.asdict() interfaces list.
Other special interfaces (virbr0, docker0, etc.) remain visible.
2026-07-19 18:21:22 +08:00
YueGuobin 4943567962 feat(builtin): NAT node returns its interface IP info in asdict()
NAT now includes an 'interfaces' field in its response, filtered to
contain only the mapped NAT interface (virbr0 on Linux, vmnet8 on
macOS/Windows). This lets connected nodes discover the NAT subnet
and gateway address without needing to list all host interfaces.

The field format mirrors Cloud.asdict(): name, type, special, and
ip_addresses list (both IPv4 and IPv6).
2026-07-19 18:10:51 +08:00
YueGuobin ddf7d4ca60 fix(controller): refresh cloud/nat node interfaces from compute on GET
Cloud and NAT nodes need live host network interface data. Previously,
GET /projects/{project_id}/nodes/{node_id} returned cached properties
from creation time, so newly added host interfaces (e.g. kernel bridges
created by EthernetSwitch nodes) were invisible until the node was
deleted and recreated.

Now the controller fetches fresh data from the compute node before
returning the response, so host interface changes are reflected
immediately. Falls back to cached data if compute is unreachable.
2026-07-19 17:41:37 +08:00
Jeremy Grossmann e1b09e683e Merge pull request #2822 from yueguobin/feature/ethernet-switch-ubridge-brctl
POC: migrate builtin EthernetSwitch from Dynamips ethsw to ubridge brctl
2026-07-17 23:49:01 +02:00
YueGuobin 7c320d6100 docs: add builtin Ethernet Switch uBridge brctl implementation document 2026-07-18 01:27:53 +08:00
YueGuobin 18a123e0b9 refactor(builtin): use deterministic bridge name from switch UUID
Replace psutil.net_if_addrs() scan with a deterministic name derived from
the switch's node UUID: gns3 + first 6 hex chars (10 chars, fits kernel
IFNAMSIZ limit of 15). Taps: <bridge>-<port> (12-13 chars).

Crash recovery: brctl delete the bridge first (best-effort), then create
fresh. Stale bridges from abnormal gns3server shutdown are automatically
reclaimed on the next start — no EEXIST or leaked interfaces.

Remove _free_iface helper and psutil import (no longer needed).
2026-07-18 01:18:32 +08:00
YueGuobin aabbee3a04 fix(builtin): bring kernel bridge UP after creation
brctl create leaves the bridge administratively DOWN. Add link set up
so the bridge actually forwards frames between enslaved ports.

Also record Docker iptables FORWARD DROP pitfall in project memory.
2026-07-18 01:11:25 +08:00
YueGuobin fd4ac4460d feat: migrate builtin EthernetSwitch from Dynamips ethsw to ubridge brctl
Replace the builtin EthernetSwitch stub with a Linux kernel bridge backed
by uBridge's brctl module. Each switch node creates one kernel bridge
(gns3br{N}) with VLAN filtering; each port is a persistent TAP enslaved to
the bridge, relayed by a per-port uBridge bridge (nio_tap <-> nio_udp).

- Access/dot1q/qinq port modes translated to brctl vlan primitives
- Compute router repointed from Dynamips to Builtin manager
- Tests updated: 21 router-level tests + 215 surrounding tests pass
- Real-kernel e2e verified: access 100 PVID untagged, dot1q trunk
  VIDs 1-4094 + native 1, qinq 802.1ad proto + 200 PVID

Ethertype 0x9100/0x9200 handling and port-count guard relaxation are
deferred pending resolution.
2026-07-18 00:21:19 +08:00
Jeremy Grossmann 22a1026633 Merge pull request #2821 from yueguobin/fix/mcp-remove-node-reload
fix(mcp): remove unreliable node_reload / node_reload_all tools
2026-07-16 12:51:29 +02:00
YueGuobin 37867a1e44 fix(mcp): remove unreliable node_reload / node_reload_all tools
For Docker nodes, reload bottoms out as a raw POST /containers/{id}/restart
to the Docker daemon, bypassing GNS3's start/stop lifecycle (uBridge
re-attach, console servers, NIC setup). The container restarts at the
Docker level but GNS3's plumbing goes out of sync, and the daemon call can
block up to the controller's 240s timeout — manifesting as "reload hangs /
no response". stop+start runs the full lifecycle and is reliable.

Remove the node_reload and node_reload_all MCP tools (tool functions,
handlers, NODE_TOOLS entry, tests, docs). The underlying REST endpoints
(POST /nodes/{id}/reload, POST /nodes/reload) are kept for native API
users. MCP callers should use node_stop + node_start (partial) or
close/open project (full restart) instead.
2026-07-16 13:27:21 +08:00
Jeremy Grossmann 013ca442f4 Merge pull request #2816 from yueguobin/feature/traffic-insight-markers
feat: add traffic-insight markers (ubridge mark filter integration)
2026-07-15 22:12:11 +02:00
grossmj c75e1ba0be Development on 2.2.61.dev1 2026-07-15 20:45:33 +02:00
Jeremy Grossmann c1e845cb22 Merge pull request #2819 from GNS3/release/v2.2.60
Release v2.2.60
2026-07-15 20:44:04 +02:00
Jeremy Grossmann 3954d6477a Merge pull request #2817 from yueguobin/fix/mcp-rbac-token-hardening
fix: thread token_version through MCP token minting and guard websocket auth failure
2026-07-15 19:20:44 +02:00
Guobin Yue 64da926d2f Merge branch '3.1' into fix/mcp-rbac-token-hardening 2026-07-16 01:03:11 +08:00
Guobin Yue 2e9073be39 Merge branch '3.1' into feature/traffic-insight-markers 2026-07-16 01:03:06 +08:00
Jeremy Grossmann 3e12ddf321 Merge pull request #2818 from yueguobin/docs/api-testing-skill
docs: add a Claude Code skill with curl patterns for testing the REST API
2026-07-15 18:57:19 +02:00
YueGuobin dc7f5d76f4 docs(api-testing): add curl-based GNS3 API testing skill 2026-07-16 00:54:33 +08:00
YueGuobin 884c40038c docs(mcp): note --no-close for node_console websocat usage
A heredoc (<<<) closes stdin at once, so websocat dropped the WebSocket
before the device's reply arrived. Add --no-close to the node_console usage
examples and the returned command field so the connection stays open while
output is read.
2026-07-16 00:51:45 +08:00
YueGuobin d4389b7068 fix(rbac): guard None current_user on websocket auth failure
get_current_active_user_from_websocket returns None after closing the socket
on an auth failure (revoked token, bad credentials, inactive user).
has_privilege_on_websocket dereferenced current_user.is_superadmin without a
None check, so any websocket auth failure surfaced as an AttributeError
traceback instead of a clean close. Bail out early when current_user is None,
mirroring the guard already present in ws_console.
2026-07-16 00:51:45 +08:00
YueGuobin 3d6d9a3396 fix(mcp): thread token_version into console/download token minting
Token revocation is a strict version check (token_data.token_version !=
user.token_version). create_access_token defaults token_version to 0, so the
short-lived JWTs minted for the console WebSocket URL (nodes) and the download
URLs (symbols, links) carried ver=0. Any user who had logged out at least once
(token_version >= 1, e.g. the default admin) got tokens rejected as "revoked"
on first use.

e433991cf fixed this in _resolve_token's API-key branch but missed these three
independent minting sites. Now resolve token_version during _resolve_token (the
JWT branch decodes it, the API-key branch reads user.token_version), carry it
through gns3_ctx, and pass it at every minting call.
2026-07-16 00:51:45 +08:00
YueGuobin 89e11d265f fix(mcp): pass user.token_version when generating temp JWT from API key
_resolve_token generated a temp JWT with a hardcoded ver=0 after
validating the API key. Users who had logged out at least once
(token_version >= 1) would hit "Token has been revoked" 401 on
every MCP tool call, because the REST auth chain rejects ver=0
when the user's token_version no longer matches.

Fix: pass the user's actual token_version to create_access_token
so the temp JWT carries the correct ver claim.
2026-07-16 00:51:45 +08:00
YueGuobin 45d18de8af fix(marker): avoid reentrant-lock deadlock when uBridge starts lazily
_start_ubridge configures the marker sink via _ubridge_configure_marker_sink, which used _ubridge_send. But _start_ubridge itself is reached THROUGH _ubridge_send when uBridge is started lazily — e.g. linking a stopped node. _ubridge_send's @locking lock (___ubridge_send_lock) is non-reentrant, so the nested _ubridge_send('marker sink') deadlocked forever, making the NIO create POST time out after 120s. Started nodes never hit this (uBridge already running, no _start_ubridge), so only stopped-node links (notably dynamips) hung.

_ubridge_configure_marker_sink now talks to self._ubridge_hypervisor.send directly: it runs inside _start_ubridge right after connect(), so uBridge is already up and the raw send is safe, with no reentrant lock acquisition.
2026-07-16 00:39:52 +08:00
YueGuobin 08f4b5ae0d docs(marker): document per-link attribution and IOU multi-interface
Add a Per-link attribution section (link field semantics, pcap path keyed on link) with an IOU subsection explaining the shared IOL-BRIDGE and per-port filter lists. Note that the marker.match link_id comes from the signal link= field, and call out shared capture-side node behavior in Notes.
2026-07-16 00:39:52 +08:00
YueGuobin 794bfef450 fix(marker): per-link attribution via link field in MARK signals
A uBridge MARK signal carries only node= and filter= (no bridge/link field), so when one node is the capture side for several links that share a marker name (always the case for global-{name} definitions on a multi-interface node) the signals were indistinguishable and the (node, filter) registry collapsed them to a single link.

The mark filter is now stamped with its link id (mark <bpf> ... link <link_id>); uBridge echoes it verbatim (link=<link_id>) and the listener uses the signal's link= as the authoritative link_id of the marker.match event, falling back to the registry only for legacy signals without it. base_node and iou apply paths pass link_id; covered by two new listener tests.
2026-07-16 00:39:52 +08:00
YueGuobin 27356da62d fix(marker): set nio.markers on iou/dynamips/cloud nio update
The PUT /nio (update) endpoints for iou, dynamips and cloud copied nio.filters from the request body but never copied nio.markers. So adding a marker (e.g. a project-level global definition) to an already-created link of these node types left nio.markers empty: no mark filter was installed on uBridge and no marker.match signal was emitted. vpcs/qemu/docker already set it; mirror them (getattr for the Union NIO types).
2026-07-16 00:39:52 +08:00
YueGuobin a732d8d521 test(marker): include marker_definitions in project-asdict expectations
project.asdict() and project_to_topology() gained a marker_definitions key.
Two tests that assert exact dict equality needed the new field.
2026-07-16 00:39:52 +08:00
YueGuobin 213738f0a5 test(marker): add controller + API route tests for traffic-insight
19 controller-layer tests covering start/stop/update_marker (storage,
inheritance guards + bypasses, partial-update preservation of render
hints), project-def CRUD (fan-out, sync, delete-cleanup regression),
apply_defs_to_new_link, persist_markers/asdict, and aggregation.

15 API-route tests covering per-link create/update/delete (201/200/204),
global-prefix rejection on create (409 regression), bad-format rejection
(422), PUT/DELETE-on-inherited guard (409 regression), project-def CRUD
endpoints, and the aggregation view.

All 34 tests pass when run as part of the full suite.
2026-07-16 00:39:52 +08:00
YueGuobin cbb4854d3d docs(marker): add traffic-insight feature documentation
Document the marker feature following the gns3-documentation standard:
overview, Mermaid architecture and business-process diagrams, API endpoint
tables, request/response and field reference, error responses, and notes
covering the key design points — immutable name (rename = delete + create),
global-prefix reservation, read-only inherited markers, render hints
(color / highlight_duration), supported node types, and persistence.

Indexed under Features in docs/README.md.
2026-07-16 00:39:52 +08:00
YueGuobin b21780605e fix(marker): reject reserved global-prefix names only on create
The global-prefix reservation belongs on the create path, where it keeps
user-chosen names disjoint from inherited global-{name} markers. It was
implemented as a field_validator on MarkerCreate / MarkerDefinitionCreate,
which also back the PUT update bodies — so updating an inherited marker
while echoing its name in the body tripped the validator with a generic
422 before the controller could return the actionable 409 ("inherited,
use the marker-definitions API").

Drop the schema validators and enforce the prefix in the two create routes
(ControllerError -> 409). PUT no longer validates the body name (ignored
anyway — the target is the {name} path param), so editing an inherited
marker now reaches the controller's inheritance guard and returns the
clear 409. The name-format regex stays on the schema (still 422).
2026-07-16 00:39:52 +08:00
YueGuobin cdc27c37a7 fix(marker): clean inherited markers on def delete
delete_marker_definition removed the def but left the inherited global-*
copies on every link: it called stop_marker(), which rejects inherited
markers (409), and the ControllerError was swallowed as a warning. The
orphaned copies were in-memory only (_persist_markers filters inherited
markers) so a restart hid the symptom, but during a running session they
were undeletable via either the per-link or project API.

Add the same inherited=True bypass that update_marker already has, and
pass it from the def-delete fan-out so the copies are removed for real.
2026-07-16 00:39:52 +08:00
YueGuobin 98bcd5eddd feat(marker): add highlight_duration render hint
Per-marker UI hint (milliseconds, ge=1) for how long the Web UI keeps a
marker highlighted after a match. Mirrors color: stored on the link,
persisted in the topology, inherited via project-level definitions, and
never sent to uBridge. Omitted = null = frontend uses its own default.

The default is intentionally NOT set in the schema: MarkerCreate also
backs PUT updates, so a schema default would make every partial update
silently reset the value. None means "not provided" (keep existing on
update / frontend decides on create).
2026-07-16 00:39:52 +08:00
YueGuobin 179f072c33 fix(marker): fix inheritance hook, topology load, update sync, and asdict
Bugs found via end-to-end testing of project-level marker definitions:

1. New links didn't inherit — apply_defs_to_new_link is async but was
   called without await in UDPLink.create().

2. Project load crashed — load_project passed marker_definitions to
   Project.__init__. Now popped in load_project and restored separately
   in Project.open() (it backs a read-only property).

3. PUT on a definition didn't sync to links — update_marker's guard
   rejected even the project-layer sync call. Added an `inherited`
   bypass flag used by update_marker_definition.

4. _ubridge_add_marker_filter raised re.PatternError — the name regex
   used (?i)(?!global) look-around, invalid in Python's re module.
   Dropped the prefix check there: "global-*" names are legitimate at
   the uBridge boundary (inherited definitions); forbidden only at the
   user-facing schema.

5. GET /links hid inherited markers — asdict()'s runtime branch used
   _persist_markers() (which filters inherited markers). Restored
   self._markers for the runtime branch; only the topology_dump branch
   filters (inherited markers are rebuilt from definitions on load).

6. Duplicate "already exists" warnings on project open — open() fanned
   out definitions to all links, but UDPLink.create() had already done
   so via its inheritance hook. Removed the redundant fan-out in open().
2026-07-16 00:39:52 +08:00
YueGuobin 84845f8504 fix(marker): export MarkerDefinitionCreate from schemas package 2026-07-16 00:39:52 +08:00
YueGuobin 686f96b748 fix(marker): replace pydantic lookahead pattern with field_validator
pydantic_core (Rust engine) does not support regex look-around.
Keep the character-class pattern for basic safety and enforce the
"global" prefix reservation via a field_validator instead.
2026-07-16 00:39:52 +08:00
YueGuobin 31991fe359 feat(marker): add project-level marker definition inheritance
Project-level marker definitions fan out to every link (existing and new).
A definition is stored once on Project._marker_definitions; when applied to
a link the marker is named "global-{def_name}" — the "global" prefix was
pre-reserved in the schema, so inherited and per-link markers can never
collide, nor will their registry keys.

Key behavior:
- POST /projects/{pid}/marker-definitions → fan out to all existing links
- PUT  /projects/{pid}/marker-definitions/{name} → sync all inherited copies
- DELETE → remove every inherited copy from every link
- New links auto-inherit all active defs (hook in UDPLink.create)
- Per-link DELETE/PUT of a "global-*" marker is rejected (409)
- Inherited markers are NOT persisted in the topology; they are re-created
  from _marker_definitions on project load
- Compute side is untouched — the marker reaches uBridge via the existing
  start_marker→update→NIO→ubridge pipeline

Files:
- controller/project.py — _marker_definitions + CRUD + fanout + topology load
- controller/link.py — Link.inherit_marker() + asdict() filter
- controller/udp_link.py — guards on stop/update + create() inheritance hook
- controller/topology.py — persist marker_definitions in project topology
- schemas/controller/links.py — MarkerDefinitionCreate schema
- api/routes/controller/projects.py — REST endpoints (marker-definitions)
2026-07-16 00:39:52 +08:00
YueGuobin 369badc6c2 fix(marker): reserve "global" name prefix for future marker inheritance 2026-07-16 00:39:51 +08:00
YueGuobin d3527c00de fix(marker): extend marker support to dynamips and cloud nodes
Dynamips needed its own NIO base class extended with a `_markers`
attribute and `markers` property (mirroring the existing `filters`
pattern), plus marker propagation from the controller-supplied NIO
settings through NIOUDP.create/update to the common destination
NIO that base_node._ubridge_apply_markers reads.

Cloud simply needed the missing _ubridge_apply_markers calls after
_ubridge_apply_filters in its two ubridge connection methods, same
pattern as the earlier docker fix.

Both types are now added back to _MARKER_CAPABLE_TYPES.
2026-07-16 00:39:51 +08:00
YueGuobin e4be98984c fix(marker): enable marker support for docker nodes
Docker's _connect_nio and adapter_update_nio_binding applied packet
filters but never called _ubridge_apply_markers, so markers silently
did nothing on Docker links despite docker being in the allowlist.
Add the missing calls (same pattern as the IOU fix).

Also narrow _MARKER_CAPABLE_TYPES to the four types that actually
implement marker support — vpcs, qemu, docker, iou — removing
dynamips, virtualbox, vmware, and cloud which have no marker pathway
and would silently fail when selected as the capture side.
2026-07-16 00:39:51 +08:00
YueGuobin f7fb43a4b9 fix(marker): tolerate port conflict on marker listener startup
When the configured marker_listen_port (default 3070) is already in
use, fall back to an OS-assigned port instead of crashing the server.
If even port 0 fails, log the error and continue without marker
signals — traffic insight degrades rather than taking down startup.
2026-07-16 00:39:51 +08:00
YueGuobin 7e4600b5a1 fix(marker): validate marker name and expose enabled via REST
Add a name pattern constraint to MarkerCreate schema (alphanumeric +
_.-) to prevent injection into uBridge commands, with a matching
defense-in-depth check in _ubridge_add_marker_filter for hand-edited
topology files. Also add the missing "enabled" field to MarkerCreate
and pass it through the PUT route so the documented toggle actually
works.
2026-07-16 00:39:51 +08:00
YueGuobin a495b5ddf2 fix(marker): uniquify auto-generated marker names per link
Auto-generating `marker-{link.id[:8]}` collided on the second anonymous
marker on the same link (start_marker rejects duplicate names). Append a
short uuid hex suffix so multiple anonymous markers coexist.
2026-07-16 00:39:51 +08:00
YueGuobin f2360f85fc feat(marker): add project-level marker aggregation endpoint
Add a read-only `markers` property on Project that flattens every link's
markers into a single dict keyed by "{link_id}/{name}", each entry
carrying the parent link_id and capture-side node_id. Expose it via
GET /projects/{pid}/markers (Project.Audit) so the frontend can fetch all
markers in one round-trip instead of enumerating links first.

Also surface a marker count in project.stats().
2026-07-16 00:39:51 +08:00
YueGuobin ab5b7444d4 feat(marker): add IOU node marker support via iol_bridge
IOU uses iol_bridge (not bridge) with extra bay/unit positional
arguments in add_packet_filter. Add _ubridge_apply_markers override
that mirrors the existing _ubridge_apply_filters pattern, called at
the same three sites: _networking, adapter_add_nio_binding, and
adapter_update_nio_binding.
2026-07-16 00:39:51 +08:00
YueGuobin c62b9b0283 refactor(marker): converge to filter single-path model, remove dual-apply endpoints
Markers now follow exactly the same apply pattern as packet filters:
state lives in Link._markers, application goes through NIO
(update() -> PUT /nio -> _ubridge_apply_markers). The former
immediate-apply REST endpoints (/markers/start, /markers/stop on
the compute side) and the per-node start_marker/stop_marker methods
are removed — they were a legacy of the original capture-inspired
design and have been superseded by the NIO flow.

Changes:
- controller/udp_link: start_marker/stop_marker/update_marker now
  set _markers state + call self.update() (mirrors update_filters).
  Removed _marker_capture_nodes runtime dict and its helpers.
- controller/project: _create_link_from_topology_data restores
  _markers directly from persisted data (with BPF validation,
  like filter reload). No long calls start_marker during load.
- compute: _ubridge_apply_markers swallows BPF compile errors
  (warn+skip), matching _ubridge_apply_filters behaviour so a
  single bad expression cannot break link creation / node restart.
- Removed: /markers/start,stop endpoints (6 handlers across
  vpcs/qemu/docker route files), node start_marker/stop_marker
  methods (3 VM files), _ubridge_delete_marker_filter,
  _marker_capture_nodes, MarkerDelete schema.

Net: ~280 lines of dead code removed; marker and packet filter now
share a single, unified apply path via the NIO.
2026-07-16 00:39:51 +08:00
YueGuobin 88c03d1431 fix(marker): restore markers from topology on project load
_create_link_from_topology_data now restores traffic-insight markers
(mirroring how filters are restored via update_filters), so markers —
including their color — survive project close/reopen and server restart.

Guard start_marker's uBridge POST with 'if self._created' (exactly as
update_filters guards its update() call): during project load the link
is not yet created, so only _markers state is recorded and the marker
is applied once via the NIO flow in create()/_ubridge_apply_markers —
no double application.
2026-07-16 00:39:51 +08:00
YueGuobin 3f15ac38f6 feat(marker): add optional color field to marker data model
Add a user-chosen hex color (e.g. '#ff5722') to each marker, stored in
Link._markers alongside bpf/tag/enabled. The color:

- Is serialized with the link (asdict) and persisted in the topology, so
  it survives project reload and is consistent across devices.
- Never reaches uBridge (compute ignores it) — it is purely a Web UI
  concern.
- Can be updated independently of the BPF (color-only changes skip the
  ubridge round-trip).

MarkerCreate schema gains color: Optional[str]; start_marker and
update_marker carry it through; REST create/update endpoints pass it.

The marker.match WebSocket event stays lean (no color) — the Web UI maps
filter -> color from the link state it already holds.
2026-07-16 00:39:51 +08:00
YueGuobin b8f49696cc fix(marker): use allowlist in _choose_marker_side, align with _get_filter_node
Replace the complex tier-based fallback (which leaked into unsupported
switch types on the final tiers) with a simple allowlist of
_MARKER_CAPABLE_TYPES, mirroring _get_filter_node in link.py.

Also accept stopped nodes — like packet filters, markers are stored on
the NIO and applied when the node starts via _ubridge_apply_markers.
2026-07-16 00:39:51 +08:00
YueGuobin 239f42f5aa fix(marker): stop auto-deleting markers on node stop in node_updated
Markers are persistent link-scoped features (like packet filters), not
transient runtime features (like capture).  Tearing them down on every
node stop breaks stop/start restore: the NIO-based re-application in
_ubridge_apply_markers works correctly on restart, but node_updated
was prematurely calling stop_marker which erased _markers state and
dumped an empty topology.

Now markers survive node stop transparently — the ubridge filter dies
with the process but is re-added from nio.markers in
add_ubridge_udp_connection when the node starts again, mirroring
exactly how packet filters behave.
2026-07-16 00:39:51 +08:00
YueGuobin 7c90fa9e64 fix(marker): ride markers on NIO so they survive node restart and project reload
Mirror the packet-filter lifecycle: marker specs now live on the NIO
(next to filters), ride in link_data from controller to compute on
every NIO create/update, and are reapplied by _ubridge_apply_markers
in add_ubridge_udp_connection (bridge creation / node restart) and
update_ubridge_udp_connection (NIO update — following the preceding
reset_packet_filters so markers survive filter changes).

Changes:
- NIO / NIOUDP: _markers property + asdict
- schemas/compute/nios.py: UDPNIO.markers field
- base_manager.create_nio: nio.markers from settings
- PUT /nio routes (vpcs/qemu/docker): nio.markers update
- base_node: _ubridge_apply_markers(bridge_name, nio) iterates
  nio.markers, computes pcap path, calls _ubridge_add_marker_filter
  + MarkerManager.register; called after _ubridge_apply_filters
- controller udp_link: _get_node_markers + _markers_for_node (route
  by capture_node_id); markers in create() and update() link_data
- /markers/start,stop endpoints: mirror spec onto nio.markers so
  the marker survives a subsequent node stop/start without a PUT
- tests: add markers field to NIO data expectations

This covers:
- Node restart: NIO persists, add_ubridge_udp_connection re-applies
- Filter update: reset wipes markers, _ubridge_apply_markers re-adds
- Project reload: create() carries markers in link_data → create_nio
- Immediate create: endpoint sets nio.markers immediately
2026-07-16 00:39:51 +08:00
YueGuobin 540e1c5678 config: set marker_listen_port default to 3070 and document in config templates
Change the marker UDP sink default from port=0 (OS-chosen) to 3070 for
predictable firewall/debugging/monitoring.

Add marker_listen_host and marker_listen_port with documentation to
config_samples/gns3_server.conf and scripts/gns3_server.conf.
2026-07-16 00:39:51 +08:00
YueGuobin 88465e872c fix(marker): add _choose_marker_side to exclude non-uBridge node types
_choose_capture_side prefers BUILTIN_NODE_TYPES which includes
ethernet_switch/ethernet_hub/NAT etc. — nodes that do not use uBridge
and therefore have no bridge for a marker filter to attach to.

Add _choose_marker_side with a _MARKER_UNSUPPORTED_TYPES blocklist
so marker capture-side selection skips NIO-only switches and picks
the uBridge-backed node (VPCS/QEMU/Docker/etc.) instead.

Verified end-to-end: PC1↔Switch↔PC2 links with arp markers correctly
pick VPCS nodes; MARK signals flow through to WebSocket and pcaps
are written.
2026-07-16 00:39:51 +08:00
YueGuobin d2e9823e6c feat: implement traffic insight marker backend (ubridge mark filter)
Add compute-side marker subsystem that receives ubridge UDP MARK signals
and forwards them as project-scoped notifications to the web UI for
real-time traffic coloring. Matched packets are always saved to per-link
pcaps for future replay.

Key components:
- gns3server/compute/marker/: MarkerManager (singleton, UDP listener +
  O(1) registry keyed by (node_id, filter_name)) + MarkerListener
  (DatagramProtocol parsing MARK lines per ubridge integration contract)
- gns3server/compute/base_node.py: marker sink/node config at ubridge
  startup; shared _ubridge_add_marker_filter / _ubridge_delete_marker_filter
- Per-node start_marker/stop_marker: VPCS (VPCS-{id}), QEMU
  (QEMU-{id}-{adapter}), Docker (bridge{adapter})
- Compute REST /markers/start + /markers/stop (vpcs/qemu/docker route files)
- Controller Link._markers state + UDPLink.start_marker/stop_marker/
  update_marker (mirror capture pattern: BPF validation, _choose_capture_side,
  node.post forwarding, topology persistence)
- Controller REST GET/POST/DELETE/PUT /v3/projects/{p}/links/{l}/markers
- Config: marker_listen_host / marker_listen_port in ServerSettings
- Signal routing: creation-time registry O(1) lookup, no node-table scan;
  project-scoped WS stream (not global); event payload always carries
  project_id for frontend scoping

Tests: 14 unit tests (registry, listener parsing, UDP round-trip);
562 existing tests pass with zero regressions.
2026-07-16 00:39:51 +08:00
grossmj 803cc17410 Release v2.2.60 2026-07-15 18:36:26 +02:00
grossmj f088c55cfe Sync appliances 2026-07-15 18:32:12 +02:00
Jeremy Grossmann 8a8864c90b Merge pull request #2814 from GNS3/bugfix/2783
fix(import): project import does not move images symlinks into place
2026-07-15 09:53:15 +02:00
grossmj 1986bce538 Only set IOU images to be executable when importing images 2026-07-15 09:43:34 +02:00
grossmj c528d2cd95 fix(import): project import does not move images symlinks into place 2026-07-13 18:19:52 +02:00
Jeremy Grossmann 7529ee4168 Merge pull request #2813 from GNS3/uefi-improvements
UEFI improvements for Qemu VMs
2026-07-13 12:05:07 +02:00
grossmj 61c7f4469d Search for system UEFI that is compatible with Python < 3.12 2026-07-13 11:34:29 +02:00
grossmj 896b4fc56a UEFI improvements for Qemu VMs
* Add OVMF firmware directory configuration
* Automatically add a Random Number Generator (RNG) device when using uefi option is enabled
2026-07-13 11:08:17 +02:00
grossmj a58fa8602b remove wrongly merged code 2026-07-13 10:05:01 +02:00
YueGuobin d8f8b6ec70 fix(docker): handle container name conflict automatically
When a Docker container with the same name already exists (e.g., from a
   previous crashed GNS3 session), Docker returns a 409 Conflict error
   when trying to create a new container with that name. This causes the
   project open operation to fail.

   This fix adds automatic cleanup of stale containers when encountering
   a name conflict:
   - Added DockerHttp409Error exception class
   - Updated http_query to detect 409 status codes
   - Modified create() to remove conflicting containers and retry

   Fixes the issue where opening a project fails with:
   "Docker has returned an error: 409 Conflict. The container name
   '/GNS3.xxx' is already in use by container 'xxx'"
2026-07-13 09:57:55 +02:00
grossmj 01d72aed9a Merge branch '2.2' into 3.1
# Conflicts:
#	gns3server/controller/__init__.py
#	gns3server/controller/project.py
2026-07-13 09:32:07 +02:00
Jeremy Grossmann b256d5a5f6 Merge pull request #2812 from GNS3/bugfix/2788
fix: gns3-server crashes on startup if "Open this project in the background" is active but there is a problem with that project
2026-07-12 13:50:11 +02:00
grossmj ea09cc2724 fix: gns3-server crashes on startup if "Open this project in the background" is active but there is a problem with that project 2026-07-12 13:46:12 +02:00
grossmj e3e0a511b6 Handle HTTPNotFound exception when retrieving compute status 2026-07-12 11:08:13 +02:00
grossmj 0ecaab4da0 Replace ControllerForbiddenError with aiohttp.web.HTTPForbidden for project deletion error handling 2026-07-12 11:05:15 +02:00
YueGuobinandClaude Opus 4.6 40644f33e6 Fix: Check compute connectivity before open() during project deletion
Previously the disconnected compute check ran after open(), which would
block for 120s trying to connect to unreachable remote computes before
rejecting the deletion. Now reads the topology file directly to extract
compute IDs and checks connectivity before calling open(), enabling
immediate rejection of deletions involving offline computes.

Also removes the redundant post-open() check since the early check
covers both opened and closed project states.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-12 10:57:56 +02:00
grossmj a9cce209d7 Development on 3.1.0.dev5 2026-07-10 10:01:35 +02:00
Jeremy Grossmann 94a131dab3 Merge pull request #2810 from yueguobin/feature/cloud-host-interface-ip-addresses
Expose full host interface IP addresses and link state in cloud node API
2026-07-10 09:19:54 +02:00
YueGuobin 0ed1715fe6 Use typed InterfaceStatus enum and simplify stats fallback in interfaces()
- Change `status: str` to `status: InterfaceStatus` (str Enum) in the
  HostInterface schema, matching the existing IPAddressFamily pattern.
- Use default-then-override pattern in interfaces() to eliminate the
  `else` branch and deduplicate defaults.
2026-07-10 14:55:21 +08:00
YueGuobin 9721660cc3 Expose full interface address list and link state in cloud node API
Each host interface surfaced by the cloud node now reports:
- ip_addresses: every IPv4 and IPv6 address (previously only a single
  IPv4 was collected internally and then dropped before the response)
- status / speed / mtu / flags: operational state and link attributes
  sourced from psutil.net_if_stats(), with flags normalized to a list

The legacy ip_address / netmask / mac_address fields are preserved so
existing callers (compute link detection, GNS3 VM, VMware, has_netmask)
keep working. The new fields travel through the existing interfaces
payload that the controller forwards verbatim, so no controller-side
change is required and the PUT / ports_mapping flow is unaffected.
2026-07-10 13:56:07 +08:00
grossmj a66740e154 Add script to create and push annotated git tags from version.py 2026-07-09 11:02:33 +02:00
grossmj 913d021705 Release v3.1.0a4 2026-07-09 10:40:38 +02:00
grossmj d64fdd71a9 Bundle web-ui v3.1.0a4 2026-07-09 08:40:05 +02:00
Jeremy Grossmann bdd73be6d6 Merge pull request #2808 from yueguobin/docs/jwt-refresh-token-config-example
Add jwt_refresh_token_expire_minutes to sample configuration
2026-07-09 08:34:25 +02:00
Guobin Yue 61ce59788b Merge branch '3.1' into docs/jwt-refresh-token-config-example 2026-07-09 14:19:14 +08:00
YueGuobin cccc3e5b77 Add jwt_refresh_token_expire_minutes to sample configuration
The sample gns3_server.conf documented jwt_access_token_expire_minutes but omitted jwt_refresh_token_expire_minutes (default 43200 minutes, 30 days) defined in ControllerSettings. Add it for parity with the schema.
2026-07-09 14:15:49 +08:00
grossmj 6fba497734 Remove deleted web-ui files from git 2026-07-08 22:40:27 +02:00
grossmj 54540e7ef7 Merge remote-tracking branch 'origin/3.1' into 3.1 2026-07-08 21:49:20 +02:00
grossmj 5992a284c9 Bundle web-ui v3.1.0a4 2026-07-08 21:48:48 +02:00
grossmj a2ee5389af Bundle web-ui v3.1.0a4 2026-07-08 20:11:59 +02:00
grossmj 4548d9053a Sync appliances 2026-07-08 18:45:07 +02:00
grossmj 2d3bb89e58 Merge remote-tracking branch 'origin/2.2' into 3.1 2026-07-08 10:33:00 +02:00
grossmj 809d4cc609 Merge branch 'master' into 2.2
# Conflicts:
#	.github/workflows/docker-build.yml
2026-07-08 10:30:53 +02:00
Jeremy Grossmann 89e4e836c9 Merge pull request #2806 from my-code-is-grey/2798-dockerhub
#2798 - DockerHub adjustments
2026-07-08 05:17:04 +02:00
my-code-is-grey 69ddcf553f .dockerignore ignore .dockerignore 2026-07-07 22:37:24 -04:00
my-code-is-grey 1e1ec5aae8 typo made me reconsider and move out to env 2026-07-07 21:58:49 -04:00
my-code-is-grey e30e6435ca defaulted DOCKERHUB_ORG repository variable 2026-07-07 21:48:13 -04:00
my-code-is-grey 77700f361c 2798 2026-07-07 01:30:58 -04:00
grossmj e3e3a7e729 Update fastapi to v0.139.0 2026-07-06 19:21:56 +02:00
grossmj 6415d2e56c Update dependencies 2026-07-06 19:06:25 +02:00
grossmj 424ae55c38 Revert "Update dependencies"
This reverts commit b6793a50f3.
2026-07-06 18:38:48 +02:00
grossmj b6793a50f3 Update dependencies 2026-07-06 18:31:52 +02:00
grossmj 562108748a Remove API docs from 2.2 after merging 2026-07-06 18:17:00 +02:00
grossmj d54b16df53 Merge branch '2.2' into 3.1
# Conflicts:
#	gns3server/handlers/api/controller/template_handler.py
#	tests/handlers/api/controller/test_template.py
2026-07-06 18:13:17 +02:00
Jeremy Grossmann ad4aa1d49b Merge pull request #2803 from GNS3/update-ga-workflows
Update GitHub Actions workflows
2026-07-06 18:10:17 +02:00
grossmj 8499ed3d65 Update GitHub Actions workflows 2026-07-06 18:05:28 +02:00
Jeremy Grossmann a0ca8761c0 Merge pull request #2802 from yueguobin/fix-web-wireshark-docker-xpra6.x
Fix web-wireshark docker build broken by xpra 6.5 release
2026-07-06 17:32:31 +02:00
Guobin Yue 277640a335 Merge branch '3.1' into fix-web-wireshark-docker-xpra6.x 2026-07-06 13:35:44 +08:00
YueGuobin f5a72532ac Fix web-wireshark docker build broken by xpra 6.5 release
Replace the `xpra=6.4*` version clamp on the apt-get install command
line with APT pinning via /etc/apt/preferences.d/pin-xpra.

The previous specifier stopped resolving once xpra 6.5 was released:
the xpra metapackage was pinned to 6.4 while its sub-packages
(xpra-client, xpra-server, xpra-codecs, ...) were unconstrained and
defaulted to 6.5, so apt could not satisfy the dependency graph and
the "Build Docker images" action failed.

Pinning all xpra* packages at the preferences level constrains the
whole set consistently (xpra-html5 is handled separately since it
follows a different version scheme, locked to v19). Blocking every
other version with priority -1 ensures that, if the pinned version
ever disappears from the repository, apt fails loudly instead of
silently upgrading to the latest release.

Fixes GNS3/gns3-registry#1044
2026-07-06 13:31:07 +08:00
Jeremy Grossmann 7650409ab6 Merge pull request #2797 from yueguobin/feature/optional-ai-support-2794
Make AI features (AI Copilot + MCP) optional via [ai-features] extra
2026-06-27 19:44:27 +02:00
YueGuobin e0ef63ebc1 Update CI to install [ai-features,dev] instead of [ai-copilot,dev]
The [ai-copilot] extra has been replaced by [ai-features] which
includes both AI Copilot and MCP dependencies.
2026-06-27 22:19:39 +08:00
YueGuobin 292b60efaa Make AI features (AI Copilot + MCP) optional via [ai-features] extra
- Move fastmcp from core requirements.txt to mcp-requirements.txt
- Add MCP_AVAILABLE feature flag in agent/__init__.py (graceful degradation)
- Guard MCP imports/registration in server.py and tasks.py
- Replace ai-copilot/mcp/ai-support extras with single ai-features extra
- Add stub MCP routes returning 501 when MCP is not installed
- Add gns3server-uninstall-ai-features CLI command
- Remove old gns3server-uninstall-ai-copilot command
- Update all error messages and docs to reference ai-features

Closes #2794
2026-06-27 10:14:43 +08:00
Jeremy Grossmann 58d3055dd3 Merge pull request #2792 from yueguobin/feature/refresh-token-mechanism-2786
Add stateless JWT refresh token mechanism
2026-06-24 00:26:07 +02:00
YueGuobin 0dd2fb4173 Add refresh token mechanism documentation under docs/features/ 2026-06-23 23:36:37 +08:00
YueGuobin c8c14af9f4 Add /refresh to allowed public endpoints list in route auth test 2026-06-23 23:19:07 +08:00
YueGuobin 0e8d0cb87b Add stateless JWT refresh token mechanism
- New config: Controller.jwt_refresh_token_expire_minutes (default 30 days)
- New endpoint: POST /v3/access/users/refresh (public, unauthenticated)
- Login/authenticate responses now include refresh_token
- AuthService: _create_token helper, create_refresh_token, get_token_data
  now parses type claim (token_use) for token classification
- Security: refresh tokens rejected on HTTP + WebSocket access paths;
  /refresh strictly requires type=='refresh'
- Logout works for free via existing token_version mechanism
- Tests: 9 new TestRefreshToken cases, all passing; 34 existing tests
  still pass (no regressions)
2026-06-23 22:39:11 +08:00
Jeremy Grossmann 4ffab7abee Merge pull request #2791 from yueguobin/fix/mcp-test-issues
Fix MCP testing issues: appliance_install version, handler unit tests, parameter validation
2026-06-17 21:07:59 +02:00
YueGuobin 471e65e48c Add project/node/link handler tests: 39 total, covering list/get/create/delete/start/stop/suspend/reload/console/update/fields 2026-06-18 00:07:14 +08:00
YueGuobin 5cc826baff Add MCP handler unit tests with mocked Gns3Connector
Tests cover:
- get_nodes_handler: fields filter, invalid fields rejection
- create_node_handler: name passthrough, fields filter, fields validation,
  batch mode inherited template_id, missing project_id
- create_link_handler: compact format, standard format, fields validation
- appliance_get, appliance_install with version
- list_templates: fields filter, invalid field error
2026-06-18 00:02:38 +08:00
YueGuobin dc3a386961 Add MCP tool parameter consistency tests 2026-06-17 23:59:40 +08:00
YueGuobin ead9747e1b Fix appliance_install: add version parameter 2026-06-17 23:03:28 +08:00
Jeremy Grossmann 2e75c9319b Merge pull request #2787 from yueguobin/optimize/project-open-speed
Performance: accelerate project opening with parallel link creation and batch UDP port allocation
2026-06-16 23:59:41 +02:00
YueGuobin f16eba183a Add coordinate system note to docs 2026-06-16 23:30:59 +08:00
YueGuobin 0d21f294f6 Document canvas coordinate system in node_create x/y params 2026-06-16 23:16:17 +08:00
YueGuobin 00647d092c Update docstring: batch concurrency from 10 to 100 2026-06-16 23:05:17 +08:00
YueGuobin da35eaa014 Remove unused import time 2026-06-16 23:04:05 +08:00
YueGuobin 10ff5b2283 Add list type check before nodes[0] access in _normalize_link_nodes 2026-06-16 23:00:47 +08:00
YueGuobin 0d45bfbae5 Add fields type validation in create handlers 2026-06-16 22:58:36 +08:00
YueGuobin 390a40ce80 Use pop() instead of pop(0) for O(1) port removal 2026-06-16 22:56:42 +08:00
YueGuobin 59e5f8dd2f Fix review issues: key_prefix length, count validation, WAL log, timeout comment, pointless temp var 2026-06-16 22:50:22 +08:00
YueGuobin 533d3d5b71 Fix: pass name from TemplateUsage to add_node_from_template
The controller endpoint received name via TemplateUsage but never
passed it to add_node_from_template, causing the name parameter to
be silently ignored. name is optional - if omitted the template's
default_name_format is used.
2026-06-16 22:43:46 +08:00
YueGuobin 68567118f1 Remove unused imports (logging, log, select) 2026-06-16 22:25:07 +08:00
YueGuobin af94c1729b Add warning for unconsumed pre-allocated UDP ports after link creation
Ports are properly released on project close (compute.post /close).
The warning alerts if any pre-allocated ports go unused within a session.
2026-06-16 22:22:46 +08:00
Jeremy Grossmann 7d5d4767bf Remove FIXME comment about middleware in server.py
Removed commented out FIXME note regarding middleware issues.
2026-06-16 12:20:55 +02:00
YueGuobin f7abb62603 Rename device_command_run_handler → device_show_run_handler to match tool name 2026-06-16 12:44:12 +08:00
YueGuobin fbb8924a8e Rename device_command_run → device_show_run for clarity
The name device_command_run was ambiguous and easily confused with
device_config_send. device_show_run makes the read-only intent explicit,
forming a clear pair: config_send (write) vs show_run (read).
2026-06-16 12:27:35 +08:00
YueGuobin e627481ec2 Fix device_command_run KeyError('commands'): tool desc said show_commands but backend expects commands
- Tool description and param schema now use 'commands' (matching ExecuteMultipleDeviceCommands)
- Updated error message in handler
- Clarified in docstring that this is read-only; use device_config_send for config changes
2026-06-16 12:23:15 +08:00
YueGuobin ec58bea24b Fix template_list return type annotation to match _run_handler_sync envelope
template_list was annotated as dict[str, Any] but _run_handler_sync always
returns a list envelope ([{"type":"text",...}]). Other tools use
list[dict[str, Any]] consistently. This caused a Pydantic dict_type
validation error on the client.
2026-06-16 12:08:07 +08:00
YueGuobin a126f910d4 Add performance optimization documentation 2026-06-16 01:14:12 +08:00
YueGuobin 48b7ea8bf0 Update MCP service docs: API key format, auth flow, tool parameters, concurrency 2026-06-16 01:13:01 +08:00
YueGuobin d407b29fe9 Increase HTTP connection pool to 500/1000 2026-06-16 00:58:12 +08:00
YueGuobin 74192f454f Increase BATCH_MAX_WORKERS and Pool concurrency from 20 to 100 2026-06-16 00:49:08 +08:00
YueGuobin 286b2407cd Remove final timing artifact in projects.py 2026-06-16 00:44:54 +08:00
YueGuobin bd15c8d019 Remove remaining dead timing variables and imports 2026-06-16 00:40:03 +08:00
YueGuobin a8cc01af46 Remove database warmup (proven ineffective - real bottleneck was bcrypt blocking event loop) 2026-06-16 00:35:42 +08:00
YueGuobin ccb629f48f Clean up all timing/debug logs
Remove all [MCP-TIMING] and [CTRL-TIMING] log lines, timing middleware,
and related import time statements across 11 files.
2026-06-16 00:34:11 +08:00
YueGuobin 8ef70d9fac Add memory: import validation best practice 2026-06-16 00:25:45 +08:00
YueGuobin 7d8ab399da Fix: add missing UUID imports 2026-06-16 00:24:37 +08:00
YueGuobin d6b337bc3a Generate fresh JWT on API key auth instead of returning raw key
When MCP client authenticates with API key, _resolve_token now
returns a newly generated JWT. This JWT is stored in the ContextVar
and used by all subsequent tool handler API calls - zero extra bcrypt.
2026-06-16 00:09:36 +08:00
YueGuobin f522c947bc Optimize API key auth: O(1) lookup via UUID-embedded key format
New format: gns3_<api_key_id>_<random_secret>
- Auth extracts api_key_id from token → single DB query by UUID → one bcrypt
- No more scanning all keys (was O(n) with bcrypt per key)
- bcrypt.checkpw offloaded to thread pool to prevent event loop blocking
- Legacy gns3_<random> format removed (compatibility break)
2026-06-15 23:56:36 +08:00
YueGuobin 796a2e6ca8 Fix: offload bcrypt.checkpw to thread pool to prevent blocking event loop
bcrypt.checkpw is CPU-bound (~1.3s per call) and was running
synchronously inside the async event loop, blocking ALL concurrent
requests. With 5 API keys and 10 concurrent requests, this caused
~13s delay before any handler could start.
2026-06-15 23:49:28 +08:00
YueGuobin 04f0657661 Add timing to API key auth path and log api_keys count 2026-06-15 23:45:26 +08:00
YueGuobin 3c41dea48f Replace SELECT 1 warmup with full database file read to warm OS page cache
Reading the entire DB file into OS page cache eliminates the 8-14s
cold-start penalty on the first ORM query (users/templates/api_keys).
Previous SELECT 1 only warmed the connection pool, not the file cache.
2026-06-15 23:43:41 +08:00
YueGuobin 5502b8f8e5 Add timing logs to auth dependency chain to identify 6s pre-handler delay 2026-06-15 23:35:12 +08:00
YueGuobin b3619a3c00 Add granular timing to get_template: separate execute vs fetch time 2026-06-15 23:33:19 +08:00
YueGuobin e90fb0774c Warm up database connection pool on startup to avoid 8s cold-start penalty on first API request 2026-06-15 23:26:28 +08:00
YueGuobin 6d3cfc7d1e Fix: register WAL PRAGMA on sync_engine instead of Engine class for async compat
- Move event listener from module-level @Engine to engine.sync_engine
- Add WAL mode verification log on startup
- Remove unused Engine class-level listener
2026-06-15 23:21:50 +08:00
YueGuobin 08e0dbc122 Add timing logs to get_template to identify DB query bottleneck 2026-06-15 23:18:00 +08:00
YueGuobin fa9aa5a9cc Add timing middleware to log slow requests (>1s) with [CTRL-TIMING] prefix 2026-06-15 23:13:36 +08:00
YueGuobin a855cb34f1 Fix: _time → time in compute.py timing log 2026-06-15 23:06:10 +08:00
YueGuobin ea4bb2c1fb Add [CTRL-TIMING] logs to controller create_node flow
Timing logs cover:
- create_node_from_template (entry, get_template, add_node, total)
- add_node_from_template (entry to done)
- _create_node (project_setup, node.create, total)
- Node.create (compute_post timing)
- compute._session.request (actual HTTP to compute)
2026-06-15 23:04:01 +08:00
YueGuobin e5797c3da0 Fix: pass template_id to batch mode handler so top-level template_id works as default 2026-06-15 22:57:40 +08:00
YueGuobin cef6dc6bd2 Add detailed timing logs to MCP node creation and HTTP client
Logs with [MCP-TIMING] prefix at:
- create_node_handler entry, setup, http_call start/end, total
- http_call entry, auth, response
- _authenticate_v3 entry, done, fail
2026-06-15 22:51:43 +08:00
YueGuobin 58ca6b1dd4 Add fields filter to template_list tool with description for AI 2026-06-15 22:11:14 +08:00
YueGuobin b79fb89866 Fix: add missing _filter_link_response function
Previously the function was referenced in create_link_handler
but never defined, causing NameError at runtime.
2026-06-15 21:51:17 +08:00
YueGuobin 1bb010d274 Pass name parameter through to controller API when creating node from template 2026-06-15 21:50:14 +08:00
YueGuobin eb3ac64016 Add validation to compact link format with clear error messages
- Validate array length (must be exactly 6)
- Validate node_id types at positions 0 and 3
- Return descriptive errors so AI can self-correct
2026-06-15 21:46:27 +08:00
YueGuobin a8a83051ff Add compact array format for link node entries to reduce token usage
Supports both standard and compact formats:
  Standard: [{"node_id": "uuid", "adapter_number": 0, "port_number": 0}]
  Compact:  ["uuid", 0, 0, "uuid", 0, 0] - 3x less tokens
2026-06-15 21:42:26 +08:00
YueGuobin 27a489a3fd Add fields filter to link_create tool
- Default response: link_id, link_type, nodes (3 fields vs full 13)
- Available fields listed in tool description for AI
2026-06-15 13:50:04 +08:00
YueGuobin 3465c39eaa Reduce md5sum cache write failure log level from error to warning
This is a non-fatal issue that occurs during concurrent project close
when multiple nodes try to write their md5sum cache while the project
directory is being cleaned up.
2026-06-15 13:27:14 +08:00
YueGuobin 11bf2d8cef Add fields filter to node_create tool description for AI
- Expose optional 'fields' parameter to AI via tool description
- List all available node fields in the parameter description
- Mention inherited template_id in batch mode
2026-06-15 13:22:32 +08:00
YueGuobin f4c8f9bf61 Optimize MCP create_node: support inherited template_id and default fields filter
- Batch mode now inherits top-level template_id as default per-node
- Both single and batch modes filter response to minimal fields by default
  (node_id, name, node_type, status, console)
- Optional 'fields' parameter overrides the default field set
2026-06-15 13:21:06 +08:00
YueGuobin 6cbe676b18 Increase MCP HTTP client timeout from 10s to 30s
Dynamips node creation (e.g., Cisco 7200 with multiple adapters)
can exceed the previous 10-second timeout, causing MCP tools to
fail with Read timed out errors.
2026-06-15 13:14:11 +08:00
YueGuobin 49d2c271ca Enable SQLite WAL mode to fix 'database is locked' errors under concurrent API requests 2026-06-15 13:02:17 +08:00
YueGuobin 650be3565f Cache IOU image default values per image path to avoid redundant subprocess calls
- Add class-level caches for loader and default RAM/NVRAM values
- When multiple IOU nodes use the same image, only the first one runs
  the ld-linux --verify and iou-image -h subprocesses
- Subsequent nodes reuse cached results, saving ~2 subprocess calls per node
- Cache is populated only on successful subprocess execution to avoid
  overwriting explicitly set ram/nvram values with stale cached defaults
2026-06-15 12:59:51 +08:00
YueGuobin 650f95af54 Increase node and link creation concurrency from 5 to 20 2026-06-15 12:50:59 +08:00
YueGuobin e29e02a8c6 Fix: revert IOU lock optimization, serialize IOU node creation for correct application_id assignment
The _iou_id_lock must cover _create_node() because get_next_application_id()
checks in-memory nodes (self._nodes), which are only registered after
_create_node() completes. Without this serialization, concurrent IOU
node creation produces duplicate application IDs.
2026-06-15 12:49:01 +08:00
YueGuobin 5d0284e7be Performance: accelerate project opening with parallel link creation and batch UDP port allocation
- Narrow IOU lock scope to only cover application_id allocation,
  allowing concurrent IOU node creation via Pool(concurrency=5)
- Parallelize link creation during project.open() using Pool(concurrency=5)
  instead of sequential processing
- Add batch UDP port allocation endpoint on compute to allocate N ports
  in a single HTTP call
- Pre-allocate UDP ports per compute before link creation during project
  loading, reducing HTTP round-trips
- UDPLink.create() falls back to individual port allocation if no
  pre-allocated port is available
2026-06-15 12:42:18 +08:00
Jeremy Grossmann b605d71a8a Merge pull request #2785 from yueguobin/mcp/batch-operations
MCP batch operations, field filtering, Jinja2 templates, download security, and WebSocket fixes
2026-06-15 00:47:17 +02:00
YueGuobin 97483310b1 docs: Update link tool descriptions for batch ops and fields filter 2026-06-14 22:51:06 +08:00
YueGuobin be69670333 feat: Add batch link_ids to link_delete/link_reset, fields filter to link_list 2026-06-14 22:43:44 +08:00
YueGuobin d6c362b3f0 feat: Add fields filter to link_list 2026-06-14 22:37:10 +08:00
YueGuobin 3b42eea112 feat: Add batch node_ids to node_delete 2026-06-14 22:28:28 +08:00
YueGuobin e17d298bc7 fix: Convert http to ws scheme in node_console WebSocket URL 2026-06-14 21:57:58 +08:00
YueGuobin 80d5ae91a4 docs: Update link capture batch ops and console short-lived JWT note 2026-06-14 21:47:25 +08:00
YueGuobin f14d30cb7e feat: Add batch link_ids to link_capture_download 2026-06-14 21:36:24 +08:00
YueGuobin 6df9374a4c feat: Add batch link_ids to link_capture_start/stop 2026-06-14 21:32:47 +08:00
YueGuobin e02f1a8cd0 fix: Store username in gns3_ctx during auth, use for short-lived download JWTs
_ jw t_username_var set in _resolve_token for both JWT and API key auth.
Passed to handlers via gns3_ctx['jwt_username']. No raw key exposure,
no fake-user fallback.
2026-06-14 14:00:41 +08:00
YueGuobin 678b1868f5 fix: Generate independent short-lived JWT for pcap download
No longer depends on the original token type (JWT or API key).
Always creates a fresh 10-min JWT for the download URL.
2026-06-14 13:49:24 +08:00
YueGuobin 754eab9609 docs: Clarify why nodes must be started for device config tools 2026-06-14 13:41:04 +08:00
YueGuobin 9f3a6a7e58 docs: Remove Claude Desktop section from MCP doc 2026-06-14 13:28:04 +08:00
YueGuobin f4b8de725c docs: Add device config workflow mermaid diagram 2026-06-14 13:20:25 +08:00
YueGuobin aabf54b6d1 docs: Add best practices section for device config — template usage, error checking, config backup 2026-06-14 13:11:15 +08:00
YueGuobin 647c5c0e65 docs: Add snapshot prerequisite and suppress telnetlib3 noise 2026-06-14 12:41:14 +08:00
YueGuobin b23f2181c4 docs: Update MCP doc with batch ops, field filtering, and Jinja2 template 2026-06-14 12:33:00 +08:00
YueGuobin c3c78f99a1 revert: Remove _configs_map changes in tools_v2 (handled by template renderer now) 2026-06-14 12:15:30 +08:00
YueGuobin c42b3a59bf fix: Merge commands for duplicate device_names in _configs_map
Dict comprehension overwrote earlier entries when the same device
appeared multiple times, causing all entries' outputs to collapse
into the last one. Now commands are appended for duplicate names.
2026-06-14 02:00:06 +08:00
YueGuobin fb032ade78 fix: Actually pass template param to device_config/command handlers
template was defined in the tool signature but omitted from
the params dict passed to the handler, making Jinja2 rendering
completely non-functional.
2026-06-14 01:40:48 +08:00
YueGuobin fed40c683e fix: Correct Jinja2 template commands_field per tool type
config_tools_nornir expects config_commands, while
display_tools_nornir and vpcs_tools_netmiko expect commands.
Render template now uses the correct field name.
2026-06-14 01:33:23 +08:00
YueGuobin 8aa7f2bde5 feat: Jinja2 template support in device_command_run 2026-06-14 00:27:49 +08:00
YueGuobin 06e1511773 feat: Jinja2 template support in device_config_send
- Add optional 'template' param with Jinja2 syntax
- Each device entry can use 'vars' dict instead of 'config_commands'
- Template rendered per device, merged with existing commands
- Rendering errors returned inline for AI self-correction
2026-06-14 00:27:13 +08:00
YueGuobin c647805cfb feat: Add batch node_ids support to node_start/stop/reload/suspend
- Each tool accepts either node_id (single) or node_ids (batch)
- Batch mode runs actions in parallel via ThreadPoolExecutor
- Useful for starting/stopping nodes by topology region
2026-06-14 00:14:34 +08:00
YueGuobin 3d3c0eb8db feat: Add fields filter to appliance_list
Appliances list can be very large (hundreds of entries). Use
fields=["name","category"] to return only what the AI needs.
2026-06-13 23:59:12 +08:00
YueGuobin cd8bb7cca2 feat: Add fields filter to node_list
- Matches same VALID_NODE_FIELDS as node_get
- Return only selected fields per node to save tokens
2026-06-13 23:54:24 +08:00
YueGuobin b9cc2d8bee feat: node_get fields filter — match controller Node schema fields 2026-06-13 23:26:41 +08:00
YueGuobin 8f8abe4106 fix: Set auto_close=False on project_create so projects stay open when clients disconnect 2026-06-13 23:13:31 +08:00
YueGuobin 2c1a83114d feat: Add batch mode to node_create and link_create (parallel, max 10 workers)
- node_create accepts nodes=[{template_id, x, y, name?}] for batch creation
- link_create accepts links=[{nodes, link_type?, filters?}] for batch creation
- Uses ThreadPoolExecutor for parallel REST API calls
- Max 10 concurrent workers per batch, backward compatible with single mode
2026-06-13 22:29:27 +08:00
Jeremy Grossmann 16d3f28cf5 Merge pull request #2782 from yueguobin/mcp/device-config
MCP (Model Context Protocol) integration — 82 tools with device config, API key auth, and bug fixes
2026-06-11 23:57:08 +02:00
YueGuobin a69485befa docs: Update MCP service documentation with 82 tools and API Key setup 2026-06-12 00:26:56 +08:00
YueGuobin f2e5d69a2c fix: Pass API key directly instead of generating short-lived JWT
REST API auth already supports gns3_ keys, so there's no need
to create a temporary 5-min JWT. The raw API key is passed
through as the Bearer token, eliminating token expiry issues.
2026-06-12 00:25:01 +08:00
Guobin Yue 47fa1bbe95 Merge branch '3.1' into mcp/device-config 2026-06-12 00:05:59 +08:00
YueGuobin a98707e91d chore: Remove redundant migration fixup
f0b0de2a9_add_api_keys_table already includes updated_at column,
so the fixup migration f0b0de2a9b is unnecessary.
2026-06-11 23:51:07 +08:00
YueGuobin 8ecc35193d feat: API key lifecycle — revoke/restore/delete
- POST /{id}/revoke: 吊销, revoked=True, 立即失效
- POST /{id}/restore: 恢复, revoked=False, 重新生效
- DELETE /{id}:       永久删除, 不可逆
- 列表接口显示所有 key 包括已吊销的
- 认证时过滤 revoked=False
2026-06-11 23:32:19 +08:00
YueGuobin 062e8dfbc4 chore: Remove unused revoke_api_key from repository 2026-06-11 23:28:43 +08:00
YueGuobin 700d71d675 fix: Rename revoke_api_key → delete_api_key 2026-06-11 23:28:05 +08:00
YueGuobin 9d441517fd fix: Hard-delete API keys instead of soft delete (revoked flag)
Removing the soft-delete approach — revoked keys are now deleted
from the database entirely via DELETE endpoint. This prevents the
api_keys table from accumulating stale records.
2026-06-11 23:27:43 +08:00
YueGuobin bfef0a36e3 feat: Support API keys in REST API authentication (reuse gns3_ prefix keys)
API keys can now be used in Authorization: Bearer header
for all REST API endpoints, not just MCP.
2026-06-11 23:21:49 +08:00
YueGuobin aed8830052 fix: Lazily access db engine for API key validation
_db_engine is not available when register_starlette_routes() is
called (it's set later during lifespan startup). Store the app
reference instead and access app.state._db_engine lazily.
2026-06-11 23:18:05 +08:00
YueGuobin d75746e029 fix: Add missing updated_at column to api_keys table
BaseTable base class includes updated_at, but the initial
migration didn't create the column. Added fixup migration.
2026-06-11 23:13:50 +08:00
YueGuobin f8340cb355 fix: Export ApiKeyCreate from schemas package 2026-06-11 22:56:44 +08:00
YueGuobin a79bc7dd20 feat: Add API Key support for MCP authentication
- New db model: api_keys table with bcrypt-hashed keys
- New API: POST/GET/DELETE /v3/access/api-keys endpoints
- MCP _resolve_token: validates API keys, resolves to 5-min JWT
- API keys inherit the creating user's RBAC permissions
- MCP auth supports both JWT (24h) and API key (permanent) tokens
2026-06-11 22:54:03 +08:00
YueGuobin 8e9afbcf92 fix: Validate JWT token exp claim — was silently ignored after migration to joserfc
joserfc.jwt.decode() does not validate the exp claim by default,
so expired tokens were accepted indefinitely. Added explicit check
after decoding. See issue #2781.
2026-06-11 22:32:12 +08:00
YueGuobin 0b2c784b81 docs: Fix appliance_install description - it does NOT download images 2026-06-11 12:13:31 +08:00
YueGuobin f0b0de2a91 docs: Add MCP sharing warnings to shared gns3_copilot modules 2026-06-11 12:11:13 +08:00
YueGuobin b69ff17850 docs: Fix image_install description - it auto-creates templates from uploaded images, not downloads 2026-06-11 12:00:36 +08:00
YueGuobin 05f12e1f15 docs: Clarify image_prune description - only removes unreferenced images 2026-06-11 01:42:27 +08:00
Jeremy Grossmann 5df385fabe Merge pull request #2780 from yueguobin/mcp/node-file-streaming
feat: Node file streaming, recursive listing, file type detection, file delete, and MCP file tools
2026-06-10 19:41:30 +02:00
YueGuobin d0960812aa docs: Clarify symbol_delete limitations (built-in vs custom symbols) 2026-06-11 01:35:14 +08:00
YueGuobin c2aae9529b fix: Add image field to template_create, document type-specific params in description 2026-06-11 01:34:34 +08:00
YueGuobin 828a770489 fix: Remove .json() calls on 204 responses for prune/install images 2026-06-11 01:28:19 +08:00
YueGuobin 9d7c482288 fix: Skip always-running nodes in start_all/stop_all
Always-running node types (Ethernet switch, Cloud, NAT, etc.)
return 405 when start/stop is called. is_always_running() already
tracks these types; start_all/stop_all now skip them.
2026-06-11 01:26:43 +08:00
YueGuobin 3db275b199 docs: Add GNS3 SVG rendering quirks to drawing_create description 2026-06-11 00:48:54 +08:00
YueGuobin 4b9572d565 docs: Add SVG shape examples to drawing_create tool description 2026-06-11 00:39:37 +08:00
YueGuobin 4266583922 fix: Add missing rotation parameter to drawing_update MCP tool
Parameter was defined in create_drawing but omitted from
update_drawing, causing rotation changes to be silently ignored.
2026-06-11 00:36:37 +08:00
YueGuobin 66975f54f7 fix: Map MCP device_command_run parameter to tool's expected field name
MCP sent 'device_commands' but the internal display_tools_nornir
expects 'device_configs'. Renamed parameter for consistency.
2026-06-11 00:14:00 +08:00
YueGuobin 571853599f docs: Update link_reset description with accurate UDP connection behavior 2026-06-10 23:59:45 +08:00
YueGuobin e7aa228ead fix: Update link_reset description to match actual behavior (delete + recreate) 2026-06-10 23:56:30 +08:00
YueGuobin 4ec80f8989 fix: Remove unsupported description param from project_create
GNS3 REST API ProjectCreate schema does not have a description field.
The parameter was silently ignored; now removed to avoid confusion.
2026-06-10 23:32:27 +08:00
YueGuobin 80e64ebbae fix: Fix symbol_get/upload/delete handlers for correct API paths
- get_symbol: URL was missing '/raw' suffix + tried .json() on binary SVG
  → now returns download URL + curl command (like download_capture_file)
- upload_symbol: URL was missing '/raw' suffix
  → now accepts SVG content string and POSTs to correct path
- delete_symbol: returns 204 No Content, .json() would fail
  → removed .json() call (just returns message)
2026-06-10 23:01:27 +08:00
YueGuobin d431ff6eaa feat: Log registered MCP tools at startup 2026-06-10 22:53:51 +08:00
YueGuobin 658a8d112b docs: Clarify compute tools descriptions about local compute vs database computes
compute_list and compute_get only return database-registered computes.
The built-in local compute is returned by server_statistics instead.
2026-06-10 22:44:54 +08:00
YueGuobin 8b014693a8 fix: Type compute_id as uuid.UUID to reject non-UUID values at MCP input layer
Previously compute_id was typed as str, so 'local' would pass MCP validation
and reach the controller API where it crashed. Now uuid.UUID type ensures
Pydantic rejects any non-UUID string before the handler runs.
2026-06-10 22:38:38 +08:00
YueGuobin 67fa65a9e0 fix: Require UUID for compute_get/images, remove 'local' string default
The end /v3/computes/{compute_id} expects the compute_id to be a
valid UUID. Previously the MCP tool defaulted to the string 'local',
which caused a ValueError in the database layer. Now compute_id is
required and callers must use compute_list first to resolve names to UUIDs.
2026-06-10 22:31:17 +08:00
YueGuobin e7038f1ae3 feat: Add device configuration MCP tools (config_send, command_run, vpcs_config_set)
- Add jwt_token/url parameters to get_device_ports_from_topology() and GNS3TopologyTool
  for MCP handler compatibility (backward compatible, auto-detection fallback)
- Add jwt_token/url pass-through to ExecuteMultipleDeviceConfigCommands,
  ExecuteMultipleDeviceCommands, and VPCSCommands _run() methods
- Create MCP handler device_config.py wrapping the 3 device config tools
- Register as device_config_send, device_command_run, vpcs_config_set
2026-06-10 14:34:35 +08:00
YueGuobin 87e14cc49e refactor: unify MCP tool naming to <module>_<action> convention
All 79 tools renamed for consistent alphabetical grouping:
- project_list, project_get, project_create, project_delete ...
- node_list, node_get, node_start, node_stop, node_file_list ...
- link_list, link_create, link_capture_start, link_capture_stop ...
- template_list, template_get ...
- snapshot_list, snapshot_create, snapshot_delete ...
- drawing_list, drawing_create, drawing_update ...
- symbol_list, symbol_get, symbol_upload ...
- appliance_list, appliance_get ...
- image_list, image_get, image_delete ...
- server_version, server_statistics ...
- compute_list, compute_get, compute_images ...
2026-06-10 14:07:34 +08:00
YueGuobin 41594faf43 feat: Add symbol upload/delete, project load, and locked check MCP tools 2026-06-10 13:59:39 +08:00
YueGuobin b786f0b7eb feat: Add image management MCP tools
- Image tools: get_images, get_image, delete_image, prune_images, install_images
2026-06-10 13:58:42 +08:00
YueGuobin e4d2faec37 feat: Add symbol and appliance MCP tools
- Symbol tools: get_symbols, get_symbol, get_symbol_dimensions, get_default_symbols
- Appliance tools: get_appliances, get_appliance, install_appliance
2026-06-10 13:57:52 +08:00
YueGuobin db9f645c98 feat: Add node bulk ops, project lock, and server info MCP tools
- Node bulk: start_all_nodes, stop_all_nodes, suspend_all_nodes, reload_all_nodes
- Node advanced: duplicate_node, isolate_node, unisolate_node, get_node_links
- Project: lock_project, unlock_project
- Server: get_version, get_statistics
2026-06-10 13:56:43 +08:00
YueGuobin e7fbb3ec10 feat: Add snapshot and drawing MCP tools
- Add snapshot tools: get_snapshots, create_snapshot, delete_snapshot, restore_snapshot
- Add drawing tools: get_drawings, create_drawing, get_drawing, update_drawing, delete_drawing
2026-06-10 13:55:13 +08:00
YueGuobin acce79d243 refactor: unify MCP handlers to use http_call directly, relocate node file ops to Node class
- Move node file operations (list_files, delete_file) from Gns3Connector to Node class
- Add link capture/reset operations (reset, start_capture, stop_capture) to Link class
- Convert all MCP handlers to use conn.http_call() directly instead of
  Gns3Connector/Node/Link abstraction methods
- Register 4 new MCP tools: reset_link, start_capture, stop_capture,
  download_capture_file
2026-06-10 13:47:10 +08:00
YueGuobin 28b06f37c4 feat: Add node file operations as MCP tools (list, get, write, delete)
- Add list_node_files, get_node_file, write_node_file, delete_node_file methods to Gns3Connector
- Add MCP handlers with offset/limit line-based reading for get_node_file
- Auto-truncate files >50KB with truncated flag in response
- Rich metadata returned (total_lines, total_bytes, has_more, etc.)
- Tool docstrings guide AI to check file sizes before reading chunked
2026-06-10 12:16:35 +08:00
YueGuobin 066b7076c0 Add comment about rootful Docker permissions at container start
Rootful Docker recreates volume mount points as root on start,
preventing the GNS3 process from writing files into node directories
while the container is running. self._fix_permissions() would resolve
this but is currently only called at container stop time.
2026-06-10 00:46:10 +08:00
YueGuobin ce95c89b64 Fix _fix_permissions test: set process.returncode=0 and update assertion
- Set returncode=0 on mock process to follow the success path
- Update assertion to include stderr=asyncio.subprocess.PIPE parameter
2026-06-10 00:44:12 +08:00
YueGuobin eb15c7138b Fix list_node_files PermissionError on os.scandir
Wrap os.scandir() in try-except to return empty list instead of
crashing when a node directory is not readable.
2026-06-10 00:15:26 +08:00
YueGuobin 1a307edbca Fix _fix_permissions error handling and list_node_files PermissionError
- _fix_permissions: capture stderr, check returncode, only set
  _permissions_fixed on success instead of silently marking as fixed
- list_node_files: wrap os.scandir in try-except to handle
  PermissionError gracefully
2026-06-10 00:14:40 +08:00
YueGuobin e659b64bf0 Add async_iterable_to_stream utility to avoid aiohttp compatibility issues
Create async_iterable_to_stream() in gns3server.utils.asyncio that
converts an async iterable to an aiohttp StreamReader via a background
feeder task. This bypasses aiohttp's AsyncIterablePayload which can
cause 'Connection reset by peer' with certain HTTP servers.

Use it in _run_http_query for the __aiter__ data path.
2026-06-09 23:55:59 +08:00
YueGuobin 8e340f0ce8 Add descriptive detail to 403 errors in compute file endpoints
Both is_safe_path rejection and PermissionError were returning 403
without a detail message, making them indistinguishable in logs.
Add specific detail strings to each:
- is_safe_path: 'Path is outside the project directory'
- PermissionError (write): 'Permission denied writing to ...'
- PermissionError (delete): 'Permission denied deleting ...'
2026-06-09 23:33:35 +08:00
YueGuobin 16a9064eb8 Fix silent file write failure in write_compute_project_file
The inner try-except caught OSError/UnicodeEncodeError with 'pass',
silently swallowing all write failures and returning HTTP 204 as if
the file was written successfully.

Remove the nested try-except and let errors propagate properly:
- OSError → 500 with error detail
- PermissionError → 403 (already handled)
- FileNotFoundError → 404 (already handled)
2026-06-09 23:27:45 +08:00
YueGuobin cbb21e8e40 feat: Node file streaming, recursive listing, file type detection, and file delete
- Stream file GET/POST through controller without buffering in memory
- Add recursive and subdirectory filtering to node file listing
- Replace file extension with magic-based file type detection
- Add DELETE endpoint for node and project files
- Include directories in listing response
- Add params and stream support to http_query
- Fix lambda closures, streamer exception scope, and delete error codes
2026-06-09 22:52:50 +08:00
Jeremy Grossmann c70a4660c1 Merge pull request #2777 from yueguobin/mcp/project-tools
Add MCP project tools: update, duplicate, and README operations
2026-06-07 18:11:16 +02:00
YueGuobin f91d7b18e9 Update README tool descriptions: .txt → .md
Only description strings changed to tell AI the content is Markdown
format (.md). The actual handler still reads/writes README.txt for
Web UI compatibility.
2026-06-07 23:32:26 +08:00
YueGuobin 4ca1a80adb Update README tool descriptions to mention Markdown format
Web UI supports rendering Markdown in README.txt, so note this in the
tool descriptions and content parameter.
2026-06-07 23:23:38 +08:00
YueGuobin 4921b98948 Add MCP project tools: update, duplicate, and README operations
- Add update_project and duplicate_project MCP tools
- Add get_project_readme and update_project_readme tools
- Add Gns3Connector methods: update_project, duplicate_project,
  get_project_file, write_project_file
2026-06-07 23:14:07 +08:00
grossmj 95b1a35300 Development on 3.1.0.dev4 2026-06-07 00:13:59 +02:00
grossmj aad66e8ede Release v3.1.0a3 2026-06-06 23:19:24 +02:00
grossmj 259b7824af Bundle web-ui v3.1.0a3 2026-06-06 23:11:04 +02:00
grossmj bd22911066 Sync appliances 2026-06-06 22:53:12 +02:00
Jeremy Grossmann c6a6cd9ad9 Merge pull request #2776 from yueguobin/fix/mcp-template-update-kwargs-handling
Fix MCP tool parameter handling for nested kwargs
2026-06-06 18:52:12 +02:00
Guobin Yue fbe6b280d5 Merge branch '3.1' into fix/mcp-template-update-kwargs-handling 2026-06-07 00:37:02 +08:00
YueGuobin 012df5af57 docs: add simplified MCP tool description guide to project memory
Added a focused, concise memory document about MCP tool description
location. Simplified from 180 lines to 45 lines to capture only the
essential information needed for future conversations.

Key points documented:
- MCP tool descriptions are defined in @mcp.tool() functions in __init__.py
- NOT in *_TOOLS arrays in individual module files
- Must restart GNS3 server to see description updates
- Description requirements: explicit formats, ranges, and examples

This replaces the verbose 180-line version with a practical 45-line
guide focused on answering: 'Where should I write tool descriptions so
the AI can see them?'
2026-06-07 00:32:39 +08:00
YueGuobin 1f985a2823 fix: update MCP link tools descriptions with detailed filter info
Updated the actual MCP tool definitions in __init__.py with detailed
filter parameter descriptions. The previous update to links.py was incorrect
because MCP tools are defined via @mcp.tool() decorators in __init__.py.

Changes:
- Updated update_link tool docstring with comprehensive filter information
- Updated create_link tool filters parameter description
- Added all 5 filter types with proper array format requirements
- Added parameter ranges and usage examples

Filters now properly documented:
- frequency_drop: [N] (N: -1 to 32767)
- packet_loss: [rate] (rate: 0 to 100)
- delay: [ms, jitter] (milliseconds)
- corrupt: [rate] (rate: 0 to 100)
- bpf: [expression] (BPF syntax)

This will prevent TypeError: 'int' object is not iterable errors and
help users understand the correct filter format.
2026-06-07 00:20:29 +08:00
YueGuobin eac6c9f17d docs: improve MCP link tools filter parameter descriptions
Updated the filters parameter description in create_link and update_link
MCP tools to specify the required array format and provide examples.

Changes:
- Updated create_link filters description with array format requirements
- Updated update_link filters description with array format and example
- Added specific filter types: frequency_drop, packet_loss, delay, corrupt, bpf

This helps users understand the correct format:
- frequency_drop: [N] (drop every Nth packet)
- packet_loss: [rate] (packet loss percentage)
- delay: [ms, jitter] (latency and jitter in milliseconds)
- corrupt: [rate] (packet corruption percentage)
- bpf: [expression] (Berkeley Packet Filter)

Prevents TypeError: 'int' object is not iterable errors.
2026-06-06 23:46:22 +08:00
YueGuobin 3ba11c8bff fix: correct MCP nodes and links tool parameter handling for nested kwargs
Extended the kwargs parameter handling fix to nodes and links MCP tools,
which had the same nested kwargs structure issue as templates.

Changes:
- Modified update_node_handler to extract params from nested kwargs
- Modified update_link_handler to extract params from nested kwargs

This ensures that node and link updates through MCP tools work correctly,
allowing proper modification of node and link properties.
2026-06-06 23:31:32 +08:00
YueGuobin 0d22d275fb fix: correct MCP template tool parameter handling for nested kwargs
Fixed an issue where MCP template tools (update_template, create_template)
were not correctly handling nested kwargs parameter structure from MCP clients.

The problem occurred when MCP clients passed parameters in the format:
  {'template_id': 'xxx', 'kwargs': {'adapters': 3}}

The original code was passing the entire kwargs dictionary as a parameter,
instead of extracting the actual update parameters from within it.

Changes:
- Modified update_template_handler to extract params from nested kwargs
- Modified create_template_handler to handle the same issue

This fix ensures that template updates through MCP tools now work correctly,
allowing proper modification of template properties like adapters count.
2026-06-06 23:26:21 +08:00
Jeremy Grossmann 93293a327d Merge pull request #2775 from yueguobin/fix/mcp-initialization-ready-check
Feature: add MCP server initialization check with ready state tracking
2026-06-06 16:56:18 +02:00
Guobin Yue 0a97e85a21 Merge branch '3.1' into fix/mcp-initialization-ready-check 2026-06-06 22:48:36 +08:00
Jeremy Grossmann 8e50d3f31b Merge pull request #2774 from cristian-ciobanu/3.1
fix(templates): Database error detected when saving a template with a disk image change
2026-06-06 16:40:52 +02:00
YueGuobin 3958279b8a feat: add client information logging to MCP connection rejection
Add reusable utility for extracting client information from ASGI scope:
- Create gns3server/utils/request_utils.py with extract_client_info()
- Extract client IP, port, path, method, and authenticated username
- Format comprehensive log messages with client context

Benefits:
- Better observability for connection rejection events
- Reusable utility for other modules
- Consistent client logging format across codebase
- Helps diagnose timing issues during server startup

Usage:
  from gns3server.utils.request_utils import extract_client_info
  client_info = extract_client_info(scope, auth_service)
  log.warning(f"Connection rejected - Client: {client_info['host']}:{client_info['port']} ({client_info['user_info']})")
2026-06-06 22:35:43 +08:00
YueGuobin 9e5b575d1c refactor: replace MCP ready state polling with asyncio.Event
Replace the global variable + lock + polling mechanism with asyncio.Event
pattern for MCP server ready state tracking.

Benefits:
- Eliminates race conditions (Event.set() is thread-safe)
- Event-driven notification instead of polling (no 50x sleep overhead)
- Fixes test contamination from global state
- Reduces code from 54 lines to 30 lines
- Uses standard asyncio primitive for this pattern
2026-06-06 22:24:10 +08:00
Cristi 03b3078271 fix: revert duplicate image check to fix failing tests 2026-06-06 15:04:06 +03:00
Cristi 1e40fa1b70 fix (templates): Add ordering to handle the duplicate cases gracefully 2026-06-06 14:46:27 +03:00
Cristi da3a34144b fix(templates): Database error detected when saving a template with a disk image change 2026-06-06 14:21:15 +03:00
YueGuobin 67a960fbce fix: return 503 error on MCP server ready timeout
Instead of allowing connections to proceed when server initialization
times out, return HTTP 503 Service Unavailable to prevent MCP
protocol initialization errors.

This prevents the original "Received request before initialization
was complete" errors when GNS3 server startup takes longer than
the 5-second timeout.
2026-06-06 15:23:52 +08:00
YueGuobin b1514edf88 fix: add MCP server ready check to prevent initialization errors
Add server ready state tracking for MCP service to prevent
"Received request before initialization was complete" errors
when clients connect before GNS3 server completes startup.

Changes:
- Add MCP server ready state management with wait/notify mechanism
- Modify auth wrapper to wait for server initialization before accepting connections
- Set MCP server ready flag after GNS3 startup completes

This ensures MCP protocol initialization handshake only occurs
after GNS3 server is fully initialized, preventing race
conditions during server startup.
2026-06-06 15:21:17 +08:00
Jeremy Grossmann 13d1563762 Merge pull request #2772 from yueguobin/feature/mcp-transport-security-config
feat: add configurable MCP transport security settings via gns3_server.conf
2026-06-05 19:05:23 +02:00
YueGuobin 1dcbd9199f docs: document MCP server URL host resolution behavior
Add documentation explaining how _server_url() resolves the host
when Server.host is 0.0.0.0 or :: — using the default route
interface IP instead of hardcoding 127.0.0.1.
2026-06-06 00:52:21 +08:00
YueGuobin db9772ca7a fix: resolve MCP server URL host via default route IP when bound to 0.0.0.0
When GNS3 server is configured to listen on 0.0.0.0 (all interfaces),
_server_url() was hardcoding 127.0.0.1, making the WebSocket console
URL unreachable from remote MCP clients.

Use the UDP connect trick (connect to 8.8.8.8:80 without sending data)
to discover the default route interface IP, which is the address remote
clients can actually reach.
2026-06-06 00:48:22 +08:00
YueGuobin 4d87b57f18 docs: add MCP transport security documentation
Added Transport Security section to MCP service documentation covering:
- Default behaviour (disabled, allow all hosts)
- How to enable protection via gns3_server.conf
- Protection mechanism (Host header validation)
- DNS rebinding attack prevention explanation
- Behaviour summary table
2026-06-05 23:40:01 +08:00
YueGuobin 0e6db9a7b6 fix: correct MCP transport security config to actually allow all hosts by default
The MCP library's TransportSecurityMiddleware only supports exact host
matches or "host:*" port wildcards. It does NOT support a standalone "*"
wildcard to mean "allow all hosts" — setting allowed_hosts=["*"] would
reject every connection because no Host header equals "*".

Worse, when transport_security=None was passed to FastMCP while its default
host is "127.0.0.1", FastMCP would auto-enable protection with strict
localhost-only rules, overriding GNS3's intent to allow all hosts.

Root cause analysis:
- FastMCP auto-enables DNS rebinding protection when host is localhost
  and no explicit TransportSecuritySettings is provided
- GNS3 was passing transport_security=None (indirectly via FastMCP's default)
  when protection was disabled, triggering the auto-enable
- The TransportSecuritySettings "allowed_hosts" list does NOT support "*"
  as a catch-all wildcard

This fix:
1. Always pass an explicit TransportSecuritySettings to FastMCP
   - Disabled: TransportSecuritySettings(enable_dns_rebinding_protection=False)
   - Enabled: TransportSecuritySettings(enable_dns_rebinding_protection=True, ...)
2. Restore mcp_allowed_hosts and mcp_allowed_origins config fields
3. Set mcp_enable_dns_rebinding_protection default to False (allow all hosts)

Behaviour:
- Default (no config change): all hosts can connect to MCP server
- With mcp_enable_dns_rebinding_protection=true: only configured hosts
- Aligns with GNS3 server's 0.0.0.0 binding policy
2026-06-05 23:20:57 +08:00
YueGuobin bb39238f02 feat: add configurable MCP transport security settings via gns3_server.conf
Add MCP transport security configuration to gns3_server.conf with permissive
defaults that align with GNS3's design philosophy and VM distribution requirements.

## Changes

### 1. Configuration Schema (gns3server/schemas/config.py)
- Added MCP transport security fields to ServerSettings class:
  - mcp_enable_dns_rebinding_protection (bool, default: True)
  - mcp_allowed_hosts (list[str], default: ["*"])
  - mcp_allowed_origins (list[str], default: ["*"])
- Added field validators to handle comma-separated string input

### 2. MCP Server Initialization (gns3server/api/routes/mcp/__init__.py)
- Import TransportSecuritySettings from mcp.server.transport_security
- Added _create_mcp_server() function to read configuration
- Updated FastMCP instantiation to use configured security settings

### 3. Configuration Sample (gns3server/config_samples/gns3_server.conf)
- Added MCP transport security settings section
- Documented default behavior and security options
- Provided examples for different use cases

## Design Philosophy

**Default: Allow All Hosts** (matches GNS3's 0.0.0.0 binding):
- VM distribution works out-of-the-box
- Users can access from any network location
- Security-conscious users can restrict when needed

**Security: Optional Restriction**:
Users can configure specific hosts for enhanced security:
``ini
mcp_allowed_hosts = 127.0.0.1:*,localhost:*,192.168.1.3:*
mcp_allowed_origins = http://127.0.0.1:*,http://localhost:*,http://192.168.1.3:*
```

## Benefits

- Flexible: Users can configure based on security requirements
- User-friendly: Default matches GNS3's 0.0.0.0 binding philosophy
- Maintainable: No code changes needed for different deployment scenarios
- Secure: DNS rebinding protection remains enabled with configurable hosts

## Related

- Issue #2771
- FastMCP DNS rebinding protection design
- Existing skills configuration in ServerSettings
2026-06-05 23:06:06 +08:00
Jeremy Grossmann 68f63597eb Merge pull request #2769 from yueguobin/fix/load-feature-skills
fix: add load_feature_skills() to properly load network planning features
2026-06-05 16:58:06 +02:00
YueGuobin e76f3970ca fix: add load_feature_skills() to properly load network planning features
The feature directory contains network planning and design functionalities
(e.g., topology_planner), not device-specific features. These were not being
loaded because load_device_skills() only scanned the device directory.

Changes:
- Added new load_feature_skills() method in SkillsLoader
- Modified reload_skills() to load both device and feature directories
- Device skills: device-specific configurations (e.g., VPCS)
- Feature skills: network planning functionalities (e.g., topology planner)
- Both are now properly loaded into SKILLS_REGISTRY

This ensures that network planning features like topology_planner are available
via the device_skills tool with proper category classification.
2026-06-05 22:25:17 +08:00
Jeremy Grossmann 6497db6f30 Merge pull request #2768 from yueguobin/feature/mcp-service
feat: MCP (Model Context Protocol) service with SSE transport
2026-06-05 10:25:58 +02:00
YueGuobin 24953c1712 refactor: convert all MCP tool parameter descriptions to Annotated+Field
- Replace Args: docstring blocks with Annotated[str, Field(description=...)]
  so parameter descriptions appear in inputSchema.properties.*.description
- mcp.server.fastmcp does not parse Args: blocks from docstrings;
  only Annotated with pydantic Field injects descriptions into the
  structured JSON Schema visible to AI clients via tools/list
- Remove redundant Args: blocks from docstrings (info moved to Field)
- Restore full 4-step websocat workflow in get_node_console_info docstring
  with connection, command sending, response receiving, and timeout
2026-06-05 14:02:58 +08:00
YueGuobin 62db5caf6a fix: remove platformdirs upper bound to resolve fastmcp-slim dependency conflict
- Remove platformdirs<3 constraint (was for Debian packaging only)
- fastmcp-slim>=3.4.0 requires platformdirs>=4.0.0
- Debian packaging is no longer a concern
2026-06-05 13:38:25 +08:00
YueGuobin 4c44a32db7 docs: update get_node_console_info description with websocat connection workflow 2026-06-05 13:17:23 +08:00
YueGuobin b415ae0af6 fix: add missing fastmcp dependency to resolve CI test failures
- Add fastmcp>=3.4.0 to requirements.txt
- Fixes ModuleNotFoundError: No module named 'mcp'
- Resolves CI test failures in compute routes tests
2026-06-05 12:32:27 +08:00
YueGuobin a056fa3450 refactor: move all imports to top of __init__.py 2026-06-05 01:16:17 +08:00
YueGuobin f1078a97e2 test: add /v3/mcp/ to allowed public endpoints 2026-06-05 01:07:20 +08:00
YueGuobin f9f7ba7a53 docs: update memory record to 30 tools 2026-06-05 00:20:34 +08:00
YueGuobin 25dc1e9009 docs: update MCP feature doc to cover all 30 tools and console WS 2026-06-05 00:20:13 +08:00
YueGuobin f34de4c075 fix: remove console_host/port from get_node_console_info
Return only ws_url + websocat command to avoid LLM misinterpreting
direct telnet connection.
2026-06-04 23:53:03 +08:00
YueGuobin 8775583b83 feat: add get_node_console_info tool
Returns console type, host, port and a suggested command (e.g. telnet,
vncviewer) for connecting to a node's console. Total: 30 tools.
2026-06-04 23:45:34 +08:00
YueGuobin 0c572f2e9c docs: add MCP service design memory record 2026-06-04 23:25:22 +08:00
YueGuobin e416ef8d5e feat: add 3 Compute MCP tools
- Add list_computes, get_compute, get_compute_images
- Total MCP tools: 29
2026-06-04 23:22:14 +08:00
YueGuobin 6889f51737 feat: add 5 Template MCP tools
Add list_templates, get_template, create_template, update_template,
delete_template. Total MCP tools: 26.
2026-06-04 23:21:01 +08:00
YueGuobin cad216705a feat: add Node and Link MCP tools, update copyright
- Add 9 node tools and 5 link tools
- Update copyright year to 2026, add author
2026-06-04 23:19:00 +08:00
YueGuobin 5b23ac81d0 docs: add MCP service feature documentation 2026-06-04 23:09:44 +08:00
YueGuobin ccdd307d54 docs: add MCP service feature documentation 2026-06-04 23:09:08 +08:00
YueGuobin 1e9b3d5879 feat: complete MCP SSE transport with JWT auth
- SSE endpoint at /v3/mcp/transport/sse with token auth
- Supports Authorization: Bearer header and ?token= query param
- JWT validated via GNS3 auth_service, stored in contextvars
- Tool handlers use GNS3 REST API via Gns3Connector with JWT token
- 7 project tools: list_projects, get_project, create_project,
  delete_project, open_project, close_project, get_project_stats
- Claude Code: claude mcp add --transport sse ... -H 'Authorization: Bearer <jwt>'
- Claude Desktop: SSE URL with ?token=<jwt>
2026-06-04 23:06:20 +08:00
YueGuobin 19e7533cd7 feat: support Authorization header and query param for MCP token
- SSE endpoint supports both Authorization: Bearer header and ?token= query param
- Claude Code can use headers (no URL exposure)
- Claude Desktop (EventSource) can use ?token= URL param
2026-06-04 22:38:49 +08:00
YueGuobin 55b3a7d622 feat: implement standard MCP protocol with SSE transport
- Use FastMCP (Anthropic MCP SDK) for tool registration and SSE transport
- Mount SSE app under /v3/mcp/transport with JWT token authentication
- Token passed via ?token=<jwt> query parameter on SSE connection
- Token validated against GNS3 auth_service and stored in contextvars
- Tool handlers create Gns3Connector with JWT token to call GNS3 REST API
- 7 project tools: list_projects, get_project, create_project, delete_project,
  open_project, close_project, get_project_stats
- Unauthenticated SSE connections return 401
2026-06-04 22:19:43 +08:00
YueGuobin 7086db4226 feat: add MCP (Model Context Protocol) service with project tools
- Add MCPTool/MCPToolRegistry system for centralized tool registration
- Add 7 project-related MCP tools: list_projects, get_project, create_project,
  delete_project, open_project, close_project, get_project_stats
- Tools use Gns3Connector (custom_gns3fy) to call GNS3 REST API via HTTP loopback,
  keeping the MCP layer decoupled from controller internals
- Handlers run in thread pool via asyncio.to_thread() to avoid blocking
  the event loop on synchronous requests calls
- Unified POST /v3/mcp/execute endpoint with JWT authentication
2026-06-04 13:53:05 +08:00
Jeremy Grossmann 6c7e8fb370 Merge pull request #2767 from yueguobin/fix/docker-container-status-detection
Add running project check for fast duplication
2026-06-03 18:44:22 +02:00
YueGuobin fb1a5a3385 Add project memory: Docker container stop delay analysis 2026-06-03 22:47:48 +08:00
YueGuobin e8e1930e0b Remove extra blank line from merge 2026-06-03 22:22:03 +08:00
YueGuobin 4adaba8e8c Revert container state detection in create()
The _get_container_state() call in create() has no practical effect:
Docker's POST /containers/create only creates the container without
starting it, so a newly created container can never be in 'running'
or 'paused' state.
2026-06-03 22:20:34 +08:00
YueGuobin 2adc09604c Fix Docker VM tests for container status detection on node creation
The create() method now calls _get_container_state() after creating the
container to detect if the container is already running. Mock this method
in all test_create_* tests so the new code path doesn't fail with
KeyError: 'State' when the Docker.query mock response lacks a State field.
2026-06-03 22:17:43 +08:00
Guobin Yue 9712e9328a Merge branch '3.1' into fix/docker-container-status-detection 2026-06-03 21:49:56 +08:00
Jeremy Grossmann 305f1cad21 Merge pull request #2766 from yueguobin/fix/web-ui-update-script-remote-check
Fix web-ui update script to handle custom GitHub URL changes
2026-06-03 09:30:01 +02:00
YueGuobin 7ab04a0f9a Add running project check for fast duplication 2026-06-03 12:20:26 +08:00
YueGuobin b6e1f84740 Move running project check before fast duplication
Move the is_running() check from _fast_duplication() to
duplicate() to avoid the error message being wrapped by
the except Exception handler. This ensures the error
message is clean and prevents wasted fast duplication
attempts on running projects.
2026-06-03 12:18:28 +08:00
YueGuobin c4440d882d Add running project check for fast duplication
Add is_running() check at the beginning of _fast_duplication()
to prevent duplicating a project while nodes are running.
Previously, only the export/import fallback path had this check,
which meant running nodes were not detected when fast duplication
succeeded. This aligns with the duplicate API behavior and
provides a consistent safeguard against data inconsistencies.
2026-06-03 12:15:27 +08:00
YueGuobin 45b5f8d7e2 Fix Docker container status detection on node creation
When a Docker container is created (e.g., when loading a project), the node
status should reflect the actual container state. Previously, the node status
was always set to 'stopped' even if the container was already running.

This fix checks the container state after creation and updates the node
status accordingly:
- If container is running: status = 'started'
- If container is paused: status = 'suspended'
- If container is exited: status = 'stopped' (default)

This ensures that project.is_running() correctly detects running Docker
containers when attempting to export/duplicate a project, fixing the issue
where running Docker nodes were not detected and prompted for shutdown.
2026-06-03 11:51:55 +08:00
YueGuobin d4fb45f862 Fix web-ui update script to handle custom GitHub URL changes
When using --url parameter with a different GitHub repository, the script
now checks if the existing clone's remote matches the provided URL. If they
don't match, it removes the old clone and re-clones from the new URL.

This prevents errors when switching between different forks or branches
of the gns3-web-ui repository.
2026-06-03 11:25:00 +08:00
Jeremy Grossmann 1d7feaf994 Merge pull request #2765 from GNS3/upgrade-dependencies
Upgrade dependencies
2026-06-02 18:11:51 +02:00
grossmj 5f3978c57c Upgrade pytest-asyncio to v1.4.0 2026-06-02 18:07:41 +02:00
grossmj 18e20f0918 Upgrade dependencies 2026-06-02 17:48:16 +02:00
Jeremy Grossmann 5ab05fb688 Merge pull request #2762 from yueguobin/fix/docker-container-rename
Fix unnecessary Docker container recreation when renaming a project
2026-06-02 09:34:11 +02:00
YueGuobin 0ba180ad1d Fix unnecessary Docker container recreation when renaming a project
When renaming a project that has running Docker containers, the containers
were unnecessarily stopped, removed, and recreated, even though the project
name change doesn't affect container configuration.

Root cause:
- Client sends complete project object including variables: [] during rename
- Controller unconditionally notified all computes about the update
- Docker nodes rebuild containers on any project update notification

Solution:
- Only notify compute nodes when variables field has actual content
- Treat None and [] as semantically equivalent (no variables)
- Empty variables don't affect running containers, so no need to update

Impact:
- Project rename operations no longer trigger ~7 second container rebuilds
- Only actual variable changes trigger container recreation
- Fixes issue #2760
2026-06-02 13:04:44 +08:00
grossmj 06b02981df Development on 3.1.0.dev3 2026-06-02 00:04:50 +02:00
Jeremy Grossmann 1f98d18ae7 Merge pull request #2761 from yueguobin/fix/project-rename-filename-sync
Fix project rename and duplicate issues (#2759)
2026-06-01 23:59:24 +02:00
YueGuobin b5d3556add Fix project rename and duplicate issues
Fixes #2759

When renaming a project:
- Update self._filename to match the new project name
- Rename the .gns3 file on disk to keep it in sync
- Add error handling for file rename failures

When duplicating a project:
- Use self._filename (actual filename) instead of self.name
- This handles the case where a project has been renamed
- Prevents 'No such file or directory' errors

The root cause was that project renaming only updated the project name
in memory and in the .gns3 file content, but did not update the actual
.gns3 filename. This caused duplicate operations to fail because they
tried to read a file with the new name that didn't exist.
2026-06-01 22:19:44 +08:00
Jeremy Grossmann 12347b24aa Merge branch '3.1' into base-configs-3.0 2026-06-01 11:39:40 +02:00
Jeremy Grossmann ed6273413e Merge pull request #2758 from yueguobin/verify-resource-pool-deletion
Fix: properly delete resource records when deleting resource pools
2026-06-01 10:55:26 +02:00
YueGuobin f3aaa902c6 Fix double deletion issue in remove_resource_from_pool
Remove duplicate delete_resource call from remove_resource_from_pool since
the API layer already handles resource deletion. This prevents conflicts
where the API layer tries to delete a resource that was already deleted by
the repository layer.

The complete fix is now:
- remove_resource_from_pool: Only removes resource from pool (API handles deletion)
- delete_resource_pool: Deletes all resource records before deleting pool
2026-06-01 12:46:15 +08:00
YueGuobin d2848a600e Complete fix: delete resource records when deleting resource pool
This completes the fix from PR #2315 by ensuring that when a resource
pool is deleted, all associated resource records are also deleted from
the resources table, preventing orphaned resource records.

Changes:
- Modified delete_resource_pool() to first delete all resource records
  in the pool before deleting the pool itself
- This complements the existing fix in remove_resource_from_pool() which
  handles single resource removal
2026-06-01 12:43:56 +08:00
YueGuobin fcb76a4b01 Apply fix from PR #2315: delete resource from resource table when removing from pool 2026-06-01 12:30:03 +08:00
grossmj bc15e3331b Add privilege dependencies for the new API endpoints 2026-05-31 19:26:47 +02:00
grossmj 87f5a4560a Remove test config files 2026-05-31 19:26:12 +02:00
Jeremy Grossmann db605120fa Merge pull request #2741 from Volo6uev/issue-2735/api-base-config
api endpoint implemented
2026-05-31 19:10:55 +02:00
grossmj 18eb3d9c48 Remove deprecated 'PermissionsStartOnly' setting for Systemd service. Ref #1830 2026-05-31 19:06:03 +02:00
Jeremy Grossmann b5536d61a1 Merge pull request #2756 from yueguobin/fix/ghost-docker-node-vnc-timeout
Fix Docker node management issues and optimize variable update performance
2026-05-31 18:59:00 +02:00
YueGuobin b4daddd1c7 Optimize project loading by implementing parallel node creation
This change significantly improves project loading performance, especially for
topologies with multiple Docker containers or other node types.

Changes:
- Modified project.open() method to use parallel node creation
- Replaced serial node creation loop with Pool-based parallel processing
- Set concurrency limit to 5 to avoid overwhelming the system
- Maintains backward compatibility with existing functionality

Performance improvements:
- Projects with 6 Docker containers: 60-70% faster loading time
- Reduced from ~4-5 seconds to ~1-2 seconds for typical multi-node topologies
- Better resource utilization through concurrent node creation

Technical details:
- Uses existing Pool utility class (concurrency=5)
- Preserves node creation order where required
- Maintains error handling and rollback capabilities
- No changes to node creation logic itself, only parallelization

Testing:
- Syntax validation passed
- Compatible with existing project.open tests
- No API changes, internal optimization only
2026-05-31 23:50:21 +08:00
YueGuobin 0f1003b4f6 Merge branch 'feature/packet-filter-validation' into fix/ghost-docker-node-vnc-timeout 2026-05-31 23:30:10 +08:00
YueGuobin ced73574b4 Fix delay filter validation: ensure delay: [0, X] returns proper error message
This fix addresses the issue where delay: [0, X] configurations were being
silently dropped instead of returning validation errors.

Changes:
- Created new utility function filter_inactive_filters() in packet_filter_validation.py
- Implemented smart filtering logic for delay filter that checks both latency and jitter:
  * delay: [0, 0] → User wants to disable delay, filter out silently
  * delay: [0, X] where X > 0 → Invalid config, keep for validation error
  * delay: [X, X] where X > 0 → Normal configuration, validate normally
- Simplified link.py update_filters() method to use the new utility function
- Added comprehensive tests for the new filtering logic

Before this fix:
- delay: [0, 100] would be silently dropped with no error message
- Users wouldn't know their configuration was invalid

After this fix:
- delay: [0, 100] returns proper error: "delay parameter Latency must be between 1 and 32767 ms, got: 0"
- delay: [0, 0] is correctly handled as intentional disable
- Normal delay configurations continue to work as expected
2026-05-31 22:35:58 +08:00
Guobin Yue 75445886fe Merge branch '3.1' into fix/ghost-docker-node-vnc-timeout 2026-05-31 22:18:28 +08:00
Jeremy Grossmann cc416f20d5 Merge pull request #2754 from yueguobin/feature/packet-filter-validation
Add packet filter parameter validation to prevent ubridge errors
2026-05-30 23:01:17 +02:00
YueGuobin 92a0fa6cd7 Fix packet filter validation tests: use correct ubridge filter type names 2026-05-31 00:38:03 +08:00
YueGuobin 5733705615 Update default web-ui branch from master-3.0 to 3.1 2026-05-31 00:25:09 +08:00
YueGuobin 2584d17067 Update tests to match show_interface_labels default change
The default value of show_interface_labels has been changed to True.
Update test expectations to match this new default.
2026-05-31 00:09:33 +08:00
YueGuobin 21bba7f4b2 Set default value of show_interface_labels to True
Change the default value of show_interface_labels from False to True for better user experience, as interface labels are commonly used in network topology visualization.
2026-05-30 22:49:42 +08:00
YueGuobin 598029face Optimize project variable updates to use parallel node processing
Performance improvement for project variable updates when multiple containers
are present. Previously, nodes were updated serially in a for loop, causing:
- 5 containers: ~35 seconds (7s per container)
- 10 containers: ~70 seconds
- 20 containers: ~140 seconds (2min 20sec)

Changed to parallel processing using asyncio.gather(), reducing total time
to the duration of the slowest single node update (~7 seconds regardless
of container count).

The change maintains error handling with return_exceptions=True to ensure
one node's update failure doesn't prevent others from completing.

This is particularly important for users with large topologies containing
many Docker containers that need to be recreated when project variables change.

Related to issue #2755 ghost node timeout fix.
2026-05-30 22:24:34 +08:00
YueGuobin 8c1dbdf079 Fix ghost Docker nodes causing 60-second VNC timeout on variable updates
When a Docker node is deleted, the compute node's DELETE endpoint only calls
node.delete() which removes the working directory but does not remove the node
object from the project's self._nodes collection. This causes ghost nodes to
remain in memory.

When project variables are updated, the code iterates through ALL nodes in
memory and calls update() on them. For ghost nodes with VNC configuration, this
triggers VNC startup attempts, resulting in 60-second timeouts waiting for X11
socket files that don't exist.

The fix adds await node.project.remove_node(node) to ensure the node object is
removed from the project's node collection when deleted, matching the behavior
of other node types that use manager.delete_node() which already calls
project.remove_node().

This resolves the issue where updating project variables after deleting a VNC
Docker container would timeout with: 'x11 socket file "/tmp/.X11-unix/X100"
does not exist'

Fixes issue #2755
2026-05-30 22:15:40 +08:00
YueGuobin c2dd480edd Fix Docker container variable compatibility with Pydantic models
When updating project variables while Docker containers are running, the
system now properly handles both dictionary-format variables and Pydantic
Variable objects. This prevents AttributeError when containers are recreated
after variable updates.

Changes:
- Modified DockerVM.create() to detect and handle Pydantic Variable objects
- Updated _format_env() method to support both variable formats
- Maintains backward compatibility with existing dictionary format

Fixes error: AttributeError: 'Variable' object has no attribute 'get'
2026-05-30 13:26:36 +08:00
YueGuobin 0aa0467b02 Fix delay latency minimum: ubridge rejects latency <= 0
Align validation rules with ubridge source: delay latency must be > 0
(packet_filter.c delay_setup line 182). Update FILTERS definition in
link.py and test cases accordingly.
2026-05-30 01:21:00 +08:00
Jeremy Grossmann 458eff6020 Merge branch '3.1' into feature/packet-filter-validation 2026-05-29 19:16:59 +02:00
YueGuobin 7bcb96a368 Improve packet filter validation: use tcpdump, handle multi-line BPF, safe project load
Changes:
- Replace tshark BPF validation with tcpdump -d (calls pcap_compile
  internally like ubridge, returns instantly without waiting for traffic)
- Support multi-line BPF expressions: split on newlines and validate
  each line individually
- Always validate, never save invalid filters on error
- Drop invalid filters during project load with warning (prevents
  old topologies with bad filters from failing to open)
- Simplify test cases (no longer depend on tshark availability)
2026-05-30 01:02:21 +08:00
Jeremy Grossmann 7d718e66c5 Merge pull request #2749 from yueguobin/feature/update-gns3-skills-repo
chore: update GNS3 skills repository to official organization
2026-05-29 17:57:14 +02:00
YueGuobin 87f38b1507 Add packet filter parameter validation to prevent ubridge errors
This commit implements comprehensive parameter validation for GNS3 packet
filters at the API layer, preventing invalid parameters from reaching
ubridge and causing technical errors.

## Changes

### New Features
- **Add packet filter validation module** (`gns3server/utils/packet_filter_validation.py`)
  - Validate parameter ranges for all filter types:
    - frequency_drop: -1 to 32767
    - packet_loss: 0 to 100%
    - delay: 0 to 32767ms (latency + jitter)
    - corrupt: 0 to 100%
  - **BPF syntax validation** using tshark (method from gns3_copilot)
  - Parameter count and type validation
  - User-friendly error messages with parameter details

### Integration
- **Integrate validation into Link.update_filters()** (`gns3server/controller/link.py`)
  - Validate parameters before applying filters
  - Raise ControllerError with clear error messages for invalid input
  - Prevent invalid parameters from reaching ubridge

### Testing
- **Add comprehensive test suite** (`tests/utils/test_packet_filter_validation.py`)
  - 15 test cases covering all validation scenarios
  - Tests for valid/invalid parameters, edge cases, and BPF syntax
  - All tests passing

## Benefits
- **Better UX**: Clear, actionable error messages instead of ubridge technical errors
- **Prevents crashes**: Validate parameters at API layer before reaching ubridge
- **Consistent validation**: Centralized validation logic for all filter types
- **BPF safety**: Syntax validation using tshark prevents invalid BPF expressions
2026-05-29 23:39:44 +08:00
YueGuobin 64471575f7 chore: update GNS3 skills repository to official organization
Update the default download repository address for GNS3 skills from
yueguobin/GNS3-Skills to gns3/gns3-skills to use the official
organization repository.

This affects:
- Default skills_repo_url in server configuration schema
- Skills manager default repository URL
- Skills configuration defaults
- All documentation references
2026-05-29 22:04:48 +08:00
Jeremy Grossmann 5d12c26836 Merge pull request #2750 from yueguobin/feature/simple-user-isolation
feat: implement simple user isolation based on project ownership
2026-05-29 11:50:13 +02:00
YueGuobin 06e31fad16 docs: update RBAC user isolation design doc to match actual implementation
Align the design document with the actual codebase:
- Correct pseudo-code to use get_accessible_project_ids() batch API
- Add super admin bypass section
- Clarify seen_project_ids is now a lightweight dedup set
- Add get_accessible_project_ids() implementation details
- Update base branch reference and add MEMORY.md index entry
2026-05-29 00:56:23 +08:00
YueGuobin 1a9a16552b docs: update skills repo URL in command-security.md 2026-05-28 14:22:04 +08:00
YueGuobin 5c8523f321 docs: remove Chinese overview docs, keep only English versions 2026-05-28 14:18:57 +08:00
YueGuobin 305eb58b1c docs: add overview docs for packet analysis, fault injection, and AI assistant 2026-05-28 14:18:24 +08:00
YueGuobin 27eca929cf feat: add mermaid-to-SVG conversion script with environment setup 2026-05-28 14:05:58 +08:00
YueGuobin 67c6f3c970 test: fix privilege count assertions after adding LLMConfig privileges
The LLMConfig.Audit and LLMConfig.Modify privileges added to the User
role increased the default privilege count from 25 to 27. Update test
assertions to match the new counts.
2026-05-28 01:24:22 +08:00
YueGuobin c66867e5ff docs: add user node limit roadmap 2026-05-28 00:02:52 +08:00
YueGuobin 9825a7b271 test: fix RBAC test to match implementation logic
- Update test_list_projects_in_resource_pool to use independent AsyncClient
- Fix test logic to match actual RBAC implementation:
  - Direct ACE + created_by match = accessible
  - Resource pool ACE = accessible (no created_by check)
  - Deduplication prevents duplicate projects
- Test now verifies: pool-only access → pool + /projects access → /projects-only access
- Add missing imports for ASGIWebSocketTransport and auth_service
2026-05-28 00:02:25 +08:00
Jeremy Grossmann 7dcb49fe45 Merge branch '3.1' into base-configs-3.0 2026-05-27 17:43:04 +02:00
YueGuobin adba0f17e5 test: add --prefix and --cleanup-only parameters to benchmark script
- Add --prefix parameter to specify custom project name prefix for testing/cleanup
- Add --cleanup-only parameter to clean up test projects without running benchmark
- Add timing statistics to cleanup showing total time and throughput
- Simplify cleanup output to single summary line instead of per-project printing
2026-05-27 23:37:34 +08:00
YueGuobin 2efcc619b1 perf: batch RBAC permission checking for GET /projects
Replace per-project check_user_has_privilege calls with a single
batch method that performs 3 fixed DB queries regardless of project
count. Reduces GET /projects response time for 10000 projects from
~12s to ~290ms (40x improvement).
2026-05-27 13:27:35 +08:00
YueGuobin 0022d1ad62 test: add benchmark script for GET /projects performance testing 2026-05-27 12:53:42 +08:00
YueGuobin 1eca8c5c94 fix: prevent duplicate projects when user projects are in resource pools
Fixed a bug where projects created by a user that are also in a resource pool
the user has access to would appear twice in the GET /projects response.

Changes:
- Add seen_project_ids set to track already added projects
- Check for duplicates before adding projects in Step 2 (user projects)
- Check for duplicates before adding projects in Step 3 (resource pool projects)

This ensures each project appears only once regardless of whether it's user-created
or shared via resource pool.
2026-05-27 10:47:36 +08:00
YueGuobin ca5db7567c fix: check both regular ACEs and resource pool ACEs for proper access control
This fix addresses a critical issue in the RBAC permission checking logic
introduced in PR #2750. When a project is shared through a resource pool,
both the project creator (with regular ACEs like "All endpoints") and the
shared user (with resource pool ACEs) should be able to see the project.

Changes:
- Modified `check_user_has_privilege` in `gns3server/db/repositories/rbac.py`
- Changed from if-elif (exclusive) to sequential (inclusive) checking
- Now checks regular ACEs first, then resource pool ACEs
- Both types of ACEs can grant access (OR logic instead of XOR)

This fixes the scenario where:
1. user100 has "All endpoints" ACE and creates a project
2. user100 shares the project via resource pool with user200
3. Both users should see the project (user100 via regular ACE, user200 via pool ACE)

Related to PR #2750 - RBAC user isolation implementation.
2026-05-27 00:17:52 +08:00
YueGuobin e6581dfe68 test: update RBAC test to use test_user.username for user isolation
Updated test_list_projects_in_resource_pool to:
- Add test_user fixture parameter
- Use test_user.username when creating projects to match created_by filtering
- Switch from authorized_client to client (admin user) for consistency

This aligns the test with the new user isolation architecture where
projects are filtered by created_by == current_user.username.

Note: This test currently fails with 401 due to test infrastructure
JWT authentication issues, but the core RBAC functionality is verified
by TestPrivileges (13 tests passing).
2026-05-26 23:46:28 +08:00
YueGuobin 5e5c464347 docs: add Phase 9 and 10 user self-registration and email service 2026-05-26 22:59:30 +08:00
YueGuobin 7bfb39dcc9 docs: add Phase 8 per-user project namespace to roadmap 2026-05-26 22:58:18 +08:00
YueGuobin a11a95330b docs: add Phase 7 resource pool renaming to roadmap 2026-05-26 22:56:25 +08:00
YueGuobin 2062136d12 feat: add alembic migration for LLMConfig privileges
Add migration to insert LLMConfig.Audit, LLMConfig.Modify, LLMConfig.Allocate
privileges into existing databases and associate LLMConfig.Audit/Modify
with the default User role.
2026-05-26 14:28:54 +08:00
YueGuobin 9fca2181a8 feat: add independent LLMConfig permissions for AI profile management
Add new privilege definitions:
- LLMConfig.Audit - View LLM model configurations
- LLMConfig.Modify - Update LLM model configurations
- LLMConfig.Allocate - Create/delete LLM model configurations

Add LLMConfig.Audit and LLMConfig.Modify to default User role so that
regular users can manage their own AI profiles without needing the
User.Manager role.

User-scoped LLM config endpoints now use LLMConfig.* permissions.
Group-scoped LLM config endpoints retain Group.* permissions.
2026-05-26 14:24:41 +08:00
YueGuobin 7e33772983 docs: add Phase 6 frontend permission query API to roadmap 2026-05-26 14:08:01 +08:00
YueGuobin 90d2263d2d docs: add Phase 5 ACE architecture refactoring plan to roadmap
Add future plan for ACE architecture refactoring to support:
- ACE name and description fields for better management
- Multiple paths and resource pools in a single ACE entry
- Solving ACE explosion problem (N groups × M pools)
- Updated permission checking logic for the new structure
2026-05-26 13:59:28 +08:00
YueGuobin f7a69bd546 feat: remove resource pools from 'all endpoints' list
Remove resource pools from the ACE endpoints list to prevent accidental
access through the 'all endpoints' option. Resource pools must be
explicitly configured for team sharing to maintain clear security
boundaries and prevent unintended exposure of shared projects.

This change aligns the UI behavior with the actual permission checking
logic where 'path: /' does not grant resource pool access.
2026-05-26 13:31:04 +08:00
YueGuobin 5e4d9e057e refactor: add efficient get_aces_for_path method for resource pool checks
Add a new repository method get_aces_for_path() that:
- Queries ACEs for a specific path at database level (more efficient)
- Preloads related user, group, and role objects to prevent 500 errors
- Keeps original get_aces() method unchanged to avoid performance impact

This improves both performance and code clarity for resource pool
deletion safety checks.
2026-05-26 12:59:37 +08:00
YueGuobin 6fcbb8e57b feat: prevent deletion of resource pools used by ACE configurations
Add safety check to prevent deletion of resource pools that are being
used by ACE configurations. If an attempt is made to delete a resource
pool that has ACE rules referencing it, the API returns a 400 error
with detailed information showing which users/groups are using the
pool and their roles.

The error message only shows the resource pool name for a clean,
user-friendly experience without exposing internal path details.
2026-05-26 12:54:48 +08:00
YueGuobin 9e5423f3f1 docs: update RBAC user isolation roadmap and add design memory
Updated the roadmap document to reflect the implemented three-step
permission check logic in the feature/simple-user-isolation branch.

Added comprehensive design memory documenting:
- Core problem and design conflicts
- Three-step permission check implementation
- Key design decisions and advantages
- Use cases and scenarios
- Design evolution process
2026-05-26 12:46:36 +08:00
YueGuobin ee36a9aec2 feat: fix permission check logic to properly handle ACE and user isolation
Implement the correct three-step permission check logic:
- Step 1: ACE check - basic access permission (get projects user has ACE for)
- Step 2: Filter ace_projects by created_by - user's own projects (project sharing only through resource pools)
- Step 3: Resource pool projects (projects shared through resource pools)

This fixes the design flaw where:
- ACE check could bypass user isolation with broad ACE configurations
- seen_project_ids mechanism prevented proper layered checking
- Project sharing was confused with direct ACE configuration

The new logic ensures:
- User isolation works even with broad ACE (path='/', propagate=True)
- Project sharing is only available through resource pools (clear design)
- Proper layered checking without seen blocking mechanism
2026-05-26 12:33:08 +08:00
Volobue b5e44b5403 added mock user in api endpoints tests 2026-05-25 22:17:14 +03:00
Volobue 01511f3aef added endpoint to v3 branch 2026-05-25 21:37:04 +03:00
Volobue b74ebafb8f api endpoint implemented 2026-05-25 20:07:05 +03:00
YueGuobin 265d0ff860 feat: implement layered permission checks for proper user isolation and sharing
Implement a three-layer permission system:
- Layer 1: ACE strategy check (explicitly authorized/shared projects)
- Layer 2: Ownership check (user's own projects based on created_by)
- Layer 3: Resource pools (team shared projects)

This approach:
- Resolves the conflict between ACE and user isolation
- Enables project sharing via ACE (other users can grant access)
- Maintains default user isolation via ownership
- Prevents duplicate projects in results
- Preserves resource pool functionality
2026-05-26 00:31:19 +08:00
Jeremy Grossmann 0e2045d4e8 Merge branch '3.1' into feature/simple-user-isolation 2026-05-25 18:23:59 +02:00
grossmj ee0e52b930 Enable GA tests on branch 3.1 2026-05-25 18:23:40 +02:00
YueGuobin 7c0b465b74 feat: implement simple user isolation based on project ownership
Users can only see projects they created (created_by field).
Super admins see all projects.
Resource pool projects continue to work as before.
2026-05-26 00:16:45 +08:00
grossmj e5636b0186 Update files to reflect 3.1 branch name 2026-05-25 18:10:23 +02:00
grossmj b3ef07e523 Update .whitesource 2026-05-25 18:07:50 +02:00
Jeremy Grossmann 0fdeb34f1a Merge pull request #2747 from yueguobin/feature/rbac-user-isolation-roadmap
docs: add RBAC user isolation roadmap
2026-05-25 04:40:04 +08:00
YueGuobinandClaude Opus 4.7 9e989566f5 docs: add RBAC user isolation roadmap
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 00:40:42 +08:00
Jeremy Grossmann ec55b9f089 Merge pull request #2746 from yueguobin/feature/gns3-copilot-packet-filter
feat: add packet filter management tool for GNS3-Copilot fault injection
2026-05-25 00:00:09 +08:00
YueGuobin ee80df9e9c docs: add AIOps fault injection testing pipeline roadmap
Roadmap for duplicating company network architecture into GNS3 and
building an automated pipeline to inject faults and validate AIOps
diagnosis, with traffic injection support for enhanced realism.
2026-05-24 23:25:01 +08:00
YueGuobin 045db5bdd8 fix: clean BPF syntax error message and specify loopback interface
- Add -i lo to tshark command to avoid "(null)" interface in errors
- Strip "for interface" suffix from error message for cleaner output
2026-05-24 23:00:41 +08:00
YueGuobin ce58e9adc8 feat: add BPF syntax validation using tshark
- Add _validate_bpf_syntax() method to validate BPF expressions
- Use tshark with 1-second timeout for syntax checking
- Check for "Invalid" in output to detect syntax errors
- Validate BPF filters before applying them to links
- Handle tshark not installed scenario gracefully
- Support both single and multiple BPF expressions
- Return detailed error messages for syntax validation failures
2026-05-24 22:49:59 +08:00
YueGuobin e71931d096 feat: add show_filters_icon parameter to packet filter tool
- Add show_filters_icon parameter with default value False
- Pass show_filters_icon to link.update() in set and clear operations
- Update tool description to explain default behavior
- Remove "clear" action from description to simplify interface
- Hide filter icon in GNS3 Web UI by default for cleaner UI during fault injection
2026-05-24 22:46:52 +08:00
YueGuobinandClaude Sonnet 4.6 dc04c29e7e docs: add GNS3 appliance loading mechanism to memory
Document how GNS3 loads appliance files from builtin and custom directories with priority rules, including storage locations and the design rationale that allows users to customize devices without losing changes during registry updates.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-24 22:31:28 +08:00
YueGuobin 8ba1f064d2 feat: add packet filter management tool for GNS3-Copilot fault injection
Add comprehensive packet filter management functionality to GNS3-Copilot,
enabling AI-powered fault injection scenarios with network simulation
capabilities like latency, packet loss, and corruption.

## Changes

### New Features
- **GNS3PacketFilterTool**: New LangChain tool for managing packet filters
  on GNS3 links with support for delay, packet loss, corruption,
  frequency_drop, and BPF filtering
  - Actions: get_available, set, get, clear
  - Integrated into troubleshooting_injection mode for fault scenarios

### API Integration
- **Link.available_filters()**: Added method to custom_gns3fy.py Link class
  - Queries available filter types for specific links
  - API v3+ only (raises ValueError for v2 connectors)
  - Returns filter definitions with parameters and constraints

### Tool Integration
- Added GNS3PacketFilterTool to TROUBLESHOOTING_INJECTION_MODE_TOOLS
- Positioned as 3rd tool in fault injection workflow
- Optimized for troubleshooting practice scenarios

## Files Modified
- gns3server/agent/gns3_copilot/agent/gns3_copilot.py
- gns3server/agent/gns3_copilot/gns3_client/custom_gns3fy.py
- gns3server/agent/gns3_copilot/tools_v2/__init__.py

## Files Added
- gns3server/agent/gns3_copilot/tools_v2/gns3_packet_filter.py

## Testing
- All validation tests passed
- Version checking verified (v3+ only)
- Tool integration confirmed in troubleshooting mode
2026-05-24 22:31:28 +08:00
Jeremy Grossmann 279ea9eed3 Merge pull request #2745 from yueguobin/feat/link-show-filters-icon
feat: add show_filters_icon property to Link for controlling Web UI filter icon display
2026-05-24 22:05:04 +08:00
YueGuobin fca4459e5f fix: update test_json expected output to include show_filters_icon field 2026-05-24 20:48:58 +08:00
YueGuobin 31d8046e30 fix: add getattr fallback to show_filters_icon property for backward compatibility 2026-05-23 17:09:50 +08:00
YueGuobin a1f4942746 feat: add show_filters_icon property to Link for controlling Web UI filter icon display
This commit adds a new  property to the Link class, allowing users to control whether filter icons are displayed in the Web UI at the individual link level.

**Changes:**
- Added  attribute to Link class (default: True)
- Added  property getter
- Added  method for updating the property
- Updated  to include the new field with backward compatibility
- Added  field to LinkBase schema using Optional[bool] = Field(True, ...) pattern
- Updated API routes to handle the new field in create and update operations
- Added loading logic for show_filters_icon in project.open() to preserve settings when reopening projects

**Schema Definition:**
Uses the same pattern as the  field:

**API Impact:**
- POST /v3/projects/{project_id}/links - accepts  in request body
- PUT /v3/projects/{project_id}/links/{link_id} - can update
- GET /v3/projects/{project_id}/links/{link_id} - returns  field

**Future Applications:**
This feature provides granular control for future AI fault injection modules to manage link-level protocol failures while maintaining clean UI presentation.
2026-05-23 16:57:39 +08:00
YueGuobin 367c565d7c fix: ensure show_filters_icon is always returned in API responses
Set default value for show_filters_icon in LinkBase schema to ensure the field is always included in API responses, even when response_model_exclude_unset=True is used.
2026-05-23 15:21:44 +08:00
YueGuobin aa6845b32b feat: add show_filters_icon property to Link for controlling Web UI filter icon display
This commit adds a new `show_filters_icon` property to the Link class, allowing users to control whether filter icons are displayed in the Web UI at the individual link level.

**Changes:**
- Added `_show_filters_icon` attribute to Link class (default: True)
- Added `show_filters_icon` property getter
- Added `update_show_filters_icon()` method for updating the property
- Updated `asdict()` to include the new field in both topology and regular dumps
- Added `show_filters_icon` field to LinkBase schema
- Updated API routes to handle the new field in create and update operations

**API Impact:**
- POST /v3/projects/{project_id}/links - accepts `show_filters_icon` in request body
- PUT /v3/projects/{project_id}/links/{link_id} - can update `show_filters_icon`
- GET /v3/projects/{project_id}/links/{link_id} - returns `show_filters_icon` field

**Future Applications:**
This feature provides granular control for future AI fault injection modules to manage link-level protocol failures while maintaining clean UI presentation.
2026-05-23 14:57:58 +08:00
Jeremy Grossmann 8719e12c59 Merge pull request #2742 from yueguobin/fix/paramiko-dependency-conflict
fix: remove explicit paramiko pin to resolve dependency conflict with netmiko
2026-05-19 19:32:18 +08:00
YueGuobinandClaude Opus 4.7 cd737cc5ed fix: remove explicit paramiko pin to resolve dependency conflict with netmiko
paramiko>=5.0.0 conflicts with netmiko 4.7.0 which requires paramiko<5.0.
Remove the explicit pin; paramiko is pulled in transitively via netmiko.
To fix CVE-2026-44405, upgrade paramiko separately:
  pip install "paramiko>=5.0.0"

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-19 08:52:19 +08:00
Volobue 9106ac4522 api endpoint implemented 2026-05-18 21:49:01 +03:00
Jeremy Grossmann ca2004e715 Merge pull request #2738 from yueguobin/docs/skills-editor-api
Documentation improvements and Skills Editor API roadmap
2026-05-19 01:13:20 +08:00
Jeremy Grossmann b65ce034e5 Merge pull request #2739 from yueguobin/fix/unclosed-client-session-websocket-proxy
fix: close DockerHTTPClient session to prevent UnixConnector leak in Web Wireshark
2026-05-19 01:12:22 +08:00
Jeremy Grossmann 529144901f Merge pull request #2740 from yueguobin/fix/cve-2026-44405-paramiko
fix: update netmiko to 4.7.0 and pin paramiko>=5.0.0 to fix CVE-2026-44405
2026-05-19 00:59:03 +08:00
Guobin Yue 0e45d62c06 Merge branch '3.0' into fix/cve-2026-44405-paramiko 2026-05-19 00:08:38 +08:00
YueGuobinandClaude Opus 4.7 6e7cab3fec fix: update netmiko to 4.7.0 and pin paramiko>=5.0.0 to fix CVE-2026-44405
CVE-2026-44405 allows SHA-1 signature algorithm in paramiko through
4.0.0. netmiko 4.7.0 still permits paramiko<5.0, so pip resolves to
the vulnerable 4.0.0. Explicitly pinning paramiko>=5.0.0 ensures the
fix commit a448945 is included.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-19 00:07:18 +08:00
Guobin Yue 11fe3a24cf Merge branch '3.0' into fix/unclosed-client-session-websocket-proxy 2026-05-19 00:02:26 +08:00
YueGuobin 868e317d24 fix: close DockerHTTPClient session to prevent UnixConnector leak in Web Wireshark
The Web Wireshark WebSocket endpoint created a WebWiresharkManager but
never called close(), leaving the DockerHTTPClient's ClientSession with
UnixConnector unclosed when users closed the browser tab.

Also switch asyncio.wait in WebSocket proxy from ALL_COMPLETED to
FIRST_COMPLETED to avoid blocking cleanup when one direction disconnects.
2026-05-18 23:59:56 +08:00
Guobin Yue 7444551a66 Merge branch '3.0' into docs/skills-editor-api 2026-05-18 23:42:31 +08:00
YueGuobinandClaude Sonnet 4.6 ef16c87a02 docs: add Web Wireshark Docker image optimization analysis
Add comprehensive roadmap document for optimizing the gns3/web-wireshark
Docker image size from current 2GB to target 1.5GB (25% reduction).

Key findings:
- 570MB of cleanable files identified (locales, docs, ibus, dev packages)
- Compression analysis shows limited benefit (1-5%) vs file cleanup (25%)
- Detailed implementation plan with safe optimization phases
- Focus on transfer optimization and build-time compression strategies

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 22:58:06 +08:00
grossmj ba79f1c226 Fix Docker build and publish API doc actions 2026-05-13 17:19:08 +08:00
grossmj 7cf25bc0d5 Release v3.1.0a2 2026-05-13 00:34:05 +08:00
grossmj 7bb0a2a863 Bundle web-ui v3.1.0a2 2026-05-13 00:29:25 +08:00
grossmj 748b8a1571 Sync appliances 2026-05-13 00:25:27 +08:00
YueGuobinandClaude Opus 4.7 c5a59acc40 docs: move packet analysis from roadmap to implemented
Rewrite the packet-analysis document based on actual code:
protocol-oriented analysis with tshark, 40+ supported protocols,
two-tool architecture (skills query + capture analysis), field
validation, and hot reload support.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 22:59:34 +08:00
YueGuobin fd08885d94 docs: reorganize roadmap files into dedicated subdirectory 2026-05-12 22:53:35 +08:00
YueGuobinandClaude Opus 4.7 0ee370d6ab docs: add Skills Editor API roadmap
Document the planned CRUD + PR management API for GNS3 skills
(prompts, fault injection, device skills, packet analysis),
enabling the Web UI to browse, edit, save, and contribute
skill changes back to the upstream repository.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 22:52:23 +08:00
Jeremy Grossmann c71dfbb6b5 Merge pull request #2729 from yueguobin/feature/troubleshooting-issue-injection
feat: add fault injection system and external skills repository
2026-05-12 22:24:00 +08:00
YueGuobin 30a140a9e3 fix: surface tshark stderr errors to LLM, not just log them 2026-05-12 15:10:27 +08:00
YueGuobin b79fea918d fix: hint LLM about -c behavior when tshark returns empty with -c 2026-05-12 14:54:50 +08:00
YueGuobin 09b0fbcfea fix: update search_fields examples to use existing field names 2026-05-12 14:48:37 +08:00
YueGuobin 73e0de7706 fix: reject multi-keyword search_fields, single keyword only 2026-05-12 14:47:00 +08:00
YueGuobin c357d52fbf fix: document action/query params in PacketAnalysisTool description 2026-05-12 14:36:21 +08:00
YueGuobin 2a34d0ef58 feat: add search_fields action to PacketAnalysisTool for real-time field lookup 2026-05-12 14:33:13 +08:00
YueGuobin 32c3b4cd35 feat: validate tshark -e field names before running analysis 2026-05-12 14:27:16 +08:00
YueGuobin 9c23f9037c fix: document -c behavior caveat in PacketAnalysisTool description 2026-05-12 14:00:58 +08:00
YueGuobinandClaude Opus 4.7 b7a0b611df fix: demote loader-level load logs to DEBUG, info already reported by manager
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 13:52:05 +08:00
YueGuobinandClaude Opus 4.7 89da58fe1b fix: promote skills/prompts load logs to INFO level
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 13:46:36 +08:00
YueGuobin c5af9ce389 fix: load packet analysis protocols during skills initialization
- Add reload_packet_analysis_protocols() call in _ensure_skills_manager()
- PACKET_ANALYSIS_REGISTRY was empty because it was never loaded
2026-05-12 13:41:52 +08:00
YueGuobin 67b3b778b8 feat: add PacketAnalysisSkillsTool
- Register packet_analysis_skills as a LangChain tool for LLM
- LLM can query protocol field definitions before calling packet_analysis
- Follows the same pattern as DeviceSkillsTool and InjectionSkillsTool
2026-05-12 13:37:59 +08:00
YueGuobin 2a38816d3d fix: remove remaining quoted example from tool input docstring 2026-05-12 13:30:18 +08:00
YueGuobin b991125c6c fix: use shlex.split for tshark_args parsing
- Replace str.split() with shlex.split() to handle quoted arguments
- LLM may pass args with or without quotes, both work correctly now
2026-05-12 13:29:31 +08:00
YueGuobin e0c2d64b7c feat: add packet analysis workflow to tool description
- Update PacketAnalysisTool.description with step-by-step workflow
- Instruct LLM to query packet_analysis skills first, then construct tshark_args
2026-05-12 13:13:24 +08:00
YueGuobin 2c5cfdbee8 refactor: remove old PacketCaptureTool, replaced by PacketAnalysisTool
- Delete packet_capture_tools.py (analyze_packets with packet_number)
- PacketAnalysisTool is more flexible and can do everything the old tool could
- Update tool registration to use only PacketAnalysisTool
- Cleaner, more unified packet analysis interface
2026-05-12 10:19:25 +08:00
YueGuobin 84b9742045 feat: add protocol-oriented packet analysis tool
- Add PacketAnalysisTool that accepts user-provided tshark arguments
- LLM constructs tshark commands based on protocol knowledge from skills
- Add PACKET_ANALYSIS_REGISTRY for protocol definitions
- Add load_packet_analysis_protocols() to SkillsLoader
- Add get_packet_analysis_protocol() and list functions to registry
- Register PacketAnalysisTool in teaching and lab automation modes
- Update SkillsManager to reload packet analysis protocols

Related: GNS3-Skills commit 7bc45d2
2026-05-12 09:59:10 +08:00
YueGuobin 6596011799 docs: add roadmap docs for planned features
Close GitHub issues and document as roadmaps instead:
- #2731: user preferences API
- #2732: server settings REST API
- #2733: injection fault tracking
- packet analysis: protocol-oriented analysis architecture
2026-05-12 00:03:07 +08:00
YueGuobin a408148073 feat: require superadmin for /reload/skills endpoint
Only administrators can reload skills from the external repository.
Non-admin users receive 403 Forbidden.
2026-05-11 22:59:28 +08:00
YueGuobinandClaude Opus 4.7 ae1a444677 fix: remove background init timeout and allow skills reload retry
- Remove 5s thread.join() timeout so git clone/pull is not truncated
- Remove _init_complete flag, reset _init_in_progress on failure instead
- Let /reload/skills API retry initialization after network failure
- Raise GIT_HTTP_LOW_SPEED_LIMIT from 1 KB/s to 10 KB/s

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-11 22:51:57 +08:00
YueGuobinandClaude Sonnet 4.6 27051a89ad deps: add PyYAML and GitPython to ai-requirements.txt
Add missing dependencies for the external skills repository feature:
- PyYAML>=6.0.0: for loading YAML skill files
- GitPython>=3.1.0: for Git operations (clone/pull) on skills repo

These dependencies are required by the SkillsManager and SkillsLoader
classes introduced in commit e68cc17e.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 15:59:52 +08:00
YueGuobin e68cc17ed5 feat: add fault injection system and external skills repository
## Summary

Add a complete fault injection system for GNS3 Copilot, migrate all
skills from local Python files to an external Git repository with
hot reload support, and restructure Copilot API under /copilot/.

## Key Changes

### Fault Injection
- New troubleshooting_injection mode with InjectionSkillsTool
- 368 fault scenarios across 39 protocol categories
- Context-based filtering (LLM must pass topology protocols)

### External Skills Repository
- SkillsManager: Git clone/pull, version tracking, smart updates
- SkillsLoader: YAML skills + Markdown prompts from external repo
- Hot reload via POST /copilot/reload/skills
- Configurable via gns3_server.conf

### Architecture
- API unified under /copilot/ prefix
- SkillsManager moved from Controller to agent module
- Lazy initialization with startup background preload
- Per-command Git timeout, smart update checks
- Forbidden commands hot-reloadable from external repo
- 32 INFO logs downgraded to DEBUG
2026-05-11 01:24:55 +08:00
Jeremy Grossmann 534bb5eded Merge pull request #2728 from yueguobin/feature/disable-deepseek-thinking-mode
fix: Disable DeepSeek thinking mode to avoid reasoning_content errors
2026-05-09 22:41:27 +08:00
YueGuobinandClaude Sonnet 4.6 156072118f fix: Disable DeepSeek thinking mode to avoid reasoning_content errors
DeepSeek models (deepseek-v4-flash/pro) enable thinking mode by default,
which returns reasoning_content that must be passed back to the API in
subsequent requests. This causes 400 errors in multi-turn conversations
when the reasoning_content is not properly handled.

This commit disables thinking mode by passing extra_body={"thinking": {"type": "disabled"}}
as an explicit parameter to DeepSeek models, preventing the reasoning_content
field from being generated.

Modified:
- create_base_model(): Add extra_body parameter with thinking mode disabled
- create_title_model(): Add extra_body parameter with thinking mode disabled

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 22:31:50 +08:00
grossmj 8b3d136a84 Update packages in requirements.txt 2026-05-09 18:41:02 +08:00
grossmj 97bf94e06e Merge branch '2.2' into 3.0
# Conflicts:
#	CHANGELOG
#	gns3server/compute/iou/iou_vm.py
#	gns3server/crash_report.py
#	gns3server/utils/asyncio/telnet_server.py
#	gns3server/version.py
#	requirements.txt
#	setup.py
2026-05-09 18:23:00 +08:00
Jeremy Grossmann 32959f1e04 Merge pull request #2724 from yueguobin/fix/import-project-skip-offline-computes
fix: Skip offline compute nodes during project import
2026-05-09 13:59:53 +08:00
YueGuobinandClaude Sonnet 4.6 5c7018283f fix: Skip offline compute nodes during project import
When importing a project, the round-robin logic would attempt to distribute
nodes across all configured compute resources, including offline ones. This
caused import failures when any remote compute was unreachable.

This fix filters the compute list to only include connected computes before
round-robin distribution. If no remote computes are connected, all nodes
are assigned to the local compute.

This matches the approach used in project._get_disconnected_computes() and
prevents the issue where importing a project fails with:
"Cannot connect to compute 'X' with request POST /projects"

Fixes issue introduced in commit 90e3a8d6 (2017) which added round-robin
load balancing without considering offline compute nodes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:46:58 +08:00
Jeremy Grossmann ae2fa31c5e Merge pull request #2721 from yueguobin/feature/list-node-files-api
feat: Add API endpoint to list node files with metadata
2026-05-09 00:04:16 +08:00
YueGuobin b4220d55c6 fix: Import status module in project.py
Fix F821 undefined name 'status' error by importing the status
module from fastapi. This resolves build errors when using
status.HTTP_403_FORBIDDEN and status.HTTP_404_NOT_FOUND.
2026-05-08 23:58:43 +08:00
Guobin Yue cd732ab4b5 Merge branch '3.0' into feature/list-node-files-api 2026-05-08 23:51:56 +08:00
Jeremy Grossmann 14c2172f1b Merge pull request #2722 from yueguobin/fix/console-stopping-error
fix: Handle connection errors when stopping node console
2026-05-08 23:43:39 +08:00
YueGuobin 3752b2bac8 fix: Handle connection errors when stopping node console
Add exception handling in stop_wrap_console to gracefully handle
ConnectionResetError, BrokenPipeError, and OSError when waiting
for console writer to close.

This prevents 500 errors when stopping QEMU nodes if the console
connection is reset before the writer finishes closing.

Fixes race condition where QEMU process exits and closes connections
before the console writer cleanup completes.
2026-05-08 23:30:41 +08:00
YueGuobinandClaude Sonnet 4.6 e6df144ae8 refactor: Remove MD5 calculation from node files listing
Remove MD5 checksum calculation from the node files API since
disk image files are dynamic and change frequently. MD5 calculation
was also causing significant performance overhead.

Changes:
- Remove md5sum field from NodeFile schema
- Remove MD5 calculation from list_node_files method
- Improve error handling for timestamp conversion
- Simplify code by removing lambda functions

Performance improvement:
- Response time reduced from ~0.5-1s to ~0.017s (30-60x faster)
- Especially beneficial for large files and multiple files

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 23:05:40 +08:00
YueGuobinandClaude Sonnet 4.6 fae7b2eabe fix: Export NodeFile schema to schemas module
Add NodeFile to the schemas __init__.py exports to fix
import error when starting the server.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 22:58:45 +08:00
YueGuobinandClaude Sonnet 4.6 51a4c73e21 feat: Add detailed metadata to node files listing
Enhance the node files API to include comprehensive file metadata:
- File size in bytes
- File creation time (ISO 8601 format)
- File modification time (ISO 8601 format)
- File extension

Create new NodeFile schema to support these additional fields
while keeping the existing ProjectFile schema for backward compatibility.

This provides users with better information to manage and identify
files in the Web UI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 22:57:19 +08:00
YueGuobinandClaude Sonnet 4.6 5b75d633aa feat: Add API endpoint to list node files
Implement issue #2719 - Add API endpoint to list project files

- Add GET /v3/projects/{project_id}/nodes/{node_id}/files endpoint
- Add list_node_files() method to Project class
- Add security checks to prevent path traversal
- Filter out .ghost temporary files
- Return file paths with MD5 checksums
- Require Node.Audit privilege

This allows users to discover dynamically created files
such as QEMU disk images created via the disk image API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 22:52:17 +08:00
Jeremy Grossmann 713dbb7f45 Merge pull request #2718 from GNS3/2.2
Release v2.2.59
2026-05-08 22:03:17 +08:00
grossmj c3d23d7cae Development on 2.2.60.dev1 2026-05-08 20:03:31 +08:00
grossmj a1edf74fcf Merge branch 'master' into 2.2 2026-05-08 19:49:53 +08:00
grossmj 59a95edd3d Release v2.2.59 2026-05-08 19:06:04 +08:00
grossmj e34a7afe79 Sync appliances 2026-05-08 19:03:14 +08:00
Jeremy Grossmann 2dcbe0f1c2 Merge pull request #2717 from GNS3/bugfix/2709
Fix snapshot restore does not work after export/import of project
2026-05-08 18:59:37 +08:00
grossmj b1c8ce8b58 Remove unneeded code 2026-05-08 18:56:56 +08:00
grossmj b304511b5e Fix snapshot restore does not work after export/import of project 2026-05-08 18:47:47 +08:00
grossmj 163d4e0534 Update requirements.txt & python_requires 2026-05-08 17:01:10 +08:00
grossmj 72f64ee60b Warn to use 64-bit IOU images. Fixes #2716
32-bit IOU image support has been removed from the GNS3 VM.
2026-05-08 16:37:06 +08:00
Jeremy Grossmann cd03abb7ed Merge pull request #2669 from arl1984/fix/telnet-proxy-reader-cleanup-2344
Fix telnet console silent-proxy hang on non-ConnectionError exit (#2344)
2026-05-08 15:42:51 +08:00
grossmj 6551bf56f8 Tweak and merge changes from 3.0 branch 2026-05-08 15:36:40 +08:00
Jeremy Grossmann e0860555ba Merge pull request #2713 from yueguobin/fix/docker-container-deletion-error-handling
Fix: Improve Docker container deletion error logging
2026-05-07 22:17:23 +08:00
YueGuobinandClaude Sonnet 4.6 822abbe671 Fix: Improve Docker container deletion error logging
When closing a Docker node, if container deletion fails, the error
is silently ignored. This can lead to stale containers remaining on
the system and causing 409 conflicts when reopening projects.

Changes:
- Distinguish between 404 (container already removed, normal) and
  other DockerError (deletion failed, needs attention)
- Log warning when deletion fails with error details
- Add comment explaining stale containers will be cleaned up on
  project open (via automatic 409 conflict resolution)

This improves observability without blocking project close operations.
The root cause of stale containers can now be diagnosed from logs.

Fixes #2708

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-07 12:45:05 +08:00
Jeremy Grossmann 3928377f80 Merge pull request #2710 from yueguobin/fix/compute-connection-timeout
Fix: Check compute connectivity before open() during project deletion
2026-05-07 11:36:45 +08:00
Jeremy Grossmann 0d88ff87e7 Merge pull request #2712 from yueguobin/fix/compute-reconnection-backoff
Fix: Add exponential backoff to compute reconnection attempts
2026-05-07 11:35:46 +08:00
YueGuobin efe6913d69 Fix: Add exponential backoff to compute reconnection attempts
When a remote compute is unreachable, the controller now uses exponential
backoff for reconnection attempts: 5s, 10s, 20s, 40s, 80s, then caps at
300s (5 minutes). Previously it retried every 5 seconds indefinitely.

User-initiated operations (open project, start node) still trigger an
immediate connection attempt, so recovery is not delayed in practice.

Related: #2704
2026-05-07 11:17:55 +08:00
YueGuobinandClaude Opus 4.6 f1b11e7cae Fix: Check compute connectivity before node creation in open()
When opening a closed project with nodes on an offline remote compute,
open() would block for 120s trying to connect before eventually
failing. Now checks compute connectivity after loading the topology
file but before creating nodes, allowing immediate failure with a
clear error message.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-07 10:48:18 +08:00
YueGuobinandClaude Opus 4.6 30ebde0cb0 Fix: Check compute connectivity before open() during project deletion
Previously the disconnected compute check ran after open(), which would
block for 120s trying to connect to unreachable remote computes before
rejecting the deletion. Now reads the topology file directly to extract
compute IDs and checks connectivity before calling open(), enabling
immediate rejection of deletions involving offline computes.

Also removes the redundant post-open() check since the early check
covers both opened and closed project states.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-07 09:58:20 +08:00
YueGuobinandClaude Opus 4.6 8ecd449dd8 Fix: Add error logging when closing/deleting projects on computes
Previously errors during close() and delete_on_computes() were silently
swallowed without any logging, making it difficult to diagnose failures
when remote computes are unreachable.

- close(): log warning instead of silent pass
- delete_on_computes(): wrap HTTP DELETE in try/except with warning log

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-07 09:37:51 +08:00
YueGuobin 4d06b2b7b6 Fix: Skip add_compute for existing computes when opening projects 2026-05-06 13:24:27 +08:00
YueGuobin 0f4b0c567e feat: add AI provider config files to .gitignore
Add deepseek, claude, gemini, and grok settings files to .gitignore to prevent accidental commits of local AI provider configurations.
2026-05-06 13:24:13 +08:00
Jeremy Grossmann 255d1e4057 Merge pull request #2705 from yueguobin/fix/project-deletion-compute-status-check
Fix: Check compute connection status before project deletion
2026-05-06 09:39:41 +08:00
YueGuobin a67cd66b60 Revert "Fix: Handle project deletion errors in autoidlepc"
This reverts commit c92188cad4.
2026-05-06 00:59:05 +08:00
YueGuobin c92188cad4 Fix: Handle project deletion errors in autoidlepc
When deleting the AUTOIDLEPC project in autoidlepc(), if a remote compute
is disconnected, the deletion fails with ControllerForbiddenError due to
our compute connection status check. This causes:
- The project to remain in the controller's memory
- Subsequent autoidlepc calls to fail
- User must restart the server to recover

This fix adds exception handling to ensure the project is always removed
from the controller's memory, even if deletion fails.

Related to #2703
2026-05-06 00:47:24 +08:00
YueGuobinandClaude Sonnet 4.6 9c868911e1 Fix: Use _computes instead of _project_created_on_compute for deletion check
The initial fix used _project_created_on_compute to check for disconnected
computes before deletion, but this set gets reset during project.open(),
causing the check to fail.

Now uses self._computes which is loaded from the topology file and
persists through the open() call.

Related: #2703

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-06 00:19:55 +08:00
YueGuobinandClaude Sonnet 4.6 4444da9ffa Fix: Check compute connection status before project deletion
This commit addresses issue #2703 where deleting a project with nodes
on remote compute nodes would result in long waits with no feedback
if those computes were unreachable.

Changes:
1. Compute connection status updates on connection failure
   - When a compute fails to connect, update connected=False and last_error
   - Send compute.updated notification to UI so users can see status
   - This allows Web UI to display real-time connection status

2. Project deletion checks compute status before attempting deletion
   - Check all computes used by the project are connected
   - If any compute is disconnected, immediately reject deletion
   - Provide clear error message indicating which computes are offline
   - This prevents long timeouts and gives users immediate feedback

Benefits:
- Immediate feedback instead of 120-second timeouts
- Clear error messages about which computes are disconnected
- Prevents orphaned resources on offline computes
- Improves user experience by avoiding silent waits

Related: #2703

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-06 00:05:10 +08:00
Jeremy Grossmann ee7c5d6d59 Merge pull request #2693 from cristian-ciobanu/ssh-console-support
feat(compute): add SSH console type support
2026-05-05 19:03:57 +08:00
Jeremy Grossmann 82d630a122 Merge branch '3.0' into ssh-console-support 2026-05-04 17:17:10 +08:00
Jeremy Grossmann 56455d3a27 Merge pull request #2701 from yueguobin/fix/iou-remote-path-validation
fix: convert absolute image paths to relative paths for remote compute nodes
2026-05-04 16:36:23 +08:00
Guobin Yue 069e139d2d Merge branch '3.0' into fix/iou-remote-path-validation 2026-05-04 15:51:30 +08:00
YueGuobin 1f5ba3cada chore: remove internal debug documentation 2026-05-04 15:48:46 +08:00
YueGuobin 38f5e8df76 fix: extend remote compute image path conversion to all node types
Previously only the "path" field was converted to a relative path for remote computes, causing issues with QEMU and VMware nodes that use different image path fields. Now all known image path fields are converted, ensuring consistent behavior across IOU, QEMU, Dynamips, and VMware node types.
2026-05-04 15:39:16 +08:00
Jeremy Grossmann 5f9b469679 Merge pull request #2700 from yueguobin/fix/issue-2680-static-directory
fix: handle directory access for static web-ui routes
2026-05-03 17:56:50 +08:00
YueGuobinandClaude Sonnet 4.6 6ff4f575bb docs: add analysis for issue #2676 remote compute image path validation
Add detailed documentation explaining:
- Problem description and root cause
- Architecture overview (Controller vs Compute)
- Why automatic upload is not triggered
- Affected node types
- Solution implementation
- Testing scenarios

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 17:47:25 +08:00
YueGuobinandClaude Sonnet 4.6 11fb5de84e fix: send relative image path to remote compute nodes
When creating a node on a remote compute, the controller now sends
only the image filename instead of the absolute local path. The remote
compute will search for the image in its own configured images_directories.

If the image is not found, the compute returns ImageMissingError, which
triggers the controller's automatic image upload mechanism.

This fixes issue #2676 where remote computes reject paths from the
controller due to mismatched directory prefixes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 17:44:47 +08:00
YueGuobinandClaude Sonnet 4.6 952ef66a6b fix: handle directory access for static web-ui routes
When accessing /static/web-ui without trailing slash, the request would
fail with "RuntimeError: File at path ... is not a file" because:

1. The route /static/web-ui/{file_path:path} doesn't match paths without
   trailing slash (Starlette's path regex requires the /)
2. The request falls through to StaticFiles mount, which tries to serve
   the directory as a file

This fix:
- Sets html=True on StaticFiles mount to automatically redirect directory
  URLs to trailing slash versions
- Adds os.path.isdir() check to handle empty file_path gracefully

Fixes #2680

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 17:21:03 +08:00
Jeremy Grossmann c45f6d9416 Merge pull request #2699 from yueguobin/fix/template-duplicate-name-validation
Fix duplicate template name validation when updating templates
2026-05-03 12:09:59 +08:00
YueGuobin 1d715e146b fix: add duplicate name validation when updating templates
Fixes #1658

Add validation in the update_template method to check if a template
with the same name already exists before updating. This prevents
users from creating duplicate template names by editing existing
templates.

The check excludes the current template being edited to allow
updating other properties without changing the name.
2026-05-03 11:08:20 +08:00
Jeremy Grossmann c3d202abc9 Merge branch '3.0' into ssh-console-support 2026-05-02 22:10:28 +08:00
Jeremy Grossmann 165209ba24 Merge pull request #2698 from yueguobin/fix/compute-update-password-none-error
fix: avoid AttributeError when password is not provided in compute update
2026-05-02 22:07:34 +08:00
Jeremy Grossmann b87ab8f86f Merge pull request #2696 from yueguobin/fix/rfc1123-error-message
fix: improve hostname validation error messages with allowed characters
2026-05-02 18:30:49 +08:00
YueGuobinandClaude Opus 4.6 a6dcab8110 fix: avoid AttributeError when password is not provided in compute update
When updating a compute without providing a password field,
compute_update.password is None and calling .get_secret_value()
on it causes a 500 Internal Server Error. Only set the password
in update_values when a password is actually provided.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-02 15:37:46 +08:00
YueGuobinandClaude Opus 4.6 ba220780a9 fix: improve hostname validation error messages with allowed characters
When a hostname validation fails, the error message now includes
the allowed character set to help users provide valid names.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-02 13:09:04 +08:00
Cristi e6de7cb60a feat(compute): Fix Dynamips to listen on the internal port when ssh is used for auxiliary console 2026-04-30 23:50:15 +03:00
CristiandCopilot d7c78f5744 feat(compute): Fix ambiguous log message related to the active console transport.
Co-authored-by: Copilot <copilot@github.com>
2026-04-30 18:07:23 +03:00
Cristi 9d451eb652 feat(compute): Updated server host key to use the secure ssh-ed25519 instead of old ssh-rsa algorithm 2026-04-30 17:56:05 +03:00
Cristi cc05eb5850 feat(compute): Fix missing SSH wrapping for auxiliary console 2026-04-30 17:47:54 +03:00
Cristi 8ff1698bfb feat(compute): Added AsyncioSSHServer dedicated tests 2026-04-30 16:35:21 +03:00
Cristi d64eca0418 feat(compute): add SSH console type support 2026-04-30 15:52:47 +03:00
Jeremy Grossmann f0ef65b8e8 Merge pull request #2688 from yueguobin/fix/llm-model-config-error-message
fix: include error details in LLM model config + docs: add troubleshooting guides
2026-04-25 16:30:47 +08:00
Guobin Yue c583462d2d Merge branch '3.0' into fix/llm-model-config-error-message 2026-04-25 16:26:59 +08:00
YueGuobin c688223e1b docs: add troubleshooting guide for WebSocket console authentication failure
Add documentation explaining the common issue where WebSocket console connections to remote compute nodes fail due to credential mismatch. The guide describes the error symptoms, root cause (Controller forwarding its own credentials to Compute), and solution (ensuring matching compute_username and compute_password in Compute's configuration).
2026-04-25 16:08:53 +08:00
YueGuobinandClaude Opus 4.6 d81a7410e2 docs: remove misleading Controller Hostname Unreachable section
This section was misleading - the actual root cause is host=0.0.0.0
causing controller to register as 127.0.0.1, not hostname resolution.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-25 15:55:37 +08:00
YueGuobinandClaude Opus 4.6 e88b12086a docs: clarify host=0.0.0.0 causes controller to register as 127.0.0.1
Add details about the "No common subnet" error when Controller's host
is set to 0.0.0.0, and how to verify via /v3/version endpoint.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-25 15:54:12 +08:00
YueGuobinandClaude Opus 4.6 f9d1ccc701 docs: add troubleshooting for controller hostname unreachable
Add documentation for the "No common subnet" error when the
controller's hostname in /etc/hosts resolves to an unreachable
or stale IP address.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-25 15:44:20 +08:00
grossmj 27d4c43ad9 Back on development on 3.1.0.dev2 2026-04-25 13:31:35 +08:00
grossmj c6e2bfa9b6 Release v3.1.0a1 2026-04-25 13:19:19 +08:00
grossmj 7fc5601405 Bundle web-ui v3.1.0a1 2026-04-25 12:58:56 +08:00
YueGuobinandClaude Opus 4.6 517764a7ac fix: include error details in LLM model config failure response
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-25 01:48:45 +08:00
grossmj ee43a5cdf5 Bump version to 3.1.0.dev2 2026-04-24 21:56:37 +08:00
grossmj ccf11d8f8b Sync appliances 2026-04-24 21:55:22 +08:00
Jeremy Grossmann b4f31222d0 Merge pull request #2685 from yueguobin/replace-wireshark-with-gns3server-web-wireshark-setup-in-installation-command
docs: replace 'wireshark' with 'gns3server-web-wireshark-setup' in installation commands
2026-04-24 10:51:53 +08:00
YueGuobinandClaude Opus 4.6 4dab36c08c docs: replace 'wireshark' with 'gns3server-web-wireshark-setup' in installation commands
The CLI entry point was renamed from 'wireshark' to 'gns3server-web-wireshark-setup'
to avoid conflicts with the system wireshark package.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-24 10:33:41 +08:00
Jeremy Grossmann d59c82892a Merge pull request #2684 from yueguobin/fix/wireshark-cli-name-conflict
fix: rename wireshark CLI to gns3-wireshark to avoid conflict
2026-04-24 00:48:57 +08:00
grossmj 640910ec8e Use gns3server-web-wireshark-setup in docs 2026-04-24 00:48:00 +08:00
grossmj 6fac4c8c8a Merge branch '3.0' into fork/yueguobin/fix/wireshark-cli-name-conflict
# Conflicts:
#	pyproject.toml
2026-04-24 00:42:48 +08:00
YueGuobinandClaude Opus 4.6 0b7c40a14a fix: rename wireshark CLI to gns3-wireshark to avoid conflict
The wireshark CLI entry point creates /usr/bin/wireshark which conflicts
with the system wireshark package on Arch Linux (owned by wireshark-qt),
causing package installation to fail.

Rename to gns3-wireshark to follow the same naming convention as other
GNS3 CLI tools (gns3server, gns3vmnet).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-24 00:04:21 +08:00
grossmj 746cc8531c Create an executable named gns3server-web-wireshark-setup instead of wireshark. Fixes #2683 2026-04-24 00:00:45 +08:00
Jeremy Grossmann 9d0bb4a592 Merge pull request #2682 from yueguobin/fix/packet-capture-https-skip-cert
fix(copilot): add verify=False for HTTPS with self-signed certs
2026-04-23 23:44:25 +08:00
YueGuobinandClaude Opus 4.6 d6d4f7262e fix(copilot): add verify=False for HTTPS with self-signed certs
Add verify=False to requests.get() when capture_url uses HTTPS protocol,
to skip SSL certificate validation. This allows PacketCaptureTool to
work with GNS3 servers using self-signed certificates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-23 23:20:41 +08:00
Jeremy Grossmann f06b5fff21 Merge pull request #2681 from yueguobin/fix/wireshark-curl-https-skip-cert
fix(web-wireshark): add -k flag to curl for HTTPS with self-signed certs
2026-04-23 23:04:08 +08:00
Guobin Yue b49e6836cc Merge branch '3.0' into fix/wireshark-curl-https-skip-cert 2026-04-23 22:55:26 +08:00
YueGuobinandClaude Opus 4.6 fd6868efce fix(web-wireshark): add -k flag to curl for HTTPS with self-signed certs
Add -k flag to curl command when capture_stream_url uses HTTPS protocol,
to skip SSL certificate validation. This allows Web Wireshark to work
with GNS3 servers using self-signed certificates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-23 22:52:51 +08:00
grossmj 85bbc74f7d Fix using SSL with remote computes. Fixes https://github.com/GNS3/gns3-gui/issues/3733 2026-04-23 21:59:51 +08:00
Jeremy Grossmann 98477371d2 Merge pull request #2679 from yueguobin/wireshark-alias
feat(web-wireshark): add container IP retrieval and Docker API version compatibility fixes
2026-04-22 23:47:43 +08:00
YueGuobinandClaude Sonnet 4.6 a189a98723 docs(web-wireshark): add Docker API compatibility documentation
- Document API version negotiation mechanism (1.44 with fallback to 1.40)
- Add tested configurations table for Docker 20.10 and 29.3+
- Document container IP retrieval dual-strategy approach
- Explain KeyError bug fix for missing NetworkSettings.Networks field

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 23:30:33 +08:00
YueGuobinandClaude Sonnet 4.6 117025a8a2 fix(web-wireshark): improve Docker API version compatibility with fallback mechanism
- Change initial API version from 1.40 to 1.44 to support Docker 29.3+ (API 1.54)
- Add automatic fallback to API 1.40 for older Docker versions (20.10, API 1.41)
- Fixes "client version 1.40 is too old. Minimum supported API version is 1.44" error
- Ensures compatibility across Docker versions 20.10 (API 1.41) to 29.3+ (API 1.54+)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 23:12:56 +08:00
YueGuobin 6c96e7493f feat(web-wireshark): add method to retrieve container IP and refactor websocket endpoint
- Add `get_container_ip` method to WebWiresharkManager for retrieving container IP addresses in wireshark network
- Method uses Docker API as primary approach with container command execution as fallback
- Refactor web_wireshark_websocket endpoint to use new method instead of direct Docker queries
- Improve error handling and logging for container IP retrieval failures
2026-04-22 22:35:50 +08:00
YueGuobin e304b19076 fix: lower Docker minimum API version to 1.40 for broader compatibility
The minimum Docker API version was reduced from 1.44 to 1.40 to support older Docker installations that do not provide API version 1.44, ensuring the web Wireshark agent can connect to a wider range of Docker environments.
2026-04-22 16:08:05 +08:00
YueGuobinandClaude Opus 4.6 2240012402 Rename gns3-wireshark-setup to wireshark for simpler command
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-22 14:13:19 +08:00
Jeremy Grossmann aac9b25b98 Merge pull request #2677 from yueguobin/add-url-branch-parameters
Add --url and --branch parameters to update-bundled-web-ui.sh
2026-04-21 22:54:01 +08:00
YueGuobinandClaude Opus 4.6 e6939513db Add --url and --branch parameters to update-bundled-web-ui.sh
Support specifying custom GitHub repository URL and branch name
when syncing WebUI with gns3server.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-21 22:24:46 +08:00
grossmj 8387b484cc Sync appliances 2026-04-21 18:15:00 +08:00
Jeremy Grossmann 0ac8fd9ff3 Merge pull request #2675 from GNS3/upgrade-dependencies
Upgrade dependencies
2026-04-21 14:00:04 +08:00
grossmj f0f09e0e2c Upgrade alembic version 2026-04-21 13:25:42 +08:00
grossmj 7d84c33598 Upgrade code dependencies 2026-04-21 13:21:45 +08:00
grossmj 19c0bbbc56 Merge remote-tracking branch 'origin/3.0' into 3.0 2026-04-21 13:16:59 +08:00
grossmj 5d60626175 Bundle dev version of the web-ui 2026-04-21 13:16:36 +08:00
Jeremy Grossmann 8311936bc8 Merge pull request #2674 from yueguobin/docs/api-error-responses
docs: add API error response format documentation
2026-04-21 12:48:44 +08:00
YueGuobinandClaude Opus 4.6 56921e10d2 docs: add API error response format documentation
Document unified error response format across all GNS3 API endpoints,
including HTTP status codes, error types, and client-side error handling
patterns.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-21 12:46:48 +08:00
Jeremy Grossmann 5ee26f5ed7 Merge pull request #2673 from yueguobin/fix/swagger-ui-cve-2018-25031
fix: use local swagger-ui static files to resolve CVE-2018-25031
2026-04-21 00:58:52 +08:00
Guobin Yue 1ea8ab0cf8 Merge branch '3.0' into fix/swagger-ui-cve-2018-25031 2026-04-21 00:54:27 +08:00
YueGuobinandClaude Opus 4.6 46d712723a fix: use local swagger-ui static files to resolve CVE-2018-25031
Replace CDN-hosted swagger-ui-dist@3.30.0 with local static files
from gns3server/static/ to address CVE-2018-25031 vulnerability.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-21 00:43:59 +08:00
Jeremy Grossmann a366007f72 Merge pull request #2672 from yueguobin/fix/update-bundled-web-ui-script
Fix update-bundled-web-ui script for newer Angular CLI
2026-04-21 00:41:49 +08:00
YueGuobinandClaude Opus 4.6 85b8b04d72 fix: update bundled web-ui script for newer Angular CLI
- Remove deprecated --build-optimizer flag
- Update copy path from dist/ to dist/browser/ for new Angular build output

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-21 00:39:08 +08:00
Guobin YueandClaude Opus 4.6 df28036a68 fix(deps): resolve 3 security vulnerabilities (CVE-2025-71176, CVE-2026-40347, CVE-2018-25031) (#2671)
* fix(deps): upgrade pytest to 9.0.3 to fix CVE-2025-71176

CVE-2025-71176: pytest 9.0.2 and earlier versions have a local
security vulnerability due to predictable temporary directory naming.

- pytest: 8.4.2 → 9.0.3
- Python 3.10+ is now required

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(deps): upgrade python-multipart to 0.0.26 to fix CVE-2026-40347

CVE-2026-40347: python-multipart < 0.0.26 has a denial of
service vulnerability when parsing multipart data.

- python-multipart: 0.0.22 → 0.0.26

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(deps): upgrade swagger-ui to 4.1.3 to fix CVE-2018-25031

CVE-2018-25031: swagger-ui < 4.1.3 has a spoofing vulnerability
where remote attackers can display remote OpenAPI definitions
via crafted URLs.

- swagger-ui: 3.30.0 → 4.1.3
- Updated swagger-ui-bundle.js and swagger-ui.css

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(api): change FastAPI version from "v3" to "3.0.0" for Swagger UI 4.1.3 compatibility

Swagger UI 4.x enforces stricter version format validation.
The version "v3" is not accepted by the new validator.

Changed from:
- version="v3"

To:
- version="3.0.0"

This affects both controller and compute API definitions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(deps): upgrade swagger-ui to 5.32.4

Fixes "Unable to render this definition" error when loading OpenAPI 3.0
docs. Swagger UI 3.x had incomplete OpenAPI 3.0 support.

- swagger-ui: 3.19.1 → 5.32.4 (latest)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 00:35:15 +08:00
f07d21c511 feat(web-wireshark): add implementation plan for script-driven (#2666)
* feat(web-wireshark): add implementation plan for script-driven integration

Add comprehensive implementation plan for Web Wireshark integration using script-driven approach. The plan outlines:

- Background and motivation for script-based Web Wireshark integration
- Core workflow from API request to WebSocket proxy connection
- Management script architecture with WebWiresharkManager class
- Docker container configuration and resource management
- Performance analysis and optimization recommendations
- Network setup and management procedures

Key features include:
- JWT token extraction from Authorization header
- Automatic GNS3 server URL detection
- Container lifecycle management per project
- Xpra session isolation per link
- WebSocket proxy integration through gns3server
- Resource monitoring and scaling guidelines

The implementation enables users to start Web Wireshark sessions via POST requests with `wireshark: true` parameter, providing a unified web-based packet capture interface.

* feat(web-wireshark): implement script-driven Web Wireshark integration

Add complete Web Wireshark container integration with script-driven architecture:

- Create manage_wireshark.py script for Docker container management
- Add LinkCapture schema with wireshark boolean field
- Update Link class with wireshark and jwt_token parameters
- Implement WebSocket proxy endpoint for xpra HTML5 client
- Add cleanup logic in Project class for container lifecycle

Key features:
- Script-driven container and xpra session management
- JWT token extraction from Authorization header
- GNS3 URL auto-detection with Controller/Config fallback
- WebSocket proxy for unified access through gns3server
- Proper cleanup on project close/delete

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(web-wireshark): use aiohttp instead of docker SDK

Replace docker-py SDK with direct Docker HTTP API calls using aiohttp,
following GNS3's existing architecture pattern in gns3server/compute/docker/.

Changes:
- Create DockerHTTPClient class using aiohttp + Unix socket
- Implement all Docker operations as async methods
- Remove dependency on docker Python SDK
- Use Docker API v1.44 via /var/run/docker.sock

Benefits:
- No additional dependencies required
- Consistent with GNS3's async architecture
- Better integration with existing codebase

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(web-wireshark): add dynamic Docker API version detection

Implement dynamic Docker API version detection following GNS3's pattern
in gns3server/compute/docker/__init__.py.

Changes:
- Add DOCKER_MINIMUM_API_VERSION and DOCKER_PREFERRED_API_VERSION constants
- Remove hardcoded "v1.44" prefix (now using "1.44" format)
- Add _check_connection() method to detect Docker daemon version
- Dynamically select API version based on daemon capabilities
- Add proper error handling for version mismatches

Behavior:
- Initialize with minimum API version (1.40)
- On first connection, detect Docker daemon API version
- Use preferred API version (1.44) if supported
- Fall back to daemon's minimum API version if needed
- Raise error if daemon version is too old

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(web-wireshark): add optional --verbose logging parameter

Remove hardcoded logging.basicConfig() and add --verbose parameter
to control log output, following GNS3's logging pattern.

Changes:
- Remove logging.basicConfig(level=logging.INFO) from module level
- Add --verbose/-v command line parameter
- Configure logging only when --verbose is specified
- Use structured log format with timestamp when verbose

Behavior:
- When called by GNS3: uses GNS3's logging configuration (default)
- When run standalone: no output unless --verbose is specified
- With --verbose: shows detailed logs with timestamps

This prevents the script from interfering with GNS3's logging
configuration while still allowing debug output when needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(web-wireshark): fix code quality issues from static analysis

Fix issues identified by flake8 and pylint static analysis:

Import and formatting fixes:
- Remove unused 'time' import
- Fix import order (stdlib before third-party)
- Remove unnecessary f-strings without placeholders
- Split long line to comply with flake8 (max line length: 127)

Code structure improvements:
- Remove unnecessary 'else' after 'return'
- Add 'from e' to exception re-raising for better tracebacks

Code quality metrics:
- pylint score: 8.11/10 → 9.97/10 (+1.86)
- flake8: 0 errors (max line length: 127, per CI/CD standard)
- All critical issues resolved

Note: R0914 (too-many-locals) warning is informational only,
code remains clear and maintainable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-wireshark): export LinkCapture schema in schemas module

Add LinkCapture to the schemas module exports in __init__.py to fix
the AttributeError when starting the GNS3 server.

Error was:
  AttributeError: module 'gns3server.schemas' has no attribute 'LinkCapture'

This was caused by adding the LinkCapture class to controller/links.py
but forgetting to export it in the schemas __init__.py.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(web-wireshark): add --image parameter for custom Docker images

Add optional --image parameter to allow using custom Docker images
for testing, instead of requiring gns3/web-wireshark:latest.

Changes:
- Add --image parameter to start command (default: gns3/web-wireshark:latest)
- Update get_or_create_container() to accept image parameter
- Update start_wireshark_session() to accept and pass image parameter
- Update TEST.md with examples of using custom images

Usage:
  # Using default image
  python3 manage_wireshark.py start --project-id x --link-id y --jwt-token z

  # Using custom image (for testing)
  python3 manage_wireshark.py start --project-id x --link-id y --jwt-token z --image ubuntu:latest

This makes it easier to test the script without having the official
gns3/web-wireshark image available.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(web-wireshark): add configurable resource limits and increase session capacity

Add configurable resource parameters and increase session support from 10 to 100.

Resource parameters:
- --memory: Memory limit (default: 2g)
- --memory-swap: Memory swap limit (default: same as memory)
- --cpus: CPU cores (default: 1.0)
- --pids-limit: Process limit (default: 1000)

Other improvements:
- Fix CPU quota calculation: 1000000 microseconds = 1.0 CPU core
  (was incorrectly 100000 = 0.1 CPU core)
- Add health check: xpra list command
- Add log configuration: json-file, max-size=10m, max-file=3
- Increase session capacity: 10 → 100 concurrent sessions
- Port range: 12300-12399 (was 12300-12309)
- Display range: :100-:199 (was :100-:109)

This matches the docker run command parameters and provides
better resource management and monitoring capabilities.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-wireshark): update UDPLink.start_capture() to accept new parameters

Update UDPLink.start_capture() to accept the new wireshark and jwt_token
parameters and pass them to the parent Link class.

This fixes the TypeError when starting capture on UDP links:
  TypeError: UDPLink.start_capture() got an unexpected keyword argument 'wireshark'

The UDPLink class overrides start_capture() but didn't include the new
parameters added for Web Wireshark support.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs: add project memory infrastructure and JWT token flow documentation

- Add memory skill for recording project knowledge
- Document JWT token flow in Web Wireshark integration
- Update .gitignore to track skills and memory directories

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-wireshark): fix Docker API URL format and use docker exec CLI

- Add "v" prefix to Docker API URL (http://docker/v{version}/...)
- Remove unused exec_create/exec_start API methods
- Fix Healthcheck.Test format to ["CMD-SHELL", "command"]
- Use docker exec CLI instead of Docker API for command execution
- This aligns with GNS3 docker_vm pattern

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-wireshark): improve xpra session management and error handling

- Check and clean up existing sessions before starting new xpra session
- Add verification that xpra session started successfully
- Use docker exec CLI consistently instead of Docker API
- Improve error logging and diagnostics

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(web-wireshark): add timeout handling and container health checks

- Add REQUEST_TIMEOUT constant for Docker HTTP API requests
- Add asyncio.timeout wrapper for Docker API calls to prevent hanging
- Add _is_container_healthy() to check container responsiveness
- Add _exec_in_container() helper with timeout support
- Improve unhealthy container handling with force remove
- Add detailed logging for container health state

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(web-wireshark): split monolithic file into modules

Split manage_wireshark.py into:
- docker_client.py: Docker HTTP API client
- manager.py: WebWiresharkManager with session management
- manage_wireshark.py: CLI entry point

This improves code organization and maintainability.
Keep all timeout handling and health check logic.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-wireshark): correct NanoCpus calculation to use nanoseconds

NanoCpus in Docker API requires nanoseconds (1 CPU = 1000000000),
not the previous incorrect multiplier of 100000.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-wireshark): fix xpra command quoting and session verification

- Add quotes around --xvfb parameter value to preserve spaces
- Fix session verification to check display number instead of session name
  (xpra list shows "LIVE session at :185" not session name)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-wireshark): run wireshark in background with &

Wireshark with curl pipeline runs continuously, so it must run
in background to avoid blocking. Also reduced timeout since we
don't wait for completion.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(web-wireshark): add deterministic hash for display/port allocation

Use MD5-based hash instead of Python's hash() which is randomized
across process restarts. This ensures display and port numbers are
stable when the GNS3 server restarts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: extract link_id_to_port to shared utils module

Move deterministic hash functions to gns3server/utils/port_allocator.py
to avoid code duplication and ensure consistent algorithm across
manager.py and links.py.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(links): move import to top of file

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(web-wireshark): use correct Config access pattern like gns3-copilot

The Server config is an object with .protocol.value, .host, .port
attributes, not a dict. Fixes URL detection failing and falling back
to 127.0.0.1:3080 which doesn't work from inside containers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add UUID validation utility and use in manage_wireshark

Create gns3server/utils/uuid_validator.py with validate_uuid function
to validate UUID format and catch typos early with clear error messages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use argparse.ArgumentTypeError for proper error message display

Previously ValueError was used which only showed 'invalid validate_uuid value'.
Now shows the full helpful message about expected format.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(web-wireshark): add --dpi=96 to xpra start parameters

Set standard DPI for better display scaling in web Wireshark.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: update TEST.md with correct project-id format

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(web-wireshark): hide xpra shutdown menu via XPRA_CLIENT_CAN_SHUTDOWN

Prevents users from accidentally shutting down the xpra server
through the HTML5 client menu.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(memory): add xpra-html5-client configuration reference

Document xpra HTML5 client configuration including:
- URL parameters for toolbar menu control
- default-settings.txt parameters (Features, Connection, Advanced)
- Server-controlled submenu items
- XPRA_CLIENT_CAN_SHUTDOWN environment variable
- Background image customization

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(web-wireshark): remove xpra stop cleanup to avoid zombie processes

xpra stop can leave zombie processes and timeout. Instead, let xpra start
reuse the display directly (it will overwrite existing session).

Also added --file-transfer=no, --printing=no, --sound=no to disable
unneeded features.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(web-wireshark): add fullscreen_button=false to default-settings.txt

Also remove --file-transfer/--printing/--sound from command line since
xpra doesn't support these options. Configure in default-settings.txt instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(web-wireshark): update xpra background with GNS3 branding

- Replace default xpra background with GNS3 icon and modern gradient
- Update CSS to use SVG background image with cover sizing
- Change background color to gradient from #021d3a to linear gradient
- Improves visual integration with GNS3 web interface

* style: add GPLv3 headers with copyright to web_wireshark modules

Co-Authored-By: YueGuobin <yueguobin@gmail.com>

* feat(docker): use Alibaba Cloud Debian mirror for faster builds in China

Use mirrors.aliyun.com for Debian packages with correct paths for
both main repo (/debian) and security repo (/debian-security).

Co-Authored-By: YueGuobin <yueguobin@gmail.com>

* feat(web-wireshark): update xpra background styling and echo command

- Replace `echo -e` with `printf` for better POSIX compatibility in Dockerfile
- Update xpra HTML5 client background to use GNS3 icon with light gradient
- Change background color from dark blue to light gray for improved visibility

* fix(web-wireshark): use pkill to stop xpra sessions

Replace 'xpra stop :{display}' with 'pkill -f "xpra.*:{display}"' for more reliable session termination.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(web-wireshark): start Wireshark in fullscreen mode

Add --fullscreen flag to Wireshark startup command to prevent window decorations and improve user experience.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-wireshark): use carrier-grade NAT subnet to avoid conflicts

Change Docker network subnet from 172.28.0.0/16 to 100.64.1.0/24 to avoid conflicts with common private networks.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(web-wireshark): update subnet to /22 for 1000+ projects

Change Docker network subnet from 100.64.1.0/24 to 100.64.0.0/22 to support 1000+ projects (1022 available IPs). Update all documentation to reflect the new subnet.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-wireshark): replace CGNAT subnet with configurable private subnet

Replace the CGNAT address block (100.64.0.0/22) with a standard private
subnet (172.31.0.0/22) to avoid network access issues. Many networks and
ISPs block or cannot route CGNAT addresses.

Changes:
- Add WebWiresharkSettings to config schema with configurable subnet
- Read network_subnet from config, default to 172.31.0.0/22
- Add Web Wireshark configuration section to sample config

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-wireshark): comment out dpi setting

Allow xpra to use default DPI settings for better display scaling.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-wireshark): set DPI in Xvfb to prevent scaling warnings

Add -dpi 96 to Xvfb startup command to match xpra's expected DPI
and avoid "scaling problems" warnings.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-wireshark): remove custom Xvfb to use default Xorg-dummy

Remove --xvfb parameter to let xpra use the default Xorg-dummy driver,
which properly handles DPI changes when resize-display is enabled and
prevents DPI mismatch warnings.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* revert(web-wireshark): restore Xvfb configuration

Restore the Xvfb display server configuration for xpra sessions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(web-wireshark): enhance xpra HTML5 default settings

Update xpra HTML5 configuration to disable additional unused features:
- audio, keyboard, mediasource, aurora, http-stream
- Improve readability using heredoc format instead of printf

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(api): add RBAC authentication to web wireshark WebSocket endpoint

- Add `has_privilege_on_websocket` dependency to enforce Link.Capture privilege
- Include current_user parameter in endpoint for user identification and logging
- Update endpoint documentation to reflect token requirement and privilege
- Add test example WebSocket URL in TEST.md for reference

* refactor(web-wireshark): add generic WebSocket proxy and disable xpra HTML

- Add websocket_to_websocket.py utility for binary WebSocket proxy
- Disable xpra HTML server with --html=no flag
- Update manager return value: url -> ws_url
- Simplify WebSocket proxy implementation in links.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(web-wireshark): remove unnecessary sleep delays

Remove 2-second sleep delays after container start and xpra initialization.
Health checks and verification happen immediately, improving startup time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(agent): prevent blocking during project close

- Add 5s timeout for AgentService.checkpointer_conn.close()
- Optimize stop_all_sessions: single command instead of 100 docker exec calls
- Prevents indefinite hang when closing projects

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web-wireshark): return errors to client on startup failure

- Throw ControllerError when Web Wireshark startup fails
- Include detailed error messages from script stderr
- Support both ws_url and url response formats
- Client now receives proper error response instead of success

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(web-wireshark): parallelize startup and remove unnecessary waits

- Parallel execution: container info query + xpra start
- Remove xpra list verification (unnecessary)
- Fire-and-forget Wireshark startup (no timeout wait)
- Startup time reduced from ~7s to ~1s

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(web-wireshark): add stop-container and delete-container commands

- Add stop-container command to stop container on project close
- Add delete-container command to delete container on project delete
- Enables proper container lifecycle management
- Containers are now stopped (not deleted) when project closes
- Containers are deleted when project is deleted

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(project): simplify close flow by removing redundant xpra session cleanup

Since docker stop with timeout=0 already force-kills all processes in the
container, explicitly stopping xpra sessions before stopping the container
is unnecessary. Remove the _cleanup_web_wireshark_xpra_sessions() call
to reduce subprocess overhead and simplify the close flow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* perf(docker): force kill containers on stop by default

Change docker stop timeout from 10 seconds to 0 (immediate SIGKILL).
Web Wireshark containers don't need graceful shutdown since they have
no persistent state.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(web-wireshark): enable WebSocket protocol for xpra connection

- Change xpra bind from --bind-tcp to --bind-ws for WebSocket support
- Enable HTML client with --html=on for xpra WebSocket server
- Pass binary subprotocol when connecting to xpra container

The xpra WebSocket server requires the 'binary' subprotocol during
handshake. Without it, the server returns 403 with message:
"client does not support 'binary' protocol".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(web-wireshark): add WebSocket subprotocol negotiation support

Fix WebSocket proxy for xpra container by implementing proper subprotocol
negotiation. The xpra client requires the server to respond with the
negotiated subprotocol (binary) in the WebSocket handshake response.

Changes:
- Modified get_current_active_user_from_websocket to extract client's
  requested subprotocols from Sec-WebSocket-Protocol header
- Updated websocket.accept() call to include negotiated subprotocol
- Enhanced websocket_proxy to support subprotocol parameter
- Improved logging for WebSocket connection debugging

This fixes the issue where xpra clients would immediately disconnect
after connection due to missing subprotocol in server response.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore(web-wireshark): remove verbose debug logging

Remove excessive debug logs from WebSocket proxy implementation while
keeping essential logs for troubleshooting:
- Keep: subprotocol negotiation, connection establishment, errors
- Remove: verbose step-by-step debugging information

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(web-wireshark): translate Chinese comments and docs to English

- Translate all Chinese comments in project.py, link.py, and links.py to English
- Convert TEST.md and maunal-test.md documentation to English
- Remove obsolete IMPLEMENTATION_PLAN.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(docker): lock xpra version to 6.4.3 for reproducible builds

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(docker): improve Dockerfile with TZ, no-install-recommends, and labels

- Add TZ=UTC timezone setting
- Use --no-install-recommends to reduce image size
- Add LABEL metadata for maintainer and description

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(docs): rename and consolidate documentation

- Rename TEST.md to WEB_WIRESHARK.md for clearer naming
- Merge maunal-test.md content into main document under "Manual Testing" section
- Remove obsolete gns3_icon_black.svg

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(docker): add xpra-x11 package and remove --no-install-recommends

- Add xpra-x11 package required for seamless mode
- Remove --no-install-recommends to ensure all recommended packages are installed

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(api): add capture file download endpoint

Add GET /{link_id}/capture/file endpoint to download PCAP capture files.
Supports downloading while capture is active (streaming).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(web_wireshark): improve session cleanup and add known issues

- Add cleanup of existing processes before starting xpra session to prevent
  "another window manager seems to be running" errors
- Stop all associated processes (xpra, wireshark, Xvfb) when stopping sessions
- Document known issues in WEB_WIRESHARK.md including JWT token security,
  duplicate code, and other implementation concerns

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(web_wireshark): add restart wireshark API endpoint

- Add POST /links/{link_id}/capture/wireshark/restart endpoint
- Add restart command to manage_wireshark.py CLI
- Add _restart_web_wireshark method in link.py controller
- Restart simply calls start_wireshark_session which handles cleanup

This allows users to recover after accidentally closing the Wireshark
window without having to stop and restart the entire capture.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add Web Wireshark business process documentation

Document the Web Wireshark feature including architecture diagrams,
business processes for capture start/stop, container lifecycle,
WebSocket connection flow, and session management.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(web_wireshark): fix container network access and improve session cleanup

- Fix GNS3 server URL detection to use container gateway IP (172.31.0.1)
  instead of localhost/127.0.0.1/0.0.0.0 which don't work from containers
- Add import socket module for gateway IP conversion
- Move urllib.parse import to file header (code style improvement)
- Add logging for Web Wireshark startup (capture stream URL, display, etc.)
- Fix X lock file cleanup to prevent "Server is already active" errors
- Use exec pkill to reduce zombie processes from docker exec bash
- Add _cleanup_x_lock() method to remove X lock files after stopping sessions
- Update link.py to always log subprocess stderr for debugging

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web_wireshark): avoid zombie pkill processes by using Docker API

Replace pkill commands with Docker API-based process killing to prevent
zombie process accumulation in Web Wireshark containers.

Changes:
- Add DockerHTTPClient.list_processes() method to query container processes
- Rewrite _kill_process_tree() to use Docker API instead of pkill
- Unify all process cleanup to use _kill_process_tree() method
- Add logging for killed processes (count and PIDs)
- Include fallback to pkill if Docker API fails

This prevents the accumulation of zombie pkill processes that occurred
when using bash -c "pkill -9 -f pattern" commands.

Testing: Started and stopped multiple capture sessions, verified no new
pkill zombie processes are created during cleanup.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web_wireshark): use docker-init and cleanup socket files

Enable Docker init system (tini) as PID 1 and clean up xpra socket
files when stopping sessions to prevent zombie processes and leftover
sockets.

Changes:
- Add "Init": True to container host_config to use docker-init (tini)
- Extend _cleanup_x_lock() to remove xpra socket files
  * /run/user/1000/xpra/{display}/socket
  * /run/user/1000/xpra/*-{display}
  * /home/gns3/.xpra/*-{display}

Benefits:
- docker-init (tini) automatically reaps orphan processes, eliminating
  zombie process accumulation (tested: 53 zombies -> 0 zombies)
- Socket cleanup prevents xpra list from showing UNKNOWN sessions
- Cleaner container state after stopping sessions

Testing:
- Started and stopped 3 capture sessions
- Verified 0 zombie processes after stopping
- Verified xpra list shows "No xpra sessions found" (no UNKNOWN)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(web_wireshark): add comprehensive CLI script documentation

Add detailed module docstring to manage_wireshark.py explaining:
- Script purpose: CLI tool for manual management, debugging, and testing
- Important note: Use WebWiresharkManager directly for programmatic access
- Usage examples for all commands (start, stop, restart, stop-all, delete)
- Available commands list with descriptions
- Output format specification (JSON to stdout/stderr)
- Help information for getting command-specific usage

This clarifies that the script is intended as a CLI utility, not for
subprocess calls from within GNS3 server code.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(web_wireshark): use direct API calls instead of subprocess

Replace subprocess calls to manage_wireshark.py with direct API calls
to WebWiresharkManager, eliminating subprocess overhead and JSON parsing.

Changes:
- Add import for WebWiresharkManager
- Refactor _start_web_wireshark() to use direct API call
- Refactor _stop_web_wireshark() to use direct API call
- Refactor _restart_web_wireshark() to use direct API call

Benefits:
- Code reduction: 78 lines (-68%)
- Better performance: No subprocess creation overhead
- Better logging: Manager logs directly to GNS3 logging system
- Simpler error handling: Direct exceptions instead of return codes
- No JSON parsing: Direct Python objects
- Resource cleanup: Added finally blocks to ensure manager.close()

Testing:
- All Web Wireshark operations work correctly
- Logs now appear in GNS3 server logs instead of stderr
- Error handling improved with proper exception propagation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web_wireshark): resolve circular import with delayed imports

Fix circular import error by moving WebWiresharkManager import from
module level to function level (delayed import).

Circular dependency was:
  link.py → WebWiresharkManager → Controller → link.py

Solution:
  - Remove top-level import of WebWiresharkManager
  - Add delayed imports in each method that uses it:
    * _start_web_wireshark()
    * _stop_web_wireshark()
    * _restart_web_wireshark()

This allows the controller module to fully initialize before importing
WebWiresharkManager, breaking the circular dependency.

Error before fix:
  ImportError: cannot import name 'Controller' from partially initialized
  module 'gns3server.controller' (most likely due to a circular import)

Testing:
  - Successfully imports Link module
  - GNS3 server starts without import errors

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: move imports to module level and remove Controller dependency

Remove circular dependency between link.py and manager.py by:

1. Remove Controller import from manager.py
   - Manager no longer imports Controller
   - URL detection now relies on Config or default only
   - link.py passes capture_stream_url to manager

2. Move WebWiresharkManager import to module level in link.py
   - No longer need delayed imports since no circular dependency
   - Clean, standard Python import pattern

3. Remove redundant Config import in ensure_network()
   - Config was already imported at module level

Changes:
- manager.py: -24 lines (removed _get_gns3_url_from_controller and redundant import)
- link.py: -30 lines net reduction after moving import to top

Testing:
- All imports work without circular dependency errors
- GNS3 server starts successfully

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(docker_client): use text response for list_processes API

Docker API /containers/{id}/top returns plain text, not JSON.
The generic _request method auto-parses JSON which caused:
  "'dict' object has no attribute 'strip'"

Fix by using session.get() directly with response.text()
instead of the generic _request method.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web_wireshark): use correct capture stream URL endpoint

Remove passing capture_stream_url from link.py to manager.py.
Manager.py auto-detects URL using correct endpoint:
  /v3/projects/{project_id}/links/{link_id}/capture/stream

This endpoint supports JWT authentication, unlike the compute URL
(/v3/compute/...) which requires compute credentials.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web_wireshark): use container-local PIDs for process killing

- Docker API returns host PIDs, not container PIDs - use pgrep inside
  container to get correct PIDs for kill
- Fix list_processes to parse Docker API JSON response correctly
- Remove unnecessary exec prefix in shell commands
- Add logging for debugging process killing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* perf(web_wireshark): optimize process killing with single pgrep call

Replace sequential _kill_process_tree calls with a new _kill_process_tree_batch
method that combines all patterns into a single regex. This reduces docker exec
calls from 8 to 1, improving performance from ~8 seconds to ~0.9 seconds.

Changes:
- Add _kill_process_tree_batch() method with combined regex pattern matching
- Update start_wireshark_session() to use batch cleanup
- Update stop_wireshark_session() to use batch cleanup
- Update stop_all_sessions() to use batch cleanup

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(web_wireshark): disable HTML5 client to improve xpra startup speed

Change --html=on to --html=off to disable the xpra HTML5 client interface.
This reduces xpra startup time from ~6.4s to ~2.9s (55% improvement) while
keeping the WebSocket server functional for browser connections.

The HTML5 client is not needed as we only require the WebSocket endpoint
for remote display forwarding.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(web_wireshark): optimize container health check using Docker built-in status

Replace manual docker exec ping with Docker's built-in health check status
to reduce container startup latency by ~1 second.

Changes:
- Use container["Health"]["Status"] instead of manual ping check
- Skip health check for containers with "healthy" or "starting" status
- Only verify containers with "unhealthy" status
- Keep manual health check for newly started containers

This reduces the container verification time from ~1s to near-zero
for healthy running containers while maintaining safety for unhealthy ones.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(web_wireshark): get gateway IP from Docker API instead of container exec

Replace slow docker exec method with fast Docker Network API call to get
gateway IP. This reduces gateway detection from ~850ms to ~1ms (1000x faster).

Changes:
- Use docker.get_network() API to get gateway from IPAM config
- Keep container exec methods as fallback if Docker API fails
- Remove container_id requirement (not needed for Docker API method)

The Docker network gateway is shared by all containers in the network,
so querying the network is more efficient than querying each container.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(web_wireshark): remove unreliable nameserver fallback for gateway detection

Remove the fallback method that reads /etc/resolv.conf nameserver as gateway.
This method is unreliable because:
- nameserver is not necessarily the gateway (could be upstream DNS)
- Many systems use 127.0.0.53 (systemd-resolved) or 127.0.0.1
- Even when not local, it could be public DNS (8.8.8.8, 1.1.1.1)

The current fallback strategy is sufficient:
1. Docker Network API (fast, reliable)
2. /proc/net/route (standard gateway detection)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(web_wireshark): use host perspective for process killing (37x faster)

Replace docker exec with host perspective process management for killing
container processes. This reduces process cleanup from ~850ms to ~23ms.

Key optimization:
- Get container init PID using docker inspect (fast)
- Use pgrep -P <init_pid> to find child processes from host perspective
- Kill processes directly using host PID (no docker exec needed)

Performance improvements:
- Process finding: 850ms → 23ms (37x faster)
- Process killing: 850ms → <1ms (850x faster)
- File checking: 850ms → 0.003ms (280,000x faster)

Fallback to docker exec method if host perspective fails.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(web_wireshark): use host perspective for file cleanup (891x faster)

Replace docker exec with direct filesystem access from host perspective
for cleaning up X lock files and xpra sockets. This reduces cleanup time
from ~890ms to ~1ms (891x faster).

Key optimization:
- Access /proc/<container_pid>/root/ directly from host
- Use os.remove() and glob.glob() instead of docker exec rm -f
- Fallback to docker exec if host perspective fails

This complements the earlier optimization for process killing, further
reducing startup time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Revert "perf(web_wireshark): use host perspective for file cleanup (891x faster)"

* fix(web_wireshark): recursively kill all descendant processes to prevent orphans

Previous implementation only killed direct children of container init using
`pgrep -P <init_pid>`, but xpra spawns child processes (Xvfb, pulseaudio,
ibus-daemon) that become grandchildren and were not being terminated.

This fix walks the entire process tree recursively to find and kill all
descendant processes, ensuring complete cleanup without orphans.

Performance: ~20-40ms (only 10-20ms slower than previous 23ms, but ensures
thorough cleanup).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(web_wireshark): add comprehensive startup/shutdown performance metrics

Add detailed performance characteristics section documenting:
- Startup performance breakdown (before/after optimization)
- Shutdown performance breakdown (before/after optimization)
- First startup vs subsequent startup comparison
- Measured production data
- Key optimization techniques

Performance improvements: 67% faster startup, 78% faster shutdown,
zero orphan processes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web_wireshark): ensure container stops on project close

Remove dependency on _web_wireshark_container_created flag which gets
reset when project is reloaded, causing container to not stop on project close.

Now directly attempts to stop container on every project close. If container
doesn't exist, the script handles it gracefully. This is simpler and more
reliable than maintaining a flag.

Also removed unnecessary _cleanup_web_wireshark_xpra_sessions() call since
stopping the container automatically terminates all xpra sessions inside.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web_wireshark): ensure container deletion on project delete

Remove dependency on _web_wireshark_container_created flag from
_cleanup_web_wireshark_container() to ensure container is deleted
when project is deleted, even if project was reloaded and flag was reset.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(web_wireshark): remove unused _web_wireshark_container_created flag

This flag was set but never read for any conditionals since we changed
the stop/delete methods to directly attempt the operation. It's purely
redundant code that adds confusion.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(web_wireshark): skip unnecessary cleanup using fast host perspective check

Add _check_residuals_exist() function that uses host perspective (~20ms)
to check if residual processes or socket files exist before cleanup.

Returns (has_process_residuals, has_socket_residuals) tuple to allow
selective cleanup - if only sockets need cleaning, skip the slower
process tree killing.

For new containers with no residuals, this saves ~3 seconds of
unnecessary docker exec calls.

Socket cleanup always uses docker exec for safety (avoiding accidental
host filesystem deletion from /proc/<pid>/root/ path errors).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(web_wireshark): increase container exec timeout from 5s to 10s

xpra start with Xvfb initialization can take longer than 5 seconds in Docker containers, causing timeout failures. Increasing timeout to 10 seconds to allow sufficient startup time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* perf(web_wireshark): optimize cleanup and document docker exec limitations

- Combine X lock and xpra socket cleanup into single docker exec call
  Reduces exec calls from 2 to 1 per stop operation, improving performance
  when stopping multiple capture sessions.

- Document Docker exec performance limitations with test data showing
  parallel exec is 42% slower than serial due to Docker daemon's
  internal queuing mechanism.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(links): prevent AssertionError in stream_pcap during rapid stop_capture

Fix race condition where stream_pcap checks link.capturing (True) but
link.capture_node becomes None before accessing link.compute.

This happens when stop_capture() is called and rapidly cleans up
_capture_node while WebSocket requests are still processing.

Now check both capturing and capture_node to ensure capture is still active.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(links): prevent AssertionError in stream_pcap during rapid stop_capture

Fix race condition where stream_pcap checks link.capturing (True) but
link.capture_node becomes None before accessing link.compute.

This happens when stop_capture() is called and rapidly cleans up
_capture_node while WebSocket requests are still processing.

Now check both capturing and capture_node to ensure capture is still active,
and log at DEBUG level since this is expected behavior during rapid stop.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(web_wireshark): load container config from settings

- Load memory, cpus, and pids_limit from WebWireshark config section
- Apply config only on container creation (_start_web_wireshark)
- Skip config on restart (_restart_web_wireshark) as container exists

Users can now configure container resources in gns3_server.conf:

[WebWireshark]
memory = 4g
cpus = 2.0
pids_limit = 2000

If not configured, defaults (2g, 1.0, 1000) are used.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(web_wireshark): add container statistics to /v3/statistics endpoint

Add Web Wireshark container monitoring to the statistics API for
better visibility into packet capture sessions.

Changes:
- Create new stats.py module with collect_webwireshark_stats()
  * Collects container info (status, project, active sessions)
  * Gets resource limits (memory, CPU, PIDs) from container config
  * Gets resource usage via docker stats (memory, CPU, PIDs)
  * Properly closes aiohttp connections to avoid leaks

- Update /v3/statistics endpoint to include webwireshark data
  * total_containers: Total number of Web Wireshark containers
  * running_containers: Number of containers currently running
  * active_sessions: Total active capture sessions
  * containers: Array with per-container details

- Update statistics-api.md documentation
  * Change URL from /v1/statistics to /v3/statistics
  * Add webwireshark field descriptions and examples
  * Add dashboard integration for Web Wireshark monitoring

Example response:
{
  "webwireshark": {
    "total_containers": 1,
    "running_containers": 1,
    "active_sessions": 2,
    "containers": [{
      "project_id": "...",
      "project_name": "test",
      "container_id": "6edc9029bac0",
      "status": "running",
      "memory_limit": "4.0 GB",
      "cpu_limit": "4.0",
      "pids_limit": 4000,
      "memory": "535.7MiB / 4GiB",
      "cpu": "0.29%",
      "pids": 124
    }]
  }
}

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(project): use WebWiresharkManager directly instead of subprocess

Replace subprocess calls to manage_wireshark.py with direct
WebWiresharkManager method calls in project.py. This eliminates
unnecessary process overhead and maintains consistent usage pattern
with link.py.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(logging): use %-formatting instead of f-strings in WebWireshark methods

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(web_wireshark): use info level log when container not found

Change log level from warning to info when container doesn't exist
during stop/delete operations, and fix f-string without placeholders.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(logging): lower log level for client disconnection

Change log level from warning to debug when client possibly
disconnects, as this is normal behavior when users close
browser tabs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(linting): resolve ruff warnings and errors

- Remove unused variable proc (fire-and-forget subprocess)
- Remove duplicate nodes property definition
- Remove extraneous f-prefix from f-strings without placeholders
- Change bare except to except Exception
- Remove unused imports (sys, asyncio, json)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(web_wireshark): improve error message when Docker image not found

When creating a container fails due to missing image, provide helpful
error message with docker pull command and local build instructions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(schema): add wireshark field to Link schema

Add wireshark boolean field to Link Pydantic schema so it is
included in API responses when Web Wireshark session is active.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(link): add wireshark state tracking

Track whether Web Wireshark is running on a link by adding
_wireshark boolean property, updated on start/stop operations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(chat): add session abort functionality

Add ability to abort streaming chat sessions via REST API.

Changes:
- Add abort flag to MessagesState for tracking abort requests
- Add session_id to state for abort event correlation
- Add _abort_flags dict and check/set/clear functions in gns3_copilot.py
- Add abort_handler_node to generate aborted tool messages
- Modify should_continue to route to abort_handler_node when aborting
- Add abort_session() method in AgentService
- Add POST /sessions/{session_id}/abort API endpoint
- Clear abort flag at stream start

When abort is triggered during streaming:
- If LLM has tool_calls pending, abort_handler_node generates
  aborted tool messages to maintain checkpoint consistency
- Prevents "insufficient tool messages" error on resume

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(chat): send tool_end event when stream is aborted

When a stream is aborted during tool execution, yield proper tool_end
events for aborted tools instead of a generic abort event. This maintains
compatibility with the frontend's expected tool_start/tool_end flow.

Changes:
- Add stream_aborted tracking flag
- After stream loop, check abort flag and yield tool_end events
- Add "abort" to ChatResponse type enum (unused but available)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(web_wireshark): add gns3-wireshark-setup command for Docker image setup

Add a new entry point script that allows users to setup the Web
Wireshark Docker image with a single command:

    pip install gns3-server && gns3-wireshark-setup

The script will:
1. Try to pull gns3/web-wireshark:latest from Docker Hub
2. If pull fails, build the image locally using the included Dockerfile
3. Show raw docker pull/build output for full visibility

Files changed:
- Add setup_wireshark_image.py (new entry point script)
- Update pyproject.toml (add gns3-wireshark-setup entry point)
- Update documentation (README.md, WEB_WIRESHARK.md, web-wireshark-business-process.md)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(appliance): add tags field support for appliances and templates

- Add tags field to ApplianceV1_6 and ApplianceV8 schemas
- Add tags propagation from appliance config to template
- Simplify VPCS builtin template tags

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix test in test_link.py

* Fix link.py after failed tests

* feat(copilot): add device skills system for LLM context injection

- Add skills module with registry and DeviceSkillsTool
- Skills organized by vendor/series for easy extensibility
- Support device_type, category (device/protocol/feature), operation (config/diagnosis)
- First implementation: VPCS skill (gns3_vpcs_telnet)
- Skills tool integrated into teaching_assistant and lab_automation modes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(vpcs): change prompt detection log from warning to debug

The "VPCS prompt not clearly detected" message is not an error,
connection works fine. Change to debug level to reduce noise.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(memory): add uBridge permission issue documentation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(copilot): include link_id in links_summary output

Return link_id in the links_summary method so that the AI can
identify which link to analyze when using the packet capture tool.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(copilot): add packet capture analysis tool

Add PacketCaptureTool that allows AI to analyze packets from an active
GNS3 capture. The tool downloads capture files from the GNS3 server
and runs tshark analysis to help users understand network traffic.

Features:
- Download capture file from /capture/file endpoint
- Run tshark with custom arguments for flexible analysis
- Support analyzing specific packets (e.g., "explain packet #42")
- Support protocol statistics, traffic analysis, and expert info

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(copilot): use shlex.split to properly parse quoted tshark arguments

tshark_args may contain quoted filter expressions like "-Y \"frame.number == 24\"".
Using plain str.split() breaks these quoted arguments, causing tshark warnings
about conflicting display filters. Use shlex.split() to correctly parse quoted
arguments.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(copilot): simplify PacketCaptureTool to accept only packet_number

Instead of exposing complex tshark arguments to the LLM, the tool now
accepts a simple packet_number parameter and internally constructs the
tshark command with verbose output. This prevents the LLM from generating
incorrect tshark parameters while still providing detailed packet analysis.

Changes:
- Remove tshark_args and max_lines parameters
- Accept only packet_number as input
- Internal command: tshark -r <file> -Y "frame.number == N" -V
- Returns complete packet structure without line limits

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add CC BY-SA 4.0 license headers to documentation files

Add SPDX license headers referencing docs/LICENSE file to all markdown
and HTML documentation files in the docs directory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(copilot): add topology planner skill

Add new skill for automatic network lab topology planning:
- IOU as default image, 10.0.0.0/8 IP range, max 10 nodes
- Node naming convention: R/S/PC + number
- 7-step workflow for create/rename/link/start/verify/config
- IP allocation rules and troubleshooting guide

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(copilot): add node positioning rules to topology planner skill

Add grid-based positioning for GNS3 nodes:
- Minimum 250px distance between nodes
- Grid layout: 4 columns, 300x250px spacing
- Formula: x = -400 + col * 300, y = -200 + row * 250
- Update workflow params_required to include x, y coordinates

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(copilot): remove unused helper functions from topology planner skill

Delete Python helper functions that cannot be serialized to JSON:
- calculate_node_positions()
- get_position_for_node()
- allocate_subnet()
- allocate_ip()
- TOPOLOGY_CREATION_STEPS (duplicate of workflow in skill dict)

LLM should use formula descriptions in skill to calculate positions/IPs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(copilot): add topology_planner workflow to lab automation prompt

Add device_skills tool to AVAILABLE TOOLS table.
Add TOPOLOGY PLANNING WORKFLOW section explaining:
- How to query topology_planner skill
- Default conventions (IOU, 10.0.0.0/8, naming rules, position formula)
- 7-step workflow for topology creation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(copilot): support name parameter in gns3_create_node tool

- Add optional 'name' field to node creation, allowing direct naming
- Update Node constructor to pass name parameter
- Remove separate rename step (step_3) from topology planner workflow
- Renumber workflow steps: 1-2-3-4-5-6 (skipping old step_3 rename)

This eliminates the need for a separate gns3_update_node_name_tool call
when creating nodes with predefined names.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(copilot): update topology workflow to 6 steps

Remove rename step since name can be set directly in create_gns3_node.
Workflow changed from 7 steps to 6 steps.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(copilot): replace grid positioning with topology-based layout

Redesign node positioning rules to follow classic network topologies:
- Star: hub at center, spokes radiating outward
- Ring: nodes in circular arrangement
- Bus: linear chain of nodes
- Mesh: grid pattern for interconnected nodes
- Hierarchical: three-tier (Core → Distribution → Access)
- Linear P2P: point-to-point WAN links in a line

Remove old grid formula. Update prompt and workflow to reflect new approach.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(copilot): add note about combining topology types

LLM can mix topology types as needed, e.g., star + linear_p2p for WAN segments.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(copilot): remove language matching rules from prompts

Remove language matching rules that cause inconsistent responses:
- "User writes in Chinese → Respond in Chinese"
- "User writes in English → Respond in English"

These created ambiguity and led to inconsistent language behavior.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(copilot): remove Chinese from response template

Replace mixed Chinese/English headers with English only:
- "操作总结 / Operation Summary" → "Operation Summary"
- "详细信息 / Details" → "Details"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(copilot): remove Chinese mentions from prompts

Remove references to "Chinese" language in title_prompt.py:
- "Generates concise Chinese or English titles" → "Generates concise titles"
- "If the content is predominantly in Chinese, generate a Chinese title" → removed

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(copilot): use short_name for link labels and support short port names

- Match ports by both name and short_name for flexibility
- Use short_name (e.g. "e0/0") instead of full name (e.g. "Ethernet0/0")
  for link labels, improving visual clarity in dense topologies

* feat(topology): replace hardcoded interface names with dynamic placeholders

- Update topology planner skill template to use {short_name} placeholder
- Allows dynamic interface name generation based on device type
- Improves template flexibility for different network device configurations

* refactor(topology): use node-pair IP format and hyphenated naming

- Use 10.0.{node_pair}.x format for P2P links (e.g., 10.0.12.x for R-1-R-2)
- Change node naming to hyphenated format: R-1, R-2, SW-1, PC-1
- Use {short_name} as placeholder for port names in output template
- Remove unused IP_SUBNET_POOL constant
- Simplify ip_planning rules to intuitive format description

* fix(web_wireshark): raise minimum Docker API version to 1.44

Docker daemon requires API version 1.44+, so the minimum
supported version should match rather than defaulting to 1.40.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(web_wireshark): call ensure_network in start_wireshark_session

Ensure Docker network exists before creating container, so that
Web Wireshark works when started via API without going through CLI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(agent): improve Docker image setup with network error detection

- Add `is_network_error()` function to detect network-related failures in Docker pull output
- Modify `pull_image()` to capture output and return success status along with output
- Enhance main logic to detect network errors and skip local build when Docker Hub is inaccessible
- Provide helpful suggestions for network issues (Docker mirror, VPN, manual image transfer)
- Improve error messages to differentiate between network failures and other errors

* feat(docs): add Ubuntu 24.04 development setup guide

Add a comprehensive development environment setup guide for Ubuntu 24.04. The guide includes steps to install dependencies via PPA, configure Docker with mirror accelerators for users in mainland China, set up user permissions, and run the server from source with a Python virtual environment. This provides a clear, step-by-step reference for new contributors and developers.

* feat(docs): add pip mirror note for China mainland users

Add a comment in the development setup documentation suggesting the use of the Aliyun PyPI mirror for users in China mainland to improve installation speed and reliability. This helps overcome network restrictions and slow downloads from the default PyPI repository.

* feat(docs): add LVM root partition expansion guide

Add optional section to development setup documentation with instructions for expanding the root partition when using LVM. This helps developers resolve low disk space issues by utilizing unallocated space in the volume group. The guide includes commands to check LVM status, extend the logical volume, and verify the changes.

* feat(docs): add tshark to development setup dependencies

Add tshark to the apt install command in the development setup documentation. Tshark is required for packet capture functionality in GNS3, ensuring the development environment has all necessary tools for network analysis and debugging.

* feat: add GNS3 documentation skill with standardized structure

Add new documentation skill file defining standards for GNS3 technical documentation. The skill establishes a structured approach focusing on architecture diagrams, flow diagrams, and text descriptions while prohibiting code examples and user scenarios. This ensures consistent, high-quality documentation across the GNS3 ecosystem.

* feat(docs): update documentation skill to focus on server technical docs

Update the GNS3 documentation skill to specifically target server technical documentation under the docs/ directory. The revised standard emphasizes high-level understanding over implementation details, using ASCII diagrams for architecture and business processes, API endpoint tables, and measured performance data. Code specifics are intentionally omitted, directing readers to the codebase for implementation details.

* feat(docs): rewrite documentation skill and statistics API doc

- Rewrite gns3-documentation skill to match actual server-side doc style:
  focus on architecture/flow diagrams, API data, skip code details
- Switch diagram standard from ASCII to Mermaid (GitHub native rendering)
- Rewrite statistics-api.md with Mermaid architecture and sequence diagrams,
  add missing webwireshark container fields (memory_limit, cpu_limit, pids_limit),
  remove speculative content

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(docs): update VNC WebSocket console documentation with Mermaid diagrams

- Replace ASCII diagrams with Mermaid flowcharts for better visualization
- Clarify connection flow between browser, controller, compute, and VNC server
- Update authentication details and endpoint descriptions
- Add missing documentation for packet capture workflow
- Improve overall readability and maintainability

* docs: add AI disclaimer to documentation files

Add a standardized disclaimer to multiple documentation files indicating that the content has been organized by AI with reference to actual code. The disclaimer warns users that AI can make mistakes and advises verification against the source code when in doubt. This improves transparency about the documentation's origin and encourages careful usage.

* feat(docs): update chat API documentation with new features and clarifications

- Update title from "Design Document" to reflect current implementation status
- Add new features: session abort, copilot modes (teaching_assistant, lab_automation_assistant), session pinning
- Enhance architecture diagram with LangGraph StateGraph details including abort_handler_node, title_generator_node, and conditional edges
- Clarify statistics tracking: add ai_response_counted flag, filter title_generator_node from LLM counts, explain incremental token counting
- Improve documentation accuracy for real-time statistics collection and token calculation methods

* feat(docs): restructure command security documentation with visual workflows

- Replace verbose implementation details with concise architecture overview
- Add Mermaid diagrams to visualize command filtering and multi-line expansion flows
- Simplify configuration instructions and remove redundant examples
- Consolidate file structure and function reference tables for clarity
- Maintain all security principles while improving readability and maintainability

* feat(context): refactor context window management documentation for clarity

- Reorganize documentation with improved structure and visual diagrams
- Add Mermaid diagrams to illustrate architecture and trimming process
- Simplify content while maintaining technical accuracy
- Update token counting and trimming strategy explanations
- Enhance readability with better formatting and component tables

* feat(docs): update LLM model configs documentation

- Change config column type from JSONB to JSON (JSONB on PostgreSQL)
- Update provider table to mark base_url as required with planned optional status
- Add note about base_url being currently required in API schema
- Clarify GET own configs endpoint returns plain array
- Add copilot_mode field to update request schema
- Document update limitations for context_strategy and copilot_mode fields

* feat(docs): add GNS3StartNodeQuickTool and update node creation examples

- Add documentation for new `GNS3StartNodeQuickTool` (`start_gns3_node_quick`) that starts nodes without waiting for boot completion
- Update node creation example to include optional `name` field in template placement
- Clarify dynamic wait time strategy for `GNS3StartNodeTool` and contrast with quick tool
- Update tool file structure to reflect new configuration, display, and packet capture tools
- Fix truncated line in suspend tool documentation

* feat(docs): add comprehensive documentation structure with CC BY-SA 4.0 license

Add initial documentation directory with detailed README.md that outlines:
- Dual license structure (CC BY-SA 4.0 for docs, GPLv3 for code)
- Complete directory structure for features, AI Copilot, and bugs
- Feature documentation covering Controller+Compute setup, Statistics API, VNC WebSocket console, and Web Wireshark
- AI Copilot implemented features including Chat API, LLM model configs, command security, and multi-vendor device support

This provides organized technical documentation for the GNS3 server project with proper licensing and feature coverage.

* docs: update multi-vendor device support documentation

- Simplify vendor support table by removing redundant platform column
- Add VPCS as a simulator in status column
- Improve VPCS driver diagram with Mermaid syntax and detailed components
- Document VPCS-specific Netmiko parameters (fast_cli, global_delay_factor)
- Update VPCS tool usage example with connection options structure
- Remove outdated Nornir configuration approaches, reference current implementation
- Add file reference for VPCS tools location
- Clean up documentation structure for better readability

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: YueGuobin <yueguobin@gmail.com>
Co-authored-by: Jeremy Grossmann <grossmj@gns3.net>
2026-04-20 22:56:55 +08:00
arl1984 de0d050689 Fix telnet console silent-proxy hang on non-ConnectionError exit (#2344)
The run() cleanup block was guarded by `except (ConnectionError, OSError):`,
so exits via asyncio.CancelledError or any other exception type skipped
cleanup. Result: `_reader_process` stays pinned to the dead reader and
`_get_reader()` returns None for every subsequent client — the silent-proxy
symptom described in #2344.

Convert the except block to try/finally so cleanup always runs, regardless
of how `_process()` exits. Also:
- catch asyncio.CancelledError + generic Exception (with log.exception) so
  unexpected failures don't swallow the cleanup
- reset `_current_read = None` after cancellation
- use `dict.pop(..., None)` instead of `del` to avoid KeyError races if
  the broadcast loop's timeout handler already removed the entry

Triggering pattern observed in practice: a diagnostic tool opens a console,
sends a few commands, and closes abruptly (e.g. from a test harness or
orchestration script that cancels its Task). If the `_process()` task was
awaiting on one of the `network_read` / `reader_read` futures at the time
of cancellation, the CancelledError propagates up through `run()` and
bypasses the ConnectionError-only except clause. The proxy accepts future
connections (the listen socket is still alive) but never forwards any data
because `_reader_process` never got reset.

Validated against gns3/gns3-server:latest (2.2.56.1) running a 10-scenario
sequential regression batch that previously hung reliably on the 4th
sp_v1 / L3VPN scenario and now completes cleanly across all 10.
2026-04-16 14:46:49 +00:00
grossmj 80cc85e5d5 Fix exception when custom adapters is not set 2026-04-15 14:17:12 +08:00
grossmj 737a174ae5 Merge branch '2.2' into 3.0
# Conflicts:
#	gns3server/controller/project.py
#	gns3server/crash_report.py
#	gns3server/version.py
2026-04-13 11:45:26 +08:00
Jeremy Grossmann d59709071a Merge pull request #2668 from GNS3/release/v2.2.58.1
Release v2.2.58.1
2026-04-12 21:18:23 +08:00
grossmj ae3ae46151 Development on 2.2.59.dev2 2026-04-12 21:17:17 +08:00
grossmj 58fc6a8478 Release v2.2.58.1 2026-04-12 20:59:20 +08:00
grossmj d9b8d0d1ff Sync appliances 2026-04-12 20:49:56 +08:00
grossmj 2b0b31b402 Merge branch 'master' into 2.2 2026-04-12 20:47:14 +08:00
grossmj 66ce3ec280 Fix bug when custom_adapters is None in port_factory.py 2026-04-12 14:05:56 +08:00
grossmj a040acfaa5 Have optional body in start node API endpoint 2026-04-12 14:04:04 +08:00
grossmj 9c988b34d0 Fix snapshots after merging 2026-04-11 19:54:49 +08:00
grossmj 0d3fad8c8f Merge branch '2.2' into 3.0
# Conflicts:
#	gns3server/controller/__init__.py
#	gns3server/controller/export_project.py
#	gns3server/controller/project.py
#	gns3server/controller/snapshot.py
#	gns3server/crash_report.py
#	gns3server/schemas/snapshot.py
#	gns3server/static/web-ui/26.34a83d9cee7db39cd0de.js
#	gns3server/static/web-ui/index.html
#	gns3server/static/web-ui/main.d74d38241f0602d1b3e0.js
#	gns3server/version.py
#	gns3server/web/route.py
#	tests/handlers/api/controller/test_project.py
#	tests/handlers/api/controller/test_snapshot.py
2026-04-11 19:22:05 +08:00
grossmj 4a47151526 Development on 2.2.59.dev1 2026-04-11 18:05:54 +08:00
Jeremy Grossmann adeec10188 Merge pull request #2667 from GNS3/release/v2.2.58
Release v2.2.58
2026-04-11 17:42:38 +08:00
Jeremy Grossmann e747deae88 Merge branch 'master' into release/v2.2.58 2026-04-11 17:37:31 +08:00
Jeremy GrossmannandCopilot Autofix powered by AI 264fffbc6c Potential fix for pull request finding 'CodeQL / Information exposure through an exception'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-04-11 17:34:53 +08:00
grossmj a295d653b8 Release v2.2.58 2026-04-10 20:36:01 +08:00
grossmj c6eec799fb Remove old files 2026-04-10 20:27:46 +08:00
grossmj 8503840307 Bundle web-ui v2.2.58 2026-04-10 20:24:16 +08:00
grossmj c4dbecc1ea Sync appliances 2026-04-10 20:11:16 +08:00
Jeremy Grossmann 3d71967983 Merge pull request #2665 from GNS3/refector-snapshots
Snapshot refactoring
2026-04-10 20:01:45 +08:00
grossmj 161e016594 Remove log message 2026-04-10 19:58:55 +08:00
grossmj 82f795c9ee Add some additional tests 2026-04-10 19:50:42 +08:00
grossmj 2670c68467 Last adjustments for import/export 2026-04-10 18:58:21 +08:00
grossmj 6f7730c455 Update snapshots when duplicating a project 2026-04-09 23:18:55 +08:00
grossmj 274cef6cf5 Add description for snapshots 2026-04-09 20:25:44 +08:00
grossmj fbea9b655d Fix issue when restoring a snapshot of a duplicated project 2026-04-08 22:16:47 +08:00
grossmj 6869f2d71d Snapshot refactoring 2026-04-08 20:02:01 +08:00
Jeremy Grossmann da5a1e9172 Merge pull request #2664 from yueguobin/fix/symbol-cors-headers
Fix/symbol cors headers
2026-04-07 22:27:47 +08:00
YueGuobin 8ab3243c79 fix(symbols): add authentication to symbol raw endpoint
Enable Symbol.Audit privilege requirement for the symbol file download
endpoint. This was previously disabled as a workaround for a web-ui bug.
2026-04-07 20:56:20 +08:00
YueGuobin 8041d2906c fix(symbols): add CORS headers to symbol file responses
FileResponse does not properly include CORS headers when served from
a different origin. This fix manually adds the Access-Control-Allow-Origin
header based on the request origin.

Fixes CORS policy errors when accessing symbol files from frontend
applications running on localhost:4200 or other origins.
2026-04-07 20:22:15 +08:00
Jeremy Grossmann ae8ef84668 Merge pull request #2663 from cristian-ciobanu/project-monitoring
Enhancement: Refresh projects list without restarting the GNS3 server
2026-04-07 18:09:29 +08:00
grossmj 5b04fd0336 Allow to find new project in sub dirs 2026-04-07 18:05:32 +08:00
Jeremy GrossmannandCopilot de8abc0af0 Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-07 17:37:34 +08:00
Jeremy GrossmannandCopilot 725cd0428c Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-07 17:35:33 +08:00
Jeremy Grossmann fdbb7bb70f Merge branch '3.0' into project-monitoring 2026-04-07 15:10:42 +08:00
grossmj e86a05a2ef Fix tests 2026-04-07 15:09:10 +08:00
Jeremy Grossmann 2ce7662c71 Merge branch '3.0' into project-monitoring 2026-04-07 12:58:56 +08:00
Cristi 104f56cb55 Enhancement: Refresh projects list without restarting the GNS3 server 2026-04-07 00:21:56 +03:00
grossmj b4edfdb063 Merge branch '2.2' into 3.0
# Conflicts:
#	gns3server/appliances/infix.gns3a
#	gns3server/appliances/juniper-vJunos-router.gns3a
#	gns3server/controller/ports/port.py
#	gns3server/crash_report.py
#	gns3server/utils/asyncio/telnet_server.py
#	gns3server/version.py
2026-04-06 23:36:03 +08:00
Jeremy Grossmann 2e1db7de6f Merge pull request #2662 from UmmmAGoodName/3.0
Fix addressing issue #2005 including tests
2026-04-06 20:10:47 +08:00
grossmj d97aedb625 Fix tests 2026-04-06 20:03:38 +08:00
grossmj b07685666b Regenerate DB migration script 2026-04-06 19:55:11 +08:00
grossmj b1852f7bcf Fix accessing username in authentication.py 2026-04-06 19:37:17 +08:00
UmmmAGoodName 4cf8b79692 Fix addressing issue #2005 including tests 2026-04-05 21:51:52 +02:00
Jeremy Grossmann e871a1aed6 Merge pull request #2661 from GNS3/revert-2660-backport-telnetlib3
Revert "Backport telnetlib3 changes"
2026-04-05 22:02:05 +08:00
Jeremy Grossmann b222fba42c Revert "Backport telnetlib3 changes" 2026-04-05 21:58:21 +08:00
Jeremy Grossmann f9957b1d90 Merge pull request #2660 from GNS3/backport-telnetlib3
Backport telnetlib3 changes
2026-04-05 21:08:43 +08:00
grossmj bc6374f7fc Add keepalive mechanism using the NOP command 2026-04-05 13:30:38 +08:00
Cristi c739a16d70 Improved exception handling and telnet client disconnect 2026-04-05 13:29:35 +08:00
Cristi 7684ce60ce Backport: Re-write current Telnet server implementation using telnetlib3 library 2026-04-05 13:29:25 +08:00
Jeremy Grossmann 1d10bda22e Merge pull request #2659 from GNS3/appliance-v8
Add support for appliance format version 8
2026-04-04 20:14:23 +08:00
grossmj 51eb88ead1 Use get method for registry_version to avoid KeyError 2026-04-04 20:09:31 +08:00
grossmj 630c2d811d Add support for appliance format version 8 2026-04-04 20:04:42 +08:00
grossmj ef546a46f9 Sync appliances 2026-04-04 19:55:13 +08:00
Jeremy Grossmann 9290b37292 Merge pull request #2654 from yueguobin/feature/extended-statistics
Feature/extended statistics
2026-04-03 21:43:50 +08:00
Guobin Yue 2a11ab6ba3 Merge branch '3.0' into feature/extended-statistics 2026-04-03 21:39:38 +08:00
Jeremy Grossmann 55fa7bb10c Merge pull request #2658 from GNS3/convert-db-fields-to-json
Convert database template fields from PickleType to JSON
2026-04-03 21:06:29 +08:00
grossmj 1dc31127d9 Convert database template fields from PickleType to JSON 2026-04-03 20:59:57 +08:00
Jeremy Grossmann d9f12366bc Merge branch '3.0' into feature/extended-statistics 2026-04-01 12:54:54 +08:00
YueGuobinandClaude Opus 4.6 050b7a80ab docs: add Controller + Compute setup guide
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-01 02:08:18 +08:00
Jeremy Grossmann c9f4f7983b Merge pull request #2645 from cristian-ciobanu/telnetlib3
Enhancement: Re-write current Telnet server implementation using telnetlib3 library
2026-03-31 20:22:59 +08:00
grossmj 0b6f17cf8e Add keepalive mechanism using the NOP command 2026-03-31 18:30:59 +08:00
grossmj 38ebaaa90a Remove requests dependency from dev-requirements.txt 2026-03-31 17:30:41 +08:00
grossmj 9a06830f88 telnetlib3 is a shared dependency with AI Copilot 2026-03-31 17:22:34 +08:00
Jeremy Grossmann bebf070e4d Merge branch '3.0' into telnetlib3 2026-03-31 14:33:35 +08:00
Guobin Yue 02dc2b9a41 Merge branch '3.0' into feature/extended-statistics 2026-03-30 22:48:39 +08:00
Jeremy Grossmann d40e292047 Merge pull request #2655 from yueguobin/fix/device-type-error-message
fix(copilot): improve device_type error message with tested types and…
2026-03-30 14:42:14 +08:00
YueGuobin 4cd74901a5 fix(copilot): improve device_type error message with tested types and config instructions
Add tested device types (cisco_ios_telnet, gns3_huawei_telnet_ce, gns3_ruijie_telnet)
to the error message and include Web UI configuration path.
2026-03-30 14:09:36 +08:00
YueGuobin d64e4e28e3 docs: add future optimization for per-compute node statistics 2026-03-30 00:23:35 +08:00
YueGuobin b379747aeb docs: add statistics API documentation
Document the extended /statistics endpoint with complete
response schema and field descriptions.
2026-03-30 00:09:09 +08:00
YueGuobin 33fdfd65ba feat(statistics): distinguish open vs closed project nodes
Closed project nodes don't have status info in topology JSON.
Add open_project_nodes and closed_project_nodes counts to
help clarify why by_status may be empty.
2026-03-30 00:06:50 +08:00
YueGuobin b4caffe13b fix(statistics): handle closed projects where nodes/links are dicts
Open projects store Node/Link objects, but closed projects store
dictionaries loaded from topology JSON. This fix checks the type
before accessing attributes.
2026-03-29 23:59:17 +08:00
YueGuobin 8b4c3a3517 feat(controller): extend /statistics API with project, node and link stats
Return aggregated statistics:
- computes: per-compute CPU/memory/disk stats (existing)
- projects: total, opened, closed counts
- nodes: total count with breakdown by type and status
- links: total count with capturing count
2026-03-29 23:49:36 +08:00
Jeremy Grossmann 3dbae9585c Merge pull request #2652 from GNS3/fix/2651
Use FastAPI Lifespan Events
2026-03-28 10:02:41 +08:00
grossmj 46f77eb7d5 Remove old add_event_handler 2026-03-28 09:59:29 +08:00
grossmj 01a68fb23a Use FastAPI Lifespan Events 2026-03-28 09:54:22 +08:00
Jeremy Grossmann 89d665f22b Merge pull request #2650 from yueguobin/feature/control-offset
feat(link): add control_offset field to LinkStyle
2026-03-28 08:08:10 +08:00
YueGuobin 7f9416a070 feat(link): add control_offset field to LinkStyle 2026-03-28 02:11:30 +08:00
grossmj b8764b29a5 Upgrade FastAPI to v0.135.2 2026-03-27 10:57:59 +08:00
Jeremy Grossmann 3b261ebbfc Merge pull request #2646 from yueguobin/feature/symbol-delete-api
feat(symbols): add DELETE API for removing custom symbol files
2026-03-27 10:51:51 +08:00
YueGuobin e651ad589b feat(symbols): add DELETE API for removing custom symbol files
- Add DELETE /v3/symbols/{symbol_id:path} endpoint
- Prevent deletion of built-in symbols (:/ prefix or in resource directory)
- Clear symbol size cache after deletion
- Require Symbol.Allocate privilege
2026-03-27 02:28:27 +08:00
Cristi 7f3468dcd3 Improved exception handling and telnet client disconnect 2026-03-24 15:23:00 +02:00
Cristi 83558d4d9f Enhancement: Re-write current Telnet server implementation using telnetlib3 library 2026-03-24 13:23:45 +02:00
Jeremy Grossmann 4701712a86 Merge pull request #2644 from GNS3/release/v2.2.57
Release v2.2.57
2026-03-24 10:10:22 +08:00
grossmj e5ff9e7f00 Development on 2.2.58.dev1 2026-03-24 08:35:49 +08:00
grossmj 7adb0c5226 Release v2.2.57 2026-03-23 09:45:46 +08:00
grossmj 667833dda5 Merge remote-tracking branch 'origin/master' into 2.2 2026-03-23 09:38:13 +08:00
grossmj 646ff00ade Sync appliances 2026-03-23 09:37:15 +08:00
Jeremy Grossmann 77bea35298 Merge pull request #2641 from cristian-ciobanu/3.0
Enhancement Added link type support on the server side
2026-03-20 13:15:03 +08:00
Jeremy Grossmann 53051dd481 Merge pull request #2631 from yueguobin/feature/ai-copilot-bridge
feature/ai-copilot-bridge Branch Summary
2026-03-20 13:05:50 +08:00
YueGuobin 77a38cef03 docs: clarify system scope - baseline config with tool handoff
Clarify the positioning of the template-based system:

Scope (0 → 1):
- Topology provisioning (nodes + links)
- Baseline configuration (IP, routing protocols)
- Management access setup (SSH/HTTPS/NETCONF)
- Basic security configuration

Handoff to Production Tools (1 → N):
- Terraform (Infrastructure as Code)
- REST API (modern device management)
- NETCONF/YANG (standardized configuration)
- Network Controllers (SDN, APIC, etc.)

This clarifies that:
1. The system focuses on rapid environment preparation
2. It complements rather than replaces production tools
3. It's ideal for labs, testing, training, and initial setup
4. Advanced config management is handled by specialized tools
2026-03-20 10:00:23 +08:00
YueGuobin 76c84798dc docs: refactor roadmap - reduce code, add comprehensive diagrams
Major documentation improvements:

Reduced Code Content (~60% reduction):
- Simplified Core Components section with concept-focused descriptions
- Condensed Large-Scale Topology section (kept concepts, removed implementation)
- Streamlined Node Creation Templates (kept workflows, removed tool code)
- Simplified Technical Considerations (kept diagrams, removed code)
- Streamlined Testing Strategy (kept test cases, removed test code)

Added Comprehensive Diagrams:
- System Architecture Overview (4-layer architecture)
- HITL State Transition Diagram (complete workflow with interrupts)
- REST API Endpoints (6 new endpoints with request/response formats)
- SSE Progress Stream (real-time event types)
- Data Flow Diagram (7-step process from user to results)
- Error Handling Flow (4 error types with recovery strategies)
- Template Lifecycle Management (5 stages from DRAFT to ARCHIVED)
- Automatic Layout Strategies (4 layout types with visual examples)
- Auto-Linking Strategies (4 link pattern types with diagrams)
- Security Considerations (3-layer validation approach)
- Error Categories (4 error types with recovery flows)
- Testing Strategy (unit, integration, E2E test scenarios)

Benefits:
- Easier to understand for non-technical stakeholders
- Focus on concepts and workflows rather than implementation
- Visual documentation aids comprehension
- Maintained all critical technical information
- Improved readability and structure
2026-03-20 09:54:58 +08:00
YueGuobin 382b2aefa9 docs: add link creation templates section to roadmap
Add comprehensive link creation templates documentation covering:
- Batch link creation workflow with HITL confirmations
- Link template schema with pattern-based connectivity
- Common topology patterns (Spine-Leaf, Ring, Mesh, Star, Three-tier)
- Intelligent port allocation strategies (round-robin, optimized)
- Performance benchmarks showing 99.9% token savings for large topologies
- Complete workflow example combining all three template systems
2026-03-20 09:44:03 +08:00
YueGuobin 3b4bc93e85 docs: add node creation templates section to roadmap
Added comprehensive node creation template system including:
- Batch node creation workflow (HITL with preview)
- Node template schema (groups, positioning, auto-linking)
- Automatic positioning algorithms (grid, spine-leaf, hierarchical)
- Auto-linking strategies (mesh, paired, linear)
- Batch parallel execution (20-50 concurrent)
- Performance benchmarks for 100-1000+ nodes
- Complete enterprise data center example (724 nodes, 4280 links)

Key benefits:
- 98% token savings for node creation (5000 → 100 tokens for 100 nodes)
- 90% time savings (10 min → 1 min for 100 nodes)
- Auto-linking eliminates manual connection setup
- Combined workflow: Create + configure 724 nodes in ~10 minutes

Updated implementation phases to include Phase 2.5 for node creation templates.
2026-03-20 09:35:45 +08:00
YueGuobin a627d45ca6 docs: add large-scale topology support (1000+ nodes) to roadmap
Added comprehensive section on large-scale topology support including:
- Direct execution mode (skip AI, use rule engine)
- Batch parallel execution (50-100 concurrent)
- Intelligent parameter generation (0 token cost)
- Real-time progress streaming
- Configuration summary generation
- Performance benchmarks for 1000+ nodes
- Multiple addressing schemes (sequential, VLAN-based, hierarchical)

Key benefits:
- 99.5% token savings for 1000 devices (150K → 400 tokens)
- 88% time savings (50 min → 3-5 min)
- Enables rapid provisioning of training labs, CI/CD testing, disaster recovery drills

Updated implementation phases to reflect large-scale support priorities.
2026-03-20 09:26:45 +08:00
YueGuobin 5c42a59413 docs: add template-based configuration roadmap with HITL workflow
Add comprehensive roadmap for implementing Jinja2 template-based
configuration with Human-in-the-Loop confirmations.

Key features:
- Three-step HITL workflow (Template → Parameters → Execute)
- 70-80% token savings for multi-device configurations
- Enhanced safety through human review at each step
- Template reusability across projects

Implementation plan:
- Phase 1: Core MVP (3-5 days)
- Phase 2: UX Enhancement (2-3 days)
- Phase 3: Template Library (2-3 days)
- Phase 4: Advanced Features (3-4 days)
2026-03-20 01:18:56 +08:00
YueGuobin 1378e3b5ec docs: add supported providers list and future enhancements
Add comprehensive documentation for LLM model providers:
- Provider list with default base URLs and requirements
- Examples for different providers (OpenAI, Anthropic, Ollama, Azure)
- Guidelines on when to specify base_url
- Future enhancements section for optional base_url field
2026-03-19 10:12:21 +08:00
YueGuobin 01d24403d3 feat: improve error handling for API configuration errors
Add error_handler utility to detect and format HTML error responses
from misconfigured API base URLs. When the API returns HTML (indicating
configuration issues), users now see a helpful message instead of raw
HTML content.
2026-03-19 00:46:49 +08:00
grossmj c5325fdee4 Install ai-copilot in testing.yml 2026-03-18 16:17:07 +08:00
grossmj 333386ac8b Use status.HTTP_501_NOT_IMPLEMENTED from FastAPI 2026-03-18 16:08:47 +08:00
YueGuobin cd1d28729a feat: make AI Copilot an optional dependency
This change makes AI Copilot features optional to reduce installation
size and support restricted environments.

Changes:
- Split AI dependencies into ai-requirements.txt
- Add ai-copilot optional dependency in pyproject.toml
- Add import protection in gns3server/agent/__init__.py
- Return 501 for AI endpoints when dependencies not installed
- Add gns3server-uninstall-ai-copilot command for cleanup
- Update README with installation and uninstallation instructions

Installation:
- Basic: pip install gns3-server
- With AI: pip install gns3-server[ai-copilot]
- Development: pip install gns3-server[ai-copilot,dev]

Uninstallation:
- gns3server-uninstall-ai-copilot
2026-03-18 15:21:23 +08:00
YueGuobin 0b741dd834 test: simplify class registration assertions in custom netmiko tests
Refactored test assertions for Huawei CE, Ruijie, and VPCS telnet drivers to compare only class names instead of full module paths. This change reduces test brittleness when classes are imported via different module paths while maintaining validation of correct class registration in CLASS_MAPPER and CLASS_MAPPER_BASE.
2026-03-17 22:45:02 +08:00
YueGuobin 55c7d249fb feat(tests): improve class comparison in custom netmiko tests
Refactor test assertions to compare classes by module and name instead of direct object comparison. This ensures tests remain reliable when classes are imported via different paths, preventing false failures due to import variations. Changes applied to Huawei CE, Ruijie, and VPCS telnet driver tests.
2026-03-17 22:33:28 +08:00
YueGuobin e1f25ce63b fix(tests): replace assertIs with assertEqual in custom netmiko tests
Changed test assertions from `assertIs` to `assertEqual` for class mapper comparisons in Huawei CE, Ruijie, and VPCS telnet driver tests. This ensures proper equality checking rather than identity checking, which is more appropriate for class comparisons in these test cases.
2026-03-17 22:06:53 +08:00
YueGuobin 2557614ba8 refactor(llm-configs):
remove unused reserved_jsonb fields

   Remove the three reserved JSONB fields (reserved_jsonb_1, reserved_jsonb_2, reserved_jsonb_3) from the llm_model_configs
   table. These fields were planned for future use but are no longer needed.
2026-03-17 21:43:48 +08:00
Cristi de1b539fb6 Enhancement Added link type support on the server side 2026-03-17 15:29:58 +02:00
YueGuobin a240620443 fix(custom_netmiko): use
assertIs instead of assertEqual for class comparison

   Fix test failures in custom Netmiko driver tests by using assertIs instead
   of assertEqual when comparing class objects registered in CLASS_MAPPER.

   The issue occurred because test files add project root to sys.path, allowing
   the same module to be imported with different paths (e.g., gns3_copilot...
   vs gns3server.agent.gns3_copilot...). assertEqual compares class __module__
   attributes which differ based on import path, while assertIs checks object
   identity which correctly identifies them as the same class.

   Modified files:
   - test_huawei_ce.py
   - test_ruijie_telnet.py
   - test_vpcs_telnet.py
2026-03-17 21:24:39 +08:00
YueGuobin bd06d3a08e feat(vpcs_tools_netmiko): update Nornir import to use core module
Update the import statement for Nornir from `nornir.core.nornir` to `nornir.core` to align with the latest Nornir library structure and avoid potential import errors. This ensures compatibility with updated Nornir versions.
2026-03-17 21:16:15 +08:00
YueGuobin a2a52c8a7d feat(vpcs_tools_netmiko): import Nornir class for type hints
Add import of Nornir class from nornir.core.nornir to support type annotations and improve code clarity in the VPCS tools module.
2026-03-17 16:31:44 +08:00
Jeremy Grossmann 8f325b4127 Merge branch '3.0' into feature/ai-copilot-bridge 2026-03-17 13:41:21 +08:00
grossmj cb18b60c5d Ignore HTTPException received when closing a project 2026-03-17 13:23:19 +08:00
Jeremy Grossmann 0d1341b4e0 Merge pull request #2640 from UmmmAGoodName/Dymamips_fix-3.0
Fixed issue #1605 regarding Cisco slots causing configs to break
2026-03-17 12:54:45 +08:00
grossmj 4f863f3767 Fix typo 2026-03-17 12:51:36 +08:00
grossmj 0aa6566e50 Prevent router to be duplicated when running and add tests 2026-03-17 12:49:38 +08:00
UmmmAGoodName 95bd76a212 Fixed issue #1605 regarding Cisco slots causing configs to break 2026-03-16 19:09:32 +01:00
YueGuobinandClaude Sonnet 4.5 a2c1a913ca docs: add VNC WebSocket console documentation
Add comprehensive documentation for VNC WebSocket console support in GNS3.

Topics covered:
- Architecture and connection flow
- API endpoints (Controller and Compute)
- WebSocket data forwarding implementation
- Supported node types (QEMU and Docker)
- Authentication and authorization
- Configuration examples
- Troubleshooting guide
- Performance considerations
- Security best practices
- Comparison with SPICE

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-17 01:28:52 +08:00
YueGuobinandClaude Sonnet 4.5 64d3c02d42 refactor(compute): remove SPICE WebSocket console support
Remove all SPICE WebSocket-related code due to frontend dependency issues
with spice-html5 library (missing RSAKey/BigInteger implementations).

Changes:
- Remove start_spice_websocket_console() from BaseNode
- Remove SPICE WebSocket endpoints from QEMU and Docker compute APIs
- Remove SPICE WebSocket proxy endpoint from controller API
- Remove WebSocket subprotocol handling from authentication layers
- Remove SPICE documentation

The SPICE console type remains functional for direct connections,
but WebSocket proxy support has been removed.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-17 01:15:26 +08:00
YueGuobin c98f802362 feat(compute): add SPICE WebSocket console support
Add WebSocket-based console support for SPICE protocol, enabling
   browser-based graphical console access with enhanced features
   like clipboard sharing, USB redirection, and audio streaming.

   Changes:
   - Add start_spice_websocket_console() method in BaseNode for SPICE
     WebSocket to TCP bridging with bidirectional binary forwarding
   - Add /console/spice WebSocket endpoints in QEMU and Docker compute APIs
   - Add /console/spice WebSocket proxy endpoint in controller API
   - Add comprehensive API documentation in docs/features/

   Supported console types:
   - spice: Basic SPICE protocol support
   - spice+agent: SPICE with spice-vdagent for enhanced features

   Architecture:
   - Browser WebSocket → Controller (JWT + RBAC) → Compute (Basic Auth)
   - No external websockify processes required
   - Consistent with existing VNC WebSocket implementation
2026-03-16 22:21:25 +08:00
YueGuobinandClaude Sonnet 4.5 ba92b405da feat: add VNC WebSocket console support for Docker and QEMU nodes
Add VNC console WebSocket endpoints for Docker and QEMU nodes:
- Add /console/vnc WebSocket endpoint to compute API (Docker & QEMU)
- Add /console/vnc WebSocket endpoint to controller API
- Implement start_vnc_websocket_console() in BaseNode
- Forward VNC WebSocket traffic between controller and compute layers

The implementation provides bidirectional WebSocket to TCP forwarding
for VNC protocol (RFB) connections, allowing browser-based VNC console
access to containers and VMs.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-16 16:40:00 +08:00
Guobin Yue 09aa12cf94 Merge branch '3.0' into feature/ai-copilot-bridge 2026-03-15 23:13:15 +08:00
YueGuobin ffcfa4ccb1 feat(telnet_server): improve error handling and connection management
- Catch OSError alongside ConnectionError in connection processing to handle more network errors
- Move client_info retrieval inside try block to prevent AttributeError on failed connections
- Replace bare except with specific exceptions (OSError, ConnectionError, asyncio.TimeoutError)
- Improve error logging to include specific exception details and client information
2026-03-15 23:07:06 +08:00
Jeremy Grossmann 6ac4c27bc1 Merge pull request #2639 from yueguobin/fix/telnet-server-connection-race-condition-3.0
Fix/telnet server connection race condition 3.0
2026-03-15 23:01:57 +08:00
Jeremy Grossmann 632b3daab7 Merge pull request #2635 from yueguobin/fix/iou-suspend-405
fix(iou): return 405 error for unsupported suspend operation
2026-03-15 22:47:42 +08:00
grossmj 6e7738a14f Return 405 error for unsupported actions for other node types 2026-03-15 22:38:51 +08:00
YueGuobin b71b323fd4 feat(agent): improve tiktoken initialization and device registration
- Configure tiktoken cache directory to isolate encoding files
- Add logging and timing for tiktoken initialization process
- Make Huawei CE and Ruijie Telnet device registration idempotent
- Add duplicate registration prevention with global flags
- Add logging for device type registration status
- Update .gitignore to exclude tiktoken cache files
2026-03-15 21:25:14 +08:00
YueGuobin 9a01e2194e chore: remove telnet race condition test files and documentation
Remove the bug report documentation and stress test files for the telnet server connection race condition issue.
2026-03-15 13:40:34 +08:00
YueGuobin 4f8daced17 Merge branch 'fix/telnet-server-connection-race-condition-3.0' into feature/ai-copilot-bridge 2026-03-15 13:38:40 +08:00
Guobin Yue 08b579ff83 Merge branch '3.0' into fix/telnet-server-connection-race-condition-3.0 2026-03-15 01:17:30 +08:00
Guobin Yue 9aa3f3b7a0 Merge branch '3.0' into feature/ai-copilot-bridge 2026-03-15 00:37:17 +08:00
YueGuobin 1aced27dc1 feat(docs): add telnet server connection race condition bug report
Add detailed bug report documenting a race condition in the telnet server's broadcast logic. The bug occurs when a client disconnects while the server is iterating through connections to broadcast data, causing an uncaught OSError from getpeername() call.

Key details included:
- Error logs showing OSError: [Errno 107] Transport endpoint is not connected
- Architecture diagram illustrating the telnet proxy server setup
- Root cause analysis showing the race condition timeline
- Problematic code location in telnet_server.py line 305
- Exception hierarchy explanation showing why OSError isn't caught
- Impact assessment and proposed solutions

This documentation will help track and resolve the issue where client disconnections during broadcast cause unhandled exceptions.
2026-03-15 00:19:51 +08:00
YueGuobin cda5fdd94b feat(telnet_server): improve error handling and connection management
- Catch OSError alongside ConnectionError in connection processing to handle more network errors
- Move client_info retrieval inside try block to prevent AttributeError on failed connections
- Replace bare except with specific exceptions (OSError, ConnectionError, asyncio.TimeoutError)
- Improve error logging to include specific exception details and client information
2026-03-15 00:19:51 +08:00
YueGuobin e4faf673af fix(telnet): handle connection race condition during broadcast
Fixes two critical issues in telnet server when clients rapidly
   connect/disconnect during broadcast operations:

   1. **OSError [Errno 107]**: Transport endpoint not connected
      - Root cause: getpeername() called outside try block
      - Fix: Move getpeername() inside try block and catch OSError
      - Expand exception handling to include OSError and TimeoutError

   2. **KeyError**: Double deletion from connections dictionary
      - Root cause: Connection deleted in broadcast loop, then deleted
        again in top-level exception handler
      - Fix: Use dict.pop(key, None) instead of del dict[key]

   **Changes**:
   - Line 216: Add OSError to top-level exception handler
   - Line 227-228: Use pop() to avoid KeyError on double deletion
   - Line 305-316: Move getpeername() inside try block, expand
     exception types, use pop() for safe deletion

   **Impact**:
   - Prevents unhandled exceptions from propagating to asyncio event loop
   - Ensures proper resource cleanup even with race conditions
   - No resource leaks or zombie connections
   - Long-lived clients unaffected by rapid client disconnects

   **Test Script**:
   - Added stress test script to reproduce the issue
   - Supports IOU-L3, VPCS, and generic device types
   - Rapid clients send broadcast-triggering commands (show/run/OSPF)

   **Related**: Fixes bug documented in docs/bugs/telnet-server-connection-race-condition.md
2026-03-15 00:19:51 +08:00
Jeremy Grossmann ad784fbf6f Merge pull request #2637 from UmmmAGoodName/3.0
Minor improvements
2026-03-14 19:10:51 +08:00
Jeremy Grossmann c6be1a28ae Change first() to one_or_none() for image query 2026-03-14 19:06:27 +08:00
YueGuobin 02b20e4ff3 feat(docs): add telnet server connection race condition bug report
Add detailed bug report documenting a race condition in the telnet server's broadcast logic. The bug occurs when a client disconnects while the server is iterating through connections to broadcast data, causing an uncaught OSError from getpeername() call.

Key details included:
- Error logs showing OSError: [Errno 107] Transport endpoint is not connected
- Architecture diagram illustrating the telnet proxy server setup
- Root cause analysis showing the race condition timeline
- Problematic code location in telnet_server.py line 305
- Exception hierarchy explanation showing why OSError isn't caught
- Impact assessment and proposed solutions

This documentation will help track and resolve the issue where client disconnections during broadcast cause unhandled exceptions.
2026-03-14 15:47:36 +08:00
YueGuobin c3b18f4ce2 feat(copilot): add dynamic wait time calculation for node startup
Optimize GNS3StartNodeTool with device-type-aware wait time calculation
   to significantly reduce startup time for fast devices (VPCS, IOU).

   Changes:
   - Add NODE_STARTUP_TIME configuration
     * VPCS: 15s base + 2s per additional node
     * IOU: 25s base + 3s per additional node
     * Other devices: 120s base + 10s per additional node (conservative)

   - Add calculate_startup_time() function
     * Detects device types via node.node_type
     * Uses fast startup time if all nodes are VPCS/IOU
     * Uses conservative time if any slow device present
     * Logs selected strategy and detected types

   - Optimize GNS3StartNodeTool._run() method
     * Retrieve node info (including node_type) before starting
     * Calculate wait time based on detected device types
     * Send start commands after info retrieval
     * Use calculated wait time for progress bar
     * Reuse collected node objects for status retrieval

   Performance improvements:
   - 1 VPCS node: 140s → 15s (89% faster)
   - 5 VPCS nodes: 180s → 23s (87% faster)
   - 1 IOU node: 140s → 25s (82% faster)
   - 5 IOU nodes: 180s → 37s (79% faster)
   - Mixed VPCS/IOU: 180s → 33s (82% faster)

   Documentation:
   - Update node-control-tools.md with dynamic wait time strategy
   - Add device type comparison table
   - Document performance improvements
   - Update changelog

   Code quality:
   - All comments in English
   - flake8 check passed
   - mypy check passed
2026-03-14 15:24:27 +08:00
YueGuobin a946ef2d69 eat(copilot): add dynamic wait time calculation for node startup
Optimize GNS3StartNodeTool with device-type-aware wait time calculation
   to significantly reduce startup time for fast devices (VPCS, IOU).

   Changes:
   - Add NODE_STARTUP_TIME configuration
     * VPCS: 10s base + 2s per additional node
     * IOU: 20s base + 3s per additional node
     * Other devices: 120s base + 10s per additional node (conservative)

   - Add calculate_startup_time() function
     * Detects device types via node.node_type
     * Uses fast startup time if all nodes are VPCS/IOU
     * Uses conservative time if any slow device present
     * Logs selected strategy and detected types

   - Optimize GNS3StartNodeTool._run() method
     * Retrieve node info (including node_type) before starting
     * Calculate wait time based on detected device types
     * Send start commands after info retrieval
     * Use calculated wait time for progress bar
     * Reuse collected node objects for status retrieval

   Performance improvements:
   - 1 VPCS node: 140s → 10s (93% faster)
   - 5 VPCS nodes: 180s → 18s (90% faster)
   - 1 IOU node: 140s → 20s (86% faster)
   - 5 IOU nodes: 180s → 32s (82% faster)
   - Mixed VPCS/IOU: 180s → 28s (84% faster)

   Documentation:
   - Update node-control-tools.md with dynamic wait time strategy
   - Add device type comparison table
   - Document performance improvements
   - Update changelog

   Code quality:
   - All comments in English
   - flake8 check passed
   - mypy check passed
2026-03-14 15:14:28 +08:00
YueGuobin df8338401e feat(copilot): filter built-in utility templates from GNS3TemplateTool
Add filtering logic to exclude built-in utility templates that are not
   useful for network device configuration, making AI focus on actual network
   devices.

   - Add FILTERED_TEMPLATES constant with 6 utility template types
     * atm_switch - ATM switch
     * cloud - Cloud
     * ethernet_hub - Ethernet hub
     * ethernet_switch - Ethernet switch (built-in)
     * frame_relay_switch - Frame Relay switch
     * nat - NAT device

   - Add should_filter_template() function
     * Exact match on template_type field
     * Simple and reliable filtering logic

   - Update GNS3TemplateTool
     * Apply filtering in _run() method
     * Log filtered count for transparency
     * Update tool description

   - Update documentation (node-control-tools.md)
     * Document filtered template types
     * List retained template types
     * Add changelog entry

   - Code quality
     * All comments in English
     * flake8 check passed
     * mypy check passed
2026-03-14 14:54:12 +08:00
YueGuobin e2e2e23cf7 feat(copilot): add VPCS Telnet driver with Netmiko and ANSI code stripping
Implement custom VPCS driver and unified tool architecture:

   - Add VPCSTelnet custom driver (vpcs_telnet.py)
     - No authentication (direct console access like VPCS behavior)
     - Simple prompt pattern matching (PC\d+>)
     - Automatic ANSI escape code stripping for clean output
     - Config mode methods return empty (VPCS has no config modes)

   - Replace vpcs_tools_telnetlib3.py with vpcs_tools_netmiko.py
     - Migrate from telnetlib3 to Netmiko + Nornir architecture
     - Unified tool architecture matching config/display tools
     - Improved code consistency and maintainability

   - Add comprehensive test coverage (test_vpcs_telnet.py)
     - 30 unit tests covering all VPCS driver functionality
     - Tests for ANSI code stripping, telnet_login, send_command
     - Tests for device registration and initialization

   - Update VPCS built-in template (services/templates.py)
     - Add platform:vpcs and device_type:gns3_vpcs_telnet tags
     - Automatic driver selection without manual configuration

   - Update documentation (docs/)
     - multi-vendor-device-support.md: VPCS driver documentation
     - netmiko_devices.md: Add VPCS to supported devices list
     - README.md: Update multi-vendor support description
2026-03-14 14:25:37 +08:00
YueGuobin bbe57f34b9 feat(llm): hide api_key from all API responses and document max_tokens as reserved
Security Enhancement:
   - API keys are now always filtered to null in all LLM config API responses
   - Created LLMModelConfigDataWithoutSecret schema for response validation
   - Added _filter_api_key_from_config() helper in API routes
   - Modified repository to always hide api_key in get_user_effective_configs()
   - Update/create operations still accept and store api_key securely

   Documentation Updates:
   - Updated API key visibility section to reflect new security policy
   - Updated all response examples to show api_key as null
   - Marked max_tokens field as reserved for future use
   - Added "Reserved Fields" section explaining unused fields

   This defense-in-depth approach prevents API keys from being leaked through
   logs, browser devtools, or network monitoring.
2026-03-14 13:12:22 +08:00
YueGuobin a58089cf48 fix(copilot): add exit command warning to prevent session disconnection
Add warnings about 'exit' command in three locations to prevent AI from
   disconnecting Telnet/SSH sessions during command execution:

   - Prompt system rules (lab_automation_assistant_prompt.py)
   - Display tool description (display_tools_nornir.py)
   - Config tool description (config_tools_nornir.py)

   The 'exit' command disconnects the session and causes all subsequent
   commands in the batch to fail. This prevents users from viewing
   command outputs properly.
2026-03-14 00:34:06 +08:00
YueGuobin c76982b29e refactor(copilot): rename Huawei CE driver and add Ruijie tests with format fixes
- Rename huawei_telnet_ce to gns3_huawei_telnet_ce for consistent naming
   - Add comprehensive test suite for RuijieTelnetEnhanced driver (10 tests)
   - Update list_netmiko_telnet_devices.py to include gns3_ruijie_telnet
   - Fix all flake8 format issues (line length, unused imports, variables)
   - Update documentation to reflect device type rename
2026-03-14 00:08:13 +08:00
YueGuobinandClaude Sonnet 4.5 6e88582ff1 docs(gns3-copilot): update documentation for Ruijie driver and new features
- Add gns3_ruijie_telnet to supported devices in netmiko_devices.md
- Document custom RuijieTelnetEnhanced driver with hybrid strategy
  - Preprocessing: auto-insert 'yes' after known interactive commands
  - Fast path: batch send (2-3s for 13 commands)
  - Fallback: one-by-one with real-time prompt detection
- Add AAA/password configuration prohibition documentation
  - System-level enforcement in lab_automation_assistant_prompt
  - Categories: AAA, login passwords, console/VTY auth, encryption
- Document multi-line command handling for banner and similar commands
- Update device_type tag validation with error feedback
- Update multi-vendor support to include Ruijie platform

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-13 23:56:57 +08:00
YueGuobin 4f4ddb6504 feat(ruijie-telnet): enhance interactive command handling with hybrid strategy
Add pre-processing of known interactive commands and implement hybrid send strategy for Ruijie Telnet devices. The new approach:
- Pre-processes configuration commands to automatically insert 'yes' responses after known interactive commands (router-id, erase, delete, format, reload, boot system)
- Implements hybrid strategy: first attempts fast batch send with pre-processed commands, then falls back to one-by-one send with real-time prompt detection if batch fails
- Maintains backward compatibility while improving reliability for interactive configuration scenarios

This improves configuration reliability for Ruijie devices that frequently require confirmation prompts during configuration changes.
2026-03-13 23:51:12 +08:00
YueGuobin ad771e2927 feat: add Ruijie telnet driver support to custom Netmiko tools
- Import and register Ruijie telnet device type in config_tools_nornir.py
- Import and register Ruijie telnet device type in display_tools_nornir.py
- Update custom_netmiko __init__.py to include RuijieTelnetEnhanced driver
- Add Ruijie telnet driver to __all__ exports for proper module exposure
- Extends custom Netmiko support to handle Ruijie devices with interactive prompt handling
2026-03-13 23:33:14 +08:00
YueGuobin 8d9fce1c12 feat(agent): enhance configuration safety and multiline command handling
- Add explicit AAA/password configuration prohibition to safety reminders
- Implement multiline command expansion for banner and similar commands
- Add error handling for devices with missing device_type tags
- Improve logging for configuration errors and multiline expansions
2026-03-13 22:24:36 +08:00
YueGuobin 364ff93783 efactor(copilot): adopt Nornir best
practice for multi-vendor device support

   Refactor Nornir configuration to use host-level connection_options
   instead of dynamic groups, following Nornir's configuration priority
   model (host > group > defaults).

   **Problem:**
   Previous implementation used first device's configuration for all
   devices, causing Cisco devices to use Huawei driver and vice versa.

   **Solution:**
   - Each host now has device-specific connection_options at host level
   - Single generic "network_devices" group for shared settings
   - Host-level config automatically overrides group-level config

   **Changes:**
   - Remove: _get_nornir_groups_config() helper function
   - Remove: _get_nornir_group() helper function
   - Simplify: _initialize_nornir() to use single generic group
   - Update: get_gns3_device_port.py() to return host-level config
   - Reserve: platform field for future NAPALM/scrapli plugin support

   **Benefits:**
   - Cleaner code structure (no dynamic group creation)
   - Follows Nornir best practice ("configuration proximity")
   - Easy to extend with new device types
   - Properly handles mixed-vendor topologies
2026-03-13 09:32:18 +08:00
YueGuobin 44f802ae4f fix: update netmiko platforms after custom device registration
Netmiko's `ssh_dispatcher` calculates platform lists at module import
time. When custom device types (like Huawei CE) are registered
dynamically, these cached lists become stale and do not include the
new platforms.

This change imports `netmiko.ssh_dispatcher` and recalculates the
`platforms`, `platforms_base`, and `telnet_platforms` attributes to
ensure Netmiko recognizes the custom device types.
2026-03-12 23:38:24 +08:00
YueGuobin effea3fb26 feat(copilot): add Netmiko device list generator and simplify Huawei driver
- Add script to generate Markdown documentation of all Netmiko supported
     devices (SSH and Telnet), grouped by platform with source attribution
   - Highlight custom GNS3-Copilot devices with "Custom ✨" marker
   - Auto-generate device list to docs/gns3-copilot/netmiko_devices.md
   - Simplify HuaweiTelnetCE driver: remove redundant aliases
     (huawei_ce, huawei_telnet_ce_telnet), keep only huawei_telnet_ce
   - Update tests to match simplified device registration
2026-03-12 23:03:19 +08:00
YueGuobin 50fa4f6e2a refactor(huawei_ce): disable mypy and refactor driver
Disable mypy type checking for the Huawei CE driver module due to
Netmiko library limitations. Netmiko lacks type stubs (py.typed) and
uses dynamic attributes, which causes unresolved import and attribute
errors in static analysis.

Added a comprehensive comment block at the top of the file explaining
the rationale for disabling mypy to prevent future confusion.

Additionally, performed code cleanup including:
- Reformatted module docstrings and comments for better readability
- Added missing imports (importlib, logging)
- Removed unused typing imports (Optional)
- Refactored variable assignments in send_config_set for clarity
2026-03-12 22:38:21 +08:00
Mr.GIitch 302878b391 Merge branch 'GNS3:3.0' into 3.0 2026-03-12 14:44:43 +01:00
YueGuobin db164e9d41 feat(copilot): add multi-vendor device support with custom Huawei driver
Add comprehensive multi-vendor support for GNS3 network automation,
   including a custom Netmiko driver for Huawei CloudEngine devices.

   Features:
   - Custom HuaweiTelnetCE driver for GNS3 emulation (no authentication)
   - Auto-commit before exit to prevent [Y/N/C] prompts
   - Dynamic device type detection from GNS3 node tags
   - Support for both Cisco IOS and Huawei devices
   - Proper VRP command handling (system-view, return confirmation)

   Implementation:
   - New package: utils/custom_netmiko/
     - huawei_ce.py: Huawei CloudEngine driver
     - tests/test_huawei_ce.py: Unit tests (9/9 passing)
     - README.md: Driver development guide
   - Updated tools for multi-vendor support:
     - display_tools_nornir.py: Dynamic group generation
     - config_tools_nornir.py: Multi-vendor config commands
     - get_gns3_device_port.py: Device port extraction
   - Documentation: multi-vendor-device-support.md

   Limitations:
   - huawei_telnet_ce driver requires devices without authentication
   - For devices with username/password, use standard huawei_telnet driver

   Co-Authored-By: Yue Guobin <yueguobin@outlook.com>"
2026-03-12 16:44:32 +08:00
Guobin Yue ab88bcd57e Merge branch '3.0' into feature/ai-copilot-bridge 2026-03-12 16:12:13 +08:00
grossmj 3abecc8492 Fix IOU tests 2026-03-12 14:18:35 +08:00
grossmj 27ce1a4e25 Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	gns3server/schemas/iou_template.py
#	tests/handlers/api/compute/test_iou.py
#	tests/handlers/api/controller/test_template.py
2026-03-12 14:11:05 +08:00
Jeremy Grossmann 5c63928636 Merge pull request #2634 from GNS3/update-iol-ram-nvram
Deactivate 'use default IOU values' by default and update RAM/NVRAM values
2026-03-12 13:53:04 +08:00
grossmj f41cc24383 Fix for duplicating Qemu nodes 2026-03-12 13:24:30 +08:00
Jeremy Grossmann 9aca6aece3 Merge pull request #2636 from yueguobin/fix/acl-endpoint-paths
fix(acl): correct endpoint paths for users, groups, and roles
2026-03-12 12:31:59 +08:00
UmmmAGoodName afeb6a3196 Minor optimizations to logic 2026-03-11 18:51:32 +01:00
UmmmAGoodName 99e3529c93 Fixed an issue with the server causing errors when uploading large images + Preparation for a smarter check when uploading images 2026-03-11 18:50:43 +01:00
UmmmAGoodName 19503ab4f7 Implementet a smarter node_type system 2026-03-11 18:49:36 +01:00
UmmmAGoodName 4a45a68703 Changed errors to new type, was causing warnings 2026-03-11 18:48:36 +01:00
YueGuobin 75a4585c20 docs: restructure copilot docs and update node tools
Simplify the documentation structure in `README.md` by removing the
`todo/` directory reference and detailed design documents for planned
features. Consolidate future roadmap items into a high-level summary
under "Future Enhancements".

Update `node-control-tools.md` to include documentation for new topology
management tools (create node, create link, get template, rename node)
and reflect updated API imports for `Link` support.
2026-03-12 00:23:00 +08:00
Guobin Yue 085a485503 Merge branch '3.0' into fix/acl-endpoint-paths 2026-03-11 23:34:18 +08:00
YueGuobin f2fd725e07 fix(acl): correct endpoint paths for users, groups, and roles
Fix the path mismatch between /acl/endpoints API and actual routes:
- Users: /users/{id} → /access/users/{id}
- Groups: /groups/{id} → /access/groups/{id}
- Roles: /roles/{id} → /access/roles/{id}

This fixes the error where creating ACE entries fails with:
"Path '/groups/{id}' doesn't match any existing endpoint"

The actual routes are registered under /access/ prefix, but the
endpoints API was returning paths without the prefix.

Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-11 23:23:52 +08:00
YueGuobin 5155e10214 feat(deps): update langchain ecosystem and add langsmith SDK
- Upgrade langchain packages to latest versions (langchain 1.2.10, langgraph 1.0.9, etc.)
- Add langsmith SDK for enhanced observability and tracing
- Update Pillow to 12.1.1 for image processing improvements
- Maintain compatibility with existing AI and automation framework
2026-03-11 22:18:31 +08:00
YueGuobin e82bce9e16 feat(docs): add TOSCA-based topology description proposal
Add comprehensive documentation proposing TOSCA (Topology and Orchestration Specification for Cloud Applications) as the standard format for GNS3 network topologies. The document outlines strategic benefits including standardized YAML descriptions, toolchain ecosystem integration, Git-based workflows, and template reuse capabilities. This initiative aims to modernize GNS3 topology management, improve user experience, and align with industry best practices for network automation and orchestration.
2026-03-11 22:13:23 +08:00
YueGuobinandClaude Sonnet 4.5 0056bdd19b fix(iou): return 405 error for unsupported suspend operation
Fixed IOU node suspend API to return proper HTTP 405 Method Not Allowed
error instead of misleading 204 No Content response.

Changes:
- Added HTTPException import to iou_nodes.py
- Fixed suspend_iou_node route from /stop to /suspend (bug fix)
- Changed response from 204 No Content to 405 Method Not Allowed
- Added clear error message: "Suspend is not supported for IOU nodes"

This fix ensures clients receive explicit feedback when attempting to
suspend IOU nodes, which do not support suspend functionality.

Related issue: IOU nodes previously returned 404 when suspend was called
due to incorrect route registration.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-11 15:14:23 +08:00
YueGuobin ad3621db0f feat(gns3-copilot): add node stop and suspend tools for lab automation
- Add GNS3StopNodeTool and GNS3SuspendNodeTool to lab automation assistant mode
- Update tools_v2 __init__.py to export new node control tools
- Document node control tools in README with key features and implementation status
- Update last modified date in documentation

The new tools provide complete node lifecycle control for automated lab workflows, including stopping nodes for shutdown and suspending nodes while preserving state.
2026-03-11 14:44:51 +08:00
YueGuobin 1cd4aeb2a2 feat(schemas): simplify LLM model config descriptions
- Remove redundant "(REQUIRED)" and detailed explanations from context_limit field descriptions
- Shorten copilot_mode descriptions by removing parenthetical details about mode capabilities
- Maintain field formatting consistency across LLMModelConfigData, Create, and Update schemas
2026-03-11 10:17:06 +08:00
YueGuobin 10a827f0c5 feat(api): enhance max_tokens field with robust null handling
- Update LLMModelConfigUpdate schema to accept Union[int, str] for max_tokens
- Add field validator to gracefully handle various null representations:
  - Convert string "null" to null
  - Convert empty strings to null
  - Convert numeric strings to integers
  - Accept proper JSON null values
- Update documentation to reflect new behavior and explain robust null handling
- Prevents validation errors from incorrect null serialization by clients
2026-03-11 10:14:06 +08:00
YueGuobinandClaude Sonnet 4.5 e452f75ac3 docs: reorganize AI Copilot documentation structure
- Add README.md with documentation overview and structure guide
- Move implemented designs to docs/gns3-copilot/implemented/:
  - chat-api.md (from ai-chat-api-design.md)
  - llm-model-configs.md (from llm-model-configs-api.md)
  - command-security.md
  - context-window-management.md
- Add Jinja2 configuration template system design documents:
  - jinja2-config-templates-system.md
  - config-templates-implementation-guide.md
  - ai-prompting-for-config-templates.md
- Remove obsolete documents (acl-web-ui, Chinese RBAC doc)

This reorganization makes it clearer which features are implemented
vs planned, following the established documentation structure.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-10 23:20:24 +08:00
YueGuobin a6c33061c4 delete doc 2026-03-10 22:51:57 +08:00
YueGuobin 578f79cd6b feat(docs): add AI prompting guide for configuration templates
Add comprehensive documentation for AI prompting to generate structured configuration data for Jinja2 templates. The guide includes:

- Core system prompt with critical rules for structured data generation
- Supported vendors and OS types (Cisco, Juniper, Huawei, Arista, Mikrotik)
- Detailed schemas for network features including OSPF, BGP, VLANs, ACLs, and more
- Example prompts and expected structured outputs
- Integration with the configuration renderer system

This documentation ensures AI-generated configurations follow the required structured format for proper template rendering, preventing direct configuration text generation and maintaining consistency across different network device vendors.
2026-03-10 22:38:41 +08:00
grossmj 0624c1b945 Deactivate 'use default IOU values' by default and update RAM/NVRAM values 2026-03-10 20:28:00 +08:00
YueGuobin 2dcdadf5fa docs: add mypy type checking issues todo document
Add comprehensive documentation for mypy static type checking
issues found in gns3_copilot module.

Document includes:
- 35 type errors across 9 files
- Detailed error descriptions and line numbers
- 3-phase fix strategy (High/Medium/Low priority)
- Mypy configuration recommendations
- Resource links for type hints and mypy

Related files:
- chat_sessions_repository.py (8 errors)
- context_manager.py (5 errors)
- agent_service.py (5 errors)
- gns3_topology_reader.py (2 errors)
- message_converters.py (1 error)
- connector_factory.py (1 error)

Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-10 01:20:42 +08:00
YueGuobin 71802a8064 style: remove unused List import from llm_model_configs.py
Remove unused typing.List import flagged by ruff F401.

Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-10 01:11:37 +08:00
YueGuobin 5e9ec23a45 style: remove unused imports and fix E712 comparisons in llm_model_configs.py
Remove 5 unused imports flagged by ruff F401:
- sqlalchemy.func, sqlalchemy.or_
- sqlalchemy.ext.asyncio.AsyncSession
- sqlalchemy.orm.selectinload
- json

Fix 4 E712 errors - avoid equality comparisons to True:
- Use models.LLMModelConfig.is_default instead of == True

Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-10 01:10:19 +08:00
YueGuobin 7d031a6802 style: remove unused imports and variables in project.py
Remove unused import and exception variable flagged by ruff:
- Remove unused 'sys' import (F401)
- Remove unused exception variable 'e' in except clause (F841)

Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-10 01:08:20 +08:00
YueGuobin c0aec3f979 style: remove unused imports from llm_model_configs.py
Remove 3 unused imports that were flagged by ruff F401:
- fastapi.Response
- fastapi.security.OAuth2PasswordRequestForm
- gns3server.controller.controller_error.ControllerError

Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-10 01:07:23 +08:00
YueGuobin 7c3b832bca style: fix E501 line-too-long errors in gns3_copilot
Fix all 423 E501 line length violations across 26 files to comply with
   PEP 8 88-character line limit.

   Changes:
   - Split long f-strings across multiple lines
   - Break long docstring descriptions and parameter lists
   - Split markdown table rows and list examples
   - Break long URL construction f-strings
   - Split long logger messages and comments
   - Add noqa: E501 for SVG strings (cannot be split)

   Modified files:
   - agent/: context_manager.py, gns3_copilot.py, model_factory.py
   - gns3_client/: connector_factory.py, context_helpers.py, custom_gns3fy.py,
                   gns3_project_info.py, gns3_topology_reader.py
   - prompts/: __init__.py, lab_automation_assistant_prompt.py,
               prompt_loader.py, teaching_assistant_prompt.py
   - tools_v2/: __init__.py, config_tools_nornir.py, display_tools_nornir.py,
                gns3_create_link.py, gns3_create_node.py, gns3_get_node_temp.py,
                gns3_start_node.py, gns3_update_node_name.py,
                vpcs_tools_telnetlib3.py
   - utils/: __init__.py, command_filter.py, get_gns3_device_port.py,
             gns3_drawing_utils.py, llm_config_helper.py, message_converters.py,
             parse_tool_content.py, tool_call_stream.py

   All files now pass ruff E501 checks.

   Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-10 01:05:17 +08:00
Guobin Yue 25f2d1b56b Merge branch '3.0' into feature/ai-copilot-bridge 2026-03-09 21:50:37 +08:00
YueGuobin 7c2083d227 feat: update .gitignore to track project context and development docs
- Add exception for .claude/development.md to allow tracking development documentation
- Comment out PROJECT_CONTEXT.md exclusion to enable version control of project context file
- Create PROJECT_CONTEXT.md with comprehensive project overview for AI assistant support
- Document project structure, AI copilot data flow, SSE event types, and code standards
- Provide flake8 static analysis guidelines and common error fixes
2026-03-09 21:46:58 +08:00
grossmj 69eafda777 Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	gns3server/version.py
2026-03-09 17:12:24 +08:00
grossmj b658f7ff68 Make sure the node shows as stopped when the wrap console cannot be stopped. Ref https://github.com/GNS3/gns3-registry/pull/1010 2026-03-09 17:10:00 +08:00
YueGuobin b936a52f6f docs: add LangGraph Agent and tool output serialization documentation
Added new section in ai-chat-api-design.md explaining:

- LangGraph Agent architecture and main components
- tool_node function and tool output serialization mechanism
- Why serialization happens in tool_node (not agent_service)
- Tool output data flow diagram showing SSE and history paths
- Explanation of JSON format consistency across streaming and storage

This documentation clarifies the fix for single-quotes issue in
conversation history and helps future maintenance.

Co-Authored-By: YueGuobin <yueguobin@outlook.com>
2026-03-09 15:54:44 +08:00
YueGuobin 9b85417a82 fix(copilot): serialize tool output to JSON in ToolMessage for history
Fixed tool output serialization in tool_node function to ensure
ToolMessage.content is always in JSON format, not Python str() representation.

This fixes the issue where conversation history showed tool outputs
with single quotes (Python format) instead of standard JSON.

Changes:
- Added json import to gns3_copilot.py
- Modified tool_node() to serialize observation to JSON before creating ToolMessage
- Ensures both SSE streaming and history storage use consistent JSON format

Root cause: ToolMessage was created with raw dict/list objects, which
LangChain converted to Python str() representation when saving to history.

Co-Authored-By: YueGuobin <yueguobin@outlook.com>
2026-03-09 15:50:36 +08:00
YueGuobin 8babaccd80 feat: add PROJECT_CONTEXT.md to gitignore
Add PROJECT_CONTEXT.md to the .gitignore file to prevent it from being tracked in version control. This file likely contains project-specific context or configuration that should not be committed to the repository.
2026-03-09 15:34:31 +08:00
YueGuobin 0f2d0e2859 fix(copilot): serialize tool output to standard JSON format for frontend parsing
Changed tool output serialization in AgentService._convert_event_to_chunk()
from str() to json.dumps() to ensure structured data (dict/list) is properly
formatted as standard JSON instead of Python string representation.

Changes:
- Added json import to agent_service.py
- Modified on_tool_end event handling to use json.dumps(output, ensure_ascii=False, indent=2)
- Updated ai-chat-api-design.md to document tool_output format

Benefits:
- Frontend can parse tool results with standard JSON.parse()
- Chinese and non-ASCII characters are preserved (not escaped)
- Formatted output (indent=2) improves readability

Co-Authored-By: YueGuobin <yueguobin@outlook.com>
2026-03-09 15:15:50 +08:00
YueGuobin 13a032ea2c chore: update author name and copyright headers
Updated the author name and copyright statements across the
gns3_copilot module. The name has been standardized from
"Guobin Yue" to "Yue Guobin (岳国宾)" to reflect the correct
author attribution including Chinese characters.
2026-03-09 11:46:28 +08:00
Guobin Yue d7e5fe065e Merge branch '3.0' into feature/ai-copilot-bridge 2026-03-08 13:20:27 +08:00
YueGuobin 9ba02e9436 feat(docs): enhance AI chat API documentation with examples and details
- Improve POST /chat endpoint documentation with request/response examples
- Add session ID management flow explanation
- Enhance GET /sessions endpoint with query parameters and response example
- Update GET /sessions/{session_id}/history with detailed response structure
- Format parameters as tables for better readability
- Clarify session ID usage in streaming conversations
2026-03-08 02:27:48 +08:00
YueGuobin fd34ef494f feat: add .claude directory to gitignore for API key security
Add .claude/ directory to .gitignore to prevent accidental
commit of Claude Code settings which may contain sensitive
API keys. This ensures development environment configuration
files with potential credentials are excluded from version
control.
2026-03-07 23:52:16 +08:00
Jeremy Grossmann cd51afb01f Merge pull request #2633 from GNS3/revert-2628-console
Revert "Fix QEMU serial console artifacts by filtering ANSI CPR responses in …"
2026-03-07 22:24:17 +08:00
Jeremy Grossmann 14ffa2c2d6 Revert "Fix QEMU serial console artifacts by filtering ANSI CPR responses in …" 2026-03-07 22:22:16 +08:00
grossmj 73ffb22aa2 Bundle web-ui v3.1.0.dev1 2026-03-07 17:47:44 +08:00
Jeremy Grossmann f81ab0cfbe Merge branch '3.0' into feature/ai-copilot-bridge 2026-03-07 13:00:39 +08:00
Jeremy Grossmann 0cedab1742 Merge pull request #2632 from GNS3/drop-python3.9
Drop support for Python 3.9
2026-03-07 11:19:03 +08:00
Guobin Yue 1cc48c1fb0 Merge branch '3.0' into feature/ai-copilot-bridge 2026-03-07 01:57:08 +08:00
YueGuobin 23b073a7bc docs: add troubleshooting guide for datetime timezone issue
Add comprehensive documentation for handling datetime timezone discrepancies between backend and frontend. The issue arises when backend returns naive datetime strings without timezone suffixes, causing JavaScript to parse them as local time instead of UTC. The guide includes problem description, root cause analysis, three frontend solutions (dayjs UTC parsing, manual 'Z' suffix addition, and global Axios interceptor), backend context, and testing checklist.
2026-03-07 01:42:18 +08:00
grossmj 679c690903 Drop support for Python 3.9 2026-03-06 19:03:53 +08:00
Jeremy Grossmann 09aaf43dd1 Merge pull request #2628 from Raizo62/console
Fix QEMU serial console artifacts by filtering ANSI CPR responses in …
2026-03-06 18:24:34 +08:00
Jeremy Grossmann 5755a48fa3 Merge pull request #2630 from UmmmAGoodName/3.0
Fixed the path traversal check
2026-03-06 18:05:57 +08:00
YueGuobin 3a287de8c1 docs: add ACL Web UI implementation guide for frontend developers
Add comprehensive implementation guide for ACL management feature in GNS3 Web UI. The guide includes:

- Feature overview with core functionality and user flow
- Detailed API endpoint specifications for ACE CRUD operations
- Data structures and relationships
- Frontend implementation guide with component architecture
- UI/UX design recommendations and wireframes
- Common usage scenarios and error handling patterns
- Example code with React components and API service layer

This document serves as a reference for frontend developers implementing ACL management interface, covering everything from API integration to user interface design.
2026-03-06 14:19:19 +08:00
YueGuobin 8660e9c8db feat(docs): add RBAC + ACL implementation guide for GNS3 Server
Add comprehensive documentation for the two-tier permission control system combining RBAC and ACL features. The guide covers system overview, core concepts, data model, permission check flow, usage examples, best practices, and common issues. This documentation is applicable for GNS3 Server v3.0+ and provides implementation details for administrators managing user permissions.
2026-03-06 13:07:15 +08:00
YueGuobin 3d781236c6 delete depreacted dir 2026-03-06 11:08:35 +08:00
YueGuobin 0f61a06557 feat(docs): clarify API key visibility for group configurations
Add detailed section explaining that plaintext API keys for group configurations are never exposed through the application API, even to super admins. This clarifies the security design where group configs are intended for inheritance only, not manual viewing, while maintaining internal decryption for inheritance functionality.
2026-03-06 11:06:18 +08:00
YueGuobin 903e48aecc feat(docs): clarify API key visibility rules and encryption details
Update the LLM model configs API documentation to provide clearer explanations of API key visibility controls and encryption behavior. The changes include:

- Enhanced visibility table with more specific scenarios and encryption states
- Added detailed rules explaining when API keys are visible as plaintext, hidden (null), or shown encrypted
- Clarified super admin capabilities and database-level access
- Updated JSON examples to reflect actual encrypted values and null placeholders
- Added important notes about Fernet encryption, on-the-fly decryption, and group config behavior

These updates provide better transparency about security measures and help users understand what to expect when viewing different types of configurations.
2026-03-06 10:05:17 +08:00
YueGuobin 4941dc13d8 feat(api): enhance /me endpoint with groups, pools, and ACEs
Add comprehensive user context to the `/me` endpoint by including group memberships, accessible resource pools, and access control entries (ACEs). This enables users to view inherited configurations, available resources, and their permissions directly from the API.

Key additions:
- Group membership details with inherited configs
- Resource pool access information
- ACE visibility for permission transparency
- Support for user-selectable group default configurations

The enhancement addresses user needs for better visibility into permissions and accessible resources within the system.
2026-03-06 09:50:34 +08:00
YueGuobin eb7a72f5e2 feat(docs): update copilot_mode value in LLM model configs API example
Changed the example value of `copilot_mode` from "lab_assistant" to "lab_automation_assistant" in the API documentation to reflect the correct mode name used in the system.
2026-03-06 02:00:02 +08:00
YueGuobin 77ef87f468 feat(prompts): rename prompt modules for clarity and consistency
Rename base_prompt.py to teaching_assistant_prompt.py and lab_assistant_prompt.py to lab_automation_assistant_prompt.py to better reflect their purposes. Update all imports and references accordingly to maintain consistency across the codebase. This improves code readability and aligns naming with the actual functionality of each prompt module.
2026-03-06 01:58:54 +08:00
YueGuobin dd3b06bd75 feat(docs): remove reference tool command from model configs API documentation
The `python scripts/show_model_context_limits.py` command was removed from the documentation as it is no longer necessary or relevant for users configuring model context limits. The remaining content still provides clear guidance on calculating context limits in K tokens.
2026-03-06 01:55:46 +08:00
Guobin Yue 87f8f22524 Merge branch '3.0' into feature/ai-copilot-bridge 2026-03-06 01:42:47 +08:00
YueGuobin 216b3d7219 docs: translate AI chat API design document to English
Translate the GNS3 Copilot Agent Chat API design document from Chinese to English to improve accessibility for international contributors and align with project documentation standards. The translation covers all sections including overview, core features, architecture design, API endpoints, and response formats.
2026-03-06 01:14:52 +08:00
YueGuobin 8fa6d6810a feat: add multi-user concurrency control design document
Add comprehensive design document outlining the race condition issue when multiple users simultaneously operate on the same network device via GNS3-Copilot Agent. The document details the problem statement, affected components, and proposes two solutions: device-level mutex lock (recommended) and connection pooling with session isolation. It includes implementation details, API changes, and UI considerations to ensure safe concurrent operations.
2026-03-06 01:08:45 +08:00
YueGuobin af6d87426b feat(agent): add runtime control parameters for agent behavior
Add support for runtime control parameters `max_iterations` and `max_tool_calls` in the chat API to allow users to dynamically adjust agent behavior per request. This addresses current limitations where iteration limits and tool call constraints are hardcoded, providing flexibility for complex tasks and cost control.
2026-03-06 00:54:33 +08:00
YueGuobin c317932f1a feat(security): add forbidden command filtering for device configurations
- Import filter_forbidden_commands utility from command_filter module
- Add _filter_forbidden_commands_from_device_configs method to filter out restricted commands before execution
- Store blocked commands information and log filtered commands for audit purposes
- Update _process_task_results to include blocked commands info in response
- Prevent execution of potentially dangerous commands while maintaining transparency about filtered content
2026-03-06 00:30:17 +08:00
YueGuobin f1b496df49 docs: add troubleshooting guide for force kill residual processes
Add comprehensive documentation for troubleshooting issues caused by using `kill -9` on gns3server processes. The guide explains the root cause where SIGKILL prevents proper cleanup of child processes like dynamips and vpcs, leading to port conflicts and resource allocation errors when restarting. It provides multiple solutions including manual process cleanup, proper shutdown procedures, and preventive measures with example scripts. This documentation helps users resolve common issues with Dynamips VM creation failures, undefined project_id errors, and TCP port warnings.
2026-03-05 23:39:59 +08:00
UmmmAGoodName 6ff4d46838 Fixed the path traversal check 2026-03-05 16:38:55 +01:00
YueGuobin aaba2d6181 docs: update node name from generate_title to title_generator_node
Updated references to the title generation node in both documentation and code:
- Changed node name from `generate_title` to `title_generator_node` in API design documentation
- Updated filtering logic in agent service to exclude `title_generator_node` from LLM call statistics, token counting, and frontend streaming events
- Maintains same functionality while using more descriptive node name for clarity
2026-03-05 23:19:20 +08:00
YueGuobin a7bc68d6e1 feat(docs): reorganize documentation and add orphan tool calls recovery plan
- Move command-security.md and hitl-implementation-plan.md to todo/ directory
- Add new documentation for orphan tool calls recovery with detailed problem analysis and implementation plan
- Document scenarios causing orphan tool calls and propose fix strategy using LangGraph API
2026-03-05 23:07:02 +08:00
YueGuobin b8cb0bdcb0 feat(agent): exclude generate_title node from LLM call and token statistics
Filter out internal LangGraph 'generate_title' node from LLM call counting and token usage tracking to avoid inflating statistics with internal operations. This ensures metrics only reflect user-facing AI interactions.
2026-03-05 23:01:41 +08:00
YueGuobin 511b155da8 feat(ai-chat-api): implement streaming tool calls with incremental parameter accumulation
- Update API documentation to reflect new streaming tool call mechanism
- Add `message_id` optional field to content and tool_call events
- Change tool_call structure from array to single object with incremental updates
- Add `tool_call_id` to tool_start events for better event correlation
- Implement ToolCallStreamAccumulator class to handle parameter accumulation
- Provide frontend example code for handling streaming tool calls
- Maintain backward compatibility with existing session_id tracking
2026-03-05 22:29:01 +08:00
YueGuobin 1de7d00db3 feat(agent): update SSE event schema and handle tool calls from LLM
- Update AI chat API documentation with revised SSE event schema
- Add support for multiple tool calls in `tool_call` events
- Include `session_id` in all event types for better session tracking
- Implement `on_chat_model_end` handler to process LLM tool call decisions
- Update example JSON payloads to reflect new schema structure
2026-03-05 22:03:27 +08:00
YueGuobin 54775faaf4 feat(copilot): rename copilot modes for clarity and add mode-specific tools
- Rename "teaching" mode to "teaching_assistant" for better clarity
- Rename "lab_assistant" mode to "lab_automation_assistant" to reflect expanded capabilities
- Implement mode-specific tool sets: teaching_assistant gets read-only diagnostic tools only, while lab_automation_assistant gets full diagnostic and configuration tools
- Update API documentation examples to reflect new mode names
- Maintain backward compatibility with default tool set initialization
2026-03-05 21:44:36 +08:00
YueGuobin 28f2fe03e6 feat: suppress nornir INFO logs to reduce console verbosity
Set logging level for nornir.core and nornir loggers to WARNING in config_tools_nornir.py and display_tools_nornir.py. This prevents nornir from logging task execution at INFO level to the console, as the logging={"enabled": False} parameter in InitNornir only disables plugin internal logs.
2026-03-05 21:04:46 +08:00
YueGuobin 2485bfec46 docs: add temperature parameter documentation and code formatting improvements
- Added `temperature` parameter to Chat API documentation with implementation notes
- Improved code formatting in context_manager.py with consistent string quotes and line breaks
- Added section on future runtime LLM parameter override capabilities
- Updated API schemas to include temperature parameter (currently unused but reserved for future implementation)
2026-03-05 20:58:14 +08:00
YueGuobin 7f8364fbf5 refactor: move context helpers to separate module for better organization
- Extract context variable management functions from connector_factory.py to new context_helpers.py module
- Update imports in gns3_copilot.py, agent_service.py, and __init__.py to use new module
- Remove inline imports and ensure consistent access to context helpers
- Improves code organization and maintainability by separating concerns
2026-03-05 20:40:16 +08:00
YueGuobin 16559d2ce2 feat(agent): reorganize imports and improve code structure
- Standardize import organization with clear sections (standard library, third-party, local)
- Move sys.path modification to top of imports for better clarity
- Fix circular import issues by reordering imports
- Remove duplicate imports and ensure proper import grouping
- Maintain all existing functionality while improving code readability
2026-03-05 18:06:36 +08:00
YueGuobin 08b5b44a50 feat(copilot): add copilot_mode field to LLM model configs API
- Add `copilot_mode` field to request examples in API documentation
- Update prompt loader to read `copilot_mode` from flattened config structure
- Support both "teaching" and "lab_assistant" modes for different assistant behaviors
2026-03-05 16:41:47 +08:00
YueGuobin a777c36a29 feat(copilot): add copilot_mode field and config command tool
- Add `copilot_mode` field to LLM model configs API with "teaching" (diagnostics only) and "lab_assistant" (full configuration access) modes
- Introduce new `ExecuteMultipleDeviceConfigCommands` tool for executing configuration commands on multiple devices
- Include `tags` field in node data structure for enhanced project management
- Update API documentation examples to reflect new `copilot_mode` field and context limit additions
2026-03-05 16:31:02 +08:00
YueGuobin 8dc8facb13 docs: simplify context window management documentation
Removed extensive implementation details and configuration examples from the context window management documentation. The document now focuses on core permissions and prohibitions for tool usage, providing a clearer and more concise reference for allowed and forbidden actions. This streamlines the documentation to essential guidelines only.
2026-03-05 14:05:58 +08:00
Jeremy Grossmann ea68d8b959 Merge pull request #2629 from GNS3/image-handling
Fix image handling
2026-03-05 12:43:47 +08:00
grossmj 4606613673 Fix image handling 2026-03-05 12:38:28 +08:00
YueGuobin a8044cc1c2 feat: add copyright and author attribution to source files
Add copyright notice and author attribution to multiple Python files in the gns3-copilot module. This ensures proper licensing attribution and clarifies authorship for the project files.
2026-03-05 11:34:00 +08:00
YueGuobin 2779734de4 feat: remove redundant project attribution comments from GNS3-Copilot modules
Removed repetitive "This module is part of the GNS3-Copilot project" and GitHub URL comments from multiple module docstrings. These comments were redundant since the project information is already established in the main package documentation. This cleanup improves code readability and reduces maintenance overhead by eliminating duplicate attribution statements across the codebase.
2026-03-05 11:26:54 +08:00
YueGuobin d12b2df306 feat(docs): update context window management documentation
- Refactor system message structure to combine system prompt and topology info using template variables
- Update token calculation process with merged system message approach
- Clarify priority order for message retention during context window management
- Add detailed token counting implementation using tiktoken library
- Include boundary case handling for system message exceeding budget
- Add SPDX license headers to tool files for proper licensing documentation
2026-03-05 11:23:11 +08:00
YueGuobin ed5fd7d17c feat(agent): inject topology into system prompt via template variable
Refactor context preparation to embed topology information directly into the system prompt using a {{topology_info}} placeholder, replacing the previous approach of appending a separate SystemMessage. This consolidates context into a single system message and simplifies token accounting. The logging is updated to reflect the new token breakdown, showing combined system prompt tokens (base + topology) instead of separate components.
2026-03-05 11:05:51 +08:00
YueGuobin 33df674cbf feat(agent): remove GNS3TopologyTool from available tools list
The GNS3TopologyTool has been removed from the list of available tools for the agent. This change simplifies the toolset by eliminating a tool that is no longer needed or supported in the current workflow.
2026-03-05 10:55:10 +08:00
YueGuobin 7fcdb57615 feat(context-manager): improve context window management with tool token accounting
Enhanced the context window management system to properly account for tool definition tokens when trimming messages. The key changes include:

- Updated `trim_messages_for_context` function to accept `tool_tokens` parameter
- Modified token budget allocation logic to subtract tool tokens before message trimming
- Added detailed documentation explaining the token budget distribution between messages and tool definitions
- Implemented prioritized trimming strategy that preserves system messages and recent conversation history
- Added boundary case handling for scenarios where system messages or tools exceed available budget

The improvements ensure more accurate context window management by accounting for the ~1000-2000 tokens typically consumed by tool definitions that LangChain automatically includes in LLM requests.
2026-03-05 01:50:48 +08:00
YueGuobin f8aa653ef9 feat(context-manager): add tiktoken dependency and improve token counting accuracy
- Add tiktoken as a required dependency for accurate token counting
- Update documentation with installation instructions and token counting strategy
- Improve logging to include tool definition token estimates
- Enhance error handling to fail fast when tiktoken is not available
- Update context manager to use tiktoken's cl100k_base encoding for GPT-4 compatibility
2026-03-05 01:01:34 +08:00
YueGuobin 7368ac098a docs: add context limit and strategy to LLM model configs API
- Add `context_limit` as required field for LLM model configurations
- Add `context_strategy` as optional field with three trimming strategies
- Update API documentation with detailed examples for GPT-4o and Claude 3.5 Sonnet
- Clarify that context limit is specified in K tokens (thousands of tokens)
- Update example payloads to reflect current model versions and new fields
2026-03-05 00:40:46 +08:00
YueGuobin 6a9f745c13 feat(copilot): add HITL confirmation for sensitive operations
- Move AI chat API design document to gns3-copilot directory
- Add comprehensive HITL implementation plan with architecture and workflow
- Introduce human-in-the-loop confirmation for dangerous configuration commands
- Include state management, API endpoints, and frontend integration details
- Provide testing plan and deployment steps for the new feature
2026-03-04 23:53:50 +08:00
YueGuobin 4e28adffb6 feat(copilot): add tool response normalization utility
Add `normalize_tool_response` function to standardize tool output formats for consistent frontend display. The function converts various response types (dict, list, string) into a unified structure with success metrics, detailed data arrays, and metadata. This ensures backward compatibility while providing predictable response formats for UI components.
2026-03-04 23:41:29 +08:00
YueGuobin f64daa96cd feat(agent): streamline conversation flow logic in copilot
- Remove unused `llm_calls` variable and redundant logging in `should_continue`
- Simplify title generation routing by eliminating unnecessary condition checks
- Improve code clarity and maintainability by focusing on essential flow control
2026-03-04 23:34:14 +08:00
YueGuobin 4a017b1a7f feat(chat): add session pinning feature with database migration
- Add `pinned` column to chat_sessions table with default FALSE
- Implement database migration for existing installations using PRAGMA table_info
- Create composite index for pinned + updated_at sorting
- Add pin/unpin API endpoints (PUT/DELETE /sessions/{id}/pin)
- Update session listing to sort by pinned status then updated_at
- Extend ChatSessionsRepository with pin_session method
- Update API documentation to reflect new pinning functionality

The feature allows users to pin important chat sessions to the top of the list. Sessions are sorted with pinned sessions first (by updated_at), followed by regular sessions (by updated_at). Database migration ensures backward compatibility with existing installations.
2026-03-04 23:18:30 +08:00
YueGuobin f688a2d5c0 feat(agent): enhance message handling with ID generation and format conversion
- Add message ID generation for initial HumanMessage creation
- Implement message converters for LangChain/OpenAI format interoperability
- Update documentation with detailed message format specifications
- Refactor AgentService to use centralized message conversion utilities
- Ensure tool_calls format compliance with OpenAI API standards
2026-03-04 23:05:43 +08:00
YueGuobin 6b73f00281 docs: update AI chat API design with detailed statistics collection
Update the AI chat API design documentation to provide comprehensive details about statistics collection during streaming conversations. The documentation now includes:

1. **Enhanced statistics collection logic**:
   - Clarified message_count increments for user messages, AI responses, and tool results
   - Added LLM call tracking via on_chat_model_start events
   - Detailed token counting methodology using LangGraph's usage_metadata

2. **Improved implementation details**:
   - Added specific event handlers for statistics collection
   - Explained LangGraph's cumulative token counting behavior
   - Provided real-world examples of token accumulation

3. **Updated data models**:
   - Enhanced ChatSession model documentation with field descriptions
   - Separated fields into categories (basic, statistics, timestamps, reserved)

4. **Refined architecture documentation**:
   - Added detailed flow for stream_chat method
   - Documented statistics collection mechanism during SSE streaming
   - Explained batch update strategy to reduce database writes

The changes ensure developers understand how conversation statistics are collected, processed, and stored without impacting streaming performance.
2026-03-04 22:33:28 +08:00
YueGuobin 48e2793719 feat(agent): initialize checkpointer before chat session retrieval
Add initialization check for `_checkpointer_conn` in `AgentService` to ensure the checkpointer database connection exists before attempting to retrieve chat sessions. This prevents potential null reference errors when the checkpointer hasn't been initialized yet during agent operations.
2026-03-04 22:09:09 +08:00
YueGuobin 03ab9cdf6c feat(chat-api): refactor design document with concise architecture overview
- Replace detailed implementation plan with high-level architecture design
- Focus on core features: project isolation, streaming responses, session management
- Remove FlowNet-Lab reference and implementation specifics
- Streamline document from 1172 to 483 lines for better maintainability
2026-03-04 21:58:23 +08:00
Raizo62 590eedeb46 Fix QEMU serial console artifacts by filtering ANSI CPR responses in telnet proxy
- filter incoming cursor position report sequences (`ESC[<row>;<col>R`) in `AsyncioTelnetServer`
- prevent stray `;1R;80R` characters on late telnet console attach after VM boot
- keep QEMU serial transport unchanged (`-serial telnet`) for BIOS compatibility
2026-03-04 14:26:52 +01:00
Jeremy Grossmann 2843e05e01 Merge pull request #2627 from GNS3/bugfix/2622
Qemu VMs: only compute MD5 checksums for existing disks
2026-03-04 18:57:52 +08:00
grossmj 8b245c96f8 Qemu VMs: only compute MD5 checksums for existing disks 2026-03-04 18:54:52 +08:00
YueGuobin b6c343c5b8 feat(agent): add GNS3UpdateNodeNameTool to copilot tools
Introduce a new tool for updating node names within the GNS3 copilot agent. This expands the agent's capabilities to include node name modifications alongside existing node and link management operations.
2026-03-04 17:55:16 +08:00
YueGuobin 0648b53a45 feat(gns3_copilot): enhance topology logging with structured details
Improve logging in GNS3 topology reader to provide more structured and informative output. The previous single-line JSON log has been replaced with separate info and debug logs:
- Info log now includes project ID, project name, node count, and link count
- Debug log retains the full topology details for deeper inspection
This makes it easier to monitor topology retrieval in production while keeping detailed data available for debugging.
2026-03-04 17:43:41 +08:00
YueGuobin ef712ad314 feat(copilot): simplify project context handling and remove mode parameter
- Remove `selected_project` tuple from MessagesState as it's no longer needed
- Replace tuple-based project selection with direct `project_id` from config
- Remove unused `mode` parameter from llm_call function
- Update topology retrieval to use project_id directly from configurable settings
- Streamline context messages by removing redundant project info formatting
- Move project_id from metadata to configurable section in agent_service config
2026-03-04 17:40:17 +08:00
grossmj 5b00529666 Copy config files and database from a previous version if it exists 2026-03-04 17:23:10 +08:00
Jeremy Grossmann a20e10f75a Merge pull request #2625 from markparonyan/3.0
feat(compute): option to disable compute authentication
2026-03-04 16:40:58 +08:00
grossmj 7010baccee Add enable_http_auth in gns3_server.conf sample 2026-03-04 16:35:26 +08:00
YueGuobin 4fa29f3bee feat(agent): add GNS3ProjectInfoTool to gns3_client exports
Add GNS3ProjectInfoTool to the __all__ list in gns3_client/__init__.py to make it available for import. This ensures the tool is properly exposed as part of the public API for use in copilot agent modules.
2026-03-04 16:06:13 +08:00
YueGuobin a278a6091c feat(agent): add comprehensive logging to LLM and tool execution nodes
- Add info and debug logging to llm_call node for tracking LLM invocations and configuration
- Add error handling and logging to tool_node for tool execution failures
- Add startup logging to stream_chat method with session details
- Improve observability of agent workflow and debugging capabilities
2026-03-04 15:46:12 +08:00
YueGuobin 1e9cf3fcbb feat(agent): make JWT token optional in GNS3 connector factory
The `get_gns3_connector` function now accepts an optional `jwt_token` parameter. If not provided, the token will be retrieved from context, improving flexibility for scenarios where authentication is handled externally or deferred.
2026-03-04 15:38:50 +08:00
YueGuobin 289ae0cddf feat(agent): remove redundant done message from stream_chat
The `stream_chat` method was yielding a "done" message after streaming all chunks, but this is unnecessary as the streaming completion is already indicated by the end of the stream. Removing this redundant message simplifies the response handling and aligns with typical streaming patterns.
2026-03-04 13:57:03 +08:00
YueGuobin 425ce43bdd feat(chat): add project status checks to chat endpoints
Add project status validation to all chat API endpoints to ensure the project is opened before allowing chat operations. This prevents unauthorized access and ensures chat functionality only works with active projects.

- Check project.status == "opened" in stream_chat, list_sessions, get_history, and delete_session endpoints
- Return HTTP 403 FORBIDDEN with descriptive error message if project is not opened
- Update docstring for stream_chat endpoint to document the requirement
2026-03-04 13:55:09 +08:00
YueGuobin eec4ebe3fd feat(copilot): replace LangGraph config with request-scoped context variables
- Refactor `llm_call` and `generate_title` nodes to retrieve `llm_config` from request-scoped context variable instead of LangGraph config
- Remove `jwt_token` and `llm_config` from LangGraph configurable parameters in `AgentService.stream`
- Add `set_current_llm_config` and `get_current_llm_config` functions to `connector_factory` and export them in `__init__.py`
- Update `tool_node` to no longer extract `jwt_token` from config as it is now handled via context variable
- Improves thread safety and decouples configuration from LangGraph's state management
2026-03-04 13:49:17 +08:00
YueGuobin b05e6a71b4 feat(copilot): refactor JWT token handling and improve metadata tracking
- Move JWT token from state to configurable context for better security and request isolation
- Add user_id parameter to agent service for enhanced metadata tracking
- Update checkpoint directory name from .gns3-copilot to gns3-copilot
- Implement context-aware JWT token management using ContextVar
- Improve tool node to extract JWT token from config instead of state
2026-03-04 13:36:16 +08:00
YueGuobin 5e9aac7514 feat(agent): handle AIMessageChunk content access in copilot streaming
Update agent service to properly access content from AIMessageChunk objects during chat model streaming. Instead of using dictionary get method on the chunk, now use getattr to directly access the content attribute, ensuring compatibility with the AIMessageChunk object structure.
2026-03-04 13:23:07 +08:00
YueGuobin b780bfaf53 feat(agent): refactor LLM configuration handling to use centralized config
- Replace separate user_id and jwt_token parameters with unified llm_config dict
- Simplify model factory to accept llm_config directly instead of fetching from API
- Update llm_call and generate_title nodes to extract llm_config from LangGraph config
- Remove deprecated API fetching logic from model factory
- Maintain backward compatibility for existing tool usage patterns

This change centralizes LLM configuration management, reducing API calls and improving performance by passing configuration directly from the API layer rather than fetching it repeatedly.
2026-03-04 13:13:54 +08:00
YueGuobin 4765e54d7a feat: improve async handling in LLM config retrieval
Refactor get_llm_config to handle various async contexts more robustly. The function now properly checks for running event loops using asyncio.get_running_loop() and handles edge cases when called from thread pools or existing loops. This prevents RuntimeErrors and ensures reliable execution in both sync and async environments.
2026-03-04 12:47:37 +08:00
YueGuobin d8c3a56503 feat(agent): remove environment variable fallback for LLM configuration
Remove support for environment variable fallback in LLM model configuration. The configuration now strictly follows:
1. Provided llm_config dictionary (highest priority)
2. Fetch from llm_model_configs system via connector_factory (requires user_id and jwt_token)

This change ensures consistent configuration management and eliminates the outdated environment variable approach. When no configuration is found, a clear ValueError is raised with appropriate error messages.
2026-03-04 12:45:09 +08:00
YueGuobin 2df05dff6e feat(api): nest chat endpoints under projects and refactor dependencies
- Change chat router prefix from `/chat` to `/projects/{project_id}/chat`
- Add `dep_project` dependency to inject Project instance into endpoints
- Remove manual project validation in `stream_chat` and use dependency instead
- Add placeholder `list_sessions` endpoint for future session listing
- Update `get_history` endpoint to use project dependency and adjust path
- Improve code organization and error handling for project retrieval
2026-03-04 12:43:36 +08:00
YueGuobin 479afc01b0 feat(api): add Query parameter description for chat session listing
Add explicit Query parameter with description to the `list_sessions` endpoint for better API documentation and clarity. The `project_id` parameter now includes a descriptive label indicating it is a GNS3 project ID.
2026-03-04 12:35:38 +08:00
YueGuobin 2fe30c69e6 feat(agent): update config parameter type to RunnableConfig in copilot nodes
- Change config parameter type from dict to RunnableConfig | None in llm_call and generate_title functions
- This improves type safety and aligns with LangChain's RunnableConfig usage
- No functional changes, only type annotations updated for better integration
2026-03-04 12:29:31 +08:00
YueGuobin dde2a00c5b feat(agent): update import paths for gns3_copilot modules
Updated import statements across multiple agent files to use absolute paths starting with `gns3server.agent.gns3_copilot` instead of relative `gns3_copilot` imports. This ensures proper module resolution within the gns3-server package structure and prevents import errors when the agent is executed from different contexts.
2026-03-04 12:27:50 +08:00
YueGuobin 73de248381 feat(copilot): add user-aware LLM calls and project cleanup
- Modified `llm_call` and `generate_title` functions to accept `config` parameter, extracting `user_id` and `jwt_token` for per-user LLM configuration and API authentication
- Updated `create_base_model_with_tools` and `create_title_model` calls to pass user authentication details
- Added `jwt_token` to state for tool usage in GNS3 API calls
- Integrated chat router into controller API routes under `/chat` endpoint
- Implemented `_cleanup_copilot_agent` method in `Project` class to remove AgentService resources upon project closure, preventing resource leaks
- Enhanced error handling in agent cleanup to avoid interrupting project close operations
2026-03-04 12:26:19 +08:00
YueGuobin e1f1bb7d9f feat(api): add AI chat API design document for GNS3 Copilot integration
This commit introduces a comprehensive design document outlining the implementation of AI Chat API for GNS3 Copilot Agent within GNS3 Server. The document provides:

- Overview and background on existing components including GNS3 Copilot Agent, LLM configuration management, and API framework
- Reference implementation details from FlowNet-Lab project
- Architecture design with clear component interactions between frontend clients and backend services
- RESTful API specifications for chat streaming, session management, and history retrieval
- Implementation details covering project-based agent management, SQLite checkpoint storage, and LangGraph integration
- Security considerations and deployment guidelines

The design enables clients to interact with GNS3 Copilot Agent through standardized APIs, supporting real-time chat streaming and persistent conversation sessions per project.
2026-03-04 12:08:47 +08:00
YueGuobin a1466dad0d feat(gns3_copilot): remove GNS3UpdateDrawingTool from client exports
Removed GNS3UpdateDrawingTool import and from __all__ list in gns3_client __init__.py to clean up the public interface and eliminate unused or deprecated components.
2026-03-04 01:17:20 +08:00
YueGuobin d525e5cad0 feat: rename FlowNet-Lab to GNS3-Copilot across codebase
Update all references from FlowNet-Lab to GNS3-Copilot in package names, documentation, and logging. This includes:
- Module and package __init__.py files
- License headers and file descriptions
- Log messages and internal comments
- Remove deprecated tools: GNS3CreateAreaDrawingTool and LinuxTelnetBatchTool

The renaming aligns with the project's new branding while maintaining all existing functionality.
2026-03-04 00:55:29 +08:00
YueGuobin 95b309c7af feat: add GNS3 Copilot AI agent dependencies to requirements
Add comprehensive dependencies for the new GNS3 Copilot AI agent feature. This includes:
- Core AI and automation frameworks (LangChain, LangGraph)
- Multiple model provider integrations (OpenAI, Anthropic, Google, AWS, Ollama, DeepSeek, xAI)
- Network automation tools (Netmiko, Nornir)
- Supporting libraries for telnet, HTTP requests, authentication, and image processing

The dependencies are organized into a dedicated section with clear comments for maintainability.
2026-03-04 00:49:46 +08:00
YueGuobin db9db56e7a feat(agent): enhance LLM config loading with priority system
Refactor model factory to support three configuration sources in priority order:
1. Direct llm_config dictionary parameter
2. Fetch from llm_model_configs system via connector_factory (requires user_id and jwt_token)
3. Environment variables as fallback for backward compatibility

This improves flexibility by allowing runtime configuration while maintaining compatibility with existing environment-based setups.
2026-03-04 00:47:35 +08:00
YueGuobin 15dc1e7ea0 feat(agent): remove checkpoint utilities from public API
Removed checkpoint-related imports and exports from the agent package's __init__.py to clean up the public API. This change reduces unnecessary exposure of internal checkpoint utility functions, focusing the public interface on the core agent_builder functionality.
2026-03-04 00:35:15 +08:00
YueGuobin cabcf4f1cb feat(gns3-copilot): add get_gns3_server_host utility and integrate into Nornir tools
- Add `get_gns3_server_host()` function to `connector_factory.py` for extracting GNS3 server hostname from controller, config, or default URL
- Export new function in `__init__.py` for public API access
- Replace `os.getenv("GNS3_SERVER_HOST", "127.0.0.1")` calls with `get_gns3_server_host()` in Nornir configuration tools (`config_tools_nornir.py`, `display_tools_nornir.py`)
- Ensures consistent host detection across tools using the same priority logic as `get_gns3_connector`
2026-03-04 00:28:52 +08:00
YueGuobin 8917ef39c9 feat: remove environment variable usage for Nornir credentials
Removed GNS3_SERVER_USERNAME and GNS3_SERVER_PASSWORD environment variables from Nornir configuration tools. Credentials are now set to empty strings by default, simplifying configuration and removing dependency on environment variables for authentication.
2026-03-04 00:22:47 +08:00
YueGuobin 983ed32663 feat(copilot): refactor prompts module for GNS3 Copilot
- Rename module description from FlowNet-Lab to GNS3 Copilot
- Replace linux_specialist_prompt and experiment_deploy_prompt imports with base_prompt
- Update __all__ export list to include SYSTEM_PROMPT instead of removed prompts
- Improve module docstring with detailed component descriptions and available prompts
- Maintain dynamic version management and module metadata
2026-03-03 23:31:38 +08:00
YueGuobin 4857cff59c feat(agent): refactor LLM configuration to support new llm_model_configs system
- Remove direct logging of LLM config from gns3_copilot.py
- Update model_factory to accept configuration from llm_model_configs dictionary
- Add fallback to environment variables for backward compatibility
- Centralize configuration loading in _load_llm_config function
2026-03-03 23:26:20 +08:00
YueGuobin 3c54a7d90a feat(gns3-copilot): add factory function to create GNS3 connector with LLM config
Introduce `get_gns3_connector_with_llm_config` as a convenience function that combines the creation of a GNS3 API connector and retrieval of the user's default LLM configuration. This simplifies initialization for operations requiring both GNS3 connectivity and AI model settings, reducing boilerplate code in callers. The function returns a dictionary containing the connector and LLM config, or None on failure.
2026-03-03 23:19:00 +08:00
YueGuobin 46b262a02c feat(agent): add gns3-copilot AI assistant integration module
Integrate the gns3-copilot AI assistant module to provide intelligent
   automation and interaction capabilities for GNS3 network emulation.

   Key components:
   - AI agent framework with LLM integration (supports Qwen vision model)
   - GNS3 client library for project topology management
   - Extensive prompt templates for various network operation scenarios
   - Tool library for node creation, linking, configuration, and management
   - Support for English level assessment (A1-C2) and specialized personas
   - Network drawing and topology visualization tools
   - Linux device automation via Nornir/Telnetlib
   - Window controller for UI interaction

   Features:
   - Multi-modal AI agent with vision capabilities
   - Automated network topology deployment and configuration
   - Interactive node and drawing management
   - File-based project operations (read, write, list)
   - Specialized prompts for different scenarios and skill levels
   - Comprehensive tool set for network device management
2026-03-03 23:08:07 +08:00
YueGuobin 7a2d15cb64 feat: clarify default LLM model config selection logic
Updated documentation and implementation to clearly define the priority order for selecting default LLM model configurations. The logic now explicitly states:
1. User's config marked with `is_default: true` (highest priority)
2. Group's config marked with `is_default: true`
3. First config in the list (user configs come before group configs)

This ensures consistent behavior between the API documentation and the actual implementation in the repository code.
2026-03-03 22:56:21 +08:00
YueGuobin f4723cd083 docs: restructure LLM model configs API documentation for nested config object
Update API documentation to reflect new nested `config` object structure in LLM model configurations endpoints. The response format now encapsulates provider-specific fields (provider, base_url, model, temperature, api_key, max_tokens) within a `config` object, while moving ownership metadata (user_id, group_id, is_default, version, created_at, updated_at) to the top level. This aligns the user-facing endpoints with the group configuration structure and improves API consistency.
2026-03-03 22:47:36 +08:00
YueGuobin 176b39145b docs: update LLM model configs API documentation for group endpoints
- Add note clarifying that GET endpoints for groups return the same structure as user endpoints
- Document LLMModelConfigListResponse schema with default configuration selection logic
- Add comprehensive example for GET group configurations endpoint
- Update endpoint numbering to accommodate new group endpoints
- Ensure consistency between user and group API documentation
2026-03-03 22:39:38 +08:00
YueGuobin 2e4c42fadc feat(api): clarify default LLM model config selection logic
- Update API documentation to explicitly describe default configuration selection logic
- Add fallback behavior in repository to use first config when no default is marked
- Clarify difference between `/default` endpoint and `default_config` field
- Document that `default_config` is never null when configs list is not empty
2026-03-03 22:27:28 +08:00
YueGuobin b06593cf2a feat(docs): add default config endpoints and clarify config inheritance
- Add `/default` endpoints for users and groups to retrieve default LLM model configurations
- Update documentation to clarify that users receive both own and inherited configurations
- Improve response examples to show combined configs with source field
- Add 404 response example for missing default configurations
- Fix optimistic locking documentation formatting
2026-03-03 22:15:17 +08:00
YueGuobin 8fb4bf1117 feat(db): manually set timestamps for SQLite compatibility
Add explicit datetime.utcnow() assignments for created_at and updated_at fields in LLMModelConfigsRepository to ensure proper timestamp handling with SQLite. This addresses issues where SQLite may not automatically populate these timestamp fields during create and update operations. Changes applied to both user and group configuration methods for consistency.
2026-03-03 21:35:26 +08:00
YueGuobin aef337e86f feat(config): load secret files even without main config
When no configuration file is found or readable, the server now still loads secret files. This ensures that secret configurations (like API keys or passwords) are available even if the primary config file is missing, improving robustness and security in environments where secrets are stored separately.
2026-03-03 18:16:37 +08:00
YueGuobin 8e43ad4912 feat(db): replace PostgreSQL-specific JSONB with cross-database JSON type
- Change `JSONB` columns to generic `JSON` in LLMModelConfig model for database compatibility
- Update migration to use `sa.String(32)` for UUID and `sa.JSON()` for config fields
- Remove PostgreSQL dialect imports to support multiple database backends
- Keep existing constraints and indexes with PostgreSQL-specific annotations where needed
2026-03-03 18:03:16 +08:00
YueGuobin 3632641127 feat(db): update migration dependency to correct previous revision
The migration file `20260303_create_llm_model_configs_table.py` had an incorrect `down_revision` reference. Changed from `7ceeddd9c9a8` to `98083573d011` to properly link to the previous migration in the Alembic history. This ensures the migration chain is consistent and rollbacks work as expected.
2026-03-03 17:58:48 +08:00
YueGuobin fc7275878e feat(db): replace deferred unique constraints with partial unique indexes for LLM model configs
Replace deferred UniqueConstraints with partial unique indexes for user and group default configurations in the LLM model configs table. This change improves performance and ensures at most one default config per user/group while maintaining data integrity. The migration script has been updated accordingly to create and drop the new indexes.
2026-03-03 17:57:32 +08:00
YueGuobin d42fff49a5 feat(docs): enhance LLM model configs API documentation with schema updates
- Add `model_type` field to database schema with supported values (text, vision, stt, tts, multimodal, embedding, reranking, other)
- Add `name` field as table-level column for indexing and filtering
- Add reserved JSONB fields for future extensibility
- Update API request/response schemas to include `model_type` and `name` fields
- Add new `LLMModelConfigWithSource` schema for detailed configuration responses
- Update usage examples to reflect new required fields
- Improve database constraints and indexing documentation
2026-03-03 17:54:30 +08:00
YueGuobin b968920a75 feat(docs): add optimistic locking documentation for LLM model configs API
Add comprehensive documentation for optimistic locking implementation in the LLM model configurations API. The update includes:

- Added "Optimistic Locking" feature to the overview section
- Added `version` field to the database schema table
- Updated `LLMModelConfigUpdate` schema to include `expected_version` parameter
- Updated `LLMModelConfigResponse` schema to include `version` field
- Added detailed examples for update operations with optimistic locking
- Included conflict handling workflow and client retry instructions

This documentation ensures users understand how to prevent concurrent modification conflicts when updating LLM model configurations.
2026-03-03 17:39:04 +08:00
YueGuobin 161b7feb01 feat(api): add optimistic locking to LLM model config updates
- Include `version` field in all LLM model config response schemas
- Add `expected_version` parameter to update endpoints for optimistic locking
- Handle concurrent modification errors with HTTP 409 Conflict status
- Update both user and group config endpoints consistently
2026-03-03 16:59:12 +08:00
YueGuobin 9315f06e1f feat(config): add encryption key loading for sensitive data
Add a new method `_load_encryption_key` to initialize encryption for sensitive data such as API keys. This method is called within `_load_secret_files` alongside JWT secret key loading, ensuring encryption is set up during server configuration. The change enhances security by enabling encryption for secrets stored in the server's secrets directory.
2026-03-03 16:46:23 +08:00
YueGuobin 8fad27d904 feat(db): improve database initialization and migration handling
- Add smarter database state detection during initialization to handle new databases, existing databases with new features, and old databases needing migration
- Make migration for llm_model_configs table idempotent to prevent conflicts when table already exists from code
- Add detailed logging for different database initialization scenarios
- Import sqlalchemy module for database inspection capabilities
2026-03-03 16:40:57 +08:00
YueGuobin bbd34f1f67 feat(api): add LLM model configurations endpoint and schemas
- Introduce new API route `/access` for managing LLM model configurations
- Add LLMModelConfig model to database models
- Include comprehensive schemas for LLM model config CRUD operations
- Register new router with tags for LLM Model Configurations
2026-03-03 16:34:48 +08:00
Mark Paronyan c48c89d309 tests(compute): test compute works with disabled auth 2026-03-03 00:30:00 +03:00
Mark Paronyan 8c3aaa78f5 feat(compute): option to disable compute authentication 2026-03-03 00:28:57 +03:00
Jeremy Grossmann d3710e8e9a Merge pull request #2621 from markparonyan/issue-2521
handle None port name to prevent TypeError in short_name method
2026-03-01 12:31:17 +08:00
Mark Paronyan 0da1a6639a tests(controller,port): short_name finishes successfully when port name is None 2026-02-28 21:45:08 +03:00
Mark Paronyan 729eb43115 fix(controller,port): handle None port name to prevent TypeError in short_name method 2026-02-28 21:45:00 +03:00
Jeremy Grossmann dce4a71004 Merge pull request #2619 from yueguobin/fix/docker-container-name-conflict
fix(docker): handle container name conflict automatically
2026-02-28 12:56:24 +08:00
grossmj 151feb369f Development on 3.1.0.dev1 2026-02-28 11:35:46 +08:00
YueGuobin ce90b2b92c fix(docker): handle container name conflict automatically
When a Docker container with the same name already exists (e.g., from a
   previous crashed GNS3 session), Docker returns a 409 Conflict error
   when trying to create a new container with that name. This causes the
   project open operation to fail.

   This fix adds automatic cleanup of stale containers when encountering
   a name conflict:
   - Added DockerHttp409Error exception class
   - Updated http_query to detect 409 status codes
   - Modified create() to remove conflicting containers and retry

   Fixes the issue where opening a project fails with:
   "Docker has returned an error: 409 Conflict. The container name
   '/GNS3.xxx' is already in use by container 'xxx'"
2026-02-27 23:35:11 +08:00
Jeremy Grossmann a0116902a6 Merge pull request #2618 from yueguobin/fix-port-nonename-error
feat(virtualbox): add fallback for VBoxManage executable detection
2026-02-27 00:39:31 +08:00
YueGuobin a343ace637 Revert: restore original port name handling logic 2026-02-27 00:12:30 +08:00
YueGuobin fe73d0e935 Merge branch 'fix-vboxmanage-case-sensitivity' into fix-port-nonename-error 2026-02-26 10:02:43 +08:00
YueGuobin c55fd9e066 feat(virtualbox): add fallback for VBoxManage executable detection
Add an additional fallback check for the VBoxManage executable using the capitalized "VBoxManage" command name. This improves compatibility on systems where the executable may be case-sensitive or installed with a different naming convention, ensuring the VirtualBox manager can locate the necessary binary when the lowercase "vboxmanage" is not found.
2026-02-26 09:57:51 +08:00
YueGuobin cde4bdc114 feat: simplify port name assignment logic
Refactor port name assignment to use `or` operator for fallback values, improving code readability and consistency. Changes applied in `node.py` and `port_factory.py` to handle custom adapter settings more cleanly.
2026-02-25 22:41:22 +08:00
YueGuobin f6777064db feat(ports): add null checks for port name in short name generation
Add conditional checks to ensure `self._name` is not None before performing string operations in the `short_name` property. This prevents potential AttributeError exceptions when port name is undefined, improving robustness and compatibility with edge cases in port configuration.
2026-02-25 22:21:05 +08:00
grossmj 4c6d40fd4b Upgrade httpx_ws 2026-02-25 19:20:18 +08:00
grossmj 4a2c1b7b23 Fix installing pytest-asyncio with Python 3.9 2026-02-25 19:06:56 +08:00
grossmj 47a7324eb5 Fix tests 2026-02-25 19:00:57 +08:00
Jeremy Grossmann 4519130a8c Merge pull request #2614 from yueguobin/3.0
Implement tags for nodes and templates
2026-02-25 00:33:15 +08:00
grossmj ef7c271a9a Development on 3.0.7.dev1 2026-02-24 21:32:36 +08:00
grossmj 4d85e517b4 Development on 2.2.57.dev2 2026-02-24 21:21:03 +08:00
grossmj ddacd956b2 Merge remote-tracking branch 'yueguobin/3.0' into fork/yueguobin/3.0 2026-02-23 23:32:15 +08:00
grossmj 547e68fd2b Remove unneeded rbac_repo in get_templates 2026-02-23 23:31:35 +08:00
Jeremy Grossmann 11e36cb32c Merge branch '3.0' into 3.0 2026-02-23 23:28:47 +08:00
grossmj 4d35477e46 Add/fix tests 2026-02-23 23:27:36 +08:00
grossmj a9525503c6 Set up database schema migration 2026-02-23 23:27:10 +08:00
grossmj bdcb6445c7 Use lists for tags instead of dicts 2026-02-23 23:26:42 +08:00
Jeremy Grossmann c49aa761ab Merge pull request #2617 from GNS3/bugfix/2560
Fix for resizing Qemu VMs disks
2026-02-23 17:18:57 +08:00
Jeremy Grossmann c88ac0c595 Merge branch '3.0' into bugfix/2560 2026-02-23 17:14:38 +08:00
grossmj 8f470fbe55 Update FastAPI dependency to support Python 3.9 2026-02-23 17:14:16 +08:00
grossmj c66e851498 Fix for resizing Qemu VM disks 2026-02-23 17:10:15 +08:00
grossmj a6043f0fc6 Upgrade dependencies 2026-02-23 16:48:26 +08:00
Guobin Yue 66f32cecf2 Merge branch 'GNS3:3.0' into 3.0 2026-02-23 01:37:05 +08:00
YueGuobin c05f667f3d feat(templates): safely access settings dict keys
Use dict.get() method to safely access 'image' and 'path' keys in settings dictionary to avoid KeyError exceptions when these keys are missing. This improves error handling and prevents server crashes when creating templates with incomplete settings.
2026-02-23 01:02:32 +08:00
YueGuobin e27ddf2cdf feat(api): fix ETag handling and JSON serialization in template endpoint
- Use `default=str` in `json.dumps` to handle non-serializable objects
- Return proper HTTP 304 response with ETag header instead of raising exception
- Ensure consistent ETag generation for template caching
2026-02-22 23:43:47 +08:00
YueGuobin a3616e6d33 feat(api): add tag filtering to nodes and templates endpoints
- Add optional `tags` query parameter to `/nodes` and `/templates` endpoints
- Support filtering by tags in format "key:value" with multiple tags ANDed together
- Example: `?tags=vendor:cisco&tags=model:7200` filters nodes/templates with both tags
- Maintain backward compatibility for existing API usage without tags parameter
2026-02-22 22:52:28 +08:00
Jeremy Grossmann 22c6b4b961 Merge pull request #2616 from StupidTreee/feature/project-created-by
Add 'created_by' field to track project creator
2026-02-22 18:33:54 +08:00
grossmj be4b87e8cd Update project schema and fix tests 2026-02-22 18:13:08 +08:00
Felix Schulz 8ab34924b5 Add 'created_by' field to track project creator 2026-02-21 22:15:21 +01:00
Jeremy Grossmann 1958deff09 Merge pull request #2615 from GNS3/bugfix/2532
Fix updating packet filters
2026-02-21 21:45:12 +08:00
grossmj 5e3ca29c58 Fix updating packet filters 2026-02-21 21:42:22 +08:00
grossmj 60abbade27 Revert merged changes from 2.2 introduced by PR #2607 2026-02-21 21:35:12 +08:00
YueGuobinandClaude Sonnet 4.5 56ece7ce88 Add vendor, model and netmiko_device_type fields to templates
Add three new optional fields to the template schema:
- vendor: Device vendor (e.g., Cisco, Juniper, Huawei)
- model: Device model (e.g., ISR4451-X, MX204, NE40E)
- netmiko_device_type: Netmiko device type for automation (e.g., cisco_ios, juniper_junos, huawei)

These fields enable AI/LLM integration to determine the appropriate
Netmiko device type when automating network device configuration.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-20 23:38:00 +08:00
grossmj c5b9f379fe Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	Dockerfile
#	dev-requirements.txt
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/version.py
#	requirements.txt
2026-02-20 16:28:53 +08:00
Jeremy Grossmann 3b95859b79 Merge pull request #2612 from dalearbo/fix/busybox-musl-ldd-check
fix: busybox static link detection on Alpine/musl
2026-02-20 16:01:01 +08:00
Jeremy Grossmann 126a4099e6 Merge pull request #2613 from dalearbo/fix/emit-threadpool-runtime-error
fix: handle RuntimeError in notification_manager emit()
2026-02-20 15:59:01 +08:00
grossmj 47cc827c1f Remove some files not related to PR 2026-02-20 15:56:08 +08:00
grossmj 715b7032bc Fix tests 2026-02-20 15:51:49 +08:00
Jeremy Grossmann bc1729b255 Merge branch '2.2' into fix/busybox-musl-ldd-check 2026-02-20 15:24:54 +08:00
Dale Arbogast 279f05a14a fix: handle RuntimeError in notification_manager emit()
asyncio.get_running_loop() raises RuntimeError when emit() is called
from a thread without an active event loop (e.g. when the disk space
check in project_manager triggers a log.warning notification from a
threadpool worker).

This causes a 500 Internal Server Error on project creation/open when
disk usage is high, instead of the intended warning notification.

Wrap the emit loop in try/except RuntimeError so that notifications
are silently dropped when no event loop is available, rather than
crashing the request.

Fixes #2502
Fixes #2505
2026-02-19 16:49:50 -05:00
Dale Arbogast 3dd4184193 fix: busybox static link detection on Alpine/musl
On musl-based systems (Alpine), ldd returns exit code 0 for static
binaries, unlike glibc which returns 1. This causes install_busybox()
to reject all busybox binaries as "dynamically linked" on Alpine.

Fix by also accepting binaries whose executable name contains "static"
(i.e. busybox-static, busybox.static), which are the first two
candidates checked by the function. The generic "busybox" fallback
still relies on the ldd return code check.
2026-02-19 16:44:01 -05:00
grossmj 013ec341b3 Fix tests 2026-02-16 19:32:47 +08:00
grossmj bfed330ec2 Fix bug when updating compute password to be empty 2026-02-16 19:23:46 +08:00
grossmj 0a118dc9e7 Cancel tasks if controller cannot be started 2026-02-16 19:16:06 +08:00
grossmj 28e7c014c8 Enable parallel compression with zstandard. Fixes #2324 2026-02-16 18:28:11 +08:00
Jeremy Grossmann 2b4a05eedd Merge pull request #2608 from RishikesavanRamesh/patch-1
Add dynamips to Dockerfile dependencies
2026-02-12 21:13:51 +08:00
Jeremy Grossmann 898a9ce208 Merge branch 'master' into patch-1 2026-02-12 21:13:38 +08:00
grossmj 74bc3b5144 Upgrade dependencies 2026-02-11 18:11:16 +08:00
grossmj aabfd08524 Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	CHANGELOG
#	dev-requirements.txt
#	gns3server/crash_report.py
#	gns3server/version.py
#	requirements.txt
2026-02-11 17:52:14 +08:00
grossmj dd87ccff4d Upgrade pytest and jsonschema dependencies 2026-02-11 17:38:49 +08:00
grossmj c1f8202d7c Revert "Upgrade python-multipart to v0.0.22. Fixes #2610"
This reverts commit fdcdce7d0d.
2026-02-11 17:14:03 +08:00
grossmj fdcdce7d0d Upgrade python-multipart to v0.0.22. Fixes #2610 2026-02-11 17:07:28 +08:00
Jeremy Grossmann 78c7d36b1c Merge pull request #2606 from GNS3/release/v2.2.56.1
Release v2.2.56.1
2026-02-11 16:52:43 +08:00
grossmj 44d4a426b5 Upgrade sentry-sdk and psutil dependencies 2026-02-11 16:46:21 +08:00
RishikesavanRamesh 3273f7e992 Add dynamips to Dockerfile dependencies 2026-02-06 20:34:44 +05:30
Jeremy Grossmann d2d19b43c2 Merge pull request #2607 from polsson86/update-pci-device-id-computation
Take populated disks into consideration when calculating PCI device ID
2026-02-06 12:45:30 +08:00
Patrik Olsson fe2014854e Take populated disks into consideration when calculating PCI device ID 2026-02-05 15:10:57 +01:00
grossmj 3915918c41 Release v3.0.6 2026-01-28 22:30:36 +08:00
grossmj a72980b2d9 Sync appliances 2026-01-28 22:24:10 +08:00
grossmj 600d3d38da Fix creating Qemu disk image. Fixes #2542 2026-01-28 19:36:22 +08:00
grossmj ebe4fdc5b5 Development on 2.2.57.dev2 2026-01-28 18:44:51 +08:00
grossmj 7f45cc6f47 Disable checking for available disk space. Ref #2548 2026-01-28 18:32:42 +08:00
grossmj 43d7e8a437 Release v2.2.56.1 2026-01-28 17:00:07 +08:00
grossmj 833b5a9ea8 Sync appliances 2026-01-28 16:52:30 +08:00
grossmj 05dda0117f Merge branch 'master' into 2.2 2026-01-28 16:00:14 +08:00
Jeremy Grossmann 6f0dfecc3d Merge pull request #2605 from iskanred/bugfix/2604
Fix telnet keepalive options on macOS
2026-01-28 15:56:13 +08:00
grossmj abaeab6242 Upgrade greenlet and sqlalchemy dependencies 2026-01-26 19:50:42 +08:00
Iskander Nafikov 2414b7a968 Fix telnet keepalive options on macOS 2026-01-26 00:31:03 +08:00
grossmj 1d45df6613 Upgrade dependencies 2026-01-24 21:38:58 +08:00
grossmj 69a695c9e6 Merge branch '2.2' into 3.0
# Conflicts:
#	dev-requirements.txt
#	gns3server/appliances/infix.gns3a
#	gns3server/crash_report.py
#	gns3server/run.py
#	gns3server/utils/asyncio/__init__.py
#	gns3server/version.py
#	requirements.txt
#	setup.py
2026-01-24 19:27:39 +08:00
grossmj 7b20314edb Delete random_query.py 2026-01-24 19:17:55 +08:00
grossmj 193f470eaf Upgrade dependencies 2026-01-24 19:12:58 +08:00
grossmj 4335fd7ce0 Drop Python 3.8 support 2026-01-24 18:51:44 +08:00
grossmj ef2cf277c1 Development on 2.2.57.dev1 2026-01-24 16:53:43 +08:00
Jeremy Grossmann 66db267e27 Merge pull request #2591 from GNS3/release/v2.2.56
Release v2.2.56
2026-01-24 16:49:35 +08:00
grossmj 27fbe37375 Release v2.2.56 2026-01-21 22:15:12 +08:00
grossmj 7b3fd4450e Sync appliances 2026-01-17 18:37:41 +08:00
grossmj 58d1e8f0c5 Set default location of udhcpc in "/etc/network/udhcpc". Fixes #2582 2026-01-17 18:01:42 +08:00
grossmj 8ce690ce78 Upgrade pywin32 to v311 2026-01-17 17:54:27 +08:00
grossmj c676a5bf82 Development on v2.2.56.dev1 2025-12-16 16:46:31 +08:00
grossmj 60c10e7ecc Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	.github/workflows/testing.yml
#	CHANGELOG
#	README.md
#	gns3server/appliances/infix.gns3a
#	gns3server/compute/docker/__init__.py
#	gns3server/compute/docker/docker_vm.py
#	gns3server/crash_report.py
#	gns3server/version.py
#	gns3server/web/web_server.py
#	setup.py
#	tests/controller/test_project.py
2025-11-29 11:42:02 +10:00
grossmj 37f17209d3 Release v2.2.55 2025-11-19 12:13:56 +10:00
grossmj f24134a8f5 Sync appliances 2025-11-19 12:08:29 +10:00
Jeremy Grossmann 1f2036defd Merge pull request #2575 from GNS3/bugfix/2574
Handle Docker API version requirements
2025-11-17 22:31:41 +10:00
grossmj f2aaf5ab38 Docker API version requirements and process to handle older daemons 2025-11-17 22:26:18 +10:00
Jeremy Grossmann 32f48e555c Merge pull request #2573 from GNS3/bugfix/2541
Python 3.14 compatibility
2025-11-17 11:30:42 +10:00
grossmj 942f021b58 Fix test_config.py 2025-11-17 11:26:06 +10:00
Jeremy Grossmann 4dc321454e Merge pull request #2572 from GNS3/feature/name-of-container-in-docker
Set name of container in Docker
2025-11-17 11:13:47 +10:00
grossmj 3a24318694 Use docker_name property when creating a container 2025-11-17 11:11:11 +10:00
grossmj bae6221991 Set name of container in Docker 2025-11-17 10:59:05 +10:00
grossmj c67ba6da0d Fix config tests 2025-11-17 10:21:12 +10:00
grossmj 4d23ca221f Comment out some tests in test_project.py 2025-11-16 20:52:14 +10:00
grossmj 09eab9529a Fix syntax error in testing.yml 2025-11-16 19:32:19 +10:00
grossmj 3bb23708fc Support for Python 3.14 2025-11-16 19:30:45 +10:00
grossmj 672b3c052b Downgrade aiofiles version 2025-11-16 19:25:12 +10:00
grossmj bfe7b7a735 Upgrade dependencies 2025-11-16 19:16:38 +10:00
Jeremy Grossmann d269a4c125 Merge pull request #2570 from Mytic2330/3.0-patch3
Fixed Connection closed when exporting a running project
2025-11-04 04:07:01 +01:00
Jeremy Grossmann 8ada95b9fd Merge pull request #2569 from Mytic2330/3.0
Fix database pool error and non-ASCII character project export
2025-11-04 04:05:56 +01:00
Bine d5f360c176 Update projects.py 2025-10-18 19:28:15 +02:00
Bine e817228280 Fix non-ASCII characters in project names 2025-10-18 18:10:24 +02:00
BineandUfi 4a059be480 Increase DB engine pool size and max overflow
As described in #2568 when multiple clients connect simultaneously, an excessive number of database pool connections are created. This leads to resource exhaustion and connection management issues.

We have introduced the pool_size and max_overflow parameters to better control the database connection pool.
These parameters ensure the database can handle a sufficient number of concurrent connections without over-allocating resources.

Added pool_size and max_overflow arguments to the database initialization.

This change overrides the default initialization configuration where the temporary fix was applied.

Co-Authored-By: Ufi <86113085+ufiking@users.noreply.github.com>
2025-10-17 22:17:10 +02:00
Jeremy Grossmann 230866890e Merge pull request #2556 from GNS3/bugfix/2543
Enable ip cef in IOU L2 and IOU L3 base configs
2025-09-01 18:59:10 +02:00
grossmj 355cfb7571 Enable ip cef in IOU L2 and IOU L3 base configs 2025-09-01 18:55:36 +02:00
grossmj 8dddfe056e Merge branch 'master' into 2.2 2025-09-01 18:42:02 +02:00
Jeremy Grossmann 9c33827de3 Merge pull request #2555 from GNS3/update-gh-docker-build
Update docker-build.yml
2025-09-01 18:28:04 +02:00
grossmj a34fa53823 Update tags 2025-09-01 18:27:04 +02:00
grossmj 565a76ae27 Only build and push Docker image when releasing a new version 2025-09-01 18:23:52 +02:00
grossmj dc9c3b1c5d Fix pushing Docker image to Docker hub 2025-09-01 18:17:52 +02:00
grossmj a394fb71dc Update method to login to ghcr.io 2025-09-01 17:55:46 +02:00
grossmj 6cbf3baa1b Update docker-build.yml 2025-09-01 17:49:34 +02:00
grossmj 10322135e3 Upgrade sentry-sdk package 2025-08-30 21:51:20 +02:00
grossmj 12d1d04462 Update security issues again in README.md. Fixes #2535 2025-08-29 18:28:14 +02:00
grossmj 8b4c2f5e90 Update security issues in README.md. Fixes #2535 2025-08-29 18:25:57 +02:00
grossmj 1ed549fdb1 Upgrade requests dev dependency 2025-08-29 14:01:19 +02:00
grossmj 845cad35f0 Upgrade pydantic, fastapi & sqlalchemy dependencies 2025-08-29 13:55:00 +02:00
grossmj 31613630cf Update pyproject.toml to support LICENSE file 2025-06-09 22:24:23 +02:00
Jeremy Grossmann aaffc731ee Merge pull request #2538 from Nick1200000/enhancement/docker-build
Add automated Docker build and publish workflow
2025-06-09 22:14:11 +02:00
Nick1200000 5ce55aff02 Resolving the Error 2025-06-08 14:11:01 +05:30
Nick1200000 aa1c0bb557 Resolving the Errors 2025-06-08 13:38:35 +05:30
Nick1200000 25bc2dce57 Resolving the Errors 2025-06-08 13:30:49 +05:30
Nick1200000 7e35d164dc Resolving the Errors 2025-06-08 13:26:40 +05:30
Nick1200000 8666ffaff3 Resolving the Errors 2025-06-08 13:24:38 +05:30
Nick1200000 020f6a123d Docker build CI/CD pipeline 2025-06-07 18:16:47 +05:30
grossmj 232b9c2806 Fix tests 2025-05-25 17:35:45 +02:00
Jeremy Grossmann f260f6e2b4 Merge pull request #2531 from PsyBlade42/docker-vnc-name
use GNS3 node name as vnc Desktop name
2025-05-23 14:26:08 +02:00
Thomas Schmidt a7108e1592 use GNS3 node name as vnc Desktop name
Most VNC clients use the Desktop name in their window title.
Currently this defaults to user@host which means all docker vnc
connections have identical window names of e.g.
  SSVNC: gns3@gns3server

This uses the node name instead.
2025-05-22 13:13:47 +02:00
grossmj 975ca254fd Release v3.0.5 2025-05-14 19:11:44 +02:00
grossmj 89cf5ea8e7 Bundle web-ui v3.0.5 2025-05-14 19:06:42 +02:00
grossmj bfbe2bf885 Sync appliances 2025-05-14 18:55:51 +02:00
grossmj 5c036dfd90 Upgrade uvicorn and pydantic 2025-05-14 18:54:55 +02:00
grossmj 2fbfe5385a Use Ubuntu 24.04 LTS in Dockerfile. Ref #2523 2025-05-14 18:30:43 +02:00
grossmj 2cdaa1cd06 Fix tests 2025-05-14 17:19:18 +02:00
grossmj f2080979a1 Fix tests 2025-05-14 16:56:46 +02:00
grossmj df576f0668 Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/schemas/template.py
2025-05-14 16:42:59 +02:00
grossmj c2c89b0ff3 Add missing 'nat' template type in schema. Fixes #2529 2025-05-13 20:52:44 +02:00
grossmj ef3570dc95 Merge branch 'master' into 2.2 2025-05-13 20:48:59 +02:00
grossmj 2bf3eff143 Remove debug message in tests 2025-05-13 20:47:38 +02:00
Jeremy Grossmann add8c4e8a2 Merge pull request #2526 from emmanuel-ferdman/master
Resolve deprecation warnings of regex library
2025-04-30 14:14:37 +07:00
Jeremy Grossmann 53eac771d6 Merge pull request #2528 from tobiasstein/fix-pci-bridges
Fix adding pci_bridges to qemu vms
2025-04-30 14:09:22 +07:00
grossmj 2b96e54629 Fix tests and require Qemu version >= 2.4 2025-04-30 13:38:52 +07:00
Tobias Stein cdf819f820 Fix adding pci_bridges to qemu vms
* Offset pci_device_id for network devices by 32 (formerly 4)
  to reserve first 32 IDs for non-netdev devices
  * This implies a new pci_bridge for the netdevs and
    the pci_device_ids starting at 0 (using modulo)
  * Fixes a bug in the creation of the qemu command
    when the number of qemu pcie devices (storage_controller, disks, …)
    was greater than 4 and the number of network devices
    saturated the 32 pcie ids of the root bridge
    causing qemu not to start.
2025-04-28 21:07:52 +02:00
Emmanuel Ferdman c05e6f4de4 Resolve deprecation warnings of regex library
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
2025-04-26 00:15:16 -07:00
grossmj d9dcc2738d Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	CHANGELOG
#	Dockerfile
#	README.md
#	gns3server/appliances/infix.gns3a
#	gns3server/compute/docker/docker_vm.py
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/controller/gns3vm/virtualbox_gns3_vm.py
#	gns3server/crash_report.py
#	gns3server/static/web-ui/index.html
#	gns3server/static/web-ui/main.9bcf455e62558dedfd48.js
#	gns3server/version.py
2025-04-21 20:13:31 +07:00
grossmj e32576d5ed Development on 2.2.55.dev1 2025-04-21 19:35:13 +07:00
Jeremy Grossmann bd3657d8f3 Merge pull request #2524 from GNS3/release/v2.2.54
Release v2.2.54
2025-04-21 19:30:57 +07:00
grossmj 4418d6bb2b Release v2.2.54 2025-04-21 17:15:22 +07:00
grossmj dbcafdd314 Merge branch 'master' into 2.2 2025-04-21 17:08:59 +07:00
grossmj 4dd1803a70 Bundle web-ui v2.2.54 2025-04-21 16:40:59 +07:00
grossmj 99272fb7e0 Sync appliances 2025-04-21 16:30:32 +07:00
grossmj a24d2e25ed Add new method to find the IP address of a VBox GNS3 VM + allow NAT Network 2025-04-21 16:29:32 +07:00
grossmj a5a663bc99 Remove OVMF_CODE_4M.secboot.fd and associated code 2025-04-18 17:31:31 +07:00
grossmj 2b7eb91694 Add edk2-stable202502 UEFI firmwares and try to first use firmwares from the ovmf package if installed. Fixes #2494 2025-04-18 17:13:29 +07:00
grossmj 684770a220 Prettify TPM error message 2025-04-18 15:25:50 +07:00
grossmj f6546bf1de Fix bug in qemu_stdout message 2025-04-18 15:19:06 +07:00
grossmj 355b97398b Reduce matching error for TPM result 2025-04-18 15:13:41 +07:00
grossmj 6242a0be80 Try to detect swtpm and AppArmor issue. Ref https://github.com/GNS3/gns3-gui/issues/3725 2025-04-18 15:06:49 +07:00
grossmj 18413c923c Development on 2.2.54.dev1 2025-04-18 15:05:34 +07:00
grossmj ffd628902c Fix Docker logs decoding. Ref #2522 2025-04-18 13:58:56 +07:00
grossmj ca34053125 Add delay after starting a Docker container and adding connections in uBridge. Ref #2522 2025-04-18 13:50:20 +07:00
grossmj d06f93e772 Fix TypeError when reading Docker container logs. Ref #2522 2025-04-18 13:47:29 +07:00
grossmj 90cf73e434 Fix tests 2025-04-18 13:38:01 +07:00
grossmj aa8ccf4f82 Allow an image to be uploaded to the controller again even if it is already in the database 2025-04-17 19:07:22 +07:00
grossmj 0826b9c259 Fix controller not reporting an error if invalid options are passed to Qemu command line. Fixes #2517 2025-04-17 19:00:52 +07:00
grossmj 0e8d969cd2 Merge branch '2.2' into 3.0
# Conflicts:
#	gns3server/compute/docker/__init__.py
#	gns3server/compute/docker/docker_vm.py
#	requirements.txt
2025-04-16 17:53:44 +07:00
grossmj 0e89ff56a8 Replace "Docker hub" by "Docker repository" because it is possible to use different repositories 2025-04-16 17:42:40 +07:00
grossmj ffb5bc7ffd Upgrade aiohttp to v3.11.16 2025-04-16 17:09:09 +07:00
grossmj f00f21c4bd Upgrade dependencies 2025-04-16 17:08:08 +07:00
grossmj 773076c40e Fix unable to add NAT cloud after configuring “allowed_interfaces” in configuration file. Fixes #2508 2025-04-16 16:04:57 +07:00
grossmj 5019167098 Fix interface information API endpoint for Cloud/NAT devices 2025-04-16 15:26:56 +07:00
grossmj f525916195 Merge remote-tracking branch 'origin/3.0' into 3.0 2025-04-16 14:39:59 +07:00
grossmj 2ac3e0b39f Upgrade dependencies 2025-04-16 14:39:36 +07:00
Jeremy Grossmann e532c83538 Merge pull request #2518 from markparonyan/get-link-ifaces
Get iface/udp tunnel from links with Cloud,Nat
2025-03-23 20:10:23 +07:00
Mark Paronyan f21887dbab tests(get_iface): add unittests 2025-03-23 14:28:58 +00:00
Mark Paronyan bab173ee52 feat(get_iface): add type information in response 2025-03-23 14:28:27 +00:00
Mark Paronyan ea2856289a feat: add endpoint to get iface/udp tunnel from links with Cloud,Nat 2025-03-23 13:46:27 +00:00
grossmj 8e6f5d2b57 Merge branch '2.2' into 3.0
# Conflicts:
#	requirements.txt
2025-03-18 21:34:43 +07:00
grossmj 2e9127449c Upgrade Jinja2 to v3.1.6. Fixes #2515 2025-03-18 21:33:31 +07:00
grossmj ee9f974362 Fix packet capture for links connected to a cloud node. Fixes #2513 2025-03-16 21:12:33 +07:00
grossmj c9231d7b83 Upgrade dependencies 2025-03-16 20:26:26 +07:00
Jeremy Grossmann 483719568f Merge pull request #2511 from NAKNAO-nnct/feature/update-dockerfile-ubuntu-2404
Update Dockerfile base image to Ubuntu 24.04
2025-03-08 20:26:12 +07:00
NAKNAO-nnct 018598a300 refactor 2025-03-08 20:22:06 +09:00
NAKNAO-nnct e262a46cfd rm space 2025-03-08 20:19:37 +09:00
NAKNAO-nnct 00bf1c028b Update README to Run Docker Compose 2025-03-08 14:11:42 +09:00
NAKNAO-nnct 21ebb934f4 add compose.yaml 2025-03-08 14:08:42 +09:00
NAKNAO-nnct 2ecd2cfe3b update Ubuntu24.04 2025-03-08 13:59:05 +09:00
Jeremy Grossmann ce4b3091dd Merge pull request #2507 from GNS3/fix-remote-install-script
Improvements for remote-install.sh
2025-03-03 05:55:42 +07:00
grossmj f722816849 Improvements for remote-install.sh
* Use apt-add-repository to install repository
* Add option to not upgrade the system
* Check if Python version is >= 3.9 when installing from ppa-v3 repository
2025-03-02 22:27:53 +07:00
Jeremy Grossmann 4cb38b23eb Merge pull request #2506 from FedericoTorres233/patch-1
Check if remote-install.sh is being run as root
2025-03-02 16:14:33 +08:00
Federico Torres f3e46227d8 fix: check if remote-install.sh is being run as root 2025-03-01 16:56:21 -03:00
grossmj 3ba109c651 Development on 3.0.5.dev1 2025-02-25 22:39:38 +08:00
grossmj 3a1ad8c08b Release v3.0.4 2025-02-25 18:24:26 +08:00
grossmj edc855dcb5 Sync appliances 2025-02-25 17:12:28 +08:00
grossmj 59c2eeb536 Upgrade aiohttp to v3.11.13 2025-02-25 19:01:00 +10:00
grossmj a7f2b1fddc Require minimum 8 characters for passwords 2025-02-23 12:05:19 +10:00
grossmj 1f4071af93 Bundle web-ui v3.0.4 2025-02-23 12:04:22 +10:00
grossmj dbdb4a53c7 Sync appliances 2025-02-22 21:18:41 +10:00
grossmj 2659ab2c35 Upgrade dependencies 2025-02-22 21:10:33 +10:00
grossmj 0c3f6712b4 Fix auto idle-pc for IOS templates 2025-02-22 20:14:13 +10:00
grossmj 9122cab6c1 Update schemas to add constraints for values like ram, nvram etc. 2025-02-04 10:39:55 +10:00
grossmj 2516b1952c Install custom symbols. Fixes #2492 2025-01-29 18:32:04 +10:00
grossmj f2bb785037 Remove txt files in custom symbols 2025-01-29 18:25:06 +10:00
grossmj fc938a069b Sync appliances 2025-01-29 18:24:04 +10:00
grossmj 4fd41991db Fix VMware add ubridge connection. Fixes #2490 2025-01-27 12:12:43 +10:00
grossmj cfc4bb371a Upgrade FastAPI to v0.115.7 2025-01-23 13:32:11 +10:00
grossmj ce59e84e13 Development on 3.0.4 2025-01-23 13:31:32 +10:00
grossmj b5e2bb1561 Release v3.0.3 2025-01-22 19:06:42 +10:00
grossmj 3003ed1465 Bundle web-ui v3.0.3 2025-01-22 18:57:21 +10:00
grossmj 09774a3a56 Merge branch '2.2' into 3.0
# Conflicts:
#	CHANGELOG
#	gns3server/compute/virtualbox/virtualbox_vm.py
#	gns3server/controller/compute.py
#	gns3server/crash_report.py
#	gns3server/static/web-ui/index.html
#	gns3server/static/web-ui/main.c83939cdfe3af0ec27df.js
#	gns3server/version.py
#	gns3server/web/web_server.py
#	requirements.txt
#	tests/controller/test_compute.py
2025-01-22 18:37:17 +10:00
Jeremy Grossmann 185cd01e19 Merge pull request #2488 from GNS3/release/v2.2.53
Release v2.2.53
2025-01-21 09:47:43 +07:00
grossmj 369fcaa512 Release v2.2.53 2025-01-21 11:52:11 +10:00
grossmj 7b7d3d676f Bundle web-ui v2.2.53 2025-01-21 11:43:11 +10:00
grossmj d1186b2acc Sync appliances 2025-01-21 11:29:28 +10:00
grossmj bd58196817 Add more information when patching .vbox file. Ref https://github.com/GNS3/gns3-gui/issues/3542 2025-01-20 13:22:22 +10:00
Jeremy Grossmann b3a822df8f Merge pull request #2487 from GNS3/upgrade-tests
Refactor tests and upgrade dev package requirements
2025-01-17 16:38:32 +07:00
grossmj ba4e0c945d Refactor tests and upgrade dev package requirements 2025-01-17 19:12:44 +10:00
Jeremy Grossmann f6725a37dd Merge pull request #2486 from GNS3/feature/args-parsing
Refactor command line arguments parsing
2025-01-17 08:26:05 +07:00
grossmj 8e96c4c2e8 Refactor command line arguments parsing 2025-01-17 07:57:42 +07:00
grossmj 035a104957 Fix capture on IOU/IOL links. Fixes #2477 2025-01-11 18:46:55 +07:00
grossmj 54c0b383cb Use Python 3.9 to publish API documentation 2025-01-08 00:03:18 +07:00
grossmj 7c0384a8d4 Development on 3.0.3.dev1 2025-01-07 11:12:02 +07:00
grossmj ccc8974d92 Fix tests 2024-12-30 16:14:33 +07:00
grossmj 0090ff3722 Increase timeout to run compute HTTP queries. Fixes #2461 2024-12-30 16:10:04 +07:00
grossmj 2c727c6bd2 Use 'allow_methods="*"' in aiohttp_cors.ResourceOptions(). Fixes #2459 2024-12-30 11:06:19 +07:00
grossmj 13b0caef4f Upgrade dependencies 2024-12-30 10:49:35 +07:00
695 changed files with 129478 additions and 11736 deletions

No files matched your search

+37
View File
@@ -0,0 +1,37 @@
# GNS3 Server Project Memory
> **Note**: This directory stores important project-related memories and case studies, managed with the code repository.
>
> **How to record**: Use the `/memory` skill to record important information to the project memory directory.
## Quick Reference
- **Memory directory**: `.claude/memory/`
- **Skill file**: `.claude/skills/memory/SKILL.md`
- **Main index**: `MEMORY.md` (this file)
## Topics
### Web Wireshark Integration
- **[JWT Token Flow](./web-wireshark-jwt-token-flow.md)** - JWT token transmission path in Web Wireshark
- Key point: UDPLink only passes through jwt_token, ultimately used by curl command inside Web Wireshark container to authenticate with GNS3 capture stream API
- **[Xpra HTML5 Client](./xpra-html5-client.md)** - Xpra HTML5 client menu control parameters for customizing the web interface
### RBAC & User Isolation
- **[RBAC User Isolation Design](./rbac-user-isolation-design.md)** — Three-step permission check design: ACE batch check → created_by filtering → resource pools
### Appliance Management
- **[GNS3 Appliance Loading](./gns3-appliance-loading.md)** - How GNS3 loads appliance files from builtin and custom directories with priority rules
### uBridge Permission
- **[uBridge Permission Issue](./gns3-ubridge-permission.md)** - Docker containers fail to start due to missing CAP_NET_ADMIN/CAP_NET_RAW capabilities on uBridge
- **[Docker iptables FORWARD blocks bridge](./docker-iptables-forward-bridge.md)** - Docker sets FORWARD chain to DROP, blocking kernel bridge forwarding; `sudo iptables -P FORWARD ACCEPT` to fix
### Docker Container Stop Delay
- **[Docker Container Stop Delay](./docker-container-stop-delay.md)** - Some containers take ~5s to stop because they don't handle SIGTERM (AlpiNet, OstinatoWireshark)
### MCP Service
- **[MCP Service Design](./mcp-service-design.md)** - MCP (Model Context Protocol) service architecture using FastMCP with SSE transport, JWT auth, 29 tools across 5 domains
- **[MCP Tool Description Location](./mcp-tool-description-guide.md)** - Where to define MCP tool descriptions: in `@mcp.tool()` functions in `__init__.py`, not in `*_TOOLS` arrays
### Python Code Verification
- **[Import Validation](./python-import-validation.md)** - Use actual module imports (`python -c "from ... import ..."`) instead of `py_compile` to catch missing imports
@@ -0,0 +1,41 @@
---
name: docker-container-stop-delay
description: Docker containers not responding to SIGTERM cause ~5s stop delays when closing a project
metadata:
type: reference
---
# Docker Container Stop Delay Analysis
## Background
When stopping a GNS3 project, some Docker containers take ~5s to exit while others stop instantly.
## Root Cause
Docker's `stop` command sends SIGTERM and waits `t` seconds (GNS3 sets `t=5`) before sending SIGKILL. Containers that don't handle SIGTERM are stuck waiting for the full timeout.
## Affected Containers
| Container | PID 1 | Why it's slow |
|-----------|-------|---------------|
| **AlpiNet** (alpine) | `dumb-init` → `bash -i` | Interactive bash ignores SIGTERM by design |
| **OstinatoWireshark** | `bash` (PID 1) | Linux kernel won't apply default signal actions to PID 1 without an explicit handler; interactive bash doesn't install one |
## Normal Containers (for comparison)
| Container | PID 1 | Why fast |
|-----------|-------|----------|
| Chromium | `/usr/bin/chromium` | Chromium handles SIGTERM natively |
| webterm | `dumb-init` → firefox | Firefox responds to SIGTERM immediately |
## Related Files
- `gns3-registry/docker/alpinet/Dockerfile`
- `gns3-registry/docker/ostinato-wireshark/Dockerfile`
- `gns3-registry/docker/ostinato-wireshark/entry.sh`
- `gns3-registry/docker/chromium/Dockerfile`
- `gns3-registry/docker/ipterm/web/Dockerfile`
- `gns3-server/gns3server/compute/docker/docker_vm.py:1040` — stop timeout parameter `t=5`
## Note
This is not a GNS3 server bug (except a minor `or` vs `and` logic issue at `docker_vm.py:1037` which doesn't affect behavior). The root cause is in the Docker images themselves.
See also: [[docker-container-stop-delay]]
@@ -0,0 +1,38 @@
---
name: docker-iptables-forward-bridge
description: Docker iptables FORWARD DROP blocks kernel bridge forwarding, fix and symptoms
metadata:
type: reference
---
Docker sets the iptables `FORWARD` chain default policy to `DROP` when the
Docker daemon starts. This blocks **all** forwarded traffic through Linux
kernel bridges on the host — including `gns3br{N}` bridges created by the
builtin Ethernet Switch (ubridge `brctl`).
## Symptoms
- Nodes connected to the switch can send frames into the bridge (visible in
`tcpdump -i gns3br{N}`) but never receive forwarded unicast frames.
- `bridge fdb show` may fail to learn MAC addresses (frames dropped before
the bridge learning path).
- ARP and multicast/broadcast may appear to work because they flood, but
unicast replies never reach the destination.
- OSPF Hello / CDP visible on both sides but ICMP echo reply never returns.
- `ubridge bridge get_stats` shows symmetric IN/OUT counts (relay is fine),
`bridge fdb show` shows learned MACs, `bridge link show` shows `state forwarding`
on all ports — yet unicast still doesn't work.
## Fix
Run once per host boot, or make persistent via iptables-persistent / firewall config:
```bash
sudo iptables -P FORWARD ACCEPT
```
## Related
- [[ethernet-switch-ubridge-brctl-migration]] — the kernel bridge that hits this
- [[gns3-server-linux-only]] — datapath constraint
- [[gns3-ubridge-permission]] — another host-level prerequisite (CAP_NET_ADMIN)
+20
View File
@@ -0,0 +1,20 @@
---
name: gns3-appliance-loading
description: GNS3 appliance file loading mechanism and storage locations
metadata:
type: reference
---
GNS3 loads appliance (.gns3a) files from two locations with specific priority order:
1. **Builtin appliances directory**: `~/.local/share/GNS3/appliances/`
- Stores automatically downloaded devices from GNS3 registry
- Maintained and updated by the system automatically
2. **Custom appliances directory**: `~/GNS3/appliances/`
- Stores user-customized or modified appliance files
- Manually managed by users
**Loading priority**: System loads builtin appliances first, then custom appliances. If both directories contain devices with the same `device_id`, the custom appliance overwrites the builtin one. This design allows users to customize devices without having their modifications overwritten by automatic registry updates.
**Implementation**: See `gns3server/controller/appliance_manager.py` in the `load_appliances()` method (lines 314-351).
+25
View File
@@ -0,0 +1,25 @@
# GNS3 uBridge Permission Issue
## Problem
Docker container node fails to start with error: `uBridge requires root access or the capability to interact with network adapters`
## Root Cause
- uBridge needs to create TAP network interfaces to connect Docker containers to GNS3 virtual network
- This requires `CAP_NET_ADMIN` and `CAP_NET_RAW` capabilities
- Other node types (like QEMU) may have their own network implementation and don't need uBridge
## Solution
```bash
sudo setcap cap_net_admin,cap_net_raw=eip /usr/bin/ubridge
```
## Code Locations
- uBridge path check: `gns3server/compute/base_manager.py:298` (`has_privileged_access`)
- Permission check call: `gns3server/compute/base_node.py:851`
- Error message definition: `gns3server/compute/base_node.py:852`
## Error Propagation Flow
Error is propagated via `NodeError` exception:
1. `NodeError` is caught at `gns3server/api/routes/compute/__init__.py:131-137`
2. Returns HTTP 409 with JSON: `{"message": "...", "exception": "NodeError"}`
3. Frontend should read error info from `message` field
+91
View File
@@ -0,0 +1,91 @@
---
name: mcp-service-design
description: MCP (Model Context Protocol) service architecture and tool design for GNS3 server
metadata:
type: project
---
# MCP (Model Context Protocol) Service Design
## Background
Provide a standard MCP interface for GNS3 Server, allowing AI assistants (Claude Code, Claude Desktop) to interact with GNS3 network simulations through the Model Context Protocol.
## Decision/Implementation
### Transport
- **SSE (Server-Sent Events)** with JWT token authentication
- Endpoint: `/v3/mcp/transport/sse`
- Message endpoint: `/v3/mcp/transport/messages/`
### Authentication
- JWT token obtained via `/v3/access/users/authenticate`
- Two ways to pass token:
- `Authorization: Bearer <jwt>` header (Claude Code via `-H`)
- `?token=<jwt>` query param (Claude Desktop, EventSource limitation)
- Token validated using GNS3's existing `auth_service`
- Token stored in `contextvars.ContextVar` for per-session isolation
- Python ≥ 3.9 `asyncio.to_thread` propagates contextvars to threads
### Architecture
```
Claude Code / Desktop → SSE → Auth Wrapper → FastMCP Server → Tool Handler → Gns3Connector → GNS3 REST API
```
### Tool Organization
Tools are separated by domain into individual files under `gns3server/api/routes/mcp/`:
| File | Domain | Tool Count |
|------|--------|:----------:|
| `projects.py` | Project CRUD, open/close/stats | 7 |
| `nodes.py` | Node CRUD, start/stop/reload/suspend, console WS | 10 |
| `links.py` | Link CRUD | 5 |
| `templates.py` | Template CRUD | 5 |
| `computes.py` | Compute list/get/images | 3 |
**Total: 30 tools**
### Handler Pattern
- Synchronous functions receiving `(params: dict, gns3_ctx: dict)`
- Run via `asyncio.to_thread()` to avoid blocking the event loop
- `gns3_ctx` contains `server_url` and `jwt_token`
- `Gns3Connector` is created per-handler from `custom_gns3fy`
### Token Lifetime
- Default: 1440 minutes (24 hours)
- Configurable via `jwt_access_token_expire_minutes` in `gns3_server.conf`
## Rationale
- **Why not Direct Controller calls**: MCP layer calls GNS3's own REST API through Gns3Connector, keeping full decoupling and supporting future multi-user/multi-instance scenarios
- **Why not Streamable HTTP**: Claude Code supports SSE natively via `--transport sse` with custom headers; Streamable HTTP session manager lifecycle conflicts with FastAPI mount
- **Why not stdio**: stdio is local-only; SSE supports both local and remote deployments
## Related Files
- `gns3server/api/routes/mcp/__init__.py` — FastMCP server, tool decorators, auth wrapper
- `gns3server/api/routes/mcp/projects.py` — Project tool handlers
- `gns3server/api/routes/mcp/nodes.py` — Node tool handlers
- `gns3server/api/routes/mcp/links.py` — Link tool handlers
- `gns3server/api/routes/mcp/templates.py` — Template tool handlers
- `gns3server/api/routes/mcp/computes.py` — Compute tool handlers
- `gns3server/agent/gns3_copilot/gns3_client/custom_gns3fy.py` — Gns3Connector client
- `gns3server/api/server.py:87` — MCP route registration
## Configuration
### Claude Code
```bash
claude mcp add --transport sse My_GNS3_Server \
http://host:3080/v3/mcp/transport/sse \
-H "Authorization: Bearer <jwt>"
```
### Claude Desktop
```json
{
"mcpServers": {
"My_GNS3_Server": {
"url": "http://host:3080/v3/mcp/transport/sse?token=<jwt>"
}
}
}
```
@@ -0,0 +1,43 @@
---
name: mcp-tool-description-location
description: Where to define MCP tool descriptions so AI can see them
metadata:
type: reference
---
# MCP Tool Description Location
## Key Point
MCP tool descriptions are defined in `@mcp.tool()` decorator functions in `__init__.py`, NOT in the `*_TOOLS` arrays in individual module files.
## Correct Location
**File**: `gns3server/api/routes/mcp/__init__.py`
**Example**:
```python
@mcp.tool()
async def update_link(
project_id: Annotated[str, Field(description="UUID of the project")],
link_id: Annotated[str, Field(description="UUID of the link to update")],
**kwargs: Any,
) -> list[dict[str, Any]]:
"""Update a link's properties.
Put detailed descriptions here, especially for complex parameters.
Include format requirements, ranges, and examples.
"""
# implementation
```
## Wrong Location
- ❌ `LINK_TOOLS` in `gns3server/api/routes/mcp/links.py`
- ❌ `TEMPLATE_TOOLS` in `gns3server/api/routes/mcp/templates.py`
## Activation
**Must restart GNS3 server** for description updates to take effect.
## Description Requirements
- Be explicit about data formats (arrays vs single values)
- Include parameter ranges and constraints
- Provide usage examples
- Prevent common errors in the description itself
@@ -0,0 +1,30 @@
# Python Import Validation
## Background
When checking if modified Python code is correct, `py_compile` only validates syntax (e.g., balanced parentheses, valid keywords). It does **not** catch missing imports or other runtime errors (e.g., using `UUID()` without importing `UUID`).
## Decision/Implementation
Use actual module imports to verify code correctness:
```bash
# ✅ This catches missing imports and runtime errors
venv/bin/python -c "
from gns3server.api.routes.controller.dependencies.authentication import get_user_from_token
from gns3server.api.routes.mcp.__init__ import _resolve_token
print('All imports OK')
"
# ❌ This only checks syntax, not references
venv/bin/python -c "import py_compile; py_compile.compile('file.py', doraise=True)"
```
## Related Files
`gns3server/api/routes/controller/dependencies/authentication.py` — missed `from uuid import UUID`
`gns3server/api/routes/mcp/__init__.py` — missed `from uuid import UUID`
## Why
A `NameError` at runtime is far more expensive than a failed import check. Real import testing catches the full dependency chain.
@@ -0,0 +1,231 @@
---
name: rbac-user-isolation-design
description: GNS3 RBAC user isolation design and implementation thought process
metadata:
type: project
---
## RBAC User Isolation Design Summary
### Core Problem
GNS3 3.0 has a complete RBAC framework (ACE + Role + Privilege), but lacks user isolation implementation, causing users to see resources they shouldn't have access to.
### Design Conflict
Traditional RBAC's **path permission model** fundamentally conflicts with **user data isolation**:
- **Path permission model**: Controls which paths users can access (e.g., `/projects`)
- **User data isolation**: Controls which specific resources users can access (e.g., alice's project vs bob's project)
### Final Implementation Approach
#### Three-Step Permission Check Logic
```python
# Step 1: Batch ACE + resource pool check (3 DB queries regardless of project count)
direct_ace_ids, pool_accessible_ids = await rbac_repo.get_accessible_project_ids(
current_user.user_id, "Project.Audit", all_project_ids
)
# Step 2: Filter direct ACE projects by created_by (user's own projects)
# Direct project sharing is only available through resource pools
for p in controller.projects.values():
if p.id in direct_ace_ids and p.created_by == current_user.username:
projects.append(p.asdict())
# Step 3: Resource pool projects (no created_by filter)
for p in controller.projects.values():
if p.id in pool_accessible_ids:
projects.append(p.asdict())
```
##### Super Admin Bypass
```python
if current_user.is_superadmin:
return [p.asdict() for p in controller.projects.values()]
```
Super admins skip all three steps and see every project.
##### seen_project_ids Deduplication
A simple `seen_project_ids` set prevents the same project from appearing twice when it exists in both direct ACE and pool results. This is a lightweight dedup, not the complex blocking mechanism from earlier rejected designs.
### Key Design Decisions
#### 1. ACE vs created_by Relationship
- **ACE for basic access control**: Whether user can access the system
- **created_by for user isolation**: Which specific resources user can access
- **Step 2 filtering is critical**: Even with broad ACE, created_by filtering ensures user isolation
#### 2. Project Sharing Mechanism
- **Project sharing only through resource pools**: Cannot configure ACE directly for specific projects
- **Avoids complexity of direct ACE sharing**: Prevents permission configuration chaos
#### 3. Broad ACE Fault Tolerance
```
Even with configuration: ACE: Users group + User role + "/" + propagate: True
Result: Users still only see projects they created
Reason: Step 2 created_by filtering removes other users' projects
```
### RbacRepository.get_accessible_project_ids()
The core batch-check method in `gns3server/db/repositories/rbac.py` uses 3 DB queries:
1. **User ACEs**: Direct ACE entries matching user + privilege
2. **Group ACEs**: Group-level ACE entries via `UserGroup` membership
3. **All resources with pool memberships**: Preloads all resources and their pool relationships
Then it computes two sets:
- **`direct_ace_ids`**: Project paths matching user or group ACEs (path-based check)
- **`pool_accessible_ids`**: Projects in pools the user/group can access (pool-based check)
Pool IDs are precomputed into a `pool_id -> set(project_ids)` map for O(1) lookup.
### Problems Solved
#### Problem 1: Missing User Isolation
- **Issue**: All users can see all projects
- **Solution**: Filter by created_by to implement user isolation
#### Problem 2: Broad ACE Breaking Isolation
- **Issue**: `path: "/" + propagate: true` breaks user isolation
- **Solution**: Step 2 created_by filtering ensures user isolation even with broad ACE
#### Problem 3: Permission Check Order Conflicts
- **Issue**: seen mechanism blocks subsequent checks (from earlier design phases)
- **Solution**: Simple pipeline-style filtering with lightweight dedup set
### Technical Details
#### API Layer Permission Check
```python
@router.get("/projects") # Note: no has_privilege decorator
async def get_projects(current_user=..., rbac_repo=...):
# Permission checks in business logic
```
#### Duplicate Prevention
- Simple `seen_project_ids` set for deduplication between direct ACE and pool results
- Not the complex blocking mechanism from earlier phases
#### Performance Considerations
- Super admin path: O(1) — returns all projects directly
- Regular user path: 3 fixed DB queries regardless of project count
- Path-based ACE check: O(n * m) in worst case, where n = projects, m = ACE entries
- Pool lookup: O(1) via precomputed pool->project map
### Use Cases
#### Scenario 1: Personal Use
```
alice creates project → alice only sees alice's projects ✅
bob creates project → bob only sees bob's projects ✅
No extra configuration needed, automatic user isolation
```
#### Scenario 2: Team Collaboration
```
Admin creates resource pool → adds projects → team members can access
alice creates project → alice still sees her own projects ✅
Team sharing + user isolation coexist
```
#### Scenario 3: Broad ACE Configuration
```
ACE: Users group + "/" + propagate: True
alice still only sees alice's projects ✅
User isolation unaffected by ACE configuration
```
### Design Principles
#### 1. Separation of Concerns
- **Basic access permission**: Controlled by ACE
- **Data ownership**: Controlled by created_by
- **Team sharing**: Controlled by resource pools
#### 2. Defensive Design
- User isolation remains effective even with improper ACE configuration
- Secure by default: users only see their own resources
#### 3. Simplicity
- Avoid complex seen mechanisms and mutual exclusion logic
- Clear pipeline-style check order
### Relationship with Original ACE System
#### Preserved Components
- ✅ ACE framework (permission check mechanism)
- ✅ Role and privilege definitions
- ✅ Resource pool functionality
#### Improved Components
- ✅ Added user isolation (created_by filtering)
- ✅ Clarified project sharing mechanism (only through resource pools)
- ✅ Simplified permission check logic
#### Removed Components
- ❌ Removed `/projects` path privilege check on `get_projects` route
- ❌ Removed the old complex `seen_project_ids` blocking mechanism
- ❌ Avoided "can see all non-pool projects" privilege leak
### Implementation Details
- **Main modification**: `gns3server/api/routes/controller/projects.py`
- **Function**: `get_projects()`
- **Batch check method**: `gns3server/db/repositories/rbac.py::RbacRepository.get_accessible_project_ids()`
- **Privilege dependency**: `gns3server/api/routes/controller/dependencies/rbac.py::has_privilege()`
- **Branch**: `feature/simple-user-isolation`
- **Base branch**: `master`
### Key Commits
1. Implemented basic created_by filtering
2. Implemented three-layer permission check (with logic issues)
3. Fixed to correct three-step check logic
### Design Evolution Process
#### Phase 1: Simple created_by Filtering
```python
user_projects = [p for p in all_projects() if p.created_by == user.username]
```
**Issue**: Didn't integrate with ACE system
#### Phase 2: Three-Layer Check (Wrong Version)
```python
# Used complex seen_project_ids blocking mechanism
# Issue: Broad ACE breaks user isolation
```
#### Phase 3: Three-Step Check (Correct Version)
```python
# Step 1: Batch ACE check via get_accessible_project_ids()
# Step 2: Filter direct_ace_ids by created_by
# Step 3: Resource pool projects (no created_by filter)
```
**Solution**: User isolation works even with broad ACE
### Relationship with RBAC Roadmap
This implementation addresses items mentioned in the roadmap:
- **Phase 1 (MVP)**: Basic project isolation implementation
- **Auto-ACE on create**: Still needs to be implemented separately
- **Template isolation**: Can use same design pattern
### Unsolved Issues
1. **Auto-ACE on project creation**: Part of roadmap Phase 1 — `create_project()` sets `created_by` but doesn't create ACE entries
2. **Template and image isolation**: Can apply same design pattern
3. **Default ACE configuration**: Need reasonable default permissions for Users group
### Design Limitations
1. **Project sharing only through resource pools**: No direct ACE configuration for sharing
2. **ACE configuration required**: Users need basic ACE to access system
3. **Performance considerations**: ACE check queries database for each project path
### Future Improvement Directions
1. **Auto-create ACE**: Automatically add ACE for creator when creating projects
2. **Default ACE strategy**: Configure reasonable default permissions for Users group
3. **Performance optimization**: Cache ACE check results to reduce database queries
This design achieves effective user isolation while maintaining RBAC system integrity, and solves the problem of broad ACE configurations breaking isolation.
**Key insight**: The Step 2 created_by filtering is the critical innovation that allows ACE and user isolation to coexist properly.
@@ -0,0 +1,53 @@
# Web Wireshark JWT Token Flow
## Background
Question: What is the purpose of `jwt_token` parameter in UDPLink and how is it transmitted?
## JWT Token Complete Flow
1. **User initiates request** → HTTP request with `Authorization: Bearer <jwt_token>`
2. **API layer extracts token** → `gns3server/api/routes/controller/links.py`:
```python
auth_header = http_request.headers.get("Authorization", "")
jwt_token = auth_header.replace("Bearer ", "") if auth_header else None
```
3. **Pass to Link layer** → `Link.start_capture(wireshark=True, jwt_token=xxx)`
4. **UDPLink forwards** → `UDPLink.start_capture()` calls `super().start_capture(jwt_token=jwt_token)`
5. **Start Web Wireshark container** → `Link._start_web_wireshark(jwt_token)` calls management script
6. **Container uses token** → `manage_wireshark.py` executes inside container:
```bash
curl -N -H 'Authorization: Bearer {jwt_token}' \
'http://controller:3080/v3/projects/{project_id}/links/{link_id}/capture/stream' | \
wireshark -i - -k -display :{display}
```
## Why More Verbose Than gns3-copilot?
**gns3-copilot approach**:
- All code within the same process
- Uses `contextvars` to store and retrieve token
- Downstream code directly calls `get_current_jwt_token()`, no need to pass through layers
**Web Wireshark approach**:
- Token needs to be passed across processes (management script is separate process)
- Token needs to be passed across containers (Docker container isolation)
- Cannot use `contextvars`, must pass through command line arguments
## Conclusion
UDPLink does not use this token, it only forwards it to the parent class. Ultimately used by the curl command inside the Web Wireshark container to authenticate with the GNS3 controller's capture stream API.
Token flow: **Client → GNS3 API → Link → UDPLink → Web Wireshark container → GNS3 capture stream API**
## Related Files
- `gns3server/api/routes/controller/links.py:114-115` - JWT token extraction
- `gns3server/controller/link.py:321-360` - Web Wireshark startup logic
- `gns3server/controller/udp_link.py:178` - UDPLink start_capture signature
- `gns3server/agent/web_wireshark/manage_wireshark.py:527-529` - Container curl command with JWT
+148
View File
@@ -0,0 +1,148 @@
# Xpra HTML5 Client Configuration
## Overview
Web Wireshark integration uses xpra's built-in HTML5 client to provide browser-based packet capture viewing. The xpra HTML5 client is located at `/usr/share/xpra/www` in the container.
## Configuration Methods
### 1. URL Parameters (Per-Session)
Control toolbar menu items via URL query parameters (default: true):
| Parameter | Menu Item | Description |
|-----------|-----------|-------------|
| xpramenu | Xpra Menu | Main menu (Server, Information submenus) |
| open_windows | Open Windows | List of open windows |
| fullscreen_button | Fullscreen | Fullscreen toggle button |
| keyboard_button | Keyboard | Keyboard layout/shortcuts |
| clipboard_button | Clipboard Copy | Clipboard copy functionality |
| sound_button | Audio | Audio toggle |
| cursor_lock_button | Lock Cursor | Game cursor lock mode |
```bash
# Examples
?fullscreen_button=false&sound_button=false
?xpramenu=false&keyboard_button=false&clipboard_button=false&sound_button=false&cursor_lock_button=false
```
### 2. default-settings.txt (Recommended - Global/Persistent)
Modify `html5/default-settings.txt` (INI format) for all users.
**Features:**
| Parameter | Default | Description |
|-----------|---------|-------------|
| keyboard | auto-detect | Enable keyboard input |
| keyboard_layout | us | Keyboard layout |
| clipboard | yes | Clipboard sharing |
| printing | yes | Printer forwarding |
| file_transfer | yes | File transfer |
| swap_keys | MacOS yes | Swap Command/Control keys |
| scroll_reverse_x | no | Reverse mouse X-axis |
| floating_menu | yes | Show floating menu |
| toolbar_position | top-left | Toolbar position |
| autohide | no | Auto-hide toolbar |
| sound | yes | Audio forwarding |
| video | 64-bit yes | Video decoding |
**Connection Options:**
| Parameter | Description |
|-----------|-------------|
| server | Server address |
| port | Port number |
| username | Username |
| password | Password |
| ssl | Enable SSL |
| encryption | Encryption type (AES-CBC/CTR/CFB) |
| key | AES encryption key |
| sharing | Allow session sharing |
| steal | Steal session |
| reconnect | Auto-reconnect |
| bandwidth_limit | Bandwidth limit (bits/s) |
| override_width | Client desktop width |
**Advanced Options:**
| Parameter | Default | Description |
|-----------|---------|-------------|
| audio_codec | auto-detect | Audio codec |
| encoding | auto | Image encoding (png/jpeg/webp/etc) |
| remote_logging | yes | Send logs to server |
| action | connect | Connection mode (start/shadow) |
| submit | yes | Show diagnostics on disconnect |
### 3. default_settings in Code
Set directly in `<script>` tag in index.html:
```javascript
const default_settings = {};
default_settings["xpramenu"] = false;
```
### Priority Order
URL parameter > default_settings object > default-settings.txt > code default value
## Server-Controlled Submenu Items
The **Server submenu** items are controlled by server hello message:
| Menu Item ID | Content | Control Source |
|--------------|---------|----------------|
| clock_menu_entry | Clock | server-time in hello |
| upload_menu_entry | Upload file | file-transfer or file.enabled |
| download_menu_entry | Download file | file-transfer or file.enabled |
| shutdown_menu_entry | Shutdown Server | client-shutdown in hello |
**Information submenu** (About Xpra, Session Info, Bug Report) and fixed items (Reload, Disconnect) are always displayed.
To hide "Shutdown Server", set environment variable before starting xpra:
```bash
export XPRA_CLIENT_CAN_SHUTDOWN=false
xpra start :{display} ...
```
Code: `xpra/server/base.py:40` - `CLIENT_CAN_SHUTDOWN = envbool("XPRA_CLIENT_CAN_SHUTDOWN", True)`
## Customizing Background Image
Background image is defined in `html5/css/client.css`:
```css
body {
margin: 0;
padding: 0;
overflow: hidden;
background-color: #021d3a;
background-image: url(../background.jpg);
background-position: center center;
background-repeat: no-repeat;
background-size: cover;
}
```
### Methods
1. **Replace image file**: Create/replace `html5/background.jpg`
2. **Modify CSS**: Change `html5/css/client.css` line 9:
```css
background-image: url(../your-image.jpg);
```
### Notes
- Supported formats: jpg, png, svg, etc.
- Recommended resolution: 1920x1080 or higher
- `background-size: cover` auto-stretches to fill
### Temporary Modification (via URL)
Cannot modify background via URL directly, but can use browser DevTools:
```css
body { background-image: url(your-image-url); }
```
## Related Files
- Container Dockerfile: `gns3server/agent/web_wireshark/docker/Dockerfile`
- Container HTML5 client path: `/usr/share/xpra/www`
+172
View File
@@ -0,0 +1,172 @@
---
name: gns3-api-testing
description: Use this skill when testing GNS3 server REST API endpoints with curl — covers JWT auth, common patterns, and marker/link examples.
version: 1.0.0
---
# GNS3 Server API Testing with curl
## Core Principle
Fixed routine for testing the GNS3 server API: **get a JWT token first, then send `Authorization: Bearer <token>` with every request.**
Default address `http://127.0.0.1:3080`, API prefix `/v3`.
---
## Authentication (always first)
```bash
TOKEN=$(curl -s -X POST http://127.0.0.1:3080/v3/access/users/authenticate \
-H "Content-Type: application/json" \
-d '{"username":"admin","password":"admin"}' \
| python3 -c "import sys,json; print(json.load(sys.stdin)['access_token'])")
```
Persist to a file for reuse (avoids re-logging in each time):
```bash
echo "$TOKEN" > /tmp/gns3_token.txt
TOKEN=$(cat /tmp/gns3_token.txt)
```
Then attach to every request:
```bash
AUTH="Authorization: Bearer $TOKEN"
curl -s -H "$AUTH" http://127.0.0.1:3080/v3/...
```
> **Endpoint note**: login is `/v3/access/users/authenticate`, **not** `/v3/auth/login`.
> OpenAPI spec is at `/openapi.json` (not `/v3/openapi.json`).
---
## Common Variables
```bash
BASE="http://127.0.0.1:3080/v3"
PID=<project_id>
LID=<link_id>
NID=<node_id>
AUTH="Authorization: Bearer $TOKEN"
```
---
## Generic Request Patterns
### GET (query)
```bash
curl -s -H "$AUTH" $BASE/projects/$PID/links | python3 -m json.tool
```
### POST (create) — with JSON body
```bash
curl -s -X POST -H "$AUTH" -H "Content-Type: application/json" \
-d '{"name":"foo","bpf":"icmp"}' \
$BASE/projects/$PID/links/$LID/markers
```
### HTTP status code only (body not needed)
```bash
curl -s -o /dev/null -w "HTTP %{http_code}\n" -X DELETE -H "$AUTH" \
$BASE/projects/$PID/links/$LID/markers/global-icmp
```
### Extract a field from the response
```bash
LID=$(curl -s -H "$AUTH" -X POST ... | python3 -c "import sys,json; print(json.load(sys.stdin)['link_id'])")
```
---
## Status Code Reference
| Code | Meaning |
|---|---|
| 200 | GET/PUT succeeded |
| 201 | POST created |
| 204 | DELETE succeeded (no body) |
| 401 | Not authenticated (token missing/expired) |
| 404 | Resource not found |
| 409 | Conflict (e.g. per-link edit of an inherited marker) |
| 422 | Schema validation failed (e.g. marker name starting with `global`) |
---
## Marker Cheat Sheet
### Project-level global marker definitions (inheritance)
```bash
# Create a def → fans out to every link automatically
curl -s -X POST -H "$AUTH" -H "Content-Type: application/json" \
-d '{"name":"icmp","bpf":"icmp","tag":1,"color":"#ff5722"}' \
$BASE/projects/$PID/marker-definitions
# List all defs + the link_ids each is bound to
curl -s -H "$AUTH" $BASE/projects/$PID/marker-definitions
# Update a def → syncs to every link
curl -s -X PUT -H "$AUTH" -H "Content-Type: application/json" \
-d '{"bpf":"icmp","tag":99}' \
$BASE/projects/$PID/marker-definitions/icmp
# Delete a def → removes the inherited marker from every link
curl -s -X DELETE -H "$AUTH" $BASE/projects/$PID/marker-definitions/icmp
```
### Per-link markers
```bash
# List markers on a link
curl -s -H "$AUTH" $BASE/projects/$PID/links/$LID/markers
# Create a private marker (name cannot start with "global")
curl -s -X POST -H "$AUTH" -H "Content-Type: application/json" \
-d '{"bpf":"tcp port 80"}' \
$BASE/projects/$PID/links/$LID/markers
# Delete (inherited markers return 409)
curl -s -X DELETE -H "$AUTH" $BASE/projects/$PID/links/$LID/markers/<name>
```
### Project-level aggregation query
```bash
curl -s -H "$AUTH" $BASE/projects/$PID/markers # all markers across links, flattened
```
---
## Link / Node Cheat Sheet
```bash
# List all links in a project (includes the markers field)
curl -s -H "$AUTH" $BASE/projects/$PID/links
# List nodes (check ports[].link_id to find free ports)
curl -s -H "$AUTH" $BASE/projects/$PID/nodes
# Create a VPCS
curl -s -X POST -H "$AUTH" -H "Content-Type: application/json" \
-d '{"name":"t1","node_type":"vpcs","compute_id":"local"}' \
$BASE/projects/$PID/nodes
# Start a node
curl -s -o /dev/null -X POST -H "$AUTH" $BASE/projects/$PID/nodes/$NID/start
# Create a link (both ends: node + adapter/port)
curl -s -X POST -H "$AUTH" -H "Content-Type: application/json" \
-d "{\"nodes\":[{\"node_id\":\"$N1\",\"adapter_number\":0,\"port_number\":0},{\"node_id\":\"$N2\",\"adapter_number\":0,\"port_number\":0}]}" \
$BASE/projects/$PID/links
```
> **Port occupancy**: VPCS has only one interface (port 0); once linked it cannot connect again.
> Confirm `ports[].link_id` is empty before creating a link; `"Port is already used"` means the port is taken.
---
## Gotchas
- **`POST /links` response may show `markers: []`** — the create response is serialized before the inheritance hook runs.
The inherited marker is actually applied; check `GET /links/{lid}/markers` or refresh `GET /links` to see it.
- **Restart gns3server after code changes** — the Python process does not hot-reload.
- **Wrap JSON bodies in single quotes** in the shell (double quotes inside); to interpolate a shell variable use `\"$VAR\"`.
- **Pipe long output through `python3 -m json.tool`** to pretty-print; extract fields with `python3 -c "import sys,json; ..."`.
@@ -0,0 +1,54 @@
---
name: documentation
description: Use this skill when creating or updating technical documentation under docs/ directory for GNS3 server
version: 3.0.0
---
# GNS3 Server Technical Documentation Standard
## Core Principle
Documentation answers: **What is this, how does it work at a high level.**
Code details are left to the codebase — readers can use AI to find implementation specifics.
---
## Document Structure
```markdown
# Feature Name
## Overview
[What it does, in 2-3 sentences]
## Architecture
[Mermaid diagram: components and their relationships]
## Business Process
[Mermaid sequence/flowchart: key flows]
## API Endpoints
[Table: Method | Path | Description | Privilege]
## Notes
[Known limitations, performance data, security — only when relevant]
```
---
## What to Include
- Mermaid diagrams for architecture and flow (GitHub renders natively)
- `graph` for component relationships
- `sequenceDiagram` for request/response flows
- `flowchart` for data processing steps
- API endpoint tables
- Request/response JSON examples (for interfaces)
- Performance data (only measured numbers, no estimates)
## What to Skip
- Code snippets — let readers search the codebase
- Verbose prose — use diagrams and tables
- Implementation details — link file paths instead of pasting code
+98
View File
@@ -0,0 +1,98 @@
---
name: memory
description: This skill should be used when the user asks to "record memory", "save to memory", "remember this", "create a memory", "add to memory", or discusses documenting technical decisions, architectural choices, or important case studies for the project. Use this skill to record project knowledge to the .claude/memory/ directory.
version: 1.0.0
---
# Memory Skill
## Overview
Record project-related technical decisions, important case studies, and lessons learned to the project memory directory, ensuring critical knowledge is managed with the code repository.
## When This Skill Activates
Activate this skill when user requests involve:
- "Record to memory", "save to memory", "add to memory"
- "Remember this", "create a memory"
- Documenting technical decisions, architectural choices
- Recording problem solutions and lessons learned
- Discussing API design, performance optimization, security implementations
## Memory Location
Project memory is stored in: `.claude/memory/`
## Memory Structure
```
.claude/
├── memory/
│ ├── MEMORY.md # Main index file
│ └── <topic>.md # Detailed records for specific topics
└── skills/
└── memory/
└── SKILL.md # This file
```
## What to Record
### ✅ Should Record
- Important architectural decisions and rationale
- Solutions to complex problems
- Key considerations for API design
- Performance optimization experiences
- Security-related implementation details
- Integration approaches with other systems
- Common errors and solutions
- Design changes based on user feedback
### ❌ Should Not Record
- Temporary debugging information
- Session-specific state
- Obvious code implementation details (code is documentation)
- Content already documented elsewhere
## Memory Template
```markdown
# <Topic Title>
## Background
<Why this record exists>
## Decision/Implementation
<What was done or how it was implemented>
## Rationale
<Why this approach was chosen, what alternatives were considered>
## Related Files
<Code files involved, use file_path:line_number format>
## Examples
<If there are code examples or other examples>
```
## Recording Process
1. **Identify content to record**
2. **Determine topic title** (concise, descriptive)
3. **Create record using template**
4. **Update MEMORY.md index**
5. **Save to .claude/memory/<topic>.md**
## Examples
- `web-wireshark-jwt-token-flow.md` - JWT token flow in Web Wireshark integration
- `docker-api-version-detection.md` - Docker API version dynamic detection implementation
## Best Practices
- Use clear, descriptive filenames
- Include references to related code files (with line numbers)
- Document "why" not just "what"
- Keep records updated, remove outdated ones
- Link to related memory files
+57
View File
@@ -0,0 +1,57 @@
# Version control
.git
.gitignore
.gitattributes
# CI / GitHub / Docker
.github
.whitesource
.dockerignore
# Editor / IDE
.idea
.vscode
.settings
.project
.pydevproject
.mr.developer.cfg
# Claude
.claude
# Python build artifacts
__pycache__
*.py[cod]
*.so
*.egg
*.egg-info
build/
dist/
eggs/
parts/
var/
sdist/
develop-eggs/
.installed.cfg
lib/
lib64/
.ropeproject
# Test & coverage
tests/
pytest.ini
.coveragerc
.coverage
.coverage*
.tox
.cache
.pytest_cache
nosetests.xml
# Virtualenv
env/
venv/
.venv/
# Editor backup files
*~
+40
View File
@@ -0,0 +1,40 @@
name: Bug report
description: Report a bug so we can fix it.
title: "[Bug]: "
labels: ["bug"]
body:
- type: textarea
id: what-happened
attributes:
label: What happened?
description: A clear description of the bug.
validations:
required: true
- type: textarea
id: reproduce
attributes:
label: Steps to reproduce
description: How can we reproduce this? Numbered steps if possible.
validations:
required: true
- type: textarea
id: expected
attributes:
label: Expected behavior
description: What did you expect to happen instead?
- type: input
id: version
attributes:
label: Version / commit
description: Which version or commit hash are you on?
- type: textarea
id: environment
attributes:
label: Environment
description: OS, runtime version, anything else that might be relevant.
- type: textarea
id: logs
attributes:
label: Relevant logs
description: Paste any relevant log output. This is automatically rendered as code.
render: shell
@@ -10,7 +10,7 @@ jobs:
name: Add issue to project
runs-on: ubuntu-latest
steps:
- uses: actions/add-to-project@v1.0.1
- uses: actions/add-to-project@v2
with:
project-url: https://github.com/orgs/GNS3/projects/3
github-token: ${{ secrets.ADD_NEW_ISSUES_TO_PROJECT }}
+3 -3
View File
@@ -56,11 +56,11 @@ jobs:
# your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v7
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
uses: github/codeql-action/init@v4
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
@@ -88,6 +88,6 @@ jobs:
exit 1
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
uses: github/codeql-action/analyze@v4
with:
category: "/language:${{matrix.language}}"
+68
View File
@@ -0,0 +1,68 @@
name: Build and Push Docker Image
on:
workflow_dispatch:
push:
tags:
- v2.*
- v3.*
jobs:
build:
runs-on: ubuntu-latest
env:
DOCKERHUB_ORG: ${{ vars.DOCKERHUB_ORG || 'gns3' }}
steps:
- name: Checkout code
uses: actions/checkout@v7
- name: Check for stable release
id: ver
run: |
TAG="${GITHUB_REF_NAME#v}"
echo "tag=$TAG" >> $GITHUB_OUTPUT
if echo "$TAG" | grep -qE '^[0-9]+\.[0-9]+\.[0-9]+$'; then
echo "stable=true" >> $GITHUB_OUTPUT
else
echo "stable=false" >> $GITHUB_OUTPUT
fi
- name: Set lowercase image name vars
if: steps.ver.outputs.stable == 'true'
id: names
run: |
echo "owner=$(echo '${{ github.repository_owner }}' | tr '[:upper:]' '[:lower:]')" >> "$GITHUB_OUTPUT"
echo "repo=$(echo '${{ github.event.repository.name }}' | tr '[:upper:]' '[:lower:]')" >> "$GITHUB_OUTPUT"
- name: Set up Docker Buildx
if: steps.ver.outputs.stable == 'true'
uses: docker/setup-buildx-action@v3
- name: Login to GitHub Container Registry
if: steps.ver.outputs.stable == 'true'
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Login to Docker Hub
if: steps.ver.outputs.stable == 'true'
uses: docker/login-action@v4
with:
registry: docker.io
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push
if: steps.ver.outputs.stable == 'true'
uses: docker/build-push-action@v6
with:
context: .
push: true
tags: |
${{ env.DOCKERHUB_ORG }}/${{ github.event.repository.name }}:${{ steps.ver.outputs.tag }}
${{ env.DOCKERHUB_ORG }}/${{ github.event.repository.name }}:latest
ghcr.io/${{ steps.names.outputs.owner }}/${{ steps.names.outputs.repo }}:${{ steps.ver.outputs.tag }}
ghcr.io/${{ steps.names.outputs.owner }}/${{ steps.names.outputs.repo }}:latest
@@ -12,13 +12,13 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
with:
fetch-depth: 0
ref: "gh-pages"
- uses: actions/setup-python@v5
- uses: actions/setup-python@v6
with:
python-version: 3.8
python-version: 3.12
- name: Merge changes from 3.0 branch
run: |
git config user.name github-actions
+8 -21
View File
@@ -4,45 +4,32 @@ on:
push:
branches:
- master
- 3.0
- 3.1
pull_request:
branches:
- master
- 3.0
- 3.1
jobs:
build:
runs-on: ${{ matrix.os }}
runs-on: ubuntu-latest
strategy:
matrix:
os: ["ubuntu-latest"]
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
#include:
# only test with Python 3.10 on Windows
# - os: windows-latest
# python-version: "3.10"
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
- name: Display Python version
run: python -c "import sys; print(sys.version)"
cache: pip
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install .[dev]
- name: Install Windows specific dependencies
if: runner.os == 'Windows'
run: |
python -m pip install -r win-requirements.txt
curl -O "http://www.win10pcap.org/download/Win10Pcap-v10.2-5002.msi"
msiexec /i "Win10Pcap-v10.2-5002.msi" /qn /norestart
python -m pip install .[ai-features,dev]
- name: Lint with flake8
run: |
+20
View File
@@ -67,3 +67,23 @@ env
venv
*venv
.ropeproject
# Claude Code settings (may contain API keys)
.claude/settings*.json
.claude/minmax-settings.json
.claude/zhipu-settings.json
.claude/settings.local.json
.claude/deepseek-settings.json
.claude/claude-settings.json
.claude/gemini-settings.json
.claude/grok-settings.json
!.claude/development.md # Exception: allow development docs
!.claude/skills/ # Exception: allow project skills
!.claude/memory/ # Exception: allow project memory
# PROJECT_CONTEXT.md # Commented out: allow tracking project context
# Tiktoken cache files
gns3server/agent/gns3_copilot/cache/tiktoken/
gns3.log
/configs/
+36 -5
View File
@@ -2,13 +2,44 @@
"scanSettings": {
"configMode": "AUTO",
"configExternalURL": "",
"projectToken" : "",
"baseBranches": ["master", "2.2", "3.0"]
"projectToken": "",
"baseBranches": []
},
"scanSettingsSAST": {
"enableScan": false,
"scanPullRequests": false,
"incrementalScan": true,
"baseBranches": [],
"snippetSize": 10
},
"checkRunSettings": {
"vulnerableCheckRunConclusionLevel": "failure"
"vulnerableCheckRunConclusionLevel": "failure",
"displayMode": "diff",
"useMendCheckNames": true
},
"checkRunSettingsSAST": {
"checkRunConclusionLevel": "failure",
"severityThreshold": "high"
},
"issueSettings": {
"minSeverityLevel": "LOW"
}
"minSeverityLevel": "LOW",
"issueType": "DEPENDENCY"
},
"issueSettingsSAST": {
"minSeverityLevel": "high",
"issueType": "repo"
},
"remediateSettings": {
"workflowRules": {
"enabled": true
}
},
"imageSettings":{
"imageTracing":{
"enableImageTracingPR": false,
"addRepositoryCoordinate": false,
"addDockerfilePath": false,
"addMendIdentifier": false
}
}
}
+705 -1
View File
@@ -1,5 +1,710 @@
# Change Log
## 3.1.0a5 19/08/2026
* Bundle web-ui v3.1.0a5
* Sync appliances
* copilot: sync CONSOLE_TYPES with the server ConsoleType enum
* feat: Enhance information displayed on the system status dashboard
* docker: cap GNS3_STOP_TIMEOUT at 210 s (controller stop budget)
* docker: harden the shm/devices/extra_configs/masking work (code review)
* docker: make the vendor graceful-stop grace period configurable (GNS3_STOP_TIMEOUT)
* docker: graceful stop for vendor NOS containers (SIGTERM + 60s grace)
* fix: UDP port allocation race causing link self-loop (one-way links)
* add support basic support for OpenBSD.
* docker: use the container's chown (not busybox) in vendor volume/perms path
* docker: mask systemd-udevd in privileged containers (GNS3_MASK_UDEV)
* docker: persist extra_configs in the template DB table
* docker: inject config files into containers via extra_configs
* docker: warn about low inotify/file-max and missing FUSE at connect
* docker: inject /dev/shm size and host devices via HostConfig from env
* appliance: expose custom_adapters on the v1-6 appliance model
* appliance: accept docker_exec console type in Docker appliances
* fix: recreate docker_exec console on reconnect after CLI exit
* vendor: drop the hardcoded /etc/network mount for SKIP_INIT containers
* vendor: skip /etc/network in volume bridge and permission passes
* fix: run _fix_permissions container-side on /gns3volumes mount targets
* fix: host-side permission fix + SKIP_INIT volume persistence docs
* refactor: move vendor NOS Docker support into VendorDockerVM subclass
* docker_exec console: wrap exec command in while-true loop so sr_cli restarts on quit
* Prototype: vendor NOS Docker node support (SR Linux etc.) — docker_exec console via Docker exec API (pty + hijacked HTTP + NAWS resize)
* fix: update CI configuration to use a single OS and remove Windows-specific dependencies
* fix: breaking change with dependency FastAPI v0.137.0 https://fastapi.tiangolo.com/release-notes/#specific-breaking-changes
* fix: suppress redundant console port setter log during Docker node create
* revert: restore force_close=True in compute._session()
* perf: parallelize batch NIO creation across nodes
* fix: scope marker reconcile to the current bridge/NIO
* cleanup: remove stray debug print in Dynamips node creation
* fix: reconcile marker filters in _ubridge_apply_markers (delete/update)
* log: suppress marker sink stats when no matches received
* log: add periodic marker.match throughput statistics
* marker: route marker.match to a dedicated project WS channel
* perf: raise marker UDP receive buffer from ~208KB to 8MB
* perf: move project-open marker inheritance to prepare phase
* perf: parallelise compute-side batch NIO update per node
* perf: batch update/delete marker-def fan-out too (full project-level)
* perf: batch marker-def fan-out to one PUT /nios/batch per compute
* log: add marker fan-out timing for project-open and interactive def-change
* tune: raise start_all concurrency from 3 to 10
* revert: drop _connect_nio thread-pool executor, restore async _ubridge_send
* fix: use socket.fromfd in send_batch_sync for Python 3.13 compat
* log: demote all per-node lifecycle INFO logs to DEBUG
* fix: await Dynamips.create_nio (it is async, unlike sync base)
* fix: use bound-method param count for Dynamips create_nio detection
* fix: Dynamips create_nio(node, nio_settings) takes extra arg + add tests
* log: add start/stop/close progress lines; revert start concurrency to 3
* log: lower per-node lifecycle logs in base_node to DEBUG
* perf: raise start/stop/suspend/reset-console concurrency from 3
* fix: race in concurrent node creation sending duplicate POST /projects
* log: lower per-node docker lifecycle logs to DEBUG
* log: emit progress lines for node and link loading on project open
* cleanup: remove project-open stage timing logs, lower NIO-added log to debug
* perf: skip per-link topology dump during project-open prepare
* perf: batch NIO dispatch on project open (one HTTP per compute)
* perf: cache compute.host_ip + add UDPLink.create timing log
* perf: dedicated 500-worker thread pool for ubridge batch I/O
* perf: offload per-NIO ubridge commands to thread-pool executor
* perf: enable TCP keep-alive for local compute HTTP requests
* debug: add per-command timing to _connect_nio ubridge calls
* perf: parallelize UDP port allocation and NIO creation in UDPLink.create
* fix: SIGKILL docker container on stop instead of 5s grace period
* fix: add event handling for deleted projects
* fix: tests after merging
* server: raise the open-files (RLIMIT_NOFILE) limit at startup
* marker: batch topology dumps in bulk fan-out (per-def on 500+ links was ~1 minute)
* marker: concurrent (bounded) definition fan-out
* marker: forward data_link_type on per-link marker create
* iou: skip already-installed markers in _ubridge_apply_markers (NIO update idempotency)
* marker: serial-link (WAN) support via data_link_type -> uBridge linktype
* iou: forward data_link_type on capture start so serial pcaps use the correct linktype
* ubridge: default the control transport to unix
* ubridge: require version >= 1.2.0
* fix: add missing HTTPException import
* mcp: drop direction from marker_definition tool
* marker: tighten marker name max length from 128 to 32
* marker: drop deleted marker from port NIO cache to stop empty pcap on restart
* marker: validate def BPF once, skip re-validation on inherited fan-out
* marker: point directional defs at BPF, refresh implementation doc
* marker: incremental apply, clear bridge map on uBridge stop
* marker: fine-grained filter ops, clean pcap on remove
* marker: restore direction on project load, reject tx/rx on definitions
* schema: accept direction='both' in marker schemas, normalize to None
* qemu: demote QEMU monitor connect and set_link logs to DEBUG
* marker: key _marker_filter_bridges by (name, link_id) so multi-link nodes toggle every copy
* marker: make per-filter toggle fall back to NIO rebuild when the marker isn't installed
* marker: make per-filter toggle a no-op when the marker isn't installed
* marker: rework pause/resume from project-wide to per-definition
* marker: fix PUT marker with an enabled-only body (bpf no longer required)
* marker: persist project-wide markers_paused to the .gns3 file
* marker: document pause/resume and instant enabled toggle
* marker: test enabled/pause/resume and instant toggle
* marker: drive uBridge enabled/pause/resume in real time
* marker: add direction-clear, routing, and ubridge transport tests
* marker: support clearing direction via explicit null / "both"
* marker: let callers pin the capture node via capture_node_id
* Name AF_UNIX sockets by node id for self-describing debuggability
* Detect unsupported -U flag and fail fast with a clear error
* Add configurable uBridge control channel transport (tcp/unix)
* fix(import): unvalidated symlink creation in import_project
* fix(qemu): move the bios options after the network ones
* fix(qemu): fix addition of QEMU RNG device causes interface names to change
* bugfix: docker workflow needs lowercase name
* fix(qemu): remove trailing space from RNG object argument
* Add MCP tools for traffic-insight marker feature
* Improve MCP tool descriptions for link_update, device_show_run, and link_reset
* Do not start nodes when deleting a project
* fix: correct always-true state check in DockerVM.stop()
* add bug report issue template
* Adds controller and compute API support for explicitly pulling or updating Docker images.
* feat(tests): increase maximum open file descriptors for test runs on Unix
* (feat): Implement functional Layer 1 keepalive support for IOU/IOL nodes
* Fix typo with ovmf_firmware_dir
* Fix issues after merging
* fix(builtin): hide GNS3 internal bridge interfaces from Cloud node
* feat(builtin): NAT node returns its interface IP info in asdict()
* fix(controller): refresh cloud/nat node interfaces from compute on GET
* refactor(builtin): use deterministic bridge name from switch UUID
* fix(builtin): bring kernel bridge UP after creation
* feat: migrate builtin EthernetSwitch from Dynamips ethsw to ubridge brctl
* fix(mcp): remove unreliable node_reload / node_reload_all tools
* fix(rbac): guard None current_user on websocket auth failure
* fix(mcp): thread token_version into console/download token minting
* fix(mcp): pass user.token_version when generating temp JWT from API key
* fix(marker): avoid reentrant-lock deadlock when uBridge starts lazily
* fix(marker): per-link attribution via link field in MARK signals
* fix(marker): set nio.markers on iou/dynamips/cloud nio update
* fix(marker): reject reserved global-prefix names only on create
* fix(marker): clean inherited markers on def delete
* feat(marker): add highlight_duration render hint
* fix(marker): fix inheritance hook, topology load, update sync, and asdict
* fix(marker): export MarkerDefinitionCreate from schemas package
* fix(marker): replace pydantic lookahead pattern with field_validator
* feat(marker): add project-level marker definition inheritance
* fix(marker): reserve "global" name prefix for future marker inheritance
* fix(marker): extend marker support to dynamips and cloud nodes
* fix(marker): enable marker support for docker nodes
* fix(marker): tolerate port conflict on marker listener startup
* fix(marker): validate marker name and expose enabled via REST
* fix(marker): uniquify auto-generated marker names per link
* feat(marker): add project-level marker aggregation endpoint
* feat(marker): add IOU node marker support via iol_bridge
* refactor(marker): converge to filter single-path model, remove dual-apply endpoints
* fix(marker): restore markers from topology on project load
* feat(marker): add optional color field to marker data model
* fix(marker): use allowlist in _choose_marker_side, align with _get_filter_node
* fix(marker): stop auto-deleting markers on node stop in node_updated
* fix(marker): ride markers on NIO so they survive node restart and project reload
* config: set marker_listen_port default to 3070 and document in config templates
* fix(marker): add _choose_marker_side to exclude non-uBridge node types
* Only set IOU images to be executable when importing images
* fix(import): project import does not move images symlinks into place
* Search for system UEFI that is compatible with Python < 3.12
* UEFI improvements for Qemu VMs * Add OVMF firmware directory configuration * Automatically add a Random Number Generator (RNG) device when using uefi option is enabled
* fix(docker): handle container name conflict automatically
* fix: gns3-server crashes on startup if "Open this project in the background" is active but there is a problem with that project
* Handle HTTPNotFound exception when retrieving compute status
* Replace ControllerForbiddenError with aiohttp.web.HTTPForbidden for project deletion error handling
* Fix: Check compute connectivity before open() during project deletion
* Use typed InterfaceStatus enum and simplify stats fallback in interfaces()
* Expose full interface address list and link state in cloud node API
## 2.2.61 30/07/2026
* Sync appliances
* fix(import): unvalidated symlink creation in import_project
* fix(qemu): fix addition of QEMU RNG device causes interface names to change
* fix(qemu): remove trailing space from RNG object argument
* fix: do not start nodes when deleting a project
* fix: correct always-true state check in DockerVM.stop()
## 2.2.60 15/07/2026
* Sync appliances
* Only set IOU images to be executable when importing images
* fix(import): project import does not move images symlinks into place
* Search for system UEFI that is compatible with Python < 3.12
* Add OVMF firmware directory configuration
* Automatically add a Random Number Generator (RNG) device when using uefi option is enabled
* fix(docker): handle container name conflict automatically
* fix: gns3-server crashes on startup if "Open this project in the background" is active but there is a problem with that project
* Handle HTTPNotFound exception when retrieving compute status
* Fix: Check compute connectivity before open() during project deletion
* API endpoints to manage base configuration files for templates
## 3.1.0a4 09/07/2026
* Bundle web-ui v3.1.0a4
* Add jwt_refresh_token_expire_minutes to sample configuration
* Remove deleted web-ui files from git
* .dockerignore ignore .dockerignore
* typo made me reconsider and move out to env
* defaulted DOCKERHUB_ORG repository variable
* Remove API docs from 2.2 after merging
* Update GitHub Actions workflows
* Fix web-wireshark docker build broken by xpra 6.5 release
* Update CI to install [ai-features,dev] instead of [ai-copilot,dev]
* Make AI features (AI Copilot + MCP) optional via [ai-features] extra
* Add refresh token mechanism documentation under docs/features/
* Add /refresh to allowed public endpoints list in route auth test
* Add stateless JWT refresh token mechanism
* Add project/node/link handler tests: 39 total, covering list/get/create/delete/start/stop/suspend/reload/console/update/fields
* Add MCP tool parameter consistency tests
* Fix appliance_install: add version parameter
* Add coordinate system note to docs
* Document canvas coordinate system in node_create x/y params
* Update docstring: batch concurrency from 10 to 100
* Remove unused import time
* Add list type check before nodes[0] access in _normalize_link_nodes
* Add fields type validation in create handlers
* Use pop() instead of pop(0) for O(1) port removal
* Fix review issues: key_prefix length, count validation, WAL log, timeout comment, pointless temp var
* Fix: pass name from TemplateUsage to add_node_from_template
* Remove unused imports (logging, log, select)
* Add warning for unconsumed pre-allocated UDP ports after link creation
* Remove FIXME comment about middleware in server.py
* Rename device_command_run_handler → device_show_run_handler to match tool name
* Rename device_command_run → device_show_run for clarity
* Fix device_command_run KeyError('commands'): tool desc said show_commands but backend expects commands
* Fix template_list return type annotation to match _run_handler_sync envelope
* Add performance optimization documentation
* Update MCP service docs: API key format, auth flow, tool parameters, concurrency
* Increase HTTP connection pool to 500/1000
* Increase BATCH_MAX_WORKERS and Pool concurrency from 20 to 100
* Remove final timing artifact in projects.py
* Remove remaining dead timing variables and imports
* Remove database warmup (proven ineffective - real bottleneck was bcrypt blocking event loop)
* Clean up all timing/debug logs
* Add memory: import validation best practice
* Fix: add missing UUID imports
* Generate fresh JWT on API key auth instead of returning raw key
* Optimize API key auth: O(1) lookup via UUID-embedded key format
* Fix: offload bcrypt.checkpw to thread pool to prevent blocking event loop
* Add timing to API key auth path and log api_keys count
* Replace SELECT 1 warmup with full database file read to warm OS page cache
* Add timing logs to auth dependency chain to identify 6s pre-handler delay
* Add granular timing to get_template: separate execute vs fetch time
* Warm up database connection pool on startup to avoid 8s cold-start penalty on first API request
* Fix: register WAL PRAGMA on sync_engine instead of Engine class for async compat
* Add timing logs to get_template to identify DB query bottleneck
* Add timing middleware to log slow requests (>1s) with [CTRL-TIMING] prefix
* Fix: _time → time in compute.py timing log
* Add [CTRL-TIMING] logs to controller create_node flow
* Fix: pass template_id to batch mode handler so top-level template_id works as default
* Add detailed timing logs to MCP node creation and HTTP client
* Add fields filter to template_list tool with description for AI
* Fix: add missing _filter_link_response function
* Pass name parameter through to controller API when creating node from template
* Add validation to compact link format with clear error messages
* Add compact array format for link node entries to reduce token usage
* Add fields filter to link_create tool
* Reduce md5sum cache write failure log level from error to warning
* Add fields filter to node_create tool description for AI
* Optimize MCP create_node: support inherited template_id and default fields filter
* Increase MCP HTTP client timeout from 10s to 30s
* Enable SQLite WAL mode to fix 'database is locked' errors under concurrent API requests
* Cache IOU image default values per image path to avoid redundant subprocess calls
* Increase node and link creation concurrency from 5 to 20
* Fix: revert IOU lock optimization, serialize IOU node creation for correct application_id assignment
* Performance: accelerate project opening with parallel link creation and batch UDP port allocation
* feat: Add batch link_ids to link_delete/link_reset, fields filter to link_list
* feat: Add fields filter to link_list
* feat: Add batch node_ids to node_delete
* fix: Convert http to ws scheme in node_console WebSocket URL
* feat: Add batch link_ids to link_capture_download
* feat: Add batch link_ids to link_capture_start/stop
* fix: Store username in gns3_ctx during auth, use for short-lived download JWTs
* fix: Generate independent short-lived JWT for pcap download
* revert: Remove _configs_map changes in tools_v2 (handled by template renderer now)
* fix: Merge commands for duplicate device_names in _configs_map
* fix: Actually pass template param to device_config/command handlers
* fix: Correct Jinja2 template commands_field per tool type
* feat: Jinja2 template support in device_command_run
* feat: Jinja2 template support in device_config_send
* feat: Add batch node_ids support to node_start/stop/reload/suspend
* feat: Add fields filter to appliance_list
* feat: Add fields filter to node_list
* feat: node_get fields filter — match controller Node schema fields
* fix: Set auto_close=False on project_create so projects stay open when clients disconnect
* feat: Add batch mode to node_create and link_create (parallel, max 10 workers)
* fix: Pass API key directly instead of generating short-lived JWT
* feat: API key lifecycle — revoke/restore/delete
* fix: Rename revoke_api_key → delete_api_key
* fix: Hard-delete API keys instead of soft delete (revoked flag)
* feat: Support API keys in REST API authentication (reuse gns3_ prefix keys)
* fix: Lazily access db engine for API key validation
* fix: Add missing updated_at column to api_keys table
* fix: Export ApiKeyCreate from schemas package
* feat: Add API Key support for MCP authentication
* fix: Validate JWT token exp claim — was silently ignored after migration to joserfc
* fix: Add image field to template_create, document type-specific params in description
* fix: Remove .json() calls on 204 responses for prune/install images
* fix: Skip always-running nodes in start_all/stop_all
* fix: Add missing rotation parameter to drawing_update MCP tool
* fix: Map MCP device_command_run parameter to tool's expected field name
* fix: Update link_reset description to match actual behavior (delete + recreate)
* fix: Remove unsupported description param from project_create
* fix: Fix symbol_get/upload/delete handlers for correct API paths
* feat: Log registered MCP tools at startup
* fix: Type compute_id as uuid.UUID to reject non-UUID values at MCP input layer
* fix: Require UUID for compute_get/images, remove 'local' string default
* feat: Add device configuration MCP tools (config_send, command_run, vpcs_config_set)
* refactor: unify MCP tool naming to <module>_<action> convention
* feat: Add symbol upload/delete, project load, and locked check MCP tools
* feat: Add image management MCP tools
* feat: Add symbol and appliance MCP tools
* feat: Add node bulk ops, project lock, and server info MCP tools
* feat: Add snapshot and drawing MCP tools
* refactor: unify MCP handlers to use http_call directly, relocate node file ops to Node class
* feat: Add node file operations as MCP tools (list, get, write, delete)
* Add comment about rootful Docker permissions at container start
* Fix _fix_permissions test: set process.returncode=0 and update assertion
* Fix list_node_files PermissionError on os.scandir
* Fix _fix_permissions error handling and list_node_files PermissionError
* Add async_iterable_to_stream utility to avoid aiohttp compatibility issues
* Add descriptive detail to 403 errors in compute file endpoints
* Fix silent file write failure in write_compute_project_file
* feat: Node file streaming, recursive listing, file type detection, and file delete
* Update README tool descriptions: .txt → .md
* Update README tool descriptions to mention Markdown format
* Add MCP project tools: update, duplicate, and README operations
## 3.1.0a3 06/06/2026
* Bundle web-ui v3.1.0a3
* fix: update MCP link tools descriptions with detailed filter info
* fix: correct MCP nodes and links tool parameter handling for nested kwargs
* fix: correct MCP template tool parameter handling for nested kwargs
* feat: add client information logging to MCP connection rejection
* refactor: replace MCP ready state polling with asyncio.Event
* fix: revert duplicate image check to fix failing tests
* fix (templates): Add ordering to handle the duplicate cases gracefully
* fix(templates): Database error detected when saving a template with a disk image change
* fix: return 503 error on MCP server ready timeout
* fix: add MCP server ready check to prevent initialization errors
* fix: resolve MCP server URL host via default route IP when bound to 0.0.0.0
* fix: correct MCP transport security config to actually allow all hosts by default
* feat: add configurable MCP transport security settings via gns3_server.conf
* fix: add load_feature_skills() to properly load network planning features
* refactor: convert all MCP tool parameter descriptions to Annotated+Field
* fix: remove platformdirs upper bound to resolve fastmcp-slim dependency conflict
* fix: add missing fastmcp dependency to resolve CI test failures
* refactor: move all imports to top of __init__.py
* test: add /v3/mcp/ to allowed public endpoints
* fix: remove console_host/port from get_node_console_info
* feat: add get_node_console_info tool
* feat: add 3 Compute MCP tools - Add list_computes, get_compute, get_compute_images - Total MCP tools: 29
* feat: add 5 Template MCP tools
* feat: add Node and Link MCP tools, update copyright - Add 9 node tools and 5 link tools - Update copyright year to 2026, add author
* feat: complete MCP SSE transport with JWT auth
* feat: support Authorization header and query param for MCP token
* feat: implement standard MCP protocol with SSE transport
* feat: add MCP (Model Context Protocol) service with project tools
* Add project memory: Docker container stop delay analysis
* Remove extra blank line from merge
* Revert container state detection in create()
* Fix Docker VM tests for container status detection on node creation
* Add running project check for fast duplication
* Move running project check before fast duplication
* Add running project check for fast duplication
* Fix Docker container status detection on node creation
* Fix web-ui update script to handle custom GitHub URL changes
* Fix unnecessary Docker container recreation when renaming a project
* Fix project rename and duplicate issues
* Fix double deletion issue in remove_resource_from_pool
* Complete fix: delete resource records when deleting resource pool
* Apply fix from PR #2315: delete resource from resource table when removing from pool
* Remove deprecated 'PermissionsStartOnly' setting for Systemd service. Ref #1830
* Optimize project loading by implementing parallel node creation
* Fix delay filter validation: ensure delay: [0, X] returns proper error message
* Fix packet filter validation tests: use correct ubridge filter type names
* Update tests to match show_interface_labels default change
* Set default value of show_interface_labels to True
* Optimize project variable updates to use parallel node processing
* Fix ghost Docker nodes causing 60-second VNC timeout on variable updates
* Fix Docker container variable compatibility with Pydantic models
* Fix delay latency minimum: ubridge rejects latency <= 0
* Improve packet filter validation: use tcpdump, handle multi-line BPF, safe project load
* Add packet filter parameter validation to prevent ubridge errors
* chore: update GNS3 skills repository to official organization
* docs: update RBAC user isolation design doc to match actual implementation
* docs: update skills repo URL in command-security.md
* docs: remove Chinese overview docs, keep only English versions
* docs: add overview docs for packet analysis, fault injection, and AI assistant
* feat: add mermaid-to-SVG conversion script with environment setup
* test: fix privilege count assertions after adding LLMConfig privileges
* docs: add user node limit roadmap
* test: fix RBAC test to match implementation logic
* test: add --prefix and --cleanup-only parameters to benchmark script
* perf: batch RBAC permission checking for GET /projects
* test: add benchmark script for GET /projects performance testing
* fix: prevent duplicate projects when user projects are in resource pools
* fix: check both regular ACEs and resource pool ACEs for proper access control
* test: update RBAC test to use test_user.username for user isolation
* docs: add Phase 9 and 10 user self-registration and email service
* docs: add Phase 8 per-user project namespace to roadmap
* docs: add Phase 7 resource pool renaming to roadmap
* feat: add alembic migration for LLMConfig privileges
* feat: add independent LLMConfig permissions for AI profile management
* docs: add Phase 6 frontend permission query API to roadmap
* docs: add Phase 5 ACE architecture refactoring plan to roadmap
* feat: remove resource pools from 'all endpoints' list
* refactor: add efficient get_aces_for_path method for resource pool checks
* feat: prevent deletion of resource pools used by ACE configurations
* docs: update RBAC user isolation roadmap and add design memory
* feat: fix permission check logic to properly handle ACE and user isolation
* feat: implement layered permission checks for proper user isolation and sharing
* feat: implement simple user isolation based on project ownership
* fix: clean BPF syntax error message and specify loopback interface
* feat: add BPF syntax validation using tshark
* feat: add show_filters_icon parameter to packet filter tool
* docs: add GNS3 appliance loading mechanism to memory
* feat: add packet filter management tool for GNS3-Copilot fault injection
* fix: update test_json expected output to include show_filters_icon field
* fix: add getattr fallback to show_filters_icon property for backward compatibility
* feat: add show_filters_icon property to Link for controlling Web UI filter icon display
* fix: ensure show_filters_icon is always returned in API responses
* feat: add show_filters_icon property to Link for controlling Web UI filter icon display
* fix: remove explicit paramiko pin to resolve dependency conflict with netmiko
* fix: update netmiko to 4.7.0 and pin paramiko>=5.0.0 to fix CVE-2026-44405
* fix: close DockerHTTPClient session to prevent UnixConnector leak in Web Wireshark
## 3.1.0a2 12/05/2026
* Bundle web-ui v3.1.0a2
* fix: surface tshark stderr errors to LLM, not just log them
* fix: hint LLM about -c behavior when tshark returns empty with -c
* fix: update search_fields examples to use existing field names
* fix: reject multi-keyword search_fields, single keyword only
* fix: document action/query params in PacketAnalysisTool description
* feat: add search_fields action to PacketAnalysisTool for real-time field lookup
* feat: validate tshark -e field names before running analysis
* fix: document -c behavior caveat in PacketAnalysisTool description
* fix: demote loader-level load logs to DEBUG, info already reported by manager
* fix: promote skills/prompts load logs to INFO level
* fix: load packet analysis protocols during skills initialization
* feat: add PacketAnalysisSkillsTool
* fix: remove remaining quoted example from tool input docstring
* fix: use shlex.split for tshark_args parsing
* feat: add packet analysis workflow to tool description
* refactor: remove old PacketCaptureTool, replaced by PacketAnalysisTool
* feat: add protocol-oriented packet analysis tool
* docs: add roadmap docs for planned features
* feat: require superadmin for /reload/skills endpoint
* fix: remove background init timeout and allow skills reload retry
* deps: add PyYAML and GitPython to ai-requirements.txt
* feat: add fault injection system and external skills repository
* fix: Disable DeepSeek thinking mode to avoid reasoning_content errors
* fix: Skip offline compute nodes during project import
* fix: Import status module in project.py
* fix: Handle connection errors when stopping node console
* refactor: Remove MD5 calculation from node files listing
* fix: Export NodeFile schema to schemas module
* feat: Add detailed metadata to node files listing
* feat: Add API endpoint to list node files
* Fix snapshot restore does not work after export/import of project
* Warn to use 64-bit IOU images. Fixes #2716 32-bit IOU image support has been removed from the GNS3 VM.
* Fix: Improve Docker container deletion error logging
* Fix: Add exponential backoff to compute reconnection attempts
* Fix: Check compute connectivity before node creation in open()
* Fix: Check compute connectivity before open() during project deletion
* Fix: Add error logging when closing/deleting projects on computes
* Fix: Skip add_compute for existing computes when opening projects
* feat: add AI provider config files to .gitignore
* Fix: Use _computes instead of _project_created_on_compute for deletion check
* Fix: Check compute connection status before project deletion
* fix: extend remote compute image path conversion to all node types
* fix: send relative image path to remote compute nodes
* fix: handle directory access for static web-ui routes
* fix: add duplicate name validation when updating templates
* fix: avoid AttributeError when password is not provided in compute update
* fix: improve hostname validation error messages with allowed characters
* feat(compute): Fix Dynamips to listen on the internal port when ssh is used for auxiliary console
* feat(compute): Fix ambiguous log message related to the active console transport.
* feat(compute): Updated server host key to use the secure ssh-ed25519 instead of old ssh-rsa algorithm
* feat(compute): Fix missing SSH wrapping for auxiliary console
* feat(compute): Added AsyncioSSHServer dedicated tests
* feat(compute): add SSH console type support
* fix: include error details in LLM model config failure response
* Fix telnet console silent-proxy hang on non-ConnectionError exit (#2344)
## 2.2.59 08/05/2026
* Sync appliances
* Fix snapshot restore does not work after export/import of project
* Warn to use 64-bit IOU images. Fixes #2716 32-bit IOU image support has been removed from the GNS3 VM.
* Fix telnet console silent-proxy hang on non-ConnectionError exit (#2344)
## 3.1.0a1 25/04/2026
* New completely revamped Web interface.
* Add GNS3 Copilot AI agent back-end
* Web-wireshark: packet capture and live traffic analysis in the browser
* Fix using SSL with remote computes. Fixes https://github.com/GNS3/gns3-gui/issues/3733
* Add --url and --branch parameters to update-bundled-web-ui.sh
* Fix bug when custom_adapters is None in port_factory.py
* Have optional body in start node API endpoint
* Fix snapshots after merging
* Potential fix for pull request finding 'CodeQL / Information exposure through an exception'
* Refactored the way snapshots are handled in the server
* fix(symbols): add authentication to symbol raw endpoint
* fix(symbols): add CORS headers to symbol file responses
* Allow to find new project in sub dirs
* Enhancement: Refresh projects list without restarting the GNS3 server
* Fix accessing username in authentication.py
* Improved exception handling and telnet client disconnect
* Use get method for registry_version to avoid KeyError
* Add support for appliance format version 8
* Convert database template fields from PickleType to JSON
* feat(statistics): distinguish open vs closed project nodes
* fix(statistics): handle closed projects where nodes/links are dicts
* feat(controller): extend /statistics API with project, node and link stats
* Use FastAPI Lifespan Events
* feat(link): add control_offset field to LinkStyle
* feat(symbols): add DELETE API for removing custom symbol files
* Enhancement: Re-write current Telnet server implementation using telnetlib3 library
* feat: improve error handling for API configuration errors
* Enhancement Added link type support on the server side
* Ignore HTTPException received when closing a project
* Prevent router to be duplicated when running and add tests
* Fixed issue #1605 regarding Cisco slots causing configs to break
* feat: add VNC WebSocket console support for Docker and QEMU nodes
* Return 405 error for unsupported actions for other node types
* feat(telnet_server): improve error handling and connection management
* fix(telnet): handle connection race condition during broadcast
* Change first() to one_or_none() for image query
* Fix for duplicating Qemu nodes
* Minor optimizations to logic
* Fixed an issue with the server causing errors when uploading large images + Preparation for a smarter check when uploading images
* Implemented a smarter node_type system
* docs: restructure copilot docs and update node tools
* fix(acl): correct endpoint paths for users, groups, and roles
* fix(iou): return 405 error for unsupported suspend operation
* feat(api): enhance max_tokens field with robust null handling
* feat(docs): add AI prompting guide for configuration templates
* Deactivate 'use default IOU values' by default and update RAM/NVRAM values
* Make sure the node shows as stopped when the wrap console cannot be stopped. Ref https://github.com/GNS3/gns3-registry/pull/1010
* chore: update author name and copyright headers
* feat(docs): enhance AI chat API documentation with examples and details
* feat: add .claude directory to gitignore for API key security
* Revert "Fix QEMU serial console artifacts by filtering ANSI CPR responses in …"
* docs: add troubleshooting guide for datetime timezone issue
* Drop support for Python 3.9
* feat: add multi-user concurrency control design document
* docs: add troubleshooting guide for force kill residual processes
* Fixed the path traversal check
* Fix image handling
* Qemu VMs: only compute MD5 checksums for existing disks
* Copy config files and database from a previous version if it exists
* Add enable_http_auth in gns3_server.conf sample
* feat(compute): option to disable compute authentication
* tests(controller,port): short_name finishes successfully when port name is None
* fix(controller,port): handle None port name to prevent TypeError in short_name method
* fix(docker): handle container name conflict automatically
* feat(virtualbox): add fallback for VBoxManage executable detection
* feat: simplify port name assignment logic
* feat(ports): add null checks for port name in short name generation
* Remove unneeded rbac_repo in get_templates
* Use lists for tags instead of dicts
* Fix for resizing Qemu VM disks
* feat(templates): safely access settings dict keys
* feat(api): fix ETag handling and JSON serialization in template endpoint
* feat(api): add tag filtering to nodes and templates endpoints
* Update project schema and fix tests
* Add 'created_by' field to track project creator
* Fix updating packet filters
* Add vendor, model and netmiko_device_type fields to templates
* fix: handle RuntimeError in notification_manager emit()
* fix: busybox static link detection on Alpine/musl
* Fix bug when updating compute password to be empty
* Cancel tasks if controller cannot be started
* Enable parallel compression with zstandard. Fixes #2324
* Add Dynamips to Dockerfile dependencies
* Take populated disks into consideration when calculating PCI device ID
* Fix telnet keepalive options on macOS
## 2.2.58.1 12/04/2026
* Sync appliances
## 2.2.58 10/04/2026
* Bundle web-ui v2.2.58
* Sync appliances
* Snapshot refactoring
## 2.2.57 23/03/2026
* feat(telnet_server): improve error handling and connection management
* Deactivate 'use default IOU values' by default and update RAM/NVRAM values
* Make sure the node shows as stopped when the wrap console cannot be stopped
* tests(controller,port): short_name finishes successfully when port name is None
* fix(controller,port): handle None port name to prevent TypeError in short_name method
* fix: busybox static link detection on Alpine/musl
* Take populated disks into consideration when calculating PCI device ID
## 2.2.56.1 28/01/2026
* Fix telnet keepalive options on macOS
* Upgrade dependencies
* Drop Python 3.8 support
## 3.0.6 28/01/2026
* Sync appliances
* Fix creating Qemu disk image. Fixes #2542
* Disable checking for available disk space. Ref #2548
* Set default location of udhcpc in "/etc/network/udhcpc". Fixes #2582
* Support for Python 3.14
* Fix non-ASCII characters in project names
* Increase DB engine pool size and max overflow
* Enable ip cef in IOU L2 and IOU L3 base configs
## 2.2.56 21/01/2026
* Set default location of udhcpc in "/etc/network/udhcpc". Fixes #2582
* Upgrade pywin32 to v311
## 2.2.55 19/11/2025
* Docker API version requirements and process to handle older daemons
* Use docker_name property when creating a container
* Set name of container in Docker
* Support for Python 3.14
* Enable ip cef in IOU L2 and IOU L3 base configs
* Only build and push Docker image when releasing a new version
* Fix pushing Docker image to Docker hub
* Update security issues in README.md. Fixes #2535
* Add missing 'nat' template type in schema. Fixes #2529
* Resolve deprecation warnings of regex library
## 3.0.5 14/05/2025
* Bundle web-ui v3.0.5
* Sync appliances
* Use Ubuntu 24.04 LTS in Dockerfile. Ref #2523
* Fix tests and require Qemu version >= 2.4
* Fix adding pci_bridges to qemu vms
* Resolve deprecation warnings of regex library
* Remove OVMF_CODE_4M.secboot.fd and associated code
* Add edk2-stable202502 UEFI firmwares and try to first use firmwares from the ovmf package if installed. Fixes #2494
* Prettify TPM error message
* Fix bug in qemu_stdout message
* Try to detect swtpm and AppArmor issue. Ref https://github.com/GNS3/gns3-gui/issues/3725
* Fix Docker logs decoding. Ref #2522
* Add delay after starting a Docker container and adding connections in uBridge. Ref #2522
* Fix TypeError when reading Docker container logs. Ref #2522
* Allow an image to be uploaded to the controller again even if it is already in the database
* Fix controller not reporting an error if invalid options are passed to Qemu command line. Fixes #2517
* Replace "Docker hub" by "Docker repository" because it is possible to use different repositories
* Fix unable to add NAT cloud after configuring “allowed_interfaces” in configuration file. Fixes #2508
* Fix interface information API endpoint for Cloud/NAT devices
* Upgrade Jinja2 to v3.1.6. Fixes #2515
* Fix packet capture for links connected to a cloud node. Fixes #2513
* fix: check if remote-install.sh is being run as root
## 2.2.54 21/04/2025
* Bundle web-ui v2.2.54
* Add new method to find the IP address of a VBox GNS3 VM + allow NAT Network
* Add edk2-stable202502 UEFI firmwares and try to first use firmwares from the ovmf package if installed. Fixes #2494
* Try to detect swtpm and AppArmor issue. Ref https://github.com/GNS3/gns3-gui/issues/3725
* Fix Docker logs decoding. Ref #2522
* Add delay after starting a Docker container and adding connections in uBridge. Ref #2522
* Fix TypeError when reading Docker container logs. Ref #2522
* Replace "Docker hub" by "Docker repository" because it is possible to use different repositories
* Upgrade dependencies
* Improvements for remote-install.sh
## 3.0.4 25/02/2025
* Require minimum 8 characters for passwords
* Bundle web-ui v3.0.4
* Upgrade dependencies
* Fix auto idle-pc for IOS templates
* Update schemas to add constraints for values like ram, nvram etc.
* Install custom symbols. Fixes #2492
* Fix VMware add ubridge connection. Fixes #2490
## 3.0.3 22/01/2025
* Bundle web-ui v3.0.3
* Refactor command line arguments parsing
* Fix capture on IOU/IOL links. Fixes #2477
* Use Python 3.9 to publish API documentation
* Upgrade dependencies
## 2.2.53 21/01/2025
* Bundle web-ui v2.2.53
* Add more information when patching .vbox file. Ref https://github.com/GNS3/gns3-gui/issues/3542
* Increase timeout to run compute HTTP queries. Fixes #2461
* Use 'allow_methods="*"' in aiohttp_cors.ResourceOptions(). Fixes #2459
* Upgrade dependencies
* Update remote-install.sh to support a custom repository and the deb822 source format
* Fix: do not use the iourc file if IOU licence check is not enabled
## 3.0.2 03/01/2025
* Bundle web-ui v3.0.2
@@ -31,7 +736,6 @@
## 2.2.52 02/12/2024
* Bundle web-ui v2.2.52
* Sync appliances
* Remove restrictions based on file extension when listing images and fix ELF header checks
* Fix use project name instead of ID for fast duplication when running local server. Fixes #2446
* Overwrite user resources when the originals have changed.
+4 -4
View File
@@ -1,4 +1,4 @@
FROM ubuntu:focal
FROM ubuntu:noble
WORKDIR /gns3server
@@ -31,7 +31,7 @@ RUN add-apt-repository ppa:gns3/ppa && apt update && DEBIAN_FRONTEND=noninteract
COPY . /gns3server
RUN mkdir -p ~/.config/GNS3/3.0/
RUN cp scripts/gns3_server.conf ~/.config/GNS3/3.0/
RUN mkdir -p ~/.config/GNS3/3.1/
RUN cp scripts/gns3_server.conf ~/.config/GNS3/3.1/
RUN python3 -m pip install .
RUN python3 -m pip install --break-system-packages --ignore-installed .
+370
View File
@@ -0,0 +1,370 @@
# GNS3 Server - Project Context for Claude Code
> This document provides project context information for the Claude Code AI assistant to work more efficiently.
---
## Project Overview
**Project Name**: GNS3 Server
**Description**: Network simulation server supporting the GNS3 network virtualization platform
**Primary Language**: Python
**Current Branch**: `feature/ai-copilot-bridge`
**Main Branch**: `master`
---
## Project Structure
```
gns3server/
├── agent/ # AI Copilot related modules
│ └── gns3_copilot/ # GNS3-Copilot AI Assistant
│ ├── agent/ # LangGraph Agent implementation
│ ├── tools_v2/ # Tool collection (device config, display commands, etc.)
│ ├── utils/ # Utility functions and helper modules
│ ├── gns3_client/ # GNS3 API client
│ ├── prompts/ # AI prompt templates
│ ├── agent_service.py # Agent service layer
│ └── project_agent_manager.py # Project Agent manager
│
├── api/ # API routes
│ └── routes/controller/ # Controller API
│ └── chat.py # AI Chat API endpoint
│
├── controller/ # Core controller
│ ├── project.py # Project management
│ └── ...
│
├── schemas/ # Pydantic data models
│ └── controller/
│ └── chat.py # Chat API Schema
│
├── db/ # Database module
│ └── tasks.py # Database tasks
│
└── docs/ # Documentation
└── gns3-copilot/ # AI Copilot documentation
```
---
## Key Module Descriptions
### 1. AI Copilot Data Flow
```
Tool Layer → Agent Layer → Service Layer → API Layer → Frontend
```
**Tool Output Format**: All tools return `dict` or `list[dict]`, automatically serialized to JSON in the Service layer
**Key Files**:
- `agent/gns3_copilot/tools_v2/*.py` - Tool implementations
- `agent/gns3_copilot/agent_service.py:460-473` - Event conversion logic
- `agent/gns3_copilot/utils/parse_tool_content.py` - Tool result parsing
### 2. SSE Event Types
| Type | Description |
|------|-------------|
| `content` | AI text streaming output |
| `tool_call` | LLM tool invocation (progressive parameters) |
| `tool_start` | Tool execution started |
| `tool_end` | Tool execution completed (output as JSON) |
| `error` | Error message |
| `done` | Stream ended |
### 3. Important Tools
| Tool Name | File | Function |
|-----------|------|----------|
| `ExecuteMultipleDeviceCommands` | `display_tools_nornir.py` | Read-only diagnostic commands |
| `ExecuteMultipleDeviceConfigCommands` | `config_tools_nornir.py` | Configuration commands |
| `VPCSMultiCommands` | `vpcs_tools_telnetlib3.py` | VPCS virtual PC commands |
| `GNS3TemplateTool` | `gns3_get_node_temp.py` | Get GNS3 templates |
| `GNS3CreateNodeTool` | `gns3_create_node.py` | Create nodes |
---
## Code Standards
### Static Code Analysis (Flake8)
The project uses **flake8** for static code analysis. You must run checks before committing code.
#### Running Flake8
```bash
# Activate virtual environment
source venv/bin/activate
# Check a single file
flake8 path/to/file.py
# Check the entire project
flake8 gns3server/
# Check a specific directory
flake8 gns3server/utils/
```
#### Common Flake8 Error Codes
| Code | Description | Example |
|------|-------------|---------|
| **F401** | Module imported but unused | `import os` never used |
| **F841** | Local variable assigned but never used | `x = 1` not used later |
| **F824** | `global` declaration never assigned | `global _bar` never assigned in scope |
| **E501** | Line too long (>79 characters) | Single line exceeds 79 characters |
#### Fix Examples
```python
# F401: Remove unused imports
# Bad
import os # Never used
# Good: Delete this line
# F841: Mark as intentionally unused
# Bad
def foo():
x = 1 # Never used
# Good: Use or mark as intentionally unused
def foo():
x = 1
_ = x # Mark as intentionally unused
# F824: Remove unnecessary global declaration
# Bad
def foo():
global _fernet # Never assigned in this scope
# Good: Remove global statement (if only reading)
def foo():
# Just read the global variable, no global declaration needed
pass
# E501: Break long lines
# Bad
raise RuntimeError("Encryption not initialized. Call init_encryption() first.")
# Good
raise RuntimeError(
"Encryption not initialized. Call init_encryption() first."
)
```
#### Ruff (Alternative)
[Ruff](https://docs.astral.sh/ruff/) is a faster Python linter that can replace flake8, isort, black, and more.
```bash
# Install
pip install ruff
# Usage
ruff check gns3server/ # Check code
ruff check --fix gns3server/ # Auto-fix
ruff format gns3server/ # Format code
```
### Python Code Style
```python
# 1. Type annotations
def process_data(input_data: dict[str, Any]) -> list[dict[str, Any]]:
pass
# 2. Error handling
try:
result = tool.invoke(args)
except Exception as e:
logger.error("Tool failed: %s", e, exc_info=True)
return {"error": str(e)}
# 3. JSON serialization
if not isinstance(output, str):
output = json.dumps(output, ensure_ascii=False, indent=2)
```
### Import Order
```python
# 1. Standard library
import asyncio
import json
import logging
# 2. Third-party libraries
from langchain.tools import BaseTool
from pydantic import BaseModel
# 3. Local modules
from gns3server.agent.gns3_copilot.agent import agent_builder
from gns3server.controller import Controller
```
### Copyright Header
```python
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
```
---
## Git Commit Standards
### Commit Message Format
```
<type>(<scope>): <subject>
<body>
<footer>
```
### Type Categories
| Type | Description | Example |
|------|-------------|---------|
| `feat` | New feature | `feat(copilot): add user authentication` |
| `fix` | Bug fix | `fix(api): correct JSON serialization` |
| `docs` | Documentation update | `docs: update API design doc` |
| `refactor` | Refactoring | `refactor(agent): simplify state management` |
| `chore` | Build/toolchain | `chore: update dependencies` |
### Example
```
fix(copilot): serialize tool output to standard JSON format
Changed tool output serialization in AgentService._convert_event_to_chunk()
from str() to json.dumps() to ensure structured data is properly formatted.
Changes:
- Added json import to agent_service.py
- Modified on_tool_end event handling
Benefits:
- Frontend can parse tool results with JSON.parse()
- Chinese characters are preserved (not escaped)
Co-Authored-By: YueGuobin <yueguobin@outlook.com>
```
### IMPORTANT: Use Your Own Git Account for Commits
When creating git commits, **ALWAYS** use your own git account information for authorship.
**Before committing**, verify your git configuration:
```bash
# Check current git user configuration
git config user.name
git config user.email
# If incorrect, set your own information
git config user.name "Your Name"
git config user.email "your.email@example.com"
```
**For this project**, the current configured user is:
```
Name: YueGuobin
Email: yueguobin@outlook.com
```
You may use `Co-Authored-By` to credit contributors, but the `Author` field should always be **your own account**:
```bash
git commit -m "feat: add new feature
Co-Authored-By: ContributorName <contributor@example.com>"
```
---
## Development Notes
### 1. When Modifying Tool Output Format
**Must update the following locations**:
- `agent/gns3_copilot/tools_v2/*.py` - Tool implementation
- `agent/gns3_copilot/agent_service.py` - Event conversion
- `docs/gns3-copilot/ai-chat-api-design.md` - API documentation
### 2. Data Serialization Principles
- **Tool Layer**: Return Python `dict`/`list`
- **Service Layer**: Use `json.dumps()` for serialization
- **API Layer**: Use Pydantic validation then serialize to SSE
### 3. Logging
```python
logger.info("Operation started: project_id=%s", project_id)
logger.debug("Detailed info: data=%s", data)
logger.error("Error occurred: %s", error, exc_info=True)
```
### 4. Security Considerations
- Prohibit dangerous commands (reload, erase startup-config, etc.)
- Use `filter_forbidden_commands()` to filter commands
- Validate user input (project_id format, etc.)
---
## Common Commands
```bash
# Run tests
pytest tests/
# Static code analysis (must run before committing)
source venv/bin/activate
flake8 gns3server/
# Check a single file
flake8 path/to/file.py
# Syntax check
python3 -m py_compile gns3server/agent/gns3_copilot/agent_service.py
# Check git status
git status
# View git history
git log --oneline -15
# View file modification history
git blame file.py -L 100,120
# Commit code (verify your git user info first!)
git config user.name && git config user.email
git add files...
git commit -m "type(scope): description"
```
---
## Related Documentation
- [AI Chat API Design](./docs/gns3-copilot/ai-chat-api-design.md)
- [Command Security Policy](./docs/gns3-copilot/command-security.md)
- [LLM Model Configuration](./docs/gns3-copilot/llm-model-configs-api.md)
- [Context Window Management](./docs/gns3-copilot/context-window-management.md)
---
**Document Maintenance**: Please update this document promptly when project structure or development standards change.
+66 -5
View File
@@ -1,7 +1,7 @@
# GNS3 server repository
[![Style](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![GitHub Actions tests](https://github.com/GNS3/gns3-server/workflows/testing/badge.svg?branch=3.0)](https://github.com/GNS3/gns3-server/actions?query=workflow%3Atesting+branch%3A3.0)
[![GitHub Actions tests](https://github.com/GNS3/gns3-server/workflows/testing/badge.svg?branch=3.1)](https://github.com/GNS3/gns3-server/actions?query=workflow%3Atesting+branch%3A3.0)
[![Latest PyPi version](https://img.shields.io/pypi/v/gns3-server.svg)](https://pypi.python.org/pypi/gns3-server)
[![Snyk scanning](https://snyk.io/test/github/GNS3/gns3-server/badge.svg)](https://snyk.io/test/github/GNS3/gns3-server)
@@ -54,7 +54,50 @@ python3 -m pip install gns3-gui
python3 -m pip install gns3-server
```
The downside of this method is you will have to manually install all dependencies (see below).
#### Optional Features
GNS3 server supports optional features that can be installed as needed:
**AI Features** (Optional — includes AI Copilot and MCP):
```shell
python3 -m pip install gns3-server[ai-features]
```
AI-powered assistant for network topology design, automation, and MCP protocol support for AI agent integration.
**Development** (For contributors):
```shell
python3 -m pip install gns3-server[dev]
```
**Web Wireshark** (Optional):
```shell
pip install gns3-server && gns3server-web-wireshark-setup
```
Browser-based packet capture analysis using Wireshark in a Docker container.
**Combination Installation**:
You can install multiple optional features together:
```shell
python3 -m pip install gns3-server[ai-features,dev]
```
**Why optional?**
- Reduces installation size for users who don't need specific features
- Supports restricted environments (government, education, corporate) where certain libraries may not be allowed
- Faster installation for basic GNS3 usage
- Allows users to choose only the features they need
**Note:** If you install without optional extras, the server will work normally but optional features will be disabled. You can add features later by running the appropriate install command.
**Uninstalling AI Features:**
To remove AI Features dependencies (AI Copilot + MCP):
```shell
gns3server-uninstall-ai-features
```
This will remove all AI Copilot and MCP dependencies while keeping the core functionality intact. The server will continue to work, but AI features will be disabled.
Please see our [documentation](https://docs.gns3.com/docs/getting-started/installation/linux) for more details.
@@ -77,18 +120,30 @@ Note that Docker needs the script program (`bsdutils` or `util-linux` package),
### Setting up
These commands will install the server as well as all Python dependencies:
These commands will install the server with core Python dependencies:
```shell
git clone https://github.com/GNS3/gns3-server
cd gns3-server
git checkout 3.0
git checkout 3.1
python3 -m venv venv-gns3server
source venv-gns3server/bin/activate
python3 -m pip install .
python3 -m gns3server
```
**For AI Copilot development**, install with additional dependencies:
```shell
python3 -m pip install .[ai-features,dev]
```
**For development (tests and linting)**:
```shell
python3 -m pip install .[dev]
```
You will have to manually install other software dependencies (see above), for Dynamips, VPCS and uBridge the easiest is to install from our PPA.
### Docker container
@@ -99,6 +154,12 @@ Alternatively, you can run the GNS3 server in a container
bash scripts/docker_dev_server.sh
```
#### use Docker Compose
``` {.bash}
docker compose up -d
```
### Running tests
First, install the development dependencies:
@@ -131,4 +192,4 @@ Bug fixes or small improvements pull requests go here.
3.x development brand for the next major release.
**Never** use this branch for production. Pull requests for major new features go here.
**Never** use this branch for production. Pull requests for major new features go here.
+50
View File
@@ -0,0 +1,50 @@
# ==============================================================================
# GNS3 Copilot AI Agent Dependencies
# ==============================================================================
# Install with: pip install gns3-server[ai-features]
# Or directly: pip install -r ai-requirements.txt
# ==============================================================================
# Core AI and Automation Framework
# Note: aiosqlite is in core requirements.txt
langchain>=1.2.10
langchain-core>=1.2.16
langgraph>=1.0.9
langgraph-checkpoint>=4.0.0
langgraph-checkpoint-sqlite>=3.0.3
langgraph-checkpoint-postgres>=3.0.4
# Model Providers
langchain-openai>=1.1.10
langchain-anthropic>=1.3.4
langchain-google-genai>=4.2.1
langchain-aws>=1.3.1
langchain-ollama>=1.0.1
langchain-deepseek>=1.0.1
langchain-xai>=1.2.2
# Token Counting
tiktoken>=0.8.0
# LangSmith SDK
langsmith>=0.7.7
# Network Automation
netmiko>=4.7.0
nornir>=3.5.0
nornir-netmiko>=1.0.1
nornir-utils>=0.2.0
nornir-salt>=0.23.0
# Environment & Configuration
python-dotenv>=1.2.1
# Authentication
PyJWT>=2.10.1
psycopg-pool>=3.1.0
# Skills Repository Management
PyYAML>=6.0.0
GitPython>=3.1.0
# Note: httpx is only for testing (in dev-requirements.txt)
+7
View File
@@ -0,0 +1,7 @@
services:
gen3-server:
build:
context: .
dockerfile: Dockerfile
ports:
- "8001:3080"
+6 -7
View File
@@ -1,7 +1,6 @@
pytest==8.3.3
flake8==7.1.1
pytest-timeout==2.3.1
pytest-asyncio==0.21.2
requests==2.32.3
httpx==0.27.2 # version 0.24.1 is required by httpx_ws
httpx_ws==0.6.2
pytest==9.1.1
flake8==7.3.0
pytest-timeout==2.4.0
pytest-asyncio==1.4.0
httpx==0.28.1
httpx_ws==0.7.2 # upgrading leads to failures in tests
+443
View File
@@ -0,0 +1,443 @@
Creative Commons Attribution-ShareAlike 4.0 International
Copyright (c) 2026 GNS3 Web UI Project
=======================================================================
Creative Commons Attribution-ShareAlike 4.0 International
Copyright (C) 2025 Creative Commons Corporation
Creative Commons Corporation ("Creative Commons") is not a law firm and
does not provide legal services or legal advice. Distribution of
Creative Commons public licenses does not create a lawyer-client or
other relationship. Creative Commons makes its licenses and related
information available on an "as-is" basis. Creative Commons gives no
warranties regarding its licenses, any material licensed under their
terms and conditions, or any related information. Creative Commons
disclaims all liability for damages resulting from their use to the
fullest extent possible.
Using Creative Commons Public Licenses
Creative Commons public licenses provide a standard set of terms and
conditions that creators and other rights holders may use to share
original works of authorship and other material subject to copyright
and certain other rights specified in the public license below. The
following considerations are for informational purposes only, are not
exhaustive, and do not form part of our licenses.
Considerations for licensors: Our public licenses are
intended for use by those authorized to give the public
permission to use material in ways otherwise restricted by
copyright and certain other rights. Our licenses are
irrevocable. Licensors should read and understand the terms
and conditions of the license they choose before applying it.
Licensors should also secure all rights necessary before
applying our licenses so that the public can reuse the
material as expected. Licensors should clearly mark any
material not subject to the license. This includes other CC-
licensed material, or material used under an exception or
limitation to copyright. More considerations for licensors:
wiki.creativecommons.org/Considerations_for_licensors
Considerations for the public: Our public licenses are
intended for use by those authorized to give the public
permission to use material in ways otherwise restricted by
copyright and certain other rights. Our licenses are
irrevocable. Licensors should read and understand the terms
and conditions of the license they choose before applying it.
Licensors should also secure all rights necessary before
applying our licenses so that the public can reuse the
material as expected. Licensors should clearly mark any
material not subject to the license. This includes other CC-
licensed material, or material used under an exception or
limitation to copyright. More considerations for licensors:
wiki.creativecommons.org/Considerations_for_licensors
Creative Commons Attribution-ShareAlike 4.0 International Public
License
By exercising the Licensed Rights (defined below), You accept and agree
to be bound by the terms and conditions of this Creative Commons
Attribution-ShareAlike 4.0 International Public License ("Public
License"). To the extent this Public License may be interpreted as a
contract, You are granted the Licensed Rights in consideration of Your
acceptance of these terms and conditions, and the Licensor grants You
such rights in consideration of benefits the Licensor receives from
making the Licensed Material available under these terms and conditions.
Section 1 – Definitions.
a. Adapted Material means material subject to Copyright and Similar
Rights that is derived from or based upon the Licensed Material
and in which the Licensed Material is translated, altered,
arranged, transformed, or otherwise modified in a manner requiring
permission under the Copyright and Similar Rights held by the
Licensor. For purposes of this Public License, where the Licensed
Material is a musical work, performance, or sound recording,
Adapted Material is always produced where the Licensed Material is
synched in timed relation with a moving image.
b. Adapter's License means the license You apply to Your Copyright
and Similar Rights in Your contributions to Adapted Material in
accordance with the terms and conditions of this Public License.
c. Copyright and Similar Rights means copyright and/or similar rights
closely related to copyright including, without limitation,
publication, distribution, public performance, public display, and
adaptation rights, and rights in database extensions and
collections.
d. Licensed Material means the artistic or literary work, database,
or other material to which the Licensor applied this Public
License.
e. Licensed Rights means rights granted to You subject to the terms
and conditions of this Public License, which are limited to all
Copyright and Similar Rights that apply to Your use of the
Licensed Material and that the Licensor has authority to license.
f. Licensor means the individual(s) or entity(ies) granting rights
under this Public License.
g. Share means to provide material to the public by any means or
process that requires permission under the Licensed Rights, such
as reproduction, public display, public performance, distribution,
dissemination, communication, or importation, and to make material
available to the public including in ways that members of the
public may access the material from a place and at a time
individually chosen by them.
h. Sui Generis Database Rights means rights other than copyright
resulting from Directive 96/9/EC of the European Parliament and
of the Council of 11 March 1996 on the legal protection of
databases, as amended and/or succeeded, as well as other
essentially equivalent rights anywhere in the world.
i. You means the individual or entity exercising the Licensed Rights
under this Public License. Your has a corresponding meaning.
Section 2 – Scope.
a. License grant.
1. Subject to the terms and conditions of this Public License,
the Licensor hereby grants You a worldwide, royalty-free,
non-sublicensable, non-exclusive, irrevocable license to
exercise the Licensed Rights in the Licensed Material to:
a. reproduce and Share the Licensed Material, in whole or
in part; and
b. produce, reproduce, and Share Adapted Material.
2. Exceptions and Limitations. For the avoidance of doubt,
where Exceptions and Limitations apply to Your use, this
Public License does not apply, and You do not need to
comply with its terms and conditions.
3. Term. The term of this Public License is specified in
Section 6(a).
4. Media and formats; technical modifications allowed. The
Licensor authorizes You to exercise the Licensed Rights in
all media and formats whether now known or hereafter created,
and to make technical modifications necessary to do so. The
Licensor waives and/or agrees not to assert any right or
authority to forbid You from making technical modifications
necessary to exercise the Licensed Rights, including
technical modifications necessary to circumvent Effective
Technological Measures. For purposes of this Public License,
simply making modifications authorized by this Section 2(a)
(4) never produces Adapted Material.
5. Downstream recipients.
a. Offer from the Licensor – Licensed Material. Every
recipient of the Licensed Material automatically
receives an offer from the Licensor to exercise the
Licensed Rights under the terms and conditions of this
Public License.
b. Additional offer from the Licensor – Adapted Material.
Every recipient of Adapted Material from You
automatically receives an offer from the Licensor to
exercise the Licensed Rights in the Adapted Material
under the conditions of the Adapter's License You
apply.
6. No endorsement. Nothing in this Public License constitutes or
may be construed as permission to assert or imply that You
are, or that Your use of the Licensed Material is, connected
with, or sponsored, endorsed, or granted official status by,
the Licensor or others designated to receive attribution as
provided in Section 3(a)(1)(A)(i).
b. Other rights. Moral rights, such as the right of integrity, are
not licensed under this Public License, nor are publicity,
privacy, and/or other similar personality rights; however, to the
extent possible, the Licensor waives and/or agrees not to assert
any such rights held by the Licensor to the limited extent
necessary to allow You to exercise the Licensed Rights, but not
otherwise.
c. Patent and trademark rights are not licensed under this Public
License.
d. To the extent possible, the Licensor waives any right to collect
royalties from You for the exercise of the Licensed Rights,
whether directly or through a collecting society under any
voluntary or waivable statutory or compulsory licensing scheme.
In all other cases the Licensor expressly reserves any right to
collect such royalties.
Section 3 – License Conditions.
Your exercise of the Licensed Rights is expressly made subject to
the following conditions.
a. Attribution.
1. If You Share the Licensed Material (including in modified
form), You must:
a. retain the following if it is supplied by the
Licensor with the Licensed Material:
i. identification of the creator(s) of the
Licensed Material and any others designated to
receive attribution, in any reasonable manner
requested by the Licensor (including by
pseudonym if designated);
ii. a copyright notice;
iii. a notice that refers to this Public License;
iv. a notice that refers to the disclaimer of
warranties;
v. a URI or hyperlink to the Licensed Material to
the extent reasonably practicable;
b. indicate if You modified the Licensed Material and
retain an indication of any previous modifications; and
c. indicate the Licensed Material is licensed under this
Public License, and include the text of, or the URI
or hyperlink to, this Public License.
2. You may satisfy the conditions in Section 3(a)(1) in any
reasonable manner based on the medium, means, and context in
which You Share the Licensed Material. For example, it may
be reasonable to satisfy the conditions by providing a URI
or hyperlink to a resource that includes the required
information.
3. If requested by the Licensor, You must remove any of the
information required by Section 3(a)(1)(A) to the extent
reasonably practicable.
4. If You Share Adapted Material You produce, the Adapter's
License You apply must not prevent recipients of the
Adapted Material from complying with this Public License.
b. ShareAlike.
In addition to the conditions in Section 3(a), if You Share
Adapted Material You produce, the following conditions also apply.
1. The Adapter's License You apply must be a Creative Commons
license with the same License Elements, this version or
later, or a BY-SA Compatible License.
2. You must include the text of, or the URI or hyperlink to,
the Adapter's License You apply. You may satisfy this
condition in any reasonable manner based on the medium,
means, and context in which You Share Adapted Material.
3. You may not offer or impose any additional or different terms
or conditions on, or apply any Effective Technological
Measures to, Adapted Material that restrict exercise of the
rights granted under the Adapter's License You apply.
Section 4 – Sui Generis Database Rights.
Where the Licensed Rights include Sui Generis Database Rights that
apply to Your use of the Licensed Material:
a. for the avoidance of doubt, Section 2(a)(1) grants You the right
to extract, reuse, reproduce, and Share all or a substantial
portion of the contents of the database;
b. if You include all or a substantial portion of the database
contents in a database in which You have Sui Generis Database
Rights, then the database in which You have Sui Generis Database
Rights (but not its individual contents) is Adapted Material; and
c. You must comply with the conditions in Section 3(a) if You Share
all or a substantial portion of the contents of the database.
For the avoidance of doubt, this Section 4 supplements and does not
replace Your obligations under this Public License where the Licensed
Rights include other Copyright and Similar Rights.
Section 5 – Disclaimer of Warranties and Limitation of Liability.
a. Unless otherwise separately undertaken by the Licensor, to the
extent possible, the Licensor offers the Licensed Material as-is
and as-available, and makes no representations or warranties of
any kind concerning the Licensed Material, whether express,
implied, statutory, or other. This includes, without limitation,
warranties of title, merchantability, fitness for a particular
purpose, non-infringement, absence of latent or other defects,
accuracy, or the presence or absence of errors, whether or not
known or discoverable. Where disclaimers of warranties are not
allowed in full or in part, this disclaimer may not apply to You.
b. To the extent possible, in no event will the Licensor be liable
to You on any legal theory (including, without limitation,
negligence) or otherwise for any direct, special, indirect,
incidental, consequential, punitive, exemplary, or other losses,
costs, expenses, or damages that arise out of this Public License
or use of the Licensed Material, even if the Licensor has been
advised of the possibility of such losses, costs, expenses, or
damages. Where a limitation of liability is not allowed in full
or in part, this limitation may not apply to You.
c. The disclaimer of warranties and limitation of liability provided
above shall be interpreted in a manner that, to the extent
possible, most closely approximates an absolute disclaimer and
waiver of all liability.
Section 6 – Term and Termination.
a. This Public License applies for the term of the Copyright and
Similar Rights licensed here. However, if You fail to comply
with this Public License, then Your rights under this Public
License terminate automatically.
b. Where Your right to use the Licensed Material has terminated
under Section 6(a), it reinstates:
1. automatically as of the date the violation is cured,
provided it is cured within 30 days of Your discovery of
the violation; or
2. upon express reinstatement by the Licensor.
For the avoidance of doubt, this Section 6(b) does not affect any
right the Licensor may have to seek remedies for Your violations
of this Public License.
c. For the avoidance of doubt, the Licensor may also offer the
Licensed Material under separate terms or conditions or stop
distributing the Licensed Material at any time; however, doing so
will not terminate this Public License.
d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
License.
Section 7 – Other Terms and Conditions.
a. The Licensor shall not be bound by any additional or different
terms or conditions communicated by You unless expressly agreed.
b. Any arrangements, understandings, or agreements regarding the
Licensed Material not stated herein are separate from and
independent of the terms and conditions of this Public License.
Section 8 – Interpretation.
a. For the avoidance of doubt, this Public License does not, and
shall not be interpreted to, reduce, limit, restrict, or impose
conditions on any use of the Licensed Material that could lawfully
be made without permission under this Public License.
b. To the extent possible, if any provision of this Public License
is deemed unenforceable, it shall be automatically reformed to
the minimum extent necessary to make it enforceable. If the
provision cannot be reformed, it shall be severed from this
Public License without affecting the enforceability of the
remaining terms and conditions.
c. No term or condition of this Public License will be waived and
no failure to comply consented to unless expressly agreed to by
the Licensor.
d. Nothing in this Public License constitutes or may be interpreted
as a limitation upon, or waiver of, any privileges and immunities
that apply to the Licensor or You, including from the legal
processes of any jurisdiction or authority.
=======================================================================
Creative Commons is not a party to its public licenses. Notwithstanding,
Creative Commons may elect to apply one of its public licenses to
material it publishes and in those instances will be considered the
"Licensor." The text of the Creative Commons public licenses is
dedicated to the public domain under the CC0 Public Domain Dedication.
The text of the Creative Commons public licenses is dedicated to the
public domain under the CC0 Public Domain Dedication. Except for the
limited purpose of indicating that material is shared under a Creative
Commons public license or as otherwise permitted by the Creative Commons
policies published at creativecommons.org/policies, Creative Commons
does not authorize the use of the trademark "Creative Commons" or any
other trademark or logo of Creative Commons without its prior written
consent including, without limitation, in connection with any
unauthorized modifications to any of its public licenses or any other
arrangements, understandings, or agreements concerning use of licensed
material. For the avoidance of doubt, this paragraph does not form part
of the public licenses.
Creative Commons may be contacted at creativecommons.org.
=======================================================================
For the GNS3 Web UI documentation:
- Documentation License: CC BY-SA 4.0
- Project License: GPLv3 (see root LICENSE file)
- License URL: https://creativecommons.org/licenses/by-sa/4.0/
All documentation files in this directory are licensed under the
Creative Commons Attribution-ShareAlike 4.0 International License
(CC BY-SA 4.0).
=======================================================================
IMPORTANT: ShareAlike Requirements for Software Derivatives
Under the CC BY-SA 4.0 ShareAlike condition, if you create derivative
works based on this documentation, including software that is considered
a derivative of the documentation, you must:
1. **License your derivative work under CC BY-SA 4.0 or a compatible
license** (such as GPLv3, which is compatible with CC BY-SA 4.0)
2. **Attribute the original documentation** to the GNS3 Web UI Project
3. **Indicate if you modified the documentation** and retain
indications of previous modifications
4. **Include the license notice** and link to the CC BY-SA 4.0 license
What constitutes a derivative work:
- Translations of the documentation
- Adaptations, modifications, or enhancements to the documentation
- Software that incorporates substantial portions of the documentation
- Works based on the documentation's structure, organization, or content
Compatible licenses include:
- CC BY-SA 4.0 or later
- GPLv3 (GNU General Public License version 3)
- AGPLv3 (GNU Affero General Public License version 3)
- Other licenses approved as compatible by Creative Commons
For more information on CC BY-SA compatibility, see:
https://creativecommons.org/compatiblelicenses
If you are unsure whether your use constitutes a derivative work or
which license to apply, please consult the full CC BY-SA 4.0 license
text above or seek legal advice.
+130
View File
@@ -0,0 +1,130 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# GNS3 Server Documentation
---
## License
This documentation is licensed under the **Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0)**.
⚠️ **Important - ShareAlike Requirement**: If you create derivative works based on this documentation (including software that incorporates substantial portions of the documentation), your work must also be licensed under **CC BY-SA 4.0 or a compatible license** (such as GPLv3).
- 📄 **Full License Text**: See [docs/LICENSE](./LICENSE)
- 🔗 **License URL**: https://creativecommons.org/licenses/by-sa/4.0/
- 📖 **Compatibility**: https://creativecommons.org/compatiblelicenses
**Dual License Structure**:
- 📚 **Documentation**: CC BY-SA 4.0 (this directory)
- 💻 **Software Code**: GPLv3 (see root [LICENSE](../LICENSE))
---
Technical documentation for the GNS3 server project, covering features, AI Copilot, development setup, and known issues.
## Directory Structure
```
docs/
├── README.md # This file
├── development-setup.md # Ubuntu 24.04 development environment setup
├── openapi.json # OpenAPI specification
├── features/ # Feature documentation
│ ├── compute-controller-setup.md # Controller + Compute architecture & configuration
│ ├── statistics-api.md # Aggregated statistics API for monitoring
│ ├── vnc-websocket-console.md # Browser-based VNC console via WebSocket
│ └── web-wireshark-business-process.md # Web Wireshark (Docker + xpra packet capture)
├── gns3-copilot/ # AI Copilot feature documentation
│ ├── netmiko_devices.md # Netmiko supported devices (366 types)
│ ├── template-based-configuration-roadmap.md # Future: template-based config with HITL
│ └── implemented/ # Implemented features
│ ├── chat-api.md # Chat API (SSE, session management)
│ ├── llm-model-configs.md # LLM model configuration system
│ ├── command-security.md # Command security and filtering
│ ├── context-window-management.md # Context window optimization
│ ├── node-control-tools.md # Node start/stop/suspend tools
│ └── multi-vendor-device-support.md # Multi-vendor device support
└── bugs/ # Known issues & bug reports
└── telnet-server-connection-race-condition.md
```
---
## Features
### Controller + Compute Setup (`features/compute-controller-setup.md`)
Architecture and minimum configuration for setting up GNS3 Controller with remote Compute nodes. Covers compute node config, controller registration, and multi-compute deployment.
### Statistics API (`features/statistics-api.md`)
Aggregated server statistics API (`GET /v3/statistics`) for monitoring dashboards. Collects compute resources, project/node/link counts, and Web Wireshark container status in a single request.
### VNC WebSocket Console (`features/vnc-websocket-console.md`)
Browser-based VNC console access via WebSocket. The Controller acts as WebSocket-to-WebSocket relay, and Compute bridges WebSocket to TCP for QEMU/Docker VMs. Supports noVNC clients.
### API Error Responses (`features/api-error-responses.md`)
Unified error response format across all GNS3 API endpoints. Documents HTTP status codes, error types, and client-side error handling patterns.
### Web Wireshark (`features/web-wireshark-business-process.md`)
Web-based packet capture analysis using Docker + xpra HTML5 client. Zero-install Wireshark experience directly in the browser, integrated with GNS3 topologies.
### Marker (Traffic Insight) (`features/marker-traffic-insight.md`)
Real-time traffic insight via per-link BPF markers and project-level inherited definitions. A marker taps a link in uBridge, emitting match notifications and pcap capture on BPF hit; definitions fan out to every capable link automatically.
### Docker exec Console (Vendor NOS) (`features/docker-exec-console.md`)
Console for vendor NOS containers (SR Linux, XRd, …) whose CLI is a TUI off PID 1: runs the vendor CLI via the Docker exec API, plus `GNS3_SKIP_INIT`/`GNS3_INTERFACE_NAMES` boot knobs and SKIP_INIT volume persistence.
### Cisco XRd Control Plane (`features/vendor-nos-xrd.md`)
Cisco XRd as a GNS3 Docker router: vendor path + shm/device injection (`GNS3_SHM_SIZE`/`GNS3_DEVICES`), config-file injection (`extra_configs`), udev masking (`GNS3_MASK_UDEV`) so privileged systemd containers don't disturb the host, and the host-readiness check.
---
## GNS3 AI Copilot (`gns3-copilot/`)
### Implemented Features
| Feature | Description | Status |
|---------|-------------|--------|
| [Chat API](gns3-copilot/implemented/chat-api.md) | SSE streaming, session management, token statistics | Implemented |
| [LLM Model Configs](gns3-copilot/implemented/llm-model-configs.md) | Multi-level model config (system/group/user) | Implemented |
| [Command Security](gns3-copilot/implemented/command-security.md) | Command filtering, dangerous operation detection, HITL | Implemented |
| [Context Window Management](gns3-copilot/implemented/context-window-management.md) | Token optimization, content filtering, compression | Implemented |
| [Node Control Tools](gns3-copilot/implemented/node-control-tools.md) | Start/stop/suspend with batch ops and progress tracking | Implemented |
| [Multi-Vendor Support](gns3-copilot/implemented/multi-vendor-device-support.md) | Cisco, Huawei, Ruijie, VPCS with custom Netmiko drivers | Implemented |
### Reference
- [Netmiko Supported Devices](gns3-copilot/netmiko_devices.md) — 366 device types (154 SSH, 55 Telnet, 3 custom GNS3 drivers)
### Roadmap
- [Template-Based Configuration with HITL](gns3-copilot/template-based-configuration-roadmap.md) — Jinja2 templates with human-in-the-loop confirmations for device configuration and node creation
---
## Known Issues (`bugs/`)
- [Telnet Server Connection Race Condition](bugs/telnet-server-connection-race-condition.md) — `getpeername()` error when client disconnects during connection setup (High severity, Open)
- [Docker Link UDP Self-Loop](bugs/link-udp-self-loop.md) — intermittent one-way link (both ends handed the same UDP port by an allocation race); **fixed** (Medium severity)
---
## Development Setup (`development-setup.md`)
Quick-start guide for Ubuntu 24.04: install via PPA, set up dependencies, and run gns3-server from source.
---
## Related Documentation
- [GNS3 Server API Documentation](https://api.gns3.com/) — Interactive API docs (also available locally via `redoc.html`)
- [GNS3 Web UI Documentation](https://docs.gns3.com/)
- [LangGraph Documentation](https://langchain-ai.github.io/langgraph/)
---
_Last updated: 2026-08-14_
+101
View File
@@ -0,0 +1,101 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# Docker Link UDP Self-Loop Bug (One-Way Link)
## Bug Report
**Date**: 2026-08-14
**Severity**: Medium (one-way connectivity, CPU burn from packet duplication; intermittent)
**Status**: **Fixed** — root cause found and unit-tested (same day)
**Component**: UDP port allocation — `gns3server/compute/port_manager.py`
(`get_free_udp_port` find-then-add race); secondary: `gns3server/controller/udp_link.py`
(`_prepare` accumulated stale `_link_data` on reset)
## Symptoms
Two Docker nodes (observed with Cisco XRd; node-type agnostic) linked on the
same compute cannot ping each other. Packet capture on the link shows only **one**
side sending ARP. The other side's traffic never appears on the link at all.
## Evidence (from the live occurrence)
Captured simultaneously on both nodes' uBridge `bridge1` (see diagnostics below):
| Direction | Result |
|---|---|
| A → B | works — A's ARP requests arrive at B's bridge, B replies |
| B → A | dead — B's replies/ICMP appear only on **B's own bridge** (duplicated ×2–×3), nothing arrives at A |
| B's `ethN` counters | RX ≈ TX ≈ 5000+ — B receives its own transmissions back |
| A's `ethN` counters | TX > 0, RX = 0 — never receives anything |
## Root Cause (confirmed)
**`PortManager.get_free_udp_port` had an unguarded find-then-add sequence.**
A link allocates the UDP port for **both ends concurrently**
(`asyncio.gather` in `UDPLink._prepare` → two `POST /ports/udp`). The route
handler is a sync `def`, so FastAPI executes the two requests **in parallel
threads**. Both threads ran `find_unused_port` (socket-probing, GIL-releasing)
before either reached `_used_udp_ports.add(port)` — the set add is idempotent,
so no error was raised and **both ends were handed the same port number**:
`lport == rport` on both NIOs, a literal self-loop.
Why it was *silent* and *asymmetric*:
- uBridge sets `SO_REUSEADDR` on UDP NIO sockets (`ubridge/src/nio_udp.c`), so
the second bind of the same port **succeeds** instead of failing with
`EADDRINUSE` — link creation returned success.
- With two sockets bound to the same port, the kernel delivers to one of them
(last bound wins). The node that started later — in the live case B,
restarted ~77 s after A — received **everything**: A's packets *and* its own
transmissions echoed back. The 77 s restart did not cause the corruption; it
only decided which end starves.
- The same-compute condition is part of the trigger: both allocations hit the
same `PortManager` instance (a cross-compute link races two processes and
cannot self-collide).
A second, smaller defect was found while auditing: `UDPLink._prepare()`
**appended** to `self._link_data` but the committed NIOs are always taken from
indices 0/1 — after `reset()` (delete + create on the same object) the stale,
already-released port pair was re-committed and the freshly allocated ports
were leaked.
## Fix
| Change | Where |
|---|---|
| `threading.RLock` making find-then-add (and reserve/release) atomic for TCP and UDP | `gns3server/compute/port_manager.py` |
| `_prepare()` rebuilds `_link_data` from scratch instead of appending | `gns3server/controller/udp_link.py` |
| Regression tests: threaded allocation never returns duplicates (red on the old code); `reset()` commits the fresh mirrored pair with `lport != rport` | `tests/compute/test_port_manager.py`, `tests/controller/test_udp_link.py` |
## Diagnostics (uBridge console is the fast path)
1. **uBridge console** — each node's uBridge listens on a Unix socket
`/run/user/1000/gns3/ubridge-<node_id>.sock`; connect and send:
`bridge list` (NIO count per bridge), and
`bridge start_capture bridge<N> "/tmp/ub-<node>.pcap"` /
`bridge stop_capture bridge<N>` to capture what the bridge actually forwards.
Comparing the two ends' pcaps localizes the break immediately.
2. **Container counters** — `docker exec <cid> ip -s link show ethN`:
TX>0/RX=0 → peer never returns; RX≈TX huge with µs-scale duplicates → self-loop.
3. **UDP sockets** — `ss -uln` (no `-p`; uBridge runs setuid-root so process names
are hidden, ports are visible): a two-node link owns exactly two "orphan" UDP
ports. **One port instead of two = this bug.**
4. **Recovery** — delete and re-create the link (or stop/start both nodes);
with the fix, the corruption no longer occurs in the first place.
Note: a Docker node's in-container `ethN` is a **TAP device** whose file descriptor
lives inside uBridge (the interface is created host-side, then moved into the
container namespace and renamed). There is no veth host end to look for — do not
waste time hunting for one in the host namespace.
## Related
- `docs/features/vendor-nos-xrd.md` — troubleshooting table entry pointing here.
- Link-create batch optimization (`controller/udp_link.py`, project-open bulk path)
was exonerated: the pool path allocates sequentially in one handler and cannot
self-collide.
@@ -0,0 +1,316 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# Telnet Server Connection Race Condition Bug
## Bug Report
**Date**: 2026-03-14
**Severity**: High
**Status**: Open
**Component**: Telnet Server (`gns3server/utils/asyncio/telnet_server.py`)
## Error Logs
```
2026-03-14 15:21:31 ERROR asyncio:1879 Unhandled exception in client_connected_cb
transport: <_SelectorSocketTransport fd=67 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
File "/home/yueguobin/myCode/GNS3/gns3-server/gns3server/utils/asyncio/telnet_server.py", line 215, in run
await self._process(network_reader, network_writer, connection)
File "/home/yueguobin/myCode/GNS3/gns3-server/gns3server/utils/asyncio/telnet_server.py", line 305, in _process
client_info = connection_key.get_extra_info("socket").getpeername()
File "/usr/lib64/python3.13/asyncio/trsock.py", line 77, in getpeername
return self._sock.getpeername()
~~~~~~~~~~~~~~~~~~~~~~^^
OSError: [Errno 107] Transport endpoint is not connected
```
## Root Cause Analysis
### Architecture
The GNS3 Telnet server architecture supports multiple concurrent client connections to a single node console:
```
┌──────────────────────────────────────────────────────────┐
│ Node (VPCS/Docker/Router/Switch, etc.) │
│ - Independent process │
│ - Normal operation, processing business logic │
└────────────────────┬─────────────────────────────────────┘
│ stdout (device output)
│
┌────────────▼─────────────────────────────────────┐
│ AsyncioTelnetServer (Telnet Proxy Server) │
│ - Reads output from Node │
│ - Broadcasts to all connected clients │
│ - ← ← Bug occurs at this layer ← ← │
└────────────┬─────────────────────────────────────┘
│
┌────────────┼────────────┬──────────────────┐
│ │ │ │
Web Console Auto Script Client 1 Client 2
(long-lived) (quick disco) (normal) (normal)
```
### The Race Condition
The bug occurs in the broadcast logic when a client disconnects while the server is iterating through connections:
**Timeline**:
```
t1: Clients A, B, C connect to the same Telnet port
t2: Device has output (e.g., log message)
t3: Client A receives data and script immediately disconnects (FIN sent)
t4: Server hasn't read EOF yet (event loop hasn't checked this connection)
t5: Server iterates through connections to broadcast data
t6: When iterating to Client A, getpeername() is called → OSError!
```
### Code Location
**File**: `gns3server/utils/asyncio/telnet_server.py`
**Line**: 305
**Problematic Code**:
```python
# Line 304-305
for connection_key in list(self._connections.keys()):
client_info = connection_key.get_extra_info("socket").getpeername() # ← OSError here
connection = self._connections[connection_key]
try:
connection.writer.write(data)
await asyncio.wait_for(connection.writer.drain(), timeout=10)
except:
log.debug(f"Timeout while sending data to client: {client_info}, closing and removing from connection table.")
connection.close()
del self._connections[connection_key]
```
### The Core Issue
The `getpeername()` call is **outside** the try-except block, so any OSError from it is not caught.
Additionally, the top-level exception handler at line 216 only catches `ConnectionError`:
```python
# Line 212-227
try:
await self._write_intro(network_writer, echo=self._echo, binary=self._binary, naws=self._naws)
await connection.connected()
await self._process(network_reader, network_writer, connection)
except ConnectionError: # ← Only catches ConnectionError
async with self._lock:
network_writer.close()
if self._reader_process == network_reader:
self._reader_process = None
if self._current_read is not None:
self._current_read.cancel()
await connection.disconnected()
del self._connections[network_writer]
```
**Python Exception Hierarchy**:
```
BaseException
└─ Exception
├─ ConnectionError ← Only this is caught
│ ├─ ConnectionResetError
│ ├─ BrokenPipeError
│ └─ ...
└─ OSError ← Actually thrown! (not a subclass of ConnectionError)
└─ [Errno 107] Transport endpoint is not connected
```
Since `OSError` is **not** a subclass of `ConnectionError`, it propagates uncaught to the asyncio event loop.
## Impact Assessment
### Immediate Effects
| Impact | Severity | Description |
|--------|----------|-------------|
| Connection interrupted | 🔴 High | The connection triggering the exception is terminated |
| Resource leak | 🟠 Medium | socket/connection not properly cleaned up |
| Other clients affected | 🟡 Low | Other clients on same port may miss broadcast data |
| Service stability | 🟡 Low | Long-running may accumulate zombie connections |
### User-Reported Symptoms
After this error occurs, users report:
1. **Cannot open the affected node** - Clicking on the node fails
2. **Cannot close the node** - Close button doesn't work
3. **"Node not found" errors** - Operations on the node return 404
4. **Refresh fixes it temporarily** - Reloading the page restores functionality
### Why This Happens
When the uncaught `OSError` occurs, the cleanup code at lines 217-227 **never executes**:
```python
except ConnectionError:
async with self._lock:
network_writer.close() # ✗ Not executed
if self._reader_process == network_reader:
self._reader_process = None # ✗ Not executed
if self._current_read is not None:
self._current_read.cancel() # ✗ Not executed
await connection.disconnected() # ✗ Not executed
del self._connections[network_writer] # ✗ Not executed
```
This leads to:
- **Resource leaks**: socket and writer not closed, file descriptors leaked
- **State inconsistency**: `_connections` dictionary retains disconnected connections
- **Potential deadlocks**: Locks may not be released if exception occurs while holding them
- **Subsequent operation failures**: Future operations may access zombie connections
### Effect on Node Process
**The Node process itself is NOT affected**:
- Node continues running normally
- Node's stdout has already been read by the proxy
- The bug occurs during the broadcast phase, after data has been read
The issue is in the **Telnet Proxy layer**, not the node itself.
## Trigger Conditions
This error is more likely to occur with:
| Scenario | Probability | Reason |
|----------|-------------|--------|
| **Automated scripts** | 🔴 High | Fast connect → execute → disconnect, small time window |
| **Manual operation** | 🟡 Medium | Can occur (e.g., closing terminal, network fluctuation) |
| **Normal usage** | 🟢 Low | Human operations slower, server usually detects EOF first |
### Typical Scenario
1. User opens Web Console (long-lived connection)
2. Automated script connects → executes command → quickly disconnects
3. While script disconnects, device has output that needs broadcasting
4. During connection iteration, script connection already closed
5. `getpeername()` call fails with OSError
## Related Issues
A secondary issue was found in the error handler:
**File**: `gns3server/api/server.py`
**Line**: 162
```python
@app.exception_handler(ControllerNotFoundError)
async def controller_not_found_error_handler(request: Request, exc: ControllerNotFoundError):
log.error(f"Controller not found error in {request.url.path} ({request.method}): {exc}")
# ^^^^^^^^^^^^^^^
return JSONResponse(...)
```
**Problem**: `request.method` only exists in HTTP requests, not WebSocket connections.
When a WebSocket request triggers this exception handler:
```
ControllerNotFoundError: Node ID xxx doesn't exist
↓
Attempt to log error
↓
AttributeError: 'WebSocket' object has no attribute 'method'
```
This masks the original error with an attribute error.
## Proposed Fix
### Primary Fix (Telnet Server)
Move `getpeername()` inside the try block and catch OSError:
```python
# Lines 303-314
for connection_key in list(self._connections.keys()):
connection = self._connections[connection_key]
client_info = None
try:
client_info = connection_key.get_extra_info("socket").getpeername()
connection.writer.write(data)
await asyncio.wait_for(connection.writer.drain(), timeout=10)
except (OSError, ConnectionError, asyncio.TimeoutError) as e:
log.debug(f"Error sending data to client {client_info}: {e}, closing and removing from connection table.")
connection.close()
del self._connections[connection_key]
```
### Secondary Fix (Top-level Exception Handler)
Catch OSError in the main handler to ensure cleanup:
```python
# Lines 212-227
try:
await self._write_intro(network_writer, echo=self._echo, binary=self._binary, naws=self._naws)
await connection.connected()
await self._process(network_reader, network_writer, connection)
except (ConnectionError, OSError): # ← Add OSError
async with self._lock:
network_writer.close()
if self._reader_process == network_reader:
self._reader_process = None
if self._current_read is not None:
self._current_read.cancel()
await connection.disconnected()
del self._connections[network_writer]
```
### Tertiary Fix (API Error Handler)
Fix the WebSocket error handler:
```python
@app.exception_handler(ControllerNotFoundError)
async def controller_not_found_error_handler(request: Request, exc: ControllerNotFoundError):
method = getattr(request, 'method', 'WebSocket')
log.error(f"Controller not found error in {request.url.path} ({method}): {exc}")
return JSONResponse(
status_code=status.HTTP_404_NOT_FOUND,
content={"message": str(exc)},
)
```
## Reproduction Steps
**To be documented** after testing.
Potential reproduction scenario:
1. Start a GNS3 node with console enabled (e.g., VPCS)
2. Open web console to keep a long-lived connection
3. Run automated script that:
- Connects to the same console port
- Executes a command
- Immediately disconnects
4. While script is disconnecting, trigger device output
5. Observe the error in logs
## References
- **Files**:
- `gns3server/utils/asyncio/telnet_server.py:305` (primary issue)
- `gns3server/utils/asyncio/telnet_server.py:216` (exception handler)
- `gns3server/api/server.py:162` (secondary issue)
- **Related Commits**:
- Recent telnet-related work on feature branch
- **Error Patterns**:
- Race condition in connection management
- Incomplete exception handling in asyncio code
+146
View File
@@ -0,0 +1,146 @@
# Ubuntu 24.04 GNS3 Development Environment Setup
## Simplified Development Environment Setup
### 1. Add PPA
```bash
sudo apt update
sudo apt install software-properties-common
sudo add-apt-repository ppa:gns3/ppa
sudo apt update
```
### 2. Install Dependencies
Binary tools (Dynamips, VPCS, uBridge, etc.) are automatically installed as gns3-server dependencies
```bash
sudo apt install gns3-server tshark
```
### 3. Remove gns3-server While Keeping Dependencies
If you only want to run the server from source:
```bash
# Remove main package only, keep dependencies
sudo apt remove gns3-server
# Mark dependencies as manual to prevent autoremove deletion
sudo apt-mark manual ubridge dynamips vpcs libvirt
```
### 4. Install and Start Docker
```bash
sudo apt install docker.io
sudo systemctl enable --now docker
```
#### Configure Docker Mirror Accelerator (China Mainland)
If Docker Hub is inaccessible, configure mirror accelerators:
```bash
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<'EOF'
{
"registry-mirrors": [
"https://docker.1ms.run",
"https://docker.xuanyuan.me"
]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker
```
Verify:
```bash
docker info | grep -i mirror
```
### 5. Add User to Groups
```bash
# If running as gns3 user (created by PPA install):
sudo usermod -aG ubridge,docker,kvm gns3
# If running as your own user, add yourself to these groups:
sudo usermod -aG ubridge,docker,kvm $USER
# Then logout and login again for group membership to take effect
```
### 6. Install Python venv
```bash
sudo apt install python3.12-venv
```
### 7. Run Development Version from Source
Clone the repository (official or your fork):
```bash
# Official repository
git clone https://github.com/GNS3/gns3-server.git
# Or your forked repository
git clone https://github.com/yourname/gns3-server.git
cd gns3-server
git checkout your-branch
```
Setup virtual environment and install dependencies:
```bash
python3 -m venv venv
source venv/bin/activate
# For China mainland users, use mirror:
# pip install -e . -i https://mirrors.aliyun.com/pypi/simple/
pip install -e . && gns3server-web-wireshark-setup
pip install -e .[ai-features]
pip install -e .[dev]
```
Run the server:
```bash
python3 -m gns3server
```
## Optional: Install AI Features Development Dependencies
```bash
python3 -m pip install .[ai-features,dev]
```
## Optional: Expand LVM Root Partition
If the root partition is low on space but the volume group has unallocated space:
Check LVM status:
```bash
sudo lvm pvdisplay
sudo lvm vgdisplay
sudo lvm lvdisplay
```
Extend the root LV using all free space:
```bash
sudo lvm lvextend -l +100%FREE /dev/mapper/ubuntu--vg-ubuntu--lv
sudo resize2fs /dev/mapper/ubuntu--vg-ubuntu--lv
```
Verify:
```bash
df -h
```
+80
View File
@@ -0,0 +1,80 @@
# API Error Response Format
## Overview
GNS3 Server uses a unified error response format across all API endpoints. All error responses return a JSON object with a `message` field containing the error details.
## Response Format
```json
{
"message": "Error description here"
}
```
## Error Types
| Error Type | HTTP Status | Description |
|------------|-------------|-------------|
| `ControllerBadRequestError` | 400 | Invalid request parameters (e.g., validation failure) |
| `ControllerUnauthorizedError` | 401 | Missing or invalid authentication |
| `ControllerForbiddenError` | 403 | User lacks required privileges |
| `ControllerNotFoundError` | 404 | Resource not found |
| `ControllerError` | 409 | General conflict (e.g., duplicate name) |
| `ControllerTimeoutError` | 408 | Operation timed out |
| `ComputeConflictError` | 409 | Compute node returned a conflict |
| `RequestValidationError` | 422 | Pydantic/FastAPI validation error (field missing or type mismatch) |
| `SQLAlchemyError` | 500 | Database error |
## Error Response Examples
### Template Already Exists (409)
```json
{
"message": "A template with name 'my-template' already exists"
}
```
### Validation Error (422)
```json
{
"message": "image field is required"
}
```
### Permission Denied (403)
```json
{
"message": "Permission denied (privilege Template.Allocate is required)"
}
```
### Invalid Template Type (400)
```json
{
"message": "JSON schema error received while creating new template: ..."
}
```
## Client-Side Error Handling
When handling errors in the GNS3 Web UI or API clients:
```javascript
// Extract error message from response
function extractErrorMessage(error) {
if (error.error && error.error.message) {
return error.error.message;
}
if (error.message) {
return error.message;
}
return 'Unknown error';
}
```
## References
- Exception handlers: `gns3server/api/server.py`
- Template validation: `gns3server/schemas/controller/templates/`
- Error classes: `gns3server/controller/controller_error.py`
@@ -0,0 +1,262 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is AI-generated with reference to actual code and verified
> against real-kernel testing (Linux 7.1.2-1-default). AI can make mistakes —
> please verify against the source code when in doubt.
# Builtin Ethernet Switch — uBridge brctl Backend
## Overview
The historical GNS3 Ethernet Switch was an emulated L2 device inside Dynamips
(`ethsw`). This implementation replaces it with a **real Linux kernel bridge**
driven through uBridge's `brctl` module — one bridge per switch node. The
migration makes the switch a first-class builtin node (no Dynamips dependency)
and enables native-kernel-speed L2 switching with VLAN filtering and QinQ.
| | Old (Dynamips ethsw) | New (uBridge brctl) |
|---|---|---|
| Switching engine | Dynamips user-space emulation | Linux kernel bridge (netlink) |
| VLAN model | ethsw ACL per port | Kernel VLAN filtering + PVID/untagged |
| QinQ | 0x8100/0x88A8/0x9100/0x9200 | 0x8100 (802.1Q) / 0x88A8 (802.1ad) |
| Data path | Node NIO ↔ ethsw NIO (Dynamips) | Node NIO ↔ uBridge relay ↔ TAP ↔ kernel bridge |
| Console | Inactive (reserved TCP port) | None (console_type=none) |
| Node type | `dynamips`-routed | `builtin` (always-on) |
## Architecture
```
┌───────────┐ ┌──────────────┐ ┌───────────────┐ ┌───────────┐
│ Peer A │ │ uBridge │ │ Kernel │ │ Peer B │
│ (Dynamips │◄───►│ per-port │◄───►│ Bridge │◄───►│ (IOU / │
│ / IOU / │ UDP │ relay │ TAP │ gns3{id[:6]}│ TAP │ QEMU / …) │
│ QEMU) │ │ nio_tap↔udp │ │ vlan_filter │ │ │
└───────────┘ └──────────────┘ └──────┬────────┘ └───────────┘
│
┌─────┴─────┐
│ ... more │
│ ports │
└───────────┘
```
Each switch port is a **dual-role TAP** — uBridge holds the file descriptor as a
`nio_tap` relay endpoint, and the same TAP is enslaved to the kernel bridge via
`brctl addif`. This is the same pattern the Cloud node already uses for host
bridges (`cloud.py:_add_linux_ethernet`). uBridge is **only** the per-port UDP
transport; the kernel bridge performs the actual MAC learning, forwarding, and
VLAN filtering.
### Component map
| File | Role |
|------|------|
| `compute/builtin/nodes/ethernet_switch.py` | Node implementation |
| `api/routes/compute/ethernet_switch_nodes.py` | REST endpoints (repointed to Builtin) |
| `schemas/compute/ethernet_switch_nodes.py` | Request/response models (unchanged) |
| `controller/udp_link.py` | Link creation — pushes NIO to switch via standard adapter endpoint |
## Lifecycle
### `create()` → `start()`
1. `_start_ubridge(require_privileged_access=True)` — launch uBridge instance
2. `_ensure_bridge()`:
- Derive deterministic bridge name: `gns3` + first 6 hex chars of `self.id`
- `brctl delete` (best-effort — crash recovery, cleans stale interfaces)
- `brctl create`
- `link set … up` (bridge is DOWN after create)
- `brctl vlanfiltering … on`
### `add_nio(nio, port_number)`
Per port, one uBridge relay bridge `{node_id}-{port}` is wired:
```
bridge create {node_id}-{port}
bridge add_nio_tap {node_id}-{port} "{tap}" ← uBridge holds TAP fd
brctl addif "{bridge}" "{tap}" ← enslave to kernel bridge
brctl vlan_del/vlan_add … ← apply port VLAN mode
bridge add_nio_udp {node_id}-{port} lport rhost rport
bridge reset_packet_filters {node_id}-{port} ← from _ubridge_apply_filters
bridge start {node_id}-{port}
```
Captures and marker signals are applied via the existing `_ubridge_apply_filters`
and `_ubridge_apply_markers` helpers from `BaseNode`.
### `remove_nio(port_number)`
```
brctl delif "{bridge}" "{tap}"
bridge delete {node_id}-{port}
release_udp_port(nio.lport)
```
### `close()`
```
for each port: release UDP port
brctl delete "{self._bridge_name}" ← kernel bridge teardown
_stop_ubridge() ← destroys remaining TAPs
```
**Cleanup paths:**
| Scenario | Bridge cleanup | TAP cleanup |
|----------|---------------|-------------|
| Normal project close | `close()` → `brctl delete` | uBridge stops → TAP fd closed → kernel destroys |
| gns3server crash / kill | Next `_ensure_bridge()` → `brctl delete` before `create` | uBridge dies → TAP fd closed by kernel |
| Manual project-file deletion after crash | Leaked (no GNS3 record of `gns3{id[:6]}`) | Leaked (same — but uBridge probably dead, TAPs gone with it) |
## Port mode → VLAN translation
All VLAN operations ride on the `brctl` hypervisor module (`../ubridge/doc/brctl.md`).
The kernel bridge must have `vlan_filtering on` before any `vlan_*` call.
### access VLAN N
```
brctl vlan_del {br} {tap} 1 ← remove default PVID 1
brctl vlan_add {br} {tap} N pvid untagged
```
### dot1q trunk (native VLAN V)
A dot1q trunk in ESW is "admit all VLANs tagged, native VLAN PVID + untagged":
```
brctl vlan_del {br} {tap} 1
brctl vlan_add {br} {tap} 1 vid 4094 ← admit all VIDs tagged
brctl vlan_add {br} {tap} V pvid untagged ← override native
```
### qinq (outer VLAN O, ethertype 0x88A8)
Bridge-level (once):
```
brctl setvlanproto {br} 0x88a8 ← switch to 802.1ad (outer S-tag)
```
Port-level:
```
brctl vlan_del {br} {tap} 1
brctl vlan_add {br} {tap} O pvid untagged ← S-tag push for untagged ingress
```
Ethertype 0x8100 qinq ports are treated as plain access ports (the bridge
defaults to 0x8100; no `setvlanproto` needed). Ethertype 0x9100/0x9200 are
not supported by the kernel bridge — see § Limitations.
### Runtime reconfiguration (`update_port_settings`)
On `ports_mapping` update, existing port VLANs are reset before re-apply:
```
brctl delif {br} {tap} ← release from bridge (clears VLAN state)
brctl addif {br} {tap} ← re-enslave (resets to default PVID 1)
brctl vlan_del/vlan_add … ← apply new mode
```
This prevents stale VLAN membership from a previous mode leaking into the new
configuration (e.g., access→trunk transition leaving old access VLAN behind).
## Bridge naming
Deterministic from the switch's UUID: `gns3` + first 6 hex chars (no dashes).
```
gns3a1b2c3 ← bridge (10 chars, ≤ 15 IFNAMSIZ limit)
gns3a1b2c3-0 ← tap for port 0 (12 chars)
gns3a1b2c3-1 ← tap for port 1 (12 chars)
```
- 6 hex = 48 bits of entropy — collision risk is astronomically low even with
thousands of switches on the same host.
- **Crash recovery**: `brctl delete` (best-effort, ignore if not found) then
`brctl create` — stale interfaces from a previous abnormal shutdown are
reclaimed automatically when the switch is re-created.
## Controller integration
No controller or API contract changes are required. The migration is entirely
compute-internal:
- `node_types.BUILTIN_NODE_TYPES` already classified `ethernet_switch` as a
builtin, always-running node.
- `udp_link.create()` pushes the NIO to the switch via the standard
`POST /adapters/0/ports/{p}/nio` endpoint (same as Dynamips).
- The REST API paths, request/response schemas, and port model
(`EthernetSwitchPort`: type/vlan/ethertype) are unchanged.
- `/start`, `/stop`, `/suspend`, `/reload` return 405 (switch is always-on).
The sole observable difference: the `console` field in the response is now
`null` (the switch has no console; `console_type="none"` makes `BaseNode`
skip TCP port reservation). The old Dynamips ethsw returned an unused TCP
port number. Both are valid under `Optional[int]`.
## Known limitations
### Default PVID 1 must be deleted explicitly
A port freshly enslaved to a `vlan_filtering` bridge inherits default PVID 1
(PVID + Egress Untagged). Access/trunk mode application must issue
`vlan_del … 1` first — `vlan_add … pvid` moves the PVID but does not remove
the old PVID's membership. This matches iproute2 semantics and is documented
in `../ubridge/doc/brctl.md#limitations`.
### QinQ is outer-tag (S-VLAN) only
With `setvlanproto 0x88a8` the bridge filters on the outer S-tag; the inner
C-tag passes through transparently. Selective QinQ (inner-VLAN classification
or remapping) requires `IFLA_BRIDGE_VLAN_TUNNEL_INFO` which is not implemented.
Documented in `../ubridge/doc/brctl.md#limitations`.
### Ethertype 0x9100 / 0x9200
The GNS3 schema allows legacy QinQ ethertypes `0x9100` and `0x9200`, but the
Linux kernel bridge only supports `0x8100` (802.1Q) and `0x88a8` (802.1ad).
Configuring these on a qinq port produces a `NodeError` at creation/update
time. Handling policy (map to 0x88A8 + warn vs. reject with error) is
pending per design discussion.
### No FDB read/write
The `brctl` module exposes no `fdb_show`/`fdb_flush`. The kernel bridge
learns and ages MAC entries autonomously; uBridge has never exposed MAC-table
access and gns3-server does not consume it. Consumers that need the FDB
(e.g., a WebUI switch view) should read `/sys/class/net/<br>/brforward` or
`bridge fdb show dev <br>` directly, without uBridge involvement.
## Troubleshooting
### Docker iptables: FORWARD chain DROP
Docker sets the iptables `FORWARD` chain default policy to `DROP` when the
Docker daemon starts. This blocks **all** forwarded traffic through kernel
bridges on the host, including `gns3*` bridges.
**Symptoms**: nodes can send frames into the bridge (visible in `tcpdump -i
gns3*`) but never receive unicast replies. ARP and multicast may work
because they flood, but unicast forwarding silently fails.
**Fix**:
```bash
sudo iptables -P FORWARD ACCEPT
```
### Bridge left DOWN after creation
`brctl create` creates the bridge but leaves it administratively DOWN.
The node now sends `link set … up` after `brctl create`. If forwarding
is not working, verify:
```bash
ip -d link show gns3* | grep -E "state|vlan_filtering"
```
### Kernel version differences
This implementation has been tested on Linux 7.1.2-1-default (x86_64) with
uBridge installed via `make install` (cap_net_admin,cap_net_raw=ep). The
ubridge `brctl` module has a 168-test suite covering kernel-side VLAN
behaviour on this kernel.
+146
View File
@@ -0,0 +1,146 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# Controller + Compute Setup
This document describes the minimum configuration required to set up a GNS3 Controller with remote Compute nodes.
## Architecture Overview
- **Compute**: Runs individual nodes (QEMU, Docker, etc.) and provides resource monitoring
- **Controller**: Manages multiple computes, projects, and provides the REST API
- **Database**: Controller uses SQLite to store projects, nodes, and compute registration
## Minimum Configuration
### 1. Compute Node Configuration
Create the configuration file at `~/.config/GNS3/3.1/gns3_server.conf`:
```ini
[Server]
host = 0.0.0.0
port = 3080
compute_username = gns3
compute_password = gns3
```
Start the Compute:
```bash
gns3server
```
### 2. Controller Node Configuration
Create the configuration file at `~/.config/GNS3/3.1/gns3_server.conf`:
```ini
[Server]
host = 192.168.1.140
port = 3080
compute_username = gns3
compute_password = gns3
```
Start the Controller:
```bash
gns3server
```
### 3. Register Compute with Controller
Use the API to register a remote compute:
```bash
POST /v3/computes
{
"protocol": "http",
"host": "192.168.1.x",
"port": 3080,
"user": "gns3",
"password": "gns3"
}
```
## Important Notes
### Host Configuration
- **Controller `host`**: If set to `0.0.0.0`, the controller will register itself as `127.0.0.1`, which breaks remote compute connections
- **Symptom**: Compute nodes report "No common subnet for compute X (controller) and Y" even when on the same network
- **Solution**: Always use the actual LAN IP address for the Controller's `host` field (e.g., `host = 192.168.1.104`)
- **Dynamic IP**: If the controller machine uses DHCP, set a static lease on the router or use mDNS (`.local` domain)
### Password Configuration
- If `compute_password` is not set, a random 16-character password is auto-generated on startup
- The Controller must use the same credentials as the Compute's configuration
### Network Requirements
- Controller and Compute must be on the same LAN for cross-compute links to work
- UDP tunnel is used for cross-compute links, requiring network connectivity on UDP ports
### Configuration File Location
- Default location: `~/.config/GNS3/3.1/gns3_server.conf`
- Version `3.0` uses `~/.config/GNS3/3.0/`
## Troubleshooting
### 401 Unauthorized on Compute Connect
1. Check that Compute's `compute_username` and `compute_password` match what was passed to the API
2. Verify the Compute's configuration file is correctly loaded
3. Ensure the `[Server]` section is used (not `[Controller]`)
### No Common Subnet Error
If compute nodes report:
```
Cannot get an IP address on same subnet: No common subnet for compute X (controller) and Y
```
1. **Primary cause**: Controller's `host` is set to `0.0.0.0` - it registers as `127.0.0.1` which is unreachable from compute nodes
2. Check the controller's `/v3/version` endpoint - if `controller_host` shows `127.0.0.1`, this is the issue
3. Set the Controller's `host` to its actual LAN IP address (e.g., `192.168.1.104`)
4. Verify both machines are on the same network and can ping each other
5. Check firewall rules allow TCP/UDP communication on required ports
### WebSocket Console Authentication Failed
When connecting to a remote compute node's console via WebSocket, the connection may fail with errors in the logs:
**Controller log:**
```
New client 192.168.1.104:33268 has connected to controller console WebSocket
Forwarding console WebSocket to 'ws://192.168.1.3:3080/v3/compute/projects/.../console/ws'
Client 192.168.1.104:33268 has disconnected from controller console WebSocket
```
**Compute log:**
```
WebSocket /v3/compute/projects/.../console/ws" [accepted]
ERROR gns3server.api.routes.compute.dependencies.authentication:103 Could not authenticate while connecting to compute WebSocket: Could not validate credentials
```
**Cause**: The Controller forwards its own `compute_username` and `compute_password` credentials when connecting to the Compute's WebSocket endpoint. If the Compute node does not have matching credentials configured, authentication fails.
**Solution**: Ensure the Compute node's configuration file has matching credentials:
```ini
[Server]
enable_http_auth = True
compute_username = gns3
compute_password = your_password
```
The Controller automatically uses its configured `compute_username` and `compute_password` when establishing WebSocket connections to remote computes (see `gns3server/api/routes/controller/nodes.py` lines 627-628).
**Note**: If `compute_password` is not explicitly set in the Compute's config, a random 16-character password is auto-generated at startup, making authentication impossible for any external Controller.
+465
View File
@@ -0,0 +1,465 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# Docker exec Console (Vendor NOS Containers)
## Overview
GNS3 Docker nodes normally expose their console by attaching to the container's
PID 1 stdio. That works for CLIs that run as PID 1 (e.g. FRR's `vtysh`), but it
does **not** work for vendor NOS containers (Nokia SR Linux, Arista cEOS,
Juniper cRPD, …) whose CLI is a separate, full-screen TUI process that is *not*
on PID 1. For those, attaching to PID 1 only shows boot logs and never yields a
CLI prompt.
The `docker_exec` console type solves this. It runs a chosen command inside the
running container via the Docker exec API (with a pty) and bridges it to the
GNS3 console, so the vendor's native TUI CLI renders in the Web UI (xterm.js)
exactly as if you had run `docker exec -it <container> <cli>` in a real
terminal.
Two companion environment knobs (`GNS3_SKIP_INIT`, `GNS3_INTERFACE_NAMES`) make
the container itself boot and wire correctly for vendor NOS images. Together
they let a vendor NOS run as a first-class GNS3 Docker router node.
> Prototype status: the knobs are environment-driven and intentionally avoid
> schema changes, so existing Docker nodes (FRR, ipterm, …) are unaffected.
> `console_type: "docker_exec"` is added to the `ConsoleType` enum.
## The three environment knobs
All three are read from the node's `environment` field. Entries prefixed with
`GNS3_` are **not** forwarded into the container (existing GNS3 behaviour), so
they stay host-side configuration.
| Variable | Purpose |
|----------|---------|
| `GNS3_SKIP_INIT=1` | Do **not** prepend `/gns3/init.sh` to the entrypoint. Vendor NOS images must run their own entrypoint (e.g. SR Linux's `sr_linux`); GNS3's init script (busybox bootstrap, `ifup`, eth wait) interferes with them. |
| `GNS3_INTERFACE_NAMES=mgmt0,e1-1,e1-2,e1-3` | Rename the injected interfaces in adapter order instead of the default `eth{N}`. SR Linux expects `mgmt0` + `e1-N`; without this it does not recognise its datapath. |
| `GNS3_CONSOLE_CMD=/opt/srlinux/bin/sr_cli` | Command run by the `docker_exec` console inside the container. |
## Architecture: `VendorDockerVM` subclass
All vendor-specific logic lives in a `VendorDockerVM(DockerVM)` subclass in
`gns3server/compute/docker/vendor_docker_vm.py` — `docker_vm.py` itself stays
on its baseline behaviour and is never touched by this feature.
`DockerVM` exposes four small extension hooks (pure refactorings, zero
behaviour change for existing nodes):
| Hook | Baseline behaviour | `VendorDockerVM` override |
|------|--------------------|---------------------------|
| `_prepare_init_and_interface_env(params)` | prepend `/gns3/init.sh`, set `GNS3_MAX_ETHERNET=eth{N-1}` | conditional init.sh (`GNS3_SKIP_INIT`), `GNS3_MAX_ETHERNET` follows the interface rename |
| `_start_console_server()` | telnet/ssh/http console dispatch | adds the `docker_exec` branch |
| `_get_container_ifname(adapter_number)` | `eth{N}` | `GNS3_INTERFACE_NAMES` lookup, fallback `eth{N}` |
| `_cleanup_console_resources()` | no-op | closes the docker-exec pty socket before restart/stop |
### Class selection
The Docker manager picks the class per node in `Docker.create_node()`
(`gns3server/compute/docker/__init__.py`):
```python
def _select_node_class(self, **kwargs):
if kwargs.get("console_type") == "docker_exec":
return VendorDockerVM
return DockerVM
```
`console_type == "docker_exec"` is the **only** trigger — every other console
type (telnet, vnc, ssh, http, …) keeps using the unmodified `DockerVM`. All
vendor features are opt-in: without the `GNS3_*` environment variables a
`VendorDockerVM` instance behaves identically to `DockerVM` (init.sh still
runs, interfaces stay `eth{N}`, the exec command defaults to `/bin/sh`), so a
regular container can use `docker_exec` too.
## The `docker_exec` console type
Setting `console_type: "docker_exec"` makes the node's primary console port run
`_start_docker_exec_console()` instead of the attach-to-PID-1 path.
### Console architecture
```mermaid
graph LR
A[Web UI xterm.js] -->|console WS| B[GNS3 Compute telnet server]
B -->|binary pty stream| C[Docker exec API]
C -->|Tty:true pty| D[sr_cli / vendor CLI]
A -.->|NAWS size| B
B -.->|POST exec/.../resize| C
```
The console uses GNS3's **existing shared/broadcast telnet model**: a single
exec instance (one CLI session) is broadcast to every console client, exactly
like the primary console shares one PID 1. There is deliberately **no
per-client session isolation** — this matches how every other GNS3 console
behaves.
### Implementation
**File**: `gns3server/compute/docker/vendor_docker_vm.py` —
`_start_docker_exec_console()`
A small subclass `_LazyExecTelnetServer(AsyncioTelnetServer)` implements the
console. Key points:
1. **Lazy exec creation.** The exec is created on the **first client
connection** (`client_connected_hook`), not when the node starts. This is
essential: vendor CLIs (e.g. `sr_cli` via `prompt_toolkit`) send a
cursor-position request (`\e[6n`, CPR) during startup and block waiting for
the terminal's answer. If the exec starts at node-start time there is no
xterm.js client to answer, the probe times out, and the TUI degrades (no
status bar, "Terminal doesn't support CPR" warning). Creating the exec on
first connect means the probe runs with a real xterm.js attached, which
answers CPR → full TUI. After creation the exec is shared by all clients.
2. **Exec API with a pty.** `POST containers/{cid}/exec` with
`Tty: true`, `User: "root"` (vendor CLIs reject the image's default
unprivileged user — SR Linux returns *"User 'user' is not authorized to use
CLI"* otherwise), and `Env: ["TERM=xterm"]` (the TUI library needs a
recognised terminal).
3. **No while-true wrapper.** The command runs as `sh -c "<cmd>"` (no
restart loop). When the CLI exits (`quit`, the NOS's own idle timeout, or a
crash) the exec pty closes, the broadcast task ends, and the next client
connection **recreates** the exec (see *Reconnection*). A `while true`
wrapper would restart the CLI mid-session with no client attached to
answer its startup CPR probe, producing a blank/degraded screen on
reconnect.
### Reconnection
The exec is created lazily and **recreated on reconnect if it has died**.
`client_connected_hook` checks `_upstream_alive()` (exec id set, writer open,
broadcast task not done) before each connect:
- **First connect / dead upstream** → (re)create the exec. Because a client is
now attached, the CLI's startup CPR probe is answered by xterm.js → full
TUI. A half-dead writer is closed first to avoid a socket leak.
- **Live upstream** → reuse the existing exec, just send `Ctrl-L` to redraw
for the new client.
This is what makes the console survive `quit`, idle timeout, and CLI
crashes: the death is detected (pty EOF ends the broadcast task) and the
next connection spins up a fresh exec with a terminal present. The
`_LazyExecTelnetServer` is extracted to module level specifically so this
reconnect logic is unit-tested.
4. **Hijacked raw-HTTP start.** The exec is started with
`POST exec/{eid}/start` sent as a raw HTTP upgrade over the Docker unix
socket (`asyncio.open_unix_connection`), the same approach docker-py uses.
This is required because aiohttp's websocket client (`ws_connect`) is
rejected by Docker's exec-start endpoint (HTTP 400), while a raw POST
upgrade succeeds (101). With `Tty:true` the response body is a raw,
non-multiplexed bidirectional pty byte stream — no frame demux needed.
5. **NAWS → exec resize.** The telnet server runs with `naws=True`; the
`window_size_changed_callback` calls `POST exec/{eid}/resize?h=&w=` so the
TUI lays out for the xterm.js window size.
6. **Binary passthrough + redraw.** `binary=True` so TUI escape sequences reach
xterm.js intact; `echo=False` (the pty echoes). On every client (re)connect
a `Ctrl-L` (`\x0c`) is sent to the pty so a TUI that already drew its
screen for a previous client redraws for the new one (otherwise a
reconnect shows a blank screen until the next output).
**File**: `gns3server/compute/base_node.py` — the console WebSocket guard now
allows `docker_exec` (alongside `telnet`/`ssh`), since the WS bridge connects to
the console TCP port exactly as it does for telnet.
### Why earlier approaches failed (context)
- `script` + `docker exec -it`: the `script` pty had size 0 (no NAWS) → the TUI
could not lay out → blank.
- `docker exec -i` (no `-t`) + `sr_cli -d` (dumb mode): line-mode output was
block-buffered and visually messy.
- Direct pipe relay: telnet `CRLF` polluted line input.
The exec-API approach fixes all of these: a real pty (`Tty:true`), a real size
(NAWS resize), and a real terminal emulator (xterm.js answering CPR).
## Configuration
### SR Linux node example
```json
{
"name": "srlinux-1",
"node_type": "docker",
"image": "ghcr.io/nokia/srlinux:latest",
"adapters": 4,
"console_type": "docker_exec",
"start_command": "sudo -E bash -c 'touch /.dockerenv && /opt/srlinux/bin/sr_linux'",
"environment": "GNS3_SKIP_INIT=1\nGNS3_INTERFACE_NAMES=mgmt0,e1-1,e1-2,e1-3\nGNS3_CONSOLE_CMD=/opt/srlinux/bin/sr_cli"
}
```
- `start_command` is the SR Linux launch line (as used by containerlab).
- Connect the node's ports as usual — links still use GNS3's UDP NIO datapath
(container-agnostic); the rename only affects the in-container interface name.
- For the Web UI port **labels** to match (display `mgmt0`/`e1-1` instead of
`Ethernet0..3`), set `custom_adapters` per port
(`{"adapter_number": 0, "port_name": "mgmt0"}`, …). Port labels are a
controller-side concept, independent of the compute-side interface rename.
### Appliance (`gns3a`) packaging
A SR Linux appliance lives in `gns3-registry/appliances/srlinux.gns3a`
(`registry_version: 6`). It sets the full chassis — **35 adapters**
(`mgmt0` + `e1-1`..`e1-34`) — with matching `GNS3_INTERFACE_NAMES` and 35
`custom_adapters` entries (`mgmt0`, `e1-1`..`e1-34`) so the canvas labels,
the kernel interface names and the `ethernet-1/N` CLI names all line up.
Three appliance-schema fixes are required for this appliance to load (all on
the gns3-server side; the registry JSON schema is unchanged because its docker
block allows `additionalProperties`):
1. **`DockerConsoleType`** (`schemas/controller/appliances.py`) must include
`docker_exec`, or the Pydantic appliance model rejects the file at import.
2. **`ApplianceV1_6.custom_adapters`** must be declared on the top-level
appliance model, or `GET /appliances` (response_model=`schemas.Appliance`)
strips `custom_adapters` from the API response even though the file and the
server-side template conversion handle it. (Node creation still worked
because `appliance_to_template._add_docker_config` reads it from the raw
dict; only the GET response was lossy.)
3. `extra_volumes` rides inside the `docker` block (passed through by
`new_config.update(appliance_config["docker"])`); no schema change needed.
> **Symbol theme caveat.** An appliance `symbol` that starts with
> `:/symbols/` is forcibly rewritten at load time
> (`appliance_manager._load_appliances`) to the current theme's default for the
> appliance category — so `:/symbols/affinity/circle/blue/router_cloud.svg` (or
> `router2.svg`) becomes `:/symbols/affinity/circle/blue/router.svg`, because
> the theme maps only the canonical name `"router"`. This is intentional: it
> lets theme switching re-skin every node consistently. To use a non-default
> icon (e.g. `router_cloud`), install it as a **custom symbol** under the
> configured `symbols_path` and reference it by filename (no `:/symbols/`
> prefix) — custom symbols do not participate in re-theming. The SR Linux
> appliance uses `router.svg`.
### Persistent state
For SR Linux, persist `/etc/opt/srlinux` (config / AAA users / TLS certs) and
`/var/log/srlinux` (logs, optional) by adding them to the node's
`extra_volumes`. The image also declares its own `VOLUME` directories
(e.g. `/opt/srlinux/appmgr`), which GNS3 persists automatically.
## Volume persistence with `GNS3_SKIP_INIT`
This is the one place where skipping init.sh changes behaviour beyond boot:
`/gns3/init.sh` normally performs the volume-persistence bridge, and without it
**nothing writes through to the host** — the container writes to its overlay
filesystem and the data is lost on stop.
The bridge (see init.sh lines 35–52) has two parts:
```
host ──Docker bind mount──▶ /gns3volumes/etc/opt/srlinux (always mounted)
│ init.sh: mount --bind
▼
/etc/opt/srlinux (where the NOS writes)
```
`VendorDockerVM` replicates this for SKIP_INIT containers:
1. **`_setup_skip_init_volumes()`** — runs once per start, right after the
container is up (`VendorDockerVM.start()`). For each persistent volume it
`docker exec`s a busybox script that:
- seeds the host directory with the container's original files on first
start (`cp -a` + `.gns3_perms` marker), exactly like init.sh;
- `mount --bind /gns3volumes<path> <path>` to bridge persistent storage
back to the in-container path — on subsequent starts the persisted data
replaces the fresh overlay content;
- restores the permissions recorded in `.gns3_perms` at the previous stop
(best-effort).
2. **Container-side `_fix_permissions()` override targeting `/gns3volumes`**
— `DockerVM._fix_permissions` operates on the in-container paths
(`/etc/opt/srlinux`, …), which only resolve to persistent storage while
the `mount --bind` bridge is up; after a container restart the bridge is
gone and it would chown the overlay copy instead of the host files. It
also restarts an exited container just to chown. The override instead
runs the same busybox record/chmod/chown script **inside the container
(as root) on the `/gns3volumes<path>` paths** — the Docker bind-mount
targets, which exist for the whole container lifetime and need no bridge.
A stopped/exited container is **not** restarted: the pass is skipped and
the next start fixes ownership. It runs at start (so the controller can
read project files while the node runs) and at stop (for files written
during runtime).
> The fix must run container-side: files written by the container are
> host-side root-owned, and an unprivileged GNS3 process cannot chown them
> from the host. Container-side root (with GNS3's `UsernsMode: host`) can.
With `GNS3_SKIP_INIT`, GNS3's hardcoded `/etc/network` volume (see
`docker_vm.py` `_mount_binds()`) is dropped entirely by
`VendorDockerVM._mount_binds()`: it holds GNS3's own network config for
init.sh's `ifup`, which never runs for SKIP_INIT containers — the NOS
manages its own interfaces. The override removes the bind, filters the
volume out of `self._volumes`, and deletes the host-side skeleton directory
the base class just created. Without `GNS3_SKIP_INIT` the mount is kept
(behaviour matches the base class).
### Lifecycle summary
| Phase | Normal Docker node | `VendorDockerVM` + `GNS3_SKIP_INIT` |
|-------|--------------------|--------------------------------------|
| start | init.sh seeds + bind-mounts + restores perms (in-container, before the app starts) | `docker exec` after start: seed + bind-mount + restore perms; then container-side chown on `/gns3volumes` |
| stop | container-side `_fix_permissions` on in-container paths (restarts an exited container) | container-side `_fix_permissions` on `/gns3volumes` paths (skips dead containers, no restart) |
| volume config | identical `_mount_binds` (host → `/gns3volumes<path>`) | identical |
### Runtime ownership safety
The start-time fix pass chowns the volume files to the host user **while the
container is running** — a deliberate deviation from the standard model, where
init.sh restores container-native ownership at start and the container never
sees host-owned files during runtime. Verified harmless for SR Linux:
1. **Most processes run as root** (`sr_linux`, appmgr) — root ignores file
ownership entirely.
2. **Self-healing daemons.** SR Linux's `aaamgr` rewrites its managed files
with its own ownership at boot: after the start-time pass chowned
`etc/opt/srlinux/aaamgr_local_user.json` to the host user, the daemon
re-created it as `srlinux:srlinux` (uid 1002, mode 700) within seconds.
3. **ACL-based access.** The directory carries a default ACL
(`default:group:srlinux:rwx`, `default:other::rwx`), so named group ACL
entries grant access independently of the owner uid; the observed file ACL
(`group:srlinux:rwx`, owner `srlinux`) survives chown.
Caveat: a NOS that strictly validates ownership of its files (e.g. "SSH keys
must be root:root 600 or refuse to start") would not tolerate this. If that
ever matters, drop the start-time pass and keep only the stop-time one
(standard behaviour — the trade-off is mid-run `Permission denied` in the
file browser, identical to regular Docker nodes).
### Boot-ordering caveat
The volume bridge (`mount --bind`) is established **after** the vendor
entrypoint has started (there is no init.sh to do it before), so the NOS's
early boot reads the overlay copy of the volume paths — default image
content, not the persisted data. Whether the persisted config takes effect
depends on the NOS re-reading those files after the bridge is up (SR Linux's
daemons do re-read/write their managed files during boot, as observed).
Always verify the closed loop when adopting a new image: `save` a config →
stop the node → start it → confirm the config is actually applied, not just
present on the host.
## Troubleshooting
**1. Console shows only boot logs, no CLI**
- You are on the primary attach console. Set `console_type: "docker_exec"` and
use `GNS3_CONSOLE_CMD` to point at the vendor CLI.
**2. `User '...' is not authorized to use CLI`**
- The exec must run as root. The implementation sets `User: "root"`; if you
fork it, keep that.
**3. `Terminal doesn't support cursor position requests (CPR)`**
- This means the exec was started without an xterm.js client connected (the
startup probe had no one to answer). The lazy-start design avoids this; if you
see it, ensure the exec is created on first connect, not at node start.
**4. Reconnecting the Web console shows a blank screen**
- A `Ctrl-L` is sent on each connect to force a TUI redraw. If the TUI does not
redraw, verify the `client_connected_hook` still writes `\x0c` to the pty.
**5. `aiohttp WSServerHandshakeError: 400` on exec start**
- Do **not** use the websocket client to start an exec. Use the hijacked raw
HTTP upgrade over the unix socket (see Implementation).
**6. SR Linux data interfaces stay down**
- SR Linux defaults its data ports to `admin-state disable`; enable them in the
CLI (`interface ethernet-1/1 admin-state enable`) and bind the interface to a
network-instance before ping works. This is SR Linux behaviour, not a GNS3
issue.
**7. "Session has been idle, will logout in 300 seconds" → Connection closed**
- SR Linux's own CLI idle timeout logs the CLI out, the exec pty closes, and
the console disconnects. Reopening the console recreates the exec (see
*Reconnection*) and gives a fresh login. To keep a permanent session,
disable the timeout in the CLI: `enter candidate` →
`/system cli idle-timeout disable` → `commit now`.
**8. Controller logs `Permission denied` reading files under the node's
project directory while the node runs**
- Root-written files inside a persistent volume. The container-side
`_fix_permissions` pass runs at start (fixes the seeded files) and at stop;
files created by the container *during* runtime become readable after the
next stop.
- Concrete example: SR Linux's `aaamgr` daemon rewrites
`etc/opt/srlinux/aaamgr_local_user.json` during boot, **after** the
start-time pass, as the image's `srlinux` user (uid 1002, mode 700) — so
the host-side file stays `1002:1002` until the stop-time pass chowns it.
- The log line comes from the file-browser API chain: Web UI *Show in file
manager* → `GET /v3/projects/{pid}/nodes/{nid}/files`
(`controller/nodes.py:538`) → `project.list_node_files`
(`compute/project.py:510`), where `magic.from_file()` cannot read the
file and the `file_type` field is left empty for that entry. The MCP
`list_node_files` tool uses the same code path. Size/modified-at fields
and everything else keep working; only the type sniff and one warning
line are affected — same behaviour as any regular Docker node writing
root-owned files at runtime.
**9. Persistent volume empty on the host after `save` + stop**
- Ensure `GNS3_SKIP_INIT=1` is set (so the host-side bridge path is taken) and
the volume path is in `extra_volumes`; check the compute log for
`Volume '<path>' bound to persistent storage`.
**10. Persisted config present on the host but not applied after restart**
- The volume bridge is established after the NOS has booted (see
*Boot-ordering caveat*); the NOS may have already loaded the overlay's
default config into memory. Verify with a visible change (hostname,
interface description): `save` → stop → start → check the change took
effect. If it does not, the image needs the bridge earlier (a
vendor-specific entrypoint wrapper, not covered by this prototype).
## Limitations
1. **Shared session (broadcast).** All console clients share one CLI session
and can see each other's input — identical to GNS3's existing primary
console model. There is no per-client independent session.
2. **`reset_console` not wired.** The console-reset action only handles
`telnet`/`ssh`; it is a no-op for `docker_exec` (non-blocking; reconnect
works fine).
3. **Prototype knobs.** `GNS3_SKIP_INIT` / `GNS3_INTERFACE_NAMES` /
`GNS3_CONSOLE_CMD` are environment-driven; they are not yet first-class node
schema fields and are not declared in the appliance (`gns3a`) schema.
4. **Rootful-Docker assumption** (`UsernsMode: host`, set for all GNS3
Docker nodes) so the container-side chown acts on the host files' real
uid/gid (see the volume-persistence section).
5. **Post-boot volume bridge.** The bind-mount bridge is established after the
vendor entrypoint has started (init.sh would do it before). A NOS that
strictly requires its persisted files at its very first read may need a
different boot arrangement (see *Boot-ordering caveat*).
## References
- `gns3server/compute/docker/vendor_docker_vm.py` — `VendorDockerVM`:
`_start_docker_exec_console`, `_LazyExecTelnetServer`,
`_setup_skip_init_volumes`, container-side `_fix_permissions` on
`/gns3volumes`, `start()`.
- `gns3server/compute/docker/docker_vm.py` — `DockerVM` extension hooks
(`_prepare_init_and_interface_env`, `_start_console_server`,
`_get_container_ifname`, `_cleanup_console_resources`).
- `gns3server/compute/docker/__init__.py` — `Docker._select_node_class` /
`create_node` factory.
- `gns3server/compute/base_node.py` — console WebSocket guard.
- `gns3server/schemas/common.py` — `ConsoleType.docker_exec`.
- containerlab `nodes/srl/srl.go` — reference for SR Linux launch command and
interface naming.
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.5 | 2026-08-13 | Add appliance (`gns3a`) packaging section: 35-adapter full-chassis design, the three server-side schema fixes (DockerConsoleType, ApplianceV1_6.custom_adapters, extra_volumes passthrough), and the symbol-theme caveat (any `:/symbols/` symbol is rewritten to the category default at load). |
| 1.4 | 2026-08-13 | Reconnect fix: drop the while-true wrapper (it restarted the CLI with no client to answer CPR → blank screen on reconnect); the exec is now recreated on connect when the upstream has died. `_LazyExecTelnetServer` extracted to module level and unit-tested. |
| 1.3 | 2026-08-12 | Document runtime ownership safety (root processes, self-healing daemons, ACL evidence for SR Linux), the boot-ordering caveat (bridge after boot → verify save/stop/start closed loop), and troubleshooting #10. |
| 1.2 | 2026-08-12 | `_fix_permissions` rewritten: container-side (as root) on the `/gns3volumes` bind-mount targets instead of host-side — host-side chown cannot touch root-owned files when GNS3 is unprivileged. Dead containers are skipped instead of restarted. |
| 1.1 | 2026-08-12 | Refactor: vendor logic extracted from `DockerVM` into `VendorDockerVM` subclass with 4 hook points + class-selection factory. Add SKIP_INIT volume persistence (`_setup_skip_init_volumes` + `_fix_permissions`) and lifecycle comparison. Add troubleshooting entries for idle timeout and permission-denied files. |
| 1.0 | 2026-08-12 | Initial documentation of the `docker_exec` console and vendor NOS knobs. |
+378
View File
@@ -0,0 +1,378 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# Marker (Traffic Insight)
## Overview
A **marker** is a passive traffic-insight tap attached to a link. It runs a libpcap BPF
expression inside uBridge; on every match uBridge emits a real-time `MARK` signal and
appends the matching packet to a per-marker pcap file. Markers exist at two layers that
coexist on the same link: **per-link private markers** and **project-level definitions**
that are inherited by every capable link.
## Architecture
```mermaid
graph TB
UI["Web UI"]
subgraph Controller["Controller"]
DEF["Project definitions<br/>(inheritance templates)"]
LNK["Per-link markers"]
end
Compute["Compute Node"]
UB["uBridge<br/>mark filter"]
PCAP[("pcap file")]
LSTN["Marker listener<br/>(UDP, per compute)"]
UI -->|"REST + notifications ws"| Controller
DEF -.->|"fan-out: global-{name}"| LNK
LNK -->|"node.post /markers"| Compute
Compute --> UB
UB -->|"BPF match"| PCAP
UB -->|"UDP MARK signal"| LSTN
LSTN -->|"marker.match"| UI
```
Inheritance is a controller-only fan-out: a definition CRUD loops over links and reuses the
existing per-link marker operations, so the compute side sees an ordinary marker and is
unchanged. Each compute process runs one UDP listener serving every uBridge on that host; the
`node` and `link` fields in each signal together identify the source link (see
[Per-link attribution](#per-link-attribution)).
## Business Process
```mermaid
sequenceDiagram
participant UI as Web UI
participant C as Controller
participant L as Capable Link
participant N as Compute / uBridge
UI->>C: POST /marker-definitions {name, bpf, ...}
C->>C: store definition
loop every capable link
C->>L: start_marker("global-{name}")
L->>N: install mark filter (BPF + pcap)
end
C-->>UI: 201 + link_ids
Note over N: later: a packet matches the BPF
N->>N: emit MARK signal + append pcap
N-->>UI: marker.match notification (per-project ws)
```
Updating a definition syncs `bpf / tag / color / highlight_duration` to every inherited
copy; deleting a definition removes every inherited copy. A newly created link inherits all
existing definitions automatically.
## Per-link attribution
A uBridge `MARK` signal carries `node`, `filter`, `link`, `tag`, and `len` — but no bridge
name. When one node is the capture side for several links — the common case for a project-level
`global-{name}` marker on a multi-interface router — `node` + `filter` alone are identical
across those links, so they cannot tell the signals (or pcap files) apart. The `link` field
resolves this:
1. At install time the controller stamps each filter with its link id
(`mark <bpf> [tag <id>] link <link_id> [pcap <path>]`).
2. uBridge treats `link` as opaque and echoes it verbatim in the signal (`link=<link_id>`).
3. The listener takes the signal's `link=` as the **authoritative** `link_id` of the
`marker.match` event, falling back to its registry only for legacy signals that carry no
`link=`.
This is also why the pcap path is keyed on link —
`<project>/markers/<node_id>_<link_id>_<filter>.pcap`, not on `bridge`+`filter`: a single
uBridge bridge can serve several links, and only the link id keeps their captures distinct.
### IOU: one bridge, many interfaces
IOU runs a single `IOL-BRIDGE` per node shared by every interface, so `bridge`+`filter` are
identical across that node's links. uBridge keeps a separate filter list **per port
(bay/unit)** within the bridge, so each interface gets its own `global-{name}` filter, its own
pcap file, and its own `link=`. The shared bridge name is irrelevant to attribution. Other
capable node types (`qemu`, `docker`, `vpcs`, `cloud`) already use one bridge per link; `link`
applies uniformly to all of them.
## Direction
A `MARK` signal optionally carries `dir=<tx|rx>` — the matched packet's travel direction
**relative to the capture node** (the `node=<id>` in the same signal, i.e. the node whose
uBridge hosts the marker):
| `dir` | Ingress NIO | Meaning |
|-------|-------------|---------|
| `tx` | device side (`source_nio` on a generic bridge; the IOL instance on an IOU `IOL-BRIDGE`) | capture node is **sending** |
| `rx` | link side (`destination_nio` on a generic bridge; the NIO side on an IOU `IOL-BRIDGE`) | capture node is **receiving** |
A marker is single-sided: only the chosen capture node's uBridge installs the `mark` filter,
yet both directions of the link transit that one bridge (it carries exactly two NIOs — the
device side and the link side), so that single uBridge observes and classifies both
directions. The `marker.match` event forwards `dir` through unchanged; the Web UI combines it
with the link's two endpoints and the capture `node_id` to draw an arrow:
- `dir=tx` → `capture_node → far_node`
- `dir=rx` → `far_node → capture_node`
- `dir` absent (older uBridge) → undirected highlight (current behaviour)
Because the listener ignores unknown keys, `dir` is **additive**: an older server silently
drops it and an older uBridge simply omits it — either way the system falls back to
undirected rendering with no error.
### Choosing the capture node
Since `dir` is relative to the capture node, *which* endpoint is the observer decides what
`tx`/`rx` mean. By default the server auto-picks (first started marker-capable endpoint, in
link-endpoint order). To pin it — e.g. so `dir=tx` unambiguously means "vpcs1 is sending" —
pass `capture_node_id` on marker **create**:
```json
{ "bpf": "icmp", "direction": "tx", "capture_node_id": "<vpcs1 node uuid>" }
```
The value must be one of the link's two endpoints and a marker-capable type (`vpcs`, `qemu`,
`docker`, `iou`, `dynamips`, `cloud`); any other id is rejected with `409`. Omit it to keep
the auto-pick. The chosen id is echoed back as `capture_node_id` in the marker entry and in
each `MARK` signal's `node=<id>`, so the Web UI always knows the observer regardless of who
picked it.
`capture_node_id` is **create-only**: it is fixed once the marker exists (changing the
observer would silently flip the meaning of stored `direction`, so recreate the marker
instead). It is not accepted on project-level definitions — a definition is link-agnostic and
has no endpoints to choose from, so inherited markers always auto-pick per link.
For the same reason, a definition **rejects `direction: tx|rx`** (HTTP 409): each inherited
copy auto-picks its capture node, so a fixed tx/rx would denote different session directions
on different links. A definition is `both` only; encode the direction you want in the BPF
instead — e.g. `icmp and icmp[icmptype]==8` for echo requests, a packet-intrinsic property
that is consistent on every link regardless of capture node. tx/rx remains available on
per-link markers, where the capture node is fixed.
## Pause & resume
Two levels of silencing, both instant (no NIO rebuild, no pcap flush):
- **Per-marker (private)** — `PUT /v3/projects/{pid}/links/{lid}/markers/{name}`
with `{"enabled": false}` flips that one filter off in place (uBridge
`enable_packet_filter … off`): no signal, no pcap, but traffic still relays —
a paused `mark` is a no-op tap, not a drop. `{"enabled": true}` flips it back.
A change to `enabled` alone is a single command (the pcap identity and emitted
counter are preserved). Changing `bpf`, `tag`, or `direction` rebuilds just that
one filter (`delete_packet_filter` + add) — only that marker's own pcap reopens
(a new capture session for the new BPF); changing `color`/`highlight_duration`
is UI-only, nothing is pushed to uBridge.
- **Per-definition (inherited)** — `POST /v3/projects/{pid}/marker-definitions/{name}/pause`
and `/resume` toggle **every** inherited `global-{name}` copy across all links
at once (same `enable_packet_filter on|off`, fanned out per copy). Use to
pause or resume a whole rule independently of the others. The definition's
`paused` flag is persisted to the `.gns3` and echoed on the definition object,
so links created later inherit it already paused, and the Web UI renders the
per-rule button from server truth.
| Action | signal | pcap | sink |
|--------|--------|------|------|
| per-marker `enabled: false` | stop | stop | n/a |
| per-def `pause` (all `global-{name}` copies) | stop | stop | n/a |
| per-def `resume` | resume | resume | n/a |
## Capture files
Each marker appends matches to `<project>/project-files/markers/<node_id>_<link_id>_<filter>.pcap`.
Removing a marker — per-link `DELETE .../markers/{name}` or deleting a definition (which
removes every inherited copy) — deletes that marker's pcap too, even with the capture node
stopped (the filter is removed with `delete_packet_filter`, the file is unlinked). uBridge's
`reset_packet_filters` (run on NIO/filter changes) preserves mark filters, so unrelated
changes no longer close/reopen any marker's pcap.
## API Endpoints
All endpoints require a JWT bearer token (`POST /v3/access/users/authenticate`). The
`Auth` column lists the required privilege.
### Per-link markers
| Method | Path | Description | Auth |
|--------|------|-------------|------|
| GET | `/v3/projects/{pid}/links/{lid}/markers` | List markers on a link | Link.Audit |
| POST | `/v3/projects/{pid}/links/{lid}/markers` | Attach a marker | Link.Modify |
| PUT | `/v3/projects/{pid}/links/{lid}/markers/{name}` | Update a marker | Link.Modify |
| DELETE | `/v3/projects/{pid}/links/{lid}/markers/{name}` | Remove a marker | Link.Modify |
### Project-level definitions
| Method | Path | Description | Auth |
|--------|------|-------------|------|
| GET | `/v3/projects/{pid}/marker-definitions` | List definitions + bound `link_ids` | Project.Audit |
| POST | `/v3/projects/{pid}/marker-definitions` | Create definition (fans out to every link) | Project.Modify |
| PUT | `/v3/projects/{pid}/marker-definitions/{name}` | Update definition (syncs all copies) | Project.Modify |
| DELETE | `/v3/projects/{pid}/marker-definitions/{name}` | Delete definition (clears all copies) | Project.Modify |
| POST | `/v3/projects/{pid}/marker-definitions/{name}/pause` | Pause every inherited copy (instant, persisted) | Project.Modify |
| POST | `/v3/projects/{pid}/marker-definitions/{name}/resume` | Resume every inherited copy | Project.Modify |
### Aggregation
| Method | Path | Description | Auth |
|--------|------|-------------|------|
| GET | `/v3/projects/{pid}/markers` | All markers across links, flat | Project.Audit |
The link object returned by `GET /v3/projects/{pid}/links[/{lid}]` also carries a `markers`
field (including inherited markers), so the Web UI can render a link's markers without an
extra request.
## Request / Response
**Marker create body** (`MarkerCreate`, shared by per-link POST and PUT):
```json
{
"name": "icmp",
"bpf": "icmp",
"tag": 1,
"direction": "tx",
"capture_node_id": "a37e2235-e21f-46c9-a2ab-ba0f8c5465e6",
"color": "#ff5722",
"highlight_duration": 800,
"enabled": true
}
```
`direction` and `capture_node_id` are both optional and create-only (see
[Direction](#direction)).
**Definition create body** (`MarkerDefinitionCreate`, shared by POST and PUT):
```json
{
"name": "arp",
"bpf": "arp",
"tag": 5,
"color": "#ff5722",
"highlight_duration": 1200
}
```
**Marker entry** (returned by GET/POST/PUT, and the value of each link's `markers[name]`):
```json
{
"bpf": "icmp",
"tag": 1,
"enabled": true,
"color": "#ff5722",
"highlight_duration": 800,
"capture_node_id": "a37e2235-e21f-46c9-a2ab-ba0f8c5465e6",
"inherited_from": null
}
```
**Definition GET response** (adds `link_ids`):
```json
{
"arp": {
"bpf": "arp",
"tag": 5,
"color": null,
"highlight_duration": 1200,
"direction": null,
"paused": false,
"link_ids": ["656ed826-...", "6bd9d156-..."]
}
}
```
## Field Reference
### Marker entry
| Field | Type | Description |
|-------|------|-------------|
| `bpf` | string | libpcap BPF expression (required) |
| `tag` | int \| null | Correlation id echoed in `MARK` signals |
| `enabled` | bool | Whether the marker is active. Toggle is instant: `false` flips the uBridge filter off in place (no signal/pcap), `true` back on — no NIO rebuild (see [Pause & resume](#pause--resume)) |
| `color` | string \| null | Hex color render hint, e.g. `#ff5722` |
| `highlight_duration` | int \| null | UI highlight duration in ms after a match; `null` = UI default |
| `direction` | string \| null | `tx` / `rx` filter relative to the capture node; `null` = both |
| `capture_node_id` | string | Node whose uBridge hosts the marker — caller-set on create, else auto-picked |
| `inherited_from` | string | Source definition name — present on inherited markers only |
### Definition
| Field | Type | Description |
|-------|------|-------------|
| `bpf` | string | libpcap BPF expression (required) |
| `tag` | int \| null | Correlation id |
| `color` | string \| null | Hex color render hint |
| `highlight_duration` | int \| null | UI highlight duration in ms; `null` = UI default |
| `direction` | string \| null | `tx` / `rx` filter relative to the capture node; `null` = both |
| `paused` | bool | Per-definition mute flag — `true` mutes every inherited copy (persisted) |
| `link_ids` | string[] | Links currently carrying an inherited copy (GET only) |
### Notifications
| Event | Payload | Delivered to |
|-------|---------|--------------|
| `link.updated` | Link object (its `markers` field is the source of truth) | Project notification ws |
| `marker.match` | `project_id`, `node_id`, `link_id`, `filter`, `tag`, `ts`, `len`, `dir` | Project notification ws only |
The `marker.match` `link_id` is taken from the signal's `link=` field (authoritative); see
[Per-link attribution](#per-link-attribution). The `dir` field is the matched packet's travel
direction relative to the capture node; see [Direction](#direction).
## Error Responses
| Status | Description |
|--------|-------------|
| 401 | Not authenticated |
| 404 | Link / marker / definition not found |
| 409 | Per-link edit or delete of an inherited marker; reserved (`global`) name or duplicate name on create |
| 422 | Validation failure (name format, `highlight_duration < 1`, missing `bpf`) |
## Notes
- **Marker name is immutable.** It is the identifier across the controller, the uBridge
filter, the pcap filename, and `MARK` signal routing — so rename is a delete + recreate,
not a field update. PUT ignores the body `name`; the `{name}` path parameter identifies
the target, and only `bpf / tag / color / enabled / highlight_duration` are changeable.
Names are 1–32 chars (`[A-Za-z0-9][A-Za-z0-9_.-]*`); inherited copies carry a `global-`
prefix, so their filter names reach ~39.
- **`global` prefix reserved.** User-chosen names may not start with `global`; inherited
markers are stored as `global-{definition_name}` so the two namespaces cannot collide.
Omitting `name` on create yields an auto-generated, prefix-free name.
- **Inherited markers are read-only per-link.** PUT/DELETE on an inherited marker returns
409 — edit them through the definitions API.
- **Render hints are not enforced.** `color` and `highlight_duration` (milliseconds, `>= 1`)
are stored on the link and never sent to uBridge; `null` lets the UI apply its own
default. A partial PUT (e.g. changing only `bpf`) leaves them untouched.
- **BPF is validated once per source.** A private per-link marker validates its BPF inline
on create/update. A definition validates its BPF once at create/update (and once per
definition on project load, dropping any whose BPF has gone invalid); the inherited
fan-out to every link then skips re-validation, so creating a definition over *N* links
runs one `tcpdump -d` rather than *N*. (uBridge still runs `pcap_compile` itself at
install time, so an invalid expression can never slip through.)
- **Supported node types.** A marker needs a uBridge bridge: `vpcs`, `qemu`, `docker`,
`iou`, `dynamips`, `cloud` (one capable endpoint suffices). Types without a uBridge are
silently skipped by the inheritance fan-out. IOU uses one shared `IOL-BRIDGE` per node but
keeps filters, pcap files, and `link=` ids per port, so multi-interface nodes are handled
(see [Per-link attribution](#per-link-attribution)).
- **Shared capture-side node.** When one node hosts markers for several links (typical for
`global-*` definitions on a router), each filter is stamped with its `link_id` so signals
and pcap files stay link-distinct; the controller never collapses them to a single link.
- **Persistence.** Definitions and private markers persist in the topology; inherited
markers are re-created from definitions on project load, so reopening a project restores
the same configuration and stale inherited copies cannot survive on disk.
- **Log interpretation across node types.** Each node type logs its startup and link
operations differently — do not mistake sparse logs from one type for inactivity.
QEMU prints `set_link gns3-<N> on` via its QEMU monitor, which is the most visible
startup log among all types. VPCS, Docker, IOU, Dynamips, and Cloud each have their own
startup paths (fork + ubridge, container veth, iouyap, Dynamips hypervisor, and TAP
device respectively) and none of them emit QEMU-monitor-style logs. To verify marker
operations (toggle, pause, resume) on non-QEMU types, either inspect uBridge's
own log for `enable_packet_filter` / `marker pause` / `marker resume` commands, or
watch the gns3server log for the corresponding compute-route calls at INFO level.
+480
View File
@@ -0,0 +1,480 @@
# MCP (Model Context Protocol) Service
## Overview
GNS3 Server provides a standard [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) interface, allowing AI assistants like Claude to interact with GNS3 network simulations through SSE (Server-Sent Events) transport.
The MCP service exposes GNS3 project management operations as MCP tools that can be discovered and called by MCP clients.
## Endpoints
| Path | Method | Description |
|------|--------|-------------|
| `/v3/mcp/` | GET | MCP service metadata |
| `/v3/mcp/transport/sse` | GET | SSE stream (MCP connection) |
| `/v3/mcp/transport/messages/` | POST | JSON-RPC messages |
## Authentication
The SSE endpoint supports two types of credentials, passed the same way.
1. **Authorization header** (recommended):
```
Authorization: Bearer <jwt_or_api_key>
```
2. **Query parameter** (for clients that don't support custom headers):
```
GET /v3/mcp/transport/sse?token=<jwt_or_api_key>
```
### Option 1: JWT Token (24h expiry)
```bash
curl -X POST http://localhost:3080/v3/access/users/authenticate \
-H "Content-Type: application/json" \
-d '{"username": "admin", "password": "admin"}'
```
Default lifetime is **1440 minutes (24 hours)**. Configurable in `gns3_server.conf`:
```ini
jwt_access_token_expire_minutes = 1440 ; 24 hours
```
### Option 2: API Key (permanent, revocable) — Recommended for MCP
API keys never expire and can be revoked individually. Format: `gns3_<api_key_id>_<random_secret>` — the embedded UUID enables O(1) lookup without scanning all keys.
Create one via the REST API:
```bash
# Create an API key (requires a JWT to authenticate)
curl -X POST http://localhost:3080/v3/access/api-keys \
-H "Authorization: Bearer <your_jwt>" \
-H "Content-Type: application/json" \
-d '{"name": "MCP Production"}'
# Response: {"api_key": "gns3_550e8400-e29b-41d4-a716-446655440000_a1b2c3d4...", ...}
# ⚠️ The key is only shown once — save it immediately.
```
API key management endpoints:
| Endpoint | Description |
|----------|-------------|
| `POST /v3/access/api-keys` | Create a new key (returns plaintext once) |
| `GET /v3/access/api-keys` | List all your keys |
| `POST /v3/access/api-keys/{id}/revoke` | Revoke a key (can be restored) |
| `POST /v3/access/api-keys/{id}/restore` | Restore a revoked key |
| `DELETE /v3/access/api-keys/{id}` | Permanently delete a key |
Both JWT tokens and API keys work for MCP and REST API endpoints interchangeably.
### Authentication Flow
When connecting with an API key:
```
SSE connect → Authorization: Bearer gns3_<uuid>_<secret>
↓
MCP auth wrapper extracts UUID → single DB query → 1 bcrypt (thread pool)
↓
Generates a fresh short-lived JWT → stored in ContextVar for the session
↓
All subsequent tool handler REST API calls use this JWT → zero extra bcrypt
```
### Concurrency
| Setting | Value |
|---------|-------|
| MCP batch workers | 100 (`BATCH_MAX_WORKERS`) |
| MCP HTTP client timeout | 30s |
| HTTP connection pool (`pool_connections`/`pool_maxsize`) | 500 / 1000 |
| REST API node/link creation pool | 100 (`Pool(concurrency=100)`) |
## Available Tools
**82 tools** across 12 categories:
### Project (15)
| Tool | Description |
|------|-------------|
| `project_list` | List all projects |
| `project_get` | Get project details |
| `project_create` | Create a project |
| `project_delete` | Delete a project |
| `project_open` | Open a closed project |
| `project_close` | Close an open project |
| `project_stats` | Get project statistics |
| `project_update` | Update project properties |
| `project_duplicate` | Duplicate a project |
| `project_readme_get` | Get project README content |
| `project_readme_update` | Update project README |
| `project_lock` | Lock project (prevent edits) |
| `project_unlock` | Unlock project |
| `project_load` | Load project from path |
| `project_locked` | Check if project is locked |
### Node (22)
| Tool | Description |
|------|-------------|
| `node_list` | List all nodes (`fields` to filter columns, e.g. `["name","status"]`) |
| `node_get` | Get node details (`fields` to filter columns) |
| `node_create` | Create node(s) — single via `template_id` or batch via `nodes` array. Supports `fields` to filter response. Top-level `template_id` applies as default in batch mode. Pass `name` to override template naming. Coordinates: left-handed Cartesian (origin at canvas center, X right-positive, Y down-positive). |
| `node_delete` | Delete a node |
| `node_update` | Update node properties |
| `node_start` | Start node(s) — `node_id` or `node_ids` array |
| `node_stop` | Stop node(s) — `node_id` or `node_ids` array |
| `node_suspend` | Suspend node(s) — `node_id` or `node_ids` array |
| `node_console` | Get WebSocket console URL |
| `node_file_list` | List files in node directory |
| `node_file_get` | Read a file (with offset/limit) |
| `node_file_write` | Write a file |
| `node_file_delete` | Delete a file |
| `node_start_all` | Start all nodes |
| `node_stop_all` | Stop all nodes |
| `node_suspend_all` | Suspend all nodes |
| `node_duplicate` | Duplicate a node |
| `node_isolate` | Isolate a node (suspend links) |
| `node_unisolate` | Un-isolate a node (resume links) |
| `node_links` | List links connected to a node |
### Link (9)
| Tool | Description |
|------|-------------|
| `link_list` | List all links (`fields` to filter columns) |
| `link_get` | Get link details |
| `link_create` | Create link(s) — single via `nodes` or batch via `links` array. Nodes support compact `[id, ad, pt, id, ad, pt]` format. Supports `fields` to filter response. |
| `link_delete` | Delete link(s) — `link_id` or `link_ids` array |
| `link_update` | Update link (suspend, filters) |
| `link_reset` | Reset link(s) — `link_id` or `link_ids` array |
| `link_capture_start` | Start capture(s) — `link_id` or `link_ids` array |
| `link_capture_stop` | Stop capture(s) — `link_id` or `link_ids` array |
| `link_capture_download` | Get PCAP download URL(s) — `link_id` or `link_ids` array |
### Template (5)
| Tool | Description |
|------|-------------|
| `template_list` | List all templates. Supports `fields` to filter response columns. |
| `template_get` | Get template details |
| `template_create` | Create a template (Docker needs `image`) |
| `template_update` | Update a template |
| `template_delete` | Delete a template |
### Compute (3)
| Tool | Description |
|------|-------------|
| `compute_list` | List registered remote computes |
| `compute_get` | Get compute details (requires UUID) |
| `compute_images` | List emulator images on a compute |
### Snapshot (4)
| Tool | Description |
|------|-------------|
| `snapshot_list` | List snapshots |
| `snapshot_create` | Create a snapshot |
| `snapshot_delete` | Delete a snapshot |
| `snapshot_restore` | Restore a snapshot |
### Drawing (5)
| Tool | Description |
|------|-------------|
| `drawing_list` | List drawings on canvas |
| `drawing_get` | Get drawing details |
| `drawing_create` | Create drawing (SVG label/shape/image) |
| `drawing_update` | Update drawing (position, rotation, SVG) |
| `drawing_delete` | Delete a drawing |
### Symbol (6)
| Tool | Description |
|------|-------------|
| `symbol_list` | List all symbols |
| `symbol_get` | Get symbol download URL |
| `symbol_dimensions` | Get symbol dimensions |
| `symbol_defaults` | Get default symbol mapping |
| `symbol_upload` | Upload a custom symbol (SVG content) |
| `symbol_delete` | Delete a custom symbol (built-in: 403) |
### Appliance (3)
| Tool | Description |
|------|-------------|
| `appliance_list` | List appliances (`fields` to filter, e.g. `["name","category"]`) |
| `appliance_get` | Get appliance details |
| `appliance_install` | Create template from appliance (images must exist locally) |
### Image (5)
| Tool | Description |
|------|-------------|
| `image_list` | List all images |
| `image_get` | Get image details |
| `image_delete` | Delete an image |
| `image_prune` | Remove images not referenced by any template |
| `image_install` | Auto-create templates from uploaded images by checksum |
### Server (2)
| Tool | Description |
|------|-------------|
| `server_version` | Get GNS3 server version |
| `server_statistics` | Get server statistics (computes, projects, nodes) |
### Device Config (3)
| Tool | Description |
|------|-------------|
| `device_config_send` | Push config commands to devices via console (Nornir + Netmiko). Supports Jinja2 `template` + `vars` |
| `device_show_run` | Run read-only show commands on devices. Supports Jinja2 `template` + `vars` |
| `vpcs_config_set` | Configure VPCS devices (IP, gateway, etc.) |
The tool connects to each device's console via telnet/SSH. Nodes must be in the `started` state (use `node_start` or `node_start_all`). Device type is auto-detected from the node's `device_type:<type>` tag in GNS3.
#### Jinja2 Template Mode
Both `device_config_send` and `device_show_run` support an optional `template` parameter. When provided, each device's `vars` dict is rendered against the template to produce commands. Entries with the same `device_name` are merged into a single device session.
```python
# Direct commands (single/batch)
device_config_send(project_id, device_configs=[
{"device_name": "R1", "config_commands": ["int lo0", "ip add 1.1.1.1 255.255.255.255"]},
])
# Jinja2 template (reduces token usage for batch)
device_config_send(project_id,
template="interface lo{{ n }}\nip address {{ ip }} 255.255.255.255",
device_configs=[
{"device_name": "R1", "vars": {"n": 0, "ip": "1.1.1.1"}},
{"device_name": "R2", "vars": {"n": 0, "ip": "2.2.2.2"}},
])
# Show commands with template
device_show_run(project_id,
template="show ip route {{ protocol }}",
device_configs=[
{"device_name": "R1", "vars": {"protocol": "ospf"}},
{"device_name": "R2", "vars": {"protocol": "bgp"}},
])
```
### Best Practices
**Prefer template over direct commands for batch.** When ≥2 nodes share the same config structure with different values, use `template`+`vars` instead of writing `config_commands` per node. This reduces token usage and transcription errors.
**Batch merging.** Multiple entries with the same `device_name` are merged into a single Nornir session. The output contains all commands' results in one block. Match results by `device_name`, not list index.
**Don't rely on `status: success` alone.** It only means commands entered config mode. IOS errors (`% Invalid input`, `% overlaps`, `% Incomplete command`) appear inside `output` text — always scan for `%` lines.
**Pilot before full rollout.** Test template + vars on 1–2 devices first to verify rendering and syntax, then expand to all nodes.
**Config backup via file operations.** IOU and Dynamips nodes save startup config as a plain text file (`startup-config.cfg`) in the node directory after `write memory`. These can be backed up and restored via `node_file_get`/`node_file_write`.
```python
# Save config on device
device_show_run(project_id, device_configs=[
{"device_name": "R1", "commands": ["write memory"]},
])
# Backup
config = node_file_get(project_id, node_id, "startup-config.cfg")
# Restore if config breaks
node_file_write(project_id, node_id, "startup-config.cfg", config)
node_stop(project_id, node_id)
node_start(project_id, node_id)
```
### Device Config Workflow
```mermaid
sequenceDiagram
participant AI as AI Agent
participant MCP as MCP Handler
participant TM as Template Renderer
participant DP as Device Discovery
participant NR as Nornir
participant NM as Netmiko
participant D as Device Console
Note over AI: Decide: template or direct commands?
alt Direct commands
AI->>MCP: device_config_send(config_commands=[...])
else Jinja2 template
AI->>MCP: device_config_send(template + vars)
MCP->>TM: Render template per device
TM->>TM: Jinja2.render(**vars)
TM-->>MCP: device_configs with rendered commands
end
MCP->>DP: get_device_ports_from_topology()
DP-->>MCP: hosts_data (console port, device_type)
Note over MCP: Prepare Nornir inventory
MCP->>NR: InitNornir(hosts, threaded runner)
par Device 1 to N (parallel, max 10)
NR->>NM: netmiko_send_config(commands)
NM->>D: telnet/SSH console session
D-->>NM: command output
NM-->>NR: execution result
end
NR-->>MCP: aggregated results
MCP-->>AI: per-device results with output
```
## Configuration
### Claude Code (CLI)
```bash
# Option A: Using API key (recommended — never expires)
claude mcp add --transport sse My_GNS3_Server \
http://localhost:3080/v3/mcp/transport/sse \
-H "Authorization: Bearer gns3_a1b2c3d4..."
# Option B: Using JWT token (expires after 24h)
TOKEN=$(curl -s -X POST http://localhost:3080/v3/access/users/authenticate \
-H "Content-Type: application/json" \
-d '{"username": "admin", "password": "admin"}' | python3 -c \
"import sys,json; print(json.load(sys.stdin)['access_token'])")
claude mcp add --transport sse My_GNS3_Server \
http://localhost:3080/v3/mcp/transport/sse \
-H "Authorization: Bearer $TOKEN"
```
## Transport Security
MCP server uses FastMCP's DNS rebinding protection to prevent attackers from
exploiting DNS resolution to access the MCP endpoint through unauthorized domains.
### Default Behaviour
DNS rebinding protection is **disabled by default**, allowing connections from
any host. This aligns with GNS3 server's default `host = 0.0.0.0` binding policy,
which is designed for VM distribution scenarios where users access the server
from various network locations.
### Enabling Protection
Add to `gns3_server.conf` under the `[Server]` section:
```ini
; Enable DNS rebinding protection for MCP server
mcp_enable_dns_rebinding_protection = True
; Allowed hosts (comma-separated, "host:*" port wildcard patterns only)
mcp_allowed_hosts = 127.0.0.1:*,localhost:*,192.168.1.3:*
; Allowed origins (comma-separated)
mcp_allowed_origins = http://127.0.0.1:*,http://localhost:*,http://192.168.1.3:*
```
> **Note**: The MCP library only supports `"host:*"` port wildcard patterns
> (e.g., `"192.168.1.3:*"`). Standalone `"*"` wildcards are not supported.
### Protection Mechanism
When protection is enabled, the MCP server validates the `Host` header of
incoming SSE connection requests:
```python
# Verify the request's Host header matches allowed patterns
validate_request → check Host header → 421 Misdirected Request if invalid
```
This prevents DNS rebinding attacks:
1. Attacker registers `evil.com` pointing to your server's IP
2. User's browser makes requests to `evil.com:3080`
3. MCP server checks Host header = `"evil.com:3080"`
4. `"evil.com:3080"` is not in `allowed_hosts` → connection rejected
### Behaviour Summary
| `mcp_enable_dns_rebinding_protection` | Result |
|:---|:---|
| `False` (default) | All hosts allowed |
| `True` + correct hosts configured | Only configured hosts allowed |
| `True` + missing/wrong hosts | Connections rejected with 421 |
For public-facing MCP servers, set `allowed_hosts` to your server's domain name.
## Architecture
```mermaid
sequenceDiagram
participant Client as Claude Code
participant MCP as MCP Service
participant Auth as Auth
participant GNS3 as GNS3 REST API
Note over Client: 1. Connect with API Key or JWT
Client->>MCP: GET /sse (Authorization: Bearer <key>)
alt API Key (gns3_&lt;uuid&gt;_&lt;secret&gt;)
MCP->>Auth: Extract UUID → DB lookup → 1 bcrypt (thread pool)
Auth-->>MCP: Generate fresh JWT
else JWT
MCP->>Auth: Decode JWT
Auth-->>MCP: Token valid
end
MCP-->>Client: event: endpoint /messages/?session_id=xxx
Note over Client: 2. Initialize & Call Tools
Client->>MCP: POST /messages/ (tools/call ...)
MCP->>GNS3: HTTP request (with JWT from step 1)
GNS3-->>MCP: Response
MCP-->>Client: event: message (tool result)
```
## Internal Implementation
- **FastMCP** (Anthropic MCP SDK) is used for tool registration and SSE transport
- The SSE app is mounted as a Starlette sub-application under `/v3/mcp/transport`
- **Auth:** JWT validation via `auth_service`. API key (`gns3_<uuid>_<secret>`) extracts UUID for O(1) DB lookup, runs bcrypt in thread pool, returns a fresh JWT — subsequent calls use the JWT with zero extra bcrypt.
- Tool handlers use `Gns3Connector` (from `custom_gns3fy`) to call GNS3's own REST API, keeping the MCP layer decoupled
- The JWT token is stored in a `contextvars.ContextVar` so it is available within tool handler threads (Python ≥ 3.9 propagates contextvars through `asyncio.to_thread`)
### Console WebSocket
The `node_console` tool returns a WebSocket URL for connecting to a node's console. The URL includes a short-lived JWT (10 min) — reconnect if it expires. This endpoint is protocol-agnostic — it works for **telnet**, **ssh**, and **vnc** console types alike. The WebSocket simply proxies raw byte streams between the client and the compute node; protocol negotiation (e.g. SSH key exchange) happens on the compute side.
The WebSocket URL is constructed using the server's `_server_url()`, which resolves the host as follows:
| `Server.host` value | Resolved host in URL |
|:---|:---|
| Specific IP or hostname (e.g. `192.168.1.3`) | Used as-is |
| `0.0.0.0` (IPv4 any, default) | Detected via **default route interface IP** |
| `::` (IPv6 any) | Detected via default route interface IP |
| Detection failure | Fallback to `127.0.0.1` |
When `Server.host` is `0.0.0.0` (listen on all interfaces), the MCP server discovers the default route interface IP using a UDP socket connect to `8.8.8.8:80` — no network data is sent, the operating system simply selects the interface that would be used for the default route. This ensures the returned WebSocket URL uses a reachable address (e.g. `192.168.1.3` instead of `127.0.0.1`).
If the configured host is already a specific IP or hostname (not `0.0.0.0`), it is used directly in the URL without modification.
Use `websocat` to connect from the command line:
```bash
# The host in the URL is automatically resolved to a reachable address
websocat ws://192.168.1.3:3080/v3/projects/{project_id}/nodes/{node_id}/console/ws?token=<jwt>
```
### Source Files
| File | Purpose |
|------|---------|
| `gns3server/api/routes/mcp/__init__.py` | FastMCP server, tool decorators, SSE transport, JWT auth wrapper |
| `gns3server/api/routes/mcp/projects.py` | Project tool handlers |
| `gns3server/api/routes/mcp/nodes.py` | Node tool handlers |
| `gns3server/api/routes/mcp/links.py` | Link tool handlers |
| `gns3server/api/routes/mcp/templates.py` | Template tool handlers |
| `gns3server/api/routes/mcp/computes.py` | Compute tool handlers |
| `gns3server/api/server.py` | Mounts MCP routes via `register_starlette_routes()` |
+148
View File
@@ -0,0 +1,148 @@
# Project Open Performance
## Overview
Optimizations to accelerate project opening (`POST /projects/{id}/open`) and node creation for topologies with many nodes and links. The main bottlenecks were sequential link creation, redundant subprocess calls, and SQLite write contention.
## Before vs After
| Scenario | Before | After |
|----------|--------|-------|
| 20 IOU nodes + 20 links (project open) | ~2s | ~1s |
| 40 QEMU nodes creation (MCP batch) | ~40s | ~1-2s |
## Optimizations
### 1. Parallel Link Creation
**File:** `gns3server/controller/project.py`
Links were created sequentially during project loading, each requiring up to 5 HTTP round-trips to the compute. Now uses `Pool(concurrency=100)` for parallel creation.
```python
# Before: sequential loop
for link_data in topology.get("links", []):
link = await self.add_link(...)
await link.add_node(...)
# After: parallel Pool
pool = Pool(concurrency=100)
for link_data in topology.get("links", []):
pool.append(self._create_link_from_topology_data, link_data)
await pool.join()
```
### 2. Batch UDP Port Allocation
**Files:** `gns3server/api/routes/compute/compute.py`, `gns3server/controller/project.py`, `gns3server/controller/udp_link.py`
During project loading, all required UDP ports are pre-allocated per compute in a single batch call before link creation begins. `UDPLink.create()` checks the pre-allocated pool first, falling back to individual allocation if unavailable.
```python
# New batch endpoint
POST /projects/{id}/ports/udp/batch → {"count": N} → {"udp_ports": [...]}
```
### 3. IOU Image Subprocess Cache
**File:** `gns3server/compute/iou/iou_vm.py`
Each IOU VM creation spawned `ld-linux --verify` and `iou-image -h` subprocesses. With 20 nodes using the same image, this ran 40 redundant subprocesses. Now results are cached per image path at the class level.
```python
# Class-level caches shared across all instances
IOUVM._loader_cache = {} # image path → loader command
IOUVM._default_values_cache = {} # image path → (ram, nvram)
```
Only the first node with a given image runs the subprocesses; subsequent nodes reuse cached values.
### 4. SQLite WAL Mode
**File:** `gns3server/db/tasks.py`
Write-Ahead Logging allows concurrent reads without blocking on writes. The PRAGMA is registered on `engine.sync_engine` instead of the `Engine` class to correctly fire for async engine connections.
```python
@event.listens_for(engine.sync_engine, "connect")
def set_sqlite_pragma(dbapi_connection, connection_record):
cursor = dbapi_connection.cursor()
cursor.execute("PRAGMA journal_mode=WAL")
cursor.execute("PRAGMA foreign_keys=ON")
cursor.close()
```
Without WAL mode, concurrent API requests caused `sqlite3.OperationalError: database is locked`.
### 5. API Key Authentication O(1) Lookup
**Files:** `gns3server/api/routes/controller/api_keys.py`, `gns3server/api/routes/controller/dependencies/authentication.py`
**Old format:** `gns3_<random>` — required scanning ALL keys and running bcrypt on each (O(n)).
**New format:** `gns3_<api_key_id>_<random_secret>` — extract UUID from token, single DB query (O(1)), single bcrypt.
```python
# New auth flow
parts = token.split("_", 2)
key_id = UUID(parts[1])
secret = parts[2]
db_key = await api_keys_repo.get_api_key(key_id) # O(1) lookup
if await asyncio.to_thread(bcrypt.checkpw, secret.encode(), db_key.key_hash.encode()):
# Authenticated — 1 query + 1 bcrypt regardless of total key count
```
### 6. bcrypt in Thread Pool
**File:** `gns3server/api/routes/controller/dependencies/authentication.py`
`bcrypt.checkpw()` is CPU-bound (~1.3s per call) and was blocking the async event loop. With 5 API keys and 10 concurrent requests, this caused ~13s delay before any handler could start.
```python
# Before: blocking the event loop
if bcrypt.checkpw(token.encode(), db_key.key_hash.encode()):
...
# After: offloaded to thread pool
if await asyncio.to_thread(bcrypt.checkpw, secret.encode(), db_key.key_hash.encode()):
...
```
### 7. Concurrency Settings
| Setting | Before | After | File |
|---------|--------|-------|------|
| Node creation Pool | 5 | 100 | `controller/project.py` |
| Link creation Pool | 5 | 100 | `controller/project.py` |
| MCP BATCH_MAX_WORKERS | 10 | 100 | `api/routes/mcp/nodes.py` |
| MCP HTTP timeout | 10s | 30s | `agent/gns3_copilot/gns3_client/custom_gns3fy.py` |
| HTTP connection pool | 10 (default) | 500/1000 | `agent/gns3_copilot/gns3_client/custom_gns3fy.py` |
| Start nodes Pool | 3 | 3 (unchanged) | `controller/project.py` |
### 8. MCP Auth Returns JWT
**File:** `gns3server/api/routes/mcp/__init__.py`
When an MCP client connects with an API key, the `_resolve_token` function validates the key then returns a fresh short-lived JWT instead of the raw API key. The JWT is stored in a `ContextVar` and reused for all subsequent tool calls within the same SSE session — zero extra bcrypt.
```python
if user:
fresh_token = auth_service.create_access_token(user.username)
return fresh_token # JWT for subsequent REST API calls
```
## Related Files
| File | Changes |
|------|---------|
| `gns3server/controller/project.py` | Parallel link creation, batch UDP, Pool(100) |
| `gns3server/compute/iou/iou_vm.py` | Image subprocess cache |
| `gns3server/db/tasks.py` | WAL mode + sync_engine event listener |
| `gns3server/api/routes/compute/compute.py` | Batch UDP endpoint |
| `gns3server/controller/udp_link.py` | Pre-allocated port consumption |
| `gns3server/api/routes/controller/api_keys.py` | O(1) key format |
| `gns3server/api/routes/controller/dependencies/authentication.py` | O(1) auth + thread pool bcrypt |
| `gns3server/api/routes/mcp/__init__.py` | Auth returns JWT, tool enhancements |
| `gns3server/api/routes/mcp/nodes.py` | fields filter, inherited template_id, name passthrough |
| `gns3server/api/routes/mcp/links.py` | fields filter, compact array format |
| `gns3server/agent/gns3_copilot/gns3_client/custom_gns3fy.py` | Timeout 30s, connection pool 500/1000 |
| `gns3server/utils/images.py` | md5sum cache error → warning |
+154
View File
@@ -0,0 +1,154 @@
# Stateless JWT Refresh Token Mechanism
## Overview
GNS3 server now supports a stateless JWT refresh token mechanism for interactive sessions (e.g., Web UI). This allows clients to stay authenticated across page reloads without repeated username/password prompts, while keeping access tokens short-lived.
No new database table or migration is required — refresh tokens are signed JWTs using the same secret and algorithm as access tokens.
## Architecture
```mermaid
graph TD
Client -->|login / authenticate| API[Controller API]
API -->|access_token + refresh_token| Client
Client -->|POST /refresh| Refresh[Refresh Endpoint]
Refresh -->|new access_token + new refresh_token| Client
Client -->|Bearer access_token| Protected[Protected Endpoints]
Protected -->|401| Client
Client -->|refresh_token in body| Refresh
Refresh -->|401 if invalid/expired/revoked| Client
Refresh -->|verify type, exp, ver| AuthService[AuthService]
AuthService -->|check token_version| DB[(users table)]
```
## Business Process
### Login / Authenticate Flow
```mermaid
sequenceDiagram
participant C as Client
participant API as Controller API
participant AS as AuthService
participant DB as Database
C->>API: POST /login or /authenticate (username + password)
API->>DB: authenticate_user()
DB-->>API: user (with token_version)
API->>AS: create_access_token(user, ver)
API->>AS: create_refresh_token(user, ver)
AS-->>API: access_token (type: access, exp: 15min)
AS-->>API: refresh_token (type: refresh, exp: 30d)
API-->>C: { access_token, token_type, refresh_token }
```
### Refresh Flow (Silent Renewal)
```mermaid
sequenceDiagram
participant C as Client
participant API as Controller API
participant AS as AuthService
participant DB as Database
Note over C: access_token expired
C->>API: POST /refresh { refresh_token }
API->>AS: get_token_data(refresh_token)
AS-->>API: { username, ver, token_use: "refresh" }
API->>DB: get_user_by_username()
DB-->>API: user (with current token_version)
Note over API,DB: rejects if user not found, inactive, or token_version mismatch
API->>AS: create_access_token(user, ver)
API->>AS: create_refresh_token(user, ver)
AS-->>API: new access_token (sliding window)
AS-->>API: new refresh_token (sliding window)
API-->>C: { access_token, token_type, refresh_token }
C->>API: Retry original request with new access_token
```
### Logout — Token Revocation
```mermaid
sequenceDiagram
participant C as Client
participant API as Controller API
participant DB as Database
C->>API: POST /logout (Bearer access_token)
API->>DB: logout_user(user_id) → token_version += 1
DB-->>API: done
API-->>C: 204 No Content
Note over C, DB: All existing access and refresh tokens with old ver are now invalid
```
## API Endpoints
| Method | Path | Description | Authentication |
|--------|------|-------------|---------------|
| POST | `/v3/access/users/login` | Login with form data, returns access + refresh tokens | Public |
| POST | `/v3/access/users/authenticate` | Login with JSON, returns access + refresh tokens | Public |
| POST | `/v3/access/users/refresh` | Exchange a refresh token for a new access token + refresh token | Public (token itself proves identity) |
| POST | `/v3/access/users/logout` | Revoke all tokens for the current user | Bearer token required |
### POST /v3/access/users/refresh
**Request:**
```json
{
"refresh_token": "<refresh_token>"
}
```
**Response 200:**
```json
{
"access_token": "<new_access_token>",
"token_type": "bearer",
"refresh_token": "<new_refresh_token>"
}
```
**Error Responses:**
- `401` — Invalid, expired, or revoked refresh token
- `422` — Missing `refresh_token` field in request body
## Security Design
### Token Claims
| Claim | Access Token | Refresh Token |
|-------|-------------|---------------|
| `sub` | username | username |
| `exp` | 24h (configurable) | 30d (configurable) |
| `ver` | user's `token_version` | user's `token_version` |
| `type` | `"access"` | `"refresh"` |
### Key Security Properties
- **Type-based isolation**: Access tokens (`type: access`) are rejected by `/refresh`. Refresh tokens (`type: refresh`) are rejected by HTTP and WebSocket authentication paths. This prevents a stolen long-lived refresh token from being used directly for API access.
- **Token version integration**: Both token types carry the user's `token_version`. `logout` increments `token_version` in the database, immediately invalidating all outstanding access and refresh tokens.
- **Stateless (no replay detection)**: Since there is no `refresh_tokens` database table, a stolen refresh token remains valid until its `exp` or until the user logs out. This is an accepted trade-off for avoiding a new table and migration.
- **Sliding window**: Each `/refresh` call issues a new refresh token with a fresh expiry, keeping active sessions alive indefinitely until logout or inactivity.
### Implementation Files
- `gns3server/services/authentication.py` — `_create_token`, `create_access_token`, `create_refresh_token`, `get_token_data`
- `gns3server/api/routes/controller/users.py` — `refresh_access_token` endpoint handler
- `gns3server/api/routes/controller/dependencies/authentication.py` — `_reject_refresh_token` guard in HTTP and WebSocket paths
- `gns3server/schemas/controller/tokens.py` — `Token`, `TokenData`, `RefreshTokenRequest` models
- `gns3server/schemas/config.py` — `jwt_refresh_token_expire_minutes` configuration
## Configuration
| Setting | Default | Description |
|---------|---------|-------------|
| `Controller.jwt_access_token_expire_minutes` | 1440 (24h) | Access token TTL. Web UI recommends 15 min. |
| `Controller.jwt_refresh_token_expire_minutes` | 43200 (30d) | Refresh token TTL. |
| `Controller.jwt_secret_key` | (random) | HMAC signing key for all JWT tokens. |
## Notes
- **Web UI integration**: The client should implement a response interceptor that catches 401, silently calls `/refresh`, and retries the original request. Multiple concurrent 401s should be queued with a single refresh request.
- **No per-session revocation**: All tokens for a user share the same `token_version`. Logout revokes everything. Per-session granularity would require adding a `refresh_tokens` table.
- **Rate limiting**: `/refresh` is a public endpoint with a valid credential (the refresh token). Rate limiting is recommended if brute-force attacks are a concern.
+236
View File
@@ -0,0 +1,236 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# Statistics API
## Overview
Aggregated server statistics API for monitoring dashboards. Collects compute resources, project/node/link counts, and Web Wireshark container status in a single request.
## Architecture
```mermaid
graph TB
Client["Client (Dashboard)"]
subgraph Controller["GET /v3/statistics"]
direction TB
F1["1. Compute Resources<br/>GET compute/statistics"]
F2["2. Project/Node/Link<br/>Iterate in memory"]
F3["3. Web Wireshark<br/>Docker API + docker stats"]
end
Compute["Compute Nodes<br/>(psutil)"]
Memory["Controller<br/>(in-memory data)"]
Docker["Docker Daemon"]
Client -->|GET /v3/statistics| Controller
F1 -->|HTTP| Compute
F2 -->|read| Memory
F3 -->|Docker API| Docker
```
## Data Collection Flow
```mermaid
sequenceDiagram
participant Client
participant Controller as Controller API
participant Compute as Compute Nodes
participant Docker as Docker Daemon
Client->>Controller: GET /v3/statistics
rect rgb(240, 248, 255)
Note over Controller: 1. Compute Resources
loop For each compute
Controller->>Compute: GET /statistics
Compute-->>Controller: psutil data
Note over Controller: skip on error
end
end
rect rgb(240, 255, 240)
Note over Controller: 2. Projects / Nodes / Links
Note over Controller: Count projects by status (opened/closed)
Note over Controller: Count nodes: open→by_type+by_status, closed→by_type only
Note over Controller: Count links: total + capturing
end
rect rgb(255, 248, 240)
Note over Controller: 3. Web Wireshark
loop For opened projects only
Controller->>Docker: Query container status
Controller->>Docker: docker stats (2s timeout)
Docker-->>Controller: Container resource usage
end
end
Controller-->>Client: JSON response
```
## API Endpoints
| Method | Path | Description | Auth |
|--------|------|-------------|------|
| GET | `/v3/statistics` | Aggregated server statistics | Session |
| GET | `/v3/compute/statistics` | Single compute resource stats | Basic Auth |
## Response
```json
{
"computes": [
{
"compute_id": "local",
"compute_name": "Local",
"statistics": {
"memory_total": 16777216000,
"memory_free": 8000000000,
"memory_used": 8777216000,
"swap_total": 2147479552,
"swap_free": 1500000000,
"swap_used": 647279552,
"cpu_usage_percent": 45,
"memory_usage_percent": 52,
"swap_usage_percent": 30,
"disk_usage_percent": 67,
"load_average_percent": [12, 8, 5]
}
}
],
"projects": {
"total": 5,
"opened": 3,
"closed": 2
},
"nodes": {
"total": 42,
"open_project_nodes": 30,
"closed_project_nodes": 12,
"by_type": { "qemu": 20, "docker": 12, "dynamips": 6, "vpcs": 4 },
"by_status": { "started": 25, "stopped": 12, "suspended": 5 }
},
"links": {
"total": 38,
"capturing": 2
},
"webwireshark": {
"total_containers": 1,
"running_containers": 1,
"active_sessions": 2,
"containers": [
{
"project_id": "e16e2b51-9ba9-403b-9df4-b2915d7508a3",
"project_name": "test-project",
"container_id": "6edc9029bac0",
"status": "running",
"running": true,
"active_sessions": 2,
"memory_limit": "2.0 GB",
"cpu_limit": "1.0",
"pids_limit": 1000,
"memory": "272.5MiB / 4GiB",
"cpu": "0.23%",
"pids": 69
}
]
}
}
```
## Field Reference
### `computes[]`
| Field | Type | Description |
|-------|------|-------------|
| `compute_id` | string | Compute identifier |
| `compute_name` | string | Display name |
| `statistics` | object | Resource usage (see below) |
**`statistics` fields:**
| Field | Type | Description |
|-------|------|-------------|
| `memory_total` | int | Total RAM (bytes) |
| `memory_free` | int | Available RAM (bytes) |
| `memory_used` | int | Used RAM (bytes) |
| `swap_total` | int | Total swap (bytes) |
| `swap_free` | int | Free swap (bytes) |
| `swap_used` | int | Used swap (bytes) |
| `cpu_usage_percent` | int | CPU usage 0-100 |
| `memory_usage_percent` | int | RAM usage 0-100 |
| `swap_usage_percent` | int | Swap usage 0-100 |
| `disk_usage_percent` | int | Project dir disk usage 0-100 |
| `load_average_percent` | int[] | Load avg per core (1/5/15 min) |
### `projects`
| Field | Type | Description |
|-------|------|-------------|
| `total` | int | All projects |
| `opened` | int | Currently opened |
| `closed` | int | Currently closed |
### `nodes`
| Field | Type | Description |
|-------|------|-------------|
| `total` | int | All nodes across projects |
| `open_project_nodes` | int | Nodes in opened projects (has runtime status) |
| `closed_project_nodes` | int | Nodes in closed projects (from topology JSON, no status) |
| `by_type` | object | Count by node type (qemu, docker, etc.) |
| `by_status` | object | Count by status — **open project nodes only** |
### `links`
| Field | Type | Description |
|-------|------|-------------|
| `total` | int | All links across projects |
| `capturing` | int | Links currently capturing |
### `webwireshark`
| Field | Type | Description |
|-------|------|-------------|
| `total_containers` | int | All Wireshark containers |
| `running_containers` | int | Currently running |
| `active_sessions` | int | Active capture sessions |
| `containers` | array | Per-container details (opened projects only) |
**`containers[]` fields:**
| Field | Type | Description |
|-------|------|-------------|
| `project_id` | string | Project UUID |
| `project_name` | string | Project name |
| `container_id` | string | Docker container ID (12 chars) |
| `status` | string | Container status (running, exited, etc.) |
| `running` | bool | Is running |
| `active_sessions` | int | Active captures in this project |
| `memory_limit` | string | Configured memory limit (e.g. `"2.0 GB"`, `"unlimited"`) |
| `cpu_limit` | string | Configured CPU limit (e.g. `"1.0"`, `"unlimited"`) |
| `pids_limit` | int/string | Process limit (e.g. `1000`, `"unlimited"`) |
| `memory` | string | Live memory usage (conditional, see notes) |
| `cpu` | string | Live CPU usage (conditional, see notes) |
| `pids` | int | Live process count (conditional, see notes) |
## Error Responses
| Status | Description |
|--------|-------------|
| 401 | Unauthorized — invalid or missing session |
| 500 | Internal server error |
## Notes
- **Compute stats are best-effort**: if a compute is unreachable, it is skipped (logged as error) and other data is still returned
- **`by_status` only reflects open project nodes**: closed projects store topology in JSON without runtime status
- **Live container stats** (`memory`, `cpu`, `pids`) are only present when `docker stats` succeeds — may be absent if the Docker daemon is slow (2s timeout)
- **Web Wireshark containers** are only queried for opened projects
+208
View File
@@ -0,0 +1,208 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# Cisco XRd Control Plane (Vendor NOS Adaptation)
## Overview
Cisco XRd Control Plane runs as a first-class GNS3 Docker router node by
combining the existing vendor NOS path (`console_type: "docker_exec"` +
`GNS3_SKIP_INIT=1`, see [docker-exec-console.md](./docker-exec-console.md))
with four generic server mechanisms added for heavy/systemd NOS containers:
`/dev/shm` and host-device injection, config-file injection (`extra_configs`),
and udev masking. XRd itself is pure appliance configuration — no image
rebuild, no source patching.
## Why XRd must take the vendor path
XRd boots `/usr/sbin/init` (systemd) as PID 1. GNS3's generic init.sh
wrapper chain (`/gns3/init.sh → su → run-cmd.sh → /usr/sbin/init`) crashes
XRd's glibc loader with `Fatal glibc error: dl-call-libc-early-init.c:37
(sym != NULL)` (SIGABRT loop). With `GNS3_SKIP_INIT=1` the container runs
its native entrypoint directly and boots cleanly — same arrangement as
SR Linux.
## Architecture
```mermaid
graph TB
subgraph Appliance["XRd appliance (.gns3a) — pure configuration"]
ENV["environment: GNS3_SKIP_INIT / GNS3_CONSOLE_CMD / GNS3_MASK_UDEV / GNS3_SHM_SIZE / GNS3_DEVICES + XR_*"]
XC["extra_configs: /firstboot.cfg"]
XV["extra_volumes: /xr-storage + /xr-storage-shadow"]
end
subgraph Server["gns3-server (generic mechanisms)"]
CREATE["DockerVM.create() HostConfig"]
MASK["GNS3_MASK_UDEV → /dev/null binds"]
HOSTCFG["ShmSize / Devices"]
CFGINJ["extra_configs → RO single-file bind"]
VBRIDGE["VendorDockerVM volume bridge"]
HOSTCHK["host-readiness check (read-only)"]
end
subgraph Container["XRd container"]
SYSTEMD["systemd (/usr/sbin/init)"]
XR["XR control plane"]
XRS["/xr-storage (persisted, live data layer)"]
end
ENV --> CREATE --> SYSTEMD
ENV --> MASK & HOSTCFG
XC --> CFGINJ
XV --> VBRIDGE --> XRS
HOSTCHK -.->|"warn: inotify/file-max/fuse"| Server
```
## Mechanisms added (all generic, XRd is just the first consumer)
| Mechanism | Interface | Effect | Where |
|-----------|-----------|--------|-------|
| shm size | `GNS3_SHM_SIZE=1024` (MB) in `environment` | native `HostConfig.ShmSize` at create time — works with or without init.sh | `docker_vm.py` `create()` |
| host devices | `GNS3_DEVICES=/dev/fuse` (`docker run --device` syntax, space-separated) | native `HostConfig.Devices` | `docker_vm.py` `_format_devices()` |
| config injection | `extra_configs: [{target, content}]` (template/node/appliance schema field) | content written under the node dir, bind-mounted **read-only** at `target` — seeds NOS startup configs without rebuilding the image | `docker_vm.py` `_mount_binds()`; persisted in `docker_templates.extra_configs` (Alembic migration) |
| udev masking | `GNS3_MASK_UDEV=1` | `/dev/null` over the 5 udev systemd units **and** `/bin|/sbin|/usr/bin/udevadm` | `docker_vm.py` `create()` |
| generic unit mask | `GNS3_MASK_SYSTEMD=u1,u2` | `/dev/null` over arbitrary `/etc/systemd/system/<unit>` | `docker_vm.py` `create()` |
| graceful stop | `GNS3_STOP_TIMEOUT=60` (seconds; default 60, max 210) | explicit user stop sends SIGTERM + a grace period (Docker SIGKILLs once it expires) instead of the base class' immediate kill — systemd NOS images require a graceful shutdown; internal paths (delete/update/close) keep the immediate kill since the container is force-deleted right after. Max 210 keeps the +30 s HTTP margin inside the controller's 240 s stop budget | `vendor_docker_vm.py` `_terminate_container()` |
| host check | automatic at Docker connect | read-only `/proc` check of inotify/file-max/FUSE; warns with exact fix commands (server is unprivileged — it can only check) | `compute/docker/__init__.py` `_check_host_readiness()` |
`GNS3_*` variables are consumed host-side only and never forwarded into the
container (existing GNS3 behaviour); `XR_*` variables pass through normally.
## Host-disturbance root causes (all fixed)
A privileged systemd container can disturb the *host* desktop. Three
independent causes were isolated with plain-`docker run` A/B/C experiments
(udevd coldplug / busybox chown crash / direct `udevadm trigger`):
| Host symptom | Root cause | Fix |
|---|---|---|
| Audio muted on every node start | container `systemd-udevd` coldplug replays **all** devices it can see (privileged → host `/sys`) | `GNS3_MASK_UDEV=1` (unit masks) |
| USB reconnects (mouse notification), journal noise | XRd's own `xr_startup.sh` calls `udevadm trigger --action=add --parent-match=<usb>` (USB license-dongle probing) — a direct binary call, unit masks don't stop it | `GNS3_MASK_UDEV=1` (udevadm null-bind) |
| Same USB/journal noise + broken persistence | static busybox `chown` dlopens container NSS modules → glibc abort → per-file coredump storm → host `systemd-coredump` rescans devices | vendor volume path prefers the container's own `chown` (`vendor_docker_vm.py`) |
Diagnostics: `udevadm monitor --kernel --udev` (uevent stream),
`docker exec <cid> grep -n udevadm /opt/cisco/install-iosxr/base/etc/xr_startup.sh`.
Note: "journal corrupted" messages with varying machine-IDs come from the
*container's* journald (random machine-id per start), not the host journal.
## XRd appliance recipe
| Field | Value |
|-------|-------|
| `image` | official `ios-xr/xrd-control-plane:<ver>` — no wrapper image needed |
| `console_type` | `docker_exec` |
| `extra_volumes` | `["/xr-storage", "/xr-storage-shadow"]` |
| `extra_configs` | `{target: /firstboot.cfg, content: <XR CLI first-boot config>}` |
```
GNS3_SKIP_INIT=1
GNS3_CONSOLE_CMD=/pkg/bin/xr_cli.sh
GNS3_MASK_UDEV=1
GNS3_SHM_SIZE=1024
GNS3_DEVICES=/dev/fuse
GNS3_STOP_TIMEOUT=40
XR_FIRST_BOOT_CONFIG=/firstboot.cfg
XR_MGMT_INTERFACES=linux:eth0,xr_name=Mg0/RP0/CPU0/0,chksum,snoop_v4,snoop_v6
XR_INTERFACES=linux:eth1,xr_name=Gi0/0/0/0;linux:eth2,xr_name=Gi0/0/0/1;...
```
XRd-specific gotchas (image-side, not GNS3):
- Management interface xr_name is **`Mg0/RP0/CPU0/0`** (short prefix, `CPU0`
without slash). `MgmtEth0/RP0/CPU/0` is rejected: "not a valid
rack/slot/instance/port combination".
- `XR_INTERFACES` must list exactly `adapters − 1` data interfaces (eth0 is
management). Changing the adapter count requires regenerating the string.
- **Persistence layout**: in the *image*, `/xr-storage/{config,disk1,log,
scratch}` are symlinks into `/xr-storage-shadow` (a pristine spare copy of
the initial state). At boot the bootstrap replaces the symlinks with real
directories, and XR writes everything — committed config (`commitdb`,
`running`) included — into **`/xr-storage`**, never touching the shadow
again. This mirrors containerlab, which bind-mounts `/xr-storage`
(`nodes/xrd/xrd.go`: "persist data by mounting /xr-storage"). The
appliance persists **both** paths so writes land on host regardless of
whether they happen before or after the symlink→directory transition.
- `XR_FIRST_BOOT_CONFIG` only applies when XR's config storage is empty
(first boot). To re-seed, delete and recreate the node.
- The official image ships no default login; the first-boot config must
create one (e.g. `username admin / group root-lr / secret ...`).
- Docker mounts are fixed at container *create* time: after changing a
template's `extra_volumes`, existing nodes must be deleted and recreated
(a stop/start keeps the old mounts).
- Host sysctls (XRd's own requirements, same for containerlab):
`fs.inotify.max_user_instances=64000`, `max_user_watches=524288`,
`fs.file-max=1000000`, FUSE module loaded. GNS3 warns about these at
Docker connect; the admin raises them once. XRd also warns (non-fatal)
about `net.core.*` socket buffer sizes.
## Business process
```mermaid
sequenceDiagram
participant U as User
participant S as gns3-server
participant D as Docker daemon
participant X as XRd container
U->>S: create node from template
S->>S: parse GNS3_* env host-side
S->>D: container create (ShmSize, Devices, /dev/null binds, firstboot.cfg RO bind)
U->>S: start
S->>X: container start (native entrypoint /usr/sbin/init)
Note over X: systemd boots; udevd + udevadm masked → host untouched
S->>X: docker exec volume bridge (container's own chown)
U->>S: open console
S->>X: docker exec pty: /pkg/bin/xr_cli.sh
X-->>U: IOS XR CLI (first boot: apply /firstboot.cfg, save to /xr-storage-shadow)
```
## Troubleshooting
| Symptom | Cause / fix |
|---|---|
| Node exits 139, `Fatal glibc error ... sym != NULL` in logs | init.sh wrapper path — set `GNS3_SKIP_INIT=1` **and** `console_type: docker_exec` (the flag is only honoured on the vendor class) |
| `XR_FIRST_BOOT_CONFIG ... File not found` | env path and `extra_configs` target disagree (e.g. `/firstboot.cfg` vs `/first_boot.cfg`), or entry missing |
| Console stuck at `Username:` with no credentials | image has no default user; provide a first-boot config creating one, then **recreate** the node (first-boot only runs on empty config storage) |
| `Invalid interface entries ... XR_MGMT_INTERFACES` | use `xr_name=Mg0/RP0/CPU0/0` |
| Host audio muted / USB reconnects when the node starts | set `GNS3_MASK_UDEV=1` |
| Config lost across stop/start | `extra_volumes` must include `/xr-storage` (XR's live data layer; the shadow alone is only a pristine spare). Changing `extra_volumes` requires deleting and recreating the node — Docker mounts are fixed at create time |
| Two nodes can't ping, only one side ARPs | GNS3 link wiring bug (UDP self-loop on one end), not XRd — fixed (port-allocation race); on older builds delete and re-create the link; see [link-udp-self-loop](../bugs/link-udp-self-loop.md) |
| Compute log: busybox coredump storm | fixed by the container-chown change; verify gns3-server is current |
## Notes
- All four mechanisms are opt-in: nodes that don't set the variables or the
field get byte-identical container configuration.
- `extra_configs` is a schema field (unlike the env knobs) because the
`environment` field is line-delimited and cannot carry multi-line file
content.
- Template fields live in three places (pydantic schema, DB column, Alembic
migration) — see the `extra_configs` DB migration when adding new ones.
- `net.core.*` socket-buffer requirements are not yet part of the
host-readiness check (XRd warns about them itself, non-fatally).
## References
- `gns3server/compute/docker/docker_vm.py` — HostConfig env injection,
`_UDEV_UNITS`/`_UDEVADM_PATHS`, `extra_configs` binds, `_format_devices()`
- `gns3server/compute/docker/vendor_docker_vm.py` — vendor path, volume
bridge, container-chown
- `gns3server/compute/docker/__init__.py` — `_check_host_readiness()`
- `gns3server/schemas/common.py` — `ExtraConfig`
- `gns3server/db/models/templates.py` + `db_migrations/` — persistence
- [docker-exec-console.md](./docker-exec-console.md) — the vendor NOS base
(docker_exec console, SKIP_INIT volume persistence)
- containerlab `nodes/xrd/xrd.go` — reference for XRd env defaults and
`/xr-storage` persistence
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.4 | 2026-08-15 | Code-review hardening: stop-query HTTP timeout scales with `GNS3_STOP_TIMEOUT` (values >300 s no longer abort); overlapping mask/config bind targets deduplicated (Docker "Duplicate mount point"); `ExtraConfig.target` validated at save time and directory forms rejected; host-readiness check no longer aborts on one unreadable `/proc/sys` key; base env parser strips trailing commas; vendor env knobs re-parsed on create (PUT environment takes effect); graceful stop limited to explicit user stop (delete/update/close keep the immediate kill); extra_configs under a persisted volume warns. |
| 1.3 | 2026-08-14 | Persistence corrected: XR's live data layer is `/xr-storage` (the image's symlink farm is materialized into real directories at bootstrap; `/xr-storage-shadow` is a pristine spare) — the appliance persists both. Vendor containers now stop gracefully (SIGTERM + `GNS3_STOP_TIMEOUT` grace, default 60 s) instead of being SIGKILLed on the spot. |
| 1.2 | 2026-08-14 | Link UDP self-loop root-caused to a port-allocation race and fixed — see [link-udp-self-loop](../bugs/link-udp-self-loop.md) (now marked Fixed). |
| 1.1 | 2026-08-14 | Datapath validated end-to-end (XRd brings its own interfaces up; ARP/ICMP bidirectional). Add troubleshooting entry for the one-way-link symptom (GNS3 link UDP self-loop bug, see bugs/link-udp-self-loop.md). |
| 1.0 | 2026-08-14 | Initial documentation of the XRd control-plane adaptation: vendor path requirement, shm/devices/extra_configs/udev-mask mechanisms, host-disturbance root causes, appliance recipe. |
+402
View File
@@ -0,0 +1,402 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# VNC WebSocket Console
## Overview
GNS3 server supports WebSocket-based VNC console access, enabling browser-based graphical console connections to QEMU and Docker VMs without requiring standalone VNC client applications.
This implementation uses GNS3's API layer as a transparent WebSocket-to-TCP proxy, forwarding binary VNC protocol data between the browser and the VM's VNC server.
## Architecture
### Connection Flow
```mermaid
graph LR
A[Browser noVNC] -->|WebSocket binary| B[GNS3 Controller]
B -->|WebSocket + BasicAuth + SSL| C[GNS3 Compute]
C -->|TCP| D[QEMU/Docker VNC :5900]
```
**Controller** acts as a WebSocket-to-WebSocket relay (JWT auth, IPv6 handling).
**Compute** acts as a WebSocket-to-TCP bridge (validates node state, opens VNC TCP connection).
### Components
1. **Browser (noVNC)**
- HTML5 VNC client running in the browser
- Connects via WebSocket using `binary` subprotocol
- Handles RFB protocol (Remote Frame Buffer) for VNC
2. **GNS3 Controller API**
- WebSocket endpoint: `/v3/projects/{project_id}/nodes/{node_id}/console/vnc`
- Authentication: JWT token via query parameter
- Authorization: RBAC privilege check via `has_privilege_on_websocket("Node.Console")`
- Proxies WebSocket to compute node (WebSocket-to-WebSocket relay)
- Handles IPv6 addresses by wrapping in brackets
3. **GNS3 Compute API**
- WebSocket endpoint: `/v3/compute/projects/{project_id}/{node_type}/nodes/{node_id}/console/vnc`
- Authentication: HTTP Basic Auth via `ws_compute_authentication()`
- Establishes TCP connection to VNC server, bridges WebSocket ↔ TCP
4. **Node (QEMU/Docker)**
- VNC server listening on configured port (default: 5900+)
- RFB protocol for remote display
## API Endpoints
### Controller WebSocket Endpoint
**URL**: `ws://{controller_host}:{port}/v3/projects/{project_id}/nodes/{node_id}/console/vnc?token={jwt_token}`
**Authentication**:
- JWT token via query parameter
- User must have `Node.Console` privilege
**WebSocket Subprotocols**:
- Accepts: `binary`
- Used by noVNC for binary data transfer
**Request Example**:
```javascript
const token = "eyJ0eXAiOiJKV1QiLCJhbGc...";
const wsUrl = `ws://localhost:3080/v3/projects/${projectId}/nodes/${nodeId}/console/vnc?token=${token}`;
const ws = new WebSocket(wsUrl, 'binary');
ws.binaryType = 'arraybuffer';
```
### Compute WebSocket Endpoint
**URL**: `ws://{compute_host}:{port}/v3/compute/projects/{project_id}/{node_type}/nodes/{node_id}/console/vnc`
**Authentication**:
- HTTP Basic Auth via `ws_compute_authentication()` dependency
- Configured via `settings.Server.compute_username` (default: `gns3`) and `settings.Server.compute_password` (default: empty)
**Response**:
- Bidirectional binary WebSocket connection
- Transparent VNC protocol forwarding
## Supported Node Types
### QEMU VMs
**Console Type Configuration**:
```json
{
"console_type": "vnc",
"console": 5900,
"console_resolution": "1024x768"
}
```
**QEMU Parameters**:
```bash
-vnc :0 # VNC on display 0 (port = 5900 + display)
```
**Implementation**:
- File: `gns3server/api/routes/compute/qemu_nodes.py`
- Endpoint: `/{node_id}/console/vnc`
- Method: `start_vnc_websocket_console(websocket)`
### Docker Containers
**Console Type Configuration**:
```json
{
"console_type": "vnc",
"console": 5900,
"console_resolution": "1024x768",
"console_http_port": 8080,
"console_http_path": "/"
}
```
**Implementation**:
- File: `gns3server/api/routes/compute/docker_nodes.py`
- Endpoint: `/{node_id}/console/vnc`
- Method: `start_vnc_websocket_console(websocket)`
## WebSocket Data Forwarding
### Compute Layer (WebSocket ↔ TCP)
**Location**: `gns3server/compute/base_node.py` — `start_vnc_websocket_console()`
1. Validates node is started and `console_type == "vnc"`; closes WebSocket with code 1000 otherwise
2. Opens TCP connection to VNC server at `console_host:console_port`
3. Runs two concurrent tasks via `asyncio.wait(FIRST_COMPLETED)`:
- `ws_forward()`: WebSocket → TCP (catches `WebSocketDisconnect`)
- `vnc_forward()`: TCP → WebSocket (reads 65536-byte buffer)
4. Cancels pending tasks, closes TCP writer on completion
### Controller Layer (WebSocket ↔ WebSocket)
**Location**: `gns3server/api/routes/controller/nodes.py` — `vnc_console()`
1. Authenticates user via `has_privilege_on_websocket("Node.Console")` dependency
2. Constructs compute URL with IPv6 bracket handling
3. Connects to compute WebSocket using `aiohttp.ws_connect()` with HTTP Basic Auth and SSL context
4. Uses `asyncio.ensure_future()` for client→compute forwarding, `async for msg` iteration for compute→client
### Data Flow
```mermaid
sequenceDiagram
participant B as Browser (noVNC)
participant C as Controller
participant W as Compute
participant V as VNC Server
B->>C: WebSocket connect (binary, JWT token)
C->>W: WebSocket connect (binary, BasicAuth, SSL)
W->>V: TCP connect (asyncio.open_connection)
Note over B,V: Bidirectional binary forwarding active
B->>C: WebSocket binary frame
C->>W: aiohttp send_bytes()
W->>V: TCP write(data)
V->>W: TCP data (read 65536)
W->>C: WebSocket binary frame
C->>B: send_bytes()
```
## Authentication & Authorization
### Controller Layer
1. **Authentication**:
- JWT token validation via `has_privilege_on_websocket("Node.Console")` dependency
- Token passed as query parameter: `?token={jwt}`
2. **Authorization**:
- RBAC privilege check: `Node.Console`
- Per-node access control
3. **WebSocket Subprotocol**:
- Client requests: `binary`
- Server accepts: `binary` (if requested)
### Compute Layer
1. **Authentication**:
- HTTP Basic Auth
- Credentials from controller config
- Username: `settings.Server.compute_username`
- Password: `settings.Server.compute_password`
2. **Node Validation**:
- Check node exists
- Check node is started
- Check console type is `vnc`
## Configuration
### Server Settings
**Controller Configuration** (`gns3-server.conf`):
```ini
[Server]
host = 0.0.0.0
port = 3080
```
**Compute Configuration** (same file):
```ini
[Server]
compute_username = gns3
compute_password = # empty by default, must be set for compute auth
```
### VNC Port Range
VNC console ports are allocated from a configurable range (`gns3server/schemas/config.py`):
| Setting | Default | Range |
|---------|---------|-------|
| `vnc_console_start_port_range` | 5900 | 5900–65535 |
| `vnc_console_end_port_range` | 10000 | 5900–65535 |
Validation: `vnc_console_end_port_range` must be greater than `vnc_console_start_port_range`.
### Node Settings
**QEMU VM Example**:
```json
{
"name": "vm-1",
"node_type": "qemu",
"console_type": "vnc",
"console": 5900,
"console_resolution": "1280x720",
"properties": {
"qemu_path": "/usr/bin/qemu-system-x86_64"
}
}
```
**Docker Container Example**:
```json
{
"name": "container-1",
"node_type": "docker",
"console_type": "vnc",
"console": 5900,
"console_resolution": "1024x768",
"console_http_port": 8080
}
```
## Troubleshooting
### Common Issues
**1. "Node is not started"**
- **Symptom**: WebSocket closes immediately
- **Solution**: Start the VM before opening console
- **API Check**: `GET /v3/projects/{project_id}/nodes/{node_id}` → verify `status == "started"`
**2. "Console type is not vnc"**
- **Symptom**: WebSocket closes with error
- **Solution**: Set node console_type to "vnc"
- **API Update**: `PUT /v3/projects/{project_id}/nodes/{node_id}` with `{"console_type": "vnc"}`
**3. "Cannot connect to VNC server"**
- **Symptom**: Connection timeout
- **Possible Causes**:
- VNC server not listening
- Port conflict
- Firewall blocking connection
- **Verification**:
```bash
# Check if VNC is listening
netstat -tlnp | grep 5900
# Check QEMU process
ps aux | grep qemu
```
**4. WebSocket Subprotocol Negotiation Failed**
- **Symptom**: Connection rejected during handshake
- **Solution**: Ensure noVNC sends `binary` subprotocol
- **Browser Console Check**:
```javascript
console.log(ws.protocol); // Should be "binary"
```
**5. Authentication Errors**
- **Symptom**: HTTP 401/403 responses
- **Controller**: Check JWT token is valid and not expired
- **Compute**: Verify compute username/password in config
### Debug Logging
**Enable Detailed Logging**:
```ini
[gns3server]
debug = true
```
**Check Controller Logs**:
```bash
# Look for WebSocket connection messages
grep "VNC console WebSocket" /var/log/gns3/gns3.log
```
**Check Compute Logs**:
```bash
# Look for VNC forwarding messages
grep "Connected to VNC server" /var/log/gns3/gns3.log
```
**Browser Console**:
```javascript
// Enable noVNC debugging
RFB.messages.log = function(msg) { console.log(msg); };
```
## Security
### Authentication
1. **Controller → Client**
- JWT token with expiration
- RBAC authorization
- Privilege: `Node.Console`
2. **Controller → Compute**
- HTTP Basic Auth via `aiohttp.BasicAuth`
- SSL context from `Controller.instance().ssl_context()`
- Raises `ControllerForbiddenError` if `compute_username` is not set
3. **VNC Server**
- Optional VNC password (QEMU only)
- Configured via node properties
### Network Security
**Recommendations**:
1. Use HTTPS/WSS for production deployments
2. Firewall compute API ports
3. Use short-lived JWT tokens
4. Enable VNC password for sensitive VMs
**Example**:
```bash
# Controller with TLS
gns3server --ssl --cert /path/to/cert.pem --key /path/to/key.pem
# VNC with password
qemu-system-x86_64 -vnc :0,password
```
### WebSocket Security
**Subprotocol Validation**:
- Only `binary` subprotocol is accepted
- Prevents protocol downgrade attacks
**Origin Validation**:
- Browser enforces same-origin policy
- Configure CORS if using separate domains
## Limitations
1. **Single Client per VNC Port**
- VNC protocol supports one client at a time
- New connections disconnect existing clients
2. **No Audio Redirection**
- VNC protocol does not support audio
3. **No USB Redirection**
- VNC protocol does not support device redirection
4. **Performance Overhead**
- WebSocket-to-TCP bridging adds processing overhead compared to native VNC clients
## References
- [RFB Protocol 3.8](https://tools.ietf.org/html/rfc6143) - VNC Protocol Specification
- [noVNC Documentation](https://github.com/novnc/noVNC) - HTML5 VNC Client
- [FastAPI WebSocket](https://fastapi.tiangolo.com/advanced/websockets/) - WebSocket Implementation
- [GNS3 Documentation](https://docs.gns3.com/) - General GNS3 Documentation
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.1 | 2026-04-19 | Review against code: fix buffer size, auth details, add port range config, remove unverified content |
| 1.0 | 2026-03-17 | Initial VNC WebSocket documentation |
@@ -0,0 +1,620 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# Web Wireshark Feature - Business Process Documentation
## Overview
The **Web Wireshark** feature enables users to run Wireshark packet capture analysis directly in a web browser without requiring a desktop environment or VNC connection. This is achieved through an **xpra** (persistent remote applications) HTML5 client running inside a Docker container.
## Feature Summary
- **Core Capability**: Web-based packet capture visualization using Wireshark
- **Technology Stack**: Docker container + xpra + HTML5 WebSocket proxy
- **Target Users**: Network engineers and students who need to analyze network traffic in GNS3 topologies
- **Key Benefit**: Zero-install, browser-based packet capture analysis
---
## Installation
Before using Web Wireshark, install the GNS3 server and set up the Docker image:
```bash
# Development install
pip install -e . && gns3server-web-wireshark-setup
# Production install
pip install gns3-server && gns3server-web-wireshark-setup
```
This command will:
1. Install the gns3-server package
2. Pull the `gns3/web-wireshark:latest` image from Docker Hub
3. If pull fails, build the image locally using the included Dockerfile
The `wireshark` command shows the raw output from `docker pull` or `docker build`, allowing you to see the full progress.
---
## Architecture Overview
```mermaid
graph TD
subgraph GNS3Server["GNS3 Server"]
WebUI["Web UI<br/>(Browser)"]
Controller["GNS3 Controller"]
LinkCtrl["Link Controller<br/>(start_capture / stop_capture)"]
Manager["WebWiresharkManager<br/>(called directly, not via CLI)"]
DockerClient["DockerHTTPClient<br/>(HTTP via Unix Socket)"]
end
subgraph DockerDaemon["Docker Daemon"]
Container["gns3-wireshark-{project_id}"]
Xvfb["Xvfb (virtual framebuffer)"]
Xpra["xpra server"]
Wireshark["Wireshark"]
end
ClientBrowser["Client Browser<br/>(HTML5 Client)"]
ClientBrowser -->|REST API| WebUI
WebUI -->|capture/start| Controller
Controller --> LinkCtrl
LinkCtrl -->|start/stop/restart session| Manager
Manager -->|Docker API<br/>/var/run/docker.sock| DockerClient
DockerClient -->|container lifecycle| Container
Container --- Xvfb
Container --- Xpra
Container --- Wireshark
ClientBrowser -.->|WebSocket proxy| Xpra
Xpra -.->|xpra HTML5| ClientBrowser
style GNS3Server fill:#e8f4fd,stroke:#2196f3
style DockerDaemon fill:#fff3e0,stroke:#ff9800
style ClientBrowser fill:#e8f5e9,stroke:#4caf50
```
---
## Component Description
### 1. Web UI (Client Browser)
- User interface for starting/stopping packet capture
- Receives WebSocket URL for connecting to xpra HTML5 client
- No plugins required - pure HTML5/JavaScript
### 2. GNS3 Controller
- Orchestrates the capture workflow
- Validates user permissions (RBAC)
- Manages link capture state
### 3. manage_wireshark.py (Management CLI)
- Command-line interface for container and session management
- **For manual debugging and testing only** — the server calls `WebWiresharkManager` directly at runtime
- Handles Docker container lifecycle
- Manages xpra sessions per link
### 4. WebWiresharkManager
- Core business logic for Web Wireshark
- Handles container creation, session startup/shutdown
- Deterministic port allocation based on link_id
### 5. DockerHTTPClient
- Async HTTP client for Docker API
- Communicates via Unix socket (/var/run/docker.sock)
- Manages container lifecycle
### 6. Docker Container (gns3-wireshark-{project_id})
- Runs xpra server with HTML5 support
- Contains Xvfb (virtual framebuffer) for headless Wireshark
- Streams display to browser via WebSocket
---
## Business Processes
### Process 1: Start Packet Capture with Web Wireshark
```mermaid
sequenceDiagram
actor User
participant WebUI as Web UI
participant Controller as Controller
participant LinkCtrl as Link Controller
participant Manager as WebWiresharkManager
User->>WebUI: 1. Start Capture
WebUI->>Controller: 2. POST /capture/start
Controller->>LinkCtrl: 3. _start_web_wireshark(jwt_token)
LinkCtrl->>Manager: 4. start_wireshark_session()
Note right of Manager: 5. Ensure network exists
Note right of Manager: 6. Get/create container
Note right of Manager: 7. Start xpra session<br/>(display + port)
Note right of Manager: 8. Start Wireshark<br/>(curl | wireshark -i - -k)
Manager-->>LinkCtrl: ws_url
LinkCtrl-->>Controller: ws_url
Controller-->>WebUI: 10. {capturing: true, ws_url: "ws://..."}
WebUI-->>User: 9. ws_url
Note over User,Manager: 11-12. Browser connects via WebSocket<br/>Server proxies to container xpra
```
**API Endpoint**: `POST /v3/projects/{project_id}/links/{link_id}/capture/start`
**Request Body**:
```json
{
"wireshark": true,
"data_link_type": "DLT_EN10MB",
"capture_file_name": "capture.pcap"
}
```
**Response**:
```json
{
"id": "link-uuid",
"capturing": true,
"ws_url": "ws://192.168.1.100:14500"
}
```
### Process 2: Container Lifecycle (Per Project)
```mermaid
flowchart TD
A["Project Open"] --> B["First Link Capture"]
B --> C["Container Created<br/>(one per project)"]
C --> D["Container Running"]
subgraph SessionArchitecture["Container Session Architecture"]
direction TB
D --> E["gns3-wireshark-{project_id}"]
subgraph Link1Session["Link 1 Session"]
L1Xpra["xpra :14503"]
L1Xvfb["Xvfb :14503<br/>(1920x1080x24)"]
L1WS["Wireshark<br/>(curl | wireshark -i -)"]
L1Bind["bind-ws=0.0.0.0:14503"]
L1Xpra --- L1Xvfb
L1Xpra --- L1WS
L1Xpra --- L1Bind
end
subgraph Link2Session["Link 2 Session"]
L2Xpra["xpra :11024"]
L2Xvfb["Xvfb :11024<br/>(1920x1080x24)"]
L2WS["Wireshark<br/>(curl | wireshark -i -)"]
L2Bind["bind-ws=0.0.0.0:11024"]
L2Xpra --- L2Xvfb
L2Xpra --- L2WS
L2Xpra --- L2Bind
end
E --> Link1Session
E --> Link2Session
Note1["display = port = 10000 + hash(link_id) % 10000<br/>Each xpra creates its own Xvfb (NOT shared)"]
end
D --> F["Project Close"]
F --> G["Stop Container<br/>(all sessions terminate)"]
G --> H["Container Stopped<br/>(preserved for reuse)"]
H -->|Project Reopened| D
I["Project Delete"] --> J["Delete Container"]
J --> K["Container Removed"]
style SessionArchitecture fill:#fff8e1,stroke:#ffa000
style Link1Session fill:#e3f2fd,stroke:#1976d2
style Link2Session fill:#e8f5e9,stroke:#388e3c
```
### Process 3: WebSocket Connection Flow
```mermaid
sequenceDiagram
participant Browser as Client Browser
participant Server as GNS3 Server (API Route)
participant Container as Docker Container (xpra)
Browser->>Server: 1. WSS Connect (ws://.../web-wireshark?token=jwt)
Note right of Server: 2. Validate JWT<br/>(RBAC: Link.Capture)
Note right of Server: 3. Get container IP<br/>from Docker API
Server-->>Browser: 4. Accept connection
Note right of Server: 5. Start WebSocket proxy
Server->>Container: 6. Connect to xpra<br/>ws://container:port
Container-->>Server: 7. xpra validates subprotocol
Note over Browser,Container: 8-9. Bidirectional WebSocket proxy<br/>Browser ⟺ Server ⟺ Container
Note left of Browser: 10. HTML5 client renders<br/>Wireshark window
```
**WebSocket Endpoint**: `ws://host/v3/projects/{project_id}/links/{link_id}/capture/web-wireshark?token=<jwt_token>`
### Process 4: Capture Data Flow
```mermaid
flowchart LR
subgraph GNS3Node["GNS3 Node"]
Router["Router / Switch"]
end
subgraph ComputeNode["Compute Node"]
Capture["Link Capture Buffer"]
end
Router -->|"TAP (raw packets)"| Capture
subgraph GNS3Server["GNS3 Server"]
APIRoute["GET /capture/stream<br/>Proxies pcap stream"]
end
Capture -->|"pcap stream"| APIRoute
subgraph DockerContainer["Docker Container"]
direction TB
Curl["curl -N -H 'Authorization: Bearer {jwt}'<br/>'{server}/capture/stream'"]
Wireshark["wireshark -i - -k"]
Xvfb["Xvfb :display<br/>(1920x1080x24)"]
Xpra["xpra :display<br/>ws://0.0.0.0:{port}"]
Curl -->|"stdin pipe"| Wireshark
Wireshark -->|"renders to"| Xvfb
Xvfb -->|"X11 display"| Xpra
end
APIRoute -.->|"pcap stream<br/>(curl fetches from container)"| Curl
subgraph Browser["Client Browser"]
HTML5Client["HTML5/xpra Client"]
WSUI["Wireshark Web Interface"]
HTML5Client --> WSUI
end
Xpra <-->|"WebSocket"| HTML5Client
style GNS3Node fill:#f3e5f5,stroke:#7b1fa2
style ComputeNode fill:#e8eaf6,stroke:#283593
style GNS3Server fill:#e8f4fd,stroke:#2196f3
style DockerContainer fill:#fff3e0,stroke:#ff9800
style Browser fill:#e8f5e9,stroke:#4caf50
```
---
## Port Allocation Strategy
### Deterministic Port Mapping
```mermaid
flowchart LR
LinkID["link_id (UUID)"] --> Hash["MD5 Hash"] --> Modulo["hash % 10000"]
Modulo --> Offset["+ 10000"]
Offset --> Result["display = port<br/>Range: 10000 - 19999"]
style LinkID fill:#e3f2fd,stroke:#1976d2
style Result fill:#e8f5e9,stroke:#388e3c
```
| link_id | port/display |
|---------|-------------|
| `f233f27f-7432-49c3-9aa2-50e326a10eec` | 14503 |
| `a1b2c3d4-1234-5678-90ab-cdef12345678` | 11024 |
| `12345678-90ab-cdef-1234-567890abcdef` | 17892 |
**Benefits**:
- Same link always gets same port (deterministic)
- Display number = port number (same hash)
- No port conflicts between sessions
- Easy to predict and debug
---
## Network Architecture
```mermaid
graph TD
subgraph HostMachine["Host Machine"]
subgraph GNS3ServerHost["GNS3 Server<br/>192.168.1.100:3080"]
WSProxy["WebSocket Proxy"]
end
subgraph DockerNetwork["Docker Bridge: gns3-wireshark<br/>Subnet: 172.31.0.0/22"]
Gateway["Bridge Gateway<br/>172.31.0.1"]
Container["gns3-wireshark-{project_id}<br/>172.31.0.x"]
Gateway --- Container
end
end
Client["Client Browser"] -->|"ws://192.168.1.100:3080<br/>/capture/web-wireshark"| WSProxy
WSProxy -->|"via gateway 172.31.0.1<br/>ws://172.31.0.x:{port}"| Container
style HostMachine fill:#fafafa,stroke:#9e9e9e
style DockerNetwork fill:#e3f2fd,stroke:#1976d2
style GNS3ServerHost fill:#e8f5e9,stroke:#388e3c
```
---
## Session Management Commands
> **Note**: These commands are for manual debugging and testing only. The GNS3 server calls `WebWiresharkManager` directly at runtime.
### start
Starts Web Wireshark session for a specific link.
| Argument | Required | Default | Description |
|----------|----------|---------|-------------|
| `--project-id` | Yes | - | Project UUID |
| `--link-id` | Yes | - | Link UUID |
| `--jwt-token` | Yes | - | JWT authentication token |
| `--capture-url` | No | auto-detected | PCAP stream URL |
| `--image` | No | `gns3/web-wireshark:latest` | Docker image |
| `--memory` | No | `2g` | Memory limit |
| `--cpus` | No | `1.0` | CPU cores |
| `--pids-limit` | No | `1000` | Process limit |
```bash
python manage_wireshark.py start \
--project-id "5af0fe00-..." \
--link-id "f233f27f-..." \
--jwt-token "eyJhbG..."
```
### Other Commands
| Command | Description | Key Arguments |
|---------|-------------|---------------|
| `stop` | Stop session for a specific link | `--project-id`, `--link-id` |
| `restart` | Restart session (reopens Wireshark window) | `--project-id`, `--link-id`, `--jwt-token` |
| `stop-all` | Stop all sessions for a project | `--project-id` |
| `delete` | Delete container (alias for delete-container) | `--project-id` |
| `stop-container` | Stop container without deleting | `--project-id` |
| `delete-container` | Delete the container | `--project-id` |
---
## Project Close/Delete Workflow
```mermaid
flowchart TD
A["Project Close"] --> B["_stop_web_wireshark_container()"]
B --> C["WebWiresharkManager.stop_container(project_id)"]
C --> D["docker stop<br/>(all xpra/Xvfb/Wireshark terminate)"]
D --> E["Container Stopped<br/>(preserved for reuse)"]
E -->|Project Reopened| F["First Capture Start"]
F --> G["docker start<br/>(container already exists, fast)"]
G --> H["New xpra sessions created"]
I["Project Delete"] --> J["_cleanup_web_wireshark_container()"]
J --> K["WebWiresharkManager.delete_container(project_id)"]
K --> L["docker rm<br/>(container removed)"]
style A fill:#e3f2fd,stroke:#1976d2
style I fill:#ffebee,stroke:#c62828
style E fill:#fff8e1,stroke:#ffa000
style L fill:#ffebee,stroke:#c62828
```
---
## API Endpoints Summary
| Method | Endpoint | Description | Privilege |
|--------|----------|-------------|-----------|
| POST | `/v3/projects/{id}/links/{id}/capture/start` | Start capture with Web Wireshark | Link.Capture |
| POST | `/v3/projects/{id}/links/{id}/capture/stop` | Stop capture | Link.Capture |
| POST | `/v3/projects/{id}/links/{id}/capture/wireshark/restart` | Restart Wireshark window | Link.Capture |
| GET | `/v3/projects/{id}/links/{id}/capture/stream` | Stream PCAP data | Link.Capture |
| GET | `/v3/projects/{id}/links/{id}/capture/file` | Download PCAP file | Link.Capture |
| WS | `/v3/projects/{id}/links/{id}/capture/web-wireshark` | WebSocket proxy for xpra | Link.Capture |
---
## Security Considerations
1. **RBAC Authentication**: All endpoints require `Link.Capture` privilege
2. **JWT Token Validation**: WebSocket connections validate JWT token
3. **WebSocket Subprotocol Negotiation**: Proper xpra subprotocol handling
4. **Container Isolation**: Each project gets its own container with isolated resources
5. **Network Segmentation**: Container runs on isolated Docker network (not host network)
---
## Performance Characteristics
### Startup & Shutdown Performance
| Step | Before Optimization | After Optimization | Improvement |
|------|---------------------|-------------------|-------------|
| **Health Check** | ~1.0s | ~0s | Docker native status |
| **Gateway Detection** | 0.85s | ~0.001s | Docker API vs exec |
| **Process Cleanup** | ~850ms | ~40ms | Host perspective recursive tree walk |
| **Xpra Startup** | 6.4s | ~3s | HTML5 client disabled |
| **Wireshark Launch** | ~1s | ~1s | No change |
| **Total Startup** | **~15s** | **~5-6s** | **67% faster** |
| Step | Before Optimization | After Optimization | Improvement |
|------|---------------------|-------------------|-------------|
| **Process Termination** | ~8s | ~20-40ms | Recursive tree walk, no orphans |
| **File Cleanup** | ~850ms | ~850ms | Docker exec (safety) |
| **Total Shutdown** | **~9s** | **~2s** | **78% faster** |
#### Startup Breakdown: First vs Subsequent
| Phase | First Startup (Container stopped) | Subsequent Startup (Container running) |
|-------|-----------------------------------|----------------------------------------|
| Container startup | ~1-2s | ~0s (already running) |
| Container health check | ~1s (unhealthy→healthy) | ~0s (already healthy) |
| Gateway detection | ~0.001s | ~0.001s |
| Process cleanup | ~40ms | ~40ms |
| Xpra startup | ~3s | ~3s |
| Wireshark launch | ~1s | ~1s |
| **Total** | **~6s** | **~5s** |
#### Measured Performance Data
**Startup (from production logs):**
```
13:46:53 → 13:46:59 = 6s (first startup with container start)
13:47:38 → 13:47:43 = 5s (subsequent startup, container running)
```
**Shutdown (from production logs):**
```
13:48:15 → 13:48:17 = 2s (complete cleanup, no orphan processes)
13:48:50 → 13:48:52 = 2s (complete cleanup, no orphan processes)
```
#### Key Optimizations
- **Complete Process Cleanup**: Recursive process tree traversal eliminates orphaned processes (Xvfb, pulseaudio, ibus-daemon)
- **Fast Gateway Detection**: Docker API query instead of container exec
- **Smart Health Check**: Trust Docker built-in status, no manual ping
- **Xpra Optimization**: Disabled unnecessary HTML5 client (`--html=off`)
- **Reduced Docker Exec Calls**: Combined X lock and xpra socket cleanup into single exec call
### Docker Exec Performance Limitations
**Important**: Docker daemon has internal queuing for concurrent exec requests to the same container.
#### Test Results (Same Container)
| Test Scenario | Execution Time | Avg Per Exec |
|--------------|----------------|--------------|
| Single docker exec | 0.854s | 0.854s |
| Serial 7 docker exec | 7.225s | 1.032s |
| Parallel 7 docker exec | 10.253s | 1.465s |
**Key Finding**: Parallel execution is **42% slower** than serial execution.
```
Serial: 7.225s (7 requests processed sequentially)
Parallel: 10.253s (Docker daemon still processes sequentially + context switch overhead)
```
#### Impact on Web Wireshark
When stopping multiple capture sessions quickly:
- Each stop requires 1 docker exec (cleanup files)
- Docker daemon processes exec requests sequentially
- 7 sessions × ~1s each = ~7-10 seconds total
- User requests appear to "queue" even though they're concurrent
#### Optimization Strategy
Since Docker exec cannot be parallelized effectively:
1. **Minimize exec calls** - Already implemented: 2 calls → 1 call
2. **Accept serial processing** - No benefit to parallel execution
3. **Focus on fast exec content** - Use simple `rm -f` commands
#### Future Optimization Options
- Use Docker API instead of exec (requires container filesystem access)
- Delay cleanup to next startup (increases startup complexity)
- Batch multiple stops into single operation (requires API changes)
### Resource Usage (Per Wireshark Instance)
| Resource | Typical Usage |
|----------|---------------|
| Memory | 150-250 MB |
| CPU | 0.5-2% (idle to active) |
| Threads | ~30 threads |
| Disk I/O | Minimal |
### Container Configuration
Configured via `WebWiresharkSettings` in `gns3server/schemas/config.py`:
| Parameter | Default | Recommended | Description |
|-----------|---------|-------------|-------------|
| enabled | true | - | Enable/disable Web Wireshark feature |
| image | gns3/web-wireshark:latest | - | Docker image name |
| network_subnet | 172.31.0.0/22 | - | Docker bridge network subnet |
| Memory | 2GB | 2-4GB | Container memory limit |
| CPUs | 1.0 | 1.0-2.0 | Container CPU limit |
| PIDs Limit | 1000 | 1000 | Container process limit |
### Scaling Guidelines
| Instances | Memory | Use Case |
|-----------|--------|----------|
| 1-3 | 450-750 MB | Light projects |
| 4-6 | 600-1.5 GB | Medium projects |
| 7-10 | 1-2.5 GB | Large projects |
| 10+ | >2.5 GB | Increase memory |
---
## Docker API Compatibility
### Version Negotiation Mechanism
The Web Wireshark feature implements automatic Docker API version negotiation to ensure compatibility across different Docker versions:
1. **Primary**: Try API version 1.44 first
- Supports Docker 29.3+ (API 1.54+), which requires minimum API 1.44
2. **Fallback**: If server rejects 1.44 with 400 error, downgrade to API 1.40
- Supports Docker 20.10 (API 1.41)
### Tested Configurations
| Docker Version | API Version | API 1.40 | API 1.44 | Solution |
|----------------|-------------|----------|----------|----------|
| 20.10 | 1.41 | ✓ | ✗ | Fallback to 1.40 |
| 29.3+ | 1.54+ | ✗ | ✓ | Use 1.44 |
### Container IP Retrieval
The `get_container_ip()` method implements a dual-strategy approach for retrieving container IP addresses:
1. **Primary Method**: Query Docker Container API
- Uses `docker inspect` via HTTP API
- Safe access to `NetworkSettings.Networks` field using `.get()`
- Handles missing fields gracefully (fixes KeyError bug)
2. **Fallback Method**: Execute `hostname -I` command inside container
- Works when Docker API response lacks network information
- Compatible with containers without `ip` command
- Returns first IP address from `hostname -I` output
This dual approach ensures compatibility across different Docker API versions that may have varying response formats.
---
## Known Limitations
1. **JWT Token Visibility**: Token passed via command-line arguments (visible in `/proc/<pid>/cmdline`)
2. **Single Container**: All Wireshark instances run in a single container per project
3. **Docker Dependency**: Requires Docker daemon running on the server
4. **Browser Support**: Requires modern browser with WebSocket support
5. **Port Range**: Limited to 10,000 unique ports (10000-19999)
---
## File Structure
```
gns3server/
├── controller/
│ ├── link.py # Link capture lifecycle
│ └── project.py # Project cleanup hooks
├── api/routes/controller/
│ └── links.py # REST/WebSocket API endpoints
└── agent/web_wireshark/
├── setup_wireshark_image.py # Docker image setup tool
├── manage_wireshark.py # CLI management tool (manual/debug use only)
├── manager.py # Session management logic (called by server)
├── docker_client.py # Docker API client
├── stats.py # Container statistics collection
├── docker/
│ └── Dockerfile # Container image definition
└── WEB_WIRESHARK.md # Technical documentation
```
@@ -0,0 +1,161 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
# GNS3-Copilot AI Assistant Overview
## Overall Architecture
```mermaid
flowchart TB
subgraph "Client"
A["Web UI"] --> B["SSE Streaming"]
end
subgraph "FastAPI Route Layer"
B --> C["POST /chat/stream\nPOST /chat/inject"]
C --> D["Auth + LLM Config Loading\nSet ContextVars"]
end
subgraph "AgentService (Project-level)"
D --> E["LangGraph Agent\nStateGraph"]
E --> F["SQLite Checkpointer\ncopilot_checkpoints.db"]
end
subgraph "LangGraph Workflow"
E --> G["llm_call node\nmodel invocation"]
E --> H["tool_node\ntool execution"]
E --> I["title_generator_node\nauto title"]
E --> J["abort_handler_node\ninterrupt handling"]
end
subgraph "Three Copilot Modes"
G --> K["teaching_assistant\ndiagnostic read-only"]
G --> L["lab_automation_assistant\nfull control"]
G --> M["troubleshooting_injection\nfault injection"]
end
subgraph "LLM Config System"
D --> N["User configs\nGroup config inheritance\nAPI key encryption"]
end
```
## API Endpoints
| Endpoint | Function |
|---|---|
| `POST /v3/projects/{pid}/chat/stream` | Streaming conversation (SSE), supports three copilot modes |
| `POST /v3/projects/{pid}/chat/inject` | Fault injection entry, auto-switches to `troubleshooting_injection` mode |
| `GET /v3/projects/{pid}/chat/sessions` | List sessions (supports filtering, pagination) |
| `DELETE /v3/projects/{pid}/chat/sessions/{sid}` | Delete session |
| `PATCH /v3/projects/{pid}/chat/sessions/{sid}` | Update session (rename, pin) |
| `POST /v3/projects/{pid}/chat/sessions/{sid}/abort` | Abort an active session |
## LangGraph Agent Workflow
```mermaid
sequenceDiagram
participant U as User
participant API as FastAPI
participant AS as AgentService
participant LLM as LLM Node
participant Tool as Tool Node
participant TGen as Title Node
U->>API: send message
API->>AS: stream_chat()
AS->>AS: set ContextVars<br/>(jwt_token, llm_config)
Note over AS,LLM: llm_call node
AS->>LLM: invoke pre-compiled model
LLM->>LLM: pre_model_hook<br/>inject topology + trim context
LLM-->>AS: AI reply (may include tool_calls)
opt has tool calls
AS->>Tool: execute tools
Tool-->>AS: tool results
AS->>LLM: continue LLM call
end
opt first turn and no title
AS->>TGen: auto-generate title
TGen-->>AS: session title
end
AS-->>API: SSE streaming response
API-->>U: stream output
```
## Three Copilot Modes
### Mode Comparison
| Mode | Tool Scope | Use Case |
|---|---|---|
| `teaching_assistant` (default) | Diagnostic read-only + packet analysis + node management | Teaching demos, troubleshooting guidance |
| `lab_automation_assistant` | All tools (including config changes) | Lab automation, device configuration |
| `troubleshooting_injection` | Fault injection tool set | Troubleshooting practice, fault simulation |
### Tool Binding Details
| Tool | teaching_assistant | lab_automation_assistant | troubleshooting_injection |
|---|---|---|---|
| `GNS3TemplateTool` get templates | ✓ | ✓ | |
| `GNS3CreateNodeTool` create nodes | ✓ | ✓ | |
| `GNS3LinkTool` create links | ✓ | ✓ | |
| `GNS3StartNodeTool` start nodes | ✓ | ✓ | |
| `GNS3UpdateNodeNameTool` rename | ✓ | ✓ | |
| `GNS3StopNodeTool` stop nodes | | ✓ | |
| `GNS3SuspendNodeTool` suspend nodes | | ✓ | |
| `ExecuteMultipleDeviceCommands` read-only commands | ✓ | ✓ | ✓ |
| `ExecuteMultipleDeviceConfigCommands` config commands | | ✓ | ✓ |
| `VPCSCommands` VPCS commands | | ✓ | |
| `PacketAnalysisTool` live packet analysis | ✓ | ✓ | |
| `PacketAnalysisSkillsTool` protocol knowledge | ✓ | ✓ | |
| `DeviceSkillsTool` device skills | ✓ | ✓ | |
| `GNS3PacketFilterTool` link filters | | | ✓ |
| `InjectionSkillsTool` fault injection skills | | | ✓ |
| `GNS3TopologyTool` topology info | | | ✓ |
The mode is selected in the `llm_call` node via `copilot_mode`, which picks the corresponding tool list and binds it to the LLM model instance through `create_base_model_with_tools(mode_tools, llm_config)`.
## Context Window Management
```mermaid
flowchart LR
A["LLM call triggered"] --> B["pre_model_hook"]
B --> C["Inject topology\ninto System Prompt"]
B --> D["Estimate tool definition\ntoken cost"]
B --> E["trim_messages\nby strategy"]
E --> F["conservative 60%\nbalanced 75%\naggressive 85%"]
F --> G["Invoke LLM"]
```
- Accurate token counting via tiktoken (`cl100k_base`)
- Three trimming strategies: conservative / balanced / aggressive
- Auto-injects `{{topology_info}}` into System Prompt
## Session Management
- Per-project independent SQLite database (`gns3-copilot/copilot_checkpoints.db`)
- Supports pin, rename, delete, history query
- Auto-records token usage, message count, LLM call count
## LLM Config System
| Feature | Description |
|---|---|
| User-level configs | Each user can independently configure provider / model / api_key |
| Group inheritance | Users auto-inherit group config when no personal config is set |
| API key encryption | Auto-encrypted at database storage |
| Optimistic locking | `version` field prevents concurrent modification conflicts |
## Key Design Points
1. **Project-level Isolation** — Each GNS3 project has its own Agent instance and SQLite storage
2. **ContextVars Safe Passing** — JWT token, API key exist only in memory, auto-cleared when request ends
3. **LangGraph StateGraph** — Custom nodes + conditional edges, supports ReAct loop and recursion limits
4. **SSE Streaming** — Real-time push of content / tool_call / tool_start / tool_end / error / done events
5. **Hot Reload** — System Prompt, Skills, Protocols all support runtime reload
6. **Mode-based Tool Sets** — Three copilot modes bind different tools, safely isolated by scenario
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,224 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# Command Security Configuration
## Overview
GNS3-Copilot includes multiple security layers to prevent execution of commands that may cause issues in the lab environment:
- **Command Filtering**: Prevents commands that may timeout or lock up the console
- **Configuration Safety**: Restricts dangerous configuration changes (AAA, passwords, etc.)
- **Multi-line Command Handling**: Properly processes commands with embedded newlines (banner, etc.)
## Architecture
```mermaid
graph TD
A[Tool Request] --> B{Command Filter}
B -->|allowed| C[Nornir/Netmiko Execution]
B -->|blocked| D[blocked_commands_info]
C --> E[Process Results]
D --> E
E --> F[Return Result]
subgraph Command Filter
B1[Load forbidden_commands.txt]
B2[Prefix match per command]
B1 --> B2
end
subgraph Security Layers
S1[Layer 1: Command Filter<br/>prefix-based blocking]
S2[Layer 2: AI Prompt Rules<br/>AAA/password guidance only]
S3[Layer 3: Multi-line Expansion<br/>banner/ACL handling]
S1 --> S2 --> S3
end
```
## Business Process
### Command Filtering Flow
```mermaid
flowchart TD
Start([Tool receives commands]) --> Load[Load forbidden patterns<br/>from config file]
Load --> Loop{For each command}
Loop -->|next command| Prefix{Command starts with<br/>forbidden pattern?}
Prefix -->|yes| Block[Add to blocked_commands_info]
Prefix -->|no| Allow[Add to allowed list]
Block --> Loop
Allow --> Loop
Loop -->|all checked| Empty{Allowed list empty?}
Empty -->|yes| Return1[Return partial_success<br/>with blocked info only]
Empty -->|no| Exec[Execute allowed commands<br/>via Nornir/Netmiko]
Exec --> HasBlocked{Has blocked commands?}
HasBlocked -->|yes| Return2[Return partial_success<br/>with output + blocked info]
HasBlocked -->|no| Return3[Return success<br/>with output]
```
### Multi-line Command Expansion Flow
```mermaid
flowchart TD
A[config_tools receives commands] --> B{Command contains \n?}
B -->|yes| C[Split by newline]
C --> D[Filter empty lines]
D --> E[Add expanded lines to command list]
B -->|no| F[Keep command as-is]
E --> G[Execute via netmiko_send_config]
F --> G
```
## Tools Using Command Security
| Tool | File | Filter | Multi-line Expansion | Result Fields |
|------|------|--------|---------------------|---------------|
| `ExecuteMultipleDeviceCommands` | `display_tools_nornir.py` | Yes | No | `diagnostic_commands` |
| `ExecuteMultipleDeviceConfigCommands` | `config_tools_nornir.py` | Yes | Yes | `config_commands` |
## Command Filtering System
### Forbidden Commands Configuration
The forbidden commands list is loaded from the external [GNS3-Skills](https://github.com/gns3/gns3-skills) repository at `config/forbidden_commands.txt`.
**Format:**
- One command pattern per line
- **Prefix matching** (case-insensitive) — matches the beginning of each command
- Empty lines and lines starting with `#` are ignored
**Default patterns (used when skills repository is unavailable):**
| Pattern | Reason |
|---------|--------|
| `traceroute` | Can run 30+ seconds, exceeds tool timeout |
| `tracepath` | Similar to traceroute, long execution time |
| `tracert` | Windows traceroute, same timeout issues |
| `ping -f` | Flood ping can overwhelm lab devices |
| `debug` | Can produce overwhelming output and destabilize devices |
| `test` | May affect device stability |
### Result Format
**Status values:**
| Status | Condition |
|--------|-----------|
| `"success"` | All commands executed, none blocked |
| `"partial_success"` | Some commands blocked (including all blocked + execution succeeded) |
| `"failed"` | Execution failed (device not found, connection error, etc.) |
**Example — partial success (display tool):**
```json
{
"device_name": "R-1",
"status": "partial_success",
"output": "...",
"diagnostic_commands": ["show version", "show ip int brief"],
"blocked_commands": ["traceroute 8.8.8.8"],
"blocked_info": {
"traceroute 8.8.8.8": "Command 'traceroute 8.8.8.8' is not allowed because it matches the forbidden pattern 'traceroute'. ..."
}
}
```
> **Note:** `diagnostic_commands` is specific to display tools. Config tools use `config_commands` instead.
## Module Structure
**Command Filter:** `gns3server/agent/gns3_copilot/utils/command_filter.py`
| Function | Purpose |
|----------|---------|
| `filter_forbidden_commands(commands)` | Returns `(allowed_commands, blocked_commands_info)` |
| `is_command_forbidden(command)` | Check if a single command matches a forbidden pattern |
| `get_forbidden_commands()` | Get current forbidden patterns list |
| `reload_forbidden_commands()` | Directly load and cache commands from skills repository |
**Integration points:**
- `display_tools_nornir.py` — `_filter_forbidden_commands_from_device_configs()`
- `config_tools_nornir.py` — `_filter_forbidden_commands_from_device_configs()` + `_expand_multiline_commands()`
## Configuration Safety (AI Prompt Level)
In addition to the command filter, the AI agent is instructed via system prompt (`lab_automation_assistant_prompt.py`) to handle sensitive configuration commands with caution:
**Prompt-enforced rules:**
- **FORBIDDEN (guidance only):** `enable secret`, `username`, `aaa new-model`, `service password-encryption`, `line vty` — AI provides configuration guidance instead of executing
- **Caution required:** `reload`, `erase`, `format` — AI warns user before executing destructive operations
> These are prompt-level rules, not code-level enforcement. Users can always execute commands directly via device console or SSH/Telnet.
## Multi-line Command Handling
Commands containing `\n` are automatically split before execution by `config_tools_nornir.py:_expand_multiline_commands()`.
**Example:** `["banner motd #\nWelcome\n#"]` becomes `["banner motd #", "Welcome", "#"]`
Applies to any command with embedded newlines: `banner`, multi-line ACLs, route-maps, etc.
## Configuration
### Customizing Forbidden Commands
Edit `config/forbidden_commands.txt` in the [GNS3-Skills repository](https://github.com/gns3/gns3-skills) and push the changes, then call `POST /copilot/reload/skills` to apply them without restarting the server.
## Implementation Verification
### Test Results (Live GNS3 Environment)
**Scenario:** IOU-L2-1 (Cisco IOS L3 switch), mixed allowed/forbidden commands.
```json
{
"device_name": "IOU-L2-1",
"status": "partial_success",
"diagnostic_commands": [
"show ip route", "show ip interface brief",
"ping 10.0.0.1", "ping 10.0.0.2", "ping 10.0.0.4"
],
"blocked_commands": ["traceroute 10.0.0.2"],
"blocked_info": {
"traceroute 10.0.0.2": "Command 'traceroute 10.0.0.2' is not allowed because it matches the forbidden pattern 'traceroute'. ..."
}
}
```
| Feature | Status | Notes |
|---------|--------|-------|
| Prefix match filtering | Verified | `traceroute` correctly matched and blocked |
| Partial execution | Verified | Other commands executed successfully |
| `partial_success` status | Verified | Set correctly when blocked + succeeded |
| Multi-device support | Verified | Each device filtered independently |
| Non-blocking behavior | Verified | No tool timeouts or console issues |
## Future Enhancements
- **Regex support** for more sophisticated pattern matching
- **Per-project override files** for forbidden commands
- **Web UI configuration** for managing forbidden commands
- **Audit logging** for blocked command analysis
- **Per-command timeouts** instead of blocking
- **Interrupt mechanism** (Ctrl+C) for long-running commands
## Troubleshooting
| Problem | Solution |
|---------|----------|
| Command blocked unexpectedly | Check `blocked_commands` in result, identify matching pattern, edit `forbidden_commands.txt` |
| "File not found, using defaults" | Verify `config/forbidden_commands.txt` exists and is readable |
| Changes not taking effect | Call `POST /copilot/reload/skills` or restart server |
## Related Documentation
- [Skills Repository](skills-repository.md)
- [Fault Injection](fault-injection.md)
- [Chat API](chat-api.md)
@@ -0,0 +1,203 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# LLM Context Window Management
## Overview
GNS3 Copilot's context window management prevents LLM context overflow by trimming conversation history to fit within configured token limits. It uses tiktoken for accurate token counting, supports three context strategies (conservative/balanced/aggressive), and automatically injects GNS3 topology information into the system prompt before each LLM call.
## Architecture
```mermaid
graph TD
subgraph "context_manager.py"
A[create_pre_model_hook] --> B[_inject_topology_into_system]
A --> C[estimate_tool_tokens]
A --> D[trim_messages via LangChain]
E[count_tokens via tiktoken]
E --> C
E --> F[_count_tokens_for_message]
F --> D
end
subgraph "gns3_copilot.py"
G[llm_call node] --> H[Get topology via GNS3TopologyTool]
H --> A
A --> I[model_with_tools.invoke]
end
style A fill:#4a9eff,color:#fff
style D fill:#ff6b6b,color:#fff
style E fill:#51cf66,color:#fff
```
## Key Components
| Function | File | Purpose |
|----------|------|---------|
| `create_pre_model_hook()` | `context_manager.py` | Factory that creates the preprocessing hook |
| `_inject_topology_into_system()` | `context_manager.py` | Injects topology into system prompt via `{{topology_info}}` placeholder |
| `count_tokens()` | `context_manager.py` | Accurate token counting using tiktoken (`cl100k_base`) |
| `estimate_tool_tokens()` | `context_manager.py` | Estimates token cost of tool definitions (JSON serialization) |
| `_count_tokens_for_message()` | `context_manager.py` | Token counter adapter for LangChain's `trim_messages` |
| `llm_call()` | `gns3_copilot.py` | StateGraph node that orchestrates the full LLM call pipeline |
**Token Counting**: Uses tiktoken `cl100k_base` encoding (GPT-4 compatible, 95%+ accuracy for most modern LLMs).
**tiktoken Cache**: Encoding files are cached locally at `<agent_package>/cache/tiktoken/` to avoid repeated downloads. This is configured before tiktoken import via `TIKTOKEN_CACHE_DIR` environment variable.
**Required Dependency**: `tiktoken>=0.8.0` — if not installed, `ModuleNotFoundError` is raised at startup.
## Trimming Process
```mermaid
flowchart TD
A[llm_call node invoked] --> B[Get topology via GNS3TopologyTool]
B --> C[Create pre_model_hook]
C --> D[Inject topology into system prompt]
D --> E{topology data available?}
E -->|Yes| F[Replace placeholder with topology content]
E -->|No| G[Replace placeholder with 'No topology information available']
F --> H[Estimate tool definition tokens]
G --> H
H --> I[Calculate trim_messages budget]
I --> J[trim_messages with strategy='last']
J --> K[Return prepared messages]
K --> L[model.invoke with prepared messages]
```
### Token Budget Calculation
```mermaid
flowchart LR
A["context_limit<br/>(e.g. 128K)"] -->|"× strategy_ratio"| B["Input Budget<br/>(e.g. 96K @ balanced)"]
B -->|"− tool_tokens"| C["trim_messages Budget<br/>(includes system message)"]
C --> D["trim_messages<br/>max_tokens parameter"]
```
**Key**: `max_tokens` passed to `trim_messages` **includes** the system message. System tokens are NOT subtracted separately — `trim_messages` handles system message preservation via `include_system=True`.
### Budget Calculation Example
| Step | Calculation | Result |
|------|------------|--------|
| Context limit | 128 (configured) | 128K tokens |
| Strategy ratio | balanced = 0.75 | 128 × 1000 × 0.75 = 96,000 |
| Tool definitions | estimated from JSON schema | ~1,725 tokens |
| trim_messages budget | 96,000 − 1,725 | **94,275 tokens** (includes system) |
### Trimming Priority
| Priority | Content | Behavior |
|----------|---------|----------|
| 1 | System Message (system prompt + topology) | Always kept (`include_system=True`) |
| 2 | Latest conversation messages | Kept from newest to oldest |
| 3 | Older conversation history | Discarded first when exceeding budget |
**Note**: System prompt and topology are merged into a single `SystemMessage` via the `{{topology_info}}` template variable and cannot be trimmed independently.
### Edge Case Handling
| Scenario | Behavior |
|----------|----------|
| System + tools > input budget | ERROR log with recommendations (reduce prompt/tools, use larger model, switch to conservative). `trim_messages` is still called — `include_system=True` ensures system message is kept, but no history messages will fit. |
| Trim budget < system × 1.5 | WARNING log: minimal room for conversation history |
| Invalid context_strategy | Falls back to `balanced` with WARNING log |
| `trim_messages` throws exception | Returns untrimmed messages (graceful degradation) |
| Missing `{{topology_info}}` placeholder | Returns original messages without injection, WARNING log |
| Topology retrieval fails | Injects `"(No topology information available)"` placeholder |
## Integration with GNS3 Copilot
**File**: `gns3server/agent/gns3_copilot/agent/gns3_copilot.py`
### Why Direct Call, Not Config-based?
The agent uses a **custom StateGraph** (`StateGraph(MessagesState)`), not LangGraph's pre-built agent. LangGraph's `pre_model_hook` config parameter only works with pre-built agents like `create_react_agent`.
| Agent Type | pre_model_hook Support |
|------------|------------------------|
| `create_react_agent` | Via `config={"configurable": {"pre_model_hook": ...}}` |
| `chat_agent_executor` | Via `config={"configurable": {"pre_model_hook": ...}}` |
| **Custom StateGraph** | **Not supported** — must call directly |
### Execution Flow
In the `llm_call` node of `gns3_copilot.py`:
1. Get `llm_config` from request-scoped context variable
2. Get `project_id` from `config["configurable"]`
3. Retrieve topology via `GNS3TopologyTool._run(project_id)`
4. Select tools based on `copilot_mode` (`teaching_assistant` vs `lab_automation_assistant`)
5. Load system prompt and create `pre_model_hook`
6. **Directly call** `pre_hook({"messages": messages, "topology_info": topology_info})` to prepare messages
7. Invoke `model_with_tools.invoke(prepared_messages)`
> **Note**: The actual `llm_call` function also includes defensive checks for missing config, empty messages, and topology retrieval errors — see source code for full details.
## Strategy Implementation
### Context Strategy Ratios
Defined as `CONTEXT_STRATEGY_RATIOS` in `context_manager.py`, with default `DEFAULT_CONTEXT_STRATEGY = "balanced"`.
| Strategy | Input Ratio | Output Reserved | Calculation |
|----------|-------------|-----------------|-------------|
| Conservative | 60% | 40% | `context_limit × 1000 × 0.60` |
| Balanced | 75% | 25% | `context_limit × 1000 × 0.75` |
| Aggressive | 85% | 15% | `context_limit × 1000 × 0.85` |
## Log Output
### Normal Case (topology injected)
```
INFO: Calling pre_hook to prepare 1 messages
INFO: ✓ Topology injected: 7722 chars, nodes: ['netshoot-1', 'R1', 'R2', 'IOU-L3-1', 'IOU-L3-2']
INFO: Context ready: 2 msgs, ~3815 tokens + 1725 tools = 5540 / 128K (4.3%), strategy=conservative
INFO: Messages prepared: 1 → 2
INFO: LLM call completed: tool_calls=0
```
### When Trimming Occurs
```
INFO: Calling pre_hook to prepare 50 messages
INFO: ✓ Topology injected: 8500 chars, nodes: ['R1', 'R2', ...]
INFO: Messages trimmed: 50 → 25 msgs. Total: ~82000 tokens + 1725 tools = 83725 / 128K (65.4%), strategy=balanced
INFO: Messages prepared: 50 → 25
```
### When Topology is None
```
INFO: Calling pre_hook to prepare 1 messages
WARNING: ✗ Topology data is None, injecting placeholder
INFO: Context ready: 2 msgs, ~800 tokens + 1725 tools = 2525 / 128K (2.0%), strategy=balanced
```
## Error Handling
| Error | Behavior | Resolution |
|-------|----------|------------|
| tiktoken not installed | `ModuleNotFoundError` at startup | `pip install tiktoken>=0.8.0` |
| `context_limit` missing | `ValueError` raised | Add `context_limit` to LLM config |
| Invalid `context_limit` | `ValueError` raised | Must be positive integer |
| Invalid `context_strategy` | WARNING, fallback to `balanced` | Use `conservative`/`balanced`/`aggressive` |
| `trim_messages` failure | ERROR log, returns untrimmed messages | Check message format compatibility |
## Deprecated API
`prepare_context_messages()` is deprecated. It remains in `context_manager.py` for backward compatibility but emits `DeprecationWarning`. Use `create_pre_model_hook()` instead.
## Related Source Files
- `gns3server/agent/gns3_copilot/agent/context_manager.py` — Context management core logic
- `gns3server/agent/gns3_copilot/agent/gns3_copilot.py` — LLM call node (StateGraph)
- `gns3server/agent/gns3_copilot/agent/model_factory.py` — Model creation and tool binding
@@ -0,0 +1,104 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
# GNS3-Copilot Fault Injection Overview
## Core Flow
```mermaid
flowchart TB
subgraph "① API Trigger & Mode Switch"
A["POST /chat/inject\nUser requests fault injection"] --> B["Verify project is opened"]
B --> C["Set copilot_mode =\ntroubleshooting_injection"]
C --> D["Start Agent\nwith fault injection tool set"]
end
subgraph "② Topology Analysis & Fault Selection"
D --> E["GNS3TopologyTool\nget topology info"]
E --> F["ExecuteMultipleDeviceCommands\nget device configs"]
F --> G["InjectionSkillsTool\nquery available fault types"]
G --> H{"Injection Skills Repository\ngns3/gns3-skills"}
H --> I["Return matching fault definitions\nwith config injection commands"]
end
subgraph "③ Fault Injection"
I --> J["Choose injection method"]
J --> K["ExecuteMultipleDeviceConfigCommands\ninject config changes"]
J --> L["GNS3PacketFilterTool\ninject link-layer faults"]
end
subgraph "④ Result Confirmation"
K --> M["Verify fault is active"]
L --> M
M --> N["Document fault details\nincluding restore commands"]
end
```
## Tool Overview
| Tool | Source File | Purpose | Available Modes |
|---|---|---|---|
| `InjectionSkillsTool` | `registry.py` (skills module) | Query protocol-level fault definitions (config change commands) | troubleshooting_injection |
| `GNS3PacketFilterTool` | `gns3_packet_filter.py` | Link-layer fault injection (delay, loss, corruption, BPF) | troubleshooting_injection |
| `ExecuteMultipleDeviceConfigCommands` | `config_tools_nornir.py` | Batch device config changes | troubleshooting_injection |
| `ExecuteMultipleDeviceCommands` | `display_tools_nornir.py` | Read device configurations (read-only) | troubleshooting_injection |
| `GNS3TopologyTool` | `gns3_client` | Get project topology information | troubleshooting_injection |
## Fault Injection API
| Endpoint | Function |
|---|---|
| `POST /v3/projects/{pid}/chat/inject` | Trigger fault injection, sets `troubleshooting_injection` mode then starts Agent |
**Prerequisite**: Project must be in `opened` status, otherwise returns 403.
## GNS3PacketFilterTool Link Filters
| Filter Type | Function | Parameters |
|---|---|---|
| `delay` | Latency + jitter | `[latency(0-32767), jitter(0-32767)]` |
| `packet_loss` | Packet loss percentage | `[chance(0-100)]` |
| `corrupt` | Packet corruption percentage | `[chance(0-100)]` |
| `frequency_drop` | Drop every Nth packet | `[frequency(-1~32767)]` |
| `bpf` | Berkeley Packet Filter | expression text |
## Agent Workflow (LangGraph)
```mermaid
sequenceDiagram
participant U as User
participant API as POST /chat/inject
participant LLM as LLM Node
participant Topo as GNS3TopologyTool
participant DC as ExecuteMultipleDeviceCommands
participant CC as ExecuteMultipleDeviceConfigCommands
participant Skill as InjectionSkillsTool
participant Filter as GNS3PacketFilterTool
U->>API: Inject an OSPF fault
API->>LLM: set mode=troubleshooting_injection
LLM->>Topo: get topology
Topo-->>LLM: topology info
LLM->>DC: read device configs
DC-->>LLM: running configs
LLM->>Skill: list context=["ospf"]
Skill-->>LLM: matching fault types
LLM->>Skill: get device_type=injection_ospf
Skill-->>LLM: fault definition + injection commands
LLM->>CC: execute config injection
CC-->>LLM: injection result
LLM->>Filter: set filters={delay:[200,50]}
Filter-->>LLM: link delay injected successfully
LLM-->>U: Fault injected, restore commands included
```
## Key Design Points
1. **Dedicated API Endpoint** — `POST /chat/inject` is the dedicated entry point, automatically switching to `troubleshooting_injection` mode
2. **LLM-driven Fault Selection** — The LLM analyzes the topology then queries matching faults via `InjectionSkillsTool`; no hardcoded fault scenarios
3. **Dual-Layer Injection** — Device-level config changes + link-level network impairment, covering complete troubleshooting scenarios
4. **Fully Reversible** — Every injection includes restore commands; link filters can be cleared with `action: clear`
5. **Safety First** — BPF syntax is pre-validated via tshark; config commands are restricted by `command_filter`
6. **Context Filtering** — `InjectionSkillsTool` requires a `context` parameter, returning only faults matching the topology protocols
@@ -0,0 +1,159 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# Fault Injection Feature
## Overview
The fault injection feature enables GNS3 Copilot to automatically inject realistic network faults into GNS3 labs for troubleshooting training. The agent analyzes the topology, selects an appropriate fault based on the configured protocols, injects it, and documents the results.
## Architecture
```mermaid
graph TD
subgraph "Agent Workflow"
TG[Topology Gathering]
CFG[Config Collection]
FA[Fault Analysis]
FI[Fault Injection]
FD[Fault Documentation]
end
subgraph "Skills Repository"
IS[INJECTION_SKILLS_REGISTRY<br/>39 categories, 368 issues]
SK[injection_skills Tool]
end
subgraph "Tools"
DC[execute_multiple_device_commands]
CC[execute_multiple_device_config_commands]
GK[GNS3TopologyTool]
end
TG -->|Get topology| GK
CFG -->|Get configs| DC
FA -->|Query filtered skills| IS
FA -->|Select fault| SK
FI -->|Inject config changes| CC
FD -->|Output report| FD
```
## Injection Skills Tool
The `InjectionSkillsTool` (LangChain `BaseTool`) is the primary interface for querying available faults.
### Listing Faults (with context filter)
The LLM MUST pass topology context when listing faults:
```json
{"action": "list", "context": ["ospf", "bgp", "vlan"]}
```
This returns only faults matching the protocols found in the topology. The tool rejects calls without `context`:
```json
{
"error": "context parameter is required when action='list'",
"hint": "Analyze the topology and device configurations first...",
"available_categories": ["bgp", "interface", "mpls", "ospf", ...]
}
```
### Getting Fault Details
Token-efficient usage pattern:
```json
// Step 1: List issue names only (~300 tokens)
{"device_type": "injection_ospf", "detail": "index"}
// Step 2: Get single issue detail (~500 tokens)
{"device_type": "injection_ospf", "issue": "ospf_hello_dead_mismatch"}
```
### Parameters
| Parameter | Required | Description |
|-----------|----------|-------------|
| `action` | No (`"get"`) | `"list"` to browse, `"get"` for details |
| `context` | **Yes** for `action="list"` | Protocols found in topology: `["ospf", "bgp"]` |
| `device_type` | Yes for `action="get"` | e.g. `"injection_ospf"` |
| `detail` | No (`"full"`) | `"index"` (names), `"summary"` (+desc), `"full"` (all) |
| `issue` | No | Single issue key for targeted detail |
## Fault Injection Workflow
```mermaid
sequenceDiagram
participant LLM
participant InjectionSkillsTool
participant TopologyTool
participant DeviceCommands
LLM->>TopologyTool: Get topology info
TopologyTool-->>LLM: Topology + device types
LLM->>DeviceCommands: Get device configs
DeviceCommands-->>LLM: Running configurations
Note over LLM: Analyze which protocols are in use
LLM->>InjectionSkillsTool: {"action": "list", "context": ["ospf", "vlan"]}
InjectionSkillsTool-->>LLM: Matching fault categories
LLM->>InjectionSkillsTool: {"device_type": "injection_ospf", "detail": "index"}
InjectionSkillsTool-->>LLM: Issue names (token-efficient)
LLM->>InjectionSkillsTool: {"device_type": "injection_ospf", "issue": "ospf_hello_dead_mismatch"}
InjectionSkillsTool-->>LLM: Full fault detail + config commands
LLM->>DeviceCommands: Inject fault configuration
DeviceCommands-->>LLM: Execution result
Note over LLM: Document fault in response
```
## Injection Skills Repository
Skills are organized by protocol/category in the external [GNS3-Skills](https://github.com/gns3/gns3-skills) repository:
| Category | File | Example Issues |
|----------|------|----------------|
| OSPF | `injection/ospf_issues.yaml` | Hello/Dead mismatch, MTU mismatch, area mismatch |
| BGP | `injection/bgp_issues.yaml` | AS-path prepend, next-hop unreachable, route filtering |
| VLAN | `injection/vlan_issues.yaml` | Trunk allowed mismatch, native VLAN mismatch |
| STP | `injection/stp_issues.yaml` | Root guard, loop guard, port priority |
| MPLS | `injection/mpls_issues.yaml` | LDP session down, label binding failure |
| ... | 34 more files | 368 issues total |
## Recovery
Each injected fault includes restore commands in the documentation. The LLM always provides commands to fully revert all changes.
## API Endpoint
### POST /copilot/projects/{project_id}/chat/inject
Dedicated endpoint for fault injection. Internally sets `copilot_mode` to `troubleshooting_injection` and runs the agent.
**Request:**
```json
{
"message": "Inject a network fault for troubleshooting practice",
"session_id": "optional-session-uuid"
}
```
**Response:** Server-Sent Events (SSE) stream identical to the chat stream endpoint.
## Related Documentation
- [Skills Repository](skills-repository.md)
- [Command Security](command-security.md)
- [Chat API](chat-api.md)
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,772 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# Multi-Vendor Network Device Support
## Overview
GNS3-Copilot supports network devices from multiple vendors through Netmiko and Nornir integration. The system includes a custom Netmiko driver for Huawei devices in GNS3 emulation environments and supports dynamic device type detection.
## Supported Vendors
| Vendor | Device Type | Protocol | Status |
|--------|-------------|----------|--------|
| **Cisco** | `cisco_ios_telnet` | Telnet | ✅ Tested |
| **Huawei** | `gns3_huawei_telnet_ce` | Telnet | ✅ Tested (Custom Driver) |
| **Ruijie (锐捷)** | `gns3_ruijie_telnet` | Telnet | ✅ Tested (Custom Driver) |
| **VPCS** | `gns3_vpcs_telnet` | Telnet | ✅ Tested (Custom Driver, Simulator) |
## Custom VPCS Driver (`VPCSTelnet`)
### Problem Statement
VPCS (Virtual PC Simulator) is a lightweight virtual PC simulator used in GNS3 lab environments. Unlike network devices (routers/switches), VPCS devices:
1. **No authentication** - Direct console access without username/password
2. **Simple command interface** - No configuration modes
3. **Simple prompt pattern** - `PC1>`, `PC2>`, etc.
### Solution: Lightweight Custom Driver
```mermaid
graph TD
A[BaseConnection<br/>Netmiko base class] --> B[VPCSTelnet<br/>Custom GNS3 driver]
B --> C[No authentication]
B --> D[Simple prompt: PC\d+]
B --> E[No config mode]
B --> F[ANSI stripping]
```
**Why Not Use Standard Telnet Driver?**
- Standard drivers attempt authentication (times out)
- No support for VPCS-specific prompt patterns (`PC\d+>`)
- No need for configuration mode handling
### VPCSTelnet Implementation
#### Location
```
gns3server/agent/gns3_copilot/utils/custom_netmiko/vpcs_telnet.py
```
#### Key Features
**1. No Authentication**
```python
def telnet_login(self, pri_prompt_terminator=r"PC\d+>", ...):
# Send returns until VPCS prompt detected
for i in range(max_loops):
self.write_channel(self.RETURN)
output = self.read_channel()
if re.search(pri_prompt_terminator, output):
return output # Success - VPCS prompt detected
```
**2. Simple Prompt Recognition**
```
PC1> ip 10.10.0.12/24 10.10.0.254
PC1> ping 10.10.0.254
```
**3. No Configuration Mode**
```python
def check_config_mode(self) -> bool:
return False # VPCS has no config mode
def config_mode(self) -> str:
return "" # No config mode to enter
def exit_config_mode(self) -> str:
return "" # No config mode to exit
```
**4. No Paging**
```python
def disable_paging(self) -> str:
return "" # VPCS doesn't use paging
```
**5. Custom `send_command`**
VPCS overrides `send_command` with non-standard defaults:
- `strip_prompt=False`, `strip_command=False`, `normalize=False` — VPCS output doesn't need standard Netmiko processing
- Uses `_strip_ansi_codes()` to clean terminal escape sequences from VPCS output
**6. ANSI Escape Code Stripping**
VPCS output often contains ANSI terminal codes. The `_strip_ansi_codes()` method strips bold, underline, reset, and color codes before returning output to callers.
### VPCS Tool Usage
The VPCS driver is used by the `execute_vpcs_commands` tool:
> File: `gns3server/agent/gns3_copilot/tools_v2/vpcs_tools_netmiko.py`
**VPCS Connection Parameters:**
VPCS devices use additional Netmiko parameters for reliable connections:
| Parameter | Value | Reason |
|-----------|-------|--------|
| `fast_cli` | `False` | VPCS is slow, disable fast CLI mode |
| `global_delay_factor` | `2.0` | Double the delay between commands |
```python
"connection_options": {
"netmiko": {
"extras": {
"device_type": "gns3_vpcs_telnet",
"fast_cli": False,
"global_delay_factor": 2.0,
}
}
}
```
### VPCS Built-in Template Configuration
**✨ Automatic Tags - No Manual Configuration Required**
VPCS nodes created from the built-in template automatically include the necessary tags:
| Tag | Value | Purpose |
|-----|-------|---------|
| `device_type` | `gns3_vpcs_telnet` | Netmiko driver selection |
**Built-in Template Definition:**
> File: `gns3server/services/templates.py`
The VPCS built-in template includes `"tags": ["device_type:gns3_vpcs_telnet"]`, which is automatically applied to all VPCS nodes created from the template.
**User Benefits:**
- ✅ **No manual tagging required** — Tags are applied automatically when creating VPCS nodes
- ✅ **Automatic driver selection** — Copilot tools automatically use the correct Netmiko driver
- ✅ **Consistent behavior** — All VPCS nodes from the built-in template work identically
- ✅ **Zero configuration** — Users don't need to understand device_type tags
**How It Works:**
1. User creates a VPCS node from the built-in "VPCS" template
2. Node automatically inherits the tag: `device_type:gns3_vpcs_telnet`
3. Copilot tools read the tag and select the appropriate VPCS Netmiko driver
4. Commands execute using the VPCS-optimized driver (no authentication, simple prompts)
### Supported VPCS Commands
| Command | Description | Example |
|---------|-------------|---------|
| `ip` | Configure/show IP address | `ip 10.10.0.12/24 10.10.0.254` |
| `ping` | Test connectivity | `ping 10.10.0.254` |
| `arp` | Display ARP table | `arp` |
| `show ip` | Show IP configuration | `show ip` |
| `version` | Show VPCS version | `version` |
| `save` | Save configuration | `save` |
| `load` | Load configuration | `load` |
### Architecture Benefits
**Unified Tool Architecture:**
- ✅ Uses Nornir for connection management (same as network device tools)
- ✅ Uses Netmiko for command execution (consistent with other tools)
- ✅ Follows same patterns as `config_tools_nornir.py` and `display_tools_nornir.py`
- ✅ Simplified codebase - no need for separate telnetlib3 implementation
**Migration from telnetlib3:**
| Aspect | Old (telnetlib3) | New (Netmiko + Nornir) |
|--------|-----------------|-------------------------|
| Library | telnetlib3 | Netmiko |
| Framework | Manual threading | Nornir |
| Code Lines | ~490 lines | ~580 lines (with better structure) |
| Consistency | Unique implementation | Same as other tools |
| Maintenance | Separate code path | Unified architecture |
---
## Custom Huawei Driver (`GNS3HuaweiTelnetCE`)
### Problem Statement
GNS3-emulated Huawei devices connect via console **without requiring authentication**. Standard Netmiko drivers attempt username/password authentication, causing connection timeouts.
**Standard Driver Behavior:**
```
Telnet Connection → Wait for username prompt → Send username → Wait for password → Send password → Access
^ Times out after 20 seconds
```
**GNS3 Huawei Device:**
```
Telnet Connection → Direct access to command line (no login prompts)
<HUAWEI>
```
### Solution: Custom Driver Architecture
```mermaid
graph TD
A[BaseConnection] --> B[CiscoBaseConnection]
B --> C[HuaweiBase<br/>VRP support, system-view,<br/>prompt patterns, paging]
C --> D[GNS3HuaweiTelnetCE<br/>Overrides telnet_login only]
D --> E[Skip authentication]
D --> F[Auto-commit before exit]
D --> G[y/n auto-confirm]
```
**Why Inherit from HuaweiBase?**
- ✅ Built-in VRP (Versatile Routing Platform) command handling
- ✅ Huawei-specific configuration mode (`system-view`)
- ✅ Huawei prompt patterns (`<...>`, `[...]`)
- ✅ Huawei paging disable (`screen-length 0 temporary`)
- ✅ Minimal code changes - only override authentication
### GNS3HuaweiTelnetCE Implementation
#### Location
```
gns3server/agent/gns3_copilot/utils/custom_netmiko/huawei_ce.py
```
**Package Structure:**
```
custom_netmiko/
├── __init__.py # Package initialization, auto-registers all drivers
├── huawei_ce.py # Huawei CloudEngine custom driver
├── README.md # Driver development guide
└── tests/ # Unit tests
├── __init__.py
└── test_huawei_ce.py # Huawei CE driver tests
```
#### Key Features
1. **Skip Authentication**
- Directly detect Huawei prompt patterns
- No username/password prompts
- Connection ready in < 1 second
2. **VRP Prompt Recognition**
```
User view: <HUAWEI>
System view: [HUAWEI]
Interface: [HUAWEI-GigabitEthernet0/0/1]
```
3. **Automatic Confirmation Handling**
- Detects and responds to `[y/n]` prompts
- Example: `return` command asks "Return to user view? [y/n]:"
- Automatically sends `y` to confirm
4. **Proper Output Collection**
- Uses Netmiko's `read_channel_timing()` for reliable output
- Waits for command completion (2s no new data = done)
- 30-second absolute timeout prevents hanging
5. **Auto-Commit Before Exit**
- Automatically sends `commit` command before exiting config mode
- Prevents "Uncommitted configurations [Y/N/C]" prompt
- Ensures configuration changes are saved
#### Limitations
**Why Not Use the Standard `huawei_telnet` Driver?**
- The standard Netmiko Huawei driver (`huawei_telnet`) has known issues in GNS3 emulation environments
- This is the primary reason the custom `gns3_huawei_telnet_ce` driver was developed
- The custom driver is designed for GNS3 devices **without authentication**
- If your GNS3 Huawei device has been configured with authentication, remove it for GNS3 testing to use the custom driver
**When to Use:**
| Scenario | Recommendation |
|----------|---------------|
| GNS3 Huawei device | Always use `gns3_huawei_telnet_ce` |
| GNS3 device has auth configured | Remove auth, then use `gns3_huawei_telnet_ce` |
#### Method Overrides
**1. `telnet_login` - Skip Authentication**
```python
def telnet_login(self, pri_prompt_terminator=r"<\S+>|>\s*$",
alt_prompt_terminator=r"\[\S+\]", ...) -> str:
# Clear buffer
self.read_channel()
# Send returns until prompt detected
for i in range(max_loops):
self.write_channel(self.RETURN)
output = self.read_channel()
# Check for Huawei prompts
if re.search(pri_prompt_terminator, output):
return output # Success!
return output # Best effort
```
**2. `send_config_set` - Configuration Commands**
```python
def send_config_set(self, config_commands, **kwargs) -> str:
# Enter config mode
output += self.config_mode(config_command="system-view")
# Send all commands
for cmd in config_commands:
self.write_channel(f"{cmd}{self.RETURN}")
time.sleep(delay_factor * 0.05)
# Collect output using Netmiko standard method
output += self.read_channel_timing(read_timeout=30, last_read=2.0)
# Auto-commit before exit (prevents [Y/N/C] prompt)
self.write_channel(f"commit{self.RETURN}")
time.sleep(0.5 * self.global_delay_factor)
output += self.read_channel()
# Exit config mode
output += self.exit_config_mode()
return output
```
**3. `exit_config_mode` - Handle Confirmation**
```python
def exit_config_mode(self, exit_config="return", pattern=r"<\S+>|>\s*$") -> str:
self.write_channel(f"return{self.RETURN}")
# Look for confirmation prompt
for _ in range(20):
new_output = self.read_channel()
if re.search(r"\[y/n\]", new_output):
self.write_channel(f"y{self.RETURN}") # Auto-confirm
if re.search(pattern, new_output):
return output # Back to user view
return output
```
### Device Type Registration
The custom driver must be registered with Netmiko's global mappings:
```python
def register_custom_device_type() -> None:
import importlib
sd = importlib.import_module("netmiko.ssh_dispatcher")
# Register in CLASS_MAPPER (for ConnectHandler)
sd.CLASS_MAPPER["gns3_huawei_telnet_ce"] = GNS3HuaweiTelnetCE
sd.CLASS_MAPPER["huawei_ce"] = GNS3HuaweiTelnetCE
# Register in CLASS_MAPPER_BASE (for base class definitions)
sd.CLASS_MAPPER_BASE["gns3_huawei_telnet_ce"] = GNS3HuaweiTelnetCE
sd.CLASS_MAPPER_BASE["huawei_ce"] = GNS3HuaweiTelnetCE
# CRITICAL: Rebuild static lists
sd.platforms = list(sd.CLASS_MAPPER.keys())
sd.platforms.sort()
sd.telnet_platforms = [x for x in sd.platforms if "telnet" in x]
```
**Important: Static List Problem**
- `ssh_dispatcher.platforms` is computed at module import time
- Modifying `CLASS_MAPPER` doesn't automatically update `platforms`
- Must manually rebuild the list after registration
**Auto-Registration**
```python
# Automatically runs on module import
try:
register_custom_device_type()
except Exception as e:
logger.warning(f"Failed to register custom device type: {e}")
```
## Custom Ruijie Driver (`RuijieTelnetEnhanced`)
### Problem Statement
Ruijie (锐捷) network devices exhibit Cisco-like command syntax but have interactive prompts during configuration that can cause standard Netmiko drivers to fail.
**Common Issue - OSPF Router-ID:**
```
Router(config-router)#router-id 10.0.0.1
Change router-id and update OSPF process! [yes/no]:
```
Standard Netmiko's `send_config_set()` waits for a prompt pattern, but the `[yes/no]:` prompt doesn't match the expected config mode pattern, causing:
1. **ReadTimeout**: Netmiko times out waiting for the prompt
2. **Command Failure**: Subsequent commands are not executed
3. **Device Lockup**: Console remains in the waiting state
### Solution: Hybrid Strategy with Interactive Prompt Handling
```mermaid
graph TD
A[BaseConnection] --> B[CiscoBaseConnection]
B --> C[RuijieOSBase<br/>Netmiko built-in]
C --> D[RuijieTelnetEnhanced<br/>Interactive prompt handling]
D --> E[Preprocessing<br/>Insert yes after known commands]
D --> F[Batch send<br/>Fast path 2-3s]
D --> G[Fallback one-by-one<br/>Slow but reliable ~10s]
```
**Why Hybrid Strategy?**
1. **Preprocessing**: Automatically insert `yes` after known interactive commands
2. **Fast Path**: Batch send for most commands (2-3 seconds for 13 commands)
3. **Fallback**: One-by-one send with real-time detection (reliable but slower)
### RuijieTelnetEnhanced Implementation
#### Location
```
gns3server/agent/gns3_copilot/utils/custom_netmiko/ruijie_telnet.py
```
#### Key Features
**1. Preprocessing - Known Interactive Commands**
```python
INTERACTIVE_PATTERNS = [
re.compile(r'^router-id\s+', re.IGNORECASE), # OSPF/EIGRP/BGP router-id
re.compile(r'^erase\s+', re.IGNORECASE), # erase startup-config
re.compile(r'^delete\s+', re.IGNORECASE), # delete files
re.compile(r'^format\s+', re.IGNORECASE), # format filesystem
re.compile(r'^reload\b', re.IGNORECASE), # reload/reboot
re.compile(r'^boot\s+system\s+', re.IGNORECASE), # change boot image
]
```
**2. Hybrid Send Strategy**
```mermaid
flowchart TD
A[Input Configuration Commands] --> B[Preprocessing]
B --> B1[Detect interactive commands]
B1 --> B2[Insert 'yes' after them]
B2 --> C[Try Batch Send - Fast]
C --> C1[Write all commands rapidly]
C1 --> C2[Read output once, last_read=2.0s]
C2 --> D{Success?}
D -->|Yes| E[Return output]
D -->|No| F[Fallback: One-by-One]
F --> F1[Send each command individually]
F1 --> F2[Detect prompts after each]
F2 --> F3[Send 'yes' when needed, last_read=0.5s]
F3 --> E
```
**3. Batch Send Performance**
- **13 commands** with 1 interactive command
- **Fast path**: ~2-3 seconds (includes device processing time)
- **Fallback**: ~10 seconds (if batch fails)
**4. Real-time Detection (Fallback)**
```python
# After each command
new_output = self.read_channel_timing(read_timeout=10, last_read=0.5)
# Check for [yes/no] prompt
if re.search(r"\[yes/no\]", new_output, re.IGNORECASE):
self.write_channel(f"yes{self.RETURN}")
output += self.read_channel_timing(read_timeout=30, last_read=0.5)
```
#### GNS3 Node Tag Configuration
**For Ruijie devices in GNS3:**
```
device_type:gns3_ruijie_telnet
```
**Example Usage:**
```python
from netmiko import ConnectHandler
from gns3server.agent.gns3_copilot.utils import custom_netmiko
device = {
"device_type": "gns3_ruijie_telnet",
"host": "127.0.0.1",
"port": 5000,
}
with ConnectHandler(**device) as conn:
# These commands include router-id (interactive)
config = [
"router ospf 1",
"router-id 10.0.0.1", # Triggers [yes/no] prompt
"network 192.168.1.0 0.0.0.255 area 0",
]
# Automatically handles the [yes/no] prompt
output = conn.send_config_set(config)
```
#### Limitations
**Interactive Command Coverage:**
- **Covered**: `router-id`, `erase`, `delete`, `format`, `reload`, `boot system`
- **Not Covered**: Unknown or vendor-specific interactive prompts
- **Fallback**: If batch fails, falls back to one-by-one with real-time detection
**When to Use:**
| Scenario | Use Driver |
|----------|------------|
| GNS3 Ruijie (known interactive commands) | `gns3_ruijie_telnet` (batch + auto-fallback) |
| GNS3 Ruijie (unknown interactive prompts) | `gns3_ruijie_telnet` (auto-fallback to one-by-one) |
## Dynamic Device Type Detection
### GNS3 Node Tags
Device type is extracted from GNS3 node tags:
```
device_type:gns3_huawei_telnet_ce → Netmiko driver selection
```
**Tag Examples:**
| Vendor | Device Type Tag | Template Source |
|--------|----------------|-----------------|
| Cisco IOS | `device_type:cisco_ios_telnet` | User appliance |
| Huawei CE | `device_type:gns3_huawei_telnet_ce` | User appliance |
| Ruijie | `device_type:gns3_ruijie_telnet` | User appliance |
| **VPCS** | `device_type:gns3_vpcs_telnet` | **Built-in ✅** |
**VPCS Built-in Template:**
- VPCS has a **built-in template** with pre-configured `device_type` tag
- The tag is **automatically applied** when creating VPCS nodes
- No manual configuration required — works out of the box
- Other devices require users to import appliances and configure tags manually
### Nornir Best Practice: Host-Level Connection Configuration
The system uses **Nornir's configuration priority** (host > group > defaults) to handle multi-vendor environments efficiently:
```python
# From get_gns3_device_port.py
hosts_data[device_name] = {
"port": console_port,
"groups": ["network_devices"], # All devices share one group
"connection_options": {
"netmiko": {
"extras": {"device_type": device_type} # Device-specific driver
}
},
}
```
**Why Host-Level `connection_options`?**
- ✅ Each device has its own `device_type` (host-level config)
- ✅ All devices share common settings via group inheritance (`hostname`, `timeout`)
- ✅ No need to dynamically create multiple groups for each device type
- ✅ Cleaner code structure — single generic group for all devices
- ✅ Follows Nornir best practice: "configuration proximity"
**Configuration Priority:**
```mermaid
graph TD
A["Host Level<br/>connection_options.device_type<br/>(gns3_huawei_telnet_ce, cisco_ios_telnet, etc.)"]
B["Group Level<br/>hostname=127.0.0.1, timeout=120<br/>username, password"]
C["Defaults Level<br/>data.location=gns3"]
A -->|overrides| B
B -->|overrides| C
```
| Aspect | Old: Dynamic Groups | New: Host-Level Config |
|--------|--------------------|-----------------------|
| Group count | One per vendor type | Single `network_devices` group |
| Device type location | Group `connection_options` | Host `connection_options` |
| Complexity | Dynamic group creation logic | No dynamic group logic |
## Architecture Evolution
### Problem: Multi-Vendor Device Support
**Initial Challenge:**
```mermaid
graph TD
A[Topology: Cisco R1 + Huawei SW1 + Juniper SRX] --> B[Need: Different Netmiko drivers per device]
B --> C[Question: How to configure Nornir for multiple device types?]
```
### Solution Evolution
| Approach | Strategy | Issue |
|----------|----------|-------|
| ❌ Approach 1 | Single group with first device's `device_type` | All devices use wrong driver |
| ❌ Approach 2 | Dynamic groups per vendor (`huawei_telnet`, `cisco_telnet`, ...) | Complex logic, code duplication, not Nornir best practice |
| ✅ Approach 3 (Current) | **Host-level `connection_options`** + single generic group | Clean, follows Nornir host > group > defaults priority |
> See `get_gns3_device_port.py:get_device_ports_from_topology()` for the current implementation.
### Configuration Priority Demonstration
```mermaid
graph TD
A["Host Level (highest)<br/>device_type: gns3_huawei_telnet_ce"]
B["Group Level (middle)<br/>hostname: 127.0.0.1, timeout: 120"]
C["Defaults Level (lowest)<br/>data.location: gns3"]
A -->|overrides| B -->|overrides| C
D[Result] --> D1["Each device uses its own device_type"]
D --> D2["All share hostname, timeout from group"]
D --> D3["All share data.location from defaults"]
```
## Usage Examples
### Direct Netmiko Usage
**Custom driver auto-registers on import. No username/password needed for GNS3 devices.**
| Device Type | `device_type` | Auth | Notes |
|-------------|---------------|------|-------|
| Huawei CE | `gns3_huawei_telnet_ce` | None | Custom driver, skip auth |
| Ruijie | `gns3_ruijie_telnet` | None | Custom driver, interactive prompts handled |
| VPCS | `gns3_vpcs_telnet` | None | Custom driver, no config mode |
| Cisco IOS | `cisco_ios_telnet` | Required | Standard Netmiko driver |
### GNS3 Copilot Tool Usage
> Tools: `tools_v2/display_tools_nornir.py`, `tools_v2/config_tools_nornir.py`, `tools_v2/vpcs_tools_netmiko.py`
Each tool reads node tags to determine `device_type`, builds host-level Nornir inventory, and dispatches commands to the appropriate Netmiko driver. Multi-vendor topologies are handled transparently — each device uses its own driver.
## Module Structure
```
gns3server/agent/gns3_copilot/
├── utils/
│ ├── custom_netmiko/ # Custom Netmiko drivers package
│ │ ├── __init__.py # Auto-registers all drivers on import
│ │ ├── huawei_ce.py # Huawei CloudEngine driver
│ │ ├── ruijie_telnet.py # Ruijie enhanced driver
│ │ ├── vpcs_telnet.py # VPCS simulator driver
│ │ ├── README.md # Driver development guide
│ │ └── tests/ # Unit tests (52 total)
│ │ ├── __init__.py
│ │ ├── test_huawei_ce.py # Huawei CE driver tests (9)
│ │ ├── test_ruijie_telnet.py # Ruijie driver tests (15)
│ │ └── test_vpcs_telnet.py # VPCS driver tests (28)
│ └── get_gns3_device_port.py # Device port extraction
│ └── get_device_ports_from_topology() # Extract ports + device_type from tags
├── tools_v2/
│ ├── display_tools_nornir.py # Multi-vendor display commands
│ ├── config_tools_nornir.py # Multi-vendor config commands
│ │ └── _expand_multiline_commands() # Expand banner commands
│ └── vpcs_tools_netmiko.py # VPCS commands using Nornir + Netmiko
│ └── VPCSCommands # VPCS tool class
```
**Key Architectural Changes (2026-03-14):**
- ❌ Removed: `vpcs_tools_telnetlib3.py` - Replaced with Netmiko implementation
- ✅ Added: `vpcs_telnet.py` - Custom VPCS driver for Netmiko
- ✅ Added: `vpcs_tools_netmiko.py` - VPCS tool using Nornir + Netmiko
- ✅ Simplified: Unified tool architecture - all tools use Nornir + Netmiko
- ✅ Updated: Module structure - all tools follow same pattern
**Key Architectural Changes (2026-03-13):**
- ❌ Removed: `_get_nornir_groups_config()` - No longer needed
- ❌ Removed: `_get_nornir_group()` - No longer needed
- ✅ Simplified: `_initialize_nornir()` - Uses single generic group
- ✅ Updated: `get_gns3_device_port.py()` - Returns host-level `connection_options`
- ✅ Added: `ruijie_telnet.py` - Custom Ruijie driver with interactive prompt handling
- ✅ Added: `_expand_multiline_commands()` - Auto-expands banner and multi-line commands
- ✅ Added: device_type tag validation with error feedback (inline in `_run`)
## Unit Testing
### Test Coverage
> Test files located at: `gns3server/agent/gns3_copilot/utils/custom_netmiko/tests/`
**Total: 52 tests across 3 test files**
| Test File | Tests | Coverage Focus |
|-----------|-------|----------------|
| `test_huawei_ce.py` | 9 | Registration, inheritance, VRP methods, mock connection |
| `test_ruijie_telnet.py` | 15 | Registration, interactive patterns, preprocessing, batch/fallback |
| `test_vpcs_telnet.py` | 28 | Registration, login, command sending, ANSI stripping, config mode |
#### Huawei CE Tests (`TestHuaweiTelnetCEDriver` + `TestHuaweiTelnetCEIntegration`)
- Device type registration in CLASS_MAPPER
- Inheritance from HuaweiBase
- VRP-specific methods (`config_mode`, `check_config_mode`, `exit_config_mode`)
- Prompt pattern constants
- Mock telnet connection
#### Ruijie Tests (`TestRuijieTelnetEnhancedDriver` + `TestRuijieTelnetEnhancedIntegration`)
- Device type registration
- Inheritance from RuijieOSBase
- Interactive pattern matching (`router-id`, `erase`, `delete`, etc.)
- Preprocessing of interactive commands
- Batch and fallback send strategies
#### VPCS Tests (`TestVPCSTelnetDriver` + 5 more test classes)
- Device type registration
- Inheritance from BaseConnection
- No-auth login with `PC\d+>` prompt
- `send_command` / `send_command_timing` overrides
- ANSI escape code stripping (8 dedicated tests)
- Config mode always returns empty/false
**Running Tests:**
```bash
source venv/bin/activate
python -m pytest gns3server/agent/gns3_copilot/utils/custom_netmiko/tests/
```
**Current Test Status:** ✅ All 52 tests passing
## Related Documentation
- [Custom Netmiko README](../../../../../gns3server/agent/gns3_copilot/utils/custom_netmiko/README.md) - Driver development guide
- [Node Control Tools](./node-control-tools.md) - Device lifecycle management
- [Command Security](./command-security.md) - Command filtering and validation
- [Chat API](./chat-api.md) - Session management and SSE
## References
- [Netmiko Documentation](https://ktbyers.github.io/netmiko/)
- [Netmiko PLATFORMS.md](https://github.com/ktbyers/netmiko/blob/master/PLATFORMS.md)
- [Nornir Documentation](https://nornir.readthedocs.io/)
---
_Implementation Date: 2026-03-12_
_Last Updated: 2026-04-20 (Documentation review: fixed VPCS template tags, updated test coverage, added Mermaid diagrams, removed code bloat)_
_Status: ✅ Implemented - Custom drivers for Huawei, Ruijie, and VPCS; multi-vendor support with Cisco IOS, Huawei, Ruijie, and VPCS tested_
_Architecture: Nornir best practice - host-level connection_options with single generic group_
_Unit Tests: ✅ 52/52 passing (9 Huawei + 15 Ruijie + 28 VPCS)_
_Changelog:_
- **2026-03-14**: Added VPCS support and unified tool architecture
- Implemented `VPCSTelnet` custom Netmiko driver for VPCS simulator
- Replaced `vpcs_tools_telnetlib3.py` with `vpcs_tools_netmiko.py`
- Unified all tools to use Nornir + Netmiko architecture
- Removed dependency on telnetlib3 for VPCS devices
- Improved code consistency and maintainability
- **2026-03-13 (Evening)**: Added Ruijie enhanced driver and interactive command handling
- Implemented `RuijieTelnetEnhanced` with hybrid batch/fallback strategy
- Added automatic `yes` insertion for known interactive commands (`router-id`, `erase`, etc.)
- Achieves ~2-3 seconds for 13 commands (vs 10+ seconds for one-by-one)
- Auto-fallback to real-time detection for unknown interactive prompts
- Registered `gns3_ruijie_telnet` device type
- **2026-03-13 (Afternoon)**: Enhanced configuration safety and command handling
- Added AAA/password configuration prohibition in system prompts
- Implemented multi-line command expansion (`_expand_multiline_commands()`)
- Added `device_type` tag validation with error feedback
- Prevents execution of commands that could lock users out of devices
- Properly handles banner and other multi-line configuration commands
- **2026-03-13 (Morning)**: Refactored to use host-level `connection_options` instead of dynamic groups
- Removed `_get_nornir_groups_config()` and `_get_nornir_group()` helper functions
- Simplified `_initialize_nornir()` to use single generic group
- Updated `get_gns3_device_port.py()` to return host-level configuration
- **2026-03-12**: Initial implementation with custom Huawei driver
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,74 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
# GNS3-Copilot Real-time Packet AI Analysis Overview
## Core Flow
```mermaid
flowchart TB
subgraph "① Analysis Trigger & Knowledge Query"
A["User asks\n'e.g. Analyze OSPF neighbor state'"] --> B["LLM calls\nPacketAnalysisSkillsTool"]
B --> C{"Protocol Knowledge Repository\ngns3/gns3-skills"}
C --> D["Returns protocol definition\nfields/base_filter/check_rules"]
B --> E["LLM calls\nsearch_fields mode"]
E --> F["tshark -G fields\nfield name search"]
F --> G["Returns valid field names"]
end
subgraph "② Live Capture & Analysis"
D --> H["LLM constructs tshark_args"]
G --> H
H --> I["PacketAnalysisTool\ncapture analysis mode"]
I --> J["GET /capture/file\ndownload live PCAP"]
J --> K["Pre-validate -e field names"]
K --> L["tshark -r pcap\nrun analysis"]
L --> M["Return analysis results"]
end
```
## Tool Overview
| Tool | Source File | Purpose | Available Modes |
|---|---|---|---|
| `PacketAnalysisTool` | `packet_analysis_tool.py` | Download live PCAP + tshark analysis | teaching / lab_automation |
| `PacketAnalysisSkillsTool` | `registry.py` (skills module) | Query protocol-level analysis knowledge (fields, filters) | teaching / lab_automation |
## Agent Workflow (LangGraph)
```mermaid
sequenceDiagram
participant U as User
participant LLM as LLM Node
participant Skills as PacketAnalysisSkillsTool
participant Pcap as PacketAnalysisTool
U->>LLM: OSPF neighbors can't establish, analyze
LLM->>Skills: get protocol=ospf
Skills-->>LLM: OSPF fields, filter definitions
LLM->>Pcap: search_fields query=ospf.hello
Pcap-->>LLM: valid -e field names
LLM->>Pcap: download PCAP + tshark_args
Pcap-->>LLM: tshark output results
LLM->>LLM: analysis reveals Dead interval mismatch
LLM-->>U: OSPF Dead interval mismatch detected
```
## Server Capture API
| Endpoint | Function |
|---|---|
| `POST /v3/projects/{pid}/links/{lid}/capture/start` | Start packet capture on a link |
| `POST /v3/projects/{pid}/links/{lid}/capture/stop` | Stop packet capture |
| `GET /v3/projects/{pid}/links/{lid}/capture/file` | Download PCAP file (available even while capture is active) |
| `GET /v3/projects/{pid}/links/{lid}/capture/stream` | Stream PCAP data |
| `WS /v3/projects/{pid}/links/{lid}/capture/web-wireshark` | Web Wireshark WebSocket proxy |
## Key Design Points
1. **LLM-driven Analysis** — The LLM constructs tshark parameters itself; the framework does not hardcode protocol logic, only performs safety validation
2. **Live PCAP** — Captures can be downloaded and analyzed while running, no need to stop capturing
3. **Dual Knowledge Sources** — External repository provides protocol-specific knowledge; local tshark field registry provides exact field names
4. **Safety First** — Pre-validation of tshark field names prevents execution failures from invalid fields
@@ -0,0 +1,182 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# Protocol-Oriented Packet Analysis
## Overview
GNS3 Copilot provides protocol-oriented packet analysis that allows the AI assistant to diagnose network issues from live GNS3 captures. Protocol definitions (tshark fields, display filters, check rules) are stored as YAML files in the external GNS3-Skills repository and loaded into memory at startup.
The system exposes two LangChain tools: `PacketAnalysisSkillsTool` queries protocol definitions, and `PacketAnalysisTool` runs tshark against downloaded pcap files using the LLM-constructed arguments.
## Architecture
```mermaid
graph TD
subgraph "GNS3-Skills Repository"
YAML[packet_analysis/*.yaml<br/>40+ protocol files]
end
subgraph "GNS3 Server"
SM[SkillsManager]
SL[SkillsLoader]
REG[PACKET_ANALYSIS_REGISTRY<br/>in-memory dict]
SKILL[PacketAnalysisSkillsTool<br/>query protocol definitions]
TOOL[PacketAnalysisTool<br/>run tshark on captures]
end
subgraph "tshark"
FIELDS["tshark -G fields"<br/>live field registry]
CAP["tshark -r pcap"<br/>packet capture analysis]
end
YAML -->|load at startup / reload| SL
SL --> REG
SM --> SL
SKILL -->|read protocol fields/filters| REG
TOOL -->|validate -e fields| FIELDS
TOOL -->|download pcap + run tshark| CAP
```
## Supported Protocols
Definitions are loaded from `GNS3-Skills/packet_analysis/*.yaml`. The following protocol families are covered:
| Category | Protocols |
|----------|-----------|
| Network Layer | ip, ipv6, arp, icmp, icmpv6 |
| Transport Layer | tcp, udp |
| Data Link | ethernet, ppp, hdlc, frame_relay, vlan, isl, llc |
| Routing | ospf, eigrp, rip, bgp, isis, pim, dvmrp |
| Link Protocols | l2tp, lacp, pagp, udld |
| Infrastructure | cdp, lldp, stp, vtp, dtp |
| Management | snmp, telnet, ssh, radius, tacacs |
| Application | dns, http, bootp, dhcp |
| Tunneling / Security | gre, esp, ah, mpls, eapol, ssl, isakmp |
| Miscellaneous | nbns, slarp, ocsp, wccp, auto_rp, loop |
Each protocol YAML contains:
```yaml
name: "OSPF Packet Analysis"
description: "Analyze OSPF routing protocol packets"
protocol_key: "ospf"
display_filter: "ospf"
fields:
- label: "Source IP"
tshark_field: "ip.src"
description: "Source IPv4 address"
- label: "OSPF Message Type"
tshark_field: "ospf.msg"
description: "1=Hello, 2=DBD, 3=LSR, 4=LSU, 5=LSAck"
filter_examples:
- description: "Show OSPF Hello packets"
filter: "ospf.msg == 1"
checks:
- name: hello_dead_mismatch
severity: critical
message: "Hello/Dead Interval mismatch between neighbors"
```
## Analysis Flow
```mermaid
sequenceDiagram
participant LLM as LLM Agent
participant SKILL as PacketAnalysisSkillsTool
participant TOOL as PacketAnalysisTool
participant tshark as tshark
participant GNS3 as GNS3 Server
LLM->>SKILL: {"action": "list"}
SKILL-->>LLM: Available protocols
LLM->>SKILL: {"action": "get", "protocol": "ospf"}
SKILL-->>LLM: Fields, filters, check rules
Note over LLM: LLM constructs tshark_args<br/>using protocol knowledge
LLM->>TOOL: {project_id, link_id,<br/>tshark_args: "-Y ospf -T fields -e ip.src -e ospf.msg"}
TOOL->>TOOL: Validate -e field names<br/>against tshark -G fields
TOOL->>GNS3: Download pcap for link_id
GNS3-->>TOOL: Capture file
TOOL->>tshark: tshark -r pcap -Y ospf -T fields -e ip.src -e ospf.msg
tshark-->>TOOL: Tab-separated output
TOOL-->>LLM: Raw tshark output
```
## Tool Registration
The packet analysis tools are available in two copilot modes:
| Tool | Teaching Assistant | Lab Automation | Troubleshooting Injection |
|------|:------------------:|:--------------:|:-------------------------:|
| `PacketAnalysisTool` | Yes | Yes | No |
| `PacketAnalysisSkillsTool` | Yes | Yes | No |
## Tool Interface
### PacketAnalysisSkillsTool (`packet_analysis_skills`)
Queries protocol definitions from the `PACKET_ANALYSIS_REGISTRY`. Used before running tshark to look up valid field names, display filters, and anomaly checks.
| Action | Input | Output |
|--------|-------|--------|
| List protocols | `{"action": "list"}` | `{count, protocols: [{protocol, name, description}]}` |
| Get protocol | `{"action": "get", "protocol": "ospf"}` | Protocol definition with fields, filters, checks |
### PacketAnalysisTool (`packet_analysis`)
Runs tshark against a downloaded GNS3 capture file. Supports two modes:
**Capture analysis mode:**
| Parameter | Required | Description |
|-----------|----------|-------------|
| `project_id` | Yes | UUID of the GNS3 project |
| `link_id` | Yes | UUID of the link to analyze |
| `tshark_args` | Yes | tshark arguments (after `-r <pcap>`) |
**Field search mode:**
| Parameter | Required | Description |
|-----------|----------|-------------|
| `action` | Yes | `"search_fields"` |
| `query` | Yes | Single keyword (e.g., `"ospf.lsa"`, `"bgp.open"`) |
### Validation and Error Handling
Before downloading the capture, the tool validates `-e` field names against the live tshark field registry (`tshark -G fields`). Invalid field names are rejected early with a hint to use `search_fields`.
| Scenario | Response |
|----------|----------|
| Invalid `-e` field name | `{error, hint, invalid_fields}` |
| tshark filter/field error | `{error: "tshark argument error", hints}` |
| Empty capture file | `{error: "Capture file is empty"}` |
| No matching packets | `{result: "No matching packets found", hints}` |
| tshark timeout (30s) | `{error: "tshark timeout after 30 seconds"}` |
| tshark not installed | `{error: "tshark not installed"}` |
When `-c` is used and no results are found, the tool hints that `-c` limits total packets read (not matched count), and suggests removing it or piping to `head`.
## Hot Reload
Packet analysis protocols can be reloaded without restarting the server via the existing reload API:
```
POST /v3/copilot/reload/skills
```
This triggers `SkillsManager.reload_packet_analysis_protocols()`, which re-reads all YAML files from the `packet_analysis/` directory and updates `PACKET_ANALYSIS_REGISTRY` in place.
## Related Documentation
- [External Skills Repository](skills-repository.md)
- [Fault Injection](fault-injection.md)
- [Chat API](chat-api.md)
@@ -0,0 +1,171 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# External Skills Repository
## Overview
GNS3 Copilot loads all skills, prompts, and security configurations from an external Git repository at [github.com/gns3/gns3-skills](https://github.com/gns3/gns3-skills). This enables dynamic updates without server redeployment.
The repository provides:
- **Injection skills** (39 categories): Network fault scenarios for troubleshooting practice
- **Device skills**: Device-specific command knowledge (VPCS, etc.)
- **Feature skills**: Topology planning, network design
- **System prompts**: Agent personality and behavior definitions
- **Forbidden commands**: Security rules for command filtering
## Architecture
```mermaid
graph TD
subgraph "GNS3-Skills Repository"
YAML[injection/*.yaml<br/>device/*.yaml<br/>feature/*.yaml]
MD[prompts/*.md]
CFG[config/forbidden_commands.txt]
end
subgraph "GNS3 Server"
SM[SkillsManager]
SL[SkillsLoader]
REG[Registry<br/>SKILLS_REGISTRY<br/>INJECTION_SKILLS_REGISTRY]
PROMPT[PROMPTS_CACHE]
FC[command_filter]
end
YAML --> SL
MD --> SL
CFG --> FC
SL --> REG
SL --> PROMPT
SM --> SL
SM -->|git pull| YAML
```
## Repository Structure
```
GNS3-Skills/
├── injection/ # 39 YAML files, one per protocol/category
│ ├── ospf_issues.yaml
│ ├── bgp_issues.yaml
│ ├── vlan_issues.yaml
│ └── ...
├── device/ # Device-specific skills
│ └── vpcs.yaml
├── feature/ # Feature skills
│ └── topology_planner.yaml
├── prompts/ # System prompts (Markdown)
│ ├── teaching_assistant.md
│ ├── lab_automation_assistant.md
│ ├── troubleshooting_injection.md
│ └── title.md
└── config/ # Security configuration
└── forbidden_commands.txt
```
## Configuration
Skills repository settings are configured in `gns3_server.conf` under the `[Server]` section:
```ini
[Server]
skills_repo_url = https://github.com/gns3/gns3-skills.git
skills_repo_branch = main
skills_auto_update = true
```
| Setting | Default | Description |
|---------|---------|-------------|
| `skills_repo_url` | `https://github.com/gns3/gns3-skills.git` | Git repository URL |
| `skills_repo_branch` | `main` | Git branch to track |
| `skills_auto_update` | `true` | Automatically pull on reload |
## Initialization Flow
```mermaid
sequenceDiagram
participant Server
participant agent/__init__.py
participant SkillsManager
participant Git
Server->>agent/__init__.py: Import module
agent/__init__.py->>SkillsManager: Start background init thread
Note over SkillsManager: Thread.join(5s timeout)
SkillsManager->>Git: Check local repo exists?
alt No local repo
Git->>SkillsManager: git clone (with timeout env)
else Local repo exists
SkillsManager->>Git: Check uncommitted changes?
alt Has uncommitted changes
Git-->>SkillsManager: Warn, skip pull
else No changes
SkillsManager->>Git: git fetch (timeout 10s)
SkillsManager->>Git: Behind remote?
alt Behind
Git->>SkillsManager: git pull
else Up to date
Git-->>SkillsManager: Nothing to do
end
end
end
SkillsManager->>SkillsManager: reload_skills() - load YAML files
SkillsManager->>SkillsManager: reload_prompts() - load Markdown files
```
### Git Timeout Configuration
Git operations use per-command environment variables to prevent hanging:
```python
_GIT_TIMEOUT_ENV = {
'GIT_HTTP_TIMEOUT': '10', # Connection timeout (default: 120s)
'GIT_HTTP_LOW_SPEED_TIME': '5', # Slow speed threshold window
'GIT_HTTP_LOW_SPEED_LIMIT': '1000', # < 1 KB/s = slow → abort
}
```
These apply only to the specific `clone`/`fetch`/`pull` subprocess, not to the global environment.
## API Endpoint
### POST /copilot/reload/skills
Triggers a full reload of the skills repository. Performs one git update check, then reloads all skills, prompts, and forbidden commands from local files.
**Response:**
```json
{
"success": true,
"skills": true,
"skill_count": 39,
"prompts": true,
"prompt_count": 4,
"forbidden_commands": 6,
"version": "abc123def456..."
}
```
| Field | Description |
|-------|-------------|
| `success` | Overall success (true if skills or prompts loaded) |
| `skills` | Skills reload result |
| `skill_count` | Number of injection skills loaded |
| `prompts` | Prompts reload result |
| `prompt_count` | Number of prompts loaded |
| `forbidden_commands` | Number of forbidden command patterns |
| `version` | Git commit hash of the repository |
## Related Documentation
- [Fault Injection](fault-injection.md)
- [Command Security](command-security.md)
- [Chat API](chat-api.md)
+255
View File
@@ -0,0 +1,255 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# Netmiko Supported Devices
**Netmiko Version:** 4.6.0
**Generated:** 2026-03-14 13:10:00
**Last Updated:** 2026-03-14 (Added VPCS support)
---
## Summary
- **Total Device Types:** 366
- **SSH Devices:** 154
- **Telnet Devices:** 55
- **Custom Devices:** 3
## Custom GNS3 Drivers
GNS3-Copilot includes custom Netmiko drivers optimized for GNS3 emulation:
| Platform | Device Type | Description |
|----------|-------------|-------------|
| Huawei CE | `gns3_huawei_telnet_ce` | Huawei CloudEngine driver with no authentication |
| Ruijie | `gns3_ruijie_telnet` | Ruijie OS enhanced driver with interactive command handling |
| VPCS | `gns3_vpcs_telnet` | VPCS simulator driver with ANSI code stripping |
## SSH Supported Devices
| Platform | Device Type | Source |
|----------|-------------|--------|
| A10 | `a10_ssh` | Netmiko |
| Accedian | `accedian_ssh` | Netmiko |
| Adtran | `adtran_os_ssh` | Netmiko |
| Adva | `adva_fsp150f2_ssh` | Netmiko |
| Adva | `adva_fsp150f3_ssh` | Netmiko |
| Alaxala | `alaxala_ax26s_ssh` | Netmiko |
| Alaxala | `alaxala_ax36s_ssh` | Netmiko |
| Alcatel | `alcatel_aos_ssh` | Netmiko |
| Alcatel | `alcatel_sros_ssh` | Netmiko |
| Allied | `allied_telesis_awplus_ssh` | Netmiko |
| Apresia | `apresia_aeos_ssh` | Netmiko |
| Arista | `arista_eos_ssh` | Netmiko |
| Arris | `arris_cer_ssh` | Netmiko |
| Aruba | `aruba_aoscx_ssh` | Netmiko |
| Aruba | `aruba_os_ssh` | Netmiko |
| Aruba | `aruba_osswitch_ssh` | Netmiko |
| Aruba | `aruba_procurve_ssh` | Netmiko |
| Asterfusion | `asterfusion_asternos_ssh` | Netmiko |
| Audiocode | `audiocode_66_ssh` | Netmiko |
| Audiocode | `audiocode_72_ssh` | Netmiko |
| Audiocode | `audiocode_shell_ssh` | Netmiko |
| Avaya | `avaya_ers_ssh` | Netmiko |
| Avaya | `avaya_vsp_ssh` | Netmiko |
| Bintec | `bintec_boss_ssh` | Netmiko |
| Broadcom | `broadcom_icos_ssh` | Netmiko |
| Brocade | `brocade_fastiron_ssh` | Netmiko |
| Brocade | `brocade_fos_ssh` | Netmiko |
| Brocade | `brocade_netiron_ssh` | Netmiko |
| Brocade | `brocade_nos_ssh` | Netmiko |
| Brocade | `brocade_vdx_ssh` | Netmiko |
| Brocade | `brocade_vyos_ssh` | Netmiko |
| Calix | `calix_b6_ssh` | Netmiko |
| Casa | `casa_cmts_ssh` | Netmiko |
| Cdot | `cdot_cros_ssh` | Netmiko |
| Centec | `centec_os_ssh` | Netmiko |
| Check Point | `checkpoint_gaia_ssh` | Netmiko |
| Ciena | `ciena_saos10_ssh` | Netmiko |
| Ciena | `ciena_saos_ssh` | Netmiko |
| Ciena | `ciena_waveserver_ssh` | Netmiko |
| Cisco | `cisco_apic_ssh` | Netmiko |
| Cisco | `cisco_asa_ssh` | Netmiko |
| Cisco | `cisco_ftd_ssh` | Netmiko |
| Cisco | `cisco_ios_ssh` | Netmiko |
| Cisco | `cisco_nxos_ssh` | Netmiko |
| Cisco | `cisco_s200_ssh` | Netmiko |
| Cisco | `cisco_s300_ssh` | Netmiko |
| Cisco | `cisco_tp_ssh` | Netmiko |
| Cisco | `cisco_viptela_ssh` | Netmiko |
| Cisco | `cisco_wlc_ssh` | Netmiko |
| Cisco | `cisco_xe_ssh` | Netmiko |
| Cisco | `cisco_xr_ssh` | Netmiko |
| Cloudgenix | `cloudgenix_ion_ssh` | Netmiko |
| Corelight | `corelight_linux_ssh` | Netmiko |
| Coriant | `coriant_ssh` | Netmiko |
| Cumulus | `cumulus_linux_ssh` | Netmiko |
| Dell | `dell_dnos9_ssh` | Netmiko |
| Dell | `dell_force10_ssh` | Netmiko |
| Dell | `dell_isilon_ssh` | Netmiko |
| Dell | `dell_os10_ssh` | Netmiko |
| Dell | `dell_os6_ssh` | Netmiko |
| Dell | `dell_os9_ssh` | Netmiko |
| Dell | `dell_powerconnect_ssh` | Netmiko |
| Dell | `dell_sonic_ssh` | Netmiko |
| Digi | `digi_transport_ssh` | Netmiko |
| Dlink | `dlink_ds_ssh` | Netmiko |
| Edgecore | `edgecore_sonic_ssh` | Netmiko |
| Ekinops | `ekinops_ek360_ssh` | Netmiko |
| Eltex | `eltex_esr_ssh` | Netmiko |
| Eltex | `eltex_ssh` | Netmiko |
| Endace | `endace_ssh` | Netmiko |
| Enterasys | `enterasys_ssh` | Netmiko |
| Ericsson | `ericsson_ipos_ssh` | Netmiko |
| Ericsson | `ericsson_mltn63_ssh` | Netmiko |
| Ericsson | `ericsson_mltn66_ssh` | Netmiko |
| Extreme | `extreme_ers_ssh` | Netmiko |
| Extreme | `extreme_exos_ssh` | Netmiko |
| Extreme | `extreme_netiron_ssh` | Netmiko |
| Extreme | `extreme_nos_ssh` | Netmiko |
| Extreme | `extreme_slx_ssh` | Netmiko |
| Extreme | `extreme_ssh` | Netmiko |
| Extreme | `extreme_tierra_ssh` | Netmiko |
| Extreme | `extreme_vdx_ssh` | Netmiko |
| Extreme | `extreme_vsp_ssh` | Netmiko |
| Extreme | `extreme_wing_ssh` | Netmiko |
| F5 | `f5_linux_ssh` | Netmiko |
| F5 | `f5_ltm_ssh` | Netmiko |
| F5 | `f5_tmsh_ssh` | Netmiko |
| Fiberstore | `fiberstore_fsos_ssh` | Netmiko |
| Fiberstore | `fiberstore_fsosv2_ssh` | Netmiko |
| Fiberstore | `fiberstore_networkos_ssh` | Netmiko |
| Flexvnf | `flexvnf_ssh` | Netmiko |
| Fortinet | `fortinet_ssh` | Netmiko |
| Garderos | `garderos_grs_ssh` | Netmiko |
| Generic | `generic_ssh` | Netmiko |
| Generic | `generic_termserver_ssh` | Netmiko |
| H3C (华三) | `h3c_comware_ssh` | Netmiko |
| HP | `hp_comware_ssh` | Netmiko |
| HP | `hp_procurve_ssh` | Netmiko |
| Hillstone | `hillstone_stoneos_ssh` | Netmiko |
| Huawei | `huawei_olt_ssh` | Netmiko |
| Huawei | `huawei_smartax_ssh` | Netmiko |
| Huawei | `huawei_smartaxmmi_ssh` | Netmiko |
| Huawei | `huawei_ssh` | Netmiko |
| Huawei | `huawei_vrp_ssh` | Netmiko |
| Huawei | `huawei_vrpv8_ssh` | Netmiko |
| Infinera | `infinera_packet_ssh` | Netmiko |
| Ipinfusion | `ipinfusion_ocnos_ssh` | Netmiko |
| Juniper | `juniper_junos_ssh` | Netmiko |
| Juniper | `juniper_screenos_ssh` | Netmiko |
| Juniper | `juniper_ssh` | Netmiko |
| Keymile | `keymile_nos_ssh` | Netmiko |
| Keymile | `keymile_ssh` | Netmiko |
| Lancom | `lancom_lcossx4_ssh` | Netmiko |
| Linux | `linux_ssh` | Netmiko |
| Maipu (迈普) | `maipu_ssh` | Netmiko |
| Mellanox | `mellanox_mlnxos_ssh` | Netmiko |
| Mellanox | `mellanox_ssh` | Netmiko |
| Mikrotik | `mikrotik_routeros_ssh` | Netmiko |
| Mikrotik | `mikrotik_switchos_ssh` | Netmiko |
| Mrv | `mrv_lx_ssh` | Netmiko |
| Mrv | `mrv_optiswitch_ssh` | Netmiko |
| Nec | `nec_ix_ssh` | Netmiko |
| Netapp | `netapp_cdot_ssh` | Netmiko |
| Netgear | `netgear_prosafe_ssh` | Netmiko |
| Netscaler | `netscaler_ssh` | Netmiko |
| Nokia | `nokia_srl_ssh` | Netmiko |
| Nokia | `nokia_sros_ssh` | Netmiko |
| Oneaccess | `oneaccess_oneos_ssh` | Netmiko |
| Ovs | `ovs_linux_ssh` | Netmiko |
| Palo Alto | `paloalto_panos_ssh` | Netmiko |
| Pluribus | `pluribus_ssh` | Netmiko |
| Quanta | `quanta_mesh_ssh` | Netmiko |
| Rad | `rad_etx_ssh` | Netmiko |
| Raisecom | `raisecom_roap_ssh` | Netmiko |
| Ruckus | `ruckus_fastiron_ssh` | Netmiko |
| Ruijie (锐捷) | `ruijie_os_ssh` | Netmiko |
| Silverpeak | `silverpeak_vxoa_ssh` | Netmiko |
| Sixwind | `sixwind_os_ssh` | Netmiko |
| Sophos | `sophos_sfos_ssh` | Netmiko |
| Supermicro | `supermicro_smis_ssh` | Netmiko |
| Telcosystems | `telcosystems_binos_ssh` | Netmiko |
| Teldat | `teldat_cit_ssh` | Netmiko |
| Tplink | `tplink_jetstream_ssh` | Netmiko |
| Ubiquiti | `ubiquiti_edge_ssh` | Netmiko |
| Ubiquiti | `ubiquiti_edgerouter_ssh` | Netmiko |
| Ubiquiti | `ubiquiti_edgeswitch_ssh` | Netmiko |
| Ubiquiti | `ubiquiti_unifiswitch_ssh` | Netmiko |
| Vertiv | `vertiv_mph_ssh` | Netmiko |
| Vyatta | `vyatta_vyos_ssh` | Netmiko |
| Vyos | `vyos_ssh` | Netmiko |
| Watchguard | `watchguard_fireware_ssh` | Netmiko |
| Yamaha | `yamaha_ssh` | Netmiko |
| ZTE (中兴) | `zte_zxros_ssh` | Netmiko |
| Zyxel | `zyxel_os_ssh` | Netmiko |
## Telnet Supported Devices
| Platform | Device Type | Source |
|----------|-------------|--------|
| Adtran | `adtran_os_telnet` | Netmiko |
| Apresia | `apresia_aeos_telnet` | Netmiko |
| Arista | `arista_eos_telnet` | Netmiko |
| Aruba | `aruba_procurve_telnet` | Netmiko |
| Audiocode | `audiocode_66_telnet` | Netmiko |
| Audiocode | `audiocode_72_telnet` | Netmiko |
| Audiocode | `audiocode_shell_telnet` | Netmiko |
| Bintec | `bintec_boss_telnet` | Netmiko |
| Brocade | `brocade_fastiron_telnet` | Netmiko |
| Brocade | `brocade_netiron_telnet` | Netmiko |
| Calix | `calix_b6_telnet` | Netmiko |
| Centec | `centec_os_telnet` | Netmiko |
| Ciena | `ciena_saos_telnet` | Netmiko |
| Cisco | `cisco_ios_telnet` | Netmiko |
| Cisco | `cisco_nxos_telnet` | Netmiko |
| Cisco | `cisco_s200_telnet` | Netmiko |
| Cisco | `cisco_s300_telnet` | Netmiko |
| Cisco | `cisco_xr_telnet` | Netmiko |
| Dell | `dell_dnos6_telnet` | Netmiko |
| Dell | `dell_powerconnect_telnet` | Netmiko |
| Dlink | `dlink_ds_telnet` | Netmiko |
| Extreme | `extreme_exos_telnet` | Netmiko |
| Extreme | `extreme_netiron_telnet` | Netmiko |
| Extreme | `extreme_telnet` | Netmiko |
| Fiberstore | `fiberstore_fsosv2_telnet` | Netmiko |
| Generic | `generic_telnet` | Netmiko |
| Generic | `generic_termserver_telnet` | Netmiko |
| Genexis | `genexis_solt33_telnet` | Netmiko |
| HP | `hp_comware_telnet` | Netmiko |
| HP | `hp_procurve_telnet` | Netmiko |
| Huawei | `huawei_olt_telnet` | Netmiko |
| Huawei | `huawei_telnet` | Netmiko |
| Huawei | `gns3_huawei_telnet_ce` | Custom ✨ (GNS3) |
| Infinera | `infinera_packet_telnet` | Netmiko |
| Ipinfusion | `ipinfusion_ocnos_telnet` | Netmiko |
| Juniper | `juniper_junos_telnet` | Netmiko |
| Maipu (迈普) | `maipu_telnet` | Netmiko |
| Nec | `nec_ix_telnet` | Netmiko |
| Nokia | `nokia_sros_telnet` | Netmiko |
| Oneaccess | `oneaccess_oneos_telnet` | Netmiko |
| Optilink | `optilink_eolt11444_telnet` | Netmiko |
| Optilink | `optilink_eolt9702_telnet` | Netmiko |
| Palo Alto | `paloalto_panos_telnet` | Netmiko |
| Rad | `rad_etx_telnet` | Netmiko |
| Raisecom | `raisecom_telnet` | Netmiko |
| Ruckus | `ruckus_fastiron_telnet` | Netmiko |
| Ruijie (锐捷) | `ruijie_os_telnet` | Netmiko |
| Ruijie (锐捷) | `gns3_ruijie_telnet` | Custom ✨ (GNS3 Enhanced) |
| Supermicro | `supermicro_smis_telnet` | Netmiko |
| VPCS | `gns3_vpcs_telnet` | Custom ✨ (GNS3 VPCS Simulator) |
| Telcosystems | `telcosystems_binos_telnet` | Netmiko |
| Teldat | `teldat_cit_telnet` | Netmiko |
| Tplink | `tplink_jetstream_telnet` | Netmiko |
| Yamaha | `yamaha_telnet` | Netmiko |
| ZTE (中兴) | `zte_zxros_telnet` | Netmiko |
---
*This document was generated automatically by the Netmiko device list script.*
@@ -0,0 +1,280 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This document is a roadmap/planning document. The described features have not been implemented yet.
# AIOps Fault Injection Testing Pipeline — Roadmap
## Overview
Build a realistic testing pipeline that duplicates the company's production network architecture into GNS3, then systematically injects network faults using the AI Copilot's fault injection capabilities to validate and train the AIOps module before production deployment.
```
┌─────────────────────────────────────────────────────────────────┐
│ AIOps Testing Pipeline │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ Network │ │ Fault │ │ AIOps │ │
│ │ Duplication │───▶│ Injection │───▶│ Validation │ │
│ │ (Phase 1) │ │ (Phase 2) │ │ (Phase 3) │ │
│ └──────────────┘ └──────────────┘ └──────────────────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ GNS3 Network │ │ Fault │ │ Results & │ │
│ │ Replica │ │ Scenarios │ │ Reporting │ │
│ └──────────────┘ └──────────────┘ └──────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
```
## Core Concept
1. **Duplicate** the company's production network architecture into a GNS3 simulation environment
2. **Select** a set of fault types to test (OSPF, BGP, VxLAN, STP, packet filter, etc.)
3. **Inject** faults automatically using the AI Copilot's fault injection capabilities
4. **Validate** whether the AIOps module correctly identifies and reports each fault
5. **Loop** through all selected fault scenarios, building a comprehensive test matrix
6. **Train** the AIOps module on results to improve accuracy before production deployment
## Phase 1: Network Architecture Duplication
### Goal
Create a high-fidelity replica of the company production network in GNS3.
### Key Tasks
- [ ] **Topology Mapping**: Document production network topology (devices, links, protocols)
- [ ] **Device Selection**: Map production devices to GNS3-compatible images (Cisco IOSv, XRv, Juniper vSRX, etc.)
- [ ] **Configuration Extraction**: Export sanitized production configs (remove passwords, public IPs, sensitive data)
- [ ] **GNS3 Deployment**: Build the network in GNS3 with accurate device placement and links
- [ ] **Config Replication**: Apply adapted configurations to GNS3 devices
- [ ] **Connectivity Validation**: Verify OSPF/BGP adjacencies, VLANs, VRFs, and end-to-end reachability
- [ ] **Baseline Capture**: Record normal operation metrics (CPU, memory, interface counters, routing tables)
### Considerations
- Sanitize all production configurations before importing into GNS3
- Use environment-specific IP addressing where necessary (loopbacks, management)
- Document all deviations from production for traceability
## Phase 2: Fault Injection Pipeline
### Goal
Systematically inject network faults and validate AIOps detection using the existing AI Copilot fault injection infrastructure.
### Components
```
┌──────────────────────────────────────────────────────────────────┐
│ Fault Injection Pipeline │
├──────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────┐ │
│ │ Scenario │ │ Inject │ │ AIOps │ │ Record │ │
│ │ Selector │──▶│ Fault │──▶│ Validate │──▶│ & Report │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────────┘ │
│ │ │ │ │ │
│ │ │ │ │ │
│ └──────────────┴──────────────┴───────────────┘ │
│ │ │
│ ▼ │
│ Loop until all scenarios tested │
└──────────────────────────────────────────────────────────────────┘
```
### 2.1 Scenario Selector
- Read fault scenarios from the GNS3-Skills repository
- Support filtering by:
- Protocol (OSPF, BGP, VxLAN, STP, VLAN, etc.)
- Severity (critical, high, medium, low)
- Difficulty (beginner, intermediate, advanced)
- Track which scenarios have been tested
- Randomize selection order to avoid bias
- Exclude previously tested scenarios
### 2.2 Fault Injection
- Use existing `manage_gns3_packet_filter` tool for network-level faults
- Use existing `execute_multiple_device_config_commands` for configuration faults
- Use existing `InjectionSkillsTool` to query and select appropriate faults
- Support combined faults (multiple simultaneous issues)
- Auto-recovery between scenarios (restore baseline state)
### 2.3 AIOps Validation
- Feed network state (after fault injection) to the AIOps module
- Record AIOps diagnosis output
- Compare AIOps results against expected fault definition:
- **Correct identification**: AIOps names the exact fault
- **Partial identification**: AIOps identifies related symptoms but not root cause
- **Missed**: AIOps fails to detect any issue
- **False positive**: AIOps reports a fault that doesn't exist
### 2.4 Test Execution Flow
```
1. Reset network to clean baseline state
2. Select next untested fault scenario
3. Inject the fault into the GNS3 network
4. Wait for convergence (configurable delay)
5. Query AIOps module for diagnosis
6. Compare AIOps output with expected fault definition
7. Record result (pass/fail/partial)
8. Restore network to baseline
9. Repeat from step 2 until all scenarios completed
```
## Phase 3: Traffic Injection (Enhanced Realism)
### Goal
Add realistic network traffic to the GNS3 simulation so that AIOps has real telemetry data to analyze, rather than a static network.
### Approaches
#### 3.1 Traffic Generators in GNS3
- Deploy traffic generator appliances in GNS3 (e.g., TRex, Ostinato, Scapy on Linux nodes)
- Generate realistic traffic patterns:
- VoIP/RTP streams
- HTTP/HTTPS web traffic
- Database replication
- Routing protocol updates (OSPF hellos, BGP keepalives)
- ICMP monitoring traffic
#### 3.2 tcpreplay with Captured Traffic
- Capture real production traffic (sanitized)
- Use `tcpreplay` to replay traffic through the GNS3 network
- More realistic than synthetic traffic generators
#### 3.3 Integration with Network Monitoring
- Feed simulated device telemetry (SNMP, syslog, NetFlow) to the AIOps module
- Enable AIOps to analyze real-time telemetry during fault conditions
- Validate that AIOps can distinguish between traffic anomalies and actual faults
## Success Metrics
| Metric | Target | Measurement Method |
|--------|--------|-------------------|
| Fault detection rate | >95% | AIOps correctly identifies injected faults |
| False positive rate | <5% | AIOps reports fault when none exists |
| Time to detection | <30s | Duration from injection to AIOps alert |
| Coverage | >80% of defined scenarios | Percentage of scenarios tested |
| Accuracy improvement | Measurable per cycle | Compare pass rates across test cycles |
## Technical Architecture
```
┌────────────────────────────────────────────────────────────────────┐
│ Test Orchestrator │
│ ┌──────────────────────────────────────────────────────────────┐ │
│ │ Test Runner (Python) │ │
│ │ - Scenario selection & scheduling │ │
│ │ - Fault injection coordination │ │
│ │ - AIOps query & result collection │ │
│ │ - Report generation │ │
│ └──────────┬────────────────────────────────────────────┬───────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌────────────────────┐ ┌────────────────────┐ │
│ │ GNS3 Controller │ │ AIOps Module │ │
│ │ (gns3-server) │ │ │ │
│ │ - Network mgmt │ │ - Fault diagnosis │ │
│ │ - Fault injection │ │ - Alert detection │ │
│ │ - State queries │ │ - Root cause │ │
│ └────────────────────┘ └────────────────────┘ │
│ │ │
│ ▼ │
│ ┌────────────────────┐ │
│ │ GNS3 Network │ │
│ │ Replica │ │
│ │ - Devices │ │
│ │ - Traffic │ │
│ │ - Telemetry │ │
│ └────────────────────┘ │
└────────────────────────────────────────────────────────────────────┘
```
## Reporting
Each test cycle produces:
- **Summary report**: Pass/fail rates, coverage, trends
- **Detailed per-scenario report**: Injection details, AIOps response, comparison
- **Regression tracker**: Which scenarios regressed since last cycle
- **Accuracy trend**: Improvement or degradation over time
### Example Report Entry
```yaml
test_cycle: 7
date: "2026-06-01"
scenarios_planned: 20
scenarios_completed: 18
failed_injections: 1
skipped: 1
results:
- scenario: ospf_hello_dead_mismatch
protocol: ospf
severity: major
injection_method: device_config
target_device: R1
aiops_detection: true
aiops_diagnosis: "OSPF Hello/Dead interval mismatch between R1 and R2"
detection_latency_ms: 12000
match: exact
- scenario: packet_loss_heavy
protocol: performance
severity: high
injection_method: packet_filter
target_link: "R1 ↔ R2 (ethernet)"
aiops_detection: true
aiops_diagnosis: "High packet loss detected on link R1-R2"
detection_latency_ms: 45000
match: partial
```
## Dependencies
- [ ] GNS3 network replica ready and validated
- [ ] AI Copilot fault injection tools operational
- [ ] AIOps module query interface available
- [ ] Traffic generation tools deployed (for Phase 3)
- [ ] Test orchestrator framework (to be built)
- [ ] Result database and reporting system
## Timeline
| Phase | Duration | Deliverable |
|-------|----------|-------------|
| P1: Network Duplication | 2-4 weeks | GNS3 replica of production network |
| P2: Fault Injection Pipeline | 2-3 weeks | Automated test runner + first results |
| P3: Traffic Injection | 2-4 weeks | Realistic traffic simulation integrated |
## Status
- [ ] P1: Network architecture duplication
- [ ] Topology mapping documented
- [ ] Device configurations sanitized and adapted
- [ ] GNS3 replica deployed
- [ ] Baseline connectivity verified
- [ ] P2: Fault injection pipeline
- [ ] Scenario selection framework
- [ ] Automated fault injection
- [ ] AIOps validation interface
- [ ] Report generation
- [ ] Loop/retry mechanism
- [ ] P3: Traffic injection
- [ ] Traffic generator deployment
- [ ] Traffic pattern library
- [ ] AIOps telemetry integration
@@ -0,0 +1,64 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This document is a roadmap/planning document. The described features have not been implemented yet.
# Injection Fault Tracking — Roadmap
## Problem
When using the AI Copilot in troubleshooting injection mode across multiple chat sessions within the same project, there is no mechanism to prevent the same fault from being injected repeatedly. Each new session starts with a clean slate, unaware of what faults have already been used. This leads to:
- Duplicate injection scenarios across different sessions
- No visibility for instructors into which faults have been used
- No record of which devices were affected or whether faults were resolved
## Proposed Solution
Track injected faults per chat session and feed the history into the LLM's system prompt as context, so the model actively avoids repeating scenarios.
### 1. Storage
The existing `chat_sessions` table has `metadata` and `stats` JSON columns. After each fault injection, the agent records:
```json
// stored in chat_session.metadata
{
"injected_fault": {
"fault_type": "injection_ospf",
"issue_key": "ospf_hello_dead_mismatch",
"issue_name": "OSPF Hello/Dead Interval Mismatch",
"device": "R1",
"severity": "major",
"injected_at": "2026-05-11T23:00:00Z",
"resolved": false
}
}
```
### 2. Context Injection
When starting a new troubleshooting session, query all previous sessions for the project, filter by `copilot_mode = "troubleshooting_injection"`, extract `injected_fault` data, and inject a summary into the system prompt via a placeholder (similar to the existing `{{topology_info}}` mechanism):
```
Previously injected faults in this project:
1. OSPF Hello/Dead Interval Mismatch on R1 (2026-05-11) — resolved
2. BGP Route Reflector misconfiguration on R3 (2026-05-10) — resolved
Select a fault that has NOT been used before.
```
### 3. Prompt Update
The `troubleshooting_injection.md` prompt in the GNS3-Skills repository gains an `{{injection_history}}` placeholder, and the agent is instructed to choose a fault not in the history list.
## Status
- [ ] GNS3-Skills: add `{{injection_history}}` placeholder to `troubleshooting_injection.md`
- [ ] gns3-server: implement injection fault recording in agent chat session flow
- [ ] gns3-server: implement injection history query across sessions per project
- [ ] gns3-server: inject history into system prompt via `context_manager.py`
- [ ] gns3-server: add `exclude` parameter to `InjectionSkillsTool` for filtering used faults
@@ -0,0 +1,621 @@
# RBAC User Isolation Roadmap
## Overview
GNS3 3.0 ships with a complete RBAC framework (ACE + Role + Privilege models), but the user isolation layer is incomplete. Users can see resources they should not have access to because:
- Resource creation does not auto-grant the creator an ACE
- List endpoints return unfiltered results
- Two GET endpoints have RBAC checks bypassed via FIXME
**This document has been updated to reflect the implemented solution in `feature/simple-user-isolation` branch.**
## Current State
| Resource | Route Check | List Filtering | Auto-ACE on Create | ACE Cleanup on Delete |
|---|---|---|---|---|
| Project | `Project.Audit/Modify/Allocate` | **Implemented** — Three-step filtering | **Not needed** | Done |
| Template | `Template.Audit` **FIXME** | None — all templates returned | **Missing** | Done |
| Node | `Node.Audit/Modify/Allocate` | N/A (inherits project) | Inherits project | N/A |
| Link | `Link.Audit/Modify/Allocate` | N/A (inherits project) | Inherits project | N/A |
| Drawing | `Drawing.Audit/Modify/Allocate` | N/A (inherits project) | Inherits project | N/A |
| Snapshot | `Snapshot.Audit/Allocate/Restore` | N/A (inherits project) | Inherits project | N/A |
| Image | `Image.Audit/Allocate` | None — all images returned | **Missing** | Missing |
| Compute | `Compute.Audit` **FIXME** | None — all computes returned | N/A (shared infra) | Done |
| Appliance | `Appliance.Audit/Allocate` | None — all appliances returned | N/A (builtin) | N/A |
| Symbol | `Symbol.Audit/Allocate` | None — all symbols returned | N/A (builtin) | N/A |
## Implemented Solution: Project Isolation
### Three-Step Permission Check Logic
**Status**: ✅ Implemented in `feature/simple-user-isolation` branch
```python
# Step 1: ACE check - basic access permission
# Get projects user has ACE for
ace_projects = []
for project in controller.projects.values():
project_path = f"/projects/{project.id}"
if await rbac_repo.check_user_has_privilege(current_user.user_id, project_path, "Project.Audit"):
ace_projects.append(project)
# Step 2: Filter ace_projects by created_by - user's own projects
# Project sharing is only available through resource pools
user_projects = [p.asdict() for p in ace_projects if p.created_by == current_user.username]
projects.extend(user_projects)
# Step 3: Resource pool projects
# Projects shared through resource pools
user_pool_resources = await rbac_repo.get_user_pool_resources(current_user.user_id, "Project.Audit")
project_ids_in_pools = [str(r.resource_id) for r in user_pool_resources if r.resource_type == "project"]
pool_projects = [p.asdict() for p in controller.projects.values() if p.id in project_ids_in_pools]
projects.extend(pool_projects)
```
### Key Design Principles
1. **ACE for basic access control**: Controls whether user can access the system
2. **created_by for user isolation**: Controls which specific resources user can access
3. **Resource pools for project sharing**: The only mechanism for sharing projects between users
4. **No direct ACE sharing**: Users cannot configure ACE directly on specific projects to share them
### Advantages of This Approach
- **Fault tolerance**: Even with broad ACE configuration (`path: "/" + propagate: true`), user isolation remains effective
- **Clear separation**: Basic access, data ownership, and team sharing are clearly separated
- **Simple mechanism**: No complex auto-ACE or seen_project_ids tracking required
- **Performance**: Leverages existing created_by field, no schema changes needed
## Updated Architecture
```mermaid
graph TD
subgraph Client
WebUI
CLI
end
subgraph "Controller API"
Auth[get_current_active_user]
Routes[Resource Routes]
ACE_Check[Step 1: ACE Check]
Owner_Filter[Step 2: Filter by created_by]
Pool_Check[Step 3: Resource Pools]
end
subgraph "RBAC Engine"
ACE[(ACE table)]
Role[(Role table)]
Privilege[(Privilege table)]
Checker[check_user_has_privilege]
end
WebUI --> Auth
CLI --> Auth
Auth --> Routes
Routes --> ACE_Check
ACE_Check --> Checker
ACE_Check --> Owner_Filter
Owner_Filter --> Pool_Check
Pool_Check --> Checker
Checker --> ACE
Checker --> Role
Role --> Privilege
```
## Business Process
### Project listing with three-step filtering
```mermaid
sequenceDiagram
actor U as User
participant API as GET /projects
participant Controller as Controller
participant RBAC as RBAC Engine
U->>API: List projects
API->>API: get_current_active_user (not superadmin)
API->>RBAC: Step 1: ACE check on each project
loop Each project
RBAC-->>API: ACE results
end
API->>API: Step 2: Filter by created_by
API-->>U: User's own projects
API->>RBAC: Step 3: Resource pool projects
RBAC-->>API: Pool projects
API-->>U: Combined list (own + pool)
```
## Updated Phased Plan
### ✅ Phase 1 — MVP: Project isolation (COMPLETED)
**Goal**: Users only see projects they created or were granted access to through resource pools.
| Task | Files | Status | Detail |
|---|---|---|---|
| Fix project list filtering | `projects.py` — `get_projects()` | ✅ **Implemented** | Three-step filtering: ACE check → created_by filter → resource pools |
**Result**:
- ✅ Users can only see projects they created
- ✅ Team collaboration through resource pools works
- ✅ Fault tolerance: Works correctly even with broad ACE configurations
- ✅ No schema changes required
- ✅ ~30 lines changed
### Phase 2 — Template isolation
**Goal**: Users see their own templates + builtin templates only.
| Task | Files | Detail |
|---|---|---|
| Apply same pattern to templates | `templates.py` — `get_templates()` | Use same three-step filtering as projects |
| Uncomment `Template.Audit` | `templates.py` | Restore `has_privilege("Template.Audit")` checks |
**Dependency**: Web UI must handle 403 from `GET /templates/{id}`. Mitigation: keep builtin templates unconditionally visible so the UI always has data.
### Phase 3 — Image isolation (optional)
**Goal**: Users see only images they uploaded.
| Task | Files |
|---|---|
| Apply same pattern to images | `images.py` — `get_images()` |
| Fix image list filtering | `images.py` |
| ACE cleanup on delete | `images.py` — `delete_image()` |
### Phase 4 — Default ACE for "Users" group (optional)
**Goal**: Users in "Users" group can create/list resources without admin ACE intervention.
| Task | Detail |
|---|---|
| Default ACE on `/projects` | Grant "Users" group → User role → `/projects` (propagate=False) |
| Default ACE on `/templates` | Grant "Users" group → User role → `/templates` (propagate=False) |
## API Endpoints Changed
### Phase 1 (Implemented)
| Method | Path | Change |
|---|---|---|
| `GET` | `/v3/projects` | Three-step filtering: ACE → created_by → resource pools |
### Phase 2 (Planned)
| Method | Path | Change |
|---|---|---|
| `GET` | `/v3/templates` | Apply same three-step filtering |
| `GET` | `/v3/templates/{id}` | Restore `Template.Audit` check |
## Key Design Decisions
1. **Project sharing through resource pools only**: Users cannot configure ACE directly to share specific projects. All sharing must go through resource pools. This prevents permission configuration chaos and maintains clear ownership semantics.
2. **No auto-ACE required**: The three-step filtering logic works without needing automatic ACE creation on project creation. The created_by field provides sufficient ownership information.
3. **Fault-tolerant to ACE configuration**: Even if administrators configure broad ACE permissions (like `path: "/" + propagate: true`), user isolation remains effective because Step 2 filters by created_by.
4. **No DB migration required**: Uses existing ACE/role/privilege tables and created_by field. No schema changes needed.
5. **Performance**: Project list filtering is O(n) where n is the total number of projects. Each project requires one ACE check. Acceptable for < 500 projects. Can optimize later with batch ACE queries if needed.
## Updated References
- **Implementation**: `gns3server/api/routes/controller/projects.py` (feature/simple-user-isolation branch)
- **Discussion**: https://github.com/GNS3/gns3-server/discussions/1949
- **RBAC models**: `gns3server/db/models/acl.py`, `roles.py`, `privileges.py`
- **RBAC repository**: `gns3server/db/repositories/rbac.py`
- **Auth dependency**: `gns3server/api/routes/controller/dependencies/authentication.py`
- **RBAC dependency**: `gns3server/api/routes/controller/dependencies/rbac.py`
- **Resource pools**: `gns3server/db/models/pools.py` and `gns3server/db/repositories/pools.py`
## Implementation Notes
### What Was Implemented
The `feature/simple-user-isolation` branch implements a robust user isolation mechanism that:
1. **Integrates with existing RBAC framework** without breaking changes
2. **Leverages the created_by field** that already exists in the Project model
3. **Uses three-step pipeline filtering** to avoid complex seen_project_ids tracking
4. **Supports team collaboration** through existing resource pool functionality
5. **Is fault-tolerant to ACE misconfiguration** - broad ACE permissions don't break user isolation
### What Was Not Implemented
The original roadmap's Phase 1 included auto-ACE creation on project creation. This was determined to be unnecessary because:
- The three-step filtering logic achieves user isolation without auto-ACE
- Auto-ACE would add complexity without significant benefit
- Project sharing through resource pools is cleaner than direct ACE configuration
### Future Work
The same three-step filtering pattern can be applied to:
- **Templates**: Replace the FIXME comment with proper filtering logic
- **Images**: Apply the same pattern for user image isolation
- **Other resources**: Extend the pattern as needed
This implementation provides a solid foundation for user isolation in GNS3 3.0+ while maintaining compatibility with the existing RBAC framework.
## Phase 5 — ACE Architecture Refactoring (Future)
**Goal**: Improve ACE manageability by supporting multiple paths and resource pools in a single ACE entry.
### Current Problem
With the current design where one ACE = one path:
- **ACE explosion**: 5 user groups × 10 resource pools = 50 ACE entries
- **Management complexity**: Difficult to maintain and understand ACE purpose
- **Performance impact**: Permission checking must iterate through many ACE entries
### Proposed Solution
Redesign ACE structure to support multiple paths and resource pools in a single ACE entry:
```json
{
"name": "Development Team Access",
"description": "Full access for development team",
"ace_type": "group",
"group_id": "...",
"role_id": "...",
"paths": ["/projects", "/templates", "/images"],
"resource_pools": ["pool-id-1", "pool-id-2"],
"propagate": true,
"allowed": true
}
```
### Database Changes Required
1. **Add name and description to ACE table**:
```sql
ALTER TABLE acl ADD COLUMN name VARCHAR;
ALTER TABLE acl ADD COLUMN description TEXT;
```
2. **Create association tables**:
```sql
CREATE TABLE ace_paths (
ace_id UUID REFERENCES acl(ace_id),
path VARCHAR,
PRIMARY KEY (ace_id, path)
);
CREATE TABLE ace_pools (
ace_id UUID REFERENCES acl(ace_id),
resource_pool_id UUID REFERENCES resource_pools(resource_pool_id),
PRIMARY KEY (ace_id, resource_pool_id)
);
```
3. **Update permission checking logic** to check both paths and resource_pools tables
### Benefits
- ✅ **Reduced ACE entries**: One ACE covers multiple related paths/pools
- ✅ **Better organization**: Logical grouping with clear names and descriptions
- ✅ **Easier management**: Edit one ACE instead of multiple related entries
- ✅ **Improved performance**: Fewer ACE entries to check during permission validation
### Implementation Considerations
- **Migration path**: Need to migrate existing single-path ACEs to new structure
- **Backward compatibility**: API should support both old and new formats during transition
- **UI updates**: ACE management interface needs to support multi-path/pool selection
- **Permission checking**: Update `check_user_has_privilege` to check association tables
## Phase 6 — Frontend Permission Query API (Future)
**Goal**: Provide an API endpoint for the Web UI to query the current user's permissions, enabling dynamic UI rendering based on role and ACE configuration.
### Problem
Currently the Web UI cannot determine what the authenticated user is allowed to see or do:
- ❌ Users see menu items and buttons they don't have permission to use
- ❌ Clicking a forbidden action results in a 403 error (unexpected UX)
- ❌ No way to hide/show UI elements based on actual permissions
### Proposed Solution
Create a `GET /v3/me/permissions` endpoint that returns the current user's effective permissions:
```json
{
"user_id": "uuid",
"is_superadmin": false,
"permissions": [
{"path": "/projects", "privileges": ["Project.Audit", "Project.Allocate"]},
{"path": "/projects/{id}", "privileges": ["Project.Audit", "Project.Modify"]},
{"path": "/templates", "privileges": ["Template.Audit"]}
],
"pools": [
{"path": "/pools/{id}", "name": "Team Projects", "privileges": ["Pool.Audit"]}
]
}
```
### Benefits
- ✅ **Dynamic UI**: Frontend can hide inaccessible menus/buttons
- ✅ **Better UX**: Users only see what they can actually use
- ✅ **Reduced errors**: Fewer 403 responses from hidden operations
- ✅ **Faster feedback**: Permission checks happen at render time, not request time
### Dependencies
- **Phase 5 (ACE refactoring)** may change how permissions are stored, which would affect this API's implementation
## Phase 7 — Resource Pool Renaming (Future)
**Goal**: Rename "Resource Pool" to a more descriptive name that better reflects its actual purpose.
### Problem
The current name "Resource Pool" is too generic and doesn't clearly convey its actual function:
- ❌ **Ambiguous name**: "Resource Pool" could refer to compute pools, connection pools, etc.
- ❌ **Unclear purpose**: Users don't understand it's primarily for sharing projects
- ❌ **Discoverability**: Hard to find the right feature when looking for project sharing
### Actual Function
Resource pools in GNS3 are used for:
- **Project sharing**: Allow users to access projects created by other users
- **Team collaboration**: Enable team members to work on shared projects
- **Access control**: Provide fine-grained permissions for project access through three-step filtering (ACE check → created_by filter → resource pools)
### Proposed Name Options
| Option | Pros | Cons |
|--------|-------|-------|
| **Project Pool** | More explicit, indicates it contains projects | Still uses "pool" terminology |
| **Shared Projects** | Directly describes the function | Loses the "collection" concept |
| **Team Projects** | Emphasizes collaboration use case | Doesn't cover non-team sharing scenarios |
**Recommended**: **Project Pool** - strikes a balance between clarity and consistency with existing terminology.
### Implementation Scope
Renaming would require changes to:
- Database tables: `resource_pools` → `project_pools`
- API routes: `/v3/pools` → `/v3/project_pools`
- Schema classes and field names throughout the codebase
- All documentation and help text
- Migration script to preserve existing data
### Benefits
- ✅ **Improved discoverability**: Users can easily find the project sharing feature
- ✅ **Better onboarding**: New users understand the purpose without confusion
- ✅ **Clearer API**: API endpoints and schemas more self-documenting
### Implementation Considerations
- **Breaking change**: Requires API version bump or backward compatibility layer
- **Data migration**: Existing resource pools must be preserved during table rename
- **Documentation updates**: All references in docs, tutorials, and API specs need updating
- **UI changes**: Frontend labels and navigation menus need to match new terminology
## Phase 8 — Per-User Project Namespace (Future)
**Goal**: Allow project names to be unique per user instead of globally unique, enabling better user experience.
### Current Problem
Although Phase 1 implements user isolation (users only see projects they created or have access to through resource pools), project names remain globally unique:
- ❌ **Naming conflicts**: Alice and Bob cannot both create a project named "My Project"
- ❌ **Unnecessary restrictions**: Even though projects are isolated, users must coordinate globally unique names
- ❌ **Poor user experience**: Users get confusing error messages when trying to use common names like "Test Project"
### Proposed Solution
Change project uniqueness from global to per-user:
**Current:**
```sql
UNIQUE(name) -- Project names must be globally unique
```
**Proposed:**
```sql
UNIQUE(user_id, name) -- Project names unique per user
```
### Benefits
- ✅ **Better UX**: Users can name projects whatever they want without worrying about global conflicts
- ✅ **Natural naming**: Common names like "Test Project" or "Demo" can coexist between users
- ✅ **No coordination needed**: Teams don't need to maintain a shared project naming registry
- ✅ **Consistent isolation**: Projects are isolated both in visibility AND naming
### Implementation Scope
Changes required:
- **Database schema**: Modify Project table unique constraint from `(name)` to `(user_id, name)`
- **Migration script**: Handle existing projects with conflicting names
- **API validation**: Update project creation validation logic
- **Frontend**: Remove global name uniqueness checks from UI
### Migration Considerations
**Handling existing name conflicts:**
If the database already has projects with the same name but different users:
- Option 1: Keep existing names, only enforce uniqueness for new projects
- Option 2: Append suffixes to duplicates (e.g., "My Project (alice)", "My Project (bob)")
- Option 3: Require admin resolution for conflicts before enabling new constraint
**Recommended**: Option 1 (grandfather existing projects) for minimal disruption.
### Dependencies
- **Phase 1 (User isolation)**: Must be completed first
- Database migration required to modify unique constraint
## Phase 9 — User Self-Registration (Future)
**Goal**: Allow users to register their own accounts without requiring manual admin creation.
### Current State
Currently, user accounts can only be created by administrators or through direct database operations:
- ❌ **Admin burden**: Every new user requires manual account creation
- ❌ **Poor scalability**: Not suitable for public deployments or large organizations
- ❌ **Friction**: Users cannot immediately start using the system
### Proposed Features
**Self-Registration Flow:**
1. User provides email, username, and password
2. System validates input and creates account
3. Optional email verification to confirm email address
4. Account created with default role (typically "User" role)
5. User can immediately log in and start creating projects
**Email Verification (Optional):**
- Send verification email with confirmation link/code
- Verify email address before granting full access
- Prevent spam account creation
- Require SMTP server configuration
### Implementation Components
1. **New API endpoint**: `POST /v3/access/register` (public, no authentication required)
2. **Email service**: Integration with SMTP server for sending emails
3. **Configuration**: SMTP settings (host, port, credentials, encryption)
4. **Rate limiting**: Prevent abuse of self-registration
5. **Captcha integration**: Optional bot protection
### Configuration
```yaml
Server:
email:
enabled: true
smtp_host: smtp.example.com
smtp_port: 587
smtp_username: noreply@example.com
smtp_password: secret
use_tls: true
registration:
require_email_verification: true
default_role: "User"
allow_public_registration: true
```
### Security Considerations
- ✅ **Rate limiting**: Prevent spam account creation
- ✅ **Email verification**: Confirm email ownership
- ✅ **Default permissions**: New users get limited default role
- ✅ **Admin approval** (optional): Require admin approval before account activation
- ❌ **Superadmin creation**: Never allow self-registration as superadmin
### Dependencies
- **Phase 10 (Email service)**: SMTP integration required for email verification
## Phase 10 — Email Service Integration (Future)
**Goal**: Implement email sending capability for notifications, verification, and alerts.
### Use Cases
1. **User registration**: Email verification links/codes
2. **Password reset**: Secure password reset emails
3. **System alerts**: Error notifications, system updates
4. **Project sharing**: Notify users when projects are shared via resource pools
5. **Usage reports**: Periodic usage summaries or quota alerts
### Proposed Implementation
**Email Service Architecture:**
- Abstract email service interface
- Support multiple email providers (SMTP, SendGrid, AWS SES, etc.)
- Email templates with Jinja2 for customization
- Async email sending to avoid blocking API responses
- Email queue for retry logic on failures
**Email Templates:**
- Registration verification
- Password reset
- Project shared notification
- System alerts
- Usage reports
**API Endpoints:**
```python
# Configuration (admin only)
POST /v3/access/config/email
GET /v3/access/config/email
PUT /v3/access/config/email
# Test email (admin only)
POST /v3/access/config/email/test
# Password reset (public)
POST /v3/access/users/password/reset/request
POST /v3/access/users/password/reset/confirm
# Email verification (public)
POST /v3/access/users/verify/email
POST /v3/access/users/verify/confirm
```
### Database Schema
New table for email tracking:
```sql
CREATE TABLE email_verification_tokens (
token_id UUID PRIMARY KEY,
user_id UUID REFERENCES users(user_id) ON DELETE CASCADE,
token VARCHAR(255), # Verification code
purpose VARCHAR(50), # 'registration', 'password_reset', etc.
expires_at DATETIME,
created_at DATETIME,
used BOOLEAN DEFAULT FALSE
);
```
### Configuration
```yaml
Server:
email:
enabled: true
provider: "smtp" # or "sendgrid", "aws_ses", etc.
from_address: "noreply@example.com"
from_name: "GNS3 Server"
reply_to: "support@example.com"
smtp:
host: smtp.example.com
port: 587
username: noreply@example.com
password: encrypted_password
use_tls: true
templates_dir: /etc/gns3/email_templates
```
### Security Considerations
- ✅ **Encrypted credentials**: SMTP passwords stored encrypted in database
- ✅ **Token expiration**: Verification tokens expire after configurable time
- ✅ **Rate limiting**: Prevent email spamming
- ✅ **Async sending**: Don't block API responses on email operations
- ✅ **Retry logic**: Handle temporary email service failures
- ✅ **Privacy**: Don't expose user information in error messages
### Dependencies
- **Phase 9 (Self-registration)**: User self-registration requires email verification
- Encryption utilities for storing SMTP credentials securely
@@ -0,0 +1,67 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This document is a roadmap/planning document. The described features have not been implemented yet.
# Server Settings REST API — Roadmap
## Problem
Currently, `gns3_server.conf` can only be modified by directly editing the file on disk. There is no REST API endpoint to read or write server configuration, which prevents the Web UI from offering a settings page for server parameters.
## Proposed API
```
GET /v3/settings → Return all current server settings
PUT /v3/settings → Update and persist server settings
```
### Implementation Plan
**1. Add `save_config()` to `Config` class** (`gns3server/config.py`)
The `Config` class currently only reads configuration (via `read_config()` / `reload()`). A `save_config()` method is needed to serialize the in-memory `ServerConfig` pydantic model back to INI format and write it to disk.
Serialization details:
- `bool` → `"True"` / `"False"` (configparser convention)
- `SecretStr` → `get_secret_value()`
- `Enum` → `.value`
- `List[str]` → semi-colon for `additional_images_paths`, comma for `allowed_interfaces`
- `None` → skip
**2. Add a settings getter/setter** to `Config` to allow programmatic updates to the in-memory settings.
**3. New route file** (`gns3server/api/routes/controller/settings.py`):
- `GET /v3/settings` — returns the full `ServerConfig` as JSON (pydantic automatically masks `SecretStr` fields as `"********"`)
- `PUT /v3/settings` — accepts `ServerConfig`, merges existing secrets when placeholder values (`"********"`) are submitted, calls `save_config()`, and triggers runtime config update callbacks
Both endpoints require `get_current_active_user` for authentication.
**4. Register the new router** in `gns3server/api/routes/controller/__init__.py` under the `/settings` prefix.
### Security
- All settings endpoints require admin authentication (`get_current_active_user`)
- `SecretStr` fields (`compute_password`, `default_admin_password`, `jwt_secret_key`) are masked in responses
- On write, unchanged secrets are preserved via placeholder detection
### Related Files
| File | Role |
|------|------|
| `gns3server/config.py` | Config singleton with `read_config()` / `reload()` |
| `gns3server/schemas/config.py` | `ServerConfig` pydantic model with all 9 sub-models |
| `gns3server/api/routes/controller/__init__.py` | Controller router mounting |
| `gns3server/controller/__init__.py` | `Controller._update_config()` for runtime credential sync |
## Status
- [ ] gns3server/config.py: add `save_config()` method
- [ ] gns3server/config.py: add settings getter/setter
- [ ] gns3server/api/routes/controller/settings.py: new route file with GET and PUT endpoints
- [ ] gns3server/api/routes/controller/__init__.py: register settings router under `/settings`
- [ ] gns3server/api/routes/controller/controller.py: add notification emission on config change
@@ -0,0 +1,318 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
# Skills Editor API
## Overview
A management API that allows the Web UI to browse, edit, save, and contribute skill files (prompts, fault injection, device skills, packet analysis protocols) back to the upstream GNS3-Skills repository via Pull Requests.
Today, skills are read-only from the server's perspective — the only management endpoint is `POST /copilot/reload/skills` for hot-reloading. This API adds full CRUD operations on the local skills repository plus Git commit/push/PR workflows.
## Architecture
```mermaid
graph TD
subgraph "Web UI"
UI[Skills Editor Page]
end
subgraph "GNS3 Server API"
API[skills_editor.py<br/>/copilot/skills/*]
FM[SkillsFileManager]
SM[SkillsManager]
end
subgraph "Local Git Repo"
INJ[injection/*.yaml]
DEV[device/*.yaml]
PA[packet_analysis/*.yaml]
PRM[prompts/*.md]
CFG[config/*.txt]
end
subgraph "Remote"
GH[GitHub API<br/>Pull Requests]
REPO[gns3/gns3-skills]
end
UI -->|CRUD + PR| API
API --> FM
FM -->|file read/write| INJ
FM -->|file read/write| DEV
FM -->|file read/write| PA
FM -->|file read/write| PRM
FM -->|file read/write| CFG
FM -->|git commit/push| REPO
FM -->|create PR| GH
SM -->|hot reload| INJ
SM -->|hot reload| DEV
SM -->|hot reload| PA
SM -->|hot reload| PRM
```
## Business Process
### Edit and Contribute Flow
```mermaid
sequenceDiagram
participant UI as Web UI
participant API as Skills Editor API
participant FM as SkillsFileManager
participant Git as Local Git Repo
participant GH as GitHub
UI->>API: GET /copilot/skills
API-->>UI: List categories + file counts
UI->>API: GET /copilot/skills/injection
API-->>UI: List YAML files
UI->>API: GET /copilot/skills/injection/ospf_issues
API->>FM: read_file("injection", "ospf_issues")
FM-->>API: YAML content
API-->>UI: File content
UI->>API: PUT /copilot/skills/injection/ospf_issues<br/>{content: "..."}
API->>FM: write_file("injection", "ospf_issues", content)
FM->>Git: Write file to disk
FM-->>API: Success
API-->>UI: Updated
UI->>API: POST /copilot/skills/commit<br/>{message: "fix: update OSPF fault"}
API->>FM: commit_changes(message, files)
FM->>Git: git add + git commit
FM-->>API: Commit hash
API-->>UI: Committed
UI->>API: POST /copilot/skills/pull-request<br/>{title, body, branch}
API->>FM: create_pull_request(...)
FM->>Git: git push origin <branch>
FM->>GH: POST /repos/{owner}/{repo}/pulls
GH-->>FM: PR URL
FM-->>API: PR created
API-->>UI: PR URL
UI->>API: POST /copilot/skills/reload
API->>FM: Hot reload all registries
API-->>UI: Reloaded
```
## Valid Categories
| Category | Directory | File Extension | Content |
|----------|-----------|---------------|---------|
| `prompts` | `prompts/` | `.md` | System prompts (teaching_assistant, lab_automation_assistant, etc.) |
| `injection` | `injection/` | `.yaml` | Fault injection skill definitions (OSPF, BGP, VLAN, etc.) |
| `device` | `device/` | `.yaml` | Device-specific command knowledge |
| `packet_analysis` | `packet_analysis/` | `.yaml` | Protocol definitions for tshark-based analysis |
| `config` | `config/` | `.txt` | Security and configuration files (forbidden_commands, etc.) |
## API Endpoints
All endpoints require **superadmin** authentication. Prefix: `/v3/copilot/skills`.
| Method | Path | Description |
|--------|------|-------------|
| GET | `/copilot/skills` | List all categories with file counts |
| GET | `/copilot/skills/{category}` | List files in a category |
| GET | `/copilot/skills/{category}/{filename}` | Read file content (without extension) |
| POST | `/copilot/skills/{category}` | Create a new skill file |
| PUT | `/copilot/skills/{category}/{filename}` | Update existing file content |
| DELETE | `/copilot/skills/{category}/{filename}` | Delete a skill file |
| GET | `/copilot/skills/status` | Git status (modified/untracked/deleted files) + repo info |
| POST | `/copilot/skills/commit` | Stage and commit changes |
| POST | `/copilot/skills/push` | Push a branch to remote |
| POST | `/copilot/skills/pull-request` | Create a Pull Request (direct or via fork) |
| POST | `/copilot/skills/reload` | Hot reload all skills into memory (replaces `/reload/skills`) |
| POST | `/copilot/skills/rollback/{commit_hash}` | Rollback repository to a specific commit |
### Response Examples
**GET /copilot/skills** — List categories
```json
{
"categories": [
{"category": "injection", "file_count": 39, "path": "injection/"},
{"category": "device", "file_count": 2, "path": "device/"},
{"category": "packet_analysis", "file_count": 8, "path": "packet_analysis/"},
{"category": "prompts", "file_count": 4, "path": "prompts/"},
{"category": "config", "file_count": 1, "path": "config/"}
],
"repository": {
"repo_url": "https://github.com/gns3/gns3-skills.git",
"branch": "main",
"current_version": "abc123def456...",
"is_dirty": false
}
}
```
**GET /copilot/skills/injection** — List files in category
```json
{
"category": "injection",
"files": [
{"filename": "ospf_issues", "extension": ".yaml", "size": 4521, "last_modified": "2026-05-10T08:30:00Z"},
{"filename": "bgp_issues", "extension": ".yaml", "size": 3820, "last_modified": "2026-05-09T14:00:00Z"}
]
}
```
**GET /copilot/skills/injection/ospf_issues** — Read file content
```json
{
"category": "injection",
"filename": "ospf_issues",
"extension": ".yaml",
"content": "name: OSPF Fault Injection\n...\n",
"size": 4521,
"last_modified": "2026-05-10T08:30:00Z"
}
```
**PUT /copilot/skills/injection/ospf_issues** — Update file
```json
{
"content": "name: OSPF Fault Injection\n..."
}
```
Response:
```json
{
"category": "injection",
"filename": "ospf_issues",
"size": 4600,
"last_modified": "2026-05-12T10:00:00Z",
"status": "modified"
}
```
**POST /copilot/skills/commit** — Commit changes
Request:
```json
{
"message": "fix: update OSPF hello/dead interval fault descriptions",
"files": ["injection/ospf_issues.yaml"]
}
```
Response:
```json
{
"success": true,
"commit_hash": "def456abc789...",
"message": "fix: update OSPF hello/dead interval fault descriptions",
"files_committed": 1
}
```
**POST /copilot/skills/pull-request** — Create PR
Request:
```json
{
"title": "Fix OSPF fault injection descriptions",
"body": "Updated OSPF hello/dead interval fault descriptions for clarity.",
"branch": "fix/ospf-descriptions",
"target_branch": "main",
"fork_url": "https://github.com/user/GNS3-Skills.git"
}
```
Response:
```json
{
"success": true,
"pr_url": "https://github.com/gns3/gns3-skills/pull/42",
"pr_number": 42,
"branch": "fix/ospf-descriptions"
}
```
**GET /copilot/skills/status** — Git status
```json
{
"current_version": "abc123def456...",
"branch": "main",
"is_dirty": true,
"modified": ["injection/ospf_issues.yaml"],
"untracked": [],
"deleted": [],
"staged": [],
"available_versions": [
{"hash": "abc123def456", "message": "Add MPLS fault scenarios", "author": "dev", "date": "2026-05-10T08:00:00Z"}
]
}
```
## Implementation Plan
### New Files
| File | Purpose |
|------|---------|
| `gns3server/agent/gns3_copilot/skills/file_manager.py` | `SkillsFileManager` class — file CRUD, git commit/push, GitHub PR API |
| `gns3server/api/routes/controller/skills_editor.py` | FastAPI router with all endpoints above |
### Modified Files
| File | Change |
|------|--------|
| `gns3server/agent/gns3_copilot/skills/manager.py` | Add `get_file_manager()` method returning a `SkillsFileManager` |
| `gns3server/api/routes/controller/__init__.py` | Register `skills_editor` router under `/copilot/skills` prefix |
| `gns3server/api/routes/controller/copilot.py` | Deprecate `/reload/skills` in favor of `/skills/reload` |
### SkillsFileManager Key Methods
| Method | Returns |
|--------|---------|
| `list_categories()` | `list[{category, file_count, path}]` |
| `list_files(category)` | `list[{filename, size, last_modified}]` |
| `read_file(category, filename)` | File content as string |
| `write_file(category, filename, content)` | Write (create or update) |
| `delete_file(category, filename)` | Delete file |
| `get_git_status()` | Modified/untracked/deleted file lists |
| `commit_changes(message, files)` | git add + commit |
| `push_to_remote(branch)` | git push |
| `create_pull_request(title, body, branch, target, fork_url)` | Push + GitHub PR API |
## Security
- **Authentication**: All endpoints require superadmin (`current_user.is_superadmin` check)
- **Path traversal prevention**: Category validated against whitelist; filename sanitized (no `/`, `..`, or absolute paths)
- **File extension enforcement**: `.yaml` for injection/device/packet_analysis, `.md` for prompts, `.txt` for config
- **File size limit**: Reject files > 1MB
- **YAML validation**: Validate with `yaml.safe_load` before saving
- **GitHub token**: Required for PR creation, stored in GNS3 server config
### PR Creation Modes
1. **Direct push**: Push to a new branch on the main repo, create PR via GitHub API
2. **Fork**: Push to user's fork, create PR against upstream
The `fork_url` parameter selects the mode. When omitted, the API pushes to the same repository and creates a PR directly.
## Related Documentation
- [External Skills Repository](../implemented/skills-repository.md)
- [Fault Injection](../implemented/fault-injection.md)
- [Chat API](../implemented/chat-api.md)
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,311 @@
# User Node Limit Roadmap
## Overview
Implement a user-level node startup limit feature for GNS3 server to prevent single users from consuming excessive system resources. This feature is **disabled by default** and can be enabled through configuration files, supporting a three-tier configuration priority system.
## Problem Statement
Currently, GNS3 server has no mechanism to limit the number of nodes a user can start across all their projects. This can lead to:
- **Resource exhaustion**: A single user can consume all available system resources
- **Unfair usage**: Some users may prevent others from using the system
- **System instability**: Too many running nodes can degrade overall performance
- **Cost issues**: In cloud environments, this can lead to unexpected costs
## Solution Design
### Core Principles
1. **Default to no limits**: System maintains backward compatibility by defaulting to unrestricted usage
2. **Configuration-driven**: All limits can be controlled through configuration files
3. **Multi-tier priority**: User-specific > User group-specific > Global configuration
4. **Intelligent filtering**: Only count nodes that actually consume resources
5. **Clear error messages**: Users receive actionable feedback when limits are reached
### Technical Architecture
#### 1. Database Layer Extension
**Files**: `gns3server/db/models/users.py`
Add `max_nodes` field to both `User` and `UserGroup` models:
```python
# In User model
max_nodes = Column(Integer, nullable=True) # NULL = no limit
# In UserGroup model
max_nodes = Column(Integer, nullable=True) # NULL = no limit
```
#### 2. Configuration System Extension
**File**: `gns3server/schemas/config.py`
Add node limit configuration to `ControllerSettings`:
```python
class NodeLimitSettings(BaseModel):
enabled: bool = False # Feature toggle (default disabled)
default_max_nodes: int = 5 # Default limit when enabled
excluded_node_types: List[str] = Field(default_factory=lambda: [
"ethernet_switch", "ethernet_hub", "cloud", "nat"
])
```
#### 3. Core Service Implementation
**New File**: `gns3server/services/node_limit_service.py`
Implement the `NodeLimitService` class with key methods:
```python
class NodeLimitService:
async def get_user_active_node_count(self, username: str, excluded_types: List[str]) -> int:
"""Count user's active nodes across all projects"""
async def get_user_node_limit(self, user: User) -> Optional[int]:
"""Get user's node limit with priority logic"""
async def check_user_node_limit(self, user: User, project: Project) -> Tuple[bool, str]:
"""Check if user can start more nodes"""
```
#### 4. API Integration
**File**: `gns3server/api/routes/controller/nodes.py`
Add limit checking to node startup endpoint:
```python
@router.post("/{node_id}/start")
async def start_node(
node: Node = Depends(dep_node),
current_user: User = Depends(get_current_active_user),
node_limit_service: NodeLimitService = Depends(get_node_limit_service)
):
# Node limit check
can_start, error_msg = await node_limit_service.check_user_node_limit(
current_user, node.project
)
if not can_start:
raise HTTPException(status_code=403, detail=error_msg)
# Original startup logic
await node.start()
```
### Node Counting Logic
#### What Counts Toward the Limit
- **Status**: Only nodes in `started` or `suspended` state
- **Ownership**: Only projects where `project.created_by == current_user.username`
- **Node types**: All node types except those explicitly excluded
#### What's Excluded from the Limit
- **Always-running nodes**: Ethernet switches, hubs (nodes where `is_always_running()` returns true)
- **Infrastructure nodes**: Cloud nodes and NAT nodes
- **Stopped nodes**: Nodes in `stopped` state
- **Other users' projects**: Nodes in projects created by other users
#### Configuration Priority
```
User-specific limit (highest priority)
↓ not set
User group limit
↓ not set
Global configuration (if enabled)
↓ disabled
No limit (default)
```
### Configuration Examples
#### Scenario 1: Default No Limits (System Default)
```ini
[Controller]
node_limits_enabled = false
```
**Result**: All users have no node limits
#### Scenario 2: Enable Global Limits
```ini
[Controller]
node_limits_enabled = true
node_limits_default_max_nodes = 5
node_limits_excluded_types = ethernet_switch,ethernet_hub,cloud,nat
```
**Result**: All users limited to 5 active nodes (excluding infrastructure nodes)
#### Scenario 3: User-Specific Limits
Configuration file: `node_limits_enabled = false`
Database:
- User A: `max_nodes = 10` (limited to 10 nodes)
- User B: `max_nodes = NULL` (no limit)
- Other users: no limit
#### Scenario 4: User Group Limits
Configuration file: `node_limits_enabled = false`
Database:
- "Users" group: `max_nodes = 5`
- "Premium Users" group: `max_nodes = 20`
- "Administrators" group: `max_nodes = NULL`
**Result**: Members inherit limits from their groups
## Implementation Files
| File Path | Type | Description |
|-----------|------|-------------|
| `gns3server/db/models/users.py` | Modify | Add `max_nodes` field to User and UserGroup |
| `gns3server/db_migrations/versions/xxx_add_node_limits.py` | New | Database migration script |
| `gns3server/schemas/config.py` | Modify | Add NodeLimitSettings configuration class |
| `gns3server/schemas/controller/users.py` | Modify | Add `max_nodes` to API schemas |
| `gns3server/services/node_limit_service.py` | New | Core node limit service |
| `gns3server/api/routes/controller/nodes.py` | Modify | Add limit check to node startup |
| `gns3server/api/routes/controller/users.py` | Modify | Add user limit configuration API |
| `gns3server/api/routes/controller/groups.py` | Modify | Add group limit configuration API |
## Implementation Steps
### Phase 1: Database Layer
1. Add `max_nodes` field to `User` and `UserGroup` models
2. Create database migration file
3. Test database migration and rollback
### Phase 2: Configuration System
1. Add `NodeLimitSettings` to configuration schema
2. Update configuration file loading logic
3. Test configuration parsing and validation
### Phase 3: Core Service
1. Implement `NodeLimitService` class
2. Implement node counting logic
3. Implement limit checking logic
4. Add unit tests for service methods
### Phase 4: API Integration
1. Modify node startup endpoint to add limit check
2. Add user limit configuration endpoints
3. Add group limit configuration endpoints
4. Add user node usage statistics endpoint
### Phase 5: Testing
1. Unit tests for all core functions
2. Integration tests for API endpoints
3. End-to-end tests for complete workflows
4. Performance tests for node counting operations
## Testing Strategy
### Functional Tests
- [ ] Default state verification (no limits)
- [ ] Global limit enablement
- [ ] User-specific limits
- [ ] User group limits
- [ ] Configuration priority verification
### Boundary Tests
- [ ] Exactly at limit (can start last node)
- [ ] One over limit (startup rejected)
- [ ] Stop node then restart (should work)
- [ ] Special node type exclusion
### Integration Tests
- [ ] Multi-user concurrent startups
- [ ] Node state transitions
- [ ] Dynamic configuration changes
- [ ] Project ownership filtering
### Performance Tests
- [ ] Node counting performance with many projects
- [ ] Concurrent startup request handling
- [ ] Memory usage monitoring
## Error Messages
### User-Friendly Error Response
When a user hits their node limit:
```json
{
"detail": "节点启动限制:您当前有 5 个活跃节点,限制为 5 个。请停止一些节点后再试,或联系管理员调整限制。"
}
```
Alternative formats:
- Show current usage vs limit
- Provide action suggestions
- Include contact information for administrators
## Migration Path
### For Existing Systems
1. **Database migration**: Add new nullable fields (safe, no data loss)
2. **Configuration update**: Add new optional settings (backward compatible)
3. **API changes**: Add new optional dependency injection (no breaking changes)
4. **Behavior**: No changes to existing functionality when disabled
### Rollback Plan
If issues occur:
1. Set `node_limits_enabled = false` in configuration
2. Service automatically disables limit checking
3. System returns to pre-feature behavior
## Benefits
1. **Resource Management**: Prevent resource exhaustion
2. **Fair Usage**: Ensure equitable resource distribution
3. **Cost Control**: Manage cloud resource costs
4. **System Stability**: Maintain performance under load
5. **Flexibility**: Support different usage patterns and tiers
6. **Backward Compatible**: No impact on existing deployments
## Future Enhancements
- Per-project limits (in addition to global user limits)
- Time-based limits (different limits for different times)
- Burst limits (temporary allowance for peak usage)
- Usage quotas with reset periods (daily/weekly/monthly)
- Monitoring and alerting for limit approaching
- Administrative override capabilities
- Usage history and analytics
## Documentation Updates
- [ ] Update API documentation with new endpoints
- [ ] Add configuration guide to admin documentation
- [ ] Update user guide with limit information
- [ ] Add troubleshooting section for limit issues
- [ ] Provide migration guide for existing deployments
## Status
**Current Status**: Design Phase
**Next Steps**:
1. Review and approve this roadmap
2. Begin Phase 1 implementation (Database Layer)
3. Create detailed technical specification
4. Set up development and testing environment
---
**Document Version**: 1.0
**Last Updated**: 2026-05-27
**Author**: GNS3 Development Team
@@ -0,0 +1,60 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This document is a roadmap/planning document. The described features have not been implemented yet.
# User Preferences API — Roadmap
## Problem
Currently, user-specific preferences (UI theme, language, AI Copilot settings, workspace layout, etc.) can only be stored in browser `localStorage`. This has several drawbacks:
1. **localStorage clearing** — clearing browser data or switching devices loses all settings
2. **No cross-device sync** — users must reconfigure preferences on each device
3. **No API access** — CLI tools and third-party clients cannot read/write preferences
4. **Fragile persistence** — localStorage can be cleared by browser maintenance or incognito mode
## Proposed API
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/v3/access/users/me/preferences` | Get all preferences for current user |
| `PUT` | `/v3/access/users/me/preferences` | Merge preferences (top-level merge, not replace) |
| `DELETE` | `/v3/access/users/me/preferences` | Clear all preferences |
### Storage
A `preferences` JSON column on the existing `users` table:
- Type: `JSON`, non-nullable, default `'{}'`
- No schema enforcement — clients can store arbitrary key-value pairs
- Examples: `{"theme": "dark", "language": "zh-CN", "copilot_default_mode": "teaching_assistant"}`
### Implementation Plan
1. Add `preferences` JSON column to `User` ORM model
2. Create Alembic migration
3. Add `UserPreferencesUpdate` Pydantic schema with `extra="allow"`
4. Add `get_user_preferences()` and `update_user_preferences()` to `UsersRepository`
5. Add three endpoints to `users.py` router: `GET/PUT/DELETE /me/preferences`
### Related Files
| File | Role |
|------|------|
| `gns3server/db/models/user.py` | User ORM model |
| `gns3server/db/repositories/users.py` | UsersRepository with database operations |
| `gns3server/api/routes/controller/users.py` | User API router |
| `gns3server/schemas/` | Pydantic schemas |
| `gns3server/db/versions/` | Alembic migrations |
## Status
- [ ] Add `preferences` JSON column to User ORM model
- [ ] Create Alembic migration
- [ ] Add `UserPreferencesUpdate` schema
- [ ] Add repository methods
- [ ] Add API endpoints to users.py router
@@ -0,0 +1,372 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
> This document is a roadmap/planning document. The described features have not been implemented yet.
# Web Wireshark Docker Image Size Optimization — Roadmap
## Problem
The `gns3/web-wireshark` Docker image currently occupies **~2GB** of disk space, which impacts:
- Initial pull/download time for users
- Storage requirements on Docker Hub
- Deployment flexibility in resource-constrained environments
### Current Size Breakdown
Based on `docker history gns3/web-wireshark:latest`:
| Component | Size | Percentage |
|-----------|------|------------|
| `debian:trixie` base image | 120 MB | 6% |
| xpra + dependencies | 65.1 MB | 3.3% |
| Wireshark + GUI stack (Qt, GTK, X11) | 1.82 GB | 91% |
| **Total** | **~2 GB** | **100%** |
### Detailed File System Analysis
Analysis of container file system reveals significant cleanup opportunities:
| Directory | Size | Cleanup Potential |
|-----------|------|-------------------|
| `/usr/lib` | 1.3 GB | ~50MB (static libraries) |
| `/usr/share/locale` | 151 MB | **~145MB** (192 locales → 1) |
| `/usr/share/ibus` | 130 MB | **~130MB** (input framework) |
| `/usr/share/doc` | 76 MB | **~76MB** (documentation) |
| `/usr/share/backgrounds` | 37 MB | **~37MB** (desktop backgrounds) |
| `/usr/share/man` | 27 MB | **~27MB** (man pages) |
| `/usr/share/icons` | 16 MB | ~5MB (keep essential) |
| Development packages | ~50 MB | **~50MB** (21 `-dev` packages) |
| Static libraries (`*.a`, `*.la`) | 16 MB | **~16MB** |
| Sounds/help/perl | ~20 MB | **~20MB** |
| **Total Cleanable** | **~570 MB** | |
**Key Findings**:
- 192 locale languages installed (only need en_US)
- ibus input framework installed (not needed in headless container)
- 21 development packages with headers/headers
- 161 static library files (`.a`, `.la`)
- Complete documentation and man pages
- Desktop environment components (backgrounds, sounds)
## Proposed Optimizations
### Phase 1: Safe Optimizations (Estimated: -300~400MB)
Low-risk changes that maintain full compatibility.
#### 1. Use Slim Base Image (-50MB)
```dockerfile
FROM debian:trixie-slim # Instead of debian:trixie
```
**Impact**: Reduces base from 120MB to ~70MB
**Risk**: Low - slim variant contains all essential runtime libraries
**Testing Required**: Verify xpra and Wireshark launch without errors
#### 2. Install Without Recommended Packages (-150~200MB)
```dockerfile
RUN apt-get install -y --no-install-recommends \
wireshark-common \
wireshark \
xpra=6.4.3* \
xpra-x11 \
xvfb \
curl \
x11-utils
```
**Impact**: Prevents installation of non-essential recommended packages
**Risk**: Low - only excludes recommended packages, not required dependencies
**Testing Required**: Full functionality test (capture, analysis, WebSocket)
#### 3. Cleanup Unnecessary Files (-400~500MB)
```dockerfile
RUN apt-get install -y --no-install-recommends \
wireshark-common wireshark xpra=6.4.3* xpra-x11 xvfb curl x11-utils \
# Remove documentation and man pages
&& rm -rf /var/lib/apt/lists/* \
&& rm -rf /usr/share/doc/* /usr/share/man/* /usr/share/help/* \
# Remove unnecessary locales (keep only en_US)
&& rm -rf /usr/share/locale/* \
&& localedef -i en_US -f UTF-8 en_US.UTF-8 \
# Remove desktop environment components
&& rm -rf /usr/share/backgrounds/* /usr/share/sounds/* \
# Remove static libraries
&& find /usr/lib -name '*.a' -delete \
&& find /usr/lib -name '*.la' -delete \
# Remove unnecessary packages
&& apt-get purge -y -y \
ibus ibus-data ibus-gtk* python3-ibus-1.0 \
gnome-backgrounds \
&& apt-get autoremove -y \
&& apt-get clean
```
**Impact**: Removes documentation, locales, desktop components, input framework
**Risk**: Low - all removed components are unnecessary in headless container
**Testing Required**: Verify Wireshark GUI renders correctly without icons/themes
### Phase 2: Experimental (Requires Testing, -200~500MB)
Higher-risk optimizations that need extensive validation.
#### 4. Alpine Linux Alternative (-500MB~1GB)
```dockerfile
FROM alpine:3.19
RUN apk add --no-cache wireshark xpra xvfb curl ...
```
**Impact**: Could reduce image to ~500MB-1GB
**Risk**: **High** - Wireshark and xpra have complex Qt/GTK dependencies
**Challenges**:
- Wireshark Qt dependencies may not be available in Alpine repos
- xpra package availability and compatibility
- X11 library differences
- May require building dependencies from source
**Testing Required**:
- [ ] Verify Wireshark package availability in Alpine
- [ ] Test xpra compilation/installation on Alpine
- [ ] Validate all GUI libraries work correctly
- [ ] Full integration testing
**Recommendation**: Do not pursue unless Phase 1 insufficient
## Compression Feasibility Analysis
### Question: Can we reduce image size through compression?
**Short Answer**: **Not recommended** - limited benefit with performance trade-offs
### Current Compression Status
Docker images already use compression:
| Format | Size | Compression Rate | Use Case |
|--------|------|------------------|----------|
| Runtime size | 2.0 GB | - | Container running |
| docker save (raw) | 1.9 GB | 5% | Docker internal compression |
| docker save + gzip | 718 MB | **64%** | Standard transfer |
| docker save + xz | 563 MB | **72%** | Maximum compression |
| docker save + zstd | ~600 MB | **70%** | Fast compression |
### Binary Compression Analysis
#### File Already Stripped
All binaries already have debug symbols removed:
```bash
wireshark: ELF 64-bit... stripped
python3.13: ELF 64-bit... stripped
libc.so.6: ELF 64-bit... stripped
```
**No further stripping possible**
#### UPX Executable Compression (Limited Benefit)
Test results compressing major executables:
| Binary | Original | UPX Compressed | Savings | Startup Impact |
|--------|----------|----------------|---------|----------------|
| wireshark (11MB) | 11.0 MB | 4.2 MB | 62% | +0.3s |
| Xvfb (2.1MB) | 2.1 MB | 0.9 MB | 57% | +0.1s |
| python3.13 (6.6MB) | 6.6 MB | 2.8 MB | 58% | +0.2s |
| **Total** | **19.7 MB** | **7.9 MB** | **60%** | **+0.6s** |
**Overall Impact**: Only ~20MB savings (1%) with 0.6s startup penalty
### Why Compression Has Limited Benefit
1. **Small Executable Footprint**: Binaries are only 74MB (3.7% of image)
2. **Libraries Are Data Files**: `/usr/lib` contains mostly data, not code
3. **Already Compressed**: Docker storage drivers compress layers automatically
4. **Resource Files Dominate**: Fonts, icons, themes don't compress well
### Compression Trade-offs
| Method | Potential Savings | Performance Impact | Complexity | Risk |
|--------|-------------------|-------------------|------------|------|
| **File cleanup** | 400-500MB (20-25%) | None | Low | Low |
| UPX compression | 50-100MB (2.5-5%) | +0.6s startup | Medium | Medium |
| Layer squashing | 10-50MB (0.5-2.5%) | None | Low | Low |
| Transfer compression | 1.4GB (70%) | None (transfer only) | None | None |
### Recommendation
**Do not pursue binary compression** because:
- ✅ File cleanup is **4-10x more effective**
- ✅ No performance penalty
- ✅ Simpler build process
- ✅ Better compatibility
**For transfer/storage optimization**, use standard tools:
```bash
# For archiving (use zstd for best speed/ratio)
docker save gns3/web-wireshark:latest | \
zstd -19 -o web-wireshark.tar.zst
# For maximum compression (slow)
docker save gns3/web-wireshark:latest | \
xz -9 -T 0 > web-wireshark.tar.xz
```
## Implementation Plan
### Step 1: Create Optimized Dockerfile
Create `gns3server/agent/web_wireshark/docker/Dockerfile.optimized` with Phase 1 changes.
### Step 2: Local Testing
```bash
# Build optimized image
cd gns3server/agent/web_wireshark/docker
docker build -f Dockerfile.optimized -t gns3/web-wireshark:optimized .
# Verify size reduction
docker images | grep web-wireshark
# Test Wireshark functionality
docker run --rm gns3/web-wireshark:optimized wireshark --version
# Test xpra functionality
docker run --rm gns3/web-wireshark:optimized xpra --version
```
### Step 3: Integration Testing
```bash
# Test with actual GNS3 server
pip install . && gns3server-web-wireshark-setup
# Start a test session
python3 gns3server/agent/web_wireshark/manage_wireshark.py \
--verbose start \
--project-id "test-optimized" \
--link-id "test-link-1" \
--jwt-token "test-token" \
--image "gns3/web-wireshark:optimized"
# Verify WebSocket connectivity and packet capture
```
### Step 4: Production Rollout
1. Tag optimized image: `gns3/web-wireshark:v1.6-optimized`
2. Deploy to staging environment
3. Monitor for 1 week with real workloads
4. If stable, promote to `latest` tag
5. Keep old image available for rollback
## Testing Checklist
Before marking as complete:
- [ ] Wireshark launches without errors
- [ ] xpra HTML5 client connects successfully
- [ ] Packet capture works end-to-end
- [ ] Packet analysis and filtering functional
- [ ] WebSocket proxy integration works
- [ ] Multi-session handling tested (3+ simultaneous captures)
- [ ] Image size measured and documented
- [ ] Tested on Docker 20.10, 24.x, 29.x
- [ ] Startup time not negatively impacted
- [ ] Memory/CPU usage unchanged
- [ ] All existing tests pass
## Expected Results
### Phase 1: Safe Optimizations (Recommended)
| Metric | Current | Target | Improvement |
|--------|---------|--------|-------------|
| Image Size | 2GB | ~1.5GB | **-25% (500MB)** |
| Pull Time | 3-5 min | 2-3 min | **-40%** |
| Startup Time | 5-6s | 5-6s | No change |
| Functionality | Full | Full | No regression |
| Compatibility | All | All | No change |
### Phase 2: Experimental (If Needed)
| Metric | Current | Target | Improvement |
|--------|---------|--------|-------------|
| Image Size | 2GB | ~1GB | **-50% (1GB)** |
| Pull Time | 3-5 min | 1-2 min | **-60%** |
| Startup Time | 5-6s | 5-7s | Slight increase |
| Functionality | Full | Full | Risk of regressions |
| Compatibility | All | Alpine only | Significant testing required |
### Compression Comparison (Not Recommended)
| Method | Size | Savings | Trade-offs |
|--------|------|---------|------------|
| **File cleanup** | ~1.5GB | 500MB | None |
| UPX compression | ~1.9GB | 100MB | +0.6s startup, compatibility risk |
| Transfer compression | 600MB | 1.4GB | Transfer only, no runtime benefit |
## Related Files
| File | Current State | Changes Needed |
|------|---------------|----------------|
| `gns3server/agent/web_wireshark/docker/Dockerfile` | Current 2GB image | Create optimized variant |
| `gns3server/agent/web_wireshark/setup_wireshark_image.py` | Pulls/builds current image | Support optimized image option |
| `gns3server/schemas/config.py` | WebWiresharkSettings | Add image variant config |
| `gns3server/agent/web_wireshark/WEB_WIRESHARK.md` | Documents current image | Update with optimization notes |
## Status
### Phase 1: Safe Optimizations
- [ ] Create `Dockerfile.optimized` with slim base + --no-install-recommends + cleanup
- [ ] Local build and size verification
- [ ] Functional testing (Wireshark, xpra, WebSocket)
- [ ] Integration testing with GNS3 server
- [ ] Document actual size reduction achieved
- [ ] Deploy to staging for 1-week observation
- [ ] Promote to production if stable
### Phase 2: Experimental (Only if Phase 1 insufficient)
- [ ] Research Alpine Wireshark/xpra package availability
- [ ] Prototype Alpine build if feasible
- [ ] Extensive compatibility testing
- [ ] Performance benchmarking vs. Phase 1
## Notes
- All size estimates based on actual `docker history` and container filesystem analysis
- Phase 1 optimizations are conservative and should be safe
- Compression techniques (UPX, layer squashing) provide minimal benefit (1-5%)
- File cleanup is **10x more effective** than compression (25% vs 2.5%)
- Phase 2 requires significant research and testing effort
- Backward compatibility must be maintained during transition
- Consider maintaining both `latest` and `optimized` tags during migration period
### Key Findings from Analysis
1. **Major space waste**: 570MB of cleanable files (28.5% of image)
- Locale files: 151MB → 6MB (keep only en_US)
- ibus input framework: 130MB → 0MB (not needed in headless container)
- Documentation: 76MB → 0MB (docs, man pages, help)
- Desktop components: 37MB → 0MB (backgrounds, sounds)
- Development packages: 50MB → 0MB (21 -dev packages)
2. **Compression not viable**:
- Binaries already stripped (no debug symbols)
- UPX only saves 20MB (1%) with 0.6s startup penalty
- Docker already compresses layers internally
- Resource files (fonts, themes) don't compress well
3. **Best approach**: Clean up unnecessary files rather than compress
- 25x better compression ratio than UPX
- No performance impact
- Simpler build process
- Better compatibility
+4
View File
@@ -1,3 +1,7 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
<!DOCTYPE html>
<html>
+6 -2
View File
@@ -1,15 +1,19 @@
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
See LICENSE file for licensing information.
-->
<!DOCTYPE html>
<html>
<head>
<link type="text/css" rel="stylesheet" href="https://cdn.jsdelivr.net/npm/swagger-ui-dist@3.30.0/swagger-ui.css">
<link type="text/css" rel="stylesheet" href="/static/swagger-ui.css">
<link rel="shortcut icon" href="https://fastapi.tiangolo.com/img/favicon.png">
<title>GNS3 controller API - Swagger UI</title>
</head>
<body>
<div id="swagger-ui">
</div>
<script src="https://cdn.jsdelivr.net/npm/swagger-ui-dist@3.30.0/swagger-ui-bundle.js"></script>
<script src="/static/swagger-ui-bundle.js"></script>
<!-- `SwaggerUIBundle` is now available on the page -->
<script>
const ui = SwaggerUIBundle({
+105
View File
@@ -0,0 +1,105 @@
#
# Copyright (C) 2025 GNS3 Technologies Inc.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""
Agent module with optional AI Copilot and MCP support.
This module provides the AI Copilot and MCP (Model Context Protocol)
functionality as optional features. If the respective dependencies are
not installed, the affected features will be disabled but will not
prevent the server from starting.
Installation:
pip install gns3-server[ai-features] # Install all AI features
"""
import logging
import threading
# Feature flag: AI Copilot is available
AI_COPILOT_AVAILABLE = False
# Try to import AI Copilot components
try:
from .gns3_copilot.project_agent_manager import get_project_agent_manager
from .gns3_copilot.project_agent_manager import ProjectAgentManager
AI_COPILOT_AVAILABLE = True
# Start skills repository initialization in background.
# Clones/pulls GNS3-Skills during server startup.
# Git network timeouts are configured in SkillsManager (_GIT_TIMEOUT_ENV).
# The daemon thread auto-terminates when the process exits.
from gns3server.agent.gns3_copilot.skills.registry import _ensure_skills_manager
threading.Thread(target=_ensure_skills_manager, daemon=True).start()
except ImportError as e:
# AI dependencies not installed, disable AI Copilot feature
logging.warning(
f"AI Copilot dependencies not installed: {e}. "
"AI features will be disabled. "
"Install with: pip install gns3-server[ai-features]"
)
AI_COPILOT_AVAILABLE = False
# Provide stub functions that raise helpful errors
async def get_project_agent_manager():
"""
Get the global ProjectAgentManager singleton instance.
Raises:
RuntimeError: If AI Copilot dependencies are not installed
"""
raise RuntimeError(
"AI Copilot is not available. "
"Install AI dependencies with: pip install gns3-server[ai-features]"
)
class ProjectAgentManager:
"""
Stub class for ProjectAgentManager when AI dependencies are not installed.
"""
def __init__(self):
raise RuntimeError(
"AI Copilot is not available. "
"Install AI dependencies with: pip install gns3-server[ai-features]"
)
# Feature flag: MCP (Model Context Protocol) is available
MCP_AVAILABLE = False
# Try to import MCP dependencies
try:
import mcp.server.fastmcp # noqa: F401 — test import only
MCP_AVAILABLE = True
except ImportError:
# MCP dependencies not installed, disable MCP feature
logging.warning(
"MCP dependencies not installed. "
"MCP features will be disabled. "
"Install with: pip install gns3-server[ai-features]"
)
MCP_AVAILABLE = False
__all__ = [
"AI_COPILOT_AVAILABLE",
"MCP_AVAILABLE",
"get_project_agent_manager",
"ProjectAgentManager",
]
+44
View File
@@ -0,0 +1,44 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
GNS3-Copilot - AI-powered network automation assistant for GNS3.
This package provides a command-line interface for launching the GNS3-Copilot
Streamlit application with support for Streamlit parameter passthrough.
"""
# Dynamic version management
__version__: str = "unknown"
try:
from importlib.metadata import version
__version__ = str(version("gns3-copilot"))
except Exception:
__version__ = "unknown"
__author__ = "Yue Guobin (岳国宾)"
__description__ = "AI-powered network automation assistant for GNS3"
__url__ = "https://github.com/yueguobin/gns3-copilot"
@@ -0,0 +1,50 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
GNS3-Copilot Agent Package
This package contains the main GNS3-Copilot agent implementation for
network automation tasks using LangGraph workflow orchestration.
"""
from .gns3_copilot import agent_builder
# Dynamic version management
try:
from importlib.metadata import version
__version__ = version("gns3-copilot")
except Exception:
__version__ = "unknown"
__author__ = "Yue Guobin (岳国宾)"
__description__ = "AI-powered network automation assistant for GNS3"
__url__ = "https://github.com/yueguobin/gns3-copilot"
__all__ = [
"agent_builder",
]
@@ -0,0 +1,641 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
Context Manager for GNS3-Copilot - Using LangGraph pre_model_hook
This module provides context window management using LangGraph's recommended
pre_model_hook approach with accurate token counting:
Key Features:
- Accurate token counting using tiktoken (95%+ accuracy)
- Template variable injection for topology info
- Context strategy ratios (conservative/balanced/aggressive)
- Message trimming using LangChain's native trim_messages
- Tool definition token estimation
- Detailed token breakdown logging
Requirements:
- tiktoken>=0.8.0 (required)
"""
import json
import logging
import os
import warnings
from pathlib import Path
from typing import Any
from typing import Callable
# Configure tiktoken cache directory (must be set before importing tiktoken)
_cache_dir = Path(__file__).parent.parent / "cache" / "tiktoken"
_cache_dir.mkdir(parents=True, exist_ok=True)
os.environ["TIKTOKEN_CACHE_DIR"] = str(_cache_dir)
import tiktoken
from langchain_core.messages import BaseMessage
from langchain_core.messages import SystemMessage
from langchain_core.messages import trim_messages
logger = logging.getLogger(__name__)
# ============================================================================
# Token Counter Setup
# ============================================================================
# Initialize tiktoken encoding (required dependency)
import time
logger.debug("Initializing tiktoken encoding (cl100k_base)...")
logger.debug(f"Cache directory: {_cache_dir}")
logger.debug("This may take a moment on first run (downloading ~1.6MB encoding file from openaipublic.blob.core.windows.net)")
start_time = time.time()
_tiktoken_encoding = tiktoken.get_encoding("cl100k_base")
elapsed = time.time() - start_time
logger.debug(f"✓ tiktoken encoding loaded successfully (took {elapsed:.2f}s)")
# ============================================================================
# Constants
# ============================================================================
# Context strategy ratios (percentage of context limit to use for input)
CONTEXT_STRATEGY_RATIOS = {
"conservative": 0.60,
"balanced": 0.75,
"aggressive": 0.85,
}
DEFAULT_CONTEXT_STRATEGY = "balanced"
# Token unit conversion
TOKENS_PER_K = 1000
# ============================================================================
# Token Counting Functions
# ============================================================================
def count_tokens(text: str) -> int:
"""
Count tokens in text using tiktoken.
Uses cl100k_base encoding (GPT-4) which provides accurate counting
for most modern LLMs (OpenAI, Anthropic, DeepSeek, etc.).
Args:
text: Text to count tokens for
Returns:
Exact token count (not estimated)
"""
if not text:
return 0
return len(_tiktoken_encoding.encode(text))
def estimate_tool_tokens(tools: list[Any]) -> int:
"""
Estimate token consumption for tool definitions.
Tool definitions are serialized as JSON and sent to LLM with each request.
This function estimates their token cost.
Args:
tools: List of LangChain tool objects
Returns:
Estimated token count for all tool definitions
"""
if not tools:
return 0
total_tokens = 0
for tool in tools:
try:
# Build tool schema in OpenAI format
tool_schema = {
"type": "function",
"function": {
"name": tool.name,
"description": tool.description,
},
}
# Add parameters schema if available
if hasattr(tool, "args_schema") and tool.args_schema:
try:
# Try Pydantic v2 method (model_json_schema)
tool_schema["function"]["parameters"] = (
tool.args_schema.model_json_schema()
)
except AttributeError:
# Fallback to Pydantic v1 method (schema)
try:
tool_schema["function"]["parameters"] = (
tool.args_schema.schema()
)
except Exception:
# Both methods failed, use empty schema
tool_name = getattr(tool, "name", "unknown")
logger.debug(
"Failed to get schema for tool %s, using empty "
"parameters",
tool_name,
)
tool_schema["function"]["parameters"] = {}
except Exception as e:
# model_json_schema() raised an exception
tool_name = getattr(tool, "name", "unknown")
logger.debug(
"model_json_schema() failed for tool %s: %s, "
"trying v1 fallback",
tool_name,
e,
)
try:
tool_schema["function"]["parameters"] = (
tool.args_schema.schema()
)
except Exception:
tool_schema["function"]["parameters"] = {}
# Serialize to JSON and count tokens
schema_str = json.dumps(tool_schema, ensure_ascii=False)
tokens = count_tokens(schema_str)
total_tokens += tokens
except Exception as e:
tool_name = getattr(tool, "name", "unknown")
logger.debug(
"Failed to estimate tokens for tool %s: %s",
tool_name,
e,
)
# Rough fallback: 1000 tokens per tool
total_tokens += 1000
return total_tokens
def _count_tokens_for_message(message: BaseMessage) -> int:
"""
Token counter for LangChain's trim_messages.
This function is called by trim_messages for each message.
Args:
message: A single message (HumanMessage, AIMessage, SystemMessage,
etc.)
Returns:
Estimated token count for the message
"""
content = ""
if hasattr(message, "content") and message.content:
content = str(message.content)
return count_tokens(content)
# ============================================================================
# Pre-Model Hook Factory
# ============================================================================
def create_pre_model_hook(
system_prompt: str,
get_topology_func: Callable[[dict], Any] | None = None,
get_llm_config_func: Callable[[], dict[str, Any] | None] | None = None,
get_tools_func: Callable[[], list[Any]] | None = None,
) -> Callable[[dict], dict]:
"""
Create a pre_model_hook function for LangGraph agent.
This hook will be automatically called before each LLM invocation,
handling topology injection, tool token estimation, and message
trimming.
IMPORTANT USAGE REQUIREMENTS:
This hook MUST be passed via model.invoke() config, NOT used as
a Node return value.
CORRECT Usage:
model.invoke(messages,
config={"configurable": {"pre_model_hook": pre_hook}})
INCORRECT Usage:
# ❌ Don't use as Node return value
def my_node(state):
return pre_hook(state) # Wrong!
# ❌ Don't use with StateGraph if state uses add_messages reducer
class State(TypedDict):
messages: Annotated[list, add_messages] # Incompatible!
COMPATIBILITY:
- Works with: model.invoke(config={"configurable":
{"pre_model_hook": ...}})
- Does NOT work as: StateGraph Node return value
- Does NOT work with: Annotated[list, add_messages] state reducers
The hook returns a complete message list that overwrites the model
input, which is correct for invoke() but wrong for state updates with
add_messages.
Args:
system_prompt: System prompt template (must contain
{{topology_info}} placeholder)
get_topology_func: Function to extract topology from state
get_llm_config_func: Function to get LLM config from context
get_tools_func: Optional function to get tools list for token
estimation
Returns:
A pre_model_hook function for use with model.invoke(config=...)
Thread Safety:
Thread-safe if get_llm_config_func uses request-scoped context
(e.g., contextvars)
Performance Notes:
- SystemMessage is reconstructed on each LLM call (intentional
design)
- Overhead: ~1-2ms per call (negligible compared to LLM latency)
- Trade-off: Simplicity > micro-optimization
- In ReAct loops with multiple LLM calls, topology is re-injected
each time
- This is acceptable for GNS3-Copilot's usage patterns (low
concurrency, short conversations)
"""
def pre_model_hook(state: dict) -> dict:
"""
LangGraph pre_model_hook - called before each LLM invocation.
This function is NOT a StateGraph Node. It is a preprocessing hook
that modifies the input to the LLM, not the agent state.
Usage Context:
Called automatically by LangChain when passed via:
model.invoke(messages,
config={"configurable":
{"pre_model_hook": this}})
Args:
state: Current agent state containing messages and topology_info
Returns:
dict with 'messages' key containing prepared and trimmed messages
Note: This return value is used by LangChain to replace the model
input, NOT to update the agent state.
Raises:
ValueError: If context_limit is missing or invalid
"""
logger.debug(
"pre_model_hook invoked: messages=%d",
len(state.get("messages", [])),
)
messages = state.get("messages", [])
if not messages:
logger.debug("No messages in state, returning empty list")
return {"messages": []}
# Get LLM config
llm_config = get_llm_config_func() if get_llm_config_func else None
if not llm_config:
logger.error("LLM config not found. context_limit is required.")
raise ValueError(
"LLM config not found. context_limit is required."
)
if "context_limit" not in llm_config:
logger.error(
"context_limit not found in LLM config. "
"This is a required field. Please configure context_limit."
)
raise ValueError("context_limit is required in LLM config")
limit = llm_config["context_limit"]
if not isinstance(limit, int) or limit <= 0:
logger.error("Invalid context_limit: %s", limit)
raise ValueError(f"Invalid context_limit: {limit}")
context_limit_k = limit
strategy = llm_config.get("context_strategy", DEFAULT_CONTEXT_STRATEGY)
if strategy not in CONTEXT_STRATEGY_RATIOS:
logger.warning(
"Invalid context_strategy '%s', using '%s'",
strategy,
DEFAULT_CONTEXT_STRATEGY,
)
strategy = DEFAULT_CONTEXT_STRATEGY
# Step 1: Estimate tool tokens
tool_tokens = 0
if get_tools_func:
try:
tools = get_tools_func()
tool_tokens = estimate_tool_tokens(tools)
logger.debug(
"Tool definitions estimated at ~%d tokens (%d tools)",
tool_tokens,
len(tools),
)
except Exception as e:
logger.warning("Failed to estimate tool tokens: %s", e)
# Step 2: Inject topology into system prompt
messages_with_system = _inject_topology_into_system(
messages=messages,
system_prompt=system_prompt,
state=state,
get_topology_func=get_topology_func,
)
# Step 3: Calculate token breakdown for logging and validation
system_message = messages_with_system[0]
system_tokens = _count_tokens_for_message(system_message)
# Calculate tokens for messages_with_system (including system)
messages_with_system_tokens = sum(
_count_tokens_for_message(m) for m in messages_with_system
)
# Calculate available budget
model_limit_tokens = context_limit_k * TOKENS_PER_K
strategy_ratio = CONTEXT_STRATEGY_RATIOS[strategy]
max_input_tokens = int(model_limit_tokens * strategy_ratio)
# Calculate max tokens for trim_messages
# Important: trim_messages counts ALL messages (including system),
# so we only subtract tool_tokens here, NOT system_tokens.
# Let trim_messages handle system.
max_tokens_for_trim = max_input_tokens - tool_tokens
# Validate budget and provide actionable warnings
if system_tokens + tool_tokens > max_input_tokens: # noqa: E501
logger.error(
"System prompt (%d tokens) + tools (%d tokens) EXCEED input "
"budget (%d tokens). This will likely cause LLM call failures. " # noqa: E501
"Recommendations: 1) Reduce system prompt length, 2) Reduce "
"number of tools, 3) Use a model with larger context window, "
"or 4) Switch to 'conservative' strategy.",
system_tokens,
tool_tokens,
max_input_tokens,
)
elif max_tokens_for_trim < system_tokens * 1.5:
# Less than 1.5x system tokens means very little room for history
logger.warning(
"System prompt (%d tokens) + tools (%d tokens) leave minimal "
"room for conversation history (%d tokens remaining). "
"Consider reducing system prompt length or number of tools.",
system_tokens,
tool_tokens,
max_tokens_for_trim - system_tokens,
)
# Debug log with clear terminology
logger.debug(
"Token breakdown: system=%d, all_messages=%d (system+history), "
"tools=%d, trim_budget=%d (limit=%dK, strategy=%s)",
system_tokens,
messages_with_system_tokens,
tool_tokens,
max_tokens_for_trim,
context_limit_k,
strategy,
)
# Step 4: Trim messages to fit
# Note: max_tokens_for_trim INCLUDES system message, trim_messages
# will handle it
try:
trimmed = trim_messages(
messages=messages_with_system,
max_tokens=max_tokens_for_trim,
strategy="last",
token_counter=_count_tokens_for_message,
include_system=True,
)
# Calculate final token counts
final_total = sum(_count_tokens_for_message(m) for m in trimmed)
usage_percent = (
(final_total + tool_tokens) / model_limit_tokens * 100
)
if len(trimmed) < len(messages_with_system):
logger.info(
"Messages trimmed: %d → %d msgs. Total: ~%d tokens + %d "
"tools = %d / %dK (%.1f%%), strategy=%s",
len(messages_with_system),
len(trimmed),
final_total,
tool_tokens,
final_total + tool_tokens,
context_limit_k,
usage_percent,
strategy,
)
else:
logger.info(
"Context ready: %d msgs, ~%d tokens + %d tools = %d / %dK "
"(%.1f%%), strategy=%s",
len(trimmed),
final_total,
tool_tokens,
final_total + tool_tokens,
context_limit_k,
usage_percent,
strategy,
)
return {"messages": trimmed}
except Exception as e:
logger.error("Failed to trim messages: %s", e)
logger.warning("Returning original messages due to trimming error")
return {"messages": messages_with_system}
return pre_model_hook
def _inject_topology_into_system(
messages: list,
system_prompt: str,
state: dict,
get_topology_func: Callable[[dict], Any] | None = None,
) -> list:
"""
Inject topology information into system prompt.
Args:
messages: Message list
system_prompt: System prompt template
state: Current state
get_topology_func: Optional function to extract topology
Returns:
Message list with system message prepended
"""
if "{{topology_info}}" not in system_prompt:
logger.warning("system_prompt missing {{topology_info}} placeholder")
return messages
topology_data = None
if get_topology_func:
try:
topology_data = get_topology_func(state)
except Exception as e:
logger.warning("Failed to get topology: %s", e)
if topology_data:
topology_str = str(topology_data)
formatted_prompt = system_prompt.replace(
"{{topology_info}}", f"\n\n## Current Topology\n{topology_str}"
)
logger.info(
"✓ Topology injected: %d chars, nodes: %s",
len(topology_str),
list(topology_data.get("nodes", {}).keys())[:5],
) # Show first 5 node names
logger.debug(
"Full topology data: %s", topology_str[:500]
) # First 500 chars
else:
formatted_prompt = system_prompt.replace(
"{{topology_info}}", "(No topology information available)"
)
logger.warning("✗ Topology data is None, injecting placeholder")
# Filter out existing SystemMessage instances
non_system_messages = [
m for m in messages if not isinstance(m, SystemMessage)
]
filtered_count = len(messages) - len(non_system_messages)
if filtered_count > 0:
logger.debug(
"Filtered out %d existing SystemMessage(s)", filtered_count
)
return [SystemMessage(content=formatted_prompt)] + non_system_messages
# ============================================================================
# Legacy Compatibility (Deprecated)
# ============================================================================
def prepare_context_messages(
state_messages: list[Any],
system_prompt: str,
topology_context: str | None = None,
model_name: str = "gpt-4o",
llm_config: dict[str, Any] | None = None,
tools: list[Any] | None = None,
) -> list[Any]:
"""
**DEPRECATED**: Use create_pre_model_hook() instead.
"""
warnings.warn(
"prepare_context_messages() is deprecated. Use "
"create_pre_model_hook() instead.",
DeprecationWarning,
stacklevel=2,
)
if "{{topology_info}}" not in system_prompt:
formatted_prompt = system_prompt
elif topology_context:
formatted_prompt = system_prompt.replace(
"{{topology_info}}", f"\n\n## Current Topology\n{topology_context}"
)
else:
formatted_prompt = system_prompt.replace(
"{{topology_info}}", "(No topology information available)"
)
return [SystemMessage(content=formatted_prompt)] + state_messages
# ============================================================================
# Module Test
# ============================================================================
if __name__ == "__main__":
from langchain_core.messages import HumanMessage
print("=== Context Manager Module Test ===\n")
# Test token counting
print("Test 1: Token Counting")
print(" tiktoken encoding: cl100k_base")
test_text = "Hello world 你好世界"
tokens = count_tokens(test_text)
print(f" '{test_text}' → {tokens} tokens")
# Test hook creation
print("\nTest 2: Create pre_model_hook")
system_prompt = "You are GNS3 Copilot.\n\n{{topology_info}}"
def mock_get_topology(state):
return {"project_id": "test", "nodes": 5}
def mock_get_config():
return {"context_limit": 8, "context_strategy": "conservative"}
def mock_get_tools():
return [] # No tools for test
hook = create_pre_model_hook(
system_prompt=system_prompt,
get_topology_func=mock_get_topology,
get_llm_config_func=mock_get_config,
get_tools_func=mock_get_tools,
)
print("✓ pre_model_hook created successfully")
# Test invocation
print("\nTest 3: Invoke pre_model_hook")
test_state = {
"messages": [
HumanMessage(f"Message {i}: {'x' * 50}") for i in range(5)
],
"topology_info": {"project_id": "test123", "nodes": 3},
}
result = hook(test_state)
print(f"✓ Hook invoked: {len(result['messages'])} messages returned")
print("\n=== All Tests Passed ===")
@@ -0,0 +1,715 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
# mypy: ignore-errors
"""
GNS3 Network Automation Assistant - LangGraph Agent
This module implements the core LangGraph agent workflow for GNS3-Copilot,
an AI-powered assistant for GNS3 network automation and management.
The agent provides:
- LangGraph-based state management and workflow
- Mode-aware tool orchestration for GNS3 operations
- Context-aware conversation handling
- Automatic conversation title generation
- Integration with GNS3 topology management
Copilot Modes:
- "teaching_assistant" (default): Diagnostic tools only, no configuration
changes
- "lab_automation_assistant": Full diagnostic and configuration tools
"""
# Standard library imports
import json
import logging
import operator
from datetime import datetime
from typing import Annotated
from typing import Literal
# Third-party imports
from langchain.messages import AnyMessage
from langchain.messages import SystemMessage
from langchain.messages import ToolMessage
from langchain_core.runnables import RunnableConfig
from langgraph.graph import END
from langgraph.graph import START
from langgraph.graph import StateGraph
from langgraph.managed.is_last_step import RemainingSteps
from typing_extensions import TypedDict
# Local imports
from gns3server.agent.gns3_copilot.agent.context_manager import (
create_pre_model_hook,
)
from gns3server.agent.gns3_copilot.agent.model_factory import (
create_base_model_with_tools,
)
from gns3server.agent.gns3_copilot.agent.model_factory import (
create_title_model,
)
from gns3server.agent.gns3_copilot.gns3_client import GNS3TopologyTool
from gns3server.agent.gns3_copilot.gns3_client.context_helpers import (
get_current_llm_config,
)
from gns3server.agent.gns3_copilot.prompts import load_system_prompt
from gns3server.agent.gns3_copilot.skills.registry import get_prompt
from gns3server.agent.gns3_copilot.tools_v2 import (
ExecuteMultipleDeviceCommands,
)
from gns3server.agent.gns3_copilot.tools_v2 import (
ExecuteMultipleDeviceConfigCommands,
)
from gns3server.agent.gns3_copilot.tools_v2 import GNS3CreateNodeTool
from gns3server.agent.gns3_copilot.tools_v2 import GNS3LinkTool
from gns3server.agent.gns3_copilot.tools_v2 import GNS3PacketFilterTool
from gns3server.agent.gns3_copilot.tools_v2 import GNS3StartNodeTool
from gns3server.agent.gns3_copilot.tools_v2 import GNS3StopNodeTool
from gns3server.agent.gns3_copilot.tools_v2 import GNS3SuspendNodeTool
from gns3server.agent.gns3_copilot.tools_v2 import GNS3TemplateTool
from gns3server.agent.gns3_copilot.tools_v2 import GNS3UpdateNodeNameTool
from gns3server.agent.gns3_copilot.tools_v2.vpcs_tools_netmiko import VPCSCommands
from gns3server.agent.gns3_copilot.tools_v2 import PacketAnalysisTool
from gns3server.agent.gns3_copilot.skills import DeviceSkillsTool
from gns3server.agent.gns3_copilot.skills import InjectionSkillsTool
from gns3server.agent.gns3_copilot.skills import PacketAnalysisSkillsTool
# Set up logger for GNS3-Copilot
logger = logging.getLogger(__name__)
# Note: LLM model configuration is now managed by the new llm_model_configs
# system. The model_factory module handles model creation with configuration
# from the database.
# Define tools for different copilot modes
# Teaching assistant mode: READ-ONLY diagnostic tools only
TEACHING_ASSISTANT_MODE_TOOLS = [
GNS3TemplateTool(), # Get GNS3 node templates
GNS3CreateNodeTool(), # Create new nodes in GNS3
GNS3LinkTool(), # Create links between nodes
GNS3StartNodeTool(), # Start GNS3 nodes
GNS3UpdateNodeNameTool(), # Update node name
ExecuteMultipleDeviceCommands(), # Execute show/display/debug commands
# (READ-ONLY)
PacketAnalysisTool(), # Protocol-oriented packet analysis with tshark
PacketAnalysisSkillsTool(), # Query packet analysis protocol definitions
DeviceSkillsTool(), # Get device-specific skills and command knowledge
]
# Lab automation assistant mode: Full diagnostic AND configuration tools
LAB_AUTOMATION_ASSISTANT_MODE_TOOLS = [
GNS3TemplateTool(), # Get GNS3 node templates
GNS3CreateNodeTool(), # Create new nodes in GNS3
GNS3LinkTool(), # Create links between nodes
GNS3StartNodeTool(), # Start GNS3 nodes
GNS3StopNodeTool(), # Stop GNS3 nodes
GNS3SuspendNodeTool(), # Suspend GNS3 nodes (preserve state)
GNS3UpdateNodeNameTool(), # Update node name
ExecuteMultipleDeviceCommands(), # Execute show/display/debug commands
# (READ-ONLY)
ExecuteMultipleDeviceConfigCommands(), # Execute configuration commands
VPCSCommands(), # Execute VPCS commands using Netmiko
PacketAnalysisTool(), # Protocol-oriented packet analysis with tshark
PacketAnalysisSkillsTool(), # Query packet analysis protocol definitions
DeviceSkillsTool(), # Get device-specific skills and command knowledge
]
# Troubleshooting injection mode: Tools for injecting network faults
# Focused on configuration commands and injection skills for fault injection
TROUBLESHOOTING_INJECTION_MODE_TOOLS = [
ExecuteMultipleDeviceCommands(), # Get device configurations (READ-ONLY)
ExecuteMultipleDeviceConfigCommands(), # Inject configuration changes
GNS3PacketFilterTool(), # Manage packet filters on links (delay, loss, corrupt, etc.)
InjectionSkillsTool(), # Query injection skills and fault types
GNS3TopologyTool(), # Get topology information
]
# Default tools (legacy support - will be overridden by mode-specific tools)
tools = LAB_AUTOMATION_ASSISTANT_MODE_TOOLS
# Create combined tool lookup for tool_node (supports all modes)
# tool_node will receive tool calls based on mode-specific tools bound to the
# model
# Combine all tools from all modes and deduplicate by tool name
ALL_TOOLS = LAB_AUTOMATION_ASSISTANT_MODE_TOOLS
tools_by_name = {tool.name: tool for tool in ALL_TOOLS}
# Add troubleshooting injection tools to the global tool registry
for tool in TROUBLESHOOTING_INJECTION_MODE_TOOLS:
if tool.name not in tools_by_name:
tools_by_name[tool.name] = tool
# Log application startup
logger.info("GNS3-Copilot application starting up")
# Constants for conversation title management
DEFAULT_CONVERSATION_TITLE = "New Conversation"
UNTITLED_SESSION_FALLBACK = "Untitled Session"
TITLE_MAX_LENGTH = 40
# Abort flags storage for session-level abort tracking
# This is checked by conditional edge functions to stop the graph
_abort_flags: dict[str, bool] = {}
def check_abort_flag(session_id: str) -> bool:
"""
Check if abort flag is set for a session.
Args:
session_id: Session identifier
Returns:
True if abort is requested, False otherwise
"""
return _abort_flags.get(session_id, False)
def set_abort_flag(session_id: str):
"""
Set abort flag for a session.
Args:
session_id: Session identifier
"""
_abort_flags[session_id] = True
logger.debug("Abort flag set for session: %s", session_id)
def clear_abort_flag(session_id: str):
"""
Clear abort flag for a session.
Args:
session_id: Session identifier
"""
_abort_flags[session_id] = False
logger.debug("Abort flag cleared for session: %s", session_id)
# Define state
class MessagesState(TypedDict):
"""
GNS3-Copilot conversation state management class.
Maintains the conversation state for the LangGraph workflow, including
message history, call counters, and session titles for comprehensive
dialogue management.
Attributes:
messages: List of conversation messages with cumulative updates using
operator.add
llm_calls: Counter for tracking the number of LLM invocations
remaining_steps: Is automatically managed by LangGraph's RemainingSteps
to track and limit recursion depth.
conversation_title: Optional conversation title for session
identification and management
topology_info: Dictionary containing GNS3 project topology information
session_id: Session identifier for abort tracking
abort: Flag to signal abort request
"""
messages: Annotated[list[AnyMessage], operator.add]
llm_calls: int
remaining_steps: RemainingSteps
# Optional conversation title
conversation_title: str | None
# Store GNS3 topology information
topology_info: dict | None
# Session identifier for abort tracking
session_id: str | None
# Abort flag to signal stop request
abort: bool
# Define llm call node
def llm_call(state: dict, config: RunnableConfig | None = None):
"""
LLM decides whether to call a tool or not.
Uses pre_model_hook pattern for automatic topology injection and
message trimming, ensuring separation of concerns and complete
history preservation in state["messages"].
"""
logger.debug("LLM call node invoked")
# Get llm_config from request-scoped context variable
llm_config = get_current_llm_config()
if not llm_config:
logger.error("LLM config not found in context")
return {
"messages": [],
"llm_calls": state.get("llm_calls", 0),
"topology_info": None,
}
logger.debug(
"LLM config retrieved from context: provider=%s, model=%s",
llm_config.get("provider"),
llm_config.get("model"),
)
# Defensive check: skip LLM call if no user messages
messages = state.get("messages", [])
if not messages or len(messages) == 0:
logger.warning("No messages in state, skipping LLM call")
return {
"messages": [],
"llm_calls": state.get("llm_calls", 0),
"topology_info": None,
}
# Get project_id from config configurable (set when starting the chat)
project_id = None
topology_info = None
if config and config.get("configurable"):
project_id = config["configurable"].get("project_id")
# Retrieve topology information if available
if project_id:
try:
topology_tool = GNS3TopologyTool()
topology = topology_tool._run(project_id=project_id)
if topology and "error" not in topology:
topology_info = topology
logger.info(
"Successfully retrieved topology for project_id: %s, "
"name: %s",
project_id,
topology.get("name"),
)
else:
logger.warning(
"Failed to retrieve topology for project_id %s: %s",
project_id,
topology.get("error", "Unknown error")
if topology
else "No result",
)
except Exception as e:
logger.warning(
"Error retrieving topology for project_id %s: %s",
project_id,
e,
)
# Store topology_info in state for pre_model_hook to access
state["topology_info"] = topology_info
# Select tools based on copilot_mode
copilot_mode = llm_config.get("copilot_mode", "teaching_assistant").lower()
if copilot_mode == "lab_automation_assistant":
mode_tools = LAB_AUTOMATION_ASSISTANT_MODE_TOOLS
logger.info(
"Using LAB_AUTOMATION_ASSISTANT mode tools (includes "
"configuration tools)"
)
elif copilot_mode == "troubleshooting_injection":
mode_tools = TROUBLESHOOTING_INJECTION_MODE_TOOLS
logger.info(
"Using TROUBLESHOOTING_INJECTION mode tools (fault injection)"
)
else: # teaching_assistant mode (default)
mode_tools = TEACHING_ASSISTANT_MODE_TOOLS
logger.info(
"Using TEACHING_ASSISTANT mode tools (diagnostic tools only)"
)
# Create pre_model_hook for automatic topology injection and trimming
# Load system prompt based on copilot_mode configuration
system_prompt = load_system_prompt(llm_config)
pre_hook = create_pre_model_hook(
system_prompt=system_prompt,
get_topology_func=lambda s: s.get("topology_info"),
get_llm_config_func=get_current_llm_config,
get_tools_func=lambda: mode_tools, # Pass mode-specific tools for
# token estimation
)
# Create fresh model with tools for each LLM call
logger.debug(
"Creating model with tools: provider=%s, model=%s, mode=%s, tools=%d",
llm_config.get("provider"),
llm_config.get("model"),
copilot_mode,
len(mode_tools),
)
model_with_tools = create_base_model_with_tools(
mode_tools, llm_config=llm_config
)
# Call pre_hook directly to prepare messages (topology injection +
# trimming)
# Note: LangGraph's pre_model_hook only works with prebuilt agents, not
# custom StateGraph
logger.debug("Calling pre_hook to prepare %d messages", len(messages))
prepared_state = pre_hook(
{"messages": messages, "topology_info": topology_info}
)
prepared_messages = prepared_state["messages"]
logger.info(
"Messages prepared: %d → %d", len(messages), len(prepared_messages)
)
# Invoke model with prepared messages
response = model_with_tools.invoke(prepared_messages)
# Add metadata with created_at timestamp to AI response
if hasattr(response, "metadata"):
existing_metadata = response.metadata or {}
response.metadata = {
**existing_metadata,
"created_at": datetime.utcnow().isoformat(),
}
else:
# LangChain messages should have metadata attribute, but defensive
# fallback
try:
response.metadata = {"created_at": datetime.utcnow().isoformat()}
except Exception:
logger.warning("Could not add metadata to AI response")
logger.info(
"LLM call completed: tool_calls=%d",
len(response.tool_calls) if hasattr(response, "tool_calls") else 0,
)
return {
"messages": [response],
"llm_calls": state.get("llm_calls", 0) + 1,
"topology_info": topology_info,
"session_id": state.get("session_id"),
}
# Define generate title node
def generate_title(
state: MessagesState, config: RunnableConfig | None = None
) -> dict:
"""
Generate a conversation title using a lightweight assistant LLM
(title_model). This node is only executed when no title has been set yet
(first round only).
"""
# Get llm_config from request-scoped context variable
llm_config = get_current_llm_config()
if not llm_config:
logger.error("LLM config not found in context, cannot generate title")
return {"conversation_title": UNTITLED_SESSION_FALLBACK, "session_id": state.get("session_id")}
# Only generate a title if it hasn't been set yet
current_title = state.get("conversation_title")
if current_title in [None, "New Conversation"]:
logger.debug("Title generation triggered for session")
messages = state["messages"]
# Load title prompt from external repository
title_prompt = get_prompt("title")
if not title_prompt:
logger.error("Title prompt not found in external repository")
return {"conversation_title": UNTITLED_SESSION_FALLBACK, "session_id": state.get("session_id")}
# Build the prompt for title generation
title_prompt_messages = [
SystemMessage(content=title_prompt),
messages[0], # User's first message
messages[-1], # Assistant's final response in this turn
]
# Call the title generation model (create fresh instance for each call)
try:
title_model = create_title_model(llm_config=llm_config)
response = title_model.invoke(
title_prompt_messages,
config={"configurable": {"foo_temperature": 1.0}},
)
raw_content = response.content
new_title = raw_content.strip()
# Validate the generated title
if not new_title or len(new_title) < 3:
raise ValueError(
f"Generated title too short or empty: '{new_title}'"
)
if new_title in [
"New Conversation",
"Untitled Session",
"GNS3 Session",
]:
raise ValueError(
f"Generated title is a default value: '{new_title}'"
)
# Safety: truncate long titles and avoid line breaks
if len(new_title) > TITLE_MAX_LENGTH:
new_title = new_title[: TITLE_MAX_LENGTH - 2] + "..."
# Remove unwanted characters
new_title = (
new_title.replace("\n", " ").replace('"', "").replace("'", "")
)
logger.debug("Generated new title: %s", new_title)
return {"conversation_title": new_title, "session_id": state.get("session_id")}
except Exception as e:
logger.error(f"Title generation failed: {e}, using fallback")
# Improved fallback: Use user's first message content
if messages and len(messages) > 0:
first_message = messages[0]
if hasattr(first_message, "content"):
fallback_title = first_message.content[:30].strip()
# Remove newlines and extra spaces
fallback_title = fallback_title.replace("\n", " ").strip()
# Collapse multiple spaces
while " " in fallback_title:
fallback_title = fallback_title.replace(" ", " ")
# Truncate if needed
if len(fallback_title) > 28:
fallback_title = fallback_title[:28] + ".."
if fallback_title:
logger.info(
"Using fallback title from user message: '%s'",
fallback_title,
)
return {"conversation_title": fallback_title, "session_id": state.get("session_id")}
# Final fallback
logger.info(
"Using final fallback title: '%s'",
UNTITLED_SESSION_FALLBACK,
)
return {"conversation_title": UNTITLED_SESSION_FALLBACK, "session_id": state.get("session_id")}
# Title already exists → no update needed
return {}
# Define tool node
def tool_node(state: dict, config: RunnableConfig | None = None):
"""Performs the tool call"""
tool_calls = state["messages"][-1].tool_calls
logger.debug("Tool node invoked: tool_calls=%d", len(tool_calls))
result = []
for tool_call in tool_calls:
tool_name = tool_call["name"]
logger.debug(
"Executing tool: %s with args: %s", tool_name, tool_call["args"]
)
tool = tools_by_name[tool_name]
try:
observation = tool.invoke(tool_call["args"])
logger.debug(
"Tool %s completed: output_length=%d",
tool_name,
len(str(observation)) if observation else 0,
)
except Exception as e:
logger.error("Tool %s failed: %s", tool_name, e, exc_info=True)
observation = f"Error: {str(e)}"
# Serialize observation to JSON string if it's not already a string
# This ensures ToolMessage.content is always JSON format, not Python
# str()
if not isinstance(observation, str):
observation = json.dumps(observation, ensure_ascii=False, indent=2)
# Create ToolMessage with metadata including created_at
tool_msg = ToolMessage(
content=observation,
tool_call_id=tool_call["id"],
name=tool_call["name"],
metadata={"created_at": datetime.utcnow().isoformat()},
)
result.append(tool_msg)
return {"messages": result, "session_id": state.get("session_id")}
# Abort handler node - provides tool results when aborting
def abort_handler_node(state: dict) -> dict:
"""
Handles abort by providing tool result messages for pending tool_calls.
This ensures message history consistency and prevents checkpoint corruption.
"""
messages = state.get("messages", [])
if not messages:
return {"session_id": state.get("session_id")}
last_message = messages[-1]
if not hasattr(last_message, "tool_calls") or not last_message.tool_calls:
return {"session_id": state.get("session_id")}
result = []
for tool_call in last_message.tool_calls:
tool_msg = ToolMessage(
content=json.dumps({
"status": "aborted",
"message": "Tool execution was aborted by user",
"tool_call_id": tool_call["id"],
}),
tool_call_id=tool_call["id"],
name=tool_call["name"],
metadata={"created_at": datetime.utcnow().isoformat(), "aborted": True},
)
result.append(tool_msg)
logger.info("Abort handler: generated %d aborted tool messages", len(result))
return {"messages": result, "session_id": state.get("session_id")}
# Routing logic after the LLM node
def should_continue(
state: MessagesState,
) -> Literal["tool_node", "title_generator_node", "abort_handler_node", END]:
"""
Determine the next step after the LLM has produced a response.
- If abort flag is set → end the conversation
- If the LLM requested any tool calls → route to tool_node
- If this is the first complete turn (llm_calls == 1) and no title
exists → generate a title
- Otherwise → conversation is complete, go to END
"""
# Check abort flag first
session_id = state.get("session_id")
if session_id and check_abort_flag(session_id):
last_message = state["messages"][-1]
# If there's a tool_calls message, we need to handle it properly
if hasattr(last_message, "tool_calls") and last_message.tool_calls:
return "abort_handler_node"
return END
last_message = state["messages"][-1]
current_title = state.get("conversation_title")
# LLM requested one or more tool executions
if last_message.tool_calls:
return "tool_node"
# First full interaction completed and title not yet generated
if current_title in [None, "New Conversation"]:
return "title_generator_node"
# Normal completion (multi-turn conversation or title already exists)
return END
# Routing logic after the tool node, Check remaining_steps
def recursion_limit_continue(state: MessagesState) -> Literal["llm_call", END]:
"""
Routing logic after tool execution to prevent infinite recursion.
Determines whether to continue with another LLM call or end the
conversation based on remaining steps and message type.
Args:
state: Current conversation state with messages and remaining steps
Returns:
"llm_call" to continue processing, END to terminate conversation
Logic:
- If abort flag is set → end the conversation
- If the last message is ToolMessage and steps >= 4: continue to LLM
- Otherwise: end the conversation to prevent infinite loops
"""
# Check abort flag first
session_id = state.get("session_id")
if session_id and check_abort_flag(session_id):
return END
last_message = state["messages"][-1]
if isinstance(last_message, ToolMessage):
if state["remaining_steps"] < 4:
return END
return "llm_call"
return END
# Build and compile the agent
# Build workflow
agent_builder = StateGraph(MessagesState)
# Add nodes
agent_builder.add_node("llm_call", llm_call)
agent_builder.add_node("tool_node", tool_node)
agent_builder.add_node("title_generator_node", generate_title)
agent_builder.add_node("abort_handler_node", abort_handler_node)
# Add edges to connect nodes
agent_builder.add_edge(START, "llm_call")
# Conditional routing after LLM response
# Determines the next step based on whether LLM needs to call tools or
# generate title
agent_builder.add_conditional_edges(
"llm_call",
should_continue,
{
"tool_node": "tool_node", # Route to tool execution if LLM requested
# tools
"title_generator_node": "title_generator_node", # Generate title on
# first interaction
"abort_handler_node": "abort_handler_node", # Handle abort with tool calls
END: END, # End conversation if no tools needed
},
)
# Conditional routing after tool execution
# Prevents infinite recursion by checking remaining steps before continuing
agent_builder.add_conditional_edges(
"tool_node",
recursion_limit_continue,
{
"llm_call": "llm_call", # Continue to LLM if tools executed and steps
# remain
END: END, # End conversation to prevent infinite loops
},
)
agent_builder.add_edge("title_generator_node", END)
agent_builder.add_edge("abort_handler_node", END)
@@ -0,0 +1,258 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
Model Factory for GNS3-Copilot Agent
This module provides factory functions to create fresh LLM model instances.
Configuration is passed directly from the database.
"""
import logging
from typing import Any
from typing import Optional
from langchain.chat_models import init_chat_model
logger = logging.getLogger(__name__)
def _load_llm_config(
llm_config: Optional[dict[str, Any]] = None,
) -> dict[str, str]:
"""
Convert llm_config dict to model factory format.
Args:
llm_config: Configuration dictionary from database
Returns:
Dictionary containing model configuration.
Raises:
ValueError: If configuration is missing or invalid.
"""
if not llm_config:
raise ValueError("LLM configuration is required")
logger.info(
"Using LLM config: provider=%s, model=%s",
llm_config.get("provider"),
llm_config.get("model"),
)
return {
"model_name": llm_config.get("model", ""),
"model_provider": llm_config.get("provider", ""),
"api_key": llm_config.get("api_key", ""),
"base_url": llm_config.get("base_url", ""),
"temperature": str(llm_config.get("temperature", "0")),
}
def create_base_model(
llm_config: Optional[dict[str, Any]] = None,
) -> Any:
"""
Create a fresh base LLM model instance.
Args:
llm_config: Configuration dictionary from database
Returns:
Any: A new LLM model instance configured with current settings.
The actual type depends on the provider (e.g., ChatOpenAI, etc.).
Raises:
ValueError: If required configuration fields are missing or invalid.
RuntimeError: If model creation fails.
"""
config_vars = _load_llm_config(llm_config)
# Log the loaded configuration (mask sensitive data)
logger.info(
"Creating base model: name=%s, provider=%s, base_url=%s, "
"temperature=%s",
config_vars["model_name"],
config_vars["model_provider"],
config_vars["base_url"] if config_vars["base_url"] else "default",
config_vars["temperature"],
)
# Validate required fields
if not config_vars["model_name"]:
raise ValueError("LLM configuration requires 'model' field")
if not config_vars["model_provider"]:
raise ValueError("LLM configuration requires 'provider' field")
try:
# Prepare parameters for init_chat_model
init_params = {
"model": config_vars["model_name"],
"model_provider": config_vars["model_provider"],
"api_key": config_vars["api_key"],
"base_url": config_vars["base_url"],
"temperature": config_vars["temperature"],
"configurable_fields": "any",
"config_prefix": "foo",
}
# Disable DeepSeek thinking mode to avoid reasoning_content handling issues
# DeepSeek models enable thinking mode by default, which returns reasoning_content
# that must be passed back to the API in subsequent requests. To simplify
# message handling and avoid 400 errors, we disable it here.
if config_vars["model_provider"] == "deepseek":
init_params["extra_body"] = {"thinking": {"type": "disabled"}}
logger.debug("DeepSeek thinking mode disabled")
model = init_chat_model(**init_params)
logger.debug("Base model created successfully")
return model
except Exception as e:
logger.error("Failed to create base model: %s", e)
raise RuntimeError(f"Failed to create base model: {e}") from e
def create_title_model(
llm_config: Optional[dict[str, Any]] = None,
) -> Any:
"""
Create a fresh title generation model instance.
This creates a model instance suitable for generating conversation titles.
It uses the same configuration as the base model but with a higher
temperature for more creative output.
Args:
llm_config: Configuration dictionary from database
Returns:
Any: A new LLM model instance for title generation.
The actual type depends on the provider.
Raises:
ValueError: If required configuration fields are missing or invalid.
RuntimeError: If model creation fails.
"""
config_vars = _load_llm_config(llm_config)
logger.info(
"Creating title model: name=%s, provider=%s, base_url=%s, "
"temperature=1.0",
config_vars["model_name"],
config_vars["model_provider"],
config_vars["base_url"] if config_vars["base_url"] else "default",
)
# Validate required fields
if not config_vars["model_name"]:
raise ValueError("LLM configuration requires 'model' field")
if not config_vars["model_provider"]:
raise ValueError("LLM configuration requires 'provider' field")
try:
# Prepare parameters for init_chat_model
init_params = {
"model": config_vars["model_name"],
"model_provider": config_vars["model_provider"],
"api_key": config_vars["api_key"],
"base_url": config_vars["base_url"],
"temperature": "1.0", # Higher temperature for more creative titles
"configurable_fields": "any",
"config_prefix": "foo",
}
# Disable DeepSeek thinking mode to avoid reasoning_content handling issues
# DeepSeek models enable thinking mode by default, which returns reasoning_content
# that must be passed back to the API in subsequent requests. To simplify
# message handling and avoid 400 errors, we disable it here.
if config_vars["model_provider"] == "deepseek":
init_params["extra_body"] = {"thinking": {"type": "disabled"}}
logger.debug("DeepSeek thinking mode disabled for title model")
model = init_chat_model(**init_params)
logger.debug("Title model created successfully")
return model
except Exception as e:
logger.error("Failed to create title model: %s", e)
raise RuntimeError(f"Failed to create title model: {e}") from e
def create_model_with_tools(
model: Any,
tools: list[Any],
) -> Any:
"""
Bind tools to a model instance.
Args:
model: The base model instance.
tools: List of tools to bind to the model.
Returns:
Any: A model instance with tools bound (type varies by provider).
Raises:
RuntimeError: If tool binding fails.
"""
try:
model_with_tools = model.bind_tools(tools)
logger.debug("Model bound with %d tools successfully", len(tools))
return model_with_tools
except Exception as e:
logger.error("Failed to bind tools to model: %s", e)
raise RuntimeError(f"Failed to bind tools to model: {e}") from e
def create_base_model_with_tools(
tools: list[Any],
llm_config: Optional[dict[str, Any]] = None,
) -> Any:
"""
Create a fresh base model instance with tools bound.
This is a convenience function that combines creating the base model
and binding tools to it.
Args:
tools: List of tools to bind to the model.
llm_config: Configuration dictionary from database
Returns:
Any: A new model instance with tools bound (type varies by provider).
Raises:
ValueError: If required configuration fields are missing.
RuntimeError: If model creation or tool binding fails.
"""
base_model = create_base_model(llm_config)
return create_model_with_tools(base_model, tools)
@@ -0,0 +1,782 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
GNS3 Copilot Agent Service
Provides project-level Agent instances with SQLite checkpoint management.
Each project has its own AgentService with a dedicated checkpoint database
in the project directory.
"""
import asyncio
import json
import logging
import os
from datetime import datetime
from typing import Any
from typing import AsyncGenerator
from typing import Dict
from typing import List
from typing import Optional
from uuid import uuid4
import aiosqlite
from langchain_core.messages import HumanMessage
from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
from gns3server.agent.gns3_copilot.agent.gns3_copilot import agent_builder
from gns3server.agent.gns3_copilot.chat_sessions_repository import (
ChatSessionsRepository,
)
from gns3server.agent.gns3_copilot.gns3_client.context_helpers import (
set_current_jwt_token,
)
from gns3server.agent.gns3_copilot.gns3_client.context_helpers import (
set_current_llm_config,
)
from gns3server.agent.gns3_copilot.utils.error_handler import format_error_message
from gns3server.agent.gns3_copilot.utils.message_converters import (
convert_langchain_to_openai,
)
from gns3server.agent.gns3_copilot.utils.tool_call_stream import (
ToolCallStreamAccumulator,
)
log = logging.getLogger(__name__)
class AgentService:
"""
Project-level Agent Service with async checkpoint management.
Manages a LangGraph agent instance with SQLite-based state persistence
for a single GNS3 project.
"""
def __init__(self, project_path: str):
"""
Initialize AgentService for a project.
Args:
project_path: Path to the GNS3 project directory
"""
self.project_path = project_path
self._checkpointer: Optional[AsyncSqliteSaver] = None
self._checkpointer_conn: Optional[aiosqlite.Connection] = None
self._checkpointer_path: Optional[str] = None
self._graph = None
self._init_lock = asyncio.Lock()
self._initialized = False
def abort_session(self, session_id: str):
"""
Signal abort for a session.
Args:
session_id: Session identifier
"""
from gns3server.agent.gns3_copilot.agent.gns3_copilot import (
set_abort_flag,
)
set_abort_flag(session_id)
log.info("Abort signal set for session: %s", session_id)
def _get_checkpoint_dir(self) -> str:
"""Get or create the checkpoint directory for this project."""
checkpoint_dir = os.path.join(self.project_path, "gns3-copilot")
os.makedirs(checkpoint_dir, exist_ok=True)
return checkpoint_dir
async def _get_checkpointer(self) -> AsyncSqliteSaver:
"""
Get or create the SQLite checkpointer for this project.
Returns:
AsyncSqliteSaver instance
"""
async with self._init_lock:
if self._checkpointer is not None:
return self._checkpointer
checkpoint_dir = self._get_checkpoint_dir()
checkpointer_path = os.path.join(
checkpoint_dir, "copilot_checkpoints.db"
)
log.debug("Creating checkpointer at: %s", checkpointer_path)
# Close existing connection if switching projects
if self._checkpointer_conn:
try:
await self._checkpointer_conn.close()
log.debug("Closed previous checkpointer connection")
except Exception as e:
log.warning(
"Error closing old checkpointer connection: %s", e
)
# Create new connection
conn = await aiosqlite.connect(checkpointer_path)
# Enable WAL mode for better concurrent performance
await conn.execute("PRAGMA journal_mode=WAL;")
self._checkpointer_conn = (
conn # Save connection reference to prevent GC
)
self._checkpointer = AsyncSqliteSaver(conn)
# CRITICAL: Initialize database schema
await self._checkpointer.setup()
# Create chat_sessions table in the same database
await self._create_chat_sessions_table(conn)
self._checkpointer_path = checkpointer_path
self._initialized = True
log.info("Project checkpointer created at: %s", checkpointer_path)
return self._checkpointer
async def _create_chat_sessions_table(self, conn: aiosqlite.Connection):
"""
Create the chat_sessions table in the checkpoint database.
Args:
conn: aiosqlite connection
"""
await conn.execute("""
CREATE TABLE IF NOT EXISTS chat_sessions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
thread_id TEXT UNIQUE NOT NULL,
user_id TEXT NOT NULL,
project_id TEXT NOT NULL,
title TEXT DEFAULT 'New Conversation',
-- Statistics
message_count INTEGER DEFAULT 0,
llm_calls_count INTEGER DEFAULT 0,
input_tokens INTEGER DEFAULT 0,
output_tokens INTEGER DEFAULT 0,
total_tokens INTEGER DEFAULT 0,
-- Timestamps
last_message_at TIMESTAMP,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
-- Reserved fields (JSON strings)
metadata TEXT DEFAULT '{}',
stats TEXT DEFAULT '{}',
-- Pin feature
pinned BOOLEAN DEFAULT FALSE
)
""")
# Create indexes
await conn.execute(
"CREATE INDEX IF NOT EXISTS idx_thread_id ON "
"chat_sessions(thread_id)"
)
await conn.execute(
"CREATE INDEX IF NOT EXISTS idx_user_project ON "
"chat_sessions(user_id, project_id)"
)
# Check if pinned column exists, add it if not (migration for existing
# databases)
cursor = await conn.execute("PRAGMA table_info(chat_sessions)")
columns = await cursor.fetchall()
column_names = [col[1] for col in columns]
if "pinned" not in column_names:
log.debug("Adding pinned column to existing chat_sessions table")
await conn.execute(
"ALTER TABLE chat_sessions ADD COLUMN pinned BOOLEAN DEFAULT "
"FALSE"
)
await conn.commit()
# Create pinned index (after column is guaranteed to exist)
await conn.execute(
"CREATE INDEX IF NOT EXISTS idx_pinned_updated ON "
"chat_sessions(pinned DESC, updated_at DESC)"
)
await conn.commit()
log.debug("chat_sessions table created in checkpoint database")
async def _get_graph(self):
"""Get or compile the LangGraph agent."""
if self._graph is None:
checkpointer = await self._get_checkpointer()
self._graph = agent_builder.compile(checkpointer=checkpointer)
log.info(
"LangGraph agent compiled for project: %s", self.project_path
)
return self._graph
async def stream_chat(
self,
message: str,
session_id: str,
project_id: Optional[str] = None,
user_id: Optional[str] = None,
jwt_token: Optional[str] = None,
mode: str = "text",
llm_config: Optional[Dict[str, Any]] = None,
) -> AsyncGenerator[Dict[str, Any], None]:
"""
Stream chat responses from the agent.
Args:
message: User message
session_id: Session/thread ID for conversation continuity
project_id: GNS3 project ID (optional, for context)
user_id: User ID for metadata tracking
jwt_token: JWT token for API authentication (optional)
mode: Interaction mode (default: "text")
llm_config: LLM configuration dict (provider, model, api_key,
etc.)
Yields:
Dict containing SSE-compatible response chunks
"""
log.info(
"Stream chat started: project_id=%s, user_id=%s, session_id=%s, "
"mode=%s",
project_id,
user_id,
session_id,
mode,
)
# Ensure checkpointer is initialized
if not self._checkpointer_conn:
await self._get_checkpointer()
# Get or create chat session
repo = ChatSessionsRepository(self._checkpointer_conn)
session = await repo.get_session_by_thread(session_id)
is_new_session = session is None
if is_new_session:
# Create new session
copilot_mode = llm_config.get("copilot_mode", "teaching_assistant").lower()
session = await repo.create_session(
thread_id=session_id,
user_id=user_id or "",
project_id=project_id or "",
title="New Conversation",
copilot_mode=copilot_mode,
)
log.debug(
"Created new chat session: thread_id=%s, copilot_mode=%s",
session_id,
copilot_mode
)
# Set request-scoped context variables (memory-only, not persisted)
if jwt_token:
set_current_jwt_token(jwt_token)
log.debug("JWT token set in context")
if llm_config:
set_current_llm_config(llm_config)
log.debug(
"LLM config set in context: provider=%s, model=%s",
llm_config.get("provider"),
llm_config.get("model"),
)
# Build config - only thread-safe identifiers
# Determine recursion_limit based on copilot_mode
copilot_mode = llm_config.get("copilot_mode", "teaching_assistant").lower()
if copilot_mode == "troubleshooting_injection":
recursion_limit = 100 # Need more recursion depth for fault injection workflow
log.debug("Using extended recursion_limit for troubleshooting_injection mode: 100")
else:
recursion_limit = 25 # Default recursion limit
config = {
"configurable": {
"thread_id": session_id,
"project_id": project_id,
},
"metadata": {
"user_id": user_id,
"copilot_mode": copilot_mode, # Store mode in checkpoint metadata
},
"recursion_limit": recursion_limit,
}
# Build inputs
# Determine remaining_steps based on copilot_mode
# Troubleshooting injection mode needs more steps for multi-tool workflow
if copilot_mode == "troubleshooting_injection":
remaining_steps = 50 # Need more steps for fault injection workflow
log.debug("Using extended remaining_steps for troubleshooting_injection mode: 50")
else:
remaining_steps = 20 # Default for other modes
inputs = {
"messages": [
HumanMessage(
content=message,
id=str(uuid4()),
metadata={"created_at": datetime.utcnow().isoformat()},
)
],
"llm_calls": 0,
"remaining_steps": remaining_steps,
"session_id": session_id,
"abort": False,
}
# Clear abort flag for this session at the start of stream
from gns3server.agent.gns3_copilot.agent.gns3_copilot import (
clear_abort_flag,
)
clear_abort_flag(session_id)
log.debug("Abort flag cleared for session: %s", session_id)
# Get the compiled graph
graph = await self._get_graph()
log.debug("LangGraph graph obtained, starting stream")
# Track statistics for session update
message_count = 1 # User message
llm_calls_count = 0
input_tokens = 0
output_tokens = 0
last_message_at = datetime.utcnow().isoformat()
# Track if we've counted the AI response for this turn
ai_response_counted = False
tool_messages_counted = 0
# Initialize tool call stream accumulator for handling progressive
# tool call arguments
tool_call_accumulator = ToolCallStreamAccumulator()
# Track if stream was aborted
stream_aborted = False
# Stream events
try:
async for event in graph.astream_events(
inputs, config=config, version="v2"
):
event_type = event.get("event", "")
data = event.get("data", {})
# Track LLM calls and tokens
if event_type == "on_chat_model_start":
# Filter out title_generator_node from statistics
langgraph_node = event.get("metadata", {}).get(
"langgraph_node", ""
)
if langgraph_node != "title_generator_node":
llm_calls_count += 1
log.debug(
"LLM call started, count=%d", llm_calls_count
)
else:
log.debug(
"Skipping LLM call count for internal node: "
"title_generator_node"
)
elif event_type == "on_chat_model_end":
# Filter out title_generator_node from token counting
langgraph_node = event.get("metadata", {}).get(
"langgraph_node", ""
)
if langgraph_node == "title_generator_node":
log.debug(
"Skipping token counting for internal node: "
"title_generator_node"
)
else:
# Extract token usage from response metadata
# Try multiple possible locations where token usage
# might be stored
token_info_found = False
# Method 1: response.usage_metadata
response = data.get("response", {})
if hasattr(response, "usage_metadata"):
usage = response.usage_metadata
if usage:
input_tokens += usage.get("input_tokens", 0)
output_tokens += usage.get("output_tokens", 0)
token_info_found = True
# Method 2: output.usage_metadata
if not token_info_found:
output_msg = data.get("output", {})
if hasattr(output_msg, "usage_metadata"):
usage = output_msg.usage_metadata
if usage:
input_tokens += usage.get(
"input_tokens", 0
)
output_tokens += usage.get(
"output_tokens", 0
)
token_info_found = True
# Method 3: Check data directly for token usage fields
if not token_info_found:
if "input_tokens" in data:
input_tokens += data.get("input_tokens", 0)
if "output_tokens" in data:
output_tokens += data.get("output_tokens", 0)
if (
"input_tokens" in data
or "output_tokens" in data
):
token_info_found = True
# Count AI response as one message (only once per turn)
if not ai_response_counted:
message_count += 1
ai_response_counted = True
# Track tool messages
elif event_type == "on_tool_end":
message_count += 1 # Tool result message
tool_messages_counted += 1
log.debug(
"Tool message counted, message_count=%d", message_count
)
# Convert event to chunk for SSE streaming
# Use accumulator for on_chat_model_stream events to handle
# progressive tool calls
# Filter out internal nodes (title_generator_node) from
# streaming to frontend
langgraph_node = event.get("metadata", {}).get(
"langgraph_node", ""
)
if langgraph_node == "title_generator_node":
# Skip all events from the title_generator_node (internal
# use only)
log.debug(
"Skipping event from internal node: "
"title_generator_node"
)
continue
if event_type == "on_chat_model_stream":
chunks = tool_call_accumulator.process_event(event)
for chunk in chunks:
# Add session_id to each chunk
chunk["session_id"] = session_id
log.debug(
"Yielding accumulated chunk: type=%s",
chunk.get("type"),
)
yield chunk
else:
# Use stateless converter for other events
chunk = self._convert_event_to_chunk(event, session_id)
if chunk:
log.debug("Yielding chunk: type=%s", chunk.get("type"))
yield chunk
# Check if stream was aborted and yield tool_end events for aborted tools
from gns3server.agent.gns3_copilot.agent.gns3_copilot import (
check_abort_flag,
)
if check_abort_flag(session_id):
log.info("Stream aborted, yielding tool_end events: session_id=%s", session_id)
# Get final state to find aborted tool messages
final_state = await graph.aget_state(config)
if final_state and "messages" in final_state.values:
for msg in final_state.values["messages"]:
# Check if this is an aborted tool message
if hasattr(msg, "metadata") and msg.metadata.get("aborted"):
yield {
"type": "tool_end",
"tool_name": getattr(msg, "name", "unknown"),
"tool_output": msg.content if hasattr(msg, "content") else "",
"session_id": session_id,
}
# Update session statistics after successful stream
await repo.update_session(
thread_id=session_id,
message_count=message_count,
llm_calls_count=llm_calls_count,
input_tokens=input_tokens,
output_tokens=output_tokens,
total_tokens=input_tokens + output_tokens,
last_message_at=last_message_at,
)
log.info(
"Session statistics updated: thread_id=%s, messages=%d, "
"llm_calls=%d, tokens=%d+%d=%d",
session_id,
message_count,
llm_calls_count,
input_tokens,
output_tokens,
input_tokens + output_tokens,
)
# Sync auto-generated title from checkpoint state
final_state = await graph.aget_state(config)
if final_state and "conversation_title" in final_state.values:
generated_title = final_state.values["conversation_title"]
current_session = await repo.get_session_by_thread(session_id)
if (
current_session
and current_session.title != generated_title
):
await repo.update_session(
thread_id=session_id, title=generated_title
)
log.info(
"Auto-generated title synced: thread_id=%s, title=%s",
session_id,
generated_title,
)
except Exception as e:
log.error("Error in stream_chat: %s", e, exc_info=True)
yield {
"type": "error",
"error": format_error_message(e),
"session_id": session_id,
}
def _convert_event_to_chunk(
self, event: Dict[str, Any], session_id: str
) -> Optional[Dict[str, Any]]:
"""
Convert LangGraph event to API response chunk.
Args:
event: LangGraph event from astream_events
session_id: Session ID for the response
Returns:
Dict for SSE response or None if event should be filtered
Note:
on_chat_model_stream events are handled by
ToolCallStreamAccumulator before calling this method, so they are
not processed here.
"""
event_type = event.get("event", "")
data = event.get("data", {})
if event_type == "on_tool_start":
# Tool execution started
# Extract tool_call_id from event metadata to associate with
# tool_call event
tool_call_id = event.get("metadata", {}).get("tool_call_id", "")
return {
"type": "tool_start",
"tool_name": event.get("name", ""),
"tool_call_id": tool_call_id,
"session_id": session_id,
}
elif event_type == "on_tool_end":
# Tool execution completed
# Extract tool output and convert to JSON string
output = data.get("output", "")
# Convert output to JSON string if it's not already a string
# This ensures dict/list outputs are properly serialized for
# frontend parsing
if not isinstance(output, str):
output = json.dumps(output, ensure_ascii=False, indent=2)
return {
"type": "tool_end",
"tool_name": event.get("name", ""),
"tool_output": output,
"session_id": session_id,
}
return None
async def get_history(
self, session_id: str, limit: int = 100
) -> Dict[str, Any]:
"""
Get conversation history for a session.
Args:
session_id: Session/thread ID
limit: Maximum number of messages to retrieve
Returns:
Dict containing thread_id, title, and messages
"""
config = {"configurable": {"thread_id": session_id}}
try:
graph = await self._get_graph()
state = await graph.aget_state(config)
if state and "messages" in state.values:
messages = []
for msg in state.values["messages"][-limit:]:
messages.append(self._convert_message_to_dict(msg))
title = state.values.get(
"conversation_title", "New Conversation"
)
return {
"thread_id": session_id,
"title": title,
"messages": messages,
}
except Exception as e:
log.error("Error getting history: %s", e, exc_info=True)
return {
"thread_id": session_id,
"title": "New Conversation",
"messages": [],
}
def _convert_message_to_dict(self, msg) -> Dict[str, Any]:
"""Convert a LangChain message to OpenAI-compatible dict format."""
return convert_langchain_to_openai(msg)
async def list_sessions(
self, user_id: Optional[str] = None, copilot_mode: Optional[str] = None, limit: int = 100
) -> List[Dict[str, Any]]:
"""
List chat sessions for this project.
Args:
user_id: Filter by user ID (optional)
copilot_mode: Filter by copilot mode (optional)
limit: Maximum number of sessions to return
Returns:
List of session dictionaries
"""
if not self._checkpointer_conn:
await self._get_checkpointer()
repo = ChatSessionsRepository(self._checkpointer_conn)
sessions = await repo.list_sessions(user_id=user_id, copilot_mode=copilot_mode, limit=limit)
return [s.to_dict() for s in sessions]
async def delete_session(self, session_id: str) -> bool:
"""
Delete a chat session and its checkpoints.
Args:
session_id: Thread ID to delete
Returns:
True if deleted, False if not found
"""
if not self._checkpointer_conn:
await self._get_checkpointer()
repo = ChatSessionsRepository(self._checkpointer_conn)
return await repo.delete_session(session_id)
async def rename_session(
self, session_id: str, new_title: str
) -> Optional[Dict[str, Any]]:
"""
Rename a chat session.
Args:
session_id: Thread ID
new_title: New title
Returns:
Updated session dictionary or None
"""
if not self._checkpointer_conn:
await self._get_checkpointer()
repo = ChatSessionsRepository(self._checkpointer_conn)
session = await repo.update_session(
thread_id=session_id, title=new_title
)
return session.to_dict() if session else None
async def pin_session(
self, session_id: str, pinned: bool = True
) -> Optional[Dict[str, Any]]:
"""
Pin or unpin a chat session.
Args:
session_id: Thread ID
pinned: True to pin, False to unpin
Returns:
Updated session dictionary or None
"""
if not self._checkpointer_conn:
await self._get_checkpointer()
repo = ChatSessionsRepository(self._checkpointer_conn)
session = await repo.pin_session(thread_id=session_id, pinned=pinned)
return session.to_dict() if session else None
async def close(self):
"""
Close the checkpointer connection and cleanup resources.
"""
async with self._init_lock:
if self._checkpointer_conn:
try:
# Add timeout to prevent blocking on database close
await asyncio.wait_for(
self._checkpointer_conn.close(),
timeout=5.0
)
log.debug(
"Checkpointer connection closed for: %s",
self.project_path,
)
except asyncio.TimeoutError:
log.warning(
"Checkpointer connection close timeout for: %s (forcing cleanup)",
self.project_path,
)
except Exception as e:
log.warning("Error closing checkpointer connection: %s", e)
finally:
self._checkpointer_conn = None
self._checkpointer = None
self._graph = None
self._initialized = False
@@ -0,0 +1,447 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
Chat Sessions Repository for managing chat session data.
Provides CRUD operations for the chat_sessions table in the project's
checkpoint database.
"""
import json
import logging
from datetime import datetime
from typing import Any
from typing import Dict
from typing import List
from typing import Optional
import aiosqlite
log = logging.getLogger(__name__)
class ChatSession:
"""Chat session model."""
def __init__(
self,
id: Optional[int] = None,
thread_id: str = "",
user_id: str = "",
project_id: str = "",
title: str = "New Conversation",
message_count: int = 0,
llm_calls_count: int = 0,
input_tokens: int = 0,
output_tokens: int = 0,
total_tokens: int = 0,
last_message_at: Optional[str] = None,
created_at: Optional[str] = None,
updated_at: Optional[str] = None,
metadata: str = "{}",
stats: str = "{}",
pinned: bool = False,
):
self.id = id
self.thread_id = thread_id
self.user_id = user_id
self.project_id = project_id
self.title = title
self.message_count = message_count
self.llm_calls_count = llm_calls_count
self.input_tokens = input_tokens
self.output_tokens = output_tokens
self.total_tokens = total_tokens
self.last_message_at = last_message_at
self.created_at = created_at
self.updated_at = updated_at
self.metadata = metadata
self.stats = stats
self.pinned = pinned
def to_dict(self) -> Dict[str, Any]:
"""Convert to dictionary."""
return {
"id": self.id,
"thread_id": self.thread_id,
"user_id": self.user_id,
"project_id": self.project_id,
"title": self.title,
"message_count": self.message_count,
"llm_calls_count": self.llm_calls_count,
"input_tokens": self.input_tokens,
"output_tokens": self.output_tokens,
"total_tokens": self.total_tokens,
"last_message_at": self.last_message_at,
"created_at": self.created_at,
"updated_at": self.updated_at,
"metadata": json.loads(self.metadata) if self.metadata else {},
"stats": json.loads(self.stats) if self.stats else {},
"pinned": self.pinned,
}
class ChatSessionsRepository:
"""
Repository for managing chat sessions in the checkpoint database.
"""
def __init__(self, conn: aiosqlite.Connection):
"""
Initialize repository with a database connection.
Args:
conn: aiosqlite connection to the checkpoint database
"""
self.conn = conn
async def create_session(
self,
thread_id: str,
user_id: str,
project_id: str,
title: str = "New Conversation",
copilot_mode: Optional[str] = None,
) -> ChatSession:
"""
Create a new chat session.
Args:
thread_id: Unique thread identifier
user_id: User ID
project_id: Project ID
title: Session title
copilot_mode: Copilot mode (optional)
Returns:
Created ChatSession
"""
now = datetime.utcnow().isoformat()
# Build metadata JSON
metadata = {"copilot_mode": copilot_mode} if copilot_mode else {}
metadata_json = json.dumps(metadata)
cursor = await self.conn.execute(
"""
INSERT INTO chat_sessions (
thread_id, user_id, project_id, title,
metadata, created_at, updated_at
) VALUES (?, ?, ?, ?, ?, ?, ?)
""",
(thread_id, user_id, project_id, title, metadata_json, now, now),
)
await self.conn.commit()
session_id = cursor.lastrowid
log.info(
"Created chat session: id=%s, thread_id=%s, copilot_mode=%s",
session_id, thread_id, copilot_mode
)
return await self.get_session_by_id(session_id)
async def get_session_by_id(
self, session_id: int
) -> Optional[ChatSession]:
"""
Get a session by its database ID.
Args:
session_id: Database row ID
Returns:
ChatSession or None
"""
cursor = await self.conn.execute(
"SELECT * FROM chat_sessions WHERE id = ?", (session_id,)
)
row = await cursor.fetchone()
if row:
return self._row_to_session(row)
return None
async def get_session_by_thread(
self, thread_id: str
) -> Optional[ChatSession]:
"""
Get a session by thread_id.
Args:
thread_id: Thread identifier
Returns:
ChatSession or None
"""
cursor = await self.conn.execute(
"SELECT * FROM chat_sessions WHERE thread_id = ?", (thread_id,)
)
row = await cursor.fetchone()
if row:
return self._row_to_session(row)
return None
async def list_sessions(
self,
user_id: Optional[str] = None,
project_id: Optional[str] = None,
copilot_mode: Optional[str] = None,
limit: int = 100,
) -> List[ChatSession]:
"""
List sessions with optional filters.
Args:
user_id: Filter by user ID
project_id: Filter by project ID
copilot_mode: Filter by copilot mode (metadata field)
limit: Maximum number of sessions to return
Returns:
List of ChatSession
"""
query = "SELECT * FROM chat_sessions"
params = []
conditions = []
if user_id:
conditions.append("user_id = ?")
params.append(user_id)
if project_id:
conditions.append("project_id = ?")
params.append(project_id)
if copilot_mode:
# Filter by JSON metadata field
conditions.append("json_extract(metadata, '$.copilot_mode') = ?")
params.append(copilot_mode)
if conditions:
query += " WHERE " + " AND ".join(conditions)
# Sort by pinned status first, then by updated_at
query += " ORDER BY pinned DESC, updated_at DESC LIMIT ?"
params.append(limit)
cursor = await self.conn.execute(query, params)
rows = await cursor.fetchall()
return [self._row_to_session(row) for row in rows]
async def update_session(
self,
thread_id: str,
title: Optional[str] = None,
message_count: Optional[int] = None,
llm_calls_count: Optional[int] = None,
input_tokens: Optional[int] = None,
output_tokens: Optional[int] = None,
total_tokens: Optional[int] = None,
last_message_at: Optional[str] = None,
) -> Optional[ChatSession]:
"""
Update a session.
Args:
thread_id: Thread identifier
title: New title
message_count: Increment message count
llm_calls_count: Increment LLM call count
input_tokens: Add to input tokens
output_tokens: Add to output tokens
total_tokens: Add to total tokens
last_message_at: Last message timestamp
Returns:
Updated ChatSession or None
"""
updates = []
params = []
now = datetime.utcnow().isoformat()
if title is not None:
updates.append("title = ?")
params.append(title)
if message_count is not None:
updates.append("message_count = message_count + ?")
params.append(message_count)
if llm_calls_count is not None:
updates.append("llm_calls_count = llm_calls_count + ?")
params.append(llm_calls_count)
if input_tokens is not None:
updates.append("input_tokens = input_tokens + ?")
params.append(input_tokens)
if output_tokens is not None:
updates.append("output_tokens = output_tokens + ?")
params.append(output_tokens)
if total_tokens is not None:
updates.append("total_tokens = total_tokens + ?")
params.append(total_tokens)
if last_message_at is not None:
updates.append("last_message_at = ?")
params.append(last_message_at)
if not updates:
return await self.get_session_by_thread(thread_id)
updates.append("updated_at = ?")
params.append(now)
params.append(thread_id)
query = (
f"UPDATE chat_sessions SET {', '.join(updates)} WHERE thread_id "
f"= ?"
)
await self.conn.execute(query, params)
await self.conn.commit()
log.debug("Updated chat session: thread_id=%s", thread_id)
return await self.get_session_by_thread(thread_id)
async def delete_session(self, thread_id: str) -> bool:
"""
Delete a session by thread_id.
Args:
thread_id: Thread identifier
Returns:
True if deleted, False if not found
"""
# First, delete the checkpoint data
await self.conn.execute(
"DELETE FROM checkpoints WHERE thread_id = ?", (thread_id,)
)
# Then delete the session
cursor = await self.conn.execute(
"DELETE FROM chat_sessions WHERE thread_id = ?", (thread_id,)
)
await self.conn.commit()
deleted = cursor.rowcount > 0
if deleted:
log.info(
"Deleted chat session and checkpoints: thread_id=%s", thread_id
)
return deleted
async def delete_all_sessions(self, project_id: str) -> int:
"""
Delete all sessions for a project.
Args:
project_id: Project ID
Returns:
Number of sessions deleted
"""
# Get all thread_ids for this project
cursor = await self.conn.execute(
"SELECT thread_id FROM chat_sessions WHERE project_id = ?",
(project_id,),
)
rows = await cursor.fetchall()
thread_ids = [row[0] for row in rows]
# Delete checkpoints and sessions
for thread_id in thread_ids:
await self.conn.execute(
"DELETE FROM checkpoints WHERE thread_id = ?", (thread_id,)
)
cursor = await self.conn.execute(
"DELETE FROM chat_sessions WHERE project_id = ?", (project_id,)
)
await self.conn.commit()
deleted_count = cursor.rowcount
if deleted_count > 0:
log.info(
"Deleted %d sessions for project: %s",
deleted_count,
project_id,
)
return deleted_count
async def pin_session(
self, thread_id: str, pinned: bool = True
) -> Optional[ChatSession]:
"""
Pin or unpin a session.
Args:
thread_id: Thread identifier
pinned: True to pin, False to unpin
Returns:
Updated ChatSession or None
"""
now = datetime.utcnow().isoformat()
await self.conn.execute(
"UPDATE chat_sessions SET pinned = ?, updated_at = ? WHERE "
"thread_id = ?",
(1 if pinned else 0, now, thread_id),
)
await self.conn.commit()
log.debug(
"Session pin status updated: thread_id=%s, pinned=%s",
thread_id,
pinned,
)
return await self.get_session_by_thread(thread_id)
def _row_to_session(self, row) -> ChatSession:
"""Convert database row to ChatSession object."""
return ChatSession(
id=row[0],
thread_id=row[1],
user_id=row[2],
project_id=row[3],
title=row[4],
message_count=row[5],
llm_calls_count=row[6],
input_tokens=row[7],
output_tokens=row[8],
total_tokens=row[9],
last_message_at=row[10],
created_at=row[11],
updated_at=row[12],
metadata=row[13],
stats=row[14],
pinned=bool(row[15]) if len(row) > 15 else False,
)
@@ -0,0 +1,22 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# Copyright (C) 2025 Yue Guobin
#
"""
Configuration package for GNS3 Copilot.
"""
from .skills_config import (
SKILLS_CONFIG,
get_skills_config,
update_skills_config,
)
__all__ = [
"SKILLS_CONFIG",
"get_skills_config",
"update_skills_config",
]
@@ -0,0 +1,86 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin
# Author: Yue Guobin
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
Skills Configuration
This module contains configuration for the external skills repository.
"""
from gns3server.config import Config
# Default skills repository configuration
SKILLS_CONFIG = {
# Git repository URL for skills
"repo_url": "https://github.com/gns3/gns3-skills.git",
# Git branch to use
"branch": "main",
# Automatically pull updates on reload
"auto_update": True,
# Enable external skills loading
# If False, use the built-in hardcoded skills
"enabled": True,
}
def get_skills_config() -> dict:
"""
Get the skills configuration.
Priority:
1. GNS3 server config (gns3_server.conf → [Server] → skills_*)
2. Hardcoded defaults
Returns:
Dictionary containing skills configuration
"""
config = SKILLS_CONFIG.copy()
try:
server = Config.instance().settings.Server
if server.skills_repo_url:
config["repo_url"] = server.skills_repo_url
if server.skills_repo_branch:
config["branch"] = server.skills_repo_branch
config["auto_update"] = server.skills_auto_update
except Exception:
pass
return config
def update_skills_config(**kwargs):
"""
Update skills configuration.
Args:
**kwargs: Configuration key-value pairs to update
"""
for key, value in kwargs.items():
if key in SKILLS_CONFIG:
SKILLS_CONFIG[key] = value
else:
raise ValueError(f"Unknown configuration key: {key}")
@@ -0,0 +1,98 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
GNS3 Client Package
This package provides a Python interface for interacting with GNS3 servers.
Adapted from the upstream gns3fy project with modifications for compatibility
with langchain and reduced dependency conflicts.
Main classes:
- Gns3Connector: Connector for GNS3 server API interaction
- Project: GNS3 Project management
- Node: GNS3 Node management
- Link: GNS3 Link management
- GNS3TopologyTool: GNS3 topology reading tool
- GNS3ProjectInfoTool: GNS3 project info tool
Main functions:
- get_gns3_connector: Factory function to create Gns3Connector
- get_gns3_connector_with_llm_config: Create connector AND retrieve LLM config
- get_gns3_server_host: Get GNS3 server hostname from Controller or Config
- get_llm_config: Get LLM model configuration for a user
Upstream gns3fy: https://github.com/davidban77/gns3fy
"""
from .connector_factory import get_gns3_connector
from .connector_factory import get_gns3_connector_with_llm_config
from .connector_factory import get_gns3_server_host
from .connector_factory import get_llm_config
from .context_helpers import get_current_jwt_token
from .context_helpers import get_current_llm_config
from .context_helpers import set_current_jwt_token
from .context_helpers import set_current_llm_config
from .custom_gns3fy import CONSOLE_TYPES
from .custom_gns3fy import LINK_TYPES
from .custom_gns3fy import NODE_TYPES
from .custom_gns3fy import Gns3Connector
from .custom_gns3fy import Link
from .custom_gns3fy import Node
from .custom_gns3fy import Project
from .gns3_project_info import GNS3ProjectInfoTool
from .gns3_topology_reader import GNS3TopologyTool
# Dynamic version management
try:
from importlib.metadata import version
__version__ = version("gns3-copilot")
except Exception:
__version__ = "unknown"
__author__ = "Yue Guobin (岳国宾)"
__description__ = "AI-powered network automation assistant for GNS3"
__url__ = "https://github.com/yueguobin/gns3-copilot"
__all__ = [
"Gns3Connector",
"Project",
"Node",
"Link",
"NODE_TYPES",
"CONSOLE_TYPES",
"LINK_TYPES",
"GNS3TopologyTool",
"GNS3ProjectInfoTool",
"get_gns3_connector",
"get_gns3_connector_with_llm_config",
"get_gns3_server_host",
"get_llm_config",
"set_current_jwt_token",
"get_current_jwt_token",
"set_current_llm_config",
"get_current_llm_config",
]
@@ -0,0 +1,476 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
GNS3 Connector Factory Module
This module provides factory functions for creating Gns3Connector instances
with JWT token authentication and context-aware configuration management.
⚠️ WARNING: This module is shared with the MCP (Model Context Protocol) service.
The get_gns3_connector() function is used by both gns3-copilot and MCP.
Modifications must be tested with BOTH.
Features:
- Context variable based request-scoped data management (JWT tokens, LLM
config)
- Auto-detection of GNS3 server URL from Controller/Config
- Fallback URL strategy for flexible deployment
- LLM configuration retrieval for users
"""
# Standard library imports
import asyncio
import concurrent.futures
import logging
from typing import Optional
from uuid import UUID
from gns3server.agent.gns3_copilot.gns3_client.context_helpers import (
get_current_jwt_token,
)
# Local imports
from gns3server.agent.gns3_copilot.gns3_client.custom_gns3fy import (
Gns3Connector,
)
logger = logging.getLogger(__name__)
# Fallback default URL
DEFAULT_GNS3_URL = "http://127.0.0.1:3080"
def _get_url_from_controller() -> Optional[str]:
"""Try to get GNS3 server URL from running Controller instance.
Returns:
URL string if successful, None otherwise
"""
try:
from gns3server.controller import Controller
controller = Controller.instance()
local_compute = controller.get_compute("local")
url = (
f"{local_compute.protocol}://{local_compute.host}:"
f"{local_compute.port}"
)
logger.debug(
"Got GNS3 URL from Controller: %s (protocol=%s, host=%s, port=%s)",
url,
local_compute.protocol,
local_compute.host,
local_compute.port,
)
return url
except ImportError as e:
logger.debug("Cannot import Controller: %s", str(e))
return None
except AttributeError as e:
logger.debug("Controller instance not available: %s", str(e))
return None
except KeyError as e:
logger.debug("Local compute not found in Controller: %s", str(e))
return None
except Exception as e:
logger.warning(
"Unexpected error getting URL from Controller: %s", str(e)
)
return None
def _get_url_from_config() -> Optional[str]:
"""Try to get GNS3 server URL from Config settings.
Returns:
URL string if successful, None otherwise
"""
try:
from gns3server.config import Config
server_config = Config.instance().settings.Server
url = (
f"{server_config.protocol.value}://{server_config.host}:"
f"{server_config.port}"
)
logger.debug(
"Got GNS3 URL from Config: %s (protocol=%s, host=%s, port=%s)",
url,
server_config.protocol.value,
server_config.host,
server_config.port,
)
return url
except ImportError as e:
logger.debug("Cannot import Config: %s", str(e))
return None
except AttributeError as e:
logger.debug("Config settings not available: %s", str(e))
return None
except Exception as e:
logger.warning("Unexpected error getting URL from Config: %s", str(e))
return None
def get_gns3_connector(
jwt_token: Optional[str] = None, url: Optional[str] = None
) -> Optional[Gns3Connector]:
"""Create and return a Gns3Connector instance with JWT authentication.
URL Resolution Strategy (in order):
1. Explicitly provided `url` parameter
2. Runtime configuration from Controller.instance().compute("local")
3. Static configuration from Config.instance().settings.Server
4. Fallback to DEFAULT_GNS3_URL (http://127.0.0.1:3080)
Args:
jwt_token: JWT token for authentication (optional, will be retrieved
from context if not provided)
url: GNS3 server URL (optional, auto-detected if not provided)
Returns:
Gns3Connector instance if parameters are valid, None otherwise
Example:
# Auto-detect URL from Controller or Config
connector = get_gns3_connector(jwt_token="your_jwt_token")
# Or specify custom URL
connector = get_gns3_connector(
jwt_token="your_jwt_token",
url="http://custom-server:3080"
)
if connector:
projects = connector.projects
else:
logger.error("Failed to create GNS3 connector")
"""
try:
# Validate JWT token
# If not provided, try to get from current request context
if not jwt_token:
jwt_token = get_current_jwt_token()
if not jwt_token:
logger.error("JWT token is required")
return None
# Resolve URL with fallback strategy
if url is None:
logger.debug("No URL provided, attempting auto-detection...")
# Strategy 1: Try to get from running Controller
url = _get_url_from_controller()
if url:
logger.info("Using GNS3 server URL from Controller: %s", url)
else:
# Strategy 2: Fall back to Config
logger.debug("Controller not available, trying Config...")
url = _get_url_from_config()
if url:
logger.info("Using GNS3 server URL from Config: %s", url)
else:
# Strategy 3: Use default fallback
logger.debug("Config not available, using default URL")
url = DEFAULT_GNS3_URL
logger.warning(
"Using fallback default URL: %s. This may not be "
"correct if your GNS3 server is configured "
"differently. Consider providing the URL explicitly.",
url,
)
else:
logger.info("Using explicitly provided GNS3 server URL: %s", url)
# Validate URL
if not url:
logger.error("Failed to resolve GNS3 server URL")
return None
# Create connector
logger.debug("Creating Gns3Connector with URL=%s", url)
connector = Gns3Connector(
url=url,
jwt_token=jwt_token,
api_version=3,
)
logger.info("Successfully created Gns3Connector for URL: %s", url)
return connector
except Exception as e:
logger.error(
"Failed to create Gns3Connector: %s", str(e), exc_info=True
)
return None
async def get_gns3_connector_with_llm_config(
user_id, jwt_token: str, url: Optional[str] = None, app=None
) -> Optional[dict]:
"""
Create Gns3Connector and retrieve LLM model configuration for the user.
This is a convenience function that combines:
1. get_gns3_connector() - Create GNS3 API connector
2. get_llm_config() - Retrieve user's default LLM config with API key
Args:
user_id: User UUID (can be string or UUID object)
jwt_token: JWT token for authentication
url: GNS3 server URL (optional, auto-detected if not provided)
app: FastAPI application instance (optional, for direct database
access)
Returns:
Dictionary with keys:
- connector: Gns3Connector instance
- llm_config: Dict with provider, api_key, model, etc.
Or None if failed
Example:
result = await get_gns3_connector_with_llm_config(user_id, jwt_token)
if result:
connector = result["connector"]
llm_config = result["llm_config"]
# Use connector for GNS3 operations
projects = connector.projects
# Use LLM config for AI operations
provider = llm_config["provider"]
api_key = llm_config["api_key"]
model = llm_config["model"]
else:
logger.error("Failed to initialize GNS3 connector or LLM config")
"""
try:
# Convert user_id to UUID if it's a string
if isinstance(user_id, str):
user_id = UUID(user_id)
# Step 1: Create GNS3 connector
connector = get_gns3_connector(jwt_token=jwt_token, url=url)
if not connector:
logger.error("Failed to create GNS3 connector")
return None
# Step 2: Detect URL if not provided
if url is None:
url = _detect_url_for_api()
# Step 3: Get LLM config
llm_config = get_llm_config(
user_id=user_id, jwt_token=jwt_token, app=app
)
if not llm_config:
logger.warning(f"No LLM config found for user {user_id}")
# Still return result with connector only
return {"connector": connector, "llm_config": None}
logger.info(
f"Successfully initialized GNS3 connector and LLM config for "
f"user {user_id}: connector_url={connector.url}, "
f"llm_provider={llm_config.get('provider')}, "
f"llm_model={llm_config.get('model')}"
)
return {"connector": connector, "llm_config": llm_config}
except Exception as e:
logger.error(
f"Failed to get GNS3 connector with LLM config: {e}", exc_info=True
)
return None
def _detect_url_for_api() -> Optional[str]:
"""
Detect GNS3 server URL for API calls.
Uses the same priority order as get_gns3_connector:
1. Controller.instance().compute("local")
2. Config.instance().settings.Server
3. Fallback to DEFAULT_GNS3_URL
Returns:
URL string, or None if detection failed
"""
logger.debug("Detecting GNS3 server URL for API calls")
# Try Controller first
url = _get_url_from_controller()
if url:
logger.debug("Using URL from Controller for API call: %s", url)
return url
# Try Config
url = _get_url_from_config()
if url:
logger.debug("Using URL from Config for API call: %s", url)
return url
# Fallback
logger.debug("Using fallback URL for API call: %s", DEFAULT_GNS3_URL)
return DEFAULT_GNS3_URL
def get_gns3_server_host() -> str:
"""
Get GNS3 server hostname from Controller or Config.
This is a convenience function for extracting the hostname only, useful
for Nornir tools that need the GNS3 server address.
Uses the same priority order as get_gns3_connector:
1. Controller.instance().compute("local")
2. Config.instance().settings.Server
3. Fallback to DEFAULT_GNS3_URL host
Returns:
Hostname/IP address string
Example:
from gns3server.agent.gns3_copilot.gns3_client import (
get_gns3_server_host,
)
host = get_gns3_server_host()
print(f"GNS3 server host: {host}")
"""
url = _detect_url_for_api()
# Extract host from URL
# URL format: protocol://host:port
try:
# Remove protocol prefix
host_part = url.split("://")[1]
# Extract host (before the port)
host = host_part.split(":")[0]
logger.debug("Extracted GNS3 server host: %s from URL: %s", host, url)
return host
except Exception as e:
logger.warning(
"Failed to extract host from URL %s: %s, using fallback", url, e
)
return DEFAULT_GNS3_URL.split("://")[1].split(":")[0]
def get_llm_config(user_id, jwt_token: str, app=None) -> Optional[dict]:
"""
Get LLM model configuration for a user.
This function retrieves the user's LLM configuration from the database.
When `app` is provided, it directly accesses the database (bypassing API
restrictions). Otherwise, it falls back to API call (which may mask
group config API keys).
Args:
user_id: User UUID (can be string or UUID object)
jwt_token: JWT token for authentication
app: FastAPI application instance (optional, for direct database
access)
Returns:
Dictionary with LLM configuration keys (provider, model, api_key,
etc.), or None if not found.
Example:
from gns3server.agent.gns3_copilot.gns3_client import (
get_llm_config,
)
config = get_llm_config(user_id, jwt_token)
if config:
provider = config['provider']
model = config['model']
api_key = config['api_key']
"""
try:
# Convert user_id to UUID if it's a string
if isinstance(user_id, str):
user_id = UUID(user_id)
# If app is provided, use direct database access (preferred)
if app is not None:
from gns3server.agent.gns3_copilot.utils.llm_config_helper import (
get_user_llm_config_with_app,
)
# Run the async function in sync context
try:
loop = asyncio.get_running_loop()
# We're in an async context with a running loop
# This shouldn't happen since this is a sync function
with concurrent.futures.ThreadPoolExecutor() as executor:
future = executor.submit(
asyncio.run, get_user_llm_config_with_app(user_id, app)
)
return future.result(timeout=10)
except RuntimeError:
# No running event loop - we're in a sync context
pass
# Try to get an existing event loop, or create a new one
try:
loop = asyncio.get_event_loop()
if loop.is_running():
# Loop is running but not the running loop (edge case)
with concurrent.futures.ThreadPoolExecutor() as executor:
future = executor.submit(
asyncio.run,
get_user_llm_config_with_app(user_id, app),
)
return future.result(timeout=10)
else:
# Loop exists but not running - use it
return loop.run_until_complete(
get_user_llm_config_with_app(user_id, app)
)
except RuntimeError:
# No event loop exists - create a new one
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
return loop.run_until_complete(
get_user_llm_config_with_app(user_id, app)
)
finally:
loop.close()
# Fallback: No app provided, try API call (will mask group config API
# keys)
logger.warning(
"No app provided for get_llm_config, group config API keys may be "
"masked"
)
return None
except Exception as e:
logger.error("Failed to get LLM config for user %s: %s", user_id, e)
return None
@@ -0,0 +1,135 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
Request-Scoped Context Helpers for GNS3-Copilot
This module provides context variable management for request-scoped data
(JWT tokens and LLM configurations) using Python's contextvars module.
These functions are separated into their own module to avoid circular imports
between agent_service.py, gns3_copilot.py, and gns3_client modules.
Key Features:
- Thread-safe and async-safe request-scoped context management
- Automatic cleanup when request context ends
- No manual cleanup required
Usage:
from gns3server.agent.gns3_copilot.gns3_client.context_helpers import (
set_current_jwt_token,
get_current_jwt_token,
set_current_llm_config,
get_current_llm_config,
)
# In request handler
set_current_jwt_token(token)
set_current_llm_config(config)
# In downstream code
token = get_current_jwt_token()
config = get_current_llm_config()
"""
import logging
from contextvars import ContextVar
from typing import Optional
logger = logging.getLogger(__name__)
# Context variables for request-scoped data
# Automatically cleaned up when request context ends
_jwt_token_context: ContextVar[Optional[str]] = ContextVar(
"_jwt_token_context", default=None
)
_llm_config_context: ContextVar[Optional[dict]] = ContextVar(
"_llm_config_context", default=None
)
def set_current_jwt_token(token: str) -> None:
"""Set the JWT token for the current request context.
Args:
token: JWT token string
"""
_jwt_token_context.set(token)
logger.debug("JWT token set in context")
def get_current_jwt_token() -> Optional[str]:
"""Get the JWT token for the current request context.
Returns:
JWT token string if available, None otherwise
"""
token = _jwt_token_context.get()
if token:
logger.debug("JWT token retrieved from context")
else:
logger.warning("JWT token not found in context")
return token
def set_current_llm_config(config: dict) -> None:
"""Set the LLM config for the current request context.
Args:
config: LLM configuration dictionary with provider, model, api_key,
etc.
"""
_llm_config_context.set(config)
logger.debug(
"LLM config set in context: provider=%s, model=%s",
config.get("provider"),
config.get("model"),
)
def get_current_llm_config() -> Optional[dict]:
"""Get the LLM config for the current request context.
Returns:
LLM configuration dict if available, None otherwise
"""
config = _llm_config_context.get()
if config:
logger.debug(
"LLM config retrieved from context: provider=%s, model=%s",
config.get("provider"),
config.get("model"),
)
else:
logger.warning("LLM config not found in context")
return config
__all__ = [
"set_current_jwt_token",
"get_current_jwt_token",
"set_current_llm_config",
"get_current_llm_config",
]
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,178 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
GNS3 Project Info Tool
This module provides a LangChain BaseTool to retrieve basic information of a
specific GNS3 project by project ID. Returns project name, status, node count,
and link count.
"""
import logging
from pprint import pprint
from typing import Any
from langchain.tools import BaseTool
from gns3server.agent.gns3_copilot.gns3_client import Project
from gns3server.agent.gns3_copilot.gns3_client import get_gns3_connector
# Configure logging
logger = logging.getLogger(__name__)
class GNS3ProjectInfoTool(BaseTool):
"""LangChain tool for retrieving GNS3 project basic information."""
name: str = "gns3_project_info"
description: str = """
Retrieves basic information of a GNS3 project including name, status,
node count and link count.
Input: `project_id` (str, required): UUID of the GNS3 project.
Output: Dictionary with:
- `project_id`: Project UUID
- `name`: Project name
- `status`: Project status (opened/closed)
- `node_count`: Number of nodes in the project
- `link_count`: Number of links in the project
Returns a tuple format: (name, project_id, node_count, link_count, status)
"""
def _run(
self,
tool_input: Any = None,
run_manager: Any = None,
project_id: str | None = None,
) -> dict:
"""
Synchronous method to retrieve basic information of a specific GNS3 project.
Args:
tool_input : Input parameters, typically a dict or Pydantic model.
run_manager : Callback manager for tool run.
project_id : The UUID of the specific GNS3 project.
Returns:
dict: A dictionary containing project info (name, project_id, node_count,
link_count, status), or an error dictionary if an exception occurs
or project_id is not provided.
"""
# Log received input
logger.info(
"Received tool_input: %s, project_id: %s", tool_input, project_id
)
try:
# Validate project_id parameter
if not project_id:
logger.error("project_id parameter is required.")
return {
"error": (
"project_id parameter is required. Please provide a valid "
"project UUID."
)
}
# Initialize Gns3Connector using factory function
logger.debug("Connecting to GNS3 server...")
server = get_gns3_connector()
if server is None:
logger.error("Failed to create GNS3 connector")
return {
"error": (
"Failed to connect to GNS3 server. Please check your "
"configuration."
)
}
# Use the provided project_id directly
logger.info(
f"Retrieving project info for project_id: {project_id}"
)
project = Project(project_id=project_id, connector=server)
project.get() # Load project details
# Get node and link counts
nodes_inventory = project.nodes_inventory()
links_summary = project.links_summary(is_print=False)
node_count = len(nodes_inventory) if nodes_inventory else 0
link_count = len(links_summary) if links_summary else 0
# Build result in tuple format consistent with GNS3ProjectList
result = {
"project_id": project.project_id,
"name": project.name,
"status": project.status,
"node_count": node_count,
"link_count": link_count,
"tuple": (
project.name,
project.project_id,
node_count,
link_count,
project.status,
),
}
# Log result
logger.info(
"Project info retrieved: name=%s, status=%s, nodes=%d, links=%d",
project.name,
project.status,
node_count,
link_count,
)
return result
except Exception as e:
logger.error("Error retrieving GNS3 project info: %s", str(e))
return {"error": f"Failed to retrieve project info: {str(e)}"}
if __name__ == "__main__":
# Test the tool
tool = GNS3ProjectInfoTool()
# Example usage with project_id
# Replace with an actual project UUID from your GNS3 server
example_project_id = "0c0fde25-6ead-4413-a283-ea8fd2324291"
print("Testing GNS3ProjectInfoTool with project_id...")
result = tool._run(project_id=example_project_id)
pprint(result)
# Test without project_id (should return error)
print("\nTesting without project_id (should return error)...")
error_result = tool._run()
pprint(error_result)
@@ -0,0 +1,189 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
GNS3 Topology Reader Tool
This module provides a LangChain BaseTool to retrieve the topology of a
specific GNS3 project by project ID. Returns nodes, links, and project
metadata.
⚠️ WARNING: This module is shared with the MCP (Model Context Protocol) service.
GNS3TopologyTool._run() is called by MCP device config handlers.
The jwt_token/url parameters were added for MCP compatibility.
Modifications must be tested with BOTH gns3-copilot AND MCP.
"""
import copy
import logging
from pprint import pprint
from typing import Any
from langchain.tools import BaseTool
from gns3server.agent.gns3_copilot.gns3_client import Project
from gns3server.agent.gns3_copilot.gns3_client import get_gns3_connector
# Configure logging
logger = logging.getLogger(__name__)
# Define LangChain tool class
class GNS3TopologyTool(BaseTool):
"""LangChain tool for retrieving GNS3 project topology information."""
name: str = "gns3_topology_reader"
description: str = """
Retrieves the topology of a GNS3 project including nodes and links.
Input: `project_id` (str, required): UUID of the GNS3 project.
Output: Dictionary with:
- `project_id`, `name`, `status`: Project metadata
- `nodes`: Dict of node details (node_id, name, ports, console_port,
type, etc.)
- `links`: List of link connections
Use this to understand network structure before making changes.
"""
def _run(
self,
tool_input: Any = None,
run_manager: Any = None,
project_id: str | None = None,
jwt_token: str | None = None,
url: str | None = None,
) -> dict:
"""
Synchronous method to retrieve the topology of a specific GNS3 project.
Args:
tool_input: Input parameters, typically a dict or Pydantic model
containing server_url.
run_manager: Callback manager for tool run.
project_id: The UUID of the specific GNS3 project to retrieve
topology from.
jwt_token: JWT token for authentication (used by MCP handlers).
url: GNS3 server URL (used by MCP handlers).
Returns:
dict: A dictionary containing the project ID, name, status, nodes,
and links, or an error dictionary if an exception occurs
or project_id is not provided.
"""
# Log received input
logger.info(
"Received tool_input: %s, project_id: %s", tool_input, project_id
)
try:
# Validate project_id parameter
if not project_id:
logger.error("project_id parameter is required.")
return {
"error": "project_id parameter is required. "
"Please provide a valid project UUID."
}
# Initialize Gns3Connector using factory function
# jwt_token/url can be passed explicitly (e.g. from MCP handlers)
# or auto-detected (e.g. from gns3-copilot agent)
logger.debug("Connecting to GNS3 server...")
server = get_gns3_connector(jwt_token=jwt_token, url=url)
if server is None:
logger.error("Failed to create GNS3 connector")
return {
"error": "Failed to connect to GNS3 server. Please check "
"your configuration."
}
# Use the provided project_id directly
logger.info(f"Retrieving topology for project_id: {project_id}")
project = Project(project_id=project_id, connector=server)
project.get() # Load project details
# Get topology JSON: includes nodes (devices), links, etc.
topology = {
"project_id": project.project_id,
"name": project.name,
"status": project.status,
"nodes": self._clean_nodes_ports(
copy.deepcopy(project.nodes_inventory())
),
"links": project.links_summary(is_print=False),
}
# Log topology result
logger.info(
"Topology retrieved: project_id=%s, name=%s, nodes=%d, "
"links=%d",
topology.get("project_id"),
topology.get("name"),
len(topology.get("nodes", {})),
len(topology.get("links", [])),
)
logger.debug("Topology details: %s", topology)
return topology
except Exception as e:
logger.error("Error retrieving GNS3 topology: %s", str(e))
return {"error": f"Failed to retrieve topology: {str(e)}"}
def _clean_nodes_ports(self, data: dict) -> dict:
"""
Clean and simplify the nodes data structure.
Simplify each node's ports list to only keep name and short_name
fields.
"""
for node in data.values(): # Iterate through R-1, R-2, R-3, R-4
if "ports" in node and isinstance(node["ports"], list):
node["ports"] = [
{"name": port["name"], "short_name": port["short_name"]}
for port in node["ports"]
]
return data
if __name__ == "__main__":
# Test the tool
tool = GNS3TopologyTool()
# Example usage with project_id
# Replace with an actual project UUID from your GNS3 server
example_project_id = "0c0fde25-6ead-4413-a283-ea8fd2324291"
print("Testing GNS3TopologyTool with project_id...")
result = tool._run(project_id=example_project_id)
pprint(result)
# Test without project_id (should return error)
print("\nTesting without project_id (should return error)...")
error_result = tool._run()
pprint(error_result)
@@ -0,0 +1,158 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
Project Agent Manager
Manages AgentService instances for GNS3 projects using a singleton pattern.
Each project has its own AgentService with a dedicated SQLite checkpoint
database.
"""
import asyncio
import logging
from typing import Dict
from typing import Optional
from gns3server.agent.gns3_copilot.agent_service import AgentService
log = logging.getLogger(__name__)
class ProjectAgentManager:
"""
Singleton manager for project-level Agent services.
Manages the lifecycle of AgentService instances, ensuring that each project
has exactly one AgentService instance. Handles cleanup of resources when
projects are closed.
"""
_instance: Optional["ProjectAgentManager"] = None
_lock: asyncio.Lock = asyncio.Lock()
_agents: Dict[str, AgentService] = {}
def __new__(cls):
if cls._instance is None:
cls._instance = super().__new__(cls)
cls._instance._agents = {}
cls._instance._lock = asyncio.Lock()
return cls._instance
async def get_agent(
self, project_id: str, project_path: str
) -> AgentService:
"""
Get or create an AgentService for a project.
Args:
project_id: GNS3 project ID
project_path: Path to the GNS3 project directory
Returns:
AgentService instance for the project
"""
async with self._lock:
if project_id not in self._agents:
log.info(
"Creating new AgentService for project: %s at %s",
project_id,
project_path,
)
self._agents[project_id] = AgentService(project_path)
return self._agents[project_id]
async def remove_agent(self, project_id: str):
"""
Remove and cleanup an AgentService for a project.
Should be called when a project is closed to free resources.
Args:
project_id: GNS3 project ID
"""
async with self._lock:
if project_id in self._agents:
log.info("Removing AgentService for project: %s", project_id)
agent = self._agents.pop(project_id)
await agent.close()
async def close_all(self):
"""
Close all AgentService instances and cleanup resources.
Should be called on server shutdown.
"""
async with self._lock:
log.info(
"Closing all AgentService instances (%d projects)",
len(self._agents),
)
for project_id, agent in self._agents.items():
log.debug("Closing AgentService for project: %s", project_id)
await agent.close()
self._agents.clear()
def has_agent(self, project_id: str) -> bool:
"""
Check if an AgentService exists for a project.
Args:
project_id: GNS3 project ID
Returns:
True if AgentService exists, False otherwise
"""
return project_id in self._agents
@property
def active_projects(self) -> list[str]:
"""
Get list of project IDs with active AgentService instances.
Returns:
List of project IDs
"""
return list(self._agents.keys())
# Global singleton instance
_project_agent_manager: Optional[ProjectAgentManager] = None
_manager_lock = asyncio.Lock()
async def get_project_agent_manager() -> ProjectAgentManager:
"""
Get the global ProjectAgentManager singleton instance.
Returns:
ProjectAgentManager instance
"""
global _project_agent_manager
async with _manager_lock:
if _project_agent_manager is None:
_project_agent_manager = ProjectAgentManager()
log.info("ProjectAgentManager singleton created")
return _project_agent_manager
@@ -0,0 +1,52 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
Prompts Module for GNS3-Copilot
This package provides system prompts loading utilities for the GNS3-Copilot AI agent.
All system prompts are now loaded from the external GNS3-Skills repository:
https://github.com/gns3/gns3-skills
Available prompts (loaded from external repository):
- lab_automation_assistant.md: Lab automation mode (diagnostics + config)
- teaching_assistant.md: Teaching assistant mode (diagnostics only)
- troubleshooting_injection.md: Fault injection specialist
- title.md: Title generation prompt template
Use the SkillsManager to load these prompts:
from gns3server.agent.gns3_copilot.skills.manager import skills_manager
# Load prompts
lab_prompt = skills_manager.load_prompt("lab_automation_assistant")
teaching_prompt = skills_manager.load_prompt("teaching_assistant")
injection_prompt = skills_manager.load_prompt("troubleshooting_injection")
title_prompt = skills_manager.load_prompt("title")
"""
from .prompt_loader import load_system_prompt
__all__ = ["load_system_prompt"]
@@ -0,0 +1,115 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (� Yue Guobin)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
Prompt Loader for GNS3-Copilot
This module provides utilities for loading system prompts from the external
GNS3-Skills repository.
Supports multiple prompt variants based on LLM model configuration.
Available Modes (controlled by config.copilot_mode in llm_model_configs):
- "teaching_assistant" (default): Teaching assistant mode - diagnostics only,
no configuration
- "lab_automation_assistant": Full lab automation assistant mode - diagnostics
and configuration enabled
- "troubleshooting_injection": Troubleshooting issue injection mode
All prompts are loaded from the external GNS3-Skills repository via SkillsManager.
"""
import logging
from gns3server.agent.gns3_copilot.skills.registry import get_prompt
logger = logging.getLogger(__name__)
def load_system_prompt(llm_config: dict | None = None) -> str:
"""
Load the system prompt for GNS3-Copilot from the external repository.
The prompt mode is controlled by the `copilot_mode` field in the LLM
model config:
- "teaching_assistant" (default): Teaching assistant mode - diagnostics
only, no configuration
- "lab_automation_assistant": Full lab automation assistant mode -
diagnostics and configuration enabled
- "troubleshooting_injection": Troubleshooting issue injection mode -
inject network faults for practice
Prompts are loaded from the external GNS3-Skills repository via SkillsManager.
Args:
llm_config: LLM model configuration dictionary (flattened structure
from get_user_llm_config_full)
Returns:
str: The system prompt string, or empty string if not found.
"""
if not llm_config:
logger.debug(
"No LLM config provided, using default TEACHING_ASSISTANT "
"prompt mode"
)
return _load_prompt("teaching_assistant")
# llm_config is a flattened dict with copilot_mode at the top level
# Example: {"provider": "...", "model": "...", "copilot_mode": "...", ...}
mode = llm_config.get("copilot_mode", "teaching_assistant").lower()
if mode == "lab_automation_assistant":
logger.debug(
"Using LAB_AUTOMATION_ASSISTANT prompt mode (diagnostics + "
"configuration)"
)
return _load_prompt("lab_automation_assistant")
elif mode == "troubleshooting_injection":
logger.debug(
"Using TROUBLESHOOTING_INJECTION prompt mode (fault injection)"
)
return _load_prompt("troubleshooting_injection")
else:
logger.debug("Using TEACHING_ASSISTANT prompt mode (diagnostics only)")
return _load_prompt("teaching_assistant")
def _load_prompt(prompt_name: str) -> str:
"""
Load a prompt from the external skills repository.
Args:
prompt_name: Name of the prompt (e.g., "teaching_assistant")
Returns:
Prompt content as string, or empty string if not found
"""
prompt = get_prompt(prompt_name)
if prompt:
logger.debug(f"Loaded prompt '{prompt_name}' from external repository")
return prompt
logger.warning(f"Prompt '{prompt_name}' not found in external repository")
return ""
@@ -0,0 +1,72 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin
# Author: Yue Guobin
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
Skills Package
This package provides skills management for GNS3 Copilot.
All skills are loaded from the external GNS3-Skills repository.
Directory Structure:
- skills/
- registry.py # SKILLS_REGISTRY and get_skill()
- manager.py # SkillsManager - Git clone/pull and hot reload
- loader.py # SkillsLoader - YAML/Markdown file loading
"""
from .registry import (
SKILLS_REGISTRY,
INJECTION_SKILLS_REGISTRY,
get_skill,
get_injection_skill,
DeviceSkillsTool,
InjectionSkillsTool,
PacketAnalysisSkillsTool,
set_skills_manager,
get_skills_manager,
reload_injection_skills,
reload_forbidden_commands,
reload_skills_repository,
get_skills_repository_info,
)
from .manager import SkillsManager
from .loader import SkillsLoader
__all__ = [
"SKILLS_REGISTRY",
"INJECTION_SKILLS_REGISTRY",
"get_skill",
"get_injection_skill",
"DeviceSkillsTool",
"InjectionSkillsTool",
"PacketAnalysisSkillsTool",
"SkillsManager",
"SkillsLoader",
"set_skills_manager",
"get_skills_manager",
"reload_injection_skills",
"reload_skills_repository",
"get_skills_repository_info",
]
@@ -0,0 +1,320 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin
# Author: Yue Guobin
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
Skills Loader
This module provides functionality to load skills from YAML files.
Supports loading injection skills and device/feature skills.
"""
import logging
import os
from pathlib import Path
from typing import Any, Dict
try:
import yaml
except ImportError:
yaml = None
logger = logging.getLogger(__name__)
class SkillsLoader:
"""
Load skills from YAML files in the skills directory.
This loader reads YAML files and converts them to the skill dictionary
format expected by the skills registry.
"""
def __init__(self, skills_dir: str):
"""
Initialize the skills loader.
Args:
skills_dir: Path to the skills directory containing YAML files
"""
self.skills_dir = Path(skills_dir)
if not self.skills_dir.exists():
logger.warning(f"Skills directory does not exist: {self.skills_dir}")
def load_injection_skills(self) -> Dict[str, Dict[str, Any]]:
"""
Load all injection skills from YAML files.
Returns:
Dictionary mapping skill keys to skill definitions
"""
if yaml is None:
logger.error("PyYAML is not installed. Cannot load skills from YAML.")
return {}
skills = {}
injection_dir = self.skills_dir / "injection"
if not injection_dir.exists():
logger.warning(f"Injection skills directory not found: {injection_dir}")
return {}
for yaml_file in injection_dir.glob("*.yaml"):
try:
skill_data = self._load_yaml(yaml_file)
if not skill_data:
logger.warning(f"Skipping empty YAML file: {yaml_file}")
continue
# Generate key from filename (e.g., "ospf_issues.yaml" -> "injection_ospf")
skill_key = f"injection_{yaml_file.stem}"
skills[skill_key] = skill_data
logger.debug(f"Loaded injection skill: {skill_key} from {yaml_file}")
except Exception as e:
logger.error(f"Failed to load skill from {yaml_file}: {e}")
logger.debug(f"Loaded {len(skills)} injection skills from {injection_dir}")
return skills
def load_device_skills(self) -> Dict[str, Dict[str, Any]]:
"""
Load all device skills from YAML files.
Returns:
Dictionary mapping skill keys to skill definitions
"""
if yaml is None:
logger.error("PyYAML is not installed. Cannot load skills from YAML.")
return {}
skills = {}
device_dir = self.skills_dir / "device"
if not device_dir.exists():
logger.warning(f"Device skills directory not found: {device_dir}")
return {}
for yaml_file in device_dir.glob("*.yaml"):
try:
skill_data = self._load_yaml(yaml_file)
if not skill_data:
logger.warning(f"Skipping empty YAML file: {yaml_file}")
continue
# Use device_type from YAML content as the key
# Fallback to filename stem if device_type not present
skill_key = skill_data.get("device_type") if isinstance(skill_data, dict) else None
if not skill_key:
skill_key = yaml_file.stem
logger.warning(f"No device_type in {yaml_file}, using filename '{skill_key}' as key")
skills[skill_key] = skill_data
logger.debug(f"Loaded device skill: {skill_key} from {yaml_file}")
except Exception as e:
logger.error(f"Failed to load skill from {yaml_file}: {e}")
logger.debug(f"Loaded {len(skills)} device skills from device directory")
return skills
def load_feature_skills(self) -> Dict[str, Dict[str, Any]]:
"""
Load all feature skills from YAML files.
Feature skills are network planning and design functionalities
(e.g., topology planner), not device-specific features.
Returns:
Dictionary mapping skill keys to skill definitions
"""
if yaml is None:
logger.error("PyYAML is not installed. Cannot load skills from YAML.")
return {}
skills = {}
feature_dir = self.skills_dir / "feature"
if not feature_dir.exists():
logger.warning(f"Feature skills directory not found: {feature_dir}")
return {}
for yaml_file in feature_dir.glob("*.yaml"):
try:
skill_data = self._load_yaml(yaml_file)
if not skill_data:
logger.warning(f"Skipping empty YAML file: {yaml_file}")
continue
# Use device_type from YAML content as the key
# Fallback to filename stem if device_type not present
skill_key = skill_data.get("device_type") if isinstance(skill_data, dict) else None
if not skill_key:
skill_key = yaml_file.stem
logger.warning(f"No device_type in {yaml_file}, using filename '{skill_key}' as key")
skills[skill_key] = skill_data
logger.debug(f"Loaded feature skill: {skill_key} from {yaml_file}")
except Exception as e:
logger.error(f"Failed to load feature skill from {yaml_file}: {e}")
logger.debug(f"Loaded {len(skills)} feature skills from feature directory")
return skills
def load_prompt(self, prompt_name: str) -> str:
"""
Load a prompt from a markdown file.
Args:
prompt_name: Name of the prompt file (without .md extension)
Returns:
Prompt content as string, or empty string if not found
"""
prompts_dir = self.skills_dir / "prompts"
prompt_file = prompts_dir / f"{prompt_name}.md"
if not prompt_file.exists():
logger.warning(f"Prompt file not found: {prompt_file}")
return ""
try:
with open(prompt_file, "r", encoding="utf-8") as f:
content = f.read()
logger.debug(f"Loaded prompt: {prompt_name} from {prompt_file}")
return content
except Exception as e:
logger.error(f"Failed to load prompt from {prompt_file}: {e}")
return ""
def load_forbidden_commands(self) -> list:
"""
Load forbidden commands from the config directory.
Returns:
List of forbidden command patterns, or empty list if not found
"""
config_dir = self.skills_dir / "config"
config_file = config_dir / "forbidden_commands.txt"
if not config_file.exists():
logger.warning(f"Forbidden commands file not found: {config_file}")
return []
try:
commands = []
with open(config_file, "r", encoding="utf-8") as f:
for line in f:
line = line.strip()
if not line or line.startswith("#"):
continue
commands.append(line.lower())
logger.debug(f"Loaded {len(commands)} forbidden commands from {config_file}")
return commands
except Exception as e:
logger.error(f"Failed to load forbidden commands from {config_file}: {e}")
return []
def load_packet_analysis_protocols(self) -> Dict[str, Dict[str, Any]]:
"""
Load packet analysis protocol definitions from YAML files.
Returns:
Dictionary mapping protocol keys to protocol definitions.
Each protocol contains tshark_field, display_filter, and filter_examples.
"""
if yaml is None:
logger.error("PyYAML is not installed. Cannot load packet analysis protocols.")
return {}
protocols = {}
packet_analysis_dir = self.skills_dir / "packet_analysis"
if not packet_analysis_dir.exists():
logger.debug(f"Packet analysis directory not found: {packet_analysis_dir}")
return {}
for yaml_file in packet_analysis_dir.glob("*.yaml"):
try:
protocol_data = self._load_yaml(yaml_file)
if not protocol_data:
logger.warning(f"Skipping empty YAML file: {yaml_file}")
continue
# Use protocol_key field from YAML as the key
protocol_key = protocol_data.get("protocol_key")
if not protocol_key:
logger.warning(f"No 'protocol_key' field in {yaml_file}, skipping")
continue
protocols[protocol_key] = protocol_data
logger.debug(f"Loaded packet analysis protocol: {protocol_key} from {yaml_file}")
except Exception as e:
logger.error(f"Failed to load protocol from {yaml_file}: {e}")
logger.debug(f"Loaded {len(protocols)} packet analysis protocols from {packet_analysis_dir}")
return protocols
def _load_yaml(self, file_path: Path) -> Dict[str, Any]:
"""
Load a YAML file and return its content.
Args:
file_path: Path to the YAML file
Returns:
Parsed YAML content as dictionary, or empty dict if file is empty
"""
with open(file_path, "r", encoding="utf-8") as f:
data = yaml.safe_load(f)
if not isinstance(data, dict):
logger.warning(f"YAML file {file_path} is empty or has invalid format")
return {}
return data
def validate_skill_format(self, skill_data: Dict[str, Any]) -> bool:
"""
Validate that a skill dictionary has the required fields.
Args:
skill_data: Skill dictionary to validate
Returns:
True if valid, False otherwise
"""
required_fields = ["name", "description", "issues"]
for field in required_fields:
if field not in skill_data:
logger.error(f"Skill missing required field: {field}")
return False
if not isinstance(skill_data["issues"], dict):
logger.error("Skill 'issues' field must be a dictionary")
return False
for issue_key, issue_data in skill_data["issues"].items():
if not isinstance(issue_data, dict):
logger.error(f"Issue '{issue_key}' must be a dictionary")
return False
issue_required = ["name", "description"]
for field in issue_required:
if field not in issue_data:
logger.error(f"Issue '{issue_key}' missing required field: {field}")
return False
return True
@@ -0,0 +1,467 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin
# Author: Yue Guobin
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
Skills Manager
This module provides functionality to manage the skills repository,
including Git operations and hot reload of skills.
"""
import logging
import os
from pathlib import Path
from typing import Optional, Dict, Any
try:
import git
except ImportError:
git = None
from gns3server.config import Config
from .loader import SkillsLoader
logger = logging.getLogger(__name__)
# Git timeout settings for network operations.
# Applied per-command via the `env` parameter to avoid polluting
# the global process environment.
_GIT_TIMEOUT_ENV = {
'GIT_HTTP_TIMEOUT': '10', # Connection timeout (default: 120s)
'GIT_HTTP_LOW_SPEED_TIME': '5', # Slow speed threshold window
'GIT_HTTP_LOW_SPEED_LIMIT': '10240', # < 10 KB/s = slow → abort
}
class SkillsManager:
"""
Manage skills repository and hot reload functionality.
This class handles:
- Cloning the skills repository
- Pulling latest updates
- Hot reloading skills into memory
- Version tracking
"""
def __init__(
self,
repo_url: str = None,
branch: str = "main",
auto_update: bool = False
):
"""
Initialize the skills manager.
Args:
repo_url: Git repository URL (default: https://github.com/gns3/gns3-skills.git)
branch: Git branch to use (default: "main")
auto_update: Whether to automatically pull updates on reload
"""
if repo_url is None:
repo_url = "https://github.com/gns3/gns3-skills.git"
# Get local path from GNS3 config directory
config_dir = Config.instance().config_dir
local_path = os.path.join(config_dir, "skills")
self.repo_url = repo_url
self.local_path = Path(local_path)
self.branch = branch
self.auto_update = auto_update
self.loader = SkillsLoader(str(local_path))
self._repo: Optional["git.Repo"] = None
self._prompt_count = 0
if git is None:
logger.warning("GitPython is not installed. Skills management features will be limited.")
def initialize(self) -> bool:
"""
Initialize the skills repository.
Behavior:
- If no local repo → clone from remote
- If local repo exists → open and run _update_if_needed()
Returns:
True if successful, False otherwise
"""
try:
if not self.local_path.exists():
logger.info(f"Creating skills directory: {self.local_path}")
self.local_path.mkdir(parents=True, exist_ok=True)
if not (self.local_path / ".git").exists():
return self._clone()
# Repo exists, open it
try:
self._repo = git.Repo(self.local_path)
except Exception as e:
logger.error(f"Failed to open existing repository: {e}")
return False
self._update_if_needed()
return True
except Exception as e:
logger.error(f"Failed to initialize skills repository: {e}")
return False
def _update_if_needed(self) -> None:
"""
Check local repo status and pull updates if safe.
- Uncommitted changes → warn and skip
- Behind remote → pull
- Up to date → nothing
- Network error → use existing files, log warning
"""
if git is None or self._repo is None:
return
# Check for uncommitted changes
if self._repo.is_dirty(untracked_files=True):
logger.warning(
"Skills repository has uncommitted changes, skipping pull. "
"Commit or stash changes in %s to enable automatic updates.",
self.local_path
)
return
# Fetch remote
try:
origin = self._repo.remotes.origin
origin.fetch(env=_GIT_TIMEOUT_ENV)
except Exception as e:
logger.warning(f"Failed to fetch remote: {e}. Using local files.")
return
# Check if behind and pull
try:
behind_commits = list(self._repo.iter_commits(
f'{self.branch}..origin/{self.branch}'
))
if behind_commits:
logger.info(
"Skills repository is behind by %d commit(s), pulling...",
len(behind_commits)
)
origin.pull(self.branch, env=_GIT_TIMEOUT_ENV)
logger.info(f"Updated to commit {self.get_current_version()}")
else:
logger.info("Skills repository is up to date")
except Exception as e:
logger.warning(f"Failed to pull updates: {e}. Using local files.")
def _clone(self) -> bool:
"""
Clone the skills repository.
Returns:
True if successful, False otherwise
"""
if git is None:
logger.error("GitPython is not installed. Cannot clone repository.")
return False
try:
self._repo = git.Repo.clone_from(
self.repo_url,
self.local_path,
branch=self.branch,
env=_GIT_TIMEOUT_ENV
)
logger.info(f"Successfully cloned skills repository to {self.local_path}")
return True
except git.GitCommandError as e:
logger.error(f"Git clone failed: {e}")
return False
def reload_skills(self) -> bool:
"""
Hot reload skills from YAML files into the registry.
Loads the latest skill definitions from YAML files and updates
the INJECTION_SKILLS_REGISTRY.
Returns:
True if successful, False otherwise
"""
try:
# Import here to avoid circular dependency
from .registry import INJECTION_SKILLS_REGISTRY, SKILLS_REGISTRY
# Load new injection skills from YAML files
new_injection_skills = self.loader.load_injection_skills()
if not new_injection_skills:
logger.warning("No injection skills loaded, keeping existing skills")
return False
# Validate injection skills
for skill_key, skill_data in new_injection_skills.items():
if not self.loader.validate_skill_format(skill_data):
logger.error(f"Invalid skill format for {skill_key}, skipping")
continue
# Load new device skills from YAML files
new_device_skills = self.loader.load_device_skills()
# Load new feature skills from YAML files
new_feature_skills = self.loader.load_feature_skills()
# Merge device and feature skills into single registry
all_skills = {}
all_skills.update(new_device_skills)
all_skills.update(new_feature_skills)
# Update registries (safe replace - never leaves dict empty)
for k in list(INJECTION_SKILLS_REGISTRY):
if k not in new_injection_skills:
del INJECTION_SKILLS_REGISTRY[k]
INJECTION_SKILLS_REGISTRY.update(new_injection_skills)
for k in list(SKILLS_REGISTRY):
if k not in all_skills:
del SKILLS_REGISTRY[k]
SKILLS_REGISTRY.update(all_skills)
logger.info(f"Loaded {len(new_injection_skills)} injection skills, {len(new_device_skills)} device skills, and {len(new_feature_skills)} feature skills")
return True
except Exception as e:
logger.error(f"Failed to reload skills: {e}")
return False
def reload_packet_analysis_protocols(self) -> bool:
"""
Hot reload packet analysis protocol definitions from YAML files.
Loads the latest protocol definitions from YAML files and updates
the PACKET_ANALYSIS_REGISTRY.
Returns:
True if successful, False otherwise
"""
try:
from .registry import PACKET_ANALYSIS_REGISTRY
# Load new packet analysis protocols from YAML files
new_protocols = self.loader.load_packet_analysis_protocols()
# Update registry (safe replace - never leaves dict empty)
for k in list(PACKET_ANALYSIS_REGISTRY):
if k not in new_protocols:
del PACKET_ANALYSIS_REGISTRY[k]
PACKET_ANALYSIS_REGISTRY.update(new_protocols)
logger.info(f"Successfully reloaded {len(new_protocols)} packet analysis protocols")
return True
except Exception as e:
logger.error(f"Failed to reload packet analysis protocols: {e}")
return False
def reload_prompts(self) -> bool:
"""
Hot reload prompts from Markdown files.
Loads the latest prompt definitions from Markdown files in
the skills repository.
Returns:
True if successful, False otherwise
"""
try:
# Available prompt names
prompt_names = [
"lab_automation_assistant",
"teaching_assistant",
"troubleshooting_injection",
"title"
]
# Load all prompts
loaded_count = 0
for prompt_name in prompt_names:
prompt_content = self.loader.load_prompt(prompt_name)
if prompt_content:
loaded_count += 1
else:
logger.warning(f"Failed to load prompt: {prompt_name}")
if loaded_count == 0:
logger.warning("No prompts loaded, keeping existing prompts")
return False
self._prompt_count = loaded_count
logger.info(f"Loaded {loaded_count} prompts")
return True
except Exception as e:
logger.error(f"Failed to reload prompts: {e}")
return False
def load_prompt(self, prompt_name: str) -> str:
"""
Load a specific prompt from the skills repository.
Args:
prompt_name: Name of the prompt (without .md extension)
Returns:
Prompt content as string, or empty string if not found
"""
try:
return self.loader.load_prompt(prompt_name)
except Exception as e:
logger.error(f"Failed to load prompt '{prompt_name}': {e}")
return ""
def load_forbidden_commands(self) -> list:
"""
Load forbidden command patterns from the skills repository.
Returns:
List of forbidden command patterns, or empty list if not found
"""
try:
return self.loader.load_forbidden_commands()
except Exception as e:
logger.error(f"Failed to load forbidden commands: {e}")
return []
def get_current_version(self) -> str:
"""
Get the current git commit hash of the skills repository.
Returns:
Commit hash as string, or empty string if not available
"""
if git is None or self._repo is None:
try:
self._repo = git.Repo(self.local_path)
except Exception:
return ""
try:
return self._repo.head.commit.hexsha
except Exception:
return ""
def get_skill_count(self) -> int:
"""
Get the number of currently loaded injection skills.
Returns:
Number of skills in the registry
"""
try:
from .registry import INJECTION_SKILLS_REGISTRY
return len(INJECTION_SKILLS_REGISTRY)
except Exception:
return 0
def get_prompt_count(self) -> int:
"""
Get the number of currently loaded prompts.
Returns:
Number of prompts loaded
"""
return self._prompt_count
def get_repository_info(self) -> Dict[str, Any]:
"""
Get information about the skills repository.
Returns:
Dictionary containing repository information
"""
return {
"repo_url": self.repo_url,
"local_path": str(self.local_path),
"branch": self.branch,
"current_version": self.get_current_version(),
"skill_count": self.get_skill_count(),
"prompt_count": self.get_prompt_count(),
"auto_update": self.auto_update,
"is_initialized": (self.local_path / ".git").exists()
}
def rollback(self, commit_hash: str) -> bool:
"""
Rollback the skills repository to a specific commit.
Args:
commit_hash: Git commit hash to rollback to
Returns:
True if successful, False otherwise
"""
if git is None:
logger.error("GitPython is not installed. Cannot rollback.")
return False
try:
if self._repo is None:
self._repo = git.Repo(self.local_path)
self._repo.git.reset("--hard", commit_hash)
logger.info(f"Successfully rolled back to commit {commit_hash}")
# Reload skills after rollback
return self.reload_skills()
except git.GitCommandError as e:
logger.error(f"Git rollback failed: {e}")
return False
def get_available_versions(self, limit: int = 10) -> list:
"""
Get a list of recent commit hashes.
Args:
limit: Maximum number of commits to return
Returns:
List of commit information dictionaries
"""
if git is None:
return []
try:
if self._repo is None:
self._repo = git.Repo(self.local_path)
commits = []
for commit in self._repo.iter_commits(max_count=limit):
commits.append({
"hash": commit.hexsha,
"message": commit.message.strip(),
"author": str(commit.author),
"date": commit.committed_datetime.isoformat()
})
return commits
except Exception as e:
logger.error(f"Failed to get commit history: {e}")
return []
@@ -0,0 +1,857 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin
# Author: Yue Guobin
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
Skill Registry and Tools
This module provides:
- SKILLS_REGISTRY: Device/feature skills (VPCS, topology, etc.)
- INJECTION_SKILLS_REGISTRY: Fault injection skills only
- get_skill(): Function to retrieve device/feature skills
- get_injection_skill(): Function to retrieve injection skills
- DeviceSkillsTool: LangChain tool for device/feature skills
- InjectionSkillsTool: LangChain tool for injection skills
"""
import json
import logging
from typing import Any
from langchain.tools import BaseTool
from langchain_core.callbacks import CallbackManagerForToolRun
# command_filter imports are done locally in functions
# to avoid circular import (command_filter also imports from registry)
logger = logging.getLogger(__name__)
# Device/Feature skills registry - loaded from external repository
SKILLS_REGISTRY: dict[str, dict[str, Any]] = {}
# Injection skills registry - fault injection skills only
# This registry can be hot-reloaded via SkillsManager
INJECTION_SKILLS_REGISTRY: dict[str, dict[str, Any]] = {}
# Packet analysis protocols registry - loaded from external repository
# Contains protocol definitions for tshark-based packet analysis
PACKET_ANALYSIS_REGISTRY: dict[str, dict[str, Any]] = {}
# Global skills manager instance for hot reload
_skills_manager = None
_init_in_progress = False
def set_skills_manager(manager):
"""
Set the global skills manager instance.
The skills manager handles Git operations and hot reload of skills.
Args:
manager: SkillsManager instance
"""
global _skills_manager
_skills_manager = manager
def _ensure_skills_manager():
"""
Initialize the SkillsManager (idempotent, retryable on failure).
Reads config, creates SkillsManager, clones/pulls repo,
and loads skills/prompts into memory. Safe to call from
background threads - uses _init_in_progress to prevent
concurrent initialization.
On failure, resets _init_in_progress so future calls
(e.g., /reload/skills API) can retry. On success, the
manager is stored in _skills_manager and subsequent
calls return immediately.
"""
global _skills_manager, _init_in_progress
if _skills_manager is not None:
return
if _init_in_progress:
return
_init_in_progress = True
try:
from gns3server.agent.gns3_copilot.configs.skills_config import get_skills_config
from gns3server.agent.gns3_copilot.skills.manager import SkillsManager
config = get_skills_config()
if not config.get("enabled", False):
logger.info("External skills repository is disabled")
return
logger.info("Initializing SkillsManager")
manager = SkillsManager(
repo_url=config.get("repo_url"),
branch=config.get("branch", "main"),
auto_update=config.get("auto_update", False)
)
if not manager.initialize():
logger.error("Failed to initialize skills repository")
return
if manager.reload_skills():
logger.debug(f"Loaded {manager.get_skill_count()} injection skills")
else:
logger.warning("Failed to reload injection skills")
if manager.reload_prompts():
logger.debug(f"Loaded {manager.get_prompt_count()} prompts")
else:
logger.warning("Failed to reload prompts")
if manager.reload_packet_analysis_protocols():
logger.debug(f"Loaded {len(PACKET_ANALYSIS_REGISTRY)} packet analysis protocols")
else:
logger.warning("Failed to reload packet analysis protocols")
_skills_manager = manager
logger.debug("SkillsManager initialized successfully")
except Exception as e:
logger.error(f"Error initializing skills manager: {e}", exc_info=True)
finally:
_init_in_progress = False
def get_skills_manager():
"""
Get the global skills manager instance, initializing on first access.
Returns:
SkillsManager instance or None
"""
_ensure_skills_manager()
return _skills_manager
def reload_skills_repository() -> dict[str, Any]:
"""
Reload the entire skills repository.
Performs one git update check, then reloads all skills, prompts,
and forbidden commands from local files.
Returns:
Dictionary with combined reload results.
"""
manager = get_skills_manager()
if manager is None:
return {
"success": False,
"message": "Skills manager not initialized",
}
# One git update for the entire repository
manager._update_if_needed()
# Reload everything from local files
skills_ok = manager.reload_skills()
prompts_ok = manager.reload_prompts()
protocols_ok = manager.reload_packet_analysis_protocols()
# Reload forbidden commands (local import to avoid circular dependency)
from gns3server.agent.gns3_copilot.utils.command_filter import reload_forbidden_commands as _reload_fc
from gns3server.agent.gns3_copilot.utils.command_filter import get_forbidden_commands
_reload_fc()
forbidden_commands = get_forbidden_commands()
return {
"success": skills_ok or prompts_ok or protocols_ok,
"skills": skills_ok,
"skill_count": manager.get_skill_count(),
"prompts": prompts_ok,
"prompt_count": manager.get_prompt_count(),
"protocols": protocols_ok,
"protocol_count": len(PACKET_ANALYSIS_REGISTRY),
"forbidden_commands": len(forbidden_commands),
"version": manager.get_current_version(),
}
def reload_injection_skills() -> dict[str, Any]:
"""
Trigger hot reload of injection skills.
This function uses the global skills manager to pull latest changes
from the skills repository and reload the INJECTION_SKILLS_REGISTRY.
Returns:
Dictionary with status information:
{
"success": bool,
"message": str,
"skill_count": int,
"version": str
}
"""
manager = get_skills_manager()
if manager is None:
return {
"success": False,
"message": "Skills manager not initialized",
"skill_count": len(INJECTION_SKILLS_REGISTRY),
"version": ""
}
try:
success = manager.reload_skills()
return {
"success": success,
"message": "Skills reloaded successfully" if success else "Failed to reload skills",
"skill_count": manager.get_skill_count(),
"version": manager.get_current_version()
}
except Exception as e:
logger.error(f"Error during skills reload: {e}")
return {
"success": False,
"message": f"Error: {str(e)}",
"skill_count": len(INJECTION_SKILLS_REGISTRY),
"version": ""
}
def reload_prompts() -> dict[str, Any]:
"""
Trigger hot reload of system prompts.
This function uses the global skills manager to pull latest changes
from the skills repository and reload prompts from disk.
Returns:
Dictionary with status information:
{
"success": bool,
"message": str,
"prompt_count": int,
"version": str
}
"""
manager = get_skills_manager()
if manager is None:
return {
"success": False,
"message": "Skills manager not initialized",
"prompt_count": 0,
"version": ""
}
try:
success = manager.reload_prompts()
return {
"success": success,
"message": "Prompts reloaded successfully" if success else "Failed to reload prompts",
"prompt_count": manager.get_prompt_count(),
"version": manager.get_current_version()
}
except Exception as e:
logger.error(f"Error during prompts reload: {e}")
manager = get_skills_manager()
return {
"success": False,
"message": f"Error: {str(e)}",
"prompt_count": manager.get_prompt_count() if manager else 0,
"version": ""
}
def reload_forbidden_commands() -> dict[str, Any]:
"""
Hot reload forbidden commands from the skills repository.
Directly loads and caches commands from the skills repository.
Returns:
Dictionary with status information:
{
"success": bool,
"message": str,
"command_count": int,
"version": str
}
"""
try:
from gns3server.agent.gns3_copilot.utils.command_filter import reload_forbidden_commands as _reload
from gns3server.agent.gns3_copilot.utils.command_filter import get_forbidden_commands
_reload()
commands = get_forbidden_commands()
manager = get_skills_manager()
return {
"success": True,
"message": "Forbidden commands reloaded",
"command_count": len(commands),
"version": manager.get_current_version() if manager else ""
}
except Exception as e:
logger.error(f"Error during forbidden commands reload: {e}")
return {
"success": False,
"message": f"Error: {str(e)}",
"command_count": 0,
"version": ""
}
def get_prompt(prompt_name: str) -> str:
"""
Get a system prompt by name, always loading from disk.
Args:
prompt_name: Name of the prompt (e.g., "teaching_assistant")
Returns:
Prompt content as string, or empty string if not found
"""
# Always load from skills manager (no cache), triggers lazy init
manager = get_skills_manager()
if manager:
try:
prompt = manager.load_prompt(prompt_name)
if prompt:
return prompt
except Exception as e:
logger.error(f"Error loading prompt '{prompt_name}': {e}")
logger.warning(f"Prompt not found: {prompt_name}")
return ""
def get_skills_repository_info() -> dict[str, Any]:
"""
Get information about the skills repository.
Returns:
Dictionary with repository information
"""
manager = get_skills_manager()
if manager is None:
return {
"initialized": False,
"message": "Skills manager not initialized"
}
return manager.get_repository_info()
def get_skill(
device_type: str,
category: str | None = None,
detail: str = "full",
issue: str | None = None,
) -> dict[str, Any]:
"""
Get skill by device_type, with configurable detail level.
Args:
device_type: The device type identifier
category: Optional category filter
detail: Detail level - "index" (names only), "summary" (+desc/sev/diff), "full" (all)
issue: Optional specific issue key to retrieve
Returns:
Skill dictionary (detail varies by level), or error dict
"""
skill = SKILLS_REGISTRY.get(device_type, {})
if not skill:
for did, s in SKILLS_REGISTRY.items():
if s.get("name", "").lower() == device_type.lower():
skill = s
break
if not skill:
return {
"error": f"Unknown device_type: {device_type}",
"available_device_types": list(SKILLS_REGISTRY.keys()),
}
if category:
skill_category = skill.get("category", "")
if category.lower() != skill_category.lower():
return {
"error": f"device_type '{device_type}' is not in category '{category}'",
"device_category": skill.get("category"),
"available_in_category": [
did for did, s in SKILLS_REGISTRY.items()
if s.get("category", "").lower() == category.lower()
],
}
issues = skill.get("issues", {})
# Single issue lookup (most token-efficient)
if issue:
issue_def = issues.get(issue)
if not issue_def:
return {
"error": f"Unknown issue '{issue}' in {device_type}",
"available_issues": list(issues.keys()),
}
return {
"device_type": device_type,
"skill_name": skill.get("name"),
"issue": {issue: issue_def},
}
if detail == "index":
# Minimal: only issue keys and names (90%+ token savings)
return {
"device_type": device_type,
"name": skill.get("name"),
"description": skill.get("description"),
"issues": {k: v["name"] for k, v in issues.items()},
}
if detail == "summary":
# Moderate: names + description + severity + difficulty
return {
"device_type": device_type,
"name": skill.get("name"),
"description": skill.get("description"),
"issues": {
k: {
"name": v["name"],
"description": v.get("description", ""),
"severity": v.get("severity", ""),
"difficulty": v.get("difficulty", ""),
}
for k, v in issues.items()
},
}
# Full detail (original behavior)
result = dict(skill)
result["device_type"] = device_type
return result
def list_available_skills(category: str | None = None) -> list[dict[str, str]]:
"""List all available device/feature skills, optionally filtered by category."""
skills = []
for did, skill in SKILLS_REGISTRY.items():
if category:
if skill.get("category", "").lower() == category.lower():
skills.append({
"device_type": did,
"name": skill.get("name", did),
"category": skill.get("category"),
})
else:
skills.append({
"device_type": did,
"name": skill.get("name", did),
"category": skill.get("category"),
})
return skills
def get_injection_skill(
device_type: str,
detail: str = "full",
issue: str | None = None,
) -> dict[str, Any]:
"""
Get injection fault skill by device_type, with configurable detail level.
Args:
device_type: The injection fault type (e.g., "injection_ospf")
detail: Detail level - "index" (names only), "summary" (+desc/sev/diff), "full" (all)
issue: Optional specific issue key to retrieve
Returns:
Skill dictionary (detail varies by level), or error dict
"""
skill = INJECTION_SKILLS_REGISTRY.get(device_type, {})
if not skill:
return {
"error": f"Unknown injection fault type: {device_type}",
"available_fault_types": list(INJECTION_SKILLS_REGISTRY.keys()),
"hint": "Use {'action': 'list'} to see all available fault types"
}
issues = skill.get("issues", {})
# Single issue lookup (most token-efficient)
if issue:
issue_def = issues.get(issue)
if not issue_def:
return {
"error": f"Unknown issue '{issue}' in {device_type}",
"available_issues": list(issues.keys()),
}
return {
"device_type": device_type,
"skill_name": skill.get("name"),
"issue": {issue: issue_def},
}
if detail == "index":
# Minimal: only issue keys and names (90%+ token savings)
return {
"device_type": device_type,
"name": skill.get("name"),
"description": skill.get("description"),
"issues": {k: v["name"] for k, v in issues.items()},
}
if detail == "summary":
# Moderate: names + description + severity + difficulty
return {
"device_type": device_type,
"name": skill.get("name"),
"description": skill.get("description"),
"issues": {
k: {
"name": v["name"],
"description": v.get("description", ""),
"severity": v.get("severity", ""),
"difficulty": v.get("difficulty", ""),
}
for k, v in issues.items()
},
}
# Full detail (original behavior)
result = dict(skill)
result["device_type"] = device_type
return result
def list_available_injection_skills(context: list[str] | None = None) -> list[dict[str, str]]:
"""
List available injection fault skills, optionally filtered by context.
Args:
context: List of protocol/service keywords (e.g., ["ospf", "bgp", "vlan"]).
Only returns skills whose category matches any keyword.
Returns:
List of skill info dicts with device_type, name, and category.
"""
skills = []
for did, skill in INJECTION_SKILLS_REGISTRY.items():
category = skill.get("category", "")
if context:
# Match if skill category contains any context keyword
category_lower = category.lower()
if not any(kw.lower() in category_lower or kw.lower() in did.lower() for kw in context):
continue
skills.append({
"device_type": did,
"name": skill.get("name", did),
"category": category,
})
return skills
def get_packet_analysis_protocol(protocol: str) -> dict[str, Any]:
"""
Get a packet analysis protocol definition.
Args:
protocol: The protocol key (e.g., "ospf", "bgp", "icmp")
Returns:
Protocol definition dictionary with available_fields, base_filter, etc.
Returns error dict if protocol not found.
"""
protocol_data = PACKET_ANALYSIS_REGISTRY.get(protocol)
if not protocol_data:
# Try case-insensitive match
for key, data in PACKET_ANALYSIS_REGISTRY.items():
if key.lower() == protocol.lower():
protocol_data = data
protocol = key
break
if not protocol_data:
return {
"error": f"Unknown protocol: {protocol}",
"available_protocols": list(PACKET_ANALYSIS_REGISTRY.keys()),
}
return protocol_data
def list_available_packet_analysis_protocols() -> list[dict[str, str]]:
"""
List all available packet analysis protocols.
Returns:
List of protocol info dicts with protocol, name, and description.
"""
protocols = []
for key, data in PACKET_ANALYSIS_REGISTRY.items():
protocols.append({
"protocol": key,
"name": data.get("name", key),
"description": data.get("description", ""),
})
return protocols
class DeviceSkillsTool(BaseTool):
"""
LangChain tool for querying device/feature skills.
Use this tool to get device command knowledge, topology planning skills, etc.
For fault injection skills, use InjectionSkillsTool instead.
"""
name: str = "device_skills"
description: str = """
Get or list device and feature specific skills.
Provides access to device command knowledge (VPCS), topology planning, etc.
For fault injection skills, use the injection_skills tool.
INPUT FORMAT (JSON string):
{
"action": "get", # "get" (default) or "list"
"device_type": "gns3_vpcs_telnet", # Required for action="get"
"detail": "full" # "full" (default) for complete skill information
}
For action="list":
{"action": "list"} # Lists all available device/feature skills
"""
def _run(
self,
tool_input: str | dict[str, Any],
run_manager: CallbackManagerForToolRun | None = None,
**kwargs: Any,
) -> str:
"""Execute the device skills lookup."""
logger.debug("DeviceSkillsTool invoked with input: %s", tool_input)
if isinstance(tool_input, str):
try:
params = json.loads(tool_input)
except json.JSONDecodeError as e:
return json.dumps({
"error": f"Invalid JSON input: {e}",
"hint": 'Expected format: {"device_type": "xxx"} or {"action": "list"}'
}, ensure_ascii=False, indent=2)
else:
params = tool_input
action = params.get("action", "get")
if action == "list":
skills = list_available_skills()
return json.dumps({
"count": len(skills),
"skills": skills
}, ensure_ascii=False, indent=2)
device_type = params.get("device_type")
if not device_type:
return json.dumps({
"error": "Missing required field: device_type",
"available_device_types": list(SKILLS_REGISTRY.keys()),
"hint": 'Use {"action": "list"} to see all available device types'
}, ensure_ascii=False, indent=2)
category = params.get("category")
detail = params.get("detail", "full")
issue = params.get("issue")
skill = get_skill(device_type, category, detail=detail, issue=issue)
return json.dumps(skill, ensure_ascii=False, indent=2)
class InjectionSkillsTool(BaseTool):
"""
LangChain tool for querying fault injection skills.
Use this tool to list available injection fault types and get fault details.
"""
name: str = "injection_skills"
description: str = """
Get or list network fault injection skills for troubleshooting practice.
REQUIRED: When action="list", you MUST always pass "context" with the
protocols/services found in your topology analysis.
Example: {"action": "list", "context": ["ospf", "bgp", "mpls", "vlan", "stp"]}
TOKEN-EFFICIENT USAGE:
1. List faults for YOUR topology protocols (REQUIRED):
{"action": "list", "context": ["ospf", "bgp"]}
2. Get specific fault details:
{"device_type": "injection_ospf", "issue": "ospf_hello_dead_mismatch"}
{"device_type": "injection_ospf", "detail": "index"}
PARAMETERS:
- action: "list" or "get"
- context: [str] - REQUIRED for action="list". Protocols from your topology.
- device_type: Required for action="get" (e.g., "injection_ospf")
- detail: "index" | "summary" | "full"
- issue: Get single fault detail by key
"""
def _run(
self,
tool_input: str | dict[str, Any],
run_manager: CallbackManagerForToolRun | None = None,
**kwargs: Any,
) -> str:
"""Execute the injection skills lookup."""
logger.debug("InjectionSkillsTool invoked with input: %s", tool_input)
if isinstance(tool_input, str):
try:
params = json.loads(tool_input)
except json.JSONDecodeError as e:
return json.dumps({
"error": f"Invalid JSON input: {e}",
"hint": 'Expected format: {"device_type": "xxx"} or {"action": "list"}'
}, ensure_ascii=False, indent=2)
else:
params = tool_input
action = params.get("action", "get")
if action == "list":
context = params.get("context")
if not context or not isinstance(context, list) or len(context) == 0:
return json.dumps({
"error": "context parameter is required when action='list'",
"hint": "Analyze the topology and device configurations first, "
"then pass the protocols/services you found as context. "
'Example: {"action": "list", "context": ["ospf", "bgp", "vlan"]}',
"available_categories": sorted(set(
skill.get("category", "")
for skill in INJECTION_SKILLS_REGISTRY.values()
))
}, ensure_ascii=False, indent=2)
skills = list_available_injection_skills(context=context)
logger.debug(f"Injection skills filtered by context={context}: {len(skills)} matching")
return json.dumps({
"count": len(skills),
"total_available": len(INJECTION_SKILLS_REGISTRY),
"context": context,
"fault_types": skills
}, ensure_ascii=False, indent=2)
device_type = params.get("device_type")
if not device_type:
return json.dumps({
"error": "Missing required field: device_type",
"available_fault_types": list(INJECTION_SKILLS_REGISTRY.keys()),
"hint": 'Use {"action": "list"} to see all available fault types'
}, ensure_ascii=False, indent=2)
detail = params.get("detail", "full")
issue = params.get("issue")
skill = get_injection_skill(device_type, detail=detail, issue=issue)
return json.dumps(skill, ensure_ascii=False, indent=2)
class PacketAnalysisSkillsTool(BaseTool):
"""
LangChain tool for querying packet analysis protocol definitions.
Use this tool to list available protocols and get protocol-specific
tshark fields, display filters, and check rules.
"""
name: str = "packet_analysis_skills"
description: str = """
Get or list packet analysis protocol definitions.
Before calling packet_analysis tool, use this to query the protocol's
available tshark fields, display filters, and check rules.
USAGE:
- List available protocols:
{"action": "list"}
- Get protocol definition with fields:
{"action": "get", "protocol": "ospf"}
PARAMETERS:
- action: "list" or "get" (required)
- protocol: Protocol key for action="get" (e.g., "ospf", "bgp", "arp", "icmp")
"""
def _run(
self,
tool_input: str | dict[str, Any],
run_manager: CallbackManagerForToolRun | None = None,
**kwargs: Any,
) -> str:
"""Execute the packet analysis skills lookup."""
logger.debug("PacketAnalysisSkillsTool invoked with input: %s", tool_input)
if isinstance(tool_input, str):
try:
params = json.loads(tool_input)
except json.JSONDecodeError as e:
return json.dumps({
"error": f"Invalid JSON input: {e}",
"hint": 'Expected format: {"action": "get", "protocol": "ospf"}'
}, ensure_ascii=False, indent=2)
else:
params = tool_input
action = params.get("action", "get")
if action == "list":
protocols = list_available_packet_analysis_protocols()
return json.dumps({
"count": len(protocols),
"protocols": protocols
}, ensure_ascii=False, indent=2)
protocol = params.get("protocol")
if not protocol:
return json.dumps({
"error": "Missing required field: protocol",
"available_protocols": list(PACKET_ANALYSIS_REGISTRY.keys()),
}, ensure_ascii=False, indent=2)
result = get_packet_analysis_protocol(protocol)
return json.dumps(result, ensure_ascii=False, indent=2)
@@ -0,0 +1,96 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
GNS3-Copilot Tools Package
This package provides various tools for interacting with GNS3 network simulator:
- Device configuration command execution
- Display command execution
- Multiple device command execution using Nornir
- VPCS device configuration using Netmiko
- Node and link management
- Packet filter management
Main modules:
- config_tools_nornir: Multiple device configuration command execution tool using Nornir
- display_tools_nornir: Multiple device command execution tool using Nornir
- vpcs_tools_netmiko: VPCS device configuration tool using Netmiko
- gns3_create_node: GNS3 node creation tool
- gns3_create_link: GNS3 link creation tool
- gns3_start_node: GNS3 node startup tool
- gns3_get_node_temp: GNS3 template retrieval tool
- gns3_update_node_name: GNS3 node name update tool
- gns3_packet_filter: GNS3 packet filter management tool
Note: GNS3TopologyTool is now available from gns3_client package
Author: Yue Guobin (岳国宾)
"""
# Import main tool classes
from .config_tools_nornir import ExecuteMultipleDeviceConfigCommands
from .display_tools_nornir import ExecuteMultipleDeviceCommands
from .gns3_create_link import GNS3LinkTool
from .gns3_create_node import GNS3CreateNodeTool
from .gns3_get_node_temp import GNS3TemplateTool
from .gns3_packet_filter import GNS3PacketFilterTool
from .gns3_start_node import GNS3StartNodeQuickTool
from .gns3_start_node import GNS3StartNodeTool
from .gns3_stop_node import GNS3StopNodeTool
from .gns3_suspend_node import GNS3SuspendNodeTool
from .gns3_update_node_name import GNS3UpdateNodeNameTool
from .packet_analysis_tool import PacketAnalysisTool
# Dynamic version management
try:
from importlib.metadata import version
__version__ = version("gns3-copilot")
except Exception:
__version__ = "unknown"
__author__ = "Yue Guobin (岳国宾)"
__description__ = "AI-powered network automation assistant for GNS3"
__url__ = "https://github.com/yueguobin/gns3-copilot"
# Export main tool classes
__all__ = [
"ExecuteMultipleDeviceConfigCommands",
"ExecuteMultipleDeviceCommands",
"GNS3CreateNodeTool",
"GNS3LinkTool",
"GNS3PacketFilterTool",
"GNS3StartNodeTool",
"GNS3StartNodeQuickTool",
"GNS3StopNodeTool",
"GNS3SuspendNodeTool",
"GNS3UpdateNodeNameTool",
"GNS3TemplateTool",
"PacketAnalysisTool",
]
# Package initialization message
# print(f"GNS3-Copilot Tools package loaded (version {__version__})")
@@ -0,0 +1,789 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
This module provides a tool to execute configuration commands on multiple
devices in a GNS3 topology using Nornir.
⚠️ WARNING: This module is shared with the MCP (Model Context Protocol) service.
ExecuteMultipleDeviceConfigCommands._run() is called by the MCP device_config_send handler.
The jwt_token/url parameters were added for MCP compatibility.
Modifications must be tested with BOTH gns3-copilot AND MCP.
"""
import json
import logging
import re
from typing import Any
from langchain.tools import BaseTool
from langchain_core.callbacks import CallbackManagerForToolRun
from netmiko.exceptions import ReadTimeout
from nornir import InitNornir
from nornir.core import Nornir
from nornir.core.task import AggregatedResult
from nornir.core.task import Result
from nornir.core.task import Task
from nornir_netmiko.tasks import netmiko_send_config
from gns3server.agent.gns3_copilot.gns3_client import get_gns3_server_host
from gns3server.agent.gns3_copilot.utils import get_device_ports_from_topology
from gns3server.agent.gns3_copilot.utils.command_filter import (
filter_forbidden_commands,
)
# Import custom Netmiko device types for GNS3 emulation
# This registers gns3_huawei_telnet_ce and other custom device types
# NOTE: Must be imported BEFORE any Nornir operations to ensure device types are registered
from gns3server.agent.gns3_copilot.utils import custom_netmiko # noqa: F401
# Explicitly register custom device types to ensure they are available
# This is a safety measure in case the auto-registration on import doesn't work
try:
from gns3server.agent.gns3_copilot.utils.custom_netmiko import huawei_ce
from gns3server.agent.gns3_copilot.utils.custom_netmiko.ruijie_telnet import (
register_custom_device_type as register_ruijie_device_type,
)
# Re-register to ensure device types are available
huawei_ce.register_custom_device_type()
register_ruijie_device_type()
# CRITICAL: Update netmiko.ssh_dispatcher platforms lists
# The platforms variable is calculated at module import time in ssh_dispatcher
# We need to update it after registering custom device types.
# NOTE: netmiko.ssh_dispatcher is aliased to ConnectHandler function,
# so we use importlib to get the actual module
import importlib
sd = importlib.import_module("netmiko.ssh_dispatcher")
# Recalculate platforms lists to include custom device types
sd.platforms = list(sd.CLASS_MAPPER.keys())
sd.platforms.sort()
sd.platforms_base = list(sd.CLASS_MAPPER_BASE.keys())
sd.platforms_base.sort()
sd.telnet_platforms = [x for x in sd.platforms if "telnet" in x]
# Update platform strings used in error messages
sd.platforms_str = "\n" + "\n".join(sd.platforms_base)
sd.telnet_platforms_str = "\n" + "\n".join(sd.telnet_platforms)
except Exception:
# Fail silently - the import-time registration should have worked
pass
# config log
logger = logging.getLogger(__name__)
# Suppress nornir INFO logs in console (reduce verbosity)
# The logging={"enabled": False} in InitNornir only disables plugin
# internal logs, but nornir.core still logs task execution at INFO level.
# Set to WARNING to suppress these.
logging.getLogger("nornir.core").setLevel(logging.WARNING)
logging.getLogger("nornir").setLevel(logging.WARNING)
# Local Nornir configuration functions for network devices
def _get_nornir_defaults() -> dict[str, Any]:
"""Get Nornir default configuration."""
return {"data": {"location": "gns3"}}
class ExecuteMultipleDeviceConfigCommands(BaseTool):
"""
A tool to execute configuration commands on multiple devices in a GNS3
topology using Nornir.
This class uses Nornir to manage connections and execute configuration
commands on multiple devices concurrently.
IMPORTANT SAFETY NOTE:
This tool is intended for configuration changes only. Use extreme caution
when executing configuration commands.
"""
name: str = "execute_multiple_device_config_commands"
description: str = """
Executes CONFIGURATION commands on multiple devices in the current GNS3
topology.
Use this tool ONLY for changing device settings (e.g., 'configure',
'interface', 'ip address', 'router ospf').
For viewing information, use the 'execute_multiple_device_commands' tool.
Input should be a JSON object containing project_id and device configs.
Example input:
{
"project_id": "<PROJECT_UUID>",
"device_configs": [
{
"device_name": "R-1",
"config_commands": [
"interface Loopback0",
"ip address 1.1.1.1 255.255.255.255",
"description CONFIG_BY_TOOL"
]
},
{
"device_name": "R-2",
"config_commands": [
"interface Loopback0",
"ip address 2.2.2.2 255.255.255.255",
"description CONFIG_BY_TOOL"
]
}
]
}
Returns a list of dicts with device name and config results.
IMPORTANT SAFETY WARNING:
Do NOT use this tool for dangerous operations that could reboot, erase,
or factory-reset devices.
Forbidden operations include but are not limited to:
- reload / reboot commands
- write erase / erase startup-config
- format / erase nvram / delete flash:
- boot system commands
- factory-reset commands
- any commands that require user confirmation prompts
- NEVER use 'exit' command - this disconnects the session and breaks subsequent commands
"""
def _run(
self,
tool_input: str, # or Union[str, List[Any], Dict[str, Any]]
run_manager: CallbackManagerForToolRun | None = None,
jwt_token: str | None = None,
url: str | None = None,
**kwargs: Any,
) -> list[dict[str, Any]]:
"""
Executes configuration commands on multiple devices in GNS3 topology.
Args:
tool_input (str): A JSON string containing project_id and device
configuration commands to execute.
jwt_token: JWT token for GNS3 API auth (MCP handlers).
url: GNS3 server URL (MCP handlers).
Returns:
List[Dict[str, Any]]: A list of dicts containing device names and
configuration results.
"""
# Log received input
logger.debug("Received input: %s", tool_input)
# Validate input
device_configs_list, project_id = self._validate_tool_input(tool_input)
if (
isinstance(device_configs_list, list)
and len(device_configs_list) > 0
and "error" in device_configs_list[0]
):
return device_configs_list
# Filter forbidden commands and store blocked commands info
device_configs_list, blocked_commands_map = (
self._filter_forbidden_commands_from_device_configs(
device_configs_list
)
)
# Expand multiline commands (e.g., banner commands with embedded newlines)
# This converts commands like "banner motd #\nline1\nline2\n#" into
# separate commands: ["banner motd #", "line1", "line2", "#"]
device_configs_list = self._expand_multiline_commands(
device_configs_list
)
# Create a mapping of device names to their configuration commands
device_configs_map = self._configs_map(device_configs_list)
# Prepare device hosts data
try:
hosts_data = self._prepare_device_hosts_data(
device_configs_list, project_id, jwt_token=jwt_token, url=url
)
except ValueError as e:
logger.error("Failed to prepare device hosts data: %s", e)
return [{"error": str(e)}]
# Check if any devices have errors (e.g., missing device_type tag)
error_devices = {
name: data
for name, data in hosts_data.items()
if "error" in data
}
if error_devices:
logger.error(
"Devices with configuration errors: %s",
list(error_devices.keys())
)
return [
{
"device_name": name,
"status": "failed",
"error": data["error"]
}
for name, data in error_devices.items()
]
# Initialize Nornir
try:
dynamic_nr = self._initialize_nornir(hosts_data)
except ValueError as e:
logger.error("Failed to initialize Nornir: %s", e)
return [{"error": str(e)}]
results = []
# Execute all devices concurrently in a single run
try:
task_result = dynamic_nr.run(
task=self._run_all_device_configs_with_single_retry,
device_configs_map=device_configs_map,
)
# Process results for all devices
results = self._process_task_results(
device_configs_list,
hosts_data,
task_result,
blocked_commands_map,
)
except Exception as e:
# Overall execution failed
logger.error(
"Error executing configurations on all devices: %s", e
)
return [{"error": f"Execution error: {str(e)}"}]
logger.info(
"Multiple device configuration execution completed. Results: %s",
json.dumps(results, indent=2, ensure_ascii=False),
)
return results
def _run_all_device_configs_with_single_retry(
self, task: Task, device_configs_map: dict[str, list[str]]
) -> Result:
"""Execute configuration commands with single retry mechanism."""
device_name = task.host.name
config_commands = device_configs_map.get(device_name, [])
if not config_commands:
return Result(
host=task.host, result="No configuration commands to execute"
)
try:
_result = task.run(
task=netmiko_send_config, config_commands=config_commands
)
return Result(host=task.host, result=_result.result)
except ReadTimeout as e:
# Log ReadTimeout exception with full details
logger.error(
"ReadTimeout occurred for device %s: %s",
device_name,
str(e),
)
return Result(
host=task.host,
result=f"Configuration failed (ReadTimeout): {str(e)}",
failed=True,
)
except Exception as e:
# Handle Cisco IOSv L2 where '#' prompt char may be delayed,
# causing Netmiko failures. Implements retry logic.
if "netmiko_send_config (failed)" in str(e):
_result = task.run(
task=netmiko_send_config, config_commands=config_commands
)
return Result(host=task.host, result=_result.result)
# Log any other exceptions with full details
logger.error(
"Configuration failed for device %s: %s (Exception type: %s)",
device_name,
str(e),
type(e).__name__,
)
return Result(
host=task.host,
result=f"Configuration failed (Unhandled Exception): {str(e)}",
failed=True,
)
def _validate_tool_input(
self, tool_input: str | bytes | list[Any] | dict[str, Any]
) -> tuple[list[dict[str, Any]], str | None]:
"""
Validate device config command input, handling new and legacy formats.
Supports new format with project_id and device_configs.
Args:
tool_input: Input from the LangChain/LangGraph tool call.
Returns:
Tuple of (device_configs_list, project_id) or (error_list, None)
"""
parsed_input = None
# Compatibility Check and Parsing ---
# Check if the input is a string (or bytes) which needs to be parsed.
if isinstance(tool_input, (str, bytes, bytearray)):
# Handle models (like DeepSeek) that return a raw JSON string.
try:
parsed_input = json.loads(tool_input)
logger.debug("Successfully parsed tool input from JSON string.")
except json.JSONDecodeError as e:
logger.error(
"Invalid JSON string received as tool input: %s", e
)
return (
[{"error": f"Invalid JSON string input from model: {e}"}],
None,
)
else:
# Handle standard models (like GPT/OpenAI) where the framework
# has already parsed the JSON into a Python object (dict or list).
parsed_input = tool_input
logger.info(
"Using tool input directly as type: %s",
type(parsed_input).__name__,
)
# Handle new format: {"project_id": "...", "device_configs": [...]}
if isinstance(parsed_input, dict):
project_id = parsed_input.get("project_id")
device_configs = parsed_input.get("device_configs")
# Validate project_id
if not project_id:
error_msg = "Missing required 'project_id' field in input"
logger.error(error_msg)
return ([{"error": error_msg}], None)
if not self._validate_project_id(project_id):
error_msg = (
f"Invalid project_id format: {project_id}. "
"Expected UUID format."
)
logger.error(error_msg)
return ([{"error": error_msg}], None)
# Validate device_configs
if not isinstance(device_configs, list):
error_msg = "'device_configs' must be an array"
logger.error(error_msg)
return ([{"error": error_msg}], None)
if not device_configs:
logger.warning("Device configs list is empty.")
return [], project_id
return device_configs, project_id
# Handle legacy format: [...]
elif isinstance(parsed_input, list):
logger.warning(
"Using legacy format without project_id. "
"Please use new format with project_id."
)
return parsed_input, None
else:
error_msg = (
"Tool input must be a JSON object with 'project_id' and "
f"'device_configs' fields, or a legacy JSON array, but got "
f"{type(parsed_input).__name__}"
)
logger.error(error_msg)
return ([{"error": error_msg}], None)
def _validate_project_id(self, project_id: str) -> bool:
"""
Validate project_id format (UUID).
Args:
project_id: The project ID to validate
Returns:
True if valid UUID format, False otherwise
"""
uuid_pattern = (
r"^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$"
)
return bool(re.match(uuid_pattern, project_id, re.IGNORECASE))
def _filter_forbidden_commands_from_device_configs(
self, device_configs_list: list[dict[str, Any]]
) -> tuple[list[dict[str, Any]], dict[str, dict[str, str]]]:
"""
Filter out forbidden commands from device configurations.
Args:
device_configs_list: List of device configs with config_commands.
Returns:
Tuple of (filtered_configs, blocked_map):
- filtered_configs: Device configs with forbidden removed.
- blocked_map: Dict mapping device names to blocked info.
"""
filtered_list = []
blocked_commands_map: dict[str, dict[str, str]] = {}
for device_config in device_configs_list:
device_name = device_config["device_name"]
commands = device_config["config_commands"]
# Filter commands
allowed_commands, blocked_info = filter_forbidden_commands(
commands
)
# Update device config with allowed commands only
filtered_config = device_config.copy()
filtered_config["config_commands"] = allowed_commands
filtered_list.append(filtered_config)
# Store blocked commands info if any
if blocked_info:
blocked_commands_map[device_name] = blocked_info
logger.info(
"Device %s: %d command(s) blocked: %s",
device_name,
len(blocked_info),
list(blocked_info.keys()),
)
return filtered_list, blocked_commands_map
def _expand_multiline_commands(
self, device_configs_list: list[dict[str, Any]]
) -> list[dict[str, Any]]:
"""
Expand commands that contain embedded newline characters.
Some commands (like banner) are passed as single strings with embedded
newlines. This method splits them into separate commands for proper
execution by netmiko_send_config.
Note: netmiko_send_config automatically handles entering config mode,
so any config mode entry commands (configure terminal, system-view, etc.)
are left as-is and will be safely ignored by the device.
Example:
Input: ["banner motd #\nline1\nline2\n#", "exit"]
Output: ["banner motd #", "line1", "line2", "#", "exit"]
Args:
device_configs_list: List of device configs with config_commands.
Returns:
List of device configs with multiline commands expanded.
"""
expanded_list = []
for device_config in device_configs_list:
device_name = device_config["device_name"]
config_commands = device_config["config_commands"]
expanded_commands = []
for cmd in config_commands:
# Check if command contains embedded newlines
if "\n" in cmd:
# Split by newline and expand
lines = cmd.split("\n")
# Filter out empty lines but keep all content lines
expanded_commands.extend([line for line in lines if line.strip()])
logger.info(
"Device %s: Expanded multiline command (%d lines)",
device_name,
len(lines),
)
else:
# Keep single-line commands as-is
expanded_commands.append(cmd)
# Create updated device config
expanded_config = device_config.copy()
expanded_config["config_commands"] = expanded_commands
expanded_list.append(expanded_config)
return expanded_list
def _configs_map(
self, device_config_list: list[dict[str, Any]]
) -> dict[str, list[str]]:
"""Create a mapping of device names to their configuration commands."""
device_configs_map = {}
for device_config in device_config_list:
device_name = device_config["device_name"]
config_commands = device_config["config_commands"]
device_configs_map[device_name] = config_commands
return device_configs_map
def _prepare_device_hosts_data(
self,
device_config_list: list[dict[str, Any]],
project_id: str | None = None,
jwt_token: str | None = None,
url: str | None = None,
) -> dict[str, dict[str, Any]]:
"""Prepare device hosts data from topology information."""
# Extract device names list
device_names = [
device_config["device_name"]
for device_config in device_config_list
]
# Get device port information with project_id
hosts_data = get_device_ports_from_topology(
device_names, project_id, jwt_token=jwt_token, url=url
)
if not hosts_data:
error_msg = (
"Failed to get device info from topology or no valid devices. "
f"Project ID: {project_id}, Devices: {device_names}"
)
raise ValueError(error_msg)
# Check for missing devices
missing_devices = set(device_names) - set(hosts_data.keys())
if missing_devices:
logger.warning(
"Some devices not found in topology (Project ID: %s): %s",
project_id or "default",
missing_devices,
)
return hosts_data
def _initialize_nornir(
self, hosts_data: dict[str, dict[str, Any]]
) -> Nornir:
"""
Initialize Nornir with the provided hosts data.
Each host now has its own connection_options (device_type), so we only
need a single generic group for shared configuration (hostname, timeout, etc.).
This is the Nornir best practice for multi-vendor environments.
"""
try:
defaults = _get_nornir_defaults()
gns3_host = get_gns3_server_host()
# Create a single generic group for shared configuration
# Individual device types are handled at host level via connection_options
groups_data = {
"network_devices": {
"hostname": gns3_host,
"timeout": 120,
"username": "",
"password": "",
}
}
# Log device types being configured
device_types = [
host["connection_options"]["netmiko"]["extras"]["device_type"]
for host in hosts_data.values()
]
logger.info(
"Initializing Nornir: host=%s, device_types=%s, hosts=%d",
gns3_host,
list(set(device_types)),
len(hosts_data),
)
return InitNornir(
inventory={
"plugin": "DictInventory",
"options": {
"hosts": hosts_data,
"groups": groups_data,
"defaults": defaults,
},
},
runner={
"plugin": "threaded",
"options": {"num_workers": 10},
},
logging={"enabled": False},
)
except Exception as e:
logger.error("Failed to initialize Nornir: %s", e)
raise ValueError(f"Failed to initialize Nornir: {e}") from e
def _process_task_results(
self,
device_configs_list: list[dict[str, Any]],
hosts_data: dict[str, dict[str, Any]],
task_result: AggregatedResult,
blocked_commands_map: dict[str, dict[str, str]],
) -> list[dict[str, Any]]:
"""Process the task results and format them for return."""
results = []
for device_config in device_configs_list:
device_name = device_config["device_name"]
config_commands = device_config["config_commands"]
blocked_commands_info = blocked_commands_map.get(device_name, {})
# Check if device is in topology
if device_name not in hosts_data:
device_result = {
"device_name": device_name,
"status": "failed",
"error": (
f"Device '{device_name}' not found in topology or "
"missing console_port"
),
}
# Add blocked commands info if any
if blocked_commands_info:
device_result["blocked_commands"] = list(
blocked_commands_info.keys()
)
device_result["blocked_info"] = blocked_commands_info
results.append(device_result)
continue
# Check if device has execution results
if device_name not in task_result:
device_result = {
"device_name": device_name,
"status": "failed",
"error": (
f"Device '{device_name}' not found in task results"
),
}
# Add blocked commands info if any
if blocked_commands_info:
device_result["blocked_commands"] = list(
blocked_commands_info.keys()
)
device_result["blocked_info"] = blocked_commands_info
results.append(device_result)
continue
# Process execution results
multi_result = task_result[device_name]
device_result = {"device_name": device_name}
if multi_result[0].failed:
# Execution failed
device_result["status"] = "failed"
device_result["error"] = (
f"Configuration execution failed: {multi_result[0].result}"
)
device_result["output"] = multi_result[0].result
else:
# Execution successful
device_result["status"] = "success"
device_result["output"] = multi_result[0].result
device_result["config_commands"] = config_commands
# Add blocked commands info if any
if blocked_commands_info:
device_result["blocked_commands"] = list(
blocked_commands_info.keys()
)
device_result["blocked_info"] = blocked_commands_info
# Update status if some commands were blocked but succeeded
if device_result["status"] == "success":
device_result["status"] = "partial_success"
results.append(device_result)
return results
if __name__ == "__main__":
# Example usage with new format
# example tool_input with project_id
input_paras = json.dumps(
{
"project_id": "<PROJECT_UUID>",
"device_configs": [
{
"device_name": "R-1",
"config_commands": [
"interface Loopback1110",
"ip address 201.201.201.201 255.255.255.255",
"description CONFIG_BY_TOOL",
],
},
{
"device_name": "R-2",
"config_commands": [
"interface Loopback1110",
"ip address 202.202.202.202 255.255.255.255",
"description CONFIG_BY_TOOL",
],
},
{
"device_name": "SW-2",
"config_commands": [
"interface Loopback1110",
"ip address 202.202.202.202 255.255.255.255",
"description CONFIG_BY_TOOL",
],
},
{
"device_name": "SW-1",
"config_commands": [
"interface Loopback1110",
"ip address 202.202.202.202 255.255.255.255",
"description CONFIG_BY_TOOL",
],
},
],
}
)
exe_config = ExecuteMultipleDeviceConfigCommands()
failed_count = 0
for _i in range(0, 5):
exe_results = exe_config._run(tool_input=input_paras)
for result in exe_results:
for result in exe_results:
if result.get("status") == "failed":
failed_count += 1
print(f"Failed Count: {failed_count}")
# print("Execution results:")
# print(json.dumps(result, indent=2, ensure_ascii=False))
@@ -0,0 +1,715 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
This module provides a tool to execute display commands on multiple devices
in a GNS3 topology using Nornir.
⚠️ WARNING: This module is shared with the MCP (Model Context Protocol) service.
ExecuteMultipleDeviceCommands._run() is called by the MCP device_command_run handler.
The jwt_token/url parameters were added for MCP compatibility.
Modifications must be tested with BOTH gns3-copilot AND MCP.
"""
import json
import logging
import re
from typing import Any
from langchain.tools import BaseTool
from langchain_core.callbacks import CallbackManagerForToolRun
from netmiko.exceptions import ReadTimeout
from nornir import InitNornir
from nornir.core import Nornir
from nornir.core.task import AggregatedResult
from nornir.core.task import Result
from nornir.core.task import Task
from nornir_netmiko.tasks import netmiko_multiline
from gns3server.agent.gns3_copilot.gns3_client import get_gns3_server_host
from gns3server.agent.gns3_copilot.utils import get_device_ports_from_topology
from gns3server.agent.gns3_copilot.utils.command_filter import (
filter_forbidden_commands,
)
# Import custom Netmiko device types for GNS3 emulation
# This registers gns3_huawei_telnet_ce and other custom device types
# NOTE: Must be imported BEFORE any Nornir operations to ensure device types are registered
from gns3server.agent.gns3_copilot.utils import custom_netmiko # noqa: F401
# Explicitly register custom device types to ensure they are available
# This is a safety measure in case the auto-registration on import doesn't work
try:
from gns3server.agent.gns3_copilot.utils.custom_netmiko import huawei_ce
from gns3server.agent.gns3_copilot.utils.custom_netmiko.ruijie_telnet import (
register_custom_device_type as register_ruijie_device_type,
)
# Re-register to ensure device types are available
huawei_ce.register_custom_device_type()
register_ruijie_device_type()
# CRITICAL: Update netmiko.ssh_dispatcher platforms lists
# The platforms variable is calculated at module import time in ssh_dispatcher
# We need to update it after registering custom device types.
# NOTE: netmiko.ssh_dispatcher is aliased to ConnectHandler function,
# so we use importlib to get the actual module
import importlib
sd = importlib.import_module("netmiko.ssh_dispatcher")
# Recalculate platforms lists to include custom device types
sd.platforms = list(sd.CLASS_MAPPER.keys())
sd.platforms.sort()
sd.platforms_base = list(sd.CLASS_MAPPER_BASE.keys())
sd.platforms_base.sort()
sd.telnet_platforms = [x for x in sd.platforms if "telnet" in x]
# Update platform strings used in error messages
sd.platforms_str = "\n" + "\n".join(sd.platforms_base)
sd.telnet_platforms_str = "\n" + "\n".join(sd.telnet_platforms)
except Exception:
# Fail silently - the import-time registration should have worked
pass
# config log
logger = logging.getLogger(__name__)
# Suppress nornir INFO logs in console (reduce verbosity)
# The logging={"enabled": False} in InitNornir only disables plugin
# internal logs, but nornir.core still logs task execution at INFO level.
# Set to WARNING to suppress these.
logging.getLogger("nornir.core").setLevel(logging.WARNING)
logging.getLogger("nornir").setLevel(logging.WARNING)
# Local Nornir configuration functions for network devices
def _get_nornir_defaults() -> dict[str, Any]:
"""Get Nornir default configuration."""
return {"data": {"location": "gns3"}}
class ExecuteMultipleDeviceCommands(BaseTool):
"""
A READ-ONLY diagnostic tool for viewing network device configurations.
**CRITICAL: DIAGNOSIS ONLY - NO CONFIGURATION PERMISSIONS**
This tool is exclusively designed for read-only operations to inspect and
diagnose network devices. It CANNOT and MUST NOT be used for configuration.
**Allowed Command Types:**
- Display commands: show, display (Cisco/Huawei)
- Debug commands: debug ip routing, debug ospf events
- Verification commands: ping, traceroute, telnet
- Status commands: show version, show running-config, show ip route
**Strictly Prohibited:**
- Configuration mode: configure terminal, config t
- Interface configuration: interface, ip address
- Protocol configuration: router ospf, router bgp
- Any command that modifies device state
"""
name: str = "execute_multiple_device_commands"
description: str = """
**READ-ONLY DIAGNOSTIC TOOL** - View network device configurations.
Use this tool to inspect device information without making any changes.
**PERMITTED USE CASES:**
- View device status: show version, show running-config
- Check routing: show ip route, show ip ospf neighbor, show bgp summary
- Interface status: show ip interface brief, show interfaces
- Protocol diagnostics: show ospf database, show bgp routes, debug commands
- Connectivity testing: ping, traceroute
**STRICTLY FORBIDDEN:**
- NO configuration commands (configure terminal, interface, router, etc.)
- NO commands that modify device state
- NEVER use 'exit' command - this disconnects the session and breaks subsequent commands
- If you need to configure, provide guidance to the student instead
**Input Format:**
{
"project_id": "<PROJECT_UUID>",
"device_configs": [
{
"device_name": "R-1",
"commands": ["show version", "show ip interface brief"]
},
{
"device_name": "R-2",
"commands": ["show version", "show ip ospf neighbor"]
}
]
}
**Returns:** List of device outputs for diagnostic analysis.
"""
def _run(
self,
tool_input: str | bytes | list[Any] | dict[str, Any],
run_manager: CallbackManagerForToolRun | None = None,
jwt_token: str | None = None,
url: str | None = None,
**kwargs: Any,
) -> list[dict[str, Any]]:
"""
Executes READ-ONLY diagnostic commands on multiple devices in GNS3.
This method only permits display/show commands and does not allow
configuration changes to network devices.
Args:
tool_input: JSON string with project_id and diagnostic commands.
jwt_token: JWT token for GNS3 API auth (MCP handlers).
url: GNS3 server URL (MCP handlers).
Returns:
List[Dict]: A list of dicts with device names and outputs.
"""
# Log received input
logger.debug("Received input: %s", tool_input)
# Validate input
device_configs_list, project_id = self._validate_tool_input(tool_input)
if (
isinstance(device_configs_list, list)
and len(device_configs_list) > 0
and "error" in device_configs_list[0]
):
return device_configs_list
# Filter forbidden commands and store blocked commands info
device_configs_list, blocked_commands_map = (
self._filter_forbidden_commands_from_device_configs(
device_configs_list
)
)
# Create a mapping of device names to their display commands
device_configs_map = self._configs_map(device_configs_list)
# Prepare device hosts data
try:
hosts_data = self._prepare_device_hosts_data(
device_configs_list, project_id, jwt_token=jwt_token, url=url
)
except ValueError as e:
logger.error("Failed to prepare device hosts data: %s", e)
return [{"error": str(e)}]
# Check if any devices have errors (e.g., missing device_type tag)
error_devices = {
name: data
for name, data in hosts_data.items()
if "error" in data
}
if error_devices:
logger.error(
"Devices with configuration errors: %s",
list(error_devices.keys())
)
return [
{
"device_name": name,
"status": "failed",
"error": data["error"]
}
for name, data in error_devices.items()
]
# Initialize Nornir
try:
dynamic_nr = self._initialize_nornir(hosts_data)
except ValueError as e:
logger.error("Failed to initialize Nornir: %s", e)
return [{"error": str(e)}]
results = []
# Execute all devices concurrently in a single run
try:
task_result = dynamic_nr.run(
task=self._run_all_device_configs_with_single_retry,
device_configs_map=device_configs_map,
)
# Process results for all devices
results = self._process_task_results(
device_configs_list,
hosts_data,
task_result,
blocked_commands_map,
)
except Exception as e:
# Overall execution failed
logger.error("Error executing display on all devices: %s", e)
return [{"error": f"Execution error: {str(e)}"}]
logger.debug(
"Multiple device display execution completed. Results: %s",
json.dumps(results, indent=2, ensure_ascii=False),
)
return results
def _run_all_device_configs_with_single_retry(
self, task: Task, device_configs_map: dict[str, list[str]]
) -> Result:
"""Execute READ-ONLY diagnostic commands with single retry."""
device_name = task.host.name
diagnostic_commands = device_configs_map.get(device_name, [])
if not diagnostic_commands:
return Result(
host=task.host, result="No diagnostic commands to execute"
)
try:
_result = task.run(
task=netmiko_multiline,
commands=diagnostic_commands,
enable=True,
read_timeout=60,
)
return Result(host=task.host, result=_result.result)
except ReadTimeout as e:
# Log ReadTimeout exception with full details
logger.error(
"ReadTimeout occurred for device %s: %s",
device_name,
str(e),
)
return Result(
host=task.host,
result=f"diagnostic command failed (ReadTimeout): {str(e)}",
failed=True,
)
except Exception as e:
# Handle Cisco IOSv L2 where '#' prompt char may be delayed,
# causing Netmiko failures. Implements retry logic.
if "netmiko_multiline (failed)" in str(e):
_result = task.run(
task=netmiko_multiline,
commands=diagnostic_commands,
enable=True,
read_timeout=60,
)
return Result(host=task.host, result=_result.result)
# Log any other exceptions with full details
logger.error(
"diagnostic command failed for device %s: %s (Exception: %s)",
device_name,
str(e),
type(e).__name__,
)
return Result(
host=task.host,
result=f"diagnostic command failed (Unhandled): {str(e)}",
failed=True,
)
def _validate_tool_input(
self, tool_input: str | bytes | list[Any] | dict[str, Any]
) -> tuple[list[dict[str, Any]], str | None]:
"""
Validate diagnostic command input for read-only device inspection.
Handles both new and legacy input formats. Supports new format with
project_id and device_configs, as well as legacy array format.
Args:
tool_input: Input from LangChain/LangGraph tool call.
Returns:
Tuple of (device_configs_list, project_id) or (error_list, None)
"""
parsed_input = None
# Compatibility Check and Parsing ---
# Check if the input is a string (or bytes) which needs to be parsed.
if isinstance(tool_input, (str, bytes, bytearray)):
# Handle models (like DeepSeek) that return a raw JSON string.
try:
parsed_input = json.loads(tool_input)
logger.info("Successfully parsed tool input from JSON string.")
except json.JSONDecodeError as e:
logger.error(
"Invalid JSON string received as tool input: %s", e
)
return (
[{"error": f"Invalid JSON string input from model: {e}"}],
None,
)
else:
# Handle standard models (like GPT/OpenAI) where the framework
# has already parsed the JSON into a Python object (dict or list).
parsed_input = tool_input
logger.info(
"Using tool input directly as type: %s",
type(parsed_input).__name__,
)
# Handle new format: {"project_id": "...", "device_configs": [...]}
if isinstance(parsed_input, dict):
project_id = parsed_input.get("project_id")
device_configs = parsed_input.get("device_configs")
# Validate project_id
if not project_id:
error_msg = "Missing required 'project_id' field in input"
logger.error(error_msg)
return ([{"error": error_msg}], None)
if not self._validate_project_id(project_id):
error_msg = f"Invalid project_id: {project_id}. Expected UUID."
logger.error(error_msg)
return ([{"error": error_msg}], None)
# Validate device_configs
if not isinstance(device_configs, list):
error_msg = "'device_configs' must be an array"
logger.error(error_msg)
return ([{"error": error_msg}], None)
if not device_configs:
logger.warning("Device configs list is empty.")
return [], project_id
return device_configs, project_id
# Handle legacy format: [...]
elif isinstance(parsed_input, list):
logger.warning(
"Legacy input format without project_id. Use new format."
)
return parsed_input, None
else:
error_msg = (
"Tool input must be JSON with project_id and device_configs, "
f"or legacy JSON array, got {type(parsed_input).__name__}"
)
logger.error(error_msg)
return ([{"error": error_msg}], None)
def _validate_project_id(self, project_id: str) -> bool:
"""
Validate project_id format (UUID).
Args:
project_id: The project ID to validate
Returns:
True if valid UUID format, False otherwise
"""
uuid_pattern = (
r"^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$"
)
return bool(re.match(uuid_pattern, project_id, re.IGNORECASE))
def _filter_forbidden_commands_from_device_configs(
self, device_configs_list: list[dict[str, Any]]
) -> tuple[list[dict[str, Any]], dict[str, dict[str, str]]]:
"""
Filter out forbidden commands from device configurations.
Args:
device_configs_list: List of device configurations with commands.
Returns:
Tuple of (filtered_configs, blocked_map):
- filtered_configs: Device configs with forbidden removed.
- blocked_map: Dict mapping device names to blocked commands.
"""
filtered_list = []
blocked_commands_map: dict[str, dict[str, str]] = {}
for device_config in device_configs_list:
device_name = device_config["device_name"]
commands = device_config["commands"]
# Filter commands
allowed_commands, blocked_info = filter_forbidden_commands(
commands
)
# Update device config with allowed commands only
filtered_config = device_config.copy()
filtered_config["commands"] = allowed_commands
filtered_list.append(filtered_config)
# Store blocked commands info if any
if blocked_info:
blocked_commands_map[device_name] = blocked_info
logger.info(
"Device %s: %d command(s) blocked: %s",
device_name,
len(blocked_info),
list(blocked_info.keys()),
)
return filtered_list, blocked_commands_map
def _configs_map(
self, device_config_list: list[dict[str, Any]]
) -> dict[str, list[str]]:
"""Create a mapping of device names to their diagnostic commands."""
device_diagnostic_map = {}
for device_config in device_config_list:
device_name = device_config["device_name"]
diagnostic_commands = device_config["commands"]
device_diagnostic_map[device_name] = diagnostic_commands
return device_diagnostic_map
def _prepare_device_hosts_data(
self,
device_config_list: list[dict[str, Any]],
project_id: str | None = None,
jwt_token: str | None = None,
url: str | None = None,
) -> dict[str, dict[str, Any]]:
"""Prepare device hosts data from topology information."""
# Extract device names list
device_names = [
device_config["device_name"]
for device_config in device_config_list
]
# Get device port information with project_id
hosts_data = get_device_ports_from_topology(
device_names, project_id, jwt_token=jwt_token, url=url
)
if not hosts_data:
error_msg = (
f"Failed to get device info from topology. "
f"Project: {project_id}, Devices: {device_names}"
)
raise ValueError(error_msg)
# Check for missing devices
missing_devices = set(device_names) - set(hosts_data.keys())
if missing_devices:
logger.warning(
"Some devices not found in topology (Project ID: %s): %s",
project_id or "default",
missing_devices,
)
return hosts_data
def _initialize_nornir(
self, hosts_data: dict[str, dict[str, Any]]
) -> Nornir:
"""
Initialize Nornir with the provided hosts data.
Each host now has its own connection_options (device_type), so we only
need a single generic group for shared configuration (hostname, timeout, etc.).
This is the Nornir best practice for multi-vendor environments.
"""
try:
defaults = _get_nornir_defaults()
gns3_host = get_gns3_server_host()
# Create a single generic group for shared configuration
# Individual device types are handled at host level via connection_options
groups_data = {
"network_devices": {
"hostname": gns3_host,
"timeout": 120,
"username": "",
"password": "",
}
}
# Log device types being configured
device_types = [
host["connection_options"]["netmiko"]["extras"]["device_type"]
for host in hosts_data.values()
]
logger.info(
"Initializing Nornir: host=%s, device_types=%s, hosts=%d",
gns3_host,
list(set(device_types)),
len(hosts_data),
)
return InitNornir(
inventory={
"plugin": "DictInventory",
"options": {
"hosts": hosts_data,
"groups": groups_data,
"defaults": defaults,
},
},
runner={
"plugin": "threaded",
"options": {"num_workers": 10},
},
logging={"enabled": False},
)
except Exception as e:
logger.error("Failed to initialize Nornir: %s", e)
raise ValueError(f"Failed to initialize Nornir: {e}") from e
def _process_task_results(
self,
device_configs_list: list[dict[str, Any]],
hosts_data: dict[str, dict[str, Any]],
task_result: AggregatedResult,
blocked_commands_map: dict[str, dict[str, str]],
) -> list[dict[str, Any]]:
"""Process the task results and format them for return."""
results = []
for device_config in device_configs_list:
device_name = device_config["device_name"]
diagnostic_commands = device_config["commands"]
blocked_commands_info = blocked_commands_map.get(device_name, {})
# Check if device is in topology
if device_name not in hosts_data:
device_result = {
"device_name": device_name,
"status": "failed",
"error": (
f"Device '{device_name}' not found in topology "
"or missing console_port"
),
}
# Add blocked commands info if any
if blocked_commands_info:
device_result["blocked_commands"] = list(
blocked_commands_info.keys()
)
device_result["blocked_info"] = blocked_commands_info
results.append(device_result)
continue
# Check if device has execution results
if device_name not in task_result:
device_result = {
"device_name": device_name,
"status": "failed",
"error": (
f"Device '{device_name}' not found in task results"
),
}
# Add blocked commands info if any
if blocked_commands_info:
device_result["blocked_commands"] = list(
blocked_commands_info.keys()
)
device_result["blocked_info"] = blocked_commands_info
results.append(device_result)
continue
# Process execution results
multi_result = task_result[device_name]
device_result = {"device_name": device_name}
if multi_result[0].failed:
# Execution failed
device_result["status"] = "failed"
device_result["error"] = (
f"Diagnostic command failed: {multi_result[0].result}"
)
device_result["output"] = multi_result[0].result
else:
# Execution successful
device_result["status"] = "success"
device_result["output"] = multi_result[0].result
device_result["diagnostic_commands"] = diagnostic_commands
# Add blocked commands info if any
if blocked_commands_info:
device_result["blocked_commands"] = list(
blocked_commands_info.keys()
)
device_result["blocked_info"] = blocked_commands_info
# Update status if some commands were blocked but succeeded
if device_result["status"] == "success":
device_result["status"] = "partial_success"
results.append(device_result)
return results
if __name__ == "__main__":
# Example usage with new format
device_commands = json.dumps(
{
"project_id": "<PROJECT_UUID>",
"device_configs": [
{
"device_name": "R-2",
"commands": ["show version", "show ip interface brief"],
},
{
"device_name": "R-1",
"commands": ["show version", "show ip interface brief"],
},
{
"device_name": "SW-1",
"commands": ["show version", "show ip interface brief"],
},
{
"device_name": "SW-2",
"commands": ["show version", "show ip interface brief"],
},
],
}
)
exe_cmd = ExecuteMultipleDeviceCommands()
failed_count = 0
for _i in range(0, 1):
exe_results = exe_cmd._run(tool_input=device_commands)
for result in exe_results:
for result in exe_results:
if result.get("status") == "failed":
failed_count += 1
print(f"Failed Count: {failed_count}")
# print("Execution results:")
# print(json.dumps(result, indent=2, ensure_ascii=False))
@@ -0,0 +1,319 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
GNS3 link creation tool for connecting network nodes.
Provides functionality to create links between nodes in GNS3 projects
using the GNS3 API connector.
"""
import json
import logging
from pprint import pprint
from typing import Any
from langchain.tools import BaseTool
from langchain_core.callbacks import CallbackManagerForToolRun
from gns3server.agent.gns3_copilot.gns3_client import Link
from gns3server.agent.gns3_copilot.gns3_client import get_gns3_connector
# Configure logging
logger = logging.getLogger(__name__)
class GNS3LinkTool(BaseTool):
"""
Tool for creating network links between GNS3 nodes.
Creates one or more links between specified nodes in a GNS3 project
by connecting their network ports. Supports batch link creation
with error handling for individual link failures.
"""
name: str = "create_gns3_link"
description: str = """
Creates one or more links between nodes in a GNS3 project.
Input: A JSON string with:
- `project_id` (str): The UUID of the GNS3 project.
- `links` (list): A non-empty array of link definitions, each containing:
- `node_id1` (str): UUID of the first node.
- `port1` (str): Port name of the first node (e.g., 'Ethernet0/0').
- `node_id2` (str): UUID of the second node.
- `port2` (str): Port name of the second node (e.g., 'Ethernet0/0').
Note: Port names must match those from `gns3_topology_reader` tool.
Example Input:
{
"project_id": "uuid-of-project",
"links": [
{
"node_id1": "uuid-of-node1",
"port1": "Ethernet0/0",
"node_id2": "uuid-of-node2",
"port2": "Ethernet0/0"
}
]
}
Output: A list of dicts, each containing:
- For successful links: link_id, node_id1, port1, node_id2, port2.
- For failed links: error(str) with error message.
Example Output:
[
{
"link_id": "uuid-of-link",
"node_id1": "uuid-of-node1",
"port1" : "Ethernet0/0",
"node_id2": "uuid-of-node2",
"port2": "Ethernet0/0"
},
{
"error": "Port not found in link 1"
},
]
"""
def _run(
self,
tool_input: str,
run_manager: CallbackManagerForToolRun | None = None,
) -> list[dict[str, Any]]:
"""
Creates one or multiple links between nodes in a GNS3 project.
Args:
tool_input: JSON string with project_id and links array.
run_manager: LangChain run manager (unused).
Returns:
list: A list with created link details or error messages.
"""
# Log received input
logger.debug("Received input: %s", tool_input)
try:
# Parse input JSON
input_data = json.loads(tool_input)
project_id = input_data.get("project_id")
links_data = input_data.get("links", [])
# Validate input
if not project_id:
logger.error("Missing required field: project_id")
return [{"error": "Missing required field: project_id"}]
if not isinstance(links_data, list) or len(links_data) == 0:
logger.error("Invalid links data: must be a non-empty array")
return [
{"error": "Invalid links data: must be a non-empty array"}
]
# Initialize Gns3Connector using factory function
logger.info("Connecting to GNS3 server...")
gns3_server = get_gns3_connector()
if gns3_server is None:
logger.error("Failed to create GNS3 connector")
return [
{
"error": (
"Failed to connect to GNS3 server. "
"Please check your configuration."
)
}
]
created_links = []
# Process each link definition
for i, link_data in enumerate(links_data):
try:
logger.info("Creating link %d/%d", i + 1, len(links_data))
# Extract link parameters
node_id1 = link_data.get("node_id1")
port1 = link_data.get("port1")
node_id2 = link_data.get("node_id2")
port2 = link_data.get("port2")
# Validate link parameters
if not all([node_id1, port1, node_id2, port2]):
error_msg = (
f"Missing required fields in link definition {i}"
)
logger.error(error_msg)
created_links.append({"error": error_msg})
continue
# Get node details
node1 = gns3_server.get_node(
project_id=project_id, node_id=node_id1
)
node2 = gns3_server.get_node(
project_id=project_id, node_id=node_id2
)
if not node1 or not node2:
error_msg = f"Node not found in link {i}"
logger.error(error_msg)
created_links.append({"error": error_msg})
continue
# Find port information - match by name or short_name
port1_info = next(
(
port
for port in node1.get("ports", [])
if port.get("name") == port1 or port.get("short_name") == port1
),
None,
)
port2_info = next(
(
port
for port in node2.get("ports", [])
if port.get("name") == port2 or port.get("short_name") == port2
),
None,
)
if not port1_info or not port2_info:
error_msg = f"Port not found in link {i}"
logger.error(error_msg)
created_links.append({"error": error_msg})
continue
# Create the link
link = Link(
project_id=project_id,
connector=gns3_server,
nodes=[
{
"node_id": node_id1,
"adapter_number": port1_info.get(
"adapter_number", 0
),
"port_number": port1_info.get(
"port_number", 0
),
"label": {"text": port1_info.get("short_name") or port1},
},
{
"node_id": node_id2,
"adapter_number": port2_info.get(
"adapter_number", 0
),
"port_number": port2_info.get(
"port_number", 0
),
"label": {"text": port2_info.get("short_name") or port2},
},
],
)
link.create()
link.get()
# Collect link details
link_info = {
"link_id": link.link_id,
"node_id1": node_id1,
"port1": port1,
"node_id2": node_id2,
"port2": port2,
}
created_links.append(link_info)
except Exception as e:
error_msg = f"Failed to create link {i}: {str(e)}"
logger.error(error_msg)
created_links.append({"error": error_msg})
# Log final results
success_count = len(
[link for link in created_links if "error" not in link]
)
logger.info(
"Link creation completed: %d successful, %d failed",
success_count,
len(links_data) - success_count,
)
return created_links
except json.JSONDecodeError as e:
logger.error("Invalid JSON input: %s", e)
return [{"error": f"Invalid JSON input: {e}"}]
except Exception as e:
logger.error("Failed to process link creation: %s", e)
return [{"error": f"Failed to process link creation: {str(e)}"}]
if __name__ == "__main__":
# Test with single link
single_link_input = json.dumps(
{
"project_id": "your-project-uuid",
"links": [
{
"node_id1": "your-node1-uuid",
"port1": "Ethernet0/0",
"node_id2": "your-node2-uuid",
"port2": "Ethernet0/0",
}
],
}
)
# Test with multiple links
multiple_links_input = json.dumps(
{
"project_id": "your-project-uuid",
"links": [
{
"node_id1": "your-node1-uuid",
"port1": "Ethernet0/0",
"node_id2": "your-node2-uuid",
"port2": "Ethernet0/0",
},
{
"node_id1": "your-node1-uuid",
"port1": "Ethernet0/1",
"node_id2": "your-node3-uuid",
"port2": "Ethernet0/0",
},
],
}
)
tool = GNS3LinkTool()
print("=== Testing Single Link Creation ===")
result = tool._run(single_link_input)
pprint(result)
print("\n=== Testing Multiple Links Creation ===")
result = tool._run(multiple_links_input)
pprint(result)
@@ -0,0 +1,325 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
GNS3 node creation tool for network topology building.
Provides functionality to create multiple nodes in GNS3 projects
using specified templates and coordinates through the GNS3 API.
"""
import json
import logging
from pprint import pprint
from typing import Any
from langchain.tools import BaseTool
from langchain_core.callbacks import CallbackManagerForToolRun
from gns3server.agent.gns3_copilot.gns3_client import Node
from gns3server.agent.gns3_copilot.gns3_client import get_gns3_connector
# Configure logging
logger = logging.getLogger(__name__)
class GNS3CreateNodeTool(BaseTool):
"""
A LangChain tool to create multiple nodes in a GNS3 project
using specified templates and coordinates.
**Input:**
A JSON object with project_id and array of nodes with template_id,
x, y coordinates, and optional name.
Example input:
{
"project_id": "uuid-of-project",
"nodes": [
{
"template_id": "uuid-of-template",
"x": 100,
"y": -200,
"name": "R1"
},
{
"template_id": "uuid-of-template2",
"x": -200,
"y": 300,
"name": "R2"
}
]
}
**Output:**
A dictionary containing the creation results for all nodes.
Example output:
{
"project_id": "uuid-of-project",
"created_nodes": [
{
"node_id": "uuid-of-node1",
"name": "R1",
"status": "success"
},
{
"node_id": "uuid-of-node2",
"name": "R2",
"status": "success"
}
],
"total_nodes": 2,
"successful_nodes": 2,
"failed_nodes": 0
}
If error occurs during validation, returns dict with error message.
"""
name: str = "create_gns3_node"
description: str = """
Creates multiple nodes in a GNS3 project using templates and coordinates.
Input is a JSON object with project_id and array of nodes.
Each node requires: template_id, x, y. Optional: name (to set node name directly).
Example input:
{
"project_id": "uuid-of-project",
"nodes": [
{
"template_id": "uuid-of-template",
"x": 100,
"y": -200,
"name": "R1"
},
{
"template_id": "uuid-of-template2",
"x": -200,
"y": 300,
"name": "R2"
}
]
}
IMPORTANT: Ensure distance between any two nodes is greater than 250 px.
This spacing is necessary to display interface numbers clearly for better
topology visualization.
Returns a dictionary with creation results for all nodes, including
success/failure status.
If the operation fails during input validation, returns a dictionary with
an error message.
"""
def _run(
self,
tool_input: str,
run_manager: CallbackManagerForToolRun | None = None,
**kwargs: Any,
) -> dict[str, Any]:
"""
Creates nodes in a GNS3 project with templates and coordinates.
Args:
tool_input: A JSON string with project_id and an array of nodes.
run_manager: LangChain run manager (unused).
Returns:
dict: A dictionary with creation results for all nodes or an error
message.
"""
# Log received input
logger.info("Received input: %s", tool_input)
try:
# Parse input JSON
input_data = json.loads(tool_input)
project_id = input_data.get("project_id")
nodes = input_data.get("nodes", [])
# Validate input
if not project_id:
logger.error("Invalid input: Missing project_id.")
return {"error": "Missing project_id."}
if not isinstance(nodes, list) or len(nodes) == 0:
logger.error("Invalid input: nodes must be a non-empty array.")
return {"error": "nodes must be a non-empty array."}
# Validate each node in the array
for i, node_data in enumerate(nodes):
if not isinstance(node_data, dict):
logger.error(
"Invalid input: Node %d must be a dictionary.", i + 1
)
return {"error": f"Node {i + 1} must be a dictionary."}
template_id = node_data.get("template_id")
x = node_data.get("x")
y = node_data.get("y")
name = node_data.get("name")
if not all(
[
template_id,
isinstance(x, (int, float)),
isinstance(y, (int, float)),
]
):
logger.error(
"Invalid input: Node %d missing or invalid "
"template_id, x, or y.",
i + 1,
)
return {
"error": f"Node {i + 1} missing or invalid "
f"template_id, x, or y."
}
# Initialize Gns3Connector using factory function
logger.info("Connecting to GNS3 server...")
gns3_server = get_gns3_connector()
if gns3_server is None:
logger.error("Failed to create GNS3 connector")
return {
"error": "Failed to connect to GNS3 server. "
"Please check your configuration."
}
# Create nodes
logger.info(
"Creating %d nodes in project %s...", len(nodes), project_id
)
results: list[dict[str, Any]] = []
for i, node_data in enumerate(nodes):
try:
template_id = node_data.get("template_id")
x = node_data.get("x")
y = node_data.get("y")
name = node_data.get("name")
logger.info(
"Creating node %d/%d with template %s at (%s, %s), name=%s...",
i + 1,
len(nodes),
template_id,
x,
y,
name,
)
# Create node
node = Node(
project_id=project_id,
template_id=template_id,
x=x,
y=y,
name=name,
connector=gns3_server,
)
node.create()
# Retrieve node details
node.get()
node_info = {
"node_id": node.node_id,
"name": node.name,
"status": "success",
}
results.append(node_info)
except Exception as e:
error_info = {
"error": f"Node {i + 1} creation failed: {str(e)}",
"status": "failed",
}
results.append(error_info)
logger.error("Failed to create node %d: %s", i + 1, e)
# Continue with next node even if one fails
# Calculate summary statistics
successful_nodes = len(
[r for r in results if r.get("status") == "success"]
)
failed_nodes = len(
[r for r in results if r.get("status") == "failed"]
)
# Prepare final result
final_result = {
"project_id": project_id,
"created_nodes": results,
"total_nodes": len(nodes),
"successful_nodes": successful_nodes,
"failed_nodes": failed_nodes,
}
# Log the final result
logger.info(
"Node creation completed: %d successful, %d failed, %d total.",
successful_nodes,
failed_nodes,
len(nodes),
)
# Return JSON-formatted result
return final_result
except json.JSONDecodeError as e:
logger.error("Invalid JSON input: %s", e)
return {"error": f"Invalid JSON input: {e}"}
except Exception as e:
logger.error("Failed to process node creation request: %s", e)
return {
"error": f"Failed to process node creation request: {str(e)}"
}
if __name__ == "__main__":
# Test the tool locally with multiple nodes
test_input = json.dumps(
{
# TODO: Replace with actual project UUID
"project_id": "d7fc094c-685e-4db1-ac11-5e33a1b2e066",
"nodes": [
{
# TODO: Replace with actual template UUID
"template_id": "b923a635-b7cc-4cb5-9a86-9357e04c02f7",
"x": 100,
"y": -200,
"name": "R1",
},
{
# TODO: Replace with actual template UUID
"template_id": "b923a635-b7cc-4cb5-9a86-9357e04c02f7",
"x": 200,
"y": -300,
"name": "R2",
},
],
}
)
tool = GNS3CreateNodeTool()
result = tool._run(test_input)
pprint(result)
@@ -0,0 +1,183 @@
# SPDX-License-Identifier: GPL-3.0-or-later
#
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
#
# This file is part of GNS3-Copilot project.
#
# GNS3-Copilot is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your
# option) any later version.
#
# GNS3-Copilot is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
#
# Copyright (C) 2025 Yue Guobin (岳国宾)
# Author: Yue Guobin (岳国宾)
#
# Project Home: https://github.com/yueguobin/gns3-copilot
#
"""
GNS3 template retrieval tool for device discovery.
Provides functionality to retrieve all available device templates
from a GNS3 server, including template names, IDs, and types.
Filters out built-in utility templates that are not useful for network labs.
"""
import json
import logging
from pprint import pprint
from typing import Any
from langchain.tools import BaseTool
from langchain_core.callbacks import CallbackManagerForToolRun
from gns3server.agent.gns3_copilot.gns3_client import get_gns3_connector
# Configure logging
logger = logging.getLogger(__name__)
# Built-in templates to filter out (utility templates, not actual network devices)
FILTERED_TEMPLATES = {
"atm_switch", # ATM switch
"cloud", # Cloud
"ethernet_hub", # Ethernet hub
"ethernet_switch", # Ethernet switch (built-in)
"frame_relay_switch", # Frame Relay switch
"nat", # NAT device
}
def should_filter_template(template: dict[str, Any]) -> bool:
"""
Determine whether a template should be filtered out.
Filter condition:
1. Template type is in the filter list (exact match)
Args:
template: Template dictionary
Returns:
True if template should be filtered, False if it should be kept
"""
template_type = template.get("template_type", "")
# Check if template_type is in filter list
# This is the most reliable way to identify built-in utility templates
return template_type in FILTERED_TEMPLATES
class GNS3TemplateTool(BaseTool):
"""
LangChain tool to retrieve available device templates from GNS3 server.
Connects to GNS3 server and extracts name, template_id, and template_type.
Filters out built-in utility templates that are not useful for network labs.
**Input:**
No input required. Connects to GNS3 server at default URL.
**Output:**
Dict with list of dicts (name, template_id, template_type).
If error, returns dict with error message.
**Filtered Templates:**
The following built-in utility templates are excluded:
- ATM switch
- Cloud
- Ethernet hub
- Ethernet switch (built-in)
- Frame Relay switch
- NAT
"""
name: str = "get_gns3_templates"
description: str = """
Retrieves available device templates from GNS3 server for network labs.
Returns dict with list of dicts (name, template_id, template_type).
Filters out built-in utility templates (ATM switch, Cloud, Ethernet hub,
Ethernet switch, Frame Relay switch, NAT) as they are not useful for
network device configuration.
No input required.
If connection fails, returns dict with error message.
"""
def _run(
self,
tool_input: str = "",
run_manager: CallbackManagerForToolRun | None = None,
) -> dict[str, Any]:
"""
Connects to GNS3 server and retrieves available device templates.
Args:
tool_input: Optional input (not used).
run_manager: LangChain run manager (unused).
Returns:
dict: Dict with templates list or error message.
"""
try:
# Initialize Gns3Connector using factory function
logger.info("Connecting to GNS3 server...")
gns3_server = get_gns3_connector()
if gns3_server is None:
logger.error("Failed to create GNS3 connector")
return {
"error": (
"Failed to connect to GNS3 server. "
"Please check your configuration."
)
}
# Retrieve all available templates
templates = gns3_server.get_templates()
# Filter out utility templates and extract relevant info
template_info = []
filtered_count = 0
for template in templates:
# Check if template should be filtered
if should_filter_template(template):
filtered_count += 1
logger.debug("Filtered out template: %s", template.get("name"))
continue
# Extract name, template_id, and template_type
template_info.append({
"name": template.get("name", "N/A"),
"template_id": template.get("template_id", "N/A"),
"template_type": template.get("template_type", "N/A"),
})
# Return JSON-formatted result with full logging
result = {"templates": template_info}
logger.info(
"Template retrieval completed. Total: %d, Filtered: %d, Remaining: %d",
len(templates),
filtered_count,
len(template_info),
)
logger.debug("Result: %s", json.dumps(result, indent=2, ensure_ascii=False))
return result
except Exception as e:
logger.error(
"Failed to connect to GNS3 server or retrieve templates: %s", e
)
return {"error": f"Failed to retrieve templates: {str(e)}"}
if __name__ == "__main__":
# Test's tool locally
tool = GNS3TemplateTool()
result = tool._run("")
pprint(result)
Loaded 100 of 695 files, more files were not shown because too many files have changed in this diff. Show more