1693 Commits

Author SHA1 Message Date
YueGuobin
a8c96dd3f7
fix: keep port_number in Docker NIO dispatch of the batch endpoints
The project-open bulk path (_add_nio_binding / _get_existing_nio /
_update_nio_binding in routes/compute/projects.py) dropped port_number
for Docker nodes, unlike the per-node routes and the IOU branch. With
multi-port docker adapters (iol-runner nodes model 4 ports per adapter,
0ca9ccc63) every batched NIO landed on port 0 where Adapter.add_nio()
silently overwrites — the last entry per node won — so reopening a
project clobbered the port-0 links with the port-1 NIOs: links ended up
cross-wired between the wrong node pairs and the real links died
(IOL direct-link ping failures after close/reopen, EXCESSCOLL storms
from the phantom loops).
2026-09-06 00:15:43 +08:00
YueGuobin
27b55b72f9
fix: keep upstream link-carrier commands off unix-socket NIO bridges
Upstream #2870 added TAP carrier control (set_nio_tap_carrier) and a
busybox interface-status monitor to the Docker base class. Both assume
TAP wiring: unix-socket NIO bridges carry no TAP NIO, so uBridge rejects
the carrier command ('bridge has no TAP NIO') and every link create,
update or delete on a running iol-runner node would fail; the monitor
polls eth{adapter} interfaces that do not exist in the container's
network namespace (or misreports the Docker default eth0 as adapter 0).

VendorDockerVM now short-circuits _set_adapter_carrier and the interface
monitor under GNS3_UNIX_SOCKET_NIO; TAP-wired vendor nodes keep the base
behavior.
2026-09-05 22:21:18 +08:00
YueGuobin
e12ef7f272
tests: expect port_number in adapter carrier calls
Upstream #2870's carrier tests assert the two-argument
_set_adapter_carrier call; the 4-port-unit commit threads port_number
through every carrier call site, so single-port adapters now pass 0.
2026-09-05 21:58:45 +08:00
YueGuobin
e318afef86
feat: ship an iol-xe-base.txt base config for IOL Docker nodes
Follows the IOU pattern: the file lives in gns3server/configs/ and the
controller installs it into the user's configs directory on startup
(never overwriting a customized copy). Referenced by the documented
GNS3_IOL_STARTUP_CONFIG=iol-xe-base.txt template knob — without it a
fresh install had no file for the knob to resolve and nodes booted to
the initial configuration dialog.

Also fixes the configs directory named in the docs (~/GNS3/configs,
the configs_path server setting default — not ~/.config/GNS3/...).
2026-09-05 21:54:52 +08:00
YueGuobin
5685f40dd6
fix: keep payload-delivered state across container (re)creation
create() re-runs _parse_vendor_environment on every (re)create — and a
stop removes the container, so every start recreates it. Resetting the
controller-allocated application id there silently flipped nodes to the
hash fallback after their first stop/start (MAC change, plus collision
risk with the allocation pool), and dropped any pending startup-config
delivered by a PUT.

The application id and startup-config state now default via class
attributes instead of being re-initialized by the parser, and the hash
fallback is removed outright: starting an IOL node without a coordinated
allocation raises an actionable error — an uncoordinated id could
collide with the pool and blackhole traffic as a MAC loop.

Found in the E2E run: node booted app id 512, came back as 566 (hash
fallback) after one stop/start; the PUT-ed config edit vanished the same
way.
2026-09-05 21:54:52 +08:00
YueGuobin
8b3aafbbdc
feat: IOU-style startup-config for IOL Docker nodes
Templates reference a config file with the GNS3_IOL_STARTUP_CONFIG
environment knob; the controller materializes the file content into
startup_config_content on node creation (sent once, knob consumed —
the same pattern as the IOU startup_config mapping). The compute builds
the content into the node's nvram_<app id> at the next start using the
IOU nvram_import utility (IOL and IOU share the nvram container format,
verified against iol-xe 17.18.02): valid config at boot, no setup
dialog, %h hostname substitution, hostname rewrite on rename.

Semantics verified against the runner: IOL boots from NVRAM whenever it
holds a config, so a plain stop/start never re-applies the startup
config and 'write memory' survives restarts; an explicit content edit
(PUT) is re-applied on the next start and wins over the saved config,
like IOU.
2026-09-05 21:54:52 +08:00
YueGuobin
a8fa529252
feat: add tag-keyed aggregate replay over paused markers' pcaps
Markers on different links sharing a tag form one distributed capture
session. Once every marker under the tag is paused (409 otherwise), three
read-only endpoints replay it:

- GET .../markers/tags/{tag}/replay/range merges the per-marker pcaps by
  scanning 16-byte record headers only (no tshark) into a timestamp-
  ordered frame list (per-second buckets above a 5000-frame cap)
- GET .../replay/frames?ts=&window_ms= returns frames in [T, T+window];
  an empty window is a normal empty array
- GET .../replay/frame/detail lazily decodes one frame the user opened:
  raw bytes for the hex view read straight from the pcap, protocol tree
  from 'tshark -T pdml' mapped isomorphically to JSON (every attribute
  survives, values stay strings). tshark reads a /tmp scratch copy with a
  scratch HOME — hardened profiles deny it the project directory.

Sort key is (ts, source file, frame number): ts is not unique across a
merge. The ts parameter round-trips as the exact string from the frame
list. Round-trip tests pin the PDML→JSON fidelity (element count and
attribute coverage).
2026-09-05 21:54:52 +08:00
YueGuobin
5c285f631b
fix: normalize marker.match event tag to int to match the REST schema
The WS event carried the tag as str (verbatim from the MARK signal) when
present and as int when falling back to the registry, so one tag reached
consumers as two different values. Parse the signal's decimal (the exact
value we installed via 'mark <bpf> tag <id>') and keep the registered int
on malformed input; None only when neither side carries one.
2026-09-05 21:54:52 +08:00
YueGuobin
51b3c81859
fix: make the patched get_default_project_directory order-safe in tests
A product module first-imported while run_around_tests' autouse
monkeypatch of gns3server.utils.path.get_default_project_directory is
active (e.g. 'from gns3server.api.server import app' inside a test body)
binds the patched lambda into its own namespace forever: module-level
from-imports capture the object by value. Once the patch is reverted,
that module keeps calling the stale lambda, which closed over the very
test's tmppath — a directory deleted at that test's teardown. Every
later test hitting psutil.disk_usage(get_default_project_directory())
then fails with FileNotFoundError, but only when the importing test runs
before the API test files (full-suite collection order hides it).

The replacement now resolves Config.instance().settings at call time
instead of closing over the tmppath, so a frozen reference stays
correct; the test that triggered this imports gns3_app at module top
so no product import ever happens inside a patched window.

Also pins pytest-random-order (inert without --random-order) and
documents the order-independence rules for new tests in the
gns3-api-test-writing skill, including the known-red legacy files
whose tests share rows sequentially.
2026-09-05 21:54:52 +08:00
YueGuobin
0119373690
feat: allocate IOL Docker application IDs from a pool disjoint from IOU
IOL interface MACs derive from the application ID (aabb.cc{app}{iface}),
so ids must be unique across opened projects sharing computes — the same
reason IOU has its allocator. IOL Docker nodes draw from the upper half
(512-1022, netiomux's fixed peer is 1023) so the two node types can
neither collide nor starve each other; IOU behavior is unchanged.

The controller sniffs the same GNS3_IOL_RUNNER environment marker the
compute uses to select IOLDockerVM (both the nested-properties and
template/top-level-kwarg shapes), stores the id in node properties like
IOU does, and passes it through the Docker create payload. Without an
allocation the compute falls back to a stable node-derived id in the
same upper range.
2026-09-05 21:54:52 +08:00
YueGuobin
b17d021bf6
fix: derive per-node IOL app id so linked routers get distinct MACs
IOL derives interface MACs from its application ID. With a constant
local-app every node shared the same MACs, and linked routers dropped
each other's frames as MAC loops — ARP never resolved and pings were
100% lost even though the whole uBridge datapath was forwarding.

Derive local-app from the node UUID (stable across restarts, kept in
1..1022) and align remote-app with the netiomux convention CML uses
(1023). Verified end to end: two iol-xe nodes, Ethernet0/0 link,
ARP resolves both ways, ping 4/5 (first loss = ARP), graceful
stop/start keeps the NVRAM config.
2026-09-05 21:54:51 +08:00
YueGuobin
0ca9ccc637
feat: model IOL adapters as 4-port units like the IOU node type
IOL interfaces come in 4-port units (Ethernet0/0-3, Ethernet1/0-3, ...),
addressed like IOU as (adapter_number, port_number 0-3):

- IOLDockerVM builds EthernetAdapter(interfaces=4) per adapter and asks
  the runner for adapters x 4 interfaces (num-eth).
- DockerVM threads port_number through the NIO/capture API (the compute
  routes parsed it from the URL but dropped it); single-port adapters
  keep the historical bridge{N} names and command sequence, multi-port
  adapters get one bridge per port (bridge{a}_{p}).
- The unix-socket NIO wiring addresses sockets flat across adapters:
  adapter x ports-per-adapter + port, so single-port images keep their
  exact socket layout.
- The controller port list for GNS3_IOL_RUNNER docker nodes is generated
  by StandardPortFactory with the IOU naming (Ethernet{segment0}/{port0},
  segment size 4); plain docker nodes keep eth{N}.
2026-09-05 21:54:51 +08:00
YueGuobin
9b55164064
revert IOL port-name override; document custom_adapters instead
_get_container_ifname only names the kernel interface inside the
container's network namespace (single call site: docker move_to_ns) -
IOL never uses that path, and the controller port list never reads it.
Name the ports through the template's custom_adapters instead (the
mechanism SR Linux appliances use for mgmt0/e1-1): adapter N shows as
Ethernet{N/4}/{N%4}, matching the IOS CLI, with no server-side changes.
2026-09-05 21:54:12 +08:00
YueGuobin
4c818c6b37
feat: show IOL-style interface names on iol-runner ports
Name ports after the IOL interface they map to (Ethernet0/0, one 4-port
unit per adapter range) via the _get_container_ifname override point, so
the GNS3 UI matches the IOS CLI. Display only — the flat adapter number
remains the socket index.
2026-09-05 21:54:12 +08:00
YueGuobin
d04a99f934
refactor: wire unix-socket NIOs through a per-node runtime directory
uBridge cannot reach sockets bind-mounted from the node's projects
directory: the path exceeds AF_UNIX's 107-byte sun_path cap. The /proc
detour from the previous commit is a dead end too - the runner drops
from root to the server uid, which makes the process non-dumpable and
its /proc/<pid>/root unreadable for the unprivileged server.

Instead, mirror how CML itself runs the image (source=<scratch>/tmp,
target=/tmp in its node definition): bind a per-node directory from the
runtime directory (<XDG_RUNTIME_DIR>/gns3/unixio/<node-id>, next to the
uBridge control sockets) at the image's socket directory. The path is
short, the directory is owned by the server user (whom the agent drops
its privileges to), and it is removed with the node. A socket directory
covered by a persisted volume keeps working as before.

IOLDockerVM persists only /tmp/run (nested bind at /tmp/run) and cleans
stale sockets/netio dirs from the socket directory on start, skipping
the cleanup when the container is already running. A failed wiring now
stops uBridge so the next start does not hit 'bridge already exist'.
2026-09-05 21:54:12 +08:00
YueGuobin
165f271bb8
refactor: reach unix-socket NIOs through the container root in /proc
The unix-socket NIO no longer requires the socket directory to be a
persisted volume: uBridge references the sockets through
/proc/<container-pid>/root<dir>/..., which is always well under the
107-byte sun_path cap (a project directory path alone exceeds it) and
leaves the sockets ephemeral in the container's own filesystem.

This replaces the runtime-dir symlink alias and its cleanup, and the
mount-time volume enforcement. IOLDockerVM now persists only /tmp/run
(the IOL working directory: startup-config + NVRAM) as a nested bind
instead of the whole /tmp; stale socket cleanup is gone with it, as
containers are recreated on every start.
2026-09-05 21:54:12 +08:00
YueGuobin
42e26717e6
fix: alias long unix-socket NIO paths through the runtime dir
AF_UNIX sun_path caps at 107 bytes and node volume directories
(projects/<uuid>/project-files/docker/<uuid>/tmp) exceed it — uBridge
rejects the NIO with 'invalid file path size'. When the wiring path is
too long, create <XDG_RUNTIME_DIR>/gns3/unixio-<node-id> as a symlink
to the real volume directory (same trick as the uBridge control
socket) and reference the alias in the uBridge commands; the symlink is
removed when uBridge stops. Found during E2E with iol-xe:17-18-02.
2026-09-05 21:54:12 +08:00
YueGuobin
e272ad915b
feat: add IOL (iol-runner) Docker node support
Run Cisco CML containerized IOL images (e.g. iol-xe/iol-xe:17-18-02,
driven by virl.lab/cmd/iol-runner) as GNS3 Docker router nodes:

- VendorDockerVM: generic GNS3_UNIX_SOCKET_NIO/GNS3_UNIX_SOCKET_DIR knobs
  wiring adapters through AF_UNIX datagram socket pairs
  (add_nio_unix cNN.sock sNN.sock) instead of TAP + move_to_ns; node
  creation fails if the socket dir is not a persisted volume.
- IOLDockerVM (GNS3_IOL_RUNNER=1): forces skip-init + unix-socket NIO +
  /config,/tmp volumes, writes iol-config.json on every start (num-eth
  tracks adapters, runner drops to the server uid/gid so the sockets are
  reachable), pre-creates tmp/run, cleans stale sockets after unclean
  kills, and makes reload a graceful stop + full start (NVRAM flush +
  rewiring). Console stays telnet on PID 1 stdio.
- Manager selects the node class from console_type or GNS3_* environment
  markers (create-time, like console_type).
- Image-free tests (25) and feature documentation.
2026-09-05 21:54:12 +08:00
Cristi
5d08cc92a4 fix: Fix failing tests and adress the MCP 2.x incompatibility failures 2026-09-04 17:37:36 +03:00
Cristi
28f0d9c009 feat: Add Docker link carrier and interface status support 2026-09-04 09:45:41 +03:00
Jeremy Grossmann
d1b4de6b8f
Merge pull request #2868 from yueguobin/feat/template-delete-usage-guard
feat: forbid deleting templates and images still used by projects
2026-08-31 19:03:28 +02:00
YueGuobin
823cf37287
fix: list every project in template/image in-use refusal, not just the first
The usage scans broke out of the whole (project, node) iteration at the
first matching node, so deleting a template or image used by several
projects reported only one project name while the error message says
'one or more projects'. Keep scanning and list each project once.

Caught on a live server: an IOU L3 template used by two projects
(one of them closed) was refused citing only one of them.
2026-08-31 23:04:00 +08:00
YueGuobin
eed981fee3
fix: tear down controller console WebSocket forwarding cleanly on client disconnect
Closing a web console while the node kept streaming output crashed the
ws_console/vnc_console handlers with an uncaught WebSocketDisconnect from
the compute-to-client send path (only the opposite direction was guarded),
producing a full ASGI traceback on every console close.

Restructure both endpoints as symmetric forwarding tasks managed with
asyncio.wait(FIRST_COMPLETED): exceptions from either direction are
collected as task exceptions, the peer task is cancelled, the compute
WebSocket is closed, and the client is notified. The receive loops now
close and log on every exit path instead of only in the exception branch.

Tests patch the in-memory ASGI transport to deliver a conformant
websocket.disconnect on client close (it sends a non-conformant
websocket.close that starlette receive() rejects).
2026-08-31 22:43:41 +08:00
YueGuobin
13548deae8
feat: forbid deleting templates and images still used by projects
Deleting a template with prune_images, deleting an image, or pruning
orphan images only checked template references — a project node still
pointing at the image (e.g. via hda_disk_image_backing_file) was left
with a dangling reference and the project could no longer be opened.

- Add controller helpers scanning every known project (opened projects
  via in-memory nodes, closed projects via their .gns3 file) for
  template and image usage; unreadable topologies are skipped
- Guard DELETE /templates/{id}, DELETE /templates/{id}?prune_images,
  DELETE /images/{path} and /images/prune with a 409 listing the
  project names
- Run all template-delete checks before any mutation so a refused
  deletion cannot leave the template gone while its images survive
2026-08-31 22:30:42 +08:00
YueGuobin
3cf9dbff5b
fix: handle client disconnect in compute console WebSocket forwarding
The compute-side telnet_forward (and vnc_forward) let the
WebSocketDisconnect that starlette raises from send_bytes once the peer
is gone escape to the generic task-exception handler. Its str() is
empty, so every client disconnect during active node output logged a
message-less warning:

  WARNING gns3server.compute.base_node:658 Exception while forwarding
  WebSocket data to TELNET server:

Catch WebSocketDisconnect in both forwarders and log it at INFO as the
normal end of a session, and format the remaining task exceptions with
!r so their type stays visible. Mirrors the controller-side fix for
the same scenario (2324dcd74).
2026-08-29 10:09:05 +08:00
YueGuobin
d1edfbe5e8
fix: replace Bearer JWTs with path-bound access tickets in MCP download tools
link_capture_download and get_symbol embedded a 10-min JWT in the
Authorization header of the curl command they return; LLM clients
retyping that command corrupted the long token — the same failure
class as the console WebSocket URLs fixed in the previous commit.

Generalize the ticket store (console_tickets.py -> access_tickets.py,
ConsoleTicketService -> AccessTicketService): a ticket now binds to
either a node's console endpoints (WebSocket, matched against route
path params) or one exact REST resource path (capture file, symbol
image). The two binding modes are isolated — a node-bound ticket
cannot authenticate a REST resource and vice versa.

get_user_from_token redeems path-bound tickets through the existing
token parameter / Bearer header, matched exactly against
request.url.path, then reuses the shared user lookup and token_version
revocation checks. Download URLs embed ?token=<ticket> and the curl
commands no longer carry a Bearer header.
2026-08-29 01:10:19 +08:00
YueGuobin
6d6b5351fa
fix: issue short-lived console tickets instead of JWTs in node_console MCP tool
LLM clients transcribing the console WebSocket URL into shell commands
reliably corrupted the ~200-char JWT embedded in it (dropped header
segment -> "MissingAlgorithmError: Missing 'alg' value in header" on
every connection attempt). The node_console tool now mints a short
random ticket ("gns3t_" + 16 urlsafe chars, 10 min TTL, multi-use)
stored server-side and bound to the node's console endpoints:

- new ConsoleTicketService (gns3server/services/console_tickets.py),
  in-memory store with lazy expiry sweeps
- get_current_active_user_from_websocket redeems tickets through the
  existing "token" query parameter, gated on websocket.path_params so a
  ticket only authenticates the console/ws and console/vnc routes of
  the node it was minted for; the JWT path is unchanged
- redemption reuses the existing user lookup, token_version revocation
  and is_active checks, so logging out invalidates outstanding tickets
- vnc_url no longer embeds the full session JWT
- the tool docstring now tells clients to run the returned command
  verbatim instead of reconstructing the URL
2026-08-29 01:00:12 +08:00
YueGuobin
2324dcd744
fix: handle client disconnect in console WebSocket forwarding
The compute-to-client forwarding loops in ws_console and vnc_console had
no WebSocketDisconnect handling: when a client (WebUI, or an MCP-driven
websocat session killed by timeout) disconnected while the compute was
still streaming console output, the next send raised WebSocketDisconnect
that leaked all the way up to uvicorn as an ERROR-level ASGI traceback.

Catch it and log at info level, symmetric with the receive-side handlers.
2026-08-28 23:35:33 +08:00
YueGuobin
df25e037ea
fix: accept the 'local' compute id in compute tools
The compute_get/compute_images MCP tools typed compute_id as a UUID, so
passing 'local' (the actual id of the built-in compute, which the
compute_images description itself pointed to) was rejected by schema
validation. Both tools now take a string defaulting to 'local', and the
compute_get REST route resolves 'local' through the controller since the
local compute has no database entry.
2026-08-26 00:44:57 +08:00
YueGuobin
6703e50487
fix: unify the device tool error contract
The device tools reported failures in three shapes: topology-level
entries with only an error key, per-device entries with status 'error'
plus the reason under output (VPCS tool only), and raw exceptions
leaking out of template rendering. Every in-band error entry now
carries status 'failed' and an error message, and invalid Jinja2
templates are reported in-band instead of escaping the handler.
2026-08-26 00:39:38 +08:00
YueGuobin
2951af6eab
fix: reject non-VPCS nodes in the VPCS config tool
VPCS syntax typed into another node's console is silently discarded
(IOS answers % Invalid input) while the tool still reports success.
get_device_ports_from_topology now carries the GNS3 node type through
to callers, and VPCSCommands fails device preparation with a per-device
error unless the node type is vpcs.
2026-08-26 00:36:47 +08:00
YueGuobin
8a8314ab29
fix: dedupe and report automatic template creation from images
install_appliances_from_image relied on the name+version pair check in
TemplatesService, so the same appliance reached through a second image
(the CSR1000v case) created a second template sharing the name. The auto
path now skips when any template with the same name exists, whatever the
version, and returns a manifest of created and skipped candidates;
POST /images/install replies 200 with that manifest instead of an empty
204, and the image_install MCP tool surfaces it.
2026-08-26 00:04:57 +08:00
YueGuobin
888afdccbd
fix: return the created template from appliance install
POST /appliances/{id}/install replied 204 with an empty body, so the MCP
appliance_install tool crashed with 'Expecting value: line 1 column 1'
while the template had actually been created. The route now returns the
created template (201, response_model=schemas.Template), _create_template
propagates it, and the MCP handler parses the body defensively so an
empty reply degrades to a plain success message.
2026-08-25 23:39:38 +08:00
YueGuobin
73e5e27c7b
fix: keep default node naming aligned with batch submission order
The controller assigns default names (R-1, R-2, ...) and console ports
in request arrival order. A parallel batch fan-out lets thread scheduling
decide that order, so the first submitted node could end up as R-2.
Batches that rely on default naming (any node without a name) are now
created sequentially; batches with explicit names stay parallel. The
node_create tool description documents the ordering semantics and tells
callers to correlate nodes by node_id.
2026-08-25 22:22:08 +08:00
YueGuobin
636abde16c
fix: keep node file content byte-faithful in node_file_get
Splitting the file with keepends=False and rejoining with newlines
dropped the trailing newline of the last line (and every \r of CRLF
files), so returned content was shorter than the file on disk and did
not round-trip. Split with keepends=True and join the selected lines
verbatim; pagination semantics are unchanged.
2026-08-25 21:33:21 +08:00
YueGuobin
57b5baed7f
fix: keep submission order and unify status in MCP batch handlers
Batch node/link creation collected results with as_completed, so the
response order followed completion rather than the submitted array and
callers could not correlate entries. Collect in submission order via
pool.map, and report batch deletes as status=success like every other
batch action (the message still says what was deleted).
2026-08-25 21:32:09 +08:00
YueGuobin
c9bc635996
fix: propagate 405 when suspending a node without suspend support
Suspending a single VPCS/IOU node returned a fake 204: the controller
route swallowed the compute 405 that the node types honestly raise, so
callers saw success while the node stayed started. Surface the 405
instead. The best-effort swallow on suspend_all is kept (and now covered
by a test) since mixed projects legitimately contain always-running node
types.
2026-08-25 21:29:53 +08:00
YueGuobin
97e7a79117
fix: report empty projects as not locked
GET /projects/{id}/locked returned True for a project with no drawings
or nodes: both loops ran zero times and the fallback return won. Locking
and unlocking such a project always succeeded while the state stayed
"locked", so it could never be unlocked.

Report a project with nothing to lock as not locked, and re-check the
state after unlock in the route tests.
2026-08-25 13:45:36 +08:00
YueGuobin
f31bfffefc
feat: expose data_link_type on the link_marker MCP tool (create-only)
Per-link markers on serial links need the WAN encapsulation (e.g.
DLT_C_HDLC) so the BPF compiles against the right link layer — the
REST API already accepts it, but the MCP tool never forwarded it.
Create passes it through; update ignores it (changing it would
invalidate the capture file), matching the REST schema semantics.
2026-08-25 13:29:39 +08:00
YueGuobin
702fc1f6d9
fix: never allow the projects directory to become a project directory
Loading a .gns3 placed directly in the projects root registered the
shared projects directory as the project path (load_project derives
the path from the file's parent directory). Deleting such an entry ran
rmtree on the projects directory itself, wiping every project until a
root-owned file stopped it, and left a zombie entry in the controller.

Three layers of protection:

- Controller.load_project() refuses a .gns3 whose parent directory is
  the projects directory; the normal subdirectory layout is unaffected
- the Project.path setter rejects the projects directory itself and
  its ancestors, closing the same hole for POST/PUT with an explicit
  path
- Project.delete() uses realpath + commonpath instead of commonprefix:
  entries whose path is the projects root are refused, and sibling
  directories sharing a string prefix (/srv/projects-evil vs
  /srv/projects) are no longer treated as inside the projects dir

Also removes the project_load MCP tool: loading by raw server
filesystem path is a footgun for automated clients; projects can still
be opened by project_id via the remaining tools.
2026-08-25 13:17:37 +08:00
YueGuobin
629bb9194f
refactor: sink shared REST handlers into gns3_client, drop gns3fy wrappers
- Slim custom_gns3fy.py to connector-only Gns3Connector and rename to
  connector.py; delete the unused Node/Link/Project dataclasses and
  endpoint wrapper methods (~3000 lines)
- Move the MCP node/link handler implementations into
  gns3_client/api_handlers.py as the shared REST client layer consumed
  by both the MCP service and copilot tools; add available_filters
  handler (exposed as link_available_filters MCP tool) and
  build_gns3_ctx() for copilot callers
- Rewrite the tools_v2 node/link tools on top of the handlers: batch
  lifecycle actions now run in parallel, node creation is a single
  POST, project-wide status reads replace per-node GETs
- Port Project.nodes_inventory/links_summary aggregation into
  project_inventory.py (output shape preserved) and rewrite the
  topology reader / project info tools on it, dropping the unused
  stats/snapshots/drawings calls
- Delete the dead mcp/nodes.py and mcp/links.py (NODE_TOOLS/LINK_TOOLS
  had no consumers; __init__ imports handlers from api_handlers)
- Retarget mcp handler tests to patch api_handlers._get_connector and
  replace test_custom_gns3fy.py with inventory contract tests
2026-08-25 09:15:32 +08:00
YueGuobin
1649fb7b7a
api: add endpoint serving the project .gns3 topology file
GET /v3/projects/{project_id}/gns3file returns the raw .gns3 file as
application/json so the WebUI can render topology thumbnails without
opening the project. Adds a public Project.topology_file property.
2026-08-24 22:19:09 +08:00
YueGuobin
e0962a59ed
feat: add GET/PUT /v3/settings server settings API
GET returns all settings sections except the deprecated
VirtualBox/VMware ones (Server.Audit privilege); secrets are
masked and Controller.jwt_secret_key is excluded entirely.
PUT applies a partial update (Server.Modify privilege): masked
or empty secrets mean unchanged, null removes the option, the
response carries the new values plus restart_required, and a
settings.updated notification is emitted. New
Server.Audit/Server.Modify privileges are seeded into the
Administrator role at table creation.
2026-08-23 18:54:31 +08:00
YueGuobin
7d3cbf1023
feat: add config read-modify-write update and harden file watcher
Config.update_config() applies submitted options to the main
configuration file via configparser read-modify-write: unknown
options are preserved, null removes an option, the merged view
of all files is validated as ServerConfig before anything is
written (a bad file would kill the FileWatcher polling loop),
and the write is atomic (.tmp + os.replace, mode 0600). Options
whose effective value is owned by a later configuration file
raise ConfigConflictError instead of writing a no-op. The
reload logic is factored into reload_and_notify() and the file
watcher callback is exception-guarded so polling never dies.
2026-08-23 18:54:23 +08:00
YueGuobin
d3ceb453a6
fix: forward auto_close in create_project_handler
The MCP project_create tool has passed auto_close=False since 8f8abe410
(2026-06-13), but create_project_handler only forwarded {"name": name}
to the REST API — auto_close was silently dropped and the controller's
Project.__init__ default (True, unchanged since 2016) won. Every project
created via MCP since June has auto_close=true on disk and closes when
the last client disconnects.

- projects.py: forward auto_close when present in params
- test_handlers.py: assert the forwarded json_data (with and without
  auto_close)
- test_tool_params.py: the tool/handler param consistency test never
  actually checked anything — three blind spots now fixed:
  1. dispatch is asyncio.to_thread(_run_handler_sync, ...) whose
     node.func is an Attribute, not a Name — no call ever matched
  2. tools that build 'params' as a variable before passing it were
     skipped; now the initial dict literal is resolved (extra-passed
     direction only)
  3. tool functions are async defs (ast.AsyncFunctionDef) but the
  enclosing-function lookup only matched ast.FunctionDef, so tool_name
  was always None
  Also: map the two marker handlers missing from HANDLER_FILES, skip
  handlers that forward params.items() generically (wildcard), union
  passed keys across multi-branch dispatches (node_create single/batch),
  and drop two dead helpers.

Verified: full suite 1568 passed; reverting the handler fix turns both
test_tool_handler_param_consistency and test_create red.
2026-08-23 08:57:39 +08:00
YueGuobin
641d10177a
refactor: move MCP service from api/routes to agent package
MCP is an optional AI feature that already depends on
agent.gns3_copilot (Gns3Connector, nornir/netmiko tools) and whose
MCP_AVAILABLE feature flag lives in gns3server/agent. Moving it there
collocates all AI features under one tree and removes AI code from the
core REST routes.

- git mv gns3server/api/routes/mcp -> gns3server/agent/mcp (no content changes)
- api/server.py, core/tasks.py: update import paths
- tests: tests/api/routes/mcp -> tests/agent/mcp, rewrite patch BASE and
  handler imports; fix MCP_DIR depth in test_tool_params.py
- agent/__init__.py: probe the SDK via importlib.import_module so the
  top-level name "mcp" is not bound in the agent namespace (it would
  shadow the new gns3server.agent.mcp subpackage and break
  'from gns3server.agent import mcp')
- docs: update source file paths

Verified: full suite 1567 passed; 82 MCP tools registered, SSE mounted
at /v3/mcp/transport.
2026-08-22 19:32:13 +08:00
YueGuobin
e98c51889e
fix: notification ping starved under sustained event load
NotificationQueue.get only generated a synthetic ping when the queue
was idle for the full timeout. Under sustained event load (e.g. a
project with markers matching at 15-260 events/s) the queue never
idled, so compute notification streams never carried a ping and the
controller stopped emitting compute.updated: clients lost compute
statistics until the event flow paused or the server restarted.

A ping is now guaranteed at least every timeout seconds regardless of
event flow: when the ping deadline is reached the next get() returns a
ping ahead of queued events (pings only carry statistics, so skipping
ahead of real events is harmless). Both the compute stream (compute
CPU/memory/disk stats -> compute.updated) and the controller stream
(idle keepalive) benefit.
2026-08-22 00:50:49 +08:00
YueGuobin
c709d74826
fix: compute notification stream silently died on uncaught exceptions
Two exception paths could permanently kill the compute notification
chain (no more compute.updated events, no reconnection until a server
restart):

- connect() only caught ComputeError, but _run_http_query translates
  HTTP status errors (401/403/404/...) into sibling ControllerError
  subclasses (and a raw fastapi HTTPException for unexpected statuses).
  Those escaped the fire-and-forget connect() task started at controller
  startup and died silently. Now they notify clients, schedule an
  exponential-backoff retry, and still re-raise for explicit callers.
  The dead web.HTTP* except branches (never reached since
  _run_http_query converts HTTP errors itself) are removed.

- _connect_notification() only caught aiohttp.ClientError. A malformed
  frame (e.g. missing 'action') or any error raised while dispatching a
  compute event (e.g. a pydantic ValidationError in
  node.parse_node_response) escaped the task, skipped the reconnect
  scheduling placed after the try block, and killed the stream forever.
  Now any exception is logged with its traceback (the gather() future
  holding it was never retrieved, so nothing was ever printed) and the
  reconnect scheduling + final compute.updated emit live in the finally
  block so every exit path recovers.

Also moves the usage-stats reset before the disconnect log line so the
emitted compute.updated snapshot is consistent.
2026-08-22 00:36:24 +08:00
YueGuobin
89d7f866cb
docker: replace vendor SKIP_INIT exec volume bridge with create-time direct binds
The SKIP_INIT volume bridge replicated init.sh's seed + mount --bind script
via docker exec *after* the container started. That copied the mechanism but
not the invariant that makes init.sh safe — the entrypoint position, which
guarantees the volume is in place before the application runs. The exec runs
concurrently with the NOS boot, so whether the NOS loaded its persisted
config or the overlay's factory copy was a timing race:

- single node stop/start on an idle system won it (exec ~1s, SR Linux reads
  its startup config at ~2-4s) — the save/stop/start round-trip passed;
- a server restart + project reload lost it (concurrent node starts queue on
  the Docker API, delaying the exec by seconds) — SR Linux booted factory
  while the persisted config.json sat intact on the host;
- XRd was immune (systemd boots tens of seconds before XR touches
  /xr-storage), which is why the race was never observed on it.

Replace the bridge entirely:

- new DockerVM._prepare_volumes hook (no-op in the base class) runs in
  create() after the image is present, before the container is created;
  VendorDockerVM overrides it to seed each volume's host directory from the
  image (throwaway docker create container + docker cp -a, nothing
  executes). The .gns3_perms marker gates the seeding: a volume that ever
  started is never re-seeded, so saved configuration is never overwritten
  with factory content (also the upgrade path for existing nodes).
- VendorDockerVM._mount_binds now binds the volumes directly at their real
  in-container paths (/etc/opt/srlinux) instead of /gns3volumes aliases, so
  the persisted config is visible to the NOS from the very first process.
- _setup_skip_init_volumes and its start() call are gone; the container-side
  _fix_permissions targets the volume paths directly (the direct binds
  exist for the whole container lifetime, unlike the old bridge).

The volume-list computation (validation + overlap de-duplication) moves
into DockerVM._persistent_volume_list so create-time seeding and _mount_binds
cannot drift apart.
2026-08-22 00:09:13 +08:00
YueGuobin
6aeb5dbda5
feat(copilot): device skills per-topic split layout and topic retrieval
SkillsLoader.load_device_skills() now supports two device layouts: the
existing single file and a split directory (device/<device>/_base.yaml +
one YAML per protocol topic). Topic files are merged into the device
skill under 'topics', keyed by their 'topic' field; mismatched
device_type, missing _base.yaml and duplicate topics are handled with
explicit log-and-skip.

get_skill() and DeviceSkillsTool gain a 'topic' parameter following the
injection list -> index -> issue pattern. Topic bodies are never
returned without an explicit topic request - index/summary/full all
serve a topic index instead - so growing a device with new protocol
topics no longer grows the token cost of device-level lookups.

Also fix reload_skills() to actually drop injection skills that fail
validate_skill_format() instead of logging 'skipping' and merging them
anyway.
2026-08-21 21:41:51 +08:00