Commit Graph
384 Commits
Author SHA1 Message Date
YueGuobin 17020b7f7b iou: forward data_link_type on capture start so serial pcaps use the correct linktype
The IOU compute capture/start route received data_link_type in the NodeCapture body but dropped it when calling node.start_capture(), so iou_vm.start_capture always defaulted to DLT_EN10MB -- every IOU serial capture (Cisco HDLC / PPP / Frame Relay) was written as an Ethernet pcap. Dynamips, cloud and the L2 switches already forward this value; IOU was the only serial-capable node that omitted it. One-argument fix: the node method already accepts and forwards data_link_type to 'iol_bridge start_capture', so only the route was missing it.

Verified: IOU serial captures now produce the expected linktype (C_HDLC / PPP_SERIAL / FRELAY) instead of Ethernet.
2026-08-07 00:35:39 +08:00
YueGuobin 309d388b0b Add MCP tools for traffic-insight marker feature
Add 2 new MCP tools to expose the marker (traffic-insight) REST API:

- link_marker: per-link marker CRUD (create/update/delete)
  POST/PUT/DELETE /projects/{pid}/links/{lid}/markers
- marker_definition: project-level marker definition CRUD (create/update/delete/list)
  POST/PUT/DELETE/GET /projects/{pid}/marker-definitions
  Create auto-fans out global-{name} to every link

Read operations use existing link_get (returns markers dict).
2026-07-26 13:01:06 +08:00
YueGuobin a8c6546c44 Improve MCP tool descriptions for link_update, device_show_run, and link_reset
- link_update: add bidirectional filter effect note, packet loss formula
  (packet_loss [50] ≈ 75% observed), filter clearing syntax (filters: {}),
  and ARP-also-filtered warning with static ARP recommendation
- device_show_run: add prerequisite section — device_type:<type> tag
  required, Docker/Linux nodes unsupported (use node_console)
- link_reset: clarify that filter state machines (e.g. frequency_drop
  counters) restart while filter configuration is preserved
2026-07-26 12:50:14 +08:00
Cristi 2abc7e09a3 Adds controller and compute API support for explicitly pulling or updating Docker images. 2026-07-24 16:40:20 +03:00
YueGuobin ddf7d4ca60 fix(controller): refresh cloud/nat node interfaces from compute on GET
Cloud and NAT nodes need live host network interface data. Previously,
GET /projects/{project_id}/nodes/{node_id} returned cached properties
from creation time, so newly added host interfaces (e.g. kernel bridges
created by EthernetSwitch nodes) were invisible until the node was
deleted and recreated.

Now the controller fetches fresh data from the compute node before
returning the response, so host interface changes are reflected
immediately. Falls back to cached data if compute is unreachable.
2026-07-19 17:41:37 +08:00
YueGuobin fd4ac4460d feat: migrate builtin EthernetSwitch from Dynamips ethsw to ubridge brctl
Replace the builtin EthernetSwitch stub with a Linux kernel bridge backed
by uBridge's brctl module. Each switch node creates one kernel bridge
(gns3br{N}) with VLAN filtering; each port is a persistent TAP enslaved to
the bridge, relayed by a per-port uBridge bridge (nio_tap <-> nio_udp).

- Access/dot1q/qinq port modes translated to brctl vlan primitives
- Compute router repointed from Dynamips to Builtin manager
- Tests updated: 21 router-level tests + 215 surrounding tests pass
- Real-kernel e2e verified: access 100 PVID untagged, dot1q trunk
  VIDs 1-4094 + native 1, qinq 802.1ad proto + 200 PVID

Ethertype 0x9100/0x9200 handling and port-count guard relaxation are
deferred pending resolution.
2026-07-18 00:21:19 +08:00
YueGuobin 37867a1e44 fix(mcp): remove unreliable node_reload / node_reload_all tools
For Docker nodes, reload bottoms out as a raw POST /containers/{id}/restart
to the Docker daemon, bypassing GNS3's start/stop lifecycle (uBridge
re-attach, console servers, NIC setup). The container restarts at the
Docker level but GNS3's plumbing goes out of sync, and the daemon call can
block up to the controller's 240s timeout — manifesting as "reload hangs /
no response". stop+start runs the full lifecycle and is reliable.

Remove the node_reload and node_reload_all MCP tools (tool functions,
handlers, NODE_TOOLS entry, tests, docs). The underlying REST endpoints
(POST /nodes/{id}/reload, POST /nodes/reload) are kept for native API
users. MCP callers should use node_stop + node_start (partial) or
close/open project (full restart) instead.
2026-07-16 13:27:21 +08:00
Jeremy Grossmann 013ca442f4 Merge pull request #2816 from yueguobin/feature/traffic-insight-markers
feat: add traffic-insight markers (ubridge mark filter integration)
2026-07-15 22:12:11 +02:00
YueGuobin 884c40038c docs(mcp): note --no-close for node_console websocat usage
A heredoc (<<<) closes stdin at once, so websocat dropped the WebSocket
before the device's reply arrived. Add --no-close to the node_console usage
examples and the returned command field so the connection stays open while
output is read.
2026-07-16 00:51:45 +08:00
YueGuobin d4389b7068 fix(rbac): guard None current_user on websocket auth failure
get_current_active_user_from_websocket returns None after closing the socket
on an auth failure (revoked token, bad credentials, inactive user).
has_privilege_on_websocket dereferenced current_user.is_superadmin without a
None check, so any websocket auth failure surfaced as an AttributeError
traceback instead of a clean close. Bail out early when current_user is None,
mirroring the guard already present in ws_console.
2026-07-16 00:51:45 +08:00
YueGuobin 3d6d9a3396 fix(mcp): thread token_version into console/download token minting
Token revocation is a strict version check (token_data.token_version !=
user.token_version). create_access_token defaults token_version to 0, so the
short-lived JWTs minted for the console WebSocket URL (nodes) and the download
URLs (symbols, links) carried ver=0. Any user who had logged out at least once
(token_version >= 1, e.g. the default admin) got tokens rejected as "revoked"
on first use.

e433991cf fixed this in _resolve_token's API-key branch but missed these three
independent minting sites. Now resolve token_version during _resolve_token (the
JWT branch decodes it, the API-key branch reads user.token_version), carry it
through gns3_ctx, and pass it at every minting call.
2026-07-16 00:51:45 +08:00
YueGuobin 89e11d265f fix(mcp): pass user.token_version when generating temp JWT from API key
_resolve_token generated a temp JWT with a hardcoded ver=0 after
validating the API key. Users who had logged out at least once
(token_version >= 1) would hit "Token has been revoked" 401 on
every MCP tool call, because the REST auth chain rejects ver=0
when the user's token_version no longer matches.

Fix: pass the user's actual token_version to create_access_token
so the temp JWT carries the correct ver claim.
2026-07-16 00:51:45 +08:00
YueGuobin 27356da62d fix(marker): set nio.markers on iou/dynamips/cloud nio update
The PUT /nio (update) endpoints for iou, dynamips and cloud copied nio.filters from the request body but never copied nio.markers. So adding a marker (e.g. a project-level global definition) to an already-created link of these node types left nio.markers empty: no mark filter was installed on uBridge and no marker.match signal was emitted. vpcs/qemu/docker already set it; mirror them (getattr for the Union NIO types).
2026-07-16 00:39:52 +08:00
YueGuobin b21780605e fix(marker): reject reserved global-prefix names only on create
The global-prefix reservation belongs on the create path, where it keeps
user-chosen names disjoint from inherited global-{name} markers. It was
implemented as a field_validator on MarkerCreate / MarkerDefinitionCreate,
which also back the PUT update bodies — so updating an inherited marker
while echoing its name in the body tripped the validator with a generic
422 before the controller could return the actionable 409 ("inherited,
use the marker-definitions API").

Drop the schema validators and enforce the prefix in the two create routes
(ControllerError -> 409). PUT no longer validates the body name (ignored
anyway — the target is the {name} path param), so editing an inherited
marker now reaches the controller's inheritance guard and returns the
clear 409. The name-format regex stays on the schema (still 422).
2026-07-16 00:39:52 +08:00
YueGuobin 98bcd5eddd feat(marker): add highlight_duration render hint
Per-marker UI hint (milliseconds, ge=1) for how long the Web UI keeps a
marker highlighted after a match. Mirrors color: stored on the link,
persisted in the topology, inherited via project-level definitions, and
never sent to uBridge. Omitted = null = frontend uses its own default.

The default is intentionally NOT set in the schema: MarkerCreate also
backs PUT updates, so a schema default would make every partial update
silently reset the value. None means "not provided" (keep existing on
update / frontend decides on create).
2026-07-16 00:39:52 +08:00
YueGuobin 31991fe359 feat(marker): add project-level marker definition inheritance
Project-level marker definitions fan out to every link (existing and new).
A definition is stored once on Project._marker_definitions; when applied to
a link the marker is named "global-{def_name}" — the "global" prefix was
pre-reserved in the schema, so inherited and per-link markers can never
collide, nor will their registry keys.

Key behavior:
- POST /projects/{pid}/marker-definitions → fan out to all existing links
- PUT  /projects/{pid}/marker-definitions/{name} → sync all inherited copies
- DELETE → remove every inherited copy from every link
- New links auto-inherit all active defs (hook in UDPLink.create)
- Per-link DELETE/PUT of a "global-*" marker is rejected (409)
- Inherited markers are NOT persisted in the topology; they are re-created
  from _marker_definitions on project load
- Compute side is untouched — the marker reaches uBridge via the existing
  start_marker→update→NIO→ubridge pipeline

Files:
- controller/project.py — _marker_definitions + CRUD + fanout + topology load
- controller/link.py — Link.inherit_marker() + asdict() filter
- controller/udp_link.py — guards on stop/update + create() inheritance hook
- controller/topology.py — persist marker_definitions in project topology
- schemas/controller/links.py — MarkerDefinitionCreate schema
- api/routes/controller/projects.py — REST endpoints (marker-definitions)
2026-07-16 00:39:52 +08:00
YueGuobin 7e4600b5a1 fix(marker): validate marker name and expose enabled via REST
Add a name pattern constraint to MarkerCreate schema (alphanumeric +
_.-) to prevent injection into uBridge commands, with a matching
defense-in-depth check in _ubridge_add_marker_filter for hand-edited
topology files. Also add the missing "enabled" field to MarkerCreate
and pass it through the PUT route so the documented toggle actually
works.
2026-07-16 00:39:51 +08:00
YueGuobin a495b5ddf2 fix(marker): uniquify auto-generated marker names per link
Auto-generating `marker-{link.id[:8]}` collided on the second anonymous
marker on the same link (start_marker rejects duplicate names). Append a
short uuid hex suffix so multiple anonymous markers coexist.
2026-07-16 00:39:51 +08:00
YueGuobin f2360f85fc feat(marker): add project-level marker aggregation endpoint
Add a read-only `markers` property on Project that flattens every link's
markers into a single dict keyed by "{link_id}/{name}", each entry
carrying the parent link_id and capture-side node_id. Expose it via
GET /projects/{pid}/markers (Project.Audit) so the frontend can fetch all
markers in one round-trip instead of enumerating links first.

Also surface a marker count in project.stats().
2026-07-16 00:39:51 +08:00
YueGuobin c62b9b0283 refactor(marker): converge to filter single-path model, remove dual-apply endpoints
Markers now follow exactly the same apply pattern as packet filters:
state lives in Link._markers, application goes through NIO
(update() -> PUT /nio -> _ubridge_apply_markers). The former
immediate-apply REST endpoints (/markers/start, /markers/stop on
the compute side) and the per-node start_marker/stop_marker methods
are removed — they were a legacy of the original capture-inspired
design and have been superseded by the NIO flow.

Changes:
- controller/udp_link: start_marker/stop_marker/update_marker now
  set _markers state + call self.update() (mirrors update_filters).
  Removed _marker_capture_nodes runtime dict and its helpers.
- controller/project: _create_link_from_topology_data restores
  _markers directly from persisted data (with BPF validation,
  like filter reload). No long calls start_marker during load.
- compute: _ubridge_apply_markers swallows BPF compile errors
  (warn+skip), matching _ubridge_apply_filters behaviour so a
  single bad expression cannot break link creation / node restart.
- Removed: /markers/start,stop endpoints (6 handlers across
  vpcs/qemu/docker route files), node start_marker/stop_marker
  methods (3 VM files), _ubridge_delete_marker_filter,
  _marker_capture_nodes, MarkerDelete schema.

Net: ~280 lines of dead code removed; marker and packet filter now
share a single, unified apply path via the NIO.
2026-07-16 00:39:51 +08:00
YueGuobin 3f15ac38f6 feat(marker): add optional color field to marker data model
Add a user-chosen hex color (e.g. '#ff5722') to each marker, stored in
Link._markers alongside bpf/tag/enabled. The color:

- Is serialized with the link (asdict) and persisted in the topology, so
  it survives project reload and is consistent across devices.
- Never reaches uBridge (compute ignores it) — it is purely a Web UI
  concern.
- Can be updated independently of the BPF (color-only changes skip the
  ubridge round-trip).

MarkerCreate schema gains color: Optional[str]; start_marker and
update_marker carry it through; REST create/update endpoints pass it.

The marker.match WebSocket event stays lean (no color) — the Web UI maps
filter -> color from the link state it already holds.
2026-07-16 00:39:51 +08:00
YueGuobin 7c90fa9e64 fix(marker): ride markers on NIO so they survive node restart and project reload
Mirror the packet-filter lifecycle: marker specs now live on the NIO
(next to filters), ride in link_data from controller to compute on
every NIO create/update, and are reapplied by _ubridge_apply_markers
in add_ubridge_udp_connection (bridge creation / node restart) and
update_ubridge_udp_connection (NIO update — following the preceding
reset_packet_filters so markers survive filter changes).

Changes:
- NIO / NIOUDP: _markers property + asdict
- schemas/compute/nios.py: UDPNIO.markers field
- base_manager.create_nio: nio.markers from settings
- PUT /nio routes (vpcs/qemu/docker): nio.markers update
- base_node: _ubridge_apply_markers(bridge_name, nio) iterates
  nio.markers, computes pcap path, calls _ubridge_add_marker_filter
  + MarkerManager.register; called after _ubridge_apply_filters
- controller udp_link: _get_node_markers + _markers_for_node (route
  by capture_node_id); markers in create() and update() link_data
- /markers/start,stop endpoints: mirror spec onto nio.markers so
  the marker survives a subsequent node stop/start without a PUT
- tests: add markers field to NIO data expectations

This covers:
- Node restart: NIO persists, add_ubridge_udp_connection re-applies
- Filter update: reset wipes markers, _ubridge_apply_markers re-adds
- Project reload: create() carries markers in link_data → create_nio
- Immediate create: endpoint sets nio.markers immediately
2026-07-16 00:39:51 +08:00
YueGuobin d2e9823e6c feat: implement traffic insight marker backend (ubridge mark filter)
Add compute-side marker subsystem that receives ubridge UDP MARK signals
and forwards them as project-scoped notifications to the web UI for
real-time traffic coloring. Matched packets are always saved to per-link
pcaps for future replay.

Key components:
- gns3server/compute/marker/: MarkerManager (singleton, UDP listener +
  O(1) registry keyed by (node_id, filter_name)) + MarkerListener
  (DatagramProtocol parsing MARK lines per ubridge integration contract)
- gns3server/compute/base_node.py: marker sink/node config at ubridge
  startup; shared _ubridge_add_marker_filter / _ubridge_delete_marker_filter
- Per-node start_marker/stop_marker: VPCS (VPCS-{id}), QEMU
  (QEMU-{id}-{adapter}), Docker (bridge{adapter})
- Compute REST /markers/start + /markers/stop (vpcs/qemu/docker route files)
- Controller Link._markers state + UDPLink.start_marker/stop_marker/
  update_marker (mirror capture pattern: BPF validation, _choose_capture_side,
  node.post forwarding, topology persistence)
- Controller REST GET/POST/DELETE/PUT /v3/projects/{p}/links/{l}/markers
- Config: marker_listen_host / marker_listen_port in ServerSettings
- Signal routing: creation-time registry O(1) lookup, no node-table scan;
  project-scoped WS stream (not global); event payload always carries
  project_id for frontend scoping

Tests: 14 unit tests (registry, listener parsing, UDP round-trip);
562 existing tests pass with zero regressions.
2026-07-16 00:39:51 +08:00
YueGuobin 292b60efaa Make AI features (AI Copilot + MCP) optional via [ai-features] extra
- Move fastmcp from core requirements.txt to mcp-requirements.txt
- Add MCP_AVAILABLE feature flag in agent/__init__.py (graceful degradation)
- Guard MCP imports/registration in server.py and tasks.py
- Replace ai-copilot/mcp/ai-support extras with single ai-features extra
- Add stub MCP routes returning 501 when MCP is not installed
- Add gns3server-uninstall-ai-features CLI command
- Remove old gns3server-uninstall-ai-copilot command
- Update all error messages and docs to reference ai-features

Closes #2794
2026-06-27 10:14:43 +08:00
YueGuobin 0e8d0cb87b Add stateless JWT refresh token mechanism
- New config: Controller.jwt_refresh_token_expire_minutes (default 30 days)
- New endpoint: POST /v3/access/users/refresh (public, unauthenticated)
- Login/authenticate responses now include refresh_token
- AuthService: _create_token helper, create_refresh_token, get_token_data
  now parses type claim (token_use) for token classification
- Security: refresh tokens rejected on HTTP + WebSocket access paths;
  /refresh strictly requires type=='refresh'
- Logout works for free via existing token_version mechanism
- Tests: 9 new TestRefreshToken cases, all passing; 34 existing tests
  still pass (no regressions)
2026-06-23 22:39:11 +08:00
YueGuobin ead9747e1b Fix appliance_install: add version parameter 2026-06-17 23:03:28 +08:00
YueGuobin 0d21f294f6 Document canvas coordinate system in node_create x/y params 2026-06-16 23:16:17 +08:00
YueGuobin 00647d092c Update docstring: batch concurrency from 10 to 100 2026-06-16 23:05:17 +08:00
YueGuobin 10ff5b2283 Add list type check before nodes[0] access in _normalize_link_nodes 2026-06-16 23:00:47 +08:00
YueGuobin 0d45bfbae5 Add fields type validation in create handlers 2026-06-16 22:58:36 +08:00
YueGuobin 59e5f8dd2f Fix review issues: key_prefix length, count validation, WAL log, timeout comment, pointless temp var 2026-06-16 22:50:22 +08:00
YueGuobin 533d3d5b71 Fix: pass name from TemplateUsage to add_node_from_template
The controller endpoint received name via TemplateUsage but never
passed it to add_node_from_template, causing the name parameter to
be silently ignored. name is optional - if omitted the template's
default_name_format is used.
2026-06-16 22:43:46 +08:00
YueGuobin 68567118f1 Remove unused imports (logging, log, select) 2026-06-16 22:25:07 +08:00
Jeremy Grossmann 7d5d4767bf Remove FIXME comment about middleware in server.py
Removed commented out FIXME note regarding middleware issues.
2026-06-16 12:20:55 +02:00
YueGuobin f7abb62603 Rename device_command_run_handler → device_show_run_handler to match tool name 2026-06-16 12:44:12 +08:00
YueGuobin fbb8924a8e Rename device_command_run → device_show_run for clarity
The name device_command_run was ambiguous and easily confused with
device_config_send. device_show_run makes the read-only intent explicit,
forming a clear pair: config_send (write) vs show_run (read).
2026-06-16 12:27:35 +08:00
YueGuobin e627481ec2 Fix device_command_run KeyError('commands'): tool desc said show_commands but backend expects commands
- Tool description and param schema now use 'commands' (matching ExecuteMultipleDeviceCommands)
- Updated error message in handler
- Clarified in docstring that this is read-only; use device_config_send for config changes
2026-06-16 12:23:15 +08:00
YueGuobin ec58bea24b Fix template_list return type annotation to match _run_handler_sync envelope
template_list was annotated as dict[str, Any] but _run_handler_sync always
returns a list envelope ([{"type":"text",...}]). Other tools use
list[dict[str, Any]] consistently. This caused a Pydantic dict_type
validation error on the client.
2026-06-16 12:08:07 +08:00
YueGuobin 74192f454f Increase BATCH_MAX_WORKERS and Pool concurrency from 20 to 100 2026-06-16 00:49:08 +08:00
YueGuobin 286b2407cd Remove final timing artifact in projects.py 2026-06-16 00:44:54 +08:00
YueGuobin bd15c8d019 Remove remaining dead timing variables and imports 2026-06-16 00:40:03 +08:00
YueGuobin ccb629f48f Clean up all timing/debug logs
Remove all [MCP-TIMING] and [CTRL-TIMING] log lines, timing middleware,
and related import time statements across 11 files.
2026-06-16 00:34:11 +08:00
YueGuobin 7d8ab399da Fix: add missing UUID imports 2026-06-16 00:24:37 +08:00
YueGuobin d6b337bc3a Generate fresh JWT on API key auth instead of returning raw key
When MCP client authenticates with API key, _resolve_token now
returns a newly generated JWT. This JWT is stored in the ContextVar
and used by all subsequent tool handler API calls - zero extra bcrypt.
2026-06-16 00:09:36 +08:00
YueGuobin f522c947bc Optimize API key auth: O(1) lookup via UUID-embedded key format
New format: gns3_<api_key_id>_<random_secret>
- Auth extracts api_key_id from token → single DB query by UUID → one bcrypt
- No more scanning all keys (was O(n) with bcrypt per key)
- bcrypt.checkpw offloaded to thread pool to prevent event loop blocking
- Legacy gns3_<random> format removed (compatibility break)
2026-06-15 23:56:36 +08:00
YueGuobin 796a2e6ca8 Fix: offload bcrypt.checkpw to thread pool to prevent blocking event loop
bcrypt.checkpw is CPU-bound (~1.3s per call) and was running
synchronously inside the async event loop, blocking ALL concurrent
requests. With 5 API keys and 10 concurrent requests, this caused
~13s delay before any handler could start.
2026-06-15 23:49:28 +08:00
YueGuobin 04f0657661 Add timing to API key auth path and log api_keys count 2026-06-15 23:45:26 +08:00
YueGuobin 5502b8f8e5 Add timing logs to auth dependency chain to identify 6s pre-handler delay 2026-06-15 23:35:12 +08:00
YueGuobin fa9aa5a9cc Add timing middleware to log slow requests (>1s) with [CTRL-TIMING] prefix 2026-06-15 23:13:36 +08:00
YueGuobin ea4bb2c1fb Add [CTRL-TIMING] logs to controller create_node flow
Timing logs cover:
- create_node_from_template (entry, get_template, add_node, total)
- add_node_from_template (entry to done)
- _create_node (project_setup, node.create, total)
- Node.create (compute_post timing)
- compute._session.request (actual HTTP to compute)
2026-06-15 23:04:01 +08:00