379 Commits

Author SHA1 Message Date
YueGuobin
fd4ac4460d
feat: migrate builtin EthernetSwitch from Dynamips ethsw to ubridge brctl
Replace the builtin EthernetSwitch stub with a Linux kernel bridge backed
by uBridge's brctl module. Each switch node creates one kernel bridge
(gns3br{N}) with VLAN filtering; each port is a persistent TAP enslaved to
the bridge, relayed by a per-port uBridge bridge (nio_tap <-> nio_udp).

- Access/dot1q/qinq port modes translated to brctl vlan primitives
- Compute router repointed from Dynamips to Builtin manager
- Tests updated: 21 router-level tests + 215 surrounding tests pass
- Real-kernel e2e verified: access 100 PVID untagged, dot1q trunk
  VIDs 1-4094 + native 1, qinq 802.1ad proto + 200 PVID

Ethertype 0x9100/0x9200 handling and port-count guard relaxation are
deferred pending resolution.
2026-07-18 00:21:19 +08:00
YueGuobin
37867a1e44
fix(mcp): remove unreliable node_reload / node_reload_all tools
For Docker nodes, reload bottoms out as a raw POST /containers/{id}/restart
to the Docker daemon, bypassing GNS3's start/stop lifecycle (uBridge
re-attach, console servers, NIC setup). The container restarts at the
Docker level but GNS3's plumbing goes out of sync, and the daemon call can
block up to the controller's 240s timeout — manifesting as "reload hangs /
no response". stop+start runs the full lifecycle and is reliable.

Remove the node_reload and node_reload_all MCP tools (tool functions,
handlers, NODE_TOOLS entry, tests, docs). The underlying REST endpoints
(POST /nodes/{id}/reload, POST /nodes/reload) are kept for native API
users. MCP callers should use node_stop + node_start (partial) or
close/open project (full restart) instead.
2026-07-16 13:27:21 +08:00
Jeremy Grossmann
013ca442f4
Merge pull request #2816 from yueguobin/feature/traffic-insight-markers
feat: add traffic-insight markers (ubridge mark filter integration)
2026-07-15 22:12:11 +02:00
YueGuobin
884c40038c
docs(mcp): note --no-close for node_console websocat usage
A heredoc (<<<) closes stdin at once, so websocat dropped the WebSocket
before the device's reply arrived. Add --no-close to the node_console usage
examples and the returned command field so the connection stays open while
output is read.
2026-07-16 00:51:45 +08:00
YueGuobin
d4389b7068
fix(rbac): guard None current_user on websocket auth failure
get_current_active_user_from_websocket returns None after closing the socket
on an auth failure (revoked token, bad credentials, inactive user).
has_privilege_on_websocket dereferenced current_user.is_superadmin without a
None check, so any websocket auth failure surfaced as an AttributeError
traceback instead of a clean close. Bail out early when current_user is None,
mirroring the guard already present in ws_console.
2026-07-16 00:51:45 +08:00
YueGuobin
3d6d9a3396
fix(mcp): thread token_version into console/download token minting
Token revocation is a strict version check (token_data.token_version !=
user.token_version). create_access_token defaults token_version to 0, so the
short-lived JWTs minted for the console WebSocket URL (nodes) and the download
URLs (symbols, links) carried ver=0. Any user who had logged out at least once
(token_version >= 1, e.g. the default admin) got tokens rejected as "revoked"
on first use.

e433991cf fixed this in _resolve_token's API-key branch but missed these three
independent minting sites. Now resolve token_version during _resolve_token (the
JWT branch decodes it, the API-key branch reads user.token_version), carry it
through gns3_ctx, and pass it at every minting call.
2026-07-16 00:51:45 +08:00
YueGuobin
89e11d265f
fix(mcp): pass user.token_version when generating temp JWT from API key
_resolve_token generated a temp JWT with a hardcoded ver=0 after
validating the API key. Users who had logged out at least once
(token_version >= 1) would hit "Token has been revoked" 401 on
every MCP tool call, because the REST auth chain rejects ver=0
when the user's token_version no longer matches.

Fix: pass the user's actual token_version to create_access_token
so the temp JWT carries the correct ver claim.
2026-07-16 00:51:45 +08:00
YueGuobin
27356da62d
fix(marker): set nio.markers on iou/dynamips/cloud nio update
The PUT /nio (update) endpoints for iou, dynamips and cloud copied nio.filters from the request body but never copied nio.markers. So adding a marker (e.g. a project-level global definition) to an already-created link of these node types left nio.markers empty: no mark filter was installed on uBridge and no marker.match signal was emitted. vpcs/qemu/docker already set it; mirror them (getattr for the Union NIO types).
2026-07-16 00:39:52 +08:00
YueGuobin
b21780605e
fix(marker): reject reserved global-prefix names only on create
The global-prefix reservation belongs on the create path, where it keeps
user-chosen names disjoint from inherited global-{name} markers. It was
implemented as a field_validator on MarkerCreate / MarkerDefinitionCreate,
which also back the PUT update bodies — so updating an inherited marker
while echoing its name in the body tripped the validator with a generic
422 before the controller could return the actionable 409 ("inherited,
use the marker-definitions API").

Drop the schema validators and enforce the prefix in the two create routes
(ControllerError -> 409). PUT no longer validates the body name (ignored
anyway — the target is the {name} path param), so editing an inherited
marker now reaches the controller's inheritance guard and returns the
clear 409. The name-format regex stays on the schema (still 422).
2026-07-16 00:39:52 +08:00
YueGuobin
98bcd5eddd
feat(marker): add highlight_duration render hint
Per-marker UI hint (milliseconds, ge=1) for how long the Web UI keeps a
marker highlighted after a match. Mirrors color: stored on the link,
persisted in the topology, inherited via project-level definitions, and
never sent to uBridge. Omitted = null = frontend uses its own default.

The default is intentionally NOT set in the schema: MarkerCreate also
backs PUT updates, so a schema default would make every partial update
silently reset the value. None means "not provided" (keep existing on
update / frontend decides on create).
2026-07-16 00:39:52 +08:00
YueGuobin
31991fe359
feat(marker): add project-level marker definition inheritance
Project-level marker definitions fan out to every link (existing and new).
A definition is stored once on Project._marker_definitions; when applied to
a link the marker is named "global-{def_name}" — the "global" prefix was
pre-reserved in the schema, so inherited and per-link markers can never
collide, nor will their registry keys.

Key behavior:
- POST /projects/{pid}/marker-definitions → fan out to all existing links
- PUT  /projects/{pid}/marker-definitions/{name} → sync all inherited copies
- DELETE → remove every inherited copy from every link
- New links auto-inherit all active defs (hook in UDPLink.create)
- Per-link DELETE/PUT of a "global-*" marker is rejected (409)
- Inherited markers are NOT persisted in the topology; they are re-created
  from _marker_definitions on project load
- Compute side is untouched — the marker reaches uBridge via the existing
  start_marker→update→NIO→ubridge pipeline

Files:
- controller/project.py — _marker_definitions + CRUD + fanout + topology load
- controller/link.py — Link.inherit_marker() + asdict() filter
- controller/udp_link.py — guards on stop/update + create() inheritance hook
- controller/topology.py — persist marker_definitions in project topology
- schemas/controller/links.py — MarkerDefinitionCreate schema
- api/routes/controller/projects.py — REST endpoints (marker-definitions)
2026-07-16 00:39:52 +08:00
YueGuobin
7e4600b5a1
fix(marker): validate marker name and expose enabled via REST
Add a name pattern constraint to MarkerCreate schema (alphanumeric +
_.-) to prevent injection into uBridge commands, with a matching
defense-in-depth check in _ubridge_add_marker_filter for hand-edited
topology files. Also add the missing "enabled" field to MarkerCreate
and pass it through the PUT route so the documented toggle actually
works.
2026-07-16 00:39:51 +08:00
YueGuobin
a495b5ddf2
fix(marker): uniquify auto-generated marker names per link
Auto-generating `marker-{link.id[:8]}` collided on the second anonymous
marker on the same link (start_marker rejects duplicate names). Append a
short uuid hex suffix so multiple anonymous markers coexist.
2026-07-16 00:39:51 +08:00
YueGuobin
f2360f85fc
feat(marker): add project-level marker aggregation endpoint
Add a read-only `markers` property on Project that flattens every link's
markers into a single dict keyed by "{link_id}/{name}", each entry
carrying the parent link_id and capture-side node_id. Expose it via
GET /projects/{pid}/markers (Project.Audit) so the frontend can fetch all
markers in one round-trip instead of enumerating links first.

Also surface a marker count in project.stats().
2026-07-16 00:39:51 +08:00
YueGuobin
c62b9b0283
refactor(marker): converge to filter single-path model, remove dual-apply endpoints
Markers now follow exactly the same apply pattern as packet filters:
state lives in Link._markers, application goes through NIO
(update() -> PUT /nio -> _ubridge_apply_markers). The former
immediate-apply REST endpoints (/markers/start, /markers/stop on
the compute side) and the per-node start_marker/stop_marker methods
are removed — they were a legacy of the original capture-inspired
design and have been superseded by the NIO flow.

Changes:
- controller/udp_link: start_marker/stop_marker/update_marker now
  set _markers state + call self.update() (mirrors update_filters).
  Removed _marker_capture_nodes runtime dict and its helpers.
- controller/project: _create_link_from_topology_data restores
  _markers directly from persisted data (with BPF validation,
  like filter reload). No long calls start_marker during load.
- compute: _ubridge_apply_markers swallows BPF compile errors
  (warn+skip), matching _ubridge_apply_filters behaviour so a
  single bad expression cannot break link creation / node restart.
- Removed: /markers/start,stop endpoints (6 handlers across
  vpcs/qemu/docker route files), node start_marker/stop_marker
  methods (3 VM files), _ubridge_delete_marker_filter,
  _marker_capture_nodes, MarkerDelete schema.

Net: ~280 lines of dead code removed; marker and packet filter now
share a single, unified apply path via the NIO.
2026-07-16 00:39:51 +08:00
YueGuobin
3f15ac38f6
feat(marker): add optional color field to marker data model
Add a user-chosen hex color (e.g. '#ff5722') to each marker, stored in
Link._markers alongside bpf/tag/enabled. The color:

- Is serialized with the link (asdict) and persisted in the topology, so
  it survives project reload and is consistent across devices.
- Never reaches uBridge (compute ignores it) — it is purely a Web UI
  concern.
- Can be updated independently of the BPF (color-only changes skip the
  ubridge round-trip).

MarkerCreate schema gains color: Optional[str]; start_marker and
update_marker carry it through; REST create/update endpoints pass it.

The marker.match WebSocket event stays lean (no color) — the Web UI maps
filter -> color from the link state it already holds.
2026-07-16 00:39:51 +08:00
YueGuobin
7c90fa9e64
fix(marker): ride markers on NIO so they survive node restart and project reload
Mirror the packet-filter lifecycle: marker specs now live on the NIO
(next to filters), ride in link_data from controller to compute on
every NIO create/update, and are reapplied by _ubridge_apply_markers
in add_ubridge_udp_connection (bridge creation / node restart) and
update_ubridge_udp_connection (NIO update — following the preceding
reset_packet_filters so markers survive filter changes).

Changes:
- NIO / NIOUDP: _markers property + asdict
- schemas/compute/nios.py: UDPNIO.markers field
- base_manager.create_nio: nio.markers from settings
- PUT /nio routes (vpcs/qemu/docker): nio.markers update
- base_node: _ubridge_apply_markers(bridge_name, nio) iterates
  nio.markers, computes pcap path, calls _ubridge_add_marker_filter
  + MarkerManager.register; called after _ubridge_apply_filters
- controller udp_link: _get_node_markers + _markers_for_node (route
  by capture_node_id); markers in create() and update() link_data
- /markers/start,stop endpoints: mirror spec onto nio.markers so
  the marker survives a subsequent node stop/start without a PUT
- tests: add markers field to NIO data expectations

This covers:
- Node restart: NIO persists, add_ubridge_udp_connection re-applies
- Filter update: reset wipes markers, _ubridge_apply_markers re-adds
- Project reload: create() carries markers in link_data → create_nio
- Immediate create: endpoint sets nio.markers immediately
2026-07-16 00:39:51 +08:00
YueGuobin
d2e9823e6c
feat: implement traffic insight marker backend (ubridge mark filter)
Add compute-side marker subsystem that receives ubridge UDP MARK signals
and forwards them as project-scoped notifications to the web UI for
real-time traffic coloring. Matched packets are always saved to per-link
pcaps for future replay.

Key components:
- gns3server/compute/marker/: MarkerManager (singleton, UDP listener +
  O(1) registry keyed by (node_id, filter_name)) + MarkerListener
  (DatagramProtocol parsing MARK lines per ubridge integration contract)
- gns3server/compute/base_node.py: marker sink/node config at ubridge
  startup; shared _ubridge_add_marker_filter / _ubridge_delete_marker_filter
- Per-node start_marker/stop_marker: VPCS (VPCS-{id}), QEMU
  (QEMU-{id}-{adapter}), Docker (bridge{adapter})
- Compute REST /markers/start + /markers/stop (vpcs/qemu/docker route files)
- Controller Link._markers state + UDPLink.start_marker/stop_marker/
  update_marker (mirror capture pattern: BPF validation, _choose_capture_side,
  node.post forwarding, topology persistence)
- Controller REST GET/POST/DELETE/PUT /v3/projects/{p}/links/{l}/markers
- Config: marker_listen_host / marker_listen_port in ServerSettings
- Signal routing: creation-time registry O(1) lookup, no node-table scan;
  project-scoped WS stream (not global); event payload always carries
  project_id for frontend scoping

Tests: 14 unit tests (registry, listener parsing, UDP round-trip);
562 existing tests pass with zero regressions.
2026-07-16 00:39:51 +08:00
YueGuobin
292b60efaa
Make AI features (AI Copilot + MCP) optional via [ai-features] extra
- Move fastmcp from core requirements.txt to mcp-requirements.txt
- Add MCP_AVAILABLE feature flag in agent/__init__.py (graceful degradation)
- Guard MCP imports/registration in server.py and tasks.py
- Replace ai-copilot/mcp/ai-support extras with single ai-features extra
- Add stub MCP routes returning 501 when MCP is not installed
- Add gns3server-uninstall-ai-features CLI command
- Remove old gns3server-uninstall-ai-copilot command
- Update all error messages and docs to reference ai-features

Closes #2794
2026-06-27 10:14:43 +08:00
YueGuobin
0e8d0cb87b
Add stateless JWT refresh token mechanism
- New config: Controller.jwt_refresh_token_expire_minutes (default 30 days)
- New endpoint: POST /v3/access/users/refresh (public, unauthenticated)
- Login/authenticate responses now include refresh_token
- AuthService: _create_token helper, create_refresh_token, get_token_data
  now parses type claim (token_use) for token classification
- Security: refresh tokens rejected on HTTP + WebSocket access paths;
  /refresh strictly requires type=='refresh'
- Logout works for free via existing token_version mechanism
- Tests: 9 new TestRefreshToken cases, all passing; 34 existing tests
  still pass (no regressions)
2026-06-23 22:39:11 +08:00
YueGuobin
ead9747e1b
Fix appliance_install: add version parameter 2026-06-17 23:03:28 +08:00
YueGuobin
0d21f294f6
Document canvas coordinate system in node_create x/y params 2026-06-16 23:16:17 +08:00
YueGuobin
00647d092c
Update docstring: batch concurrency from 10 to 100 2026-06-16 23:05:17 +08:00
YueGuobin
10ff5b2283
Add list type check before nodes[0] access in _normalize_link_nodes 2026-06-16 23:00:47 +08:00
YueGuobin
0d45bfbae5
Add fields type validation in create handlers 2026-06-16 22:58:36 +08:00
YueGuobin
59e5f8dd2f
Fix review issues: key_prefix length, count validation, WAL log, timeout comment, pointless temp var 2026-06-16 22:50:22 +08:00
YueGuobin
533d3d5b71
Fix: pass name from TemplateUsage to add_node_from_template
The controller endpoint received name via TemplateUsage but never
passed it to add_node_from_template, causing the name parameter to
be silently ignored. name is optional - if omitted the template's
default_name_format is used.
2026-06-16 22:43:46 +08:00
YueGuobin
68567118f1
Remove unused imports (logging, log, select) 2026-06-16 22:25:07 +08:00
Jeremy Grossmann
7d5d4767bf
Remove FIXME comment about middleware in server.py
Removed commented out FIXME note regarding middleware issues.
2026-06-16 12:20:55 +02:00
YueGuobin
f7abb62603
Rename device_command_run_handler → device_show_run_handler to match tool name 2026-06-16 12:44:12 +08:00
YueGuobin
fbb8924a8e
Rename device_command_run → device_show_run for clarity
The name device_command_run was ambiguous and easily confused with
device_config_send. device_show_run makes the read-only intent explicit,
forming a clear pair: config_send (write) vs show_run (read).
2026-06-16 12:27:35 +08:00
YueGuobin
e627481ec2
Fix device_command_run KeyError('commands'): tool desc said show_commands but backend expects commands
- Tool description and param schema now use 'commands' (matching ExecuteMultipleDeviceCommands)
- Updated error message in handler
- Clarified in docstring that this is read-only; use device_config_send for config changes
2026-06-16 12:23:15 +08:00
YueGuobin
ec58bea24b
Fix template_list return type annotation to match _run_handler_sync envelope
template_list was annotated as dict[str, Any] but _run_handler_sync always
returns a list envelope ([{"type":"text",...}]). Other tools use
list[dict[str, Any]] consistently. This caused a Pydantic dict_type
validation error on the client.
2026-06-16 12:08:07 +08:00
YueGuobin
74192f454f
Increase BATCH_MAX_WORKERS and Pool concurrency from 20 to 100 2026-06-16 00:49:08 +08:00
YueGuobin
286b2407cd
Remove final timing artifact in projects.py 2026-06-16 00:44:54 +08:00
YueGuobin
bd15c8d019
Remove remaining dead timing variables and imports 2026-06-16 00:40:03 +08:00
YueGuobin
ccb629f48f
Clean up all timing/debug logs
Remove all [MCP-TIMING] and [CTRL-TIMING] log lines, timing middleware,
and related import time statements across 11 files.
2026-06-16 00:34:11 +08:00
YueGuobin
7d8ab399da
Fix: add missing UUID imports 2026-06-16 00:24:37 +08:00
YueGuobin
d6b337bc3a
Generate fresh JWT on API key auth instead of returning raw key
When MCP client authenticates with API key, _resolve_token now
returns a newly generated JWT. This JWT is stored in the ContextVar
and used by all subsequent tool handler API calls - zero extra bcrypt.
2026-06-16 00:09:36 +08:00
YueGuobin
f522c947bc
Optimize API key auth: O(1) lookup via UUID-embedded key format
New format: gns3_<api_key_id>_<random_secret>
- Auth extracts api_key_id from token → single DB query by UUID → one bcrypt
- No more scanning all keys (was O(n) with bcrypt per key)
- bcrypt.checkpw offloaded to thread pool to prevent event loop blocking
- Legacy gns3_<random> format removed (compatibility break)
2026-06-15 23:56:36 +08:00
YueGuobin
796a2e6ca8
Fix: offload bcrypt.checkpw to thread pool to prevent blocking event loop
bcrypt.checkpw is CPU-bound (~1.3s per call) and was running
synchronously inside the async event loop, blocking ALL concurrent
requests. With 5 API keys and 10 concurrent requests, this caused
~13s delay before any handler could start.
2026-06-15 23:49:28 +08:00
YueGuobin
04f0657661
Add timing to API key auth path and log api_keys count 2026-06-15 23:45:26 +08:00
YueGuobin
5502b8f8e5
Add timing logs to auth dependency chain to identify 6s pre-handler delay 2026-06-15 23:35:12 +08:00
YueGuobin
fa9aa5a9cc
Add timing middleware to log slow requests (>1s) with [CTRL-TIMING] prefix 2026-06-15 23:13:36 +08:00
YueGuobin
ea4bb2c1fb
Add [CTRL-TIMING] logs to controller create_node flow
Timing logs cover:
- create_node_from_template (entry, get_template, add_node, total)
- add_node_from_template (entry to done)
- _create_node (project_setup, node.create, total)
- Node.create (compute_post timing)
- compute._session.request (actual HTTP to compute)
2026-06-15 23:04:01 +08:00
YueGuobin
e5797c3da0
Fix: pass template_id to batch mode handler so top-level template_id works as default 2026-06-15 22:57:40 +08:00
YueGuobin
cef6dc6bd2
Add detailed timing logs to MCP node creation and HTTP client
Logs with [MCP-TIMING] prefix at:
- create_node_handler entry, setup, http_call start/end, total
- http_call entry, auth, response
- _authenticate_v3 entry, done, fail
2026-06-15 22:51:43 +08:00
YueGuobin
58ca6b1dd4
Add fields filter to template_list tool with description for AI 2026-06-15 22:11:14 +08:00
YueGuobin
b79fb89866
Fix: add missing _filter_link_response function
Previously the function was referenced in create_link_handler
but never defined, causing NameError at runtime.
2026-06-15 21:51:17 +08:00
YueGuobin
1bb010d274
Pass name parameter through to controller API when creating node from template 2026-06-15 21:50:14 +08:00