381 Commits

Author SHA1 Message Date
Cristi
2abc7e09a3 Adds controller and compute API support for explicitly pulling or updating Docker images. 2026-07-24 16:40:20 +03:00
YueGuobin
ddf7d4ca60
fix(controller): refresh cloud/nat node interfaces from compute on GET
Cloud and NAT nodes need live host network interface data. Previously,
GET /projects/{project_id}/nodes/{node_id} returned cached properties
from creation time, so newly added host interfaces (e.g. kernel bridges
created by EthernetSwitch nodes) were invisible until the node was
deleted and recreated.

Now the controller fetches fresh data from the compute node before
returning the response, so host interface changes are reflected
immediately. Falls back to cached data if compute is unreachable.
2026-07-19 17:41:37 +08:00
YueGuobin
fd4ac4460d
feat: migrate builtin EthernetSwitch from Dynamips ethsw to ubridge brctl
Replace the builtin EthernetSwitch stub with a Linux kernel bridge backed
by uBridge's brctl module. Each switch node creates one kernel bridge
(gns3br{N}) with VLAN filtering; each port is a persistent TAP enslaved to
the bridge, relayed by a per-port uBridge bridge (nio_tap <-> nio_udp).

- Access/dot1q/qinq port modes translated to brctl vlan primitives
- Compute router repointed from Dynamips to Builtin manager
- Tests updated: 21 router-level tests + 215 surrounding tests pass
- Real-kernel e2e verified: access 100 PVID untagged, dot1q trunk
  VIDs 1-4094 + native 1, qinq 802.1ad proto + 200 PVID

Ethertype 0x9100/0x9200 handling and port-count guard relaxation are
deferred pending resolution.
2026-07-18 00:21:19 +08:00
YueGuobin
37867a1e44
fix(mcp): remove unreliable node_reload / node_reload_all tools
For Docker nodes, reload bottoms out as a raw POST /containers/{id}/restart
to the Docker daemon, bypassing GNS3's start/stop lifecycle (uBridge
re-attach, console servers, NIC setup). The container restarts at the
Docker level but GNS3's plumbing goes out of sync, and the daemon call can
block up to the controller's 240s timeout — manifesting as "reload hangs /
no response". stop+start runs the full lifecycle and is reliable.

Remove the node_reload and node_reload_all MCP tools (tool functions,
handlers, NODE_TOOLS entry, tests, docs). The underlying REST endpoints
(POST /nodes/{id}/reload, POST /nodes/reload) are kept for native API
users. MCP callers should use node_stop + node_start (partial) or
close/open project (full restart) instead.
2026-07-16 13:27:21 +08:00
Jeremy Grossmann
013ca442f4
Merge pull request #2816 from yueguobin/feature/traffic-insight-markers
feat: add traffic-insight markers (ubridge mark filter integration)
2026-07-15 22:12:11 +02:00
YueGuobin
884c40038c
docs(mcp): note --no-close for node_console websocat usage
A heredoc (<<<) closes stdin at once, so websocat dropped the WebSocket
before the device's reply arrived. Add --no-close to the node_console usage
examples and the returned command field so the connection stays open while
output is read.
2026-07-16 00:51:45 +08:00
YueGuobin
d4389b7068
fix(rbac): guard None current_user on websocket auth failure
get_current_active_user_from_websocket returns None after closing the socket
on an auth failure (revoked token, bad credentials, inactive user).
has_privilege_on_websocket dereferenced current_user.is_superadmin without a
None check, so any websocket auth failure surfaced as an AttributeError
traceback instead of a clean close. Bail out early when current_user is None,
mirroring the guard already present in ws_console.
2026-07-16 00:51:45 +08:00
YueGuobin
3d6d9a3396
fix(mcp): thread token_version into console/download token minting
Token revocation is a strict version check (token_data.token_version !=
user.token_version). create_access_token defaults token_version to 0, so the
short-lived JWTs minted for the console WebSocket URL (nodes) and the download
URLs (symbols, links) carried ver=0. Any user who had logged out at least once
(token_version >= 1, e.g. the default admin) got tokens rejected as "revoked"
on first use.

e433991cf fixed this in _resolve_token's API-key branch but missed these three
independent minting sites. Now resolve token_version during _resolve_token (the
JWT branch decodes it, the API-key branch reads user.token_version), carry it
through gns3_ctx, and pass it at every minting call.
2026-07-16 00:51:45 +08:00
YueGuobin
89e11d265f
fix(mcp): pass user.token_version when generating temp JWT from API key
_resolve_token generated a temp JWT with a hardcoded ver=0 after
validating the API key. Users who had logged out at least once
(token_version >= 1) would hit "Token has been revoked" 401 on
every MCP tool call, because the REST auth chain rejects ver=0
when the user's token_version no longer matches.

Fix: pass the user's actual token_version to create_access_token
so the temp JWT carries the correct ver claim.
2026-07-16 00:51:45 +08:00
YueGuobin
27356da62d
fix(marker): set nio.markers on iou/dynamips/cloud nio update
The PUT /nio (update) endpoints for iou, dynamips and cloud copied nio.filters from the request body but never copied nio.markers. So adding a marker (e.g. a project-level global definition) to an already-created link of these node types left nio.markers empty: no mark filter was installed on uBridge and no marker.match signal was emitted. vpcs/qemu/docker already set it; mirror them (getattr for the Union NIO types).
2026-07-16 00:39:52 +08:00
YueGuobin
b21780605e
fix(marker): reject reserved global-prefix names only on create
The global-prefix reservation belongs on the create path, where it keeps
user-chosen names disjoint from inherited global-{name} markers. It was
implemented as a field_validator on MarkerCreate / MarkerDefinitionCreate,
which also back the PUT update bodies — so updating an inherited marker
while echoing its name in the body tripped the validator with a generic
422 before the controller could return the actionable 409 ("inherited,
use the marker-definitions API").

Drop the schema validators and enforce the prefix in the two create routes
(ControllerError -> 409). PUT no longer validates the body name (ignored
anyway — the target is the {name} path param), so editing an inherited
marker now reaches the controller's inheritance guard and returns the
clear 409. The name-format regex stays on the schema (still 422).
2026-07-16 00:39:52 +08:00
YueGuobin
98bcd5eddd
feat(marker): add highlight_duration render hint
Per-marker UI hint (milliseconds, ge=1) for how long the Web UI keeps a
marker highlighted after a match. Mirrors color: stored on the link,
persisted in the topology, inherited via project-level definitions, and
never sent to uBridge. Omitted = null = frontend uses its own default.

The default is intentionally NOT set in the schema: MarkerCreate also
backs PUT updates, so a schema default would make every partial update
silently reset the value. None means "not provided" (keep existing on
update / frontend decides on create).
2026-07-16 00:39:52 +08:00
YueGuobin
31991fe359
feat(marker): add project-level marker definition inheritance
Project-level marker definitions fan out to every link (existing and new).
A definition is stored once on Project._marker_definitions; when applied to
a link the marker is named "global-{def_name}" — the "global" prefix was
pre-reserved in the schema, so inherited and per-link markers can never
collide, nor will their registry keys.

Key behavior:
- POST /projects/{pid}/marker-definitions → fan out to all existing links
- PUT  /projects/{pid}/marker-definitions/{name} → sync all inherited copies
- DELETE → remove every inherited copy from every link
- New links auto-inherit all active defs (hook in UDPLink.create)
- Per-link DELETE/PUT of a "global-*" marker is rejected (409)
- Inherited markers are NOT persisted in the topology; they are re-created
  from _marker_definitions on project load
- Compute side is untouched — the marker reaches uBridge via the existing
  start_marker→update→NIO→ubridge pipeline

Files:
- controller/project.py — _marker_definitions + CRUD + fanout + topology load
- controller/link.py — Link.inherit_marker() + asdict() filter
- controller/udp_link.py — guards on stop/update + create() inheritance hook
- controller/topology.py — persist marker_definitions in project topology
- schemas/controller/links.py — MarkerDefinitionCreate schema
- api/routes/controller/projects.py — REST endpoints (marker-definitions)
2026-07-16 00:39:52 +08:00
YueGuobin
7e4600b5a1
fix(marker): validate marker name and expose enabled via REST
Add a name pattern constraint to MarkerCreate schema (alphanumeric +
_.-) to prevent injection into uBridge commands, with a matching
defense-in-depth check in _ubridge_add_marker_filter for hand-edited
topology files. Also add the missing "enabled" field to MarkerCreate
and pass it through the PUT route so the documented toggle actually
works.
2026-07-16 00:39:51 +08:00
YueGuobin
a495b5ddf2
fix(marker): uniquify auto-generated marker names per link
Auto-generating `marker-{link.id[:8]}` collided on the second anonymous
marker on the same link (start_marker rejects duplicate names). Append a
short uuid hex suffix so multiple anonymous markers coexist.
2026-07-16 00:39:51 +08:00
YueGuobin
f2360f85fc
feat(marker): add project-level marker aggregation endpoint
Add a read-only `markers` property on Project that flattens every link's
markers into a single dict keyed by "{link_id}/{name}", each entry
carrying the parent link_id and capture-side node_id. Expose it via
GET /projects/{pid}/markers (Project.Audit) so the frontend can fetch all
markers in one round-trip instead of enumerating links first.

Also surface a marker count in project.stats().
2026-07-16 00:39:51 +08:00
YueGuobin
c62b9b0283
refactor(marker): converge to filter single-path model, remove dual-apply endpoints
Markers now follow exactly the same apply pattern as packet filters:
state lives in Link._markers, application goes through NIO
(update() -> PUT /nio -> _ubridge_apply_markers). The former
immediate-apply REST endpoints (/markers/start, /markers/stop on
the compute side) and the per-node start_marker/stop_marker methods
are removed — they were a legacy of the original capture-inspired
design and have been superseded by the NIO flow.

Changes:
- controller/udp_link: start_marker/stop_marker/update_marker now
  set _markers state + call self.update() (mirrors update_filters).
  Removed _marker_capture_nodes runtime dict and its helpers.
- controller/project: _create_link_from_topology_data restores
  _markers directly from persisted data (with BPF validation,
  like filter reload). No long calls start_marker during load.
- compute: _ubridge_apply_markers swallows BPF compile errors
  (warn+skip), matching _ubridge_apply_filters behaviour so a
  single bad expression cannot break link creation / node restart.
- Removed: /markers/start,stop endpoints (6 handlers across
  vpcs/qemu/docker route files), node start_marker/stop_marker
  methods (3 VM files), _ubridge_delete_marker_filter,
  _marker_capture_nodes, MarkerDelete schema.

Net: ~280 lines of dead code removed; marker and packet filter now
share a single, unified apply path via the NIO.
2026-07-16 00:39:51 +08:00
YueGuobin
3f15ac38f6
feat(marker): add optional color field to marker data model
Add a user-chosen hex color (e.g. '#ff5722') to each marker, stored in
Link._markers alongside bpf/tag/enabled. The color:

- Is serialized with the link (asdict) and persisted in the topology, so
  it survives project reload and is consistent across devices.
- Never reaches uBridge (compute ignores it) — it is purely a Web UI
  concern.
- Can be updated independently of the BPF (color-only changes skip the
  ubridge round-trip).

MarkerCreate schema gains color: Optional[str]; start_marker and
update_marker carry it through; REST create/update endpoints pass it.

The marker.match WebSocket event stays lean (no color) — the Web UI maps
filter -> color from the link state it already holds.
2026-07-16 00:39:51 +08:00
YueGuobin
7c90fa9e64
fix(marker): ride markers on NIO so they survive node restart and project reload
Mirror the packet-filter lifecycle: marker specs now live on the NIO
(next to filters), ride in link_data from controller to compute on
every NIO create/update, and are reapplied by _ubridge_apply_markers
in add_ubridge_udp_connection (bridge creation / node restart) and
update_ubridge_udp_connection (NIO update — following the preceding
reset_packet_filters so markers survive filter changes).

Changes:
- NIO / NIOUDP: _markers property + asdict
- schemas/compute/nios.py: UDPNIO.markers field
- base_manager.create_nio: nio.markers from settings
- PUT /nio routes (vpcs/qemu/docker): nio.markers update
- base_node: _ubridge_apply_markers(bridge_name, nio) iterates
  nio.markers, computes pcap path, calls _ubridge_add_marker_filter
  + MarkerManager.register; called after _ubridge_apply_filters
- controller udp_link: _get_node_markers + _markers_for_node (route
  by capture_node_id); markers in create() and update() link_data
- /markers/start,stop endpoints: mirror spec onto nio.markers so
  the marker survives a subsequent node stop/start without a PUT
- tests: add markers field to NIO data expectations

This covers:
- Node restart: NIO persists, add_ubridge_udp_connection re-applies
- Filter update: reset wipes markers, _ubridge_apply_markers re-adds
- Project reload: create() carries markers in link_data → create_nio
- Immediate create: endpoint sets nio.markers immediately
2026-07-16 00:39:51 +08:00
YueGuobin
d2e9823e6c
feat: implement traffic insight marker backend (ubridge mark filter)
Add compute-side marker subsystem that receives ubridge UDP MARK signals
and forwards them as project-scoped notifications to the web UI for
real-time traffic coloring. Matched packets are always saved to per-link
pcaps for future replay.

Key components:
- gns3server/compute/marker/: MarkerManager (singleton, UDP listener +
  O(1) registry keyed by (node_id, filter_name)) + MarkerListener
  (DatagramProtocol parsing MARK lines per ubridge integration contract)
- gns3server/compute/base_node.py: marker sink/node config at ubridge
  startup; shared _ubridge_add_marker_filter / _ubridge_delete_marker_filter
- Per-node start_marker/stop_marker: VPCS (VPCS-{id}), QEMU
  (QEMU-{id}-{adapter}), Docker (bridge{adapter})
- Compute REST /markers/start + /markers/stop (vpcs/qemu/docker route files)
- Controller Link._markers state + UDPLink.start_marker/stop_marker/
  update_marker (mirror capture pattern: BPF validation, _choose_capture_side,
  node.post forwarding, topology persistence)
- Controller REST GET/POST/DELETE/PUT /v3/projects/{p}/links/{l}/markers
- Config: marker_listen_host / marker_listen_port in ServerSettings
- Signal routing: creation-time registry O(1) lookup, no node-table scan;
  project-scoped WS stream (not global); event payload always carries
  project_id for frontend scoping

Tests: 14 unit tests (registry, listener parsing, UDP round-trip);
562 existing tests pass with zero regressions.
2026-07-16 00:39:51 +08:00
YueGuobin
292b60efaa
Make AI features (AI Copilot + MCP) optional via [ai-features] extra
- Move fastmcp from core requirements.txt to mcp-requirements.txt
- Add MCP_AVAILABLE feature flag in agent/__init__.py (graceful degradation)
- Guard MCP imports/registration in server.py and tasks.py
- Replace ai-copilot/mcp/ai-support extras with single ai-features extra
- Add stub MCP routes returning 501 when MCP is not installed
- Add gns3server-uninstall-ai-features CLI command
- Remove old gns3server-uninstall-ai-copilot command
- Update all error messages and docs to reference ai-features

Closes #2794
2026-06-27 10:14:43 +08:00
YueGuobin
0e8d0cb87b
Add stateless JWT refresh token mechanism
- New config: Controller.jwt_refresh_token_expire_minutes (default 30 days)
- New endpoint: POST /v3/access/users/refresh (public, unauthenticated)
- Login/authenticate responses now include refresh_token
- AuthService: _create_token helper, create_refresh_token, get_token_data
  now parses type claim (token_use) for token classification
- Security: refresh tokens rejected on HTTP + WebSocket access paths;
  /refresh strictly requires type=='refresh'
- Logout works for free via existing token_version mechanism
- Tests: 9 new TestRefreshToken cases, all passing; 34 existing tests
  still pass (no regressions)
2026-06-23 22:39:11 +08:00
YueGuobin
ead9747e1b
Fix appliance_install: add version parameter 2026-06-17 23:03:28 +08:00
YueGuobin
0d21f294f6
Document canvas coordinate system in node_create x/y params 2026-06-16 23:16:17 +08:00
YueGuobin
00647d092c
Update docstring: batch concurrency from 10 to 100 2026-06-16 23:05:17 +08:00
YueGuobin
10ff5b2283
Add list type check before nodes[0] access in _normalize_link_nodes 2026-06-16 23:00:47 +08:00
YueGuobin
0d45bfbae5
Add fields type validation in create handlers 2026-06-16 22:58:36 +08:00
YueGuobin
59e5f8dd2f
Fix review issues: key_prefix length, count validation, WAL log, timeout comment, pointless temp var 2026-06-16 22:50:22 +08:00
YueGuobin
533d3d5b71
Fix: pass name from TemplateUsage to add_node_from_template
The controller endpoint received name via TemplateUsage but never
passed it to add_node_from_template, causing the name parameter to
be silently ignored. name is optional - if omitted the template's
default_name_format is used.
2026-06-16 22:43:46 +08:00
YueGuobin
68567118f1
Remove unused imports (logging, log, select) 2026-06-16 22:25:07 +08:00
Jeremy Grossmann
7d5d4767bf
Remove FIXME comment about middleware in server.py
Removed commented out FIXME note regarding middleware issues.
2026-06-16 12:20:55 +02:00
YueGuobin
f7abb62603
Rename device_command_run_handler → device_show_run_handler to match tool name 2026-06-16 12:44:12 +08:00
YueGuobin
fbb8924a8e
Rename device_command_run → device_show_run for clarity
The name device_command_run was ambiguous and easily confused with
device_config_send. device_show_run makes the read-only intent explicit,
forming a clear pair: config_send (write) vs show_run (read).
2026-06-16 12:27:35 +08:00
YueGuobin
e627481ec2
Fix device_command_run KeyError('commands'): tool desc said show_commands but backend expects commands
- Tool description and param schema now use 'commands' (matching ExecuteMultipleDeviceCommands)
- Updated error message in handler
- Clarified in docstring that this is read-only; use device_config_send for config changes
2026-06-16 12:23:15 +08:00
YueGuobin
ec58bea24b
Fix template_list return type annotation to match _run_handler_sync envelope
template_list was annotated as dict[str, Any] but _run_handler_sync always
returns a list envelope ([{"type":"text",...}]). Other tools use
list[dict[str, Any]] consistently. This caused a Pydantic dict_type
validation error on the client.
2026-06-16 12:08:07 +08:00
YueGuobin
74192f454f
Increase BATCH_MAX_WORKERS and Pool concurrency from 20 to 100 2026-06-16 00:49:08 +08:00
YueGuobin
286b2407cd
Remove final timing artifact in projects.py 2026-06-16 00:44:54 +08:00
YueGuobin
bd15c8d019
Remove remaining dead timing variables and imports 2026-06-16 00:40:03 +08:00
YueGuobin
ccb629f48f
Clean up all timing/debug logs
Remove all [MCP-TIMING] and [CTRL-TIMING] log lines, timing middleware,
and related import time statements across 11 files.
2026-06-16 00:34:11 +08:00
YueGuobin
7d8ab399da
Fix: add missing UUID imports 2026-06-16 00:24:37 +08:00
YueGuobin
d6b337bc3a
Generate fresh JWT on API key auth instead of returning raw key
When MCP client authenticates with API key, _resolve_token now
returns a newly generated JWT. This JWT is stored in the ContextVar
and used by all subsequent tool handler API calls - zero extra bcrypt.
2026-06-16 00:09:36 +08:00
YueGuobin
f522c947bc
Optimize API key auth: O(1) lookup via UUID-embedded key format
New format: gns3_<api_key_id>_<random_secret>
- Auth extracts api_key_id from token → single DB query by UUID → one bcrypt
- No more scanning all keys (was O(n) with bcrypt per key)
- bcrypt.checkpw offloaded to thread pool to prevent event loop blocking
- Legacy gns3_<random> format removed (compatibility break)
2026-06-15 23:56:36 +08:00
YueGuobin
796a2e6ca8
Fix: offload bcrypt.checkpw to thread pool to prevent blocking event loop
bcrypt.checkpw is CPU-bound (~1.3s per call) and was running
synchronously inside the async event loop, blocking ALL concurrent
requests. With 5 API keys and 10 concurrent requests, this caused
~13s delay before any handler could start.
2026-06-15 23:49:28 +08:00
YueGuobin
04f0657661
Add timing to API key auth path and log api_keys count 2026-06-15 23:45:26 +08:00
YueGuobin
5502b8f8e5
Add timing logs to auth dependency chain to identify 6s pre-handler delay 2026-06-15 23:35:12 +08:00
YueGuobin
fa9aa5a9cc
Add timing middleware to log slow requests (>1s) with [CTRL-TIMING] prefix 2026-06-15 23:13:36 +08:00
YueGuobin
ea4bb2c1fb
Add [CTRL-TIMING] logs to controller create_node flow
Timing logs cover:
- create_node_from_template (entry, get_template, add_node, total)
- add_node_from_template (entry to done)
- _create_node (project_setup, node.create, total)
- Node.create (compute_post timing)
- compute._session.request (actual HTTP to compute)
2026-06-15 23:04:01 +08:00
YueGuobin
e5797c3da0
Fix: pass template_id to batch mode handler so top-level template_id works as default 2026-06-15 22:57:40 +08:00
YueGuobin
cef6dc6bd2
Add detailed timing logs to MCP node creation and HTTP client
Logs with [MCP-TIMING] prefix at:
- create_node_handler entry, setup, http_call start/end, total
- http_call entry, auth, response
- _authenticate_v3 entry, done, fail
2026-06-15 22:51:43 +08:00
YueGuobin
58ca6b1dd4
Add fields filter to template_list tool with description for AI 2026-06-15 22:11:14 +08:00