5207 Commits

Author SHA1 Message Date
YueGuobin
f2360f85fc
feat(marker): add project-level marker aggregation endpoint
Add a read-only `markers` property on Project that flattens every link's
markers into a single dict keyed by "{link_id}/{name}", each entry
carrying the parent link_id and capture-side node_id. Expose it via
GET /projects/{pid}/markers (Project.Audit) so the frontend can fetch all
markers in one round-trip instead of enumerating links first.

Also surface a marker count in project.stats().
2026-07-16 00:39:51 +08:00
YueGuobin
ab5b7444d4
feat(marker): add IOU node marker support via iol_bridge
IOU uses iol_bridge (not bridge) with extra bay/unit positional
arguments in add_packet_filter. Add _ubridge_apply_markers override
that mirrors the existing _ubridge_apply_filters pattern, called at
the same three sites: _networking, adapter_add_nio_binding, and
adapter_update_nio_binding.
2026-07-16 00:39:51 +08:00
YueGuobin
c62b9b0283
refactor(marker): converge to filter single-path model, remove dual-apply endpoints
Markers now follow exactly the same apply pattern as packet filters:
state lives in Link._markers, application goes through NIO
(update() -> PUT /nio -> _ubridge_apply_markers). The former
immediate-apply REST endpoints (/markers/start, /markers/stop on
the compute side) and the per-node start_marker/stop_marker methods
are removed — they were a legacy of the original capture-inspired
design and have been superseded by the NIO flow.

Changes:
- controller/udp_link: start_marker/stop_marker/update_marker now
  set _markers state + call self.update() (mirrors update_filters).
  Removed _marker_capture_nodes runtime dict and its helpers.
- controller/project: _create_link_from_topology_data restores
  _markers directly from persisted data (with BPF validation,
  like filter reload). No long calls start_marker during load.
- compute: _ubridge_apply_markers swallows BPF compile errors
  (warn+skip), matching _ubridge_apply_filters behaviour so a
  single bad expression cannot break link creation / node restart.
- Removed: /markers/start,stop endpoints (6 handlers across
  vpcs/qemu/docker route files), node start_marker/stop_marker
  methods (3 VM files), _ubridge_delete_marker_filter,
  _marker_capture_nodes, MarkerDelete schema.

Net: ~280 lines of dead code removed; marker and packet filter now
share a single, unified apply path via the NIO.
2026-07-16 00:39:51 +08:00
YueGuobin
88c03d1431
fix(marker): restore markers from topology on project load
_create_link_from_topology_data now restores traffic-insight markers
(mirroring how filters are restored via update_filters), so markers —
including their color — survive project close/reopen and server restart.

Guard start_marker's uBridge POST with 'if self._created' (exactly as
update_filters guards its update() call): during project load the link
is not yet created, so only _markers state is recorded and the marker
is applied once via the NIO flow in create()/_ubridge_apply_markers —
no double application.
2026-07-16 00:39:51 +08:00
YueGuobin
3f15ac38f6
feat(marker): add optional color field to marker data model
Add a user-chosen hex color (e.g. '#ff5722') to each marker, stored in
Link._markers alongside bpf/tag/enabled. The color:

- Is serialized with the link (asdict) and persisted in the topology, so
  it survives project reload and is consistent across devices.
- Never reaches uBridge (compute ignores it) — it is purely a Web UI
  concern.
- Can be updated independently of the BPF (color-only changes skip the
  ubridge round-trip).

MarkerCreate schema gains color: Optional[str]; start_marker and
update_marker carry it through; REST create/update endpoints pass it.

The marker.match WebSocket event stays lean (no color) — the Web UI maps
filter -> color from the link state it already holds.
2026-07-16 00:39:51 +08:00
YueGuobin
b8f49696cc
fix(marker): use allowlist in _choose_marker_side, align with _get_filter_node
Replace the complex tier-based fallback (which leaked into unsupported
switch types on the final tiers) with a simple allowlist of
_MARKER_CAPABLE_TYPES, mirroring _get_filter_node in link.py.

Also accept stopped nodes — like packet filters, markers are stored on
the NIO and applied when the node starts via _ubridge_apply_markers.
2026-07-16 00:39:51 +08:00
YueGuobin
239f42f5aa
fix(marker): stop auto-deleting markers on node stop in node_updated
Markers are persistent link-scoped features (like packet filters), not
transient runtime features (like capture).  Tearing them down on every
node stop breaks stop/start restore: the NIO-based re-application in
_ubridge_apply_markers works correctly on restart, but node_updated
was prematurely calling stop_marker which erased _markers state and
dumped an empty topology.

Now markers survive node stop transparently — the ubridge filter dies
with the process but is re-added from nio.markers in
add_ubridge_udp_connection when the node starts again, mirroring
exactly how packet filters behave.
2026-07-16 00:39:51 +08:00
YueGuobin
7c90fa9e64
fix(marker): ride markers on NIO so they survive node restart and project reload
Mirror the packet-filter lifecycle: marker specs now live on the NIO
(next to filters), ride in link_data from controller to compute on
every NIO create/update, and are reapplied by _ubridge_apply_markers
in add_ubridge_udp_connection (bridge creation / node restart) and
update_ubridge_udp_connection (NIO update — following the preceding
reset_packet_filters so markers survive filter changes).

Changes:
- NIO / NIOUDP: _markers property + asdict
- schemas/compute/nios.py: UDPNIO.markers field
- base_manager.create_nio: nio.markers from settings
- PUT /nio routes (vpcs/qemu/docker): nio.markers update
- base_node: _ubridge_apply_markers(bridge_name, nio) iterates
  nio.markers, computes pcap path, calls _ubridge_add_marker_filter
  + MarkerManager.register; called after _ubridge_apply_filters
- controller udp_link: _get_node_markers + _markers_for_node (route
  by capture_node_id); markers in create() and update() link_data
- /markers/start,stop endpoints: mirror spec onto nio.markers so
  the marker survives a subsequent node stop/start without a PUT
- tests: add markers field to NIO data expectations

This covers:
- Node restart: NIO persists, add_ubridge_udp_connection re-applies
- Filter update: reset wipes markers, _ubridge_apply_markers re-adds
- Project reload: create() carries markers in link_data → create_nio
- Immediate create: endpoint sets nio.markers immediately
2026-07-16 00:39:51 +08:00
YueGuobin
540e1c5678
config: set marker_listen_port default to 3070 and document in config templates
Change the marker UDP sink default from port=0 (OS-chosen) to 3070 for
predictable firewall/debugging/monitoring.

Add marker_listen_host and marker_listen_port with documentation to
config_samples/gns3_server.conf and scripts/gns3_server.conf.
2026-07-16 00:39:51 +08:00
YueGuobin
88465e872c
fix(marker): add _choose_marker_side to exclude non-uBridge node types
_choose_capture_side prefers BUILTIN_NODE_TYPES which includes
ethernet_switch/ethernet_hub/NAT etc. — nodes that do not use uBridge
and therefore have no bridge for a marker filter to attach to.

Add _choose_marker_side with a _MARKER_UNSUPPORTED_TYPES blocklist
so marker capture-side selection skips NIO-only switches and picks
the uBridge-backed node (VPCS/QEMU/Docker/etc.) instead.

Verified end-to-end: PC1↔Switch↔PC2 links with arp markers correctly
pick VPCS nodes; MARK signals flow through to WebSocket and pcaps
are written.
2026-07-16 00:39:51 +08:00
YueGuobin
d2e9823e6c
feat: implement traffic insight marker backend (ubridge mark filter)
Add compute-side marker subsystem that receives ubridge UDP MARK signals
and forwards them as project-scoped notifications to the web UI for
real-time traffic coloring. Matched packets are always saved to per-link
pcaps for future replay.

Key components:
- gns3server/compute/marker/: MarkerManager (singleton, UDP listener +
  O(1) registry keyed by (node_id, filter_name)) + MarkerListener
  (DatagramProtocol parsing MARK lines per ubridge integration contract)
- gns3server/compute/base_node.py: marker sink/node config at ubridge
  startup; shared _ubridge_add_marker_filter / _ubridge_delete_marker_filter
- Per-node start_marker/stop_marker: VPCS (VPCS-{id}), QEMU
  (QEMU-{id}-{adapter}), Docker (bridge{adapter})
- Compute REST /markers/start + /markers/stop (vpcs/qemu/docker route files)
- Controller Link._markers state + UDPLink.start_marker/stop_marker/
  update_marker (mirror capture pattern: BPF validation, _choose_capture_side,
  node.post forwarding, topology persistence)
- Controller REST GET/POST/DELETE/PUT /v3/projects/{p}/links/{l}/markers
- Config: marker_listen_host / marker_listen_port in ServerSettings
- Signal routing: creation-time registry O(1) lookup, no node-table scan;
  project-scoped WS stream (not global); event payload always carries
  project_id for frontend scoping

Tests: 14 unit tests (registry, listener parsing, UDP round-trip);
562 existing tests pass with zero regressions.
2026-07-16 00:39:51 +08:00
grossmj
a9cce209d7
Development on 3.1.0.dev5 2026-07-10 10:01:35 +02:00
YueGuobin
0ed1715fe6
Use typed InterfaceStatus enum and simplify stats fallback in interfaces()
- Change `status: str` to `status: InterfaceStatus` (str Enum) in the
  HostInterface schema, matching the existing IPAddressFamily pattern.
- Use default-then-override pattern in interfaces() to eliminate the
  `else` branch and deduplicate defaults.
2026-07-10 14:55:21 +08:00
YueGuobin
9721660cc3
Expose full interface address list and link state in cloud node API
Each host interface surfaced by the cloud node now reports:
- ip_addresses: every IPv4 and IPv6 address (previously only a single
  IPv4 was collected internally and then dropped before the response)
- status / speed / mtu / flags: operational state and link attributes
  sourced from psutil.net_if_stats(), with flags normalized to a list

The legacy ip_address / netmask / mac_address fields are preserved so
existing callers (compute link detection, GNS3 VM, VMware, has_netmask)
keep working. The new fields travel through the existing interfaces
payload that the controller forwards verbatim, so no controller-side
change is required and the PUT / ports_mapping flow is unaffected.
2026-07-10 13:56:07 +08:00
grossmj
913d021705
Release v3.1.0a4 2026-07-09 10:40:38 +02:00
grossmj
d64fdd71a9
Bundle web-ui v3.1.0a4 2026-07-09 08:40:05 +02:00
Guobin Yue
61ce59788b
Merge branch '3.1' into docs/jwt-refresh-token-config-example 2026-07-09 14:19:14 +08:00
YueGuobin
cccc3e5b77
Add jwt_refresh_token_expire_minutes to sample configuration
The sample gns3_server.conf documented jwt_access_token_expire_minutes but omitted jwt_refresh_token_expire_minutes (default 43200 minutes, 30 days) defined in ControllerSettings. Add it for parity with the schema.
2026-07-09 14:15:49 +08:00
grossmj
54540e7ef7
Merge remote-tracking branch 'origin/3.1' into 3.1 2026-07-08 21:49:20 +02:00
grossmj
5992a284c9
Bundle web-ui v3.1.0a4 2026-07-08 21:48:48 +02:00
grossmj
a2ee5389af
Bundle web-ui v3.1.0a4 2026-07-08 20:11:59 +02:00
grossmj
4548d9053a
Sync appliances 2026-07-08 18:45:07 +02:00
Guobin Yue
277640a335
Merge branch '3.1' into fix-web-wireshark-docker-xpra6.x 2026-07-06 13:35:44 +08:00
YueGuobin
f5a72532ac
Fix web-wireshark docker build broken by xpra 6.5 release
Replace the `xpra=6.4*` version clamp on the apt-get install command
line with APT pinning via /etc/apt/preferences.d/pin-xpra.

The previous specifier stopped resolving once xpra 6.5 was released:
the xpra metapackage was pinned to 6.4 while its sub-packages
(xpra-client, xpra-server, xpra-codecs, ...) were unconstrained and
defaulted to 6.5, so apt could not satisfy the dependency graph and
the "Build Docker images" action failed.

Pinning all xpra* packages at the preferences level constrains the
whole set consistently (xpra-html5 is handled separately since it
follows a different version scheme, locked to v19). Blocking every
other version with priority -1 ensures that, if the pinned version
ever disappears from the repository, apt fails loudly instead of
silently upgrading to the latest release.

Fixes GNS3/gns3-registry#1044
2026-07-06 13:31:07 +08:00
YueGuobin
292b60efaa
Make AI features (AI Copilot + MCP) optional via [ai-features] extra
- Move fastmcp from core requirements.txt to mcp-requirements.txt
- Add MCP_AVAILABLE feature flag in agent/__init__.py (graceful degradation)
- Guard MCP imports/registration in server.py and tasks.py
- Replace ai-copilot/mcp/ai-support extras with single ai-features extra
- Add stub MCP routes returning 501 when MCP is not installed
- Add gns3server-uninstall-ai-features CLI command
- Remove old gns3server-uninstall-ai-copilot command
- Update all error messages and docs to reference ai-features

Closes #2794
2026-06-27 10:14:43 +08:00
YueGuobin
0e8d0cb87b
Add stateless JWT refresh token mechanism
- New config: Controller.jwt_refresh_token_expire_minutes (default 30 days)
- New endpoint: POST /v3/access/users/refresh (public, unauthenticated)
- Login/authenticate responses now include refresh_token
- AuthService: _create_token helper, create_refresh_token, get_token_data
  now parses type claim (token_use) for token classification
- Security: refresh tokens rejected on HTTP + WebSocket access paths;
  /refresh strictly requires type=='refresh'
- Logout works for free via existing token_version mechanism
- Tests: 9 new TestRefreshToken cases, all passing; 34 existing tests
  still pass (no regressions)
2026-06-23 22:39:11 +08:00
YueGuobin
ead9747e1b
Fix appliance_install: add version parameter 2026-06-17 23:03:28 +08:00
YueGuobin
0d21f294f6
Document canvas coordinate system in node_create x/y params 2026-06-16 23:16:17 +08:00
YueGuobin
00647d092c
Update docstring: batch concurrency from 10 to 100 2026-06-16 23:05:17 +08:00
YueGuobin
da35eaa014
Remove unused import time 2026-06-16 23:04:05 +08:00
YueGuobin
10ff5b2283
Add list type check before nodes[0] access in _normalize_link_nodes 2026-06-16 23:00:47 +08:00
YueGuobin
0d45bfbae5
Add fields type validation in create handlers 2026-06-16 22:58:36 +08:00
YueGuobin
390a40ce80
Use pop() instead of pop(0) for O(1) port removal 2026-06-16 22:56:42 +08:00
YueGuobin
59e5f8dd2f
Fix review issues: key_prefix length, count validation, WAL log, timeout comment, pointless temp var 2026-06-16 22:50:22 +08:00
YueGuobin
533d3d5b71
Fix: pass name from TemplateUsage to add_node_from_template
The controller endpoint received name via TemplateUsage but never
passed it to add_node_from_template, causing the name parameter to
be silently ignored. name is optional - if omitted the template's
default_name_format is used.
2026-06-16 22:43:46 +08:00
YueGuobin
68567118f1
Remove unused imports (logging, log, select) 2026-06-16 22:25:07 +08:00
YueGuobin
af94c1729b
Add warning for unconsumed pre-allocated UDP ports after link creation
Ports are properly released on project close (compute.post /close).
The warning alerts if any pre-allocated ports go unused within a session.
2026-06-16 22:22:46 +08:00
Jeremy Grossmann
7d5d4767bf
Remove FIXME comment about middleware in server.py
Removed commented out FIXME note regarding middleware issues.
2026-06-16 12:20:55 +02:00
YueGuobin
f7abb62603
Rename device_command_run_handler → device_show_run_handler to match tool name 2026-06-16 12:44:12 +08:00
YueGuobin
fbb8924a8e
Rename device_command_run → device_show_run for clarity
The name device_command_run was ambiguous and easily confused with
device_config_send. device_show_run makes the read-only intent explicit,
forming a clear pair: config_send (write) vs show_run (read).
2026-06-16 12:27:35 +08:00
YueGuobin
e627481ec2
Fix device_command_run KeyError('commands'): tool desc said show_commands but backend expects commands
- Tool description and param schema now use 'commands' (matching ExecuteMultipleDeviceCommands)
- Updated error message in handler
- Clarified in docstring that this is read-only; use device_config_send for config changes
2026-06-16 12:23:15 +08:00
YueGuobin
ec58bea24b
Fix template_list return type annotation to match _run_handler_sync envelope
template_list was annotated as dict[str, Any] but _run_handler_sync always
returns a list envelope ([{"type":"text",...}]). Other tools use
list[dict[str, Any]] consistently. This caused a Pydantic dict_type
validation error on the client.
2026-06-16 12:08:07 +08:00
YueGuobin
d407b29fe9
Increase HTTP connection pool to 500/1000 2026-06-16 00:58:12 +08:00
YueGuobin
74192f454f
Increase BATCH_MAX_WORKERS and Pool concurrency from 20 to 100 2026-06-16 00:49:08 +08:00
YueGuobin
286b2407cd
Remove final timing artifact in projects.py 2026-06-16 00:44:54 +08:00
YueGuobin
bd15c8d019
Remove remaining dead timing variables and imports 2026-06-16 00:40:03 +08:00
YueGuobin
a8cc01af46
Remove database warmup (proven ineffective - real bottleneck was bcrypt blocking event loop) 2026-06-16 00:35:42 +08:00
YueGuobin
ccb629f48f
Clean up all timing/debug logs
Remove all [MCP-TIMING] and [CTRL-TIMING] log lines, timing middleware,
and related import time statements across 11 files.
2026-06-16 00:34:11 +08:00
YueGuobin
7d8ab399da
Fix: add missing UUID imports 2026-06-16 00:24:37 +08:00
YueGuobin
d6b337bc3a
Generate fresh JWT on API key auth instead of returning raw key
When MCP client authenticates with API key, _resolve_token now
returns a newly generated JWT. This JWT is stored in the ContextVar
and used by all subsequent tool handler API calls - zero extra bcrypt.
2026-06-16 00:09:36 +08:00