mirror of
https://github.com/GNS3/gns3-server.git
synced 2026-08-27 20:40:13 +03:00
80 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
YueGuobin
|
72dc10a669
|
controller: allow markers and packet filters on Ethernet switch links
The brctl Ethernet switch runs a per-port uBridge relay, so it can host the mark filter and packet filters like any uBridge-backed node. Add ethernet_switch to _MARKER_CAPABLE_TYPES and _get_filter_node, narrow the UDPLink.update() NIO-PUT skip down to the Dynamips-hosted ethernet_hub, and expose the matching compute endpoints: PUT nio (filter/marker reapply) plus the per-marker toggle/pause/resume/delete/rebuild routes. The ethernet_hub keeps its exclusion: its routes still wire into the Dynamips hub, which has no uBridge of its own. |
||
|
YueGuobin
|
350f2b24e7
|
fix: UDP port allocation race causing link self-loop (one-way links)
PortManager.get_free_udp_port had an unguarded find-then-add sequence. A link allocates both ends concurrently (asyncio.gather in UDPLink._prepare -> two POST /ports/udp) and FastAPI runs the sync route handler in a threadpool, so both threads could probe the same 'free' port before either recorded it — handing lport == rport to both ends. uBridge sets SO_REUSEADDR on UDP NIO sockets, so the double bind succeeds silently and the kernel delivers everything to the last-bound socket: one node starves, the other echoes to itself. Make every TCP/UDP allocate/reserve/release path atomic with an RLock, and rebuild _link_data in UDPLink._prepare so reset() commits the fresh port pair instead of re-sending the stale, already-released one. Regression tests: threaded barrier allocation never returns duplicates (red on the old code, UDP and TCP); reset() leaves exactly one mirrored NIO pair per side with lport != rport (red on the old code). |
||
|
YueGuobin
|
57558508bf
|
perf: batch update/delete marker-def fan-out too (full project-level)
per-def operations are project-level — create, update AND delete all
modify the marker policy on every link — so all three must batch, not
just create.
Extend memory_only to update_marker and stop_marker (merge/delete into
_markers + refresh _link_data, no per-link HTTP), and route
update_marker_definition and delete_marker_definition through the same
two-phase path as create: memory-only per link, then one
PUT /projects/{id}/nios/batch per compute.
Trade-off: a full reapply resets every marker's pcap on the link (the
old precise update_marker/stop_marker preserved sibling pcaps). For
project-level policy changes this is acceptable — real-time insight
matters more than pcap continuity, and batching turns 5000+ round-trips
into one per compute.
|
||
|
YueGuobin
|
48991f2d29
|
perf: batch marker-def fan-out to one PUT /nios/batch per compute
When a marker definition is created (or re-applied on data_link_type
change), the fan-out used to call inherit_marker -> update() on every
link, issuing one PUT /nio per link end (5000+ round-trips on a 2500-link
project). On started nodes each round-trip also reconfigured uBridge.
Two-phase fan-out:
- inherit_marker/start_marker gain memory_only: writes the marker into
link._markers and refreshes _link_data without any HTTP/emit/dump.
- _apply_def_to_all_links applies memory-only to every link, then
_batch_update_link_nios groups the updated NIO specs by compute and
sends a single PUT /projects/{id}/nios/batch per compute.
compute: new PUT /projects/{id}/nios/batch endpoint with _get_existing_nio
+ _update_nio_binding dispatch (mirrors create_batch_nios), re-applies
filters+markers to uBridge on started nodes.
Precise per-marker operations (update_marker bpf change, stop_marker on
def delete) are untouched — they deliberately avoid a full reapply to
preserve sibling marker pcaps.
|
||
|
YueGuobin
|
38c49a655c
|
perf: batch NIO dispatch on project open (one HTTP per compute)
Project open used to create each link by issuing two NIO POSTs from the
controller to the compute — ~5000 HTTP round-trips for a 2500-link
topology, all funnelling through the single shared controller/compute
event loop and capping throughput near 12 links/s.
Replace it with a bulk path:
- UDPLink split into _prepare() (local: ports, peer addrs, link_data)
and _commit_nios() (dispatch). create() = prepare + commit (interactive).
- Link.add_node gains batch=True: attach both nodes without triggering
per-link NIO HTTP.
- compute: new POST /projects/{id}/nios/batch endpoint with a unified
_add_nio_binding dispatch across node types (docker/qemu/iou/vpcs/
builtin differ in signature).
- project.open: prepare all links locally, group NIO entries by compute,
send each compute a single /nios/batch, then finalise (wire node/port
refs, mark created, notify, apply marker defs) in parallel.
Cuts controller->compute HTTP from O(links) to O(computes). Test added
for the batch endpoint.
|
||
|
YueGuobin
|
96d4d82716
|
perf: cache compute.host_ip + add UDPLink.create timing log
host_ip resolved socket.gethostbyname on every access with no cache. get_ip_on_same_subnet touches host_ip 2-4 times per link, so opening a 2500-link project issued thousands of blocking DNS calls inline on the event loop — freezing all concurrent link coroutines each time. This is the most likely cause of the 12-link/s throughput (1000x below what Pool(concurrency=100) should deliver) and the burst+pause pattern. - compute.host_ip: cache the resolution in _host_ip_cache, invalidate on host setter change - UDPLink.create: timing log splitting get_ip / ports / nio so the next project-open confirms where time actually goes |
||
|
YueGuobin
|
acc9e670af
|
perf: parallelize UDP port allocation and NIO creation in UDPLink.create
Previously the two sides' UDP port reservations and NIO tunnel POSTs were issued sequentially, even though they are independent once the peer addresses are known — each node talks to its own compute/uBridge with no shared lock, so the HTTP round-trips can overlap. - Port allocation: gather _allocate_port for both computes - NIO creation: gather both node.post calls together - Error handling: if either side fails, roll back whichever side succeeded (previously only node2-fails→cleanup-node1) before re-raising the first error Batch loading (open/import project) benefits most because links are created at high concurrency and the per-link wall-clock time is dominated by the sum of its two sequential NIO POSTs. |
||
|
YueGuobin
|
ad8bac8328
|
marker: batch topology dumps in bulk fan-out (per-def on 500+ links was ~1 minute)
Every per-link marker operation (start_marker / stop_marker / update_marker) called Project.dump() -- a full topology serialization + file write. A definition fan-out over 500 links therefore wrote the whole topology 500+ times (each blocking the event loop), which dominated the observed ~1 minute; the NIO round-trips themselves were negligible. Add a dump: bool = True parameter to the three per-link operations and inherit_marker (matching the existing dump param on Link.add_node). The bulk paths -- definition create fan-out, definition-update sync and re-fan-out, definition-delete cleanup, pause/resume, new-link inheritance -- pass dump=False and their caller dumps once after. apply_defs_to_new_link suppresses per-def dumps too: link create / project open dump once after, so opening a 500-link project with N definitions no longer does 500xN topology writes. |
||
|
YueGuobin
|
3322233658
|
marker: serial-link (WAN) support via data_link_type -> uBridge linktype
Markers now work on serial links (Cisco HDLC / PPP / Frame Relay / ATM), not just Ethernet. A marker carries a data_link_type (default DLT_EN10MB); at the uBridge boundary it becomes the 'mark ... linktype <dlt>' keyword so the BPF compiles and the pcap is written with the matching link-layer. - MarkerCreate / MarkerDefinitionCreate gain data_link_type (default DLT_EN10MB). Per-link it is create-only; definitions are updatable (a change re-fans-out). - base_node._marker_linktype() normalizes the GNS3 DLT name (strip DLT_, uppercase, None for EN10MB). Single source is SerialPort.data_link_types, so Cisco PPP -> PPP_SERIAL (50), matching the capture path -- no second mapping table. - _ubridge_add_marker_filter (generic) and the IOU marker loop append 'linktype <dlt>'. - Definition fan-out branches on link_type in inherit_marker: Ethernet is always EN10MB; a serial link uses the definition's WAN encapsulation, or is SKIPPED when none was chosen (an EN10MB pcap on serial is undecodable). One definition covers a mixed topology. - MCP marker_definition exposes data_link_type (None = not forwarded). - No uBridge rebuild on a data_link_type change -- only that one marker's filter is swapped (delete + re-add), mirroring a BPF change; reset_packet_filters preserves sibling mark filters. Requires the uBridge build with 'mark ... linktype' support. |
||
|
YueGuobin
|
75f278228b
|
marker: drop deleted marker from port NIO cache to stop empty pcap on restart
Deleting a marker while its node was stopped, then starting the node, recreated
an empty pcap. Root cause: delete_marker_capture removed the uBridge filter and
the pcap file but not the marker spec cached on the port NIO (nio.markers) —
the data source _ubridge_apply_markers reads on node start. The stale spec
reinstalled the marker when uBridge came up.
This was a regression from switching stop_marker off update() (which re-sent
the NIO and implicitly refreshed nio.markers) to the fine-grained
node.delete(/markers/{name}) path.
Fix: make the delete port-aware so the compute can locate the NIO. The DELETE
marker route becomes /adapters/{a}/ports/{p}/markers/{name} across all six
node types; the handler resolves the NIO via get_nio and passes it to
delete_marker_capture, which now pops the marker from nio.markers. get_nio
works regardless of uBridge state, so the stopped-node case is covered. The
controller's stop_marker targets the capture side's adapter/port.
|
||
|
YueGuobin
|
1a0ce51f38
|
marker: validate def BPF once, skip re-validation on inherited fan-out
A marker definition validated its BPF N times — once per link in the fan-out (start_marker runs validate_bpf_syntax on every copy), spawning one tcpdump -d subprocess per link for the same expression. Definitions did not validate BPF at all; only direction was checked. Move the single validation point to the definition layer (create/update), and validate each definition's BPF on project load (dropping any that have gone invalid, like private markers). The inherited fan-out (start_marker) and def sync (update_marker) now skip validate_bpf_syntax for inherited copies, since the BPF comes from an already-validated definition. Private per-link markers still validate inline as before. uBridge still runs pcap_compile at install, so an invalid expression can never slip through. Creating a definition over N links now runs one tcpdump instead of N. |
||
|
YueGuobin
|
caec71aa71
|
marker: fine-grained filter ops, clean pcap on remove
Deleting or updating a marker no longer triggers a full NIO reapply
(reset_packet_filters + re-add), which closed/reopened every sibling
marker's pcap via uBridge. Instead operate on single filters:
- stop_marker: bridge delete_packet_filter + unlink the pcap (works with
the node stopped; filter removal is skipped, the file is still deleted).
- update_marker: bpf/tag/direction → rebuild just that filter (delete + add);
enabled → instant toggle; color/highlight_duration → stored only.
- compute delete_marker_capture / rebuild_marker_filter + per-node routes
(DELETE /markers/{name}, PUT /markers/{name}/rebuild) + MarkerRebuild schema.
IOU overrides _ubridge_delete_marker_filter for iol_bridge; rebuild reuses
the already-overridden add/delete/set, so IOU needs no rebuild override.
|
||
|
YueGuobin
|
1eeee024bd
|
marker: rework pause/resume from project-wide to per-definition
The project-wide mute (POST /markers/pause|resume + _markers_paused) paused
every marker with one button. The actual need is per-rule control: pause one
definition and toggle only its inherited global-{name} copies across all links.
- Drop project-level: _markers_paused (init/asdict/load/start_all), the
pause_all/resume_all_markers methods, and the /markers/pause|resume routes.
- Add per-definition: a persisted `paused` flag on each definition;
pause/resume_marker_definition fan out update_marker(enabled) to every
global-{name} copy — instant, via the existing enable_packet_filter toggle
(no NIO rebuild, pcap/emitted preserved). New links inherit a paused
definition already off (inherit_marker passes enabled=not paused).
- start_marker takes an enabled kwarg; update_marker's enabled-only short-circuit
now also covers inherited copies so def pause/resume is instant.
- Routes: POST /marker-definitions/{name}/pause|resume.
- Docs + tests updated.
|
||
|
YueGuobin
|
3d06c4e22f
|
marker: drive uBridge enabled/pause/resume in real time
Wire gns3-server to uBridge's real-time marker controls (contract
../ubridge/doc/gns3server-integration.md §3.2), in three layers:
A. enabled reaches uBridge — _markers_for_node no longer drops disabled
markers controller-side; it carries `enabled` in the NIO spec. apply
installs every marker then issues `enable_packet_filter … off` for the
disabled ones (base_node bridge / iou iol_bridge; old-ubridge errors
downgrade to a warning so a toggle can't break link create).
B. instant per-filter toggle — apply records name→bridge so a new
`_ubridge_set_marker_filter_state` can flip a running filter with
`enable_packet_filter on|off` (iou overrides for iol_bridge + bay/unit).
Each marker-capable node type gains PUT /markers/{name}; update_marker
short-circuits to it when only `enabled` changes (no NIO rebuild, no pcap
flush), falling back to reset+reapply if the route is unavailable.
C. global pause/resume — `_ubridge_marker_pause/resume` send `marker pause`
/ `marker resume` direct to the hypervisor (pause stops signal+pcap,
resume instant, sink retained). Six node-type routes add POST
/markers/pause|resume; project.pause_all/resume_all_markers fan out to
each capture node (deduped, best-effort); REST exposes
POST /projects/{id}/markers/pause|resume.
Toggling enabled and pause/resume are now both instant — only marker create
or a bpf change still go through reset+reapply.
|
||
|
YueGuobin
|
37cb9f0a9c
|
marker: support clearing direction via explicit null / "both"
direction was settable but not clearable: once a marker or project-level definition had direction=tx/rx, no update path could return it to "both directions", and the definition-sync fan-out silently kept stale values on every inherited copy. Introduce a _UNSET sentinel (link.py) distinct from None so updaters can tell "caller omitted direction" (preserve) from "caller passed None" (clear). Thread it through UDPLink.update_marker and Project.update_marker_definition; the two REST routes use Pydantic v2 model_fields_set to translate an explicit JSON null into the sentinel. MCP follows with a "both" token: link_marker / marker_definition handlers map direction="both" to a null in the REST body (tri-state: omit=preserve, tx/rx=set, both=clear), and the tool descriptions/docstrings document it. Backward compatible: omitting direction or passing tx/rx behaves exactly as before; only an explicit null / "both" clears. |
||
|
YueGuobin
|
71fa778d50
|
marker: let callers pin the capture node via capture_node_id
A marker is single-sided — only the chosen capture node's uBridge installs the mark filter — and dir=tx|rx is interpreted from that node's perspective. Until now the observer was always auto-picked (_choose_marker_side), so dir=tx meant "the auto-chosen endpoint is sending", which is unpredictable and makes the direction filter hard to render meaningfully in the Web UI. Add an optional create-only capture_node_id to MarkerCreate: when set, the marker is pinned to that endpoint's uBridge (validated as a link endpoint and a marker-capable type); when omitted, behavior is unchanged (auto-pick). The chosen id is already echoed back as capture_node_id and in MARK signals, so the UI can always render the observer regardless of who picked it. capture_node_id is create-only (changing it would silently flip the meaning of stored direction; recreate instead) and is not accepted on project-level definitions — they are link-agnostic and have no endpoints, so inherited markers keep auto-picking per link. Plumbed through REST create_marker, the MCP link_marker tool, and base Link.start_marker. update_marker does not forward it. |
||
|
YueGuobin
|
6749b872fa
|
marker: forward dir= from ubridge and add per-marker direction filter
Direction field in marker.match events
=======================================
Read ubridge dir=<tx|rx> from MARK signal datagrams and forward it
as a "dir" key in the marker.match notification event. The field is
additive -- older ubridge builds omit it and the parser leaves it null,
so consumers fall back to undirected rendering with no version coupling.
Semantics are relative to the capture node (the signal's node=<id>):
tx = capture node is sending (ingressed device-side NIO), rx = it is
receiving (ingressed link-side NIO).
Per-marker direction filter (opt-in, server-side pipeline)
==========================================================
Add a direction field to MarkerCreate and MarkerDefinitionCreate
schemas ("tx" | "rx" | null). Plumb it through the full pipeline:
Schema -> controller (start_marker/update_marker, marker_entry,
_markers_for_node, update_marker_definition sync)
-> REST/MCP handlers -> compute _ubridge_add_marker_filter +
IOU _ubridge_apply_markers -> bridge add_packet_filter dir <tx|rx>
When set, ubridge only fires the mark handler (signal + pcap) for
packets matching the chosen direction. null (default/legacy) = both
directions -- zero behavioural change for existing markers.
Docs and tests
==============
- docs/features/marker-traffic-insight.md: signal format updated,
new Direction section with NIO mapping, arrow mapping, and additive
compatibility note.
- tests/compute/marker/test_marker_manager.py: 3 new parser tests
(dir tx/rx/absent) plus existing test extended to assert dir=None.
13 files, +123/-22, 72 tests pass (zero breakage)
|
||
|
YueGuobin
|
cdc27c37a7
|
fix(marker): clean inherited markers on def delete
delete_marker_definition removed the def but left the inherited global-* copies on every link: it called stop_marker(), which rejects inherited markers (409), and the ControllerError was swallowed as a warning. The orphaned copies were in-memory only (_persist_markers filters inherited markers) so a restart hid the symptom, but during a running session they were undeletable via either the per-link or project API. Add the same inherited=True bypass that update_marker already has, and pass it from the def-delete fan-out so the copies are removed for real. |
||
|
YueGuobin
|
98bcd5eddd
|
feat(marker): add highlight_duration render hint
Per-marker UI hint (milliseconds, ge=1) for how long the Web UI keeps a marker highlighted after a match. Mirrors color: stored on the link, persisted in the topology, inherited via project-level definitions, and never sent to uBridge. Omitted = null = frontend uses its own default. The default is intentionally NOT set in the schema: MarkerCreate also backs PUT updates, so a schema default would make every partial update silently reset the value. None means "not provided" (keep existing on update / frontend decides on create). |
||
|
YueGuobin
|
179f072c33
|
fix(marker): fix inheritance hook, topology load, update sync, and asdict
Bugs found via end-to-end testing of project-level marker definitions: 1. New links didn't inherit — apply_defs_to_new_link is async but was called without await in UDPLink.create(). 2. Project load crashed — load_project passed marker_definitions to Project.__init__. Now popped in load_project and restored separately in Project.open() (it backs a read-only property). 3. PUT on a definition didn't sync to links — update_marker's guard rejected even the project-layer sync call. Added an `inherited` bypass flag used by update_marker_definition. 4. _ubridge_add_marker_filter raised re.PatternError — the name regex used (?i)(?!global) look-around, invalid in Python's re module. Dropped the prefix check there: "global-*" names are legitimate at the uBridge boundary (inherited definitions); forbidden only at the user-facing schema. 5. GET /links hid inherited markers — asdict()'s runtime branch used _persist_markers() (which filters inherited markers). Restored self._markers for the runtime branch; only the topology_dump branch filters (inherited markers are rebuilt from definitions on load). 6. Duplicate "already exists" warnings on project open — open() fanned out definitions to all links, but UDPLink.create() had already done so via its inheritance hook. Removed the redundant fan-out in open(). |
||
|
YueGuobin
|
31991fe359
|
feat(marker): add project-level marker definition inheritance
Project-level marker definitions fan out to every link (existing and new).
A definition is stored once on Project._marker_definitions; when applied to
a link the marker is named "global-{def_name}" — the "global" prefix was
pre-reserved in the schema, so inherited and per-link markers can never
collide, nor will their registry keys.
Key behavior:
- POST /projects/{pid}/marker-definitions → fan out to all existing links
- PUT /projects/{pid}/marker-definitions/{name} → sync all inherited copies
- DELETE → remove every inherited copy from every link
- New links auto-inherit all active defs (hook in UDPLink.create)
- Per-link DELETE/PUT of a "global-*" marker is rejected (409)
- Inherited markers are NOT persisted in the topology; they are re-created
from _marker_definitions on project load
- Compute side is untouched — the marker reaches uBridge via the existing
start_marker→update→NIO→ubridge pipeline
Files:
- controller/project.py — _marker_definitions + CRUD + fanout + topology load
- controller/link.py — Link.inherit_marker() + asdict() filter
- controller/udp_link.py — guards on stop/update + create() inheritance hook
- controller/topology.py — persist marker_definitions in project topology
- schemas/controller/links.py — MarkerDefinitionCreate schema
- api/routes/controller/projects.py — REST endpoints (marker-definitions)
|
||
|
YueGuobin
|
d3527c00de
|
fix(marker): extend marker support to dynamips and cloud nodes
Dynamips needed its own NIO base class extended with a `_markers` attribute and `markers` property (mirroring the existing `filters` pattern), plus marker propagation from the controller-supplied NIO settings through NIOUDP.create/update to the common destination NIO that base_node._ubridge_apply_markers reads. Cloud simply needed the missing _ubridge_apply_markers calls after _ubridge_apply_filters in its two ubridge connection methods, same pattern as the earlier docker fix. Both types are now added back to _MARKER_CAPABLE_TYPES. |
||
|
YueGuobin
|
e4be98984c
|
fix(marker): enable marker support for docker nodes
Docker's _connect_nio and adapter_update_nio_binding applied packet filters but never called _ubridge_apply_markers, so markers silently did nothing on Docker links despite docker being in the allowlist. Add the missing calls (same pattern as the IOU fix). Also narrow _MARKER_CAPABLE_TYPES to the four types that actually implement marker support — vpcs, qemu, docker, iou — removing dynamips, virtualbox, vmware, and cloud which have no marker pathway and would silently fail when selected as the capture side. |
||
|
YueGuobin
|
c62b9b0283
|
refactor(marker): converge to filter single-path model, remove dual-apply endpoints
Markers now follow exactly the same apply pattern as packet filters: state lives in Link._markers, application goes through NIO (update() -> PUT /nio -> _ubridge_apply_markers). The former immediate-apply REST endpoints (/markers/start, /markers/stop on the compute side) and the per-node start_marker/stop_marker methods are removed — they were a legacy of the original capture-inspired design and have been superseded by the NIO flow. Changes: - controller/udp_link: start_marker/stop_marker/update_marker now set _markers state + call self.update() (mirrors update_filters). Removed _marker_capture_nodes runtime dict and its helpers. - controller/project: _create_link_from_topology_data restores _markers directly from persisted data (with BPF validation, like filter reload). No long calls start_marker during load. - compute: _ubridge_apply_markers swallows BPF compile errors (warn+skip), matching _ubridge_apply_filters behaviour so a single bad expression cannot break link creation / node restart. - Removed: /markers/start,stop endpoints (6 handlers across vpcs/qemu/docker route files), node start_marker/stop_marker methods (3 VM files), _ubridge_delete_marker_filter, _marker_capture_nodes, MarkerDelete schema. Net: ~280 lines of dead code removed; marker and packet filter now share a single, unified apply path via the NIO. |
||
|
YueGuobin
|
88c03d1431
|
fix(marker): restore markers from topology on project load
_create_link_from_topology_data now restores traffic-insight markers (mirroring how filters are restored via update_filters), so markers — including their color — survive project close/reopen and server restart. Guard start_marker's uBridge POST with 'if self._created' (exactly as update_filters guards its update() call): during project load the link is not yet created, so only _markers state is recorded and the marker is applied once via the NIO flow in create()/_ubridge_apply_markers — no double application. |
||
|
YueGuobin
|
3f15ac38f6
|
feat(marker): add optional color field to marker data model
Add a user-chosen hex color (e.g. '#ff5722') to each marker, stored in Link._markers alongside bpf/tag/enabled. The color: - Is serialized with the link (asdict) and persisted in the topology, so it survives project reload and is consistent across devices. - Never reaches uBridge (compute ignores it) — it is purely a Web UI concern. - Can be updated independently of the BPF (color-only changes skip the ubridge round-trip). MarkerCreate schema gains color: Optional[str]; start_marker and update_marker carry it through; REST create/update endpoints pass it. The marker.match WebSocket event stays lean (no color) — the Web UI maps filter -> color from the link state it already holds. |
||
|
YueGuobin
|
b8f49696cc
|
fix(marker): use allowlist in _choose_marker_side, align with _get_filter_node
Replace the complex tier-based fallback (which leaked into unsupported switch types on the final tiers) with a simple allowlist of _MARKER_CAPABLE_TYPES, mirroring _get_filter_node in link.py. Also accept stopped nodes — like packet filters, markers are stored on the NIO and applied when the node starts via _ubridge_apply_markers. |
||
|
YueGuobin
|
239f42f5aa
|
fix(marker): stop auto-deleting markers on node stop in node_updated
Markers are persistent link-scoped features (like packet filters), not transient runtime features (like capture). Tearing them down on every node stop breaks stop/start restore: the NIO-based re-application in _ubridge_apply_markers works correctly on restart, but node_updated was prematurely calling stop_marker which erased _markers state and dumped an empty topology. Now markers survive node stop transparently — the ubridge filter dies with the process but is re-added from nio.markers in add_ubridge_udp_connection when the node starts again, mirroring exactly how packet filters behave. |
||
|
YueGuobin
|
7c90fa9e64
|
fix(marker): ride markers on NIO so they survive node restart and project reload
Mirror the packet-filter lifecycle: marker specs now live on the NIO (next to filters), ride in link_data from controller to compute on every NIO create/update, and are reapplied by _ubridge_apply_markers in add_ubridge_udp_connection (bridge creation / node restart) and update_ubridge_udp_connection (NIO update — following the preceding reset_packet_filters so markers survive filter changes). Changes: - NIO / NIOUDP: _markers property + asdict - schemas/compute/nios.py: UDPNIO.markers field - base_manager.create_nio: nio.markers from settings - PUT /nio routes (vpcs/qemu/docker): nio.markers update - base_node: _ubridge_apply_markers(bridge_name, nio) iterates nio.markers, computes pcap path, calls _ubridge_add_marker_filter + MarkerManager.register; called after _ubridge_apply_filters - controller udp_link: _get_node_markers + _markers_for_node (route by capture_node_id); markers in create() and update() link_data - /markers/start,stop endpoints: mirror spec onto nio.markers so the marker survives a subsequent node stop/start without a PUT - tests: add markers field to NIO data expectations This covers: - Node restart: NIO persists, add_ubridge_udp_connection re-applies - Filter update: reset wipes markers, _ubridge_apply_markers re-adds - Project reload: create() carries markers in link_data → create_nio - Immediate create: endpoint sets nio.markers immediately |
||
|
YueGuobin
|
88465e872c
|
fix(marker): add _choose_marker_side to exclude non-uBridge node types
_choose_capture_side prefers BUILTIN_NODE_TYPES which includes ethernet_switch/ethernet_hub/NAT etc. — nodes that do not use uBridge and therefore have no bridge for a marker filter to attach to. Add _choose_marker_side with a _MARKER_UNSUPPORTED_TYPES blocklist so marker capture-side selection skips NIO-only switches and picks the uBridge-backed node (VPCS/QEMU/Docker/etc.) instead. Verified end-to-end: PC1↔Switch↔PC2 links with arp markers correctly pick VPCS nodes; MARK signals flow through to WebSocket and pcaps are written. |
||
|
YueGuobin
|
d2e9823e6c
|
feat: implement traffic insight marker backend (ubridge mark filter)
Add compute-side marker subsystem that receives ubridge UDP MARK signals
and forwards them as project-scoped notifications to the web UI for
real-time traffic coloring. Matched packets are always saved to per-link
pcaps for future replay.
Key components:
- gns3server/compute/marker/: MarkerManager (singleton, UDP listener +
O(1) registry keyed by (node_id, filter_name)) + MarkerListener
(DatagramProtocol parsing MARK lines per ubridge integration contract)
- gns3server/compute/base_node.py: marker sink/node config at ubridge
startup; shared _ubridge_add_marker_filter / _ubridge_delete_marker_filter
- Per-node start_marker/stop_marker: VPCS (VPCS-{id}), QEMU
(QEMU-{id}-{adapter}), Docker (bridge{adapter})
- Compute REST /markers/start + /markers/stop (vpcs/qemu/docker route files)
- Controller Link._markers state + UDPLink.start_marker/stop_marker/
update_marker (mirror capture pattern: BPF validation, _choose_capture_side,
node.post forwarding, topology persistence)
- Controller REST GET/POST/DELETE/PUT /v3/projects/{p}/links/{l}/markers
- Config: marker_listen_host / marker_listen_port in ServerSettings
- Signal routing: creation-time registry O(1) lookup, no node-table scan;
project-scoped WS stream (not global); event payload always carries
project_id for frontend scoping
Tests: 14 unit tests (registry, listener parsing, UDP round-trip);
562 existing tests pass with zero regressions.
|
||
|
YueGuobin
|
5d0284e7be
|
Performance: accelerate project opening with parallel link creation and batch UDP port allocation
- Narrow IOU lock scope to only cover application_id allocation, allowing concurrent IOU node creation via Pool(concurrency=5) - Parallelize link creation during project.open() using Pool(concurrency=5) instead of sequential processing - Add batch UDP port allocation endpoint on compute to allocate N ports in a single HTTP call - Pre-allocate UDP ports per compute before link creation during project loading, reducing HTTP round-trips - UDPLink.create() falls back to individual port allocation if no pre-allocated port is available |
||
Guobin Yue
|
f07d21c511
|
feat(web-wireshark): add implementation plan for script-driven (#2666)
* feat(web-wireshark): add implementation plan for script-driven integration Add comprehensive implementation plan for Web Wireshark integration using script-driven approach. The plan outlines: - Background and motivation for script-based Web Wireshark integration - Core workflow from API request to WebSocket proxy connection - Management script architecture with WebWiresharkManager class - Docker container configuration and resource management - Performance analysis and optimization recommendations - Network setup and management procedures Key features include: - JWT token extraction from Authorization header - Automatic GNS3 server URL detection - Container lifecycle management per project - Xpra session isolation per link - WebSocket proxy integration through gns3server - Resource monitoring and scaling guidelines The implementation enables users to start Web Wireshark sessions via POST requests with `wireshark: true` parameter, providing a unified web-based packet capture interface. * feat(web-wireshark): implement script-driven Web Wireshark integration Add complete Web Wireshark container integration with script-driven architecture: - Create manage_wireshark.py script for Docker container management - Add LinkCapture schema with wireshark boolean field - Update Link class with wireshark and jwt_token parameters - Implement WebSocket proxy endpoint for xpra HTML5 client - Add cleanup logic in Project class for container lifecycle Key features: - Script-driven container and xpra session management - JWT token extraction from Authorization header - GNS3 URL auto-detection with Controller/Config fallback - WebSocket proxy for unified access through gns3server - Proper cleanup on project close/delete Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web-wireshark): use aiohttp instead of docker SDK Replace docker-py SDK with direct Docker HTTP API calls using aiohttp, following GNS3's existing architecture pattern in gns3server/compute/docker/. Changes: - Create DockerHTTPClient class using aiohttp + Unix socket - Implement all Docker operations as async methods - Remove dependency on docker Python SDK - Use Docker API v1.44 via /var/run/docker.sock Benefits: - No additional dependencies required - Consistent with GNS3's async architecture - Better integration with existing codebase Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web-wireshark): add dynamic Docker API version detection Implement dynamic Docker API version detection following GNS3's pattern in gns3server/compute/docker/__init__.py. Changes: - Add DOCKER_MINIMUM_API_VERSION and DOCKER_PREFERRED_API_VERSION constants - Remove hardcoded "v1.44" prefix (now using "1.44" format) - Add _check_connection() method to detect Docker daemon version - Dynamically select API version based on daemon capabilities - Add proper error handling for version mismatches Behavior: - Initialize with minimum API version (1.40) - On first connection, detect Docker daemon API version - Use preferred API version (1.44) if supported - Fall back to daemon's minimum API version if needed - Raise error if daemon version is too old Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web-wireshark): add optional --verbose logging parameter Remove hardcoded logging.basicConfig() and add --verbose parameter to control log output, following GNS3's logging pattern. Changes: - Remove logging.basicConfig(level=logging.INFO) from module level - Add --verbose/-v command line parameter - Configure logging only when --verbose is specified - Use structured log format with timestamp when verbose Behavior: - When called by GNS3: uses GNS3's logging configuration (default) - When run standalone: no output unless --verbose is specified - With --verbose: shows detailed logs with timestamps This prevents the script from interfering with GNS3's logging configuration while still allowing debug output when needed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web-wireshark): fix code quality issues from static analysis Fix issues identified by flake8 and pylint static analysis: Import and formatting fixes: - Remove unused 'time' import - Fix import order (stdlib before third-party) - Remove unnecessary f-strings without placeholders - Split long line to comply with flake8 (max line length: 127) Code structure improvements: - Remove unnecessary 'else' after 'return' - Add 'from e' to exception re-raising for better tracebacks Code quality metrics: - pylint score: 8.11/10 → 9.97/10 (+1.86) - flake8: 0 errors (max line length: 127, per CI/CD standard) - All critical issues resolved Note: R0914 (too-many-locals) warning is informational only, code remains clear and maintainable. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): export LinkCapture schema in schemas module Add LinkCapture to the schemas module exports in __init__.py to fix the AttributeError when starting the GNS3 server. Error was: AttributeError: module 'gns3server.schemas' has no attribute 'LinkCapture' This was caused by adding the LinkCapture class to controller/links.py but forgetting to export it in the schemas __init__.py. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web-wireshark): add --image parameter for custom Docker images Add optional --image parameter to allow using custom Docker images for testing, instead of requiring gns3/web-wireshark:latest. Changes: - Add --image parameter to start command (default: gns3/web-wireshark:latest) - Update get_or_create_container() to accept image parameter - Update start_wireshark_session() to accept and pass image parameter - Update TEST.md with examples of using custom images Usage: # Using default image python3 manage_wireshark.py start --project-id x --link-id y --jwt-token z # Using custom image (for testing) python3 manage_wireshark.py start --project-id x --link-id y --jwt-token z --image ubuntu:latest This makes it easier to test the script without having the official gns3/web-wireshark image available. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web-wireshark): add configurable resource limits and increase session capacity Add configurable resource parameters and increase session support from 10 to 100. Resource parameters: - --memory: Memory limit (default: 2g) - --memory-swap: Memory swap limit (default: same as memory) - --cpus: CPU cores (default: 1.0) - --pids-limit: Process limit (default: 1000) Other improvements: - Fix CPU quota calculation: 1000000 microseconds = 1.0 CPU core (was incorrectly 100000 = 0.1 CPU core) - Add health check: xpra list command - Add log configuration: json-file, max-size=10m, max-file=3 - Increase session capacity: 10 → 100 concurrent sessions - Port range: 12300-12399 (was 12300-12309) - Display range: :100-:199 (was :100-:109) This matches the docker run command parameters and provides better resource management and monitoring capabilities. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): update UDPLink.start_capture() to accept new parameters Update UDPLink.start_capture() to accept the new wireshark and jwt_token parameters and pass them to the parent Link class. This fixes the TypeError when starting capture on UDP links: TypeError: UDPLink.start_capture() got an unexpected keyword argument 'wireshark' The UDPLink class overrides start_capture() but didn't include the new parameters added for Web Wireshark support. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: add project memory infrastructure and JWT token flow documentation - Add memory skill for recording project knowledge - Document JWT token flow in Web Wireshark integration - Update .gitignore to track skills and memory directories Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): fix Docker API URL format and use docker exec CLI - Add "v" prefix to Docker API URL (http://docker/v{version}/...) - Remove unused exec_create/exec_start API methods - Fix Healthcheck.Test format to ["CMD-SHELL", "command"] - Use docker exec CLI instead of Docker API for command execution - This aligns with GNS3 docker_vm pattern Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): improve xpra session management and error handling - Check and clean up existing sessions before starting new xpra session - Add verification that xpra session started successfully - Use docker exec CLI consistently instead of Docker API - Improve error logging and diagnostics Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(web-wireshark): add timeout handling and container health checks - Add REQUEST_TIMEOUT constant for Docker HTTP API requests - Add asyncio.timeout wrapper for Docker API calls to prevent hanging - Add _is_container_healthy() to check container responsiveness - Add _exec_in_container() helper with timeout support - Improve unhealthy container handling with force remove - Add detailed logging for container health state Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web-wireshark): split monolithic file into modules Split manage_wireshark.py into: - docker_client.py: Docker HTTP API client - manager.py: WebWiresharkManager with session management - manage_wireshark.py: CLI entry point This improves code organization and maintainability. Keep all timeout handling and health check logic. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): correct NanoCpus calculation to use nanoseconds NanoCpus in Docker API requires nanoseconds (1 CPU = 1000000000), not the previous incorrect multiplier of 100000. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): fix xpra command quoting and session verification - Add quotes around --xvfb parameter value to preserve spaces - Fix session verification to check display number instead of session name (xpra list shows "LIVE session at :185" not session name) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): run wireshark in background with & Wireshark with curl pipeline runs continuously, so it must run in background to avoid blocking. Also reduced timeout since we don't wait for completion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web-wireshark): add deterministic hash for display/port allocation Use MD5-based hash instead of Python's hash() which is randomized across process restarts. This ensures display and port numbers are stable when the GNS3 server restarts. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: extract link_id_to_port to shared utils module Move deterministic hash functions to gns3server/utils/port_allocator.py to avoid code duplication and ensure consistent algorithm across manager.py and links.py. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(links): move import to top of file Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(web-wireshark): use correct Config access pattern like gns3-copilot The Server config is an object with .protocol.value, .host, .port attributes, not a dict. Fixes URL detection failing and falling back to 127.0.0.1:3080 which doesn't work from inside containers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add UUID validation utility and use in manage_wireshark Create gns3server/utils/uuid_validator.py with validate_uuid function to validate UUID format and catch typos early with clear error messages. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use argparse.ArgumentTypeError for proper error message display Previously ValueError was used which only showed 'invalid validate_uuid value'. Now shows the full helpful message about expected format. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(web-wireshark): add --dpi=96 to xpra start parameters Set standard DPI for better display scaling in web Wireshark. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: update TEST.md with correct project-id format Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(web-wireshark): hide xpra shutdown menu via XPRA_CLIENT_CAN_SHUTDOWN Prevents users from accidentally shutting down the xpra server through the HTML5 client menu. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(memory): add xpra-html5-client configuration reference Document xpra HTML5 client configuration including: - URL parameters for toolbar menu control - default-settings.txt parameters (Features, Connection, Advanced) - Server-controlled submenu items - XPRA_CLIENT_CAN_SHUTDOWN environment variable - Background image customization Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(web-wireshark): remove xpra stop cleanup to avoid zombie processes xpra stop can leave zombie processes and timeout. Instead, let xpra start reuse the display directly (it will overwrite existing session). Also added --file-transfer=no, --printing=no, --sound=no to disable unneeded features. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(web-wireshark): add fullscreen_button=false to default-settings.txt Also remove --file-transfer/--printing/--sound from command line since xpra doesn't support these options. Configure in default-settings.txt instead. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(web-wireshark): update xpra background with GNS3 branding - Replace default xpra background with GNS3 icon and modern gradient - Update CSS to use SVG background image with cover sizing - Change background color to gradient from #021d3a to linear gradient - Improves visual integration with GNS3 web interface * style: add GPLv3 headers with copyright to web_wireshark modules Co-Authored-By: YueGuobin <yueguobin@gmail.com> * feat(docker): use Alibaba Cloud Debian mirror for faster builds in China Use mirrors.aliyun.com for Debian packages with correct paths for both main repo (/debian) and security repo (/debian-security). Co-Authored-By: YueGuobin <yueguobin@gmail.com> * feat(web-wireshark): update xpra background styling and echo command - Replace `echo -e` with `printf` for better POSIX compatibility in Dockerfile - Update xpra HTML5 client background to use GNS3 icon with light gradient - Change background color from dark blue to light gray for improved visibility * fix(web-wireshark): use pkill to stop xpra sessions Replace 'xpra stop :{display}' with 'pkill -f "xpra.*:{display}"' for more reliable session termination. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web-wireshark): start Wireshark in fullscreen mode Add --fullscreen flag to Wireshark startup command to prevent window decorations and improve user experience. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): use carrier-grade NAT subnet to avoid conflicts Change Docker network subnet from 172.28.0.0/16 to 100.64.1.0/24 to avoid conflicts with common private networks. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs(web-wireshark): update subnet to /22 for 1000+ projects Change Docker network subnet from 100.64.1.0/24 to 100.64.0.0/22 to support 1000+ projects (1022 available IPs). Update all documentation to reflect the new subnet. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): replace CGNAT subnet with configurable private subnet Replace the CGNAT address block (100.64.0.0/22) with a standard private subnet (172.31.0.0/22) to avoid network access issues. Many networks and ISPs block or cannot route CGNAT addresses. Changes: - Add WebWiresharkSettings to config schema with configurable subnet - Read network_subnet from config, default to 172.31.0.0/22 - Add Web Wireshark configuration section to sample config Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): comment out dpi setting Allow xpra to use default DPI settings for better display scaling. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): set DPI in Xvfb to prevent scaling warnings Add -dpi 96 to Xvfb startup command to match xpra's expected DPI and avoid "scaling problems" warnings. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): remove custom Xvfb to use default Xorg-dummy Remove --xvfb parameter to let xpra use the default Xorg-dummy driver, which properly handles DPI changes when resize-display is enabled and prevents DPI mismatch warnings. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * revert(web-wireshark): restore Xvfb configuration Restore the Xvfb display server configuration for xpra sessions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web-wireshark): enhance xpra HTML5 default settings Update xpra HTML5 configuration to disable additional unused features: - audio, keyboard, mediasource, aurora, http-stream - Improve readability using heredoc format instead of printf Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(api): add RBAC authentication to web wireshark WebSocket endpoint - Add `has_privilege_on_websocket` dependency to enforce Link.Capture privilege - Include current_user parameter in endpoint for user identification and logging - Update endpoint documentation to reflect token requirement and privilege - Add test example WebSocket URL in TEST.md for reference * refactor(web-wireshark): add generic WebSocket proxy and disable xpra HTML - Add websocket_to_websocket.py utility for binary WebSocket proxy - Disable xpra HTML server with --html=no flag - Update manager return value: url -> ws_url - Simplify WebSocket proxy implementation in links.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web-wireshark): remove unnecessary sleep delays Remove 2-second sleep delays after container start and xpra initialization. Health checks and verification happen immediately, improving startup time. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(agent): prevent blocking during project close - Add 5s timeout for AgentService.checkpointer_conn.close() - Optimize stop_all_sessions: single command instead of 100 docker exec calls - Prevents indefinite hang when closing projects Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): return errors to client on startup failure - Throw ControllerError when Web Wireshark startup fails - Include detailed error messages from script stderr - Support both ws_url and url response formats - Client now receives proper error response instead of success Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web-wireshark): parallelize startup and remove unnecessary waits - Parallel execution: container info query + xpra start - Remove xpra list verification (unnecessary) - Fire-and-forget Wireshark startup (no timeout wait) - Startup time reduced from ~7s to ~1s Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web-wireshark): add stop-container and delete-container commands - Add stop-container command to stop container on project close - Add delete-container command to delete container on project delete - Enables proper container lifecycle management - Containers are now stopped (not deleted) when project closes - Containers are deleted when project is deleted Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(project): simplify close flow by removing redundant xpra session cleanup Since docker stop with timeout=0 already force-kills all processes in the container, explicitly stopping xpra sessions before stopping the container is unnecessary. Remove the _cleanup_web_wireshark_xpra_sessions() call to reduce subprocess overhead and simplify the close flow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * perf(docker): force kill containers on stop by default Change docker stop timeout from 10 seconds to 0 (immediate SIGKILL). Web Wireshark containers don't need graceful shutdown since they have no persistent state. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(web-wireshark): enable WebSocket protocol for xpra connection - Change xpra bind from --bind-tcp to --bind-ws for WebSocket support - Enable HTML client with --html=on for xpra WebSocket server - Pass binary subprotocol when connecting to xpra container The xpra WebSocket server requires the 'binary' subprotocol during handshake. Without it, the server returns 403 with message: "client does not support 'binary' protocol". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(web-wireshark): add WebSocket subprotocol negotiation support Fix WebSocket proxy for xpra container by implementing proper subprotocol negotiation. The xpra client requires the server to respond with the negotiated subprotocol (binary) in the WebSocket handshake response. Changes: - Modified get_current_active_user_from_websocket to extract client's requested subprotocols from Sec-WebSocket-Protocol header - Updated websocket.accept() call to include negotiated subprotocol - Enhanced websocket_proxy to support subprotocol parameter - Improved logging for WebSocket connection debugging This fixes the issue where xpra clients would immediately disconnect after connection due to missing subprotocol in server response. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore(web-wireshark): remove verbose debug logging Remove excessive debug logs from WebSocket proxy implementation while keeping essential logs for troubleshooting: - Keep: subprotocol negotiation, connection establishment, errors - Remove: verbose step-by-step debugging information Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web-wireshark): translate Chinese comments and docs to English - Translate all Chinese comments in project.py, link.py, and links.py to English - Convert TEST.md and maunal-test.md documentation to English - Remove obsolete IMPLEMENTATION_PLAN.md Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore(docker): lock xpra version to 6.4.3 for reproducible builds Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore(docker): improve Dockerfile with TZ, no-install-recommends, and labels - Add TZ=UTC timezone setting - Use --no-install-recommends to reduce image size - Add LABEL metadata for maintainer and description Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(docs): rename and consolidate documentation - Rename TEST.md to WEB_WIRESHARK.md for clearer naming - Merge maunal-test.md content into main document under "Manual Testing" section - Remove obsolete gns3_icon_black.svg Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(docker): add xpra-x11 package and remove --no-install-recommends - Add xpra-x11 package required for seamless mode - Remove --no-install-recommends to ensure all recommended packages are installed Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(api): add capture file download endpoint Add GET /{link_id}/capture/file endpoint to download PCAP capture files. Supports downloading while capture is active (streaming). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(web_wireshark): improve session cleanup and add known issues - Add cleanup of existing processes before starting xpra session to prevent "another window manager seems to be running" errors - Stop all associated processes (xpra, wireshark, Xvfb) when stopping sessions - Document known issues in WEB_WIRESHARK.md including JWT token security, duplicate code, and other implementation concerns Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(web_wireshark): add restart wireshark API endpoint - Add POST /links/{link_id}/capture/wireshark/restart endpoint - Add restart command to manage_wireshark.py CLI - Add _restart_web_wireshark method in link.py controller - Restart simply calls start_wireshark_session which handles cleanup This allows users to recover after accidentally closing the Wireshark window without having to stop and restart the entire capture. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add Web Wireshark business process documentation Document the Web Wireshark feature including architecture diagrams, business processes for capture start/stop, container lifecycle, WebSocket connection flow, and session management. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(web_wireshark): fix container network access and improve session cleanup - Fix GNS3 server URL detection to use container gateway IP (172.31.0.1) instead of localhost/127.0.0.1/0.0.0.0 which don't work from containers - Add import socket module for gateway IP conversion - Move urllib.parse import to file header (code style improvement) - Add logging for Web Wireshark startup (capture stream URL, display, etc.) - Fix X lock file cleanup to prevent "Server is already active" errors - Use exec pkill to reduce zombie processes from docker exec bash - Add _cleanup_x_lock() method to remove X lock files after stopping sessions - Update link.py to always log subprocess stderr for debugging Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web_wireshark): avoid zombie pkill processes by using Docker API Replace pkill commands with Docker API-based process killing to prevent zombie process accumulation in Web Wireshark containers. Changes: - Add DockerHTTPClient.list_processes() method to query container processes - Rewrite _kill_process_tree() to use Docker API instead of pkill - Unify all process cleanup to use _kill_process_tree() method - Add logging for killed processes (count and PIDs) - Include fallback to pkill if Docker API fails This prevents the accumulation of zombie pkill processes that occurred when using bash -c "pkill -9 -f pattern" commands. Testing: Started and stopped multiple capture sessions, verified no new pkill zombie processes are created during cleanup. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web_wireshark): use docker-init and cleanup socket files Enable Docker init system (tini) as PID 1 and clean up xpra socket files when stopping sessions to prevent zombie processes and leftover sockets. Changes: - Add "Init": True to container host_config to use docker-init (tini) - Extend _cleanup_x_lock() to remove xpra socket files * /run/user/1000/xpra/{display}/socket * /run/user/1000/xpra/*-{display} * /home/gns3/.xpra/*-{display} Benefits: - docker-init (tini) automatically reaps orphan processes, eliminating zombie process accumulation (tested: 53 zombies -> 0 zombies) - Socket cleanup prevents xpra list from showing UNKNOWN sessions - Cleaner container state after stopping sessions Testing: - Started and stopped 3 capture sessions - Verified 0 zombie processes after stopping - Verified xpra list shows "No xpra sessions found" (no UNKNOWN) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs(web_wireshark): add comprehensive CLI script documentation Add detailed module docstring to manage_wireshark.py explaining: - Script purpose: CLI tool for manual management, debugging, and testing - Important note: Use WebWiresharkManager directly for programmatic access - Usage examples for all commands (start, stop, restart, stop-all, delete) - Available commands list with descriptions - Output format specification (JSON to stdout/stderr) - Help information for getting command-specific usage This clarifies that the script is intended as a CLI utility, not for subprocess calls from within GNS3 server code. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web_wireshark): use direct API calls instead of subprocess Replace subprocess calls to manage_wireshark.py with direct API calls to WebWiresharkManager, eliminating subprocess overhead and JSON parsing. Changes: - Add import for WebWiresharkManager - Refactor _start_web_wireshark() to use direct API call - Refactor _stop_web_wireshark() to use direct API call - Refactor _restart_web_wireshark() to use direct API call Benefits: - Code reduction: 78 lines (-68%) - Better performance: No subprocess creation overhead - Better logging: Manager logs directly to GNS3 logging system - Simpler error handling: Direct exceptions instead of return codes - No JSON parsing: Direct Python objects - Resource cleanup: Added finally blocks to ensure manager.close() Testing: - All Web Wireshark operations work correctly - Logs now appear in GNS3 server logs instead of stderr - Error handling improved with proper exception propagation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web_wireshark): resolve circular import with delayed imports Fix circular import error by moving WebWiresharkManager import from module level to function level (delayed import). Circular dependency was: link.py → WebWiresharkManager → Controller → link.py Solution: - Remove top-level import of WebWiresharkManager - Add delayed imports in each method that uses it: * _start_web_wireshark() * _stop_web_wireshark() * _restart_web_wireshark() This allows the controller module to fully initialize before importing WebWiresharkManager, breaking the circular dependency. Error before fix: ImportError: cannot import name 'Controller' from partially initialized module 'gns3server.controller' (most likely due to a circular import) Testing: - Successfully imports Link module - GNS3 server starts without import errors Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor: move imports to module level and remove Controller dependency Remove circular dependency between link.py and manager.py by: 1. Remove Controller import from manager.py - Manager no longer imports Controller - URL detection now relies on Config or default only - link.py passes capture_stream_url to manager 2. Move WebWiresharkManager import to module level in link.py - No longer need delayed imports since no circular dependency - Clean, standard Python import pattern 3. Remove redundant Config import in ensure_network() - Config was already imported at module level Changes: - manager.py: -24 lines (removed _get_gns3_url_from_controller and redundant import) - link.py: -30 lines net reduction after moving import to top Testing: - All imports work without circular dependency errors - GNS3 server starts successfully Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(docker_client): use text response for list_processes API Docker API /containers/{id}/top returns plain text, not JSON. The generic _request method auto-parses JSON which caused: "'dict' object has no attribute 'strip'" Fix by using session.get() directly with response.text() instead of the generic _request method. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web_wireshark): use correct capture stream URL endpoint Remove passing capture_stream_url from link.py to manager.py. Manager.py auto-detects URL using correct endpoint: /v3/projects/{project_id}/links/{link_id}/capture/stream This endpoint supports JWT authentication, unlike the compute URL (/v3/compute/...) which requires compute credentials. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web_wireshark): use container-local PIDs for process killing - Docker API returns host PIDs, not container PIDs - use pgrep inside container to get correct PIDs for kill - Fix list_processes to parse Docker API JSON response correctly - Remove unnecessary exec prefix in shell commands - Add logging for debugging process killing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * perf(web_wireshark): optimize process killing with single pgrep call Replace sequential _kill_process_tree calls with a new _kill_process_tree_batch method that combines all patterns into a single regex. This reduces docker exec calls from 8 to 1, improving performance from ~8 seconds to ~0.9 seconds. Changes: - Add _kill_process_tree_batch() method with combined regex pattern matching - Update start_wireshark_session() to use batch cleanup - Update stop_wireshark_session() to use batch cleanup - Update stop_all_sessions() to use batch cleanup Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web_wireshark): disable HTML5 client to improve xpra startup speed Change --html=on to --html=off to disable the xpra HTML5 client interface. This reduces xpra startup time from ~6.4s to ~2.9s (55% improvement) while keeping the WebSocket server functional for browser connections. The HTML5 client is not needed as we only require the WebSocket endpoint for remote display forwarding. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web_wireshark): optimize container health check using Docker built-in status Replace manual docker exec ping with Docker's built-in health check status to reduce container startup latency by ~1 second. Changes: - Use container["Health"]["Status"] instead of manual ping check - Skip health check for containers with "healthy" or "starting" status - Only verify containers with "unhealthy" status - Keep manual health check for newly started containers This reduces the container verification time from ~1s to near-zero for healthy running containers while maintaining safety for unhealthy ones. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web_wireshark): get gateway IP from Docker API instead of container exec Replace slow docker exec method with fast Docker Network API call to get gateway IP. This reduces gateway detection from ~850ms to ~1ms (1000x faster). Changes: - Use docker.get_network() API to get gateway from IPAM config - Keep container exec methods as fallback if Docker API fails - Remove container_id requirement (not needed for Docker API method) The Docker network gateway is shared by all containers in the network, so querying the network is more efficient than querying each container. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web_wireshark): remove unreliable nameserver fallback for gateway detection Remove the fallback method that reads /etc/resolv.conf nameserver as gateway. This method is unreliable because: - nameserver is not necessarily the gateway (could be upstream DNS) - Many systems use 127.0.0.53 (systemd-resolved) or 127.0.0.1 - Even when not local, it could be public DNS (8.8.8.8, 1.1.1.1) The current fallback strategy is sufficient: 1. Docker Network API (fast, reliable) 2. /proc/net/route (standard gateway detection) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web_wireshark): use host perspective for process killing (37x faster) Replace docker exec with host perspective process management for killing container processes. This reduces process cleanup from ~850ms to ~23ms. Key optimization: - Get container init PID using docker inspect (fast) - Use pgrep -P <init_pid> to find child processes from host perspective - Kill processes directly using host PID (no docker exec needed) Performance improvements: - Process finding: 850ms → 23ms (37x faster) - Process killing: 850ms → <1ms (850x faster) - File checking: 850ms → 0.003ms (280,000x faster) Fallback to docker exec method if host perspective fails. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web_wireshark): use host perspective for file cleanup (891x faster) Replace docker exec with direct filesystem access from host perspective for cleaning up X lock files and xpra sockets. This reduces cleanup time from ~890ms to ~1ms (891x faster). Key optimization: - Access /proc/<container_pid>/root/ directly from host - Use os.remove() and glob.glob() instead of docker exec rm -f - Fallback to docker exec if host perspective fails This complements the earlier optimization for process killing, further reducing startup time. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Revert "perf(web_wireshark): use host perspective for file cleanup (891x faster)" * fix(web_wireshark): recursively kill all descendant processes to prevent orphans Previous implementation only killed direct children of container init using `pgrep -P <init_pid>`, but xpra spawns child processes (Xvfb, pulseaudio, ibus-daemon) that become grandchildren and were not being terminated. This fix walks the entire process tree recursively to find and kill all descendant processes, ensuring complete cleanup without orphans. Performance: ~20-40ms (only 10-20ms slower than previous 23ms, but ensures thorough cleanup). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs(web_wireshark): add comprehensive startup/shutdown performance metrics Add detailed performance characteristics section documenting: - Startup performance breakdown (before/after optimization) - Shutdown performance breakdown (before/after optimization) - First startup vs subsequent startup comparison - Measured production data - Key optimization techniques Performance improvements: 67% faster startup, 78% faster shutdown, zero orphan processes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web_wireshark): ensure container stops on project close Remove dependency on _web_wireshark_container_created flag which gets reset when project is reloaded, causing container to not stop on project close. Now directly attempts to stop container on every project close. If container doesn't exist, the script handles it gracefully. This is simpler and more reliable than maintaining a flag. Also removed unnecessary _cleanup_web_wireshark_xpra_sessions() call since stopping the container automatically terminates all xpra sessions inside. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web_wireshark): ensure container deletion on project delete Remove dependency on _web_wireshark_container_created flag from _cleanup_web_wireshark_container() to ensure container is deleted when project is deleted, even if project was reloaded and flag was reset. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web_wireshark): remove unused _web_wireshark_container_created flag This flag was set but never read for any conditionals since we changed the stop/delete methods to directly attempt the operation. It's purely redundant code that adds confusion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web_wireshark): skip unnecessary cleanup using fast host perspective check Add _check_residuals_exist() function that uses host perspective (~20ms) to check if residual processes or socket files exist before cleanup. Returns (has_process_residuals, has_socket_residuals) tuple to allow selective cleanup - if only sockets need cleaning, skip the slower process tree killing. For new containers with no residuals, this saves ~3 seconds of unnecessary docker exec calls. Socket cleanup always uses docker exec for safety (avoiding accidental host filesystem deletion from /proc/<pid>/root/ path errors). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web_wireshark): increase container exec timeout from 5s to 10s xpra start with Xvfb initialization can take longer than 5 seconds in Docker containers, causing timeout failures. Increasing timeout to 10 seconds to allow sufficient startup time. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web_wireshark): optimize cleanup and document docker exec limitations - Combine X lock and xpra socket cleanup into single docker exec call Reduces exec calls from 2 to 1 per stop operation, improving performance when stopping multiple capture sessions. - Document Docker exec performance limitations with test data showing parallel exec is 42% slower than serial due to Docker daemon's internal queuing mechanism. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(links): prevent AssertionError in stream_pcap during rapid stop_capture Fix race condition where stream_pcap checks link.capturing (True) but link.capture_node becomes None before accessing link.compute. This happens when stop_capture() is called and rapidly cleans up _capture_node while WebSocket requests are still processing. Now check both capturing and capture_node to ensure capture is still active. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(links): prevent AssertionError in stream_pcap during rapid stop_capture Fix race condition where stream_pcap checks link.capturing (True) but link.capture_node becomes None before accessing link.compute. This happens when stop_capture() is called and rapidly cleans up _capture_node while WebSocket requests are still processing. Now check both capturing and capture_node to ensure capture is still active, and log at DEBUG level since this is expected behavior during rapid stop. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web_wireshark): load container config from settings - Load memory, cpus, and pids_limit from WebWireshark config section - Apply config only on container creation (_start_web_wireshark) - Skip config on restart (_restart_web_wireshark) as container exists Users can now configure container resources in gns3_server.conf: [WebWireshark] memory = 4g cpus = 2.0 pids_limit = 2000 If not configured, defaults (2g, 1.0, 1000) are used. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web_wireshark): add container statistics to /v3/statistics endpoint Add Web Wireshark container monitoring to the statistics API for better visibility into packet capture sessions. Changes: - Create new stats.py module with collect_webwireshark_stats() * Collects container info (status, project, active sessions) * Gets resource limits (memory, CPU, PIDs) from container config * Gets resource usage via docker stats (memory, CPU, PIDs) * Properly closes aiohttp connections to avoid leaks - Update /v3/statistics endpoint to include webwireshark data * total_containers: Total number of Web Wireshark containers * running_containers: Number of containers currently running * active_sessions: Total active capture sessions * containers: Array with per-container details - Update statistics-api.md documentation * Change URL from /v1/statistics to /v3/statistics * Add webwireshark field descriptions and examples * Add dashboard integration for Web Wireshark monitoring Example response: { "webwireshark": { "total_containers": 1, "running_containers": 1, "active_sessions": 2, "containers": [{ "project_id": "...", "project_name": "test", "container_id": "6edc9029bac0", "status": "running", "memory_limit": "4.0 GB", "cpu_limit": "4.0", "pids_limit": 4000, "memory": "535.7MiB / 4GiB", "cpu": "0.29%", "pids": 124 }] } } Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(project): use WebWiresharkManager directly instead of subprocess Replace subprocess calls to manage_wireshark.py with direct WebWiresharkManager method calls in project.py. This eliminates unnecessary process overhead and maintains consistent usage pattern with link.py. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(logging): use %-formatting instead of f-strings in WebWireshark methods Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(web_wireshark): use info level log when container not found Change log level from warning to info when container doesn't exist during stop/delete operations, and fix f-string without placeholders. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(logging): lower log level for client disconnection Change log level from warning to debug when client possibly disconnects, as this is normal behavior when users close browser tabs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(linting): resolve ruff warnings and errors - Remove unused variable proc (fire-and-forget subprocess) - Remove duplicate nodes property definition - Remove extraneous f-prefix from f-strings without placeholders - Change bare except to except Exception - Remove unused imports (sys, asyncio, json) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(web_wireshark): improve error message when Docker image not found When creating a container fails due to missing image, provide helpful error message with docker pull command and local build instructions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(schema): add wireshark field to Link schema Add wireshark boolean field to Link Pydantic schema so it is included in API responses when Web Wireshark session is active. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(link): add wireshark state tracking Track whether Web Wireshark is running on a link by adding _wireshark boolean property, updated on start/stop operations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(chat): add session abort functionality Add ability to abort streaming chat sessions via REST API. Changes: - Add abort flag to MessagesState for tracking abort requests - Add session_id to state for abort event correlation - Add _abort_flags dict and check/set/clear functions in gns3_copilot.py - Add abort_handler_node to generate aborted tool messages - Modify should_continue to route to abort_handler_node when aborting - Add abort_session() method in AgentService - Add POST /sessions/{session_id}/abort API endpoint - Clear abort flag at stream start When abort is triggered during streaming: - If LLM has tool_calls pending, abort_handler_node generates aborted tool messages to maintain checkpoint consistency - Prevents "insufficient tool messages" error on resume Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(chat): send tool_end event when stream is aborted When a stream is aborted during tool execution, yield proper tool_end events for aborted tools instead of a generic abort event. This maintains compatibility with the frontend's expected tool_start/tool_end flow. Changes: - Add stream_aborted tracking flag - After stream loop, check abort flag and yield tool_end events - Add "abort" to ChatResponse type enum (unused but available) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(web_wireshark): add gns3-wireshark-setup command for Docker image setup Add a new entry point script that allows users to setup the Web Wireshark Docker image with a single command: pip install gns3-server && gns3-wireshark-setup The script will: 1. Try to pull gns3/web-wireshark:latest from Docker Hub 2. If pull fails, build the image locally using the included Dockerfile 3. Show raw docker pull/build output for full visibility Files changed: - Add setup_wireshark_image.py (new entry point script) - Update pyproject.toml (add gns3-wireshark-setup entry point) - Update documentation (README.md, WEB_WIRESHARK.md, web-wireshark-business-process.md) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(appliance): add tags field support for appliances and templates - Add tags field to ApplianceV1_6 and ApplianceV8 schemas - Add tags propagation from appliance config to template - Simplify VPCS builtin template tags Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix test in test_link.py * Fix link.py after failed tests * feat(copilot): add device skills system for LLM context injection - Add skills module with registry and DeviceSkillsTool - Skills organized by vendor/series for easy extensibility - Support device_type, category (device/protocol/feature), operation (config/diagnosis) - First implementation: VPCS skill (gns3_vpcs_telnet) - Skills tool integrated into teaching_assistant and lab_automation modes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(vpcs): change prompt detection log from warning to debug The "VPCS prompt not clearly detected" message is not an error, connection works fine. Change to debug level to reduce noise. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(memory): add uBridge permission issue documentation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(copilot): include link_id in links_summary output Return link_id in the links_summary method so that the AI can identify which link to analyze when using the packet capture tool. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(copilot): add packet capture analysis tool Add PacketCaptureTool that allows AI to analyze packets from an active GNS3 capture. The tool downloads capture files from the GNS3 server and runs tshark analysis to help users understand network traffic. Features: - Download capture file from /capture/file endpoint - Run tshark with custom arguments for flexible analysis - Support analyzing specific packets (e.g., "explain packet #42") - Support protocol statistics, traffic analysis, and expert info Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(copilot): use shlex.split to properly parse quoted tshark arguments tshark_args may contain quoted filter expressions like "-Y \"frame.number == 24\"". Using plain str.split() breaks these quoted arguments, causing tshark warnings about conflicting display filters. Use shlex.split() to correctly parse quoted arguments. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(copilot): simplify PacketCaptureTool to accept only packet_number Instead of exposing complex tshark arguments to the LLM, the tool now accepts a simple packet_number parameter and internally constructs the tshark command with verbose output. This prevents the LLM from generating incorrect tshark parameters while still providing detailed packet analysis. Changes: - Remove tshark_args and max_lines parameters - Accept only packet_number as input - Internal command: tshark -r <file> -Y "frame.number == N" -V - Returns complete packet structure without line limits Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add CC BY-SA 4.0 license headers to documentation files Add SPDX license headers referencing docs/LICENSE file to all markdown and HTML documentation files in the docs directory. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(copilot): add topology planner skill Add new skill for automatic network lab topology planning: - IOU as default image, 10.0.0.0/8 IP range, max 10 nodes - Node naming convention: R/S/PC + number - 7-step workflow for create/rename/link/start/verify/config - IP allocation rules and troubleshooting guide Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(copilot): add node positioning rules to topology planner skill Add grid-based positioning for GNS3 nodes: - Minimum 250px distance between nodes - Grid layout: 4 columns, 300x250px spacing - Formula: x = -400 + col * 300, y = -200 + row * 250 - Update workflow params_required to include x, y coordinates Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(copilot): remove unused helper functions from topology planner skill Delete Python helper functions that cannot be serialized to JSON: - calculate_node_positions() - get_position_for_node() - allocate_subnet() - allocate_ip() - TOPOLOGY_CREATION_STEPS (duplicate of workflow in skill dict) LLM should use formula descriptions in skill to calculate positions/IPs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(copilot): add topology_planner workflow to lab automation prompt Add device_skills tool to AVAILABLE TOOLS table. Add TOPOLOGY PLANNING WORKFLOW section explaining: - How to query topology_planner skill - Default conventions (IOU, 10.0.0.0/8, naming rules, position formula) - 7-step workflow for topology creation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(copilot): support name parameter in gns3_create_node tool - Add optional 'name' field to node creation, allowing direct naming - Update Node constructor to pass name parameter - Remove separate rename step (step_3) from topology planner workflow - Renumber workflow steps: 1-2-3-4-5-6 (skipping old step_3 rename) This eliminates the need for a separate gns3_update_node_name_tool call when creating nodes with predefined names. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(copilot): update topology workflow to 6 steps Remove rename step since name can be set directly in create_gns3_node. Workflow changed from 7 steps to 6 steps. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(copilot): replace grid positioning with topology-based layout Redesign node positioning rules to follow classic network topologies: - Star: hub at center, spokes radiating outward - Ring: nodes in circular arrangement - Bus: linear chain of nodes - Mesh: grid pattern for interconnected nodes - Hierarchical: three-tier (Core → Distribution → Access) - Linear P2P: point-to-point WAN links in a line Remove old grid formula. Update prompt and workflow to reflect new approach. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(copilot): add note about combining topology types LLM can mix topology types as needed, e.g., star + linear_p2p for WAN segments. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(copilot): remove language matching rules from prompts Remove language matching rules that cause inconsistent responses: - "User writes in Chinese → Respond in Chinese" - "User writes in English → Respond in English" These created ambiguity and led to inconsistent language behavior. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(copilot): remove Chinese from response template Replace mixed Chinese/English headers with English only: - "操作总结 / Operation Summary" → "Operation Summary" - "详细信息 / Details" → "Details" Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(copilot): remove Chinese mentions from prompts Remove references to "Chinese" language in title_prompt.py: - "Generates concise Chinese or English titles" → "Generates concise titles" - "If the content is predominantly in Chinese, generate a Chinese title" → removed Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(copilot): use short_name for link labels and support short port names - Match ports by both name and short_name for flexibility - Use short_name (e.g. "e0/0") instead of full name (e.g. "Ethernet0/0") for link labels, improving visual clarity in dense topologies * feat(topology): replace hardcoded interface names with dynamic placeholders - Update topology planner skill template to use {short_name} placeholder - Allows dynamic interface name generation based on device type - Improves template flexibility for different network device configurations * refactor(topology): use node-pair IP format and hyphenated naming - Use 10.0.{node_pair}.x format for P2P links (e.g., 10.0.12.x for R-1-R-2) - Change node naming to hyphenated format: R-1, R-2, SW-1, PC-1 - Use {short_name} as placeholder for port names in output template - Remove unused IP_SUBNET_POOL constant - Simplify ip_planning rules to intuitive format description * fix(web_wireshark): raise minimum Docker API version to 1.44 Docker daemon requires API version 1.44+, so the minimum supported version should match rather than defaulting to 1.40. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(web_wireshark): call ensure_network in start_wireshark_session Ensure Docker network exists before creating container, so that Web Wireshark works when started via API without going through CLI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(agent): improve Docker image setup with network error detection - Add `is_network_error()` function to detect network-related failures in Docker pull output - Modify `pull_image()` to capture output and return success status along with output - Enhance main logic to detect network errors and skip local build when Docker Hub is inaccessible - Provide helpful suggestions for network issues (Docker mirror, VPN, manual image transfer) - Improve error messages to differentiate between network failures and other errors * feat(docs): add Ubuntu 24.04 development setup guide Add a comprehensive development environment setup guide for Ubuntu 24.04. The guide includes steps to install dependencies via PPA, configure Docker with mirror accelerators for users in mainland China, set up user permissions, and run the server from source with a Python virtual environment. This provides a clear, step-by-step reference for new contributors and developers. * feat(docs): add pip mirror note for China mainland users Add a comment in the development setup documentation suggesting the use of the Aliyun PyPI mirror for users in China mainland to improve installation speed and reliability. This helps overcome network restrictions and slow downloads from the default PyPI repository. * feat(docs): add LVM root partition expansion guide Add optional section to development setup documentation with instructions for expanding the root partition when using LVM. This helps developers resolve low disk space issues by utilizing unallocated space in the volume group. The guide includes commands to check LVM status, extend the logical volume, and verify the changes. * feat(docs): add tshark to development setup dependencies Add tshark to the apt install command in the development setup documentation. Tshark is required for packet capture functionality in GNS3, ensuring the development environment has all necessary tools for network analysis and debugging. * feat: add GNS3 documentation skill with standardized structure Add new documentation skill file defining standards for GNS3 technical documentation. The skill establishes a structured approach focusing on architecture diagrams, flow diagrams, and text descriptions while prohibiting code examples and user scenarios. This ensures consistent, high-quality documentation across the GNS3 ecosystem. * feat(docs): update documentation skill to focus on server technical docs Update the GNS3 documentation skill to specifically target server technical documentation under the docs/ directory. The revised standard emphasizes high-level understanding over implementation details, using ASCII diagrams for architecture and business processes, API endpoint tables, and measured performance data. Code specifics are intentionally omitted, directing readers to the codebase for implementation details. * feat(docs): rewrite documentation skill and statistics API doc - Rewrite gns3-documentation skill to match actual server-side doc style: focus on architecture/flow diagrams, API data, skip code details - Switch diagram standard from ASCII to Mermaid (GitHub native rendering) - Rewrite statistics-api.md with Mermaid architecture and sequence diagrams, add missing webwireshark container fields (memory_limit, cpu_limit, pids_limit), remove speculative content Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(docs): update VNC WebSocket console documentation with Mermaid diagrams - Replace ASCII diagrams with Mermaid flowcharts for better visualization - Clarify connection flow between browser, controller, compute, and VNC server - Update authentication details and endpoint descriptions - Add missing documentation for packet capture workflow - Improve overall readability and maintainability * docs: add AI disclaimer to documentation files Add a standardized disclaimer to multiple documentation files indicating that the content has been organized by AI with reference to actual code. The disclaimer warns users that AI can make mistakes and advises verification against the source code when in doubt. This improves transparency about the documentation's origin and encourages careful usage. * feat(docs): update chat API documentation with new features and clarifications - Update title from "Design Document" to reflect current implementation status - Add new features: session abort, copilot modes (teaching_assistant, lab_automation_assistant), session pinning - Enhance architecture diagram with LangGraph StateGraph details including abort_handler_node, title_generator_node, and conditional edges - Clarify statistics tracking: add ai_response_counted flag, filter title_generator_node from LLM counts, explain incremental token counting - Improve documentation accuracy for real-time statistics collection and token calculation methods * feat(docs): restructure command security documentation with visual workflows - Replace verbose implementation details with concise architecture overview - Add Mermaid diagrams to visualize command filtering and multi-line expansion flows - Simplify configuration instructions and remove redundant examples - Consolidate file structure and function reference tables for clarity - Maintain all security principles while improving readability and maintainability * feat(context): refactor context window management documentation for clarity - Reorganize documentation with improved structure and visual diagrams - Add Mermaid diagrams to illustrate architecture and trimming process - Simplify content while maintaining technical accuracy - Update token counting and trimming strategy explanations - Enhance readability with better formatting and component tables * feat(docs): update LLM model configs documentation - Change config column type from JSONB to JSON (JSONB on PostgreSQL) - Update provider table to mark base_url as required with planned optional status - Add note about base_url being currently required in API schema - Clarify GET own configs endpoint returns plain array - Add copilot_mode field to update request schema - Document update limitations for context_strategy and copilot_mode fields * feat(docs): add GNS3StartNodeQuickTool and update node creation examples - Add documentation for new `GNS3StartNodeQuickTool` (`start_gns3_node_quick`) that starts nodes without waiting for boot completion - Update node creation example to include optional `name` field in template placement - Clarify dynamic wait time strategy for `GNS3StartNodeTool` and contrast with quick tool - Update tool file structure to reflect new configuration, display, and packet capture tools - Fix truncated line in suspend tool documentation * feat(docs): add comprehensive documentation structure with CC BY-SA 4.0 license Add initial documentation directory with detailed README.md that outlines: - Dual license structure (CC BY-SA 4.0 for docs, GPLv3 for code) - Complete directory structure for features, AI Copilot, and bugs - Feature documentation covering Controller+Compute setup, Statistics API, VNC WebSocket console, and Web Wireshark - AI Copilot implemented features including Chat API, LLM model configs, command security, and multi-vendor device support This provides organized technical documentation for the GNS3 server project with proper licensing and feature coverage. * docs: update multi-vendor device support documentation - Simplify vendor support table by removing redundant platform column - Add VPCS as a simulator in status column - Improve VPCS driver diagram with Mermaid syntax and detailed components - Document VPCS-specific Netmiko parameters (fast_cli, global_delay_factor) - Update VPCS tool usage example with connection options structure - Remove outdated Nornir configuration approaches, reference current implementation - Add file reference for VPCS tools location - Clean up documentation structure for better readability --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: YueGuobin <yueguobin@gmail.com> Co-authored-by: Jeremy Grossmann <grossmj@gns3.net> |
||
UmmmAGoodName
|
19503ab4f7 | Implementet a smarter node_type system | ||
grossmj
|
d3860ba84b
|
Merge branch '2.2' into 3.0
# Conflicts: # CHANGELOG # dev-requirements.txt # gns3server/compute/qemu/__init__.py # gns3server/compute/vmware/__init__.py # gns3server/controller/snapshot.py # gns3server/crash_report.py # gns3server/handlers/api/compute/atm_switch_handler.py # gns3server/run.py # gns3server/static/web-ui/26.77d4bfd104f37c42e028.js # gns3server/static/web-ui/index.html # gns3server/static/web-ui/runtime.415291667f70565cd8ef.js # gns3server/utils/__init__.py # gns3server/utils/images.py # gns3server/utils/interfaces.py # gns3server/version.py # gns3server/web/web_server.py # pytest.ini # requirements.txt # scripts/update-bundled-web-ui.sh # setup.py # tests/api/routes/compute/test_dynamips_nodes.py # tests/compute/builtin/nodes/test_cloud.py # tests/compute/docker/test_docker.py # tests/compute/docker/test_docker_vm.py # tests/compute/dynamips/test_dynamips_manager.py # tests/compute/dynamips/test_dynamips_router.py # tests/compute/iou/test_iou_vm.py # tests/compute/qemu/test_qcow2.py # tests/compute/qemu/test_qemu_manager.py # tests/compute/qemu/test_qemu_vm.py # tests/compute/test_base_node.py # tests/compute/test_manager.py # tests/compute/test_project.py # tests/compute/traceng/test_traceng_vm.py # tests/compute/virtualbox/test_virtualbox_manager.py # tests/compute/virtualbox/test_virtualbox_vm.py # tests/compute/vmware/test_vmware_manager.py # tests/compute/vmware/test_vmware_vm.py # tests/compute/vpcs/test_vpcs_vm.py # tests/conftest.py # tests/controller/gns3vm/test_remote_gns3_vm.py # tests/controller/gns3vm/test_virtualbox_gns3_vm.py # tests/controller/gns3vm/test_vmware_gns3_vm.py # tests/controller/test_export_project.py # tests/controller/test_gns3vm.py # tests/controller/test_import_project.py # tests/test_config.py # tests/utils/test_asyncio.py # tests/utils/test_images.py # tests/web/test_response.py |
||
|
grossmj
|
35b75f0f7f
|
Fix link capture for ATM switch. Fixes https://github.com/GNS3/gns3-gui/issues/3570 | ||
|
grossmj
|
8bae921587 | Fix packet capture | ||
|
grossmj
|
c021e21309 | Use black with -l 120 param. | ||
|
grossmj
|
f928738bd5 | Use pyupgrade with --py36-plus param. | ||
|
grossmj
|
eb3cb8a41f | Migration to FastAPI | ||
|
grossmj
|
f97d346c34 | Support to reset links. Fixes https://github.com/GNS3/gns3-server/issues/1620 | ||
|
grossmj
|
dde01317e0 | Fix packet filter not working for Ethernet switch and Ethernet hub. Fixes https://github.com/GNS3/gns3-gui/issues/2754 | ||
|
grossmj
|
2764828f38 |
Refactor how clients access PCAP capture files. Fixes https://github.com/GNS3/gns3-gui/issues/2438.
* The PCAP file is directly accessed if controller and client are on the same host. * The PCAP file is streamed from the compute server to the client with the controller as a proxy when the controller is remote for the client. |
||
|
grossmj
|
76af98404a | Drop Python 3.4 and switch to async / await syntax for asyncio. Fixes #1425 | ||
|
grossmj
|
033f005bde |
Merge branch '2.1' into 2.2
# Conflicts: # gns3server/compute/docker/docker_vm.py # gns3server/version.py |
||
|
grossmj
|
d0ed6931d8 | Catch exception: ZIP does not support timestamps before 1980. Ref #1360. | ||
|
grossmj
|
8b91894fa4 |
Improve suspend a link for Qemu and VirtualBox VMs.
A suspended link will be unplugged allowing the VMs to be notified of the change. |
||
|
grossmj
|
ca3f6fb4b8 | Update some sentences related to suspending a link. | ||
Julien Duponchelle
|
5fd842e54d
|
Suspend link | ||
|
Julien Duponchelle
|
08423eff96
|
Support packet filtering for VPCS
https://github.com/GNS3/gns3-gui/issues/765 |