mirror of
https://github.com/GNS3/gns3-server.git
synced 2026-09-01 15:54:03 +03:00
feat(web-wireshark): add implementation plan for script-driven (#2666)
* feat(web-wireshark): add implementation plan for script-driven integration Add comprehensive implementation plan for Web Wireshark integration using script-driven approach. The plan outlines: - Background and motivation for script-based Web Wireshark integration - Core workflow from API request to WebSocket proxy connection - Management script architecture with WebWiresharkManager class - Docker container configuration and resource management - Performance analysis and optimization recommendations - Network setup and management procedures Key features include: - JWT token extraction from Authorization header - Automatic GNS3 server URL detection - Container lifecycle management per project - Xpra session isolation per link - WebSocket proxy integration through gns3server - Resource monitoring and scaling guidelines The implementation enables users to start Web Wireshark sessions via POST requests with `wireshark: true` parameter, providing a unified web-based packet capture interface. * feat(web-wireshark): implement script-driven Web Wireshark integration Add complete Web Wireshark container integration with script-driven architecture: - Create manage_wireshark.py script for Docker container management - Add LinkCapture schema with wireshark boolean field - Update Link class with wireshark and jwt_token parameters - Implement WebSocket proxy endpoint for xpra HTML5 client - Add cleanup logic in Project class for container lifecycle Key features: - Script-driven container and xpra session management - JWT token extraction from Authorization header - GNS3 URL auto-detection with Controller/Config fallback - WebSocket proxy for unified access through gns3server - Proper cleanup on project close/delete Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web-wireshark): use aiohttp instead of docker SDK Replace docker-py SDK with direct Docker HTTP API calls using aiohttp, following GNS3's existing architecture pattern in gns3server/compute/docker/. Changes: - Create DockerHTTPClient class using aiohttp + Unix socket - Implement all Docker operations as async methods - Remove dependency on docker Python SDK - Use Docker API v1.44 via /var/run/docker.sock Benefits: - No additional dependencies required - Consistent with GNS3's async architecture - Better integration with existing codebase Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web-wireshark): add dynamic Docker API version detection Implement dynamic Docker API version detection following GNS3's pattern in gns3server/compute/docker/__init__.py. Changes: - Add DOCKER_MINIMUM_API_VERSION and DOCKER_PREFERRED_API_VERSION constants - Remove hardcoded "v1.44" prefix (now using "1.44" format) - Add _check_connection() method to detect Docker daemon version - Dynamically select API version based on daemon capabilities - Add proper error handling for version mismatches Behavior: - Initialize with minimum API version (1.40) - On first connection, detect Docker daemon API version - Use preferred API version (1.44) if supported - Fall back to daemon's minimum API version if needed - Raise error if daemon version is too old Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web-wireshark): add optional --verbose logging parameter Remove hardcoded logging.basicConfig() and add --verbose parameter to control log output, following GNS3's logging pattern. Changes: - Remove logging.basicConfig(level=logging.INFO) from module level - Add --verbose/-v command line parameter - Configure logging only when --verbose is specified - Use structured log format with timestamp when verbose Behavior: - When called by GNS3: uses GNS3's logging configuration (default) - When run standalone: no output unless --verbose is specified - With --verbose: shows detailed logs with timestamps This prevents the script from interfering with GNS3's logging configuration while still allowing debug output when needed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web-wireshark): fix code quality issues from static analysis Fix issues identified by flake8 and pylint static analysis: Import and formatting fixes: - Remove unused 'time' import - Fix import order (stdlib before third-party) - Remove unnecessary f-strings without placeholders - Split long line to comply with flake8 (max line length: 127) Code structure improvements: - Remove unnecessary 'else' after 'return' - Add 'from e' to exception re-raising for better tracebacks Code quality metrics: - pylint score: 8.11/10 → 9.97/10 (+1.86) - flake8: 0 errors (max line length: 127, per CI/CD standard) - All critical issues resolved Note: R0914 (too-many-locals) warning is informational only, code remains clear and maintainable. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): export LinkCapture schema in schemas module Add LinkCapture to the schemas module exports in __init__.py to fix the AttributeError when starting the GNS3 server. Error was: AttributeError: module 'gns3server.schemas' has no attribute 'LinkCapture' This was caused by adding the LinkCapture class to controller/links.py but forgetting to export it in the schemas __init__.py. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web-wireshark): add --image parameter for custom Docker images Add optional --image parameter to allow using custom Docker images for testing, instead of requiring gns3/web-wireshark:latest. Changes: - Add --image parameter to start command (default: gns3/web-wireshark:latest) - Update get_or_create_container() to accept image parameter - Update start_wireshark_session() to accept and pass image parameter - Update TEST.md with examples of using custom images Usage: # Using default image python3 manage_wireshark.py start --project-id x --link-id y --jwt-token z # Using custom image (for testing) python3 manage_wireshark.py start --project-id x --link-id y --jwt-token z --image ubuntu:latest This makes it easier to test the script without having the official gns3/web-wireshark image available. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web-wireshark): add configurable resource limits and increase session capacity Add configurable resource parameters and increase session support from 10 to 100. Resource parameters: - --memory: Memory limit (default: 2g) - --memory-swap: Memory swap limit (default: same as memory) - --cpus: CPU cores (default: 1.0) - --pids-limit: Process limit (default: 1000) Other improvements: - Fix CPU quota calculation: 1000000 microseconds = 1.0 CPU core (was incorrectly 100000 = 0.1 CPU core) - Add health check: xpra list command - Add log configuration: json-file, max-size=10m, max-file=3 - Increase session capacity: 10 → 100 concurrent sessions - Port range: 12300-12399 (was 12300-12309) - Display range: :100-:199 (was :100-:109) This matches the docker run command parameters and provides better resource management and monitoring capabilities. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): update UDPLink.start_capture() to accept new parameters Update UDPLink.start_capture() to accept the new wireshark and jwt_token parameters and pass them to the parent Link class. This fixes the TypeError when starting capture on UDP links: TypeError: UDPLink.start_capture() got an unexpected keyword argument 'wireshark' The UDPLink class overrides start_capture() but didn't include the new parameters added for Web Wireshark support. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: add project memory infrastructure and JWT token flow documentation - Add memory skill for recording project knowledge - Document JWT token flow in Web Wireshark integration - Update .gitignore to track skills and memory directories Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): fix Docker API URL format and use docker exec CLI - Add "v" prefix to Docker API URL (http://docker/v{version}/...) - Remove unused exec_create/exec_start API methods - Fix Healthcheck.Test format to ["CMD-SHELL", "command"] - Use docker exec CLI instead of Docker API for command execution - This aligns with GNS3 docker_vm pattern Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): improve xpra session management and error handling - Check and clean up existing sessions before starting new xpra session - Add verification that xpra session started successfully - Use docker exec CLI consistently instead of Docker API - Improve error logging and diagnostics Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(web-wireshark): add timeout handling and container health checks - Add REQUEST_TIMEOUT constant for Docker HTTP API requests - Add asyncio.timeout wrapper for Docker API calls to prevent hanging - Add _is_container_healthy() to check container responsiveness - Add _exec_in_container() helper with timeout support - Improve unhealthy container handling with force remove - Add detailed logging for container health state Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web-wireshark): split monolithic file into modules Split manage_wireshark.py into: - docker_client.py: Docker HTTP API client - manager.py: WebWiresharkManager with session management - manage_wireshark.py: CLI entry point This improves code organization and maintainability. Keep all timeout handling and health check logic. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): correct NanoCpus calculation to use nanoseconds NanoCpus in Docker API requires nanoseconds (1 CPU = 1000000000), not the previous incorrect multiplier of 100000. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): fix xpra command quoting and session verification - Add quotes around --xvfb parameter value to preserve spaces - Fix session verification to check display number instead of session name (xpra list shows "LIVE session at :185" not session name) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): run wireshark in background with & Wireshark with curl pipeline runs continuously, so it must run in background to avoid blocking. Also reduced timeout since we don't wait for completion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web-wireshark): add deterministic hash for display/port allocation Use MD5-based hash instead of Python's hash() which is randomized across process restarts. This ensures display and port numbers are stable when the GNS3 server restarts. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: extract link_id_to_port to shared utils module Move deterministic hash functions to gns3server/utils/port_allocator.py to avoid code duplication and ensure consistent algorithm across manager.py and links.py. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(links): move import to top of file Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(web-wireshark): use correct Config access pattern like gns3-copilot The Server config is an object with .protocol.value, .host, .port attributes, not a dict. Fixes URL detection failing and falling back to 127.0.0.1:3080 which doesn't work from inside containers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add UUID validation utility and use in manage_wireshark Create gns3server/utils/uuid_validator.py with validate_uuid function to validate UUID format and catch typos early with clear error messages. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use argparse.ArgumentTypeError for proper error message display Previously ValueError was used which only showed 'invalid validate_uuid value'. Now shows the full helpful message about expected format. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(web-wireshark): add --dpi=96 to xpra start parameters Set standard DPI for better display scaling in web Wireshark. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: update TEST.md with correct project-id format Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(web-wireshark): hide xpra shutdown menu via XPRA_CLIENT_CAN_SHUTDOWN Prevents users from accidentally shutting down the xpra server through the HTML5 client menu. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(memory): add xpra-html5-client configuration reference Document xpra HTML5 client configuration including: - URL parameters for toolbar menu control - default-settings.txt parameters (Features, Connection, Advanced) - Server-controlled submenu items - XPRA_CLIENT_CAN_SHUTDOWN environment variable - Background image customization Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(web-wireshark): remove xpra stop cleanup to avoid zombie processes xpra stop can leave zombie processes and timeout. Instead, let xpra start reuse the display directly (it will overwrite existing session). Also added --file-transfer=no, --printing=no, --sound=no to disable unneeded features. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(web-wireshark): add fullscreen_button=false to default-settings.txt Also remove --file-transfer/--printing/--sound from command line since xpra doesn't support these options. Configure in default-settings.txt instead. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(web-wireshark): update xpra background with GNS3 branding - Replace default xpra background with GNS3 icon and modern gradient - Update CSS to use SVG background image with cover sizing - Change background color to gradient from #021d3a to linear gradient - Improves visual integration with GNS3 web interface * style: add GPLv3 headers with copyright to web_wireshark modules Co-Authored-By: YueGuobin <yueguobin@gmail.com> * feat(docker): use Alibaba Cloud Debian mirror for faster builds in China Use mirrors.aliyun.com for Debian packages with correct paths for both main repo (/debian) and security repo (/debian-security). Co-Authored-By: YueGuobin <yueguobin@gmail.com> * feat(web-wireshark): update xpra background styling and echo command - Replace `echo -e` with `printf` for better POSIX compatibility in Dockerfile - Update xpra HTML5 client background to use GNS3 icon with light gradient - Change background color from dark blue to light gray for improved visibility * fix(web-wireshark): use pkill to stop xpra sessions Replace 'xpra stop :{display}' with 'pkill -f "xpra.*:{display}"' for more reliable session termination. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web-wireshark): start Wireshark in fullscreen mode Add --fullscreen flag to Wireshark startup command to prevent window decorations and improve user experience. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): use carrier-grade NAT subnet to avoid conflicts Change Docker network subnet from 172.28.0.0/16 to 100.64.1.0/24 to avoid conflicts with common private networks. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs(web-wireshark): update subnet to /22 for 1000+ projects Change Docker network subnet from 100.64.1.0/24 to 100.64.0.0/22 to support 1000+ projects (1022 available IPs). Update all documentation to reflect the new subnet. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): replace CGNAT subnet with configurable private subnet Replace the CGNAT address block (100.64.0.0/22) with a standard private subnet (172.31.0.0/22) to avoid network access issues. Many networks and ISPs block or cannot route CGNAT addresses. Changes: - Add WebWiresharkSettings to config schema with configurable subnet - Read network_subnet from config, default to 172.31.0.0/22 - Add Web Wireshark configuration section to sample config Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): comment out dpi setting Allow xpra to use default DPI settings for better display scaling. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): set DPI in Xvfb to prevent scaling warnings Add -dpi 96 to Xvfb startup command to match xpra's expected DPI and avoid "scaling problems" warnings. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): remove custom Xvfb to use default Xorg-dummy Remove --xvfb parameter to let xpra use the default Xorg-dummy driver, which properly handles DPI changes when resize-display is enabled and prevents DPI mismatch warnings. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * revert(web-wireshark): restore Xvfb configuration Restore the Xvfb display server configuration for xpra sessions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web-wireshark): enhance xpra HTML5 default settings Update xpra HTML5 configuration to disable additional unused features: - audio, keyboard, mediasource, aurora, http-stream - Improve readability using heredoc format instead of printf Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(api): add RBAC authentication to web wireshark WebSocket endpoint - Add `has_privilege_on_websocket` dependency to enforce Link.Capture privilege - Include current_user parameter in endpoint for user identification and logging - Update endpoint documentation to reflect token requirement and privilege - Add test example WebSocket URL in TEST.md for reference * refactor(web-wireshark): add generic WebSocket proxy and disable xpra HTML - Add websocket_to_websocket.py utility for binary WebSocket proxy - Disable xpra HTML server with --html=no flag - Update manager return value: url -> ws_url - Simplify WebSocket proxy implementation in links.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web-wireshark): remove unnecessary sleep delays Remove 2-second sleep delays after container start and xpra initialization. Health checks and verification happen immediately, improving startup time. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(agent): prevent blocking during project close - Add 5s timeout for AgentService.checkpointer_conn.close() - Optimize stop_all_sessions: single command instead of 100 docker exec calls - Prevents indefinite hang when closing projects Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web-wireshark): return errors to client on startup failure - Throw ControllerError when Web Wireshark startup fails - Include detailed error messages from script stderr - Support both ws_url and url response formats - Client now receives proper error response instead of success Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web-wireshark): parallelize startup and remove unnecessary waits - Parallel execution: container info query + xpra start - Remove xpra list verification (unnecessary) - Fire-and-forget Wireshark startup (no timeout wait) - Startup time reduced from ~7s to ~1s Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web-wireshark): add stop-container and delete-container commands - Add stop-container command to stop container on project close - Add delete-container command to delete container on project delete - Enables proper container lifecycle management - Containers are now stopped (not deleted) when project closes - Containers are deleted when project is deleted Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(project): simplify close flow by removing redundant xpra session cleanup Since docker stop with timeout=0 already force-kills all processes in the container, explicitly stopping xpra sessions before stopping the container is unnecessary. Remove the _cleanup_web_wireshark_xpra_sessions() call to reduce subprocess overhead and simplify the close flow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * perf(docker): force kill containers on stop by default Change docker stop timeout from 10 seconds to 0 (immediate SIGKILL). Web Wireshark containers don't need graceful shutdown since they have no persistent state. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(web-wireshark): enable WebSocket protocol for xpra connection - Change xpra bind from --bind-tcp to --bind-ws for WebSocket support - Enable HTML client with --html=on for xpra WebSocket server - Pass binary subprotocol when connecting to xpra container The xpra WebSocket server requires the 'binary' subprotocol during handshake. Without it, the server returns 403 with message: "client does not support 'binary' protocol". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(web-wireshark): add WebSocket subprotocol negotiation support Fix WebSocket proxy for xpra container by implementing proper subprotocol negotiation. The xpra client requires the server to respond with the negotiated subprotocol (binary) in the WebSocket handshake response. Changes: - Modified get_current_active_user_from_websocket to extract client's requested subprotocols from Sec-WebSocket-Protocol header - Updated websocket.accept() call to include negotiated subprotocol - Enhanced websocket_proxy to support subprotocol parameter - Improved logging for WebSocket connection debugging This fixes the issue where xpra clients would immediately disconnect after connection due to missing subprotocol in server response. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore(web-wireshark): remove verbose debug logging Remove excessive debug logs from WebSocket proxy implementation while keeping essential logs for troubleshooting: - Keep: subprotocol negotiation, connection establishment, errors - Remove: verbose step-by-step debugging information Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web-wireshark): translate Chinese comments and docs to English - Translate all Chinese comments in project.py, link.py, and links.py to English - Convert TEST.md and maunal-test.md documentation to English - Remove obsolete IMPLEMENTATION_PLAN.md Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore(docker): lock xpra version to 6.4.3 for reproducible builds Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore(docker): improve Dockerfile with TZ, no-install-recommends, and labels - Add TZ=UTC timezone setting - Use --no-install-recommends to reduce image size - Add LABEL metadata for maintainer and description Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(docs): rename and consolidate documentation - Rename TEST.md to WEB_WIRESHARK.md for clearer naming - Merge maunal-test.md content into main document under "Manual Testing" section - Remove obsolete gns3_icon_black.svg Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(docker): add xpra-x11 package and remove --no-install-recommends - Add xpra-x11 package required for seamless mode - Remove --no-install-recommends to ensure all recommended packages are installed Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(api): add capture file download endpoint Add GET /{link_id}/capture/file endpoint to download PCAP capture files. Supports downloading while capture is active (streaming). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(web_wireshark): improve session cleanup and add known issues - Add cleanup of existing processes before starting xpra session to prevent "another window manager seems to be running" errors - Stop all associated processes (xpra, wireshark, Xvfb) when stopping sessions - Document known issues in WEB_WIRESHARK.md including JWT token security, duplicate code, and other implementation concerns Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(web_wireshark): add restart wireshark API endpoint - Add POST /links/{link_id}/capture/wireshark/restart endpoint - Add restart command to manage_wireshark.py CLI - Add _restart_web_wireshark method in link.py controller - Restart simply calls start_wireshark_session which handles cleanup This allows users to recover after accidentally closing the Wireshark window without having to stop and restart the entire capture. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add Web Wireshark business process documentation Document the Web Wireshark feature including architecture diagrams, business processes for capture start/stop, container lifecycle, WebSocket connection flow, and session management. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(web_wireshark): fix container network access and improve session cleanup - Fix GNS3 server URL detection to use container gateway IP (172.31.0.1) instead of localhost/127.0.0.1/0.0.0.0 which don't work from containers - Add import socket module for gateway IP conversion - Move urllib.parse import to file header (code style improvement) - Add logging for Web Wireshark startup (capture stream URL, display, etc.) - Fix X lock file cleanup to prevent "Server is already active" errors - Use exec pkill to reduce zombie processes from docker exec bash - Add _cleanup_x_lock() method to remove X lock files after stopping sessions - Update link.py to always log subprocess stderr for debugging Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web_wireshark): avoid zombie pkill processes by using Docker API Replace pkill commands with Docker API-based process killing to prevent zombie process accumulation in Web Wireshark containers. Changes: - Add DockerHTTPClient.list_processes() method to query container processes - Rewrite _kill_process_tree() to use Docker API instead of pkill - Unify all process cleanup to use _kill_process_tree() method - Add logging for killed processes (count and PIDs) - Include fallback to pkill if Docker API fails This prevents the accumulation of zombie pkill processes that occurred when using bash -c "pkill -9 -f pattern" commands. Testing: Started and stopped multiple capture sessions, verified no new pkill zombie processes are created during cleanup. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web_wireshark): use docker-init and cleanup socket files Enable Docker init system (tini) as PID 1 and clean up xpra socket files when stopping sessions to prevent zombie processes and leftover sockets. Changes: - Add "Init": True to container host_config to use docker-init (tini) - Extend _cleanup_x_lock() to remove xpra socket files * /run/user/1000/xpra/{display}/socket * /run/user/1000/xpra/*-{display} * /home/gns3/.xpra/*-{display} Benefits: - docker-init (tini) automatically reaps orphan processes, eliminating zombie process accumulation (tested: 53 zombies -> 0 zombies) - Socket cleanup prevents xpra list from showing UNKNOWN sessions - Cleaner container state after stopping sessions Testing: - Started and stopped 3 capture sessions - Verified 0 zombie processes after stopping - Verified xpra list shows "No xpra sessions found" (no UNKNOWN) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs(web_wireshark): add comprehensive CLI script documentation Add detailed module docstring to manage_wireshark.py explaining: - Script purpose: CLI tool for manual management, debugging, and testing - Important note: Use WebWiresharkManager directly for programmatic access - Usage examples for all commands (start, stop, restart, stop-all, delete) - Available commands list with descriptions - Output format specification (JSON to stdout/stderr) - Help information for getting command-specific usage This clarifies that the script is intended as a CLI utility, not for subprocess calls from within GNS3 server code. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web_wireshark): use direct API calls instead of subprocess Replace subprocess calls to manage_wireshark.py with direct API calls to WebWiresharkManager, eliminating subprocess overhead and JSON parsing. Changes: - Add import for WebWiresharkManager - Refactor _start_web_wireshark() to use direct API call - Refactor _stop_web_wireshark() to use direct API call - Refactor _restart_web_wireshark() to use direct API call Benefits: - Code reduction: 78 lines (-68%) - Better performance: No subprocess creation overhead - Better logging: Manager logs directly to GNS3 logging system - Simpler error handling: Direct exceptions instead of return codes - No JSON parsing: Direct Python objects - Resource cleanup: Added finally blocks to ensure manager.close() Testing: - All Web Wireshark operations work correctly - Logs now appear in GNS3 server logs instead of stderr - Error handling improved with proper exception propagation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web_wireshark): resolve circular import with delayed imports Fix circular import error by moving WebWiresharkManager import from module level to function level (delayed import). Circular dependency was: link.py → WebWiresharkManager → Controller → link.py Solution: - Remove top-level import of WebWiresharkManager - Add delayed imports in each method that uses it: * _start_web_wireshark() * _stop_web_wireshark() * _restart_web_wireshark() This allows the controller module to fully initialize before importing WebWiresharkManager, breaking the circular dependency. Error before fix: ImportError: cannot import name 'Controller' from partially initialized module 'gns3server.controller' (most likely due to a circular import) Testing: - Successfully imports Link module - GNS3 server starts without import errors Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor: move imports to module level and remove Controller dependency Remove circular dependency between link.py and manager.py by: 1. Remove Controller import from manager.py - Manager no longer imports Controller - URL detection now relies on Config or default only - link.py passes capture_stream_url to manager 2. Move WebWiresharkManager import to module level in link.py - No longer need delayed imports since no circular dependency - Clean, standard Python import pattern 3. Remove redundant Config import in ensure_network() - Config was already imported at module level Changes: - manager.py: -24 lines (removed _get_gns3_url_from_controller and redundant import) - link.py: -30 lines net reduction after moving import to top Testing: - All imports work without circular dependency errors - GNS3 server starts successfully Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(docker_client): use text response for list_processes API Docker API /containers/{id}/top returns plain text, not JSON. The generic _request method auto-parses JSON which caused: "'dict' object has no attribute 'strip'" Fix by using session.get() directly with response.text() instead of the generic _request method. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web_wireshark): use correct capture stream URL endpoint Remove passing capture_stream_url from link.py to manager.py. Manager.py auto-detects URL using correct endpoint: /v3/projects/{project_id}/links/{link_id}/capture/stream This endpoint supports JWT authentication, unlike the compute URL (/v3/compute/...) which requires compute credentials. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web_wireshark): use container-local PIDs for process killing - Docker API returns host PIDs, not container PIDs - use pgrep inside container to get correct PIDs for kill - Fix list_processes to parse Docker API JSON response correctly - Remove unnecessary exec prefix in shell commands - Add logging for debugging process killing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * perf(web_wireshark): optimize process killing with single pgrep call Replace sequential _kill_process_tree calls with a new _kill_process_tree_batch method that combines all patterns into a single regex. This reduces docker exec calls from 8 to 1, improving performance from ~8 seconds to ~0.9 seconds. Changes: - Add _kill_process_tree_batch() method with combined regex pattern matching - Update start_wireshark_session() to use batch cleanup - Update stop_wireshark_session() to use batch cleanup - Update stop_all_sessions() to use batch cleanup Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web_wireshark): disable HTML5 client to improve xpra startup speed Change --html=on to --html=off to disable the xpra HTML5 client interface. This reduces xpra startup time from ~6.4s to ~2.9s (55% improvement) while keeping the WebSocket server functional for browser connections. The HTML5 client is not needed as we only require the WebSocket endpoint for remote display forwarding. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web_wireshark): optimize container health check using Docker built-in status Replace manual docker exec ping with Docker's built-in health check status to reduce container startup latency by ~1 second. Changes: - Use container["Health"]["Status"] instead of manual ping check - Skip health check for containers with "healthy" or "starting" status - Only verify containers with "unhealthy" status - Keep manual health check for newly started containers This reduces the container verification time from ~1s to near-zero for healthy running containers while maintaining safety for unhealthy ones. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web_wireshark): get gateway IP from Docker API instead of container exec Replace slow docker exec method with fast Docker Network API call to get gateway IP. This reduces gateway detection from ~850ms to ~1ms (1000x faster). Changes: - Use docker.get_network() API to get gateway from IPAM config - Keep container exec methods as fallback if Docker API fails - Remove container_id requirement (not needed for Docker API method) The Docker network gateway is shared by all containers in the network, so querying the network is more efficient than querying each container. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web_wireshark): remove unreliable nameserver fallback for gateway detection Remove the fallback method that reads /etc/resolv.conf nameserver as gateway. This method is unreliable because: - nameserver is not necessarily the gateway (could be upstream DNS) - Many systems use 127.0.0.53 (systemd-resolved) or 127.0.0.1 - Even when not local, it could be public DNS (8.8.8.8, 1.1.1.1) The current fallback strategy is sufficient: 1. Docker Network API (fast, reliable) 2. /proc/net/route (standard gateway detection) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web_wireshark): use host perspective for process killing (37x faster) Replace docker exec with host perspective process management for killing container processes. This reduces process cleanup from ~850ms to ~23ms. Key optimization: - Get container init PID using docker inspect (fast) - Use pgrep -P <init_pid> to find child processes from host perspective - Kill processes directly using host PID (no docker exec needed) Performance improvements: - Process finding: 850ms → 23ms (37x faster) - Process killing: 850ms → <1ms (850x faster) - File checking: 850ms → 0.003ms (280,000x faster) Fallback to docker exec method if host perspective fails. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web_wireshark): use host perspective for file cleanup (891x faster) Replace docker exec with direct filesystem access from host perspective for cleaning up X lock files and xpra sockets. This reduces cleanup time from ~890ms to ~1ms (891x faster). Key optimization: - Access /proc/<container_pid>/root/ directly from host - Use os.remove() and glob.glob() instead of docker exec rm -f - Fallback to docker exec if host perspective fails This complements the earlier optimization for process killing, further reducing startup time. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Revert "perf(web_wireshark): use host perspective for file cleanup (891x faster)" * fix(web_wireshark): recursively kill all descendant processes to prevent orphans Previous implementation only killed direct children of container init using `pgrep -P <init_pid>`, but xpra spawns child processes (Xvfb, pulseaudio, ibus-daemon) that become grandchildren and were not being terminated. This fix walks the entire process tree recursively to find and kill all descendant processes, ensuring complete cleanup without orphans. Performance: ~20-40ms (only 10-20ms slower than previous 23ms, but ensures thorough cleanup). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs(web_wireshark): add comprehensive startup/shutdown performance metrics Add detailed performance characteristics section documenting: - Startup performance breakdown (before/after optimization) - Shutdown performance breakdown (before/after optimization) - First startup vs subsequent startup comparison - Measured production data - Key optimization techniques Performance improvements: 67% faster startup, 78% faster shutdown, zero orphan processes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web_wireshark): ensure container stops on project close Remove dependency on _web_wireshark_container_created flag which gets reset when project is reloaded, causing container to not stop on project close. Now directly attempts to stop container on every project close. If container doesn't exist, the script handles it gracefully. This is simpler and more reliable than maintaining a flag. Also removed unnecessary _cleanup_web_wireshark_xpra_sessions() call since stopping the container automatically terminates all xpra sessions inside. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web_wireshark): ensure container deletion on project delete Remove dependency on _web_wireshark_container_created flag from _cleanup_web_wireshark_container() to ensure container is deleted when project is deleted, even if project was reloaded and flag was reset. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(web_wireshark): remove unused _web_wireshark_container_created flag This flag was set but never read for any conditionals since we changed the stop/delete methods to directly attempt the operation. It's purely redundant code that adds confusion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web_wireshark): skip unnecessary cleanup using fast host perspective check Add _check_residuals_exist() function that uses host perspective (~20ms) to check if residual processes or socket files exist before cleanup. Returns (has_process_residuals, has_socket_residuals) tuple to allow selective cleanup - if only sockets need cleaning, skip the slower process tree killing. For new containers with no residuals, this saves ~3 seconds of unnecessary docker exec calls. Socket cleanup always uses docker exec for safety (avoiding accidental host filesystem deletion from /proc/<pid>/root/ path errors). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(web_wireshark): increase container exec timeout from 5s to 10s xpra start with Xvfb initialization can take longer than 5 seconds in Docker containers, causing timeout failures. Increasing timeout to 10 seconds to allow sufficient startup time. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * perf(web_wireshark): optimize cleanup and document docker exec limitations - Combine X lock and xpra socket cleanup into single docker exec call Reduces exec calls from 2 to 1 per stop operation, improving performance when stopping multiple capture sessions. - Document Docker exec performance limitations with test data showing parallel exec is 42% slower than serial due to Docker daemon's internal queuing mechanism. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(links): prevent AssertionError in stream_pcap during rapid stop_capture Fix race condition where stream_pcap checks link.capturing (True) but link.capture_node becomes None before accessing link.compute. This happens when stop_capture() is called and rapidly cleans up _capture_node while WebSocket requests are still processing. Now check both capturing and capture_node to ensure capture is still active. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(links): prevent AssertionError in stream_pcap during rapid stop_capture Fix race condition where stream_pcap checks link.capturing (True) but link.capture_node becomes None before accessing link.compute. This happens when stop_capture() is called and rapidly cleans up _capture_node while WebSocket requests are still processing. Now check both capturing and capture_node to ensure capture is still active, and log at DEBUG level since this is expected behavior during rapid stop. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web_wireshark): load container config from settings - Load memory, cpus, and pids_limit from WebWireshark config section - Apply config only on container creation (_start_web_wireshark) - Skip config on restart (_restart_web_wireshark) as container exists Users can now configure container resources in gns3_server.conf: [WebWireshark] memory = 4g cpus = 2.0 pids_limit = 2000 If not configured, defaults (2g, 1.0, 1000) are used. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(web_wireshark): add container statistics to /v3/statistics endpoint Add Web Wireshark container monitoring to the statistics API for better visibility into packet capture sessions. Changes: - Create new stats.py module with collect_webwireshark_stats() * Collects container info (status, project, active sessions) * Gets resource limits (memory, CPU, PIDs) from container config * Gets resource usage via docker stats (memory, CPU, PIDs) * Properly closes aiohttp connections to avoid leaks - Update /v3/statistics endpoint to include webwireshark data * total_containers: Total number of Web Wireshark containers * running_containers: Number of containers currently running * active_sessions: Total active capture sessions * containers: Array with per-container details - Update statistics-api.md documentation * Change URL from /v1/statistics to /v3/statistics * Add webwireshark field descriptions and examples * Add dashboard integration for Web Wireshark monitoring Example response: { "webwireshark": { "total_containers": 1, "running_containers": 1, "active_sessions": 2, "containers": [{ "project_id": "...", "project_name": "test", "container_id": "6edc9029bac0", "status": "running", "memory_limit": "4.0 GB", "cpu_limit": "4.0", "pids_limit": 4000, "memory": "535.7MiB / 4GiB", "cpu": "0.29%", "pids": 124 }] } } Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(project): use WebWiresharkManager directly instead of subprocess Replace subprocess calls to manage_wireshark.py with direct WebWiresharkManager method calls in project.py. This eliminates unnecessary process overhead and maintains consistent usage pattern with link.py. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(logging): use %-formatting instead of f-strings in WebWireshark methods Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(web_wireshark): use info level log when container not found Change log level from warning to info when container doesn't exist during stop/delete operations, and fix f-string without placeholders. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(logging): lower log level for client disconnection Change log level from warning to debug when client possibly disconnects, as this is normal behavior when users close browser tabs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(linting): resolve ruff warnings and errors - Remove unused variable proc (fire-and-forget subprocess) - Remove duplicate nodes property definition - Remove extraneous f-prefix from f-strings without placeholders - Change bare except to except Exception - Remove unused imports (sys, asyncio, json) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(web_wireshark): improve error message when Docker image not found When creating a container fails due to missing image, provide helpful error message with docker pull command and local build instructions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(schema): add wireshark field to Link schema Add wireshark boolean field to Link Pydantic schema so it is included in API responses when Web Wireshark session is active. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(link): add wireshark state tracking Track whether Web Wireshark is running on a link by adding _wireshark boolean property, updated on start/stop operations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(chat): add session abort functionality Add ability to abort streaming chat sessions via REST API. Changes: - Add abort flag to MessagesState for tracking abort requests - Add session_id to state for abort event correlation - Add _abort_flags dict and check/set/clear functions in gns3_copilot.py - Add abort_handler_node to generate aborted tool messages - Modify should_continue to route to abort_handler_node when aborting - Add abort_session() method in AgentService - Add POST /sessions/{session_id}/abort API endpoint - Clear abort flag at stream start When abort is triggered during streaming: - If LLM has tool_calls pending, abort_handler_node generates aborted tool messages to maintain checkpoint consistency - Prevents "insufficient tool messages" error on resume Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(chat): send tool_end event when stream is aborted When a stream is aborted during tool execution, yield proper tool_end events for aborted tools instead of a generic abort event. This maintains compatibility with the frontend's expected tool_start/tool_end flow. Changes: - Add stream_aborted tracking flag - After stream loop, check abort flag and yield tool_end events - Add "abort" to ChatResponse type enum (unused but available) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(web_wireshark): add gns3-wireshark-setup command for Docker image setup Add a new entry point script that allows users to setup the Web Wireshark Docker image with a single command: pip install gns3-server && gns3-wireshark-setup The script will: 1. Try to pull gns3/web-wireshark:latest from Docker Hub 2. If pull fails, build the image locally using the included Dockerfile 3. Show raw docker pull/build output for full visibility Files changed: - Add setup_wireshark_image.py (new entry point script) - Update pyproject.toml (add gns3-wireshark-setup entry point) - Update documentation (README.md, WEB_WIRESHARK.md, web-wireshark-business-process.md) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(appliance): add tags field support for appliances and templates - Add tags field to ApplianceV1_6 and ApplianceV8 schemas - Add tags propagation from appliance config to template - Simplify VPCS builtin template tags Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix test in test_link.py * Fix link.py after failed tests * feat(copilot): add device skills system for LLM context injection - Add skills module with registry and DeviceSkillsTool - Skills organized by vendor/series for easy extensibility - Support device_type, category (device/protocol/feature), operation (config/diagnosis) - First implementation: VPCS skill (gns3_vpcs_telnet) - Skills tool integrated into teaching_assistant and lab_automation modes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(vpcs): change prompt detection log from warning to debug The "VPCS prompt not clearly detected" message is not an error, connection works fine. Change to debug level to reduce noise. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(memory): add uBridge permission issue documentation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(copilot): include link_id in links_summary output Return link_id in the links_summary method so that the AI can identify which link to analyze when using the packet capture tool. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(copilot): add packet capture analysis tool Add PacketCaptureTool that allows AI to analyze packets from an active GNS3 capture. The tool downloads capture files from the GNS3 server and runs tshark analysis to help users understand network traffic. Features: - Download capture file from /capture/file endpoint - Run tshark with custom arguments for flexible analysis - Support analyzing specific packets (e.g., "explain packet #42") - Support protocol statistics, traffic analysis, and expert info Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(copilot): use shlex.split to properly parse quoted tshark arguments tshark_args may contain quoted filter expressions like "-Y \"frame.number == 24\"". Using plain str.split() breaks these quoted arguments, causing tshark warnings about conflicting display filters. Use shlex.split() to correctly parse quoted arguments. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(copilot): simplify PacketCaptureTool to accept only packet_number Instead of exposing complex tshark arguments to the LLM, the tool now accepts a simple packet_number parameter and internally constructs the tshark command with verbose output. This prevents the LLM from generating incorrect tshark parameters while still providing detailed packet analysis. Changes: - Remove tshark_args and max_lines parameters - Accept only packet_number as input - Internal command: tshark -r <file> -Y "frame.number == N" -V - Returns complete packet structure without line limits Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add CC BY-SA 4.0 license headers to documentation files Add SPDX license headers referencing docs/LICENSE file to all markdown and HTML documentation files in the docs directory. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(copilot): add topology planner skill Add new skill for automatic network lab topology planning: - IOU as default image, 10.0.0.0/8 IP range, max 10 nodes - Node naming convention: R/S/PC + number - 7-step workflow for create/rename/link/start/verify/config - IP allocation rules and troubleshooting guide Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(copilot): add node positioning rules to topology planner skill Add grid-based positioning for GNS3 nodes: - Minimum 250px distance between nodes - Grid layout: 4 columns, 300x250px spacing - Formula: x = -400 + col * 300, y = -200 + row * 250 - Update workflow params_required to include x, y coordinates Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(copilot): remove unused helper functions from topology planner skill Delete Python helper functions that cannot be serialized to JSON: - calculate_node_positions() - get_position_for_node() - allocate_subnet() - allocate_ip() - TOPOLOGY_CREATION_STEPS (duplicate of workflow in skill dict) LLM should use formula descriptions in skill to calculate positions/IPs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(copilot): add topology_planner workflow to lab automation prompt Add device_skills tool to AVAILABLE TOOLS table. Add TOPOLOGY PLANNING WORKFLOW section explaining: - How to query topology_planner skill - Default conventions (IOU, 10.0.0.0/8, naming rules, position formula) - 7-step workflow for topology creation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(copilot): support name parameter in gns3_create_node tool - Add optional 'name' field to node creation, allowing direct naming - Update Node constructor to pass name parameter - Remove separate rename step (step_3) from topology planner workflow - Renumber workflow steps: 1-2-3-4-5-6 (skipping old step_3 rename) This eliminates the need for a separate gns3_update_node_name_tool call when creating nodes with predefined names. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(copilot): update topology workflow to 6 steps Remove rename step since name can be set directly in create_gns3_node. Workflow changed from 7 steps to 6 steps. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(copilot): replace grid positioning with topology-based layout Redesign node positioning rules to follow classic network topologies: - Star: hub at center, spokes radiating outward - Ring: nodes in circular arrangement - Bus: linear chain of nodes - Mesh: grid pattern for interconnected nodes - Hierarchical: three-tier (Core → Distribution → Access) - Linear P2P: point-to-point WAN links in a line Remove old grid formula. Update prompt and workflow to reflect new approach. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(copilot): add note about combining topology types LLM can mix topology types as needed, e.g., star + linear_p2p for WAN segments. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(copilot): remove language matching rules from prompts Remove language matching rules that cause inconsistent responses: - "User writes in Chinese → Respond in Chinese" - "User writes in English → Respond in English" These created ambiguity and led to inconsistent language behavior. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(copilot): remove Chinese from response template Replace mixed Chinese/English headers with English only: - "操作总结 / Operation Summary" → "Operation Summary" - "详细信息 / Details" → "Details" Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(copilot): remove Chinese mentions from prompts Remove references to "Chinese" language in title_prompt.py: - "Generates concise Chinese or English titles" → "Generates concise titles" - "If the content is predominantly in Chinese, generate a Chinese title" → removed Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(copilot): use short_name for link labels and support short port names - Match ports by both name and short_name for flexibility - Use short_name (e.g. "e0/0") instead of full name (e.g. "Ethernet0/0") for link labels, improving visual clarity in dense topologies * feat(topology): replace hardcoded interface names with dynamic placeholders - Update topology planner skill template to use {short_name} placeholder - Allows dynamic interface name generation based on device type - Improves template flexibility for different network device configurations * refactor(topology): use node-pair IP format and hyphenated naming - Use 10.0.{node_pair}.x format for P2P links (e.g., 10.0.12.x for R-1-R-2) - Change node naming to hyphenated format: R-1, R-2, SW-1, PC-1 - Use {short_name} as placeholder for port names in output template - Remove unused IP_SUBNET_POOL constant - Simplify ip_planning rules to intuitive format description * fix(web_wireshark): raise minimum Docker API version to 1.44 Docker daemon requires API version 1.44+, so the minimum supported version should match rather than defaulting to 1.40. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(web_wireshark): call ensure_network in start_wireshark_session Ensure Docker network exists before creating container, so that Web Wireshark works when started via API without going through CLI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(agent): improve Docker image setup with network error detection - Add `is_network_error()` function to detect network-related failures in Docker pull output - Modify `pull_image()` to capture output and return success status along with output - Enhance main logic to detect network errors and skip local build when Docker Hub is inaccessible - Provide helpful suggestions for network issues (Docker mirror, VPN, manual image transfer) - Improve error messages to differentiate between network failures and other errors * feat(docs): add Ubuntu 24.04 development setup guide Add a comprehensive development environment setup guide for Ubuntu 24.04. The guide includes steps to install dependencies via PPA, configure Docker with mirror accelerators for users in mainland China, set up user permissions, and run the server from source with a Python virtual environment. This provides a clear, step-by-step reference for new contributors and developers. * feat(docs): add pip mirror note for China mainland users Add a comment in the development setup documentation suggesting the use of the Aliyun PyPI mirror for users in China mainland to improve installation speed and reliability. This helps overcome network restrictions and slow downloads from the default PyPI repository. * feat(docs): add LVM root partition expansion guide Add optional section to development setup documentation with instructions for expanding the root partition when using LVM. This helps developers resolve low disk space issues by utilizing unallocated space in the volume group. The guide includes commands to check LVM status, extend the logical volume, and verify the changes. * feat(docs): add tshark to development setup dependencies Add tshark to the apt install command in the development setup documentation. Tshark is required for packet capture functionality in GNS3, ensuring the development environment has all necessary tools for network analysis and debugging. * feat: add GNS3 documentation skill with standardized structure Add new documentation skill file defining standards for GNS3 technical documentation. The skill establishes a structured approach focusing on architecture diagrams, flow diagrams, and text descriptions while prohibiting code examples and user scenarios. This ensures consistent, high-quality documentation across the GNS3 ecosystem. * feat(docs): update documentation skill to focus on server technical docs Update the GNS3 documentation skill to specifically target server technical documentation under the docs/ directory. The revised standard emphasizes high-level understanding over implementation details, using ASCII diagrams for architecture and business processes, API endpoint tables, and measured performance data. Code specifics are intentionally omitted, directing readers to the codebase for implementation details. * feat(docs): rewrite documentation skill and statistics API doc - Rewrite gns3-documentation skill to match actual server-side doc style: focus on architecture/flow diagrams, API data, skip code details - Switch diagram standard from ASCII to Mermaid (GitHub native rendering) - Rewrite statistics-api.md with Mermaid architecture and sequence diagrams, add missing webwireshark container fields (memory_limit, cpu_limit, pids_limit), remove speculative content Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(docs): update VNC WebSocket console documentation with Mermaid diagrams - Replace ASCII diagrams with Mermaid flowcharts for better visualization - Clarify connection flow between browser, controller, compute, and VNC server - Update authentication details and endpoint descriptions - Add missing documentation for packet capture workflow - Improve overall readability and maintainability * docs: add AI disclaimer to documentation files Add a standardized disclaimer to multiple documentation files indicating that the content has been organized by AI with reference to actual code. The disclaimer warns users that AI can make mistakes and advises verification against the source code when in doubt. This improves transparency about the documentation's origin and encourages careful usage. * feat(docs): update chat API documentation with new features and clarifications - Update title from "Design Document" to reflect current implementation status - Add new features: session abort, copilot modes (teaching_assistant, lab_automation_assistant), session pinning - Enhance architecture diagram with LangGraph StateGraph details including abort_handler_node, title_generator_node, and conditional edges - Clarify statistics tracking: add ai_response_counted flag, filter title_generator_node from LLM counts, explain incremental token counting - Improve documentation accuracy for real-time statistics collection and token calculation methods * feat(docs): restructure command security documentation with visual workflows - Replace verbose implementation details with concise architecture overview - Add Mermaid diagrams to visualize command filtering and multi-line expansion flows - Simplify configuration instructions and remove redundant examples - Consolidate file structure and function reference tables for clarity - Maintain all security principles while improving readability and maintainability * feat(context): refactor context window management documentation for clarity - Reorganize documentation with improved structure and visual diagrams - Add Mermaid diagrams to illustrate architecture and trimming process - Simplify content while maintaining technical accuracy - Update token counting and trimming strategy explanations - Enhance readability with better formatting and component tables * feat(docs): update LLM model configs documentation - Change config column type from JSONB to JSON (JSONB on PostgreSQL) - Update provider table to mark base_url as required with planned optional status - Add note about base_url being currently required in API schema - Clarify GET own configs endpoint returns plain array - Add copilot_mode field to update request schema - Document update limitations for context_strategy and copilot_mode fields * feat(docs): add GNS3StartNodeQuickTool and update node creation examples - Add documentation for new `GNS3StartNodeQuickTool` (`start_gns3_node_quick`) that starts nodes without waiting for boot completion - Update node creation example to include optional `name` field in template placement - Clarify dynamic wait time strategy for `GNS3StartNodeTool` and contrast with quick tool - Update tool file structure to reflect new configuration, display, and packet capture tools - Fix truncated line in suspend tool documentation * feat(docs): add comprehensive documentation structure with CC BY-SA 4.0 license Add initial documentation directory with detailed README.md that outlines: - Dual license structure (CC BY-SA 4.0 for docs, GPLv3 for code) - Complete directory structure for features, AI Copilot, and bugs - Feature documentation covering Controller+Compute setup, Statistics API, VNC WebSocket console, and Web Wireshark - AI Copilot implemented features including Chat API, LLM model configs, command security, and multi-vendor device support This provides organized technical documentation for the GNS3 server project with proper licensing and feature coverage. * docs: update multi-vendor device support documentation - Simplify vendor support table by removing redundant platform column - Add VPCS as a simulator in status column - Improve VPCS driver diagram with Mermaid syntax and detailed components - Document VPCS-specific Netmiko parameters (fast_cli, global_delay_factor) - Update VPCS tool usage example with connection options structure - Remove outdated Nornir configuration approaches, reference current implementation - Add file reference for VPCS tools location - Clean up documentation structure for better readability --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: YueGuobin <yueguobin@gmail.com> Co-authored-by: Jeremy Grossmann <grossmj@gns3.net>
This commit is contained in:
parent
80cc85e5d5
commit
f07d21c511
20
.claude/memory/MEMORY.md
Normal file
20
.claude/memory/MEMORY.md
Normal file
@ -0,0 +1,20 @@
|
||||
# GNS3 Server Project Memory
|
||||
|
||||
> **Note**: This directory stores important project-related memories and case studies, managed with the code repository.
|
||||
>
|
||||
> **How to record**: Use the `/memory` skill to record important information to the project memory directory.
|
||||
|
||||
## Quick Reference
|
||||
- **Memory directory**: `.claude/memory/`
|
||||
- **Skill file**: `.claude/skills/memory/SKILL.md`
|
||||
- **Main index**: `MEMORY.md` (this file)
|
||||
|
||||
## Topics
|
||||
|
||||
### Web Wireshark Integration
|
||||
- **[JWT Token Flow](./web-wireshark-jwt-token-flow.md)** - JWT token transmission path in Web Wireshark
|
||||
- Key point: UDPLink only passes through jwt_token, ultimately used by curl command inside Web Wireshark container to authenticate with GNS3 capture stream API
|
||||
- **[Xpra HTML5 Client](./xpra-html5-client.md)** - Xpra HTML5 client menu control parameters for customizing the web interface
|
||||
|
||||
### uBridge Permission
|
||||
- **[uBridge Permission Issue](./gns3-ubridge-permission.md)** - Docker containers fail to start due to missing CAP_NET_ADMIN/CAP_NET_RAW capabilities on uBridge
|
||||
25
.claude/memory/gns3-ubridge-permission.md
Normal file
25
.claude/memory/gns3-ubridge-permission.md
Normal file
@ -0,0 +1,25 @@
|
||||
# GNS3 uBridge Permission Issue
|
||||
|
||||
## Problem
|
||||
Docker container node fails to start with error: `uBridge requires root access or the capability to interact with network adapters`
|
||||
|
||||
## Root Cause
|
||||
- uBridge needs to create TAP network interfaces to connect Docker containers to GNS3 virtual network
|
||||
- This requires `CAP_NET_ADMIN` and `CAP_NET_RAW` capabilities
|
||||
- Other node types (like QEMU) may have their own network implementation and don't need uBridge
|
||||
|
||||
## Solution
|
||||
```bash
|
||||
sudo setcap cap_net_admin,cap_net_raw=eip /usr/bin/ubridge
|
||||
```
|
||||
|
||||
## Code Locations
|
||||
- uBridge path check: `gns3server/compute/base_manager.py:298` (`has_privileged_access`)
|
||||
- Permission check call: `gns3server/compute/base_node.py:851`
|
||||
- Error message definition: `gns3server/compute/base_node.py:852`
|
||||
|
||||
## Error Propagation Flow
|
||||
Error is propagated via `NodeError` exception:
|
||||
1. `NodeError` is caught at `gns3server/api/routes/compute/__init__.py:131-137`
|
||||
2. Returns HTTP 409 with JSON: `{"message": "...", "exception": "NodeError"}`
|
||||
3. Frontend should read error info from `message` field
|
||||
53
.claude/memory/web-wireshark-jwt-token-flow.md
Normal file
53
.claude/memory/web-wireshark-jwt-token-flow.md
Normal file
@ -0,0 +1,53 @@
|
||||
# Web Wireshark JWT Token Flow
|
||||
|
||||
## Background
|
||||
|
||||
Question: What is the purpose of `jwt_token` parameter in UDPLink and how is it transmitted?
|
||||
|
||||
## JWT Token Complete Flow
|
||||
|
||||
1. **User initiates request** → HTTP request with `Authorization: Bearer <jwt_token>`
|
||||
|
||||
2. **API layer extracts token** → `gns3server/api/routes/controller/links.py`:
|
||||
```python
|
||||
auth_header = http_request.headers.get("Authorization", "")
|
||||
jwt_token = auth_header.replace("Bearer ", "") if auth_header else None
|
||||
```
|
||||
|
||||
3. **Pass to Link layer** → `Link.start_capture(wireshark=True, jwt_token=xxx)`
|
||||
|
||||
4. **UDPLink forwards** → `UDPLink.start_capture()` calls `super().start_capture(jwt_token=jwt_token)`
|
||||
|
||||
5. **Start Web Wireshark container** → `Link._start_web_wireshark(jwt_token)` calls management script
|
||||
|
||||
6. **Container uses token** → `manage_wireshark.py` executes inside container:
|
||||
```bash
|
||||
curl -N -H 'Authorization: Bearer {jwt_token}' \
|
||||
'http://controller:3080/v3/projects/{project_id}/links/{link_id}/capture/stream' | \
|
||||
wireshark -i - -k -display :{display}
|
||||
```
|
||||
|
||||
## Why More Verbose Than gns3-copilot?
|
||||
|
||||
**gns3-copilot approach**:
|
||||
- All code within the same process
|
||||
- Uses `contextvars` to store and retrieve token
|
||||
- Downstream code directly calls `get_current_jwt_token()`, no need to pass through layers
|
||||
|
||||
**Web Wireshark approach**:
|
||||
- Token needs to be passed across processes (management script is separate process)
|
||||
- Token needs to be passed across containers (Docker container isolation)
|
||||
- Cannot use `contextvars`, must pass through command line arguments
|
||||
|
||||
## Conclusion
|
||||
|
||||
UDPLink does not use this token, it only forwards it to the parent class. Ultimately used by the curl command inside the Web Wireshark container to authenticate with the GNS3 controller's capture stream API.
|
||||
|
||||
Token flow: **Client → GNS3 API → Link → UDPLink → Web Wireshark container → GNS3 capture stream API**
|
||||
|
||||
## Related Files
|
||||
|
||||
- `gns3server/api/routes/controller/links.py:114-115` - JWT token extraction
|
||||
- `gns3server/controller/link.py:321-360` - Web Wireshark startup logic
|
||||
- `gns3server/controller/udp_link.py:178` - UDPLink start_capture signature
|
||||
- `gns3server/agent/web_wireshark/manage_wireshark.py:527-529` - Container curl command with JWT
|
||||
148
.claude/memory/xpra-html5-client.md
Normal file
148
.claude/memory/xpra-html5-client.md
Normal file
@ -0,0 +1,148 @@
|
||||
# Xpra HTML5 Client Configuration
|
||||
|
||||
## Overview
|
||||
|
||||
Web Wireshark integration uses xpra's built-in HTML5 client to provide browser-based packet capture viewing. The xpra HTML5 client is located at `/usr/share/xpra/www` in the container.
|
||||
|
||||
## Configuration Methods
|
||||
|
||||
### 1. URL Parameters (Per-Session)
|
||||
|
||||
Control toolbar menu items via URL query parameters (default: true):
|
||||
|
||||
| Parameter | Menu Item | Description |
|
||||
|-----------|-----------|-------------|
|
||||
| xpramenu | Xpra Menu | Main menu (Server, Information submenus) |
|
||||
| open_windows | Open Windows | List of open windows |
|
||||
| fullscreen_button | Fullscreen | Fullscreen toggle button |
|
||||
| keyboard_button | Keyboard | Keyboard layout/shortcuts |
|
||||
| clipboard_button | Clipboard Copy | Clipboard copy functionality |
|
||||
| sound_button | Audio | Audio toggle |
|
||||
| cursor_lock_button | Lock Cursor | Game cursor lock mode |
|
||||
|
||||
```bash
|
||||
# Examples
|
||||
?fullscreen_button=false&sound_button=false
|
||||
?xpramenu=false&keyboard_button=false&clipboard_button=false&sound_button=false&cursor_lock_button=false
|
||||
```
|
||||
|
||||
### 2. default-settings.txt (Recommended - Global/Persistent)
|
||||
|
||||
Modify `html5/default-settings.txt` (INI format) for all users.
|
||||
|
||||
**Features:**
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------|---------|-------------|
|
||||
| keyboard | auto-detect | Enable keyboard input |
|
||||
| keyboard_layout | us | Keyboard layout |
|
||||
| clipboard | yes | Clipboard sharing |
|
||||
| printing | yes | Printer forwarding |
|
||||
| file_transfer | yes | File transfer |
|
||||
| swap_keys | MacOS yes | Swap Command/Control keys |
|
||||
| scroll_reverse_x | no | Reverse mouse X-axis |
|
||||
| floating_menu | yes | Show floating menu |
|
||||
| toolbar_position | top-left | Toolbar position |
|
||||
| autohide | no | Auto-hide toolbar |
|
||||
| sound | yes | Audio forwarding |
|
||||
| video | 64-bit yes | Video decoding |
|
||||
|
||||
**Connection Options:**
|
||||
|
||||
| Parameter | Description |
|
||||
|-----------|-------------|
|
||||
| server | Server address |
|
||||
| port | Port number |
|
||||
| username | Username |
|
||||
| password | Password |
|
||||
| ssl | Enable SSL |
|
||||
| encryption | Encryption type (AES-CBC/CTR/CFB) |
|
||||
| key | AES encryption key |
|
||||
| sharing | Allow session sharing |
|
||||
| steal | Steal session |
|
||||
| reconnect | Auto-reconnect |
|
||||
| bandwidth_limit | Bandwidth limit (bits/s) |
|
||||
| override_width | Client desktop width |
|
||||
|
||||
**Advanced Options:**
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------|---------|-------------|
|
||||
| audio_codec | auto-detect | Audio codec |
|
||||
| encoding | auto | Image encoding (png/jpeg/webp/etc) |
|
||||
| remote_logging | yes | Send logs to server |
|
||||
| action | connect | Connection mode (start/shadow) |
|
||||
| submit | yes | Show diagnostics on disconnect |
|
||||
|
||||
### 3. default_settings in Code
|
||||
|
||||
Set directly in `<script>` tag in index.html:
|
||||
```javascript
|
||||
const default_settings = {};
|
||||
default_settings["xpramenu"] = false;
|
||||
```
|
||||
|
||||
### Priority Order
|
||||
|
||||
URL parameter > default_settings object > default-settings.txt > code default value
|
||||
|
||||
## Server-Controlled Submenu Items
|
||||
|
||||
The **Server submenu** items are controlled by server hello message:
|
||||
|
||||
| Menu Item ID | Content | Control Source |
|
||||
|--------------|---------|----------------|
|
||||
| clock_menu_entry | Clock | server-time in hello |
|
||||
| upload_menu_entry | Upload file | file-transfer or file.enabled |
|
||||
| download_menu_entry | Download file | file-transfer or file.enabled |
|
||||
| shutdown_menu_entry | Shutdown Server | client-shutdown in hello |
|
||||
|
||||
**Information submenu** (About Xpra, Session Info, Bug Report) and fixed items (Reload, Disconnect) are always displayed.
|
||||
|
||||
To hide "Shutdown Server", set environment variable before starting xpra:
|
||||
```bash
|
||||
export XPRA_CLIENT_CAN_SHUTDOWN=false
|
||||
xpra start :{display} ...
|
||||
```
|
||||
Code: `xpra/server/base.py:40` - `CLIENT_CAN_SHUTDOWN = envbool("XPRA_CLIENT_CAN_SHUTDOWN", True)`
|
||||
|
||||
## Customizing Background Image
|
||||
|
||||
Background image is defined in `html5/css/client.css`:
|
||||
```css
|
||||
body {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
overflow: hidden;
|
||||
background-color: #021d3a;
|
||||
background-image: url(../background.jpg);
|
||||
background-position: center center;
|
||||
background-repeat: no-repeat;
|
||||
background-size: cover;
|
||||
}
|
||||
```
|
||||
|
||||
### Methods
|
||||
|
||||
1. **Replace image file**: Create/replace `html5/background.jpg`
|
||||
2. **Modify CSS**: Change `html5/css/client.css` line 9:
|
||||
```css
|
||||
background-image: url(../your-image.jpg);
|
||||
```
|
||||
|
||||
### Notes
|
||||
- Supported formats: jpg, png, svg, etc.
|
||||
- Recommended resolution: 1920x1080 or higher
|
||||
- `background-size: cover` auto-stretches to fill
|
||||
|
||||
### Temporary Modification (via URL)
|
||||
|
||||
Cannot modify background via URL directly, but can use browser DevTools:
|
||||
```css
|
||||
body { background-image: url(your-image-url); }
|
||||
```
|
||||
|
||||
## Related Files
|
||||
|
||||
- Container Dockerfile: `gns3server/agent/web_wireshark/docker/Dockerfile`
|
||||
- Container HTML5 client path: `/usr/share/xpra/www`
|
||||
54
.claude/skills/gns3-documentation/SKILL.md
Normal file
54
.claude/skills/gns3-documentation/SKILL.md
Normal file
@ -0,0 +1,54 @@
|
||||
---
|
||||
name: documentation
|
||||
description: Use this skill when creating or updating technical documentation under docs/ directory for GNS3 server
|
||||
version: 3.0.0
|
||||
---
|
||||
|
||||
# GNS3 Server Technical Documentation Standard
|
||||
|
||||
## Core Principle
|
||||
|
||||
Documentation answers: **What is this, how does it work at a high level.**
|
||||
|
||||
Code details are left to the codebase — readers can use AI to find implementation specifics.
|
||||
|
||||
---
|
||||
|
||||
## Document Structure
|
||||
|
||||
```markdown
|
||||
# Feature Name
|
||||
|
||||
## Overview
|
||||
[What it does, in 2-3 sentences]
|
||||
|
||||
## Architecture
|
||||
[Mermaid diagram: components and their relationships]
|
||||
|
||||
## Business Process
|
||||
[Mermaid sequence/flowchart: key flows]
|
||||
|
||||
## API Endpoints
|
||||
[Table: Method | Path | Description | Privilege]
|
||||
|
||||
## Notes
|
||||
[Known limitations, performance data, security — only when relevant]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## What to Include
|
||||
|
||||
- Mermaid diagrams for architecture and flow (GitHub renders natively)
|
||||
- `graph` for component relationships
|
||||
- `sequenceDiagram` for request/response flows
|
||||
- `flowchart` for data processing steps
|
||||
- API endpoint tables
|
||||
- Request/response JSON examples (for interfaces)
|
||||
- Performance data (only measured numbers, no estimates)
|
||||
|
||||
## What to Skip
|
||||
|
||||
- Code snippets — let readers search the codebase
|
||||
- Verbose prose — use diagrams and tables
|
||||
- Implementation details — link file paths instead of pasting code
|
||||
98
.claude/skills/memory/SKILL.md
Normal file
98
.claude/skills/memory/SKILL.md
Normal file
@ -0,0 +1,98 @@
|
||||
---
|
||||
name: memory
|
||||
description: This skill should be used when the user asks to "record memory", "save to memory", "remember this", "create a memory", "add to memory", or discusses documenting technical decisions, architectural choices, or important case studies for the project. Use this skill to record project knowledge to the .claude/memory/ directory.
|
||||
version: 1.0.0
|
||||
---
|
||||
|
||||
# Memory Skill
|
||||
|
||||
## Overview
|
||||
|
||||
Record project-related technical decisions, important case studies, and lessons learned to the project memory directory, ensuring critical knowledge is managed with the code repository.
|
||||
|
||||
## When This Skill Activates
|
||||
|
||||
Activate this skill when user requests involve:
|
||||
- "Record to memory", "save to memory", "add to memory"
|
||||
- "Remember this", "create a memory"
|
||||
- Documenting technical decisions, architectural choices
|
||||
- Recording problem solutions and lessons learned
|
||||
- Discussing API design, performance optimization, security implementations
|
||||
|
||||
## Memory Location
|
||||
|
||||
Project memory is stored in: `.claude/memory/`
|
||||
|
||||
## Memory Structure
|
||||
|
||||
```
|
||||
.claude/
|
||||
├── memory/
|
||||
│ ├── MEMORY.md # Main index file
|
||||
│ └── <topic>.md # Detailed records for specific topics
|
||||
└── skills/
|
||||
└── memory/
|
||||
└── SKILL.md # This file
|
||||
```
|
||||
|
||||
## What to Record
|
||||
|
||||
### ✅ Should Record
|
||||
|
||||
- Important architectural decisions and rationale
|
||||
- Solutions to complex problems
|
||||
- Key considerations for API design
|
||||
- Performance optimization experiences
|
||||
- Security-related implementation details
|
||||
- Integration approaches with other systems
|
||||
- Common errors and solutions
|
||||
- Design changes based on user feedback
|
||||
|
||||
### ❌ Should Not Record
|
||||
|
||||
- Temporary debugging information
|
||||
- Session-specific state
|
||||
- Obvious code implementation details (code is documentation)
|
||||
- Content already documented elsewhere
|
||||
|
||||
## Memory Template
|
||||
|
||||
```markdown
|
||||
# <Topic Title>
|
||||
|
||||
## Background
|
||||
<Why this record exists>
|
||||
|
||||
## Decision/Implementation
|
||||
<What was done or how it was implemented>
|
||||
|
||||
## Rationale
|
||||
<Why this approach was chosen, what alternatives were considered>
|
||||
|
||||
## Related Files
|
||||
<Code files involved, use file_path:line_number format>
|
||||
|
||||
## Examples
|
||||
<If there are code examples or other examples>
|
||||
```
|
||||
|
||||
## Recording Process
|
||||
|
||||
1. **Identify content to record**
|
||||
2. **Determine topic title** (concise, descriptive)
|
||||
3. **Create record using template**
|
||||
4. **Update MEMORY.md index**
|
||||
5. **Save to .claude/memory/<topic>.md**
|
||||
|
||||
## Examples
|
||||
|
||||
- `web-wireshark-jwt-token-flow.md` - JWT token flow in Web Wireshark integration
|
||||
- `docker-api-version-detection.md` - Docker API version dynamic detection implementation
|
||||
|
||||
## Best Practices
|
||||
|
||||
- Use clear, descriptive filenames
|
||||
- Include references to related code files (with line numbers)
|
||||
- Document "why" not just "what"
|
||||
- Keep records updated, remove outdated ones
|
||||
- Link to related memory files
|
||||
7
.gitignore
vendored
7
.gitignore
vendored
@ -69,8 +69,13 @@ venv
|
||||
.ropeproject
|
||||
|
||||
# Claude Code settings (may contain API keys)
|
||||
.claude/
|
||||
.claude/settings*.json
|
||||
.claude/minmax-settings.json
|
||||
.claude/zhipu-settings.json
|
||||
.claude/settings.local.json
|
||||
!.claude/development.md # Exception: allow development docs
|
||||
!.claude/skills/ # Exception: allow project skills
|
||||
!.claude/memory/ # Exception: allow project memory
|
||||
# PROJECT_CONTEXT.md # Commented out: allow tracking project context
|
||||
|
||||
# Tiktoken cache files
|
||||
|
||||
@ -69,6 +69,12 @@ AI-powered assistant for network topology design and automation.
|
||||
python3 -m pip install gns3-server[dev]
|
||||
```
|
||||
|
||||
**Web Wireshark** (Optional):
|
||||
```shell
|
||||
pip install gns3-server && gns3-wireshark-setup
|
||||
```
|
||||
Browser-based packet capture analysis using Wireshark in a Docker container.
|
||||
|
||||
**Combination Installation**:
|
||||
You can install multiple optional features together:
|
||||
```shell
|
||||
|
||||
443
docs/LICENSE
Normal file
443
docs/LICENSE
Normal file
@ -0,0 +1,443 @@
|
||||
Creative Commons Attribution-ShareAlike 4.0 International
|
||||
|
||||
Copyright (c) 2026 GNS3 Web UI Project
|
||||
|
||||
=======================================================================
|
||||
|
||||
Creative Commons Attribution-ShareAlike 4.0 International
|
||||
|
||||
Copyright (C) 2025 Creative Commons Corporation
|
||||
|
||||
Creative Commons Corporation ("Creative Commons") is not a law firm and
|
||||
does not provide legal services or legal advice. Distribution of
|
||||
Creative Commons public licenses does not create a lawyer-client or
|
||||
other relationship. Creative Commons makes its licenses and related
|
||||
information available on an "as-is" basis. Creative Commons gives no
|
||||
warranties regarding its licenses, any material licensed under their
|
||||
terms and conditions, or any related information. Creative Commons
|
||||
disclaims all liability for damages resulting from their use to the
|
||||
fullest extent possible.
|
||||
|
||||
Using Creative Commons Public Licenses
|
||||
|
||||
Creative Commons public licenses provide a standard set of terms and
|
||||
conditions that creators and other rights holders may use to share
|
||||
original works of authorship and other material subject to copyright
|
||||
and certain other rights specified in the public license below. The
|
||||
following considerations are for informational purposes only, are not
|
||||
exhaustive, and do not form part of our licenses.
|
||||
|
||||
Considerations for licensors: Our public licenses are
|
||||
intended for use by those authorized to give the public
|
||||
permission to use material in ways otherwise restricted by
|
||||
copyright and certain other rights. Our licenses are
|
||||
irrevocable. Licensors should read and understand the terms
|
||||
and conditions of the license they choose before applying it.
|
||||
Licensors should also secure all rights necessary before
|
||||
applying our licenses so that the public can reuse the
|
||||
material as expected. Licensors should clearly mark any
|
||||
material not subject to the license. This includes other CC-
|
||||
licensed material, or material used under an exception or
|
||||
limitation to copyright. More considerations for licensors:
|
||||
wiki.creativecommons.org/Considerations_for_licensors
|
||||
|
||||
Considerations for the public: Our public licenses are
|
||||
intended for use by those authorized to give the public
|
||||
permission to use material in ways otherwise restricted by
|
||||
copyright and certain other rights. Our licenses are
|
||||
irrevocable. Licensors should read and understand the terms
|
||||
and conditions of the license they choose before applying it.
|
||||
Licensors should also secure all rights necessary before
|
||||
applying our licenses so that the public can reuse the
|
||||
material as expected. Licensors should clearly mark any
|
||||
material not subject to the license. This includes other CC-
|
||||
licensed material, or material used under an exception or
|
||||
limitation to copyright. More considerations for licensors:
|
||||
wiki.creativecommons.org/Considerations_for_licensors
|
||||
|
||||
Creative Commons Attribution-ShareAlike 4.0 International Public
|
||||
License
|
||||
|
||||
By exercising the Licensed Rights (defined below), You accept and agree
|
||||
to be bound by the terms and conditions of this Creative Commons
|
||||
Attribution-ShareAlike 4.0 International Public License ("Public
|
||||
License"). To the extent this Public License may be interpreted as a
|
||||
contract, You are granted the Licensed Rights in consideration of Your
|
||||
acceptance of these terms and conditions, and the Licensor grants You
|
||||
such rights in consideration of benefits the Licensor receives from
|
||||
making the Licensed Material available under these terms and conditions.
|
||||
|
||||
Section 1 – Definitions.
|
||||
|
||||
a. Adapted Material means material subject to Copyright and Similar
|
||||
Rights that is derived from or based upon the Licensed Material
|
||||
and in which the Licensed Material is translated, altered,
|
||||
arranged, transformed, or otherwise modified in a manner requiring
|
||||
permission under the Copyright and Similar Rights held by the
|
||||
Licensor. For purposes of this Public License, where the Licensed
|
||||
Material is a musical work, performance, or sound recording,
|
||||
Adapted Material is always produced where the Licensed Material is
|
||||
synched in timed relation with a moving image.
|
||||
|
||||
b. Adapter's License means the license You apply to Your Copyright
|
||||
and Similar Rights in Your contributions to Adapted Material in
|
||||
accordance with the terms and conditions of this Public License.
|
||||
|
||||
c. Copyright and Similar Rights means copyright and/or similar rights
|
||||
closely related to copyright including, without limitation,
|
||||
publication, distribution, public performance, public display, and
|
||||
adaptation rights, and rights in database extensions and
|
||||
collections.
|
||||
|
||||
d. Licensed Material means the artistic or literary work, database,
|
||||
or other material to which the Licensor applied this Public
|
||||
License.
|
||||
|
||||
e. Licensed Rights means rights granted to You subject to the terms
|
||||
and conditions of this Public License, which are limited to all
|
||||
Copyright and Similar Rights that apply to Your use of the
|
||||
Licensed Material and that the Licensor has authority to license.
|
||||
|
||||
f. Licensor means the individual(s) or entity(ies) granting rights
|
||||
under this Public License.
|
||||
|
||||
g. Share means to provide material to the public by any means or
|
||||
process that requires permission under the Licensed Rights, such
|
||||
as reproduction, public display, public performance, distribution,
|
||||
dissemination, communication, or importation, and to make material
|
||||
available to the public including in ways that members of the
|
||||
public may access the material from a place and at a time
|
||||
individually chosen by them.
|
||||
|
||||
h. Sui Generis Database Rights means rights other than copyright
|
||||
resulting from Directive 96/9/EC of the European Parliament and
|
||||
of the Council of 11 March 1996 on the legal protection of
|
||||
databases, as amended and/or succeeded, as well as other
|
||||
essentially equivalent rights anywhere in the world.
|
||||
|
||||
i. You means the individual or entity exercising the Licensed Rights
|
||||
under this Public License. Your has a corresponding meaning.
|
||||
|
||||
Section 2 – Scope.
|
||||
|
||||
a. License grant.
|
||||
|
||||
1. Subject to the terms and conditions of this Public License,
|
||||
the Licensor hereby grants You a worldwide, royalty-free,
|
||||
non-sublicensable, non-exclusive, irrevocable license to
|
||||
exercise the Licensed Rights in the Licensed Material to:
|
||||
|
||||
a. reproduce and Share the Licensed Material, in whole or
|
||||
in part; and
|
||||
|
||||
b. produce, reproduce, and Share Adapted Material.
|
||||
|
||||
2. Exceptions and Limitations. For the avoidance of doubt,
|
||||
where Exceptions and Limitations apply to Your use, this
|
||||
Public License does not apply, and You do not need to
|
||||
comply with its terms and conditions.
|
||||
|
||||
3. Term. The term of this Public License is specified in
|
||||
Section 6(a).
|
||||
|
||||
4. Media and formats; technical modifications allowed. The
|
||||
Licensor authorizes You to exercise the Licensed Rights in
|
||||
all media and formats whether now known or hereafter created,
|
||||
and to make technical modifications necessary to do so. The
|
||||
Licensor waives and/or agrees not to assert any right or
|
||||
authority to forbid You from making technical modifications
|
||||
necessary to exercise the Licensed Rights, including
|
||||
technical modifications necessary to circumvent Effective
|
||||
Technological Measures. For purposes of this Public License,
|
||||
simply making modifications authorized by this Section 2(a)
|
||||
(4) never produces Adapted Material.
|
||||
|
||||
5. Downstream recipients.
|
||||
|
||||
a. Offer from the Licensor – Licensed Material. Every
|
||||
recipient of the Licensed Material automatically
|
||||
receives an offer from the Licensor to exercise the
|
||||
Licensed Rights under the terms and conditions of this
|
||||
Public License.
|
||||
|
||||
b. Additional offer from the Licensor – Adapted Material.
|
||||
Every recipient of Adapted Material from You
|
||||
automatically receives an offer from the Licensor to
|
||||
exercise the Licensed Rights in the Adapted Material
|
||||
under the conditions of the Adapter's License You
|
||||
apply.
|
||||
|
||||
6. No endorsement. Nothing in this Public License constitutes or
|
||||
may be construed as permission to assert or imply that You
|
||||
are, or that Your use of the Licensed Material is, connected
|
||||
with, or sponsored, endorsed, or granted official status by,
|
||||
the Licensor or others designated to receive attribution as
|
||||
provided in Section 3(a)(1)(A)(i).
|
||||
|
||||
b. Other rights. Moral rights, such as the right of integrity, are
|
||||
not licensed under this Public License, nor are publicity,
|
||||
privacy, and/or other similar personality rights; however, to the
|
||||
extent possible, the Licensor waives and/or agrees not to assert
|
||||
any such rights held by the Licensor to the limited extent
|
||||
necessary to allow You to exercise the Licensed Rights, but not
|
||||
otherwise.
|
||||
|
||||
c. Patent and trademark rights are not licensed under this Public
|
||||
License.
|
||||
|
||||
d. To the extent possible, the Licensor waives any right to collect
|
||||
royalties from You for the exercise of the Licensed Rights,
|
||||
whether directly or through a collecting society under any
|
||||
voluntary or waivable statutory or compulsory licensing scheme.
|
||||
In all other cases the Licensor expressly reserves any right to
|
||||
collect such royalties.
|
||||
|
||||
Section 3 – License Conditions.
|
||||
|
||||
Your exercise of the Licensed Rights is expressly made subject to
|
||||
the following conditions.
|
||||
|
||||
a. Attribution.
|
||||
|
||||
1. If You Share the Licensed Material (including in modified
|
||||
form), You must:
|
||||
|
||||
a. retain the following if it is supplied by the
|
||||
Licensor with the Licensed Material:
|
||||
|
||||
i. identification of the creator(s) of the
|
||||
Licensed Material and any others designated to
|
||||
receive attribution, in any reasonable manner
|
||||
requested by the Licensor (including by
|
||||
pseudonym if designated);
|
||||
|
||||
ii. a copyright notice;
|
||||
|
||||
iii. a notice that refers to this Public License;
|
||||
|
||||
iv. a notice that refers to the disclaimer of
|
||||
warranties;
|
||||
|
||||
v. a URI or hyperlink to the Licensed Material to
|
||||
the extent reasonably practicable;
|
||||
|
||||
b. indicate if You modified the Licensed Material and
|
||||
retain an indication of any previous modifications; and
|
||||
|
||||
c. indicate the Licensed Material is licensed under this
|
||||
Public License, and include the text of, or the URI
|
||||
or hyperlink to, this Public License.
|
||||
|
||||
2. You may satisfy the conditions in Section 3(a)(1) in any
|
||||
reasonable manner based on the medium, means, and context in
|
||||
which You Share the Licensed Material. For example, it may
|
||||
be reasonable to satisfy the conditions by providing a URI
|
||||
or hyperlink to a resource that includes the required
|
||||
information.
|
||||
|
||||
3. If requested by the Licensor, You must remove any of the
|
||||
information required by Section 3(a)(1)(A) to the extent
|
||||
reasonably practicable.
|
||||
|
||||
4. If You Share Adapted Material You produce, the Adapter's
|
||||
License You apply must not prevent recipients of the
|
||||
Adapted Material from complying with this Public License.
|
||||
|
||||
b. ShareAlike.
|
||||
|
||||
In addition to the conditions in Section 3(a), if You Share
|
||||
Adapted Material You produce, the following conditions also apply.
|
||||
|
||||
1. The Adapter's License You apply must be a Creative Commons
|
||||
license with the same License Elements, this version or
|
||||
later, or a BY-SA Compatible License.
|
||||
|
||||
2. You must include the text of, or the URI or hyperlink to,
|
||||
the Adapter's License You apply. You may satisfy this
|
||||
condition in any reasonable manner based on the medium,
|
||||
means, and context in which You Share Adapted Material.
|
||||
|
||||
3. You may not offer or impose any additional or different terms
|
||||
or conditions on, or apply any Effective Technological
|
||||
Measures to, Adapted Material that restrict exercise of the
|
||||
rights granted under the Adapter's License You apply.
|
||||
|
||||
Section 4 – Sui Generis Database Rights.
|
||||
|
||||
Where the Licensed Rights include Sui Generis Database Rights that
|
||||
apply to Your use of the Licensed Material:
|
||||
|
||||
a. for the avoidance of doubt, Section 2(a)(1) grants You the right
|
||||
to extract, reuse, reproduce, and Share all or a substantial
|
||||
portion of the contents of the database;
|
||||
|
||||
b. if You include all or a substantial portion of the database
|
||||
contents in a database in which You have Sui Generis Database
|
||||
Rights, then the database in which You have Sui Generis Database
|
||||
Rights (but not its individual contents) is Adapted Material; and
|
||||
|
||||
c. You must comply with the conditions in Section 3(a) if You Share
|
||||
all or a substantial portion of the contents of the database.
|
||||
|
||||
For the avoidance of doubt, this Section 4 supplements and does not
|
||||
replace Your obligations under this Public License where the Licensed
|
||||
Rights include other Copyright and Similar Rights.
|
||||
|
||||
Section 5 – Disclaimer of Warranties and Limitation of Liability.
|
||||
|
||||
a. Unless otherwise separately undertaken by the Licensor, to the
|
||||
extent possible, the Licensor offers the Licensed Material as-is
|
||||
and as-available, and makes no representations or warranties of
|
||||
any kind concerning the Licensed Material, whether express,
|
||||
implied, statutory, or other. This includes, without limitation,
|
||||
warranties of title, merchantability, fitness for a particular
|
||||
purpose, non-infringement, absence of latent or other defects,
|
||||
accuracy, or the presence or absence of errors, whether or not
|
||||
known or discoverable. Where disclaimers of warranties are not
|
||||
allowed in full or in part, this disclaimer may not apply to You.
|
||||
|
||||
b. To the extent possible, in no event will the Licensor be liable
|
||||
to You on any legal theory (including, without limitation,
|
||||
negligence) or otherwise for any direct, special, indirect,
|
||||
incidental, consequential, punitive, exemplary, or other losses,
|
||||
costs, expenses, or damages that arise out of this Public License
|
||||
or use of the Licensed Material, even if the Licensor has been
|
||||
advised of the possibility of such losses, costs, expenses, or
|
||||
damages. Where a limitation of liability is not allowed in full
|
||||
or in part, this limitation may not apply to You.
|
||||
|
||||
c. The disclaimer of warranties and limitation of liability provided
|
||||
above shall be interpreted in a manner that, to the extent
|
||||
possible, most closely approximates an absolute disclaimer and
|
||||
waiver of all liability.
|
||||
|
||||
Section 6 – Term and Termination.
|
||||
|
||||
a. This Public License applies for the term of the Copyright and
|
||||
Similar Rights licensed here. However, if You fail to comply
|
||||
with this Public License, then Your rights under this Public
|
||||
License terminate automatically.
|
||||
|
||||
b. Where Your right to use the Licensed Material has terminated
|
||||
under Section 6(a), it reinstates:
|
||||
|
||||
1. automatically as of the date the violation is cured,
|
||||
provided it is cured within 30 days of Your discovery of
|
||||
the violation; or
|
||||
|
||||
2. upon express reinstatement by the Licensor.
|
||||
|
||||
For the avoidance of doubt, this Section 6(b) does not affect any
|
||||
right the Licensor may have to seek remedies for Your violations
|
||||
of this Public License.
|
||||
|
||||
c. For the avoidance of doubt, the Licensor may also offer the
|
||||
Licensed Material under separate terms or conditions or stop
|
||||
distributing the Licensed Material at any time; however, doing so
|
||||
will not terminate this Public License.
|
||||
|
||||
d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
|
||||
License.
|
||||
|
||||
Section 7 – Other Terms and Conditions.
|
||||
|
||||
a. The Licensor shall not be bound by any additional or different
|
||||
terms or conditions communicated by You unless expressly agreed.
|
||||
|
||||
b. Any arrangements, understandings, or agreements regarding the
|
||||
Licensed Material not stated herein are separate from and
|
||||
independent of the terms and conditions of this Public License.
|
||||
|
||||
Section 8 – Interpretation.
|
||||
|
||||
a. For the avoidance of doubt, this Public License does not, and
|
||||
shall not be interpreted to, reduce, limit, restrict, or impose
|
||||
conditions on any use of the Licensed Material that could lawfully
|
||||
be made without permission under this Public License.
|
||||
|
||||
b. To the extent possible, if any provision of this Public License
|
||||
is deemed unenforceable, it shall be automatically reformed to
|
||||
the minimum extent necessary to make it enforceable. If the
|
||||
provision cannot be reformed, it shall be severed from this
|
||||
Public License without affecting the enforceability of the
|
||||
remaining terms and conditions.
|
||||
|
||||
c. No term or condition of this Public License will be waived and
|
||||
no failure to comply consented to unless expressly agreed to by
|
||||
the Licensor.
|
||||
|
||||
d. Nothing in this Public License constitutes or may be interpreted
|
||||
as a limitation upon, or waiver of, any privileges and immunities
|
||||
that apply to the Licensor or You, including from the legal
|
||||
processes of any jurisdiction or authority.
|
||||
|
||||
=======================================================================
|
||||
|
||||
Creative Commons is not a party to its public licenses. Notwithstanding,
|
||||
Creative Commons may elect to apply one of its public licenses to
|
||||
material it publishes and in those instances will be considered the
|
||||
"Licensor." The text of the Creative Commons public licenses is
|
||||
dedicated to the public domain under the CC0 Public Domain Dedication.
|
||||
The text of the Creative Commons public licenses is dedicated to the
|
||||
public domain under the CC0 Public Domain Dedication. Except for the
|
||||
limited purpose of indicating that material is shared under a Creative
|
||||
Commons public license or as otherwise permitted by the Creative Commons
|
||||
policies published at creativecommons.org/policies, Creative Commons
|
||||
does not authorize the use of the trademark "Creative Commons" or any
|
||||
other trademark or logo of Creative Commons without its prior written
|
||||
consent including, without limitation, in connection with any
|
||||
unauthorized modifications to any of its public licenses or any other
|
||||
arrangements, understandings, or agreements concerning use of licensed
|
||||
material. For the avoidance of doubt, this paragraph does not form part
|
||||
of the public licenses.
|
||||
|
||||
Creative Commons may be contacted at creativecommons.org.
|
||||
|
||||
=======================================================================
|
||||
|
||||
For the GNS3 Web UI documentation:
|
||||
|
||||
- Documentation License: CC BY-SA 4.0
|
||||
- Project License: GPLv3 (see root LICENSE file)
|
||||
- License URL: https://creativecommons.org/licenses/by-sa/4.0/
|
||||
|
||||
All documentation files in this directory are licensed under the
|
||||
Creative Commons Attribution-ShareAlike 4.0 International License
|
||||
(CC BY-SA 4.0).
|
||||
|
||||
=======================================================================
|
||||
|
||||
IMPORTANT: ShareAlike Requirements for Software Derivatives
|
||||
|
||||
Under the CC BY-SA 4.0 ShareAlike condition, if you create derivative
|
||||
works based on this documentation, including software that is considered
|
||||
a derivative of the documentation, you must:
|
||||
|
||||
1. **License your derivative work under CC BY-SA 4.0 or a compatible
|
||||
license** (such as GPLv3, which is compatible with CC BY-SA 4.0)
|
||||
|
||||
2. **Attribute the original documentation** to the GNS3 Web UI Project
|
||||
|
||||
3. **Indicate if you modified the documentation** and retain
|
||||
indications of previous modifications
|
||||
|
||||
4. **Include the license notice** and link to the CC BY-SA 4.0 license
|
||||
|
||||
What constitutes a derivative work:
|
||||
- Translations of the documentation
|
||||
- Adaptations, modifications, or enhancements to the documentation
|
||||
- Software that incorporates substantial portions of the documentation
|
||||
- Works based on the documentation's structure, organization, or content
|
||||
|
||||
Compatible licenses include:
|
||||
- CC BY-SA 4.0 or later
|
||||
- GPLv3 (GNU General Public License version 3)
|
||||
- AGPLv3 (GNU Affero General Public License version 3)
|
||||
- Other licenses approved as compatible by Creative Commons
|
||||
|
||||
For more information on CC BY-SA compatibility, see:
|
||||
https://creativecommons.org/compatiblelicenses
|
||||
|
||||
If you are unsure whether your use constitutes a derivative work or
|
||||
which license to apply, please consult the full CC BY-SA 4.0 license
|
||||
text above or seek legal advice.
|
||||
117
docs/README.md
Normal file
117
docs/README.md
Normal file
@ -0,0 +1,117 @@
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
|
||||
|
||||
# GNS3 Server Documentation
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
This documentation is licensed under the **Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0)**.
|
||||
|
||||
⚠️ **Important - ShareAlike Requirement**: If you create derivative works based on this documentation (including software that incorporates substantial portions of the documentation), your work must also be licensed under **CC BY-SA 4.0 or a compatible license** (such as GPLv3).
|
||||
|
||||
- 📄 **Full License Text**: See [docs/LICENSE](./LICENSE)
|
||||
- 🔗 **License URL**: https://creativecommons.org/licenses/by-sa/4.0/
|
||||
- 📖 **Compatibility**: https://creativecommons.org/compatiblelicenses
|
||||
|
||||
**Dual License Structure**:
|
||||
- 📚 **Documentation**: CC BY-SA 4.0 (this directory)
|
||||
- 💻 **Software Code**: GPLv3 (see root [LICENSE](../LICENSE))
|
||||
|
||||
---
|
||||
|
||||
Technical documentation for the GNS3 server project, covering features, AI Copilot, development setup, and known issues.
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
docs/
|
||||
├── README.md # This file
|
||||
├── development-setup.md # Ubuntu 24.04 development environment setup
|
||||
├── openapi.json # OpenAPI specification
|
||||
├── features/ # Feature documentation
|
||||
│ ├── compute-controller-setup.md # Controller + Compute architecture & configuration
|
||||
│ ├── statistics-api.md # Aggregated statistics API for monitoring
|
||||
│ ├── vnc-websocket-console.md # Browser-based VNC console via WebSocket
|
||||
│ └── web-wireshark-business-process.md # Web Wireshark (Docker + xpra packet capture)
|
||||
├── gns3-copilot/ # AI Copilot feature documentation
|
||||
│ ├── netmiko_devices.md # Netmiko supported devices (366 types)
|
||||
│ ├── template-based-configuration-roadmap.md # Future: template-based config with HITL
|
||||
│ └── implemented/ # Implemented features
|
||||
│ ├── chat-api.md # Chat API (SSE, session management)
|
||||
│ ├── llm-model-configs.md # LLM model configuration system
|
||||
│ ├── command-security.md # Command security and filtering
|
||||
│ ├── context-window-management.md # Context window optimization
|
||||
│ ├── node-control-tools.md # Node start/stop/suspend tools
|
||||
│ └── multi-vendor-device-support.md # Multi-vendor device support
|
||||
└── bugs/ # Known issues & bug reports
|
||||
└── telnet-server-connection-race-condition.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Features
|
||||
|
||||
### Controller + Compute Setup (`features/compute-controller-setup.md`)
|
||||
Architecture and minimum configuration for setting up GNS3 Controller with remote Compute nodes. Covers compute node config, controller registration, and multi-compute deployment.
|
||||
|
||||
### Statistics API (`features/statistics-api.md`)
|
||||
Aggregated server statistics API (`GET /v3/statistics`) for monitoring dashboards. Collects compute resources, project/node/link counts, and Web Wireshark container status in a single request.
|
||||
|
||||
### VNC WebSocket Console (`features/vnc-websocket-console.md`)
|
||||
Browser-based VNC console access via WebSocket. The Controller acts as WebSocket-to-WebSocket relay, and Compute bridges WebSocket to TCP for QEMU/Docker VMs. Supports noVNC clients.
|
||||
|
||||
### Web Wireshark (`features/web-wireshark-business-process.md`)
|
||||
Web-based packet capture analysis using Docker + xpra HTML5 client. Zero-install Wireshark experience directly in the browser, integrated with GNS3 topologies.
|
||||
|
||||
---
|
||||
|
||||
## GNS3 AI Copilot (`gns3-copilot/`)
|
||||
|
||||
### Implemented Features
|
||||
|
||||
| Feature | Description | Status |
|
||||
|---------|-------------|--------|
|
||||
| [Chat API](gns3-copilot/implemented/chat-api.md) | SSE streaming, session management, token statistics | Implemented |
|
||||
| [LLM Model Configs](gns3-copilot/implemented/llm-model-configs.md) | Multi-level model config (system/group/user) | Implemented |
|
||||
| [Command Security](gns3-copilot/implemented/command-security.md) | Command filtering, dangerous operation detection, HITL | Implemented |
|
||||
| [Context Window Management](gns3-copilot/implemented/context-window-management.md) | Token optimization, content filtering, compression | Implemented |
|
||||
| [Node Control Tools](gns3-copilot/implemented/node-control-tools.md) | Start/stop/suspend with batch ops and progress tracking | Implemented |
|
||||
| [Multi-Vendor Support](gns3-copilot/implemented/multi-vendor-device-support.md) | Cisco, Huawei, Ruijie, VPCS with custom Netmiko drivers | Implemented |
|
||||
|
||||
### Reference
|
||||
|
||||
- [Netmiko Supported Devices](gns3-copilot/netmiko_devices.md) — 366 device types (154 SSH, 55 Telnet, 3 custom GNS3 drivers)
|
||||
|
||||
### Roadmap
|
||||
|
||||
- [Template-Based Configuration with HITL](gns3-copilot/template-based-configuration-roadmap.md) — Jinja2 templates with human-in-the-loop confirmations for device configuration and node creation
|
||||
|
||||
---
|
||||
|
||||
## Known Issues (`bugs/`)
|
||||
|
||||
- [Telnet Server Connection Race Condition](bugs/telnet-server-connection-race-condition.md) — `getpeername()` error when client disconnects during connection setup (High severity, Open)
|
||||
|
||||
---
|
||||
|
||||
## Development Setup (`development-setup.md`)
|
||||
|
||||
Quick-start guide for Ubuntu 24.04: install via PPA, set up dependencies, and run gns3-server from source.
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [GNS3 Server API Documentation](https://api.gns3.com/) — Interactive API docs (also available locally via `redoc.html`)
|
||||
- [GNS3 Web UI Documentation](https://docs.gns3.com/)
|
||||
- [LangGraph Documentation](https://langchain-ai.github.io/langgraph/)
|
||||
|
||||
---
|
||||
|
||||
_Last updated: 2026-04-20_
|
||||
@ -1,3 +1,11 @@
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
|
||||
|
||||
|
||||
# Telnet Server Connection Race Condition Bug
|
||||
|
||||
## Bug Report
|
||||
|
||||
146
docs/development-setup.md
Normal file
146
docs/development-setup.md
Normal file
@ -0,0 +1,146 @@
|
||||
# Ubuntu 24.04 GNS3 Development Environment Setup
|
||||
|
||||
## Simplified Development Environment Setup
|
||||
|
||||
### 1. Add PPA
|
||||
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install software-properties-common
|
||||
sudo add-apt-repository ppa:gns3/ppa
|
||||
sudo apt update
|
||||
```
|
||||
|
||||
### 2. Install Dependencies
|
||||
|
||||
Binary tools (Dynamips, VPCS, uBridge, etc.) are automatically installed as gns3-server dependencies
|
||||
|
||||
```bash
|
||||
sudo apt install gns3-server tshark
|
||||
```
|
||||
|
||||
### 3. Remove gns3-server While Keeping Dependencies
|
||||
|
||||
If you only want to run the server from source:
|
||||
|
||||
```bash
|
||||
# Remove main package only, keep dependencies
|
||||
sudo apt remove gns3-server
|
||||
|
||||
# Mark dependencies as manual to prevent autoremove deletion
|
||||
sudo apt-mark manual ubridge dynamips vpcs libvirt
|
||||
```
|
||||
|
||||
### 4. Install and Start Docker
|
||||
|
||||
```bash
|
||||
sudo apt install docker.io
|
||||
sudo systemctl enable --now docker
|
||||
```
|
||||
|
||||
#### Configure Docker Mirror Accelerator (China Mainland)
|
||||
|
||||
If Docker Hub is inaccessible, configure mirror accelerators:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /etc/docker
|
||||
sudo tee /etc/docker/daemon.json <<'EOF'
|
||||
{
|
||||
"registry-mirrors": [
|
||||
"https://docker.1ms.run",
|
||||
"https://docker.xuanyuan.me"
|
||||
]
|
||||
}
|
||||
EOF
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl restart docker
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
docker info | grep -i mirror
|
||||
```
|
||||
|
||||
### 5. Add User to Groups
|
||||
|
||||
```bash
|
||||
# If running as gns3 user (created by PPA install):
|
||||
sudo usermod -aG ubridge,docker,kvm gns3
|
||||
|
||||
# If running as your own user, add yourself to these groups:
|
||||
sudo usermod -aG ubridge,docker,kvm $USER
|
||||
|
||||
# Then logout and login again for group membership to take effect
|
||||
```
|
||||
|
||||
### 6. Install Python venv
|
||||
|
||||
```bash
|
||||
sudo apt install python3.12-venv
|
||||
```
|
||||
|
||||
### 7. Run Development Version from Source
|
||||
|
||||
Clone the repository (official or your fork):
|
||||
|
||||
```bash
|
||||
# Official repository
|
||||
git clone https://github.com/GNS3/gns3-server.git
|
||||
|
||||
# Or your forked repository
|
||||
git clone https://github.com/yourname/gns3-server.git
|
||||
cd gns3-server
|
||||
git checkout your-branch
|
||||
```
|
||||
|
||||
Setup virtual environment and install dependencies:
|
||||
|
||||
```bash
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
|
||||
# For China mainland users, use mirror:
|
||||
# pip install -e . -i https://mirrors.aliyun.com/pypi/simple/
|
||||
|
||||
pip install -e . && gns3-wireshark-setup
|
||||
pip install -e .[ai-copilot]
|
||||
pip install -e .[dev]
|
||||
```
|
||||
|
||||
Run the server:
|
||||
|
||||
```bash
|
||||
python3 -m gns3server
|
||||
```
|
||||
|
||||
## Optional: Install AI Copilot Development Dependencies
|
||||
|
||||
```bash
|
||||
python3 -m pip install .[ai-copilot,dev]
|
||||
```
|
||||
|
||||
## Optional: Expand LVM Root Partition
|
||||
|
||||
If the root partition is low on space but the volume group has unallocated space:
|
||||
|
||||
Check LVM status:
|
||||
|
||||
```bash
|
||||
sudo lvm pvdisplay
|
||||
sudo lvm vgdisplay
|
||||
sudo lvm lvdisplay
|
||||
```
|
||||
|
||||
Extend the root LV using all free space:
|
||||
|
||||
```bash
|
||||
sudo lvm lvextend -l +100%FREE /dev/mapper/ubuntu--vg-ubuntu--lv
|
||||
sudo resize2fs /dev/mapper/ubuntu--vg-ubuntu--lv
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
df -h
|
||||
```
|
||||
@ -1,3 +1,11 @@
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
|
||||
|
||||
|
||||
# Controller + Compute Setup
|
||||
|
||||
This document describes the minimum configuration required to set up a GNS3 Controller with remote Compute nodes.
|
||||
|
||||
@ -1,23 +1,94 @@
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
|
||||
|
||||
|
||||
# Statistics API
|
||||
|
||||
## `GET /statistics`
|
||||
## Overview
|
||||
|
||||
Returns aggregated server statistics including compute resources, projects, nodes, and links.
|
||||
Aggregated server statistics API for monitoring dashboards. Collects compute resources, project/node/link counts, and Web Wireshark container status in a single request.
|
||||
|
||||
**Authentication:** Requires active user session
|
||||
## Architecture
|
||||
|
||||
**Method:** `GET`
|
||||
```mermaid
|
||||
graph TB
|
||||
Client["Client (Dashboard)"]
|
||||
|
||||
**URL:** `http://server:3080/v1/statistics`
|
||||
subgraph Controller["GET /v3/statistics"]
|
||||
direction TB
|
||||
F1["1. Compute Resources<br/>GET compute/statistics"]
|
||||
F2["2. Project/Node/Link<br/>Iterate in memory"]
|
||||
F3["3. Web Wireshark<br/>Docker API + docker stats"]
|
||||
end
|
||||
|
||||
### Response
|
||||
Compute["Compute Nodes<br/>(psutil)"]
|
||||
Memory["Controller<br/>(in-memory data)"]
|
||||
Docker["Docker Daemon"]
|
||||
|
||||
Client -->|GET /v3/statistics| Controller
|
||||
F1 -->|HTTP| Compute
|
||||
F2 -->|read| Memory
|
||||
F3 -->|Docker API| Docker
|
||||
```
|
||||
|
||||
## Data Collection Flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Client
|
||||
participant Controller as Controller API
|
||||
participant Compute as Compute Nodes
|
||||
participant Docker as Docker Daemon
|
||||
|
||||
Client->>Controller: GET /v3/statistics
|
||||
|
||||
rect rgb(240, 248, 255)
|
||||
Note over Controller: 1. Compute Resources
|
||||
loop For each compute
|
||||
Controller->>Compute: GET /statistics
|
||||
Compute-->>Controller: psutil data
|
||||
Note over Controller: skip on error
|
||||
end
|
||||
end
|
||||
|
||||
rect rgb(240, 255, 240)
|
||||
Note over Controller: 2. Projects / Nodes / Links
|
||||
Note over Controller: Count projects by status (opened/closed)
|
||||
Note over Controller: Count nodes: open→by_type+by_status, closed→by_type only
|
||||
Note over Controller: Count links: total + capturing
|
||||
end
|
||||
|
||||
rect rgb(255, 248, 240)
|
||||
Note over Controller: 3. Web Wireshark
|
||||
loop For opened projects only
|
||||
Controller->>Docker: Query container status
|
||||
Controller->>Docker: docker stats (2s timeout)
|
||||
Docker-->>Controller: Container resource usage
|
||||
end
|
||||
end
|
||||
|
||||
Controller-->>Client: JSON response
|
||||
```
|
||||
|
||||
## API Endpoints
|
||||
|
||||
| Method | Path | Description | Auth |
|
||||
|--------|------|-------------|------|
|
||||
| GET | `/v3/statistics` | Aggregated server statistics | Session |
|
||||
| GET | `/v3/compute/statistics` | Single compute resource stats | Basic Auth |
|
||||
|
||||
## Response
|
||||
|
||||
```json
|
||||
{
|
||||
"computes": [
|
||||
{
|
||||
"compute_id": "string",
|
||||
"compute_name": "string",
|
||||
"compute_id": "local",
|
||||
"compute_name": "Local",
|
||||
"statistics": {
|
||||
"memory_total": 16777216000,
|
||||
"memory_free": 8000000000,
|
||||
@ -42,135 +113,124 @@ Returns aggregated server statistics including compute resources, projects, node
|
||||
"total": 42,
|
||||
"open_project_nodes": 30,
|
||||
"closed_project_nodes": 12,
|
||||
"by_type": {
|
||||
"qemu": 20,
|
||||
"docker": 12,
|
||||
"dynamips": 6,
|
||||
"vpcs": 4
|
||||
},
|
||||
"by_status": {
|
||||
"started": 25,
|
||||
"stopped": 12,
|
||||
"suspended": 5
|
||||
}
|
||||
"by_type": { "qemu": 20, "docker": 12, "dynamips": 6, "vpcs": 4 },
|
||||
"by_status": { "started": 25, "stopped": 12, "suspended": 5 }
|
||||
},
|
||||
"links": {
|
||||
"total": 38,
|
||||
"capturing": 2
|
||||
},
|
||||
"webwireshark": {
|
||||
"total_containers": 1,
|
||||
"running_containers": 1,
|
||||
"active_sessions": 2,
|
||||
"containers": [
|
||||
{
|
||||
"project_id": "e16e2b51-9ba9-403b-9df4-b2915d7508a3",
|
||||
"project_name": "test-project",
|
||||
"container_id": "6edc9029bac0",
|
||||
"status": "running",
|
||||
"running": true,
|
||||
"active_sessions": 2,
|
||||
"memory_limit": "2.0 GB",
|
||||
"cpu_limit": "1.0",
|
||||
"pids_limit": 1000,
|
||||
"memory": "272.5MiB / 4GiB",
|
||||
"cpu": "0.23%",
|
||||
"pids": 69
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Field Descriptions
|
||||
## Field Reference
|
||||
|
||||
#### `computes`
|
||||
|
||||
Array of compute node statistics. Each compute reports:
|
||||
### `computes[]`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `compute_id` | string | Unique identifier for the compute |
|
||||
| `compute_name` | string | Human-readable name |
|
||||
| `statistics` | object | Resource usage statistics |
|
||||
| `compute_id` | string | Compute identifier |
|
||||
| `compute_name` | string | Display name |
|
||||
| `statistics` | object | Resource usage (see below) |
|
||||
|
||||
#### `computes[].statistics`
|
||||
**`statistics` fields:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `memory_total` | integer | Total physical memory in bytes |
|
||||
| `memory_free` | integer | Free memory in bytes |
|
||||
| `memory_used` | integer | Used memory in bytes |
|
||||
| `swap_total` | integer | Total swap space in bytes |
|
||||
| `swap_free` | integer | Free swap space in bytes |
|
||||
| `swap_used` | integer | Used swap space in bytes |
|
||||
| `cpu_usage_percent` | integer | CPU usage percentage (0-100) |
|
||||
| `memory_usage_percent` | integer | Memory usage percentage (0-100) |
|
||||
| `swap_usage_percent` | integer | Swap usage percentage (0-100) |
|
||||
| `disk_usage_percent` | integer | Disk usage percentage for project directory (0-100) |
|
||||
| `load_average_percent` | integer[] | Load average as percentage per CPU core (1/5/15 min) |
|
||||
| `memory_total` | int | Total RAM (bytes) |
|
||||
| `memory_free` | int | Available RAM (bytes) |
|
||||
| `memory_used` | int | Used RAM (bytes) |
|
||||
| `swap_total` | int | Total swap (bytes) |
|
||||
| `swap_free` | int | Free swap (bytes) |
|
||||
| `swap_used` | int | Used swap (bytes) |
|
||||
| `cpu_usage_percent` | int | CPU usage 0-100 |
|
||||
| `memory_usage_percent` | int | RAM usage 0-100 |
|
||||
| `swap_usage_percent` | int | Swap usage 0-100 |
|
||||
| `disk_usage_percent` | int | Project dir disk usage 0-100 |
|
||||
| `load_average_percent` | int[] | Load avg per core (1/5/15 min) |
|
||||
|
||||
#### `projects`
|
||||
### `projects`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `total` | integer | Total number of projects |
|
||||
| `opened` | integer | Number of projects currently opened |
|
||||
| `closed` | integer | Number of projects currently closed |
|
||||
| `total` | int | All projects |
|
||||
| `opened` | int | Currently opened |
|
||||
| `closed` | int | Currently closed |
|
||||
|
||||
#### `nodes`
|
||||
### `nodes`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `total` | integer | Total number of nodes across all projects |
|
||||
| `open_project_nodes` | integer | Nodes in opened projects (has real status) |
|
||||
| `closed_project_nodes` | integer | Nodes in closed projects (loaded from topology JSON, no status) |
|
||||
| `by_type` | object | Node count grouped by node type (qemu, docker, dynamips, etc.) |
|
||||
| `by_status` | object | Node count grouped by status (only for `open_project_nodes`) |
|
||||
| `total` | int | All nodes across projects |
|
||||
| `open_project_nodes` | int | Nodes in opened projects (has runtime status) |
|
||||
| `closed_project_nodes` | int | Nodes in closed projects (from topology JSON, no status) |
|
||||
| `by_type` | object | Count by node type (qemu, docker, etc.) |
|
||||
| `by_status` | object | Count by status — **open project nodes only** |
|
||||
|
||||
**Note on `by_status`:** Status is a runtime attribute only available for nodes in opened projects. Closed projects store topology data in JSON format which does not include runtime status. Therefore `by_status` only reflects nodes from opened projects.
|
||||
|
||||
Valid node statuses: `started`, `stopped`, `suspended`
|
||||
|
||||
#### `links`
|
||||
### `links`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `total` | integer | Total number of links across all projects |
|
||||
| `capturing` | integer | Number of links currently capturing packets |
|
||||
| `total` | int | All links across projects |
|
||||
| `capturing` | int | Links currently capturing |
|
||||
|
||||
### Example Usage
|
||||
### `webwireshark`
|
||||
|
||||
```bash
|
||||
# Get statistics
|
||||
curl -X GET http://localhost:3080/v1/statistics \
|
||||
-H "Authorization: Bearer <token>"
|
||||
```
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `total_containers` | int | All Wireshark containers |
|
||||
| `running_containers` | int | Currently running |
|
||||
| `active_sessions` | int | Active capture sessions |
|
||||
| `containers` | array | Per-container details (opened projects only) |
|
||||
|
||||
### Dashboard Integration
|
||||
**`containers[]` fields:**
|
||||
|
||||
This API is designed for monitoring dashboards that need:
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `project_id` | string | Project UUID |
|
||||
| `project_name` | string | Project name |
|
||||
| `container_id` | string | Docker container ID (12 chars) |
|
||||
| `status` | string | Container status (running, exited, etc.) |
|
||||
| `running` | bool | Is running |
|
||||
| `active_sessions` | int | Active captures in this project |
|
||||
| `memory_limit` | string | Configured memory limit (e.g. `"2.0 GB"`, `"unlimited"`) |
|
||||
| `cpu_limit` | string | Configured CPU limit (e.g. `"1.0"`, `"unlimited"`) |
|
||||
| `pids_limit` | int/string | Process limit (e.g. `1000`, `"unlimited"`) |
|
||||
| `memory` | string | Live memory usage (conditional, see notes) |
|
||||
| `cpu` | string | Live CPU usage (conditional, see notes) |
|
||||
| `pids` | int | Live process count (conditional, see notes) |
|
||||
|
||||
- **System health**: CPU, memory, disk from `computes[].statistics`
|
||||
- **Project overview**: Project counts from `projects`
|
||||
- **Node inventory**: Node counts by type and status from `nodes`
|
||||
- **Capture monitoring**: Active capture sessions from `links.capturing`
|
||||
|
||||
### Error Responses
|
||||
## Error Responses
|
||||
|
||||
| Status | Description |
|
||||
|--------|-------------|
|
||||
| 401 | Unauthorized - invalid or missing session |
|
||||
| 401 | Unauthorized — invalid or missing session |
|
||||
| 500 | Internal server error |
|
||||
|
||||
### Future Optimizations
|
||||
## Notes
|
||||
|
||||
#### Per-Compose Node Statistics
|
||||
|
||||
Currently `nodes` are aggregated globally. Future enhancement could add per-compute breakdown:
|
||||
|
||||
```json
|
||||
"nodes": {
|
||||
"total": 42,
|
||||
"by_compute": {
|
||||
"local": {
|
||||
"total": 30,
|
||||
"open_project_nodes": 20,
|
||||
"closed_project_nodes": 10,
|
||||
"by_type": { "qemu": 20, "docker": 10 }
|
||||
},
|
||||
"remote-server-1": {
|
||||
"total": 12,
|
||||
"open_project_nodes": 10,
|
||||
"closed_project_nodes": 2,
|
||||
"by_type": { "docker": 12 }
|
||||
}
|
||||
},
|
||||
"by_type": { "qemu": 20, "docker": 22 },
|
||||
"open_project_nodes": 30,
|
||||
"closed_project_nodes": 12,
|
||||
"by_type": { "qemu": 20, "docker": 22 },
|
||||
"by_status": { "started": 25, "stopped": 12, "suspended": 5 }
|
||||
}
|
||||
```
|
||||
|
||||
This requires tracking which compute each node runs on (Node._compute).
|
||||
- **Compute stats are best-effort**: if a compute is unreachable, it is skipped (logged as error) and other data is still returned
|
||||
- **`by_status` only reflects open project nodes**: closed projects store topology in JSON without runtime status
|
||||
- **Live container stats** (`memory`, `cpu`, `pids`) are only present when `docker stats` succeeds — may be absent if the Docker daemon is slow (2s timeout)
|
||||
- **Web Wireshark containers** are only queried for opened projects
|
||||
|
||||
@ -1,3 +1,11 @@
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
|
||||
|
||||
|
||||
# VNC WebSocket Console
|
||||
|
||||
## Overview
|
||||
@ -10,20 +18,15 @@ This implementation uses GNS3's API layer as a transparent WebSocket-to-TCP prox
|
||||
|
||||
### Connection Flow
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
A[Browser noVNC] -->|WebSocket binary| B[GNS3 Controller]
|
||||
B -->|WebSocket + BasicAuth + SSL| C[GNS3 Compute]
|
||||
C -->|TCP| D[QEMU/Docker VNC :5900]
|
||||
```
|
||||
┌─────────┐ WebSocket ┌─────────────┐ HTTP/WS ┌──────────┐
|
||||
│ Browser │ ◄─────────────────► │ GNS3 │ ◄──────────────► │ GNS3 │
|
||||
│ noVNC │ (wss://port) │ Controller │ (JWT + RBAC) │ Compute │
|
||||
└─────────┘ └─────────────┘ └─────┬────┘
|
||||
│
|
||||
│ TCP
|
||||
│
|
||||
┌────▼────┐
|
||||
│ QEMU │
|
||||
│ VNC │
|
||||
│ :5900 │
|
||||
└─────────┘
|
||||
```
|
||||
|
||||
**Controller** acts as a WebSocket-to-WebSocket relay (JWT auth, IPv6 handling).
|
||||
**Compute** acts as a WebSocket-to-TCP bridge (validates node state, opens VNC TCP connection).
|
||||
|
||||
### Components
|
||||
|
||||
@ -35,13 +38,14 @@ This implementation uses GNS3's API layer as a transparent WebSocket-to-TCP prox
|
||||
2. **GNS3 Controller API**
|
||||
- WebSocket endpoint: `/v3/projects/{project_id}/nodes/{node_id}/console/vnc`
|
||||
- Authentication: JWT token via query parameter
|
||||
- Authorization: RBAC privilege check (`Node.Console`)
|
||||
- Forwards WebSocket connections to compute node
|
||||
- Authorization: RBAC privilege check via `has_privilege_on_websocket("Node.Console")`
|
||||
- Proxies WebSocket to compute node (WebSocket-to-WebSocket relay)
|
||||
- Handles IPv6 addresses by wrapping in brackets
|
||||
|
||||
3. **GNS3 Compute API**
|
||||
- WebSocket endpoint: `/v3/compute/projects/{project_id}/qemu/nodes/{node_id}/console/vnc`
|
||||
- Authentication: HTTP Basic Auth
|
||||
- Transparent bidirectional binary forwarding
|
||||
- WebSocket endpoint: `/v3/compute/projects/{project_id}/{node_type}/nodes/{node_id}/console/vnc`
|
||||
- Authentication: HTTP Basic Auth via `ws_compute_authentication()`
|
||||
- Establishes TCP connection to VNC server, bridges WebSocket ↔ TCP
|
||||
|
||||
4. **Node (QEMU/Docker)**
|
||||
- VNC server listening on configured port (default: 5900+)
|
||||
@ -75,8 +79,8 @@ ws.binaryType = 'arraybuffer';
|
||||
**URL**: `ws://{compute_host}:{port}/v3/compute/projects/{project_id}/{node_type}/nodes/{node_id}/console/vnc`
|
||||
|
||||
**Authentication**:
|
||||
- HTTP Basic Auth (controller credentials)
|
||||
- Configured via `settings.Server.compute_username` and `settings.Server.compute_password`
|
||||
- HTTP Basic Auth via `ws_compute_authentication()` dependency
|
||||
- Configured via `settings.Server.compute_username` (default: `gns3`) and `settings.Server.compute_password` (default: empty)
|
||||
|
||||
**Response**:
|
||||
- Bidirectional binary WebSocket connection
|
||||
@ -125,120 +129,56 @@ ws.binaryType = 'arraybuffer';
|
||||
|
||||
## WebSocket Data Forwarding
|
||||
|
||||
### Implementation Details
|
||||
### Compute Layer (WebSocket ↔ TCP)
|
||||
|
||||
**Location**: `gns3server/compute/base_node.py:544-612`
|
||||
**Location**: `gns3server/compute/base_node.py` — `start_vnc_websocket_console()`
|
||||
|
||||
```python
|
||||
async def start_vnc_websocket_console(self, websocket):
|
||||
"""Connect to VNC console using WebSocket."""
|
||||
1. Validates node is started and `console_type == "vnc"`; closes WebSocket with code 1000 otherwise
|
||||
2. Opens TCP connection to VNC server at `console_host:console_port`
|
||||
3. Runs two concurrent tasks via `asyncio.wait(FIRST_COMPLETED)`:
|
||||
- `ws_forward()`: WebSocket → TCP (catches `WebSocketDisconnect`)
|
||||
- `vnc_forward()`: TCP → WebSocket (reads 65536-byte buffer)
|
||||
4. Cancels pending tasks, closes TCP writer on completion
|
||||
|
||||
# 1. Validation
|
||||
if self.status != "started":
|
||||
await websocket.close(code=1000)
|
||||
raise NodeError(f"Node {self.name} is not started")
|
||||
if self._console_type != "vnc":
|
||||
await websocket.close(code=1000)
|
||||
raise NodeError(f"Node {self.name} console type is not vnc")
|
||||
### Controller Layer (WebSocket ↔ WebSocket)
|
||||
|
||||
# 2. Connect to VNC server
|
||||
vnc_reader, vnc_writer = await asyncio.open_connection(
|
||||
self._manager.port_manager.console_host,
|
||||
self.console
|
||||
)
|
||||
**Location**: `gns3server/api/routes/controller/nodes.py` — `vnc_console()`
|
||||
|
||||
# 3. Bidirectional forwarding
|
||||
async def ws_forward(vnc_writer):
|
||||
# Browser → VNC: Forward WebSocket data to VNC server
|
||||
while True:
|
||||
data = await websocket.receive_bytes()
|
||||
if data:
|
||||
vnc_writer.write(data)
|
||||
await vnc_writer.drain()
|
||||
|
||||
async def vnc_forward(vnc_reader):
|
||||
# VNC → Browser: Forward VNC data to WebSocket
|
||||
while not vnc_reader.at_eof():
|
||||
data = await vnc_reader.read(4096)
|
||||
if data:
|
||||
await websocket.send_bytes(data)
|
||||
|
||||
# 4. Run both forwarding tasks
|
||||
aws = [
|
||||
asyncio.create_task(ws_forward(vnc_writer)),
|
||||
asyncio.create_task(vnc_forward(vnc_reader))
|
||||
]
|
||||
|
||||
done, pending = await asyncio.wait(aws, return_when=asyncio.FIRST_COMPLETED)
|
||||
|
||||
# 5. Cleanup
|
||||
for task in pending:
|
||||
task.cancel()
|
||||
vnc_writer.close()
|
||||
await vnc_writer.wait_closed()
|
||||
```
|
||||
1. Authenticates user via `has_privilege_on_websocket("Node.Console")` dependency
|
||||
2. Constructs compute URL with IPv6 bracket handling
|
||||
3. Connects to compute WebSocket using `aiohttp.ws_connect()` with HTTP Basic Auth and SSL context
|
||||
4. Uses `asyncio.ensure_future()` for client→compute forwarding, `async for msg` iteration for compute→client
|
||||
|
||||
### Data Flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant B as Browser (noVNC)
|
||||
participant C as Controller
|
||||
participant W as Compute
|
||||
participant V as VNC Server
|
||||
|
||||
B->>C: WebSocket connect (binary, JWT token)
|
||||
C->>W: WebSocket connect (binary, BasicAuth, SSL)
|
||||
W->>V: TCP connect (asyncio.open_connection)
|
||||
|
||||
Note over B,V: Bidirectional binary forwarding active
|
||||
|
||||
B->>C: WebSocket binary frame
|
||||
C->>W: aiohttp send_bytes()
|
||||
W->>V: TCP write(data)
|
||||
|
||||
V->>W: TCP data (read 65536)
|
||||
W->>C: WebSocket binary frame
|
||||
C->>B: send_bytes()
|
||||
```
|
||||
Browser (noVNC) GNS3 Compute QEMU VNC
|
||||
│ │ │
|
||||
│ WebSocket Frame (Binary) │ │
|
||||
├─────────────────────────────────►│ │
|
||||
│ receive_bytes() │ │
|
||||
│ │ TCP Socket │
|
||||
│ ├───────────────────────────►│
|
||||
│ │ write(data) │
|
||||
│ │ │
|
||||
│ │ TCP Socket │
|
||||
│ │◄──────────────────────────┤
|
||||
│ WebSocket Frame (Binary) │ read(4096) │
|
||||
│◄─────────────────────────────────┤ │
|
||||
│ send_bytes(data) │ │
|
||||
```
|
||||
|
||||
## Frontend Integration
|
||||
|
||||
### noVNC Integration
|
||||
|
||||
**Location**: `gns3-web-ui/src/assets/vnc-console/`
|
||||
|
||||
**Files**:
|
||||
- `index.html` - VNC console page
|
||||
- `vnc-controller.js` - VNC connection controller
|
||||
- `novnc/` - noVNC library files
|
||||
|
||||
**Connection Example**:
|
||||
```javascript
|
||||
const sc = new RFB(document.getElementById('vnc-canvas'), {
|
||||
target: vncWsUrl, // ws://controller:port/v3/projects/.../console/vnc?token=...
|
||||
credentials: { password: vncPassword }
|
||||
});
|
||||
|
||||
sc.addEventListener('connect', () => {
|
||||
console.log('VNC connected');
|
||||
});
|
||||
|
||||
sc.addEventListener('disconnect', (e) => {
|
||||
console.log('VNC disconnected:', e);
|
||||
});
|
||||
```
|
||||
|
||||
### Console Service
|
||||
|
||||
**Location**: `gns3-web-ui/src/app/services/vnc-console.service.ts`
|
||||
|
||||
**Methods**:
|
||||
- `buildVncWebSocketUrl()` - Construct WebSocket URL
|
||||
- `openVncConsole()` - Open console in new window
|
||||
- `buildVncConsolePageUrl()` - Build standalone page URL
|
||||
|
||||
## Authentication & Authorization
|
||||
|
||||
### Controller Layer
|
||||
|
||||
1. **Authentication**:
|
||||
- JWT token validation via `get_current_active_user_from_websocket()`
|
||||
- JWT token validation via `has_privilege_on_websocket("Node.Console")` dependency
|
||||
- Token passed as query parameter: `?token={jwt}`
|
||||
|
||||
2. **Authorization**:
|
||||
@ -277,9 +217,20 @@ port = 3080
|
||||
```ini
|
||||
[Server]
|
||||
compute_username = gns3
|
||||
compute_password = gns3
|
||||
compute_password = # empty by default, must be set for compute auth
|
||||
```
|
||||
|
||||
### VNC Port Range
|
||||
|
||||
VNC console ports are allocated from a configurable range (`gns3server/schemas/config.py`):
|
||||
|
||||
| Setting | Default | Range |
|
||||
|---------|---------|-------|
|
||||
| `vnc_console_start_port_range` | 5900 | 5900–65535 |
|
||||
| `vnc_console_end_port_range` | 10000 | 5900–65535 |
|
||||
|
||||
Validation: `vnc_console_end_port_range` must be greater than `vnc_console_start_port_range`.
|
||||
|
||||
### Node Settings
|
||||
|
||||
**QEMU VM Example**:
|
||||
@ -376,32 +327,6 @@ grep "Connected to VNC server" /var/log/gns3/gns3.log
|
||||
RFB.messages.log = function(msg) { console.log(msg); };
|
||||
```
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Bandwidth
|
||||
|
||||
- **Typical Usage**: 1-5 Mbps per active VNC session
|
||||
- **Full HD (1920x1080)**: Up to 10 Mbps with rapid screen changes
|
||||
- **Optimization**: Use lower resolution for slower connections
|
||||
|
||||
### Latency
|
||||
|
||||
- **Target**: < 50ms for local connections
|
||||
- **Factors**:
|
||||
- Network latency
|
||||
- WebSocket frame processing overhead
|
||||
- VNC encoding efficiency
|
||||
- **Mitigation**:
|
||||
- Use QXL driver for QEMU VMs
|
||||
- Enable VNC password authentication (reduces overhead)
|
||||
- Adjust console resolution
|
||||
|
||||
### Concurrent Connections
|
||||
|
||||
- **Multiple Clients**: Each VNC console supports one WebSocket connection
|
||||
- **Multi-viewer**: Not supported (VNC protocol limitation)
|
||||
- **Shared Sessions**: Use SPICE for multi-client support
|
||||
|
||||
## Security
|
||||
|
||||
### Authentication
|
||||
@ -412,8 +337,9 @@ RFB.messages.log = function(msg) { console.log(msg); };
|
||||
- Privilege: `Node.Console`
|
||||
|
||||
2. **Controller → Compute**
|
||||
- HTTP Basic Auth over TLS (recommended)
|
||||
- Separate compute credentials
|
||||
- HTTP Basic Auth via `aiohttp.BasicAuth`
|
||||
- SSL context from `Controller.instance().ssl_context()`
|
||||
- Raises `ControllerForbiddenError` if `compute_username` is not set
|
||||
|
||||
3. **VNC Server**
|
||||
- Optional VNC password (QEMU only)
|
||||
@ -453,50 +379,13 @@ qemu-system-x86_64 -vnc :0,password
|
||||
- New connections disconnect existing clients
|
||||
|
||||
2. **No Audio Redirection**
|
||||
- VNC doesn't support audio
|
||||
- Use SPICE for audio support
|
||||
- VNC protocol does not support audio
|
||||
|
||||
3. **No USB Redirection**
|
||||
- VNC doesn't support device redirection
|
||||
- Use SPICE for USB support
|
||||
- VNC protocol does not support device redirection
|
||||
|
||||
4. **Performance**
|
||||
- Higher CPU usage than native VNC clients
|
||||
- Binary forwarding adds processing overhead
|
||||
|
||||
## Comparison with SPICE
|
||||
|
||||
| Feature | VNC WebSocket | SPICE WebSocket |
|
||||
|---------|---------------|-----------------|
|
||||
| **Browser Support** | ✅ Excellent | ✅ Good (with spice-html5) |
|
||||
| **Audio Redirection** | ❌ No | ✅ Yes |
|
||||
| **USB Redirection** | ❌ No | ✅ Yes |
|
||||
| **Clipboard Sharing** | ⚠️ Limited | ✅ Full |
|
||||
| **Multi-monitor** | ✅ Yes | ✅ Yes |
|
||||
| **Performance** | ⚠️ Moderate | ✅ Better |
|
||||
| **Guest Agent** | ❌ No | ✅ Yes (spice-vdagent) |
|
||||
| **Stability** | ✅ Very Stable | ⚠️ Moderate |
|
||||
| **Implementation** | ✅ Complete | ❌ Removed (dependencies) |
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
Planned improvements:
|
||||
|
||||
1. **WebSocket Compression**
|
||||
- Enable per-message compression
|
||||
- Reduce bandwidth usage
|
||||
|
||||
2. **Connection Pooling**
|
||||
- Reuse WebSocket connections
|
||||
- Reduce connection overhead
|
||||
|
||||
3. **Recording Support**
|
||||
- Record VNC sessions
|
||||
- Playback functionality
|
||||
|
||||
4. **Multi-viewer Mode**
|
||||
- Read-only shared viewing
|
||||
- Teacher/student scenarios
|
||||
4. **Performance Overhead**
|
||||
- WebSocket-to-TCP bridging adds processing overhead compared to native VNC clients
|
||||
|
||||
## References
|
||||
|
||||
@ -509,5 +398,5 @@ Planned improvements:
|
||||
|
||||
| Version | Date | Changes |
|
||||
|---------|------|---------|
|
||||
| 1.1 | 2026-04-19 | Review against code: fix buffer size, auth details, add port range config, remove unverified content |
|
||||
| 1.0 | 2026-03-17 | Initial VNC WebSocket documentation |
|
||||
| 0.9 | 2026-03-15 | Added VNC WebSocket support to QEMU and Docker |
|
||||
|
||||
583
docs/features/web-wireshark-business-process.md
Normal file
583
docs/features/web-wireshark-business-process.md
Normal file
@ -0,0 +1,583 @@
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
|
||||
|
||||
|
||||
# Web Wireshark Feature - Business Process Documentation
|
||||
|
||||
## Overview
|
||||
|
||||
The **Web Wireshark** feature enables users to run Wireshark packet capture analysis directly in a web browser without requiring a desktop environment or VNC connection. This is achieved through an **xpra** (persistent remote applications) HTML5 client running inside a Docker container.
|
||||
|
||||
## Feature Summary
|
||||
|
||||
- **Core Capability**: Web-based packet capture visualization using Wireshark
|
||||
- **Technology Stack**: Docker container + xpra + HTML5 WebSocket proxy
|
||||
- **Target Users**: Network engineers and students who need to analyze network traffic in GNS3 topologies
|
||||
- **Key Benefit**: Zero-install, browser-based packet capture analysis
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
Before using Web Wireshark, install the GNS3 server and set up the Docker image:
|
||||
|
||||
```bash
|
||||
# Development install
|
||||
pip install -e . && gns3-wireshark-setup
|
||||
|
||||
# Production install
|
||||
pip install gns3-server && gns3-wireshark-setup
|
||||
```
|
||||
|
||||
This command will:
|
||||
1. Install the gns3-server package
|
||||
2. Pull the `gns3/web-wireshark:latest` image from Docker Hub
|
||||
3. If pull fails, build the image locally using the included Dockerfile
|
||||
|
||||
The `gns3-wireshark-setup` command shows the raw output from `docker pull` or `docker build`, allowing you to see the full progress.
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph GNS3Server["GNS3 Server"]
|
||||
WebUI["Web UI<br/>(Browser)"]
|
||||
Controller["GNS3 Controller"]
|
||||
LinkCtrl["Link Controller<br/>(start_capture / stop_capture)"]
|
||||
Manager["WebWiresharkManager<br/>(called directly, not via CLI)"]
|
||||
DockerClient["DockerHTTPClient<br/>(HTTP via Unix Socket)"]
|
||||
end
|
||||
|
||||
subgraph DockerDaemon["Docker Daemon"]
|
||||
Container["gns3-wireshark-{project_id}"]
|
||||
Xvfb["Xvfb (virtual framebuffer)"]
|
||||
Xpra["xpra server"]
|
||||
Wireshark["Wireshark"]
|
||||
end
|
||||
|
||||
ClientBrowser["Client Browser<br/>(HTML5 Client)"]
|
||||
|
||||
ClientBrowser -->|REST API| WebUI
|
||||
WebUI -->|capture/start| Controller
|
||||
Controller --> LinkCtrl
|
||||
LinkCtrl -->|start/stop/restart session| Manager
|
||||
Manager -->|Docker API<br/>/var/run/docker.sock| DockerClient
|
||||
DockerClient -->|container lifecycle| Container
|
||||
Container --- Xvfb
|
||||
Container --- Xpra
|
||||
Container --- Wireshark
|
||||
ClientBrowser -.->|WebSocket proxy| Xpra
|
||||
Xpra -.->|xpra HTML5| ClientBrowser
|
||||
|
||||
style GNS3Server fill:#e8f4fd,stroke:#2196f3
|
||||
style DockerDaemon fill:#fff3e0,stroke:#ff9800
|
||||
style ClientBrowser fill:#e8f5e9,stroke:#4caf50
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Component Description
|
||||
|
||||
### 1. Web UI (Client Browser)
|
||||
- User interface for starting/stopping packet capture
|
||||
- Receives WebSocket URL for connecting to xpra HTML5 client
|
||||
- No plugins required - pure HTML5/JavaScript
|
||||
|
||||
### 2. GNS3 Controller
|
||||
- Orchestrates the capture workflow
|
||||
- Validates user permissions (RBAC)
|
||||
- Manages link capture state
|
||||
|
||||
### 3. manage_wireshark.py (Management CLI)
|
||||
- Command-line interface for container and session management
|
||||
- **For manual debugging and testing only** — the server calls `WebWiresharkManager` directly at runtime
|
||||
- Handles Docker container lifecycle
|
||||
- Manages xpra sessions per link
|
||||
|
||||
### 4. WebWiresharkManager
|
||||
- Core business logic for Web Wireshark
|
||||
- Handles container creation, session startup/shutdown
|
||||
- Deterministic port allocation based on link_id
|
||||
|
||||
### 5. DockerHTTPClient
|
||||
- Async HTTP client for Docker API
|
||||
- Communicates via Unix socket (/var/run/docker.sock)
|
||||
- Manages container lifecycle
|
||||
|
||||
### 6. Docker Container (gns3-wireshark-{project_id})
|
||||
- Runs xpra server with HTML5 support
|
||||
- Contains Xvfb (virtual framebuffer) for headless Wireshark
|
||||
- Streams display to browser via WebSocket
|
||||
|
||||
---
|
||||
|
||||
## Business Processes
|
||||
|
||||
### Process 1: Start Packet Capture with Web Wireshark
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
actor User
|
||||
participant WebUI as Web UI
|
||||
participant Controller as Controller
|
||||
participant LinkCtrl as Link Controller
|
||||
participant Manager as WebWiresharkManager
|
||||
|
||||
User->>WebUI: 1. Start Capture
|
||||
WebUI->>Controller: 2. POST /capture/start
|
||||
Controller->>LinkCtrl: 3. _start_web_wireshark(jwt_token)
|
||||
LinkCtrl->>Manager: 4. start_wireshark_session()
|
||||
Note right of Manager: 5. Ensure network exists
|
||||
Note right of Manager: 6. Get/create container
|
||||
Note right of Manager: 7. Start xpra session<br/>(display + port)
|
||||
Note right of Manager: 8. Start Wireshark<br/>(curl | wireshark -i - -k)
|
||||
Manager-->>LinkCtrl: ws_url
|
||||
LinkCtrl-->>Controller: ws_url
|
||||
Controller-->>WebUI: 10. {capturing: true, ws_url: "ws://..."}
|
||||
WebUI-->>User: 9. ws_url
|
||||
Note over User,Manager: 11-12. Browser connects via WebSocket<br/>Server proxies to container xpra
|
||||
```
|
||||
|
||||
**API Endpoint**: `POST /v3/projects/{project_id}/links/{link_id}/capture/start`
|
||||
|
||||
**Request Body**:
|
||||
```json
|
||||
{
|
||||
"wireshark": true,
|
||||
"data_link_type": "DLT_EN10MB",
|
||||
"capture_file_name": "capture.pcap"
|
||||
}
|
||||
```
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"id": "link-uuid",
|
||||
"capturing": true,
|
||||
"ws_url": "ws://192.168.1.100:14500"
|
||||
}
|
||||
```
|
||||
|
||||
### Process 2: Container Lifecycle (Per Project)
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["Project Open"] --> B["First Link Capture"]
|
||||
B --> C["Container Created<br/>(one per project)"]
|
||||
C --> D["Container Running"]
|
||||
|
||||
subgraph SessionArchitecture["Container Session Architecture"]
|
||||
direction TB
|
||||
D --> E["gns3-wireshark-{project_id}"]
|
||||
|
||||
subgraph Link1Session["Link 1 Session"]
|
||||
L1Xpra["xpra :14503"]
|
||||
L1Xvfb["Xvfb :14503<br/>(1920x1080x24)"]
|
||||
L1WS["Wireshark<br/>(curl | wireshark -i -)"]
|
||||
L1Bind["bind-ws=0.0.0.0:14503"]
|
||||
L1Xpra --- L1Xvfb
|
||||
L1Xpra --- L1WS
|
||||
L1Xpra --- L1Bind
|
||||
end
|
||||
|
||||
subgraph Link2Session["Link 2 Session"]
|
||||
L2Xpra["xpra :11024"]
|
||||
L2Xvfb["Xvfb :11024<br/>(1920x1080x24)"]
|
||||
L2WS["Wireshark<br/>(curl | wireshark -i -)"]
|
||||
L2Bind["bind-ws=0.0.0.0:11024"]
|
||||
L2Xpra --- L2Xvfb
|
||||
L2Xpra --- L2WS
|
||||
L2Xpra --- L2Bind
|
||||
end
|
||||
|
||||
E --> Link1Session
|
||||
E --> Link2Session
|
||||
|
||||
Note1["display = port = 10000 + hash(link_id) % 10000<br/>Each xpra creates its own Xvfb (NOT shared)"]
|
||||
end
|
||||
|
||||
D --> F["Project Close"]
|
||||
F --> G["Stop Container<br/>(all sessions terminate)"]
|
||||
G --> H["Container Stopped<br/>(preserved for reuse)"]
|
||||
H -->|Project Reopened| D
|
||||
|
||||
I["Project Delete"] --> J["Delete Container"]
|
||||
J --> K["Container Removed"]
|
||||
|
||||
style SessionArchitecture fill:#fff8e1,stroke:#ffa000
|
||||
style Link1Session fill:#e3f2fd,stroke:#1976d2
|
||||
style Link2Session fill:#e8f5e9,stroke:#388e3c
|
||||
```
|
||||
|
||||
### Process 3: WebSocket Connection Flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Browser as Client Browser
|
||||
participant Server as GNS3 Server (API Route)
|
||||
participant Container as Docker Container (xpra)
|
||||
|
||||
Browser->>Server: 1. WSS Connect (ws://.../web-wireshark?token=jwt)
|
||||
Note right of Server: 2. Validate JWT<br/>(RBAC: Link.Capture)
|
||||
Note right of Server: 3. Get container IP<br/>from Docker API
|
||||
Server-->>Browser: 4. Accept connection
|
||||
Note right of Server: 5. Start WebSocket proxy
|
||||
Server->>Container: 6. Connect to xpra<br/>ws://container:port
|
||||
Container-->>Server: 7. xpra validates subprotocol
|
||||
Note over Browser,Container: 8-9. Bidirectional WebSocket proxy<br/>Browser ⟺ Server ⟺ Container
|
||||
Note left of Browser: 10. HTML5 client renders<br/>Wireshark window
|
||||
```
|
||||
|
||||
**WebSocket Endpoint**: `ws://host/v3/projects/{project_id}/links/{link_id}/capture/web-wireshark?token=<jwt_token>`
|
||||
|
||||
### Process 4: Capture Data Flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph GNS3Node["GNS3 Node"]
|
||||
Router["Router / Switch"]
|
||||
end
|
||||
|
||||
subgraph ComputeNode["Compute Node"]
|
||||
Capture["Link Capture Buffer"]
|
||||
end
|
||||
|
||||
Router -->|"TAP (raw packets)"| Capture
|
||||
|
||||
subgraph GNS3Server["GNS3 Server"]
|
||||
APIRoute["GET /capture/stream<br/>Proxies pcap stream"]
|
||||
end
|
||||
|
||||
Capture -->|"pcap stream"| APIRoute
|
||||
|
||||
subgraph DockerContainer["Docker Container"]
|
||||
direction TB
|
||||
Curl["curl -N -H 'Authorization: Bearer {jwt}'<br/>'{server}/capture/stream'"]
|
||||
Wireshark["wireshark -i - -k"]
|
||||
Xvfb["Xvfb :display<br/>(1920x1080x24)"]
|
||||
Xpra["xpra :display<br/>ws://0.0.0.0:{port}"]
|
||||
|
||||
Curl -->|"stdin pipe"| Wireshark
|
||||
Wireshark -->|"renders to"| Xvfb
|
||||
Xvfb -->|"X11 display"| Xpra
|
||||
end
|
||||
|
||||
APIRoute -.->|"pcap stream<br/>(curl fetches from container)"| Curl
|
||||
|
||||
subgraph Browser["Client Browser"]
|
||||
HTML5Client["HTML5/xpra Client"]
|
||||
WSUI["Wireshark Web Interface"]
|
||||
HTML5Client --> WSUI
|
||||
end
|
||||
|
||||
Xpra <-->|"WebSocket"| HTML5Client
|
||||
|
||||
style GNS3Node fill:#f3e5f5,stroke:#7b1fa2
|
||||
style ComputeNode fill:#e8eaf6,stroke:#283593
|
||||
style GNS3Server fill:#e8f4fd,stroke:#2196f3
|
||||
style DockerContainer fill:#fff3e0,stroke:#ff9800
|
||||
style Browser fill:#e8f5e9,stroke:#4caf50
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Port Allocation Strategy
|
||||
|
||||
### Deterministic Port Mapping
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
LinkID["link_id (UUID)"] --> Hash["MD5 Hash"] --> Modulo["hash % 10000"]
|
||||
Modulo --> Offset["+ 10000"]
|
||||
Offset --> Result["display = port<br/>Range: 10000 - 19999"]
|
||||
|
||||
style LinkID fill:#e3f2fd,stroke:#1976d2
|
||||
style Result fill:#e8f5e9,stroke:#388e3c
|
||||
```
|
||||
|
||||
| link_id | port/display |
|
||||
|---------|-------------|
|
||||
| `f233f27f-7432-49c3-9aa2-50e326a10eec` | 14503 |
|
||||
| `a1b2c3d4-1234-5678-90ab-cdef12345678` | 11024 |
|
||||
| `12345678-90ab-cdef-1234-567890abcdef` | 17892 |
|
||||
|
||||
**Benefits**:
|
||||
- Same link always gets same port (deterministic)
|
||||
- Display number = port number (same hash)
|
||||
- No port conflicts between sessions
|
||||
- Easy to predict and debug
|
||||
|
||||
---
|
||||
|
||||
## Network Architecture
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph HostMachine["Host Machine"]
|
||||
subgraph GNS3ServerHost["GNS3 Server<br/>192.168.1.100:3080"]
|
||||
WSProxy["WebSocket Proxy"]
|
||||
end
|
||||
|
||||
subgraph DockerNetwork["Docker Bridge: gns3-wireshark<br/>Subnet: 172.31.0.0/22"]
|
||||
Gateway["Bridge Gateway<br/>172.31.0.1"]
|
||||
Container["gns3-wireshark-{project_id}<br/>172.31.0.x"]
|
||||
Gateway --- Container
|
||||
end
|
||||
end
|
||||
|
||||
Client["Client Browser"] -->|"ws://192.168.1.100:3080<br/>/capture/web-wireshark"| WSProxy
|
||||
WSProxy -->|"via gateway 172.31.0.1<br/>ws://172.31.0.x:{port}"| Container
|
||||
|
||||
style HostMachine fill:#fafafa,stroke:#9e9e9e
|
||||
style DockerNetwork fill:#e3f2fd,stroke:#1976d2
|
||||
style GNS3ServerHost fill:#e8f5e9,stroke:#388e3c
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Session Management Commands
|
||||
|
||||
> **Note**: These commands are for manual debugging and testing only. The GNS3 server calls `WebWiresharkManager` directly at runtime.
|
||||
|
||||
### start
|
||||
|
||||
Starts Web Wireshark session for a specific link.
|
||||
|
||||
| Argument | Required | Default | Description |
|
||||
|----------|----------|---------|-------------|
|
||||
| `--project-id` | Yes | - | Project UUID |
|
||||
| `--link-id` | Yes | - | Link UUID |
|
||||
| `--jwt-token` | Yes | - | JWT authentication token |
|
||||
| `--capture-url` | No | auto-detected | PCAP stream URL |
|
||||
| `--image` | No | `gns3/web-wireshark:latest` | Docker image |
|
||||
| `--memory` | No | `2g` | Memory limit |
|
||||
| `--cpus` | No | `1.0` | CPU cores |
|
||||
| `--pids-limit` | No | `1000` | Process limit |
|
||||
|
||||
```bash
|
||||
python manage_wireshark.py start \
|
||||
--project-id "5af0fe00-..." \
|
||||
--link-id "f233f27f-..." \
|
||||
--jwt-token "eyJhbG..."
|
||||
```
|
||||
|
||||
### Other Commands
|
||||
|
||||
| Command | Description | Key Arguments |
|
||||
|---------|-------------|---------------|
|
||||
| `stop` | Stop session for a specific link | `--project-id`, `--link-id` |
|
||||
| `restart` | Restart session (reopens Wireshark window) | `--project-id`, `--link-id`, `--jwt-token` |
|
||||
| `stop-all` | Stop all sessions for a project | `--project-id` |
|
||||
| `delete` | Delete container (alias for delete-container) | `--project-id` |
|
||||
| `stop-container` | Stop container without deleting | `--project-id` |
|
||||
| `delete-container` | Delete the container | `--project-id` |
|
||||
|
||||
---
|
||||
|
||||
## Project Close/Delete Workflow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["Project Close"] --> B["_stop_web_wireshark_container()"]
|
||||
B --> C["WebWiresharkManager.stop_container(project_id)"]
|
||||
C --> D["docker stop<br/>(all xpra/Xvfb/Wireshark terminate)"]
|
||||
D --> E["Container Stopped<br/>(preserved for reuse)"]
|
||||
|
||||
E -->|Project Reopened| F["First Capture Start"]
|
||||
F --> G["docker start<br/>(container already exists, fast)"]
|
||||
G --> H["New xpra sessions created"]
|
||||
|
||||
I["Project Delete"] --> J["_cleanup_web_wireshark_container()"]
|
||||
J --> K["WebWiresharkManager.delete_container(project_id)"]
|
||||
K --> L["docker rm<br/>(container removed)"]
|
||||
|
||||
style A fill:#e3f2fd,stroke:#1976d2
|
||||
style I fill:#ffebee,stroke:#c62828
|
||||
style E fill:#fff8e1,stroke:#ffa000
|
||||
style L fill:#ffebee,stroke:#c62828
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints Summary
|
||||
|
||||
| Method | Endpoint | Description | Privilege |
|
||||
|--------|----------|-------------|-----------|
|
||||
| POST | `/v3/projects/{id}/links/{id}/capture/start` | Start capture with Web Wireshark | Link.Capture |
|
||||
| POST | `/v3/projects/{id}/links/{id}/capture/stop` | Stop capture | Link.Capture |
|
||||
| POST | `/v3/projects/{id}/links/{id}/capture/wireshark/restart` | Restart Wireshark window | Link.Capture |
|
||||
| GET | `/v3/projects/{id}/links/{id}/capture/stream` | Stream PCAP data | Link.Capture |
|
||||
| GET | `/v3/projects/{id}/links/{id}/capture/file` | Download PCAP file | Link.Capture |
|
||||
| WS | `/v3/projects/{id}/links/{id}/capture/web-wireshark` | WebSocket proxy for xpra | Link.Capture |
|
||||
|
||||
---
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **RBAC Authentication**: All endpoints require `Link.Capture` privilege
|
||||
2. **JWT Token Validation**: WebSocket connections validate JWT token
|
||||
3. **WebSocket Subprotocol Negotiation**: Proper xpra subprotocol handling
|
||||
4. **Container Isolation**: Each project gets its own container with isolated resources
|
||||
5. **Network Segmentation**: Container runs on isolated Docker network (not host network)
|
||||
|
||||
---
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
### Startup & Shutdown Performance
|
||||
|
||||
| Step | Before Optimization | After Optimization | Improvement |
|
||||
|------|---------------------|-------------------|-------------|
|
||||
| **Health Check** | ~1.0s | ~0s | Docker native status |
|
||||
| **Gateway Detection** | 0.85s | ~0.001s | Docker API vs exec |
|
||||
| **Process Cleanup** | ~850ms | ~40ms | Host perspective recursive tree walk |
|
||||
| **Xpra Startup** | 6.4s | ~3s | HTML5 client disabled |
|
||||
| **Wireshark Launch** | ~1s | ~1s | No change |
|
||||
| **Total Startup** | **~15s** | **~5-6s** | **67% faster** |
|
||||
|
||||
| Step | Before Optimization | After Optimization | Improvement |
|
||||
|------|---------------------|-------------------|-------------|
|
||||
| **Process Termination** | ~8s | ~20-40ms | Recursive tree walk, no orphans |
|
||||
| **File Cleanup** | ~850ms | ~850ms | Docker exec (safety) |
|
||||
| **Total Shutdown** | **~9s** | **~2s** | **78% faster** |
|
||||
|
||||
#### Startup Breakdown: First vs Subsequent
|
||||
|
||||
| Phase | First Startup (Container stopped) | Subsequent Startup (Container running) |
|
||||
|-------|-----------------------------------|----------------------------------------|
|
||||
| Container startup | ~1-2s | ~0s (already running) |
|
||||
| Container health check | ~1s (unhealthy→healthy) | ~0s (already healthy) |
|
||||
| Gateway detection | ~0.001s | ~0.001s |
|
||||
| Process cleanup | ~40ms | ~40ms |
|
||||
| Xpra startup | ~3s | ~3s |
|
||||
| Wireshark launch | ~1s | ~1s |
|
||||
| **Total** | **~6s** | **~5s** |
|
||||
|
||||
#### Measured Performance Data
|
||||
|
||||
**Startup (from production logs):**
|
||||
```
|
||||
13:46:53 → 13:46:59 = 6s (first startup with container start)
|
||||
13:47:38 → 13:47:43 = 5s (subsequent startup, container running)
|
||||
```
|
||||
|
||||
**Shutdown (from production logs):**
|
||||
```
|
||||
13:48:15 → 13:48:17 = 2s (complete cleanup, no orphan processes)
|
||||
13:48:50 → 13:48:52 = 2s (complete cleanup, no orphan processes)
|
||||
```
|
||||
|
||||
#### Key Optimizations
|
||||
|
||||
- **Complete Process Cleanup**: Recursive process tree traversal eliminates orphaned processes (Xvfb, pulseaudio, ibus-daemon)
|
||||
- **Fast Gateway Detection**: Docker API query instead of container exec
|
||||
- **Smart Health Check**: Trust Docker built-in status, no manual ping
|
||||
- **Xpra Optimization**: Disabled unnecessary HTML5 client (`--html=off`)
|
||||
- **Reduced Docker Exec Calls**: Combined X lock and xpra socket cleanup into single exec call
|
||||
|
||||
### Docker Exec Performance Limitations
|
||||
|
||||
**Important**: Docker daemon has internal queuing for concurrent exec requests to the same container.
|
||||
|
||||
#### Test Results (Same Container)
|
||||
|
||||
| Test Scenario | Execution Time | Avg Per Exec |
|
||||
|--------------|----------------|--------------|
|
||||
| Single docker exec | 0.854s | 0.854s |
|
||||
| Serial 7 docker exec | 7.225s | 1.032s |
|
||||
| Parallel 7 docker exec | 10.253s | 1.465s |
|
||||
|
||||
**Key Finding**: Parallel execution is **42% slower** than serial execution.
|
||||
|
||||
```
|
||||
Serial: 7.225s (7 requests processed sequentially)
|
||||
Parallel: 10.253s (Docker daemon still processes sequentially + context switch overhead)
|
||||
```
|
||||
|
||||
#### Impact on Web Wireshark
|
||||
|
||||
When stopping multiple capture sessions quickly:
|
||||
- Each stop requires 1 docker exec (cleanup files)
|
||||
- Docker daemon processes exec requests sequentially
|
||||
- 7 sessions × ~1s each = ~7-10 seconds total
|
||||
- User requests appear to "queue" even though they're concurrent
|
||||
|
||||
#### Optimization Strategy
|
||||
|
||||
Since Docker exec cannot be parallelized effectively:
|
||||
1. **Minimize exec calls** - Already implemented: 2 calls → 1 call
|
||||
2. **Accept serial processing** - No benefit to parallel execution
|
||||
3. **Focus on fast exec content** - Use simple `rm -f` commands
|
||||
|
||||
#### Future Optimization Options
|
||||
|
||||
- Use Docker API instead of exec (requires container filesystem access)
|
||||
- Delay cleanup to next startup (increases startup complexity)
|
||||
- Batch multiple stops into single operation (requires API changes)
|
||||
|
||||
### Resource Usage (Per Wireshark Instance)
|
||||
| Resource | Typical Usage |
|
||||
|----------|---------------|
|
||||
| Memory | 150-250 MB |
|
||||
| CPU | 0.5-2% (idle to active) |
|
||||
| Threads | ~30 threads |
|
||||
| Disk I/O | Minimal |
|
||||
|
||||
### Container Configuration
|
||||
|
||||
Configured via `WebWiresharkSettings` in `gns3server/schemas/config.py`:
|
||||
|
||||
| Parameter | Default | Recommended | Description |
|
||||
|-----------|---------|-------------|-------------|
|
||||
| enabled | true | - | Enable/disable Web Wireshark feature |
|
||||
| image | gns3/web-wireshark:latest | - | Docker image name |
|
||||
| network_subnet | 172.31.0.0/22 | - | Docker bridge network subnet |
|
||||
| Memory | 2GB | 2-4GB | Container memory limit |
|
||||
| CPUs | 1.0 | 1.0-2.0 | Container CPU limit |
|
||||
| PIDs Limit | 1000 | 1000 | Container process limit |
|
||||
|
||||
### Scaling Guidelines
|
||||
| Instances | Memory | Use Case |
|
||||
|-----------|--------|----------|
|
||||
| 1-3 | 450-750 MB | Light projects |
|
||||
| 4-6 | 600-1.5 GB | Medium projects |
|
||||
| 7-10 | 1-2.5 GB | Large projects |
|
||||
| 10+ | >2.5 GB | Increase memory |
|
||||
|
||||
---
|
||||
|
||||
## Known Limitations
|
||||
|
||||
1. **JWT Token Visibility**: Token passed via command-line arguments (visible in `/proc/<pid>/cmdline`)
|
||||
2. **Single Container**: All Wireshark instances run in a single container per project
|
||||
3. **Docker Dependency**: Requires Docker daemon running on the server
|
||||
4. **Browser Support**: Requires modern browser with WebSocket support
|
||||
5. **Port Range**: Limited to 10,000 unique ports (10000-19999)
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
gns3server/
|
||||
├── controller/
|
||||
│ ├── link.py # Link capture lifecycle
|
||||
│ └── project.py # Project cleanup hooks
|
||||
├── api/routes/controller/
|
||||
│ └── links.py # REST/WebSocket API endpoints
|
||||
└── agent/web_wireshark/
|
||||
├── setup_wireshark_image.py # Docker image setup tool
|
||||
├── manage_wireshark.py # CLI management tool (manual/debug use only)
|
||||
├── manager.py # Session management logic (called by server)
|
||||
├── docker_client.py # Docker API client
|
||||
├── stats.py # Container statistics collection
|
||||
├── docker/
|
||||
│ └── Dockerfile # Container image definition
|
||||
└── WEB_WIRESHARK.md # Technical documentation
|
||||
```
|
||||
@ -1,167 +0,0 @@
|
||||
# GNS3 AI Copilot Documentation
|
||||
|
||||
This directory contains design documentation, implementation guides, and future plans for the GNS3 AI Copilot feature.
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
docs/gns3-copilot/
|
||||
├── README.md # This file
|
||||
├── template-based-configuration-roadmap.md # Future: Template-based config with HITL
|
||||
└── implemented/ # Implemented features and designs
|
||||
├── chat-api.md # Chat API design (SSE, session management)
|
||||
├── llm-model-configs.md # LLM model configuration system
|
||||
├── command-security.md # Command security and filtering
|
||||
├── context-window-management.md # Context window optimization
|
||||
├── node-control-tools.md # Node start/stop/suspend tools for lab automation
|
||||
└── multi-vendor-device-support.md # Multi-vendor device support (Cisco, Huawei)
|
||||
```
|
||||
|
||||
## Implemented Features
|
||||
|
||||
### Chat API (`implemented/chat-api.md`)
|
||||
The core Chat API that enables AI-powered conversations within GNS3 projects.
|
||||
|
||||
**Key Features:**
|
||||
- Server-Sent Events (SSE) for streaming responses
|
||||
- Project-level session isolation
|
||||
- Session management (CRUD operations)
|
||||
- Statistics tracking (messages, tokens, LLM calls)
|
||||
- User-level LLM configuration
|
||||
|
||||
**Status:** ✅ Implemented
|
||||
|
||||
### LLM Model Configs (`implemented/llm-model-configs.md`)
|
||||
Multi-level LLM model configuration system.
|
||||
|
||||
**Key Features:**
|
||||
- System-wide defaults
|
||||
- Group-level configurations
|
||||
- User-level overrides
|
||||
- Runtime parameter adjustment
|
||||
- Model provider abstraction
|
||||
|
||||
**Status:** ✅ Implemented
|
||||
|
||||
### Command Security (`implemented/command-security.md`)
|
||||
Security framework for AI-generated commands.
|
||||
|
||||
**Key Features:**
|
||||
- Command filtering and validation
|
||||
- Dangerous operation detection
|
||||
- HITL (Human-in-the-Loop) confirmations
|
||||
- Audit logging
|
||||
|
||||
**Status:** ✅ Implemented
|
||||
|
||||
### Context Window Management (`implemented/context-window-management.md`)
|
||||
Optimization strategies for handling large project contexts.
|
||||
|
||||
**Key Features:**
|
||||
- Intelligent content filtering
|
||||
- Token usage optimization
|
||||
- Summary generation
|
||||
- Context compression
|
||||
|
||||
**Status:** ✅ Implemented
|
||||
|
||||
### Node Control Tools (`implemented/node-control-tools.md`)
|
||||
Tools for controlling network device lifecycle in GNS3 projects.
|
||||
|
||||
**Key Features:**
|
||||
- Start nodes with progress tracking
|
||||
- Quick start for automated workflows
|
||||
- Stop nodes for lab shutdown
|
||||
- Batch operations support
|
||||
- Mode-based access control
|
||||
|
||||
**Status:** ✅ Implemented
|
||||
|
||||
### Multi-Vendor Device Support (`implemented/multi-vendor-device-support.md`)
|
||||
Multi-vendor network device support with custom Netmiko drivers for Huawei, Ruijie, and VPCS devices.
|
||||
|
||||
**Key Features:**
|
||||
- Custom HuaweiTelnetCE driver for Huawei CloudEngine (no authentication)
|
||||
- Custom RuijieTelnetEnhanced driver for Ruijie OS (interactive command handling)
|
||||
- Custom VPCSTelnet driver for VPCS simulator (no authentication, ANSI code stripping)
|
||||
- Cisco IOS Telnet support
|
||||
- Dynamic device type detection from GNS3 tags
|
||||
- Unified Nornir + Netmiko architecture
|
||||
- Vendor-specific command handling (VRP system-view, Ruijie interactive prompts, VPCS simple prompts)
|
||||
|
||||
**Tested Vendors:**
|
||||
- Cisco IOS (Telnet)
|
||||
- Huawei CloudEngine (Telnet, custom driver)
|
||||
- Ruijie (锐捷) OS (Telnet, custom enhanced driver)
|
||||
- VPCS (Virtual PC Simulator, Telnet, custom driver)
|
||||
|
||||
**Status:** ✅ Implemented
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Template-Based Configuration with HITL (`template-based-configuration-roadmap.md`)
|
||||
**Status:** 💡 Proposed | **Target:** Next Release
|
||||
|
||||
A revolutionary approach to network device configuration using Jinja2 templates with Human-in-the-Loop confirmations.
|
||||
|
||||
**Key Features:**
|
||||
- Three-step HITL workflow (Template → Parameters → Execute)
|
||||
- 70-80% token savings for multi-device configurations
|
||||
- Template reusability across projects
|
||||
- Human review at every critical step
|
||||
- Configuration preview before execution
|
||||
|
||||
**Benefits:**
|
||||
- Massive token cost reduction
|
||||
- Enhanced safety through human oversight
|
||||
- Template library for common configurations
|
||||
- Multi-vendor support (Cisco, Huawei, H3C, etc.)
|
||||
|
||||
**Implementation Timeline:**
|
||||
- Phase 1: Core MVP (3-5 days) - Basic template workflow
|
||||
- Phase 2: UX Enhancement (2-3 days) - Review interfaces, preview
|
||||
- Phase 3: Template Library (2-3 days) - Pre-built templates, caching
|
||||
- Phase 4: Advanced Features (3-4 days) - Multi-vendor, composition, analytics
|
||||
|
||||
See [`template-based-configuration-roadmap.md`](./template-based-configuration-roadmap.md) for complete details.
|
||||
|
||||
### Other Proposed Features
|
||||
|
||||
- Vision-based Topology Creation: Create network topologies from images/diagrams
|
||||
- Enhanced HITL Workflows: Advanced confirmation patterns for other operations
|
||||
- Web UI Enhancements: Improved management interfaces
|
||||
- Configuration Diff & Comparison: Compare configurations across devices
|
||||
- Rollback & Undo: Revert configuration changes
|
||||
|
||||
## Contributing
|
||||
|
||||
When adding new documentation:
|
||||
|
||||
1. **Implementation:** Add documentation to `implemented/` when feature is complete
|
||||
2. **Naming:** Use concise names like `{feature}.md`
|
||||
|
||||
## Document Status Legend
|
||||
|
||||
| Status | Description |
|
||||
|--------|-------------|
|
||||
| ✅ Implemented | Feature is fully implemented and deployed |
|
||||
| 📋 Design Complete | Design is done, awaiting implementation |
|
||||
| 🚧 In Progress | Currently being implemented |
|
||||
| 💡 Proposed | Initial idea or proposal |
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [GNS3 Server API Documentation](https://api.gns3.com/)
|
||||
- [GNS3 Web UI Documentation](https://docs.gns3.com/)
|
||||
- [LangGraph Documentation](https://langchain-ai.github.io/langgraph/)
|
||||
- [FastAPI Documentation](https://fastapi.tiangolo.com/)
|
||||
|
||||
## Quick Links
|
||||
|
||||
- **Current Feature Branch:** `feature/ai-copilot-bridge`
|
||||
- **Main Branch:** `master`
|
||||
- **Issue Tracker:** [GitHub Issues](https://github.com/GNS3/gns3-server/issues)
|
||||
|
||||
---
|
||||
|
||||
_Last updated: 2026-03-20_
|
||||
@ -1,15 +1,25 @@
|
||||
# GNS3 Copilot Agent Chat API Design Document
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
|
||||
|
||||
|
||||
# GNS3 Copilot Agent Chat API
|
||||
|
||||
## Overview
|
||||
|
||||
This document describes the architectural design and implementation plan for the GNS3 Copilot Chat API. This API enables clients to interact with the GNS3 Copilot Agent through a RESTful interface, providing streaming conversations, session management, and project topology queries.
|
||||
This document describes the implementation of the GNS3 Copilot Chat API. This API enables clients to interact with the GNS3 Copilot Agent through a RESTful interface, providing streaming conversations, session management, session abort, and project topology queries.
|
||||
|
||||
## Core Features
|
||||
|
||||
- **Project-level Isolation**: Each GNS3 project has its own Agent instance and session storage
|
||||
- **Streaming Responses**: Uses Server-Sent Events (SSE) for real-time streaming output
|
||||
- **Session Management**: Supports session listing, renaming, deletion, and history queries
|
||||
- **Session Management**: Supports session listing, renaming, deletion, pinning, and history queries
|
||||
- **Session Abort**: Supports aborting an ongoing streaming session mid-conversation
|
||||
- **Statistics Tracking**: Automatically records message counts, LLM call counts, and token usage
|
||||
- **Copilot Modes**: Supports `teaching_assistant` (diagnostic only) and `lab_automation_assistant` (full config) modes
|
||||
- **User Isolation**: Each user has independent LLM configurations and session spaces
|
||||
|
||||
## Architecture Design
|
||||
@ -31,10 +41,19 @@ AgentService (per project)
|
||||
│ ├─ checkpoints table (LangGraph state)
|
||||
│ └─ chat_sessions table (session metadata)
|
||||
│
|
||||
└─ LangGraph Agent
|
||||
└─ LangGraph Agent (StateGraph)
|
||||
├─ llm_call node
|
||||
├─ should_continue node
|
||||
└─ tool_node (GNS3 tools)
|
||||
├─ tool_node (GNS3 tools)
|
||||
├─ title_generator_node (auto title)
|
||||
└─ abort_handler_node (abort handling)
|
||||
│
|
||||
├─ Conditional Edges (routing functions):
|
||||
│ ├─ should_continue (after llm_call)
|
||||
│ └─ recursion_limit_continue (after tool_node)
|
||||
│
|
||||
└─ Copilot Modes:
|
||||
├─ teaching_assistant (diagnostic tools only)
|
||||
└─ lab_automation_assistant (full diagnostic + config tools)
|
||||
```
|
||||
|
||||
### Project-level Checkpoint Design
|
||||
@ -184,38 +203,34 @@ Statistics are collected in real-time during conversation, and updated to `chat_
|
||||
|
||||
1. **message_count (number of messages)**
|
||||
- Initial value: 1 (user message)
|
||||
- `on_chat_model_end` event: +1 (AI complete reply, not each chunk)
|
||||
- `on_chat_model_end` event: +1 (AI complete reply, only counted once per turn via `ai_response_counted` flag, not each streaming chunk)
|
||||
- `on_tool_end` event: +1 (each tool execution result)
|
||||
|
||||
2. **llm_calls_count (number of LLM calls)**
|
||||
- Listen to `on_chat_model_start` event
|
||||
- +1 each time LLM starts generation
|
||||
- **Filtered**: `title_generator_node` events are excluded from count (internal use only)
|
||||
|
||||
3. **input_tokens (input tokens)**
|
||||
- Extracted from `usage_metadata` in `on_chat_model_end` event
|
||||
- **Important**: input_tokens returned by LangGraph already includes conversation history, accumulates previous conversation content on each LLM call
|
||||
- Example: 1st call input=8674, 2nd call input=9421 (includes 1st conversation 8674+675+system prompt increment)
|
||||
- Uses incremental addition (`+=`): each event's token count is added to the running total
|
||||
- **Filtered**: `title_generator_node` events are excluded from token counting
|
||||
- Tries multiple extraction methods: `response.usage_metadata` → `output.usage_metadata` → direct data fields
|
||||
|
||||
4. **output_tokens (output tokens)**
|
||||
- Extracted from `usage_metadata` in `on_chat_model_end` event
|
||||
- **Important**: output_tokens returned by LangGraph is also accumulated value, includes output from all LLM calls
|
||||
- Example: 1st actual output=675, 2nd actual output=9, accumulated output=684 (675+9)
|
||||
- Uses incremental addition (`+=`): each event's token count is added to the running total
|
||||
- **Filtered**: `title_generator_node` events are excluded from token counting
|
||||
|
||||
5. **total_tokens (total tokens)**
|
||||
- Calculation formula: input_tokens + output_tokens
|
||||
- Take the accumulated value from the last LLM call for calculation
|
||||
|
||||
**Statistics Example** (real data):
|
||||
- 1st LLM call (AI reply): input=8674, output=675
|
||||
- 2nd LLM call (generate title): input=9421, output=684 (accumulated value: 675+9)
|
||||
- Final storage: input_tokens=9421, output_tokens=684, total_tokens=10105
|
||||
- Note: LangGraph automatically accumulates, code can directly take the last value
|
||||
- Calculation formula: `input_tokens + output_tokens` (computed at update time)
|
||||
|
||||
**Notes**:
|
||||
- message_count counts **complete messages**, not streaming chunks
|
||||
- `ai_response_counted` flag ensures AI responses are only counted once per turn, even if multiple `on_chat_model_end` events fire
|
||||
- `title_generator_node` is completely excluded from all statistics (llm_calls, tokens, streaming output)
|
||||
- Token data depends on LLM's returned `usage_metadata`, some models may not support
|
||||
- Statistics are incrementally updated to database via `update_session` method after stream ends
|
||||
- LangGraph automatically handles input and output history accumulation, code uses the last LLM call value
|
||||
- Statistics are incrementally updated to database via `update_session` method after stream ends (using SQL `field = field + ?` syntax)
|
||||
- **Message ID handling**: Assign ID when creating initial message (`HumanMessage(id=str(uuid4()))`), messages read from checkpoint without ID are also automatically generated
|
||||
- **Format conversion**: Use `message_converters.py` module to handle conversion between LangChain and OpenAI formats, ensuring tool_calls format conforms to OpenAI specification
|
||||
|
||||
@ -241,12 +256,14 @@ Chat API uses Server-Sent Events (SSE) for streaming transmission.
|
||||
|
||||
| type | Description | Included Fields |
|
||||
|------|-------------|------------------|
|
||||
| content | AI text content (streaming) | content, message_id (optional) |
|
||||
| content | AI text content (streaming) | content, message_id (optional), session_id |
|
||||
| tool_call | LLM decides to call tool (streaming, parameters accumulated gradually) | tool_call (object, includes id, type, function), session_id, message_id (optional) |
|
||||
| tool_start | Tool starts execution | tool_name, tool_call_id, session_id |
|
||||
| tool_end | Tool execution complete | tool_name, tool_output, session_id |
|
||||
| error | Error message | error, session_id |
|
||||
| abort | Stream aborted by user | session_id |
|
||||
| done | Stream end | session_id |
|
||||
| heartbeat | *(Planned)* Heartbeat keepalive | session_id |
|
||||
|
||||
**Tool Output Format** (`tool_output` field):
|
||||
- If the tool returns a non-string type (dict, list), it is automatically serialized to JSON format using `json.dumps(obj, ensure_ascii=False, indent=2)`
|
||||
@ -254,8 +271,6 @@ Chat API uses Server-Sent Events (SSE) for streaming transmission.
|
||||
- This ensures all structured data is in standard JSON format, making it easy for the frontend to parse with `JSON.parse()`
|
||||
- Chinese and other non-ASCII characters are preserved (not escaped to `\uXXXX`)
|
||||
|
||||
| heartbeat | Heartbeat keepalive | session_id |
|
||||
|
||||
### Message Examples
|
||||
|
||||
```json
|
||||
@ -333,6 +348,9 @@ Chat API uses Server-Sent Events (SSE) for streaming transmission.
|
||||
|
||||
// Error
|
||||
{"type": "error", "error": "Project not found", "session_id": "xxx"}
|
||||
|
||||
// Stream aborted by user
|
||||
{"type": "abort", "session_id": "xxx"}
|
||||
```
|
||||
|
||||
### Streaming Tool Call Mechanism
|
||||
@ -384,14 +402,16 @@ function handleToolCallEvent(chunk) {
|
||||
}
|
||||
```
|
||||
|
||||
### Heartbeat Mechanism
|
||||
### Heartbeat Mechanism *(Planned)*
|
||||
|
||||
**Purpose**: Prevent proxy server/load balancer from disconnecting SSE connection due to timeout.
|
||||
|
||||
**Implementation**: Use `asyncio.wait` to set timeout, send `heartbeat` message after timeout, then continue waiting for next event.
|
||||
**Planned Implementation**: Use `asyncio.wait` to set timeout, send `heartbeat` message after timeout, then continue waiting for next event.
|
||||
|
||||
**Frontend Handling**: When receiving `heartbeat` message, ignore it directly, don't render anything.
|
||||
|
||||
**Note**: The `heartbeat` type is defined in the `ChatResponse` schema but not yet implemented in the streaming path. Currently, long-running tool executions may cause proxy timeouts.
|
||||
|
||||
## API Endpoints
|
||||
|
||||
All endpoints are under `/v3/projects/{project_id}/chat/` path.
|
||||
@ -403,6 +423,7 @@ All endpoints are under `/v3/projects/{project_id}/chat/` path.
|
||||
| GET | `/sessions/{session_id}/history` | Get session history |
|
||||
| PATCH | `/sessions/{session_id}` | Rename session |
|
||||
| DELETE | `/sessions/{session_id}` | Delete session |
|
||||
| POST | `/sessions/{session_id}/abort` | Abort ongoing streaming session |
|
||||
| PUT | `/sessions/{session_id}/pin` | Pin session |
|
||||
| DELETE | `/sessions/{session_id}/pin` | Unpin session |
|
||||
|
||||
@ -514,7 +535,7 @@ data: {"type": "done", "session_id": "d7e76375-6960-419a-9367-211ef64af877"}
|
||||
{
|
||||
"id": "f0247568-071d-412f-9e3e-4cbe815834ea",
|
||||
"role": "user",
|
||||
"content": "你能干点啥。",
|
||||
"content": "What can you do?",
|
||||
"metadata": {
|
||||
"created_at": "2026-03-07T17:48:07.848519"
|
||||
}
|
||||
@ -522,7 +543,7 @@ data: {"type": "done", "session_id": "d7e76375-6960-419a-9367-211ef64af877"}
|
||||
{
|
||||
"id": "lc_run--019cc969-eb81-7dd1-a894-e819daf81cd0",
|
||||
"role": "assistant",
|
||||
"content": "我可以作为GNS3网络实验的助教...",
|
||||
"content": "I can serve as a teaching assistant for GNS3 network labs...",
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": "call_00_xxx",
|
||||
@ -566,6 +587,34 @@ data: {"type": "done", "session_id": "d7e76375-6960-419a-9367-211ef64af877"}
|
||||
|
||||
**Response**: 204 No Content
|
||||
|
||||
### POST /v3/projects/{project_id}/chat/sessions/{session_id}/abort
|
||||
|
||||
**Function**: Abort an ongoing streaming session
|
||||
|
||||
**Behavior**:
|
||||
1. Sets an in-memory abort flag (`_abort_flags[session_id] = True`)
|
||||
2. The LangGraph graph checks this flag at conditional edges (`should_continue`, `recursion_limit_continue`)
|
||||
3. If abort is detected during a tool call, `abort_handler_node` generates placeholder `ToolMessage` results to maintain message history consistency
|
||||
4. The stream ends gracefully, and any aborted tool messages are yielded as `tool_end` events with `{"status": "aborted"}` content
|
||||
|
||||
**Response Example**:
|
||||
```json
|
||||
{"status": "ok", "session_id": "d7e76375-6960-419a-9367-211ef64af877"}
|
||||
```
|
||||
|
||||
**Abort Flow**:
|
||||
```
|
||||
POST /abort → set_abort_flag(session_id)
|
||||
↓
|
||||
Graph conditional edge checks flag
|
||||
↓
|
||||
┌─ Has pending tool_calls?
|
||||
│ YES → abort_handler_node (generates placeholder ToolMessages)
|
||||
│ NO → END directly
|
||||
↓
|
||||
Stream ends → yields aborted tool_end events → done
|
||||
```
|
||||
|
||||
### PUT /v3/projects/{project_id}/chat/sessions/{session_id}/pin
|
||||
|
||||
**Function**: Pin session to top of list
|
||||
@ -684,9 +733,9 @@ OpenAI-compatible message model.
|
||||
**Responsibility**: Convert between LangChain message format and OpenAI-compatible format
|
||||
|
||||
**Main Functions**:
|
||||
- `convert_langchain_to_openai()`: LangChain → OpenAI format
|
||||
- `convert_openai_to_langchain()`: OpenAI → LangChain format
|
||||
- `convert_stream_event_to_openai()`: Stream event → OpenAI SSE format
|
||||
- `convert_langchain_to_openai()`: LangChain → OpenAI format (used in `get_history` and message conversion)
|
||||
- `convert_openai_to_langchain()`: OpenAI → LangChain format (utility, not used in main streaming path)
|
||||
- `convert_stream_event_to_openai()`: *(Not used in main streaming path)* — streaming uses `ToolCallStreamAccumulator` for `on_chat_model_stream` and `AgentService._convert_event_to_chunk` for other events
|
||||
|
||||
**Key Conversion Logic**:
|
||||
|
||||
@ -718,12 +767,13 @@ OpenAI-compatible message model.
|
||||
|
||||
**Responsibility**: LangGraph-based workflow orchestration for AI conversation
|
||||
|
||||
**Main Components**:
|
||||
**Graph Nodes**:
|
||||
|
||||
1. **llm_call Node**: Invokes LLM with tools and conversation history
|
||||
- Injects topology information into system prompt
|
||||
- Handles message trimming for context window management
|
||||
- Routes to tool execution or generates response
|
||||
- Selects tools based on copilot mode (`teaching_assistant` vs `lab_automation_assistant`)
|
||||
- Creates fresh model instance with tools for each call
|
||||
|
||||
2. **tool_node Function**: Executes tool calls and returns results
|
||||
- **Critical**: Serializes tool output to JSON before creating ToolMessage
|
||||
@ -742,7 +792,47 @@ OpenAI-compatible message model.
|
||||
)
|
||||
```
|
||||
|
||||
3. **generate_title Node**: Auto-generates conversation title on first interaction
|
||||
3. **title_generator_node** (`generate_title` function): Auto-generates conversation title on first interaction
|
||||
- Uses a separate lightweight LLM (title_model) to generate a title from the first user message and assistant response
|
||||
- Title is truncated to 40 characters max
|
||||
- Fallback: uses first 30 chars of user's message if title generation fails
|
||||
- This node is **filtered out** from statistics and SSE streaming (internal use only)
|
||||
|
||||
4. **abort_handler_node**: Handles abort when pending tool_calls exist
|
||||
- Generates placeholder `ToolMessage` with `{"status": "aborted"}` content
|
||||
- Ensures message history consistency and prevents checkpoint corruption
|
||||
- Only triggered when abort flag is set and the last AI message has tool_calls
|
||||
|
||||
**Conditional Edges (Routing Functions)**:
|
||||
|
||||
- **should_continue** (after `llm_call`): Routes to `tool_node`, `title_generator_node`, `abort_handler_node`, or `END` based on:
|
||||
1. Check abort flag → `abort_handler_node` (if pending tool_calls) or `END`
|
||||
2. Has tool_calls → `tool_node`
|
||||
3. First interaction without title → `title_generator_node`
|
||||
4. Otherwise → `END`
|
||||
|
||||
- **recursion_limit_continue** (after `tool_node`): Routes to `llm_call` or `END` based on:
|
||||
1. Check abort flag → `END`
|
||||
2. Remaining steps < 4 → `END` (prevent infinite loops)
|
||||
3. Otherwise → `llm_call`
|
||||
|
||||
**State** (`MessagesState`):
|
||||
- `messages`: Conversation messages (cumulative with `operator.add`)
|
||||
- `llm_calls`: LLM invocation counter
|
||||
- `remaining_steps`: Recursion depth tracker (initial: 20)
|
||||
- `conversation_title`: Auto-generated title
|
||||
- `topology_info`: GNS3 project topology data
|
||||
- `session_id`: Session identifier (for abort tracking)
|
||||
- `abort`: Abort flag
|
||||
|
||||
**Copilot Modes**:
|
||||
|
||||
The agent supports two tool sets, selected by `copilot_mode` in the user's LLM configuration:
|
||||
|
||||
| Mode | Tools | Description |
|
||||
|------|-------|-------------|
|
||||
| `teaching_assistant` (default) | GNS3Template, GNS3CreateNode, GNS3Link, GNS3StartNode, GNS3UpdateNodeName, ExecuteMultipleDeviceCommands, PacketCapture, DeviceSkills | Read-only diagnostic + node creation |
|
||||
| `lab_automation_assistant` | All teaching_assistant tools + GNS3StopNode, GNS3SuspendNode, ExecuteMultipleDeviceConfigCommands, VPCSCommands | Full diagnostic + configuration tools |
|
||||
|
||||
**Why Serialize in tool_node?**
|
||||
|
||||
@ -780,6 +870,8 @@ ToolMessage(content=JSON_string)
|
||||
- `list_sessions`: List sessions
|
||||
- `delete_session`: Delete session
|
||||
- `rename_session`: Rename session
|
||||
- `pin_session`: Pin or unpin session
|
||||
- `abort_session`: Signal abort for a running session (sets in-memory flag)
|
||||
- `close`: Close database connection
|
||||
|
||||
**Core Flow** (stream_chat):
|
||||
@ -799,9 +891,11 @@ ToolMessage(content=JSON_string)
|
||||
- Statistics logic doesn't depend on converted SSE chunk, gets directly from original events
|
||||
|
||||
**Key Event Handling**:
|
||||
- `on_chat_model_start`: LLM call count +1
|
||||
- `on_chat_model_end`: Extract token usage (from `output.usage_metadata`), AI message count +1
|
||||
- `on_chat_model_start`: LLM call count +1 (excludes `title_generator_node`)
|
||||
- `on_chat_model_end`: Extract token usage via `response.usage_metadata` → `output.usage_metadata` → data fields (excludes `title_generator_node`), AI message count +1 (once per turn via `ai_response_counted` flag)
|
||||
- `on_tool_end`: Tool message count +1
|
||||
- `on_chat_model_stream`: Processed by `ToolCallStreamAccumulator` for progressive tool call arguments (excludes `title_generator_node` from SSE output)
|
||||
- Abort flag is cleared at stream start, checked during streaming for graceful termination
|
||||
|
||||
**Implementation Location**: `agent_service.py`
|
||||
|
||||
@ -868,8 +962,9 @@ Handle different types based on SSE message's `type` field:
|
||||
| tool_start | Optional: show tool start execution status |
|
||||
| tool_end | Create tool_result type message, display tool execution result |
|
||||
| error | Display error message |
|
||||
| abort | Mark stream as aborted, stop loading state |
|
||||
| done | Mark stream end, stop loading state |
|
||||
| heartbeat | Ignore (keepalive signal) |
|
||||
| heartbeat | *(Planned)* Ignore (keepalive signal) |
|
||||
|
||||
### Session ID Management (Important)
|
||||
|
||||
@ -977,15 +1072,15 @@ const displayTime = timestamp ? new Date(timestamp).toLocaleString() : 'Unknown'
|
||||
- Lower database lock contention
|
||||
- Improve real-time performance of streaming response
|
||||
|
||||
**Implementation Location**: `agent_service.py` lines 283-294
|
||||
**Implementation Location**: `agent_service.py` `stream_chat` method (statistics collection in event loop + batch update after stream)
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `langchain` >= 0.3.0
|
||||
- `langgraph` >= 0.2.0
|
||||
- `langchain-core`
|
||||
- `langgraph-checkpoint-sqlite` >= 3.0.1
|
||||
- `aiosqlite`
|
||||
- `fastapi`
|
||||
|
||||
## Extensibility
|
||||
|
||||
|
||||
@ -1,3 +1,11 @@
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
|
||||
|
||||
|
||||
# Command Security Configuration
|
||||
|
||||
## Overview
|
||||
@ -5,483 +13,208 @@
|
||||
GNS3-Copilot includes multiple security layers to prevent execution of commands that may cause issues in the lab environment:
|
||||
|
||||
- **Command Filtering**: Prevents commands that may timeout or lock up the console
|
||||
- **Configuration Safety**: Prohibits dangerous configuration changes (AAA, passwords, etc.)
|
||||
- **Configuration Safety**: Restricts dangerous configuration changes (AAA, passwords, etc.)
|
||||
- **Multi-line Command Handling**: Properly processes commands with embedded newlines (banner, etc.)
|
||||
|
||||
This helps:
|
||||
## Architecture
|
||||
|
||||
- **Prevent tool timeouts**: Commands like `traceroute` may run longer than the tool timeout
|
||||
- **Maintain console availability**: Long-running commands can leave the device console unavailable for subsequent commands
|
||||
- **Prevent device lockout**: AAA/password changes can lock users out of devices
|
||||
- **Ensure reliable execution**: Filtering problematic commands ensures the remaining commands can execute properly
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Tool Request] --> B{Command Filter}
|
||||
B -->|allowed| C[Nornir/Netmiko Execution]
|
||||
B -->|blocked| D[blocked_commands_info]
|
||||
C --> E[Process Results]
|
||||
D --> E
|
||||
E --> F[Return Result]
|
||||
|
||||
## Implementation Status
|
||||
subgraph Command Filter
|
||||
B1[Load forbidden_commands.txt]
|
||||
B2[Prefix match per command]
|
||||
B1 --> B2
|
||||
end
|
||||
|
||||
**Status**: ✅ **Implemented and Verified**
|
||||
|
||||
The command filtering system is fully implemented and has been tested in a live GNS3 environment with actual network devices. Key features:
|
||||
|
||||
- ✅ Simple text-based configuration file
|
||||
- ✅ Substring matching (case-insensitive)
|
||||
- ✅ Non-blocking filtering (allowed commands execute normally)
|
||||
- ✅ Detailed blocking feedback in tool results
|
||||
- ✅ Multi-device support
|
||||
- ✅ Verified with real Cisco IOS devices
|
||||
|
||||
See the [Implementation Verification](#implementation-verification) section for actual test results.
|
||||
|
||||
## Problem Context
|
||||
|
||||
### Why Filter Commands?
|
||||
|
||||
When GNS3-Copilot tools execute commands on network devices using Nornir/Netmiko, there is a timeout limit (typically 30-60 seconds). If a command exceeds this timeout:
|
||||
|
||||
1. The tool stops waiting and returns a timeout error
|
||||
2. The device console may still be executing the command
|
||||
3. Subsequent commands sent to the device fail or produce incorrect results
|
||||
4. The user may need to manually interrupt the command on the device console
|
||||
|
||||
### Example Scenario
|
||||
|
||||
```
|
||||
Time Agent Action Device Console Status
|
||||
t0 Execute: traceroute 8.8.8.8 [Command starts]
|
||||
t1 ...waiting... [Tracing...]
|
||||
t2 ...waiting... [Tracing...]
|
||||
t30 Timeout! Proceed to next tool [Still tracing!]
|
||||
t31 Execute: show ip route [Ignored or corrupted]
|
||||
t32 ❌ Command fails [Console still busy]
|
||||
subgraph Security Layers
|
||||
S1[Layer 1: Command Filter<br/>prefix-based blocking]
|
||||
S2[Layer 2: AI Prompt Rules<br/>AAA/password guidance only]
|
||||
S3[Layer 3: Multi-line Expansion<br/>banner/ACL handling]
|
||||
S1 --> S2 --> S3
|
||||
end
|
||||
```
|
||||
|
||||
## Current Implementation
|
||||
## Business Process
|
||||
|
||||
### 1. Command Filtering System
|
||||
### Command Filtering Flow
|
||||
|
||||
#### Forbidden Commands List
|
||||
|
||||
Commands are listed in a simple text file at:
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Start([Tool receives commands]) --> Load[Load forbidden patterns<br/>from config file]
|
||||
Load --> Loop{For each command}
|
||||
Loop -->|next command| Prefix{Command starts with<br/>forbidden pattern?}
|
||||
Prefix -->|yes| Block[Add to blocked_commands_info]
|
||||
Prefix -->|no| Allow[Add to allowed list]
|
||||
Block --> Loop
|
||||
Allow --> Loop
|
||||
Loop -->|all checked| Empty{Allowed list empty?}
|
||||
Empty -->|yes| Return1[Return partial_success<br/>with blocked info only]
|
||||
Empty -->|no| Exec[Execute allowed commands<br/>via Nornir/Netmiko]
|
||||
Exec --> HasBlocked{Has blocked commands?}
|
||||
HasBlocked -->|yes| Return2[Return partial_success<br/>with output + blocked info]
|
||||
HasBlocked -->|no| Return3[Return success<br/>with output]
|
||||
```
|
||||
gns3server/agent/gns3_copilot/config/forbidden_commands.txt
|
||||
|
||||
### Multi-line Command Expansion Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[config_tools receives commands] --> B{Command contains \n?}
|
||||
B -->|yes| C[Split by newline]
|
||||
C --> D[Filter empty lines]
|
||||
D --> E[Add expanded lines to command list]
|
||||
B -->|no| F[Keep command as-is]
|
||||
E --> G[Execute via netmiko_send_config]
|
||||
F --> G
|
||||
```
|
||||
|
||||
## Tools Using Command Security
|
||||
|
||||
| Tool | File | Filter | Multi-line Expansion | Result Fields |
|
||||
|------|------|--------|---------------------|---------------|
|
||||
| `ExecuteMultipleDeviceCommands` | `display_tools_nornir.py` | Yes | No | `diagnostic_commands` |
|
||||
| `ExecuteMultipleDeviceConfigCommands` | `config_tools_nornir.py` | Yes | Yes | `config_commands` |
|
||||
|
||||
## Command Filtering System
|
||||
|
||||
### Forbidden Commands Configuration
|
||||
|
||||
**File:** `gns3server/agent/gns3_copilot/config/forbidden_commands.txt`
|
||||
|
||||
**Format:**
|
||||
- One command pattern per line
|
||||
- Simple substring matching (case-insensitive)
|
||||
- **Prefix matching** (case-insensitive) — matches the beginning of each command
|
||||
- Empty lines and lines starting with `#` are ignored
|
||||
- Match is performed on the beginning of each command
|
||||
|
||||
**Example:**
|
||||
```
|
||||
# Network diagnostic commands that may timeout
|
||||
traceroute
|
||||
tracepath
|
||||
tracert
|
||||
**Default patterns (used when config file is missing):**
|
||||
|
||||
# Debug commands that may destabilize devices
|
||||
debug
|
||||
|
||||
# Test commands that may affect device stability
|
||||
test
|
||||
```
|
||||
|
||||
### Filter Behavior
|
||||
|
||||
1. **Input Commands**: `["show version", "traceroute 8.8.8.8", "show ip int brief"]`
|
||||
2. **Filtering**: `traceroute 8.8.8.8` is removed (matches `traceroute`)
|
||||
3. **Executed**: `["show version", "show ip int brief"]`
|
||||
4. **Result**: Returns successful output with blocked command information
|
||||
| Pattern | Reason |
|
||||
|---------|--------|
|
||||
| `traceroute` | Can run 30+ seconds, exceeds tool timeout |
|
||||
| `tracepath` | Similar to traceroute, long execution time |
|
||||
| `tracert` | Windows traceroute, same timeout issues |
|
||||
| `ping -f` | Flood ping can overwhelm lab devices |
|
||||
| `debug` | Can produce overwhelming output and destabilize devices |
|
||||
| `test` | May affect device stability |
|
||||
|
||||
### Result Format
|
||||
|
||||
When commands are filtered, the result includes additional fields:
|
||||
**Status values:**
|
||||
|
||||
| Status | Condition |
|
||||
|--------|-----------|
|
||||
| `"success"` | All commands executed, none blocked |
|
||||
| `"partial_success"` | Some commands blocked (including all blocked + execution succeeded) |
|
||||
| `"failed"` | Execution failed (device not found, connection error, etc.) |
|
||||
|
||||
**Example — partial success (display tool):**
|
||||
|
||||
```json
|
||||
{
|
||||
"device_name": "R-1",
|
||||
"status": "partial_success",
|
||||
"output": "R-1#show version\nCisco IOS Software...\nR-1#show ip int brief\nInterface...",
|
||||
"output": "...",
|
||||
"diagnostic_commands": ["show version", "show ip int brief"],
|
||||
"blocked_commands": ["traceroute 8.8.8.8"],
|
||||
"blocked_info": {
|
||||
"traceroute 8.8.8.8": "Command 'traceroute 8.8.8.8' is not allowed because it matches the forbidden pattern 'traceroute'. This command may run longer than the tool timeout or leave the device console unavailable for subsequent commands."
|
||||
"traceroute 8.8.8.8": "Command 'traceroute 8.8.8.8' is not allowed because it matches the forbidden pattern 'traceroute'. ..."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Status values:**
|
||||
- `"success"`: All commands executed successfully
|
||||
- `"partial_success"`: Some commands were blocked, but remaining commands executed successfully
|
||||
- `"failed"`: Command execution failed (device not found, connection error, etc.)
|
||||
> **Note:** `diagnostic_commands` is specific to display tools. Config tools use `config_commands` instead.
|
||||
|
||||
## Module Structure
|
||||
|
||||
### Command Filter Module
|
||||
**Command Filter:** `gns3server/agent/gns3_copilot/utils/command_filter.py`
|
||||
|
||||
**File:** `gns3server/agent/gns3_copilot/utils/command_filter.py`
|
||||
| Function | Purpose |
|
||||
|----------|---------|
|
||||
| `filter_forbidden_commands(commands)` | Returns `(allowed_commands, blocked_commands_info)` |
|
||||
| `is_command_forbidden(command)` | Check if a single command matches a forbidden pattern |
|
||||
| `get_forbidden_commands()` | Get current forbidden patterns list |
|
||||
| `reload_forbidden_commands()` | Clear cache, reload from file on next access |
|
||||
|
||||
**Functions:**
|
||||
- `filter_forbidden_commands(commands: list[str]) -> tuple[list[str], dict[str, str]]`
|
||||
- Returns allowed commands and blocked command information
|
||||
- `is_command_forbidden(command: str) -> bool`
|
||||
- Check if a single command is forbidden
|
||||
- `get_forbidden_commands() -> list[str]`
|
||||
- Get the current list of forbidden patterns
|
||||
- `reload_forbidden_commands() -> None`
|
||||
- Reload the forbidden commands list (useful after editing the file)
|
||||
**Integration points:**
|
||||
- `display_tools_nornir.py` — `_filter_forbidden_commands_from_device_configs()`
|
||||
- `config_tools_nornir.py` — `_filter_forbidden_commands_from_device_configs()` + `_expand_multiline_commands()`
|
||||
|
||||
### Integration Points
|
||||
## Configuration Safety (AI Prompt Level)
|
||||
|
||||
The filter is integrated into:
|
||||
- **Display Tools** (`display_tools_nornir.py`): `ExecuteMultipleDeviceCommands`
|
||||
- **Configuration Tools** (`config_tools_nornir.py`): `ExecuteMultipleDeviceConfigCommands`
|
||||
In addition to the command filter, the AI agent is instructed via system prompt (`lab_automation_assistant_prompt.py`) to handle sensitive configuration commands with caution:
|
||||
|
||||
Both tools use the same filtering logic and return format.
|
||||
**Prompt-enforced rules:**
|
||||
- **FORBIDDEN (guidance only):** `enable secret`, `username`, `aaa new-model`, `service password-encryption`, `line vty` — AI provides configuration guidance instead of executing
|
||||
- **Caution required:** `reload`, `erase`, `format` — AI warns user before executing destructive operations
|
||||
|
||||
> These are prompt-level rules, not code-level enforcement. Users can always execute commands directly via device console or SSH/Telnet.
|
||||
|
||||
## Multi-line Command Handling
|
||||
|
||||
Commands containing `\n` are automatically split before execution by `config_tools_nornir.py:_expand_multiline_commands()`.
|
||||
|
||||
**Example:** `["banner motd #\nWelcome\n#"]` becomes `["banner motd #", "Welcome", "#"]`
|
||||
|
||||
Applies to any command with embedded newlines: `banner`, multi-line ACLs, route-maps, etc.
|
||||
|
||||
## Configuration
|
||||
|
||||
### Default Forbidden Commands
|
||||
### Customizing Forbidden Commands
|
||||
|
||||
If the configuration file is not found, these defaults are used:
|
||||
- `traceroute`
|
||||
- `tracepath`
|
||||
- `tracert`
|
||||
- `ping -f`
|
||||
- `debug`
|
||||
- `test`
|
||||
|
||||
### Customizing the List
|
||||
|
||||
To add or remove forbidden commands:
|
||||
|
||||
1. Edit the configuration file:
|
||||
```bash
|
||||
nano gns3server/agent/gns3_copilot/config/forbidden_commands.txt
|
||||
```
|
||||
|
||||
2. Add your command patterns (one per line):
|
||||
```
|
||||
# My custom blocked commands
|
||||
my_dangerous_command
|
||||
another_pattern
|
||||
```
|
||||
|
||||
3. Restart GNS3 server to apply changes
|
||||
|
||||
### Reloading Without Restart
|
||||
|
||||
To reload the forbidden commands list without restarting the server:
|
||||
|
||||
```python
|
||||
from gns3server.agent.gns3_copilot.utils.command_filter import reload_forbidden_commands
|
||||
reload_forbidden_commands()
|
||||
```
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Example 1: All Commands Allowed
|
||||
|
||||
**Input:**
|
||||
```json
|
||||
{
|
||||
"project_id": "abc-123-def",
|
||||
"device_configs": [
|
||||
{
|
||||
"device_name": "R-1",
|
||||
"commands": ["show version", "show ip route"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Output:**
|
||||
```json
|
||||
{
|
||||
"device_name": "R-1",
|
||||
"status": "success",
|
||||
"output": "...",
|
||||
"diagnostic_commands": ["show version", "show ip route"]
|
||||
}
|
||||
```
|
||||
|
||||
### Example 2: Some Commands Blocked
|
||||
|
||||
**Input:**
|
||||
```json
|
||||
{
|
||||
"project_id": "abc-123-def",
|
||||
"device_configs": [
|
||||
{
|
||||
"device_name": "R-1",
|
||||
"commands": ["show version", "traceroute 8.8.8.8", "show ip route"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Output:**
|
||||
```json
|
||||
{
|
||||
"device_name": "R-1",
|
||||
"status": "partial_success",
|
||||
"output": "...",
|
||||
"diagnostic_commands": ["show version", "show ip route"],
|
||||
"blocked_commands": ["traceroute 8.8.8.8"],
|
||||
"blocked_info": {
|
||||
"traceroute 8.8.8.8": "Command 'traceroute 8.8.8.8' is not allowed because it matches the forbidden pattern 'traceroute'. This command may run longer than the tool timeout or leave the device console unavailable for subsequent commands."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Example 3: All Commands Blocked
|
||||
|
||||
**Input:**
|
||||
```json
|
||||
{
|
||||
"project_id": "abc-123-def",
|
||||
"device_configs": [
|
||||
{
|
||||
"device_name": "R-1",
|
||||
"commands": ["traceroute 8.8.8.8", "debug ip routing"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Output:**
|
||||
```json
|
||||
{
|
||||
"device_name": "R-1",
|
||||
"status": "success",
|
||||
"output": "",
|
||||
"diagnostic_commands": [],
|
||||
"blocked_commands": ["traceroute 8.8.8.8", "debug ip routing"],
|
||||
"blocked_info": {
|
||||
"traceroute 8.8.8.8": "Command 'traceroute 8.8.8.8' is not allowed because it matches the forbidden pattern 'traceroute'. This command may run longer than the tool timeout or leave the device console unavailable for subsequent commands.",
|
||||
"debug ip routing": "Command 'debug ip routing' is not allowed because it matches the forbidden pattern 'debug'. This command may run longer than the tool timeout or leave the device console unavailable for subsequent commands."
|
||||
}
|
||||
}
|
||||
```
|
||||
Edit `gns3server/agent/gns3_copilot/config/forbidden_commands.txt` and either restart the server or call `reload_forbidden_commands()` to apply changes without restart.
|
||||
|
||||
## Implementation Verification
|
||||
|
||||
### Real-World Test Results
|
||||
### Test Results (Live GNS3 Environment)
|
||||
|
||||
The command filtering system has been tested in a live GNS3 environment with actual network devices. Below are actual execution results:
|
||||
**Scenario:** IOU-L2-1 (Cisco IOS L3 switch), mixed allowed/forbidden commands.
|
||||
|
||||
**Test Scenario:**
|
||||
- Devices: IOU-L2-1, IOU-L2-2 (Cisco IOS Layer 3 switches)
|
||||
- Commands: Mixed allowed and forbidden commands
|
||||
- Forbidden pattern: `traceroute`
|
||||
|
||||
**Actual Output:**
|
||||
```json
|
||||
{
|
||||
"device_name": "IOU-L2-1",
|
||||
"status": "partial_success",
|
||||
"diagnostic_commands": [
|
||||
"show ip route",
|
||||
"show ip interface brief",
|
||||
"ping 10.0.0.1",
|
||||
"ping 10.0.0.2",
|
||||
"ping 10.0.0.4"
|
||||
"show ip route", "show ip interface brief",
|
||||
"ping 10.0.0.1", "ping 10.0.0.2", "ping 10.0.0.4"
|
||||
],
|
||||
"blocked_commands": ["traceroute 10.0.0.2"],
|
||||
"blocked_info": {
|
||||
"traceroute 10.0.0.2": "Command 'traceroute 10.0.0.2' is not allowed because it matches the forbidden pattern 'traceroute'. This command may run longer than the tool timeout or leave the device console unavailable for subsequent commands."
|
||||
"traceroute 10.0.0.2": "Command 'traceroute 10.0.0.2' is not allowed because it matches the forbidden pattern 'traceroute'. ..."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key Observations:**
|
||||
1. ✅ `traceroute` command was successfully filtered
|
||||
2. ✅ All other commands (`show`, `ping`) executed normally
|
||||
3. ✅ Status correctly set to `partial_success`
|
||||
4. ✅ Both `diagnostic_commands` (executed) and `blocked_commands` (filtered) are clearly listed
|
||||
5. ✅ Detailed blocking reason provided in `blocked_info`
|
||||
6. ✅ Tool execution continued without timeout or console lockup issues
|
||||
|
||||
### Functionality Verification Matrix
|
||||
|
||||
| Feature | Status | Notes |
|
||||
|---------|--------|-------|
|
||||
| Command filtering (substring match) | ✅ Verified | `traceroute` correctly matched and blocked |
|
||||
| Partial execution | ✅ Verified | Other commands executed successfully |
|
||||
| Return format consistency | ✅ Verified | Contains all expected fields |
|
||||
| Multi-device support | ✅ Verified | Each device filtered independently |
|
||||
| Error messages | ✅ Verified | Clear, informative blocking reasons |
|
||||
| Status field accuracy | ✅ Verified | `partial_success` set correctly |
|
||||
| Non-blocking behavior | ✅ Verified | No tool timeouts or console issues |
|
||||
| Prefix match filtering | Verified | `traceroute` correctly matched and blocked |
|
||||
| Partial execution | Verified | Other commands executed successfully |
|
||||
| `partial_success` status | Verified | Set correctly when blocked + succeeded |
|
||||
| Multi-device support | Verified | Each device filtered independently |
|
||||
| Non-blocking behavior | Verified | No tool timeouts or console issues |
|
||||
|
||||
### Benefits Confirmed
|
||||
## Future Enhancements
|
||||
|
||||
1. **Timeout Prevention**: The `traceroute` command that could have taken 30+ seconds was filtered, preventing tool timeout
|
||||
2. **Console Availability**: Since `traceroute` was not executed, the device console remained available for subsequent commands
|
||||
3. **Clear Feedback**: The LLM receives clear information about which commands were blocked and why
|
||||
4. **Partial Execution**: Useful commands (`show`, `ping`) still executed, providing valuable diagnostic information
|
||||
|
||||
## Future Enhancements (TODO)
|
||||
|
||||
### Planned Improvements
|
||||
|
||||
1. **Regex Support**: Allow more sophisticated pattern matching
|
||||
```python
|
||||
# Current: simple substring match
|
||||
"traceroute"
|
||||
|
||||
# Future: regex patterns
|
||||
"^traceroute\\s+"
|
||||
"ping\\s+.*\\s+-f"
|
||||
```
|
||||
|
||||
2. **User Override File**: Allow per-project or user-specific overrides
|
||||
```
|
||||
/etc/gns3-server/forbidden_commands_override.txt
|
||||
<project_dir>/forbidden_commands_override.txt
|
||||
```
|
||||
|
||||
3. **Web UI Configuration**: Manage forbidden commands through GNS3 web interface
|
||||
|
||||
4. **Audit Logging**: Log blocked commands for security analysis
|
||||
|
||||
5. **Per-Command Timeouts**: Configure timeouts for specific commands instead of blocking
|
||||
```python
|
||||
"command_timeouts": {
|
||||
"traceroute.*": 120,
|
||||
"debug.*": 5
|
||||
}
|
||||
```
|
||||
|
||||
6. **Interrupt Mechanism**: Send Ctrl+C to interrupt long-running commands instead of blocking
|
||||
```python
|
||||
def execute_with_timeout(cmd, timeout=30):
|
||||
try:
|
||||
return device.execute(cmd, timeout=timeout)
|
||||
except Timeout:
|
||||
device.send_break() # Ctrl+C
|
||||
return f"Command interrupted after {timeout}s"
|
||||
```
|
||||
|
||||
7. **Command State Tracking**: Track device console state to ensure availability
|
||||
```python
|
||||
device_state = {
|
||||
"console_available": True,
|
||||
"current_command": None,
|
||||
"last_prompt_seen": timestamp
|
||||
}
|
||||
```
|
||||
|
||||
### Advanced Features (Long-term)
|
||||
|
||||
- **Per-Device Filtering**: Different rules for different device types
|
||||
- **Time-Based Restrictions**: Block certain commands during specific hours
|
||||
- **Severity Levels**: Classify commands by severity (warn, block, allow)
|
||||
- **ML-Based Detection**: Learn which commands cause problems and auto-block them
|
||||
- **Regex support** for more sophisticated pattern matching
|
||||
- **Per-project override files** for forbidden commands
|
||||
- **Web UI configuration** for managing forbidden commands
|
||||
- **Audit logging** for blocked command analysis
|
||||
- **Per-command timeouts** instead of blocking
|
||||
- **Interrupt mechanism** (Ctrl+C) for long-running commands
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Commands Are Being Blocked Unexpectedly
|
||||
|
||||
**Problem:** A command you want to use is being blocked.
|
||||
|
||||
**Solution:**
|
||||
1. Check the blocked command list in the result output
|
||||
2. Identify which pattern is matching your command
|
||||
3. Edit `forbidden_commands.txt` to remove or modify the pattern
|
||||
4. Restart GNS3 server
|
||||
|
||||
### Forbidden Commands File Not Found
|
||||
|
||||
**Problem:** The system logs "Forbidden commands file not found. Using default list."
|
||||
|
||||
**Solution:**
|
||||
1. Verify the file exists at the expected location
|
||||
2. Check file permissions (should be readable by the GNS3 server process)
|
||||
3. Ensure the file is not empty
|
||||
|
||||
### Changes Not Taking Effect
|
||||
|
||||
**Problem:** You edited the file but commands are still being blocked.
|
||||
|
||||
**Solution:**
|
||||
1. Restart the GNS3 server (required to reload the configuration)
|
||||
2. Or use the `reload_forbidden_commands()` function if available in your context
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### 1. Why These Commands Are Blocked (Command Filtering)
|
||||
|
||||
| Command | Reason |
|
||||
|---------|--------|
|
||||
| `traceroute` | Can run for 30+ seconds, exceeds typical tool timeout |
|
||||
| `tracepath` | Similar to traceroute, long execution time |
|
||||
| `tracert` | Windows traceroute, same timeout issues |
|
||||
| `ping -f` | Flood ping can overwhelm lab devices |
|
||||
| `debug` | Debug commands can produce overwhelming output and destabilize devices |
|
||||
| `test` | Test commands may affect device stability |
|
||||
|
||||
### 2. Configuration Safety (Prohibited Commands)
|
||||
|
||||
In addition to timeout-based filtering, GNS3-Copilot prohibits execution of sensitive configuration commands that could lock users out of devices or cause security issues. These restrictions are enforced at the **AI agent level** through system prompts.
|
||||
|
||||
**Prohibited Configuration Categories:**
|
||||
|
||||
| Category | Commands | Reason |
|
||||
|----------|----------|--------|
|
||||
| **AAA Configuration** | `aaa new-model`, `radius-server`, `tacacs-server` | May lock users out; requires manual configuration |
|
||||
| **Login Passwords** | `enable secret`, `password`, `username ... password` | Can lock users out; security risk |
|
||||
| **Console/VTY Authentication** | `line console 0`, `line vty 0 4`, `login local` | May block console access |
|
||||
| **Password Encryption** | `service password-encryption` | Security-sensitive; manual setup required |
|
||||
| **Access Control Lists** | `access-list ... deny ip any any` (on mgmt interfaces) | Can block management access |
|
||||
| **Dangerous System Operations** | `reload`, `erase startup-config`, `format` | Destructive operations |
|
||||
|
||||
**Implementation:**
|
||||
- **System Prompt**: Restrictions are defined in `lab_automation_assistant_prompt.py`
|
||||
- **Behavior**: When AI detects these commands, it provides configuration guidance instead of execution
|
||||
- **Example Response**:
|
||||
```
|
||||
"I cannot execute AAA/password commands directly as they may lock you out.
|
||||
Here's how to configure them manually..."
|
||||
```
|
||||
|
||||
**User Override:**
|
||||
Users can manually execute these commands through:
|
||||
1. Direct device console access
|
||||
2. GNS3 device console
|
||||
3. Manual SSH/Telnet connection
|
||||
|
||||
### 3. Multi-line Command Handling
|
||||
|
||||
**Problem:** Some configuration commands contain embedded newlines (e.g., `banner motd`), which cause Netmiko to fail when processed as single strings.
|
||||
|
||||
**Solution:** The system automatically expands multi-line commands before execution.
|
||||
|
||||
**Example:**
|
||||
```python
|
||||
# Input (single string with newlines)
|
||||
["banner motd #\nWelcome\nUnauthorized access prohibited\n#"]
|
||||
|
||||
# After expansion
|
||||
["banner motd #", "Welcome", "Unauthorized access prohibited", "#"]
|
||||
```
|
||||
|
||||
**Implementation:**
|
||||
- **Location**: `config_tools_nornir.py:_expand_multiline_commands()`
|
||||
- **Detection**: Checks for `\n` newline character in commands
|
||||
- **Processing**: Splits by `\n` and filters empty lines
|
||||
- **Logging**: Records expansion for debugging
|
||||
|
||||
**Supported Commands:**
|
||||
- `banner motd`, `banner login`, `banner exec`
|
||||
- Multi-line ACLs
|
||||
- Route-maps with continue statements
|
||||
- Any command with embedded newlines
|
||||
|
||||
### Best Practices
|
||||
|
||||
1. **Education Environment**: Use the default filtering for safety
|
||||
2. **Personal Lab**: Consider which commands you actually need
|
||||
3. **Production-like Environment**: Keep restrictions enabled
|
||||
4. **Always Understand**: Before allowing a command, understand why it was blocked
|
||||
| Problem | Solution |
|
||||
|---------|----------|
|
||||
| Command blocked unexpectedly | Check `blocked_commands` in result, identify matching pattern, edit `forbidden_commands.txt` |
|
||||
| "File not found, using defaults" | Verify `config/forbidden_commands.txt` exists and is readable |
|
||||
| Changes not taking effect | Restart server or call `reload_forbidden_commands()` |
|
||||
|
||||
## Related Documentation
|
||||
|
||||
@ -489,13 +222,3 @@ Users can manually execute these commands through:
|
||||
- [GNS3-Copilot Documentation](../README.md)
|
||||
- [Contributing Guide](../../CONTRIBUTING.md)
|
||||
|
||||
## Feedback and Issues
|
||||
|
||||
If you:
|
||||
- Find commands that should be blocked by default
|
||||
- Need to allow commands for legitimate use cases
|
||||
- Have suggestions for improving the filtering system
|
||||
|
||||
Please submit an issue: https://github.com/yueguobin/gns3-copilot/issues
|
||||
|
||||
|
||||
|
||||
@ -1,255 +1,161 @@
|
||||
# LLM Context Window Management Implementation Document
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
|
||||
|
||||
|
||||
# LLM Context Window Management
|
||||
|
||||
## Overview
|
||||
|
||||
This document explains the context window management implementation mechanism for GNS3 Copilot, including message trimming, token counting, and configuration validation.
|
||||
GNS3 Copilot's context window management prevents LLM context overflow by trimming conversation history to fit within configured token limits. It uses tiktoken for accurate token counting, supports three context strategies (conservative/balanced/aggressive), and automatically injects GNS3 topology information into the system prompt before each LLM call.
|
||||
|
||||
## Implementation Architecture
|
||||
## Architecture
|
||||
|
||||
### 1. Core Modules
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph "context_manager.py"
|
||||
A[create_pre_model_hook] --> B[_inject_topology_into_system]
|
||||
A --> C[estimate_tool_tokens]
|
||||
A --> D[trim_messages via LangChain]
|
||||
E[count_tokens via tiktoken]
|
||||
E --> C
|
||||
E --> F[_count_tokens_for_message]
|
||||
F --> D
|
||||
end
|
||||
|
||||
**File Location**: `gns3server/agent/gns3_copilot/agent/context_manager.py`
|
||||
subgraph "gns3_copilot.py"
|
||||
G[llm_call node] --> H[Get topology via GNS3TopologyTool]
|
||||
H --> A
|
||||
A --> I[model_with_tools.invoke]
|
||||
end
|
||||
|
||||
#### Token Counting Strategy
|
||||
|
||||
The system uses **tiktoken** for token counting (context_manager.py:60):
|
||||
|
||||
```python
|
||||
_tiktoken_encoding = tiktoken.get_encoding("cl100k_base")
|
||||
style A fill:#4a9eff,color:#fff
|
||||
style D fill:#ff6b6b,color:#fff
|
||||
style E fill:#51cf66,color:#fff
|
||||
```
|
||||
|
||||
**Required Dependency**:
|
||||
```bash
|
||||
pip install tiktoken>=0.8.0
|
||||
## Key Components
|
||||
|
||||
| Function | File | Purpose |
|
||||
|----------|------|---------|
|
||||
| `create_pre_model_hook()` | `context_manager.py` | Factory that creates the preprocessing hook |
|
||||
| `_inject_topology_into_system()` | `context_manager.py` | Injects topology into system prompt via `{{topology_info}}` placeholder |
|
||||
| `count_tokens()` | `context_manager.py` | Accurate token counting using tiktoken (`cl100k_base`) |
|
||||
| `estimate_tool_tokens()` | `context_manager.py` | Estimates token cost of tool definitions (JSON serialization) |
|
||||
| `_count_tokens_for_message()` | `context_manager.py` | Token counter adapter for LangChain's `trim_messages` |
|
||||
| `llm_call()` | `gns3_copilot.py` | StateGraph node that orchestrates the full LLM call pipeline |
|
||||
|
||||
**Token Counting**: Uses tiktoken `cl100k_base` encoding (GPT-4 compatible, 95%+ accuracy for most modern LLMs).
|
||||
|
||||
**tiktoken Cache**: Encoding files are cached locally at `<agent_package>/cache/tiktoken/` to avoid repeated downloads. This is configured before tiktoken import via `TIKTOKEN_CACHE_DIR` environment variable.
|
||||
|
||||
**Required Dependency**: `tiktoken>=0.8.0` — if not installed, `ModuleNotFoundError` is raised at startup.
|
||||
|
||||
## Trimming Process
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[llm_call node invoked] --> B[Get topology via GNS3TopologyTool]
|
||||
B --> C[Create pre_model_hook]
|
||||
C --> D[Inject topology into system prompt]
|
||||
D --> E{topology data available?}
|
||||
E -->|Yes| F[Replace placeholder with topology content]
|
||||
E -->|No| G[Replace placeholder with 'No topology information available']
|
||||
F --> H[Estimate tool definition tokens]
|
||||
G --> H
|
||||
H --> I[Calculate trim_messages budget]
|
||||
I --> J[trim_messages with strategy='last']
|
||||
J --> K[Return prepared messages]
|
||||
K --> L[model.invoke with prepared messages]
|
||||
```
|
||||
|
||||
If tiktoken is not installed, the system will throw a `ModuleNotFoundError` at startup.
|
||||
### Token Budget Calculation
|
||||
|
||||
#### Key Functions
|
||||
|
||||
**`count_tokens(text: str) -> int`** (context_manager.py:84-100)
|
||||
- Uses tiktoken to accurately count tokens in text
|
||||
- Uses `cl100k_base` encoding
|
||||
- Returns the exact token count
|
||||
|
||||
**`estimate_tool_tokens(tools: list) -> int`** (context_manager.py:103-169)
|
||||
- Serializes tool schema to JSON
|
||||
- Uses tiktoken to count token consumption of tool definitions
|
||||
- Supports Pydantic v1/v2 compatibility
|
||||
- Falls back to 1000 tokens on failure
|
||||
|
||||
**`create_pre_model_hook(...)`** (context_manager.py:195-402)
|
||||
- Creates a preprocessing function (pre_model_hook)
|
||||
- Automatically executes before each LLM call:
|
||||
1. Injects topology information into system prompt
|
||||
2. Estimates token consumption of tool definitions
|
||||
3. Trims message history to fit context limits
|
||||
- Returns a callable function for preparing messages
|
||||
|
||||
### 2. Detailed Trimming Logic
|
||||
|
||||
#### 2.1 Token Budget Allocation
|
||||
|
||||
When calling the LLM, the content sent consists of two parts:
|
||||
|
||||
```
|
||||
Complete request sent to LLM:
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ 1. Messages (managed by us) │
|
||||
│ ├─ SystemMessage: system prompt + topology (template injection) │
|
||||
│ └─ HumanMessage/AIMessage: user messages / history messages │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ 2. Tool Definitions (LangChain adds automatically, not in messages) │
|
||||
│ ├─ Tool 1 schema (name, description, parameters) │
|
||||
│ ├─ Tool 2 schema │
|
||||
│ └─ ... (about 500-1500 tokens per tool) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A["context_limit<br/>(e.g. 128K)"] -->|"× strategy_ratio"| B["Input Budget<br/>(e.g. 96K @ balanced)"]
|
||||
B -->|"− tool_tokens"| C["trim_messages Budget<br/>(includes system message)"]
|
||||
C --> D["trim_messages<br/>max_tokens parameter"]
|
||||
```
|
||||
|
||||
**System Message Structure**:
|
||||
- Uses template variable `{{topology_info}}` to dynamically inject topology
|
||||
- System prompt contains placeholder: `"### CURRENT TOPOLOGY\n{{topology_info}}"`
|
||||
- If topology exists, replaces with actual content
|
||||
- If no topology, replaces with `"(No topology information available)"`
|
||||
**Key**: `max_tokens` passed to `trim_messages` **includes** the system message. System tokens are NOT subtracted separately — `trim_messages` handles system message preservation via `include_system=True`.
|
||||
|
||||
#### 2.2 Trimming Process
|
||||
### Budget Calculation Example
|
||||
|
||||
```
|
||||
Step 1: Calculate Input Budget
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ context_limit: 128,000 tokens (128K) │
|
||||
│ strategy: balanced (75%) │
|
||||
│ │
|
||||
│ Input budget = 128 × 1000 × 0.75 = 96,000 tokens │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
Step 2: Subtract Tool Definitions
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Input budget: 96,000 tokens │
|
||||
│ Tool definitions: 1,725 tokens │
|
||||
│ │
|
||||
│ Available for messages = 96,000 - 1,725 = 94,275 tokens │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
Step 3: trim_messages Processing
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Call LangChain's trim_messages: │
|
||||
│ - max_tokens = 94,275 (includes system message) │
|
||||
│ - strategy = "last" (keep latest messages) │
|
||||
│ - token_counter = tiktoken counting function │
|
||||
│ - include_system = True (always keep system) │
|
||||
│ │
|
||||
│ trim_messages will: │
|
||||
│ 1. Keep SystemMessage (system + topology) │
|
||||
│ 2. Starting from latest messages, keep as much history │
|
||||
│ 3. When exceeding limit, discard oldest messages │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
| Step | Calculation | Result |
|
||||
|------|------------|--------|
|
||||
| Context limit | 128 (configured) | 128K tokens |
|
||||
| Strategy ratio | balanced = 0.75 | 128 × 1000 × 0.75 = 96,000 |
|
||||
| Tool definitions | estimated from JSON schema | ~1,725 tokens |
|
||||
| trim_messages budget | 96,000 − 1,725 | **94,275 tokens** (includes system) |
|
||||
|
||||
#### 2.3 Trimming Priority
|
||||
### Trimming Priority
|
||||
|
||||
The system preserves content in the following priority order:
|
||||
| Priority | Content | Behavior |
|
||||
|----------|---------|----------|
|
||||
| 1 | System Message (system prompt + topology) | Always kept (`include_system=True`) |
|
||||
| 2 | Latest conversation messages | Kept from newest to oldest |
|
||||
| 3 | Older conversation history | Discarded first when exceeding budget |
|
||||
|
||||
| Priority | Content | Description |
|
||||
|----------|---------|-------------|
|
||||
| 1️⃣ | System Message (system prompt + topology) | Never removed |
|
||||
| 2️⃣ | Latest user message | Keep at least the last 1 |
|
||||
| 3️⃣ | Old conversation history | Discarded in chronological order |
|
||||
**Note**: System prompt and topology are merged into a single `SystemMessage` via the `{{topology_info}}` template variable and cannot be trimmed independently.
|
||||
|
||||
**Note**: System prompt and topology info are merged into one SystemMessage via template variable and cannot be separated.
|
||||
### Edge Case Handling
|
||||
|
||||
#### 2.4 Edge Case Handling
|
||||
|
||||
| Scenario | Handling |
|
||||
| Scenario | Behavior |
|
||||
|----------|----------|
|
||||
| System (including topology) > budget | Keep complete SystemMessage (cannot separate system and topology) |
|
||||
| Tools > budget | ERROR log, suggest increasing context_limit or reducing tool count |
|
||||
| All history trimmed | Keep last 1 user message |
|
||||
| System + tools > input budget | ERROR log with recommendations (reduce prompt/tools, use larger model, switch to conservative). `trim_messages` is still called — `include_system=True` ensures system message is kept, but no history messages will fit. |
|
||||
| Trim budget < system × 1.5 | WARNING log: minimal room for conversation history |
|
||||
| Invalid context_strategy | Falls back to `balanced` with WARNING log |
|
||||
| `trim_messages` throws exception | Returns untrimmed messages (graceful degradation) |
|
||||
| Missing `{{topology_info}}` placeholder | Returns original messages without injection, WARNING log |
|
||||
| Topology retrieval fails | Injects `"(No topology information available)"` placeholder |
|
||||
|
||||
**Important Notes**:
|
||||
- When system + topology exceed available budget, **both are preserved**
|
||||
- Cannot discard only topology while keeping system prompt (already merged)
|
||||
## Integration with GNS3 Copilot
|
||||
|
||||
### 3. Integration with GNS3 Copilot
|
||||
**File**: `gns3server/agent/gns3_copilot/agent/gns3_copilot.py`
|
||||
|
||||
**File Location**: `gns3server/agent/gns3_copilot/agent/gns3_copilot.py`
|
||||
### Why Direct Call, Not Config-based?
|
||||
|
||||
#### Implementation Method
|
||||
|
||||
**Key Point**: The system uses a **custom StateGraph**, not LangGraph's pre-built agent.
|
||||
|
||||
Therefore, `pre_model_hook` cannot be passed via `model.invoke(config={"configurable": {"pre_model_hook": ...}})`.
|
||||
|
||||
**Correct Usage**: **Directly call** the `pre_hook` function to prepare messages.
|
||||
|
||||
```python
|
||||
def llm_call(state: dict, config: RunnableConfig | None = None):
|
||||
"""LLM decides whether to call a tool or not."""
|
||||
|
||||
# 1. Get topology information
|
||||
project_id = config["configurable"].get("project_id")
|
||||
topology_info = None
|
||||
if project_id:
|
||||
topology_tool = GNS3TopologyTool()
|
||||
topology = topology_tool._run(project_id=project_id)
|
||||
if topology and "error" not in topology:
|
||||
topology_info = topology
|
||||
|
||||
# 2. Create pre_model_hook
|
||||
system_prompt = load_system_prompt()
|
||||
pre_hook = create_pre_model_hook(
|
||||
system_prompt=system_prompt,
|
||||
get_topology_func=lambda s: s.get("topology_info"),
|
||||
get_llm_config_func=get_current_llm_config,
|
||||
get_tools_func=lambda: tools,
|
||||
)
|
||||
|
||||
# 3. Create model with tools
|
||||
model_with_tools = create_base_model_with_tools(tools, llm_config=llm_config)
|
||||
|
||||
# 4. ⭐ Key: directly call pre_hook to prepare messages
|
||||
logger.info("Calling pre_hook to prepare %d messages", len(messages))
|
||||
prepared_state = pre_hook({"messages": messages, "topology_info": topology_info})
|
||||
prepared_messages = prepared_state["messages"]
|
||||
|
||||
# 5. Use prepared messages to call LLM
|
||||
response = model_with_tools.invoke(prepared_messages)
|
||||
|
||||
return {"messages": [response], ...}
|
||||
```
|
||||
|
||||
#### Why Not Pass via Config?
|
||||
|
||||
LangGraph's `pre_model_hook` parameter only applies to **pre-built agents**, not custom StateGraphs.
|
||||
The agent uses a **custom StateGraph** (`StateGraph(MessagesState)`), not LangGraph's pre-built agent. LangGraph's `pre_model_hook` config parameter only works with pre-built agents like `create_react_agent`.
|
||||
|
||||
| Agent Type | pre_model_hook Support |
|
||||
|------------|------------------------|
|
||||
| `create_react_agent` | ✅ Via `pre_model_hook` parameter |
|
||||
| `chat_agent_executor` | ✅ Via `pre_model_hook` parameter |
|
||||
| **Custom StateGraph** | ❌ **Not supported**, need to call directly |
|
||||
| `create_react_agent` | Via `config={"configurable": {"pre_model_hook": ...}}` |
|
||||
| `chat_agent_executor` | Via `config={"configurable": {"pre_model_hook": ...}}` |
|
||||
| **Custom StateGraph** | **Not supported** — must call directly |
|
||||
|
||||
Our implementation uses a custom StateGraph (`agent_builder = StateGraph(MessagesState)`), so we must call `pre_hook` directly.
|
||||
### Execution Flow
|
||||
|
||||
### 4. Execution Flow
|
||||
In the `llm_call` node of `gns3_copilot.py`:
|
||||
|
||||
```
|
||||
User sends message
|
||||
↓
|
||||
llm_call node is called
|
||||
↓
|
||||
Get project_id (from config["configurable"])
|
||||
↓
|
||||
Call GNS3TopologyTool._run(project_id) to get topology
|
||||
↓
|
||||
Store topology_info to state
|
||||
↓
|
||||
Create pre_model_hook (via create_pre_model_hook())
|
||||
↓
|
||||
[Key] Directly call pre_hook({"messages": messages, "topology_info": topology_info})
|
||||
├─ 1. Inject topology into system prompt
|
||||
├─ 2. Estimate tool definitions tokens
|
||||
├─ 3. Call trim_messages() to trim messages
|
||||
└─ 4. Return prepared message list
|
||||
↓
|
||||
Call model.invoke() with prepared messages
|
||||
↓
|
||||
Return LLM response
|
||||
```
|
||||
1. Get `llm_config` from request-scoped context variable
|
||||
2. Get `project_id` from `config["configurable"]`
|
||||
3. Retrieve topology via `GNS3TopologyTool._run(project_id)`
|
||||
4. Select tools based on `copilot_mode` (`teaching_assistant` vs `lab_automation_assistant`)
|
||||
5. Load system prompt and create `pre_model_hook`
|
||||
6. **Directly call** `pre_hook({"messages": messages, "topology_info": topology_info})` to prepare messages
|
||||
7. Invoke `model_with_tools.invoke(prepared_messages)`
|
||||
|
||||
---
|
||||
> **Note**: The actual `llm_call` function also includes defensive checks for missing config, empty messages, and topology retrieval errors — see source code for full details.
|
||||
|
||||
## Strategy Implementation
|
||||
|
||||
### Context Strategy Ratios
|
||||
|
||||
**Definition** (context_manager.py:68-72):
|
||||
Defined as `CONTEXT_STRATEGY_RATIOS` in `context_manager.py`, with default `DEFAULT_CONTEXT_STRATEGY = "balanced"`.
|
||||
|
||||
```python
|
||||
CONTEXT_STRATEGY_RATIOS = {
|
||||
"conservative": 0.60,
|
||||
"balanced": 0.75,
|
||||
"aggressive": 0.85,
|
||||
}
|
||||
```
|
||||
|
||||
**Default Value** (context_manager.py:74):
|
||||
```python
|
||||
DEFAULT_CONTEXT_STRATEGY = "balanced"
|
||||
```
|
||||
|
||||
### Strategy Comparison
|
||||
|
||||
| Strategy | Input Ratio | Output Reserved | Calculation Formula |
|
||||
|----------|-------------|-----------------|---------------------|
|
||||
| Strategy | Input Ratio | Output Reserved | Calculation |
|
||||
|----------|-------------|-----------------|-------------|
|
||||
| Conservative | 60% | 40% | `context_limit × 1000 × 0.60` |
|
||||
| Balanced | 75% | 25% | `context_limit × 1000 × 0.75` |
|
||||
| Aggressive | 85% | 15% | `context_limit × 1000 × 0.85` |
|
||||
|
||||
---
|
||||
|
||||
## Log Output
|
||||
|
||||
### Normal Case (topology successfully injected)
|
||||
### Normal Case (topology injected)
|
||||
|
||||
```
|
||||
INFO: Calling pre_hook to prepare 1 messages
|
||||
@ -268,7 +174,7 @@ INFO: Messages trimmed: 50 → 25 msgs. Total: ~82000 tokens + 1725 tools = 8372
|
||||
INFO: Messages prepared: 50 → 25
|
||||
```
|
||||
|
||||
### When topology is None
|
||||
### When Topology is None
|
||||
|
||||
```
|
||||
INFO: Calling pre_hook to prepare 1 messages
|
||||
@ -276,53 +182,22 @@ WARNING: ✗ Topology data is None, injecting placeholder
|
||||
INFO: Context ready: 2 msgs, ~800 tokens + 1725 tools = 2525 / 128K (2.0%), strategy=balanced
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
### tiktoken Not Installed
|
||||
| Error | Behavior | Resolution |
|
||||
|-------|----------|------------|
|
||||
| tiktoken not installed | `ModuleNotFoundError` at startup | `pip install tiktoken>=0.8.0` |
|
||||
| `context_limit` missing | `ValueError` raised | Add `context_limit` to LLM config |
|
||||
| Invalid `context_limit` | `ValueError` raised | Must be positive integer |
|
||||
| Invalid `context_strategy` | WARNING, fallback to `balanced` | Use `conservative`/`balanced`/`aggressive` |
|
||||
| `trim_messages` failure | ERROR log, returns untrimmed messages | Check message format compatibility |
|
||||
|
||||
If tiktoken is not installed, the system will throw an error at startup:
|
||||
## Deprecated API
|
||||
|
||||
```python
|
||||
ModuleNotFoundError: No module named 'tiktoken'
|
||||
```
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
pip install tiktoken>=0.8.0
|
||||
```
|
||||
|
||||
### context_limit Missing or Invalid
|
||||
|
||||
If there is no `context_limit` in the LLM configuration or the value is invalid (context_manager.py:285-295):
|
||||
|
||||
```python
|
||||
if "context_limit" not in llm_config:
|
||||
raise ValueError("context_limit is required in LLM config")
|
||||
|
||||
limit = llm_config["context_limit"]
|
||||
if not isinstance(limit, int) or limit <= 0:
|
||||
raise ValueError(f"Invalid context_limit: {limit}")
|
||||
```
|
||||
|
||||
### Trimming Failure
|
||||
|
||||
```python
|
||||
try:
|
||||
trimmed = trim_messages(...)
|
||||
except Exception as e:
|
||||
logger.error("Failed to trim messages: %s", e)
|
||||
logger.warning("Returning original messages due to trimming error")
|
||||
return {"messages": messages_with_system}
|
||||
```
|
||||
|
||||
---
|
||||
`prepare_context_messages()` is deprecated. It remains in `context_manager.py` for backward compatibility but emits `DeprecationWarning`. Use `create_pre_model_hook()` instead.
|
||||
|
||||
## Related Source Files
|
||||
|
||||
- `gns3server/agent/gns3_copilot/agent/context_manager.py` - Context management core logic
|
||||
- `gns3server/agent/gns3_copilot/agent/gns3_copilot.py` - LLM call node (StateGraph)
|
||||
- `gns3server/agent/gns3_copilot/agent/model_factory.py` - Model creation and tool binding
|
||||
|
||||
|
||||
- `gns3server/agent/gns3_copilot/agent/context_manager.py` — Context management core logic
|
||||
- `gns3server/agent/gns3_copilot/agent/gns3_copilot.py` — LLM call node (StateGraph)
|
||||
- `gns3server/agent/gns3_copilot/agent/model_factory.py` — Model creation and tool binding
|
||||
|
||||
@ -1,3 +1,11 @@
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
|
||||
|
||||
|
||||
# LLM Model Configurations API
|
||||
|
||||
## Overview
|
||||
@ -40,7 +48,7 @@ User's own config > User's group config
|
||||
| `config_id` | UUID | Primary key |
|
||||
| `name` | VARCHAR(100) | Configuration name (table-level for indexing) |
|
||||
| `model_type` | VARCHAR(50) | Model type (table-level for filtering) |
|
||||
| `config` | JSONB | Configuration data (provider, base_url, model, temperature, api_key, etc.) |
|
||||
| `config` | JSON (JSONB on PostgreSQL) | Configuration data (provider, base_url, model, temperature, api_key, etc.) |
|
||||
| `user_id` | UUID (nullable) | Foreign key to users table |
|
||||
| `group_id` | UUID (nullable) | Foreign key to user_groups table |
|
||||
| `is_default` | BOOLEAN | Default configuration flag |
|
||||
@ -77,13 +85,15 @@ GNS3-Copilot uses LangChain's `init_chat_model` function, which supports the fol
|
||||
|
||||
| Provider | Provider Value | Default base_url | base_url Required? | Notes |
|
||||
|----------|---------------|------------------|-------------------|-------|
|
||||
| OpenAI | `openai` | `https://api.openai.com/v1` | ❌ No | Most popular, supports GPT-4, GPT-3.5 |
|
||||
| Anthropic | `anthropic` | `https://api.anthropic.com` | ❌ No | Claude 3.5 Sonnet, Claude 3 Opus |
|
||||
| Google | `google` | `https://generativelanguage.googleapis.com` | ❌ No | Gemini Pro, Gemini Flash |
|
||||
| OpenAI | `openai` | `https://api.openai.com/v1` | ✅ Yes (planned optional) | Most popular, supports GPT-4, GPT-3.5 |
|
||||
| Anthropic | `anthropic` | `https://api.anthropic.com` | ✅ Yes (planned optional) | Claude 3.5 Sonnet, Claude 3 Opus |
|
||||
| Google | `google` | `https://generativelanguage.googleapis.com` | ✅ Yes (planned optional) | Gemini Pro, Gemini Flash |
|
||||
| AWS Bedrock | `aws` | Varies by region | ✅ Yes | Requires AWS configuration |
|
||||
| Ollama | `ollama` | `http://localhost:11434` | ✅ Yes | Local models, typically running on localhost |
|
||||
| DeepSeek | `deepseek` | `https://api.deepseek.com` | ❌ No | DeepSeek Chat, DeepSeek Coder |
|
||||
| xAI | `xai` | `https://api.x.ai` | ❌ No | Grok models |
|
||||
| DeepSeek | `deepseek` | `https://api.deepseek.com` | ✅ Yes (planned optional) | DeepSeek Chat, DeepSeek Coder |
|
||||
| xAI | `xai` | `https://api.x.ai` | ✅ Yes (planned optional) | Grok models |
|
||||
|
||||
> **Note:** The `base_url` field is currently **required** in the API schema for all providers. Entries marked "planned optional" indicate providers where `base_url` may become optional in a future release (see [Future Enhancements](#optional-base_url-field)). For now, use the default endpoint URL listed in the table.
|
||||
|
||||
### When to Specify `base_url`
|
||||
|
||||
@ -169,7 +179,7 @@ For a complete list of LangChain-supported providers, see: https://python.langch
|
||||
| Method | Path | Description | Privilege |
|
||||
|--------|------|-------------|-----------|
|
||||
| GET | `/v3/access/users/{user_id}/llm-model-configs` | Get user's effective configs (own + inherited) | User.Audit |
|
||||
| GET | `/v3/access/users/{user_id}/llm-model-configs/own` | Get user's own configs only | User.Audit |
|
||||
| GET | `/v3/access/users/{user_id}/llm-model-configs/own` | Get user's own configs only (returns `List[LLMModelConfigResponse]`, a plain array) | User.Audit |
|
||||
| GET | `/v3/access/users/{user_id}/llm-model-configs/default` | Get user's default configuration | User.Audit |
|
||||
| POST | `/v3/access/users/{user_id}/llm-model-configs` | Create a new configuration | User.Modify |
|
||||
| PUT | `/v3/access/users/{user_id}/llm-model-configs/{config_id}` | Update a configuration | User.Modify |
|
||||
@ -242,6 +252,7 @@ For a complete list of LangChain-supported providers, see: https://python.langch
|
||||
| `context_limit` | integer (optional) | Model context window limit in K tokens |
|
||||
| `context_strategy` | string (optional) | Context trimming strategy |
|
||||
| `is_default` | boolean (optional) | Default flag |
|
||||
| `copilot_mode` | string (optional) | GNS3-Copilot mode: "teaching_assistant" or "lab_automation_assistant" |
|
||||
| `expected_version` | integer (optional) | **Optimistic locking version** |
|
||||
|
||||
**Note:** When using `expected_version`, the API will verify the version hasn't changed since you read the data. If it has, you'll receive a 409 Conflict error.
|
||||
@ -249,6 +260,9 @@ For a complete list of LangChain-supported providers, see: https://python.langch
|
||||
**Reserved Fields:**
|
||||
- `max_tokens`: Reserved for future implementation. Currently not used by the system. The maximum output tokens are controlled automatically by the LLM provider based on the model and input size.
|
||||
|
||||
**Update Limitations:**
|
||||
- `context_strategy` and `copilot_mode` can be set to new values but **cannot be cleared to null** via update. This is because the API filters out null values before processing.
|
||||
|
||||
### LLMModelConfigResponse
|
||||
|
||||
| Field | Type | Description |
|
||||
|
||||
@ -1,3 +1,11 @@
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
|
||||
|
||||
|
||||
# Multi-Vendor Network Device Support
|
||||
|
||||
## Overview
|
||||
@ -6,12 +14,12 @@ GNS3-Copilot supports network devices from multiple vendors through Netmiko and
|
||||
|
||||
## Supported Vendors
|
||||
|
||||
| Vendor | Platform | Device Type | Protocol | Status |
|
||||
|--------|----------|-------------|----------|--------|
|
||||
| **Cisco** | `cisco_ios` | `cisco_ios_telnet` | Telnet | ✅ Tested |
|
||||
| **Huawei** | `huawei` | `gns3_huawei_telnet_ce` | Telnet | ✅ Tested (Custom Driver) |
|
||||
| **Ruijie (锐捷)** | `ruijie_os` | `gns3_ruijie_telnet` | Telnet | ✅ Tested (Custom Driver) |
|
||||
| **VPCS** | `vpcs` | `gns3_vpcs_telnet` | Telnet | ✅ Tested (Custom Driver) |
|
||||
| Vendor | Device Type | Protocol | Status |
|
||||
|--------|-------------|----------|--------|
|
||||
| **Cisco** | `cisco_ios_telnet` | Telnet | ✅ Tested |
|
||||
| **Huawei** | `gns3_huawei_telnet_ce` | Telnet | ✅ Tested (Custom Driver) |
|
||||
| **Ruijie (锐捷)** | `gns3_ruijie_telnet` | Telnet | ✅ Tested (Custom Driver) |
|
||||
| **VPCS** | `gns3_vpcs_telnet` | Telnet | ✅ Tested (Custom Driver, Simulator) |
|
||||
|
||||
## Custom VPCS Driver (`VPCSTelnet`)
|
||||
|
||||
@ -25,10 +33,13 @@ VPCS (Virtual PC Simulator) is a lightweight virtual PC simulator used in GNS3 l
|
||||
|
||||
### Solution: Lightweight Custom Driver
|
||||
|
||||
```
|
||||
BaseConnection (Netmiko base class)
|
||||
↓
|
||||
VPCSTelnet (Custom GNS3 driver)
|
||||
```mermaid
|
||||
graph TD
|
||||
A[BaseConnection<br/>Netmiko base class] --> B[VPCSTelnet<br/>Custom GNS3 driver]
|
||||
B --> C[No authentication]
|
||||
B --> D[Simple prompt: PC\d+]
|
||||
B --> E[No config mode]
|
||||
B --> F[ANSI stripping]
|
||||
```
|
||||
|
||||
**Why Not Use Standard Telnet Driver?**
|
||||
@ -81,26 +92,38 @@ def disable_paging(self) -> str:
|
||||
return "" # VPCS doesn't use paging
|
||||
```
|
||||
|
||||
**5. Custom `send_command`**
|
||||
VPCS overrides `send_command` with non-standard defaults:
|
||||
- `strip_prompt=False`, `strip_command=False`, `normalize=False` — VPCS output doesn't need standard Netmiko processing
|
||||
- Uses `_strip_ansi_codes()` to clean terminal escape sequences from VPCS output
|
||||
|
||||
**6. ANSI Escape Code Stripping**
|
||||
VPCS output often contains ANSI terminal codes. The `_strip_ansi_codes()` method strips bold, underline, reset, and color codes before returning output to callers.
|
||||
|
||||
### VPCS Tool Usage
|
||||
|
||||
The VPCS driver is used by the `execute_vpcs_commands` tool:
|
||||
|
||||
```python
|
||||
from gns3server.agent.gns3_copilot.tools_v2.vpcs_tools_netmiko import VPCSCommands
|
||||
> File: `gns3server/agent/gns3_copilot/tools_v2/vpcs_tools_netmiko.py`
|
||||
|
||||
tool = VPCSCommands()
|
||||
result = tool._run(json.dumps({
|
||||
"project_id": "<PROJECT_UUID>",
|
||||
"device_configs": [
|
||||
{
|
||||
"device_name": "PC1",
|
||||
"commands": [
|
||||
"ip 10.10.0.12/24 10.10.0.254",
|
||||
"ping 10.10.0.254"
|
||||
]
|
||||
**VPCS Connection Parameters:**
|
||||
|
||||
VPCS devices use additional Netmiko parameters for reliable connections:
|
||||
| Parameter | Value | Reason |
|
||||
|-----------|-------|--------|
|
||||
| `fast_cli` | `False` | VPCS is slow, disable fast CLI mode |
|
||||
| `global_delay_factor` | `2.0` | Double the delay between commands |
|
||||
|
||||
```python
|
||||
"connection_options": {
|
||||
"netmiko": {
|
||||
"extras": {
|
||||
"device_type": "gns3_vpcs_telnet",
|
||||
"fast_cli": False,
|
||||
"global_delay_factor": 2.0,
|
||||
}
|
||||
]
|
||||
}))
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### VPCS Built-in Template Configuration
|
||||
@ -111,34 +134,23 @@ VPCS nodes created from the built-in template automatically include the necessar
|
||||
|
||||
| Tag | Value | Purpose |
|
||||
|-----|-------|---------|
|
||||
| `platform` | `vpcs` | Platform identification |
|
||||
| `device_type` | `gns3_vpcs_telnet` | Netmiko driver selection |
|
||||
|
||||
**Built-in Template Definition:**
|
||||
```python
|
||||
# gns3server/services/templates.py
|
||||
{
|
||||
"template_id": uuid.uuid5(uuid.NAMESPACE_X500, "vpcs"),
|
||||
"template_type": "vpcs",
|
||||
"name": "VPCS",
|
||||
"default_name_format": "PC{0}",
|
||||
"category": "guest",
|
||||
"symbol": "vpcs_guest",
|
||||
"builtin": True,
|
||||
"tags": ["platform:vpcs", "device_type:gns3_vpcs_telnet"], # ✅ Auto-applied
|
||||
}
|
||||
```
|
||||
> File: `gns3server/services/templates.py`
|
||||
|
||||
The VPCS built-in template includes `"tags": ["device_type:gns3_vpcs_telnet"]`, which is automatically applied to all VPCS nodes created from the template.
|
||||
|
||||
**User Benefits:**
|
||||
- ✅ **No manual tagging required** - Tags are applied automatically when creating VPCS nodes
|
||||
- ✅ **Automatic driver selection** - Copilot tools automatically use the correct Netmiko driver
|
||||
- ✅ **Consistent behavior** - All VPCS nodes from the built-in template work identically
|
||||
- ✅ **Zero configuration** - Users don't need to understand device_type tags
|
||||
- ✅ **No manual tagging required** — Tags are applied automatically when creating VPCS nodes
|
||||
- ✅ **Automatic driver selection** — Copilot tools automatically use the correct Netmiko driver
|
||||
- ✅ **Consistent behavior** — All VPCS nodes from the built-in template work identically
|
||||
- ✅ **Zero configuration** — Users don't need to understand device_type tags
|
||||
|
||||
**How It Works:**
|
||||
1. User creates a VPCS node from the built-in "VPCS" template
|
||||
2. Node automatically inherits the tags: `platform:vpcs` and `device_type:gns3_vpcs_telnet`
|
||||
3. Copilot tools read these tags and select the appropriate VPCS Netmiko driver
|
||||
2. Node automatically inherits the tag: `device_type:gns3_vpcs_telnet`
|
||||
3. Copilot tools read the tag and select the appropriate VPCS Netmiko driver
|
||||
4. Commands execute using the VPCS-optimized driver (no authentication, simple prompts)
|
||||
|
||||
### Supported VPCS Commands
|
||||
@ -192,14 +204,14 @@ Telnet Connection → Direct access to command line (no login prompts)
|
||||
|
||||
### Solution: Custom Driver Architecture
|
||||
|
||||
```
|
||||
BaseConnection (Netmiko base class)
|
||||
↓
|
||||
CiscoBaseConnection (Cisco-style base class)
|
||||
↓
|
||||
HuaweiBase (Huawei device base class) ← Inherits VRP support
|
||||
↓
|
||||
GNS3HuaweiTelnetCE (Custom GNS3 driver) ← Overrides telnet_login only
|
||||
```mermaid
|
||||
graph TD
|
||||
A[BaseConnection] --> B[CiscoBaseConnection]
|
||||
B --> C[HuaweiBase<br/>VRP support, system-view,<br/>prompt patterns, paging]
|
||||
C --> D[GNS3HuaweiTelnetCE<br/>Overrides telnet_login only]
|
||||
D --> E[Skip authentication]
|
||||
D --> F[Auto-commit before exit]
|
||||
D --> G[y/n auto-confirm]
|
||||
```
|
||||
|
||||
**Why Inherit from HuaweiBase?**
|
||||
@ -258,20 +270,18 @@ custom_netmiko/
|
||||
|
||||
#### Limitations
|
||||
|
||||
**Authentication Requirement:**
|
||||
- The `gns3_huawei_telnet_ce` driver is designed for GNS3 devices **without authentication**
|
||||
- If your Huawei device has been configured with a username/password:
|
||||
- **Option 1**: Use the standard `huawei_telnet` driver (requires username/password)
|
||||
- **Option 2**: Remove authentication from the device for GNS3 testing
|
||||
- The driver does **not** currently auto-detect authentication requirements
|
||||
**Why Not Use the Standard `huawei_telnet` Driver?**
|
||||
- The standard Netmiko Huawei driver (`huawei_telnet`) has known issues in GNS3 emulation environments
|
||||
- This is the primary reason the custom `gns3_huawei_telnet_ce` driver was developed
|
||||
- The custom driver is designed for GNS3 devices **without authentication**
|
||||
- If your GNS3 Huawei device has been configured with authentication, remove it for GNS3 testing to use the custom driver
|
||||
|
||||
**When to Use Each Driver:**
|
||||
**When to Use:**
|
||||
|
||||
| Scenario | Use Driver | Requires Credentials? |
|
||||
|----------|-----------|----------------------|
|
||||
| GNS3 Huawei (fresh, no auth) | `gns3_huawei_telnet_ce` | ❌ No |
|
||||
| GNS3 Huawei (configured with username/password) | `huawei_telnet` | ✅ Yes |
|
||||
| Real Huawei hardware | `huawei_telnet` | ✅ Yes |
|
||||
| Scenario | Recommendation |
|
||||
|----------|---------------|
|
||||
| GNS3 Huawei device | Always use `gns3_huawei_telnet_ce` |
|
||||
| GNS3 device has auth configured | Remove auth, then use `gns3_huawei_telnet_ce` |
|
||||
|
||||
#### Method Overrides
|
||||
|
||||
@ -393,14 +403,14 @@ Standard Netmiko's `send_config_set()` waits for a prompt pattern, but the `[yes
|
||||
|
||||
### Solution: Hybrid Strategy with Interactive Prompt Handling
|
||||
|
||||
```
|
||||
BaseConnection (Netmiko base class)
|
||||
↓
|
||||
CiscoBaseConnection (Cisco-style base class)
|
||||
↓
|
||||
RuijieOSBase (Netmiko's Ruijie implementation)
|
||||
↓
|
||||
RuijieTelnetEnhanced (Custom GNS3 driver) ← Adds interactive prompt handling
|
||||
```mermaid
|
||||
graph TD
|
||||
A[BaseConnection] --> B[CiscoBaseConnection]
|
||||
B --> C[RuijieOSBase<br/>Netmiko built-in]
|
||||
C --> D[RuijieTelnetEnhanced<br/>Interactive prompt handling]
|
||||
D --> E[Preprocessing<br/>Insert yes after known commands]
|
||||
D --> F[Batch send<br/>Fast path 2-3s]
|
||||
D --> G[Fallback one-by-one<br/>Slow but reliable ~10s]
|
||||
```
|
||||
|
||||
**Why Hybrid Strategy?**
|
||||
@ -430,35 +440,22 @@ INTERACTIVE_PATTERNS = [
|
||||
```
|
||||
|
||||
**2. Hybrid Send Strategy**
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ Input Configuration Commands │
|
||||
└──────────────┬──────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────┐
|
||||
│ Step 1: Preprocessing │
|
||||
│ - Detect interactive commands │
|
||||
│ - Insert 'yes' after them │
|
||||
└──────────────┬──────────────────────┘
|
||||
↓
|
||||
┌─────────────────────────────────────┐
|
||||
│ Step 2: Try Batch Send (Fast) │
|
||||
│ - Write all commands rapidly │
|
||||
│ - Read output once │
|
||||
│ - last_read=2.0s (Netmiko standard) │
|
||||
└──────────────┬──────────────────────┘
|
||||
↓
|
||||
Success? ──Yes──→ Return output
|
||||
│
|
||||
No
|
||||
↓
|
||||
┌─────────────────────────────────────┐
|
||||
│ Step 3: Fallback (Slow but Reliable)│
|
||||
│ - Send commands one-by-one │
|
||||
│ - Detect prompts after each command │
|
||||
│ - Send 'yes' when needed │
|
||||
│ - last_read=0.5s per command │
|
||||
└─────────────────────────────────────┘
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Input Configuration Commands] --> B[Preprocessing]
|
||||
B --> B1[Detect interactive commands]
|
||||
B1 --> B2[Insert 'yes' after them]
|
||||
B2 --> C[Try Batch Send - Fast]
|
||||
C --> C1[Write all commands rapidly]
|
||||
C1 --> C2[Read output once, last_read=2.0s]
|
||||
C2 --> D{Success?}
|
||||
D -->|Yes| E[Return output]
|
||||
D -->|No| F[Fallback: One-by-One]
|
||||
F --> F1[Send each command individually]
|
||||
F1 --> F2[Detect prompts after each]
|
||||
F2 --> F3[Send 'yes' when needed, last_read=0.5s]
|
||||
F3 --> E
|
||||
```
|
||||
|
||||
**3. Batch Send Performance**
|
||||
@ -482,7 +479,6 @@ if re.search(r"\[yes/no\]", new_output, re.IGNORECASE):
|
||||
**For Ruijie devices in GNS3:**
|
||||
```
|
||||
device_type:gns3_ruijie_telnet
|
||||
platform:ruijie_os
|
||||
```
|
||||
|
||||
**Example Usage:**
|
||||
@ -514,38 +510,36 @@ with ConnectHandler(**device) as conn:
|
||||
- **Not Covered**: Unknown or vendor-specific interactive prompts
|
||||
- **Fallback**: If batch fails, falls back to one-by-one with real-time detection
|
||||
|
||||
**When to Use Each Driver:**
|
||||
**When to Use:**
|
||||
|
||||
| Scenario | Use Driver |
|
||||
|----------|------------|
|
||||
| GNS3 Ruijie (known commands) | `gns3_ruijie_telnet` (batch works) |
|
||||
| GNS3 Ruijie (unknown commands) | `gns3_ruijie_telnet` (auto-fallback) |
|
||||
| Real Ruijie hardware | `ruijie_os_telnet` (standard) |
|
||||
| GNS3 Ruijie (known interactive commands) | `gns3_ruijie_telnet` (batch + auto-fallback) |
|
||||
| GNS3 Ruijie (unknown interactive prompts) | `gns3_ruijie_telnet` (auto-fallback to one-by-one) |
|
||||
|
||||
## Dynamic Device Type Detection
|
||||
|
||||
### GNS3 Node Tags
|
||||
|
||||
Device type and platform are extracted from GNS3 node tags:
|
||||
Device type is extracted from GNS3 node tags:
|
||||
|
||||
```
|
||||
device_type:gns3_huawei_telnet_ce → Netmiko device type (precise)
|
||||
platform:huawei → Nornir platform (high-level)
|
||||
device_type:gns3_huawei_telnet_ce → Netmiko driver selection
|
||||
```
|
||||
|
||||
**Tag Examples:**
|
||||
|
||||
| Vendor | Device Type Tag | Platform Tag | Template Source |
|
||||
|--------|----------------|--------------|------------------|
|
||||
| Cisco IOS | `device_type:cisco_ios_telnet` | `platform:cisco_ios` | User appliance |
|
||||
| Huawei CE | `device_type:gns3_huawei_telnet_ce` | `platform:huawei` | User appliance |
|
||||
| Ruijie | `device_type:gns3_ruijie_telnet` | `platform:ruijie_os` | User appliance |
|
||||
| **VPCS** | `device_type:gns3_vpcs_telnet` | `platform:vpcs` | **Built-in ✅** |
|
||||
| Vendor | Device Type Tag | Template Source |
|
||||
|--------|----------------|-----------------|
|
||||
| Cisco IOS | `device_type:cisco_ios_telnet` | User appliance |
|
||||
| Huawei CE | `device_type:gns3_huawei_telnet_ce` | User appliance |
|
||||
| Ruijie | `device_type:gns3_ruijie_telnet` | User appliance |
|
||||
| **VPCS** | `device_type:gns3_vpcs_telnet` | **Built-in ✅** |
|
||||
|
||||
**VPCS Built-in Template:**
|
||||
- VPCS has a **built-in template** with pre-configured tags
|
||||
- Tags are **automatically applied** when creating VPCS nodes
|
||||
- No manual configuration required - works out of the box
|
||||
- VPCS has a **built-in template** with pre-configured `device_type` tag
|
||||
- The tag is **automatically applied** when creating VPCS nodes
|
||||
- No manual configuration required — works out of the box
|
||||
- Other devices require users to import appliances and configure tags manually
|
||||
|
||||
### Nornir Best Practice: Host-Level Connection Configuration
|
||||
@ -556,7 +550,6 @@ The system uses **Nornir's configuration priority** (host > group > defaults) to
|
||||
# From get_gns3_device_port.py
|
||||
hosts_data[device_name] = {
|
||||
"port": console_port,
|
||||
"platform": platform, # Reserved for future use (NAPALM, scrapli)
|
||||
"groups": ["network_devices"], # All devices share one group
|
||||
"connection_options": {
|
||||
"netmiko": {
|
||||
@ -570,305 +563,77 @@ hosts_data[device_name] = {
|
||||
- ✅ Each device has its own `device_type` (host-level config)
|
||||
- ✅ All devices share common settings via group inheritance (`hostname`, `timeout`)
|
||||
- ✅ No need to dynamically create multiple groups for each device type
|
||||
- ✅ Cleaner code structure - single generic group for all devices
|
||||
- ✅ Cleaner code structure — single generic group for all devices
|
||||
- ✅ Follows Nornir best practice: "configuration proximity"
|
||||
|
||||
**Configuration Priority:**
|
||||
```
|
||||
Host Level (connection_options.device_type)
|
||||
↓ OVERRIDES
|
||||
Group Level (hostname, timeout, username, password)
|
||||
↓ OVERRIDES
|
||||
Defaults Level (data.location)
|
||||
```mermaid
|
||||
graph TD
|
||||
A["Host Level<br/>connection_options.device_type<br/>(gns3_huawei_telnet_ce, cisco_ios_telnet, etc.)"]
|
||||
B["Group Level<br/>hostname=127.0.0.1, timeout=120<br/>username, password"]
|
||||
C["Defaults Level<br/>data.location=gns3"]
|
||||
A -->|overrides| B
|
||||
B -->|overrides| C
|
||||
```
|
||||
|
||||
**Before (Old Approach - Dynamic Groups):**
|
||||
```python
|
||||
# Had to create multiple groups dynamically
|
||||
groups = {
|
||||
"cisco_ios_telnet": {"device_type": "cisco_ios_telnet", ...},
|
||||
"huawei_telnet": {"device_type": "gns3_huawei_telnet_ce", ...},
|
||||
"juniper_junos": {"device_type": "juniper_junos_telnet", ...},
|
||||
}
|
||||
# Each host assigned to its vendor-specific group
|
||||
```
|
||||
|
||||
**After (Current Approach - Host-Level Config):**
|
||||
```python
|
||||
# Single group for shared settings
|
||||
groups = {
|
||||
"network_devices": {
|
||||
"hostname": "127.0.0.1",
|
||||
"timeout": 120,
|
||||
"username": "",
|
||||
"password": "",
|
||||
}
|
||||
}
|
||||
# Each host has device-specific connection_options
|
||||
# Host config overrides group config automatically
|
||||
```
|
||||
| Aspect | Old: Dynamic Groups | New: Host-Level Config |
|
||||
|--------|--------------------|-----------------------|
|
||||
| Group count | One per vendor type | Single `network_devices` group |
|
||||
| Device type location | Group `connection_options` | Host `connection_options` |
|
||||
| Complexity | Dynamic group creation logic | No dynamic group logic |
|
||||
|
||||
## Architecture Evolution
|
||||
|
||||
### Problem: Multi-Vendor Device Support
|
||||
|
||||
**Initial Challenge:**
|
||||
```
|
||||
Topology: Cisco R1 + Huawei SW1 + Juniper SRX
|
||||
↓
|
||||
Need: Different Netmiko drivers for each device
|
||||
↓
|
||||
Question: How to configure Nornir for multiple device types?
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Topology: Cisco R1 + Huawei SW1 + Juniper SRX] --> B[Need: Different Netmiko drivers per device]
|
||||
B --> C[Question: How to configure Nornir for multiple device types?]
|
||||
```
|
||||
|
||||
### Solution Evolution
|
||||
|
||||
#### ❌ Approach 1: Single Group with First Device's Type (Initial Implementation)
|
||||
| Approach | Strategy | Issue |
|
||||
|----------|----------|-------|
|
||||
| ❌ Approach 1 | Single group with first device's `device_type` | All devices use wrong driver |
|
||||
| ❌ Approach 2 | Dynamic groups per vendor (`huawei_telnet`, `cisco_telnet`, ...) | Complex logic, code duplication, not Nornir best practice |
|
||||
| ✅ Approach 3 (Current) | **Host-level `connection_options`** + single generic group | Clean, follows Nornir host > group > defaults priority |
|
||||
|
||||
```python
|
||||
# PROBLEM: Only uses first device's configuration
|
||||
def _initialize_nornir(hosts_data):
|
||||
first_device = next(iter(hosts_data.values()))
|
||||
device_type = first_device["device_type"] # Only one type!
|
||||
|
||||
return InitNornir(
|
||||
inventory={
|
||||
"options": {
|
||||
"hosts": hosts_data, # Has multiple device types
|
||||
"groups": {
|
||||
"network_devices": {
|
||||
"connection_options": {
|
||||
"netmiko": {"extras": {"device_type": device_type}}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
**Issue:** All devices use the first device's driver!
|
||||
- Cisco R1 → Uses Huawei driver (if Huawei is first) ❌
|
||||
- Huawei SW1 → Uses Cisco driver (if Cisco is first) ❌
|
||||
|
||||
#### ❌ Approach 2: Dynamic Groups (Intermediate Solution)
|
||||
|
||||
```python
|
||||
# COMPLEX: Create multiple groups dynamically
|
||||
groups = {}
|
||||
for host_data in hosts_data.values():
|
||||
device_type = host_data["device_type"]
|
||||
platform = host_data["platform"]
|
||||
group_name = f"{platform}_telnet" # e.g., "huawei_telnet"
|
||||
|
||||
if group_name not in groups:
|
||||
groups[group_name] = {
|
||||
"platform": platform,
|
||||
"connection_options": {
|
||||
"netmiko": {"extras": {"device_type": device_type}}
|
||||
}
|
||||
}
|
||||
|
||||
host_data["groups"] = [group_name]
|
||||
```
|
||||
|
||||
**Issues:**
|
||||
- Complex logic to detect and create groups
|
||||
- Code duplication in multiple files
|
||||
- Had to delete helper functions (`_get_nornir_groups_config`, `_get_nornir_group`)
|
||||
- Not following Nornir best practices
|
||||
|
||||
#### ✅ Approach 3: Host-Level Configuration (Current - Best Practice)
|
||||
|
||||
```python
|
||||
# SIMPLE: Single group + host-level device_type
|
||||
hosts_data[device_name] = {
|
||||
"port": console_port,
|
||||
"platform": platform, # Reserved for future use
|
||||
"groups": ["network_devices"], # All devices in one group
|
||||
"connection_options": { # Device-specific config
|
||||
"netmiko": {
|
||||
"extras": {"device_type": device_type}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# Single generic group for shared settings
|
||||
groups = {
|
||||
"network_devices": {
|
||||
"hostname": "127.0.0.1",
|
||||
"timeout": 120,
|
||||
"username": "",
|
||||
"password": "",
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Advantages:**
|
||||
- ✅ Clean, simple code
|
||||
- ✅ Follows Nornir best practice (host > group > defaults)
|
||||
- ✅ No dynamic group creation logic
|
||||
- ✅ Each device's `connection_options` overrides group settings automatically
|
||||
- ✅ Easy to extend with new device types
|
||||
> See `get_gns3_device_port.py:get_device_ports_from_topology()` for the current implementation.
|
||||
|
||||
### Configuration Priority Demonstration
|
||||
|
||||
```python
|
||||
# Host level (highest priority)
|
||||
host["connection_options"]["netmiko"]["extras"]["device_type"] = "gns3_huawei_telnet_ce"
|
||||
|
||||
↓ OVERRIDES
|
||||
|
||||
# Group level (middle priority)
|
||||
group["hostname"] = "127.0.0.1"
|
||||
group["timeout"] = 120
|
||||
|
||||
↓ OVERRIDES
|
||||
|
||||
# Defaults level (lowest priority)
|
||||
defaults["data"]["location"] = "gns3"
|
||||
```mermaid
|
||||
graph TD
|
||||
A["Host Level (highest)<br/>device_type: gns3_huawei_telnet_ce"]
|
||||
B["Group Level (middle)<br/>hostname: 127.0.0.1, timeout: 120"]
|
||||
C["Defaults Level (lowest)<br/>data.location: gns3"]
|
||||
A -->|overrides| B -->|overrides| C
|
||||
D[Result] --> D1["Each device uses its own device_type"]
|
||||
D --> D2["All share hostname, timeout from group"]
|
||||
D --> D3["All share data.location from defaults"]
|
||||
```
|
||||
|
||||
**Result:**
|
||||
- Each device uses its own `device_type` from host level
|
||||
- All devices share `hostname`, `timeout` from group level
|
||||
- All devices share `data.location` from defaults level
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Direct Netmiko Usage
|
||||
|
||||
**Huawei Device (Custom Driver):**
|
||||
```python
|
||||
from netmiko import ConnectHandler
|
||||
from gns3server.agent.gns3_copilot.utils import custom_netmiko
|
||||
**Custom driver auto-registers on import. No username/password needed for GNS3 devices.**
|
||||
|
||||
# Custom driver auto-registers on import
|
||||
device = {
|
||||
"device_type": "gns3_huawei_telnet_ce",
|
||||
"host": "127.0.0.1",
|
||||
"port": 5000,
|
||||
# No username/password needed!
|
||||
}
|
||||
|
||||
with ConnectHandler(**device) as conn:
|
||||
# Execute display command
|
||||
output = conn.send_command("display version")
|
||||
|
||||
# Execute configuration commands
|
||||
config = [
|
||||
"interface GE1/0/1",
|
||||
"description Uplink-to-Core",
|
||||
"undo shutdown"
|
||||
]
|
||||
output = conn.send_config_set(config)
|
||||
```
|
||||
|
||||
**Cisco IOS Device (Standard Driver):**
|
||||
```python
|
||||
from netmiko import ConnectHandler
|
||||
|
||||
device = {
|
||||
"device_type": "cisco_ios_telnet",
|
||||
"host": "127.0.0.1",
|
||||
"port": 5001,
|
||||
"username": "cisco",
|
||||
"password": "cisco",
|
||||
}
|
||||
|
||||
with ConnectHandler(**device) as conn:
|
||||
output = conn.send_command("show version")
|
||||
config = ["interface GigabitEthernet0/0", "description Test"]
|
||||
output = conn.send_config_set(config)
|
||||
```
|
||||
|
||||
### Nornir Multi-Vendor Automation
|
||||
|
||||
```python
|
||||
from nornir import InitNornir
|
||||
from gns3server.agent.gns3_copilot.utils import custom_netmiko
|
||||
|
||||
# Auto-register custom driver (happens automatically on import)
|
||||
from gns3server.agent.gns3_copilot.utils.custom_netmiko import huawei_ce
|
||||
huawei_ce.register_custom_device_type()
|
||||
|
||||
# Initialize Nornir with mixed-vendor inventory
|
||||
# Using host-level connection_options (best practice)
|
||||
inventory = {
|
||||
"plugin": "DictInventory",
|
||||
"options": {
|
||||
"hosts": {
|
||||
"huawei-sw1": {
|
||||
"hostname": "127.0.0.1",
|
||||
"port": 5001,
|
||||
"platform": "huawei", # Reserved for future use
|
||||
"groups": ["network_devices"],
|
||||
"connection_options": {
|
||||
"netmiko": {
|
||||
"extras": {"device_type": "gns3_huawei_telnet_ce"}
|
||||
}
|
||||
}
|
||||
},
|
||||
"cisco-r1": {
|
||||
"hostname": "127.0.0.1",
|
||||
"port": 5002,
|
||||
"platform": "cisco_ios",
|
||||
"groups": ["network_devices"],
|
||||
"connection_options": {
|
||||
"netmiko": {
|
||||
"extras": {"device_type": "cisco_ios_telnet"}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"groups": {
|
||||
"network_devices": {
|
||||
"hostname": "127.0.0.1", # Shared by all devices
|
||||
"timeout": 120,
|
||||
"username": "",
|
||||
"password": "",
|
||||
}
|
||||
},
|
||||
"defaults": {
|
||||
"data": {"location": "gns3"}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
nr = InitNornir(inventory=inventory)
|
||||
|
||||
# Execute commands on all devices (multi-vendor)
|
||||
result = nr.run(task=send_commands, commands=["display version"])
|
||||
|
||||
# Each device gets vendor-specific command handling
|
||||
# huawei-sw1 uses gns3_huawei_telnet_ce driver
|
||||
# cisco-r1 uses cisco_ios_telnet driver
|
||||
```
|
||||
| Device Type | `device_type` | Auth | Notes |
|
||||
|-------------|---------------|------|-------|
|
||||
| Huawei CE | `gns3_huawei_telnet_ce` | None | Custom driver, skip auth |
|
||||
| Ruijie | `gns3_ruijie_telnet` | None | Custom driver, interactive prompts handled |
|
||||
| VPCS | `gns3_vpcs_telnet` | None | Custom driver, no config mode |
|
||||
| Cisco IOS | `cisco_ios_telnet` | Required | Standard Netmiko driver |
|
||||
|
||||
### GNS3 Copilot Tool Usage
|
||||
|
||||
```python
|
||||
from gns3server.agent.gns3_copilot.tools_v2 import DisplayToolNornir
|
||||
> Tools: `tools_v2/display_tools_nornir.py`, `tools_v2/config_tools_nornir.py`, `tools_v2/vpcs_tools_netmiko.py`
|
||||
|
||||
tool = DisplayToolNornir()
|
||||
result = tool._run(json.dumps({
|
||||
"device_names": ["huawei-sw1", "cisco-r1"],
|
||||
"commands": ["display version", "show version"],
|
||||
"project_id": "project-uuid"
|
||||
}))
|
||||
|
||||
# Returns:
|
||||
# {
|
||||
# "huawei-sw1": {
|
||||
# "display version": "<Huawei output>",
|
||||
# "status": "success"
|
||||
# },
|
||||
# "cisco-r1": {
|
||||
# "show version": "<Cisco output>",
|
||||
# "status": "success"
|
||||
# }
|
||||
# }
|
||||
```
|
||||
Each tool reads node tags to determine `device_type`, builds host-level Nornir inventory, and dispatches commands to the appropriate Netmiko driver. Multi-vendor topologies are handled transparently — each device uses its own driver.
|
||||
|
||||
## Module Structure
|
||||
|
||||
@ -876,29 +641,24 @@ result = tool._run(json.dumps({
|
||||
gns3server/agent/gns3_copilot/
|
||||
├── utils/
|
||||
│ ├── custom_netmiko/ # Custom Netmiko drivers package
|
||||
│ │ ├── __init__.py # Package initialization
|
||||
│ │ ├── __init__.py # Auto-registers all drivers on import
|
||||
│ │ ├── huawei_ce.py # Huawei CloudEngine driver
|
||||
│ │ ├── ruijie_telnet.py # Ruijie enhanced driver
|
||||
│ │ ├── vpcs_telnet.py # VPCS simulator driver (NEW)
|
||||
│ │ ├── vpcs_telnet.py # VPCS simulator driver
|
||||
│ │ ├── README.md # Driver development guide
|
||||
│ │ └── tests/ # Unit tests
|
||||
│ │ └── tests/ # Unit tests (52 total)
|
||||
│ │ ├── __init__.py
|
||||
│ │ └── test_huawei_ce.py # Huawei CE driver tests
|
||||
│ └── get_gns3_device_port.py # Device port extraction with host-level config
|
||||
│ ├── _expand_multiline_commands() # Expand banner commands
|
||||
│ └── _error_handling() # device_type missing errors
|
||||
│ │ ├── test_huawei_ce.py # Huawei CE driver tests (9)
|
||||
│ │ ├── test_ruijie_telnet.py # Ruijie driver tests (15)
|
||||
│ │ └── test_vpcs_telnet.py # VPCS driver tests (28)
|
||||
│ └── get_gns3_device_port.py # Device port extraction
|
||||
│ └── get_device_ports_from_topology() # Extract ports + device_type from tags
|
||||
├── tools_v2/
|
||||
│ ├── display_tools_nornir.py # Multi-vendor display commands
|
||||
│ │ ├── _get_nornir_defaults() # Returns default Nornir config
|
||||
│ │ └── _initialize_nornir() # Single generic group + host-level device_type
|
||||
│ ├── config_tools_nornir.py # Multi-vendor config commands
|
||||
│ │ ├── _get_nornir_defaults() # Returns default Nornir config
|
||||
│ │ └── _initialize_nornir() # Single generic group + host-level device_type
|
||||
│ │ ├── _expand_multiline_commands() # Expand banner commands
|
||||
│ │ └── _error_handling() # device_type validation
|
||||
│ └── vpcs_tools_netmiko.py # VPCS commands using Nornir + Netmiko (NEW)
|
||||
│ ├── VPCSCommands # VPCS tool class
|
||||
│ └── _initialize_nornir() # VPCS device inventory setup
|
||||
│ │ └── _expand_multiline_commands() # Expand banner commands
|
||||
│ └── vpcs_tools_netmiko.py # VPCS commands using Nornir + Netmiko
|
||||
│ └── VPCSCommands # VPCS tool class
|
||||
```
|
||||
|
||||
**Key Architectural Changes (2026-03-14):**
|
||||
@ -915,80 +675,51 @@ gns3server/agent/gns3_copilot/
|
||||
- ✅ Updated: `get_gns3_device_port.py()` - Returns host-level `connection_options`
|
||||
- ✅ Added: `ruijie_telnet.py` - Custom Ruijie driver with interactive prompt handling
|
||||
- ✅ Added: `_expand_multiline_commands()` - Auto-expands banner and multi-line commands
|
||||
- ✅ Added: `_error_handling()` - Validates device_type tags, returns error if missing
|
||||
- ✅ Added: device_type tag validation with error feedback (inline in `_run`)
|
||||
|
||||
## Unit Testing
|
||||
|
||||
### Test Coverage
|
||||
|
||||
```python
|
||||
# test_netmiko_custom.py
|
||||
> Test files located at: `gns3server/agent/gns3_copilot/utils/custom_netmiko/tests/`
|
||||
|
||||
class TestGNS3HuaweiTelnetCEDriver(unittest.TestCase):
|
||||
def test_device_type_registered(self):
|
||||
"""Verify gns3_huawei_telnet_ce is in Netmiko CLASS_MAPPER"""
|
||||
from netmiko.ssh_dispatcher import CLASS_MAPPER
|
||||
self.assertIn("gns3_huawei_telnet_ce", CLASS_MAPPER)
|
||||
**Total: 52 tests across 3 test files**
|
||||
|
||||
def test_inheritance_from_huawei_base(self):
|
||||
"""Verify inherits from HuaweiBase"""
|
||||
from netmiko.huawei.huawei import HuaweiBase
|
||||
self.assertTrue(issubclass(GNS3HuaweiTelnetCE, HuaweiBase))
|
||||
| Test File | Tests | Coverage Focus |
|
||||
|-----------|-------|----------------|
|
||||
| `test_huawei_ce.py` | 9 | Registration, inheritance, VRP methods, mock connection |
|
||||
| `test_ruijie_telnet.py` | 15 | Registration, interactive patterns, preprocessing, batch/fallback |
|
||||
| `test_vpcs_telnet.py` | 28 | Registration, login, command sending, ANSI stripping, config mode |
|
||||
|
||||
def test_vrp_methods_available(self):
|
||||
"""Verify VRP-specific methods are available"""
|
||||
methods = ["config_mode", "check_config_mode", "exit_config_mode"]
|
||||
for method in methods:
|
||||
self.assertTrue(hasattr(GNS3HuaweiTelnetCE, method))
|
||||
```
|
||||
#### Huawei CE Tests (`TestHuaweiTelnetCEDriver` + `TestHuaweiTelnetCEIntegration`)
|
||||
- Device type registration in CLASS_MAPPER
|
||||
- Inheritance from HuaweiBase
|
||||
- VRP-specific methods (`config_mode`, `check_config_mode`, `exit_config_mode`)
|
||||
- Prompt pattern constants
|
||||
- Mock telnet connection
|
||||
|
||||
#### Ruijie Tests (`TestRuijieTelnetEnhancedDriver` + `TestRuijieTelnetEnhancedIntegration`)
|
||||
- Device type registration
|
||||
- Inheritance from RuijieOSBase
|
||||
- Interactive pattern matching (`router-id`, `erase`, `delete`, etc.)
|
||||
- Preprocessing of interactive commands
|
||||
- Batch and fallback send strategies
|
||||
|
||||
#### VPCS Tests (`TestVPCSTelnetDriver` + 5 more test classes)
|
||||
- Device type registration
|
||||
- Inheritance from BaseConnection
|
||||
- No-auth login with `PC\d+>` prompt
|
||||
- `send_command` / `send_command_timing` overrides
|
||||
- ANSI escape code stripping (8 dedicated tests)
|
||||
- Config mode always returns empty/false
|
||||
|
||||
**Running Tests:**
|
||||
```bash
|
||||
source venv/bin/activate
|
||||
python gns3server/agent/gns3_copilot/utils/custom_netmiko/tests/test_huawei_ce.py
|
||||
python -m pytest gns3server/agent/gns3_copilot/utils/custom_netmiko/tests/
|
||||
```
|
||||
|
||||
**Current Test Status:** ✅ All 9 tests passing
|
||||
|
||||
## Platform vs Device Type
|
||||
|
||||
### Key Concepts
|
||||
|
||||
**Platform (Nornir):**
|
||||
- High-level vendor identifier
|
||||
- **Reserved for future use** with plugins like NAPALM, scrapli
|
||||
- Used for metadata and logging
|
||||
- Examples: `huawei`, `cisco_ios`
|
||||
- ⚠️ **Not used by nornir_netmiko** (only `device_type` matters)
|
||||
|
||||
**Device Type (Netmiko):**
|
||||
- Precise driver type for Netmiko connection
|
||||
- Includes protocol information
|
||||
- **Actively used** to determine which Netmiko driver class to load
|
||||
- Examples: `gns3_huawei_telnet_ce`, `cisco_ios_telnet`
|
||||
|
||||
### Why Keep `platform` Field?
|
||||
|
||||
| Purpose | Plugin | Uses `platform`? |
|
||||
|---------|--------|------------------|
|
||||
| Connection driver | nornir_netmiko | ❌ No (uses `device_type`) |
|
||||
| Driver selection | NAPALM | ✅ Yes |
|
||||
| Driver selection | scrapli | ✅ Yes |
|
||||
| Metadata/Logging | General | ✅ Yes (future) |
|
||||
|
||||
**Conclusion:** The `platform` field is kept for:
|
||||
1. **Future plugin support** (NAPALM, scrapli)
|
||||
2. **Debugging and logging** (vendor identification)
|
||||
3. **Data completeness** (industry standard practice)
|
||||
|
||||
### Mapping
|
||||
|
||||
| Platform | Device Type | Netmiko Usage | Notes |
|
||||
|----------|-------------|---------------|-------|
|
||||
| `huawei` | `gns3_huawei_telnet_ce` | ✅ Active | Custom driver for GNS3 |
|
||||
| `cisco_ios` | `cisco_ios_telnet` | ✅ Active | Standard Netmiko driver |
|
||||
|
||||
**Important:** For nornir_netmiko, only `device_type` in `connection_options` matters. The `platform` field is informational only.
|
||||
**Current Test Status:** ✅ All 52 tests passing
|
||||
|
||||
## Related Documentation
|
||||
|
||||
@ -1007,13 +738,13 @@ python gns3server/agent/gns3_copilot/utils/custom_netmiko/tests/test_huawei_ce.p
|
||||
|
||||
_Implementation Date: 2026-03-12_
|
||||
|
||||
_Last Updated: 2026-03-14 (Added VPCS driver and unified tool architecture)_
|
||||
_Last Updated: 2026-04-20 (Documentation review: fixed VPCS template tags, updated test coverage, added Mermaid diagrams, removed code bloat)_
|
||||
|
||||
_Status: ✅ Implemented - Custom drivers for Huawei, Ruijie, and VPCS; multi-vendor support with Cisco IOS, Huawei, Ruijie, and VPCS tested_
|
||||
|
||||
_Architecture: Nornir best practice - host-level connection_options with single generic group_
|
||||
|
||||
_Unit Tests: ✅ 9/9 passing_
|
||||
_Unit Tests: ✅ 52/52 passing (9 Huawei + 15 Ruijie + 28 VPCS)_
|
||||
|
||||
_Changelog:_
|
||||
- **2026-03-14**: Added VPCS support and unified tool architecture
|
||||
@ -1038,5 +769,4 @@ _Changelog:_
|
||||
- Removed `_get_nornir_groups_config()` and `_get_nornir_group()` helper functions
|
||||
- Simplified `_initialize_nornir()` to use single generic group
|
||||
- Updated `get_gns3_device_port.py()` to return host-level configuration
|
||||
- Reserved `platform` field for future NAPALM/scrapli plugin support
|
||||
- **2026-03-12**: Initial implementation with custom Huawei driver
|
||||
|
||||
@ -1,3 +1,11 @@
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
|
||||
|
||||
|
||||
# Node and Topology Management Tools
|
||||
|
||||
## Overview
|
||||
@ -80,7 +88,8 @@ The following built-in utility templates are excluded as they are not actual net
|
||||
{
|
||||
"template_id": "uuid-of-template",
|
||||
"x": 100,
|
||||
"y": -200
|
||||
"y": -200,
|
||||
"name": "R1"
|
||||
},
|
||||
{
|
||||
"template_id": "uuid-of-template2",
|
||||
@ -98,7 +107,7 @@ The following built-in utility templates are excluded as they are not actual net
|
||||
"created_nodes": [
|
||||
{
|
||||
"node_id": "uuid-of-node1",
|
||||
"name": "NodeName1",
|
||||
"name": "R1",
|
||||
"status": "success"
|
||||
},
|
||||
{
|
||||
@ -117,6 +126,7 @@ The following built-in utility templates are excluded as they are not actual net
|
||||
- Batch create multiple nodes
|
||||
- Uses templates for consistent node configuration
|
||||
- X/Y coordinate positioning for topology layout
|
||||
- Optional `name` field to set node name directly (if omitted, GNS3 assigns default name)
|
||||
- **Important**: Ensure distance between any two nodes is greater than 250px for clear interface labels
|
||||
|
||||
**Use Cases:**
|
||||
@ -307,6 +317,58 @@ The tool automatically detects node types and calculates optimal wait times:
|
||||
- Logs device types and selected wait strategy
|
||||
- Progress bar displays calculated wait time
|
||||
|
||||
### GNS3StartNodeQuickTool 🆕
|
||||
|
||||
**Tool Name:** `start_gns3_node_quick`
|
||||
|
||||
**Description:** Starts one or multiple nodes in a GNS3 project WITHOUT waiting for startup completion. Suitable for automated deployment workflows where long waits would cause HTTP timeouts.
|
||||
|
||||
**Input:**
|
||||
```json
|
||||
{
|
||||
"project_id": "uuid-of-project",
|
||||
"node_ids": ["uuid-of-node-1", "uuid-of-node-2"]
|
||||
}
|
||||
```
|
||||
|
||||
**Output:**
|
||||
```json
|
||||
{
|
||||
"project_id": "...",
|
||||
"total_nodes": 2,
|
||||
"successful": 2,
|
||||
"failed": 0,
|
||||
"nodes": [
|
||||
{"node_id": "...", "name": "...", "status": "started"},
|
||||
{"node_id": "...", "name": "...", "status": "started"}
|
||||
],
|
||||
"note": "Start commands sent. Nodes are booting in background. Check node status later."
|
||||
}
|
||||
```
|
||||
|
||||
**Features:**
|
||||
- Batch start multiple nodes
|
||||
- No progress bar or wait time
|
||||
- Immediate API response
|
||||
- Nodes boot in background after tool returns
|
||||
- Comprehensive error handling
|
||||
|
||||
**When to Use Quick vs Regular Start:**
|
||||
|
||||
| Scenario | Use `start_gns3_node` | Use `start_gns3_node_quick` |
|
||||
|----------|----------------------|-----------------------------|
|
||||
| Interactive lab startup | ✅ | ❌ |
|
||||
| CI/CD pipeline | ❌ | ✅ |
|
||||
| HTTP timeout risk | ❌ | ✅ |
|
||||
| Need verified status | ✅ | ❌ |
|
||||
| Automated bulk deployment | ❌ | ✅ |
|
||||
|
||||
**Implementation Details:**
|
||||
- Calls `POST /projects/{project_id}/nodes/{node_id}/start` for each node
|
||||
- Returns immediately after sending start commands (no progress bar)
|
||||
- Status reflects command send result, not boot completion
|
||||
- Source: `gns3_start_node.py` (same file as `GNS3StartNodeTool`)
|
||||
|
||||
### GNS3StopNodeTool
|
||||
|
||||
**Tool Name:** `stop_gns3_node`
|
||||
@ -444,9 +506,12 @@ gns3server/agent/gns3_copilot/tools_v2/
|
||||
├── gns3_create_link.py # Link creation tool 🆕
|
||||
├── gns3_get_node_temp.py # Template retrieval tool 🆕
|
||||
├── gns3_update_node_name.py # Node rename tool 🆕
|
||||
├── gns3_start_node.py # Start node tool
|
||||
├── gns3_start_node.py # Start node tool (+ GNS3StartNodeQuickTool)
|
||||
├── gns3_stop_node.py # Stop node tool
|
||||
└── gns3_suspend_node.py # Suspend node tool
|
||||
├── gns3_suspend_node.py # Suspend node tool
|
||||
├── config_tools_nornir.py # Configuration command execution
|
||||
├── display_tools_nornir.py # Display command execution
|
||||
└── packet_capture_tools.py # Packet capture analysis
|
||||
```
|
||||
|
||||
### API Integration
|
||||
@ -478,44 +543,40 @@ node.start() # or node.stop() / node.suspend()
|
||||
|
||||
### Progress Tracking
|
||||
|
||||
**GNS3StartNodeTool** includes visual progress bar:
|
||||
**GNS3StartNodeTool** includes visual progress bar with dynamic wait time:
|
||||
|
||||
```
|
||||
Starting 3 node(s), please wait...
|
||||
[===========> ] 35.0%
|
||||
```
|
||||
|
||||
**Progress Calculation:**
|
||||
- Base duration: 140 seconds
|
||||
- Extra duration: 10 seconds per additional node
|
||||
- Formula: `total_duration = 140 + max(0, node_count - 1) * 10`
|
||||
Wait time is calculated by `calculate_startup_time()` based on node types (see Dynamic Wait Time Strategy above). No hardcoded duration.
|
||||
|
||||
**GNS3StartNodeQuickTool** does NOT include progress tracking:
|
||||
- Returns immediately after sending start commands
|
||||
- Nodes boot in background
|
||||
|
||||
**GNS3StopNodeTool** does not include progress tracking:
|
||||
- Stop operations are typically fast (< 5 seconds)
|
||||
- Immediate API response provides status feedback
|
||||
- No need for progress indication
|
||||
|
||||
**GNS3SuspendNodeTool** does not include progress tracking:
|
||||
- Suspend operations are typically fast (< 10 seconds)
|
||||
- Immediate API response provides status feedback
|
||||
- No need for progress indication
|
||||
|
||||
## Node State Transitions
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> Stopped
|
||||
Stopped --> Started : start()
|
||||
Started --> Stopped : stop()
|
||||
Started --> Suspended : suspend()
|
||||
Suspended --> Started : start() [resume]
|
||||
Suspended --> Stopped : stop()
|
||||
```
|
||||
┌─────────┐
|
||||
│ Stopped │◀─────── stop()
|
||||
└────┬────┘
|
||||
│
|
||||
│ start()
|
||||
▼
|
||||
┌──────────┐
|
||||
│ Started │───suspend()───▶ Suspended
|
||||
└──────────┘ │
|
||||
▲ │ │
|
||||
│ │ stop() │ resume()
|
||||
└──────┴──────────────────────────┘
|
||||
```
|
||||
|
||||
> **Note:** Resuming a suspended node uses `start()` — there is no separate `resume()` method.
|
||||
|
||||
**State Change Allowed Operations:**
|
||||
|
||||
@ -538,11 +599,15 @@ Note: These special node types are filtered out by GNS3TemplateTool and won't ap
|
||||
- `GNS3LinkTool` - Create links between nodes 🆕
|
||||
- `GNS3UpdateNodeNameTool` - Rename nodes 🆕
|
||||
- `GNS3StartNodeTool` - For diagnostics requiring started nodes
|
||||
- `ExecuteMultipleDeviceCommands` - Execute show/display/debug commands (READ-ONLY)
|
||||
- `PacketCaptureTool` - Analyze packets from active capture
|
||||
- `DeviceSkillsTool` - Get device-specific skills and command knowledge
|
||||
|
||||
**Capabilities:**
|
||||
- READ-ONLY diagnostic tools
|
||||
- Can create and manage topology (nodes, links, names)
|
||||
- Cannot stop or suspend nodes (prevents disruption of active labs)
|
||||
- Cannot execute configuration commands
|
||||
|
||||
### Lab Automation Assistant Mode
|
||||
|
||||
@ -554,10 +619,16 @@ Note: These special node types are filtered out by GNS3TemplateTool and won't ap
|
||||
- `GNS3StartNodeTool` - Full lab deployment
|
||||
- `GNS3StopNodeTool` - Full lab shutdown
|
||||
- `GNS3SuspendNodeTool` - Lab pause with state preservation
|
||||
- `ExecuteMultipleDeviceCommands` - Execute show/display/debug commands (READ-ONLY)
|
||||
- `ExecuteMultipleDeviceConfigCommands` - Execute configuration commands
|
||||
- `VPCSCommands` - Execute VPCS commands using Netmiko
|
||||
- `PacketCaptureTool` - Analyze packets from active capture
|
||||
- `DeviceSkillsTool` - Get device-specific skills and command knowledge
|
||||
|
||||
**Capabilities:**
|
||||
- Full diagnostic and configuration tools
|
||||
- Complete topology and lifecycle management (create, connect, start/stop/suspend)
|
||||
- Device configuration via Nornir/Netmiko
|
||||
- Automated workflows with state preservation
|
||||
- Lab snapshot capabilities for later resumption
|
||||
|
||||
@ -898,7 +969,9 @@ logger.info("Suspend command sent for node %s (%s)", node_id, node.name)
|
||||
| Create Node | < 1s per node | 0s | No | N/A |
|
||||
| Create Link | < 1s per link | 0s | No | N/A |
|
||||
| Update Name | < 1s per node | 0s | No | N/A |
|
||||
| Start | 60-180s | ~140s base | Yes | N/A |
|
||||
| Start (VPCS/IOU) | 15-37s | Dynamic | Yes | N/A |
|
||||
| Start (QEMU/etc.) | 120-160s+ | Dynamic | Yes | N/A |
|
||||
| Start Quick | < 2s | 0s | No | N/A |
|
||||
| Stop | < 5s | 0s | No | ❌ No |
|
||||
| Suspend | < 10s | 0s | No | ✅ Yes |
|
||||
|
||||
@ -911,28 +984,6 @@ logger.info("Suspend command sent for node %s (%s)", node_id, node.name)
|
||||
- Create node/link operations are fast and require no waiting
|
||||
- Template retrieval is instant with no parameters needed
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Planned Features
|
||||
|
||||
- [ ] **Quick Start Tool**: Start nodes without waiting for completion (for CI/CD)
|
||||
- [ ] **Delete Node Tool**: Remove nodes from topology
|
||||
- [ ] **Delete Link Tool**: Remove links from topology
|
||||
- [ ] **Resume Tool**: Explicit resume operation for suspended nodes
|
||||
- [ ] **Restart Tool**: Combined stop + start operation
|
||||
- [ ] **Bulk Status Check**: Query multiple nodes without stopping
|
||||
- [ ] **Conditional Stop/Suspend**: Operate only if node is in specific state
|
||||
- [ ] **Graceful Shutdown**: Send halt commands before stopping
|
||||
|
||||
### Potential Improvements
|
||||
|
||||
- [ ] Auto-layout calculation (optimal node positioning)
|
||||
- [ ] Progress tracking for long suspend operations (rare but possible)
|
||||
- [ ] Concurrent create/link operations (parallel API calls)
|
||||
- [ ] Suspend node groups by name pattern
|
||||
- [ ] Dependency-aware suspend (suspend in dependency order)
|
||||
- [ ] Auto-suspend after idle timeout
|
||||
- [ ] State snapshots (save multiple suspend states)
|
||||
|
||||
## Related Documentation
|
||||
|
||||
|
||||
@ -1,3 +1,11 @@
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
|
||||
|
||||
|
||||
|
||||
# Netmiko Supported Devices
|
||||
|
||||
**Netmiko Version:** 4.6.0
|
||||
|
||||
@ -1,3 +1,11 @@
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
|
||||
|
||||
|
||||
# Template-Based System with HITL - Future Roadmap
|
||||
|
||||
**Status:** 💡 Proposed
|
||||
|
||||
@ -1,3 +1,7 @@
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
|
||||
@ -1,3 +1,7 @@
|
||||
<!--
|
||||
SPDX-License-Identifier: CC-BY-SA-4.0
|
||||
See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
|
||||
@ -94,6 +94,8 @@ from gns3server.agent.gns3_copilot.tools_v2 import GNS3SuspendNodeTool
|
||||
from gns3server.agent.gns3_copilot.tools_v2 import GNS3TemplateTool
|
||||
from gns3server.agent.gns3_copilot.tools_v2 import GNS3UpdateNodeNameTool
|
||||
from gns3server.agent.gns3_copilot.tools_v2.vpcs_tools_netmiko import VPCSCommands
|
||||
from gns3server.agent.gns3_copilot.tools_v2 import PacketCaptureTool
|
||||
from gns3server.agent.gns3_copilot.skills import DeviceSkillsTool
|
||||
|
||||
# Set up logger for GNS3-Copilot
|
||||
logger = logging.getLogger(__name__)
|
||||
@ -112,6 +114,8 @@ TEACHING_ASSISTANT_MODE_TOOLS = [
|
||||
GNS3UpdateNodeNameTool(), # Update node name
|
||||
ExecuteMultipleDeviceCommands(), # Execute show/display/debug commands
|
||||
# (READ-ONLY)
|
||||
PacketCaptureTool(), # Analyze packets from active capture
|
||||
DeviceSkillsTool(), # Get device-specific skills and command knowledge
|
||||
]
|
||||
|
||||
# Lab automation assistant mode: Full diagnostic AND configuration tools
|
||||
@ -127,6 +131,8 @@ LAB_AUTOMATION_ASSISTANT_MODE_TOOLS = [
|
||||
# (READ-ONLY)
|
||||
ExecuteMultipleDeviceConfigCommands(), # Execute configuration commands
|
||||
VPCSCommands(), # Execute VPCS commands using Netmiko
|
||||
PacketCaptureTool(), # Analyze packets from active capture
|
||||
DeviceSkillsTool(), # Get device-specific skills and command knowledge
|
||||
]
|
||||
|
||||
# Default tools (legacy support - will be overridden by mode-specific tools)
|
||||
@ -146,6 +152,45 @@ DEFAULT_CONVERSATION_TITLE = "New Conversation"
|
||||
UNTITLED_SESSION_FALLBACK = "Untitled Session"
|
||||
TITLE_MAX_LENGTH = 40
|
||||
|
||||
# Abort flags storage for session-level abort tracking
|
||||
# This is checked by conditional edge functions to stop the graph
|
||||
_abort_flags: dict[str, bool] = {}
|
||||
|
||||
|
||||
def check_abort_flag(session_id: str) -> bool:
|
||||
"""
|
||||
Check if abort flag is set for a session.
|
||||
|
||||
Args:
|
||||
session_id: Session identifier
|
||||
|
||||
Returns:
|
||||
True if abort is requested, False otherwise
|
||||
"""
|
||||
return _abort_flags.get(session_id, False)
|
||||
|
||||
|
||||
def set_abort_flag(session_id: str):
|
||||
"""
|
||||
Set abort flag for a session.
|
||||
|
||||
Args:
|
||||
session_id: Session identifier
|
||||
"""
|
||||
_abort_flags[session_id] = True
|
||||
logger.debug("Abort flag set for session: %s", session_id)
|
||||
|
||||
|
||||
def clear_abort_flag(session_id: str):
|
||||
"""
|
||||
Clear abort flag for a session.
|
||||
|
||||
Args:
|
||||
session_id: Session identifier
|
||||
"""
|
||||
_abort_flags[session_id] = False
|
||||
logger.debug("Abort flag cleared for session: %s", session_id)
|
||||
|
||||
|
||||
# Define state
|
||||
class MessagesState(TypedDict):
|
||||
@ -165,6 +210,8 @@ class MessagesState(TypedDict):
|
||||
conversation_title: Optional conversation title for session
|
||||
identification and management
|
||||
topology_info: Dictionary containing GNS3 project topology information
|
||||
session_id: Session identifier for abort tracking
|
||||
abort: Flag to signal abort request
|
||||
"""
|
||||
|
||||
messages: Annotated[list[AnyMessage], operator.add]
|
||||
@ -179,6 +226,12 @@ class MessagesState(TypedDict):
|
||||
# Store GNS3 topology information
|
||||
topology_info: dict | None
|
||||
|
||||
# Session identifier for abort tracking
|
||||
session_id: str | None
|
||||
|
||||
# Abort flag to signal stop request
|
||||
abort: bool
|
||||
|
||||
|
||||
# Define llm call node
|
||||
def llm_call(state: dict, config: RunnableConfig | None = None):
|
||||
@ -334,6 +387,7 @@ def llm_call(state: dict, config: RunnableConfig | None = None):
|
||||
"messages": [response],
|
||||
"llm_calls": state.get("llm_calls", 0) + 1,
|
||||
"topology_info": topology_info,
|
||||
"session_id": state.get("session_id"),
|
||||
}
|
||||
|
||||
|
||||
@ -352,7 +406,7 @@ def generate_title(
|
||||
|
||||
if not llm_config:
|
||||
logger.error("LLM config not found in context, cannot generate title")
|
||||
return {"conversation_title": UNTITLED_SESSION_FALLBACK}
|
||||
return {"conversation_title": UNTITLED_SESSION_FALLBACK, "session_id": state.get("session_id")}
|
||||
|
||||
# Only generate a title if it hasn't been set yet
|
||||
current_title = state.get("conversation_title")
|
||||
@ -403,7 +457,7 @@ def generate_title(
|
||||
)
|
||||
|
||||
logger.info("Generated new title: %s", new_title)
|
||||
return {"conversation_title": new_title}
|
||||
return {"conversation_title": new_title, "session_id": state.get("session_id")}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Title generation failed: {e}, using fallback")
|
||||
@ -428,14 +482,14 @@ def generate_title(
|
||||
"Using fallback title from user message: '%s'",
|
||||
fallback_title,
|
||||
)
|
||||
return {"conversation_title": fallback_title}
|
||||
return {"conversation_title": fallback_title, "session_id": state.get("session_id")}
|
||||
|
||||
# Final fallback
|
||||
logger.info(
|
||||
"Using final fallback title: '%s'",
|
||||
UNTITLED_SESSION_FALLBACK,
|
||||
)
|
||||
return {"conversation_title": UNTITLED_SESSION_FALLBACK}
|
||||
return {"conversation_title": UNTITLED_SESSION_FALLBACK, "session_id": state.get("session_id")}
|
||||
|
||||
# Title already exists → no update needed
|
||||
return {}
|
||||
@ -481,21 +535,63 @@ def tool_node(state: dict, config: RunnableConfig | None = None):
|
||||
)
|
||||
result.append(tool_msg)
|
||||
|
||||
return {"messages": result}
|
||||
return {"messages": result, "session_id": state.get("session_id")}
|
||||
|
||||
|
||||
# Abort handler node - provides tool results when aborting
|
||||
def abort_handler_node(state: dict) -> dict:
|
||||
"""
|
||||
Handles abort by providing tool result messages for pending tool_calls.
|
||||
This ensures message history consistency and prevents checkpoint corruption.
|
||||
"""
|
||||
messages = state.get("messages", [])
|
||||
if not messages:
|
||||
return {"session_id": state.get("session_id")}
|
||||
|
||||
last_message = messages[-1]
|
||||
if not hasattr(last_message, "tool_calls") or not last_message.tool_calls:
|
||||
return {"session_id": state.get("session_id")}
|
||||
|
||||
result = []
|
||||
for tool_call in last_message.tool_calls:
|
||||
tool_msg = ToolMessage(
|
||||
content=json.dumps({
|
||||
"status": "aborted",
|
||||
"message": "Tool execution was aborted by user",
|
||||
"tool_call_id": tool_call["id"],
|
||||
}),
|
||||
tool_call_id=tool_call["id"],
|
||||
name=tool_call["name"],
|
||||
metadata={"created_at": datetime.utcnow().isoformat(), "aborted": True},
|
||||
)
|
||||
result.append(tool_msg)
|
||||
|
||||
logger.info("Abort handler: generated %d aborted tool messages", len(result))
|
||||
return {"messages": result, "session_id": state.get("session_id")}
|
||||
|
||||
|
||||
# Routing logic after the LLM node
|
||||
def should_continue(
|
||||
state: MessagesState,
|
||||
) -> Literal["tool_node", "title_generator_node", END]:
|
||||
) -> Literal["tool_node", "title_generator_node", "abort_handler_node", END]:
|
||||
"""
|
||||
Determine the next step after the LLM has produced a response.
|
||||
|
||||
- If abort flag is set → end the conversation
|
||||
- If the LLM requested any tool calls → route to tool_node
|
||||
- If this is the first complete turn (llm_calls == 1) and no title
|
||||
exists → generate a title
|
||||
- Otherwise → conversation is complete, go to END
|
||||
"""
|
||||
# Check abort flag first
|
||||
session_id = state.get("session_id")
|
||||
if session_id and check_abort_flag(session_id):
|
||||
last_message = state["messages"][-1]
|
||||
# If there's a tool_calls message, we need to handle it properly
|
||||
if hasattr(last_message, "tool_calls") and last_message.tool_calls:
|
||||
return "abort_handler_node"
|
||||
return END
|
||||
|
||||
last_message = state["messages"][-1]
|
||||
current_title = state.get("conversation_title")
|
||||
|
||||
@ -526,9 +622,15 @@ def recursion_limit_continue(state: MessagesState) -> Literal["llm_call", END]:
|
||||
"llm_call" to continue processing, END to terminate conversation
|
||||
|
||||
Logic:
|
||||
- If abort flag is set → end the conversation
|
||||
- If the last message is ToolMessage and steps >= 4: continue to LLM
|
||||
- Otherwise: end the conversation to prevent infinite loops
|
||||
"""
|
||||
# Check abort flag first
|
||||
session_id = state.get("session_id")
|
||||
if session_id and check_abort_flag(session_id):
|
||||
return END
|
||||
|
||||
last_message = state["messages"][-1]
|
||||
if isinstance(last_message, ToolMessage):
|
||||
if state["remaining_steps"] < 4:
|
||||
@ -546,6 +648,7 @@ agent_builder = StateGraph(MessagesState)
|
||||
agent_builder.add_node("llm_call", llm_call)
|
||||
agent_builder.add_node("tool_node", tool_node)
|
||||
agent_builder.add_node("title_generator_node", generate_title)
|
||||
agent_builder.add_node("abort_handler_node", abort_handler_node)
|
||||
|
||||
# Add edges to connect nodes
|
||||
agent_builder.add_edge(START, "llm_call")
|
||||
@ -560,6 +663,7 @@ agent_builder.add_conditional_edges(
|
||||
# tools
|
||||
"title_generator_node": "title_generator_node", # Generate title on
|
||||
# first interaction
|
||||
"abort_handler_node": "abort_handler_node", # Handle abort with tool calls
|
||||
END: END, # End conversation if no tools needed
|
||||
},
|
||||
)
|
||||
@ -576,3 +680,4 @@ agent_builder.add_conditional_edges(
|
||||
)
|
||||
|
||||
agent_builder.add_edge("title_generator_node", END)
|
||||
agent_builder.add_edge("abort_handler_node", END)
|
||||
|
||||
@ -91,6 +91,20 @@ class AgentService:
|
||||
self._init_lock = asyncio.Lock()
|
||||
self._initialized = False
|
||||
|
||||
def abort_session(self, session_id: str):
|
||||
"""
|
||||
Signal abort for a session.
|
||||
|
||||
Args:
|
||||
session_id: Session identifier
|
||||
"""
|
||||
from gns3server.agent.gns3_copilot.agent.gns3_copilot import (
|
||||
set_abort_flag,
|
||||
)
|
||||
|
||||
set_abort_flag(session_id)
|
||||
log.info("Abort signal set for session: %s", session_id)
|
||||
|
||||
def _get_checkpoint_dir(self) -> str:
|
||||
"""Get or create the checkpoint directory for this project."""
|
||||
checkpoint_dir = os.path.join(self.project_path, "gns3-copilot")
|
||||
@ -313,9 +327,18 @@ class AgentService:
|
||||
],
|
||||
"llm_calls": 0,
|
||||
"remaining_steps": 20,
|
||||
"mode": mode,
|
||||
"session_id": session_id,
|
||||
"abort": False,
|
||||
}
|
||||
|
||||
# Clear abort flag for this session at the start of stream
|
||||
from gns3server.agent.gns3_copilot.agent.gns3_copilot import (
|
||||
clear_abort_flag,
|
||||
)
|
||||
|
||||
clear_abort_flag(session_id)
|
||||
log.debug("Abort flag cleared for session: %s", session_id)
|
||||
|
||||
# Get the compiled graph
|
||||
graph = await self._get_graph()
|
||||
log.debug("LangGraph graph obtained, starting stream")
|
||||
@ -335,6 +358,9 @@ class AgentService:
|
||||
# tool call arguments
|
||||
tool_call_accumulator = ToolCallStreamAccumulator()
|
||||
|
||||
# Track if stream was aborted
|
||||
stream_aborted = False
|
||||
|
||||
# Stream events
|
||||
try:
|
||||
async for event in graph.astream_events(
|
||||
@ -459,6 +485,26 @@ class AgentService:
|
||||
log.debug("Yielding chunk: type=%s", chunk.get("type"))
|
||||
yield chunk
|
||||
|
||||
# Check if stream was aborted and yield tool_end events for aborted tools
|
||||
from gns3server.agent.gns3_copilot.agent.gns3_copilot import (
|
||||
check_abort_flag,
|
||||
)
|
||||
|
||||
if check_abort_flag(session_id):
|
||||
log.info("Stream aborted, yielding tool_end events: session_id=%s", session_id)
|
||||
# Get final state to find aborted tool messages
|
||||
final_state = await graph.aget_state(config)
|
||||
if final_state and "messages" in final_state.values:
|
||||
for msg in final_state.values["messages"]:
|
||||
# Check if this is an aborted tool message
|
||||
if hasattr(msg, "metadata") and msg.metadata.get("aborted"):
|
||||
yield {
|
||||
"type": "tool_end",
|
||||
"tool_name": getattr(msg, "name", "unknown"),
|
||||
"tool_output": msg.content if hasattr(msg, "content") else "",
|
||||
"session_id": session_id,
|
||||
}
|
||||
|
||||
# Update session statistics after successful stream
|
||||
await repo.update_session(
|
||||
thread_id=session_id,
|
||||
@ -688,11 +734,20 @@ class AgentService:
|
||||
async with self._init_lock:
|
||||
if self._checkpointer_conn:
|
||||
try:
|
||||
await self._checkpointer_conn.close()
|
||||
# Add timeout to prevent blocking on database close
|
||||
await asyncio.wait_for(
|
||||
self._checkpointer_conn.close(),
|
||||
timeout=5.0
|
||||
)
|
||||
log.debug(
|
||||
"Checkpointer connection closed for: %s",
|
||||
self.project_path,
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
log.warning(
|
||||
"Checkpointer connection close timeout for: %s (forcing cleanup)",
|
||||
self.project_path,
|
||||
)
|
||||
except Exception as e:
|
||||
log.warning("Error closing checkpointer connection: %s", e)
|
||||
finally:
|
||||
|
||||
@ -55,7 +55,6 @@ tracert
|
||||
|
||||
# Ping flood variants
|
||||
ping -f
|
||||
ping -f
|
||||
|
||||
# Debug commands that may destabilize devices
|
||||
debug
|
||||
|
||||
@ -2192,12 +2192,12 @@ class Project:
|
||||
|
||||
def links_summary(
|
||||
self, is_print: bool = True
|
||||
) -> list[tuple[str, str, str, str]] | None:
|
||||
) -> list[dict[str, str]] | None:
|
||||
"""
|
||||
Returns a summary of the links insode the project. If `is_print` is False, it
|
||||
will return a list of tuples like:
|
||||
Returns a summary of the links inside the project. If `is_print` is False,
|
||||
it will return a list of dicts like:
|
||||
|
||||
`[(node_a, port_a, node_b, port_b) ...]`
|
||||
`[{"link_id": "xxx", "node_a": "R1", "port_a": "Eth0/0", "node_b": "R2", "port_b": "Eth0/0"}, ...]`
|
||||
|
||||
**Required Attributes:**
|
||||
|
||||
@ -2213,7 +2213,7 @@ class Project:
|
||||
assert self.links is not None, "Links must be loaded"
|
||||
assert self.nodes is not None, "Nodes must be loaded"
|
||||
|
||||
_links_summary: list[tuple[str, str, str, str]] = []
|
||||
_links_summary: list[dict[str, str]] = []
|
||||
|
||||
for _l in self.links:
|
||||
if not _l.nodes:
|
||||
@ -2258,7 +2258,13 @@ class Project:
|
||||
if is_print:
|
||||
print(f"{endpoint_a} ---- {endpoint_b}")
|
||||
|
||||
_links_summary.append((name_a, _port_a, name_b, _port_b))
|
||||
_links_summary.append({
|
||||
"link_id": _l.link_id,
|
||||
"node_a": name_a,
|
||||
"port_a": _port_a,
|
||||
"node_b": name_b,
|
||||
"port_b": _port_b
|
||||
})
|
||||
|
||||
except (StopIteration, KeyError, AttributeError):
|
||||
# Prevent errors when list comprehension can't match data
|
||||
@ -2426,13 +2432,13 @@ class Project:
|
||||
"node_id": _node_a.node_id,
|
||||
"adapter_number": _port_a["adapter_number"],
|
||||
"port_number": _port_a["port_number"],
|
||||
"label": {"text": _port_a["name"]},
|
||||
"label": {"text": _port_a.get("short_name") or _port_a["name"]},
|
||||
},
|
||||
{
|
||||
"node_id": _node_b.node_id,
|
||||
"adapter_number": _port_b["adapter_number"],
|
||||
"port_number": _port_b["port_number"],
|
||||
"label": {"text": _port_b["name"]},
|
||||
"label": {"text": _port_b.get("short_name") or _port_b["name"]},
|
||||
},
|
||||
],
|
||||
)
|
||||
|
||||
@ -52,6 +52,7 @@ You have access to the following tools to help users:
|
||||
|
||||
| Tool | Purpose | Usage |
|
||||
|------|---------|-------|
|
||||
| `device_skills` | Query device/protocol/feature skills | Get command syntax, troubleshooting |
|
||||
| `gns3_template_reader` | Get available node templates | List templates |
|
||||
| `gns3_create_node` | Create new nodes in topology | Add routers, switches, VPCS |
|
||||
| `gns3_link_tool` | Create links between nodes | Connect topology |
|
||||
@ -96,6 +97,28 @@ Clearly communicate:
|
||||
|
||||
---
|
||||
|
||||
# TOPOLOGY PLANNING WORKFLOW
|
||||
|
||||
When user asks to create a network lab/experiment/topology:
|
||||
|
||||
1. **Query topology_planner skill**:
|
||||
```
|
||||
device_skills({"action": "get", "device_type": "topology_planner"})
|
||||
```
|
||||
|
||||
2. **Follow the skill's guidance**:
|
||||
- Use IOU image by default
|
||||
- Plan IP addressing with 10.0.0.0/8 range, /24 for LANs, /30 for P2P links
|
||||
- Use naming convention: R1, R2 for routers; S1, S2 for switches; PC1, PC2 for PCs
|
||||
- Position nodes based on topology type (star/ring/bus/mesh/hierarchical)
|
||||
- Place hub/spine nodes at center, leaf nodes radiating outward
|
||||
- Use "name" field in create_gns3_node to set names directly (no separate rename step)
|
||||
- Follow 6-step workflow: read templates → create nodes → link → start → verify → config
|
||||
|
||||
3. **Output topology plan** using the skill's output_template format
|
||||
|
||||
---
|
||||
|
||||
# COMMAND EXAMPLES
|
||||
|
||||
## Diagnostic Commands (READ-ONLY)
|
||||
@ -140,19 +163,14 @@ vlan 10
|
||||
|
||||
# RESPONSE GUIDELINES
|
||||
|
||||
1. **Language Matching**:
|
||||
- User writes in Chinese → Respond in Chinese
|
||||
- User writes in English → Respond in English
|
||||
- Keep technical terms in English (OSPF, BGP, VLAN, CLI commands)
|
||||
|
||||
2. **Clear Structure**:
|
||||
```markdown
|
||||
## 操作总结 / Operation Summary
|
||||
## Operation Summary
|
||||
|
||||
**执行的任务**: [What was done]
|
||||
**结果**: [Success/Failure]
|
||||
**Task**: [What was done]
|
||||
**Result**: [Success/Failure]
|
||||
|
||||
## 详细信息 / Details
|
||||
## Details
|
||||
[Device outputs, configurations, etc.]
|
||||
```
|
||||
|
||||
|
||||
@ -119,14 +119,6 @@ ip route, ip addr, tcpdump, ping, traceroute
|
||||
|
||||
---
|
||||
|
||||
# RESPONSE LANGUAGE
|
||||
|
||||
- User writes in Chinese → Respond in Chinese
|
||||
- User writes in English → Respond in English
|
||||
- Keep technical terms in English (OSPF, BGP, VLAN, CLI commands)
|
||||
|
||||
---
|
||||
|
||||
# CURRENT TOPOLOGY
|
||||
|
||||
{{topology_info}}
|
||||
|
||||
@ -27,14 +27,13 @@
|
||||
Title Generation Prompt for GNS3-Copilot
|
||||
|
||||
Prompt template for generating conversation titles.
|
||||
Generates concise Chinese or English titles based on conversation language.
|
||||
Generates concise titles matching the conversation language.
|
||||
|
||||
"""
|
||||
|
||||
TITLE_PROMPT = """
|
||||
Based on the following conversation records,
|
||||
analyze the language composition and generate a concise, summary title.
|
||||
If the content is predominantly in Chinese, generate a Chinese title.
|
||||
If the content is predominantly in English, generate an English title.
|
||||
Match the language of the conversation.
|
||||
Only return the title, do not include any additional explanations or punctuation:
|
||||
"""
|
||||
|
||||
49
gns3server/agent/gns3_copilot/skills/__init__.py
Normal file
49
gns3server/agent/gns3_copilot/skills/__init__.py
Normal file
@ -0,0 +1,49 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
|
||||
#
|
||||
# This file is part of GNS3-Copilot project.
|
||||
#
|
||||
# GNS3-Copilot is free software: you can redistribute it and/or modify it
|
||||
# under the terms of the GNU General Public License as published by the
|
||||
# Free Software Foundation, either version 3 of the License, or (at your
|
||||
# option) any later version.
|
||||
#
|
||||
# GNS3-Copilot is distributed in the hope that it will be useful, but
|
||||
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
|
||||
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
|
||||
# for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
|
||||
#
|
||||
# Copyright (C) 2025 Yue Guobin
|
||||
# Author: Yue Guobin
|
||||
#
|
||||
# Project Home: https://github.com/yueguobin/gns3-copilot
|
||||
#
|
||||
|
||||
"""
|
||||
Device Skills Package
|
||||
|
||||
This package provides device-specific skills for GNS3 Copilot.
|
||||
Skills are organized by vendor and product series for easy extensibility.
|
||||
|
||||
Directory Structure:
|
||||
- skills/
|
||||
- registry.py # SKILLS_REGISTRY and get_skill()
|
||||
- cisco/ # Cisco devices
|
||||
- huawei/ # Huawei devices
|
||||
- h3c/ # H3C devices
|
||||
- ruijie/ # Ruijie devices
|
||||
- vpcs/ # GNS3 VPCS
|
||||
- generic/ # Base templates
|
||||
"""
|
||||
|
||||
from .registry import SKILLS_REGISTRY, get_skill, DeviceSkillsTool
|
||||
|
||||
__all__ = [
|
||||
"SKILLS_REGISTRY",
|
||||
"get_skill",
|
||||
"DeviceSkillsTool",
|
||||
]
|
||||
13
gns3server/agent/gns3_copilot/skills/cisco/__init__.py
Normal file
13
gns3server/agent/gns3_copilot/skills/cisco/__init__.py
Normal file
@ -0,0 +1,13 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# Cisco Skill Package
|
||||
#
|
||||
# Placeholder for future Cisco device skill implementations.
|
||||
#
|
||||
# Available device types:
|
||||
# - cisco_ios_telnet: Cisco IOS Router (via Telnet)
|
||||
# - cisco_iou_telnet: Cisco IOU L2/L3 Switch
|
||||
#
|
||||
|
||||
# TODO: Implement Cisco IOS skill
|
||||
# TODO: Implement Cisco IOU skill
|
||||
18
gns3server/agent/gns3_copilot/skills/generic/__init__.py
Normal file
18
gns3server/agent/gns3_copilot/skills/generic/__init__.py
Normal file
@ -0,0 +1,18 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# Generic Skill Package
|
||||
#
|
||||
# Base templates and generic skill utilities.
|
||||
#
|
||||
|
||||
# Base skill template for reference:
|
||||
# SKILL_TEMPLATE = {
|
||||
# "device_type": "",
|
||||
# "name": "",
|
||||
# "description": "",
|
||||
# "config_commands": {},
|
||||
# "display_commands": {},
|
||||
# "notes": [],
|
||||
# "troubleshooting": {},
|
||||
# "command_aliases": {},
|
||||
# }
|
||||
11
gns3server/agent/gns3_copilot/skills/h3c/__init__.py
Normal file
11
gns3server/agent/gns3_copilot/skills/h3c/__init__.py
Normal file
@ -0,0 +1,11 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# H3C Skill Package
|
||||
#
|
||||
# Placeholder for future H3C (Hewlett Packard Enterprise) device skill implementations.
|
||||
#
|
||||
# Available device types:
|
||||
# - h3c_telnet: H3C Comware Series (via Telnet)
|
||||
#
|
||||
|
||||
# TODO: Implement H3C Comware skill
|
||||
12
gns3server/agent/gns3_copilot/skills/huawei/__init__.py
Normal file
12
gns3server/agent/gns3_copilot/skills/huawei/__init__.py
Normal file
@ -0,0 +1,12 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# Huawei Skill Package
|
||||
#
|
||||
# Placeholder for future Huawei device skill implementations.
|
||||
#
|
||||
# Available device types:
|
||||
# - huawei_telnet: Huawei NE/AR/CE Series (via Telnet)
|
||||
#
|
||||
|
||||
# TODO: Implement Huawei CloudEngine (CE) skill
|
||||
# TODO: Implement Huawei AR skill
|
||||
276
gns3server/agent/gns3_copilot/skills/registry.py
Normal file
276
gns3server/agent/gns3_copilot/skills/registry.py
Normal file
@ -0,0 +1,276 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
|
||||
#
|
||||
# This file is part of GNS3-Copilot project.
|
||||
#
|
||||
# GNS3-Copilot is free software: you can redistribute it and/or modify it
|
||||
# under the terms of the GNU General Public License as published by the
|
||||
# Free Software Foundation, either version 3 of the License, or (at your
|
||||
# option) any later version.
|
||||
#
|
||||
# GNS3-Copilot is distributed in the hope that it will be useful, but
|
||||
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
|
||||
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
|
||||
# for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
|
||||
#
|
||||
# Copyright (C) 2025 Yue Guobin
|
||||
# Author: Yue Guobin
|
||||
#
|
||||
# Project Home: https://github.com/yueguobin/gns3-copilot
|
||||
#
|
||||
|
||||
"""
|
||||
Skill Registry and DeviceSkillsTool
|
||||
|
||||
This module provides:
|
||||
- SKILLS_REGISTRY: A dictionary mapping device_type to skill definitions
|
||||
- get_skill(): Function to retrieve skill for a device_type
|
||||
- DeviceSkillsTool: LangChain tool for LLM to query skills
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
from typing import Any
|
||||
|
||||
from langchain.tools import BaseTool
|
||||
from langchain_core.callbacks import CallbackManagerForToolRun
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Import all skill modules to register them
|
||||
from gns3server.agent.gns3_copilot.skills.vpcs import VPCS_SKILL
|
||||
from gns3server.agent.gns3_copilot.skills.topology import TOPOLOGY_PLANNER_SKILL
|
||||
|
||||
# Global skill registry - maps device_type to skill definition
|
||||
SKILLS_REGISTRY: dict[str, dict[str, Any]] = {
|
||||
"gns3_vpcs_telnet": VPCS_SKILL,
|
||||
"topology_planner": TOPOLOGY_PLANNER_SKILL,
|
||||
# Add more skills here as they are implemented
|
||||
# "huawei_telnet": HUAWEI_SKILL,
|
||||
# "ruijie_telnet": RUIJIE_SKILL,
|
||||
# Protocol/Feature skills:
|
||||
# "ospf": OSPF_SKILL,
|
||||
# "bgp": BGP_SKILL,
|
||||
# "mpls": MPLS_SKILL,
|
||||
}
|
||||
|
||||
|
||||
def get_skill(
|
||||
device_type: str,
|
||||
category: str | None = None,
|
||||
operation: str = "all"
|
||||
) -> dict[str, Any]:
|
||||
"""
|
||||
Get skill by device_type, optionally filtered by category.
|
||||
|
||||
Args:
|
||||
device_type: The device type identifier (e.g., "gns3_vpcs_telnet", "huawei_telnet")
|
||||
category: Optional category filter - "device", "protocol", "feature"
|
||||
operation: Filter by operation type - "config", "diagnosis", or "all" (default)
|
||||
|
||||
Returns:
|
||||
Skill dictionary containing commands, notes, and troubleshooting info,
|
||||
or error dict if device_type not found
|
||||
"""
|
||||
skill = SKILLS_REGISTRY.get(device_type, {})
|
||||
|
||||
if not skill:
|
||||
# Try to find by name if not found by device_type
|
||||
for did, s in SKILLS_REGISTRY.items():
|
||||
if s.get("name", "").lower() == device_type.lower():
|
||||
skill = s
|
||||
break
|
||||
|
||||
if not skill:
|
||||
return {
|
||||
"error": f"Unknown device_type: {device_type}",
|
||||
"available_device_types": list(SKILLS_REGISTRY.keys()),
|
||||
}
|
||||
|
||||
# Filter by category if specified
|
||||
if category:
|
||||
skill_category = skill.get("category", "")
|
||||
if category.lower() != skill_category.lower():
|
||||
return {
|
||||
"error": f"device_type '{device_type}' is not in category '{category}'",
|
||||
"device_category": skill.get("category"),
|
||||
"available_in_category": [
|
||||
did for did, s in SKILLS_REGISTRY.items()
|
||||
if s.get("category", "").lower() == category.lower()
|
||||
],
|
||||
}
|
||||
|
||||
if operation == "config":
|
||||
return {
|
||||
"device_type": device_type,
|
||||
"name": skill.get("name"),
|
||||
"category": skill.get("category"),
|
||||
"description": skill.get("description"),
|
||||
"config_commands": skill.get("config_commands", {}),
|
||||
}
|
||||
elif operation == "diagnosis":
|
||||
return {
|
||||
"device_type": device_type,
|
||||
"name": skill.get("name"),
|
||||
"category": skill.get("category"),
|
||||
"display_commands": skill.get("display_commands", {}),
|
||||
"troubleshooting": skill.get("troubleshooting", {}),
|
||||
}
|
||||
else:
|
||||
# Return full skill
|
||||
result = dict(skill)
|
||||
result["device_type"] = device_type
|
||||
return result
|
||||
|
||||
|
||||
def list_available_skills(category: str | None = None) -> list[dict[str, str]]:
|
||||
"""
|
||||
List all available skills, optionally filtered by category.
|
||||
|
||||
Args:
|
||||
category: Optional category filter - "device", "protocol", "feature"
|
||||
|
||||
Returns:
|
||||
List of dicts with device_type, name, and category
|
||||
"""
|
||||
skills = []
|
||||
for did, skill in SKILLS_REGISTRY.items():
|
||||
if category:
|
||||
if skill.get("category", "").lower() == category.lower():
|
||||
skills.append({
|
||||
"device_type": did,
|
||||
"name": skill.get("name", did),
|
||||
"category": skill.get("category"),
|
||||
})
|
||||
else:
|
||||
skills.append({
|
||||
"device_type": did,
|
||||
"name": skill.get("name", did),
|
||||
"category": skill.get("category"),
|
||||
})
|
||||
return skills
|
||||
|
||||
|
||||
class DeviceSkillsTool(BaseTool):
|
||||
"""
|
||||
LangChain tool for querying device-specific skills.
|
||||
|
||||
Use this tool to get device-specific command syntax, examples,
|
||||
and troubleshooting guidance before executing commands.
|
||||
|
||||
Example:
|
||||
# Get VPCS skill by device_type
|
||||
tool.run('{"device_type": "gns3_vpcs_telnet"}')
|
||||
|
||||
# Get OSPF protocol skill
|
||||
tool.run('{"device_type": "ospf", "category": "protocol"}')
|
||||
|
||||
# Get only config commands
|
||||
tool.run('{"device_type": "gns3_vpcs_telnet", "operation": "config"}')
|
||||
|
||||
# Get only diagnosis commands
|
||||
tool.run('{"device_type": "gns3_vpcs_telnet", "operation": "diagnosis"}')
|
||||
|
||||
# List all available skills
|
||||
tool.run('{"action": "list"}')
|
||||
|
||||
# List skills by category
|
||||
tool.run('{"action": "list", "category": "device"}')
|
||||
"""
|
||||
|
||||
name: str = "device_skills"
|
||||
description: str = """
|
||||
Get or list device/ protocol/ feature specific skills and command knowledge.
|
||||
|
||||
Use this tool BEFORE executing device commands to understand:
|
||||
- Command syntax for the specific device type
|
||||
- Configuration command examples
|
||||
- Display/diagnostic command syntax
|
||||
- Troubleshooting guidance
|
||||
|
||||
INPUT FORMAT (JSON string):
|
||||
{
|
||||
"action": "get", # Optional: "get" (default) or "list"
|
||||
"device_type": "gns3_vpcs_telnet", # Required for action="get": device type identifier
|
||||
"category": "device", # Optional: "device", "protocol", "feature"
|
||||
"operation": "all" # Optional: "config", "diagnosis", or "all" (default)
|
||||
}
|
||||
|
||||
For action="list":
|
||||
{
|
||||
"action": "list",
|
||||
"category": "device" # Optional: filter by category
|
||||
}
|
||||
|
||||
OUTPUT:
|
||||
- Skill name and description
|
||||
- Command syntax with parameters
|
||||
- Usage examples
|
||||
- Troubleshooting tips
|
||||
- Important notes
|
||||
|
||||
Available categories:
|
||||
- "device": Device-specific skills (VPCS, routers, switches)
|
||||
- "protocol": Network protocol skills (OSPF, BGP, MPLS)
|
||||
- "feature": Feature skills (ACL, QoS, NAT)
|
||||
"""
|
||||
|
||||
def _run(
|
||||
self,
|
||||
tool_input: str | dict[str, Any],
|
||||
run_manager: CallbackManagerForToolRun | None = None,
|
||||
**kwargs: Any,
|
||||
) -> str:
|
||||
"""
|
||||
Execute the device skills lookup.
|
||||
|
||||
Args:
|
||||
tool_input: JSON string or dict with device_type and optional operation/category
|
||||
|
||||
Returns:
|
||||
JSON string with skill information or skill list
|
||||
"""
|
||||
logger.info("DeviceSkillsTool invoked with input: %s", tool_input)
|
||||
|
||||
# Parse input
|
||||
if isinstance(tool_input, str):
|
||||
try:
|
||||
params = json.loads(tool_input)
|
||||
except json.JSONDecodeError as e:
|
||||
return json.dumps({
|
||||
"error": f"Invalid JSON input: {e}",
|
||||
"hint": 'Expected format: {"device_type": "xxx"} or {"action": "list"}'
|
||||
}, ensure_ascii=False, indent=2)
|
||||
else:
|
||||
params = tool_input
|
||||
|
||||
action = params.get("action", "get")
|
||||
|
||||
if action == "list":
|
||||
category = params.get("category")
|
||||
skills = list_available_skills(category)
|
||||
return json.dumps({
|
||||
"category": category or "all",
|
||||
"skills": skills
|
||||
}, ensure_ascii=False, indent=2)
|
||||
|
||||
# Default action: "get"
|
||||
device_type = params.get("device_type")
|
||||
if not device_type:
|
||||
return json.dumps({
|
||||
"error": "Missing required field: device_type",
|
||||
"available_device_types": list(SKILLS_REGISTRY.keys()),
|
||||
"hint": 'Use {"action": "list"} to see all available device types'
|
||||
}, ensure_ascii=False, indent=2)
|
||||
|
||||
category = params.get("category")
|
||||
operation = params.get("operation", "all")
|
||||
|
||||
# Get skill
|
||||
skill = get_skill(device_type, category, operation)
|
||||
|
||||
return json.dumps(skill, ensure_ascii=False, indent=2)
|
||||
11
gns3server/agent/gns3_copilot/skills/ruijie/__init__.py
Normal file
11
gns3server/agent/gns3_copilot/skills/ruijie/__init__.py
Normal file
@ -0,0 +1,11 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# Ruijie Skill Package
|
||||
#
|
||||
# Placeholder for future Ruijie Networks device skill implementations.
|
||||
#
|
||||
# Available device types:
|
||||
# - ruijie_telnet: Ruijie RGOS (via Telnet)
|
||||
#
|
||||
|
||||
# TODO: Implement Ruijie RGOS skill
|
||||
34
gns3server/agent/gns3_copilot/skills/topology/__init__.py
Normal file
34
gns3server/agent/gns3_copilot/skills/topology/__init__.py
Normal file
@ -0,0 +1,34 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
|
||||
#
|
||||
# This file is part of GNS3-Copilot project.
|
||||
#
|
||||
# GNS3-Copilot is free software: you can redistribute it and/or modify it
|
||||
# under the terms of the GNU General Public License as published by the
|
||||
# Free Software Foundation, either version 3 of the License, or (at your
|
||||
# option) any later version.
|
||||
#
|
||||
# GNS3-Copilot is distributed in the hope that it will be useful, but
|
||||
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
|
||||
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
|
||||
# for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
|
||||
#
|
||||
# Copyright (C) 2025 Yue Guobin
|
||||
# Author: Yue Guobin
|
||||
#
|
||||
# Project Home: https://github.com/yueguobin/gns3-copilot
|
||||
#
|
||||
|
||||
"""
|
||||
Topology Planner Skill Package
|
||||
|
||||
This package provides the skill for automatic network lab topology planning.
|
||||
"""
|
||||
|
||||
from .topology_planner_skill import TOPOLOGY_PLANNER_SKILL
|
||||
|
||||
__all__ = ["TOPOLOGY_PLANNER_SKILL"]
|
||||
@ -0,0 +1,235 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
|
||||
#
|
||||
# This file is part of GNS3-Copilot project.
|
||||
#
|
||||
# GNS3-Copilot is free software: you can redistribute it and/or modify it
|
||||
# under the terms of the GNU General Public License as published by the
|
||||
# Free Software Foundation, either version 3 of the License, or (at your
|
||||
# option) any later version.
|
||||
#
|
||||
# GNS3-Copilot is distributed in the hope that it will be useful, but
|
||||
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
|
||||
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
|
||||
# for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
|
||||
#
|
||||
# Copyright (C) 2025 Yue Guobin
|
||||
# Author: Yue Guobin
|
||||
#
|
||||
# Project Home: https://github.com/yueguobin/gns3-copilot
|
||||
#
|
||||
|
||||
"""
|
||||
Topology Planner Skill for GNS3 Lab Automation
|
||||
|
||||
This skill helps users plan and create network lab topologies automatically.
|
||||
|
||||
Key Constraints:
|
||||
- Default image: IOU (L3 device)
|
||||
- Default IP range: 10.0.0.0/8 (use 10.x.x.x subnets)
|
||||
- Max 10 nodes recommended
|
||||
- Node naming: Router=R, Switch=S, PC=PC
|
||||
"""
|
||||
|
||||
# Default node naming convention
|
||||
NODE_NAMING = {
|
||||
"router": "R", # e.g., R-1, R-2, R-3
|
||||
"switch": "SW", # e.g., SW-1, SW-2
|
||||
"pc": "PC", # e.g., PC-1, PC-2
|
||||
}
|
||||
|
||||
# Default IOU template name
|
||||
DEFAULT_IOU_TEMPLATE = "IOU"
|
||||
|
||||
|
||||
# Topology Planner Skill Definition
|
||||
TOPOLOGY_PLANNER_SKILL = {
|
||||
"device_type": "topology_planner",
|
||||
"category": "feature",
|
||||
"name": "GNS3 Topology Planner",
|
||||
"description": "Automatically plan and create GNS3 network lab topologies",
|
||||
|
||||
# Default settings
|
||||
"defaults": {
|
||||
"image": "IOU",
|
||||
"ip_range": "10.0.0.0/8",
|
||||
"max_nodes": 10,
|
||||
"naming": NODE_NAMING,
|
||||
},
|
||||
|
||||
# IP planning rules
|
||||
"ip_planning": {
|
||||
"subnet_format": "10.0.{node_pair}.0/30 for P2P links, 10.0.{node_pair}.0/24 for LANs",
|
||||
"subnet_example": "10.0.12.x = R1-R2 (.1=R1, .2=R2)",
|
||||
"gateway_convention": ".1/.2 for P2P, .254 for LAN gateway",
|
||||
},
|
||||
|
||||
# Node naming conventions
|
||||
"naming_rules": {
|
||||
"router": "R{number}, e.g., R-1, R-2, R-3",
|
||||
"switch": "SW{number}, e.g., SW-1, SW-2",
|
||||
"pc": "PC{number}, e.g., PC-1, PC-2",
|
||||
"firewall": "FW{number}, e.g., FW-1",
|
||||
"loopback": "Lo{number}, e.g., Lo-0, Lo-1",
|
||||
},
|
||||
|
||||
# Node positioning rules based on topology type
|
||||
"positioning_rules": {
|
||||
"min_distance_px": 250,
|
||||
"topology_types": {
|
||||
"star": {
|
||||
"description": "Central node with peripherals around it",
|
||||
"center_node": {"x": 0, "y": 0},
|
||||
"peripheral_nodes": "Arrange in circle around center, angle = index * (360 / count)",
|
||||
"radius": 300,
|
||||
"example_3nodes": {"R-1": (0, 0), "R-2": (-300, 0), "R-3": (300, 0)},
|
||||
"example_5nodes": {"R-1": (0, 0), "R-2": (-250, -250), "R-3": (250, -250), "R-4": (-250, 250), "R-5": (250, 250)},
|
||||
},
|
||||
"ring": {
|
||||
"description": "Nodes connected in a closed loop",
|
||||
"arrangement": "Circle arrangement, equal spacing",
|
||||
"radius": 250,
|
||||
"example_3nodes": {"R-1": (0, -250), "R-2": (216, 125), "R-3": (-216, 125)},
|
||||
"example_4nodes": {"R-1": (0, -250), "R-2": (250, 0), "R-3": (0, 250), "R-4": (-250, 0)},
|
||||
},
|
||||
"bus": {
|
||||
"description": "Linear chain of nodes",
|
||||
"arrangement": "Horizontal line, equal spacing",
|
||||
"spacing_x": 300,
|
||||
"spacing_y": 0,
|
||||
"example_3nodes": {"R-1": (-300, 0), "R-2": (0, 0), "R-3": (300, 0)},
|
||||
},
|
||||
"mesh": {
|
||||
"description": "Fully or partially interconnected nodes",
|
||||
"arrangement": "Grid pattern, rows and columns",
|
||||
"cols": 2,
|
||||
"spacing_x": 300,
|
||||
"spacing_y": 250,
|
||||
"example_4nodes": {"R-1": (-150, -125), "R-2": (150, -125), "R-3": (-150, 125), "R-4": (150, 125)},
|
||||
},
|
||||
"hierarchical": {
|
||||
"description": "Three-tier: Core -> Distribution -> Access",
|
||||
"layers": {
|
||||
"core": {"y": -250, "x": 0},
|
||||
"distribution": {"y": 0, "x_offset": 200},
|
||||
"access": {"y": 250, "x_offset": 300},
|
||||
},
|
||||
"example_5nodes": {"Core": (0, -250), "Dist1": (-200, 0), "Dist2": (200, 0), "Acc1": (-300, 250), "Acc2": (300, 250)},
|
||||
},
|
||||
"linear_p2p": {
|
||||
"description": "Point-to-point links in a line (WAN links)",
|
||||
"arrangement": "Horizontal or vertical line",
|
||||
"spacing_x": 250,
|
||||
"spacing_y": 0,
|
||||
"example_3routers": {"R-1": (-250, 0), "R-2": (0, 0), "R-3": (250, 0)},
|
||||
},
|
||||
},
|
||||
"general_guidelines": [
|
||||
"Place hub/spine nodes at center (0,0) or top center",
|
||||
"Leaf/edge nodes radiate outward from center",
|
||||
"WAN routers typically on left and right sides",
|
||||
"PCs/terminals placed at outer edges",
|
||||
"Maintain minimum 250px between any two nodes",
|
||||
"Adjust positions to reflect actual network topology logic",
|
||||
"Combine topology types as needed (e.g., star + linear_p2p for WAN segments)",
|
||||
],
|
||||
},
|
||||
|
||||
# Tool call workflow
|
||||
"workflow": {
|
||||
"step_1_read_templates": {
|
||||
"tool": "gns3_template_reader",
|
||||
"purpose": "Find available IOU template name",
|
||||
"example": "List templates to identify IOU template"
|
||||
},
|
||||
"step_2_create_nodes": {
|
||||
"tool": "gns3_create_node",
|
||||
"purpose": "Create all router/switch/PC nodes",
|
||||
"params_required": ["project_id", "nodes: [{template_id, x, y, name?}]"],
|
||||
"positioning": "Choose topology type (star/ring/bus/mesh/hierarchical) based on network design. Place hub/spine at center, leaves at edges. Maintain 250px min distance.",
|
||||
"note": "Use 'name' field to set node names directly (e.g., R1, R2). No separate rename step needed."
|
||||
},
|
||||
"step_3_create_links": {
|
||||
"tool": "gns3_link_tool",
|
||||
"purpose": "Connect nodes according to topology design",
|
||||
"params_required": ["node1_id", "node1_port", "node2_id", "node2_port"]
|
||||
},
|
||||
"step_4_start_nodes": {
|
||||
"tool": "gns3_start_node_tool",
|
||||
"purpose": "Power on all nodes",
|
||||
"params_required": ["node_ids"],
|
||||
"note": "Start nodes before configuration"
|
||||
},
|
||||
"step_5_verify": {
|
||||
"tool": "execute_multiple_device_commands",
|
||||
"purpose": "Verify connectivity before config",
|
||||
"example_commands": ["ping <neighbor_ip>"]
|
||||
},
|
||||
"step_6_config": {
|
||||
"tool": "execute_multiple_device_config_commands",
|
||||
"purpose": "Apply network configuration",
|
||||
"note": "Only after verifying physical connectivity"
|
||||
},
|
||||
},
|
||||
|
||||
# Troubleshooting
|
||||
"troubleshooting": {
|
||||
"node_creation_failed": [
|
||||
"1. Check if template name is correct (use gns3_template_reader)",
|
||||
"2. Verify GNS3 server is running",
|
||||
"3. Check compute resource availability"
|
||||
],
|
||||
"link_creation_failed": [
|
||||
"1. Verify both nodes exist and have available ports",
|
||||
"2. Check if link already exists between nodes",
|
||||
"3. Confirm nodes are stopped before linking (some setups)"
|
||||
],
|
||||
"node_start_failed": [
|
||||
"1. Check if node is already running",
|
||||
"2. Verify compute resource has enough memory",
|
||||
"3. Check console port availability"
|
||||
],
|
||||
"connectivity_failed": [
|
||||
"1. Use show ip interface brief to verify IPs are configured",
|
||||
"2. Check if interfaces are administratively up (no shutdown)",
|
||||
"3. Verify cable/port mapping in GNS3 topology"
|
||||
],
|
||||
},
|
||||
|
||||
# Planning output format
|
||||
"output_template": """
|
||||
## Topology Plan
|
||||
|
||||
### Devices
|
||||
| Node | Type | Image | Description |
|
||||
|------|------|-------|-------------|
|
||||
| R-1 | Router | IOU | Core router |
|
||||
| ... | ... | ... | ... |
|
||||
|
||||
### Connections
|
||||
| Node1 | Port | Node2 | Port |
|
||||
|-------|------|-------|------|
|
||||
| R-1 | {short_name} | R-2 | {short_name} |
|
||||
| ... | ... | ... | ... |
|
||||
|
||||
### IP Addressing
|
||||
| Device | Interface | IP Address | Subnet |
|
||||
|--------|-----------|------------|--------|
|
||||
| R-1 | {short_name} | 10.0.12.1 | /30 |
|
||||
| R-2 | {short_name} | 10.0.12.2 | /30 |
|
||||
| ... | ... | ... | ... |
|
||||
|
||||
### Configuration Steps
|
||||
1. Create nodes: gns3_create_node(...)
|
||||
2. Create links: gns3_link_tool(...)
|
||||
3. Start nodes: gns3_start_node_tool(...)
|
||||
4. Verify connectivity: ping ...
|
||||
5. Apply config: execute_multiple_device_config_commands(...)
|
||||
|
||||
Note: Use "name" field in gns3_create_node to set node names directly. No separate rename step needed.
|
||||
""",
|
||||
}
|
||||
34
gns3server/agent/gns3_copilot/skills/vpcs/__init__.py
Normal file
34
gns3server/agent/gns3_copilot/skills/vpcs/__init__.py
Normal file
@ -0,0 +1,34 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
|
||||
#
|
||||
# This file is part of GNS3-Copilot project.
|
||||
#
|
||||
# GNS3-Copilot is free software: you can redistribute it and/or modify it
|
||||
# under the terms of the GNU General Public License as published by the
|
||||
# Free Software Foundation, either version 3 of the License, or (at your
|
||||
# option) any later version.
|
||||
#
|
||||
# GNS3-Copilot is distributed in the hope that it will be useful, but
|
||||
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
|
||||
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
|
||||
# for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
|
||||
#
|
||||
# Copyright (C) 2025 Yue Guobin
|
||||
# Author: Yue Guobin
|
||||
#
|
||||
# Project Home: https://github.com/yueguobin/gns3-copilot
|
||||
#
|
||||
|
||||
"""
|
||||
VPCS Skill Package
|
||||
|
||||
This package provides the skill definition for GNS3 VPCS Virtual PC Simulator.
|
||||
"""
|
||||
|
||||
from .vpcs_skill import VPCS_SKILL
|
||||
|
||||
__all__ = ["VPCS_SKILL"]
|
||||
158
gns3server/agent/gns3_copilot/skills/vpcs/vpcs_skill.py
Normal file
158
gns3server/agent/gns3_copilot/skills/vpcs/vpcs_skill.py
Normal file
@ -0,0 +1,158 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
|
||||
#
|
||||
# This file is part of GNS3-Copilot project.
|
||||
#
|
||||
# GNS3-Copilot is free software: you can redistribute it and/or modify it
|
||||
# under the terms of the GNU General Public License as published by the
|
||||
# Free Software Foundation, either version 3 of the License, or (at your
|
||||
# option) any later version.
|
||||
#
|
||||
# GNS3-Copilot is distributed in the hope that it will be useful, but
|
||||
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
|
||||
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
|
||||
# for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
|
||||
#
|
||||
# Copyright (C) 2025 Yue Guobin
|
||||
# Author: Yue Guobin
|
||||
#
|
||||
# Project Home: https://github.com/yueguobin/gns3-copilot
|
||||
#
|
||||
|
||||
"""
|
||||
VPCS Skill Definition
|
||||
|
||||
GNS3 VPCS (Virtual PC Simulator) is a lightweight virtual PC simulator
|
||||
used in GNS3 labs for testing network connectivity and basic IP configuration.
|
||||
|
||||
Key Characteristics:
|
||||
- NOT a network device (router/switch), it's a simple PC simulator
|
||||
- No authentication required (direct console access)
|
||||
- No configuration mode (commands entered directly)
|
||||
- Simple command set focused on IP configuration and connectivity testing
|
||||
|
||||
Device Type Tag: device_type:gns3_vpcs_telnet
|
||||
"""
|
||||
|
||||
# VPCS Skill Definition
|
||||
VPCS_SKILL = {
|
||||
"device_type": "gns3_vpcs_telnet",
|
||||
"category": "device", # device, protocol, or feature
|
||||
"name": "VPCS Virtual PC Simulator",
|
||||
"description": "GNS3 VPCS lightweight virtual PC simulator for testing network connectivity and basic IP configuration",
|
||||
|
||||
# Configuration Commands (no config mode needed)
|
||||
"config_commands": {
|
||||
"ip_config": {
|
||||
"syntax": "ip <address>/<mask> <gateway>",
|
||||
"example": "ip 10.10.0.12/24 10.10.0.254",
|
||||
"description": "Configure PC IP address and default gateway",
|
||||
"parameters": {
|
||||
"address": "IP address, e.g., 10.10.0.12",
|
||||
"mask": "Subnet mask in CIDR notation, e.g., 24 means 255.255.255.0",
|
||||
"gateway": "Default gateway, e.g., 10.10.0.254"
|
||||
}
|
||||
},
|
||||
"ip_dhcp": {
|
||||
"syntax": "ip dhcp",
|
||||
"description": "Obtain IP configuration from DHCP server"
|
||||
},
|
||||
"save": {
|
||||
"syntax": "save",
|
||||
"description": "Save current configuration to NVRAM (persists after reboot)"
|
||||
},
|
||||
"reset": {
|
||||
"syntax": "reset",
|
||||
"description": "Reset VPCS configuration (clears all settings)"
|
||||
},
|
||||
},
|
||||
|
||||
# Display/Diagnostic Commands
|
||||
"display_commands": {
|
||||
"show_ip": {
|
||||
"syntax": "show ip",
|
||||
"description": "Show current IP configuration (IP address, subnet mask, gateway)"
|
||||
},
|
||||
"ping": {
|
||||
"syntax": "ping <destination>",
|
||||
"example": "ping 10.10.0.254",
|
||||
"description": "Test connectivity to destination (sends 4 ICMP echo requests)"
|
||||
},
|
||||
"ping_count": {
|
||||
"syntax": "ping <destination> <count>",
|
||||
"example": "ping 10.10.0.254 10",
|
||||
"description": "Send specified number of ICMP packets"
|
||||
},
|
||||
"arp": {
|
||||
"syntax": "arp",
|
||||
"description": "Display ARP cache table"
|
||||
},
|
||||
"version": {
|
||||
"syntax": "version",
|
||||
"description": "Show VPCS version information"
|
||||
},
|
||||
"show": {
|
||||
"syntax": "show",
|
||||
"description": "Display current running configuration"
|
||||
},
|
||||
"pc_info": {
|
||||
"syntax": "pcinfo",
|
||||
"description": "Display PC hardware information"
|
||||
},
|
||||
"route": {
|
||||
"syntax": "route",
|
||||
"description": "Display routing table (static routes)"
|
||||
},
|
||||
},
|
||||
|
||||
# Important Notes
|
||||
"notes": [
|
||||
"WARNING: VPCS is NOT a network device (router/switch), it is a lightweight PC simulator!",
|
||||
"WARNING: Do NOT use router/switch config commands on VPCS (e.g., configure terminal, interface)",
|
||||
"VPCS has no config mode, commands are entered directly",
|
||||
"VPCS does not require username/password authentication, direct console access",
|
||||
"Prompt format: PC1>, PC2>, VPCS>",
|
||||
"Default sends 4 ICMP packets, use ping <ip> <count> to specify number",
|
||||
"Must execute save to persist configuration, otherwise lost after reboot",
|
||||
],
|
||||
|
||||
# Troubleshooting Guide
|
||||
"troubleshooting": {
|
||||
"ping failed": [
|
||||
"1. Use show ip to verify IP configuration is correct",
|
||||
"2. Verify target gateway is reachable (ping gateway IP)",
|
||||
"3. Ensure source and target are in same subnet or gateway is correct",
|
||||
"4. Check if link is UP (verify GNS3 topology connections)"
|
||||
],
|
||||
"configuration lost": [
|
||||
"1. VPCS configuration is lost after reboot",
|
||||
"2. Must execute save command after any configuration change",
|
||||
"3. Use show command to verify current configuration"
|
||||
],
|
||||
"cannot connect to console": [
|
||||
"1. Check if node is started in GNS3",
|
||||
"2. Verify console port mapping is correct",
|
||||
"3. Confirm telnet connection parameters are correct (IP:port)"
|
||||
],
|
||||
},
|
||||
|
||||
# Command aliases (for LLM understanding)
|
||||
"command_aliases": {
|
||||
"show ip": "show ip",
|
||||
"display ip": "show ip",
|
||||
"config ip": "ip <address>/<mask> <gateway>",
|
||||
"set ip": "ip <address>/<mask> <gateway>",
|
||||
"test connectivity": "ping <destination>",
|
||||
"ping test": "ping <destination>",
|
||||
"save config": "save",
|
||||
"save": "save",
|
||||
"reset": "reset",
|
||||
"show route": "route",
|
||||
"show arp": "arp",
|
||||
"show version": "version",
|
||||
},
|
||||
}
|
||||
@ -59,6 +59,7 @@ from .gns3_start_node import GNS3StartNodeTool
|
||||
from .gns3_stop_node import GNS3StopNodeTool
|
||||
from .gns3_suspend_node import GNS3SuspendNodeTool
|
||||
from .gns3_update_node_name import GNS3UpdateNodeNameTool
|
||||
from .packet_capture_tools import PacketCaptureTool
|
||||
|
||||
# Dynamic version management
|
||||
try:
|
||||
@ -84,6 +85,7 @@ __all__ = [
|
||||
"GNS3SuspendNodeTool",
|
||||
"GNS3UpdateNodeNameTool",
|
||||
"GNS3TemplateTool",
|
||||
"PacketCaptureTool",
|
||||
]
|
||||
|
||||
# Package initialization message
|
||||
|
||||
@ -183,12 +183,12 @@ class GNS3LinkTool(BaseTool):
|
||||
created_links.append({"error": error_msg})
|
||||
continue
|
||||
|
||||
# Find port information
|
||||
# Find port information - match by name or short_name
|
||||
port1_info = next(
|
||||
(
|
||||
port
|
||||
for port in node1.get("ports", [])
|
||||
if port.get("name") == port1
|
||||
if port.get("name") == port1 or port.get("short_name") == port1
|
||||
),
|
||||
None,
|
||||
)
|
||||
@ -196,7 +196,7 @@ class GNS3LinkTool(BaseTool):
|
||||
(
|
||||
port
|
||||
for port in node2.get("ports", [])
|
||||
if port.get("name") == port2
|
||||
if port.get("name") == port2 or port.get("short_name") == port2
|
||||
),
|
||||
None,
|
||||
)
|
||||
@ -219,7 +219,7 @@ class GNS3LinkTool(BaseTool):
|
||||
"port_number": port1_info.get(
|
||||
"port_number", 0
|
||||
),
|
||||
"label": {"text": port1},
|
||||
"label": {"text": port1_info.get("short_name") or port1},
|
||||
},
|
||||
{
|
||||
"node_id": node_id2,
|
||||
@ -229,7 +229,7 @@ class GNS3LinkTool(BaseTool):
|
||||
"port_number": port2_info.get(
|
||||
"port_number", 0
|
||||
),
|
||||
"label": {"text": port2},
|
||||
"label": {"text": port2_info.get("short_name") or port2},
|
||||
},
|
||||
],
|
||||
)
|
||||
|
||||
@ -52,7 +52,7 @@ class GNS3CreateNodeTool(BaseTool):
|
||||
|
||||
**Input:**
|
||||
A JSON object with project_id and array of nodes with template_id,
|
||||
x and y coordinates.
|
||||
x, y coordinates, and optional name.
|
||||
|
||||
Example input:
|
||||
{
|
||||
@ -61,12 +61,14 @@ class GNS3CreateNodeTool(BaseTool):
|
||||
{
|
||||
"template_id": "uuid-of-template",
|
||||
"x": 100,
|
||||
"y": -200
|
||||
"y": -200,
|
||||
"name": "R1"
|
||||
},
|
||||
{
|
||||
"template_id": "uuid-of-template2",
|
||||
"x": -200,
|
||||
"y": 300
|
||||
"y": 300,
|
||||
"name": "R2"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -79,12 +81,12 @@ class GNS3CreateNodeTool(BaseTool):
|
||||
"created_nodes": [
|
||||
{
|
||||
"node_id": "uuid-of-node1",
|
||||
"name": "NodeName1",
|
||||
"name": "R1",
|
||||
"status": "success"
|
||||
},
|
||||
{
|
||||
"node_id": "uuid-of-node2",
|
||||
"name": "NodeName2",
|
||||
"name": "R2",
|
||||
"status": "success"
|
||||
}
|
||||
],
|
||||
@ -99,6 +101,7 @@ class GNS3CreateNodeTool(BaseTool):
|
||||
description: str = """
|
||||
Creates multiple nodes in a GNS3 project using templates and coordinates.
|
||||
Input is a JSON object with project_id and array of nodes.
|
||||
Each node requires: template_id, x, y. Optional: name (to set node name directly).
|
||||
Example input:
|
||||
{
|
||||
"project_id": "uuid-of-project",
|
||||
@ -106,12 +109,14 @@ class GNS3CreateNodeTool(BaseTool):
|
||||
{
|
||||
"template_id": "uuid-of-template",
|
||||
"x": 100,
|
||||
"y": -200
|
||||
"y": -200,
|
||||
"name": "R1"
|
||||
},
|
||||
{
|
||||
"template_id": "uuid-of-template2",
|
||||
"x": -200,
|
||||
"y": 300
|
||||
"y": 300,
|
||||
"name": "R2"
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -170,6 +175,7 @@ class GNS3CreateNodeTool(BaseTool):
|
||||
template_id = node_data.get("template_id")
|
||||
x = node_data.get("x")
|
||||
y = node_data.get("y")
|
||||
name = node_data.get("name")
|
||||
|
||||
if not all(
|
||||
[
|
||||
@ -210,14 +216,16 @@ class GNS3CreateNodeTool(BaseTool):
|
||||
template_id = node_data.get("template_id")
|
||||
x = node_data.get("x")
|
||||
y = node_data.get("y")
|
||||
name = node_data.get("name")
|
||||
|
||||
logger.info(
|
||||
"Creating node %d/%d with template %s at (%s, %s)...",
|
||||
"Creating node %d/%d with template %s at (%s, %s), name=%s...",
|
||||
i + 1,
|
||||
len(nodes),
|
||||
template_id,
|
||||
x,
|
||||
y,
|
||||
name,
|
||||
)
|
||||
|
||||
# Create node
|
||||
@ -226,6 +234,7 @@ class GNS3CreateNodeTool(BaseTool):
|
||||
template_id=template_id,
|
||||
x=x,
|
||||
y=y,
|
||||
name=name,
|
||||
connector=gns3_server,
|
||||
)
|
||||
node.create()
|
||||
@ -299,12 +308,14 @@ if __name__ == "__main__":
|
||||
"template_id": "b923a635-b7cc-4cb5-9a86-9357e04c02f7",
|
||||
"x": 100,
|
||||
"y": -200,
|
||||
"name": "R1",
|
||||
},
|
||||
{
|
||||
# TODO: Replace with actual template UUID
|
||||
"template_id": "b923a635-b7cc-4cb5-9a86-9357e04c02f7",
|
||||
"x": 200,
|
||||
"y": -300,
|
||||
"name": "R2",
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
303
gns3server/agent/gns3_copilot/tools_v2/packet_capture_tools.py
Normal file
303
gns3server/agent/gns3_copilot/tools_v2/packet_capture_tools.py
Normal file
@ -0,0 +1,303 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
|
||||
#
|
||||
# This file is part of GNS3-Copilot project.
|
||||
#
|
||||
# GNS3-Copilot is free software: you can redistribute it and/or modify it
|
||||
# under the terms of the GNU General Public License as published by the
|
||||
# Free Software Foundation, either version 3 of the License, or
|
||||
# (at your option) any later version.
|
||||
#
|
||||
# GNS3-Copilot is distributed in the hope that it will be useful, but
|
||||
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
|
||||
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
|
||||
# for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
|
||||
#
|
||||
# Copyright (C) 2025 Yue Guobin (岳国宾)
|
||||
# Author: Yue Guobin (岳国宾)
|
||||
#
|
||||
# Project Home: https://github.com/yueguobin/gns3-copilot
|
||||
#
|
||||
|
||||
"""
|
||||
Packet Capture Analysis Tool
|
||||
|
||||
Analyzes specific packets from an active GNS3 capture using tshark.
|
||||
Downloads the capture file from GNS3 server and returns detailed packet information.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
|
||||
from langchain.tools import BaseTool
|
||||
from langchain_core.callbacks import CallbackManagerForToolRun
|
||||
|
||||
from gns3server.agent.gns3_copilot.gns3_client.context_helpers import (
|
||||
get_current_jwt_token,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class PacketCaptureTool(BaseTool):
|
||||
"""
|
||||
LangChain tool for analyzing a specific packet from an active GNS3 capture.
|
||||
|
||||
This tool:
|
||||
1. Downloads the capture file from GNS3 server via /capture/file endpoint
|
||||
2. Saves it to a temporary file
|
||||
3. Runs tshark with verbose output (-V) for the specified packet
|
||||
4. Returns the complete packet structure
|
||||
|
||||
Input:
|
||||
project_id (str, required): UUID of the GNS3 project
|
||||
link_id (str, required): UUID of the link to analyze
|
||||
packet_number (int, required): The packet number to analyze
|
||||
|
||||
Output:
|
||||
str: Detailed packet information in text format
|
||||
"""
|
||||
|
||||
name: str = "analyze_packets"
|
||||
description: str = """
|
||||
Analyze a specific packet from an active GNS3 capture using tshark.
|
||||
|
||||
Downloads the capture file from GNS3 server and returns detailed packet information.
|
||||
|
||||
Input (JSON format):
|
||||
- project_id (str, required): UUID of the GNS3 project
|
||||
- link_id (str, required): UUID of the link to analyze
|
||||
- packet_number (int, required): The packet number to analyze
|
||||
|
||||
Output:
|
||||
Complete packet structure in verbose text format
|
||||
|
||||
Example:
|
||||
# Analyze packet #42
|
||||
tool._run(
|
||||
project_id="xxx",
|
||||
link_id="yyy",
|
||||
packet_number=42
|
||||
)
|
||||
"""
|
||||
|
||||
def _run(
|
||||
self,
|
||||
project_id: str,
|
||||
link_id: str,
|
||||
packet_number: int,
|
||||
run_manager: CallbackManagerForToolRun | None = None,
|
||||
) -> str:
|
||||
"""
|
||||
Analyze a specific packet from an active GNS3 capture.
|
||||
|
||||
Args:
|
||||
project_id: UUID of the GNS3 project
|
||||
link_id: UUID of the link to analyze
|
||||
packet_number: The packet number to analyze
|
||||
run_manager: LangChain run manager (unused)
|
||||
|
||||
Returns:
|
||||
str: Detailed packet information
|
||||
"""
|
||||
logger.info(
|
||||
f"PacketCaptureTool invoked: project_id={project_id}, "
|
||||
f"link_id={link_id}, packet_number={packet_number}"
|
||||
)
|
||||
|
||||
# Validate inputs
|
||||
if not project_id:
|
||||
return '{"error": "project_id is required"}'
|
||||
if not link_id:
|
||||
return '{"error": "link_id is required"}'
|
||||
if packet_number is None or packet_number <= 0:
|
||||
return '{"error": "packet_number must be a positive integer"}'
|
||||
|
||||
temp_file = None
|
||||
try:
|
||||
# Download capture file
|
||||
temp_file = self._download_capture(project_id, link_id)
|
||||
if not temp_file:
|
||||
return '{"error": "Failed to download capture file"}'
|
||||
|
||||
# Check if file exists and has content
|
||||
if not os.path.exists(temp_file):
|
||||
return '{"error": "Capture file not found"}'
|
||||
|
||||
file_size = os.path.getsize(temp_file)
|
||||
if file_size == 0:
|
||||
return '{"error": "Capture file is empty, no packets captured yet"}'
|
||||
|
||||
logger.info(f"Capture file downloaded: {temp_file}, size={file_size} bytes")
|
||||
|
||||
# Run tshark verbose analysis for the specific packet
|
||||
result = self._run_tshark_verbose(temp_file, packet_number)
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"PacketCaptureTool error: {e}", exc_info=True)
|
||||
return f'{{"error": "Analysis failed: {str(e)}"}}'
|
||||
|
||||
finally:
|
||||
# Clean up temp file
|
||||
if temp_file and os.path.exists(temp_file):
|
||||
try:
|
||||
os.remove(temp_file)
|
||||
logger.debug(f"Temporary file removed: {temp_file}")
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to remove temp file: {e}")
|
||||
|
||||
def _download_capture(self, project_id: str, link_id: str) -> str | None:
|
||||
"""
|
||||
Download capture file from GNS3 server.
|
||||
|
||||
Args:
|
||||
project_id: Project UUID
|
||||
link_id: Link UUID
|
||||
|
||||
Returns:
|
||||
str: Path to temporary capture file, or None on failure
|
||||
"""
|
||||
jwt_token = get_current_jwt_token()
|
||||
if not jwt_token:
|
||||
logger.error("JWT token not found in context")
|
||||
return None
|
||||
|
||||
# Detect GNS3 server URL
|
||||
url = self._detect_gns3_url()
|
||||
if not url:
|
||||
return None
|
||||
|
||||
capture_url = f"{url}/v3/projects/{project_id}/links/{link_id}/capture/file"
|
||||
logger.info(f"Downloading capture from: {capture_url}")
|
||||
|
||||
# Create temp file
|
||||
temp_fd, temp_file = tempfile.mkstemp(suffix=".pcap", prefix="gns3_capture_")
|
||||
os.close(temp_fd)
|
||||
|
||||
try:
|
||||
import requests
|
||||
|
||||
headers = {"Authorization": f"Bearer {jwt_token}"}
|
||||
response = requests.get(
|
||||
capture_url,
|
||||
headers=headers,
|
||||
stream=True,
|
||||
timeout=30,
|
||||
)
|
||||
|
||||
if response.status_code != 200:
|
||||
logger.error(f"Failed to download capture: HTTP {response.status_code}")
|
||||
os.remove(temp_file)
|
||||
return None
|
||||
|
||||
# Write to temp file
|
||||
with open(temp_file, "wb") as f:
|
||||
for chunk in response.iter_content(chunk_size=8192):
|
||||
f.write(chunk)
|
||||
|
||||
logger.info(f"Capture file saved: {temp_file}, size={os.path.getsize(temp_file)} bytes")
|
||||
return temp_file
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to download capture: {e}", exc_info=True)
|
||||
if os.path.exists(temp_file):
|
||||
os.remove(temp_file)
|
||||
return None
|
||||
|
||||
def _detect_gns3_url(self) -> str | None:
|
||||
"""
|
||||
Detect GNS3 server URL from Controller or Config.
|
||||
|
||||
Returns:
|
||||
str: GNS3 server URL, or None on failure
|
||||
"""
|
||||
try:
|
||||
from gns3server.controller import Controller
|
||||
|
||||
controller = Controller.instance()
|
||||
local_compute = controller.get_compute("local")
|
||||
url = f"{local_compute.protocol}://{local_compute.host}:{local_compute.port}"
|
||||
logger.debug(f"Detected GNS3 URL from Controller: {url}")
|
||||
return url
|
||||
except Exception as e:
|
||||
logger.debug(f"Cannot get URL from Controller: {e}")
|
||||
|
||||
try:
|
||||
from gns3server.config import Config
|
||||
|
||||
server_config = Config.instance().settings.Server
|
||||
url = f"{server_config.protocol.value}://{server_config.host}:{server_config.port}"
|
||||
logger.debug(f"Detected GNS3 URL from Config: {url}")
|
||||
return url
|
||||
except Exception as e:
|
||||
logger.debug(f"Cannot get URL from Config: {e}")
|
||||
|
||||
# Fallback default
|
||||
default_url = "http://127.0.0.1:3080"
|
||||
logger.warning(f"Using fallback default URL: {default_url}")
|
||||
return default_url
|
||||
|
||||
def _run_tshark_verbose(self, pcap_file: str, packet_number: int) -> str:
|
||||
"""
|
||||
Run tshark verbose output for a specific packet.
|
||||
|
||||
Args:
|
||||
pcap_file: Path to the capture file
|
||||
packet_number: The packet number to analyze
|
||||
|
||||
Returns:
|
||||
str: tshark verbose output
|
||||
"""
|
||||
# Build command: tshark -r <file> -Y "frame.number == N" -V
|
||||
cmd = [
|
||||
"tshark",
|
||||
"-r", pcap_file,
|
||||
"-Y", f"frame.number == {packet_number}",
|
||||
"-V"
|
||||
]
|
||||
|
||||
logger.info(f"Running tshark: {' '.join(cmd)}")
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
)
|
||||
|
||||
output = result.stdout
|
||||
if result.stderr:
|
||||
if "tshark:" in result.stderr.lower():
|
||||
logger.warning(f"tshark stderr: {result.stderr}")
|
||||
|
||||
if not output.strip():
|
||||
return f'No packet found with frame.number == {packet_number}'
|
||||
|
||||
logger.info(f"tshark output: {len(output)} characters")
|
||||
return output
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.error("tshark timeout after 30 seconds")
|
||||
return '{"error": "tshark timeout after 30 seconds"}'
|
||||
except FileNotFoundError:
|
||||
logger.error("tshark not found. Please install tshark: apt install tshark")
|
||||
return '{"error": "tshark not installed. Please install tshark: apt install tshark"}'
|
||||
except Exception as e:
|
||||
logger.error(f"tshark execution error: {e}", exc_info=True)
|
||||
return f'{{"error": "tshark failed: {str(e)}"}}'
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Test the tool
|
||||
tool = PacketCaptureTool()
|
||||
print("Testing PacketCaptureTool...")
|
||||
print("Note: Set project_id, link_id and packet_number to test with actual GNS3 capture")
|
||||
print(tool.description)
|
||||
@ -163,7 +163,7 @@ class VPCSTelnet(BaseConnection):
|
||||
logger.debug("Error waiting for VPCS prompt: %s", e)
|
||||
|
||||
# Step 4: Return what we have (connection might still work)
|
||||
logger.warning("VPCS prompt not clearly detected, returning current output")
|
||||
logger.debug("VPCS prompt not clearly detected, returning current output")
|
||||
return return_msg
|
||||
|
||||
def session_preparation(self) -> None:
|
||||
|
||||
466
gns3server/agent/web_wireshark/WEB_WIRESHARK.md
Normal file
466
gns3server/agent/web_wireshark/WEB_WIRESHARK.md
Normal file
@ -0,0 +1,466 @@
|
||||
> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.
|
||||
|
||||
|
||||
# GNS3 Web Wireshark Container Management Guide
|
||||
|
||||
This document describes how to manage GNS3 Web Wireshark containers using the management script or Docker commands.
|
||||
|
||||
## Installation
|
||||
|
||||
Before using Web Wireshark, you need to set up the Docker image:
|
||||
|
||||
```bash
|
||||
pip install . && gns3-wireshark-setup
|
||||
```
|
||||
|
||||
This will:
|
||||
1. First try to pull the `gns3/web-wireshark:latest` image from Docker Hub
|
||||
2. If pull fails, build the image locally using the Dockerfile
|
||||
|
||||
The setup command shows the output from `docker pull` or `docker build` directly, so you can see the progress.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ Host Machine │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────┐ │
|
||||
│ │ Docker Network: gns3-wireshark │ │
|
||||
│ │ (Bridge network for container-host │ │
|
||||
│ │ communication) │ │
|
||||
│ │ │ │
|
||||
│ │ ┌──────────────────────────────────────┐ │ │
|
||||
│ │ │ Container: gns3-PROJECT-ID │ │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ │ Link 1: Display :10001, Port 10001 │ │ │
|
||||
│ │ │ Link 2: Display :10002, Port 10002 │ │ │
|
||||
│ │ │ Link 3: Display :10003, Port 10003 │ │ │
|
||||
│ │ │ ... │ │ │
|
||||
│ │ └──────────────────────────────────────┘ │ │
|
||||
│ └────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Quick Start (Using Management Script)
|
||||
|
||||
### Prerequisites
|
||||
- Docker is running
|
||||
- Virtual environment is activated: `source venv/bin/activate`
|
||||
|
||||
### Start Session
|
||||
|
||||
```bash
|
||||
# Start session (using all defaults)
|
||||
python3 gns3server/agent/web_wireshark/manage_wireshark.py \
|
||||
--verbose start \
|
||||
--project-id "5af0fe00-f39d-4985-8669-7e8c512d729c" \
|
||||
--link-id "f233f27f-7432-49c3-9aa2-50e326a10eec" \
|
||||
--jwt-token "YOUR_JWT_TOKEN"
|
||||
|
||||
# Use custom image
|
||||
python3 gns3server/agent/web_wireshark/manage_wireshark.py \
|
||||
--verbose start \
|
||||
--project-id "5af0fe00-f39d-4985-8669-7e8c512d729c" \
|
||||
--link-id "f233f27f-7432-49c3-9aa2-50e326a10eec" \
|
||||
--jwt-token "YOUR_JWT_TOKEN" \
|
||||
--image "gns3/web-wireshark:test"
|
||||
```
|
||||
|
||||
### Access Web Interface
|
||||
|
||||
After starting, access Wireshark at:
|
||||
```
|
||||
ws://<container-ip>:<port>
|
||||
```
|
||||
|
||||
### Stop Session
|
||||
|
||||
```bash
|
||||
python3 gns3server/agent/web_wireshark/manage_wireshark.py \
|
||||
stop \
|
||||
--project-id "test-project" \
|
||||
--link-id "link-1"
|
||||
```
|
||||
|
||||
### Delete Container
|
||||
|
||||
```bash
|
||||
python3 gns3server/agent/web_wireshark/manage_wireshark.py \
|
||||
delete-container \
|
||||
--project-id "test-project"
|
||||
```
|
||||
|
||||
## Resource Parameter Configuration
|
||||
|
||||
### Memory Configuration
|
||||
|
||||
```bash
|
||||
# Default 2GB memory
|
||||
--memory "2g"
|
||||
|
||||
# Custom memory
|
||||
--memory "4g" # 4GB
|
||||
--memory "512m" # 512MB
|
||||
--memory "1g" \
|
||||
--memory-swap "2g" # 1GB memory + 2GB swap
|
||||
```
|
||||
|
||||
### CPU Configuration
|
||||
|
||||
```bash
|
||||
# Default 1 CPU core
|
||||
--cpus 1.0
|
||||
|
||||
# Custom CPU
|
||||
--cpus 0.5 # 50% CPU
|
||||
--cpus 2.0 # 2 CPU cores
|
||||
--cpus 4.0 # 4 CPU cores
|
||||
```
|
||||
|
||||
### Process Limit Configuration
|
||||
|
||||
```bash
|
||||
# Default max 1000 processes
|
||||
--pids-limit 1000
|
||||
|
||||
# Custom limit
|
||||
--pids-limit 500 # Max 500 processes
|
||||
--pids-limit 2000 # Max 2000 processes
|
||||
```
|
||||
|
||||
### Complete Example
|
||||
|
||||
```bash
|
||||
python3 gns3server/agent/web_wireshark/manage_wireshark.py \
|
||||
--verbose start \
|
||||
--project-id "test-project" \
|
||||
--link-id "link-1" \
|
||||
--jwt-token "test-token" \
|
||||
--image "gns3/web-wireshark:latest" \
|
||||
--memory "4g" \
|
||||
--memory-swap "6g" \
|
||||
--cpus 2.0 \
|
||||
--pids-limit 2000
|
||||
```
|
||||
|
||||
## Parameter Reference Table
|
||||
|
||||
| Parameter | Default Value | Description | Example |
|
||||
|-----------|---------------|-------------|---------|
|
||||
| --image | gns3/web-wireshark:latest | Docker image | ubuntu:latest |
|
||||
| --memory | 2g | Memory limit | 4g, 512m |
|
||||
| --memory-swap | Same as memory | Memory swap limit | 4g, 8g |
|
||||
| --cpus | 1.0 | CPU cores | 0.5, 2.0 |
|
||||
| --pids-limit | 1000 | Process limit | 500, 2000 |
|
||||
|
||||
## Docker Network Management
|
||||
|
||||
### Create Network
|
||||
|
||||
```bash
|
||||
docker network create \
|
||||
--driver bridge \
|
||||
--subnet=100.64.0.0/22 \
|
||||
gns3-wireshark
|
||||
```
|
||||
|
||||
### Delete Network
|
||||
|
||||
```bash
|
||||
docker network rm gns3-wireshark
|
||||
```
|
||||
|
||||
**Warning:** Stop and disconnect all containers before deleting the network.
|
||||
|
||||
### View Network Information
|
||||
|
||||
```bash
|
||||
# List all Docker networks
|
||||
docker network ls
|
||||
|
||||
# View network details
|
||||
docker network inspect gns3-wireshark
|
||||
|
||||
# List containers connected to the network
|
||||
docker network inspect gns3-wireshark -f '{{range .Containers}}{{.Name}} {{end}}'
|
||||
```
|
||||
|
||||
## Performance Metrics
|
||||
|
||||
### Per Wireshark Instance Resource Usage
|
||||
|
||||
| Resource Type | Usage | Description |
|
||||
|---------------|-------|-------------|
|
||||
| **Memory** | 150-250 MB | Depends on capture traffic and number of parsed protocols |
|
||||
| **CPU** | 0.5-2% | Lower at idle, increases with high traffic |
|
||||
| **Threads** | ~30 threads | Wireshark multi-threaded architecture |
|
||||
| **Disk I/O** | Minimal | Mostly log writing |
|
||||
|
||||
### Container Resource Configuration Recommendations
|
||||
|
||||
Based on `--pids-limit 1000` and `--memory="2g"` configuration:
|
||||
|
||||
| Wireshark Instances | Estimated Threads | Estimated Memory | Recommended Use Case |
|
||||
|---------------------|-------------------|------------------|---------------------|
|
||||
| 1-3 | 120-200 threads | 450-750 MB | Lightweight projects, small topologies |
|
||||
| 4-6 | 230-290 threads | 600-1.5 GB | Medium projects, multiple network links |
|
||||
| 7-10 | 320-410 threads | 1-2.5 GB | Large projects, dense capture |
|
||||
| 10+ | >400 threads | >2.5 GB | Warning: Increase memory limit |
|
||||
|
||||
### Actual Test Data
|
||||
|
||||
**Test Environment:** 3 Wireshark instances running simultaneously
|
||||
|
||||
```
|
||||
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
|
||||
19363d29bd9d gns3-PROJECT-ID 4.59% 735.6MiB / 2GiB 35.92% 386kB / 38.6MB 0B / 15.5MB 201
|
||||
```
|
||||
|
||||
**Detailed Process Statistics:**
|
||||
- Total threads: ~204
|
||||
- Total processes: ~61
|
||||
- Per Wireshark instance: ~30 threads + 1 parent process
|
||||
|
||||
### Performance Optimization Recommendations
|
||||
|
||||
1. **Memory is the main bottleneck**, not PID limit
|
||||
- Default 2GB memory can support 6-8 Wireshark instances
|
||||
- When needing more instances, increase memory first rather than PID limit
|
||||
|
||||
2. **Start Wireshark on demand**
|
||||
- Only start Wireshark for links that need packet capture
|
||||
- Stop sessions promptly when done to release resources
|
||||
|
||||
3. **Multi-container strategy**
|
||||
- For super large projects (10+ links), consider using multiple containers
|
||||
- Each container handles 5-8 links for better resource isolation and stability
|
||||
|
||||
4. **Monitor resource usage**
|
||||
```bash
|
||||
# Real-time container resource monitoring
|
||||
docker stats gns3-PROJECT-ID
|
||||
|
||||
# Check process count inside container
|
||||
docker exec gns3-PROJECT_ID bash -c "ps -eLf | wc -l"
|
||||
```
|
||||
|
||||
## Management Commands (Docker)
|
||||
|
||||
### View All Active xpra Sessions
|
||||
|
||||
```bash
|
||||
docker exec "${CONTAINER_NAME}" xpra list
|
||||
```
|
||||
|
||||
### View Container Logs
|
||||
|
||||
```bash
|
||||
docker logs "${CONTAINER_NAME}"
|
||||
```
|
||||
|
||||
### View Processes Inside Container
|
||||
|
||||
```bash
|
||||
docker exec "${CONTAINER_NAME}" ps aux | grep -E "Xvfb|xpra"
|
||||
```
|
||||
|
||||
### Enter Container Shell
|
||||
|
||||
```bash
|
||||
docker exec -it "${CONTAINER_NAME}" /bin/bash
|
||||
```
|
||||
|
||||
### Stop Container
|
||||
|
||||
```bash
|
||||
docker stop "${CONTAINER_NAME}"
|
||||
```
|
||||
|
||||
### Delete Container
|
||||
|
||||
```bash
|
||||
docker rm "${CONTAINER_NAME}"
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Check if Container is Running
|
||||
|
||||
```bash
|
||||
docker ps | grep "${CONTAINER_NAME}"
|
||||
```
|
||||
|
||||
### Check Network Connection
|
||||
|
||||
```bash
|
||||
# Ping container from host
|
||||
docker exec "${CONTAINER_NAME}" ping -c 3 100.64.0.1
|
||||
|
||||
# Check port listening
|
||||
docker exec "${CONTAINER_NAME}" netstat -tlnp | grep xpra
|
||||
```
|
||||
|
||||
### View xpra Logs
|
||||
|
||||
```bash
|
||||
docker exec "${CONTAINER_NAME}" ls -la /tmp/sessions/
|
||||
```
|
||||
|
||||
### Restart Specific Session
|
||||
|
||||
```bash
|
||||
LINK_ID=1
|
||||
DISPLAY_ID=$LINK_ID
|
||||
|
||||
# Stop session
|
||||
docker exec "${CONTAINER_NAME}" xpra stop ":${DISPLAY_ID}"
|
||||
|
||||
# Clean up session files
|
||||
docker exec "${CONTAINER_NAME}" rm -rf "/tmp/sessions/link-${LINK_ID}"
|
||||
|
||||
# Restart (see previous start commands)
|
||||
```
|
||||
|
||||
### Thread Creation Error (QThread::start: Thread creation error)
|
||||
|
||||
**Error Message:**
|
||||
```
|
||||
QThread::start: Thread creation error (Resource temporarily unavailable)
|
||||
```
|
||||
|
||||
**Root Cause Analysis:**
|
||||
- Docker container PID limit (`--pids-limit`) actually limits thread count
|
||||
- Each Wireshark instance requires approximately 30 threads
|
||||
- Default limit of 200 may not be enough for multiple Wireshark instances
|
||||
|
||||
**Solution:**
|
||||
|
||||
```bash
|
||||
# Check current thread usage
|
||||
docker exec "${CONTAINER_NAME}" bash -c "ps -eLf | wc -l"
|
||||
|
||||
# Increase PID limit (recommended to set to 1000)
|
||||
docker update --pids-limit 1000 "${CONTAINER_NAME}"
|
||||
|
||||
# Verify new limit
|
||||
docker inspect "${CONTAINER_NAME}" --format '{{.HostConfig.PidsLimit}}'
|
||||
```
|
||||
|
||||
**Prevention:**
|
||||
- Set a reasonable PID limit when starting the container: `--pids-limit 1000`
|
||||
- Refer to "Performance Metrics" section for appropriate configuration
|
||||
|
||||
### XDG_RUNTIME_DIR Warning
|
||||
|
||||
**Warning Message:**
|
||||
```
|
||||
Warning: XDG_RUNTIME_DIR is not defined
|
||||
and '/run/user/1000' does not exist
|
||||
using '/tmp'
|
||||
```
|
||||
|
||||
**Explanation:**
|
||||
- This is a warning, not an error; Xpra falls back to using `/tmp`
|
||||
- May affect some features relying on XDG specification
|
||||
|
||||
**Solution:**
|
||||
Ensure you are using the latest Docker image, which includes the following fix:
|
||||
- Create `/run/user/1000` directory
|
||||
- Set `XDG_RUNTIME_DIR` environment variable
|
||||
|
||||
For manual fix:
|
||||
```bash
|
||||
docker exec "${CONTAINER_NAME}" mkdir -p /run/user/1000
|
||||
docker exec "${CONTAINER_NAME}" bash -c "export XDG_RUNTIME_DIR=/run/user/1000"
|
||||
```
|
||||
|
||||
## Manual Testing (Step-by-Step)
|
||||
|
||||
This section preserves the original manual testing steps used during development.
|
||||
|
||||
### Prerequisites
|
||||
- Docker is running
|
||||
- Virtual environment is activated: `source venv/bin/activate`
|
||||
|
||||
### Basic Test Commands
|
||||
|
||||
```bash
|
||||
# 1. Start session (using all defaults)
|
||||
python3 gns3server/agent/web_wireshark/manage_wireshark.py \
|
||||
--verbose start \
|
||||
--project-id "5af0fe00-f39d-4985-8669-7e8c512d729c" \
|
||||
--link-id "f233f27f-7432-49c3-9aa2-50e326a10eec" \
|
||||
--jwt-token "YOUR_JWT_TOKEN"
|
||||
|
||||
# 2. Use custom image
|
||||
python3 gns3server/agent/web_wireshark/manage_wireshark.py \
|
||||
--verbose start \
|
||||
--project-id "5af0fe00-f39d-4985-8669-7e8c512d729c" \
|
||||
--link-id "f233f27f-7432-49c3-9aa2-50e326a10eec" \
|
||||
--jwt-token "YOUR_JWT_TOKEN" \
|
||||
--image "gns3/web-wireshark:test"
|
||||
|
||||
# 3. View containers
|
||||
docker ps | grep gns3-wireshark
|
||||
docker logs gns3-wireshark-test-project
|
||||
|
||||
# 4. Stop session
|
||||
python3 gns3server/agent/web_wireshark/manage_wireshark.py \
|
||||
stop \
|
||||
--project-id "test-project" \
|
||||
--link-id "link-1"
|
||||
|
||||
# 5. Delete container
|
||||
python3 gns3server/agent/web_wireshark/manage_wireshark.py \
|
||||
delete-container \
|
||||
--project-id "test-project"
|
||||
```
|
||||
|
||||
### WebSocket Access URL
|
||||
|
||||
After starting a session, access Wireshark via WebSocket:
|
||||
```
|
||||
ws://192.168.1.140:3080/v3/projects/5af0fe00-f39d-4985-8669-7e8c512d729c/links/f233f27f-7432-49c3-9aa2-50e326a10eec/capture/web-wireshark?token=YOUR_JWT_TOKEN
|
||||
```
|
||||
|
||||
## Cleanup (If Tests Fail)
|
||||
|
||||
```bash
|
||||
# Delete test containers
|
||||
docker ps -a | grep 'gns3-wireshark-test' | awk '{print $1}' | xargs -r docker rm -f
|
||||
|
||||
# Delete test networks
|
||||
docker network ls | grep 'gns3-wireshark' | awk '{print $2}' | xargs -r docker network rm
|
||||
```
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
gns3server/agent/web_wireshark/
|
||||
├── setup_wireshark_image.py # Docker image setup tool (gns3-wireshark-setup)
|
||||
├── manage_wireshark.py # CLI management tool
|
||||
├── manager.py # Session management logic
|
||||
├── docker_client.py # Docker API client
|
||||
├── docker/
|
||||
│ └── Dockerfile # Container image definition
|
||||
└── WEB_WIRESHARK.md # This documentation
|
||||
```
|
||||
|
||||
## Known Issues
|
||||
|
||||
- JWT token is passed via command-line arguments (visible in `/proc/<pid>/cmdline`). Consider using a temporary file inside the container for improved security.
|
||||
- `cmd_delete_container` and `cmd_delete` in manage_wireshark.py are duplicate code.
|
||||
- `stop-container` and `delete-container` subcommands are defined but not registered in the commands dictionary.
|
||||
- `link_id_to_display` and `link_id_to_port` return the same value (10000-19999), which may cause confusion since xpra typically uses different display numbers and ports.
|
||||
- Container health check timeout (5 seconds) may be insufficient on slow systems.
|
||||
- Docker Unix socket connection error handling could be improved (FileNotFoundError not properly caught).
|
||||
|
||||
## Notes
|
||||
|
||||
- Default uses `gns3/web-wireshark:latest` image
|
||||
- Use `--verbose` to see detailed logs
|
||||
- Containers use 100.64.0.0/22 network
|
||||
- xpra port range: 10000-19999 (deterministic based on link_id)
|
||||
- Health check: `xpra list`
|
||||
- Log configuration: json-file, max-size=10m, max-file=3
|
||||
61
gns3server/agent/web_wireshark/docker/Dockerfile
Normal file
61
gns3server/agent/web_wireshark/docker/Dockerfile
Normal file
@ -0,0 +1,61 @@
|
||||
FROM debian:trixie
|
||||
|
||||
# Prevent interactive prompts during build
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
|
||||
# Set timezone to UTC
|
||||
ENV TZ=UTC
|
||||
|
||||
# Set UTF-8 locale to avoid Qt warnings
|
||||
ENV LANG=C.UTF-8
|
||||
ENV LC_ALL=C.UTF-8
|
||||
|
||||
# Use Alibaba Cloud Debian mirror for faster downloads in China (Debian Trixie uses deb822 format)
|
||||
RUN sed -i 's|http://deb.debian.org/debian|http://mirrors.aliyun.com/debian|g' /etc/apt/sources.list.d/debian.sources \
|
||||
&& sed -i 's|http://security.debian.org/debian-security|http://mirrors.aliyun.com/debian-security|g' /etc/apt/sources.list.d/debian.sources
|
||||
|
||||
# Add xpra official repository
|
||||
RUN apt-get update && apt-get install -y \
|
||||
ca-certificates \
|
||||
wget \
|
||||
gnupg \
|
||||
&& wget -O "/usr/share/keyrings/xpra.asc" https://xpra.org/xpra.asc \
|
||||
&& echo "deb [arch=amd64,arm64 signed-by=/usr/share/keyrings/xpra.asc] https://xpra.org trixie main" > /etc/apt/sources.list.d/xpra.list \
|
||||
&& apt-get update
|
||||
|
||||
# Install xpra and dependencies
|
||||
# Lock xpra version for reproducible builds
|
||||
RUN apt-get install -y \
|
||||
wireshark-common \
|
||||
wireshark \
|
||||
xpra=6.4.3* \
|
||||
xpra-x11 \
|
||||
xvfb \
|
||||
curl \
|
||||
x11-utils \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Metadata
|
||||
LABEL maintainer="YueGuobin <yueguobin@outlook.com>"
|
||||
LABEL description="Web Wireshark container with xpra for GNS3"
|
||||
|
||||
# Create gns3 user
|
||||
RUN groupadd -g 1000 gns3 && \
|
||||
useradd -u 1000 -g gns3 -m -s /bin/bash gns3
|
||||
|
||||
# Create sessions directory with proper permissions
|
||||
RUN mkdir -p /tmp/sessions && chmod 1777 /tmp/sessions
|
||||
|
||||
# Create XDG runtime directory for gns3 user
|
||||
RUN mkdir -p /run/user/1000 && chown -R gns3:gns3 /run/user/1000 && chmod 700 /run/user/1000
|
||||
|
||||
# Set XDG environment variables
|
||||
ENV XDG_RUNTIME_DIR=/run/user/1000
|
||||
|
||||
|
||||
# Set default user
|
||||
USER gns3
|
||||
|
||||
# No EXPOSE - port will be dynamically allocated
|
||||
# Keep container running
|
||||
CMD ["tail", "-f", "/dev/null"]
|
||||
281
gns3server/agent/web_wireshark/docker_client.py
Normal file
281
gns3server/agent/web_wireshark/docker_client.py
Normal file
@ -0,0 +1,281 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# Copyright (C) 2026 YueGuobin
|
||||
#
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU General Public License as published by
|
||||
# the Free Software Foundation, either version 3 of the License, or
|
||||
# (at your option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
"""
|
||||
Web Wireshark - Docker HTTP API client
|
||||
|
||||
This module handles Docker API operations via aiohttp Unix socket.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
|
||||
import aiohttp
|
||||
|
||||
from gns3server.utils import parse_version
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Docker API configuration
|
||||
DOCKER_SOCKET = "/var/run/docker.sock"
|
||||
DOCKER_MINIMUM_API_VERSION = "1.44"
|
||||
DOCKER_PREFERRED_API_VERSION = "1.44"
|
||||
|
||||
|
||||
class DockerHTTPClient:
|
||||
"""Docker HTTP API client using aiohttp with Unix socket connection."""
|
||||
|
||||
# API request timeout (seconds)
|
||||
REQUEST_TIMEOUT = 10
|
||||
|
||||
def __init__(self):
|
||||
self._connector = None
|
||||
self._session = None
|
||||
self._connected = False
|
||||
self._api_version = DOCKER_MINIMUM_API_VERSION
|
||||
|
||||
async def _get_connector(self):
|
||||
"""Get or create Unix socket connector."""
|
||||
if self._connector is None or self._connector.closed:
|
||||
try:
|
||||
self._connector = aiohttp.connector.UnixConnector(DOCKER_SOCKET, limit=None)
|
||||
except (aiohttp.ClientError, FileNotFoundError):
|
||||
raise RuntimeError(f"Can't connect to Docker daemon at {DOCKER_SOCKET}")
|
||||
return self._connector
|
||||
|
||||
async def _get_session(self):
|
||||
"""Get or create aiohttp session."""
|
||||
if self._session is None or self._session.closed:
|
||||
connector = await self._get_connector()
|
||||
self._session = aiohttp.ClientSession(connector=connector)
|
||||
return self._session
|
||||
|
||||
async def close(self):
|
||||
"""Close connections."""
|
||||
if self._session and not self._session.closed:
|
||||
await self._session.close()
|
||||
if self._connector and not self._connector.closed:
|
||||
await self._connector.close()
|
||||
self._connected = False
|
||||
|
||||
async def _check_connection(self):
|
||||
"""Check Docker connection and detect API version."""
|
||||
if not self._connected:
|
||||
try:
|
||||
# Get Docker version info
|
||||
docker_info = await self._request("GET", "version", check_connection=False)
|
||||
self._connected = True
|
||||
|
||||
# Parse API version
|
||||
api_version = parse_version(docker_info['ApiVersion'])
|
||||
docker_version = docker_info["Version"]
|
||||
|
||||
logger.info(f"Connected to Docker {docker_version}, API {api_version}")
|
||||
|
||||
# Check minimum version requirement
|
||||
if api_version < parse_version(DOCKER_MINIMUM_API_VERSION):
|
||||
raise RuntimeError(
|
||||
f"Docker version is {docker_version}. "
|
||||
f"GNS3 requires a minimum API version of {DOCKER_MINIMUM_API_VERSION}"
|
||||
)
|
||||
|
||||
# Use preferred API version if supported
|
||||
preferred_api_version = parse_version(DOCKER_PREFERRED_API_VERSION)
|
||||
if api_version >= preferred_api_version:
|
||||
self._api_version = DOCKER_PREFERRED_API_VERSION
|
||||
logger.info(f"Using Docker API version {self._api_version}")
|
||||
else:
|
||||
# Use Docker daemon's actual API version
|
||||
self._api_version = docker_info['ApiVersion']
|
||||
logger.info(f"Using Docker API version {self._api_version} (daemon native)")
|
||||
|
||||
except (aiohttp.ClientError, FileNotFoundError) as e:
|
||||
self._connected = False
|
||||
raise RuntimeError(f"Can't connect to Docker daemon: {e}") from e
|
||||
except KeyError as e:
|
||||
raise RuntimeError(f"Unexpected Docker API response: missing {e}") from e
|
||||
|
||||
async def _request(self, method: str, endpoint: str, **kwargs):
|
||||
"""
|
||||
Send request to Docker API.
|
||||
|
||||
Args:
|
||||
method: HTTP method
|
||||
endpoint: API endpoint (without version prefix)
|
||||
**kwargs: Other parameters for aiohttp.request
|
||||
|
||||
Returns:
|
||||
Response JSON data
|
||||
"""
|
||||
# Check connection and version on first request
|
||||
check_connection = kwargs.pop('check_connection', True)
|
||||
if check_connection and not self._connected:
|
||||
await self._check_connection()
|
||||
|
||||
session = await self._get_session()
|
||||
url = f"http://docker/v{self._api_version}/{endpoint}"
|
||||
|
||||
try:
|
||||
async with asyncio.timeout(self.REQUEST_TIMEOUT):
|
||||
async with session.request(method, url, **kwargs) as response:
|
||||
if response.status >= 300:
|
||||
error_text = await response.text()
|
||||
raise RuntimeError(f"Docker API error {response.status}: {error_text}")
|
||||
if response.status == 204: # No Content
|
||||
return None
|
||||
return await response.json()
|
||||
except asyncio.TimeoutError:
|
||||
raise RuntimeError(f"Docker API timeout after {self.REQUEST_TIMEOUT}s for {endpoint}")
|
||||
except aiohttp.ClientError as e:
|
||||
raise RuntimeError(f"Docker connection error: {e}") from e
|
||||
|
||||
async def create_network(self, name: str, driver: str = "bridge", subnet: str = None):
|
||||
"""Create Docker network."""
|
||||
data = {
|
||||
"Name": name,
|
||||
"Driver": driver
|
||||
}
|
||||
if subnet:
|
||||
data["IPAM"] = {
|
||||
"Config": [{"Subnet": subnet}]
|
||||
}
|
||||
await self._request("POST", "networks/create", json=data)
|
||||
|
||||
async def get_network(self, name: str):
|
||||
"""Get network info."""
|
||||
try:
|
||||
return await self._request("GET", f"networks/{name}")
|
||||
except RuntimeError as e:
|
||||
if "404" in str(e):
|
||||
return None
|
||||
raise
|
||||
|
||||
async def create_container(self, name: str, image: str, **kwargs):
|
||||
"""Create container."""
|
||||
data = {
|
||||
"Image": image,
|
||||
"name": name,
|
||||
"HostConfig": {},
|
||||
"NetworkingConfig": {}
|
||||
}
|
||||
|
||||
# Handle network config
|
||||
if "network" in kwargs:
|
||||
network_name = kwargs.pop("network")
|
||||
data["NetworkingConfig"]["EndpointsConfig"] = {
|
||||
network_name: {}
|
||||
}
|
||||
|
||||
# Handle environment variables
|
||||
if "environment" in kwargs:
|
||||
data["Env"] = [f"{k}={v}" for k, v in kwargs.pop("environment").items()]
|
||||
|
||||
# Handle resource limits
|
||||
if "host_config" in kwargs:
|
||||
data["HostConfig"].update(kwargs.pop("host_config"))
|
||||
else:
|
||||
# Legacy parameter format
|
||||
host_config = {}
|
||||
if "mem_limit" in kwargs:
|
||||
host_config["Memory"] = kwargs.pop("mem_limit")
|
||||
if "cpu_quota" in kwargs:
|
||||
host_config["NanoCpus"] = kwargs.pop("cpu_quota")
|
||||
if "pids_limit" in kwargs:
|
||||
host_config["PidsLimit"] = kwargs.pop("pids_limit")
|
||||
if "restart_policy" in kwargs:
|
||||
host_config["RestartPolicy"] = kwargs.pop("restart_policy")
|
||||
data["HostConfig"].update(host_config)
|
||||
|
||||
# Handle health check
|
||||
if "health_config" in kwargs:
|
||||
data["HealthCheck"] = kwargs.pop("health_config")
|
||||
|
||||
# Create container
|
||||
result = await self._request("POST", "containers/create", params={"name": name}, json=data)
|
||||
return result["Id"]
|
||||
|
||||
async def start_container(self, container_id: str):
|
||||
"""Start container."""
|
||||
await self._request("POST", f"containers/{container_id}/start")
|
||||
|
||||
async def get_container(self, name: str):
|
||||
"""Get container info."""
|
||||
try:
|
||||
return await self._request("GET", f"containers/{name}/json")
|
||||
except RuntimeError as e:
|
||||
if "404" in str(e):
|
||||
return None
|
||||
raise
|
||||
|
||||
async def stop_container(self, container_id: str, timeout: int = 0):
|
||||
"""Stop container immediately (force kill, no graceful shutdown)."""
|
||||
await self._request("POST", f"containers/{container_id}/stop", params={"t": timeout})
|
||||
|
||||
async def remove_container(self, container_id: str, force: bool = False):
|
||||
"""Remove container."""
|
||||
params = {"force": "true"} if force else {}
|
||||
await self._request("DELETE", f"containers/{container_id}", params=params)
|
||||
|
||||
async def list_processes(self, container_name: str) -> list:
|
||||
"""Get process list from container.
|
||||
|
||||
Args:
|
||||
container_name: Container name (e.g., "gns3-wireshark-xxx")
|
||||
|
||||
Returns:
|
||||
List of process dicts with keys: PID, USER, COMMAND, etc.
|
||||
"""
|
||||
if not self._connected:
|
||||
await self._check_connection()
|
||||
|
||||
session = await self._get_session()
|
||||
url = f"http://docker/v{self._api_version}/containers/{container_name}/top?ps_args=aux"
|
||||
|
||||
try:
|
||||
async with asyncio.timeout(self.REQUEST_TIMEOUT):
|
||||
async with session.get(url) as response:
|
||||
if response.status >= 300:
|
||||
error_text = await response.text()
|
||||
raise RuntimeError(f"Docker API error {response.status}: {error_text}")
|
||||
result = await response.json()
|
||||
except asyncio.TimeoutError:
|
||||
raise RuntimeError(f"Docker API timeout after {self.REQUEST_TIMEOUT}s for containers/{container_name}/top")
|
||||
except aiohttp.ClientError as e:
|
||||
raise RuntimeError(f"Docker connection error: {e}") from e
|
||||
|
||||
# Docker API returns JSON format:
|
||||
# {"Processes": [["yueguob+", "259847", ...], [...]], "Titles": ["USER", "PID", "%CPU", ...]}
|
||||
# Headers are in "Titles" key, data rows are in "Processes"
|
||||
|
||||
headers = result.get("Titles", [])
|
||||
processes_data = result.get("Processes", [])
|
||||
if not headers or not processes_data:
|
||||
return []
|
||||
|
||||
processes = []
|
||||
for values in processes_data:
|
||||
if not values or len(values) < len(headers):
|
||||
continue
|
||||
|
||||
proc = {}
|
||||
for i, header in enumerate(headers):
|
||||
if i < len(values):
|
||||
proc[header] = values[i]
|
||||
|
||||
processes.append(proc)
|
||||
|
||||
return processes
|
||||
380
gns3server/agent/web_wireshark/manage_wireshark.py
Executable file
380
gns3server/agent/web_wireshark/manage_wireshark.py
Executable file
@ -0,0 +1,380 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# Copyright (C) 2026 YueGuobin
|
||||
#
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU General Public License as published by
|
||||
# the Free Software Foundation, either version 3 of the License, or
|
||||
# (at your option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
"""
|
||||
Web Wireshark Management Script (CLI Tool)
|
||||
|
||||
This script provides a command-line interface for managing Web Wireshark containers.
|
||||
It is intended for manual management, debugging, and testing purposes.
|
||||
|
||||
NOTE: For programmatic access within GNS3 server, use WebWiresharkManager class directly
|
||||
from gns3server.agent.web_wireshark.manager instead of calling this script via subprocess.
|
||||
|
||||
Usage Examples:
|
||||
# Start a Web Wireshark session
|
||||
python manage_wireshark.py start \\
|
||||
--project-id <uuid> \\
|
||||
--link-id <uuid> \\
|
||||
--jwt-token <token>
|
||||
|
||||
# Stop a Web Wireshark session
|
||||
python manage_wireshark.py stop \\
|
||||
--project-id <uuid> \\
|
||||
--link-id <uuid>
|
||||
|
||||
# Restart a Web Wireshark session
|
||||
python manage_wireshark.py restart \\
|
||||
--project-id <uuid> \\
|
||||
--link-id <uuid> \\
|
||||
--jwt-token <token>
|
||||
|
||||
# Stop all sessions for a project
|
||||
python manage_wireshark.py stop-all --project-id <uuid>
|
||||
|
||||
# Delete project container
|
||||
python manage_wireshark.py delete --project-id <uuid>
|
||||
|
||||
Available Commands:
|
||||
start Start a new Web Wireshark session
|
||||
stop Stop a Web Wireshark session
|
||||
restart Restart a Web Wireshark session
|
||||
stop-all Stop all Web Wireshark sessions for a project
|
||||
delete Delete the Web Wireshark container
|
||||
stop-container Stop container without deleting
|
||||
delete-container Delete the container
|
||||
|
||||
Output Format:
|
||||
All commands output JSON to stdout for easy parsing:
|
||||
- Success: {"status": "...", "ws_url": "...", ...}
|
||||
- Error: {"error": "message"} (to stderr)
|
||||
|
||||
For more information on each command, use:
|
||||
python manage_wireshark.py <command> --help
|
||||
"""
|
||||
|
||||
import sys
|
||||
import json
|
||||
import argparse
|
||||
import logging
|
||||
import asyncio
|
||||
import os
|
||||
from typing import Optional
|
||||
|
||||
from gns3server.utils.uuid_validator import validate_uuid
|
||||
|
||||
# Add parent directory to path for imports when run directly
|
||||
if __name__ == "__main__":
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))))
|
||||
|
||||
try:
|
||||
from gns3server.agent.web_wireshark.manager import WebWiresharkManager
|
||||
except ImportError:
|
||||
from manager import WebWiresharkManager
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def setup_logging(verbose: bool = False):
|
||||
"""Setup logging configuration."""
|
||||
level = logging.DEBUG if verbose else logging.INFO
|
||||
logging.basicConfig(
|
||||
level=level,
|
||||
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
|
||||
)
|
||||
|
||||
|
||||
async def cmd_start(args) -> int:
|
||||
"""Start Web Wireshark session.
|
||||
|
||||
Args:
|
||||
args: Command line arguments
|
||||
|
||||
Returns:
|
||||
Exit code (0 for success)
|
||||
"""
|
||||
manager = WebWiresharkManager()
|
||||
try:
|
||||
await manager.ensure_network()
|
||||
|
||||
result = await manager.start_wireshark_session(
|
||||
project_id=args.project_id,
|
||||
link_id=args.link_id,
|
||||
jwt_token=args.jwt_token,
|
||||
capture_stream_url=args.capture_url,
|
||||
image=args.image,
|
||||
memory=args.memory,
|
||||
cpus=args.cpus,
|
||||
pids_limit=args.pids_limit
|
||||
)
|
||||
|
||||
print(json.dumps(result, indent=2))
|
||||
return 0
|
||||
except Exception as e:
|
||||
logger.error(f"Error: {e}")
|
||||
print(json.dumps({"error": str(e)}), file=sys.stderr)
|
||||
return 1
|
||||
finally:
|
||||
await manager.close()
|
||||
|
||||
|
||||
async def cmd_stop(args) -> int:
|
||||
"""Stop Web Wireshark session.
|
||||
|
||||
Args:
|
||||
args: Command line arguments
|
||||
|
||||
Returns:
|
||||
Exit code (0 for success)
|
||||
"""
|
||||
manager = WebWiresharkManager()
|
||||
try:
|
||||
await manager.stop_wireshark_session(
|
||||
project_id=args.project_id,
|
||||
link_id=args.link_id
|
||||
)
|
||||
print(json.dumps({"status": "stopped"}))
|
||||
return 0
|
||||
except Exception as e:
|
||||
logger.error(f"Error: {e}")
|
||||
print(json.dumps({"error": str(e)}), file=sys.stderr)
|
||||
return 1
|
||||
finally:
|
||||
await manager.close()
|
||||
|
||||
|
||||
async def cmd_restart(args) -> int:
|
||||
"""Restart Web Wireshark session.
|
||||
|
||||
Args:
|
||||
args: Command line arguments
|
||||
|
||||
Returns:
|
||||
Exit code (0 for success)
|
||||
"""
|
||||
manager = WebWiresharkManager()
|
||||
try:
|
||||
result = await manager.restart_wireshark_session(
|
||||
project_id=args.project_id,
|
||||
link_id=args.link_id,
|
||||
jwt_token=args.jwt_token,
|
||||
capture_stream_url=args.capture_url
|
||||
)
|
||||
print(json.dumps(result, indent=2))
|
||||
return 0
|
||||
except Exception as e:
|
||||
logger.error(f"Error: {e}")
|
||||
print(json.dumps({"error": str(e)}), file=sys.stderr)
|
||||
return 1
|
||||
finally:
|
||||
await manager.close()
|
||||
|
||||
|
||||
async def cmd_stop_all(args) -> int:
|
||||
"""Stop all Web Wireshark sessions for a project.
|
||||
|
||||
Args:
|
||||
args: Command line arguments
|
||||
|
||||
Returns:
|
||||
Exit code (0 for success)
|
||||
"""
|
||||
manager = WebWiresharkManager()
|
||||
try:
|
||||
await manager.stop_all_sessions(project_id=args.project_id)
|
||||
print(json.dumps({"status": "all sessions stopped"}))
|
||||
return 0
|
||||
except Exception as e:
|
||||
logger.error(f"Error: {e}")
|
||||
print(json.dumps({"error": str(e)}), file=sys.stderr)
|
||||
return 1
|
||||
finally:
|
||||
await manager.close()
|
||||
|
||||
|
||||
async def cmd_stop_container(args) -> int:
|
||||
"""Stop Web Wireshark container (without deleting).
|
||||
|
||||
Args:
|
||||
args: Command line arguments
|
||||
|
||||
Returns:
|
||||
Exit code (0 for success)
|
||||
"""
|
||||
manager = WebWiresharkManager()
|
||||
try:
|
||||
await manager.stop_container(project_id=args.project_id)
|
||||
print(json.dumps({"status": "stopped"}))
|
||||
return 0
|
||||
except Exception as e:
|
||||
logger.error(f"Error: {e}")
|
||||
print(json.dumps({"error": str(e)}), file=sys.stderr)
|
||||
return 1
|
||||
finally:
|
||||
await manager.close()
|
||||
|
||||
|
||||
async def cmd_delete_container(args) -> int:
|
||||
"""Delete Web Wireshark container.
|
||||
|
||||
Args:
|
||||
args: Command line arguments
|
||||
|
||||
Returns:
|
||||
Exit code (0 for success)
|
||||
"""
|
||||
manager = WebWiresharkManager()
|
||||
try:
|
||||
await manager.delete_container(project_id=args.project_id)
|
||||
print(json.dumps({"status": "deleted"}))
|
||||
return 0
|
||||
except Exception as e:
|
||||
logger.error(f"Error: {e}")
|
||||
print(json.dumps({"error": str(e)}), file=sys.stderr)
|
||||
return 1
|
||||
finally:
|
||||
await manager.close()
|
||||
|
||||
|
||||
async def cmd_delete(args) -> int:
|
||||
"""Delete Web Wireshark container.
|
||||
|
||||
Args:
|
||||
args: Command line arguments
|
||||
|
||||
Returns:
|
||||
Exit code (0 for success)
|
||||
"""
|
||||
manager = WebWiresharkManager()
|
||||
try:
|
||||
await manager.delete_container(project_id=args.project_id)
|
||||
print(json.dumps({"status": "deleted"}))
|
||||
return 0
|
||||
except Exception as e:
|
||||
logger.error(f"Error: {e}")
|
||||
print(json.dumps({"error": str(e)}), file=sys.stderr)
|
||||
return 1
|
||||
finally:
|
||||
await manager.close()
|
||||
|
||||
|
||||
def create_parser() -> argparse.ArgumentParser:
|
||||
"""Create command line argument parser."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Web Wireshark container management"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--verbose", "-v",
|
||||
action="store_true",
|
||||
help="Enable verbose logging"
|
||||
)
|
||||
|
||||
subparsers = parser.add_subparsers(dest="command", help="Commands")
|
||||
|
||||
# Start command
|
||||
start_parser = subparsers.add_parser("start", help="Start Web Wireshark session")
|
||||
start_parser.add_argument("--project-id", required=True, type=validate_uuid, help="Project ID")
|
||||
start_parser.add_argument("--link-id", required=True, type=validate_uuid, help="Link ID")
|
||||
start_parser.add_argument("--jwt-token", required=True, help="JWT token")
|
||||
start_parser.add_argument(
|
||||
"--capture-url",
|
||||
help="Capture stream URL (auto-detected if not provided)"
|
||||
)
|
||||
start_parser.add_argument(
|
||||
"--image",
|
||||
default="gns3/web-wireshark:latest",
|
||||
help="Docker image (default: gns3/web-wireshark:latest)"
|
||||
)
|
||||
start_parser.add_argument(
|
||||
"--memory",
|
||||
default="2g",
|
||||
help="Memory limit (default: 2g)"
|
||||
)
|
||||
start_parser.add_argument(
|
||||
"--cpus",
|
||||
type=float,
|
||||
default=1.0,
|
||||
help="CPU cores (default: 1.0)"
|
||||
)
|
||||
start_parser.add_argument(
|
||||
"--pids-limit",
|
||||
type=int,
|
||||
default=1000,
|
||||
help="Process limit (default: 1000)"
|
||||
)
|
||||
|
||||
# Stop command
|
||||
stop_parser = subparsers.add_parser("stop", help="Stop Web Wireshark session")
|
||||
stop_parser.add_argument("--project-id", required=True, type=validate_uuid, help="Project ID")
|
||||
stop_parser.add_argument("--link-id", required=True, type=validate_uuid, help="Link ID")
|
||||
|
||||
# Restart command
|
||||
restart_parser = subparsers.add_parser("restart", help="Restart Web Wireshark session")
|
||||
restart_parser.add_argument("--project-id", required=True, type=validate_uuid, help="Project ID")
|
||||
restart_parser.add_argument("--link-id", required=True, type=validate_uuid, help="Link ID")
|
||||
restart_parser.add_argument("--jwt-token", required=True, help="JWT token")
|
||||
restart_parser.add_argument(
|
||||
"--capture-url",
|
||||
help="Capture stream URL (auto-detected if not provided)"
|
||||
)
|
||||
|
||||
# Stop all command
|
||||
stop_all_parser = subparsers.add_parser("stop-all", help="Stop all sessions")
|
||||
stop_all_parser.add_argument("--project-id", required=True, type=validate_uuid, help="Project ID")
|
||||
|
||||
# Delete command
|
||||
delete_parser = subparsers.add_parser("delete", help="Delete container")
|
||||
delete_parser.add_argument("--project-id", required=True, type=validate_uuid, help="Project ID")
|
||||
|
||||
# Stop container command
|
||||
stop_container_parser = subparsers.add_parser("stop-container", help="Stop container (without deleting)")
|
||||
stop_container_parser.add_argument("--project-id", required=True, type=validate_uuid, help="Project ID")
|
||||
|
||||
# Delete container command
|
||||
delete_container_parser = subparsers.add_parser("delete-container", help="Delete container")
|
||||
delete_container_parser.add_argument("--project-id", required=True, type=validate_uuid, help="Project ID")
|
||||
|
||||
return parser
|
||||
|
||||
|
||||
async def main() -> int:
|
||||
"""Main entry point."""
|
||||
parser = create_parser()
|
||||
args = parser.parse_args()
|
||||
|
||||
if not args.command:
|
||||
parser.print_help()
|
||||
return 1
|
||||
|
||||
setup_logging(args.verbose)
|
||||
|
||||
commands = {
|
||||
"start": cmd_start,
|
||||
"stop": cmd_stop,
|
||||
"restart": cmd_restart,
|
||||
"stop-all": cmd_stop_all,
|
||||
"delete": cmd_delete,
|
||||
"stop-container": cmd_stop_container,
|
||||
"delete-container": cmd_delete_container,
|
||||
}
|
||||
|
||||
return await commands[args.command](args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
exit_code = asyncio.run(main())
|
||||
sys.exit(exit_code)
|
||||
1041
gns3server/agent/web_wireshark/manager.py
Normal file
1041
gns3server/agent/web_wireshark/manager.py
Normal file
File diff suppressed because it is too large
Load Diff
269
gns3server/agent/web_wireshark/setup_wireshark_image.py
Normal file
269
gns3server/agent/web_wireshark/setup_wireshark_image.py
Normal file
@ -0,0 +1,269 @@
|
||||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# Copyright (C) 2026 YueGuobin
|
||||
#
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU General Public License as published by
|
||||
# the Free Software Foundation, either version 3 of the License, or
|
||||
# (at your option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
"""
|
||||
Setup Web Wireshark Docker image.
|
||||
|
||||
This script pulls or builds the gns3/web-wireshark Docker image.
|
||||
Run with: pip install gns3server[wireshark] && gns3-wireshark-setup
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import shutil
|
||||
import subprocess
|
||||
import argparse
|
||||
|
||||
|
||||
DOCKER_IMAGE = "gns3/web-wireshark:latest"
|
||||
DOCKERFILE_NAME = "Dockerfile"
|
||||
|
||||
|
||||
def find_dockerfile():
|
||||
"""Find the Dockerfile.
|
||||
|
||||
The Dockerfile is expected to be in the 'docker' subdirectory
|
||||
relative to this script's location.
|
||||
"""
|
||||
|
||||
# This script is in: gns3server/agent/web_wireshark/setup_wireshark_image.py
|
||||
# The Dockerfile is in: gns3server/agent/web_wireshark/docker/Dockerfile
|
||||
script_dir = os.path.dirname(os.path.abspath(__file__))
|
||||
dockerfile_path = os.path.join(script_dir, "docker", DOCKERFILE_NAME)
|
||||
|
||||
if os.path.exists(dockerfile_path):
|
||||
return dockerfile_path
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def check_docker():
|
||||
"""Check if Docker is available."""
|
||||
|
||||
if not shutil.which("docker"):
|
||||
print("Error: docker command not found", file=sys.stderr)
|
||||
print("Please install Docker first: https://docs.docker.com/get-docker/", file=sys.stderr)
|
||||
return False
|
||||
|
||||
# Check if Docker daemon is running
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["docker", "info"],
|
||||
capture_output=True,
|
||||
text=True
|
||||
)
|
||||
if result.returncode != 0:
|
||||
print("Error: Docker daemon is not running", file=sys.stderr)
|
||||
print("Please start Docker and try again", file=sys.stderr)
|
||||
return False
|
||||
except Exception as e:
|
||||
print(f"Error: Cannot connect to Docker: {e}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def image_exists():
|
||||
"""Check if the Docker image already exists."""
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["docker", "image", "inspect", DOCKER_IMAGE],
|
||||
capture_output=True,
|
||||
text=True
|
||||
)
|
||||
return result.returncode == 0
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def is_network_error(output):
|
||||
"""Check if the error is network-related."""
|
||||
|
||||
network_error_patterns = [
|
||||
"connection reset",
|
||||
"connection refused",
|
||||
"timeout",
|
||||
"no route to host",
|
||||
"network unreachable",
|
||||
"failed to fetch",
|
||||
"failed to authorize",
|
||||
"i/o timeout",
|
||||
]
|
||||
output_lower = output.lower()
|
||||
return any(pattern in output_lower for pattern in network_error_patterns)
|
||||
|
||||
|
||||
def pull_image():
|
||||
"""Pull the Docker image from registry."""
|
||||
|
||||
print(f"Pulling Docker image: {DOCKER_IMAGE}")
|
||||
print("-" * 60)
|
||||
|
||||
result = subprocess.run(
|
||||
["docker", "pull", DOCKER_IMAGE],
|
||||
capture_output=True,
|
||||
text=True
|
||||
)
|
||||
|
||||
# Store output for network error detection
|
||||
pull_output = result.stdout + result.stderr
|
||||
pull_success = result.returncode == 0
|
||||
|
||||
# Print output
|
||||
print(pull_output)
|
||||
|
||||
return pull_success, pull_output
|
||||
|
||||
|
||||
def build_image(dockerfile_path):
|
||||
"""Build the Docker image locally."""
|
||||
|
||||
dockerfile_dir = os.path.dirname(dockerfile_path)
|
||||
print(f"Building Docker image: {DOCKER_IMAGE}")
|
||||
print(f"Using Dockerfile: {dockerfile_path}")
|
||||
print("-" * 60)
|
||||
|
||||
result = subprocess.run(
|
||||
["docker", "build", "-t", DOCKER_IMAGE, "-f", dockerfile_path, "."],
|
||||
cwd=dockerfile_dir,
|
||||
pass_fds=(1, 2)
|
||||
)
|
||||
|
||||
return result.returncode == 0
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Setup Web Wireshark Docker image for GNS3"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--force",
|
||||
action="store_true",
|
||||
help="Force rebuild even if image exists"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--build-only",
|
||||
action="store_true",
|
||||
help="Only build locally, skip pull"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--pull-only",
|
||||
action="store_true",
|
||||
help="Only pull from registry, skip build"
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
print("=" * 60)
|
||||
print(" GNS3 Web Wireshark Image Setup")
|
||||
print("=" * 60)
|
||||
print()
|
||||
|
||||
# Check Docker
|
||||
if not check_docker():
|
||||
sys.exit(1)
|
||||
|
||||
# Check if image already exists
|
||||
if image_exists() and not args.force:
|
||||
print(f"Image {DOCKER_IMAGE} already exists.")
|
||||
response = input("Do you want to rebuild it? [y/N]: ").strip().lower()
|
||||
if response != 'y':
|
||||
print("Setup cancelled.")
|
||||
sys.exit(0)
|
||||
print()
|
||||
|
||||
# Try strategies in order
|
||||
success = False
|
||||
pull_failed_due_to_network = False
|
||||
|
||||
if not args.build_only:
|
||||
# Strategy 1: Pull from registry
|
||||
print("[Strategy 1/2] Attempting to pull image from Docker Hub...")
|
||||
print()
|
||||
pull_ok, pull_output = pull_image()
|
||||
if pull_ok:
|
||||
success = True
|
||||
print()
|
||||
print(f"Successfully pulled {DOCKER_IMAGE}")
|
||||
else:
|
||||
# Check if it's a network error
|
||||
if is_network_error(pull_output):
|
||||
pull_failed_due_to_network = True
|
||||
print()
|
||||
print("Pull failed due to network issues.")
|
||||
print("The local build will also fail since it requires pulling the base image from Docker Hub.")
|
||||
print()
|
||||
print("Suggestions:")
|
||||
print(" 1. Configure Docker mirror accelerator (see /etc/docker/daemon.json)")
|
||||
print(" 2. Use a VPN/proxy")
|
||||
print(" 3. Manually import the image on a machine with Docker Hub access:")
|
||||
print(f" docker save -o web-wireshark.tar {DOCKER_IMAGE}")
|
||||
print(" scp web-wireshark.tar your-server:/tmp/")
|
||||
print(" docker load -i /tmp/web-wireshark.tar")
|
||||
print()
|
||||
print("Skipping local build...")
|
||||
else:
|
||||
print()
|
||||
print("Pull failed, trying local build...")
|
||||
|
||||
if not success and not args.pull_only and not pull_failed_due_to_network:
|
||||
# Strategy 2: Build locally
|
||||
# Skip if pull failed due to network - local build will also fail
|
||||
print()
|
||||
print("[Strategy 2/2] Attempting local build...")
|
||||
print()
|
||||
|
||||
dockerfile_path = find_dockerfile()
|
||||
if not dockerfile_path:
|
||||
print("Error: Cannot find Dockerfile", file=sys.stderr)
|
||||
print("Please ensure the GNS3 server package is correctly installed.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if build_image(dockerfile_path):
|
||||
success = True
|
||||
print()
|
||||
print(f"Successfully built {DOCKER_IMAGE}")
|
||||
else:
|
||||
print()
|
||||
print("Build failed.")
|
||||
|
||||
if success:
|
||||
print()
|
||||
print("=" * 60)
|
||||
print(" Setup completed successfully!")
|
||||
print("=" * 60)
|
||||
sys.exit(0)
|
||||
else:
|
||||
print()
|
||||
print("=" * 60)
|
||||
print(" Setup failed!")
|
||||
print("=" * 60)
|
||||
print()
|
||||
if pull_failed_due_to_network:
|
||||
print("Docker Hub is not accessible. Please fix the network issue and try again.")
|
||||
else:
|
||||
print("Please check the errors above and try again.")
|
||||
print("You can also manually run:")
|
||||
print(f" docker pull {DOCKER_IMAGE}")
|
||||
print(f" docker build -t {DOCKER_IMAGE} -f <dockerfile_path> .")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
134
gns3server/agent/web_wireshark/stats.py
Normal file
134
gns3server/agent/web_wireshark/stats.py
Normal file
@ -0,0 +1,134 @@
|
||||
"""
|
||||
Web Wireshark statistics collection utilities.
|
||||
|
||||
This module provides functions to collect and aggregate statistics
|
||||
about Web Wireshark containers and sessions.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import subprocess
|
||||
from typing import Dict, List, Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def collect_webwireshark_stats(projects: List) -> Dict:
|
||||
"""
|
||||
Collect Web Wireshark container statistics across all projects.
|
||||
|
||||
Args:
|
||||
projects: List of project objects to check for containers
|
||||
|
||||
Returns:
|
||||
Dictionary with aggregated statistics:
|
||||
{
|
||||
"total_containers": int,
|
||||
"running_containers": int,
|
||||
"active_sessions": int,
|
||||
"containers": [list of container details]
|
||||
}
|
||||
"""
|
||||
from .manager import WebWiresharkManager
|
||||
|
||||
stats = {
|
||||
"total_containers": 0,
|
||||
"running_containers": 0,
|
||||
"active_sessions": 0,
|
||||
"containers": []
|
||||
}
|
||||
|
||||
# Create a single manager instance and reuse it
|
||||
manager = WebWiresharkManager()
|
||||
|
||||
try:
|
||||
# Check Web Wireshark containers for each opened project
|
||||
for project in projects:
|
||||
if project.status != "opened":
|
||||
continue
|
||||
|
||||
container_name = f"gns3-wireshark-{project.id}"
|
||||
container = await manager.docker.get_container(container_name)
|
||||
|
||||
if container:
|
||||
stats["total_containers"] += 1
|
||||
|
||||
container_info = {
|
||||
"project_id": project.id,
|
||||
"project_name": project.name,
|
||||
"container_id": container["Id"][:12],
|
||||
"status": container["State"]["Status"],
|
||||
"running": container["State"]["Running"],
|
||||
}
|
||||
|
||||
if container["State"]["Running"]:
|
||||
stats["running_containers"] += 1
|
||||
|
||||
# Get container resource limits from HostConfig
|
||||
host_config = container.get("HostConfig", {})
|
||||
memory_limit = host_config.get("Memory", 0)
|
||||
cpu_quota = host_config.get("NanoCpus", 0)
|
||||
pids_limit = host_config.get("PidsLimit", 0)
|
||||
|
||||
# Count active capture sessions
|
||||
active_sessions = sum(
|
||||
1 for link in project.links.values()
|
||||
if getattr(link, "capturing", False)
|
||||
)
|
||||
stats["active_sessions"] += active_sessions
|
||||
container_info["active_sessions"] = active_sessions
|
||||
|
||||
# Add resource limits
|
||||
container_info["memory_limit"] = f"{memory_limit / (1024**3):.1f} GB" if memory_limit > 0 else "unlimited"
|
||||
container_info["cpu_limit"] = f"{cpu_quota / 1000000000:.1f}" if cpu_quota > 0 else "unlimited"
|
||||
container_info["pids_limit"] = pids_limit if pids_limit > 0 else "unlimited"
|
||||
|
||||
# Get resource usage via docker stats
|
||||
resource_stats = await _get_container_resource_stats(container["Id"])
|
||||
if resource_stats:
|
||||
container_info.update(resource_stats)
|
||||
|
||||
stats["containers"].append(container_info)
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not retrieve Web Wireshark statistics: {e}")
|
||||
|
||||
finally:
|
||||
# Always close the manager to cleanup aiohttp sessions
|
||||
await manager.close()
|
||||
|
||||
return stats
|
||||
|
||||
|
||||
async def _get_container_resource_stats(container_id: str) -> Optional[Dict]:
|
||||
"""
|
||||
Get resource usage statistics for a container.
|
||||
|
||||
Args:
|
||||
container_id: Docker container ID
|
||||
|
||||
Returns:
|
||||
Dictionary with memory, cpu, and pids, or None if failed
|
||||
"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["docker", "stats", "--no-stream", "--format",
|
||||
"{{.MemUsage}}\t{{.CPUPerc}}\t{{.PIDs}}",
|
||||
container_id],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=2
|
||||
)
|
||||
if result.returncode == 0:
|
||||
parts = result.stdout.strip().split("\t")
|
||||
if len(parts) >= 3:
|
||||
return {
|
||||
"memory": parts[0],
|
||||
"cpu": parts[1],
|
||||
"pids": int(parts[2])
|
||||
}
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.debug(f"Docker stats timeout for container {container_id[:12]}")
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to get stats for container {container_id[:12]}: {e}")
|
||||
|
||||
return None
|
||||
@ -292,6 +292,41 @@ async def delete_session(
|
||||
)
|
||||
|
||||
|
||||
@router.post(
|
||||
"/sessions/{session_id}/abort",
|
||||
status_code=status.HTTP_200_OK,
|
||||
summary="Abort a streaming session",
|
||||
description="Abort an ongoing streaming session for a specific session."
|
||||
)
|
||||
async def abort_session(
|
||||
session_id: str,
|
||||
project: Project = Depends(dep_project),
|
||||
current_user: schemas.User = Depends(get_current_active_user),
|
||||
):
|
||||
"""
|
||||
Abort a streaming session.
|
||||
|
||||
Sets the abort flag for the session, which will be checked on the next
|
||||
conditional edge evaluation. The streaming will stop at that point.
|
||||
"""
|
||||
|
||||
# Check if project is opened
|
||||
if project.status != "opened":
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_403_FORBIDDEN,
|
||||
detail=f"Project must be opened to abort chat session. Current status: {project.status}"
|
||||
)
|
||||
|
||||
# Get AgentService for this project
|
||||
agent_manager = await get_project_agent_manager()
|
||||
agent_service = await agent_manager.get_agent(str(project.id), project.path)
|
||||
|
||||
# Abort the session
|
||||
agent_service.abort_session(session_id)
|
||||
|
||||
return {"status": "ok", "session_id": session_id}
|
||||
|
||||
|
||||
@router.patch(
|
||||
"/sessions/{session_id}",
|
||||
response_model=schemas.ChatSession,
|
||||
|
||||
@ -28,6 +28,7 @@ from typing import List, Dict
|
||||
|
||||
from gns3server.config import Config
|
||||
from gns3server.controller import Controller
|
||||
from gns3server.agent.web_wireshark.stats import collect_webwireshark_stats
|
||||
from gns3server.version import __version__
|
||||
from gns3server.controller.controller_error import ControllerError, ControllerForbiddenError
|
||||
from gns3server import schemas
|
||||
@ -241,11 +242,15 @@ async def statistics() -> dict:
|
||||
"capturing": link_capturing,
|
||||
}
|
||||
|
||||
# Web Wireshark container statistics
|
||||
webwireshark_stats = await collect_webwireshark_stats(projects)
|
||||
|
||||
return {
|
||||
"computes": compute_statistics,
|
||||
"projects": project_stats,
|
||||
"nodes": node_stats,
|
||||
"links": link_stats,
|
||||
"webwireshark": webwireshark_stats,
|
||||
}
|
||||
|
||||
|
||||
|
||||
@ -90,7 +90,16 @@ async def get_current_active_user_from_websocket(
|
||||
user_repo: UsersRepository = Depends(get_repository(UsersRepository)),
|
||||
) -> Optional[schemas.User]:
|
||||
|
||||
await websocket.accept()
|
||||
# Extract requested subprotocols from headers for proper WebSocket negotiation
|
||||
# This is critical for protocols like xpra that require specific subprotocols
|
||||
scope = websocket.scope
|
||||
headers = dict(scope.get("headers", []))
|
||||
requested_protocols_header = headers.get(b"sec-websocket-protocol", b"")
|
||||
requested_protocols = [p.decode().strip() for p in requested_protocols_header.split(b",") if p.strip()]
|
||||
|
||||
# Accept the connection with the first requested subprotocol (if any)
|
||||
subprotocol = requested_protocols[0] if requested_protocols else None
|
||||
await websocket.accept(subprotocol=subprotocol)
|
||||
|
||||
try:
|
||||
token_data = auth_service.get_token_data(token)
|
||||
|
||||
@ -18,11 +18,13 @@
|
||||
API routes for links.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import multidict
|
||||
import aiohttp
|
||||
|
||||
from fastapi import APIRouter, Depends, Request, status
|
||||
from fastapi.responses import StreamingResponse
|
||||
from fastapi import APIRouter, Depends, Request, status, WebSocket
|
||||
from fastapi.responses import FileResponse, StreamingResponse
|
||||
from fastapi.encoders import jsonable_encoder
|
||||
from typing import List, Union
|
||||
from uuid import UUID
|
||||
@ -32,10 +34,12 @@ from gns3server.controller.controller_error import ControllerError
|
||||
from gns3server.db.repositories.rbac import RbacRepository
|
||||
from gns3server.controller.link import Link
|
||||
from gns3server.utils.http_client import HTTPClient
|
||||
from gns3server.utils.port_allocator import link_id_to_port
|
||||
from gns3server.utils.websocket_to_websocket import websocket_proxy
|
||||
from gns3server import schemas
|
||||
|
||||
from .dependencies.database import get_repository
|
||||
from .dependencies.rbac import has_privilege
|
||||
from .dependencies.rbac import has_privilege, has_privilege_on_websocket
|
||||
|
||||
import logging
|
||||
|
||||
@ -214,16 +218,26 @@ async def reset_link(link: Link = Depends(dep_link)) -> schemas.Link:
|
||||
response_model=schemas.Link,
|
||||
dependencies=[Depends(has_privilege("Link.Capture"))]
|
||||
)
|
||||
async def start_capture(capture_data: dict, link: Link = Depends(dep_link)) -> schemas.Link:
|
||||
async def start_capture(
|
||||
capture_data: schemas.LinkCapture,
|
||||
http_request: Request,
|
||||
link: Link = Depends(dep_link)
|
||||
) -> schemas.Link:
|
||||
"""
|
||||
Start packet capture on the link.
|
||||
|
||||
Required privilege: Link.Capture
|
||||
"""
|
||||
|
||||
# Extract JWT token from Authorization header
|
||||
auth_header = http_request.headers.get("Authorization", "")
|
||||
jwt_token = auth_header.replace("Bearer ", "") if auth_header else None
|
||||
|
||||
await link.start_capture(
|
||||
data_link_type=capture_data.get("data_link_type", "DLT_EN10MB"),
|
||||
capture_file_name=capture_data.get("capture_file_name"),
|
||||
data_link_type=capture_data.data_link_type,
|
||||
capture_file_name=capture_data.capture_file_name,
|
||||
wireshark=capture_data.wireshark,
|
||||
jwt_token=jwt_token
|
||||
)
|
||||
return link.asdict()
|
||||
|
||||
@ -243,6 +257,33 @@ async def stop_capture(link: Link = Depends(dep_link)) -> None:
|
||||
await link.stop_capture()
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{link_id}/capture/wireshark/restart",
|
||||
status_code=status.HTTP_200_OK,
|
||||
dependencies=[Depends(has_privilege("Link.Capture"))]
|
||||
)
|
||||
async def restart_wireshark(
|
||||
http_request: Request,
|
||||
link: Link = Depends(dep_link)
|
||||
) -> dict:
|
||||
"""
|
||||
Restart Wireshark window without stopping the capture.
|
||||
|
||||
This allows recovery after accidentally closing the Wireshark window.
|
||||
|
||||
Required privilege: Link.Capture
|
||||
"""
|
||||
# Extract JWT token from Authorization header
|
||||
auth_header = http_request.headers.get("Authorization", "")
|
||||
jwt_token = auth_header.replace("Bearer ", "") if auth_header else None
|
||||
|
||||
if not jwt_token:
|
||||
raise ControllerError("JWT token is required for Web Wireshark restart")
|
||||
|
||||
await link._restart_web_wireshark(jwt_token)
|
||||
return {"status": "restarted"}
|
||||
|
||||
|
||||
@router.get(
|
||||
"/{link_id}/capture/stream",
|
||||
dependencies=[Depends(has_privilege("Link.Capture"))]
|
||||
@ -254,7 +295,10 @@ async def stream_pcap(request: Request, link: Link = Depends(dep_link)) -> Strea
|
||||
Required privilege: Link.Capture
|
||||
"""
|
||||
|
||||
if not link.capturing:
|
||||
# Check both capturing flag and capture_node to avoid race condition
|
||||
# when stop_capture() sets _capture_node = None before this check completes
|
||||
if not link.capturing or not link.capture_node:
|
||||
log.info(f"Stream pcap ended for link {link.id}: capture stopped before stream completed")
|
||||
raise ControllerError("This link has no active packet capture")
|
||||
|
||||
compute = link.compute
|
||||
@ -287,6 +331,105 @@ async def stream_pcap(request: Request, link: Link = Depends(dep_link)) -> Strea
|
||||
return StreamingResponse(compute_pcap_stream(), media_type="application/vnd.tcpdump.pcap")
|
||||
|
||||
|
||||
@router.get(
|
||||
"/{link_id}/capture/file",
|
||||
dependencies=[Depends(has_privilege("Link.Capture"))],
|
||||
response_class=FileResponse
|
||||
)
|
||||
async def download_capture_file(link: Link = Depends(dep_link)):
|
||||
"""
|
||||
Download the PCAP capture file.
|
||||
|
||||
This endpoint allows downloading the capture file even while capture is active.
|
||||
The file is streamed directly, so partial data may be received if capture is still running.
|
||||
|
||||
Required privilege: Link.Capture
|
||||
"""
|
||||
if not link.capture_file_path:
|
||||
raise ControllerError("No capture file path set for this link")
|
||||
|
||||
if not os.path.exists(link.capture_file_path):
|
||||
raise ControllerError(f"Capture file not found: {link.capture_file_path}")
|
||||
|
||||
return FileResponse(
|
||||
path=link.capture_file_path,
|
||||
filename=os.path.basename(link.capture_file_path),
|
||||
media_type="application/vnd.tcpdump.pcap"
|
||||
)
|
||||
|
||||
|
||||
@router.websocket("/{link_id}/capture/web-wireshark")
|
||||
async def web_wireshark_websocket(
|
||||
websocket: WebSocket,
|
||||
link_id: str,
|
||||
project_id: str,
|
||||
current_user: schemas.User = Depends(has_privilege_on_websocket("Link.Capture"))
|
||||
):
|
||||
"""
|
||||
WebSocket proxy endpoint for xpra container (Web Wireshark).
|
||||
|
||||
Path: ws://host/v3/projects/{project_id}/links/{link_id}/capture/web-wireshark?token=<jwt_token>
|
||||
|
||||
Required privilege: Link.Capture
|
||||
|
||||
Note: The WebSocket connection is accepted by the authentication dependency
|
||||
(get_current_active_user_from_websocket) with the proper subprotocol negotiation.
|
||||
"""
|
||||
log.info(f"New WebSocket connection for project {project_id}, link {link_id}, user {current_user.username}")
|
||||
|
||||
try:
|
||||
# Get container information
|
||||
container_name = f"gns3-wireshark-{project_id}"
|
||||
|
||||
# Calculate xpra port (using deterministic hash)
|
||||
xpra_port = link_id_to_port(link_id)
|
||||
|
||||
# Get container IP
|
||||
from gns3server.compute.docker import Docker
|
||||
|
||||
docker_manager = Docker.instance()
|
||||
container_info = await docker_manager.query(
|
||||
"GET",
|
||||
f"containers/{container_name}/json"
|
||||
)
|
||||
|
||||
networks = container_info["NetworkSettings"]["Networks"]
|
||||
container_ip = None
|
||||
|
||||
# Find gns3-wireshark network
|
||||
for network_name, network_config in networks.items():
|
||||
if "wireshark" in network_name.lower():
|
||||
container_ip = network_config["IPAddress"]
|
||||
break
|
||||
|
||||
if not container_ip:
|
||||
log.error(f"Container {container_name} not found in wireshark network")
|
||||
await websocket.close(code=status.WS_1011_INTERNAL_ERROR)
|
||||
return
|
||||
|
||||
# Build container WebSocket URL
|
||||
container_ws_url = f"ws://{container_ip}:{xpra_port}"
|
||||
log.info(f"Proxying WebSocket to container: {container_ws_url}")
|
||||
|
||||
# Get client's requested subprotocols from request headers
|
||||
scope = websocket.scope
|
||||
headers = dict(scope.get("headers", []))
|
||||
requested_protocols_header = headers.get(b"sec-websocket-protocol", b"")
|
||||
requested_protocols = [p.decode().strip() for p in requested_protocols_header.split(b",") if p.strip()]
|
||||
log.info(f"Client requested subprotocols: {requested_protocols}")
|
||||
|
||||
# The WebSocket connection has already been accepted by the authentication dependency
|
||||
# with the proper subprotocol. Now we just proxy data to the backend.
|
||||
await websocket_proxy(websocket, container_ws_url, requested_protocols)
|
||||
|
||||
except Exception as e:
|
||||
log.error(f"Error in WebSocket proxy for link {link_id}: {e}")
|
||||
try:
|
||||
await websocket.close(code=status.WS_1011_INTERNAL_ERROR, reason=str(e))
|
||||
except:
|
||||
pass
|
||||
|
||||
|
||||
@router.get(
|
||||
"/{link_id}/iface",
|
||||
response_model=Union[schemas.UDPPortInfo, schemas.EthernetPortInfo],
|
||||
|
||||
@ -160,4 +160,19 @@ enable_hardware_acceleration = True
|
||||
; Require hardware acceleration in order to start VMs
|
||||
require_hardware_acceleration = False
|
||||
; Allow unsafe additional command line options
|
||||
allow_unsafe_options = False
|
||||
allow_unsafe_options = False
|
||||
|
||||
[WebWireshark]
|
||||
; Enable Web Wireshark feature (container-based Wireshark in browser)
|
||||
enabled = True
|
||||
; Docker image for Web Wireshark container
|
||||
image = gns3/web-wireshark:latest
|
||||
; Docker network subnet for Web Wireshark containers (default: 172.31.0.0/22)
|
||||
; Change this if it conflicts with your existing network
|
||||
network_subnet = 172.31.0.0/22
|
||||
; Memory limit per container (e.g., "1g", "2g", "512m")
|
||||
memory = 2g
|
||||
; CPU cores per container (e.g., 1.0, 2.0)
|
||||
cpus = 1.0
|
||||
; Process limit per container
|
||||
pids_limit = 1000
|
||||
@ -50,6 +50,9 @@ class ApplianceToTemplate:
|
||||
if "symbol" in appliance_config:
|
||||
new_template["symbol"] = appliance_config.get("symbol")
|
||||
|
||||
if "tags" in appliance_config:
|
||||
new_template["tags"] = appliance_config.get("tags")
|
||||
|
||||
if new_template.get("symbol") is None:
|
||||
if appliance_config["category"] == "guest":
|
||||
if "docker" in appliance_config:
|
||||
|
||||
@ -21,6 +21,8 @@ import uuid
|
||||
import html
|
||||
|
||||
from .controller_error import ControllerError, ControllerNotFoundError
|
||||
from gns3server.agent.web_wireshark.manager import WebWiresharkManager
|
||||
from gns3server.config import Config
|
||||
|
||||
import logging
|
||||
|
||||
@ -86,6 +88,7 @@ class Link:
|
||||
self._suspended = False
|
||||
self._filters = {}
|
||||
self._link_style = {}
|
||||
self._wireshark = False
|
||||
|
||||
@property
|
||||
def filters(self):
|
||||
@ -94,13 +97,6 @@ class Link:
|
||||
"""
|
||||
return self._filters
|
||||
|
||||
@property
|
||||
def nodes(self):
|
||||
"""
|
||||
Get the current nodes attached to this link
|
||||
"""
|
||||
return self._nodes
|
||||
|
||||
@property
|
||||
def project(self):
|
||||
"""
|
||||
@ -284,10 +280,14 @@ class Link:
|
||||
|
||||
raise NotImplementedError
|
||||
|
||||
async def start_capture(self, data_link_type="DLT_EN10MB", capture_file_name=None):
|
||||
async def start_capture(self, data_link_type="DLT_EN10MB", capture_file_name=None, wireshark=False, jwt_token=None):
|
||||
"""
|
||||
Start capture on the link
|
||||
|
||||
:param data_link_type: PCAP data link type
|
||||
:param capture_file_name: PCAP capture file name
|
||||
:param wireshark: Enable Web Wireshark
|
||||
:param jwt_token: JWT token for authentication
|
||||
:returns: Capture object
|
||||
"""
|
||||
|
||||
@ -295,14 +295,124 @@ class Link:
|
||||
self._capture_file_name = capture_file_name
|
||||
self._project.emit_notification("link.updated", self.asdict())
|
||||
|
||||
# Start Web Wireshark if requested
|
||||
if wireshark:
|
||||
await self._start_web_wireshark(jwt_token)
|
||||
|
||||
async def stop_capture(self):
|
||||
"""
|
||||
Stop capture on the link
|
||||
"""
|
||||
|
||||
# Stop Web Wireshark
|
||||
if self._capturing:
|
||||
await self._stop_web_wireshark()
|
||||
|
||||
self._capturing = False
|
||||
self._project.emit_notification("link.updated", self.asdict())
|
||||
|
||||
async def _start_web_wireshark(self, jwt_token: str):
|
||||
"""Start Web Wireshark.
|
||||
|
||||
Args:
|
||||
jwt_token: JWT authentication token
|
||||
|
||||
Raises:
|
||||
ControllerError: If startup fails
|
||||
"""
|
||||
if not jwt_token:
|
||||
raise ControllerError("JWT token is required for Web Wireshark")
|
||||
|
||||
# Load WebWireshark configuration from config file
|
||||
webwireshark_config = Config.instance().settings.WebWireshark
|
||||
|
||||
manager = WebWiresharkManager()
|
||||
try:
|
||||
log.info(f"Starting Web Wireshark for link {self.id}")
|
||||
|
||||
result = await manager.start_wireshark_session(
|
||||
project_id=self._project.id,
|
||||
link_id=self.id,
|
||||
jwt_token=jwt_token,
|
||||
memory=webwireshark_config.memory,
|
||||
cpus=webwireshark_config.cpus,
|
||||
pids_limit=webwireshark_config.pids_limit
|
||||
)
|
||||
|
||||
# Send notification
|
||||
self._project.emit_notification("link.web_wireshark_started", {
|
||||
"link_id": self.id,
|
||||
"ws_url": result.get("ws_url", result.get("url"))
|
||||
})
|
||||
|
||||
log.info(f"Web Wireshark started for link {self.id}: {result.get('ws_url')}")
|
||||
self._wireshark = True
|
||||
|
||||
except ControllerError:
|
||||
# Re-raise ControllerError to return error to client
|
||||
raise
|
||||
except Exception as e:
|
||||
error_msg = f"Error starting Web Wireshark: {str(e)}"
|
||||
log.error(error_msg)
|
||||
raise ControllerError(error_msg)
|
||||
finally:
|
||||
await manager.close()
|
||||
|
||||
async def _stop_web_wireshark(self):
|
||||
"""Stop Web Wireshark"""
|
||||
manager = WebWiresharkManager()
|
||||
try:
|
||||
log.info(f"Stopping Web Wireshark for link {self.id}")
|
||||
|
||||
await manager.stop_wireshark_session(
|
||||
project_id=self._project.id,
|
||||
link_id=self.id
|
||||
)
|
||||
|
||||
log.info(f"Web Wireshark stopped for link {self.id}")
|
||||
self._wireshark = False
|
||||
|
||||
except Exception as e:
|
||||
log.error(f"Error stopping Web Wireshark: {e}")
|
||||
finally:
|
||||
await manager.close()
|
||||
|
||||
async def _restart_web_wireshark(self, jwt_token: str):
|
||||
"""Restart Web Wireshark after window was closed.
|
||||
|
||||
Args:
|
||||
jwt_token: JWT authentication token
|
||||
|
||||
Raises:
|
||||
ControllerError: If restart fails
|
||||
"""
|
||||
# Get capture stream URL from compute
|
||||
manager = WebWiresharkManager()
|
||||
try:
|
||||
log.info(f"Restarting Web Wireshark for link {self.id}")
|
||||
|
||||
result = await manager.restart_wireshark_session(
|
||||
project_id=self._project.id,
|
||||
link_id=self.id,
|
||||
jwt_token=jwt_token
|
||||
)
|
||||
|
||||
self._project.emit_notification("link.web_wireshark_started", {
|
||||
"link_id": self.id,
|
||||
"ws_url": result.get("ws_url", result.get("url"))
|
||||
})
|
||||
|
||||
log.info(f"Web Wireshark restarted for link {self.id}: {result.get('ws_url')}")
|
||||
|
||||
except ControllerError:
|
||||
raise
|
||||
except Exception as e:
|
||||
error_msg = f"Error restarting Web Wireshark: {str(e)}"
|
||||
log.error(error_msg)
|
||||
raise ControllerError(error_msg)
|
||||
finally:
|
||||
await manager.close()
|
||||
|
||||
def pcap_streaming_url(self):
|
||||
"""
|
||||
Get the PCAP streaming URL on compute
|
||||
@ -458,4 +568,5 @@ class Link:
|
||||
"filters": self._filters,
|
||||
"suspend": self._suspended,
|
||||
"link_style": self._link_style,
|
||||
"wireshark": self._wireshark,
|
||||
}
|
||||
|
||||
@ -47,6 +47,7 @@ from ..utils.asyncio import wait_run_in_executor
|
||||
from .export_project import export_project
|
||||
from .import_project import import_project, update_snapshots, regenerate_topology_ids
|
||||
from .controller_error import ControllerError, ControllerForbiddenError, ControllerNotFoundError
|
||||
from gns3server.agent.web_wireshark.manager import WebWiresharkManager
|
||||
|
||||
import logging
|
||||
|
||||
@ -897,6 +898,9 @@ class Project:
|
||||
if not ignore_notification:
|
||||
self.emit_controller_notification("project.closed", self.asdict())
|
||||
|
||||
# Stop Web Wireshark container (all xpra sessions terminate with container)
|
||||
await self._stop_web_wireshark_container()
|
||||
|
||||
# Cleanup GNS3 Copilot AgentService for this project
|
||||
await self._cleanup_copilot_agent()
|
||||
|
||||
@ -936,6 +940,70 @@ class Project:
|
||||
except OSError as e:
|
||||
log.warning(f"Could not delete unused pictures: {e}")
|
||||
|
||||
async def _cleanup_web_wireshark_xpra_sessions(self):
|
||||
"""
|
||||
Cleanup all Web Wireshark xpra sessions (without deleting container).
|
||||
|
||||
Called when project is closed to stop all xpra sessions and Wireshark processes,
|
||||
while keeping the container for quick reuse when project is reopened.
|
||||
"""
|
||||
try:
|
||||
log.info("Stopping xpra sessions for project '%s' (%s)", self.name, self._id)
|
||||
|
||||
manager = WebWiresharkManager()
|
||||
try:
|
||||
await manager.stop_all_sessions(self._id)
|
||||
log.info("Web Wireshark xpra sessions stopped successfully")
|
||||
finally:
|
||||
await manager.close()
|
||||
|
||||
except Exception as e:
|
||||
# Don't raise exception to avoid affecting project close flow
|
||||
log.warning("Failed to cleanup xpra sessions for project '%s': %s", self.name, e)
|
||||
|
||||
async def _stop_web_wireshark_container(self):
|
||||
"""
|
||||
Stop Web Wireshark container (without deleting).
|
||||
|
||||
Called when project is closed to stop the container and free memory,
|
||||
while keeping the container for quick startup when project is reopened.
|
||||
"""
|
||||
try:
|
||||
container_name = f"gns3-wireshark-{self._id}"
|
||||
log.info("Stopping Web Wireshark container '%s' for project '%s'", container_name, self.name)
|
||||
|
||||
manager = WebWiresharkManager()
|
||||
try:
|
||||
await manager.stop_container(self._id)
|
||||
log.info("Web Wireshark container stopped successfully")
|
||||
finally:
|
||||
await manager.close()
|
||||
|
||||
except Exception as e:
|
||||
# Don't fail project close if container stop fails
|
||||
log.warning("Failed to stop container for project '%s': %s", self.name, e)
|
||||
|
||||
async def _cleanup_web_wireshark_container(self):
|
||||
"""
|
||||
Delete Web Wireshark container.
|
||||
|
||||
Called when project is deleted to stop and remove the container.
|
||||
"""
|
||||
try:
|
||||
container_name = f"gns3-wireshark-{self._id}"
|
||||
log.info("Deleting Web Wireshark container '%s' for project '%s'", container_name, self.name)
|
||||
|
||||
manager = WebWiresharkManager()
|
||||
try:
|
||||
await manager.delete_container(self._id)
|
||||
log.info("Web Wireshark container deleted successfully")
|
||||
finally:
|
||||
await manager.close()
|
||||
|
||||
except Exception as e:
|
||||
# Don't fail project delete if container cleanup fails
|
||||
log.warning("Failed to delete container for project '%s': %s", self.name, e)
|
||||
|
||||
async def _cleanup_copilot_agent(self):
|
||||
"""
|
||||
Cleanup GNS3 Copilot AgentService for this project.
|
||||
@ -947,11 +1015,11 @@ class Project:
|
||||
|
||||
agent_manager = await get_project_agent_manager()
|
||||
if agent_manager.has_agent(self._id):
|
||||
log.info(f"Cleaning up AgentService for project '{self.name}' ({self._id})")
|
||||
log.info("Cleaning up AgentService for project '%s' (%s)", self.name, self._id)
|
||||
await agent_manager.remove_agent(self._id)
|
||||
except Exception as e:
|
||||
# Don't fail project close if agent cleanup fails
|
||||
log.warning(f"Failed to cleanup AgentService for project '{self.name}': {e}")
|
||||
log.warning("Failed to cleanup AgentService for project '%s': %s", self.name, e)
|
||||
|
||||
async def delete(self):
|
||||
|
||||
@ -963,6 +1031,10 @@ class Project:
|
||||
log.warning(f"Conflict while deleting project: {e}")
|
||||
await self.delete_on_computes()
|
||||
await self.close()
|
||||
|
||||
# Delete Web Wireshark container
|
||||
await self._cleanup_web_wireshark_container()
|
||||
|
||||
try:
|
||||
project_directory = get_default_project_directory()
|
||||
if not os.path.commonprefix([project_directory, self.path]) == project_directory:
|
||||
|
||||
@ -175,7 +175,7 @@ class UDPLink(Link):
|
||||
await self.delete()
|
||||
await self.create()
|
||||
|
||||
async def start_capture(self, data_link_type="DLT_EN10MB", capture_file_name=None):
|
||||
async def start_capture(self, data_link_type="DLT_EN10MB", capture_file_name=None, wireshark=False, jwt_token=None):
|
||||
"""
|
||||
Start capture on a link
|
||||
"""
|
||||
@ -189,7 +189,7 @@ class UDPLink(Link):
|
||||
),
|
||||
data=data,
|
||||
)
|
||||
await super().start_capture(data_link_type=data_link_type, capture_file_name=capture_file_name)
|
||||
await super().start_capture(data_link_type=data_link_type, capture_file_name=capture_file_name, wireshark=wireshark, jwt_token=jwt_token)
|
||||
|
||||
async def stop_capture(self):
|
||||
"""
|
||||
|
||||
@ -20,7 +20,7 @@ from .common import ErrorMessage
|
||||
from .version import Version
|
||||
|
||||
# Controller schemas
|
||||
from .controller.links import LinkCreate, LinkUpdate, Link, UDPPortInfo, EthernetPortInfo
|
||||
from .controller.links import LinkCreate, LinkUpdate, Link, UDPPortInfo, EthernetPortInfo, LinkCapture
|
||||
from .controller.computes import ComputeCreate, ComputeUpdate, ComputeVirtualBoxVM, ComputeVMwareVM, ComputeDockerImage, AutoIdlePC, Compute
|
||||
from .controller.templates import TemplateCreate, TemplateUpdate, TemplateUsage, Template
|
||||
from .controller.images import Image, ImageType
|
||||
|
||||
@ -94,6 +94,17 @@ class VMwareSettings(BaseModel):
|
||||
return self
|
||||
|
||||
|
||||
class WebWiresharkSettings(BaseModel):
|
||||
|
||||
enabled: bool = True
|
||||
image: str = "gns3/web-wireshark:latest"
|
||||
network_subnet: str = "172.31.0.0/22"
|
||||
memory: str = "2g"
|
||||
cpus: float = 1.0
|
||||
pids_limit: int = 1000
|
||||
model_config = ConfigDict(validate_assignment=True, str_strip_whitespace=True)
|
||||
|
||||
|
||||
class ServerProtocol(str, Enum):
|
||||
|
||||
http = "http"
|
||||
@ -196,3 +207,4 @@ class ServerConfig(BaseModel):
|
||||
Qemu: QemuSettings = QemuSettings()
|
||||
VirtualBox: VirtualBoxSettings = VirtualBoxSettings()
|
||||
VMware: VMwareSettings = VMwareSettings()
|
||||
WebWireshark: WebWiresharkSettings = WebWiresharkSettings()
|
||||
|
||||
@ -634,6 +634,7 @@ class ApplianceV1_6(BaseModel):
|
||||
iou: Optional[Iou] = Field(None, title='IOU specific options')
|
||||
dynamips: Optional[Dynamips] = Field(None, title='Dynamips specific options')
|
||||
qemu: Optional[Qemu] = Field(None, title='Qemu specific options')
|
||||
tags: Optional[List[str]] = Field(None, title='User-defined metadata tags for the appliance')
|
||||
images: Optional[List[ApplianceImage]] = Field(None, title='Images for this appliance')
|
||||
versions: Optional[List[ApplianceVersion]] = Field(None, title='Versions of the appliance')
|
||||
|
||||
@ -672,6 +673,7 @@ class ApplianceV8(BaseModel):
|
||||
default_username: Optional[str] = Field(None, title='Default username for the appliance')
|
||||
default_password: Optional[str] = Field(None, title='Default password for the appliance')
|
||||
symbol: Optional[str] = Field(None, title='An optional symbol for the appliance')
|
||||
tags: Optional[List[str]] = Field(None, title='User-defined metadata tags for the appliance')
|
||||
settings: List[TemplateSetting] = Field(..., title='Settings for running the appliance')
|
||||
images: Optional[List[ApplianceImage]] = Field(None, title='Images for this appliance')
|
||||
versions: Optional[List[ApplianceVersionV8]] = Field(None, title='Versions of the appliance')
|
||||
|
||||
@ -55,7 +55,8 @@ class ChatResponse(BaseModel):
|
||||
"tool_end", # Tool execution completed
|
||||
"error", # Error message
|
||||
"done", # Stream ended
|
||||
"heartbeat" # Keep-alive signal
|
||||
"heartbeat", # Keep-alive signal
|
||||
"abort" # Stream aborted
|
||||
] = Field(..., description="Response message type")
|
||||
content: Optional[str] = Field(None, description="Text content (for type=content)")
|
||||
message_id: Optional[str] = Field(None, description="Message ID")
|
||||
|
||||
@ -96,6 +96,10 @@ class Link(LinkBase):
|
||||
None,
|
||||
description="Read only property. The compute identifier where a capture is running"
|
||||
)
|
||||
wireshark: Optional[bool] = Field(
|
||||
False,
|
||||
description="Read only property. True if a Web Wireshark session is active on the link"
|
||||
)
|
||||
|
||||
|
||||
class UDPPortInfo(BaseModel):
|
||||
@ -117,3 +121,13 @@ class EthernetPortInfo(BaseModel):
|
||||
node_id: UUID
|
||||
interface: str
|
||||
type: str
|
||||
|
||||
|
||||
class LinkCapture(BaseModel):
|
||||
"""
|
||||
Link capture data.
|
||||
"""
|
||||
|
||||
data_link_type: str = "DLT_EN10MB"
|
||||
capture_file_name: Optional[str] = None
|
||||
wireshark: bool = False
|
||||
|
||||
@ -110,7 +110,7 @@ BUILTIN_TEMPLATES = [
|
||||
"base_script_file": "vpcs_base_config.txt",
|
||||
"compute_id": None,
|
||||
"builtin": True,
|
||||
"tags": ["platform:vpcs", "device_type:gns3_vpcs_telnet"],
|
||||
"tags": ["device_type:gns3_vpcs_telnet"],
|
||||
},
|
||||
{
|
||||
"template_id": uuid.uuid5(uuid.NAMESPACE_X500, "ethernet_switch"),
|
||||
|
||||
40
gns3server/utils/port_allocator.py
Normal file
40
gns3server/utils/port_allocator.py
Normal file
@ -0,0 +1,40 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# Deterministic port allocation utilities
|
||||
#
|
||||
|
||||
import hashlib
|
||||
|
||||
|
||||
# Display/port allocation range (stable across process restarts)
|
||||
DISPLAY_RANGE = 10000
|
||||
DISPLAY_MODULO = 10000
|
||||
|
||||
|
||||
def link_id_to_display(link_id: str) -> int:
|
||||
"""Convert link_id to display number using deterministic hash.
|
||||
|
||||
Uses MD5 to ensure stable display assignment across process restarts.
|
||||
|
||||
Args:
|
||||
link_id: The link ID (UUID)
|
||||
|
||||
Returns:
|
||||
Display number (10000-19999)
|
||||
"""
|
||||
hash_value = int(hashlib.md5(link_id.encode()).hexdigest(), 16)
|
||||
return DISPLAY_RANGE + (hash_value % DISPLAY_MODULO)
|
||||
|
||||
|
||||
def link_id_to_port(link_id: str) -> int:
|
||||
"""Convert link_id to TCP port using deterministic hash.
|
||||
|
||||
Uses same algorithm as link_id_to_display for consistency.
|
||||
|
||||
Args:
|
||||
link_id: The link ID (UUID)
|
||||
|
||||
Returns:
|
||||
TCP port (10000-19999)
|
||||
"""
|
||||
return link_id_to_display(link_id)
|
||||
33
gns3server/utils/uuid_validator.py
Normal file
33
gns3server/utils/uuid_validator.py
Normal file
@ -0,0 +1,33 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# UUID validation utilities
|
||||
#
|
||||
|
||||
import argparse
|
||||
import re
|
||||
|
||||
|
||||
UUID_PATTERN = re.compile(
|
||||
r'^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$',
|
||||
re.IGNORECASE
|
||||
)
|
||||
|
||||
|
||||
def validate_uuid(value: str) -> str:
|
||||
"""Validate UUID format for argparse.
|
||||
|
||||
Args:
|
||||
value: UUID string to validate
|
||||
|
||||
Returns:
|
||||
The validated UUID string
|
||||
|
||||
Raises:
|
||||
argparse.ArgumentTypeError: If UUID format is invalid
|
||||
"""
|
||||
if not UUID_PATTERN.match(value):
|
||||
raise argparse.ArgumentTypeError(
|
||||
f"Invalid UUID format: '{value}'. "
|
||||
f"Expected 8-4-4-4-12 hex groups (e.g., 5af0fe00-f39d-4985-8669-7e8c512d729c)"
|
||||
)
|
||||
return value
|
||||
299
gns3server/utils/websocket_to_websocket.py
Normal file
299
gns3server/utils/websocket_to_websocket.py
Normal file
@ -0,0 +1,299 @@
|
||||
#
|
||||
# Copyright (C) 2026 GNS3 Technologies Inc.
|
||||
#
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU General Public License as published by
|
||||
# the Free Software Foundation, either version 3 of the License, or
|
||||
# (at your option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
"""
|
||||
WebSocket-to-WebSocket proxy utility functions.
|
||||
|
||||
Similar pattern to VNC console implementation in base_node.py:
|
||||
- Bidirectional forwarding
|
||||
- Binary data only (for protocols like xpra, VNC, RDP)
|
||||
- Graceful error handling
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import sys
|
||||
from typing import Optional
|
||||
|
||||
import aiohttp
|
||||
from fastapi import WebSocket, status
|
||||
from fastapi.websockets import WebSocketDisconnect
|
||||
from starlette.websockets import WebSocket as StarletteWebSocket
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def websocket_proxy(
|
||||
client_ws: WebSocket,
|
||||
target_url: str,
|
||||
requested_protocols: list = None,
|
||||
buffer_size: int = 65536,
|
||||
timeout: Optional[float] = None
|
||||
) -> None:
|
||||
"""
|
||||
Proxy binary WebSocket data between client and target WebSocket server.
|
||||
|
||||
Designed for binary protocols like xpra, VNC, RDP.
|
||||
Text data is not supported.
|
||||
|
||||
Args:
|
||||
client_ws: Client WebSocket connection (FastAPI)
|
||||
target_url: Target WebSocket URL to proxy to
|
||||
requested_protocols: List of subprotocols requested by client (default: ["binary"])
|
||||
buffer_size: Buffer size for binary data (default: 65536)
|
||||
timeout: Connection timeout in seconds (default: None)
|
||||
|
||||
Raises:
|
||||
aiohttp.ClientError: If connection to target fails
|
||||
"""
|
||||
client_info = f"{client_ws.client.host}:{client_ws.client.port}"
|
||||
|
||||
async def forward_client_to_target(target_ws):
|
||||
"""Client → Target: Forward binary WebSocket data."""
|
||||
try:
|
||||
while True:
|
||||
data = await client_ws.receive_bytes()
|
||||
if data:
|
||||
await target_ws.send_bytes(data)
|
||||
except WebSocketDisconnect:
|
||||
log.info(f"Client {client_info} disconnected from WebSocket proxy")
|
||||
except Exception as e:
|
||||
log.warning(f"Error forwarding client to target: {e}")
|
||||
|
||||
async def forward_target_to_client(target_ws):
|
||||
"""Target → Client: Forward binary WebSocket data."""
|
||||
try:
|
||||
async for msg in target_ws:
|
||||
if msg.type == aiohttp.WSMsgType.BINARY:
|
||||
try:
|
||||
await client_ws.send_bytes(msg.data)
|
||||
except Exception as e:
|
||||
log.debug(f"Failed to send to client (possibly disconnected): {e}")
|
||||
break
|
||||
elif msg.type == aiohttp.WSMsgType.ERROR:
|
||||
log.warning(f"Target WebSocket error: {msg.data}")
|
||||
break
|
||||
elif msg.type == aiohttp.WSMsgType.CLOSE:
|
||||
log.info("Target WebSocket closed")
|
||||
# Don't try to forward close to client if already disconnected
|
||||
break
|
||||
except Exception as e:
|
||||
log.warning(f"Error forwarding target to client: {e}")
|
||||
|
||||
try:
|
||||
# Connect to target WebSocket FIRST to negotiate subprotocol
|
||||
log.info(f"Connecting to target WebSocket: {target_url}")
|
||||
|
||||
# Use requested protocols from client (usually "binary" for xpra)
|
||||
subprotocols = requested_protocols or ["binary"]
|
||||
log.info(f"Client requested subprotocols: {subprotocols}")
|
||||
|
||||
timeout_config = {}
|
||||
if timeout:
|
||||
timeout_config = {"timeout": aiohttp.ClientTimeout(total=timeout)}
|
||||
|
||||
async with aiohttp.ClientSession() as session:
|
||||
async with session.ws_connect(target_url, protocols=subprotocols, **timeout_config) as target_ws:
|
||||
negotiated_protocol = target_ws.protocol
|
||||
log.info(f"Target WebSocket negotiated protocol: {negotiated_protocol}")
|
||||
log.info(f"WebSocket proxy established: {client_info} → {target_url}")
|
||||
|
||||
# Run both forwarding tasks in parallel
|
||||
# Similar pattern to base_node.py VNC implementation
|
||||
if sys.version_info >= (3, 11, 0):
|
||||
aws = [
|
||||
asyncio.create_task(forward_client_to_target(target_ws)),
|
||||
asyncio.create_task(forward_target_to_client(target_ws))
|
||||
]
|
||||
else:
|
||||
aws = [
|
||||
forward_client_to_target(target_ws),
|
||||
forward_target_to_client(target_ws)
|
||||
]
|
||||
|
||||
try:
|
||||
done, pending = await asyncio.wait(
|
||||
aws,
|
||||
return_when=asyncio.ALL_COMPLETED
|
||||
)
|
||||
except Exception as e:
|
||||
log.error(f"asyncio.wait raised exception: {e}")
|
||||
|
||||
# Check for exceptions
|
||||
for task in done:
|
||||
if task.exception():
|
||||
log.warning(
|
||||
f"WebSocket proxy task exception: {task.exception()}"
|
||||
)
|
||||
|
||||
# Cancel pending tasks
|
||||
for task in pending:
|
||||
task.cancel()
|
||||
|
||||
except aiohttp.ClientError as e:
|
||||
log.error(f"WebSocket proxy connection error: {e}")
|
||||
await client_ws.close(
|
||||
code=status.WS_1011_INTERNAL_ERROR,
|
||||
reason=f"Proxy connection failed: {e}"
|
||||
)
|
||||
except Exception as e:
|
||||
log.error(f"WebSocket proxy unexpected error: {e}")
|
||||
await client_ws.close(
|
||||
code=status.WS_1011_INTERNAL_ERROR,
|
||||
reason=str(e)
|
||||
)
|
||||
|
||||
|
||||
async def websocket_proxy_with_manual_accept(
|
||||
client_ws: StarletteWebSocket,
|
||||
target_url: str,
|
||||
requested_protocols: list = None,
|
||||
buffer_size: int = 65536,
|
||||
timeout: Optional[float] = None
|
||||
) -> None:
|
||||
"""
|
||||
Proxy binary WebSocket data between client and target WebSocket server.
|
||||
|
||||
This version manually accepts the client WebSocket connection after negotiating
|
||||
the subprotocol with the backend server. This is required for protocols like xpra
|
||||
that depend on proper subprotocol negotiation.
|
||||
|
||||
Args:
|
||||
client_ws: Client WebSocket connection (Starlette WebSocket)
|
||||
target_url: Target WebSocket URL to proxy to
|
||||
requested_protocols: List of subprotocols requested by client (default: ["binary"])
|
||||
buffer_size: Buffer size for binary data (default: 65536)
|
||||
timeout: Connection timeout in seconds (default: None)
|
||||
|
||||
Raises:
|
||||
aiohttp.ClientError: If connection to target fails
|
||||
"""
|
||||
client_info = f"{client_ws.client.host}:{client_ws.client.port}" if hasattr(client_ws, 'client') else "unknown"
|
||||
|
||||
async def forward_client_to_target(target_ws):
|
||||
"""Client → Target: Forward binary WebSocket data."""
|
||||
log.info("forward_client_to_target: started")
|
||||
try:
|
||||
while True:
|
||||
log.info("forward_client_to_target: waiting for client data")
|
||||
data = await client_ws.receive_bytes()
|
||||
log.info(f"forward_client_to_target: received {len(data)} bytes from client")
|
||||
if data:
|
||||
await target_ws.send_bytes(data)
|
||||
except Exception as e:
|
||||
log.info(f"Client disconnected or error: {e}")
|
||||
raise
|
||||
|
||||
async def forward_target_to_client(target_ws):
|
||||
"""Target → Client: Forward binary WebSocket data."""
|
||||
log.info("forward_target_to_client: started, about to iterate")
|
||||
try:
|
||||
async for msg in target_ws:
|
||||
log.info(f"forward_target_to_client: got message type={msg.type}")
|
||||
if msg.type == aiohttp.WSMsgType.BINARY:
|
||||
try:
|
||||
await client_ws.send_bytes(msg.data)
|
||||
except Exception as e:
|
||||
log.debug(f"Failed to send to client (possibly disconnected): {e}")
|
||||
break
|
||||
elif msg.type == aiohttp.WSMsgType.ERROR:
|
||||
log.warning(f"Target WebSocket error: {msg.data}")
|
||||
break
|
||||
elif msg.type == aiohttp.WSMsgType.CLOSE:
|
||||
log.info("Target WebSocket closed")
|
||||
break
|
||||
log.info("forward_target_to_client: iteration ended")
|
||||
except Exception as e:
|
||||
log.warning(f"Error forwarding target to client: {e}")
|
||||
|
||||
try:
|
||||
# Connect to target WebSocket FIRST to negotiate subprotocol
|
||||
log.info(f"Connecting to target WebSocket: {target_url}")
|
||||
|
||||
# Use requested protocols from client (usually "binary" for xpra)
|
||||
subprotocols = requested_protocols or ["binary"]
|
||||
log.info(f"Client requested subprotocols: {subprotocols}")
|
||||
|
||||
timeout_config = {}
|
||||
if timeout:
|
||||
timeout_config = {"timeout": aiohttp.ClientTimeout(total=timeout)}
|
||||
|
||||
async with aiohttp.ClientSession() as session:
|
||||
log.info(f"About to ws_connect to {target_url}")
|
||||
ws_conn = session.ws_connect(target_url, protocols=subprotocols, **timeout_config)
|
||||
log.info("ws_connect coroutine created, about to enter")
|
||||
async with ws_conn as target_ws:
|
||||
negotiated_protocol = target_ws.protocol
|
||||
log.info(f"Target WebSocket negotiated protocol: {negotiated_protocol}")
|
||||
log.info(f"target_ws.closed = {target_ws.closed}")
|
||||
|
||||
# Accept client WebSocket with the negotiated subprotocol
|
||||
# This is CRITICAL for xpra to work properly
|
||||
log.info(f"Accepting client WebSocket with subprotocol: {negotiated_protocol}")
|
||||
await client_ws.accept(subprotocol=negotiated_protocol)
|
||||
log.info("Client WebSocket accepted")
|
||||
|
||||
log.info(f"WebSocket proxy established: {client_info} → {target_url}")
|
||||
|
||||
# Run both forwarding tasks in parallel
|
||||
if sys.version_info >= (3, 11, 0):
|
||||
aws = [
|
||||
asyncio.create_task(forward_client_to_target(target_ws)),
|
||||
asyncio.create_task(forward_target_to_client(target_ws))
|
||||
]
|
||||
else:
|
||||
aws = [
|
||||
forward_client_to_target(target_ws),
|
||||
forward_target_to_client(target_ws)
|
||||
]
|
||||
|
||||
log.info(f"About to call asyncio.wait with {len(aws)} tasks")
|
||||
try:
|
||||
done, pending = await asyncio.wait(
|
||||
aws,
|
||||
return_when=asyncio.ALL_COMPLETED
|
||||
)
|
||||
log.info(f"asyncio.wait returned. done={len(done)}, pending={len(pending)}")
|
||||
except Exception as e:
|
||||
log.error(f"asyncio.wait raised exception: {e}")
|
||||
log.info("After asyncio.wait check")
|
||||
|
||||
# Check for exceptions
|
||||
for task in done:
|
||||
if task.exception():
|
||||
log.warning(
|
||||
f"WebSocket proxy task exception: {task.exception()}"
|
||||
)
|
||||
|
||||
# Cancel pending tasks
|
||||
for task in pending:
|
||||
log.info("Cancelling pending task")
|
||||
task.cancel()
|
||||
|
||||
log.info("Cleanup complete, exiting websocket_proxy_with_manual_accept")
|
||||
|
||||
except aiohttp.ClientError as e:
|
||||
log.error(f"WebSocket proxy connection error: {e}")
|
||||
try:
|
||||
await client_ws.close(code=status.WS_1011_INTERNAL_ERROR)
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as e:
|
||||
log.error(f"WebSocket proxy unexpected error: {e}")
|
||||
try:
|
||||
await client_ws.close(code=status.WS_1011_INTERNAL_ERROR)
|
||||
except Exception:
|
||||
pass
|
||||
@ -30,9 +30,6 @@ classifiers = [
|
||||
|
||||
dynamic = ["version", "dependencies", "optional-dependencies"]
|
||||
|
||||
[tool.setuptools]
|
||||
packages = ["gns3server"]
|
||||
|
||||
[tool.setuptools.dynamic]
|
||||
version = {attr = "gns3server.version.__version__"}
|
||||
dependencies = {file = "requirements.txt"}
|
||||
@ -54,3 +51,7 @@ ai-copilot = {file = ['ai-requirements.txt']}
|
||||
gns3server = "gns3server.main:main"
|
||||
gns3vmnet = "gns3server.utils.vmnet:main"
|
||||
gns3server-uninstall-ai-copilot = "gns3server.utils.uninstall_ai_copilot:main"
|
||||
gns3-wireshark-setup = "gns3server.agent.web_wireshark.setup_wireshark_image:main"
|
||||
|
||||
[tool.setuptools]
|
||||
packages = ["gns3server"]
|
||||
|
||||
@ -223,6 +223,7 @@ async def test_json(project, compute):
|
||||
"filters": {},
|
||||
"link_style": {},
|
||||
"suspend": False,
|
||||
'wireshark': False,
|
||||
"link_type": "ethernet",
|
||||
"capturing": False,
|
||||
"capture_file_name": None,
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user