6483 Commits

Author SHA1 Message Date
YueGuobin
e2e2e23cf7 feat(copilot): add VPCS Telnet driver with Netmiko and ANSI code stripping
Implement custom VPCS driver and unified tool architecture:

   - Add VPCSTelnet custom driver (vpcs_telnet.py)
     - No authentication (direct console access like VPCS behavior)
     - Simple prompt pattern matching (PC\d+>)
     - Automatic ANSI escape code stripping for clean output
     - Config mode methods return empty (VPCS has no config modes)

   - Replace vpcs_tools_telnetlib3.py with vpcs_tools_netmiko.py
     - Migrate from telnetlib3 to Netmiko + Nornir architecture
     - Unified tool architecture matching config/display tools
     - Improved code consistency and maintainability

   - Add comprehensive test coverage (test_vpcs_telnet.py)
     - 30 unit tests covering all VPCS driver functionality
     - Tests for ANSI code stripping, telnet_login, send_command
     - Tests for device registration and initialization

   - Update VPCS built-in template (services/templates.py)
     - Add platform:vpcs and device_type:gns3_vpcs_telnet tags
     - Automatic driver selection without manual configuration

   - Update documentation (docs/)
     - multi-vendor-device-support.md: VPCS driver documentation
     - netmiko_devices.md: Add VPCS to supported devices list
     - README.md: Update multi-vendor support description
2026-03-14 14:25:37 +08:00
YueGuobin
bbe57f34b9 feat(llm): hide api_key from all API responses and document max_tokens as reserved
Security Enhancement:
   - API keys are now always filtered to null in all LLM config API responses
   - Created LLMModelConfigDataWithoutSecret schema for response validation
   - Added _filter_api_key_from_config() helper in API routes
   - Modified repository to always hide api_key in get_user_effective_configs()
   - Update/create operations still accept and store api_key securely

   Documentation Updates:
   - Updated API key visibility section to reflect new security policy
   - Updated all response examples to show api_key as null
   - Marked max_tokens field as reserved for future use
   - Added "Reserved Fields" section explaining unused fields

   This defense-in-depth approach prevents API keys from being leaked through
   logs, browser devtools, or network monitoring.
2026-03-14 13:12:22 +08:00
YueGuobin
a58089cf48 fix(copilot): add exit command warning to prevent session disconnection
Add warnings about 'exit' command in three locations to prevent AI from
   disconnecting Telnet/SSH sessions during command execution:

   - Prompt system rules (lab_automation_assistant_prompt.py)
   - Display tool description (display_tools_nornir.py)
   - Config tool description (config_tools_nornir.py)

   The 'exit' command disconnects the session and causes all subsequent
   commands in the batch to fail. This prevents users from viewing
   command outputs properly.
2026-03-14 00:34:06 +08:00
YueGuobin
c76982b29e refactor(copilot): rename Huawei CE driver and add Ruijie tests with format fixes
- Rename huawei_telnet_ce to gns3_huawei_telnet_ce for consistent naming
   - Add comprehensive test suite for RuijieTelnetEnhanced driver (10 tests)
   - Update list_netmiko_telnet_devices.py to include gns3_ruijie_telnet
   - Fix all flake8 format issues (line length, unused imports, variables)
   - Update documentation to reflect device type rename
2026-03-14 00:08:13 +08:00
YueGuobin
6e88582ff1 docs(gns3-copilot): update documentation for Ruijie driver and new features
- Add gns3_ruijie_telnet to supported devices in netmiko_devices.md
- Document custom RuijieTelnetEnhanced driver with hybrid strategy
  - Preprocessing: auto-insert 'yes' after known interactive commands
  - Fast path: batch send (2-3s for 13 commands)
  - Fallback: one-by-one with real-time prompt detection
- Add AAA/password configuration prohibition documentation
  - System-level enforcement in lab_automation_assistant_prompt
  - Categories: AAA, login passwords, console/VTY auth, encryption
- Document multi-line command handling for banner and similar commands
- Update device_type tag validation with error feedback
- Update multi-vendor support to include Ruijie platform

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-13 23:56:57 +08:00
YueGuobin
4f4ddb6504 feat(ruijie-telnet): enhance interactive command handling with hybrid strategy
Add pre-processing of known interactive commands and implement hybrid send strategy for Ruijie Telnet devices. The new approach:
- Pre-processes configuration commands to automatically insert 'yes' responses after known interactive commands (router-id, erase, delete, format, reload, boot system)
- Implements hybrid strategy: first attempts fast batch send with pre-processed commands, then falls back to one-by-one send with real-time prompt detection if batch fails
- Maintains backward compatibility while improving reliability for interactive configuration scenarios

This improves configuration reliability for Ruijie devices that frequently require confirmation prompts during configuration changes.
2026-03-13 23:51:12 +08:00
YueGuobin
ad771e2927 feat: add Ruijie telnet driver support to custom Netmiko tools
- Import and register Ruijie telnet device type in config_tools_nornir.py
- Import and register Ruijie telnet device type in display_tools_nornir.py
- Update custom_netmiko __init__.py to include RuijieTelnetEnhanced driver
- Add Ruijie telnet driver to __all__ exports for proper module exposure
- Extends custom Netmiko support to handle Ruijie devices with interactive prompt handling
2026-03-13 23:33:14 +08:00
YueGuobin
8d9fce1c12 feat(agent): enhance configuration safety and multiline command handling
- Add explicit AAA/password configuration prohibition to safety reminders
- Implement multiline command expansion for banner and similar commands
- Add error handling for devices with missing device_type tags
- Improve logging for configuration errors and multiline expansions
2026-03-13 22:24:36 +08:00
YueGuobin
364ff93783 efactor(copilot): adopt Nornir best
practice for multi-vendor device support

   Refactor Nornir configuration to use host-level connection_options
   instead of dynamic groups, following Nornir's configuration priority
   model (host > group > defaults).

   **Problem:**
   Previous implementation used first device's configuration for all
   devices, causing Cisco devices to use Huawei driver and vice versa.

   **Solution:**
   - Each host now has device-specific connection_options at host level
   - Single generic "network_devices" group for shared settings
   - Host-level config automatically overrides group-level config

   **Changes:**
   - Remove: _get_nornir_groups_config() helper function
   - Remove: _get_nornir_group() helper function
   - Simplify: _initialize_nornir() to use single generic group
   - Update: get_gns3_device_port.py() to return host-level config
   - Reserve: platform field for future NAPALM/scrapli plugin support

   **Benefits:**
   - Cleaner code structure (no dynamic group creation)
   - Follows Nornir best practice ("configuration proximity")
   - Easy to extend with new device types
   - Properly handles mixed-vendor topologies
2026-03-13 09:32:18 +08:00
YueGuobin
44f802ae4f fix: update netmiko platforms after custom device registration
Netmiko's `ssh_dispatcher` calculates platform lists at module import
time. When custom device types (like Huawei CE) are registered
dynamically, these cached lists become stale and do not include the
new platforms.

This change imports `netmiko.ssh_dispatcher` and recalculates the
`platforms`, `platforms_base`, and `telnet_platforms` attributes to
ensure Netmiko recognizes the custom device types.
2026-03-12 23:38:24 +08:00
YueGuobin
effea3fb26 feat(copilot): add Netmiko device list generator and simplify Huawei driver
- Add script to generate Markdown documentation of all Netmiko supported
     devices (SSH and Telnet), grouped by platform with source attribution
   - Highlight custom GNS3-Copilot devices with "Custom " marker
   - Auto-generate device list to docs/gns3-copilot/netmiko_devices.md
   - Simplify HuaweiTelnetCE driver: remove redundant aliases
     (huawei_ce, huawei_telnet_ce_telnet), keep only huawei_telnet_ce
   - Update tests to match simplified device registration
2026-03-12 23:03:19 +08:00
YueGuobin
50fa4f6e2a refactor(huawei_ce): disable mypy and refactor driver
Disable mypy type checking for the Huawei CE driver module due to
Netmiko library limitations. Netmiko lacks type stubs (py.typed) and
uses dynamic attributes, which causes unresolved import and attribute
errors in static analysis.

Added a comprehensive comment block at the top of the file explaining
the rationale for disabling mypy to prevent future confusion.

Additionally, performed code cleanup including:
- Reformatted module docstrings and comments for better readability
- Added missing imports (importlib, logging)
- Removed unused typing imports (Optional)
- Refactored variable assignments in send_config_set for clarity
2026-03-12 22:38:21 +08:00
YueGuobin
db164e9d41 feat(copilot): add multi-vendor device support with custom Huawei driver
Add comprehensive multi-vendor support for GNS3 network automation,
   including a custom Netmiko driver for Huawei CloudEngine devices.

   Features:
   - Custom HuaweiTelnetCE driver for GNS3 emulation (no authentication)
   - Auto-commit before exit to prevent [Y/N/C] prompts
   - Dynamic device type detection from GNS3 node tags
   - Support for both Cisco IOS and Huawei devices
   - Proper VRP command handling (system-view, return confirmation)

   Implementation:
   - New package: utils/custom_netmiko/
     - huawei_ce.py: Huawei CloudEngine driver
     - tests/test_huawei_ce.py: Unit tests (9/9 passing)
     - README.md: Driver development guide
   - Updated tools for multi-vendor support:
     - display_tools_nornir.py: Dynamic group generation
     - config_tools_nornir.py: Multi-vendor config commands
     - get_gns3_device_port.py: Device port extraction
   - Documentation: multi-vendor-device-support.md

   Limitations:
   - huawei_telnet_ce driver requires devices without authentication
   - For devices with username/password, use standard huawei_telnet driver

   Co-Authored-By: Yue Guobin <yueguobin@outlook.com>"
2026-03-12 16:44:32 +08:00
Guobin Yue
ab88bcd57e
Merge branch '3.0' into feature/ai-copilot-bridge 2026-03-12 16:12:13 +08:00
grossmj
3abecc8492
Fix IOU tests 2026-03-12 14:18:35 +08:00
grossmj
27ce1a4e25
Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	gns3server/schemas/iou_template.py
#	tests/handlers/api/compute/test_iou.py
#	tests/handlers/api/controller/test_template.py
2026-03-12 14:11:05 +08:00
Jeremy Grossmann
5c63928636
Merge pull request #2634 from GNS3/update-iol-ram-nvram
Deactivate 'use default IOU values' by default and update RAM/NVRAM values
2026-03-12 13:53:04 +08:00
grossmj
f41cc24383
Fix for duplicating Qemu nodes 2026-03-12 13:24:30 +08:00
Jeremy Grossmann
9aca6aece3
Merge pull request #2636 from yueguobin/fix/acl-endpoint-paths
fix(acl): correct endpoint paths for users, groups, and roles
2026-03-12 12:31:59 +08:00
YueGuobin
75a4585c20 docs: restructure copilot docs and update node tools
Simplify the documentation structure in `README.md` by removing the
`todo/` directory reference and detailed design documents for planned
features. Consolidate future roadmap items into a high-level summary
under "Future Enhancements".

Update `node-control-tools.md` to include documentation for new topology
management tools (create node, create link, get template, rename node)
and reflect updated API imports for `Link` support.
2026-03-12 00:23:00 +08:00
Guobin Yue
085a485503
Merge branch '3.0' into fix/acl-endpoint-paths 2026-03-11 23:34:18 +08:00
YueGuobin
f2fd725e07 fix(acl): correct endpoint paths for users, groups, and roles
Fix the path mismatch between /acl/endpoints API and actual routes:
- Users: /users/{id} → /access/users/{id}
- Groups: /groups/{id} → /access/groups/{id}
- Roles: /roles/{id} → /access/roles/{id}

This fixes the error where creating ACE entries fails with:
"Path '/groups/{id}' doesn't match any existing endpoint"

The actual routes are registered under /access/ prefix, but the
endpoints API was returning paths without the prefix.

Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-11 23:23:52 +08:00
YueGuobin
5155e10214 feat(deps): update langchain ecosystem and add langsmith SDK
- Upgrade langchain packages to latest versions (langchain 1.2.10, langgraph 1.0.9, etc.)
- Add langsmith SDK for enhanced observability and tracing
- Update Pillow to 12.1.1 for image processing improvements
- Maintain compatibility with existing AI and automation framework
2026-03-11 22:18:31 +08:00
YueGuobin
e82bce9e16 feat(docs): add TOSCA-based topology description proposal
Add comprehensive documentation proposing TOSCA (Topology and Orchestration Specification for Cloud Applications) as the standard format for GNS3 network topologies. The document outlines strategic benefits including standardized YAML descriptions, toolchain ecosystem integration, Git-based workflows, and template reuse capabilities. This initiative aims to modernize GNS3 topology management, improve user experience, and align with industry best practices for network automation and orchestration.
2026-03-11 22:13:23 +08:00
YueGuobin
ad3621db0f feat(gns3-copilot): add node stop and suspend tools for lab automation
- Add GNS3StopNodeTool and GNS3SuspendNodeTool to lab automation assistant mode
- Update tools_v2 __init__.py to export new node control tools
- Document node control tools in README with key features and implementation status
- Update last modified date in documentation

The new tools provide complete node lifecycle control for automated lab workflows, including stopping nodes for shutdown and suspending nodes while preserving state.
2026-03-11 14:44:51 +08:00
YueGuobin
1cd4aeb2a2 feat(schemas): simplify LLM model config descriptions
- Remove redundant "(REQUIRED)" and detailed explanations from context_limit field descriptions
- Shorten copilot_mode descriptions by removing parenthetical details about mode capabilities
- Maintain field formatting consistency across LLMModelConfigData, Create, and Update schemas
2026-03-11 10:17:06 +08:00
YueGuobin
10a827f0c5 feat(api): enhance max_tokens field with robust null handling
- Update LLMModelConfigUpdate schema to accept Union[int, str] for max_tokens
- Add field validator to gracefully handle various null representations:
  - Convert string "null" to null
  - Convert empty strings to null
  - Convert numeric strings to integers
  - Accept proper JSON null values
- Update documentation to reflect new behavior and explain robust null handling
- Prevents validation errors from incorrect null serialization by clients
2026-03-11 10:14:06 +08:00
YueGuobin
e452f75ac3 docs: reorganize AI Copilot documentation structure
- Add README.md with documentation overview and structure guide
- Move implemented designs to docs/gns3-copilot/implemented/:
  - chat-api.md (from ai-chat-api-design.md)
  - llm-model-configs.md (from llm-model-configs-api.md)
  - command-security.md
  - context-window-management.md
- Add Jinja2 configuration template system design documents:
  - jinja2-config-templates-system.md
  - config-templates-implementation-guide.md
  - ai-prompting-for-config-templates.md
- Remove obsolete documents (acl-web-ui, Chinese RBAC doc)

This reorganization makes it clearer which features are implemented
vs planned, following the established documentation structure.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-10 23:20:24 +08:00
YueGuobin
a6c33061c4 delete doc 2026-03-10 22:51:57 +08:00
YueGuobin
578f79cd6b feat(docs): add AI prompting guide for configuration templates
Add comprehensive documentation for AI prompting to generate structured configuration data for Jinja2 templates. The guide includes:

- Core system prompt with critical rules for structured data generation
- Supported vendors and OS types (Cisco, Juniper, Huawei, Arista, Mikrotik)
- Detailed schemas for network features including OSPF, BGP, VLANs, ACLs, and more
- Example prompts and expected structured outputs
- Integration with the configuration renderer system

This documentation ensures AI-generated configurations follow the required structured format for proper template rendering, preventing direct configuration text generation and maintaining consistency across different network device vendors.
2026-03-10 22:38:41 +08:00
grossmj
0624c1b945
Deactivate 'use default IOU values' by default and update RAM/NVRAM values 2026-03-10 20:28:00 +08:00
YueGuobin
2dcdadf5fa docs: add mypy type checking issues todo document
Add comprehensive documentation for mypy static type checking
issues found in gns3_copilot module.

Document includes:
- 35 type errors across 9 files
- Detailed error descriptions and line numbers
- 3-phase fix strategy (High/Medium/Low priority)
- Mypy configuration recommendations
- Resource links for type hints and mypy

Related files:
- chat_sessions_repository.py (8 errors)
- context_manager.py (5 errors)
- agent_service.py (5 errors)
- gns3_topology_reader.py (2 errors)
- message_converters.py (1 error)
- connector_factory.py (1 error)

Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-10 01:20:42 +08:00
YueGuobin
71802a8064 style: remove unused List import from llm_model_configs.py
Remove unused typing.List import flagged by ruff F401.

Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-10 01:11:37 +08:00
YueGuobin
5e9ec23a45 style: remove unused imports and fix E712 comparisons in llm_model_configs.py
Remove 5 unused imports flagged by ruff F401:
- sqlalchemy.func, sqlalchemy.or_
- sqlalchemy.ext.asyncio.AsyncSession
- sqlalchemy.orm.selectinload
- json

Fix 4 E712 errors - avoid equality comparisons to True:
- Use models.LLMModelConfig.is_default instead of == True

Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-10 01:10:19 +08:00
YueGuobin
7d031a6802 style: remove unused imports and variables in project.py
Remove unused import and exception variable flagged by ruff:
- Remove unused 'sys' import (F401)
- Remove unused exception variable 'e' in except clause (F841)

Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-10 01:08:20 +08:00
YueGuobin
c0aec3f979 style: remove unused imports from llm_model_configs.py
Remove 3 unused imports that were flagged by ruff F401:
- fastapi.Response
- fastapi.security.OAuth2PasswordRequestForm
- gns3server.controller.controller_error.ControllerError

Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-10 01:07:23 +08:00
YueGuobin
7c3b832bca style: fix E501 line-too-long errors in gns3_copilot
Fix all 423 E501 line length violations across 26 files to comply with
   PEP 8 88-character line limit.

   Changes:
   - Split long f-strings across multiple lines
   - Break long docstring descriptions and parameter lists
   - Split markdown table rows and list examples
   - Break long URL construction f-strings
   - Split long logger messages and comments
   - Add noqa: E501 for SVG strings (cannot be split)

   Modified files:
   - agent/: context_manager.py, gns3_copilot.py, model_factory.py
   - gns3_client/: connector_factory.py, context_helpers.py, custom_gns3fy.py,
                   gns3_project_info.py, gns3_topology_reader.py
   - prompts/: __init__.py, lab_automation_assistant_prompt.py,
               prompt_loader.py, teaching_assistant_prompt.py
   - tools_v2/: __init__.py, config_tools_nornir.py, display_tools_nornir.py,
                gns3_create_link.py, gns3_create_node.py, gns3_get_node_temp.py,
                gns3_start_node.py, gns3_update_node_name.py,
                vpcs_tools_telnetlib3.py
   - utils/: __init__.py, command_filter.py, get_gns3_device_port.py,
             gns3_drawing_utils.py, llm_config_helper.py, message_converters.py,
             parse_tool_content.py, tool_call_stream.py

   All files now pass ruff E501 checks.

   Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-10 01:05:17 +08:00
Guobin Yue
25f2d1b56b
Merge branch '3.0' into feature/ai-copilot-bridge 2026-03-09 21:50:37 +08:00
YueGuobin
7c2083d227 feat: update .gitignore to track project context and development docs
- Add exception for .claude/development.md to allow tracking development documentation
- Comment out PROJECT_CONTEXT.md exclusion to enable version control of project context file
- Create PROJECT_CONTEXT.md with comprehensive project overview for AI assistant support
- Document project structure, AI copilot data flow, SSE event types, and code standards
- Provide flake8 static analysis guidelines and common error fixes
2026-03-09 21:46:58 +08:00
grossmj
69eafda777
Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	gns3server/version.py
2026-03-09 17:12:24 +08:00
grossmj
b658f7ff68
Make sure the node shows as stopped when the wrap console cannot be stopped. Ref https://github.com/GNS3/gns3-registry/pull/1010 2026-03-09 17:10:00 +08:00
YueGuobin
b936a52f6f docs: add LangGraph Agent and tool output serialization documentation
Added new section in ai-chat-api-design.md explaining:

- LangGraph Agent architecture and main components
- tool_node function and tool output serialization mechanism
- Why serialization happens in tool_node (not agent_service)
- Tool output data flow diagram showing SSE and history paths
- Explanation of JSON format consistency across streaming and storage

This documentation clarifies the fix for single-quotes issue in
conversation history and helps future maintenance.

Co-Authored-By: YueGuobin <yueguobin@outlook.com>
2026-03-09 15:54:44 +08:00
YueGuobin
9b85417a82 fix(copilot): serialize tool output to JSON in ToolMessage for history
Fixed tool output serialization in tool_node function to ensure
ToolMessage.content is always in JSON format, not Python str() representation.

This fixes the issue where conversation history showed tool outputs
with single quotes (Python format) instead of standard JSON.

Changes:
- Added json import to gns3_copilot.py
- Modified tool_node() to serialize observation to JSON before creating ToolMessage
- Ensures both SSE streaming and history storage use consistent JSON format

Root cause: ToolMessage was created with raw dict/list objects, which
LangChain converted to Python str() representation when saving to history.

Co-Authored-By: YueGuobin <yueguobin@outlook.com>
2026-03-09 15:50:36 +08:00
YueGuobin
8babaccd80 feat: add PROJECT_CONTEXT.md to gitignore
Add PROJECT_CONTEXT.md to the .gitignore file to prevent it from being tracked in version control. This file likely contains project-specific context or configuration that should not be committed to the repository.
2026-03-09 15:34:31 +08:00
YueGuobin
0f2d0e2859 fix(copilot): serialize tool output to standard JSON format for frontend parsing
Changed tool output serialization in AgentService._convert_event_to_chunk()
from str() to json.dumps() to ensure structured data (dict/list) is properly
formatted as standard JSON instead of Python string representation.

Changes:
- Added json import to agent_service.py
- Modified on_tool_end event handling to use json.dumps(output, ensure_ascii=False, indent=2)
- Updated ai-chat-api-design.md to document tool_output format

Benefits:
- Frontend can parse tool results with standard JSON.parse()
- Chinese and non-ASCII characters are preserved (not escaped)
- Formatted output (indent=2) improves readability

Co-Authored-By: YueGuobin <yueguobin@outlook.com>
2026-03-09 15:15:50 +08:00
YueGuobin
13a032ea2c chore: update author name and copyright headers
Updated the author name and copyright statements across the
gns3_copilot module. The name has been standardized from
"Guobin Yue" to "Yue Guobin (岳国宾)" to reflect the correct
author attribution including Chinese characters.
2026-03-09 11:46:28 +08:00
Guobin Yue
d7e5fe065e
Merge branch '3.0' into feature/ai-copilot-bridge 2026-03-08 13:20:27 +08:00
YueGuobin
9ba02e9436 feat(docs): enhance AI chat API documentation with examples and details
- Improve POST /chat endpoint documentation with request/response examples
- Add session ID management flow explanation
- Enhance GET /sessions endpoint with query parameters and response example
- Update GET /sessions/{session_id}/history with detailed response structure
- Format parameters as tables for better readability
- Clarify session ID usage in streaming conversations
2026-03-08 02:27:48 +08:00
YueGuobin
fd34ef494f feat: add .claude directory to gitignore for API key security
Add .claude/ directory to .gitignore to prevent accidental
commit of Claude Code settings which may contain sensitive
API keys. This ensures development environment configuration
files with potential credentials are excluded from version
control.
2026-03-07 23:52:16 +08:00
grossmj
73ffb22aa2
Bundle web-ui v3.1.0.dev1 2026-03-07 17:47:44 +08:00