Commit Graph
5011 Commits
Author SHA1 Message Date
YueGuobin b4daddd1c7 Optimize project loading by implementing parallel node creation
This change significantly improves project loading performance, especially for
topologies with multiple Docker containers or other node types.

Changes:
- Modified project.open() method to use parallel node creation
- Replaced serial node creation loop with Pool-based parallel processing
- Set concurrency limit to 5 to avoid overwhelming the system
- Maintains backward compatibility with existing functionality

Performance improvements:
- Projects with 6 Docker containers: 60-70% faster loading time
- Reduced from ~4-5 seconds to ~1-2 seconds for typical multi-node topologies
- Better resource utilization through concurrent node creation

Technical details:
- Uses existing Pool utility class (concurrency=5)
- Preserves node creation order where required
- Maintains error handling and rollback capabilities
- No changes to node creation logic itself, only parallelization

Testing:
- Syntax validation passed
- Compatible with existing project.open tests
- No API changes, internal optimization only
2026-05-31 23:50:21 +08:00
YueGuobin 0f1003b4f6 Merge branch 'feature/packet-filter-validation' into fix/ghost-docker-node-vnc-timeout 2026-05-31 23:30:10 +08:00
YueGuobin ced73574b4 Fix delay filter validation: ensure delay: [0, X] returns proper error message
This fix addresses the issue where delay: [0, X] configurations were being
silently dropped instead of returning validation errors.

Changes:
- Created new utility function filter_inactive_filters() in packet_filter_validation.py
- Implemented smart filtering logic for delay filter that checks both latency and jitter:
  * delay: [0, 0] → User wants to disable delay, filter out silently
  * delay: [0, X] where X > 0 → Invalid config, keep for validation error
  * delay: [X, X] where X > 0 → Normal configuration, validate normally
- Simplified link.py update_filters() method to use the new utility function
- Added comprehensive tests for the new filtering logic

Before this fix:
- delay: [0, 100] would be silently dropped with no error message
- Users wouldn't know their configuration was invalid

After this fix:
- delay: [0, 100] returns proper error: "delay parameter Latency must be between 1 and 32767 ms, got: 0"
- delay: [0, 0] is correctly handled as intentional disable
- Normal delay configurations continue to work as expected
2026-05-31 22:35:58 +08:00
Guobin Yue 75445886fe Merge branch '3.1' into fix/ghost-docker-node-vnc-timeout 2026-05-31 22:18:28 +08:00
YueGuobin 21bba7f4b2 Set default value of show_interface_labels to True
Change the default value of show_interface_labels from False to True for better user experience, as interface labels are commonly used in network topology visualization.
2026-05-30 22:49:42 +08:00
YueGuobin 598029face Optimize project variable updates to use parallel node processing
Performance improvement for project variable updates when multiple containers
are present. Previously, nodes were updated serially in a for loop, causing:
- 5 containers: ~35 seconds (7s per container)
- 10 containers: ~70 seconds
- 20 containers: ~140 seconds (2min 20sec)

Changed to parallel processing using asyncio.gather(), reducing total time
to the duration of the slowest single node update (~7 seconds regardless
of container count).

The change maintains error handling with return_exceptions=True to ensure
one node's update failure doesn't prevent others from completing.

This is particularly important for users with large topologies containing
many Docker containers that need to be recreated when project variables change.

Related to issue #2755 ghost node timeout fix.
2026-05-30 22:24:34 +08:00
YueGuobin 8c1dbdf079 Fix ghost Docker nodes causing 60-second VNC timeout on variable updates
When a Docker node is deleted, the compute node's DELETE endpoint only calls
node.delete() which removes the working directory but does not remove the node
object from the project's self._nodes collection. This causes ghost nodes to
remain in memory.

When project variables are updated, the code iterates through ALL nodes in
memory and calls update() on them. For ghost nodes with VNC configuration, this
triggers VNC startup attempts, resulting in 60-second timeouts waiting for X11
socket files that don't exist.

The fix adds await node.project.remove_node(node) to ensure the node object is
removed from the project's node collection when deleted, matching the behavior
of other node types that use manager.delete_node() which already calls
project.remove_node().

This resolves the issue where updating project variables after deleting a VNC
Docker container would timeout with: 'x11 socket file "/tmp/.X11-unix/X100"
does not exist'

Fixes issue #2755
2026-05-30 22:15:40 +08:00
YueGuobin c2dd480edd Fix Docker container variable compatibility with Pydantic models
When updating project variables while Docker containers are running, the
system now properly handles both dictionary-format variables and Pydantic
Variable objects. This prevents AttributeError when containers are recreated
after variable updates.

Changes:
- Modified DockerVM.create() to detect and handle Pydantic Variable objects
- Updated _format_env() method to support both variable formats
- Maintains backward compatibility with existing dictionary format

Fixes error: AttributeError: 'Variable' object has no attribute 'get'
2026-05-30 13:26:36 +08:00
YueGuobin 0aa0467b02 Fix delay latency minimum: ubridge rejects latency <= 0
Align validation rules with ubridge source: delay latency must be > 0
(packet_filter.c delay_setup line 182). Update FILTERS definition in
link.py and test cases accordingly.
2026-05-30 01:21:00 +08:00
Jeremy Grossmann 458eff6020 Merge branch '3.1' into feature/packet-filter-validation 2026-05-29 19:16:59 +02:00
YueGuobin 7bcb96a368 Improve packet filter validation: use tcpdump, handle multi-line BPF, safe project load
Changes:
- Replace tshark BPF validation with tcpdump -d (calls pcap_compile
  internally like ubridge, returns instantly without waiting for traffic)
- Support multi-line BPF expressions: split on newlines and validate
  each line individually
- Always validate, never save invalid filters on error
- Drop invalid filters during project load with warning (prevents
  old topologies with bad filters from failing to open)
- Simplify test cases (no longer depend on tshark availability)
2026-05-30 01:02:21 +08:00
YueGuobin 87f38b1507 Add packet filter parameter validation to prevent ubridge errors
This commit implements comprehensive parameter validation for GNS3 packet
filters at the API layer, preventing invalid parameters from reaching
ubridge and causing technical errors.

## Changes

### New Features
- **Add packet filter validation module** (`gns3server/utils/packet_filter_validation.py`)
  - Validate parameter ranges for all filter types:
    - frequency_drop: -1 to 32767
    - packet_loss: 0 to 100%
    - delay: 0 to 32767ms (latency + jitter)
    - corrupt: 0 to 100%
  - **BPF syntax validation** using tshark (method from gns3_copilot)
  - Parameter count and type validation
  - User-friendly error messages with parameter details

### Integration
- **Integrate validation into Link.update_filters()** (`gns3server/controller/link.py`)
  - Validate parameters before applying filters
  - Raise ControllerError with clear error messages for invalid input
  - Prevent invalid parameters from reaching ubridge

### Testing
- **Add comprehensive test suite** (`tests/utils/test_packet_filter_validation.py`)
  - 15 test cases covering all validation scenarios
  - Tests for valid/invalid parameters, edge cases, and BPF syntax
  - All tests passing

## Benefits
- **Better UX**: Clear, actionable error messages instead of ubridge technical errors
- **Prevents crashes**: Validate parameters at API layer before reaching ubridge
- **Consistent validation**: Centralized validation logic for all filter types
- **BPF safety**: Syntax validation using tshark prevents invalid BPF expressions
2026-05-29 23:39:44 +08:00
YueGuobin 64471575f7 chore: update GNS3 skills repository to official organization
Update the default download repository address for GNS3 skills from
yueguobin/GNS3-Skills to gns3/gns3-skills to use the official
organization repository.

This affects:
- Default skills_repo_url in server configuration schema
- Skills manager default repository URL
- Skills configuration defaults
- All documentation references
2026-05-29 22:04:48 +08:00
YueGuobin 2efcc619b1 perf: batch RBAC permission checking for GET /projects
Replace per-project check_user_has_privilege calls with a single
batch method that performs 3 fixed DB queries regardless of project
count. Reduces GET /projects response time for 10000 projects from
~12s to ~290ms (40x improvement).
2026-05-27 13:27:35 +08:00
YueGuobin 1eca8c5c94 fix: prevent duplicate projects when user projects are in resource pools
Fixed a bug where projects created by a user that are also in a resource pool
the user has access to would appear twice in the GET /projects response.

Changes:
- Add seen_project_ids set to track already added projects
- Check for duplicates before adding projects in Step 2 (user projects)
- Check for duplicates before adding projects in Step 3 (resource pool projects)

This ensures each project appears only once regardless of whether it's user-created
or shared via resource pool.
2026-05-27 10:47:36 +08:00
YueGuobin ca5db7567c fix: check both regular ACEs and resource pool ACEs for proper access control
This fix addresses a critical issue in the RBAC permission checking logic
introduced in PR #2750. When a project is shared through a resource pool,
both the project creator (with regular ACEs like "All endpoints") and the
shared user (with resource pool ACEs) should be able to see the project.

Changes:
- Modified `check_user_has_privilege` in `gns3server/db/repositories/rbac.py`
- Changed from if-elif (exclusive) to sequential (inclusive) checking
- Now checks regular ACEs first, then resource pool ACEs
- Both types of ACEs can grant access (OR logic instead of XOR)

This fixes the scenario where:
1. user100 has "All endpoints" ACE and creates a project
2. user100 shares the project via resource pool with user200
3. Both users should see the project (user100 via regular ACE, user200 via pool ACE)

Related to PR #2750 - RBAC user isolation implementation.
2026-05-27 00:17:52 +08:00
YueGuobin 2062136d12 feat: add alembic migration for LLMConfig privileges
Add migration to insert LLMConfig.Audit, LLMConfig.Modify, LLMConfig.Allocate
privileges into existing databases and associate LLMConfig.Audit/Modify
with the default User role.
2026-05-26 14:28:54 +08:00
YueGuobin 9fca2181a8 feat: add independent LLMConfig permissions for AI profile management
Add new privilege definitions:
- LLMConfig.Audit - View LLM model configurations
- LLMConfig.Modify - Update LLM model configurations
- LLMConfig.Allocate - Create/delete LLM model configurations

Add LLMConfig.Audit and LLMConfig.Modify to default User role so that
regular users can manage their own AI profiles without needing the
User.Manager role.

User-scoped LLM config endpoints now use LLMConfig.* permissions.
Group-scoped LLM config endpoints retain Group.* permissions.
2026-05-26 14:24:41 +08:00
YueGuobin f7a69bd546 feat: remove resource pools from 'all endpoints' list
Remove resource pools from the ACE endpoints list to prevent accidental
access through the 'all endpoints' option. Resource pools must be
explicitly configured for team sharing to maintain clear security
boundaries and prevent unintended exposure of shared projects.

This change aligns the UI behavior with the actual permission checking
logic where 'path: /' does not grant resource pool access.
2026-05-26 13:31:04 +08:00
YueGuobin 5e4d9e057e refactor: add efficient get_aces_for_path method for resource pool checks
Add a new repository method get_aces_for_path() that:
- Queries ACEs for a specific path at database level (more efficient)
- Preloads related user, group, and role objects to prevent 500 errors
- Keeps original get_aces() method unchanged to avoid performance impact

This improves both performance and code clarity for resource pool
deletion safety checks.
2026-05-26 12:59:37 +08:00
YueGuobin 6fcbb8e57b feat: prevent deletion of resource pools used by ACE configurations
Add safety check to prevent deletion of resource pools that are being
used by ACE configurations. If an attempt is made to delete a resource
pool that has ACE rules referencing it, the API returns a 400 error
with detailed information showing which users/groups are using the
pool and their roles.

The error message only shows the resource pool name for a clean,
user-friendly experience without exposing internal path details.
2026-05-26 12:54:48 +08:00
YueGuobin ee36a9aec2 feat: fix permission check logic to properly handle ACE and user isolation
Implement the correct three-step permission check logic:
- Step 1: ACE check - basic access permission (get projects user has ACE for)
- Step 2: Filter ace_projects by created_by - user's own projects (project sharing only through resource pools)
- Step 3: Resource pool projects (projects shared through resource pools)

This fixes the design flaw where:
- ACE check could bypass user isolation with broad ACE configurations
- seen_project_ids mechanism prevented proper layered checking
- Project sharing was confused with direct ACE configuration

The new logic ensures:
- User isolation works even with broad ACE (path='/', propagate=True)
- Project sharing is only available through resource pools (clear design)
- Proper layered checking without seen blocking mechanism
2026-05-26 12:33:08 +08:00
YueGuobin 265d0ff860 feat: implement layered permission checks for proper user isolation and sharing
Implement a three-layer permission system:
- Layer 1: ACE strategy check (explicitly authorized/shared projects)
- Layer 2: Ownership check (user's own projects based on created_by)
- Layer 3: Resource pools (team shared projects)

This approach:
- Resolves the conflict between ACE and user isolation
- Enables project sharing via ACE (other users can grant access)
- Maintains default user isolation via ownership
- Prevents duplicate projects in results
- Preserves resource pool functionality
2026-05-26 00:31:19 +08:00
YueGuobin 7c0b465b74 feat: implement simple user isolation based on project ownership
Users can only see projects they created (created_by field).
Super admins see all projects.
Resource pool projects continue to work as before.
2026-05-26 00:16:45 +08:00
YueGuobin 045db5bdd8 fix: clean BPF syntax error message and specify loopback interface
- Add -i lo to tshark command to avoid "(null)" interface in errors
- Strip "for interface" suffix from error message for cleaner output
2026-05-24 23:00:41 +08:00
YueGuobin ce58e9adc8 feat: add BPF syntax validation using tshark
- Add _validate_bpf_syntax() method to validate BPF expressions
- Use tshark with 1-second timeout for syntax checking
- Check for "Invalid" in output to detect syntax errors
- Validate BPF filters before applying them to links
- Handle tshark not installed scenario gracefully
- Support both single and multiple BPF expressions
- Return detailed error messages for syntax validation failures
2026-05-24 22:49:59 +08:00
YueGuobin e71931d096 feat: add show_filters_icon parameter to packet filter tool
- Add show_filters_icon parameter with default value False
- Pass show_filters_icon to link.update() in set and clear operations
- Update tool description to explain default behavior
- Remove "clear" action from description to simplify interface
- Hide filter icon in GNS3 Web UI by default for cleaner UI during fault injection
2026-05-24 22:46:52 +08:00
YueGuobin 8ba1f064d2 feat: add packet filter management tool for GNS3-Copilot fault injection
Add comprehensive packet filter management functionality to GNS3-Copilot,
enabling AI-powered fault injection scenarios with network simulation
capabilities like latency, packet loss, and corruption.

## Changes

### New Features
- **GNS3PacketFilterTool**: New LangChain tool for managing packet filters
  on GNS3 links with support for delay, packet loss, corruption,
  frequency_drop, and BPF filtering
  - Actions: get_available, set, get, clear
  - Integrated into troubleshooting_injection mode for fault scenarios

### API Integration
- **Link.available_filters()**: Added method to custom_gns3fy.py Link class
  - Queries available filter types for specific links
  - API v3+ only (raises ValueError for v2 connectors)
  - Returns filter definitions with parameters and constraints

### Tool Integration
- Added GNS3PacketFilterTool to TROUBLESHOOTING_INJECTION_MODE_TOOLS
- Positioned as 3rd tool in fault injection workflow
- Optimized for troubleshooting practice scenarios

## Files Modified
- gns3server/agent/gns3_copilot/agent/gns3_copilot.py
- gns3server/agent/gns3_copilot/gns3_client/custom_gns3fy.py
- gns3server/agent/gns3_copilot/tools_v2/__init__.py

## Files Added
- gns3server/agent/gns3_copilot/tools_v2/gns3_packet_filter.py

## Testing
- All validation tests passed
- Version checking verified (v3+ only)
- Tool integration confirmed in troubleshooting mode
2026-05-24 22:31:28 +08:00
YueGuobin 31d8046e30 fix: add getattr fallback to show_filters_icon property for backward compatibility 2026-05-23 17:09:50 +08:00
YueGuobin a1f4942746 feat: add show_filters_icon property to Link for controlling Web UI filter icon display
This commit adds a new  property to the Link class, allowing users to control whether filter icons are displayed in the Web UI at the individual link level.

**Changes:**
- Added  attribute to Link class (default: True)
- Added  property getter
- Added  method for updating the property
- Updated  to include the new field with backward compatibility
- Added  field to LinkBase schema using Optional[bool] = Field(True, ...) pattern
- Updated API routes to handle the new field in create and update operations
- Added loading logic for show_filters_icon in project.open() to preserve settings when reopening projects

**Schema Definition:**
Uses the same pattern as the  field:

**API Impact:**
- POST /v3/projects/{project_id}/links - accepts  in request body
- PUT /v3/projects/{project_id}/links/{link_id} - can update
- GET /v3/projects/{project_id}/links/{link_id} - returns  field

**Future Applications:**
This feature provides granular control for future AI fault injection modules to manage link-level protocol failures while maintaining clean UI presentation.
2026-05-23 16:57:39 +08:00
YueGuobin 367c565d7c fix: ensure show_filters_icon is always returned in API responses
Set default value for show_filters_icon in LinkBase schema to ensure the field is always included in API responses, even when response_model_exclude_unset=True is used.
2026-05-23 15:21:44 +08:00
YueGuobin aa6845b32b feat: add show_filters_icon property to Link for controlling Web UI filter icon display
This commit adds a new `show_filters_icon` property to the Link class, allowing users to control whether filter icons are displayed in the Web UI at the individual link level.

**Changes:**
- Added `_show_filters_icon` attribute to Link class (default: True)
- Added `show_filters_icon` property getter
- Added `update_show_filters_icon()` method for updating the property
- Updated `asdict()` to include the new field in both topology and regular dumps
- Added `show_filters_icon` field to LinkBase schema
- Updated API routes to handle the new field in create and update operations

**API Impact:**
- POST /v3/projects/{project_id}/links - accepts `show_filters_icon` in request body
- PUT /v3/projects/{project_id}/links/{link_id} - can update `show_filters_icon`
- GET /v3/projects/{project_id}/links/{link_id} - returns `show_filters_icon` field

**Future Applications:**
This feature provides granular control for future AI fault injection modules to manage link-level protocol failures while maintaining clean UI presentation.
2026-05-23 14:57:58 +08:00
Guobin Yue 11fe3a24cf Merge branch '3.0' into fix/unclosed-client-session-websocket-proxy 2026-05-19 00:02:26 +08:00
YueGuobin 868e317d24 fix: close DockerHTTPClient session to prevent UnixConnector leak in Web Wireshark
The Web Wireshark WebSocket endpoint created a WebWiresharkManager but
never called close(), leaving the DockerHTTPClient's ClientSession with
UnixConnector unclosed when users closed the browser tab.

Also switch asyncio.wait in WebSocket proxy from ALL_COMPLETED to
FIRST_COMPLETED to avoid blocking cleanup when one direction disconnects.
2026-05-18 23:59:56 +08:00
grossmj 7cf25bc0d5 Release v3.1.0a2 2026-05-13 00:34:05 +08:00
grossmj 7bb0a2a863 Bundle web-ui v3.1.0a2 2026-05-13 00:29:25 +08:00
grossmj 748b8a1571 Sync appliances 2026-05-13 00:25:27 +08:00
YueGuobin 30a140a9e3 fix: surface tshark stderr errors to LLM, not just log them 2026-05-12 15:10:27 +08:00
YueGuobin b79fea918d fix: hint LLM about -c behavior when tshark returns empty with -c 2026-05-12 14:54:50 +08:00
YueGuobin 09b0fbcfea fix: update search_fields examples to use existing field names 2026-05-12 14:48:37 +08:00
YueGuobin 73e0de7706 fix: reject multi-keyword search_fields, single keyword only 2026-05-12 14:47:00 +08:00
YueGuobin c357d52fbf fix: document action/query params in PacketAnalysisTool description 2026-05-12 14:36:21 +08:00
YueGuobin 2a34d0ef58 feat: add search_fields action to PacketAnalysisTool for real-time field lookup 2026-05-12 14:33:13 +08:00
YueGuobin 32c3b4cd35 feat: validate tshark -e field names before running analysis 2026-05-12 14:27:16 +08:00
YueGuobin 9c23f9037c fix: document -c behavior caveat in PacketAnalysisTool description 2026-05-12 14:00:58 +08:00
YueGuobinandClaude Opus 4.7 b7a0b611df fix: demote loader-level load logs to DEBUG, info already reported by manager
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 13:52:05 +08:00
YueGuobinandClaude Opus 4.7 89da58fe1b fix: promote skills/prompts load logs to INFO level
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-12 13:46:36 +08:00
YueGuobin c5af9ce389 fix: load packet analysis protocols during skills initialization
- Add reload_packet_analysis_protocols() call in _ensure_skills_manager()
- PACKET_ANALYSIS_REGISTRY was empty because it was never loaded
2026-05-12 13:41:52 +08:00
YueGuobin 67b3b778b8 feat: add PacketAnalysisSkillsTool
- Register packet_analysis_skills as a LangChain tool for LLM
- LLM can query protocol field definitions before calling packet_analysis
- Follows the same pattern as DeviceSkillsTool and InjectionSkillsTool
2026-05-12 13:37:59 +08:00
YueGuobin 2a38816d3d fix: remove remaining quoted example from tool input docstring 2026-05-12 13:30:18 +08:00