124 Commits

Author SHA1 Message Date
YueGuobin
49d2c271ca
Enable SQLite WAL mode to fix 'database is locked' errors under concurrent API requests 2026-06-15 13:02:17 +08:00
YueGuobin
8ecc35193d
feat: API key lifecycle — revoke/restore/delete
- POST /{id}/revoke: 吊销, revoked=True, 立即失效
- POST /{id}/restore: 恢复, revoked=False, 重新生效
- DELETE /{id}:       永久删除, 不可逆
- 列表接口显示所有 key 包括已吊销的
- 认证时过滤 revoked=False
2026-06-11 23:32:19 +08:00
YueGuobin
062e8dfbc4
chore: Remove unused revoke_api_key from repository 2026-06-11 23:28:43 +08:00
YueGuobin
a79bc7dd20
feat: Add API Key support for MCP authentication
- New db model: api_keys table with bcrypt-hashed keys
- New API: POST/GET/DELETE /v3/access/api-keys endpoints
- MCP _resolve_token: validates API keys, resolves to 5-min JWT
- API keys inherit the creating user's RBAC permissions
- MCP auth supports both JWT (24h) and API key (permanent) tokens
2026-06-11 22:54:03 +08:00
Cristi
03b3078271 fix: revert duplicate image check to fix failing tests 2026-06-06 15:04:06 +03:00
Cristi
1e40fa1b70 fix (templates): Add ordering to handle the duplicate cases gracefully 2026-06-06 14:46:27 +03:00
Cristi
da3a34144b fix(templates): Database error detected when saving a template with a disk image change 2026-06-06 14:21:15 +03:00
YueGuobin
f3aaa902c6
Fix double deletion issue in remove_resource_from_pool
Remove duplicate delete_resource call from remove_resource_from_pool since
the API layer already handles resource deletion. This prevents conflicts
where the API layer tries to delete a resource that was already deleted by
the repository layer.

The complete fix is now:
- remove_resource_from_pool: Only removes resource from pool (API handles deletion)
- delete_resource_pool: Deletes all resource records before deleting pool
2026-06-01 12:46:15 +08:00
YueGuobin
d2848a600e
Complete fix: delete resource records when deleting resource pool
This completes the fix from PR #2315 by ensuring that when a resource
pool is deleted, all associated resource records are also deleted from
the resources table, preventing orphaned resource records.

Changes:
- Modified delete_resource_pool() to first delete all resource records
  in the pool before deleting the pool itself
- This complements the existing fix in remove_resource_from_pool() which
  handles single resource removal
2026-06-01 12:43:56 +08:00
YueGuobin
fcb76a4b01
Apply fix from PR #2315: delete resource from resource table when removing from pool 2026-06-01 12:30:03 +08:00
YueGuobin
2efcc619b1
perf: batch RBAC permission checking for GET /projects
Replace per-project check_user_has_privilege calls with a single
batch method that performs 3 fixed DB queries regardless of project
count. Reduces GET /projects response time for 10000 projects from
~12s to ~290ms (40x improvement).
2026-05-27 13:27:35 +08:00
YueGuobin
ca5db7567c
fix: check both regular ACEs and resource pool ACEs for proper access control
This fix addresses a critical issue in the RBAC permission checking logic
introduced in PR #2750. When a project is shared through a resource pool,
both the project creator (with regular ACEs like "All endpoints") and the
shared user (with resource pool ACEs) should be able to see the project.

Changes:
- Modified `check_user_has_privilege` in `gns3server/db/repositories/rbac.py`
- Changed from if-elif (exclusive) to sequential (inclusive) checking
- Now checks regular ACEs first, then resource pool ACEs
- Both types of ACEs can grant access (OR logic instead of XOR)

This fixes the scenario where:
1. user100 has "All endpoints" ACE and creates a project
2. user100 shares the project via resource pool with user200
3. Both users should see the project (user100 via regular ACE, user200 via pool ACE)

Related to PR #2750 - RBAC user isolation implementation.
2026-05-27 00:17:52 +08:00
YueGuobin
9fca2181a8
feat: add independent LLMConfig permissions for AI profile management
Add new privilege definitions:
- LLMConfig.Audit - View LLM model configurations
- LLMConfig.Modify - Update LLM model configurations
- LLMConfig.Allocate - Create/delete LLM model configurations

Add LLMConfig.Audit and LLMConfig.Modify to default User role so that
regular users can manage their own AI profiles without needing the
User.Manager role.

User-scoped LLM config endpoints now use LLMConfig.* permissions.
Group-scoped LLM config endpoints retain Group.* permissions.
2026-05-26 14:24:41 +08:00
YueGuobin
5e4d9e057e
refactor: add efficient get_aces_for_path method for resource pool checks
Add a new repository method get_aces_for_path() that:
- Queries ACEs for a specific path at database level (more efficient)
- Preloads related user, group, and role objects to prevent 500 errors
- Keeps original get_aces() method unchanged to avoid performance impact

This improves both performance and code clarity for resource pool
deletion safety checks.
2026-05-26 12:59:37 +08:00
YueGuobin
a6dcab8110
fix: avoid AttributeError when password is not provided in compute update
When updating a compute without providing a password field,
compute_update.password is None and calling .get_secret_value()
on it causes a 500 Internal Server Error. Only set the password
in update_values when a password is actually provided.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-02 15:37:46 +08:00
UmmmAGoodName
4cf8b79692 Fix addressing issue #2005 including tests 2026-04-05 21:51:52 +02:00
grossmj
1dc31127d9
Convert database template fields from PickleType to JSON 2026-04-03 20:59:57 +08:00
YueGuobin
2557614ba8 refactor(llm-configs):
remove unused reserved_jsonb fields

   Remove the three reserved JSONB fields (reserved_jsonb_1, reserved_jsonb_2, reserved_jsonb_3) from the llm_model_configs
   table. These fields were planned for future use but are no longer needed.
2026-03-17 21:43:48 +08:00
YueGuobin
bbe57f34b9 feat(llm): hide api_key from all API responses and document max_tokens as reserved
Security Enhancement:
   - API keys are now always filtered to null in all LLM config API responses
   - Created LLMModelConfigDataWithoutSecret schema for response validation
   - Added _filter_api_key_from_config() helper in API routes
   - Modified repository to always hide api_key in get_user_effective_configs()
   - Update/create operations still accept and store api_key securely

   Documentation Updates:
   - Updated API key visibility section to reflect new security policy
   - Updated all response examples to show api_key as null
   - Marked max_tokens field as reserved for future use
   - Added "Reserved Fields" section explaining unused fields

   This defense-in-depth approach prevents API keys from being leaked through
   logs, browser devtools, or network monitoring.
2026-03-14 13:12:22 +08:00
YueGuobin
5e9ec23a45 style: remove unused imports and fix E712 comparisons in llm_model_configs.py
Remove 5 unused imports flagged by ruff F401:
- sqlalchemy.func, sqlalchemy.or_
- sqlalchemy.ext.asyncio.AsyncSession
- sqlalchemy.orm.selectinload
- json

Fix 4 E712 errors - avoid equality comparisons to True:
- Use models.LLMModelConfig.is_default instead of == True

Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-10 01:10:19 +08:00
Guobin Yue
87f8f22524
Merge branch '3.0' into feature/ai-copilot-bridge 2026-03-06 01:42:47 +08:00
grossmj
4606613673
Fix image handling 2026-03-05 12:38:28 +08:00
YueGuobin
b780bfaf53 feat(agent): refactor LLM configuration handling to use centralized config
- Replace separate user_id and jwt_token parameters with unified llm_config dict
- Simplify model factory to accept llm_config directly instead of fetching from API
- Update llm_call and generate_title nodes to extract llm_config from LangGraph config
- Remove deprecated API fetching logic from model factory
- Maintain backward compatibility for existing tool usage patterns

This change centralizes LLM configuration management, reducing API calls and improving performance by passing configuration directly from the API layer rather than fetching it repeatedly.
2026-03-04 13:13:54 +08:00
YueGuobin
7a2d15cb64 feat: clarify default LLM model config selection logic
Updated documentation and implementation to clearly define the priority order for selecting default LLM model configurations. The logic now explicitly states:
1. User's config marked with `is_default: true` (highest priority)
2. Group's config marked with `is_default: true`
3. First config in the list (user configs come before group configs)

This ensures consistent behavior between the API documentation and the actual implementation in the repository code.
2026-03-03 22:56:21 +08:00
YueGuobin
f4723cd083 docs: restructure LLM model configs API documentation for nested config object
Update API documentation to reflect new nested `config` object structure in LLM model configurations endpoints. The response format now encapsulates provider-specific fields (provider, base_url, model, temperature, api_key, max_tokens) within a `config` object, while moving ownership metadata (user_id, group_id, is_default, version, created_at, updated_at) to the top level. This aligns the user-facing endpoints with the group configuration structure and improves API consistency.
2026-03-03 22:47:36 +08:00
YueGuobin
2e4c42fadc feat(api): clarify default LLM model config selection logic
- Update API documentation to explicitly describe default configuration selection logic
- Add fallback behavior in repository to use first config when no default is marked
- Clarify difference between `/default` endpoint and `default_config` field
- Document that `default_config` is never null when configs list is not empty
2026-03-03 22:27:28 +08:00
YueGuobin
b06593cf2a feat(docs): add default config endpoints and clarify config inheritance
- Add `/default` endpoints for users and groups to retrieve default LLM model configurations
- Update documentation to clarify that users receive both own and inherited configurations
- Improve response examples to show combined configs with source field
- Add 404 response example for missing default configurations
- Fix optimistic locking documentation formatting
2026-03-03 22:15:17 +08:00
YueGuobin
8fb4bf1117 feat(db): manually set timestamps for SQLite compatibility
Add explicit datetime.utcnow() assignments for created_at and updated_at fields in LLMModelConfigsRepository to ensure proper timestamp handling with SQLite. This addresses issues where SQLite may not automatically populate these timestamp fields during create and update operations. Changes applied to both user and group configuration methods for consistency.
2026-03-03 21:35:26 +08:00
YueGuobin
8e43ad4912 feat(db): replace PostgreSQL-specific JSONB with cross-database JSON type
- Change `JSONB` columns to generic `JSON` in LLMModelConfig model for database compatibility
- Update migration to use `sa.String(32)` for UUID and `sa.JSON()` for config fields
- Remove PostgreSQL dialect imports to support multiple database backends
- Keep existing constraints and indexes with PostgreSQL-specific annotations where needed
2026-03-03 18:03:16 +08:00
YueGuobin
fc7275878e feat(db): replace deferred unique constraints with partial unique indexes for LLM model configs
Replace deferred UniqueConstraints with partial unique indexes for user and group default configurations in the LLM model configs table. This change improves performance and ensures at most one default config per user/group while maintaining data integrity. The migration script has been updated accordingly to create and drop the new indexes.
2026-03-03 17:57:32 +08:00
YueGuobin
d42fff49a5 feat(docs): enhance LLM model configs API documentation with schema updates
- Add `model_type` field to database schema with supported values (text, vision, stt, tts, multimodal, embedding, reranking, other)
- Add `name` field as table-level column for indexing and filtering
- Add reserved JSONB fields for future extensibility
- Update API request/response schemas to include `model_type` and `name` fields
- Add new `LLMModelConfigWithSource` schema for detailed configuration responses
- Update usage examples to reflect new required fields
- Improve database constraints and indexing documentation
2026-03-03 17:54:30 +08:00
YueGuobin
161b7feb01 feat(api): add optimistic locking to LLM model config updates
- Include `version` field in all LLM model config response schemas
- Add `expected_version` parameter to update endpoints for optimistic locking
- Handle concurrent modification errors with HTTP 409 Conflict status
- Update both user and group config endpoints consistently
2026-03-03 16:59:12 +08:00
YueGuobin
8fad27d904 feat(db): improve database initialization and migration handling
- Add smarter database state detection during initialization to handle new databases, existing databases with new features, and old databases needing migration
- Make migration for llm_model_configs table idempotent to prevent conflicts when table already exists from code
- Add detailed logging for different database initialization scenarios
- Import sqlalchemy module for database inspection capabilities
2026-03-03 16:40:57 +08:00
YueGuobin
bbd34f1f67 feat(api): add LLM model configurations endpoint and schemas
- Introduce new API route `/access` for managing LLM model configurations
- Add LLMModelConfig model to database models
- Include comprehensive schemas for LLM model config CRUD operations
- Register new router with tags for LLM Model Configurations
2026-03-03 16:34:48 +08:00
grossmj
bdcb6445c7
Use lists for tags instead of dicts 2026-02-23 23:26:42 +08:00
YueGuobin
a3616e6d33 feat(api): add tag filtering to nodes and templates endpoints
- Add optional `tags` query parameter to `/nodes` and `/templates` endpoints
- Support filtering by tags in format "key:value" with multiple tags ANDed together
- Example: `?tags=vendor:cisco&tags=model:7200` filters nodes/templates with both tags
- Maintain backward compatibility for existing API usage without tags parameter
2026-02-22 22:52:28 +08:00
YueGuobin
56ece7ce88 Add vendor, model and netmiko_device_type fields to templates
Add three new optional fields to the template schema:
- vendor: Device vendor (e.g., Cisco, Juniper, Huawei)
- model: Device model (e.g., ISR4451-X, MX204, NE40E)
- netmiko_device_type: Netmiko device type for automation (e.g., cisco_ios, juniper_junos, huawei)

These fields enable AI/LLM integration to determine the appropriate
Netmiko device type when automating network device configuration.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-20 23:38:00 +08:00
grossmj
bfed330ec2
Fix bug when updating compute password to be empty 2026-02-16 19:23:46 +08:00
Bine
4a059be480 Increase DB engine pool size and max overflow
As described in #2568 when multiple clients connect simultaneously, an excessive number of database pool connections are created. This leads to resource exhaustion and connection management issues.

We have introduced the pool_size and max_overflow parameters to better control the database connection pool.
These parameters ensure the database can handle a sufficient number of concurrent connections without over-allocating resources.

Added pool_size and max_overflow arguments to the database initialization.

This change overrides the default initialization configuration where the temporary fix was applied.

Co-Authored-By: Ufi <86113085+ufiking@users.noreply.github.com>
2025-10-17 22:17:10 +02:00
grossmj
ac4c5f5985
Improve tests for pruning images 2025-01-03 19:56:41 +07:00
grossmj
4758431c76
Improvements for built-in disks
* Checksum is updated in the database for updated disks.
* It is not possible to prune them.
2025-01-02 23:10:51 +07:00
grossmj
b39c7541fb
Add timeout for waiting watchdog observer thread to complete 2024-12-31 17:24:19 +07:00
grossmj
82779d816f
Use watchdog instead of watchfiles to monitor for new images on the file system 2024-12-31 17:19:32 +07:00
grossmj
4e41aefbdb
Require "Project.Audit" permission to duplicate a project and check if "Project.Allocate" permission for the destination. 2024-12-06 14:09:08 +10:00
grossmj
2718224dde
Increase SQLite timeout. Ref #2422 2024-10-31 21:31:03 +10:00
grossmj
a1666dd247
Fix MAC address support for Docker containers 2024-09-18 18:02:35 +07:00
grossmj
998898a471
Fix CPU fractional values for Docker VMs. 2024-04-03 17:15:10 +07:00
grossmj
a27db6b4eb Fix deleting resource from resource pool. Ref #2293 2023-09-28 18:38:38 +10:00
Sylvain MATHIEU OBS
0d834ee2a0 rbac fix: cannot add multiple time same privilege to a role 2023-09-18 13:34:46 +02:00
grossmj
a95dda0d1d Complete resource pool support for projects 2023-09-11 18:15:03 +07:00