139 Commits

Author SHA1 Message Date
YueGuobin
e0962a59ed
feat: add GET/PUT /v3/settings server settings API
GET returns all settings sections except the deprecated
VirtualBox/VMware ones (Server.Audit privilege); secrets are
masked and Controller.jwt_secret_key is excluded entirely.
PUT applies a partial update (Server.Modify privilege): masked
or empty secrets mean unchanged, null removes the option, the
response carries the new values plus restart_required, and a
settings.updated notification is emitted. New
Server.Audit/Server.Modify privileges are seeded into the
Administrator role at table creation.
2026-08-23 18:54:31 +08:00
YueGuobin
300c53e6fb
templates: persist appliance metadata on install
Appliance fields that describe the appliance (vendor information, default
credentials, installation instructions...) were dropped when installing a
template. Keep them in a new appliance_metadata JSON column on the
templates table, filled by the appliance-to-template conversion for both
registry v1-6 and v8 (version level values override the appliance level
ones). The nested schema allows extra fields so future registry fields
persist without a migration.
2026-08-21 21:41:42 +08:00
YueGuobin
9260108f53
templates: add netmiko_device_type for automation tools
Common template field (schema + templates table column + Alembic
migration) holding the Netmiko device type (e.g. 'cisco_xr', 'nokia_srl')
so Netmiko/Nornir based tooling can look up how to reach a node's CLI
without hard-coded vendor mappings. Free-form lowercase string on
purpose: Netmiko's platform list evolves independently of GNS3.
2026-08-21 21:41:35 +08:00
YueGuobin
088f1da77a
docker: persist extra_configs in the template DB table
The extra_configs schema field needs a column on the docker_templates table
to actually round-trip through the controller DB (the schema alone is
accepted but dropped by the SQLAlchemy model mapping). Add the JSON column
and an Alembic migration so existing databases get it on upgrade.
2026-08-14 00:20:08 +08:00
Jeremy Grossmann
909ccf8fcd
Merge branch '3.1' into base-configs-3.0 2026-08-08 22:11:39 +02:00
YueGuobin
59e5f8dd2f
Fix review issues: key_prefix length, count validation, WAL log, timeout comment, pointless temp var 2026-06-16 22:50:22 +08:00
YueGuobin
a8cc01af46
Remove database warmup (proven ineffective - real bottleneck was bcrypt blocking event loop) 2026-06-16 00:35:42 +08:00
YueGuobin
ccb629f48f
Clean up all timing/debug logs
Remove all [MCP-TIMING] and [CTRL-TIMING] log lines, timing middleware,
and related import time statements across 11 files.
2026-06-16 00:34:11 +08:00
YueGuobin
3c41dea48f
Replace SELECT 1 warmup with full database file read to warm OS page cache
Reading the entire DB file into OS page cache eliminates the 8-14s
cold-start penalty on the first ORM query (users/templates/api_keys).
Previous SELECT 1 only warmed the connection pool, not the file cache.
2026-06-15 23:43:41 +08:00
YueGuobin
b3619a3c00
Add granular timing to get_template: separate execute vs fetch time 2026-06-15 23:33:19 +08:00
YueGuobin
e90fb0774c
Warm up database connection pool on startup to avoid 8s cold-start penalty on first API request 2026-06-15 23:26:28 +08:00
YueGuobin
6d3cfc7d1e
Fix: register WAL PRAGMA on sync_engine instead of Engine class for async compat
- Move event listener from module-level @Engine to engine.sync_engine
- Add WAL mode verification log on startup
- Remove unused Engine class-level listener
2026-06-15 23:21:50 +08:00
YueGuobin
08e0dbc122
Add timing logs to get_template to identify DB query bottleneck 2026-06-15 23:18:00 +08:00
YueGuobin
49d2c271ca
Enable SQLite WAL mode to fix 'database is locked' errors under concurrent API requests 2026-06-15 13:02:17 +08:00
YueGuobin
8ecc35193d
feat: API key lifecycle — revoke/restore/delete
- POST /{id}/revoke: 吊销, revoked=True, 立即失效
- POST /{id}/restore: 恢复, revoked=False, 重新生效
- DELETE /{id}:       永久删除, 不可逆
- 列表接口显示所有 key 包括已吊销的
- 认证时过滤 revoked=False
2026-06-11 23:32:19 +08:00
YueGuobin
062e8dfbc4
chore: Remove unused revoke_api_key from repository 2026-06-11 23:28:43 +08:00
YueGuobin
a79bc7dd20
feat: Add API Key support for MCP authentication
- New db model: api_keys table with bcrypt-hashed keys
- New API: POST/GET/DELETE /v3/access/api-keys endpoints
- MCP _resolve_token: validates API keys, resolves to 5-min JWT
- API keys inherit the creating user's RBAC permissions
- MCP auth supports both JWT (24h) and API key (permanent) tokens
2026-06-11 22:54:03 +08:00
Cristi
03b3078271 fix: revert duplicate image check to fix failing tests 2026-06-06 15:04:06 +03:00
Cristi
1e40fa1b70 fix (templates): Add ordering to handle the duplicate cases gracefully 2026-06-06 14:46:27 +03:00
Cristi
da3a34144b fix(templates): Database error detected when saving a template with a disk image change 2026-06-06 14:21:15 +03:00
Jeremy Grossmann
12347b24aa
Merge branch '3.1' into base-configs-3.0 2026-06-01 11:39:40 +02:00
YueGuobin
f3aaa902c6
Fix double deletion issue in remove_resource_from_pool
Remove duplicate delete_resource call from remove_resource_from_pool since
the API layer already handles resource deletion. This prevents conflicts
where the API layer tries to delete a resource that was already deleted by
the repository layer.

The complete fix is now:
- remove_resource_from_pool: Only removes resource from pool (API handles deletion)
- delete_resource_pool: Deletes all resource records before deleting pool
2026-06-01 12:46:15 +08:00
YueGuobin
d2848a600e
Complete fix: delete resource records when deleting resource pool
This completes the fix from PR #2315 by ensuring that when a resource
pool is deleted, all associated resource records are also deleted from
the resources table, preventing orphaned resource records.

Changes:
- Modified delete_resource_pool() to first delete all resource records
  in the pool before deleting the pool itself
- This complements the existing fix in remove_resource_from_pool() which
  handles single resource removal
2026-06-01 12:43:56 +08:00
YueGuobin
fcb76a4b01
Apply fix from PR #2315: delete resource from resource table when removing from pool 2026-06-01 12:30:03 +08:00
YueGuobin
2efcc619b1
perf: batch RBAC permission checking for GET /projects
Replace per-project check_user_has_privilege calls with a single
batch method that performs 3 fixed DB queries regardless of project
count. Reduces GET /projects response time for 10000 projects from
~12s to ~290ms (40x improvement).
2026-05-27 13:27:35 +08:00
YueGuobin
ca5db7567c
fix: check both regular ACEs and resource pool ACEs for proper access control
This fix addresses a critical issue in the RBAC permission checking logic
introduced in PR #2750. When a project is shared through a resource pool,
both the project creator (with regular ACEs like "All endpoints") and the
shared user (with resource pool ACEs) should be able to see the project.

Changes:
- Modified `check_user_has_privilege` in `gns3server/db/repositories/rbac.py`
- Changed from if-elif (exclusive) to sequential (inclusive) checking
- Now checks regular ACEs first, then resource pool ACEs
- Both types of ACEs can grant access (OR logic instead of XOR)

This fixes the scenario where:
1. user100 has "All endpoints" ACE and creates a project
2. user100 shares the project via resource pool with user200
3. Both users should see the project (user100 via regular ACE, user200 via pool ACE)

Related to PR #2750 - RBAC user isolation implementation.
2026-05-27 00:17:52 +08:00
YueGuobin
9fca2181a8
feat: add independent LLMConfig permissions for AI profile management
Add new privilege definitions:
- LLMConfig.Audit - View LLM model configurations
- LLMConfig.Modify - Update LLM model configurations
- LLMConfig.Allocate - Create/delete LLM model configurations

Add LLMConfig.Audit and LLMConfig.Modify to default User role so that
regular users can manage their own AI profiles without needing the
User.Manager role.

User-scoped LLM config endpoints now use LLMConfig.* permissions.
Group-scoped LLM config endpoints retain Group.* permissions.
2026-05-26 14:24:41 +08:00
YueGuobin
5e4d9e057e
refactor: add efficient get_aces_for_path method for resource pool checks
Add a new repository method get_aces_for_path() that:
- Queries ACEs for a specific path at database level (more efficient)
- Preloads related user, group, and role objects to prevent 500 errors
- Keeps original get_aces() method unchanged to avoid performance impact

This improves both performance and code clarity for resource pool
deletion safety checks.
2026-05-26 12:59:37 +08:00
Volobue
b74ebafb8f api endpoint implemented 2026-05-25 20:07:05 +03:00
YueGuobin
a6dcab8110
fix: avoid AttributeError when password is not provided in compute update
When updating a compute without providing a password field,
compute_update.password is None and calling .get_secret_value()
on it causes a 500 Internal Server Error. Only set the password
in update_values when a password is actually provided.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-02 15:37:46 +08:00
UmmmAGoodName
4cf8b79692 Fix addressing issue #2005 including tests 2026-04-05 21:51:52 +02:00
grossmj
1dc31127d9
Convert database template fields from PickleType to JSON 2026-04-03 20:59:57 +08:00
YueGuobin
2557614ba8 refactor(llm-configs):
remove unused reserved_jsonb fields

   Remove the three reserved JSONB fields (reserved_jsonb_1, reserved_jsonb_2, reserved_jsonb_3) from the llm_model_configs
   table. These fields were planned for future use but are no longer needed.
2026-03-17 21:43:48 +08:00
YueGuobin
bbe57f34b9 feat(llm): hide api_key from all API responses and document max_tokens as reserved
Security Enhancement:
   - API keys are now always filtered to null in all LLM config API responses
   - Created LLMModelConfigDataWithoutSecret schema for response validation
   - Added _filter_api_key_from_config() helper in API routes
   - Modified repository to always hide api_key in get_user_effective_configs()
   - Update/create operations still accept and store api_key securely

   Documentation Updates:
   - Updated API key visibility section to reflect new security policy
   - Updated all response examples to show api_key as null
   - Marked max_tokens field as reserved for future use
   - Added "Reserved Fields" section explaining unused fields

   This defense-in-depth approach prevents API keys from being leaked through
   logs, browser devtools, or network monitoring.
2026-03-14 13:12:22 +08:00
YueGuobin
5e9ec23a45 style: remove unused imports and fix E712 comparisons in llm_model_configs.py
Remove 5 unused imports flagged by ruff F401:
- sqlalchemy.func, sqlalchemy.or_
- sqlalchemy.ext.asyncio.AsyncSession
- sqlalchemy.orm.selectinload
- json

Fix 4 E712 errors - avoid equality comparisons to True:
- Use models.LLMModelConfig.is_default instead of == True

Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-10 01:10:19 +08:00
Guobin Yue
87f8f22524
Merge branch '3.0' into feature/ai-copilot-bridge 2026-03-06 01:42:47 +08:00
grossmj
4606613673
Fix image handling 2026-03-05 12:38:28 +08:00
YueGuobin
b780bfaf53 feat(agent): refactor LLM configuration handling to use centralized config
- Replace separate user_id and jwt_token parameters with unified llm_config dict
- Simplify model factory to accept llm_config directly instead of fetching from API
- Update llm_call and generate_title nodes to extract llm_config from LangGraph config
- Remove deprecated API fetching logic from model factory
- Maintain backward compatibility for existing tool usage patterns

This change centralizes LLM configuration management, reducing API calls and improving performance by passing configuration directly from the API layer rather than fetching it repeatedly.
2026-03-04 13:13:54 +08:00
YueGuobin
7a2d15cb64 feat: clarify default LLM model config selection logic
Updated documentation and implementation to clearly define the priority order for selecting default LLM model configurations. The logic now explicitly states:
1. User's config marked with `is_default: true` (highest priority)
2. Group's config marked with `is_default: true`
3. First config in the list (user configs come before group configs)

This ensures consistent behavior between the API documentation and the actual implementation in the repository code.
2026-03-03 22:56:21 +08:00
YueGuobin
f4723cd083 docs: restructure LLM model configs API documentation for nested config object
Update API documentation to reflect new nested `config` object structure in LLM model configurations endpoints. The response format now encapsulates provider-specific fields (provider, base_url, model, temperature, api_key, max_tokens) within a `config` object, while moving ownership metadata (user_id, group_id, is_default, version, created_at, updated_at) to the top level. This aligns the user-facing endpoints with the group configuration structure and improves API consistency.
2026-03-03 22:47:36 +08:00
YueGuobin
2e4c42fadc feat(api): clarify default LLM model config selection logic
- Update API documentation to explicitly describe default configuration selection logic
- Add fallback behavior in repository to use first config when no default is marked
- Clarify difference between `/default` endpoint and `default_config` field
- Document that `default_config` is never null when configs list is not empty
2026-03-03 22:27:28 +08:00
YueGuobin
b06593cf2a feat(docs): add default config endpoints and clarify config inheritance
- Add `/default` endpoints for users and groups to retrieve default LLM model configurations
- Update documentation to clarify that users receive both own and inherited configurations
- Improve response examples to show combined configs with source field
- Add 404 response example for missing default configurations
- Fix optimistic locking documentation formatting
2026-03-03 22:15:17 +08:00
YueGuobin
8fb4bf1117 feat(db): manually set timestamps for SQLite compatibility
Add explicit datetime.utcnow() assignments for created_at and updated_at fields in LLMModelConfigsRepository to ensure proper timestamp handling with SQLite. This addresses issues where SQLite may not automatically populate these timestamp fields during create and update operations. Changes applied to both user and group configuration methods for consistency.
2026-03-03 21:35:26 +08:00
YueGuobin
8e43ad4912 feat(db): replace PostgreSQL-specific JSONB with cross-database JSON type
- Change `JSONB` columns to generic `JSON` in LLMModelConfig model for database compatibility
- Update migration to use `sa.String(32)` for UUID and `sa.JSON()` for config fields
- Remove PostgreSQL dialect imports to support multiple database backends
- Keep existing constraints and indexes with PostgreSQL-specific annotations where needed
2026-03-03 18:03:16 +08:00
YueGuobin
fc7275878e feat(db): replace deferred unique constraints with partial unique indexes for LLM model configs
Replace deferred UniqueConstraints with partial unique indexes for user and group default configurations in the LLM model configs table. This change improves performance and ensures at most one default config per user/group while maintaining data integrity. The migration script has been updated accordingly to create and drop the new indexes.
2026-03-03 17:57:32 +08:00
YueGuobin
d42fff49a5 feat(docs): enhance LLM model configs API documentation with schema updates
- Add `model_type` field to database schema with supported values (text, vision, stt, tts, multimodal, embedding, reranking, other)
- Add `name` field as table-level column for indexing and filtering
- Add reserved JSONB fields for future extensibility
- Update API request/response schemas to include `model_type` and `name` fields
- Add new `LLMModelConfigWithSource` schema for detailed configuration responses
- Update usage examples to reflect new required fields
- Improve database constraints and indexing documentation
2026-03-03 17:54:30 +08:00
YueGuobin
161b7feb01 feat(api): add optimistic locking to LLM model config updates
- Include `version` field in all LLM model config response schemas
- Add `expected_version` parameter to update endpoints for optimistic locking
- Handle concurrent modification errors with HTTP 409 Conflict status
- Update both user and group config endpoints consistently
2026-03-03 16:59:12 +08:00
YueGuobin
8fad27d904 feat(db): improve database initialization and migration handling
- Add smarter database state detection during initialization to handle new databases, existing databases with new features, and old databases needing migration
- Make migration for llm_model_configs table idempotent to prevent conflicts when table already exists from code
- Add detailed logging for different database initialization scenarios
- Import sqlalchemy module for database inspection capabilities
2026-03-03 16:40:57 +08:00
YueGuobin
bbd34f1f67 feat(api): add LLM model configurations endpoint and schemas
- Introduce new API route `/access` for managing LLM model configurations
- Add LLMModelConfig model to database models
- Include comprehensive schemas for LLM model config CRUD operations
- Register new router with tags for LLM Model Configurations
2026-03-03 16:34:48 +08:00
grossmj
bdcb6445c7
Use lists for tags instead of dicts 2026-02-23 23:26:42 +08:00