110 Commits

Author SHA1 Message Date
YueGuobin
a6dcab8110
fix: avoid AttributeError when password is not provided in compute update
When updating a compute without providing a password field,
compute_update.password is None and calling .get_secret_value()
on it causes a 500 Internal Server Error. Only set the password
in update_values when a password is actually provided.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-02 15:37:46 +08:00
UmmmAGoodName
4cf8b79692 Fix addressing issue #2005 including tests 2026-04-05 21:51:52 +02:00
grossmj
1dc31127d9
Convert database template fields from PickleType to JSON 2026-04-03 20:59:57 +08:00
YueGuobin
2557614ba8 refactor(llm-configs):
remove unused reserved_jsonb fields

   Remove the three reserved JSONB fields (reserved_jsonb_1, reserved_jsonb_2, reserved_jsonb_3) from the llm_model_configs
   table. These fields were planned for future use but are no longer needed.
2026-03-17 21:43:48 +08:00
YueGuobin
bbe57f34b9 feat(llm): hide api_key from all API responses and document max_tokens as reserved
Security Enhancement:
   - API keys are now always filtered to null in all LLM config API responses
   - Created LLMModelConfigDataWithoutSecret schema for response validation
   - Added _filter_api_key_from_config() helper in API routes
   - Modified repository to always hide api_key in get_user_effective_configs()
   - Update/create operations still accept and store api_key securely

   Documentation Updates:
   - Updated API key visibility section to reflect new security policy
   - Updated all response examples to show api_key as null
   - Marked max_tokens field as reserved for future use
   - Added "Reserved Fields" section explaining unused fields

   This defense-in-depth approach prevents API keys from being leaked through
   logs, browser devtools, or network monitoring.
2026-03-14 13:12:22 +08:00
YueGuobin
5e9ec23a45 style: remove unused imports and fix E712 comparisons in llm_model_configs.py
Remove 5 unused imports flagged by ruff F401:
- sqlalchemy.func, sqlalchemy.or_
- sqlalchemy.ext.asyncio.AsyncSession
- sqlalchemy.orm.selectinload
- json

Fix 4 E712 errors - avoid equality comparisons to True:
- Use models.LLMModelConfig.is_default instead of == True

Co-Authored-By: Yue Guobin <yueguobin@outlook.com>
2026-03-10 01:10:19 +08:00
Guobin Yue
87f8f22524
Merge branch '3.0' into feature/ai-copilot-bridge 2026-03-06 01:42:47 +08:00
grossmj
4606613673
Fix image handling 2026-03-05 12:38:28 +08:00
YueGuobin
b780bfaf53 feat(agent): refactor LLM configuration handling to use centralized config
- Replace separate user_id and jwt_token parameters with unified llm_config dict
- Simplify model factory to accept llm_config directly instead of fetching from API
- Update llm_call and generate_title nodes to extract llm_config from LangGraph config
- Remove deprecated API fetching logic from model factory
- Maintain backward compatibility for existing tool usage patterns

This change centralizes LLM configuration management, reducing API calls and improving performance by passing configuration directly from the API layer rather than fetching it repeatedly.
2026-03-04 13:13:54 +08:00
YueGuobin
7a2d15cb64 feat: clarify default LLM model config selection logic
Updated documentation and implementation to clearly define the priority order for selecting default LLM model configurations. The logic now explicitly states:
1. User's config marked with `is_default: true` (highest priority)
2. Group's config marked with `is_default: true`
3. First config in the list (user configs come before group configs)

This ensures consistent behavior between the API documentation and the actual implementation in the repository code.
2026-03-03 22:56:21 +08:00
YueGuobin
f4723cd083 docs: restructure LLM model configs API documentation for nested config object
Update API documentation to reflect new nested `config` object structure in LLM model configurations endpoints. The response format now encapsulates provider-specific fields (provider, base_url, model, temperature, api_key, max_tokens) within a `config` object, while moving ownership metadata (user_id, group_id, is_default, version, created_at, updated_at) to the top level. This aligns the user-facing endpoints with the group configuration structure and improves API consistency.
2026-03-03 22:47:36 +08:00
YueGuobin
2e4c42fadc feat(api): clarify default LLM model config selection logic
- Update API documentation to explicitly describe default configuration selection logic
- Add fallback behavior in repository to use first config when no default is marked
- Clarify difference between `/default` endpoint and `default_config` field
- Document that `default_config` is never null when configs list is not empty
2026-03-03 22:27:28 +08:00
YueGuobin
b06593cf2a feat(docs): add default config endpoints and clarify config inheritance
- Add `/default` endpoints for users and groups to retrieve default LLM model configurations
- Update documentation to clarify that users receive both own and inherited configurations
- Improve response examples to show combined configs with source field
- Add 404 response example for missing default configurations
- Fix optimistic locking documentation formatting
2026-03-03 22:15:17 +08:00
YueGuobin
8fb4bf1117 feat(db): manually set timestamps for SQLite compatibility
Add explicit datetime.utcnow() assignments for created_at and updated_at fields in LLMModelConfigsRepository to ensure proper timestamp handling with SQLite. This addresses issues where SQLite may not automatically populate these timestamp fields during create and update operations. Changes applied to both user and group configuration methods for consistency.
2026-03-03 21:35:26 +08:00
YueGuobin
8e43ad4912 feat(db): replace PostgreSQL-specific JSONB with cross-database JSON type
- Change `JSONB` columns to generic `JSON` in LLMModelConfig model for database compatibility
- Update migration to use `sa.String(32)` for UUID and `sa.JSON()` for config fields
- Remove PostgreSQL dialect imports to support multiple database backends
- Keep existing constraints and indexes with PostgreSQL-specific annotations where needed
2026-03-03 18:03:16 +08:00
YueGuobin
fc7275878e feat(db): replace deferred unique constraints with partial unique indexes for LLM model configs
Replace deferred UniqueConstraints with partial unique indexes for user and group default configurations in the LLM model configs table. This change improves performance and ensures at most one default config per user/group while maintaining data integrity. The migration script has been updated accordingly to create and drop the new indexes.
2026-03-03 17:57:32 +08:00
YueGuobin
d42fff49a5 feat(docs): enhance LLM model configs API documentation with schema updates
- Add `model_type` field to database schema with supported values (text, vision, stt, tts, multimodal, embedding, reranking, other)
- Add `name` field as table-level column for indexing and filtering
- Add reserved JSONB fields for future extensibility
- Update API request/response schemas to include `model_type` and `name` fields
- Add new `LLMModelConfigWithSource` schema for detailed configuration responses
- Update usage examples to reflect new required fields
- Improve database constraints and indexing documentation
2026-03-03 17:54:30 +08:00
YueGuobin
161b7feb01 feat(api): add optimistic locking to LLM model config updates
- Include `version` field in all LLM model config response schemas
- Add `expected_version` parameter to update endpoints for optimistic locking
- Handle concurrent modification errors with HTTP 409 Conflict status
- Update both user and group config endpoints consistently
2026-03-03 16:59:12 +08:00
YueGuobin
8fad27d904 feat(db): improve database initialization and migration handling
- Add smarter database state detection during initialization to handle new databases, existing databases with new features, and old databases needing migration
- Make migration for llm_model_configs table idempotent to prevent conflicts when table already exists from code
- Add detailed logging for different database initialization scenarios
- Import sqlalchemy module for database inspection capabilities
2026-03-03 16:40:57 +08:00
YueGuobin
bbd34f1f67 feat(api): add LLM model configurations endpoint and schemas
- Introduce new API route `/access` for managing LLM model configurations
- Add LLMModelConfig model to database models
- Include comprehensive schemas for LLM model config CRUD operations
- Register new router with tags for LLM Model Configurations
2026-03-03 16:34:48 +08:00
grossmj
bdcb6445c7
Use lists for tags instead of dicts 2026-02-23 23:26:42 +08:00
YueGuobin
a3616e6d33 feat(api): add tag filtering to nodes and templates endpoints
- Add optional `tags` query parameter to `/nodes` and `/templates` endpoints
- Support filtering by tags in format "key:value" with multiple tags ANDed together
- Example: `?tags=vendor:cisco&tags=model:7200` filters nodes/templates with both tags
- Maintain backward compatibility for existing API usage without tags parameter
2026-02-22 22:52:28 +08:00
YueGuobin
56ece7ce88 Add vendor, model and netmiko_device_type fields to templates
Add three new optional fields to the template schema:
- vendor: Device vendor (e.g., Cisco, Juniper, Huawei)
- model: Device model (e.g., ISR4451-X, MX204, NE40E)
- netmiko_device_type: Netmiko device type for automation (e.g., cisco_ios, juniper_junos, huawei)

These fields enable AI/LLM integration to determine the appropriate
Netmiko device type when automating network device configuration.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-20 23:38:00 +08:00
grossmj
bfed330ec2
Fix bug when updating compute password to be empty 2026-02-16 19:23:46 +08:00
Bine
4a059be480 Increase DB engine pool size and max overflow
As described in #2568 when multiple clients connect simultaneously, an excessive number of database pool connections are created. This leads to resource exhaustion and connection management issues.

We have introduced the pool_size and max_overflow parameters to better control the database connection pool.
These parameters ensure the database can handle a sufficient number of concurrent connections without over-allocating resources.

Added pool_size and max_overflow arguments to the database initialization.

This change overrides the default initialization configuration where the temporary fix was applied.

Co-Authored-By: Ufi <86113085+ufiking@users.noreply.github.com>
2025-10-17 22:17:10 +02:00
grossmj
ac4c5f5985
Improve tests for pruning images 2025-01-03 19:56:41 +07:00
grossmj
4758431c76
Improvements for built-in disks
* Checksum is updated in the database for updated disks.
* It is not possible to prune them.
2025-01-02 23:10:51 +07:00
grossmj
b39c7541fb
Add timeout for waiting watchdog observer thread to complete 2024-12-31 17:24:19 +07:00
grossmj
82779d816f
Use watchdog instead of watchfiles to monitor for new images on the file system 2024-12-31 17:19:32 +07:00
grossmj
4e41aefbdb
Require "Project.Audit" permission to duplicate a project and check if "Project.Allocate" permission for the destination. 2024-12-06 14:09:08 +10:00
grossmj
2718224dde
Increase SQLite timeout. Ref #2422 2024-10-31 21:31:03 +10:00
grossmj
a1666dd247
Fix MAC address support for Docker containers 2024-09-18 18:02:35 +07:00
grossmj
998898a471
Fix CPU fractional values for Docker VMs. 2024-04-03 17:15:10 +07:00
grossmj
a27db6b4eb Fix deleting resource from resource pool. Ref #2293 2023-09-28 18:38:38 +10:00
Sylvain MATHIEU OBS
0d834ee2a0 rbac fix: cannot add multiple time same privilege to a role 2023-09-18 13:34:46 +02:00
grossmj
a95dda0d1d Complete resource pool support for projects 2023-09-11 18:15:03 +07:00
grossmj
d53ef175f8 DB and API for resource pools 2023-09-07 17:31:11 +07:00
grossmj
e72b07bf5c Prepare resource pools table for future development 2023-09-02 18:42:40 +07:00
grossmj
170f476733 Add more built-in roles 2023-09-02 18:15:00 +07:00
grossmj
0077fd98aa Add required privileges to all endpoints 2023-09-02 17:54:24 +07:00
grossmj
f3a4ad49f4 Check for group ACEs to find user privileges 2023-08-28 18:14:34 +10:00
grossmj
3e0592520b Handle ACE propagate and allowed 2023-08-28 12:06:01 +10:00
grossmj
57197c3d1c Comment unused code 2023-08-27 18:23:10 +10:00
grossmj
60ce1172e0 Use an ACL table to check for privileges 2023-08-27 18:20:42 +10:00
grossmj
6bd855b3c5 New database schema for better RBAC 2023-08-21 21:32:23 +10:00
grossmj
b0657b39ef Upgrade dependencies 2023-08-11 18:54:05 +10:00
grossmj
9c456532d0 Pydantic v2 migration 2023-08-04 18:20:06 +10:00
grossmj
40c265152b Add UEFI boot mode option for Qemu VMs 2023-06-21 17:56:34 +09:30
grossmj
ab1f9a4385 Handle Alembic CommandError exceptions 2023-01-13 12:52:31 +08:00
grossmj
a033080418 Support for database schema migrations using alembic 2023-01-11 10:15:04 +08:00