gns3-server/docs/llm-model-configs-api.md

LLM Model Configurations API

Overview

This API provides LLM model configuration management for users and user groups with inheritance support.

Key Features

• User-level configurations: Each user can have their own LLM model configurations
• Group-level configurations: User groups can share LLM model configurations
• Inheritance: Users automatically inherit configurations from their groups (when they have no own configs)
• Default configuration: Both users and groups can set a default configuration
• API Key Encryption: API keys are automatically encrypted in the database
• Optimistic Locking: Prevents concurrent modification conflicts using version tracking

Inheritance Logic

User requests configs:
  ├─ Always return user's own configs (if any)
  └─ Always return inherited group configs (if any)

Note: Users can see both their own configurations AND configurations inherited from their groups. The source field in the response indicates the origin of each configuration.

Configuration Priority

User's own config > User's group config

---

Database Schema

Table: llm_model_configs

Column	Type	Description
config_id	UUID	Primary key
name	VARCHAR(100)	Configuration name (table-level for indexing)
model_type	VARCHAR(50)	Model type (table-level for filtering)
config	JSONB	Configuration data (provider, base_url, model, temperature, api_key, etc.)
user_id	UUID (nullable)	Foreign key to users table
group_id	UUID (nullable)	Foreign key to user_groups table
is_default	BOOLEAN	Default configuration flag
version	INTEGER	Optimistic locking version (starts at 0, increments on each update)
reserved_jsonb_1	JSONB (nullable)	Reserved field for future use
reserved_jsonb_2	JSONB (nullable)	Reserved field for future use
reserved_jsonb_3	JSONB (nullable)	Reserved field for future use
created_at	TIMESTAMP	Creation timestamp
updated_at	TIMESTAMP	Last update timestamp

Model Types

The model_type field accepts the following values:

• text - Text generation models
• vision - Vision/image understanding models
• stt - Speech-to-Text models
• tts - Text-to-Speech models
• multimodal - Multimodal models supporting multiple input types
• embedding - Text embedding models
• reranking - Reranking models
• other - Other model types

Constraints

• Each config belongs to either a user or a group (not both)
• Each user can have at most one default configuration
• Each group can have at most one default configuration
• version field is automatically incremented on each update

---

API Endpoints

User Configuration Endpoints

Method	Path	Description	Privilege
GET	/v3/access/users/{user_id}/llm-model-configs	Get user's effective configs (own + inherited)	User.Audit
GET	/v3/access/users/{user_id}/llm-model-configs/own	Get user's own configs only	User.Audit
GET	/v3/access/users/{user_id}/llm-model-configs/default	Get user's default configuration	User.Audit
POST	/v3/access/users/{user_id}/llm-model-configs	Create a new configuration	User.Modify
PUT	/v3/access/users/{user_id}/llm-model-configs/{config_id}	Update a configuration	User.Modify
DELETE	/v3/access/users/{user_id}/llm-model-configs/{config_id}	Delete a configuration	User.Modify
PUT	/v3/access/users/{user_id}/llm-model-configs/default/{config_id}	Set default configuration	User.Modify

Group Configuration Endpoints

Method	Path	Description	Privilege
GET	/v3/access/groups/{group_id}/llm-model-configs	Get all group configurations	Group.Audit
GET	/v3/access/groups/{group_id}/llm-model-configs/default	Get group's default configuration	Group.Audit
POST	/v3/access/groups/{group_id}/llm-model-configs	Create a new configuration	Group.Modify
PUT	/v3/access/groups/{group_id}/llm-model-configs/{config_id}	Update a configuration	Group.Modify
DELETE	/v3/access/groups/{group_id}/llm-model-configs/{config_id}	Delete a configuration	Group.Modify
PUT	/v3/access/groups/{group_id}/llm-model-configs/default/{config_id}	Set default configuration	Group.Modify

Note: The GET endpoints for groups return the same structure as user endpoints: configs, default_config, and total.

---

Request/Response Schemas

LLMModelConfigCreate

Required Fields:

Field	Type	Description
name	string	Configuration name (1-100 chars)
model_type	string	Model type (text, vision, stt, tts, multimodal, embedding, reranking, other)
provider	string	LLM provider (e.g., "openai", "anthropic", "ollama")
base_url	string	API base URL
model	string	Model name
temperature	float	Temperature (0.0-2.0, default: 0.7)

Optional Fields:

Field	Type	Description
api_key	string	API key (auto-encrypted)
max_tokens	integer	Max tokens for generation
is_default	boolean	Set as default (default: false)

Extra Fields: Any custom fields are supported for future extensibility.

LLMModelConfigUpdate

Field	Type	Description
name	string (optional)	Configuration name
model_type	string (optional)	Model type
provider	string (optional)	LLM provider
base_url	string (optional)	API base URL
model	string (optional)	Model name
temperature	float (optional)	Temperature
api_key	string (optional)	API key
max_tokens	integer (optional)	Max tokens
is_default	boolean (optional)	Default flag
expected_version	integer (optional)	Optimistic locking version

Note: When using expected_version, the API will verify the version hasn't changed since you read the data. If it has, you'll receive a 409 Conflict error.

LLMModelConfigResponse

Field	Type	Description
config_id	UUID	Configuration ID
name	string	Configuration name
model_type	string	Model type
config	LLMModelConfigData	Configuration data (provider, base_url, model, temperature, etc.)
user_id	UUID (nullable)	Owner user ID
group_id	UUID (nullable)	Owner group ID
is_default	boolean	Default flag
version	integer	Current version number (for optimistic locking)
created_at	TIMESTAMP	Creation time
updated_at	TIMESTAMP	Last update time

LLMModelConfigInheritedResponse

Field	Type	Description
configs	list[LLMModelConfigWithSource]	Effective configurations
default_config	LLMModelConfigWithSource (nullable)	Default configuration (never null if configs list is not empty)
total	integer	Total count

Default Configuration Selection Logic:

1. User's config marked with is_default: true (highest priority)
2. Group's config marked with is_default: true
3. First config in the list (user configs come before group configs)

LLMModelConfigListResponse

Field	Type	Description
configs	list[LLMModelConfigResponse]	Configuration list
default_config	LLMModelConfigResponse (nullable)	Default configuration (never null if configs list is not empty)
total	integer	Total count

Default Configuration Selection Logic:

1. Config marked with is_default: true
2. First config in the list (fallback if no default is marked)

Usage: This schema is used for group configuration endpoints (e.g., GET /groups/{group_id}/llm-model-configs).

LLMModelConfigWithSource

Field	Type	Description
config_id	UUID	Configuration ID
name	string	Configuration name
model_type	string	Model type
config	LLMModelConfigData	Configuration data (provider, base_url, model, temperature, etc.)
user_id	UUID (nullable)	Owner user ID
group_id	UUID (nullable)	Owner group ID
is_default	boolean	Default flag
version	integer	Optimistic locking version
created_at	TIMESTAMP	Creation time
updated_at	TIMESTAMP	Last update time
source	string	Source: "user" or "group"
group_name	string (nullable)	Group name if source is "group"

---

Usage Examples

1. Create a user configuration

curl -X POST http://localhost:3080/v3/access/users/{user_id}/llm-model-configs \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "GPT-4",
    "model_type": "text",
    "provider": "openai",
    "base_url": "https://api.openai.com/v1",
    "model": "gpt-4",
    "temperature": 0.7,
    "api_key": "sk-xxx",
    "is_default": true
  }'

Response:

{
  "config_id": "uuid-1",
  "name": "GPT-4",
  "model_type": "text",
  "config": {
    "provider": "openai",
    "base_url": "https://api.openai.com/v1",
    "model": "gpt-4",
    "temperature": 0.7,
    "api_key": "sk-xxx"
  },
  "user_id": "uuid-user",
  "group_id": null,
  "is_default": true,
  "version": 0,
  "created_at": "2026-03-03T12:00:00Z",
  "updated_at": "2026-03-03T12:00:00Z"
}

2. Create a group configuration

curl -X POST http://localhost:3080/v3/access/groups/{group_id}/llm-model-configs \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Claude-3",
    "model_type": "text",
    "provider": "anthropic",
    "base_url": "https://api.anthropic.com",
    "model": "claude-3-opus-20240229",
    "temperature": 0.7,
    "api_key": "sk-ant-xxx",
    "is_default": true
  }'

3. Get user's effective configurations (with inheritance)

curl -X GET http://localhost:3080/v3/access/users/{user_id}/llm-model-configs \
  -H "Authorization: Bearer <token>"

Response (user has both own configs and inherited group configs):

{
  "configs": [
    {
      "config_id": "uuid-1",
      "name": "GPT-4",
      "model_type": "text",
      "config": {
        "provider": "openai",
        "base_url": "https://api.openai.com/v1",
        "model": "gpt-4",
        "temperature": 0.7,
        "api_key": "sk-xxx"
      },
      "user_id": "uuid-user",
      "group_id": null,
      "is_default": true,
      "version": 0,
      "created_at": "2026-03-03T14:32:48.158880Z",
      "updated_at": "2026-03-03T14:32:48.158880Z",
      "source": "user",
      "group_name": null
    },
    {
      "config_id": "uuid-2",
      "name": "Claude-3",
      "model_type": "text",
      "config": {
        "provider": "anthropic",
        "base_url": "https://api.anthropic.com",
        "model": "claude-3-opus-20240229",
        "temperature": 0.7,
        "api_key": null
      },
      "user_id": null,
      "group_id": "uuid-group",
      "is_default": true,
      "version": 0,
      "created_at": "2026-03-03T14:32:48.158880Z",
      "updated_at": "2026-03-03T14:32:48.158880Z",
      "source": "group",
      "group_name": "Developers"
    }
  ],
  "default_config": {
    "config_id": "uuid-1",
    "name": "GPT-4",
    "model_type": "text",
    "config": {
      "provider": "openai",
      ...
    },
    "user_id": "uuid-user",
    "group_id": null,
    "is_default": true,
    "version": 0,
    ...
  },
  "total": 2
}

Note:

• User's own config shows config.api_key: "sk-xxx" (visible to owner)
• Inherited group config shows config.api_key: null (hidden from users)
• source: "user" indicates the config belongs to the user
• source: "group" indicates the config is inherited from a group
• Configuration fields are nested in the config object (same structure as group endpoints)

4. Get group configurations

curl -X GET http://localhost:3080/v3/access/groups/{group_id}/llm-model-configs \
  -H "Authorization: Bearer <token>"

Response:

{
  "configs": [
    {
      "config_id": "uuid-1",
      "name": "Claude-3",
      "model_type": "text",
      "config": {
        "provider": "anthropic",
        "base_url": "https://api.anthropic.com",
        "model": "claude-3-opus-20240229",
        "temperature": 0.7,
        "api_key": "sk-ant-xxx"
      },
      "user_id": null,
      "group_id": "uuid-group",
      "is_default": true,
      "version": 0,
      "created_at": "2026-03-03T12:00:00Z",
      "updated_at": "2026-03-03T12:00:00Z"
    },
    {
      "config_id": "uuid-2",
      "name": "GPT-4",
      "model_type": "text",
      "config": {
        "provider": "openai",
        "base_url": "https://api.openai.com/v1",
        "model": "gpt-4",
        "temperature": 0.7,
        "api_key": "sk-xxx"
      },
      "user_id": null,
      "group_id": "uuid-group",
      "is_default": false,
      "version": 0,
      "created_at": "2026-03-03T12:00:00Z",
      "updated_at": "2026-03-03T12:00:00Z"
    }
  ],
  "default_config": {
    "config_id": "uuid-1",
    "name": "Claude-3",
    "model_type": "text",
    "config": {
      "provider": "anthropic",
      ...
    },
    "user_id": null,
    "group_id": "uuid-group",
    "is_default": true,
    "version": 0,
    ...
  },
  "total": 2
}

Note: The response structure is the same as user endpoints, with configs, default_config, and total fields.

5. Update a configuration (without optimistic locking)

curl -X PUT http://localhost:3080/v3/access/users/{user_id}/llm-model-configs/{config_id} \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "temperature": 0.9,
    "max_tokens": 4000
  }'

6. Update a configuration (WITH optimistic locking)

Best practice for avoiding concurrent modification conflicts:

# Step 1: Read the config (get the current version)
curl -X GET http://localhost:3080/v3/access/users/{user_id}/llm-model-configs/own \
  -H "Authorization: Bearer <token>"

# Response includes "version": 5

# Step 2: Update with expected_version
curl -X PUT http://localhost:3080/v3/access/users/{user_id}/llm-model-configs/{config_id} \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "temperature": 0.9,
    "max_tokens": 4000,
    "expected_version": 5
  }'

# Response includes incremented "version": 6

If someone else modified the config before you:

HTTP 409 Conflict
{
  "detail": "Concurrent modification detected. Expected version 5, but current version is 6. Please retry."
}

Client retry flow:

1. Receive 409 Conflict error
2. Re-fetch the config to get the latest version
3. Apply your changes on top of the latest data
4. Retry the update with the new expected_version

7. Set default configuration

curl -X PUT http://localhost:3080/v3/access/users/{user_id}/llm-model-configs/default/{config_id} \
  -H "Authorization: Bearer <token>"

8. Get default configuration

Get the user's default configuration:

curl -X GET http://localhost:3080/v3/access/users/{user_id}/llm-model-configs/default \
  -H "Authorization: Bearer <token>"

Note: This endpoint only returns configurations explicitly marked with is_default: true. If no configuration is marked as default, it returns 404.

Response:

{
  "config_id": "uuid-1",
  "name": "GPT-4",
  "model_type": "text",
  "config": {
    "provider": "openai",
    "base_url": "https://api.openai.com/v1",
    "model": "gpt-4",
    "temperature": 0.7,
    "api_key": "sk-xxx"
  },
  "user_id": "uuid-user",
  "group_id": null,
  "is_default": true,
  "version": 0,
  "created_at": "2026-03-03T18:15:00Z",
  "updated_at": "2026-03-03T18:15:00Z"
}

If no default configuration is set:

HTTP 404 Not Found
{
  "detail": "No default LLM model configuration found for user '{user_id}'"
}

Get the group's default configuration:

curl -X GET http://localhost:3080/v3/access/groups/{group_id}/llm-model-configs/default \
  -H "Authorization: Bearer <token>"

The response format is the same as for users.

---

Important Note: This dedicated /default endpoint is different from the default_config field in the list response:

• /default endpoint: Requires explicit is_default: true flag, returns 404 if not found
• default_config field in list: Falls back to first config if no explicit default is marked

9. Delete a configuration

curl -X DELETE http://localhost:3080/v3/access/users/{user_id}/llm-model-configs/{config_id} \
  -H "Authorization: Bearer <token>"

---

Error Codes

Status	Description
200	Success
201	Created
204	Deleted (no content)
400	Bad request
401	Unauthorized
404	Not found
409	Conflict (optimistic lock violation)
500	Server error

409 Conflict Response

{
  "detail": "Concurrent modification detected. Expected version 5, but current version is 6. Please retry."
}

---

Concurrency Control

Optimistic Locking

This API uses optimistic locking to prevent concurrent modification conflicts:

1. Version Tracking: Each configuration has a version field that starts at 0 and increments on each update
2. Read-Modify-Write: When updating, clients should include the expected_version from their last read
3. Conflict Detection: If the provided version doesn't match the current version, the update is rejected with HTTP 409

When to Use Optimistic Locking

Use expected_version when:

• Multiple users/admins might modify the same configuration
• You want to prevent accidental overwrites of concurrent changes
• Building interactive UIs that display and edit configurations

Skip expected_version when:

• You're sure no one else is modifying the config
• Performance is more important than data integrity (not recommended)

---

Security Notes

API Key Encryption

All API keys are encrypted using Fernet symmetric encryption (AES-128-CBC). Encryption keys are stored in {secrets_dir}/gns3_encryption_key with 0600 permissions.

Access Control

All endpoints require appropriate privileges:

• User.Audit: View user configurations
• User.Modify: Create, update, delete user configurations
• Group.Audit: View group configurations
• Group.Modify: Create, update, delete group configurations

API Key Visibility

The API implements strict API key visibility controls to protect sensitive credentials:

Scenario	User Configs	Group Configs
User viewing own configs	Visible	Hidden
Admin viewing other users' configs	Hidden	Hidden
Viewing group configs directly	N/A	Visible

Rules:

1. Users viewing their own configs: Can see API keys in their own configurations, but NOT in inherited group configurations
2. Admins viewing other users' configs: Cannot see API keys in any user configurations (user privacy)
3. Viewing group configs: Users with Group.Audit privilege can see API keys in group configurations

Example:

// User viewing their own configs
{
  "configs": [
    {
      "config_id": "uuid-1",
      "name": "GPT-4",
      "source": "user",
      "config": {
        "api_key": "sk-xxx"  // Visible (own config)
      }
    },
    {
      "config_id": "uuid-2",
      "name": "Claude-3",
      "source": "group",
      "config": {
        "api_key": null      // Hidden (inherited from group)
      }
    }
  ]
}

// Admin viewing another user's configs
{
  "configs": [
    {
      "config_id": "uuid-1",
      "name": "GPT-4",
      "source": "user",
      "config": {
        "api_key": null      // Hidden (another user's config)
      }
    }
  ]
}

---

Migration from Old User Settings API

The old user settings API (/v3/access/users/{user_id}/profiles) stored configurations in the users.model_configs JSON column. This new API uses a dedicated table with better inheritance support and optimistic locking.

Migration strategy:

1. Run the database migration to create the llm_model_configs table
2. Optionally migrate existing data from users.model_configs to the new table
3. Update clients to use the new API endpoints
4. Update clients to handle version field and 409 Conflict errors
5. Deprecate the old /profiles endpoints

Key differences:

• Inheritance: Users without configs inherit from groups (automatic fallback)
• Optimistic locking: New version field and expected_version parameter
• Dedicated table: Better query performance and data integrity
• Transparent encryption: API keys auto-encrypted/decrypted by the API
• Model type support: New model_type field for categorizing models (text, vision, stt, tts, multimodal, embedding, reranking, other)
• Table-level indexing: name and model_type stored as table columns for efficient filtering and querying

---

Model Type Filtering

The model_type table column enables efficient filtering and querying of configurations by model type:

Common Use Cases

1. Filter by model type: Retrieve only text generation models for chat features
2. Multi-model applications: Select appropriate model based on task type (text vs vision vs embedding)
3. Model type analytics: Query and analyze usage patterns by model type
4. Type-specific defaults: Set different default models for different model types

Example: Filter text models (client-side)

# After fetching configs, filter by model_type
configs = get_user_configs(user_id)
text_models = [c for c in configs if c["model_type"] == "text"]
vision_models = [c for c in configs if c["model_type"] == "vision"]

Database Index

The model_type column is indexed for efficient queries:

CREATE INDEX idx_llm_model_configs_model_type ON llm_model_configs(model_type);

This enables fast lookups when filtering by model type, even with large datasets.

---

Reserved Fields

The llm_model_configs table includes three reserved JSONB fields for future extensibility:

Field	Type	Description
reserved_jsonb_1	JSONB (nullable)	Reserved for future use
reserved_jsonb_2	JSONB (nullable)	Reserved for future use
reserved_jsonb_3	JSONB (nullable)	Reserved for future use

Purpose: These fields are reserved for future feature development without requiring schema changes. They are currently unused in the API code but are available in the database layer for future enhancements.

Use Cases: Future features might use these fields for:

• Advanced configuration options
• Metadata storage
• Feature flags
• Extension data
• Caching computed values

Note: These fields are not exposed in the current API schemas and are reserved for internal use.