feat(api): enhance /me endpoint with groups, pools, and ACEs

Add comprehensive user context to the `/me` endpoint by including group memberships, accessible resource pools, and access control entries (ACEs). This enables users to view inherited configurations, available resources, and their permissions directly from the API.

Key additions:
- Group membership details with inherited configs
- Resource pool access information
- ACE visibility for permission transparency
- Support for user-selectable group default configurations

The enhancement addresses user needs for better visibility into permissions and accessible resources within the system.
This commit is contained in:
YueGuobin 2026-03-06 09:50:34 +08:00
parent eb7a72f5e2
commit 4941dc13d8
2 changed files with 1753 additions and 0 deletions

View File

@ -0,0 +1,823 @@
# Enhance `/me` Endpoint with Groups, Pools, and ACEs
**Document Status**: Design Phase
**Priority**: High
**Created**: 2026-03-06
**Related Docs**: [User-Selectable Group Default Config](./user-selectable-group-default-config.md)
---
## Table of Contents
- [Problem Description](#problem-description)
- [Data Model Analysis](#data-model-analysis)
- [Solution Design](#solution-design)
- [Implementation Steps](#implementation-steps)
- [API Response Structure](#api-response-structure)
- [Testing Plan](#testing-plan)
---
## Problem Description
### Current Behavior
The `/me` endpoint only returns basic user information. Users cannot easily see:
1. Which groups they belong to
2. Which resource pools they have access to
3. Their access control entries (ACEs)
### User Needs
1. **Group Membership**: Understand inherited configs and permissions
2. **Pool Access**: Know which resource pools are available
3. **ACE Visibility**: See what access control rules apply to them
---
## Data Model Analysis
### User Model Relationships
**File**: `gns3server/db/models/users.py:38-52`
```python
class User(BaseTable):
__tablename__ = "users"
user_id = Column(GUID, primary_key=True, default=generate_uuid)
username = Column(String, unique=True, index=True)
email = Column(String, unique=True, index=True)
full_name = Column(String)
hashed_password = Column(String)
last_login = Column(DateTime)
is_active = Column(Boolean, default=True)
is_superadmin = Column(Boolean, default=False)
# Relationships
groups = relationship("UserGroup", secondary=user_group_map, back_populates="users")
acl_entries = relationship("ACE") # User's direct ACEs
```
### ACE Model
**File**: `gns3server/db/models/acl.py:28-46`
```python
class ACE(BaseTable):
__tablename__ = "acl"
ace_id = Column(GUID, primary_key=True, default=generate_uuid)
ace_type = Column(String) # "user" or "group"
path = Column(String) # e.g., "/pools/{pool_id}", "/projects"
propagate = Column(Boolean, default=True)
allowed = Column(Boolean, default=True)
user_id = Column(GUID, ForeignKey('users.user_id', ondelete="CASCADE"))
user = relationship("User", back_populates="acl_entries")
group_id = Column(GUID, ForeignKey('user_groups.user_group_id', ondelete="CASCADE"))
group = relationship("UserGroup", back_populates="acl_entries")
role_id = Column(GUID, ForeignKey('roles.role_id', ondelete="CASCADE"))
role = relationship("Role", back_populates="acl_entries")
```
### Resource Pool Model
**File**: `gns3server/db/models/pools.py:46-53`
```python
class ResourcePool(BaseTable):
__tablename__ = "resource_pools"
resource_pool_id = Column(GUID, primary_key=True, default=generate_uuid)
name = Column(String, unique=True, index=True)
resources = relationship("Resource", secondary=resource_pool_map, back_populates="resource_pools")
```
### Key Relationships
```
User ────< UserGroup > (via user_group_map)
└───< ACE (user_id)
├── path = "/pools/{pool_id}" → ResourcePool
├── path = "/projects"
├── role → Role → Privilege
└── allowed (boolean)
UserGroup ────< ACE (group_id)
└─── users
```
### Pool Path Format
From `gns3server/db/repositories/rbac.py:326-327`:
```python
if ace_path.startswith("/pool"):
resource_pool_id = ace_path.split("/")[2]
```
**Pool ACE Path Format**: `/pools/{resource_pool_id}`
---
## Solution Design
### Approach
1. **Groups**: Eager load via `selectinload(User.groups)`
2. **Pools**: Extract from user and group ACEs where path starts with `/pools/`
3. **ACEs**: Aggregate user's direct ACEs and group ACEs
### Response Structure
```json
{
// ===== Basic User Info =====
"user_id": "uuid",
"username": "string",
"email": "string",
"full_name": "string",
"is_active": true,
"is_superadmin": false,
"last_login": "datetime",
"created_at": "datetime",
"updated_at": "datetime",
// ===== User Groups =====
"groups": [
{
"user_group_id": "uuid",
"name": "Developers",
"is_builtin": false,
"created_at": "datetime",
"updated_at": "datetime"
}
],
// ===== Accessible Pools =====
"pools": [
{
"resource_pool_id": "uuid",
"name": "Production Pool",
"access_source": "user", // "user" or "group"
"access_allowed": true
}
],
// ===== ACEs =====
"aces": [
{
"ace_id": "uuid",
"path": "/pools/{pool_id}",
"allowed": true,
"propagate": true,
"ace_type": "user", // "user" or "group"
"source_group_id": null, // null if ace_type is "user"
"source_group_name": null
}
]
}
```
---
## Implementation Steps
### Step 1: Update Schemas
**File**: `gns3server/schemas/controller/users.py`
```python
from typing import List, Optional
from datetime import datetime
from pydantic import ConfigDict, EmailStr, BaseModel, Field, SecretStr
from uuid import UUID
from .base import DateTimeModelMixin
class UserGroup(BaseModel):
"""User group reference."""
user_group_id: UUID
name: str
is_builtin: bool
created_at: datetime
updated_at: datetime
model_config = ConfigDict(from_attributes=True)
class ResourcePoolInfo(BaseModel):
"""Resource pool info accessible to user."""
resource_pool_id: UUID
name: str
access_source: str = Field(..., description="'user' or 'group'")
access_allowed: bool = Field(..., description="Whether access is allowed")
model_config = ConfigDict(from_attributes=True)
class ACEInfo(BaseModel):
"""Access Control Entry info."""
ace_id: UUID
path: str
allowed: bool
propagate: bool
ace_type: str = Field(..., description="'user' or 'group'")
source_group_id: Optional[UUID] = Field(None, description="Group ID if from group ACE")
source_group_name: Optional[str] = Field(None, description="Group name if from group ACE")
model_config = ConfigDict(from_attributes=True)
class UserBase(BaseModel):
"""Common user properties."""
username: Optional[str] = Field(None, min_length=3, pattern="[a-zA-Z0-9_-]+$")
is_active: bool = True
email: Optional[EmailStr] = None
full_name: Optional[str] = None
class User(DateTimeModelMixin, UserBase):
user_id: UUID
last_login: Optional[datetime] = None
is_superadmin: bool = False
# NEW FIELDS
groups: List[UserGroup] = []
pools: List[ResourcePoolInfo] = []
aces: List[ACEInfo] = []
model_config = ConfigDict(from_attributes=True)
# Other existing schemas...
class UserCreate(UserBase):
username: str = Field(..., min_length=3, pattern="[a-zA-Z0-9_-]+$")
password: SecretStr = Field(..., min_length=8, max_length=100)
class UserUpdate(UserBase):
password: Optional[SecretStr] = Field(None, min_length=8, max_length=100)
class LoggedInUserUpdate(BaseModel):
password: Optional[SecretStr] = Field(None, min_length=8, max_length=100)
email: Optional[EmailStr] = None
full_name: Optional[str] = None
class Credentials(BaseModel):
username: str
password: str
```
### Step 2: Update Repository Method
**File**: `gns3server/db/repositories/users.py`
```python
from uuid import UUID
from typing import Optional, List, Dict, Any
from sqlalchemy import select, update, delete, func
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from .base import BaseRepository
import gns3server.db.models as models
from gns3server import schemas
from gns3server.services import auth_service
import logging
log = logging.getLogger(__name__)
class UsersRepository(BaseRepository):
# ... existing methods ...
async def get_user_with_details(
self,
user_id: UUID,
include_pools: bool = True,
include_aces: bool = True
) -> Optional[Dict[str, Any]]:
"""
Get user with groups, pools, and ACEs.
Args:
user_id: User UUID
include_pools: Whether to include accessible resource pools
include_aces: Whether to include ACEs
Returns:
Dictionary with user, groups, pools, and aces
"""
# Get user with groups eagerly loaded
query = select(models.User).where(
models.User.user_id == user_id
).options(selectinload(models.User.groups))
result = await self._db_session.execute(query)
user = result.scalars().first()
if not user:
return None
# Prepare response
response = {
"user": user,
"groups": list(user.groups),
"pools": [],
"aces": []
}
if not include_pools and not include_aces:
return response
# Get user's direct ACEs
user_aces_query = select(models.ACE).where(
models.ACE.user_id == user_id
)
user_aces_result = await self._db_session.execute(user_aces_query)
user_aces = user_aces_result.scalars().all()
# Get group ACEs (inherited from user's groups)
group_aces = []
for group in user.groups:
group_aces_query = select(models.ACE).where(
models.ACE.group_id == group.user_group_id
)
group_aces_result = await self._db_session.execute(group_aces_query)
group_aces.extend(group_aces_result.scalars().all())
# Process ACEs and extract pools
pool_ids_seen = set()
if include_aces:
# Add user ACEs
for ace in user_aces:
response["aces"].append({
"ace_id": ace.ace_id,
"path": ace.path,
"allowed": ace.allowed,
"propagate": ace.propagate,
"ace_type": "user",
"source_group_id": None,
"source_group_name": None
})
# Add group ACEs
for ace in group_aces:
response["aces"].append({
"ace_id": ace.ace_id,
"path": ace.path,
"allowed": ace.allowed,
"propagate": ace.propagate,
"ace_type": "group",
"source_group_id": ace.group_id,
"source_group_name": next((g.name for g in user.groups if g.user_group_id == ace.group_id), None)
})
if include_pools:
# Extract pools from ACEs
for ace in user_aces + group_aces:
if ace.path.startswith("/pools/") and ace.allowed:
try:
pool_id = UUID(ace.path.split("/")[2])
if pool_id not in pool_ids_seen:
# Get pool info
pool_query = select(models.ResourcePool).where(
models.ResourcePool.resource_pool_id == pool_id
)
pool_result = await self._db_session.execute(pool_query)
pool = pool_result.scalars().first()
if pool:
response["pools"].append({
"resource_pool_id": pool.resource_pool_id,
"name": pool.name,
"access_source": "user" if ace.user_id else "group",
"access_allowed": ace.allowed
})
pool_ids_seen.add(pool_id)
except (ValueError, IndexError) as e:
log.warning(f"Invalid pool path format: {ace.path}, error: {e}")
return response
```
### Step 3: Update API Endpoint
**File**: `gns3server/api/routes/controller/users.py`
```python
@router.get("/me", response_model=schemas.User)
async def get_logged_in_user(
current_user: schemas.User = Depends(get_current_active_user),
users_repo: UsersRepository = Depends(get_repository(UsersRepository))
) -> schemas.User:
"""
Get the current active user (including groups, pools, and ACEs).
Returns comprehensive user information including:
- Basic user profile
- Group memberships
- Accessible resource pools
- Access control entries (ACEs)
"""
# Fetch user with all details
user_details = await users_repo.get_user_with_details(
current_user.user_id,
include_pools=True,
include_aces=True
)
if not user_details:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="User not found"
)
# Convert to schema
user = user_details["user"]
return schemas.User(
user_id=user.user_id,
username=user.username,
email=user.email,
full_name=user.full_name,
is_active=user.is_active,
is_superadmin=user.is_superadmin,
last_login=user.last_login,
created_at=user.created_at,
updated_at=user.updated_at,
groups=[schemas.UserGroup.model_validate(g) for g in user_details["groups"]],
pools=[schemas.ResourcePoolInfo(**p) for p in user_details["pools"]],
aces=[schemas.ACEInfo(**a) for a in user_details["aces"]]
)
```
---
## API Response Structure
### Complete Example
```json
{
"user_id": "550e8400-e29b-41d4-a716-446655440000",
"username": "johndoe",
"email": "john@example.com",
"full_name": "John Doe",
"is_active": true,
"is_superadmin": false,
"last_login": "2026-03-06T10:30:00Z",
"created_at": "2026-01-01T00:00:00Z",
"updated_at": "2026-03-06T10:30:00Z",
"groups": [
{
"user_group_id": "650e8400-e29b-41d4-a716-446655440001",
"name": "Developers",
"is_builtin": false,
"created_at": "2026-01-01T00:00:00Z",
"updated_at": "2026-01-01T00:00:00Z"
},
{
"user_group_id": "750e8400-e29b-41d4-a716-446655440002",
"name": "Administrators",
"is_builtin": true,
"created_at": "2026-01-01T00:00:00Z",
"updated_at": "2026-01-01T00:00:00Z"
}
],
"pools": [
{
"resource_pool_id": "850e8400-e29b-41d4-a716-446655440003",
"name": "Production Pool",
"access_source": "user",
"access_allowed": true
},
{
"resource_pool_id": "950e8400-e29b-41d4-a716-446655440004",
"name": "Development Pool",
"access_source": "group",
"access_allowed": true
}
],
"aces": [
{
"ace_id": "a50e8400-e29b-41d4-a716-446655440005",
"path": "/pools/850e8400-e29b-41d4-a716-446655440003",
"allowed": true,
"propagate": true,
"ace_type": "user",
"source_group_id": null,
"source_group_name": null
},
{
"ace_id": "b50e8400-e29b-41d4-a716-446655440006",
"path": "/projects",
"allowed": true,
"propagate": true,
"ace_type": "group",
"source_group_id": "650e8400-e29b-41d4-a716-446655440001",
"source_group_name": "Developers"
},
{
"ace_id": "c50e8400-e29b-41d4-a716-446655440007",
"path": "/pools/950e8400-e29b-41d4-a716-446655440004",
"allowed": true,
"propagate": true,
"ace_type": "group",
"source_group_id": "650e8400-e29b-41d4-a716-446655440001",
"source_group_name": "Developers"
}
]
}
```
---
## Testing Plan
### Unit Tests
#### Test `get_user_with_details` Repository Method
```python
import pytest
from uuid import uuid4
@pytest.mark.asyncio
async def test_get_user_with_groups_only(db_session, test_user, test_group):
"""Test getting user with groups only."""
from gns3server.db.repositories.users import UsersRepository
repo = UsersRepository(db_session)
result = await repo.get_user_with_details(
test_user.user_id,
include_pools=False,
include_aces=False
)
assert result is not None
assert len(result["groups"]) > 0
assert result["groups"][0].name == test_group.name
assert result["pools"] == []
assert result["aces"] == []
@pytest.mark.asyncio
async def test_get_user_with_pools_and_aces(db_session, test_user, test_pool, test_ace):
"""Test getting user with pools and ACEs."""
from gns3server.db.repositories.users import UsersRepository
repo = UsersRepository(db_session)
result = await repo.get_user_with_details(
test_user.user_id,
include_pools=True,
include_aces=True
)
assert result is not None
assert len(result["pools"]) > 0
assert result["pools"][0]["name"] == test_pool.name
assert len(result["aces"]) > 0
assert result["aces"][0]["path"].startswith("/pools/")
@pytest.mark.asyncio
async def test_get_user_with_group_pools(db_session, test_user, test_group, test_group_pool, test_group_ace):
"""Test getting user with pools inherited from groups."""
from gns3server.db.repositories.users import UsersRepository
repo = UsersRepository(db_session)
result = await repo.get_user_with_details(
test_user.user_id,
include_pools=True,
include_aces=True
)
assert result is not None
# Should have pool from group ACE
group_pools = [p for p in result["pools"] if p["access_source"] == "group"]
assert len(group_pools) > 0
```
### Integration Tests
#### Test `/me` Endpoint Response
```python
def test_get_me_with_all_details(test_client, auth_token, test_user_with_groups_and_pools):
"""Test GET /me returns groups, pools, and ACEs."""
response = test_client.get(
"/v3/access/users/me",
headers={"Authorization": f"Bearer {auth_token}"}
)
assert response.status_code == 200
data = response.json()
# Verify basic user info
assert "user_id" in data
assert "username" in data
# Verify groups
assert "groups" in data
assert isinstance(data["groups"], list)
assert len(data["groups"]) > 0
assert "user_group_id" in data["groups"][0]
assert "name" in data["groups"][0]
# Verify pools
assert "pools" in data
assert isinstance(data["pools"], list)
if len(data["pools"]) > 0:
pool = data["pools"][0]
assert "resource_pool_id" in pool
assert "name" in pool
assert "access_source" in pool
assert pool["access_source"] in ["user", "group"]
# Verify ACEs
assert "aces" in data
assert isinstance(data["aces"], list)
if len(data["aces"]) > 0:
ace = data["aces"][0]
assert "ace_id" in ace
assert "path" in ace
assert "allowed" in ace
assert "ace_type" in ace
assert ace["ace_type"] in ["user", "group"]
def test_get_me_user_with_no_groups(test_client, auth_token, test_user_no_groups):
"""Test GET /me for user with no groups."""
response = test_client.get(
"/v3/access/users/me",
headers={"Authorization": f"Bearer {auth_token}"}
)
assert response.status_code == 200
data = response.json()
assert data["groups"] == []
# May still have pools and ACEs from direct user ACEs
```
---
## Benefits
| Feature | Benefit |
|---------|---------|
| **Groups in /me** | Users see inherited configs and permissions |
| **Pools in /me** | Users know available resource pools without separate API call |
| **ACEs in /me** | Transparency - users see their access control rules |
| **Single API Call** | Frontend gets all user context in one request |
| **No Privilege Required** | Users can always see their own info |
---
## Use Cases
### 1. Frontend User Profile Page
```javascript
// Get complete user context
const response = await fetch('/v3/access/users/me', {
headers: { 'Authorization': `Bearer ${token}` }
});
const user = await response.json();
// Display groups
console.log('Member of:', user.groups.map(g => g.name));
// Display available pools
console.log('Accessible pools:', user.pools.map(p => p.name));
// Display ACE summary
console.log('ACEs:', user.aces.length);
```
### 2. LLM Config Selection UI
```javascript
// User wants to select from inherited configs
const user = await fetchCurrentUser();
// Show which configs are from which groups
user.groups.forEach(group => {
console.log(`Configs from ${group.name}:`, getGroupConfigs(group.user_group_id));
});
```
### 3. Permission Troubleshooting
```javascript
// User can't access a resource - why?
const user = await fetchCurrentUser();
// Check if user has pool access
const hasPoolAccess = user.pools.some(p => p.resource_pool_id === targetPoolId);
// Check ACEs
const relevantACEs = user.aces.filter(ace => ace.path.includes(resourcePath));
console.log('Relevant ACEs:', relevantACEs);
```
---
## Performance Considerations
| Query | Complexity | Optimization |
|-------|------------|--------------|
| Get user with groups | 1 JOIN (eager load) | Uses `selectinload` |
| Get user ACEs | 1 query | Direct index lookup |
| Get group ACEs | N queries (one per group) | Could optimize with subquery |
| Get pool details | M queries (one per unique pool) | Could batch fetch |
**Potential Optimization**:
```python
# Batch fetch all pools in one query
pool_ids = [extract_pool_id_from_ace(ace) for ace in all_aces]
pools_query = select(models.ResourcePool).where(
models.ResourcePool.resource_pool_id.in_(pool_ids)
)
pools_result = await self._db_session.execute(pools_query)
pools = {p.resource_pool_id: p for p in pools_result.scalars().all()}
```
---
## Security Considerations
### Data Exposure
| Data | Visibility | Rationale |
|------|-----------|-----------|
| Basic user info | User themselves | Already exposed in current `/me` |
| Groups | User themselves | User knows which groups they joined |
| Pools | User themselves | User knows which pools they can access |
| ACEs | User themselves | Transparency about access rules |
| Other users' data | **Hidden** | Not included in response |
### Access Control
- **Authentication Required**: Must provide valid JWT token
- **No Special Privilege**: Users can always view their own data
- **Filtering**: Only returns data for the authenticated user
---
## Future Enhancements
1. **Roles**: Add user's roles (derived from ACEs)
```json
"roles": ["User", "Auditor"]
```
2. **Effective Privileges**: Consolidated privilege list
```json
"privileges": ["Project.Audit", "Node.Create"]
```
3. **Resource Counts**: Summary of accessible resources
```json
"resources_summary": {
"projects_count": 5,
"templates_count": 3
}
```
---
## Code Changes Checklist
| File | Change Type | Description |
|------|-------------|-------------|
| `gns3server/schemas/controller/users.py` | Modify | Add UserGroup, ResourcePoolInfo, ACEInfo schemas; Update User schema |
| `gns3server/db/repositories/users.py` | Modify | Add `get_user_with_details` method |
| `gns3server/api/routes/controller/users.py` | Modify | Update `/me` endpoint to use new method |
---
**Document Version**: 1.0
**Last Updated**: 2026-03-06

View File

@ -0,0 +1,930 @@
# User-Selectable Group Default Config
**Document Status**: Design Phase
**Priority**: Medium
**Created**: 2026-03-06
**Related Docs**: [LLM Model Configs API](../llm-model-configs-api.md)
---
## Table of Contents
- [Problem Description](#problem-description)
- [Current State Analysis](#current-state-analysis)
- [Requirements Analysis](#requirements-analysis)
- [Solution Design](#solution-design)
- [Implementation Steps](#implementation-steps)
- [Code Changes Checklist](#code-changes-checklist)
- [Testing Plan](#testing-plan)
- [Risk Assessment](#risk-assessment)
---
## Problem Description
### Current Behavior
Regular users cannot select an inherited group LLM model config as their default config, even though they can see the inherited group configs in their config list.
### User Scenario
1. Administrator creates multiple LLM model configs for a user group (e.g., GPT-4, Claude 3.5, Gemini Pro)
2. The group default is set to GPT-4
3. Users inherit these configs and can see all group configs in their config list
4. Users want to use Claude 3.5 as their default, but have no way to set it via API
### Existing Code Limitation
**File**: `gns3server/db/repositories/llm_model_configs.py:194-218`
```python
async def set_user_default_config(self, user_id: UUID, config_id: UUID) -> bool:
"""Set a user's default LLM model configuration."""
# ...
# Set new default
query = update(models.LLMModelConfig).where(
and_(
models.LLMModelConfig.config_id == config_id,
models.LLMModelConfig.user_id == user_id # KEY LIMITATION
)
).values(is_default=True, updated_at=now)
```
**Problem**: The `user_id == user_id` condition restricts setting only user's own configs. Inherited group configs have `user_id` as `NULL`, so they cannot be set as default.
---
## Current State Analysis
### Current Config Retrieval Flow
```
User requests config list
GET /v3/access/users/{user_id}/llm-model-configs
get_user_effective_configs(user_id)
Returns: {
configs: [
{ source: "user", ... }, # User's own configs
{ source: "group", ... } # Inherited group configs
],
default_config: { ... } # Current default config
}
```
### Default Config Selection Priority
**Current Logic** (`llm_model_configs.py:503-520`):
1. User config marked with `is_default: true`
2. Group config marked with `is_default: true`
3. First config in the list (user configs come before group configs)
### Agent Config Retrieval Flow
**Key Discovery**: Agent retrieves config via `user_id`, doesn't care about config source.
**Flow**:
```
Agent → get_user_llm_config_full(user_id, app)
get_user_effective_configs(user_id)
Returns default config (auto-decrypts API key)
Agent uses config to call LLM
```
**Key Files**:
- `gns3server/db/tasks.py:314-406` - `get_user_llm_config_full`
- `gns3server/api/routes/controller/chat.py:122` - API entry point
### API Key Visibility Control
| Scenario | User Configs | Group Configs |
|----------|-------------|---------------|
| User viewing own configs | **Visible** | **Hidden** (`null`) |
| Admin viewing other users' configs | **Hidden** | **Hidden** |
| Agent usage (system-level) | **Visible** | **Visible** (direct DB access) |
---
## Requirements Analysis
### Functional Requirements
1. **Users can select group config as default**
- Users can set any accessible config (own or inherited) as default via API
- API endpoint remains unchanged: `PUT /v3/access/users/{user_id}/llm-model-configs/default/{config_id}`
2. **Maintain API Key Security**
- Group config API keys remain hidden when users view config list
- Agent can access and decrypt group config API keys when using
3. **Backward Compatibility**
- No impact on existing user configs
- No impact on Agent calling flow
- Config list response structure remains consistent
### Non-Functional Requirements
1. **Performance**: No significant query overhead
2. **Maintainability**: Clear code logic, easy to understand and maintain
3. **Extensibility**: Future support for config overrides (users modifying certain parameters of inherited configs)
---
## Solution Design
### Selection: Shadow Config Approach
Add `inherited_from_config_id` field to `llm_model_configs` table. When user selects a group config as default, create a "shadow config" record.
### Data Model Design
#### Table Structure Modification
**File**: `gns3server/db/models/llm_model_configs.py`
```python
class LLMModelConfig(BaseTable):
"""LLM model configuration for users and user groups."""
__tablename__ = "llm_model_configs"
config_id = Column(GUID, primary_key=True, default=generate_uuid)
name = Column(String(100), nullable=False)
model_type = Column(String(50), nullable=False)
config = Column(JSON, nullable=False)
user_id = Column(GUID, ForeignKey("users.user_id", ondelete="CASCADE"), nullable=True)
group_id = Column(GUID, ForeignKey("user_groups.user_group_id", ondelete="CASCADE"), nullable=True)
is_default = Column(Boolean, default=False, nullable=False)
version = Column(Integer, default=0, nullable=False)
# NEW FIELD: Shadow config references original group config
inherited_from_config_id = Column(
GUID,
ForeignKey("llm_model_configs.config_id", ondelete="CASCADE"),
nullable=True
)
# Relationships
inherited_from = relationship(
"LLMModelConfig",
remote_side=[config_id],
backref="shadow_configs"
)
# Constraints
__table_args__ = (
# Original constraints...
CheckConstraint(
"(user_id IS NOT NULL AND group_id IS NULL) OR "
"(user_id IS NULL AND group_id IS NOT NULL)",
name="single_owner_check"
),
# NEW CONSTRAINT: Shadow configs must belong to users
CheckConstraint(
"inherited_from_config_id IS NULL OR user_id IS NOT NULL",
name="shadow_config_belong_to_user"
),
# ... other constraints
)
```
### Shadow Config Explanation
| Field | Value | Description |
|------|-------|-------------|
| `config_id` | New UUID | Shadow config's unique identifier |
| `name` | Original group config's name | Display name |
| `model_type` | Original group config's type | Config type |
| `config` | `{"api_key": "__INHERITED_FROM_GROUP__", ...}` | Config data, API key marked with special value |
| `user_id` | Current user's ID | Belongs to user |
| `group_id` | `NULL` | Shadow config doesn't belong to group |
| `is_default` | `true` | Marked as default config |
| `inherited_from_config_id` | Original group config's ID | References original config |
### Workflow
#### 1. User Sets Group Config as Default
```
User Request: PUT /users/{user_id}/llm-model-configs/default/{group_config_id}
set_user_default_config(user_id, group_config_id)
Detects group_config_id is a group config
Creates shadow config:
- user_id = current user
- inherited_from_config_id = group_config_id
- config = original config (API key marked as "__INHERITED_FROM_GROUP__")
- is_default = true
Deletes old shadow configs and default flags
Commits to database
```
#### 2. User Views Config List
```
GET /users/{user_id}/llm-model-configs
get_user_effective_configs(user_id)
Gets user configs (including shadow configs)
For shadow configs:
- Reads complete data from original group config
- Hides API key (sets to null)
- Marks source = "user"
- Marks inherited_from = original config ID
Returns config list
```
#### 3. Agent Retrieves Config for Usage
```
Agent → get_user_llm_config_full(user_id, app)
Gets default config (detects it's a shadow config)
Gets encrypted API key from original group config
Decrypts API key
Returns complete config (including API key)
Agent uses config to call LLM
```
### Solution Advantages
| Advantage | Description |
|-----------|-------------|
| **Data Integrity** | Foreign key constraints ensure referential integrity, cascading deletes handle cleanup |
| **Backward Compatible** | No modification to existing logic, shadow config is a new feature |
| **Clear Semantics** | `inherited_from_config_id` clearly indicates inheritance relationship |
| **Unified API** | Users don't need to care about config source, just select directly |
| **Extensible** | Shadow config can add override fields in the future (e.g., user-custom parameters) |
| **No Agent Changes Required** | Agent still retrieves config via `user_id`, automatically compatible |
---
## Implementation Steps
### Step 1: Database Migration
Create new migration file: `gns3server/db_migrations/versions/{timestamp}_add_inherited_from_config_id.py`
```python
"""Add inherited_from_config_id to llm_model_configs table
Revision ID: xxx_add_inherited_from_config_id
Revises: [previous_revision_id]
Create Date: 2026-03-06
This migration adds support for shadow configs, allowing users to select
inherited group configurations as their default.
"""
from alembic import op
import sqlalchemy as sa
def upgrade():
# Add the new column
op.add_column(
'llm_model_configs',
sa.Column(
'inherited_from_config_id',
sa.GUID(),
nullable=True
)
)
# Create foreign key constraint
op.create_foreign_key(
'fk_llm_configs_inherited_from',
'llm_model_configs', 'llm_model_configs',
['inherited_from_config_id'], ['config_id'],
ondelete='CASCADE'
)
# Add check constraint: shadow configs must belong to users
op.execute("""
ALTER TABLE llm_model_configs
ADD CONSTRAINT shadow_config_belong_to_user
CHECK (inherited_from_config_id IS NULL OR user_id IS NOT NULL)
""")
def downgrade():
# Remove constraints and column
op.execute("ALTER TABLE llm_model_configs DROP CONSTRAINT shadow_config_belong_to_user")
op.drop_constraint('fk_llm_configs_inherited_from', 'llm_model_configs', type_='foreignkey')
op.drop_column('llm_model_configs', 'inherited_from_config_id')
```
### Step 2: Modify Data Model
**File**: `gns3server/db/models/llm_model_configs.py`
Add to `LLMModelConfig` class:
- `inherited_from_config_id` field
- `inherited_from` relationship
- `shadow_config_belong_to_user` constraint
### Step 3: Modify Repository Layer
**File**: `gns3server/db/repositories/llm_model_configs.py`
#### 3.1 Modify `set_user_default_config` Method
```python
async def set_user_default_config(self, user_id: UUID, config_id: UUID) -> bool:
"""
Set a user's default LLM model configuration.
Supports setting inherited group configs as default via shadow configs.
Args:
user_id: User UUID
config_id: Configuration UUID (can be user's own or inherited group config)
Returns:
True if successful, False if config not found or not accessible
"""
from gns3server.utils.encryption import is_encrypted
# Check if config is accessible to user
effective = await self.get_user_effective_configs(
user_id,
current_user_id=user_id
)
accessible_config_ids = {c["config_id"] for c in effective["configs"]}
if config_id not in accessible_config_ids:
return False
# Get the original config
result = await self._db_session.execute(
select(models.LLMModelConfig).where(
models.LLMModelConfig.config_id == config_id
)
)
orig_config = result.scalars().first()
if not orig_config:
return False
now = datetime.utcnow()
if orig_config.user_id == user_id:
# Scenario 1: User selects their own config
# Use the existing is_default mechanism
# Delete old shadow configs
await self._db_session.execute(
delete(models.LLMModelConfig)
.where(
and_(
models.LLMModelConfig.user_id == user_id,
models.LLMModelConfig.inherited_from_config_id.isnot(None)
)
)
)
# Clear all user default flags
await self._db_session.execute(
update(models.LLMModelConfig)
.where(
and_(
models.LLMModelConfig.user_id == user_id,
models.LLMModelConfig.is_default == True
)
)
.values(is_default=False, updated_at=now)
)
# Set new default
await self._db_session.execute(
update(models.LLMModelConfig)
.where(
and_(
models.LLMModelConfig.config_id == config_id,
models.LLMModelConfig.user_id == user_id
)
)
.values(is_default=True, updated_at=now)
)
else:
# Scenario 2: User selects a group config - create shadow config
# Clear all user default flags
await self._db_session.execute(
update(models.LLMModelConfig)
.where(models.LLMModelConfig.user_id == user_id)
.values(is_default=False)
)
# Delete old shadow configs
await self._db_session.execute(
delete(models.LLMModelConfig)
.where(
and_(
models.LLMModelConfig.user_id == user_id,
models.LLMModelConfig.inherited_from_config_id.isnot(None)
)
)
)
# Copy config data, but mark API key as inherited
shadow_config_data = orig_config.config.copy()
shadow_config_data["api_key"] = "__INHERITED_FROM_GROUP__"
# Create shadow config
shadow_config = models.LLMModelConfig(
name=orig_config.name,
model_type=orig_config.model_type,
config=shadow_config_data,
user_id=user_id,
group_id=None,
is_default=True,
inherited_from_config_id=config_id,
version=0,
created_at=now,
updated_at=now
)
self._db_session.add(shadow_config)
await self._db_session.commit()
return True
```
#### 3.2 Modify `get_user_effective_configs` Method
Add special logic for shadow config handling:
```python
# In get_user_effective_configs method
# Process user configs (including shadow configs)
user_configs = await self.get_user_configs(user_id)
# Build map of group configs for shadow config resolution
group_configs_map = {}
group_names_map = {}
for group in user_groups:
configs = await self.get_group_configs(group.user_group_id)
if configs:
group_configs_map[group.user_group_id] = configs
group_names_map[group.user_group_id] = group.name
# Flatten group configs for easy access
all_group_configs = {}
for configs in group_configs_map.values():
for config in configs:
all_group_configs[config.config_id] = config
configs_with_source = []
# Process each user config
for config in user_configs:
if config.inherited_from_config_id:
# This is a shadow config - resolve from parent group config
parent_config = all_group_configs.get(config.inherited_from_config_id)
if parent_config:
config_dict = parent_config.config.copy()
# Hide API key in shadow configs (users viewing their own configs)
if "api_key" in config_dict:
config_dict["api_key"] = None
configs_with_source.append({
"config_id": config.config_id,
"name": config.name,
"model_type": config.model_type,
"config": config_dict,
"user_id": config.user_id,
"group_id": None,
"is_default": config.is_default,
"version": config.version,
"created_at": config.created_at,
"updated_at": config.updated_at,
"source": "user",
"inherited_from": config.inherited_from_config_id,
"group_name": group_names_map.get(parent_config.group_id)
})
else:
# Regular user config - existing logic
config_dict = config.config.copy()
# API key visibility control
if "api_key" in config_dict and config_dict["api_key"]:
if is_viewing_own:
try:
if is_encrypted(config_dict["api_key"]):
config_dict["api_key"] = decrypt(config_dict["api_key"])
except Exception as e:
log.warning(f"Failed to decrypt API key: {e}")
config_dict["api_key"] = None
else:
config_dict["api_key"] = None
configs_with_source.append({
"config_id": config.config_id,
"name": config.name,
"model_type": config.model_type,
"config": config_dict,
"user_id": config.user_id,
"group_id": config.group_id,
"is_default": config.is_default,
"version": config.version,
"created_at": config.created_at,
"updated_at": config.updated_at,
"source": "user",
"inherited_from": None,
"group_name": None
})
# Add inherited group configs (exclude those already shadowed)
shadow_inherited_ids = {
c["inherited_from"]
for c in configs_with_source
if c["inherited_from"]
}
for group_id, configs in group_configs_map.items():
for config in configs:
if config.config_id in shadow_inherited_ids:
continue # Already shadowed, don't duplicate
config_dict = config.config.copy()
if "api_key" in config_dict:
config_dict["api_key"] = None
configs_with_source.append({
"config_id": config.config_id,
"name": config.name,
"model_type": config.model_type,
"config": config_dict,
"user_id": None,
"group_id": config.group_id,
"is_default": config.is_default,
"version": config.version,
"created_at": config.created_at,
"updated_at": config.updated_at,
"source": "group",
"inherited_from": None,
"group_name": group_names_map[group_id]
})
# Select default_config (shadow configs have priority since marked is_default=true)
default_config = None
for config in configs_with_source:
if config["is_default"] and config["source"] == "user":
default_config = config
break
if default_config is None:
for config in configs_with_source:
if config["is_default"] and config["source"] == "group":
default_config = config
break
if default_config is None and configs_with_source:
default_config = configs_with_source[0]
return {
"configs": configs_with_source,
"default_config": default_config
}
```
### Step 4: Modify System-Level Config Retrieval
**File**: `gns3server/db/tasks.py`
Modify `get_user_llm_config_full` function to add shadow config API key decryption logic:
```python
async def get_user_llm_config_full(user_id: str, app: FastAPI) -> Optional[dict]:
"""
Get user's full LLM configuration with decrypted API key for Copilot.
This is a system-level function that bypasses API security restrictions.
It retrieves the complete configuration including decrypted API keys,
even for inherited group configurations and shadow configs.
Args:
user_id: User UUID
app: FastAPI application instance
Returns:
Dictionary with LLM configuration (provider, model, api_key, etc.)
or None if not found.
"""
from uuid import UUID
from gns3server.db.repositories.llm_model_configs import LLMModelConfigsRepository
from gns3server.utils.encryption import decrypt, is_encrypted
try:
user_uuid = UUID(user_id) if isinstance(user_id, str) else user_id
async with AsyncSession(app.state._db_engine, expire_on_commit=False) as session:
repo = LLMModelConfigsRepository(session)
# Get effective configs (own + inherited from groups)
result = await repo.get_user_effective_configs(
user_uuid,
current_user_id=user_uuid,
current_user_is_superadmin=False
)
if not result or not result.get("default_config"):
log.warning(f"No default LLM configuration found for user {user_id}")
return None
default_config = result["default_config"]
config_id = default_config["config_id"]
source = default_config["source"]
inherited_from = default_config.get("inherited_from")
# Get full config from database
full_config = await repo.get_user_config(config_id)
if not full_config:
log.error(f"Failed to retrieve full config: config_id={config_id}")
return None
# Decrypt API key
config_data = full_config.config.copy()
inherited_from_config_id = full_config.inherited_from_config_id
# If shadow config, get API key from parent group config
if inherited_from_config_id:
parent_config = await repo.get_group_config(inherited_from_config_id)
if parent_config and "api_key" in parent_config.config:
try:
encrypted_key = parent_config.config["api_key"]
if encrypted_key and is_encrypted(encrypted_key):
config_data["api_key"] = decrypt(encrypted_key)
log.debug(f"Decrypted API key from inherited group config for user {user_id}")
else:
config_data["api_key"] = encrypted_key
except Exception as e:
log.error(f"Failed to decrypt inherited API key: {e}")
config_data["api_key"] = None
else:
log.error(f"Parent group config not found for shadow config: {inherited_from_config_id}")
config_data["api_key"] = None
else:
# Regular user config - decrypt API key directly
if "api_key" in config_data and config_data["api_key"]:
try:
if is_encrypted(config_data["api_key"]):
config_data["api_key"] = decrypt(config_data["api_key"])
log.debug(f"Successfully decrypted API key for user {user_id}")
except Exception as e:
log.error(f"Failed to decrypt API key: {e}")
config_data["api_key"] = None
# Build configuration dict
llm_config = {
"config_id": str(full_config.config_id),
"name": full_config.name,
"model_type": str(full_config.model_type),
"source": source,
"inherited_from": str(inherited_from_config_id) if inherited_from_config_id else None,
"group_name": default_config.get("group_name"),
"user_id": str(full_config.user_id) if full_config.user_id else None,
"group_id": str(full_config.group_id) if full_config.group_id else None,
**config_data
}
# Validate required fields
if not llm_config.get("provider"):
log.error(f"LLM config missing 'provider' field: {config_id}")
return None
if not llm_config.get("model"):
log.error(f"LLM config missing 'model' field: {config_id}")
return None
log.info(
f"Retrieved LLM config for user {user_id}: "
f"provider={llm_config.get('provider')}, model={llm_config.get('model')}, "
f"source={source}, inherited_from={inherited_from_config_id}"
)
return llm_config
except Exception as e:
log.error(f"Failed to retrieve LLM config for user {user_id}: {e}", exc_info=True)
return None
```
### Step 5: Update Schema (Optional)
If you want to display `inherited_from` field in API response, update relevant Schema:
**File**: `gns3server/schemas/controller/chat.py` or corresponding schema file
```python
class LLMModelConfigWithSource(BaseModel):
"""LLM model configuration with source information."""
config_id: UUID
name: str
model_type: str
config: Dict[str, Any]
user_id: Optional[UUID] = None
group_id: Optional[UUID] = None
is_default: bool
version: int
created_at: datetime
updated_at: datetime
source: str # "user" or "group"
group_name: Optional[str] = None
inherited_from: Optional[UUID] = None # NEW FIELD
```
### Step 6: Update API Documentation
**File**: `docs/gns3-copilot/llm-model-configs-api.md`
Add `inherited_from` field description in response schema section:
```markdown
### LLMModelConfigWithSource
| Field | Type | Description |
|-------|------|-------------|
| ...
| `inherited_from` | UUID (nullable) | For shadow configs, the ID of the inherited group config |
```
---
## Code Changes Checklist
### Files to Modify
| File Path | Change Type | Description |
|-----------|-------------|-------------|
| `gns3server/db/models/llm_model_configs.py` | Modify | Add `inherited_from_config_id` field and relationship |
| `gns3server/db/repositories/llm_model_configs.py` | Modify | Modify `set_user_default_config` and `get_user_effective_configs` |
| `gns3server/db/tasks.py` | Modify | Modify `get_user_llm_config_full` to support shadow configs |
| `gns3server/schemas/...` | Modify (Optional) | Add `inherited_from` field to Schema |
| `gns3server/db_migrations/versions/...` | New | Database migration file |
| `docs/gns3-copilot/llm-model-configs-api.md` | Modify | Update API documentation |
### New Files
| File Path | Description |
|-----------|-------------|
| `gns3server/db_migrations/versions/{timestamp}_add_inherited_from_config_id.py` | Database migration |
---
## Testing Plan
### Unit Tests
#### 1. Test `set_user_default_config`
- **Test 1.1**: User sets their own config as default
- Input: User's config ID
- Expected: `is_default=true`, old shadow configs deleted
- **Test 1.2**: User sets group config as default
- Input: Group config ID
- Expected: Shadow config created, `inherited_from_config_id` points to group config
- **Test 1.3**: User switches default config (from own to group config)
- Input: Group config ID
- Expected: Old shadow config deleted, new shadow config created
- **Test 1.4**: User sets non-existent config as default
- Input: Invalid config ID
- Expected: Returns `False`
- **Test 1.5**: User sets inaccessible config as default
- Input: Other user's group config ID
- Expected: Returns `False`
#### 2. Test `get_user_effective_configs`
- **Test 2.1**: User with only own configs
- Expected: Returns user configs, no `inherited_from` field
- **Test 2.2**: User with inherited group configs, no default set
- Expected: Returns user configs + group configs, `default_config` is first user config or first group config
- **Test 2.3**: User set group config as default (shadow config)
- Expected: Shadow config `source="user"`, `is_default=true`, `inherited_from` points to group config, API key is `null`
- **Test 2.4**: User viewing own configs (API key visibility)
- Expected: Own config shows API key, shadow config and group config hide API key
#### 3. Test `get_user_llm_config_full`
- **Test 3.1**: User using own default config
- Expected: Returns config with decrypted API key
- **Test 3.2**: User using shadow config (group config)
- Expected: Retrieves and decrypts API key from original group config
- **Test 3.3**: Shadow config's original group config deleted
- Expected: Returns `None` or appropriate error handling
### Integration Tests
#### 1. API Endpoint Tests
- **Test 1.1**: `PUT /users/{user_id}/llm-model-configs/default/{group_config_id}`
- Request: Set group config as default
- Expected: Returns 200, config set as default
- **Test 1.2**: `GET /users/{user_id}/llm-model-configs`
- Expected: Shadow config appears in list, `source="user"`, `inherited_from` field exists
- **Test 1.3**: `GET /users/{user_id}/llm-model-configs/default`
- Expected: Returns shadow config
#### 2. Agent Integration Tests
- **Test 2.1**: User using shadow config calls Agent
- Expected: Agent successfully retrieves config and calls LLM
- **Test 2.2**: Multiple users using same group config as default
- Expected: Each user has their own shadow config, no interference
### Security Tests
- **Test 1**: User views config list, shadow config's API key is hidden
- **Test 2**: Admin views other user's config, API key is hidden
- **Test 3**: User cannot set other user's config as default
- **Test 4**: Cascading delete: Group config deleted, shadow config auto-deleted
---
## Risk Assessment
### Technical Risks
| Risk | Impact | Probability | Mitigation |
|------|--------|-------------|------------|
| Database migration failure | High | Low | Thoroughly test migration script, prepare rollback plan |
| Shadow config out of sync with original config | Medium | Medium | Shadow config dynamically reads from original config, real-time sync |
| API key decryption failure | High | Low | Add error handling and logging |
| Performance impact | Low | Low | Limited number of shadow configs, negligible performance impact |
### Business Risks
| Risk | Impact | Probability | Mitigation |
|------|--------|-------------|------------|
| User confusion (shadow config vs own config) | Medium | Medium | Clearly indicate inheritance source in UI |
| Users unaware of group config updates | Low | Low | Document behavior, or add config version notification in the future |
### Compatibility Risks
| Risk | Impact | Probability | Mitigation |
|------|--------|-------------|------------|
| Existing API clients incompatible with `inherited_from` field | Low | Low | Field is optional, old clients can ignore it |
| Agent doesn't support shadow config | High | Low | Agent retrieves config via `user_id`, automatically compatible |
---
## Future Enhancements
### Potential Future Features
1. **Config Overrides**: Allow users to override certain parameters in shadow config (e.g., `temperature`)
2. **Change Notifications**: Notify users when group config is updated
3. **Config Version Tracking**: Record change history of configs
4. **Config Recommendations**: Recommend default configs based on usage patterns
### Related Features
- Support user config templates (create own config based on group config)
- Config import/export functionality
- Batch config management
---
## References
- [LLM Model Configs API](../llm-model-configs-api.md)
- [AI Chat API Design](../ai-chat-api-design.md)
- SQLAlchemy Foreign Key: https://docs.sqlalchemy.org/en/14/core/metadata.html
- Alembic Migrations: https://alembic.sqlalchemy.org/en/latest/tutorial.html
---
**Document Version**: 1.0
**Last Updated**: 2026-03-06