diff --git a/docs/gns3-copilot/ai-chat-api-design.md b/docs/gns3-copilot/ai-chat-api-design.md index 3ccbbd7ed..bad946f9c 100644 --- a/docs/gns3-copilot/ai-chat-api-design.md +++ b/docs/gns3-copilot/ai-chat-api-design.md @@ -404,42 +404,152 @@ All endpoints are under `/v3/projects/{project_id}/chat/` path. **Function**: Streaming conversation interface **Request Parameters**: -- message: User message content -- session_id: Session ID (optional, creates new session if not provided) -- stream: Enable streaming response (default true) -- temperature: LLM temperature parameter (Note: currently unused, reserved for future implementation. Actual temperature is read from user's database LLM configuration) -- mode: Interaction mode (currently only supports "text") +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| message | string | Yes | User message content | +| session_id | string | No | Session ID (creates new session if not provided) | +| stream | boolean | No | Enable streaming response (default true) | +| temperature | float | No | LLM temperature (reserved, currently unused) | +| mode | string | No | Interaction mode (fixed as "text", reserved for future expansion) | -**Response**: SSE stream, contains multiple types of messages (see message format above) +**Request Example**: +```json +// First message (new session) +{ + "message": "Hello, can you help me?", + "stream": true +} + +// Subsequent messages (continue session) +{ + "message": "Show me the network topology", + "session_id": "d7e76375-6960-419a-9367-211ef64af877", + "stream": true +} +``` + +**Response**: SSE stream, contains multiple types of messages (see message format section) + +**Session ID Management**: +- **First message**: Do not send `session_id` in request, backend generates a new UUID +- **Retrieve session_id**: Each SSE message (including `done` message) contains `session_id` field +- **Subsequent messages**: Include the saved `session_id` in request body to continue conversation +- **Example flow**: + 1. First request: `{"message": "hello", "stream": true}` → generates new session + 2. Get `session_id` from SSE response: `{"type": "done", "session_id": "xxx-xxx-xxx"}` + 3. Second request: `{"message": "how are you?", "session_id": "xxx-xxx-xxx", "stream": true}` **Project Status Check**: Only allows conversation when project status is "opened" +**Response Example** (SSE stream): +``` +data: {"type": "content", "content": "Hello", "session_id": "d7e76375-6960-419a-9367-211ef64af877"} + +data: {"type": "content", "content": "! I can help", "session_id": "d7e76375-6960-419a-9367-211ef64af877"} + +data: {"type": "done", "session_id": "d7e76375-6960-419a-9367-211ef64af877"} +``` + ### GET /v3/projects/{project_id}/chat/sessions **Function**: List all sessions in a project +**Query Parameters**: +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| user_id | string | No | Filter by user ID | +| limit | int | No | Maximum number of sessions (default 100) | + **Response**: Session list, includes statistics (message count, token usage, etc.), sorted by pin status and update time +**Response Example**: +```json +[ + { + "id": 1, + "thread_id": "d7e76375-6960-419a-9367-211ef64af877", + "user_id": "admin", + "project_id": "a0f46d81-e564-443c-b321-2cdebe80e321", + "title": "GNS3 Topology Assistance", + "message_count": 4, + "llm_calls_count": 2, + "input_tokens": 8500, + "output_tokens": 1200, + "total_tokens": 9700, + "last_message_at": "2026-03-08T01:34:07", + "created_at": "2026-03-07T17:48:07", + "updated_at": "2026-03-08T01:34:07", + "metadata": {}, + "stats": {}, + "pinned": false + } +] +``` + ### GET /v3/projects/{project_id}/chat/sessions/{session_id}/history **Function**: Get complete history of a session -**Parameters**: +**Path Parameters**: - session_id: Session ID -- limit: Maximum number of messages (default 100) -**Response**: -- thread_id: Session ID -- title: Session title -- messages: Message list (OpenAI format) -- llm_calls: Number of LLM calls +**Query Parameters**: +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| limit | int | No | Maximum number of messages (default 100) | + +**Response Example**: +```json +{ + "thread_id": "d7e76375-6960-419a-9367-211ef64af877", + "title": "GNS3 Topology Assistance", + "messages": [ + { + "id": "f0247568-071d-412f-9e3e-4cbe815834ea", + "role": "user", + "content": "你能干点啥。", + "metadata": { + "created_at": "2026-03-07T17:48:07.848519" + } + }, + { + "id": "lc_run--019cc969-eb81-7dd1-a894-e819daf81cd0", + "role": "assistant", + "content": "我可以作为GNS3网络实验的助教...", + "tool_calls": [ + { + "id": "call_00_xxx", + "type": "function", + "function": { + "name": "get_gns3_topology", + "arguments": {} + } + } + ], + "metadata": {} + } + ], + "created_at": null, + "updated_at": null, + "llm_calls": 2 +} +``` ### PATCH /v3/projects/{project_id}/chat/sessions/{session_id} **Function**: Rename session **Request Parameters**: -- title: New title (1-255 characters) +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| title | string | Yes | New title (1-255 characters) | + +**Request Example**: +```json +{ + "title": "New Session Title" +} +``` **Response**: Updated session information @@ -453,13 +563,31 @@ All endpoints are under `/v3/projects/{project_id}/chat/` path. **Function**: Pin session to top of list -**Response**: Updated session information (includes pinned=true) +**Response Example**: +```json +{ + "id": 1, + "thread_id": "d7e76375-6960-419a-9367-211ef64af877", + "title": "GNS3 Topology Assistance", + "pinned": true, + ... +} +``` ### DELETE /v3/projects/{project_id}/chat/sessions/{session_id}/pin **Function**: Unpin session -**Response**: Updated session information (includes pinned=false) +**Response Example**: +```json +{ + "id": 1, + "thread_id": "d7e76375-6960-419a-9367-211ef64af877", + "title": "GNS3 Topology Assistance", + "pinned": false, + ... +} +``` **Sorting Rules**: - Pinned sessions (pinned=true) appear at the front @@ -523,7 +651,9 @@ OpenAI-compatible message model. - id: str - Message unique identifier (auto-generated or inherited from LangChain message) - role: Literal["user", "assistant", "system", "tool"] - Message role - content: str - Message content (supports text, JSON string) -- created_at: str - Creation time (ISO 8601) +- metadata: Optional[Dict] - Message metadata (includes created_at and other custom fields) + - created_at: str - Message creation time (ISO 8601 format) + - Other custom fields can be added as needed **Tool-related Fields**: - name: Optional[str] - Tool message name (tool message) @@ -533,8 +663,10 @@ OpenAI-compatible message model. - type: Literal["function"] - Fixed as "function" - function: Dict - Contains name and arguments (dict or JSON string) -**Metadata**: -- metadata: Optional[Dict] - Additional message metadata +**Important Notes**: +- Message creation time is stored in `metadata.created_at` field +- Frontend should read `metadata.created_at` for message timestamp +- Historical messages may not have `created_at` in metadata (empty `{}`) ## Core Components @@ -555,12 +687,19 @@ OpenAI-compatible message model. - Auto-generate UUID if message has no ID - Ensure all returned messages have unique identifier -2. **Tool Calls Format Conversion** +2. **Metadata and Timestamp Handling** + - Extract entire `metadata` dict from LangChain message + - Message creation time stored in `metadata.created_at` field (ISO 8601 format) + - No top-level `created_at` field in returned message + - Frontend should read `message.metadata.created_at` for timestamp + - Historical messages without metadata will have empty `{}` + +3. **Tool Calls Format Conversion** - LangChain format: `{'name': 'xxx', 'args': {...}, 'id': 'yyy', 'type': 'tool_call'}` - OpenAI format: `{'id': 'yyy', 'type': 'function', 'function': {'name': 'xxx', 'arguments': '{...}'}}` - Automatically convert `args` object to JSON string (if needed) -3. **Content Type Handling** +4. **Content Type Handling** - Supports string, dict, list types - Non-string types automatically converted to JSON string @@ -583,7 +722,7 @@ OpenAI-compatible message model. 2. Get or create chat session (from `chat_sessions` table) 3. Set ContextVars (JWT token, LLM config) 4. Build LangGraph config -5. Create initial message with ID: `HumanMessage(content=message, id=str(uuid4()))` +5. Create initial message with ID and timestamp: `HumanMessage(content=message, id=str(uuid4()), metadata={"created_at": datetime.utcnow().isoformat()})` 6. Stream Agent execution, collecting statistics simultaneously 7. Update session statistics to database after stream ends 8. Sync auto-generated title @@ -667,6 +806,55 @@ Handle different types based on SSE message's `type` field: | done | Mark stream end, stop loading state | | heartbeat | Ignore (keepalive signal) | +### Session ID Management (Important) + +The frontend must properly manage session_id to maintain conversation continuity: + +1. **First request**: Do not include `session_id` in request body +2. **Save session_id**: Extract `session_id` from each SSE message (especially the `done` message) +3. **Subsequent requests**: Include the saved `session_id` in request body to continue the conversation +4. **State management**: Store `session_id` in React state/localStorage to persist across page refreshes + +**Example**: +```javascript +// First message +const response = await fetch('/chat/stream', { + method: 'POST', + body: JSON.stringify({ message: 'Hello', stream: true }) +}); + +// Get session_id from first done message +let sessionId = null; +for await (const chunk of reader) { + const data = JSON.parse(chunk.data); + if (data.type === 'done') { + sessionId = data.session_id; + break; + } +} + +// Subsequent messages - include session_id +await fetch('/chat/stream', { + method: 'POST', + body: JSON.stringify({ message: 'Continue conversation', session_id: sessionId, stream: true }) +}); +``` + +### Message Timestamp + +Each message includes a timestamp in the `metadata` field: + +- **Field location**: `message.metadata.created_at` +- **Format**: ISO 8601 (e.g., `"2026-03-08T01:33:17.848519"`) +- **Note**: Historical messages may have empty `metadata` ({}) if created before this feature was added + +**Example**: +```javascript +// Read message timestamp +const timestamp = message.metadata?.created_at; +const displayTime = timestamp ? new Date(timestamp).toLocaleString() : 'Unknown'; +``` + ### Error Handling - Network error: Show retry option diff --git a/docs/gns3-copilot/todo/sse-interruption-and-agent-cancellation.md b/docs/gns3-copilot/todo/sse-interruption-and-agent-cancellation.md new file mode 100644 index 000000000..16000170a --- /dev/null +++ b/docs/gns3-copilot/todo/sse-interruption-and-agent-cancellation.md @@ -0,0 +1,240 @@ +# SSE Connection Interruption and Agent Cancellation Design + +## Overview + +This document describes the behavior when SSE connection is interrupted during agent execution, how statistics are handled, and strategies for graceful agent cancellation. + +## Current Behavior Analysis + +### What Happens When SSE Connection Drops + +| Component | Behavior | Persists After Disconnect | +|-----------|----------|---------------------------| +| LangGraph Checkpoint | Auto-saved after each node completes | ✅ Yes | +| Messages in conversation | Saved to checkpoint | ✅ Yes | +| Session statistics (message_count, tokens, etc.) | Not updated | ❌ Lost | +| Auto-generated title | Not synced | ❌ Lost | + +### LangGraph Checkpoint Mechanism + +LangGraph automatically saves checkpoint after each node completes: + +``` +llm_call (AI generates response) + ↓ checkpoint saved +should_continue (decides if tools needed) + ↓ checkpoint saved +tool_node (executes tools) + ↓ checkpoint saved +llm_call (processes tool results) + ... +``` + +**Important**: Checkpoint is saved at node boundaries, not during node execution. + +## Statistics Tracking Issue + +### Current Implementation + +```python +message_count = 1 # User message +llm_calls_count = 0 + +async for event in graph.astream_events(...): + if event_type == "on_chat_model_start": + llm_calls_count += 1 + elif event_type == "on_chat_model_end": + message_count += 1 + elif event_type == "on_tool_end": + message_count += 1 +``` + +Statistics are calculated during streaming and only persisted after successful completion: + +```python +try: + async for event in graph.astream_events(...): + yield chunk +except Exception as e: + yield {"type": "error", ...} +# Statistics update - only runs on successful completion! +await repo.update_session(message_count=message_count, ...) +``` + +### Problem + +When connection drops mid-stream: +- Statistics are calculated in-memory but never persisted +- Values may be incomplete/inaccurate (e.g., 2 LLM calls made but only 1 counted) + +## Graceful Shutdown Strategy + +### Recommended: try/finally Approach + +Add `try/finally` to ensure statistics are updated even on disconnection: + +```python +async def stream_chat(...): + try: + async for event in graph.astream_events(inputs, config=config, version="v2"): + try: + yield chunk # May raise exception on client disconnect + except Exception: + log.info("Client disconnected, stopping stream") + break + except Exception as e: + yield {"type": "error", "error": str(e)} + finally: + # Always update statistics, even on disconnect + await repo.update_session( + thread_id=session_id, + message_count=message_count, + llm_calls_count=llm_calls_count, + ... + ) +``` + +### Benefits + +- Statistics are recorded even on disconnection +- Title sync attempt on every request +- Minimal performance overhead (single DB write) + +### Trade-offs + +- Statistics may be inaccurate if disconnection happens mid-processing +- If LLM call fails, partial statistics still recorded + +## Agent Cancellation Analysis + +### Scenarios and Impact + +| Cancellation Timing | State | Issue | +|---------------------|-------|-------| +| Before llm_call | User message sent | No response, no issue | +| After llm_call, has tool_call | AI requested tool execution | ⚠️ Has tool_call, no tool_result | +| During tool_node | Tool executing | May partially execute | +| After tool_node | Tool result returned | Clean state | + +### Key Concern: Orphan tool_calls + +The most dangerous scenario: AI generates `tool_call` but execution hasn't started: + +```json +// Incomplete message: +{ + "role": "assistant", + "tool_calls": [{"name": "execute_command", "arguments": "..."}] + // No corresponding ToolMessage! +} +``` + +### LangGraph Cancellation Handling + +LangGraph handles cancellation automatically: + +1. **Checkpoint at node boundaries**: Messages are saved after each node completes +2. **Cancellation preserves state**: When cancelled, checkpoint is saved automatically +3. **Message consistency**: Either complete (tool_call + ToolMessage) or no tool_call + +```python +# When cancellation happens: +async def stream_chat(...): + try: + async for event in graph.astream_events(...): + yield chunk + except CancelledError: + # LangGraph auto-saves checkpoint before raising + log.info("Request cancelled, checkpoint saved") + finally: + await repo.update_session(...) +``` + +### Handling Incomplete Messages + +When reconnecting, check for incomplete messages: + +```python +async def get_history(session_id): + state = await graph.aget_state(config) + messages = state.values["messages"] + + # Check for orphan tool_calls + last_msg = messages[-1] if messages else None + if last_msg and last_msg.tool_calls and not has_tool_result(messages): + # Handle incomplete message + # Option 1: Show as "interrupted" + # Option 2: Auto-resume tool execution + # Option 3: Ask user to retry +``` + +## Frontend Integration + +### Handling Disconnection + +```javascript +// On connection close: +window.addEventListener('beforeunload', () => { + // Connection will close, server will handle cleanup +}); + +// On reconnect - fetch history: +const history = await fetch(`/chat/sessions/${sessionId}/history`); +const data = await history.json(); + +// Check for incomplete messages +if (data.messages.length > 0) { + const lastMsg = data.messages[data.messages.length - 1]; + if (lastMsg.tool_calls && !lastMsg.content) { + // Message was interrupted - handle appropriately + showWarning("Previous response was interrupted"); + } +} +``` + +## Future Enhancements + +### Optional: Cancel Endpoint + +For explicit cancellation (not just disconnection): + +```python +# Request management +request_manager = RequestManager() + +@router.post("/stream/{request_id}/cancel") +async def cancel_stream(request_id: str): + request_manager.cancel(request_id) + +# In stream_chat: +async def stream_chat(request_id: str, ...): + request_manager.register(request_id) + try: + async for event in graph.astream_events(...): + if request_manager.is_cancelled(request_id): + break + yield chunk + finally: + request_manager.unregister(request_id) +``` + +**Complexity**: Requires request ID tracking, state management, and coordination. + +**Current recommendation**: Not necessary - disconnection naturally stops the stream. + +## Summary + +| Aspect | Current Behavior | Recommended Fix | +|--------|-----------------|-----------------| +| Messages | Auto-saved to checkpoint | Already correct | +| Statistics | Lost on disconnect | Add try/finally | +| Title sync | Lost on disconnect | Add try/finally | +| Cancellation | Handled by LangGraph | Already correct | +| Incomplete messages | Handled on reconnect | Document frontend handling | + +## Action Items + +1. [ ] Add try/finally to ensure statistics update +2. [ ] Add client disconnect detection in yield loop +3. [ ] Document frontend handling for incomplete messages +4. [ ] Test reconnection scenario with tool_call interruption diff --git a/gns3server/agent/gns3_copilot/agent/gns3_copilot.py b/gns3server/agent/gns3_copilot/agent/gns3_copilot.py index 50c0deb1c..587653eeb 100644 --- a/gns3server/agent/gns3_copilot/agent/gns3_copilot.py +++ b/gns3server/agent/gns3_copilot/agent/gns3_copilot.py @@ -47,6 +47,7 @@ Copilot Modes: # Standard library imports import logging import operator +from datetime import datetime from typing import Annotated from typing import Literal @@ -262,6 +263,17 @@ def llm_call(state: dict, config: RunnableConfig | None = None): # Invoke model with prepared messages response = model_with_tools.invoke(prepared_messages) + # Add metadata with created_at timestamp to AI response + if hasattr(response, "metadata"): + existing_metadata = response.metadata or {} + response.metadata = {**existing_metadata, "created_at": datetime.utcnow().isoformat()} + else: + # LangChain messages should have metadata attribute, but defensive fallback + try: + response.metadata = {"created_at": datetime.utcnow().isoformat()} + except Exception: + logger.warning("Could not add metadata to AI response") + logger.info("LLM call completed: tool_calls=%d", len(response.tool_calls) if hasattr(response, "tool_calls") else 0) return { @@ -371,7 +383,16 @@ def tool_node(state: dict, config: RunnableConfig | None = None): except Exception as e: logger.error("Tool %s failed: %s", tool_name, e, exc_info=True) observation = f"Error: {str(e)}" - result.append(ToolMessage(content=observation, tool_call_id=tool_call["id"], name=tool_call["name"])) + + # Create ToolMessage with metadata including created_at + tool_msg = ToolMessage( + content=observation, + tool_call_id=tool_call["id"], + name=tool_call["name"], + metadata={"created_at": datetime.utcnow().isoformat()} + ) + result.append(tool_msg) + return {"messages": result} diff --git a/gns3server/agent/gns3_copilot/agent_service.py b/gns3server/agent/gns3_copilot/agent_service.py index 034259250..aad466361 100644 --- a/gns3server/agent/gns3_copilot/agent_service.py +++ b/gns3server/agent/gns3_copilot/agent_service.py @@ -276,7 +276,13 @@ class AgentService: # Build inputs inputs = { - "messages": [HumanMessage(content=message, id=str(uuid4()))], + "messages": [ + HumanMessage( + content=message, + id=str(uuid4()), + metadata={"created_at": datetime.utcnow().isoformat()}, + ) + ], "llm_calls": 0, "remaining_steps": 20, "mode": mode, diff --git a/gns3server/agent/gns3_copilot/utils/message_converters.py b/gns3server/agent/gns3_copilot/utils/message_converters.py index 87e1d353e..dc599e893 100644 --- a/gns3server/agent/gns3_copilot/utils/message_converters.py +++ b/gns3server/agent/gns3_copilot/utils/message_converters.py @@ -31,7 +31,6 @@ Converts between LangChain messages and OpenAI-compatible format. import json import uuid -from datetime import datetime from typing import Any from typing import Dict @@ -66,15 +65,13 @@ def convert_langchain_to_openai(lc_message) -> Dict[str, Any]: if msg_id is None: msg_id = str(uuid.uuid4()) - # Get timestamp - timestamp = getattr(lc_message, "created_at", None) - if timestamp is None: - timestamp = datetime.utcnow().isoformat() - elif hasattr(timestamp, "isoformat"): - timestamp = timestamp.isoformat() + # Get metadata from message (including created_at) + metadata = getattr(lc_message, "metadata", None) or {} + if not isinstance(metadata, dict): + metadata = {} - # Base message structure - base_msg = {"id": msg_id, "created_at": timestamp, "metadata": {}} + # Base message structure (no top-level created_at, only metadata) + base_msg = {"id": msg_id, "metadata": metadata} # Convert based on message type if isinstance(lc_message, HumanMessage): diff --git a/gns3server/schemas/controller/chat.py b/gns3server/schemas/controller/chat.py index ed8950b18..eeab07472 100644 --- a/gns3server/schemas/controller/chat.py +++ b/gns3server/schemas/controller/chat.py @@ -75,8 +75,7 @@ class OpenAIMessage(BaseModel): name: Optional[str] = Field(None, description="Tool message name") tool_call_id: Optional[str] = Field(None, description="Associated tool call ID (for tool messages)") tool_calls: Optional[List[OpenAIToolCall]] = Field(None, description="Tool calls (for assistant messages)") - metadata: Dict[str, Any] = Field(default_factory=dict, description="Message metadata") - created_at: str = Field(..., description="Message timestamp (ISO 8601)") + metadata: Dict[str, Any] = Field(default_factory=dict, description="Message metadata (includes created_at)") class ConversationHistory(BaseModel):