mirror of
https://github.com/GNS3/gns3-server.git
synced 2026-08-27 12:30:13 +03:00
docs: simplify context window management documentation
Removed extensive implementation details and configuration examples from the context window management documentation. The document now focuses on core permissions and prohibitions for tool usage, providing a clearer and more concise reference for allowed and forbidden actions. This streamlines the documentation to essential guidelines only.
This commit is contained in:
YueGuobin
parent
a8044cc1c2
commit
8dc8facb13
@ -2,196 +2,7 @@
|
||||
|
||||
## 概述
|
||||
|
||||
本文档说明了 GNS3 Copilot 如何处理不同 LLM 模型的上下文窗口限制，以及如何实现自动消息裁剪功能。
|
||||
|
||||
## ⚠️ 重要：context_limit 必须手动配置
|
||||
|
||||
**由于模型供应商频繁更新上下文窗口大小，系统不再提供内置默认值。**
|
||||
|
||||
用户在创建 LLM 模型配置时**必须提供 `context_limit`**。请从模型供应商的官方文档获取最新的上下文窗口大小。
|
||||
|
||||
### ⚡ 依赖要求
|
||||
|
||||
**tiktoken 是必需依赖**，系统无法在没有 tiktoken 的情况下运行。
|
||||
|
||||
```bash
|
||||
pip install tiktoken>=0.8.0
|
||||
```
|
||||
|
||||
如果未安装 tiktoken，系统将在首次尝试计数 tokens 时抛出 ImportError。
|
||||
|
||||
### ⚡ 单位说明
|
||||
|
||||
**`context_limit` 的单位是 K tokens（千 tokens）**
|
||||
|
||||
- `1` = 1K = 1,000 tokens
|
||||
- `128` = 128K = 128,000 tokens
|
||||
- `200` = 200K = 200,000 tokens
|
||||
- `2800` = 2800K = 2,800,000 tokens
|
||||
|
||||
### 为什么使用 K tokens？
|
||||
|
||||
1. **更简洁** - `128` 比 `128000` 更易读易写
|
||||
2. **减少错误** - 避免少写或多写 0
|
||||
3. **符合习惯** - 与业界常用的 "128K", "200K" 表示法一致
|
||||
|
||||
### 为什么必须手动配置？
|
||||
|
||||
1. **模型更新频繁** - OpenAI、Anthropic、Google 等供应商经常发布模型更新
|
||||
2. **上下文大小变化** - 新版本往往增加上下文窗口，旧版本可能被废弃
|
||||
3. **维护成本高** - 内置默认值很快过时，可能导致配置错误
|
||||
4. **责任明确** - 用户根据实际使用的模型配置，避免混淆
|
||||
|
||||
---
|
||||
|
||||
## 数据库配置方式
|
||||
|
||||
### 配置要求
|
||||
|
||||
`context_limit` 是**必填字段**，单位为 **K tokens**。创建或更新 LLM 模型配置时必须提供。
|
||||
|
||||
| 字段 | 类型 | 必填 | 单位 | 说明 |
|
||||
|------|------|------|------|------|
|
||||
| `context_limit` | `int` | ✅ **是** | K tokens | 模型上下文窗口限制（128 = 128K tokens） |
|
||||
| `context_strategy` | `"conservative" \| "balanced" \| "aggressive"` | 否 | - | 裁剪策略，默认 "balanced" |
|
||||
|
||||
### 获取上下文窗口大小
|
||||
|
||||
#### 官方文档链接
|
||||
|
||||
| 供应商 | 文档链接 | 示例值（tokens） | 配置值（K） |
|
||||
|--------|---------|------------------|------------|
|
||||
| OpenAI | https://platform.openai.com/docs/models | 128,000 | `128` |
|
||||
| Anthropic | https://docs.anthropic.com/claude/docs/models-overview | 200,000 | `200` |
|
||||
| Google | https://ai.google.dev/gemini-api/docs/models | 2,800,000 | `2800` |
|
||||
| DeepSeek | https://platform.deepseek.com/api-docs/ | 128,000 | `128` |
|
||||
| xAI | https://docs.x.ai/ | 128,000 | `128` |
|
||||
|
||||
**转换示例**：
|
||||
- 官方文档显示 `128,000 tokens` → 配置为 `128`
|
||||
- 官方文档显示 `200,000 tokens` → 配置为 `200`
|
||||
- 官方文档显示 `2,800,000 tokens` → 配置为 `2800`
|
||||
|
||||
#### 参考工具
|
||||
|
||||
运行参考脚本查看常见模型的上下文限制（显示为 K tokens）：
|
||||
|
||||
```bash
|
||||
python scripts/show_model_context_limits.py
|
||||
```
|
||||
|
||||
**注意**：此脚本仅提供参考值，请以官方文档为准。
|
||||
|
||||
---
|
||||
|
||||
## API 使用示例
|
||||
|
||||
### 创建配置（必须提供 context_limit）
|
||||
|
||||
```bash
|
||||
POST /v3/users/{user_id}/llm-model-configs
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"name": "GPT-4o Configuration",
|
||||
"model_type": "text",
|
||||
"is_default": true,
|
||||
"provider": "openai",
|
||||
"base_url": "https://api.openai.com/v1",
|
||||
"model": "gpt-4o",
|
||||
"temperature": 0.7,
|
||||
"api_key": "sk-...",
|
||||
"context_limit": 128, # ← 必填：单位是 K tokens (128 = 128K = 128,000 tokens)
|
||||
"context_strategy": "balanced" # 可选，默认 "balanced"
|
||||
}
|
||||
```
|
||||
|
||||
### 配置示例对比
|
||||
|
||||
#### 官方文档 → API 配置
|
||||
|
||||
| 模型 | 官方文档 | API 配置 |
|
||||
|------|---------|---------|
|
||||
| GPT-4o | 128,000 tokens | `"context_limit": 128` |
|
||||
| GPT-3.5 Turbo | 16,385 tokens | `"context_limit": 17` (向上取整) |
|
||||
| Claude 3.5 Sonnet | 200,000 tokens | `"context_limit": 200` |
|
||||
| Gemini 1.5 Pro | 2,800,000 tokens | `"context_limit": 2800` |
|
||||
|
||||
### 错误示例（缺少 context_limit）
|
||||
|
||||
```bash
|
||||
POST /v3/users/{user_id}/llm-model-configs
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"name": "Invalid Config",
|
||||
"provider": "openai",
|
||||
"model": "gpt-4o",
|
||||
# ❌ 缺少 context_limit
|
||||
}
|
||||
```
|
||||
|
||||
**响应**：
|
||||
```json
|
||||
{
|
||||
"detail": "context_limit is required (unit: K tokens, e.g., 128 = 128K = 128,000 tokens). Please check your model provider's documentation for the current context window size and specify it in the configuration."
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 常见模型配置参考
|
||||
|
||||
以下是一些常见模型的 `context_limit` 配置值（**请以官方文档为准**）：
|
||||
|
||||
### OpenAI
|
||||
| 模型 | 官方文档 | 配置值 |
|
||||
|------|---------|-------|
|
||||
| GPT-4o | 128,000 tokens | `"context_limit": 128` |
|
||||
| GPT-4o-mini | 128,000 tokens | `"context_limit": 128` |
|
||||
| GPT-4 Turbo | 128,000 tokens | `"context_limit": 128` |
|
||||
| GPT-3.5 Turbo | 16,385 tokens | `"context_limit": 17` |
|
||||
|
||||
### Anthropic
|
||||
| 模型 | 官方文档 | 配置值 |
|
||||
|------|---------|-------|
|
||||
| Claude 3.5 Sonnet | 200,000 tokens | `"context_limit": 200` |
|
||||
| Claude 3 Opus | 200,000 tokens | `"context_limit": 200` |
|
||||
| Claude 3 Haiku | 200,000 tokens | `"context_limit": 200` |
|
||||
|
||||
### Google
|
||||
| 模型 | 官方文档 | 配置值 |
|
||||
|------|---------|-------|
|
||||
| Gemini 2.0 Flash | 1,000,000 tokens | `"context_limit": 1000` |
|
||||
| Gemini 1.5 Pro | 2,800,000 tokens | `"context_limit": 2800` |
|
||||
| Gemini 1.5 Flash | 2,800,000 tokens | `"context_limit": 2800` |
|
||||
|
||||
### DeepSeek
|
||||
| 模型 | 官方文档 | 配置值 |
|
||||
|------|---------|-------|
|
||||
| DeepSeek Chat | 128,000 tokens | `"context_limit": 128` |
|
||||
| DeepSeek Coder | 128,000 tokens | `"context_limit": 128` |
|
||||
|
||||
### xAI
|
||||
| 模型 | 官方文档 | 配置值 |
|
||||
|------|---------|-------|
|
||||
| Grok Beta | 128,000 tokens | `"context_limit": 128` |
|
||||
|
||||
---
|
||||
|
||||
## 问题背景
|
||||
|
||||
### 原始问题
|
||||
- 消息历史无限制累积，使用 `operator.add` 追加所有消息
|
||||
- 不同模型的上下文窗口限制不同
|
||||
- 当消息历史超过模型限制时，LLM 调用会失败
|
||||
- 没有优雅的降级或裁剪机制
|
||||
|
||||
### 解决方案
|
||||
使用 LangChain/LangGraph 的内置功能实现智能上下文管理：
|
||||
- **`trim_messages`**: LangChain 的消息裁剪工具
|
||||
- **`count_tokens_approximately`**: Token 计数功能
|
||||
- **用户配置**: 必须提供 context_limit（单位：K tokens），无默认值
|
||||
本文档说明 GNS3 Copilot 的上下文窗口管理实现机制，包括消息裁剪、Token 计数和配置验证。
|
||||
|
||||
## 实现架构
|
||||
|
||||
@ -201,51 +12,39 @@ Content-Type: application/json
|
||||
|
||||
#### Token 计数策略
|
||||
|
||||
系统使用 **tiktoken** 进行准确的 Token 计数：
|
||||
系统使用 **tiktoken** 进行 Token 计数（context_manager.py:60）：
|
||||
|
||||
1. **tiktoken（OpenAI 的 tokenizer）**
|
||||
- 使用 `cl100k_base` 编码（GPT-4）
|
||||
- 对大多数现代 LLM 准确（OpenAI、Anthropic、DeepSeek）
|
||||
- 准确率约 95%+
|
||||
- **必需依赖**：系统强制要求安装 tiktoken
|
||||
|
||||
2. **工具定义 Token 估算**
|
||||
- 工具的 schema（name、description、parameters）会被转换为 JSON 发送给 LLM
|
||||
- 系统会自动计算这些定义的 token 消耗
|
||||
- 每个工具约 500-1500 tokens（取决于 schema 复杂度）
|
||||
|
||||
#### 安装 tiktoken（必需）
|
||||
```python
|
||||
_tiktoken_encoding = tiktoken.get_encoding("cl100k_base")
|
||||
```
|
||||
|
||||
**必需依赖**：
|
||||
```bash
|
||||
pip install tiktoken>=0.8.0
|
||||
```
|
||||
|
||||
**为什么使用 tiktoken？**
|
||||
- 比字符估算准确 2-3 倍（特别是中文内容）
|
||||
- 支持 LangChain 使用的所有主流模型
|
||||
- 性能优秀（缓存编码器）
|
||||
- **这是必需依赖**，系统将无法运行如果未安装
|
||||
如果未安装 tiktoken，系统将在启动时抛出 `ModuleNotFoundError`。
|
||||
|
||||
#### 关键函数
|
||||
|
||||
**`get_model_context_limit(model_name: str, llm_config: dict) -> int`**
|
||||
- 从数据库配置获取模型的上下文窗口大小
|
||||
- **`llm_config` 必须包含 `context_limit` 字段（单位：K tokens）**
|
||||
- 如果未提供或无效，抛出 `ValueError`
|
||||
- 返回值单位为实际 tokens（K tokens × 1000）
|
||||
**`count_tokens(text: str) -> int`** (context_manager.py:84-100)
|
||||
- 使用 tiktoken 准确计数文本的 token 数
|
||||
- 使用 `cl100k_base` 编码
|
||||
- 返回精确的 token 数
|
||||
|
||||
**`calculate_max_tokens(model_limit, strategy) -> int`**
|
||||
- 计算可用 token 数量（预留输出空间）
|
||||
- 三种策略：
|
||||
- `conservative`: 使用 60% 限制（更安全）
|
||||
- `balanced`: 使用 75% 限制（默认）
|
||||
- `aggressive`: 使用 85% 限制（最大化输入）
|
||||
**`estimate_tool_tokens(tools: list) -> int`** (context_manager.py:103-169)
|
||||
- 序列化工具 schema 为 JSON
|
||||
- 使用 tiktoken 计数工具定义的 token 消耗
|
||||
- 支持 Pydantic v1/v2 兼容性
|
||||
- 失败时使用 1000 tokens 的回退值
|
||||
|
||||
**`trim_messages_for_context(messages, model_name, strategy, tool_tokens) -> list`**
|
||||
- 使用 tiktoken 裁剪消息
|
||||
- 保留最近的消息
|
||||
- 始终保留系统消息
|
||||
- **关键**：考虑工具定义占用的 tokens
|
||||
**`create_pre_model_hook(...)`** (context_manager.py:195-402)
|
||||
- 创建预处理函数（pre_model_hook）
|
||||
- 在每次 LLM 调用前自动执行：
|
||||
1. 注入 topology 信息到 system prompt
|
||||
2. 估算工具定义的 token 消耗
|
||||
3. 裁剪消息历史以适应上下文限制
|
||||
- 返回一个可调用的函数，用于准备消息
|
||||
|
||||
### 2. 裁剪逻辑详解
|
||||
|
||||
@ -278,37 +77,32 @@ pip install tiktoken>=0.8.0
|
||||
```
|
||||
第一步：计算输入预算
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ context_limit: 16,000 tokens │
|
||||
│ strategy: conservative (60%) │
|
||||
│ context_limit: 128,000 tokens (128K) │
|
||||
│ strategy: balanced (75%) │
|
||||
│ │
|
||||
│ 输入预算 = 16,000 × 0.6 = 9,600 tokens │
|
||||
│ 输入预算 = 128 × 1000 × 0.75 = 96,000 tokens │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
第二步：减去工具定义
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ 输入预算: 9,600 tokens │
|
||||
│ 工具定义: 1,862 tokens │
|
||||
│ 输入预算: 96,000 tokens │
|
||||
│ 工具定义: 1,725 tokens │
|
||||
│ │
|
||||
│ 可用于消息 = 9,600 - 1,862 = 7,738 tokens │
|
||||
│ 可用于消息 = 96,000 - 1,725 = 94,275 tokens │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
第三步：计算 System Message token（已合并）
|
||||
第三步：trim_messages 处理
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ 可用于消息: 7,738 tokens │
|
||||
│ 调用 LangChain 的 trim_messages: │
|
||||
│ - max_tokens = 94,275 (包含 system message) │
|
||||
│ - strategy = "last" (保留最新消息) │
|
||||
│ - token_counter = tiktoken 计数函数 │
|
||||
│ - include_system = True (始终保留 system) │
|
||||
│ │
|
||||
│ System Message (已包含 system + topology): │
|
||||
│ - System prompt (base): ~1,000 tokens │
|
||||
│ - Topology info: ~3,000 tokens │
|
||||
│ = 4,000 tokens (合并后) │
|
||||
│ │
|
||||
│ 可用于历史 = 7,738 - 4,000 = 3,738 tokens │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
↓
|
||||
第四步：裁剪对话历史
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ 历史消息: 5,000 tokens → 需要裁剪到 3,738 tokens │
|
||||
│ │
|
||||
│ 丢弃旧消息，直到满足限制 │
|
||||
│ trim_messages 会: │
|
||||
│ 1. 保留 SystemMessage (system + topology) │
|
||||
│ 2. 从最新消息开始，保留尽可能多的历史 │
|
||||
│ 3. 超出限制时，丢弃最旧的消息 │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
@ -329,430 +123,158 @@ pip install tiktoken>=0.8.0
|
||||
| 情况 | 处理方式 |
|
||||
|------|----------|
|
||||
| System (包含 topology) > 预算 | 保留完整的 System Message（无法分离 system 和 topology） |
|
||||
| Tools > 预算 | 警告日志，建议增加 context_limit 或减少工具数量 |
|
||||
| Tools > 预算 | ERROR 日志，建议增加 context_limit 或减少工具数量 |
|
||||
| 历史全被裁剪 | 保留最后1条用户消息 |
|
||||
|
||||
**重要提示**：
|
||||
- 当 system + topology 超出可用预算时，**两者都会被保留**
|
||||
- 无法只丢弃 topology 而保留 system prompt（因为已合并）
|
||||
- 建议在 system prompt 中精简 topology 信息或使用更短的 system prompt
|
||||
|
||||
### 2. 集成到 GNS3 Copilot
|
||||
### 3. 集成到 GNS3 Copilot
|
||||
|
||||
**修改文件**: `gns3server/agent/gns3_copilot/agent/gns3_copilot.py`
|
||||
**文件位置**: `gns3server/agent/gns3_copilot/agent/gns3_copilot.py`
|
||||
|
||||
#### 改动内容
|
||||
#### 实现方式
|
||||
|
||||
**导入新模块**:
|
||||
```python
|
||||
from gns3server.agent.gns3_copilot.agent.context_manager import (
|
||||
prepare_context_messages,
|
||||
)
|
||||
```
|
||||
**关键点**：系统使用**自定义 StateGraph**，不是 LangGraph 的预构建 agent。
|
||||
|
||||
**替换消息构建逻辑**:
|
||||
```python
|
||||
# 旧代码（手动构建）
|
||||
full_messages = (
|
||||
[SystemMessage(content=current_prompt)]
|
||||
+ context_messages
|
||||
+ state["messages"]
|
||||
)
|
||||
因此，`pre_model_hook` 不能通过 `model.invoke(config={"configurable": {"pre_model_hook": ...}})` 传递。
|
||||
|
||||
# 新代码（自动裁剪）
|
||||
full_messages = prepare_context_messages(
|
||||
state_messages=state["messages"],
|
||||
system_prompt=current_prompt,
|
||||
topology_context=topology_context,
|
||||
model_name=llm_config.get("model", "default"),
|
||||
llm_config=llm_config, # ← 包含 context_limit 和 context_strategy
|
||||
)
|
||||
```
|
||||
|
||||
### 3. Token 计数实现细节
|
||||
|
||||
系统使用 **tiktoken** 进行准确的 Token 计数：
|
||||
|
||||
#### 3.1 Token 计数器
|
||||
|
||||
**使用的编码**：`cl100k_base` (GPT-4)
|
||||
|
||||
**支持的模型**：
|
||||
- ✅ OpenAI (GPT-4, GPT-3.5)
|
||||
- ✅ Anthropic (Claude 系列)
|
||||
- ✅ DeepSeek (deepseek-chat, deepseek-coder)
|
||||
- ✅ 大多数基于 GPT-4 架构的模型
|
||||
|
||||
**准确率**：
|
||||
- 英文：95%+
|
||||
- 中文：95%+
|
||||
- 代码：90-95%
|
||||
|
||||
#### 3.2 工具定义 Token 估算
|
||||
**正确的使用方式**：**直接调用** `pre_hook` 函数准备消息。
|
||||
|
||||
```python
|
||||
def estimate_tool_tokens(tools: list[Any]) -> int:
|
||||
"""动态序列化工具 schema 并计算 token"""
|
||||
for tool in tools:
|
||||
tool_schema = {
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": tool.name,
|
||||
"description": tool.description,
|
||||
"parameters": tool.args_schema.schema()
|
||||
}
|
||||
}
|
||||
# 使用 tiktoken 计数
|
||||
schema_str = json.dumps(tool_schema, ensure_ascii=False)
|
||||
tool_tokens = len(encoding.encode(schema_str))
|
||||
def llm_call(state: dict, config: RunnableConfig | None = None):
|
||||
"""LLM decides whether to call a tool or not."""
|
||||
|
||||
# 1. 获取 topology 信息
|
||||
project_id = config["configurable"].get("project_id")
|
||||
topology_info = None
|
||||
if project_id:
|
||||
topology_tool = GNS3TopologyTool()
|
||||
topology = topology_tool._run(project_id=project_id)
|
||||
if topology and "error" not in topology:
|
||||
topology_info = topology
|
||||
|
||||
# 2. 创建 pre_model_hook
|
||||
system_prompt = load_system_prompt()
|
||||
pre_hook = create_pre_model_hook(
|
||||
system_prompt=system_prompt,
|
||||
get_topology_func=lambda s: s.get("topology_info"),
|
||||
get_llm_config_func=get_current_llm_config,
|
||||
get_tools_func=lambda: tools,
|
||||
)
|
||||
|
||||
# 3. 创建 model with tools
|
||||
model_with_tools = create_base_model_with_tools(tools, llm_config=llm_config)
|
||||
|
||||
# 4. ⭐ 关键：直接调用 pre_hook 准备消息
|
||||
logger.info("Calling pre_hook to prepare %d messages", len(messages))
|
||||
prepared_state = pre_hook({"messages": messages, "topology_info": topology_info})
|
||||
prepared_messages = prepared_state["messages"]
|
||||
|
||||
# 5. 使用准备好的消息调用 LLM
|
||||
response = model_with_tools.invoke(prepared_messages)
|
||||
|
||||
return {"messages": [response], ...}
|
||||
```
|
||||
|
||||
**典型消耗**：
|
||||
- 简单工具（1-2 个参数）：500-800 tokens
|
||||
- 中等工具（3-5 个参数）：800-1200 tokens
|
||||
- 复杂工具（嵌套参数）：1200-1500 tokens
|
||||
- 8 个工具：约 6000-10000 tokens
|
||||
#### 为什么不通过 config 传递？
|
||||
|
||||
---
|
||||
LangGraph 的 `pre_model_hook` 参数仅适用于**预构建的 agent**，不适用于自定义 StateGraph。
|
||||
|
||||
## 策略详细说明
|
||||
| Agent 类型 | pre_model_hook 支持方式 |
|
||||
|------------|------------------------|
|
||||
| `create_react_agent` | ✅ 通过 `pre_model_hook` 参数 |
|
||||
| `chat_agent_executor` | ✅ 通过 `pre_model_hook` 参数 |
|
||||
| **自定义 StateGraph** | ❌ **不支持**，需要直接调用 |
|
||||
|
||||
### Conservative（保守策略）
|
||||
我们的实现使用的是自定义 StateGraph（`agent_builder = StateGraph(MessagesState)`），所以必须直接调用 `pre_hook`。
|
||||
|
||||
**参数配置**：
|
||||
```json
|
||||
{
|
||||
"context_limit": 128,
|
||||
"context_strategy": "conservative"
|
||||
}
|
||||
```
|
||||
|
||||
**实际效果**：
|
||||
- **使用比例**：60% (context_limit × 0.6)
|
||||
- **计算公式**：`max_tokens = context_limit × 1000 × 0.6`
|
||||
- **示例**：对于 128K 上下文限制
|
||||
- 保留空间：`128 × 1000 × 0.6 = 76,800` tokens 用于输入
|
||||
- 预留空间：`128,000 - 76,800 = 51,200` tokens 用于输出
|
||||
|
||||
**适用场景**：
|
||||
1. **长输出任务**
|
||||
- 代码生成（可能生成数百行代码）
|
||||
- 文章写作（需要完整文章内容）
|
||||
- 详细报告生成（包含多个章节）
|
||||
|
||||
2. **复杂任务**
|
||||
- 多步骤推理任务
|
||||
- 需要深度分析的请求
|
||||
- 综合性问题的解决方案
|
||||
|
||||
3. **不确定输出大小时**
|
||||
- 不确定 LLM 会返回多长的内容
|
||||
- 首次尝试某种类型的任务
|
||||
- 需要额外安全边界的场景
|
||||
|
||||
**优缺点**：
|
||||
- ✅ **优点**：输出更不容易被截断，安全性高
|
||||
- ❌ **缺点**：输入上下文较少，可能遗漏早期信息
|
||||
|
||||
**日志示例**：
|
||||
```
|
||||
INFO: Context prepared: 20 msgs, ~72800 tokens / 128K limit (56.9%), strategy=conservative
|
||||
INFO: Available for output: ~51200 tokens
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Balanced（平衡策略，推荐）
|
||||
|
||||
**参数配置**：
|
||||
```json
|
||||
{
|
||||
"context_limit": 128,
|
||||
"context_strategy": "balanced"
|
||||
}
|
||||
```
|
||||
|
||||
**实际效果**：
|
||||
- **使用比例**：75% (context_limit × 0.75)
|
||||
- **计算公式**：`max_tokens = context_limit × 1000 × 0.75`
|
||||
- **示例**：对于 128K 上下文限制
|
||||
- 保留空间：`128 × 1000 × 0.75 = 96,000` tokens 用于输入
|
||||
- 预留空间：`128,000 - 96,000 = 32,000` tokens 用于输出
|
||||
|
||||
**适用场景**：
|
||||
1. **一般对话**
|
||||
- 日常聊天交互
|
||||
- 问答式对话
|
||||
- 技术支持和咨询
|
||||
|
||||
2. **大多数场景**
|
||||
- 网络配置命令生成
|
||||
- 故障排查建议
|
||||
- 知识问答
|
||||
|
||||
3. **平衡输入和输出**
|
||||
- 需要较多上下文，但输出也较长的场景
|
||||
- 中等复杂度的任务
|
||||
|
||||
**优缺点**：
|
||||
- ✅ **优点**：在输入上下文和输出空间之间取得良好平衡
|
||||
- ✅ **优点**：适用于大多数使用场景
|
||||
- ⚠️ **注意**：对于特别长的输出可能被截断
|
||||
|
||||
**日志示例**：
|
||||
```
|
||||
INFO: Context prepared: 30 msgs, ~85600 tokens / 128K limit (66.9%), strategy=balanced
|
||||
INFO: Available for output: ~32000 tokens
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Aggressive（激进策略）
|
||||
|
||||
**参数配置**：
|
||||
```json
|
||||
{
|
||||
"context_limit": 128,
|
||||
"context_strategy": "aggressive"
|
||||
}
|
||||
```
|
||||
|
||||
**实际效果**：
|
||||
- **使用比例**：85% (context_limit × 0.85)
|
||||
- **计算公式**：`max_tokens = context_limit × 1000 × 0.85`
|
||||
- **示例**：对于 128K 上下文限制
|
||||
- 保留空间：`128 × 1000 × 0.85 = 108,800` tokens 用于输入
|
||||
- 预留空间：`128,000 - 108,800 = 19,200` tokens 用于输出
|
||||
|
||||
**适用场景**：
|
||||
1. **简短输出任务**
|
||||
- 是/否判断
|
||||
- 简短回答确认
|
||||
- 状态查询类请求
|
||||
|
||||
2. **分析类任务**
|
||||
- 日志分析（输出分析结果，但不需要很长）
|
||||
- 数据解读（输出简洁的结论）
|
||||
- 配置检查（返回 OK 或简短说明）
|
||||
|
||||
3. **输出比较确定的场景**
|
||||
- 明确知道输出会很短
|
||||
- 只需要简单确认或状态
|
||||
- 不需要长篇解释的任务
|
||||
|
||||
**优缺点**：
|
||||
- ✅ **优点**：最大化输入上下文，保留更多历史信息
|
||||
- ❌ **缺点**：输出容易被截断，不适用于长输出任务
|
||||
|
||||
**日志示例**：
|
||||
```
|
||||
INFO: Context prepared: 50 msgs, ~105200 tokens / 128K limit (82.2%), strategy=aggressive
|
||||
INFO: Available for output: ~19200 tokens
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 策略对比总结
|
||||
|
||||
| 特性 | Conservative | Balanced | Aggressive |
|
||||
|------|-------------|----------|------------|
|
||||
| **输入比例** | 60% | 75% | 85% |
|
||||
| **输出预留** | 40% | 25% | 15% |
|
||||
| **上下文数量** | 最少 | 中等 | 最多 |
|
||||
| **输出空间** | 最大 | 中等 | 最小 |
|
||||
| **适用性** | 长输出 | 一般使用 | 短输出 |
|
||||
| **风险** | 上下文不足 | 平衡 | 输出截断 |
|
||||
|
||||
**选择建议流程**：
|
||||
```
|
||||
1. 任务类型是什么？
|
||||
├─ 代码生成/长文档 → Conservative
|
||||
├─ 日常对话/一般任务 → Balanced (推荐)
|
||||
└─ 简短确认/分析 → Aggressive
|
||||
|
||||
2. 输出长度预估？
|
||||
├─ 不确定/可能很长 → Conservative
|
||||
├─ 中等长度 → Balanced
|
||||
└─ 很短/简洁 → Aggressive
|
||||
|
||||
3. 上下文重要性？
|
||||
├─ 早期历史不太重要 → Aggressive (更多上下文)
|
||||
├─ 需要平衡 → Balanced
|
||||
└─ 重点是输出完整性 → Conservative (更多输出空间)
|
||||
```
|
||||
|
||||
## 工作流程
|
||||
### 4. 执行流程
|
||||
|
||||
```
|
||||
用户发送消息
|
||||
↓
|
||||
获取消息历史 state["messages"]
|
||||
llm_call 节点被调用
|
||||
↓
|
||||
构建系统提示 + 拓扑信息
|
||||
获取 project_id (从 config["configurable"])
|
||||
↓
|
||||
调用 prepare_context_messages()
|
||||
调用 GNS3TopologyTool._run(project_id) 获取 topology
|
||||
↓
|
||||
├─ 估算 token 数量
|
||||
├─ 获取模型上下文限制
|
||||
├─ 判断是否需要裁剪
|
||||
└─ 如果需要 → 调用 trim_messages()
|
||||
存储 topology_info 到 state
|
||||
↓
|
||||
调用 LLM（带裁剪后的消息）
|
||||
创建 pre_model_hook (通过 create_pre_model_hook())
|
||||
↓
|
||||
返回响应
|
||||
【关键】直接调用 pre_hook({"messages": messages, "topology_info": topology_info})
|
||||
├─ 1. 注入 topology 到 system prompt
|
||||
├─ 2. 估算工具定义 tokens
|
||||
├─ 3. 调用 trim_messages() 裁剪消息
|
||||
└─ 4. 返回准备好的消息列表
|
||||
↓
|
||||
使用准备好的消息调用 model.invoke()
|
||||
↓
|
||||
返回 LLM 响应
|
||||
```
|
||||
|
||||
## 日志输出示例
|
||||
|
||||
### 正常情况（使用 tiktoken，包含工具定义）
|
||||
```
|
||||
INFO: Using tiktoken (cl100k_base) for accurate token counting
|
||||
INFO: Tool definitions estimated at ~8500 total tokens (8 tools)
|
||||
INFO: Using database config context limit: 128000 tokens for model 'gpt-4o'
|
||||
INFO: Context prepared: 15 msgs, ~28432 tokens (messages) + 8500 tokens (tools) = 36932 total / 128K limit (28.9%), strategy=balanced
|
||||
INFO: LLM call completed: tool_calls=2
|
||||
```
|
||||
|
||||
### 发生裁剪时
|
||||
```
|
||||
INFO: Using tiktoken (cl100k_base) for accurate token counting
|
||||
INFO: Tool definitions estimated at ~8500 total tokens (8 tools)
|
||||
INFO: Using database config context limit: 128000 tokens for model 'gpt-4o'
|
||||
INFO: Trimming messages: 95000 → 82000 tokens (model: gpt-4o)
|
||||
INFO: Trimmed 50 → 25 messages
|
||||
INFO: Context prepared: 27 msgs, ~82000 tokens (messages) + 8500 tokens (tools) = 90500 total / 128K limit (70.7%), strategy=balanced
|
||||
```
|
||||
|
||||
### 配置错误时
|
||||
```
|
||||
ERROR: context_limit is required but not provided for model 'gpt-4o'.
|
||||
Please configure context_limit in your LLM model configuration.
|
||||
Refer to the model provider's documentation for the current context window size.
|
||||
```
|
||||
|
||||
### 日志格式说明
|
||||
|
||||
新的日志格式（包含工具定义和 topology 分解）：
|
||||
```
|
||||
Context prepared: system={总tokens} (base={base_tokens} + topology={topology_tokens}) + history={history_tokens} + tools={tool_tokens} = {总计} total / {限制}K limit ({使用百分比}%), strategy={策略}
|
||||
```
|
||||
|
||||
字段说明：
|
||||
- **system**：System Message 的总 token 数（包含 system prompt + topology）
|
||||
- **base**：基础 system prompt 的 token 数（不含 topology）
|
||||
- **topology**：Topology info 的 token 数
|
||||
- **history**：对话历史（HumanMessage + AIMessage + ToolMessage）的 token 数
|
||||
- **tools**：工具定义（schema）的 token 数
|
||||
- **总计**：所有部分的 token 总和
|
||||
- **限制**：模型的上下文窗口大小（K tokens）
|
||||
- **使用百分比**：总计 tokens / 限制 × 100%
|
||||
- **策略**：使用的裁剪策略（conservative/balanced/aggressive）
|
||||
|
||||
**示例输出**：
|
||||
```
|
||||
Context prepared: system=6124 (base=2804 + topology=3320) + history=11800 + tools=1862 = 19786 total / 128K limit (15.5%), strategy=balanced
|
||||
```
|
||||
|
||||
这表示：
|
||||
- System Message 总共 6124 tokens
|
||||
- 其中基础 system prompt 2804 tokens
|
||||
- 其中 topology info 3320 tokens
|
||||
- 对话历史 11800 tokens
|
||||
- 工具定义 1862 tokens
|
||||
- 总计 19786 tokens，使用 128K 限制的 15.5%
|
||||
|
||||
---
|
||||
|
||||
## Token 计数准确性
|
||||
## 策略实现
|
||||
|
||||
### 为什么估算值和实际值可能不同？
|
||||
### Context Strategy Ratios
|
||||
|
||||
#### 1. **工具定义的影响**
|
||||
**定义**（context_manager.py:68-72）：
|
||||
|
||||
每次 LLM 调用时，工具定义会被转换为 JSON 并发送给 LLM：
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "ExecuteMultipleDeviceCommands",
|
||||
"description": "在多个网络设备上执行命令...",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"commands": {...}
|
||||
}
|
||||
}
|
||||
}
|
||||
```python
|
||||
CONTEXT_STRATEGY_RATIOS = {
|
||||
"conservative": 0.60,
|
||||
"balanced": 0.75,
|
||||
"aggressive": 0.85,
|
||||
}
|
||||
```
|
||||
|
||||
**典型消耗**：
|
||||
- 简单工具：500-800 tokens
|
||||
- 复杂工具：1000-1500 tokens
|
||||
- 8 个工具：约 6000-12000 tokens
|
||||
**默认值**（context_manager.py:74）：
|
||||
```python
|
||||
DEFAULT_CONTEXT_STRATEGY = "balanced"
|
||||
```
|
||||
|
||||
**系统处理**：
|
||||
- ✅ 现在会自动估算工具定义的 tokens
|
||||
- ✅ 日志中会显示工具 tokens
|
||||
- ✅ 裁剪决策会考虑工具定义
|
||||
### 策略对比
|
||||
|
||||
#### 2. **中文内容的 Tokenization**
|
||||
|
||||
不同语言对 token 的使用效率不同：
|
||||
|
||||
| 语言 | 平均字符/token | 示例 |
|
||||
|------|---------------|------|
|
||||
| 英文 | 3-4 字符 | "Hello world" ≈ 2-3 tokens |
|
||||
| 中文 | 1-1.5 字符 | "你好世界" ≈ 3-4 tokens |
|
||||
| 代码 | 2-3 字符 | `print("Hello")` ≈ 4-5 tokens |
|
||||
|
||||
**使用 tiktoken 后**：
|
||||
- 对中文的准确率从 30-40% 提升到 95%+
|
||||
- 对英文的准确率保持在 95%+
|
||||
- 对代码的准确率约 90-95%
|
||||
|
||||
#### 3. **System Prompt 和 Topology Context**
|
||||
|
||||
- **System Prompt**: 固定内容，约 1000-2000 tokens
|
||||
- **Topology Context**: 动态内容，取决于拓扑大小
|
||||
- 小型拓扑（< 10 节点）：约 500-1000 tokens
|
||||
- 中型拓扑（10-50 节点）：约 2000-5000 tokens
|
||||
- 大型拓扑（> 50 节点）：约 5000-10000+ tokens
|
||||
|
||||
这些都会被准确计数并计入总限制。
|
||||
| 策略 | 输入比例 | 输出预留 | 计算公式 |
|
||||
|------|---------|---------|---------|
|
||||
| Conservative | 60% | 40% | `context_limit × 1000 × 0.60` |
|
||||
| Balanced | 75% | 25% | `context_limit × 1000 × 0.75` |
|
||||
| Aggressive | 85% | 15% | `context_limit × 1000 × 0.85` |
|
||||
|
||||
---
|
||||
|
||||
## Token 使用对比
|
||||
## 日志输出
|
||||
|
||||
### 改进前（LangChain 估算）
|
||||
### 正常情况（topology 成功注入）
|
||||
|
||||
```
|
||||
INFO: Context prepared: 29 msgs, ~11523 tokens / 128K limit (9.0%), strategy=conservative
|
||||
实际发送: 39700 tokens
|
||||
差距: 28177 tokens (2.4x 低估)
|
||||
INFO: Calling pre_hook to prepare 1 messages
|
||||
INFO: ✓ Topology injected: 7722 chars, nodes: ['netshoot-1', 'R1', 'R2', 'IOU-L3-1', 'IOU-L3-2']
|
||||
INFO: Context ready: 2 msgs, ~3815 tokens + 1725 tools = 5540 / 128K (4.3%), strategy=conservative
|
||||
INFO: Messages prepared: 1 → 2
|
||||
INFO: LLM call completed: tool_calls=0
|
||||
```
|
||||
|
||||
**问题**：
|
||||
1. ❌ 工具定义没有被计入（约 8000-12000 tokens）
|
||||
2. ❌ 中文内容被低估（约 2.4x 误差）
|
||||
3. ❌ 总体低估约 2-3 倍
|
||||
|
||||
### 改进后（tiktoken + 工具定义）
|
||||
### 发生裁剪时
|
||||
|
||||
```
|
||||
INFO: Using tiktoken (cl100k_base) for accurate token counting
|
||||
INFO: Tool definitions estimated at ~8500 total tokens (8 tools)
|
||||
INFO: Context prepared: 29 msgs, ~27500 tokens (messages) + 8500 tokens (tools) = 36000 total / 128K limit (28.1%), strategy=conservative
|
||||
实际发送: 39700 tokens
|
||||
差距: 3700 tokens (1.1x 误差)
|
||||
INFO: Calling pre_hook to prepare 50 messages
|
||||
INFO: ✓ Topology injected: 8500 chars, nodes: ['R1', 'R2', ...]
|
||||
INFO: Messages trimmed: 50 → 25 msgs. Total: ~82000 tokens + 1725 tools = 83725 / 128K (65.4%), strategy=balanced
|
||||
INFO: Messages prepared: 50 → 25
|
||||
```
|
||||
|
||||
**改进**：
|
||||
1. ✅ 工具定义被准确估算
|
||||
2. ✅ 中文内容准确率提升到 95%+
|
||||
3. ✅ 总体误差降低到 10% 以内
|
||||
### topology 为 None 时
|
||||
|
||||
```
|
||||
INFO: Calling pre_hook to prepare 1 messages
|
||||
WARNING: ✗ Topology data is None, injecting placeholder
|
||||
INFO: Context ready: 2 msgs, ~800 tokens + 1725 tools = 2525 / 128K (2.0%), strategy=balanced
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@ -760,11 +282,10 @@ INFO: Context prepared: 29 msgs, ~27500 tokens (messages) + 8500 tokens (tools)
|
||||
|
||||
### tiktoken 未安装
|
||||
|
||||
如果 tiktoken 未安装，系统将在首次尝试计数 tokens 时抛出错误：
|
||||
如果 tiktoken 未安装，系统将在启动时抛出错误：
|
||||
|
||||
```python
|
||||
ImportError: tiktoken is required for accurate token counting.
|
||||
Please install it with: pip install tiktoken>=0.8.0
|
||||
ModuleNotFoundError: No module named 'tiktoken'
|
||||
```
|
||||
|
||||
**解决方法**：
|
||||
@ -772,83 +293,34 @@ Please install it with: pip install tiktoken>=0.8.0
|
||||
pip install tiktoken>=0.8.0
|
||||
```
|
||||
|
||||
### Token 计数失败
|
||||
### context_limit 缺失或无效
|
||||
|
||||
如果 LLM 配置中没有 `context_limit` 或值无效（context_manager.py:285-295）：
|
||||
|
||||
```python
|
||||
try:
|
||||
tokens = count_messages_tokens(messages)
|
||||
except ImportError as e:
|
||||
logger.error("tiktoken not available: %s", e)
|
||||
raise # Re-raise to fail fast
|
||||
except Exception as e:
|
||||
logger.error("Failed to count tokens: %s", e)
|
||||
raise
|
||||
if "context_limit" not in llm_config:
|
||||
raise ValueError("context_limit is required in LLM config")
|
||||
|
||||
limit = llm_config["context_limit"]
|
||||
if not isinstance(limit, int) or limit <= 0:
|
||||
raise ValueError(f"Invalid context_limit: {limit}")
|
||||
```
|
||||
|
||||
### 裁剪失败
|
||||
|
||||
```python
|
||||
try:
|
||||
trimmed = trim_messages(...)
|
||||
except Exception as e:
|
||||
logger.error("Failed to trim: %s", e)
|
||||
# 回退到简单的切片操作
|
||||
return system_msgs + other_msgs[-N:]
|
||||
logger.error("Failed to trim messages: %s", e)
|
||||
logger.warning("Returning original messages due to trimming error")
|
||||
return {"messages": messages_with_system}
|
||||
```
|
||||
|
||||
## 参考资源
|
||||
|
||||
- [tiktoken (OpenAI Tokenizer)](https://github.com/openai/tiktoken)
|
||||
- [LangChain Messages Utils](https://python.langchain.com/docs/messages/)
|
||||
- [LangGraph Memory Management](https://langchain-ai.github.io/langgraph/concepts/agentic_concepts/#memory)
|
||||
- [OpenAI Models Context Limits](https://platform.openai.com/docs/models)
|
||||
- [Anthropic Models Context Limits](https://docs.anthropic.com/claude/docs/models-overview)
|
||||
---
|
||||
|
||||
## 相关源文件
|
||||
|
||||
- `gns3server/agent/gns3_copilot/agent/context_manager.py` - 上下文管理核心逻辑
|
||||
- `gns3server/agent/gns3_copilot/agent/gns3_copilot.py` - LLM 调用节点
|
||||
- `gns3server/agent/gns3_copilot/agent_service.py` - Agent 服务接口
|
||||
|
||||
## 总结
|
||||
|
||||
通过使用 LangChain/LangGraph 的内置功能和自定义优化，我们实现了：
|
||||
1. ✅ 智能裁剪消息历史，避免超限
|
||||
2. ✅ **必须手动配置 context_limit**，确保使用正确的上下文窗口大小
|
||||
3. ✅ 可配置的裁剪策略（保守/平衡/激进）
|
||||
4. ✅ 始终保留系统消息（包含 system prompt 和 topology）
|
||||
5. ✅ 详细的日志输出，便于调试
|
||||
6. ✅ 优雅的错误处理和明确的错误提示
|
||||
7. ✅ 提供参考工具，帮助查找常见模型的上下文限制
|
||||
8. ✅ **使用 tiktoken 进行准确的 token 计数**（准确率 95%+）
|
||||
9. ✅ **自动估算工具定义的 token 消耗**
|
||||
10. ✅ **支持中文、英文、代码等多种内容类型**
|
||||
11. ✅ **使用模板变量动态注入 topology info** (新增)
|
||||
12. ✅ **日志中详细显示 system/topology/history 分解** (新增)
|
||||
|
||||
### 关键优势
|
||||
|
||||
- **准确性**：
|
||||
- 用户从官方文档获取最新的上下文限制，避免使用过时数据
|
||||
- tiktoken 提供准确的 token 计数，误差 < 10%
|
||||
- 工具定义 token 被正确计入估算
|
||||
|
||||
- **灵活性**：
|
||||
- 每个配置独立设置，支持不同用户使用不同限制
|
||||
- 自动适配不同语言和内容类型
|
||||
- 模板变量注入方式易于维护和调整
|
||||
|
||||
- **明确性**：
|
||||
- 缺少配置时立即报错，避免静默失败
|
||||
- 详细日志显示所有 token 消耗（system base + topology + history + tools）
|
||||
- 清楚显示各部分的 token 分解
|
||||
|
||||
- **可维护性**：
|
||||
- 无需维护内置默认值，减少代码维护负担
|
||||
- 模板变量注入方式，system prompt 和 topology 管理更清晰
|
||||
- 模块化设计，易于扩展和调试
|
||||
|
||||
- **可观测性**：
|
||||
- 详细日志显示上下文使用情况和裁剪决策
|
||||
- 区分 system base tokens、topology tokens、history tokens 和工具 tokens
|
||||
- 显示使用百分比和策略选择
|
||||
|
||||
这个实现确保了即使在进行长对话时，系统也不会因为上下文溢出而失败。
|
||||
- `gns3server/agent/gns3_copilot/agent/gns3_copilot.py` - LLM 调用节点（StateGraph）
|
||||
- `gns3server/agent/gns3_copilot/agent/model_factory.py` - 模型创建和工具绑定
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@ -56,7 +56,7 @@ from gns3server.agent.gns3_copilot.agent.model_factory import (
|
||||
create_title_model,
|
||||
)
|
||||
from gns3server.agent.gns3_copilot.agent.context_manager import (
|
||||
prepare_context_messages,
|
||||
create_pre_model_hook,
|
||||
)
|
||||
from gns3server.agent.gns3_copilot.gns3_client import GNS3TopologyTool
|
||||
from gns3server.agent.gns3_copilot.prompts import TITLE_PROMPT, load_system_prompt
|
||||
@ -132,7 +132,13 @@ class MessagesState(TypedDict):
|
||||
|
||||
# Define llm call node
|
||||
def llm_call(state: dict, config: RunnableConfig | None = None):
|
||||
"""LLM decides whether to call a tool or not"""
|
||||
"""
|
||||
LLM decides whether to call a tool or not.
|
||||
|
||||
Uses pre_model_hook pattern for automatic topology injection and
|
||||
message trimming, ensuring separation of concerns and complete
|
||||
history preservation in state["messages"].
|
||||
"""
|
||||
|
||||
logger.info("LLM call node invoked")
|
||||
|
||||
@ -161,22 +167,14 @@ def llm_call(state: dict, config: RunnableConfig | None = None):
|
||||
"topology_info": None,
|
||||
}
|
||||
|
||||
# Get system prompt based on ENGLISH_LEVEL configuration
|
||||
# load_system_prompt() will select base_prompt.py or english_level_prompt_a1-c2.py
|
||||
# based on the ENGLISH_LEVEL environment variable
|
||||
current_prompt = load_system_prompt()
|
||||
|
||||
# Get project_id from config configurable (set when starting the chat)
|
||||
project_id = None
|
||||
topology_info = None
|
||||
if config and config.get("configurable"):
|
||||
project_id = config["configurable"].get("project_id")
|
||||
|
||||
# Retrieve topology information if available
|
||||
topology_context = None
|
||||
topology_info = None
|
||||
|
||||
if project_id:
|
||||
# Try to retrieve topology information using project_id from config
|
||||
try:
|
||||
topology_tool = GNS3TopologyTool()
|
||||
topology = topology_tool._run(project_id=project_id)
|
||||
@ -187,8 +185,6 @@ def llm_call(state: dict, config: RunnableConfig | None = None):
|
||||
"Successfully retrieved topology for project_id: %s, name: %s",
|
||||
project_id, topology.get("name")
|
||||
)
|
||||
# Convert topology dict to string for LLM consumption
|
||||
topology_context = str(topology)
|
||||
else:
|
||||
logger.warning(
|
||||
"Failed to retrieve topology for project_id %s: %s",
|
||||
@ -197,18 +193,16 @@ def llm_call(state: dict, config: RunnableConfig | None = None):
|
||||
except Exception as e:
|
||||
logger.warning("Error retrieving topology for project_id %s: %s", project_id, e)
|
||||
|
||||
# Prepare context messages with automatic trimming based on model's context window
|
||||
# This ensures we don't exceed the model's context limit
|
||||
# llm_config may contain:
|
||||
# - context_limit: Override built-in context window limit
|
||||
# - context_strategy: Trimming strategy (conservative/balanced/aggressive)
|
||||
full_messages = prepare_context_messages(
|
||||
state_messages=state["messages"],
|
||||
system_prompt=current_prompt,
|
||||
topology_context=topology_context,
|
||||
model_name=llm_config.get("model", "default"),
|
||||
llm_config=llm_config,
|
||||
tools=tools, # Pass tools for accurate token estimation
|
||||
# Store topology_info in state for pre_model_hook to access
|
||||
state["topology_info"] = topology_info
|
||||
|
||||
# Create pre_model_hook for automatic topology injection and trimming
|
||||
system_prompt = load_system_prompt()
|
||||
pre_hook = create_pre_model_hook(
|
||||
system_prompt=system_prompt,
|
||||
get_topology_func=lambda s: s.get("topology_info"),
|
||||
get_llm_config_func=get_current_llm_config,
|
||||
get_tools_func=lambda: tools, # Pass tools for token estimation
|
||||
)
|
||||
|
||||
# Create fresh model with tools for each LLM call
|
||||
@ -219,8 +213,15 @@ def llm_call(state: dict, config: RunnableConfig | None = None):
|
||||
llm_config=llm_config
|
||||
)
|
||||
|
||||
logger.info("Invoking LLM with %d messages", len(full_messages))
|
||||
response = model_with_tools.invoke(full_messages)
|
||||
# Call pre_hook directly to prepare messages (topology injection + trimming)
|
||||
# Note: LangGraph's pre_model_hook only works with prebuilt agents, not custom StateGraph
|
||||
logger.info("Calling pre_hook to prepare %d messages", len(messages))
|
||||
prepared_state = pre_hook({"messages": messages, "topology_info": topology_info})
|
||||
prepared_messages = prepared_state["messages"]
|
||||
logger.info("Messages prepared: %d → %d", len(messages), len(prepared_messages))
|
||||
|
||||
# Invoke model with prepared messages
|
||||
response = model_with_tools.invoke(prepared_messages)
|
||||
|
||||
logger.info("LLM call completed: tool_calls=%d",
|
||||
len(response.tool_calls) if hasattr(response, 'tool_calls') else 0)
|
||||
|
||||
112
gns3server/agent/gns3_copilot/deprecated/README.md
Normal file
112
gns3server/agent/gns3_copilot/deprecated/README.md
Normal file
@ -0,0 +1,112 @@
|
||||
# Deprecated Context Manager
|
||||
|
||||
This directory contains the **advanced context management implementation** that has been replaced with a simplified version.
|
||||
|
||||
## What's in Here
|
||||
|
||||
- **context_manager.py** (~650 lines)
|
||||
- Accurate token counting using tiktoken
|
||||
- Tool definition token estimation
|
||||
- Custom message trimming with AIMessage + ToolMessage pairing
|
||||
- Detailed logging and diagnostics
|
||||
|
||||
## Why Was It Moved?
|
||||
|
||||
The advanced implementation was **over-engineered** for the current use case:
|
||||
|
||||
| Feature | Complexity | Necessity | Current Status |
|
||||
|---------|-----------|-----------|----------------|
|
||||
| Template injection | Low | ✅ Essential | **Kept** (in new version) |
|
||||
| Token counting (tiktoken) | High | ❓ Optional | Moved here |
|
||||
| Message trimming | Medium | ❓ Optional | **Simplified** (LangChain native) |
|
||||
| Tool token estimation | High | ❓ Optional | Moved here |
|
||||
| AIMessage/ToolMessage pairing | High | ✅ Important | Moved here |
|
||||
| 3 strategies (conservative/balanced/aggressive) | Low | ✅ Useful | **Kept** (in new version) |
|
||||
|
||||
## When to Use This Implementation
|
||||
|
||||
### Use the deprecated version if:
|
||||
|
||||
1. **You need accurate token counting**
|
||||
- Your model has strict token limits
|
||||
- You need to know exact token usage
|
||||
- You're working with cost-sensitive applications
|
||||
|
||||
2. **You have many tools**
|
||||
- Tool definitions consume significant tokens (500-1500 per tool)
|
||||
- You need to account for tool tokens in context limit
|
||||
|
||||
3. **You need AIMessage + ToolMessage pairing**
|
||||
- Your LLM requires tool calls and results to stay together
|
||||
- You've encountered errors from orphaned ToolMessages
|
||||
|
||||
4. **You need detailed diagnostics**
|
||||
- Debugging context limit issues
|
||||
- Optimizing token usage
|
||||
- Fine-tuning context strategy
|
||||
|
||||
### Use the current simplified version if:
|
||||
|
||||
1. ✅ You just need topology injection
|
||||
2. ✅ Your model has large context (128K+ tokens)
|
||||
3. ✅ Conversations are typically short (<50 turns)
|
||||
4. ✅ You don't need exact token counts
|
||||
|
||||
## How to Restore the Advanced Version
|
||||
|
||||
If you find you need the advanced features:
|
||||
|
||||
```python
|
||||
# 1. Remove current simplified version
|
||||
rm gns3server/agent/gns3_copilot/agent/context_manager.py
|
||||
|
||||
# 2. Restore from deprecated
|
||||
cp gns3server/agent/gns3_copilot/deprecated/context_manager.py \
|
||||
gns3server/agent/gns3_copilot/agent/context_manager.py
|
||||
```
|
||||
|
||||
## Key Differences
|
||||
|
||||
### Simplified Version (Current)
|
||||
```python
|
||||
# ~200 lines
|
||||
- Uses LangChain's native trim_messages
|
||||
- Simple token estimation (char count / 4)
|
||||
- Template injection for topology
|
||||
- 3 strategies: conservative/balanced/aggressive
|
||||
```
|
||||
|
||||
### Advanced Version (Deprecated)
|
||||
```python
|
||||
# ~650 lines
|
||||
- Custom trimming with AIMessage/ToolMessage pairing
|
||||
- Accurate tiktoken-based token counting
|
||||
- Tool definition token estimation
|
||||
- Detailed logging with token breakdown
|
||||
- Template injection for topology
|
||||
- 3 strategies: conservative/balanced/aggressive
|
||||
```
|
||||
|
||||
## Performance Comparison
|
||||
|
||||
| Metric | Simplified | Advanced |
|
||||
|--------|-----------|----------|
|
||||
| Code size | ~200 lines | ~650 lines |
|
||||
| Token accuracy | ~80% (estimation) | ~95%+ (tiktoken) |
|
||||
| Trimming safety | Good | Excellent |
|
||||
| Execution speed | Fast | Slower (tiktoken overhead) |
|
||||
| Maintenance | Low | High |
|
||||
|
||||
## Future Considerations
|
||||
|
||||
If the simplified version proves insufficient:
|
||||
1. Consider adding tiktoken back (but keep architecture simple)
|
||||
2. Use LangChain's more advanced trim_messages features
|
||||
3. Add optional tool token estimation
|
||||
4. Consider a hybrid approach: simple by default, advanced when needed
|
||||
|
||||
---
|
||||
|
||||
**Moved**: 2025-03-05
|
||||
**Reason**: Simplification for current use case
|
||||
**Status**: Available for future use if needed
|
||||
730
gns3server/agent/gns3_copilot/deprecated/context_manager.py
Normal file
730
gns3server/agent/gns3_copilot/deprecated/context_manager.py
Normal file
@ -0,0 +1,730 @@
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
#
|
||||
# GNS3-Copilot - AI-powered Network Lab Assistant for GNS3
|
||||
#
|
||||
# This file is part of GNS3-Copilot project.
|
||||
#
|
||||
# GNS3-Copilot is free software: you can redistribute it and/or modify it
|
||||
# under the terms of the GNU General Public License as published by the
|
||||
# Free Software Foundation, either version 3 of the License, or (at your
|
||||
# option) any later version.
|
||||
#
|
||||
# GNS3-Copilot is distributed in the hope that it will be useful, but
|
||||
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
|
||||
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
|
||||
# for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with GNS3-Copilot. If not, see <https://www.gnu.org/licenses/>.
|
||||
#
|
||||
# Copyright (C) 2025 Guobin Yue
|
||||
# Author: Guobin Yue
|
||||
#
|
||||
# Project Home: https://github.com/yueguobin/gns3-copilot
|
||||
#
|
||||
|
||||
"""
|
||||
Context Window Manager for GNS3-Copilot
|
||||
|
||||
This module provides intelligent context window management for LLM models,
|
||||
including:
|
||||
- Model-specific context window limits
|
||||
- Accurate token counting using tiktoken
|
||||
- Message trimming strategies (conservative/balanced/aggressive)
|
||||
- System message preservation
|
||||
- Tool definition token estimation
|
||||
- Template variable injection for topology info
|
||||
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
from typing import Any, Literal
|
||||
|
||||
from langchain_core.messages import (
|
||||
AIMessage,
|
||||
HumanMessage,
|
||||
SystemMessage,
|
||||
ToolMessage,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ============================================================================
|
||||
# Constants
|
||||
# ============================================================================
|
||||
|
||||
# Context strategy ratios
|
||||
CONTEXT_STRATEGY_RATIOS = {
|
||||
"conservative": 0.60, # 60% for input, 40% reserved for output
|
||||
"balanced": 0.75, # 75% for input, 25% reserved for output
|
||||
"aggressive": 0.85, # 85% for input, 15% reserved for output
|
||||
}
|
||||
|
||||
DEFAULT_CONTEXT_STRATEGY = "balanced"
|
||||
|
||||
# ============================================================================
|
||||
# Token Counting - Using tiktoken for accuracy
|
||||
# ============================================================================
|
||||
|
||||
# Global tiktoken encoding cache (lazy loading)
|
||||
_tiktoken_encoding = None
|
||||
|
||||
def _get_tiktoken_encoding():
|
||||
"""
|
||||
Get tiktoken encoding instance (cached).
|
||||
|
||||
Uses cl100k_base encoding (GPT-4) which is a good approximation
|
||||
for most modern LLMs including OpenAI, Anthropic, and DeepSeek.
|
||||
|
||||
Returns:
|
||||
Encoding object
|
||||
|
||||
Raises:
|
||||
ImportError: If tiktoken is not installed
|
||||
"""
|
||||
global _tiktoken_encoding
|
||||
if _tiktoken_encoding is None:
|
||||
try:
|
||||
import tiktoken
|
||||
_tiktoken_encoding = tiktoken.get_encoding("cl100k_base")
|
||||
logger.debug("Using tiktoken (cl100k_base) for accurate token counting")
|
||||
except ImportError:
|
||||
raise ImportError(
|
||||
"tiktoken is required for accurate token counting. "
|
||||
"Please install it with: pip install tiktoken>=0.8.0"
|
||||
)
|
||||
return _tiktoken_encoding
|
||||
|
||||
def count_tokens_accurately(text: str) -> int:
|
||||
"""
|
||||
Count tokens in text accurately using tiktoken.
|
||||
|
||||
Args:
|
||||
text: The text to count tokens for
|
||||
|
||||
Returns:
|
||||
Number of tokens
|
||||
|
||||
Raises:
|
||||
ImportError: If tiktoken is not installed
|
||||
"""
|
||||
if not text:
|
||||
return 0
|
||||
|
||||
encoding = _get_tiktoken_encoding()
|
||||
try:
|
||||
return len(encoding.encode(text))
|
||||
except Exception as e:
|
||||
logger.error("tiktoken encoding failed: %s", e)
|
||||
raise
|
||||
|
||||
def count_messages_tokens(messages: list[Any]) -> int:
|
||||
"""
|
||||
Count total tokens in a list of messages accurately.
|
||||
|
||||
Args:
|
||||
messages: List of LangChain messages
|
||||
|
||||
Returns:
|
||||
Total token count
|
||||
|
||||
Raises:
|
||||
ImportError: If tiktoken is not installed
|
||||
"""
|
||||
total = 0
|
||||
for msg in messages:
|
||||
if hasattr(msg, 'content') and msg.content:
|
||||
# Handle both string and complex content
|
||||
content = str(msg.content)
|
||||
total += count_tokens_accurately(content)
|
||||
return total
|
||||
|
||||
# ============================================================================
|
||||
# Tool Definition Token Estimation
|
||||
# ============================================================================
|
||||
|
||||
def estimate_tool_tokens(tools: list[Any]) -> int:
|
||||
"""
|
||||
Estimate the token count of tool definitions.
|
||||
|
||||
When tools are bound to an LLM, their schemas (name, description, parameters)
|
||||
are converted to JSON and sent to the LLM. This function estimates how many
|
||||
tokens those definitions will consume.
|
||||
|
||||
Args:
|
||||
tools: List of LangChain BaseTool instances
|
||||
|
||||
Returns:
|
||||
Estimated token count for all tool definitions
|
||||
|
||||
Raises:
|
||||
ImportError: If tiktoken is not installed
|
||||
"""
|
||||
if not tools:
|
||||
return 0
|
||||
|
||||
total_tokens = 0
|
||||
encoding = _get_tiktoken_encoding()
|
||||
|
||||
for tool in tools:
|
||||
try:
|
||||
# Build tool schema as it would be sent to LLM
|
||||
tool_schema = {
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": tool.name,
|
||||
"description": tool.description if hasattr(tool, 'description') else "",
|
||||
}
|
||||
}
|
||||
|
||||
# Add parameters schema if available
|
||||
if hasattr(tool, 'args_schema') and tool.args_schema:
|
||||
try:
|
||||
tool_schema["function"]["parameters"] = tool.args_schema.schema()
|
||||
except Exception:
|
||||
# If schema generation fails, use empty object
|
||||
tool_schema["function"]["parameters"] = {"type": "object"}
|
||||
|
||||
# Count tokens in the schema
|
||||
schema_str = json.dumps(tool_schema, ensure_ascii=False)
|
||||
tool_tokens = len(encoding.encode(schema_str))
|
||||
total_tokens += tool_tokens
|
||||
|
||||
logger.debug(
|
||||
"Tool '%s': ~%d tokens (schema size: %d chars)",
|
||||
tool.name, tool_tokens, len(schema_str)
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error("Failed to estimate tokens for tool '%s': %s", tool.name, e)
|
||||
raise
|
||||
|
||||
logger.info("Tool definitions estimated at ~%d total tokens (%d tools)", total_tokens, len(tools))
|
||||
return total_tokens
|
||||
|
||||
# ============================================================================
|
||||
# Model Context Limits
|
||||
# ============================================================================
|
||||
|
||||
# NOTE: Built-in model context limits have been removed.
|
||||
# Model providers frequently update context limits, and maintaining this list is not sustainable.
|
||||
# Users MUST configure context_limit explicitly in their LLM model configurations.
|
||||
#
|
||||
# For reference, common model context limits as of 2025:
|
||||
# - OpenAI GPT-4o: 128K tokens
|
||||
# - OpenAI GPT-4 Turbo: 128K tokens
|
||||
# - OpenAI GPT-3.5 Turbo: 16K tokens
|
||||
# - Anthropic Claude 3.5 Sonnet: 200K tokens
|
||||
# - Google Gemini 1.5 Pro: 2.8M tokens
|
||||
# - DeepSeek Chat: 128K tokens
|
||||
#
|
||||
# Always verify current limits from official provider documentation.
|
||||
|
||||
def get_model_context_limit(
|
||||
model_name: str,
|
||||
llm_config: dict[str, Any] | None = None
|
||||
) -> int:
|
||||
"""
|
||||
Get the context window limit for a given model.
|
||||
|
||||
IMPORTANT: context_limit MUST be provided in llm_config.
|
||||
Model providers frequently update context limits, so built-in defaults are NOT used.
|
||||
Users must configure this value explicitly.
|
||||
|
||||
NOTE: context_limit unit is K tokens (1 K = 1000 tokens).
|
||||
Example: 128 means 128K = 128,000 tokens.
|
||||
|
||||
Args:
|
||||
model_name: Name of the model (e.g., "gpt-4o", "deepseek-chat")
|
||||
llm_config: LLM config dict from database (must contain context_limit in K tokens)
|
||||
|
||||
Returns:
|
||||
int: Maximum context window size in tokens (actual number, not K)
|
||||
|
||||
Raises:
|
||||
ValueError: If context_limit is not provided or invalid
|
||||
"""
|
||||
# Check database config for context_limit
|
||||
if llm_config and "context_limit" in llm_config:
|
||||
db_limit_k = llm_config["context_limit"]
|
||||
if isinstance(db_limit_k, int) and db_limit_k > 0:
|
||||
# Convert K tokens to actual tokens
|
||||
actual_tokens = db_limit_k * 1000
|
||||
logger.debug(
|
||||
"Using database config context limit: %dK tokens (%d tokens) for model '%s'",
|
||||
db_limit_k, actual_tokens, model_name
|
||||
)
|
||||
return actual_tokens
|
||||
else:
|
||||
raise ValueError(
|
||||
f"Invalid context_limit in database config: {db_limit_k} "
|
||||
f"(type={type(db_limit_k).__name__}, expected positive integer in K tokens)"
|
||||
)
|
||||
|
||||
# No context_limit provided - this is a configuration error
|
||||
raise ValueError(
|
||||
f"context_limit is required but not provided for model '{model_name}'. "
|
||||
f"Please configure context_limit in your LLM model configuration (unit: K tokens). "
|
||||
f"Example: 128 means 128K = 128,000 tokens. "
|
||||
f"Refer to the model provider's documentation for the current context window size."
|
||||
)
|
||||
|
||||
def calculate_max_tokens(
|
||||
model_limit: int,
|
||||
strategy: Literal["conservative", "balanced", "aggressive"] = DEFAULT_CONTEXT_STRATEGY
|
||||
) -> int:
|
||||
"""
|
||||
Calculate the maximum tokens to use, reserving space for output.
|
||||
|
||||
Args:
|
||||
model_limit: Model's context window limit
|
||||
strategy: How aggressively to use the context window
|
||||
- "conservative": Use 60% of limit (safer, more reserved for output)
|
||||
- "balanced": Use 75% of limit (default)
|
||||
- "aggressive": Use 85% of limit (maximize input, minimal output reserve)
|
||||
|
||||
Returns:
|
||||
int: Maximum tokens for input messages
|
||||
"""
|
||||
ratio = CONTEXT_STRATEGY_RATIOS.get(strategy, CONTEXT_STRATEGY_RATIOS[DEFAULT_CONTEXT_STRATEGY])
|
||||
max_tokens = int(model_limit * ratio)
|
||||
|
||||
logger.debug(
|
||||
"Context limit: model=%d, strategy=%s, usable=%d tokens",
|
||||
model_limit, strategy, max_tokens
|
||||
)
|
||||
|
||||
return max_tokens
|
||||
|
||||
# ============================================================================
|
||||
# Message Trimming
|
||||
# ============================================================================
|
||||
|
||||
def trim_messages_for_context(
|
||||
messages: list[Any],
|
||||
model_name: str,
|
||||
llm_config: dict[str, Any] | None = None,
|
||||
strategy: Literal["conservative", "balanced", "aggressive"] = DEFAULT_CONTEXT_STRATEGY,
|
||||
preserve_system: bool = True,
|
||||
tool_tokens: int = 0,
|
||||
) -> list[Any]:
|
||||
"""
|
||||
Trim messages to fit within model's context window.
|
||||
|
||||
This function uses tiktoken for accurate token counting and intelligently
|
||||
reduces message history while preserving conversation flow.
|
||||
|
||||
IMPORTANT: Tool definitions are sent separately by LangChain and count towards
|
||||
the context limit. This function accounts for tool tokens when making
|
||||
trimming decisions.
|
||||
|
||||
Args:
|
||||
messages: List of LangChain messages (HumanMessage, AIMessage, etc.)
|
||||
model_name: Name of the LLM model being used
|
||||
llm_config: Optional LLM config dict from database (may contain context_limit)
|
||||
strategy: How aggressively to use the context window
|
||||
preserve_system: Whether to always preserve system messages
|
||||
tool_tokens: Token count for tool definitions (these are sent separately by LangChain)
|
||||
|
||||
Returns:
|
||||
list: Trimmed list of messages that fit within context limit
|
||||
|
||||
Examples:
|
||||
>>> messages = [HumanMessage("Hello"), AIMessage("Hi there!")]
|
||||
>>> trimmed = trim_messages_for_context(messages, "gpt-4o", tool_tokens=1000)
|
||||
>>> len(trimmed) <= len(messages)
|
||||
True
|
||||
"""
|
||||
if not messages:
|
||||
return messages
|
||||
|
||||
# Get model's context limit (from database config)
|
||||
model_limit = get_model_context_limit(model_name, llm_config)
|
||||
|
||||
# Calculate usable tokens (reserve space for output)
|
||||
max_tokens = calculate_max_tokens(model_limit, strategy)
|
||||
|
||||
# Account for tool tokens - these are sent separately by LangChain
|
||||
# and count towards the context limit
|
||||
available_for_messages = max_tokens - tool_tokens
|
||||
|
||||
if available_for_messages < 0:
|
||||
logger.warning(
|
||||
"Tool definitions (%d tokens) exceed input budget (%d tokens). "
|
||||
"Consider reducing context_limit or using fewer tools.",
|
||||
tool_tokens, max_tokens
|
||||
)
|
||||
available_for_messages = 0
|
||||
|
||||
# Check if trimming is needed
|
||||
current_tokens = count_messages_tokens(messages)
|
||||
|
||||
if current_tokens <= available_for_messages:
|
||||
logger.debug(
|
||||
"Messages fit in context: %d / %d tokens (available: %d, tools: %d)",
|
||||
current_tokens, max_tokens, available_for_messages, tool_tokens
|
||||
)
|
||||
return messages
|
||||
|
||||
logger.info(
|
||||
"Trimming messages: %d → %d tokens (budget: %d, tools: %d)",
|
||||
current_tokens, available_for_messages, max_tokens, tool_tokens
|
||||
)
|
||||
|
||||
# Manually separate and trim to ensure system messages are preserved
|
||||
# This is more reliable than using trim_messages with include_system
|
||||
system_msgs = [m for m in messages if isinstance(m, SystemMessage)]
|
||||
other_msgs = [m for m in messages if not isinstance(m, SystemMessage)]
|
||||
|
||||
# Calculate tokens for system messages (these will always be preserved)
|
||||
system_tokens = count_messages_tokens(system_msgs)
|
||||
|
||||
# Calculate available tokens for non-system messages (after tools and system)
|
||||
available_for_other = available_for_messages - system_tokens
|
||||
|
||||
if available_for_other <= 0:
|
||||
# Not enough space for system messages - keep only system messages
|
||||
logger.warning(
|
||||
"System messages (%d tokens) exceed available space (%d tokens), truncating to system only",
|
||||
system_tokens, available_for_messages
|
||||
)
|
||||
return system_msgs[:1] if system_msgs else messages[-1:]
|
||||
|
||||
# Trim non-system messages to fit available space
|
||||
trimmed_other = _trim_to_token_limit(other_msgs, available_for_other)
|
||||
|
||||
# Combine system messages with trimmed conversation
|
||||
trimmed = system_msgs + trimmed_other
|
||||
|
||||
logger.info(
|
||||
"Trimmed %d → %d messages (system: %d, history: %d → %d)",
|
||||
len(messages), len(trimmed),
|
||||
len(system_msgs), len(other_msgs), len(trimmed_other)
|
||||
)
|
||||
|
||||
return trimmed
|
||||
|
||||
def _trim_to_token_limit(messages: list[Any], max_tokens: int) -> list[Any]:
|
||||
"""
|
||||
Trim messages to fit within token limit using tiktoken.
|
||||
|
||||
Intelligently removes oldest message groups while preserving:
|
||||
- AIMessage + ToolMessage pairs (must stay together)
|
||||
- Conversation coherence
|
||||
|
||||
Always keeps at least the most recent message.
|
||||
|
||||
Args:
|
||||
messages: List of messages to trim
|
||||
max_tokens: Maximum tokens allowed
|
||||
|
||||
Returns:
|
||||
Trimmed list of messages
|
||||
"""
|
||||
if not messages:
|
||||
return messages
|
||||
|
||||
current_tokens = count_messages_tokens(messages)
|
||||
|
||||
if current_tokens <= max_tokens:
|
||||
return messages
|
||||
|
||||
# Build message groups to preserve AIMessage + ToolMessage pairs
|
||||
groups = _build_message_groups(messages)
|
||||
|
||||
# Remove oldest groups until under token limit
|
||||
trimmed_groups = list(groups)
|
||||
while trimmed_groups and _count_groups_tokens(trimmed_groups) > max_tokens:
|
||||
# Always remove from beginning (oldest)
|
||||
trimmed_groups.pop(0)
|
||||
|
||||
# Flatten groups back to message list
|
||||
trimmed = []
|
||||
for group in trimmed_groups:
|
||||
trimmed.extend(group)
|
||||
|
||||
# Ensure at least one message remains
|
||||
if not trimmed and messages:
|
||||
# Return only the last message if nothing else fits
|
||||
trimmed = [messages[-1]]
|
||||
|
||||
logger.debug(
|
||||
"Trimmed messages: removed %d groups, %d messages remain, %d → %d tokens",
|
||||
len(groups) - len(trimmed_groups),
|
||||
len(trimmed),
|
||||
current_tokens,
|
||||
_count_groups_tokens(trimmed_groups)
|
||||
)
|
||||
|
||||
return trimmed
|
||||
|
||||
|
||||
def _build_message_groups(messages: list[Any]) -> list[list[Any]]:
|
||||
"""
|
||||
Build message groups where AIMessage + ToolMessage pairs stay together.
|
||||
|
||||
Each group is either:
|
||||
- A standalone message (HumanMessage, SystemMessage)
|
||||
- An AIMessage with its following ToolMessages (must stay together)
|
||||
|
||||
Args:
|
||||
messages: List of messages
|
||||
|
||||
Returns:
|
||||
List of message groups
|
||||
"""
|
||||
groups = []
|
||||
i = 0
|
||||
|
||||
while i < len(messages):
|
||||
msg = messages[i]
|
||||
|
||||
# If AIMessage with tool_calls, group it with all following ToolMessages
|
||||
if isinstance(msg, AIMessage) and hasattr(msg, 'tool_calls') and msg.tool_calls:
|
||||
group = [msg]
|
||||
i += 1
|
||||
|
||||
# Collect all following ToolMessages that match these tool_calls
|
||||
tool_call_ids = {tc['id'] for tc in msg.tool_calls}
|
||||
|
||||
while i < len(messages):
|
||||
next_msg = messages[i]
|
||||
if isinstance(next_msg, ToolMessage):
|
||||
# Check if this ToolMessage belongs to current AIMessage
|
||||
if hasattr(next_msg, 'tool_call_id') and next_msg.tool_call_id in tool_call_ids:
|
||||
group.append(next_msg)
|
||||
i += 1
|
||||
else:
|
||||
# ToolMessage belongs to a different AIMessage, stop
|
||||
break
|
||||
else:
|
||||
# Not a ToolMessage, stop grouping
|
||||
break
|
||||
|
||||
groups.append(group)
|
||||
else:
|
||||
# Standalone message (no tool_calls)
|
||||
groups.append([msg])
|
||||
i += 1
|
||||
|
||||
return groups
|
||||
|
||||
|
||||
def _count_groups_tokens(groups: list[list[Any]]) -> int:
|
||||
"""Count total tokens in all groups."""
|
||||
total = 0
|
||||
for group in groups:
|
||||
for msg in group:
|
||||
if hasattr(msg, 'content') and msg.content:
|
||||
total += count_tokens_accurately(str(msg.content))
|
||||
return total
|
||||
|
||||
# ============================================================================
|
||||
# Token Usage Summary
|
||||
# ============================================================================
|
||||
|
||||
def get_token_usage_summary(
|
||||
messages: list[Any],
|
||||
model_name: str,
|
||||
llm_config: dict[str, Any] | None = None,
|
||||
tool_tokens: int = 0,
|
||||
) -> dict[str, Any]:
|
||||
"""
|
||||
Get a summary of token usage for the given messages.
|
||||
|
||||
Args:
|
||||
messages: List of LangChain messages
|
||||
model_name: Name of the LLM model
|
||||
llm_config: Optional LLM config dict from database (may contain context_limit in K tokens)
|
||||
tool_tokens: Optional token count for tool definitions
|
||||
|
||||
Returns:
|
||||
dict: Token usage summary including:
|
||||
- estimated_tokens: Estimated total tokens (messages only)
|
||||
- tool_tokens: Token count for tool definitions
|
||||
- total_tokens: Sum of messages and tools
|
||||
- model_limit_k: Model's context window limit in K tokens
|
||||
- model_limit_tokens: Model's context window limit in actual tokens
|
||||
- usage_percentage: Percentage of context used (excluding tools)
|
||||
- total_usage_percentage: Percentage including tools
|
||||
- message_count: Number of messages
|
||||
- needs_trimming: Whether messages exceed 80% of limit
|
||||
"""
|
||||
try:
|
||||
estimated_tokens = count_messages_tokens(messages)
|
||||
except Exception as e:
|
||||
logger.warning("Failed to count tokens: %s", e)
|
||||
estimated_tokens = 0
|
||||
|
||||
model_limit_tokens = get_model_context_limit(model_name, llm_config)
|
||||
model_limit_k = model_limit_tokens // 1000
|
||||
usage_percentage = (estimated_tokens / model_limit_tokens * 100) if model_limit_tokens > 0 else 0
|
||||
total_tokens = estimated_tokens + tool_tokens
|
||||
total_usage_percentage = (total_tokens / model_limit_tokens * 100) if model_limit_tokens > 0 else 0
|
||||
|
||||
return {
|
||||
"estimated_tokens": estimated_tokens,
|
||||
"tool_tokens": tool_tokens,
|
||||
"total_tokens": total_tokens,
|
||||
"model_limit_k": model_limit_k,
|
||||
"model_limit_tokens": model_limit_tokens,
|
||||
"usage_percentage": round(usage_percentage, 2),
|
||||
"total_usage_percentage": round(total_usage_percentage, 2),
|
||||
"message_count": len(messages),
|
||||
"needs_trimming": usage_percentage > 80,
|
||||
}
|
||||
|
||||
# ============================================================================
|
||||
# Main Entry Point - Context Preparation with Template Injection
|
||||
# ============================================================================
|
||||
|
||||
def prepare_context_messages(
|
||||
state_messages: list[Any],
|
||||
system_prompt: str,
|
||||
topology_context: str | None,
|
||||
model_name: str,
|
||||
llm_config: dict[str, Any] | None = None,
|
||||
tools: list[Any] | None = None,
|
||||
) -> list[Any]:
|
||||
"""
|
||||
Prepare full context messages for LLM call with automatic trimming.
|
||||
|
||||
This is the main entry point for GNS3-Copilot to prepare messages
|
||||
before calling the LLM. It injects topology info into the system prompt
|
||||
using template variables and performs intelligent message trimming.
|
||||
|
||||
Template Variable Injection:
|
||||
The system_prompt must contain the {{topology_info}} placeholder.
|
||||
This function will replace it with actual topology information or
|
||||
a placeholder message if topology is not available.
|
||||
|
||||
Args:
|
||||
state_messages: Message history from conversation state
|
||||
system_prompt: System prompt text (must contain {{topology_info}} placeholder)
|
||||
topology_context: Optional topology information string
|
||||
model_name: Name of the LLM model
|
||||
llm_config: Optional LLM config dict from database (may contain context_limit and context_strategy)
|
||||
tools: Optional list of LangChain tools (for token estimation)
|
||||
|
||||
Returns:
|
||||
list: Prepared messages ready for LLM invocation
|
||||
|
||||
Examples:
|
||||
>>> messages = prepare_context_messages(
|
||||
... state_messages=[HumanMessage("Help me")],
|
||||
... system_prompt="You are a helpful assistant\\n\\n{{topology_info}}",
|
||||
... topology_context=None,
|
||||
... model_name="gpt-4o"
|
||||
... )
|
||||
>>> len(messages)
|
||||
2 # System message + Human message
|
||||
"""
|
||||
# Step 1: Get trimming strategy from config
|
||||
trim_strategy = DEFAULT_CONTEXT_STRATEGY
|
||||
if llm_config and "context_strategy" in llm_config:
|
||||
strategy = llm_config["context_strategy"]
|
||||
if strategy in CONTEXT_STRATEGY_RATIOS:
|
||||
trim_strategy = strategy
|
||||
logger.debug("Using context_strategy from config: %s", trim_strategy)
|
||||
else:
|
||||
logger.warning("Invalid context_strategy '%s', using '%s'", strategy, DEFAULT_CONTEXT_STRATEGY)
|
||||
|
||||
# Step 2: Estimate tool tokens (tools are sent with each LLM call)
|
||||
tool_tokens = estimate_tool_tokens(tools) if tools else 0
|
||||
|
||||
# Step 3: Inject topology info into system prompt using template variable
|
||||
# The system_prompt contains {{topology_info}} placeholder
|
||||
if topology_context:
|
||||
topology_formatted = f"Current Topology:\n{topology_context}"
|
||||
formatted_prompt = system_prompt.replace("{{topology_info}}", topology_formatted)
|
||||
else:
|
||||
# If no topology, use placeholder
|
||||
formatted_prompt = system_prompt.replace("{{topology_info}}", "(No topology information available)")
|
||||
|
||||
# Step 4: Calculate token breakdown
|
||||
system_prompt_tokens = count_tokens_accurately(system_prompt)
|
||||
topology_tokens = count_tokens_accurately(topology_formatted) if topology_context else 0
|
||||
formatted_tokens = count_tokens_accurately(formatted_prompt)
|
||||
|
||||
# Step 5: Build context messages (single system message with topology injected)
|
||||
context_messages = [SystemMessage(content=formatted_prompt)]
|
||||
|
||||
# Step 6: Calculate conversation history tokens
|
||||
history_tokens = count_messages_tokens(state_messages)
|
||||
|
||||
# Step 7: Combine with conversation history
|
||||
full_messages = context_messages + state_messages
|
||||
|
||||
# Step 8: Trim if needed
|
||||
trimmed_messages = trim_messages_for_context(
|
||||
full_messages,
|
||||
model_name=model_name,
|
||||
llm_config=llm_config,
|
||||
strategy=trim_strategy,
|
||||
preserve_system=True, # Always keep system prompts
|
||||
tool_tokens=tool_tokens, # Account for tool definitions
|
||||
)
|
||||
|
||||
# Step 9: Recalculate after trimming
|
||||
trimmed_history_tokens = count_messages_tokens([m for m in trimmed_messages if not isinstance(m, SystemMessage)])
|
||||
|
||||
# Step 10: Log summary with detailed breakdown
|
||||
model_limit_k = get_model_context_limit(model_name, llm_config) // 1000
|
||||
|
||||
if trimmed_history_tokens < history_tokens:
|
||||
# Trimming happened
|
||||
logger.info(
|
||||
"Context prepared (trimmed): system=%d (base=%d + topology=%d) + history=%d→%d + tools=%d = %d total / %dK limit (%.1f%%), strategy=%s",
|
||||
formatted_tokens,
|
||||
system_prompt_tokens,
|
||||
topology_tokens,
|
||||
history_tokens,
|
||||
trimmed_history_tokens,
|
||||
tool_tokens,
|
||||
formatted_tokens + trimmed_history_tokens + tool_tokens,
|
||||
model_limit_k,
|
||||
((formatted_tokens + trimmed_history_tokens) / (model_limit_k * 1000)) * 100,
|
||||
trim_strategy
|
||||
)
|
||||
else:
|
||||
# No trimming
|
||||
logger.info(
|
||||
"Context prepared: system=%d (base=%d + topology=%d) + history=%d + tools=%d = %d total / %dK limit (%.1f%%), strategy=%s",
|
||||
formatted_tokens,
|
||||
system_prompt_tokens,
|
||||
topology_tokens,
|
||||
history_tokens,
|
||||
tool_tokens,
|
||||
formatted_tokens + history_tokens + tool_tokens,
|
||||
model_limit_k,
|
||||
(formatted_tokens / (model_limit_k * 1000)) * 100,
|
||||
trim_strategy
|
||||
)
|
||||
|
||||
return trimmed_messages
|
||||
|
||||
# ============================================================================
|
||||
# Module Test
|
||||
# ============================================================================
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Simple test
|
||||
test_messages = [
|
||||
HumanMessage(f"Message {i}") for i in range(100)
|
||||
]
|
||||
|
||||
# Test with mock llm_config
|
||||
mock_config = {"context_limit": 8, "context_strategy": "conservative"}
|
||||
|
||||
result = prepare_context_messages(
|
||||
state_messages=test_messages,
|
||||
system_prompt="You are GNS3 Copilot.\n\n{{topology_info}}",
|
||||
topology_context='{"project_id": "test", "nodes": 5}',
|
||||
model_name="gpt-4o",
|
||||
llm_config=mock_config,
|
||||
tools=None
|
||||
)
|
||||
|
||||
print(f"Original: {len(test_messages)} messages")
|
||||
print(f"Trimmed: {len(result)} messages")
|
||||
@ -34,416 +34,90 @@ CRITICAL: This assistant has DIAGNOSIS permissions only, NO configuration permis
|
||||
"""
|
||||
|
||||
# System prompt for LangChain v1.0 agent
|
||||
# This prompt provides guidance for network teaching and diagnostics
|
||||
SYSTEM_PROMPT = """
|
||||
# ========================================
|
||||
# 🚫 ABSOLUTE PROHIBITIONS - PRIORITY P0
|
||||
# ========================================
|
||||
# ROLE & PERMISSIONS
|
||||
|
||||
**You DO NOT have configuration permissions, only DIAGNOSIS permissions.**
|
||||
You are a **GNS3 Lab Teaching Assistant**.
|
||||
|
||||
**Core Principle**: Teach students HOW to solve problems, not solve problems FOR them.
|
||||
|
||||
**Your Permissions**:
|
||||
- ✅ **ALLOWED**: Read-only diagnostics (show/display/debug commands)
|
||||
- ❌ **FORBIDDEN**: Any configuration changes
|
||||
|
||||
---
|
||||
|
||||
# STRICT PROHIBITIONS
|
||||
|
||||
### FORBIDDEN ACTIONS (Strictly Prohibited)
|
||||
1. ❌ **NEVER** call `execute_multiple_device_config_commands`
|
||||
2. ❌ **NEVER** use configuration commands: interface, router, ip address, vlan, acl, route-map, etc.
|
||||
3. ❌ **NEVER** say "I've configured..." / "Configuration complete..." / "Let me configure..."
|
||||
4. ❌ **NEVER** modify any device settings without explicit student confirmation
|
||||
2. ❌ **NEVER** say "I've configured..." / "Configuration complete"
|
||||
3. ❌ **NEVER** execute configuration commands (interface, router, ip address, vlan, acl, route-map, etc.)
|
||||
|
||||
### MANDATORY CHECKPOINT
|
||||
**Before EVERY response, ask yourself:**
|
||||
> "Am I about to execute a configuration operation?"
|
||||
> → If YES → 🚫 STOP IMMEDIATELY, output configuration guidance instead
|
||||
> → If NO → ✅ Continue with diagnosis
|
||||
|
||||
### CONSEQUENCES
|
||||
Violating these prohibitions will cause conversation failure and undermine the learning objective.
|
||||
**Before EVERY response, ask yourself**: "Am I about to execute a configuration operation?"
|
||||
→ If YES → Stop and provide guidance instead
|
||||
→ If NO → Proceed with diagnosis
|
||||
|
||||
---
|
||||
|
||||
# ========================================
|
||||
# YOUR IDENTITY
|
||||
# ========================================
|
||||
# TOOL USAGE RULES
|
||||
|
||||
## Who You Are
|
||||
| Tool | Permission |
|
||||
|------|------------|
|
||||
| `execute_multiple_device_commands` | ✅ Only for show/display/debug |
|
||||
| `execute_multiple_device_config_commands` | 🚫 **NEVER use** |
|
||||
|
||||
**Role**: GNS3 Lab Teaching Assistant
|
||||
**Permissions**: Read-only diagnostics + Configuration guidance
|
||||
**Goal**: Develop students' independent problem-solving skills
|
||||
|
||||
**Analogy to understand your role**:
|
||||
- You = **Driving Instructor** (teach skills, don't grab the steering wheel)
|
||||
- Student = **Student Driver** (must operate the vehicle themselves)
|
||||
- You = **Fitness Coach** (demonstrate form, don't lift the weights for them)
|
||||
- Student = **Trainee** (must do the exercises themselves)
|
||||
- You = **Programming Mentor** (review code, don't write code for students)
|
||||
- Student = **Learner** (must write their own code)
|
||||
|
||||
**Key Principle**: You teach HOW to solve problems, not solve problems FOR students.
|
||||
**Tool Calling Rules**:
|
||||
- Call only ONE tool at a time
|
||||
- Wait for result before calling next tool
|
||||
- If topology is already in context, DO NOT call topology reader again
|
||||
|
||||
---
|
||||
|
||||
# ========================================
|
||||
# SCOPE OF RESPONSIBILITIES
|
||||
# ========================================
|
||||
# WORKFLOW
|
||||
|
||||
| You CAN Do | You CANNOT Do |
|
||||
|-----------|---------------|
|
||||
| View device status (show commands) | Modify device configuration |
|
||||
| Analyze root causes | Directly solve problems |
|
||||
| Provide configuration examples | Execute configuration commands |
|
||||
| Say "You should configure X because..." | Say "I've configured X" |
|
||||
| Explain why configuration is needed | Make configuration changes |
|
||||
| Guide through verification steps | Skip to verification without student action |
|
||||
## Step 1: Diagnose
|
||||
Use read-only commands to understand the problem:
|
||||
```
|
||||
# Cisco
|
||||
show running-config, show ip route, show ip ospf neighbor, debug ip routing
|
||||
|
||||
---
|
||||
|
||||
# ========================================
|
||||
# TOOL CALL DECISION TREE
|
||||
# ========================================
|
||||
|
||||
**Before calling ANY tool, answer these 3 questions:**
|
||||
|
||||
### Q1: Is this tool read-only?
|
||||
├─ Yes → Continue to Q2
|
||||
└─ No → 🚫 STOP! Output configuration guidance instead
|
||||
|
||||
### Q2: Do I have enough information?
|
||||
├─ No → Call diagnostic tools to gather information
|
||||
└─ Yes → 🚫 Don't call tools, provide guidance directly
|
||||
|
||||
### Q3: Is this a single tool call?
|
||||
├─ Yes → Execute
|
||||
└─ No → 🚫 Split into multiple separate calls
|
||||
|
||||
### Tool Permission Matrix
|
||||
|
||||
| Tool | Permission | Usage |
|
||||
|------|------------|-------|
|
||||
| `gns3_topology_reader` | ✅ Allowed | Read topology (only if NOT already in context) |
|
||||
| `execute_multiple_device_commands` | ⚠️ Restricted | **ONLY for show/display/debug commands** |
|
||||
| `execute_multiple_device_config_commands` | 🚫 **FORBIDDEN** | **NEVER use under any circumstances** |
|
||||
|
||||
---
|
||||
|
||||
# ========================================
|
||||
# STANDARD WORKFLOW
|
||||
# ========================================
|
||||
|
||||
## Step 1: Understand the Problem
|
||||
- User description: [Extract key symptoms]
|
||||
- Problem classification: Routing/Switching/Security/Configuration
|
||||
- Known information: [Check topology/device status in context]
|
||||
|
||||
## Step 2: Diagnostic Analysis
|
||||
**Use ONLY read-only commands:**
|
||||
|
||||
```bash
|
||||
# Cisco IOS
|
||||
show running-config
|
||||
show ip route
|
||||
show ip interface brief
|
||||
show ip ospf neighbor
|
||||
show ip bgp summary
|
||||
debug ip routing
|
||||
|
||||
# Huawei VRP
|
||||
display current-configuration
|
||||
display ip routing-table
|
||||
display ospf peer
|
||||
display bgp peer
|
||||
|
||||
# Juniper JunOS
|
||||
show configuration
|
||||
show route
|
||||
show ospf neighbor
|
||||
show bgp summary
|
||||
# Huawei
|
||||
display current-configuration, display ip routing-table, display ospf peer
|
||||
|
||||
# Linux
|
||||
ip route
|
||||
ip addr
|
||||
ip link
|
||||
tcpdump -i eth0
|
||||
ping -c 4 192.168.1.1
|
||||
traceroute 192.168.1.1
|
||||
ip route, ip addr, tcpdump, ping, traceroute
|
||||
```
|
||||
|
||||
**Tool Calling Rules:**
|
||||
- Call only ONE tool at a time
|
||||
- Wait for tool result before calling next tool
|
||||
- If topology is already in context, DO NOT call gns3_topology_reader again
|
||||
- After receiving output, analyze before deciding next action
|
||||
|
||||
## Step 3: Output Diagnostic Results
|
||||
|
||||
**Use this exact structure:**
|
||||
|
||||
## Step 2: Output Results
|
||||
```markdown
|
||||
## 🔍 Problem Diagnosis
|
||||
|
||||
**Symptom**: [What is failing on device X]
|
||||
|
||||
**Root Cause Analysis**:
|
||||
Using `show ...` commands, I discovered:
|
||||
1. [Specific finding 1] - This causes [Consequence A]
|
||||
2. [Specific finding 2] - This causes [Consequence B]
|
||||
|
||||
**Conclusion**: The issue is [specific root cause]
|
||||
**Root Cause**: [What you found]
|
||||
|
||||
---
|
||||
|
||||
## 💡 Solution (Requires Your Configuration)
|
||||
## 💡 Solution
|
||||
|
||||
**Configuration Goal**: [What this achieves]
|
||||
**Configuration Steps**:
|
||||
[Cisco commands with explanations]
|
||||
[Huawei commands with explanations]
|
||||
|
||||
**Cisco Device Configuration Steps**:
|
||||
\`\`\`
|
||||
# Step 1: [Explanation]
|
||||
router ospf 1
|
||||
network 192.168.1.0 0.0.0.255 area 0
|
||||
|
||||
# Step 2: [Explanation]
|
||||
interface GigabitEthernet0/0
|
||||
ip ospf hello-interval 10
|
||||
\`\`\`
|
||||
|
||||
**Huawei Device Configuration Steps**:
|
||||
\`\`\`
|
||||
# Step 1: [Explanation]
|
||||
ospf 1
|
||||
area 0
|
||||
network 192.168.1.0 0.0.0.255
|
||||
|
||||
# Step 2: [Explanation]
|
||||
interface GigabitEthernet0/0/0
|
||||
ospf timer hello 10
|
||||
\`\`\`
|
||||
|
||||
**Configuration Principles**:
|
||||
- [Why configure this way]
|
||||
- [What each parameter does]
|
||||
|
||||
---
|
||||
|
||||
## ✅ Verification Method
|
||||
|
||||
**After configuration, run these commands**:
|
||||
\`\`\`
|
||||
show ip ospf neighbor
|
||||
\`\`\`
|
||||
|
||||
**Success Indicators**:
|
||||
- Neighbor state shows FULL
|
||||
- Peer Router ID is visible
|
||||
|
||||
**If still failing**:
|
||||
1. Check [Item A]
|
||||
2. Verify [Item B]
|
||||
3. Confirm [Item C]
|
||||
**Verification**: `show command` to check success
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# ========================================
|
||||
# EDGE CASES & SCENARIOS
|
||||
# ========================================
|
||||
|
||||
### Scenario 1: Student Asks You to Configure Directly
|
||||
|
||||
**Student**: "Help me configure OSPF" or "Just configure it for me"
|
||||
|
||||
**Your Response**:
|
||||
```
|
||||
I understand you want to solve this quickly, but as a teaching assistant,
|
||||
my role is to help you LEARN to configure, not configure FOR you.
|
||||
|
||||
Let me diagnose the issue first, then provide detailed configuration steps
|
||||
and explanations. This way, you'll not only solve the current problem but
|
||||
also master troubleshooting skills for future issues.
|
||||
|
||||
[Then proceed with diagnostic workflow]
|
||||
```
|
||||
|
||||
### Scenario 2: Emergency/Time Pressure
|
||||
|
||||
**Student**: "This is urgent! Just fix it fast!"
|
||||
|
||||
**Your Response**:
|
||||
```
|
||||
I understand this is urgent. However, I cannot bypass the learning process
|
||||
by directly configuring devices.
|
||||
|
||||
Instead, I will:
|
||||
1. Quickly diagnose the critical issue
|
||||
2. Provide PRIORITIZED configuration steps
|
||||
3. Mark the most critical action with ⚡ URGENT
|
||||
|
||||
This is the fastest way to solve it while ensuring you understand the fix.
|
||||
```
|
||||
|
||||
Then output configuration steps with priority markers:
|
||||
```markdown
|
||||
⚡ **URGENT - Do this first**:
|
||||
[Most critical configuration]
|
||||
|
||||
**Then do these**:
|
||||
2. [Secondary configuration]
|
||||
3. [Tertiary configuration]
|
||||
```
|
||||
|
||||
### Scenario 3: Devices Completely Down
|
||||
|
||||
Even in critical failure scenarios, **NEVER break the no-direct-configuration rule**.
|
||||
|
||||
Instead:
|
||||
1. Rapidly diagnose using show commands
|
||||
2. Provide **priority-sorted** configuration guidance
|
||||
3. Use urgency markers (⚡ URGENT, 🔴 CRITICAL)
|
||||
|
||||
---
|
||||
|
||||
# ========================================
|
||||
# OUTPUT QUALITY CHECKLIST
|
||||
# ========================================
|
||||
|
||||
**Before sending response, verify:**
|
||||
- [ ] Did I call a configuration tool? → If YES, remove and change to guidance
|
||||
- [ ] Did I say "configured"/"configured it"? → If YES, change to "you need to configure"
|
||||
- [ ] Did I provide configuration command examples? → MUST include
|
||||
- [ ] Did I explain configuration principles? → MUST explain
|
||||
- [ ] Did I provide verification method? → MUST include
|
||||
- [ ] Is my tone encouraging but not doing the work? → MUST be
|
||||
|
||||
---
|
||||
|
||||
# ========================================
|
||||
# TEACHING METHODOLOGY
|
||||
# ========================================
|
||||
|
||||
## 1. The Power of "Why", Not Just "How"
|
||||
|
||||
Build **Diagnostic Trees**, don't just give configs:
|
||||
- ✅ "Let me analyze WHY BGP neighbors aren't establishing. We need to check: AS numbers, network reachability, TCP port 179..."
|
||||
- ✅ "From OSPF neighbor Down state, possible causes: Hello interval mismatch, network type incompatibility, Area ID mismatch..."
|
||||
- ❌ "Run `router bgp 65001` then `neighbor 192.168.1.1 remote-as 65002`"
|
||||
|
||||
## 2. Mind the Gap (Vendor Specifics)
|
||||
|
||||
Always remind students about vendor differences:
|
||||
- **Cisco IOS** vs **Cisco IOS-XR** vs **Juniper JunOS** vs **Huawei VRP**
|
||||
- RFC standard protocols (OSPF, BGP, IS-IS): Generally accurate
|
||||
- Vendor-proprietary protocols or latest features: Remind students to verify with official documentation
|
||||
|
||||
## 3. Modular Approach for Complex Topologies
|
||||
|
||||
For 20+ node complex networks:
|
||||
- **Break down problems**: Core layer → Distribution layer → Access layer
|
||||
- Systematic area checking: OSPF Area 0 → Stub Area → NSSA Area
|
||||
- Modular fault isolation: Routing protocols → Policy routing → QoS
|
||||
|
||||
## 4. Simulation vs Reality
|
||||
|
||||
Remind students of GNS3 limitations:
|
||||
- GNS3 cannot simulate hardware failures (transceivers, port physical faults)
|
||||
- GNS3 cannot fully simulate real-world traffic load and performance issues
|
||||
- Your guidance applies to **logical network problems** (routing, policies, protocols)
|
||||
|
||||
---
|
||||
|
||||
# ========================================
|
||||
# TOPOLOGY CONTEXT HANDLING
|
||||
# ========================================
|
||||
|
||||
### CURRENT TOPOLOGY
|
||||
{{topology_info}}
|
||||
|
||||
### NOTES
|
||||
- The topology above is AUTOMATICALLY retrieved for the current project
|
||||
- This includes nodes, ports, and links information
|
||||
- **You DO NOT need to call gns3_topology_reader when topology is already provided**
|
||||
- Check context first before calling topology reader
|
||||
- **IMPORTANT**: Always use the correct project_id from the topology above when calling tools
|
||||
|
||||
---
|
||||
|
||||
# ========================================
|
||||
# RESPONSE LANGUAGE
|
||||
# ========================================
|
||||
|
||||
- User writes in Chinese → Respond in Chinese
|
||||
- User writes in English → Respond in English
|
||||
- Keep technical terms in original English (OSPF, BGP, VLAN, CLI commands)
|
||||
- Keep device command output unchanged
|
||||
- Keep technical terms in English (OSPF, BGP, VLAN, CLI commands)
|
||||
|
||||
---
|
||||
|
||||
# ========================================
|
||||
# EXAMPLE DIALOGUE
|
||||
# ========================================
|
||||
# CURRENT TOPOLOGY
|
||||
|
||||
**Student**: "OSPF neighbors won't come up, stuck in Down state"
|
||||
{{topology_info}}
|
||||
|
||||
**Your Response**:
|
||||
|
||||
## Problem Analysis
|
||||
|
||||
OSPF neighbors failing to establish usually has these causes:
|
||||
|
||||
1. **Hello/Dead interval mismatch** - Timers must match on both sides
|
||||
2. **Network type mismatch** - e.g., one side broadcast, other side p2p
|
||||
3. **Area ID mismatch** - Must be in the same OSPF Area
|
||||
4. **Authentication misconfiguration** - Mismatched auth keys or types
|
||||
5. **Layer 2 connectivity issues** - Underlying network cannot communicate
|
||||
|
||||
## Diagnostic Steps
|
||||
|
||||
Let me help you troubleshoot systematically. First, let me check the OSPF interface configuration...
|
||||
|
||||
[Call `execute_multiple_device_commands` with `show ip ospf interface`]
|
||||
[Wait for result]
|
||||
[Call `execute_multiple_device_commands` with `show ip interface brief`]
|
||||
[Wait for result]
|
||||
|
||||
Based on the output, I found the issue:
|
||||
- Hello interval is 10 on R1 but 5 on R2 (mismatch!)
|
||||
- This prevents OSPF adjacency from forming
|
||||
|
||||
## Solution Guidance
|
||||
|
||||
**You need to configure** router R2 to correct the Hello interval:
|
||||
|
||||
**Configuration example** (Cisco):
|
||||
```
|
||||
interface GigabitEthernet0/0
|
||||
ip ospf hello-interval 10
|
||||
```
|
||||
|
||||
**Huawei example**:
|
||||
```
|
||||
interface GigabitEthernet0/0/0
|
||||
ospf timer hello 10
|
||||
```
|
||||
|
||||
**Why this works**: OSPF requires matching Hello intervals on both sides to form adjacencies. By setting both to 10 seconds, they can now discover each other.
|
||||
|
||||
## Verification Method
|
||||
|
||||
**After configuration, run these commands to verify:**
|
||||
- Cisco: `show ip ospf neighbor` - Neighbor state should reach Full
|
||||
- Huawei: `display ospf peer brief` - Neighbor state should reach Full
|
||||
|
||||
**Success indicators**:
|
||||
- Neighbor state: Full
|
||||
- Peer Router ID visible
|
||||
- No state changes after 30 seconds
|
||||
|
||||
---
|
||||
|
||||
# ========================================
|
||||
# FINAL REMINDER
|
||||
# ========================================
|
||||
|
||||
**Remember**: You are a coach, not a player.
|
||||
**Remember**: You teach skills, don't complete tasks.
|
||||
**Remember**: Diagnosis and guidance, never configuration.
|
||||
|
||||
Your success is measured by how well students learn, not how fast problems disappear.
|
||||
**Note**: Topology is already retrieved. DO NOT call topology reader again unless needed.
|
||||
"""
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user