Files
gns3-server/docs/gns3-copilot/implemented/context-window-management.md
T
YueGuobin 6a4cbe39c1 refactor(copilot): reuse MCP tools via a bridge; extract shared device runners
The copilot and the MCP service maintained two parallel wrapper layers
(LangChain BaseTool classes vs @mcp.tool() functions) over the same
handlers in gns3_client/api_handlers.py. Descriptions and parameters
drifted apart (compute_id and link_type were missing on the copilot
side). This makes the MCP registration the single source of truth and
removes the duplicated wrappers.

tools_v2/mcp_bridge.py adapts the MCP server's tool registration into
LangChain tools: names, descriptions and argument schemas come from the
MCP tools. The pool is filtered to project-scoped tools
(MCP_ALLOWED_TOOLS) because the copilot runs inside one open project —
project management, compute/appliance/image administration and server
info tools stay out. The bridge carries the copilot's request JWT into
the MCP contextvars (username/token_version decoded from the token so
ticket-minting tools work) and runs the async MCP functions from the
copilot's sync tool node on either thread flavor.

Mode toolsets (teaching / lab_automation / troubleshooting_injection)
are now composed from adapted MCP tools plus a few copilot-native ones
that add value the MCP versions lack: link_create (port-name
resolution), wait_seconds, packet_analysis(+skills), and the topology
reader. Eight orphaned LangChain wrappers are deleted.

The device command tools had the dependency inverted (the MCP handlers
instantiated the copilot LangChain classes and round-tripped through
JSON strings). The Nornir/Netmiko classes moved verbatim (git mv) into
the shared layer as plain runners — DeviceConfigCommandRunner,
DeviceShowCommandRunner, VpcsCommandRunner — with _run(json_string)
becoming run(params, gns3_ctx). The optional Jinja2 template rendering
moved from the MCP handlers into the runners (and into
utils/device_configs.render_device_template), so both entry paths get
it; the MCP handlers are now pure delegation. This also drops the
json.dumps/json.loads detour the MCP path used to take.

test_tool_params consistency gate extended: a handler that forwards the
whole params dict to a callee is a wildcard (same rationale as the
params.items() case) — the per-key check cannot see through delegation.
2026-11-03 22:04:57 +08:00

8.8 KiB
Raw Blame History

This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.

LLM Context Window Management

Overview

GNS3 Copilot's context window management prevents LLM context overflow by trimming conversation history to fit within configured token limits. It uses tiktoken for accurate token counting, supports three context strategies (conservative/balanced/aggressive), and automatically injects GNS3 topology information into the system prompt before each LLM call.

Architecture

graph TD
    subgraph "context_manager.py"
        A[create_pre_model_hook] --> B[_inject_topology_into_system]
        A --> C[estimate_tool_tokens]
        A --> D[trim_messages via LangChain]
        E[count_tokens via tiktoken]
        E --> C
        E --> F[_count_tokens_for_message]
        F --> D
    end

    subgraph "gns3_copilot.py"
        G[llm_call node] --> H[Get topology via gns3_topology_reader]
        H --> A
        A --> I[model_with_tools.invoke]
    end

    style A fill:#4a9eff,color:#fff
    style D fill:#ff6b6b,color:#fff
    style E fill:#51cf66,color:#fff

Key Components

Function File Purpose
create_pre_model_hook() context_manager.py Factory that creates the preprocessing hook
_inject_topology_into_system() context_manager.py Injects topology into system prompt via {{topology_info}} placeholder
count_tokens() context_manager.py Accurate token counting using tiktoken (cl100k_base)
estimate_tool_tokens() context_manager.py Estimates token cost of tool definitions (JSON serialization)
_count_tokens_for_message() context_manager.py Token counter adapter for LangChain's trim_messages
llm_call() gns3_copilot.py StateGraph node that orchestrates the full LLM call pipeline

Token Counting: Uses tiktoken cl100k_base encoding (GPT-4 compatible, 95%+ accuracy for most modern LLMs).

tiktoken Cache: Encoding files are cached locally at <agent_package>/cache/tiktoken/ to avoid repeated downloads. This is configured before tiktoken import via TIKTOKEN_CACHE_DIR environment variable.

Required Dependency: tiktoken>=0.8.0 — if not installed, ModuleNotFoundError is raised at startup.

Trimming Process

flowchart TD
    A[llm_call node invoked] --> B[Get topology via gns3_topology_reader]
    B --> C[Create pre_model_hook]
    C --> D[Inject topology into system prompt]
    D --> E{topology data available?}
    E -->|Yes| F[Replace placeholder with topology content]
    E -->|No| G[Replace placeholder with 'No topology information available']
    F --> H[Estimate tool definition tokens]
    G --> H
    H --> I[Calculate trim_messages budget]
    I --> J[trim_messages with strategy='last']
    J --> K[Return prepared messages]
    K --> L[model.invoke with prepared messages]

Token Budget Calculation

flowchart LR
    A["context_limit<br/>(e.g. 128K)"] -->|"× strategy_ratio"| B["Input Budget<br/>(e.g. 96K @ balanced)"]
    B -->|"− tool_tokens"| C["trim_messages Budget<br/>(includes system message)"]
    C --> D["trim_messages<br/>max_tokens parameter"]

Key: max_tokens passed to trim_messages includes the system message. System tokens are NOT subtracted separately — trim_messages handles system message preservation via include_system=True.

Budget Calculation Example

Step Calculation Result
Context limit 128 (configured) 128K tokens
Strategy ratio balanced = 0.75 128 × 1000 × 0.75 = 96,000
Tool definitions estimated from JSON schema ~1,725 tokens
trim_messages budget 96,000 − 1,725 94,275 tokens (includes system)

Trimming Priority

Priority Content Behavior
1 System Message (system prompt + topology) Always kept (include_system=True)
2 Latest conversation messages Kept from newest to oldest
3 Older conversation history Discarded first when exceeding budget

Note: System prompt and topology are merged into a single SystemMessage via the {{topology_info}} template variable and cannot be trimmed independently.

Edge Case Handling

Scenario Behavior
System + tools > input budget ERROR log with recommendations (reduce prompt/tools, use larger model, switch to conservative). trim_messages is still called — include_system=True ensures system message is kept, but no history messages will fit.
Trim budget < system × 1.5 WARNING log: minimal room for conversation history
Invalid context_strategy Falls back to balanced with WARNING log
trim_messages throws exception Returns untrimmed messages (graceful degradation)
Missing {{topology_info}} placeholder Returns original messages without injection, WARNING log
Topology retrieval fails Injects "(No topology information available)" placeholder

Integration with GNS3 Copilot

File: gns3server/agent/gns3_copilot/agent/gns3_copilot.py

Why Direct Call, Not Config-based?

The agent uses a custom StateGraph (StateGraph(MessagesState)), not LangGraph's pre-built agent. LangGraph's pre_model_hook config parameter only works with pre-built agents like create_react_agent.

Agent Type pre_model_hook Support
create_react_agent Via config={"configurable": {"pre_model_hook": ...}}
chat_agent_executor Via config={"configurable": {"pre_model_hook": ...}}
Custom StateGraph Not supported — must call directly

Execution Flow

In the llm_call node of gns3_copilot.py:

  1. Get llm_config from request-scoped context variable
  2. Get project_id from config["configurable"]
  3. Retrieve topology via gns3_topology_reader._run(project_id)
  4. Select tools based on copilot_mode (teaching_assistant vs lab_automation_assistant)
  5. Load system prompt and create pre_model_hook
  6. Directly call pre_hook({"messages": messages, "topology_info": topology_info}) to prepare messages
  7. Invoke model_with_tools.invoke(prepared_messages)

Note

: The actual llm_call function also includes defensive checks for missing config, empty messages, and topology retrieval errors — see source code for full details.

Strategy Implementation

Context Strategy Ratios

Defined as CONTEXT_STRATEGY_RATIOS in context_manager.py, with default DEFAULT_CONTEXT_STRATEGY = "balanced".

Strategy Input Ratio Output Reserved Calculation
Conservative 60% 40% context_limit × 1000 × 0.60
Balanced 75% 25% context_limit × 1000 × 0.75
Aggressive 85% 15% context_limit × 1000 × 0.85

Log Output

Normal Case (topology injected)

INFO: Calling pre_hook to prepare 1 messages
INFO: ✓ Topology injected: 7722 chars, nodes: ['netshoot-1', 'R1', 'R2', 'IOU-L3-1', 'IOU-L3-2']
INFO: Context ready: 2 msgs, ~3815 tokens + 1725 tools = 5540 / 128K (4.3%), strategy=conservative
INFO: Messages prepared: 1 → 2
INFO: LLM call completed: tool_calls=0

When Trimming Occurs

INFO: Calling pre_hook to prepare 50 messages
INFO: ✓ Topology injected: 8500 chars, nodes: ['R1', 'R2', ...]
INFO: Messages trimmed: 50 → 25 msgs. Total: ~82000 tokens + 1725 tools = 83725 / 128K (65.4%), strategy=balanced
INFO: Messages prepared: 50 → 25

When Topology is None

INFO: Calling pre_hook to prepare 1 messages
WARNING: ✗ Topology data is None, injecting placeholder
INFO: Context ready: 2 msgs, ~800 tokens + 1725 tools = 2525 / 128K (2.0%), strategy=balanced

Error Handling

Error Behavior Resolution
tiktoken not installed ModuleNotFoundError at startup pip install tiktoken>=0.8.0
context_limit missing ValueError raised Add context_limit to LLM config
Invalid context_limit ValueError raised Must be positive integer
Invalid context_strategy WARNING, fallback to balanced Use conservative/balanced/aggressive
trim_messages failure ERROR log, returns untrimmed messages Check message format compatibility

Deprecated API

prepare_context_messages() is deprecated. It remains in context_manager.py for backward compatibility but emits DeprecationWarning. Use create_pre_model_hook() instead.

  • gns3server/agent/gns3_copilot/agent/context_manager.py — Context management core logic
  • gns3server/agent/gns3_copilot/agent/gns3_copilot.py — LLM call node (StateGraph)
  • gns3server/agent/gns3_copilot/agent/model_factory.py — Model creation and tool binding