mirror of
https://github.com/GNS3/gns3-server.git
synced 2026-11-05 06:41:13 +02:00
feat(context-manager): improve context window management with tool token accounting
Enhanced the context window management system to properly account for tool definition tokens when trimming messages. The key changes include: - Updated `trim_messages_for_context` function to accept `tool_tokens` parameter - Modified token budget allocation logic to subtract tool tokens before message trimming - Added detailed documentation explaining the token budget distribution between messages and tool definitions - Implemented prioritized trimming strategy that preserves system messages and recent conversation history - Added boundary case handling for scenarios where system messages or tools exceed available budget The improvements ensure more accurate context window management by accounting for the ~1000-2000 tokens typically consumed by tool definitions that LangChain automatically includes in LLM requests.
This commit is contained in:
1 parent
f8aa653ef9
commit
7fcdb57615
6 files changed
+220
-66
No files matched your search
@@ -355,7 +355,7 @@ if __name__ == "__main__":
|
||||
print("=== Testing single node startup ===")
|
||||
test_input_single = json.dumps(
|
||||
{
|
||||
"project_id": "f32ebf3d-ef8c-4910-b0d6-566ed828cd24", # Replace with actual project UUID
|
||||
"project_id": "<PROJECT_UUID>", # Replace with actual project UUID
|
||||
"node_ids": [
|
||||
"fbeda109-9a74-4d8c-a749-cc3847911a90"
|
||||
], # Replace with actual node UUID
|
||||
@@ -369,7 +369,7 @@ if __name__ == "__main__":
|
||||
print("\n=== Testing multiple nodes startup ===")
|
||||
test_input_multiple = json.dumps(
|
||||
{
|
||||
"project_id": "f32ebf3d-ef8c-4910-b0d6-566ed828cd24", # Replace with actual project UUID
|
||||
"project_id": "<PROJECT_UUID>", # Replace with actual project UUID
|
||||
"node_ids": [
|
||||
"fbeda109-9a74-4d8c-a749-cc3847911a90", # Replace with actual node UUIDs
|
||||
"another-node-uuid-here",
|
||||
|
||||
Reference in new issue
Block a user