feat(context-manager): improve context window management with tool token accounting

Enhanced the context window management system to properly account for tool definition tokens when trimming messages. The key changes include:

- Updated `trim_messages_for_context` function to accept `tool_tokens` parameter
- Modified token budget allocation logic to subtract tool tokens before message trimming
- Added detailed documentation explaining the token budget distribution between messages and tool definitions
- Implemented prioritized trimming strategy that preserves system messages and recent conversation history
- Added boundary case handling for scenarios where system messages or tools exceed available budget

The improvements ensure more accurate context window management by accounting for the ~1000-2000 tokens typically consumed by tool definitions that LangChain automatically includes in LLM requests.
This commit is contained in:
YueGuobin committed 2026-03-05 01:50:48 +08:00
1 parent f8aa653ef9
commit 7fcdb57615
6 files changed
+220 -66

No files matched your search

@@ -355,7 +355,7 @@ if __name__ == "__main__":
print("=== Testing single node startup ===")
test_input_single = json.dumps(
{
"project_id": "f32ebf3d-ef8c-4910-b0d6-566ed828cd24", # Replace with actual project UUID
"project_id": "<PROJECT_UUID>", # Replace with actual project UUID
"node_ids": [
"fbeda109-9a74-4d8c-a749-cc3847911a90"
], # Replace with actual node UUID
@@ -369,7 +369,7 @@ if __name__ == "__main__":
print("\n=== Testing multiple nodes startup ===")
test_input_multiple = json.dumps(
{
"project_id": "f32ebf3d-ef8c-4910-b0d6-566ed828cd24", # Replace with actual project UUID
"project_id": "<PROJECT_UUID>", # Replace with actual project UUID
"node_ids": [
"fbeda109-9a74-4d8c-a749-cc3847911a90", # Replace with actual node UUIDs
"another-node-uuid-here",