- Refactor system message structure to combine system prompt and topology info using template variables
- Update token calculation process with merged system message approach
- Clarify priority order for message retention during context window management
- Add detailed token counting implementation using tiktoken library
- Include boundary case handling for system message exceeding budget
- Add SPDX license headers to tool files for proper licensing documentation
Enhanced the context window management system to properly account for tool definition tokens when trimming messages. The key changes include:
- Updated `trim_messages_for_context` function to accept `tool_tokens` parameter
- Modified token budget allocation logic to subtract tool tokens before message trimming
- Added detailed documentation explaining the token budget distribution between messages and tool definitions
- Implemented prioritized trimming strategy that preserves system messages and recent conversation history
- Added boundary case handling for scenarios where system messages or tools exceed available budget
The improvements ensure more accurate context window management by accounting for the ~1000-2000 tokens typically consumed by tool definitions that LangChain automatically includes in LLM requests.
- Add tiktoken as a required dependency for accurate token counting
- Update documentation with installation instructions and token counting strategy
- Improve logging to include tool definition token estimates
- Enhance error handling to fail fast when tiktoken is not available
- Update context manager to use tiktoken's cl100k_base encoding for GPT-4 compatibility
- Add `context_limit` as required field for LLM model configurations
- Add `context_strategy` as optional field with three trimming strategies
- Update API documentation with detailed examples for GPT-4o and Claude 3.5 Sonnet
- Clarify that context limit is specified in K tokens (thousands of tokens)
- Update example payloads to reflect current model versions and new fields
- Move AI chat API design document to gns3-copilot directory
- Add comprehensive HITL implementation plan with architecture and workflow
- Introduce human-in-the-loop confirmation for dangerous configuration commands
- Include state management, API endpoints, and frontend integration details
- Provide testing plan and deployment steps for the new feature
