Commit Graph
7057 Commits
Author SHA1 Message Date
YueGuobin ec58bea24b Fix template_list return type annotation to match _run_handler_sync envelope
template_list was annotated as dict[str, Any] but _run_handler_sync always
returns a list envelope ([{"type":"text",...}]). Other tools use
list[dict[str, Any]] consistently. This caused a Pydantic dict_type
validation error on the client.
2026-06-16 12:08:07 +08:00
YueGuobin a126f910d4 Add performance optimization documentation 2026-06-16 01:14:12 +08:00
YueGuobin 48b7ea8bf0 Update MCP service docs: API key format, auth flow, tool parameters, concurrency 2026-06-16 01:13:01 +08:00
YueGuobin d407b29fe9 Increase HTTP connection pool to 500/1000 2026-06-16 00:58:12 +08:00
YueGuobin 74192f454f Increase BATCH_MAX_WORKERS and Pool concurrency from 20 to 100 2026-06-16 00:49:08 +08:00
YueGuobin 286b2407cd Remove final timing artifact in projects.py 2026-06-16 00:44:54 +08:00
YueGuobin bd15c8d019 Remove remaining dead timing variables and imports 2026-06-16 00:40:03 +08:00
YueGuobin a8cc01af46 Remove database warmup (proven ineffective - real bottleneck was bcrypt blocking event loop) 2026-06-16 00:35:42 +08:00
YueGuobin ccb629f48f Clean up all timing/debug logs
Remove all [MCP-TIMING] and [CTRL-TIMING] log lines, timing middleware,
and related import time statements across 11 files.
2026-06-16 00:34:11 +08:00
YueGuobin 8ef70d9fac Add memory: import validation best practice 2026-06-16 00:25:45 +08:00
YueGuobin 7d8ab399da Fix: add missing UUID imports 2026-06-16 00:24:37 +08:00
YueGuobin d6b337bc3a Generate fresh JWT on API key auth instead of returning raw key
When MCP client authenticates with API key, _resolve_token now
returns a newly generated JWT. This JWT is stored in the ContextVar
and used by all subsequent tool handler API calls - zero extra bcrypt.
2026-06-16 00:09:36 +08:00
YueGuobin f522c947bc Optimize API key auth: O(1) lookup via UUID-embedded key format
New format: gns3_<api_key_id>_<random_secret>
- Auth extracts api_key_id from token → single DB query by UUID → one bcrypt
- No more scanning all keys (was O(n) with bcrypt per key)
- bcrypt.checkpw offloaded to thread pool to prevent event loop blocking
- Legacy gns3_<random> format removed (compatibility break)
2026-06-15 23:56:36 +08:00
YueGuobin 796a2e6ca8 Fix: offload bcrypt.checkpw to thread pool to prevent blocking event loop
bcrypt.checkpw is CPU-bound (~1.3s per call) and was running
synchronously inside the async event loop, blocking ALL concurrent
requests. With 5 API keys and 10 concurrent requests, this caused
~13s delay before any handler could start.
2026-06-15 23:49:28 +08:00
YueGuobin 04f0657661 Add timing to API key auth path and log api_keys count 2026-06-15 23:45:26 +08:00
YueGuobin 3c41dea48f Replace SELECT 1 warmup with full database file read to warm OS page cache
Reading the entire DB file into OS page cache eliminates the 8-14s
cold-start penalty on the first ORM query (users/templates/api_keys).
Previous SELECT 1 only warmed the connection pool, not the file cache.
2026-06-15 23:43:41 +08:00
YueGuobin 5502b8f8e5 Add timing logs to auth dependency chain to identify 6s pre-handler delay 2026-06-15 23:35:12 +08:00
YueGuobin b3619a3c00 Add granular timing to get_template: separate execute vs fetch time 2026-06-15 23:33:19 +08:00
YueGuobin e90fb0774c Warm up database connection pool on startup to avoid 8s cold-start penalty on first API request 2026-06-15 23:26:28 +08:00
YueGuobin 6d3cfc7d1e Fix: register WAL PRAGMA on sync_engine instead of Engine class for async compat
- Move event listener from module-level @Engine to engine.sync_engine
- Add WAL mode verification log on startup
- Remove unused Engine class-level listener
2026-06-15 23:21:50 +08:00
YueGuobin 08e0dbc122 Add timing logs to get_template to identify DB query bottleneck 2026-06-15 23:18:00 +08:00
YueGuobin fa9aa5a9cc Add timing middleware to log slow requests (>1s) with [CTRL-TIMING] prefix 2026-06-15 23:13:36 +08:00
YueGuobin a855cb34f1 Fix: _time → time in compute.py timing log 2026-06-15 23:06:10 +08:00
YueGuobin ea4bb2c1fb Add [CTRL-TIMING] logs to controller create_node flow
Timing logs cover:
- create_node_from_template (entry, get_template, add_node, total)
- add_node_from_template (entry to done)
- _create_node (project_setup, node.create, total)
- Node.create (compute_post timing)
- compute._session.request (actual HTTP to compute)
2026-06-15 23:04:01 +08:00
YueGuobin e5797c3da0 Fix: pass template_id to batch mode handler so top-level template_id works as default 2026-06-15 22:57:40 +08:00
YueGuobin cef6dc6bd2 Add detailed timing logs to MCP node creation and HTTP client
Logs with [MCP-TIMING] prefix at:
- create_node_handler entry, setup, http_call start/end, total
- http_call entry, auth, response
- _authenticate_v3 entry, done, fail
2026-06-15 22:51:43 +08:00
YueGuobin 58ca6b1dd4 Add fields filter to template_list tool with description for AI 2026-06-15 22:11:14 +08:00
YueGuobin b79fb89866 Fix: add missing _filter_link_response function
Previously the function was referenced in create_link_handler
but never defined, causing NameError at runtime.
2026-06-15 21:51:17 +08:00
YueGuobin 1bb010d274 Pass name parameter through to controller API when creating node from template 2026-06-15 21:50:14 +08:00
YueGuobin eb3ac64016 Add validation to compact link format with clear error messages
- Validate array length (must be exactly 6)
- Validate node_id types at positions 0 and 3
- Return descriptive errors so AI can self-correct
2026-06-15 21:46:27 +08:00
YueGuobin a8a83051ff Add compact array format for link node entries to reduce token usage
Supports both standard and compact formats:
  Standard: [{"node_id": "uuid", "adapter_number": 0, "port_number": 0}]
  Compact:  ["uuid", 0, 0, "uuid", 0, 0] - 3x less tokens
2026-06-15 21:42:26 +08:00
YueGuobin 27a489a3fd Add fields filter to link_create tool
- Default response: link_id, link_type, nodes (3 fields vs full 13)
- Available fields listed in tool description for AI
2026-06-15 13:50:04 +08:00
YueGuobin 3465c39eaa Reduce md5sum cache write failure log level from error to warning
This is a non-fatal issue that occurs during concurrent project close
when multiple nodes try to write their md5sum cache while the project
directory is being cleaned up.
2026-06-15 13:27:14 +08:00
YueGuobin 11bf2d8cef Add fields filter to node_create tool description for AI
- Expose optional 'fields' parameter to AI via tool description
- List all available node fields in the parameter description
- Mention inherited template_id in batch mode
2026-06-15 13:22:32 +08:00
YueGuobin f4c8f9bf61 Optimize MCP create_node: support inherited template_id and default fields filter
- Batch mode now inherits top-level template_id as default per-node
- Both single and batch modes filter response to minimal fields by default
  (node_id, name, node_type, status, console)
- Optional 'fields' parameter overrides the default field set
2026-06-15 13:21:06 +08:00
YueGuobin 6cbe676b18 Increase MCP HTTP client timeout from 10s to 30s
Dynamips node creation (e.g., Cisco 7200 with multiple adapters)
can exceed the previous 10-second timeout, causing MCP tools to
fail with Read timed out errors.
2026-06-15 13:14:11 +08:00
YueGuobin 49d2c271ca Enable SQLite WAL mode to fix 'database is locked' errors under concurrent API requests 2026-06-15 13:02:17 +08:00
YueGuobin 650be3565f Cache IOU image default values per image path to avoid redundant subprocess calls
- Add class-level caches for loader and default RAM/NVRAM values
- When multiple IOU nodes use the same image, only the first one runs
  the ld-linux --verify and iou-image -h subprocesses
- Subsequent nodes reuse cached results, saving ~2 subprocess calls per node
- Cache is populated only on successful subprocess execution to avoid
  overwriting explicitly set ram/nvram values with stale cached defaults
2026-06-15 12:59:51 +08:00
YueGuobin 650f95af54 Increase node and link creation concurrency from 5 to 20 2026-06-15 12:50:59 +08:00
YueGuobin e29e02a8c6 Fix: revert IOU lock optimization, serialize IOU node creation for correct application_id assignment
The _iou_id_lock must cover _create_node() because get_next_application_id()
checks in-memory nodes (self._nodes), which are only registered after
_create_node() completes. Without this serialization, concurrent IOU
node creation produces duplicate application IDs.
2026-06-15 12:49:01 +08:00
YueGuobin 5d0284e7be Performance: accelerate project opening with parallel link creation and batch UDP port allocation
- Narrow IOU lock scope to only cover application_id allocation,
  allowing concurrent IOU node creation via Pool(concurrency=5)
- Parallelize link creation during project.open() using Pool(concurrency=5)
  instead of sequential processing
- Add batch UDP port allocation endpoint on compute to allocate N ports
  in a single HTTP call
- Pre-allocate UDP ports per compute before link creation during project
  loading, reducing HTTP round-trips
- UDPLink.create() falls back to individual port allocation if no
  pre-allocated port is available
2026-06-15 12:42:18 +08:00
Jeremy Grossmann b605d71a8a Merge pull request #2785 from yueguobin/mcp/batch-operations
MCP batch operations, field filtering, Jinja2 templates, download security, and WebSocket fixes
2026-06-15 00:47:17 +02:00
YueGuobin 97483310b1 docs: Update link tool descriptions for batch ops and fields filter 2026-06-14 22:51:06 +08:00
YueGuobin be69670333 feat: Add batch link_ids to link_delete/link_reset, fields filter to link_list 2026-06-14 22:43:44 +08:00
YueGuobin d6c362b3f0 feat: Add fields filter to link_list 2026-06-14 22:37:10 +08:00
YueGuobin 3b42eea112 feat: Add batch node_ids to node_delete 2026-06-14 22:28:28 +08:00
YueGuobin e17d298bc7 fix: Convert http to ws scheme in node_console WebSocket URL 2026-06-14 21:57:58 +08:00
YueGuobin 80d5ae91a4 docs: Update link capture batch ops and console short-lived JWT note 2026-06-14 21:47:25 +08:00
YueGuobin f14d30cb7e feat: Add batch link_ids to link_capture_download 2026-06-14 21:36:24 +08:00
YueGuobin 6df9374a4c feat: Add batch link_ids to link_capture_start/stop 2026-06-14 21:32:47 +08:00