Commit Graph
5162 Commits
Author SHA1 Message Date
YueGuobin bd15c8d019 Remove remaining dead timing variables and imports 2026-06-16 00:40:03 +08:00
YueGuobin a8cc01af46 Remove database warmup (proven ineffective - real bottleneck was bcrypt blocking event loop) 2026-06-16 00:35:42 +08:00
YueGuobin ccb629f48f Clean up all timing/debug logs
Remove all [MCP-TIMING] and [CTRL-TIMING] log lines, timing middleware,
and related import time statements across 11 files.
2026-06-16 00:34:11 +08:00
YueGuobin 7d8ab399da Fix: add missing UUID imports 2026-06-16 00:24:37 +08:00
YueGuobin d6b337bc3a Generate fresh JWT on API key auth instead of returning raw key
When MCP client authenticates with API key, _resolve_token now
returns a newly generated JWT. This JWT is stored in the ContextVar
and used by all subsequent tool handler API calls - zero extra bcrypt.
2026-06-16 00:09:36 +08:00
YueGuobin f522c947bc Optimize API key auth: O(1) lookup via UUID-embedded key format
New format: gns3_<api_key_id>_<random_secret>
- Auth extracts api_key_id from token → single DB query by UUID → one bcrypt
- No more scanning all keys (was O(n) with bcrypt per key)
- bcrypt.checkpw offloaded to thread pool to prevent event loop blocking
- Legacy gns3_<random> format removed (compatibility break)
2026-06-15 23:56:36 +08:00
YueGuobin 796a2e6ca8 Fix: offload bcrypt.checkpw to thread pool to prevent blocking event loop
bcrypt.checkpw is CPU-bound (~1.3s per call) and was running
synchronously inside the async event loop, blocking ALL concurrent
requests. With 5 API keys and 10 concurrent requests, this caused
~13s delay before any handler could start.
2026-06-15 23:49:28 +08:00
YueGuobin 04f0657661 Add timing to API key auth path and log api_keys count 2026-06-15 23:45:26 +08:00
YueGuobin 3c41dea48f Replace SELECT 1 warmup with full database file read to warm OS page cache
Reading the entire DB file into OS page cache eliminates the 8-14s
cold-start penalty on the first ORM query (users/templates/api_keys).
Previous SELECT 1 only warmed the connection pool, not the file cache.
2026-06-15 23:43:41 +08:00
YueGuobin 5502b8f8e5 Add timing logs to auth dependency chain to identify 6s pre-handler delay 2026-06-15 23:35:12 +08:00
YueGuobin b3619a3c00 Add granular timing to get_template: separate execute vs fetch time 2026-06-15 23:33:19 +08:00
YueGuobin e90fb0774c Warm up database connection pool on startup to avoid 8s cold-start penalty on first API request 2026-06-15 23:26:28 +08:00
YueGuobin 6d3cfc7d1e Fix: register WAL PRAGMA on sync_engine instead of Engine class for async compat
- Move event listener from module-level @Engine to engine.sync_engine
- Add WAL mode verification log on startup
- Remove unused Engine class-level listener
2026-06-15 23:21:50 +08:00
YueGuobin 08e0dbc122 Add timing logs to get_template to identify DB query bottleneck 2026-06-15 23:18:00 +08:00
YueGuobin fa9aa5a9cc Add timing middleware to log slow requests (>1s) with [CTRL-TIMING] prefix 2026-06-15 23:13:36 +08:00
YueGuobin a855cb34f1 Fix: _time → time in compute.py timing log 2026-06-15 23:06:10 +08:00
YueGuobin ea4bb2c1fb Add [CTRL-TIMING] logs to controller create_node flow
Timing logs cover:
- create_node_from_template (entry, get_template, add_node, total)
- add_node_from_template (entry to done)
- _create_node (project_setup, node.create, total)
- Node.create (compute_post timing)
- compute._session.request (actual HTTP to compute)
2026-06-15 23:04:01 +08:00
YueGuobin e5797c3da0 Fix: pass template_id to batch mode handler so top-level template_id works as default 2026-06-15 22:57:40 +08:00
YueGuobin cef6dc6bd2 Add detailed timing logs to MCP node creation and HTTP client
Logs with [MCP-TIMING] prefix at:
- create_node_handler entry, setup, http_call start/end, total
- http_call entry, auth, response
- _authenticate_v3 entry, done, fail
2026-06-15 22:51:43 +08:00
YueGuobin 58ca6b1dd4 Add fields filter to template_list tool with description for AI 2026-06-15 22:11:14 +08:00
YueGuobin b79fb89866 Fix: add missing _filter_link_response function
Previously the function was referenced in create_link_handler
but never defined, causing NameError at runtime.
2026-06-15 21:51:17 +08:00
YueGuobin 1bb010d274 Pass name parameter through to controller API when creating node from template 2026-06-15 21:50:14 +08:00
YueGuobin eb3ac64016 Add validation to compact link format with clear error messages
- Validate array length (must be exactly 6)
- Validate node_id types at positions 0 and 3
- Return descriptive errors so AI can self-correct
2026-06-15 21:46:27 +08:00
YueGuobin a8a83051ff Add compact array format for link node entries to reduce token usage
Supports both standard and compact formats:
  Standard: [{"node_id": "uuid", "adapter_number": 0, "port_number": 0}]
  Compact:  ["uuid", 0, 0, "uuid", 0, 0] - 3x less tokens
2026-06-15 21:42:26 +08:00
YueGuobin 27a489a3fd Add fields filter to link_create tool
- Default response: link_id, link_type, nodes (3 fields vs full 13)
- Available fields listed in tool description for AI
2026-06-15 13:50:04 +08:00
YueGuobin 3465c39eaa Reduce md5sum cache write failure log level from error to warning
This is a non-fatal issue that occurs during concurrent project close
when multiple nodes try to write their md5sum cache while the project
directory is being cleaned up.
2026-06-15 13:27:14 +08:00
YueGuobin 11bf2d8cef Add fields filter to node_create tool description for AI
- Expose optional 'fields' parameter to AI via tool description
- List all available node fields in the parameter description
- Mention inherited template_id in batch mode
2026-06-15 13:22:32 +08:00
YueGuobin f4c8f9bf61 Optimize MCP create_node: support inherited template_id and default fields filter
- Batch mode now inherits top-level template_id as default per-node
- Both single and batch modes filter response to minimal fields by default
  (node_id, name, node_type, status, console)
- Optional 'fields' parameter overrides the default field set
2026-06-15 13:21:06 +08:00
YueGuobin 6cbe676b18 Increase MCP HTTP client timeout from 10s to 30s
Dynamips node creation (e.g., Cisco 7200 with multiple adapters)
can exceed the previous 10-second timeout, causing MCP tools to
fail with Read timed out errors.
2026-06-15 13:14:11 +08:00
YueGuobin 49d2c271ca Enable SQLite WAL mode to fix 'database is locked' errors under concurrent API requests 2026-06-15 13:02:17 +08:00
YueGuobin 650be3565f Cache IOU image default values per image path to avoid redundant subprocess calls
- Add class-level caches for loader and default RAM/NVRAM values
- When multiple IOU nodes use the same image, only the first one runs
  the ld-linux --verify and iou-image -h subprocesses
- Subsequent nodes reuse cached results, saving ~2 subprocess calls per node
- Cache is populated only on successful subprocess execution to avoid
  overwriting explicitly set ram/nvram values with stale cached defaults
2026-06-15 12:59:51 +08:00
YueGuobin 650f95af54 Increase node and link creation concurrency from 5 to 20 2026-06-15 12:50:59 +08:00
YueGuobin e29e02a8c6 Fix: revert IOU lock optimization, serialize IOU node creation for correct application_id assignment
The _iou_id_lock must cover _create_node() because get_next_application_id()
checks in-memory nodes (self._nodes), which are only registered after
_create_node() completes. Without this serialization, concurrent IOU
node creation produces duplicate application IDs.
2026-06-15 12:49:01 +08:00
YueGuobin 5d0284e7be Performance: accelerate project opening with parallel link creation and batch UDP port allocation
- Narrow IOU lock scope to only cover application_id allocation,
  allowing concurrent IOU node creation via Pool(concurrency=5)
- Parallelize link creation during project.open() using Pool(concurrency=5)
  instead of sequential processing
- Add batch UDP port allocation endpoint on compute to allocate N ports
  in a single HTTP call
- Pre-allocate UDP ports per compute before link creation during project
  loading, reducing HTTP round-trips
- UDPLink.create() falls back to individual port allocation if no
  pre-allocated port is available
2026-06-15 12:42:18 +08:00
YueGuobin be69670333 feat: Add batch link_ids to link_delete/link_reset, fields filter to link_list 2026-06-14 22:43:44 +08:00
YueGuobin d6c362b3f0 feat: Add fields filter to link_list 2026-06-14 22:37:10 +08:00
YueGuobin 3b42eea112 feat: Add batch node_ids to node_delete 2026-06-14 22:28:28 +08:00
YueGuobin e17d298bc7 fix: Convert http to ws scheme in node_console WebSocket URL 2026-06-14 21:57:58 +08:00
YueGuobin f14d30cb7e feat: Add batch link_ids to link_capture_download 2026-06-14 21:36:24 +08:00
YueGuobin 6df9374a4c feat: Add batch link_ids to link_capture_start/stop 2026-06-14 21:32:47 +08:00
YueGuobin e02f1a8cd0 fix: Store username in gns3_ctx during auth, use for short-lived download JWTs
_ jw t_username_var set in _resolve_token for both JWT and API key auth.
Passed to handlers via gns3_ctx['jwt_username']. No raw key exposure,
no fake-user fallback.
2026-06-14 14:00:41 +08:00
YueGuobin 678b1868f5 fix: Generate independent short-lived JWT for pcap download
No longer depends on the original token type (JWT or API key).
Always creates a fresh 10-min JWT for the download URL.
2026-06-14 13:49:24 +08:00
YueGuobin 647c5c0e65 docs: Add snapshot prerequisite and suppress telnetlib3 noise 2026-06-14 12:41:14 +08:00
YueGuobin c3c78f99a1 revert: Remove _configs_map changes in tools_v2 (handled by template renderer now) 2026-06-14 12:15:30 +08:00
YueGuobin c42b3a59bf fix: Merge commands for duplicate device_names in _configs_map
Dict comprehension overwrote earlier entries when the same device
appeared multiple times, causing all entries' outputs to collapse
into the last one. Now commands are appended for duplicate names.
2026-06-14 02:00:06 +08:00
YueGuobin fb032ade78 fix: Actually pass template param to device_config/command handlers
template was defined in the tool signature but omitted from
the params dict passed to the handler, making Jinja2 rendering
completely non-functional.
2026-06-14 01:40:48 +08:00
YueGuobin fed40c683e fix: Correct Jinja2 template commands_field per tool type
config_tools_nornir expects config_commands, while
display_tools_nornir and vpcs_tools_netmiko expect commands.
Render template now uses the correct field name.
2026-06-14 01:33:23 +08:00
YueGuobin 8aa7f2bde5 feat: Jinja2 template support in device_command_run 2026-06-14 00:27:49 +08:00
YueGuobin 06e1511773 feat: Jinja2 template support in device_config_send
- Add optional 'template' param with Jinja2 syntax
- Each device entry can use 'vars' dict instead of 'config_commands'
- Template rendered per device, merged with existing commands
- Rendering errors returned inline for AI self-correction
2026-06-14 00:27:13 +08:00
YueGuobin c647805cfb feat: Add batch node_ids support to node_start/stop/reload/suspend
- Each tool accepts either node_id (single) or node_ids (batch)
- Batch mode runs actions in parallel via ThreadPoolExecutor
- Useful for starting/stopping nodes by topology region
2026-06-14 00:14:34 +08:00