7053 Commits

Author SHA1 Message Date
YueGuobin
74192f454f
Increase BATCH_MAX_WORKERS and Pool concurrency from 20 to 100 2026-06-16 00:49:08 +08:00
YueGuobin
286b2407cd
Remove final timing artifact in projects.py 2026-06-16 00:44:54 +08:00
YueGuobin
bd15c8d019
Remove remaining dead timing variables and imports 2026-06-16 00:40:03 +08:00
YueGuobin
a8cc01af46
Remove database warmup (proven ineffective - real bottleneck was bcrypt blocking event loop) 2026-06-16 00:35:42 +08:00
YueGuobin
ccb629f48f
Clean up all timing/debug logs
Remove all [MCP-TIMING] and [CTRL-TIMING] log lines, timing middleware,
and related import time statements across 11 files.
2026-06-16 00:34:11 +08:00
YueGuobin
8ef70d9fac
Add memory: import validation best practice 2026-06-16 00:25:45 +08:00
YueGuobin
7d8ab399da
Fix: add missing UUID imports 2026-06-16 00:24:37 +08:00
YueGuobin
d6b337bc3a
Generate fresh JWT on API key auth instead of returning raw key
When MCP client authenticates with API key, _resolve_token now
returns a newly generated JWT. This JWT is stored in the ContextVar
and used by all subsequent tool handler API calls - zero extra bcrypt.
2026-06-16 00:09:36 +08:00
YueGuobin
f522c947bc
Optimize API key auth: O(1) lookup via UUID-embedded key format
New format: gns3_<api_key_id>_<random_secret>
- Auth extracts api_key_id from token → single DB query by UUID → one bcrypt
- No more scanning all keys (was O(n) with bcrypt per key)
- bcrypt.checkpw offloaded to thread pool to prevent event loop blocking
- Legacy gns3_<random> format removed (compatibility break)
2026-06-15 23:56:36 +08:00
YueGuobin
796a2e6ca8
Fix: offload bcrypt.checkpw to thread pool to prevent blocking event loop
bcrypt.checkpw is CPU-bound (~1.3s per call) and was running
synchronously inside the async event loop, blocking ALL concurrent
requests. With 5 API keys and 10 concurrent requests, this caused
~13s delay before any handler could start.
2026-06-15 23:49:28 +08:00
YueGuobin
04f0657661
Add timing to API key auth path and log api_keys count 2026-06-15 23:45:26 +08:00
YueGuobin
3c41dea48f
Replace SELECT 1 warmup with full database file read to warm OS page cache
Reading the entire DB file into OS page cache eliminates the 8-14s
cold-start penalty on the first ORM query (users/templates/api_keys).
Previous SELECT 1 only warmed the connection pool, not the file cache.
2026-06-15 23:43:41 +08:00
YueGuobin
5502b8f8e5
Add timing logs to auth dependency chain to identify 6s pre-handler delay 2026-06-15 23:35:12 +08:00
YueGuobin
b3619a3c00
Add granular timing to get_template: separate execute vs fetch time 2026-06-15 23:33:19 +08:00
YueGuobin
e90fb0774c
Warm up database connection pool on startup to avoid 8s cold-start penalty on first API request 2026-06-15 23:26:28 +08:00
YueGuobin
6d3cfc7d1e
Fix: register WAL PRAGMA on sync_engine instead of Engine class for async compat
- Move event listener from module-level @Engine to engine.sync_engine
- Add WAL mode verification log on startup
- Remove unused Engine class-level listener
2026-06-15 23:21:50 +08:00
YueGuobin
08e0dbc122
Add timing logs to get_template to identify DB query bottleneck 2026-06-15 23:18:00 +08:00
YueGuobin
fa9aa5a9cc
Add timing middleware to log slow requests (>1s) with [CTRL-TIMING] prefix 2026-06-15 23:13:36 +08:00
YueGuobin
a855cb34f1
Fix: _time → time in compute.py timing log 2026-06-15 23:06:10 +08:00
YueGuobin
ea4bb2c1fb
Add [CTRL-TIMING] logs to controller create_node flow
Timing logs cover:
- create_node_from_template (entry, get_template, add_node, total)
- add_node_from_template (entry to done)
- _create_node (project_setup, node.create, total)
- Node.create (compute_post timing)
- compute._session.request (actual HTTP to compute)
2026-06-15 23:04:01 +08:00
YueGuobin
e5797c3da0
Fix: pass template_id to batch mode handler so top-level template_id works as default 2026-06-15 22:57:40 +08:00
YueGuobin
cef6dc6bd2
Add detailed timing logs to MCP node creation and HTTP client
Logs with [MCP-TIMING] prefix at:
- create_node_handler entry, setup, http_call start/end, total
- http_call entry, auth, response
- _authenticate_v3 entry, done, fail
2026-06-15 22:51:43 +08:00
YueGuobin
58ca6b1dd4
Add fields filter to template_list tool with description for AI 2026-06-15 22:11:14 +08:00
YueGuobin
b79fb89866
Fix: add missing _filter_link_response function
Previously the function was referenced in create_link_handler
but never defined, causing NameError at runtime.
2026-06-15 21:51:17 +08:00
YueGuobin
1bb010d274
Pass name parameter through to controller API when creating node from template 2026-06-15 21:50:14 +08:00
YueGuobin
eb3ac64016
Add validation to compact link format with clear error messages
- Validate array length (must be exactly 6)
- Validate node_id types at positions 0 and 3
- Return descriptive errors so AI can self-correct
2026-06-15 21:46:27 +08:00
YueGuobin
a8a83051ff
Add compact array format for link node entries to reduce token usage
Supports both standard and compact formats:
  Standard: [{"node_id": "uuid", "adapter_number": 0, "port_number": 0}]
  Compact:  ["uuid", 0, 0, "uuid", 0, 0] - 3x less tokens
2026-06-15 21:42:26 +08:00
YueGuobin
27a489a3fd
Add fields filter to link_create tool
- Default response: link_id, link_type, nodes (3 fields vs full 13)
- Available fields listed in tool description for AI
2026-06-15 13:50:04 +08:00
YueGuobin
3465c39eaa
Reduce md5sum cache write failure log level from error to warning
This is a non-fatal issue that occurs during concurrent project close
when multiple nodes try to write their md5sum cache while the project
directory is being cleaned up.
2026-06-15 13:27:14 +08:00
YueGuobin
11bf2d8cef
Add fields filter to node_create tool description for AI
- Expose optional 'fields' parameter to AI via tool description
- List all available node fields in the parameter description
- Mention inherited template_id in batch mode
2026-06-15 13:22:32 +08:00
YueGuobin
f4c8f9bf61
Optimize MCP create_node: support inherited template_id and default fields filter
- Batch mode now inherits top-level template_id as default per-node
- Both single and batch modes filter response to minimal fields by default
  (node_id, name, node_type, status, console)
- Optional 'fields' parameter overrides the default field set
2026-06-15 13:21:06 +08:00
YueGuobin
6cbe676b18
Increase MCP HTTP client timeout from 10s to 30s
Dynamips node creation (e.g., Cisco 7200 with multiple adapters)
can exceed the previous 10-second timeout, causing MCP tools to
fail with Read timed out errors.
2026-06-15 13:14:11 +08:00
YueGuobin
49d2c271ca
Enable SQLite WAL mode to fix 'database is locked' errors under concurrent API requests 2026-06-15 13:02:17 +08:00
YueGuobin
650be3565f
Cache IOU image default values per image path to avoid redundant subprocess calls
- Add class-level caches for loader and default RAM/NVRAM values
- When multiple IOU nodes use the same image, only the first one runs
  the ld-linux --verify and iou-image -h subprocesses
- Subsequent nodes reuse cached results, saving ~2 subprocess calls per node
- Cache is populated only on successful subprocess execution to avoid
  overwriting explicitly set ram/nvram values with stale cached defaults
2026-06-15 12:59:51 +08:00
YueGuobin
650f95af54
Increase node and link creation concurrency from 5 to 20 2026-06-15 12:50:59 +08:00
YueGuobin
e29e02a8c6
Fix: revert IOU lock optimization, serialize IOU node creation for correct application_id assignment
The _iou_id_lock must cover _create_node() because get_next_application_id()
checks in-memory nodes (self._nodes), which are only registered after
_create_node() completes. Without this serialization, concurrent IOU
node creation produces duplicate application IDs.
2026-06-15 12:49:01 +08:00
YueGuobin
5d0284e7be
Performance: accelerate project opening with parallel link creation and batch UDP port allocation
- Narrow IOU lock scope to only cover application_id allocation,
  allowing concurrent IOU node creation via Pool(concurrency=5)
- Parallelize link creation during project.open() using Pool(concurrency=5)
  instead of sequential processing
- Add batch UDP port allocation endpoint on compute to allocate N ports
  in a single HTTP call
- Pre-allocate UDP ports per compute before link creation during project
  loading, reducing HTTP round-trips
- UDPLink.create() falls back to individual port allocation if no
  pre-allocated port is available
2026-06-15 12:42:18 +08:00
Jeremy Grossmann
b605d71a8a
Merge pull request #2785 from yueguobin/mcp/batch-operations
MCP batch operations, field filtering, Jinja2 templates, download security, and WebSocket fixes
2026-06-15 00:47:17 +02:00
YueGuobin
97483310b1
docs: Update link tool descriptions for batch ops and fields filter 2026-06-14 22:51:06 +08:00
YueGuobin
be69670333
feat: Add batch link_ids to link_delete/link_reset, fields filter to link_list 2026-06-14 22:43:44 +08:00
YueGuobin
d6c362b3f0
feat: Add fields filter to link_list 2026-06-14 22:37:10 +08:00
YueGuobin
3b42eea112
feat: Add batch node_ids to node_delete 2026-06-14 22:28:28 +08:00
YueGuobin
e17d298bc7
fix: Convert http to ws scheme in node_console WebSocket URL 2026-06-14 21:57:58 +08:00
YueGuobin
80d5ae91a4
docs: Update link capture batch ops and console short-lived JWT note 2026-06-14 21:47:25 +08:00
YueGuobin
f14d30cb7e
feat: Add batch link_ids to link_capture_download 2026-06-14 21:36:24 +08:00
YueGuobin
6df9374a4c
feat: Add batch link_ids to link_capture_start/stop 2026-06-14 21:32:47 +08:00
YueGuobin
e02f1a8cd0
fix: Store username in gns3_ctx during auth, use for short-lived download JWTs
_ jw t_username_var set in _resolve_token for both JWT and API key auth.
Passed to handlers via gns3_ctx['jwt_username']. No raw key exposure,
no fake-user fallback.
2026-06-14 14:00:41 +08:00
YueGuobin
678b1868f5
fix: Generate independent short-lived JWT for pcap download
No longer depends on the original token type (JWT or API key).
Always creates a fresh 10-min JWT for the download URL.
2026-06-14 13:49:24 +08:00
YueGuobin
754eab9609
docs: Clarify why nodes must be started for device config tools 2026-06-14 13:41:04 +08:00
YueGuobin
9f3a6a7e58
docs: Remove Claude Desktop section from MCP doc 2026-06-14 13:28:04 +08:00