Commit Graph
196 Commits
Author SHA1 Message Date
YueGuobin 42e0a5dd5b fix(copilot): bound LLM requests with a timeout and one retry
A field log showed one request stayed silent for 604s (the OpenAI SDK
default timeout), then the SDK retried on a fresh connection and got
200 OK in under a second. The abort signal did not cancel the in-flight
HTTP call, so the turn could look hung for ten minutes.

- set timeout (default 120s, request_timeout in the LLM config wins),
  max_retries=1 and stream_usage=True for the providers that accept them
  (openai, deepseek, anthropic, xai). ollama and google-genai reject
  these kwargs, so they keep the old behavior.
- log the elapsed time of each LLM call. A slow first token or an SDK
  retry now shows up in the log line.
- add request_timeout to the LLM config schemas and docs (JSON column,
  no migration).
- tests: cover the defaults, the override, invalid values, provider
  filtering and the title model.
2026-11-03 23:57:07 +08:00
YueGuobin 5a294461e8 fix(copilot): make the step budget real, configurable and visible
The graph passed remaining_steps=20 (50 for injection) in the run inputs.
RemainingSteps is a LangGraph managed value derived from recursion_limit,
so the input was ignored and the real budget was 25 (100 for injection).
No one could see those numbers, and a turn that hit the cap ended on a
bare tool result with no explanation.

- Remove the dead remaining_steps input.
- Add resolve_recursion_limit(): max_steps from the LLM model config wins;
  otherwise 25 teaching_assistant, 50 lab_automation_assistant, 100
  troubleshooting_injection.
- When fewer than 4 steps remain after a tool run, route to the new
  budget_wrapup node. The next llm_call binds no tools and gets a wrap-up
  instruction, so the turn ends with a short summary of what was done and
  what remains.
- should_continue skips the title call after a wrap-up (no step left).
- budget_exhausted is reset at the start of every user turn.
- Add max_steps to the LLM config schemas. The config is a JSON column,
  so no migration is needed.
- Tests: 16 new tests cover the resolver, the routing, the tool-free
  wrap-up call and the removed input. Docs updated.
2026-11-03 23:16:26 +08:00
YueGuobin e8462507a5 feat(copilot): surface console, snapshot, capture and marker tools per mode
The bridge made 49 project-scoped MCP tools available, but the mode
tables still mirrored the old copilot capability set. Expand them:

- lab_automation_assistant: node_console_exec (raw console for nodes
  without a device_type tag), node_list, snapshot_list/create/delete/
  restore
- teaching_assistant: node_list/node_get, link_list, link_marker
  (read-only diagnostics plus traffic-insight markers)
- troubleshooting_injection: link_capture_start/stop/download and
  marker_definition (inject -> capture -> verify loop)

Also fix the tools_by_name registry: it was built from the lab and
injection tables only, which worked while teaching was a strict subset
of lab. Teaching-only tools would have raised KeyError in tool_node
when the LLM called them; the registry now unions all three mode
tables. A new test pins each mode's exact toolset, checks that no
administration tool leaks in, and asserts every referenced name is
resolvable.
2026-11-03 22:57:09 +08:00
YueGuobin 6a4cbe39c1 refactor(copilot): reuse MCP tools via a bridge; extract shared device runners
The copilot and the MCP service maintained two parallel wrapper layers
(LangChain BaseTool classes vs @mcp.tool() functions) over the same
handlers in gns3_client/api_handlers.py. Descriptions and parameters
drifted apart (compute_id and link_type were missing on the copilot
side). This makes the MCP registration the single source of truth and
removes the duplicated wrappers.

tools_v2/mcp_bridge.py adapts the MCP server's tool registration into
LangChain tools: names, descriptions and argument schemas come from the
MCP tools. The pool is filtered to project-scoped tools
(MCP_ALLOWED_TOOLS) because the copilot runs inside one open project —
project management, compute/appliance/image administration and server
info tools stay out. The bridge carries the copilot's request JWT into
the MCP contextvars (username/token_version decoded from the token so
ticket-minting tools work) and runs the async MCP functions from the
copilot's sync tool node on either thread flavor.

Mode toolsets (teaching / lab_automation / troubleshooting_injection)
are now composed from adapted MCP tools plus a few copilot-native ones
that add value the MCP versions lack: link_create (port-name
resolution), wait_seconds, packet_analysis(+skills), and the topology
reader. Eight orphaned LangChain wrappers are deleted.

The device command tools had the dependency inverted (the MCP handlers
instantiated the copilot LangChain classes and round-tripped through
JSON strings). The Nornir/Netmiko classes moved verbatim (git mv) into
the shared layer as plain runners — DeviceConfigCommandRunner,
DeviceShowCommandRunner, VpcsCommandRunner — with _run(json_string)
becoming run(params, gns3_ctx). The optional Jinja2 template rendering
moved from the MCP handlers into the runners (and into
utils/device_configs.render_device_template), so both entry paths get
it; the MCP handlers are now pure delegation. This also drops the
json.dumps/json.loads detour the MCP path used to take.

test_tool_params consistency gate extended: a handler that forwards the
whole params dict to a callee is a wildcard (same rationale as the
params.items() case) — the per-key check cannot see through delegation.
2026-11-03 22:04:57 +08:00
YueGuobin 9c27a293d0 refactor(copilot): align tool names with MCP convention
Rename five copilot tools to the MCP <object>_<verb> naming so both
agent surfaces expose the same names:

- create_gns3_node              -> node_create
- create_gns3_link              -> link_create
- create_gns3_text_drawing      -> drawing_create_text
- create_gns3_rectangle_drawing -> drawing_create_rectangle
- create_gns3_ellipse_drawing   -> drawing_create_ellipse

Also close two capability gaps found during the comparison:

- node_create: accept optional per-node compute_id (passed through to
  the shared handler; before, every node landed on local)
- link_create: accept optional per-link link_type (ethernet default,
  serial supported; before, serial links were impossible)

All tools keep calling the same shared handlers in
gns3_client/api_handlers.py, so the rename has no effect on execution
semantics. Old names appeared nowhere else (prompts, tests, mode tool
lists are clean); docs updated accordingly.
2026-11-03 15:30:44 +08:00
YueGuobin 7bf18ac2e8 feat(mcp): node_console_exec — one-shot console session via the host's websocat
Replaces node_console (which handed a URL + ticket to the client for a
bash websocat chain) with a tool that runs the whole session server-side:
spawn `timeout N websocat -t --no-close <console-ws-url>`, feed the
command lines through stdin (paced, CRLF), read stdout until the timeout
kills websocat, return the collected output as plain text.

- plain text replies: "output: ..." or bare error text — no JSON
  envelope, no status field (the server cannot judge command success)
- raw output: UTF-8 decode with replacement only, no telnet/ANSI
  stripping; control keys are real characters (description teaches
  \u0003, no \x03 translation)
- output funnel: grep=<python regex> filter; <=50 lines return as-is;
  beyond that a 10-line sample plus hints — regrep=true re-filters the
  node's cached capture (no re-run), full=true returns the last 400
  lines / 64 KiB, and the full capture is written to the node's files
  (console-capture.txt) with a ticketed curl download URL
- get_node_console_info_handler removed with its hashlib/TTL imports;
  the node-bound console ticket redemption path stays (comment updated,
  no current producer)
2026-11-02 22:52:46 +08:00
yueguobin 92feea2a51 test(vpcs): assert the pcname-derived prompt pattern
The window-size tests still asserted the old hardcoded PC\d+> default;
the driver now derives the prompt from the node name.
2026-11-01 23:09:38 +08:00
yueguobin efc097f0ad feat(mcp): injection_skill_get — expose fault-injection skills to MCP
The skills repository carries per-protocol fault-injection knowledge
(symptoms, injection commands, verification steps) that only the
copilot could read through InjectionSkillsTool. Add the MCP counterpart
with the same multi-call protocol:

  list (context REQUIRED) -> category index -> single fault

The list step is topology-filtered: context matches a skill's category,
key and declared protocols (case-insensitive), so aliases such as ebgp,
ospfv3 and stp select the right category instead of relying only on key
substrings. Fault bodies stay gated behind detail/issue requests.

Fault category keys are derived from repository filenames
(injection_<file>_issues). Correct the stale injection_ospf examples in
the registry docstrings, the loader comment and the copilot docs, which
pointed at keys that never existed.

Docs: 84 tools; new Fault Injection Skills section.
2026-11-01 23:09:38 +08:00
yueguobin 21fb84e91c fix(skills): load categories independently and support non-git checkouts
SkillsManager refused to start when the local skills directory was not a
git repository (cloning into a non-empty path fails), and reload_skills()
returned early when injection/ was missing, so device and feature skills
never loaded. reload_packet_analysis_protocols() had the opposite bug:
an empty result wiped the registry while still reporting success.

- initialize(): accept a non-empty directory without .git as a local-only
  source (skip automatic updates, keep loading files); degrade to local
  files when GitPython is unavailable.
- _update_if_needed(): re-create/clone when the local directory vanished
  or moved, instead of reusing a stale handle forever.
- reload_skills(): injection, device and feature load independently; a
  missing category keeps its previously loaded entries.
- reload_packet_analysis_protocols(): an empty/missing directory keeps
  the existing registry and reports failure.
- reload_skills_repository(): drop the console guard cache so
  config/console_guard.txt applies on reload.

MCP device_skill_get now exposes the category filter, documents the
single-file (topic-less) layout, and returns a sections list for
index/summary of skills that have no topics/issues. An uninitialized
skills repository fails with a clear error instead of an empty list.
2026-11-01 23:09:38 +08:00
yueguobin abb59b3d06 test(vpcs): mock read_channel in the strip-ansi test
The send_command read loop polls read_channel (netmiko decorates it
with a session-lock wrapper that only __init__ can satisfy), so the
old read_until_pattern mock died with AttributeError on instances
built via __new__.
2026-11-01 23:09:38 +08:00
yueguobin 68c8c0f1a0 fix(vpcs): wire pcname into the nornir inventory; keep partial output on read timeout
The pcname kwarg was added to VPCSTelnet but never passed by the only
production instantiation path, so a node renamed away from PC<n> never
matched the prompt pattern and every command stalled the full read
timeout. Pass pcname via the netmiko connection extras (the host key is
the node name).

send_command now owns its read loop instead of delegating to netmiko's
read_until_pattern: that helper raises ReadTimeout discarding accumulated
output, so a command slower than the timeout returned an empty string
that looked like success. The partial output is returned instead.
2026-11-01 23:09:38 +08:00
yueguobin c988ede19c docs: state the Linux 6.6 requirement of the netem seed filter
Aligns with uBridge's tc.md correction: TCA_NETEM_PRNG_SEED (uapi slot 14,
added in Linux 6.6 — not the nonexistent TCA_NETEM_SEED at "5.1") is what
the seed filter rides on; pre-6.6 kernels parse qdisc options
non-strictly and silently drop the attribute, so seed determinism is
unavailable there while every other netem knob applies. Stated in both
filter tables and the seed section of packet-filters.md, the seed rows of
the MCP and copilot tool descriptions, and the kernel-impairment spec.
2026-10-26 13:23:05 +08:00
Mark Paronyan b7e6c02647 refactor: ruff autofixes 2026-10-17 11:43:36 +03:00
YueGuobin f47cb56495 fix: dial per-node console hosts in MCP device config tools
The device tools (device_config_send / device_show_run / vpcs_config_set)
built their Nornir inventory with only the console port and inherited the
group hostname, which is always the controller host. Console ports of
nodes on remote computes are bound on the compute's own host, so those
nodes were unreachable from the MCP tools - only local-compute nodes
worked.

Pass the per-node console_host through the project inventory and inject
it as a host-level Nornir hostname (which overrides the group's
controller-wide default). Missing or wildcard (0.0.0.0/::) values keep
the group inheritance, so single-machine setups behave exactly as
before. VPCS builds its own host entries, so it carries the hostname
over explicitly.
2026-10-16 10:29:19 +08:00
YueGuobin 23f117de51 style: apply ruff format to drawing builder files 2026-10-10 14:34:28 +08:00
YueGuobin 5e8272fdd8 feat(agent): share Web UI compatible drawing builders between MCP and copilot
The text drawing tool was MCP-only; the copilot needs the same
capability and its area-annotation helper carried the same Web UI
compat bugs. Split into three layers with a single implementation:

- gns3server/utils/drawings.py: pure SVG builders (text, rectangle,
  ellipse) with full input validation (ValueError) — the Web UI
  format knowledge lives in core utils next to picture.py
- api_handlers.py: shared REST handlers (create_text_drawing_handler
  moved here, create_rectangle/ellipse_drawing_handler added) — the
  layer both consumers already share
- thin shells: MCP tools (drawing_create_text unchanged, plus new
  drawing_create_rectangle/_ellipse) and three LangChain tools
  (tools_v2/gns3_create_drawing.py) registered in both copilot mode
  lists

Shape builder details:

- rectangle: solid fill or "none", stroke block emitted only when a
  stroke is given, rx corner radius, dasharray validated against the
  Web UI's Qt remap table (QtDasharrayFixer rewrites those patterns)
- ellipse: bounding-box API; cx/cy derived as offsets inside the
  drawing box (the Web UI treats them as box-relative, not canvas
  coordinates); odd sizes round down and the actual size is returned

gns3_drawing_utils now delegates to the shared builders, fixing the
px-blind sizing, the ignored text-anchor/x/y centering and the
missing XML escaping in the area-annotation path.
2026-10-09 11:12:18 +08:00
YueGuobin c3c025ab2a fix: resolve the code-review findings on the kernel datapath
Compute:
- Track the anchor that owns uBridge's single AF_PACKET capture slot: a
  second kernel port's capture is refused with a clear error (its flag
  rolled back), a failed start no longer wedges the port, and only the
  owning port's stop issues the process-wide `capture stop_kernel`
  (docker/qemu/iou/dynamips/switch, via the mixin's slot helpers).
- Unwind failed starts instead of leaking uBridge and the anchor TAPs:
  QEMU's stop body runs under try/finally, QEMU and IOU start the datapath
  inside the launch try, Dynamips undoes a half-attached port (the retry
  used to be short-circuited into a silent black hole), and Docker deletes
  the veth pair when the namespace move fails.
- Restore the relay filter guard on IOU (kernel-only filters raise
  NodeError, an uncompilable BPF line degrades to a warning), validate the
  eBPF/quota/window/bpf filter shapes before touching the bridge (short or
  non-numeric values surfaced as 500s after partial state), and refuse a
  second NIO on an occupied Docker/QEMU port (the guard IOU and Dynamips
  already had).
- Probe fixes: `_probe_bridge_tap` survives a failing uBridge start, and
  the probe caches key on ctime too, so a chmod/setcap remedy re-probes.

Controller:
- Parse GNS3_UNIX_SOCKET_NIO like the compute does (line-anchored, truthy
  value) and mirror the compute's class selection in kernel_anchor_type.
- Read partial capability reports defensively; make BridgeNIO.bridge
  required-but-nullable so an omitted key is a 422 rather than a KeyError.
- Strip the packet_loss correlation slot even when it is zero: the relay
  filter takes exactly one argument and rejected `packet_loss 10 0`.

Ethernet switch: skip ports whose absorbed anchor does not exist yet (the
join is deferred by design) and roll the ports_mapping back when the VLAN
reconcile fails, so a retry still diffs against the old mapping.

Agent: reject device config entries without a device_name instead of
silently dropping them (a malformed batch used to execute partially while
reporting success).

Also split the netem builder to clear C901 and drop the two per-file ruff
ignores the PR had added, keeping the remove-only policy.
2026-10-06 19:55:21 +08:00
YueGuobin 7d04d08dba fix: repair imports lost in the 3.1 rebase and meet the new lint baseline
The rebase onto upstream 3.1 (mypy + ruff I001/F401 waves) left the tree
short of the newly enforced baselines in three ways:

* contextlib / UbridgeError imports dropped as false duplicates during
  conflict resolution left NameErrors in iou_vm and qemu_vm (F821);
  the Dynamips router grabbed the compute-level NIOTAP instead of the
  hypervisor one, breaking TAP anchoring (test failure).
* unused imports (KERNEL_UNSUPPORTED_FILTERS, UbridgeError, List) and
  unsorted blocks left behind where the stack's later commits moved the
  last usage elsewhere — ruff I001/F401/RUF022 now enforce these.
* mypy, now baseline-free: cache dict annotations in tc_probe, the
  locators candidate list, the resolved-params None branch in the MCP
  runner, and validate_bpf_syntax keeps the upstream Dict[str, Any]
  return (the strict Optional[str] was wrong for the "valid" bool entry).

The e2e raw-frame probe resolves its compiler via shutil.which (S607)
and zips with strict= (B905); the remaining subprocess call is S603 by
nature and gets a per-file ignore like other test files.

ruff check/format clean, mypy at upstream parity (no new errors),
full unit suite 2167 passed.
2026-10-06 00:21:11 +08:00
YueGuobin e13e5f3804 fix(tools): merge same-device entries before device command execution
A batch carrying the same device twice lost all but the last entry's
commands: the per-device command map kept only the last entry while
result processing iterated every original entry, so each row reported
the last entry's output under its own command list.

c3c78f99a consolidated the merge into the MCP template renderer, but
that only runs when a template is passed — the MCP direct-commands
path and gns3-copilot's direct _run path were left uncovered.
Normalize once in each tool's _run (single point covering all entry
paths, restoring the c42b3a59b coverage) and keep the renderer merge
as boundary normalization. Blocked-command info now also accumulates
across same-device entries instead of last-winning in the per-device
map.
2026-10-06 00:21:11 +08:00
YueGuobin 6633304e97 fix(mcp): let link_list return kernel_datapath
The tool descriptions have always advertised kernel_datapath as a
requestable field, but the handler's whitelist did not contain it: asking
for it was rejected as an invalid field, so the MCP clients could not see
which links run on kernel interfaces at all.

Also record what seeding a netem experiment on IOU costs: IOL emits its own
CDP/keepalive frames and they draw from the netem RNG stream, so a
seeded run only reproduces after `no cdp run` (verified bit-identical pcaps).
2026-10-06 00:21:09 +08:00
YueGuobin 1db9f171ca fix(controller): re-arm window_drop and expose kernel_datapath
Three fixes from live-calibrating the kernel impairment filters on an
isolated instance (Docker-to-Docker veth link, per-sequence evidence):

- update_filters skipped an unchanged filters dict as a no-op, so a
  one-shot window_drop outage that had already passed could never be
  re-armed: re-applying the same values produced no datapath command at
  all. A PUT still carrying window_drop now reconciles, restarting the
  schedule from the apply as documented. Measured: identical re-apply
  moves the outage window from seq 50-70 to 70-91 (arm at +8s of the
  new apply).

- gemodel parameter semantics were documented backwards in the
  validation rules, available_filters, the MCP/copilot descriptions and
  the feature docs. Measured against the kernel (per-direction counters,
  offered = sent + dropped): p is the good-to-bad transition chance, r
  the bad-to-good transition, 1-h the loss chance in the bad state (the
  good state loses nothing) - mean one-way loss p/(p+r)*(1-h), e.g.
  [10,30,100] -> 24.8%/25.7% measured vs 25% predicted, [5,45,100] ->
  10.8% both ways vs 10%.

- kernel_datapath was silently stripped from every REST/MCP link payload
  by the response model (LinkBase carried no such field), so clients
  could not tell a kernel link from a relay one.

Also verified on the same setup: the 3-argument window_drop form works
exactly as specified ([3000,5000,100] -> a single seq 31-55 outage,
[20000,6000,100] -> seq 115-144), and both generations of the command
builder emit the parameters unpadded.

Adds an API test for the field and controller tests for the no-op skip
and the window_drop re-arm exemption.
2026-10-06 00:21:08 +08:00
YueGuobin 935c00802f docs(mcp): refresh filter vocabulary in the MCP/copilot tool descriptions
link_update, link_available_filters and the copilot packet-filter tool
still advertised the pre-kernel-datapath five types. List the full
thirteen with positional parameter formats, ranges and kernel-only
marking, point at link_available_filters as the per-link authority
(topology + compute capability gated), and document window_drop's
apply-relative schedule restart. Also add kernel_datapath to the link
field lists.

Description-only: handlers pass filters through unchanged.
2026-10-06 00:21:08 +08:00
Mark Paronyan b622ccae7b refactor: declare explicit public API in package __init__ modules 2026-10-03 19:43:59 +03:00
Mark Paronyan 306d8fbde4 refactor: remove unused imports 2026-10-03 19:43:58 +03:00
Mark Paronyan 61caa20524 style: sort imports in agent and remaining compute modules 2026-10-02 12:43:26 +03:00
Mark Paronyan 06899c92fc refactor: use iterable unpacking instead of concatenation 2026-10-01 19:57:46 +03:00
Mark Paronyan c9416d2569 style: remove quotes from type annotations 2026-10-01 19:57:45 +03:00
Mark Paronyan 396407827e style: drop redundant open mode arguments 2026-10-01 19:57:44 +03:00
Mark Paronyan d356a68987 refactor: replace aliased OSError subclasses 2026-10-01 19:57:44 +03:00
Mark Paronyan 0416ad8228 style: sort __all__ declarations 2026-10-01 19:57:42 +03:00
Mark Paronyan c5b5091f5e refactor: use explicit conversion in f-strings 2026-10-01 19:57:41 +03:00
Mark Paronyan 9e5ab02147 style: remove unused noqa directives 2026-10-01 19:57:40 +03:00
Jeremy Grossmann 726da8383a Merge branch '3.1' into mypy-agent-gns3_copilot-plus 2026-09-29 13:20:27 +02:00
Jeremy Grossmann 468ad4e85f Merge branch '3.1' into mypy-agent 2026-09-29 13:19:30 +02:00
Jeremy Grossmann 4bcaf596df Merge pull request #2910 from markparonyan/mypy-agent-copilot-utils
fix(typing): resolve mypy errors in agent.gns3_copilot.utils
2026-09-29 12:48:03 +02:00
Jeremy Grossmann b73128d791 Merge branch '3.1' into mypy-agent-copilot-tools-analysis 2026-09-29 12:45:07 +02:00
Jeremy Grossmann 77a323e1e8 Merge branch '3.1' into mypy-agent-copilot-tools-nornir 2026-09-29 12:38:40 +02:00
Jeremy Grossmann 7b40064658 Merge branch '3.1' into mypy-agent-copilot-skills 2026-09-29 12:30:40 +02:00
Jeremy Grossmann 6aac6c8e34 Merge pull request #2906 from markparonyan/mypy-agent-copilot-client
fix(typing): resolve mypy errors in agent.gns3_copilot.gns3_client
2026-09-29 12:21:33 +02:00
Mark Paronyan 780b7f2ab6 fix(typing): resolve mypy errors in agent node lifecycle tools 2026-09-29 09:46:32 +03:00
Mark Paronyan bc27c4814f fix(typing): resolve mypy errors in agent.gns3_copilot.gns3_client.api_handlers 2026-09-29 09:46:31 +03:00
Mark Paronyan 9882eba088 fix(typing): resolve mypy errors in agent.gns3_copilot.chat_sessions_repository 2026-09-29 09:46:31 +03:00
Mark Paronyan 2d712e0a90 fix(typing): resolve mypy errors in agent.gns3_copilot.utils.custom_netmiko.scripts.list_netmiko_telnet_devices 2026-09-29 09:23:01 +03:00
Mark Paronyan 4ae6b5338d fix(typing): resolve mypy errors in agent.gns3_copilot.utils.parse_tool_content 2026-09-29 08:52:38 +03:00
Mark Paronyan 5cb1b8e3d2 fix(typing): resolve mypy errors in agent.gns3_copilot.utils.message_converters 2026-09-29 08:52:37 +03:00
Mark Paronyan cd7f5ab8e9 fix(typing): resolve mypy errors in agent.gns3_copilot.agent.model_factory 2026-09-29 08:52:33 +03:00
Mark Paronyan a251ca73ac fix(typing): resolve mypy errors in agent.gns3_copilot.agent.context_manager 2026-09-29 08:52:33 +03:00
Mark Paronyan f8144494f6 fix(typing): resolve mypy errors in agent.gns3_copilot.tools_v2.vpcs_tools_netmiko 2026-09-29 08:48:02 +03:00
Mark Paronyan d2fd238383 fix(typing): resolve mypy errors in agent.gns3_copilot.tools_v2.gns3_packet_filter 2026-09-29 08:48:02 +03:00
Mark Paronyan 67f68c9676 fix(typing): resolve mypy errors in agent.gns3_copilot.tools_v2.packet_analysis_tool 2026-09-29 08:48:02 +03:00