1661 Commits

Author SHA1 Message Date
YueGuobin
888afdccbd
fix: return the created template from appliance install
POST /appliances/{id}/install replied 204 with an empty body, so the MCP
appliance_install tool crashed with 'Expecting value: line 1 column 1'
while the template had actually been created. The route now returns the
created template (201, response_model=schemas.Template), _create_template
propagates it, and the MCP handler parses the body defensively so an
empty reply degrades to a plain success message.
2026-08-25 23:39:38 +08:00
YueGuobin
73e5e27c7b
fix: keep default node naming aligned with batch submission order
The controller assigns default names (R-1, R-2, ...) and console ports
in request arrival order. A parallel batch fan-out lets thread scheduling
decide that order, so the first submitted node could end up as R-2.
Batches that rely on default naming (any node without a name) are now
created sequentially; batches with explicit names stay parallel. The
node_create tool description documents the ordering semantics and tells
callers to correlate nodes by node_id.
2026-08-25 22:22:08 +08:00
YueGuobin
636abde16c
fix: keep node file content byte-faithful in node_file_get
Splitting the file with keepends=False and rejoining with newlines
dropped the trailing newline of the last line (and every \r of CRLF
files), so returned content was shorter than the file on disk and did
not round-trip. Split with keepends=True and join the selected lines
verbatim; pagination semantics are unchanged.
2026-08-25 21:33:21 +08:00
YueGuobin
57b5baed7f
fix: keep submission order and unify status in MCP batch handlers
Batch node/link creation collected results with as_completed, so the
response order followed completion rather than the submitted array and
callers could not correlate entries. Collect in submission order via
pool.map, and report batch deletes as status=success like every other
batch action (the message still says what was deleted).
2026-08-25 21:32:09 +08:00
YueGuobin
c9bc635996
fix: propagate 405 when suspending a node without suspend support
Suspending a single VPCS/IOU node returned a fake 204: the controller
route swallowed the compute 405 that the node types honestly raise, so
callers saw success while the node stayed started. Surface the 405
instead. The best-effort swallow on suspend_all is kept (and now covered
by a test) since mixed projects legitimately contain always-running node
types.
2026-08-25 21:29:53 +08:00
YueGuobin
97e7a79117
fix: report empty projects as not locked
GET /projects/{id}/locked returned True for a project with no drawings
or nodes: both loops ran zero times and the fallback return won. Locking
and unlocking such a project always succeeded while the state stayed
"locked", so it could never be unlocked.

Report a project with nothing to lock as not locked, and re-check the
state after unlock in the route tests.
2026-08-25 13:45:36 +08:00
YueGuobin
f31bfffefc
feat: expose data_link_type on the link_marker MCP tool (create-only)
Per-link markers on serial links need the WAN encapsulation (e.g.
DLT_C_HDLC) so the BPF compiles against the right link layer — the
REST API already accepts it, but the MCP tool never forwarded it.
Create passes it through; update ignores it (changing it would
invalidate the capture file), matching the REST schema semantics.
2026-08-25 13:29:39 +08:00
YueGuobin
702fc1f6d9
fix: never allow the projects directory to become a project directory
Loading a .gns3 placed directly in the projects root registered the
shared projects directory as the project path (load_project derives
the path from the file's parent directory). Deleting such an entry ran
rmtree on the projects directory itself, wiping every project until a
root-owned file stopped it, and left a zombie entry in the controller.

Three layers of protection:

- Controller.load_project() refuses a .gns3 whose parent directory is
  the projects directory; the normal subdirectory layout is unaffected
- the Project.path setter rejects the projects directory itself and
  its ancestors, closing the same hole for POST/PUT with an explicit
  path
- Project.delete() uses realpath + commonpath instead of commonprefix:
  entries whose path is the projects root are refused, and sibling
  directories sharing a string prefix (/srv/projects-evil vs
  /srv/projects) are no longer treated as inside the projects dir

Also removes the project_load MCP tool: loading by raw server
filesystem path is a footgun for automated clients; projects can still
be opened by project_id via the remaining tools.
2026-08-25 13:17:37 +08:00
YueGuobin
629bb9194f
refactor: sink shared REST handlers into gns3_client, drop gns3fy wrappers
- Slim custom_gns3fy.py to connector-only Gns3Connector and rename to
  connector.py; delete the unused Node/Link/Project dataclasses and
  endpoint wrapper methods (~3000 lines)
- Move the MCP node/link handler implementations into
  gns3_client/api_handlers.py as the shared REST client layer consumed
  by both the MCP service and copilot tools; add available_filters
  handler (exposed as link_available_filters MCP tool) and
  build_gns3_ctx() for copilot callers
- Rewrite the tools_v2 node/link tools on top of the handlers: batch
  lifecycle actions now run in parallel, node creation is a single
  POST, project-wide status reads replace per-node GETs
- Port Project.nodes_inventory/links_summary aggregation into
  project_inventory.py (output shape preserved) and rewrite the
  topology reader / project info tools on it, dropping the unused
  stats/snapshots/drawings calls
- Delete the dead mcp/nodes.py and mcp/links.py (NODE_TOOLS/LINK_TOOLS
  had no consumers; __init__ imports handlers from api_handlers)
- Retarget mcp handler tests to patch api_handlers._get_connector and
  replace test_custom_gns3fy.py with inventory contract tests
2026-08-25 09:15:32 +08:00
YueGuobin
1649fb7b7a
api: add endpoint serving the project .gns3 topology file
GET /v3/projects/{project_id}/gns3file returns the raw .gns3 file as
application/json so the WebUI can render topology thumbnails without
opening the project. Adds a public Project.topology_file property.
2026-08-24 22:19:09 +08:00
YueGuobin
e0962a59ed
feat: add GET/PUT /v3/settings server settings API
GET returns all settings sections except the deprecated
VirtualBox/VMware ones (Server.Audit privilege); secrets are
masked and Controller.jwt_secret_key is excluded entirely.
PUT applies a partial update (Server.Modify privilege): masked
or empty secrets mean unchanged, null removes the option, the
response carries the new values plus restart_required, and a
settings.updated notification is emitted. New
Server.Audit/Server.Modify privileges are seeded into the
Administrator role at table creation.
2026-08-23 18:54:31 +08:00
YueGuobin
7d3cbf1023
feat: add config read-modify-write update and harden file watcher
Config.update_config() applies submitted options to the main
configuration file via configparser read-modify-write: unknown
options are preserved, null removes an option, the merged view
of all files is validated as ServerConfig before anything is
written (a bad file would kill the FileWatcher polling loop),
and the write is atomic (.tmp + os.replace, mode 0600). Options
whose effective value is owned by a later configuration file
raise ConfigConflictError instead of writing a no-op. The
reload logic is factored into reload_and_notify() and the file
watcher callback is exception-guarded so polling never dies.
2026-08-23 18:54:23 +08:00
YueGuobin
d3ceb453a6
fix: forward auto_close in create_project_handler
The MCP project_create tool has passed auto_close=False since 8f8abe410
(2026-06-13), but create_project_handler only forwarded {"name": name}
to the REST API — auto_close was silently dropped and the controller's
Project.__init__ default (True, unchanged since 2016) won. Every project
created via MCP since June has auto_close=true on disk and closes when
the last client disconnects.

- projects.py: forward auto_close when present in params
- test_handlers.py: assert the forwarded json_data (with and without
  auto_close)
- test_tool_params.py: the tool/handler param consistency test never
  actually checked anything — three blind spots now fixed:
  1. dispatch is asyncio.to_thread(_run_handler_sync, ...) whose
     node.func is an Attribute, not a Name — no call ever matched
  2. tools that build 'params' as a variable before passing it were
     skipped; now the initial dict literal is resolved (extra-passed
     direction only)
  3. tool functions are async defs (ast.AsyncFunctionDef) but the
  enclosing-function lookup only matched ast.FunctionDef, so tool_name
  was always None
  Also: map the two marker handlers missing from HANDLER_FILES, skip
  handlers that forward params.items() generically (wildcard), union
  passed keys across multi-branch dispatches (node_create single/batch),
  and drop two dead helpers.

Verified: full suite 1568 passed; reverting the handler fix turns both
test_tool_handler_param_consistency and test_create red.
2026-08-23 08:57:39 +08:00
YueGuobin
641d10177a
refactor: move MCP service from api/routes to agent package
MCP is an optional AI feature that already depends on
agent.gns3_copilot (Gns3Connector, nornir/netmiko tools) and whose
MCP_AVAILABLE feature flag lives in gns3server/agent. Moving it there
collocates all AI features under one tree and removes AI code from the
core REST routes.

- git mv gns3server/api/routes/mcp -> gns3server/agent/mcp (no content changes)
- api/server.py, core/tasks.py: update import paths
- tests: tests/api/routes/mcp -> tests/agent/mcp, rewrite patch BASE and
  handler imports; fix MCP_DIR depth in test_tool_params.py
- agent/__init__.py: probe the SDK via importlib.import_module so the
  top-level name "mcp" is not bound in the agent namespace (it would
  shadow the new gns3server.agent.mcp subpackage and break
  'from gns3server.agent import mcp')
- docs: update source file paths

Verified: full suite 1567 passed; 82 MCP tools registered, SSE mounted
at /v3/mcp/transport.
2026-08-22 19:32:13 +08:00
YueGuobin
e98c51889e
fix: notification ping starved under sustained event load
NotificationQueue.get only generated a synthetic ping when the queue
was idle for the full timeout. Under sustained event load (e.g. a
project with markers matching at 15-260 events/s) the queue never
idled, so compute notification streams never carried a ping and the
controller stopped emitting compute.updated: clients lost compute
statistics until the event flow paused or the server restarted.

A ping is now guaranteed at least every timeout seconds regardless of
event flow: when the ping deadline is reached the next get() returns a
ping ahead of queued events (pings only carry statistics, so skipping
ahead of real events is harmless). Both the compute stream (compute
CPU/memory/disk stats -> compute.updated) and the controller stream
(idle keepalive) benefit.
2026-08-22 00:50:49 +08:00
YueGuobin
c709d74826
fix: compute notification stream silently died on uncaught exceptions
Two exception paths could permanently kill the compute notification
chain (no more compute.updated events, no reconnection until a server
restart):

- connect() only caught ComputeError, but _run_http_query translates
  HTTP status errors (401/403/404/...) into sibling ControllerError
  subclasses (and a raw fastapi HTTPException for unexpected statuses).
  Those escaped the fire-and-forget connect() task started at controller
  startup and died silently. Now they notify clients, schedule an
  exponential-backoff retry, and still re-raise for explicit callers.
  The dead web.HTTP* except branches (never reached since
  _run_http_query converts HTTP errors itself) are removed.

- _connect_notification() only caught aiohttp.ClientError. A malformed
  frame (e.g. missing 'action') or any error raised while dispatching a
  compute event (e.g. a pydantic ValidationError in
  node.parse_node_response) escaped the task, skipped the reconnect
  scheduling placed after the try block, and killed the stream forever.
  Now any exception is logged with its traceback (the gather() future
  holding it was never retrieved, so nothing was ever printed) and the
  reconnect scheduling + final compute.updated emit live in the finally
  block so every exit path recovers.

Also moves the usage-stats reset before the disconnect log line so the
emitted compute.updated snapshot is consistent.
2026-08-22 00:36:24 +08:00
YueGuobin
89d7f866cb
docker: replace vendor SKIP_INIT exec volume bridge with create-time direct binds
The SKIP_INIT volume bridge replicated init.sh's seed + mount --bind script
via docker exec *after* the container started. That copied the mechanism but
not the invariant that makes init.sh safe — the entrypoint position, which
guarantees the volume is in place before the application runs. The exec runs
concurrently with the NOS boot, so whether the NOS loaded its persisted
config or the overlay's factory copy was a timing race:

- single node stop/start on an idle system won it (exec ~1s, SR Linux reads
  its startup config at ~2-4s) — the save/stop/start round-trip passed;
- a server restart + project reload lost it (concurrent node starts queue on
  the Docker API, delaying the exec by seconds) — SR Linux booted factory
  while the persisted config.json sat intact on the host;
- XRd was immune (systemd boots tens of seconds before XR touches
  /xr-storage), which is why the race was never observed on it.

Replace the bridge entirely:

- new DockerVM._prepare_volumes hook (no-op in the base class) runs in
  create() after the image is present, before the container is created;
  VendorDockerVM overrides it to seed each volume's host directory from the
  image (throwaway docker create container + docker cp -a, nothing
  executes). The .gns3_perms marker gates the seeding: a volume that ever
  started is never re-seeded, so saved configuration is never overwritten
  with factory content (also the upgrade path for existing nodes).
- VendorDockerVM._mount_binds now binds the volumes directly at their real
  in-container paths (/etc/opt/srlinux) instead of /gns3volumes aliases, so
  the persisted config is visible to the NOS from the very first process.
- _setup_skip_init_volumes and its start() call are gone; the container-side
  _fix_permissions targets the volume paths directly (the direct binds
  exist for the whole container lifetime, unlike the old bridge).

The volume-list computation (validation + overlap de-duplication) moves
into DockerVM._persistent_volume_list so create-time seeding and _mount_binds
cannot drift apart.
2026-08-22 00:09:13 +08:00
YueGuobin
6aeb5dbda5
feat(copilot): device skills per-topic split layout and topic retrieval
SkillsLoader.load_device_skills() now supports two device layouts: the
existing single file and a split directory (device/<device>/_base.yaml +
one YAML per protocol topic). Topic files are merged into the device
skill under 'topics', keyed by their 'topic' field; mismatched
device_type, missing _base.yaml and duplicate topics are handled with
explicit log-and-skip.

get_skill() and DeviceSkillsTool gain a 'topic' parameter following the
injection list -> index -> issue pattern. Topic bodies are never
returned without an explicit topic request - index/summary/full all
serve a topic index instead - so growing a device with new protocol
topics no longer grows the token cost of device-level lookups.

Also fix reload_skills() to actually drop injection skills that fail
validate_skill_format() instead of logging 'skipping' and merging them
anyway.
2026-08-21 21:41:51 +08:00
YueGuobin
65e8eb9e28
docker: don't lose a client size that races the exec creation
A browser's terminal-size control frame (NAWS through the console telnet
server) can arrive while client_connected_hook is still creating the
exec; the resize is a no-op then, and the tall default applied after
creation would overwrite it, leaving the session at 511x10000 until the
user resizes.

Record sizes received before the exec exists and prefer them over the
tall default once creation finishes. The recorded size is cleared when
the last client disconnects, together with the restore-to-default.
2026-08-21 21:41:51 +08:00
YueGuobin
5741e85b65
docker: add GNS3_CONSOLE_RESIZE knob for paging CLIs
The exec behind a docker_exec console is shared by every console client,
so a browser's terminal-size resize (WS control frames -> NAWS) also
changes the geometry concurrent netmiko sessions see. SR Linux doesn't
care (no pager, no hard wrapping), but CLIs that page on the PTY window
size (IOS-XR) would park at --More-- again the moment a browser is
connected.

Split the client-driven NAWS path (_on_naws) from the internal resize
(_resize_exec): GNS3_CONSOLE_RESIZE=0 makes the console ignore client
resizes entirely and keep the tall 511x10000 no-paging default, while
the creation-time default and the restore-on-last-disconnect still go
through the internal path. XRd appliance templates should set it.
2026-08-21 21:41:51 +08:00
YueGuobin
5188ae625a
tests: pick a free console port instead of hardcoding 5011
A dev machine running a real gns3server (or qemu) can already listen on
5011; reserve_tcp_port then silently replaces it with the next free port
and test_console fails on the exact-echo assertion. Pick a port that is
actually free on the host first, like test_change_console_port does.
2026-08-21 21:41:51 +08:00
YueGuobin
72dc10a669
controller: allow markers and packet filters on Ethernet switch links
The brctl Ethernet switch runs a per-port uBridge relay, so it can host
the mark filter and packet filters like any uBridge-backed node. Add
ethernet_switch to _MARKER_CAPABLE_TYPES and _get_filter_node, narrow the
UDPLink.update() NIO-PUT skip down to the Dynamips-hosted ethernet_hub,
and expose the matching compute endpoints: PUT nio (filter/marker
reapply) plus the per-marker toggle/pause/resume/delete/rebuild routes.

The ethernet_hub keeps its exclusion: its routes still wire into the
Dynamips hub, which has no uBridge of its own.
2026-08-21 21:41:51 +08:00
YueGuobin
210103058f
docker: don't recreate containers on empty-string property PUTs
Web clients serialize empty form fields as "" while unset values are
stored as None on the node. The bare != diff in the update handler then
sees a phantom change on every full PUT and recreates the container for
nothing -- even when the user only changed a controller-only field such
as netmiko_device_type.

Normalize at the schema boundary ("" -> None for start_command,
environment and extra_hosts; "" -> "/" for console_http_path), make
the setters apply the same canonicalization, and create nodes through
the setters instead of bypassing them in __init__ so both paths store
identical values.
2026-08-21 21:41:47 +08:00
YueGuobin
200ccf0dfe
mcp: fingerprint short-lived console tokens for copy-corruption checks
node_console_info now returns token_sha256_prefix (sha256, first 8 hex
chars) and token_ttl_seconds alongside the console WebSocket URL, and
controller WebSocket auth rejections include the sha256 prefix of the
token as received. Comparing the two immediately distinguishes a token
corrupted in transfer from server-side rejection causes (expired,
revoked, bad signature).
2026-08-21 21:41:47 +08:00
YueGuobin
704b5d80c2
auth: split JWT validation errors by failure cause
get_token_data used to raise the same "Could not validate credentials"
for every JWT-level failure (bad signature, expired, malformed), which
made console WebSocket auth failures impossible to tell apart. Return a
distinct detail per cause and log the underlying exception plus the
unverified header alg value on rejection.
2026-08-21 21:41:47 +08:00
YueGuobin
fd7594f62e
docker: give the docker_exec console a tall default PTY geometry
The docker_exec console resized its exec PTY to 80x24 until a client
sent NAWS. CLIs that page on the PTY window size instead of the
terminal length (the IOS-XR pager) therefore parked long output at
--More-- for clients that never negotiate NAWS — netmiko, bare telnet —
making copilot device commands time out on XRd.

Default the exec to 511x10000 instead (511 matches netmiko's own
'terminal width 511' convention): no paging and no hard wrapping for
non-NAWS clients, while real NAWS clients keep resizing to their actual
geometry as before.

Also updates the project memory record with the confirmed root cause
and the fix.
2026-08-21 21:41:46 +08:00
YueGuobin
19f20e8d75
copilot: log into devices using the node default credentials
The vendored gns3fy Node model dropped default_username/default_password
from the API response, and the nornir groups hardcoded empty
credentials, so drivers that require authentication (gns3_ruijie_telnet,
stock netmiko SSH/telnet) could not log in.

Carry the per-node credentials through nodes_inventory() into the
nornir hosts data at host level, where they override the group's empty
fallback. Missing or cleared ("") values keep inheriting from the
group, so no-auth drivers are unaffected.
2026-08-21 21:41:42 +08:00
YueGuobin
38742ad4b6
tests: sign class-scoped client tokens with the default JWT secret
Since aef337e86 the config loader generates a random JWT secret even
without a main config file, so the class-scoped client/authorized_client
fixtures signed their tokens with a key that the autouse
run_around_tests fixture would immediately replace with the default
one for every test function. Every controller API test using those
fixtures was failing with 401 (BadSignature).

Sign both fixtures explicitly with DEFAULT_JWT_SECRET_KEY to match
the secret enforced at request time.
2026-08-21 21:41:42 +08:00
YueGuobin
7e8c515a3c
api: expose installed netmiko device types for the web UI
New GET /v3/netmiko/device_types endpoint returns the device types
supported by the netmiko library installed on the server, including
the gns3-copilot custom drivers, so the web UI can populate the
netmiko_device_type dropdown on templates and nodes.

The list is read at runtime from netmiko's ssh_dispatcher.CLASS_MAPPER
registry (the same table ConnectHandler dispatches on), filtered to
drop the '_ssh' aliases and the 'autodetect' pseudo type, and cached
for the process lifetime. Returns 501 when netmiko is not installed
(ai-features extra).
2026-08-21 21:41:42 +08:00
YueGuobin
92d4e3d378
nodes: per-node default credentials seeded from the template
Add default_username/default_password as controller-only node properties
(the netmiko_device_type pattern): they are not sent to the compute,
persist with the project topology and can be updated or cleared per
node. Creating a node from a template seeds them from the template
appliance metadata, and the metadata itself is dropped there so it
never leaks into the node properties.
2026-08-21 21:41:42 +08:00
YueGuobin
300c53e6fb
templates: persist appliance metadata on install
Appliance fields that describe the appliance (vendor information, default
credentials, installation instructions...) were dropped when installing a
template. Keep them in a new appliance_metadata JSON column on the
templates table, filled by the appliance-to-template conversion for both
registry v1-6 and v8 (version level values override the appliance level
ones). The nested schema allows extra fields so future registry fields
persist without a migration.
2026-08-21 21:41:42 +08:00
YueGuobin
1c68a52856
fix: address appliance v8 install review findings
- install: resolve the image directory from the version's settings type and
  skip image handling for docker appliances; guard appliance.images
- appliance schema: validate template_properties against template_type,
  align cpu_throttling with the qemu template, add kvm and version idlepc
- conversion: map IOU image to path, kvm disable to accel=tcg, inherit only
  same-type default settings, symbol fallback from the effective category,
  template_properties cannot override structural fields
- allow clearing netmiko_device_type with an empty string
- download the template symbol regardless of the level it is defined at and
  give qemu guests a default symbol
2026-08-21 21:41:42 +08:00
YueGuobin
435aa4c257
copilot: prefer netmiko_device_type over the device_type:<type> tag
The vendored gns3fy Node model and its nodes_inventory() now carry the
node's netmiko_device_type field, and get_device_ports_from_topology()
resolves the Netmiko device type from it first, falling back to the
device_type:<type> tag. Nodes created from a template inherit the value
from the template automatically, so automation tooling gets the correct
Netmiko driver without tags.
2026-08-21 21:41:35 +08:00
YueGuobin
f524e9a713
appliance: seed netmiko_device_type from the appliance file
Both the v1-6 and v8 appliance models accept an optional top-level
netmiko_device_type, and ApplianceToTemplate copies it into the created
template so installed appliances carry the automation hint end to end.
2026-08-21 21:41:35 +08:00
YueGuobin
d64f47afa4
nodes: per-node netmiko_device_type (controller-only)
netmiko_device_type follows the CONTROLLER_ONLY_PROPERTIES pattern
(like console_auto_start): a node created from a template inherits the
template value, PUT /nodes can override it inside a topology, updates
never round-trip to the compute, and the value persists in the project
topology file.
2026-08-21 21:41:35 +08:00
YueGuobin
9260108f53
templates: add netmiko_device_type for automation tools
Common template field (schema + templates table column + Alembic
migration) holding the Netmiko device type (e.g. 'cisco_xr', 'nokia_srl')
so Netmiko/Nornir based tooling can look up how to reach a node's CLI
without hard-coded vendor mappings. Free-form lowercase string on
purpose: Netmiko's platform list evolves independently of GNS3.
2026-08-21 21:41:35 +08:00
YueGuobin
254721dbda
appliance: close the v8 gaps for Docker vendor appliances
DockerPropertiesV8 now accepts custom_adapters (already available to
v1-6 top-level appliances and to Qemu v8 properties), so port-named
Docker appliances (XRd, SR Linux) can move to the v8 format without
losing their interface naming.

Appliance.type resolves the node type from the v8 settings template_type
(default set first) instead of misclassifying every v8 appliance as
qemu, and _get_default_symbol applies the docker guest symbol to v8
Docker guest appliances.
2026-08-21 21:41:35 +08:00
YueGuobin
57fe549773
appliance: implement install support for registry version 8
new_template() now converts the v8 settings[] format per the spec in
gns3-registry#734: settings selection (version name reference, then the
default set, then a single set), inherit_default_properties merging, and
template_properties expansion with category/usage/symbol resolved from
template_properties > version > appliance levels. Undefined properties
are left out so controller template defaults apply.

Registry versions 1-6 keep the existing top-level emulator block path.
2026-08-21 21:41:35 +08:00
YueGuobin
f43a717b20
copilot: sync CONSOLE_TYPES with the server ConsoleType enum
The vendored gns3fy copy keeps its type lists as literals (the module is
shared with the standalone MCP service and cannot import server enums),
and CONSOLE_TYPES had drifted: 'ssh' and 'docker_exec' were missing while
both are valid server-side. Impact: the copilot topology reader validates
the whole node list in one pydantic pass, so a single vendor NOS node
(console_type 'docker_exec') made it drop the entire project and return
zero devices to every copilot device tool.

Add the missing values plus drift tests asserting the vendored lists
cover the server enums (skipped when ai-features extras are absent).
2026-08-15 23:32:24 +08:00
Guobin Yue
74a51a1f9f
Merge branch '3.1' into code-review-fixes 2026-08-15 22:39:42 +08:00
Cristi
a35df76384 feat: Enhance information displayed on the system status dashboard 2026-08-14 23:36:03 +03:00
YueGuobin
110e041e56
docker: cap GNS3_STOP_TIMEOUT at 210 s (controller stop budget)
The 600 s clamp was unreachable in practice: the controller's stop
request times out at 240 s (controller/node.py) and the Docker stop
query gets the value +30 s as its HTTP timeout, so anything above 210
would abort upstream first and surface an error while the stop keeps
running server-side. Cap at the derived ceiling and document the chain
in the clamp and the docstring.
2026-08-15 01:25:50 +08:00
YueGuobin
9604c85fda
docker: harden the shm/devices/extra_configs/masking work (code review)
Nine fixes from a review of the docker-shm-devices diff:

* GNS3_STOP_TIMEOUT >300 s aborted at the manager's default HTTP timeout
  before Docker finished the stop — the stop query now gets a timeout
  with a margin over the grace period.
* Overlapping bind targets (GNS3_MASK_UDEV + GNS3_MASK_SYSTEMD on the
  same unit, a unit named twice, an extra_configs target equal to a
  masked unit) made Docker reject the create with 'Duplicate mount
  point' — Mounts are deduplicated by target.
* ExtraConfig.target now carries a pydantic validator (absolute file
  path, no '..'), so bad targets 422 at template-save time instead of
  failing at node-create time after a multi-GB image pull; directory
  forms ('/', '/etc/') are also rejected by the runtime guard instead
  of raising IsADirectoryError (raw 500).
* _check_host_readiness skipped every remaining check when one
  /proc/sys key was unreadable (mid-loop return) — now continues.
* The base-class GNS3_* env parser strips trailing commas like the
  vendor parser, so 'GNS3_MASK_UDEV=1,' composed from a list still
  activates.
* Vendor env knobs are re-parsed on every create(), so a PUT to the
  node's environment takes effect on the next (re)create.
* The graceful SIGTERM stop is now limited to the explicit user stop
  route; delete/update/close/crash-cleanup keep the immediate kill
  (those paths force-delete or recreate the container right after).
* An extra_configs target beneath a persisted volume is shadowed by the
  volume bind — warn at create time.
2026-08-15 00:52:55 +08:00
YueGuobin
62076c1727
docker: make the vendor graceful-stop grace period configurable (GNS3_STOP_TIMEOUT)
The 60 s SIGTERM grace was hardcoded, unlike every other vendor knob
(GNS3_SHM_SIZE, GNS3_DEVICES, GNS3_MASK_UDEV, ...) which rides the
environment line. Parse GNS3_STOP_TIMEOUT=<seconds> (default 60,
clamped to 1-600, invalid values keep the default) and use it in
VendorDockerVM._terminate_container().
2026-08-14 22:11:34 +08:00
YueGuobin
e9339faaa7
docker: graceful stop for vendor NOS containers (SIGTERM + 60s grace)
DockerVM.stop() terminated containers with an immediate SIGKILL — fine
for init.sh-based containers whose state is persisted beforehand, but a
systemd NOS (Cisco XRd, SR Linux) needs a graceful shutdown and treats
the abrupt kill as an unclean shutdown (exit 137 on every stop).

Extract the final termination into _terminate_container() and override
it in VendorDockerVM: POST /containers/{id}/stop?t=60 sends SIGTERM and
waits for systemd to stop services; Docker itself SIGKILLs the
container once the grace period expires, so no fallback is needed.
Docker's 304 (already stopped) is swallowed.
2026-08-14 22:02:58 +08:00
YueGuobin
350f2b24e7
fix: UDP port allocation race causing link self-loop (one-way links)
PortManager.get_free_udp_port had an unguarded find-then-add sequence.
A link allocates both ends concurrently (asyncio.gather in
UDPLink._prepare -> two POST /ports/udp) and FastAPI runs the sync
route handler in a threadpool, so both threads could probe the same
'free' port before either recorded it — handing lport == rport to both
ends. uBridge sets SO_REUSEADDR on UDP NIO sockets, so the double bind
succeeds silently and the kernel delivers everything to the last-bound
socket: one node starves, the other echoes to itself.

Make every TCP/UDP allocate/reserve/release path atomic with an RLock,
and rebuild _link_data in UDPLink._prepare so reset() commits the fresh
port pair instead of re-sending the stale, already-released one.

Regression tests: threaded barrier allocation never returns duplicates
(red on the old code, UDP and TCP); reset() leaves exactly one mirrored
NIO pair per side with lport != rport (red on the old code).
2026-08-14 21:31:58 +08:00
YueGuobin
e8b51aa12c
docker: also null-bind udevadm in GNS3_MASK_UDEV
Masking the udev systemd units stopped the daemon's coldplug (host audio
resets), but host USB devices still reconnected on every XRd start. A/B
testing with plain `docker run` isolated the trigger: XRd's own
xr_startup.sh calls udevadm directly (USB license-dongle probing, e.g.
`udevadm trigger --action=add --parent-match=<usb device>`), which
synthesizes uevents into the host kernel from the privileged container --
no udevd required.

GNS3_MASK_UDEV=1 now also binds /dev/null over the udevadm binary
(/bin, /sbin, /usr/bin). Verified with a plain-run experiment: with the
bind, host udev monitor shows zero usb/input/hid/sound events during XRd
boot (only normal docker veth traffic), and XRd itself boots to running
state -- it does not need udevadm under GNS3 (interfaces are pre-created
veths).
2026-08-14 19:09:48 +08:00
YueGuobin
347537f1b3
docker: mask systemd-udevd in privileged containers (GNS3_MASK_UDEV)
A privileged systemd-based NOS container (Cisco XRd boots /usr/sbin/init)
runs systemd-udevd, which on startup coldplugs every device it can reach.
In privileged mode that includes the HOST's USB/input/audio/disk devices,
so every XRd start reconnects USB, mutes audio, and disrupts the host
journal -- highly disruptive on Linux desktops (caught in the act: the
container's udevd was even rescanning the host BTRFS root device).

XRd doesn't need udev (its interfaces are pre-created by GNS3 veth and
mapped via XR_INTERFACES). Add two opt-in env vars, consumed host-side at
container create time in the inherited DockerVM.create (so VendorDockerVM
nodes get it too):

  GNS3_MASK_UDEV=1        -> bind /dev/null over the udevd unit, its two
                             activation sockets, and the coldplug/settle
                             trigger services
  GNS3_MASK_SYSTEMD=u1,u2 -> bind /dev/null over arbitrary units in
                             /etc/systemd/system/ (comma/semicolon list)

Only injected when set, so ordinary nodes are unaffected.
2026-08-14 01:12:21 +08:00
YueGuobin
08e37a4509
docker: inject config files into containers via extra_configs
Add an `extra_configs` field (list of {target, content}) to the docker
node/template/appliance schemas. For each entry GNS3 writes `content` to a
file in the node working directory and bind-mounts it read-only at `target`
inside the container.

This lets a NOS appliance seed its startup config without rebuilding the
image: XRd points XR_FIRST_BOOT_CONFIG at an injected /firstboot.cfg, FRR at
/etc/frr/frr.conf, etc. The bind is a single-file mount applied at create
time, so it works for both the generic init.sh path and vendor nodes that
skip init.sh (console_type=docker_exec). Entries are only injected when
present, so ordinary nodes are unaffected.

The content can't go through `environment` (it is line-delimited, one var per
line), hence a dedicated field -- the same plumbing shape as extra_volumes.
2026-08-13 23:33:34 +08:00
YueGuobin
2bee34031c
docker: warn about low inotify/file-max and missing FUSE at connect
Add a read-only _check_host_readiness() that runs once after the Docker
daemon connection is established. It reads /proc/sys inotify/file-max
limits and /proc/filesystems (for FUSE), and logs a warning with the exact
commands to fix when they are too low for heavy containers -- XRd wants
~4000 inotify instances per node against a stock default of 128.

The server runs unprivileged (only the setuid ubridge helper has root), so
it can only check, not set; the warning tells the admin exactly what to
raise once. Stays silent when the limits are already sufficient.
2026-08-13 22:39:48 +08:00