Commit Graph
286 Commits
Author SHA1 Message Date
YueGuobin c1e03194d3 api: list Docker image details and delete images
GET /computes/{compute_id}/docker/images now returns the daemon image
id, size and creation date in addition to the tag, so a UI can show
and compare images instead of bare names. The fields are optional to
keep the response compatible with older computes.

Add DELETE /computes/{compute_id}/docker/images/{image} which removes
the image from the compute Docker daemon. The image name is passed as
a path so names with a slash or a tag separator work. A daemon that
refuses because a container uses the image answers a conflict; the
force query parameter removes it anyway.

The compute also drops the controller digest to local id translation
recorded when the image was synced, since it no longer matches
anything on the daemon.
2026-11-06 14:22:22 +08:00
YueGuobin 4c00d223e4 fix: pass the leading slash to the compute load endpoint path
getUrl joins the path directly after /v3/compute, so a path without
the leading slash produces /v3/computedocker/images/load and the
compute answers 404. Match the path shape used by forward() and the
image sync path.
2026-11-06 12:46:21 +08:00
YueGuobin 092202615e api: add controller endpoint to load Docker images on a compute
Add POST /computes/{compute_id}/docker/images/load. It streams an
uploaded docker save tar (plain or gzip) to the compute, which loads
it into its Docker daemon. Clients can now install image packages
such as iol-xe-17-18-02.tar.gz without SSH access to the server.

The body is proxied without buffering (async iterable to chunked
transfer) and runs with no total timeout, like the existing image
sync path between computes.
2026-11-06 10:00:17 +08:00
YueGuobin 3ddfe0254d api: add endpoint returning the current user's privileges
Add GET /access/users/me/privileges. It returns the effective privileges
of the logged in user, flattened across all roles assigned to the user
(directly or through group membership). Superadmins get the full
privilege list.

This gives clients a way to render their UI based on permissions. The
server stays the authority for permission checks: a privilege in the
list can still be denied on a specific resource path.
2026-11-05 23:05:28 +08:00
YueGuobin 517b0b0d85 fix: attribute duplicated project to the user who duplicated it
duplicate() kept the original author in created_by. The project list
filters on created_by, so a non-admin user could not see the copy they
just made. Pass the caller through both duplication paths, the same way
import already does since #3017.
2026-11-05 19:22:27 +08:00
YueGuobin a56fce4c36 fix: do not monkeypatch attributes on Starlette State in tests
pytest 9.2 reads the old value of a monkeypatched attribute from the
instance __dict__. Starlette State keeps attributes in an internal
dict, so the old value looks unset and the teardown deletes the
attribute for the rest of the test session. The first test that
touches app.state.exiting passes but removes it, and every later
monkeypatch.setattr on it fails with AttributeError.

Use plain assignments instead.
2026-11-04 09:22:15 +08:00
YueGuobin 52c19002f2 fix: bound graceful shutdown and stop notification WebSockets on exit
By default uvicorn waits for open connections forever. A long-lived
streaming response keeps the server waiting after Ctrl+C. One common
case is the MCP Streamable HTTP transport: the client opens a
standalone GET request and the server answers with an SSE stream
(not the old SSE transport). A second Ctrl+C sets force_exit. Uvicorn
then skips the lifespan shutdown, so project and controller cleanup
does not run.

Add timeout_graceful_shutdown. After the timeout uvicorn cancels the
remaining tasks and runs the lifespan shutdown. Also make the four
notification WebSocket handlers stop when app.state.exiting is set.
The HTTP stream handlers already do this.
2026-11-04 00:32:34 +08:00
grossmj ac65698eea refactor: update authentication handling to use aiohttp.encode_basic_auth and adjust request options + fix other pytest warnings 2026-10-31 22:00:21 +07:00
grossmj c7e085e23c refactor: remove local gns3vm support and remove more platform-specific code 2026-10-31 17:51:37 +07:00
grossmj bad48286a5 refactor: remove platform-specific code for Darwin and Windows. Clean up Qemu options handling 2026-10-31 16:49:23 +07:00
tikD1337andClaude Opus 5.5 09b8420354 fix(api): keep the LLM stub from shadowing /access routes without AI extras
Without the AI dependencies the LLM model configuration router is replaced
by a stub that answers 501. It was a catch-all ("/{path:path}") included
under the /access prefix, so it also answered every /access route included
after it: creating an API key (POST /v3/access/api-keys) failed with "AI
Copilot is not available", which left API keys unusable on servers without
the AI extras.

The LLM stub now covers only the LLM model configuration paths, under
/access/users/{user_id}/ and /access/groups/{group_id}/. The stub routers
are built by build_ai_stub_routers() so tests can check them with or
without the extras, and add_stub_routes() takes the paths to stub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 10:35:12 +01:00
Jeremy Grossmann dc6a9b3c7e Merge pull request #3028 from tikD1337/fix/switch-stale-bridge
fix(ethernet-switch): release a leftover bridge's ports before the crash-recovery delete
2026-10-24 17:31:11 +02:00
tikD1337andClaude Opus 5.5 d440849886 fix(ethernet-switch): release a leftover bridge's ports before the crash-recovery delete
A server killed before close() leaves the switch's kernel bridge behind
with its ports still enslaved: a peer's persistent anchor TAP and cascade
veth ends outlive uBridge. uBridge's brctl delete refuses a bridge that
still has ports (EBUSY), so the best-effort recovery delete in
_ensure_bridge() failed silently and brctl create hit EEXIST: the switch,
and the project holding it, could no longer be opened (HTTP 409 "Could
not create bridge gns3xxxxxx: File exists").

List the leftover bridge's ports from /sys/class/net/<bridge>/brif and
release each with brctl delif before the delete. uBridge's brctl show
reports no ports, and the switch already reads sysfs on the same host.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:10:13 +02:00
tikD1337andClaude Opus 5.5 53164eccee fix(qemu): turn off kvmclock and TSC deadline timer on Hyper-V hosts without a usable TSC
When the compute is itself a Hyper-V guest without an invariant TSC (e.g.
the GNS3 VM on hosts whose CPU lacks TSC scaling), Linux falls back to the
hyperv_clocksource_tsc_page clocksource and the guest timekeeping KVM
derives from the TSC is broken with recent kernels:

- kvmclock runs thousands of times too fast: Linux guests (MikroTik
  RouterOS, ...) panic with "watchdog: BUG: soft lockup - CPU#0 stuck for
  29604s" and reboot in a loop right after boot;
- TSC deadline timers fire far too late: guests with -cpu host hang.

The only workaround so far was -machine accel=tcg, i.e. no KVM at all.

Add a kvmclock setting to the [Qemu] section (auto, enabled, disabled).
In auto mode, KVM guests started on a host whose clocksource is a Hyper-V
one get kvmclock and tsc-deadline turned off: on the -cpu given in the
additional options (unless it sets these features itself), otherwise on
QEMU's default CPU model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 15:38:10 +02:00
Jeremy Grossmann 68b5e89bfa Merge branch '3.1' into 3.1 2026-10-19 18:11:29 +02:00
Cristi a9f5306f16 feat(cloud): attach TAP interfaces to existing Linux bridges 2026-10-19 15:42:18 +03:00
Jeremy Grossmann e57acfb7b1 Merge pull request #3011 from cristian-ciobanu/feat-#3049
feat(qemu): support two CD/DVD images and consistent ISO handling
2026-10-19 04:42:18 +02:00
Cristi 58e5591935 feat(qemu): support SATA CD-ROM interfaces
- Add per-drive IDE/SATA selection with backward-compatible IDE defaults
- Persist interfaces in nodes, templates, and appliance imports
- Add database migration support for versioned and unversioned databases
- Prevent partial updates when rejecting live interface changes
- Validate interfaces and support SATA media changes through the monitor
2026-10-18 22:07:37 +03:00
YueGuobin 33b28a3b5f fix: fetch remote-compute marker pcaps in tag replay
Marker pcaps are written by uBridge on the compute that hosts the capture
node. The tag replay only looked in the project folder on the controller.
So a source from a remote compute showed count 0 and no frames, with no
error. Its frame detail returned 404 and said nothing ever matched (#3010).

Each source now gets its pcap through the compute of its capture node.
Local files are read as before. Remote files are fetched over HTTP on
every acquire into a small cache (32 idle files, cleared at shutdown).
A file is replaced only when its bytes changed. This keeps the old mtime,
so the sharkd session for that file stays alive.

When a compute cannot be reached, its source now shows an error message
in the timeline instead of just showing nothing. The detail endpoint now
returns an error instead of 404.
2026-10-18 16:36:11 +08:00
Cristi 84b4c05b9f feat(qemu): support two CD/DVD images and consistent ISO handling 2026-10-17 23:08:11 +03:00
Mark Paronyan b7e6c02647 refactor: ruff autofixes 2026-10-17 11:43:36 +03:00
YueGuobin 21b866aeed feat: cross-compute VXLAN kernel datapath
Cross-compute links can ride kernel VXLAN tunnels instead of the uBridge
UDP relay: each endpoint creates one VXLAN device (uBridge's rtnetlink
vxlan module) enslaved into the same per-link Linux bridge the same-host
datapath uses, so filters, markers, capture and suspend keep running on
the anchor unchanged. Opt-in (Server.enable_vxlan_cross_compute, default
off) because the tunnel caps the usable MTU at the underlay minus the
outer headers; the relay stays the default and the fallback.

The controller derives what the pair must agree on: one VNI per link
(upper VNI space, per-link random salt) and deterministic device names
(gvl{link}{node}) that make crashed-run leftovers claimable
(show -> delete -> create). Same-host compute pairs, switch endpoints
and serial/unix-socket containers stay on the relay.

A VNI the kernel refuses self-heals on every path it can surface:
interactive create retries a fresh salt then degrades to the relay;
project open re-dispatches refused batch entries the same way; node
start recognises the refused device, stops the half-started node,
re-wires both ends on a fresh VNI (or degrades the link) and retries
the start. The device MTU is derived from the lower device (dev=) and
the per-link bridge inherits it - no configuration. The capability
probe (ubridge_vxlan in /capabilities) runs one scratch create/delete
on a scratch port that never equals the configured dstport.

The e2e scenario runs two computes on one machine (a network namespace
joined by a veth pair as the fabric) and needs root.
2026-10-17 14:11:00 +08:00
Guobin Yue d2b851f148 Merge branch '3.1' into fix/node-update-while-running 2026-10-16 16:56:40 +08:00
YueGuobin 1090a32bbb fix: refuse Docker node updates that recreate a running container
Updating any Docker node property destroys and recreates the container
(the route calls DockerVM.update() as soon as the property diff is
non-empty), so it discards everything written inside it. On a running
node that happened silently, as a side effect of an unrelated edit: the
user only saw the node restart.

Refuse the update while the container is up and say what stopping it is
for, like QEMU, IOU, Dynamips, VPCS, VirtualBox and VMware already do
for their own restart-requiring settings. The check runs before anything
is applied — a refused update leaves the node untouched — and only when
the PUT really changes a property, since Web clients send the whole node
back on every edit.

is_running() is not reusable for the question: it reports a paused
container as not running and stops crashed containers as a side effect,
so DockerVM gains is_up() for "running or paused".
2026-10-16 16:49:11 +08:00
YueGuobin 6734d2d666 feat: add auto_connect flag and disconnect endpoint for computes
Computes can now be flagged auto_connect in the database. The controller
connects to flagged computes in the background at startup (reusing the
one-second delayed fire-and-forget mechanism), and enabling the flag via
PUT /computes/{id} triggers a background connection without the request
ever waiting for it. Disabling the flag does not drop an existing
connection.

Add POST /computes/{id}/disconnect to tear a compute down manually: the
notification stream and HTTP session are closed and every reconnection
path (retry backoff, notification stream reconnect, lazy connection in
http_query) is blocked until connect() is called again or the controller
restarts with auto_connect enabled. Disconnecting a compute used by an
opened project is rejected with 409 unless force=true; forcing only
drops the controller-side control channel, nodes keep running unmanaged.
The local compute cannot be disconnected.

Also make test_compute_list create its own compute (it relied on rows
from earlier tests and failed under reordering), and stop scheduling the
delayed add_compute connection during tests: the timer leaked real
connection attempts into other tests' event loops.
2026-10-15 22:50:22 +08:00
YueGuobin 3e6a53c4b9 style: apply ruff format to the files this branch touches 2026-10-14 22:13:58 +08:00
YueGuobin 9e0d5050db feat: concurrent kernel captures, one per interface
uBridge's capture module now serves one AF_PACKET capture per interface,
any number per process, and `capture stop_kernel` takes the interface it
stops — this rides that (breaking, paired-binary) protocol change.

The server-side single-slot tracker (_kernel_capture_ifc with its
reserve/release/owned_by helpers) is gone: same-port duplicates were
already refused by the NIO capturing flag, uBridge answers EALREADY on a
same-interface duplicate, and per-interface stops removed the process-
wide stop the ownership gating guarded against.

The docker e2e drives ten concurrent captures on one node's uBridge and
asserts a per-interface stop freezes exactly the stopped half while the
live half keeps recording.
2026-10-14 21:52:34 +08:00
YueGuobin 90917c4a2c fix(docker): accept the pinned image id when the stores report different ids
Docker ids are not comparable across image stores: for the same content a
classic store reports the config digest while a containerd store (or an
image loaded from an OCI archive) reports the manifest digest. The
controller pins the id of its own daemon, so a compute whose store reports
another id for that content answered ImageMissingError forever: every node
creation streamed the image again (save/load round trip) without ever
converging, then failed with "The image 'x' is missing" after the six
retries. Docker 29 makes this easy to hit, since the id a daemon reports
now depends on how the image entered it.

The load endpoint now takes the tag and the digest pinned by the
controller, and the compute records how its own daemon identifies the
image it just received. Node creation accepts the pin through that pair,
and only while the tag still resolves to the synced content: a tag moved
to some other image, or an image the controller never pushed here, still
counts as missing. The pair lives in the compute process, so a restarted
compute syncs once more.

Computes that predate the new endpoint parameters keep the previous
behaviour, as do controllers that do not send them.
2026-10-13 17:10:17 +08:00
YueGuobin c7ffe919dc feat: VPCS kernel datapath
A VPCS node's single adapter anchors on a persistent TAP the server
creates (tap create + set_owner via uBridge) and the vpcs process
attaches to by name: -e -d <gp{id8}e0p0> -n, the same
server-creates/guest-attaches shape as QEMU's netdev. -n (a vpcs fork
feature) keeps the guest's ip command from ifconfig'ing the anchor on
the host; the vpcs build is probed once per binary identity
(vpcs_tap capability, usage scan) and gates kernel links together
with ubridge_tap — anything else keeps the node on the relay (the
legacy local UDP tunnel on a stock binary, an AF_PACKET endpoint on
the anchor with a capable one).

VPCSVM takes KernelDatapathMixin: _connect_nio dispatches kernel
(enslave into the per-link bridge) / relay-on-anchor / legacy,
captures pick capture start_kernel vs the bridge engine, stop runs
under an execute lock and terminates the process before deleting the
anchor TAPs (a held device refuses deletion), and the duplicate-port
guard backstops the controller's. The NIO routes accept nio_bridge
and write suspend back on update.

_delete_anchor_taps verifies each tap is gone from sysfs and retries
a bounded three times: a holder can release its fd a moment after its
process was reaped (vpcs's daemon parent kills its console child and
exits first), so the unconditional delete raced EBUSY and leaked the
anchor intermittently.

The vpcs side needs IFNAMSIZ (16) instead of its own IFNAMESIZ (12)
for the -d name buffer — anchor names are 14 chars and truncated to a
nonexistent device, which TUNSETIFF then tries to create without
CAP_NET_ADMIN (fixed in the vpcs fork).

e2e (tests/e2e/test_vpcs_kernel_datapath.py): kernel and relay
scenarios green against the real binary — real ICMP both ways, §E.1/§
E.2 on the anchors, delay/suspend/capture, link delete/re-create,
node stop/restart self-heal, zero leftovers; the relay control passes
with both the fork binary and a stock one.
2026-10-12 17:19:57 +08:00
grossmj dff5fbfd7d fix: enable Layer 1 keepalives by default at the template level 2026-10-11 16:50:14 +07:00
Jeremy Grossmann 11c10dd85c Merge branch '3.1' into fix/node-lifecycle-status-flood 2026-10-10 06:49:50 +02:00
grossmj 4fc601e919 refactor: remove unused link filter schemas and fix test_add_v8_version_appliance 2026-10-10 06:40:52 +02:00
grossmj 6004e06499 Merge branch '3.1' into fork/markparonyan/api-issue-10-0eztih
# Conflicts:
#	gns3server/schemas/__init__.py
2026-10-10 06:39:15 +02:00
YueGuobin 0796d455a0 test: make the version output test order-independent
controller_host is recorded on the mounted compute sub-application
when the controller queries the compute capabilities; the test relied
on another test having made that call first and failed when run on
its own. Set the state directly in the test.
2026-10-10 00:29:17 +08:00
YueGuobin 82015dcb5c fix(controller): stop flooding clients with transitional node statuses
Every node lifecycle call (start/stop) emitted a "starting"/"stopping"
node.updated notification on entry, and for the whole duration of the
call asdict() masked the real status with the transitional one — every
compute event forwarded in that window was re-emitted as a duplicate
transitional status.

Stop-all/start-all on a 13-node project pushed two transitional events
per node before the final state. Clients that do not render these
transitional statuses (the Web UI) showed nodes flickering through an
unexpected suspended-looking state before reaching started/stopped.

Remove the transitional notification and the status masking: clients
see the real statuses again (started/stopped), from the forwarded
compute events plus the controller-side confirmation. The wait option
on the lifecycle routes, the strict stop error handling and
wait_for_status() are unchanged.
2026-10-10 00:24:20 +08:00
Guobin Yue 7df7ce1c96 Merge branch '3.1' into fix/duplicate-device-configs-merge 2026-10-09 21:23:38 +08:00
Jeremy Grossmann fbe93c4c4e Merge pull request #2954 from markparonyan/api-issue-05-2xftmd
feat(api): add strict_names and batch node and link creation
2026-10-09 14:39:10 +02:00
YueGuobin 1e0d123f4b Merge branch '3.1' into fix/duplicate-device-configs-merge
Resolve conflicts against the upstream wave (PRs #2952-#2965, incl. the
typed-response/schema hardening and the drop of Python 3.10):

- controller/node.py: keep upstream's _transitioning/_confirm_status
  lifecycle guard; the kernel-anchor link wiring (node_started) runs
  after the transition completes.
- controller/link.py: keep upstream's add_node rollback (release_ports
  on create failure) and delete via release_ports; release_ports now
  resolves ports live (same staleness reason as _bind_members) and
  discards the node link unconditionally.
- schemas/controller/links.py: extend LinkFilterType with the eight
  kernel-datapath filter types and LinkFilterParameter.type with "str"
  so the newly typed /available_filters response validates; Link.filters
  stays a free-form dict — the vocabulary is capability-gated per link
  and validated by the controller (409), not by a strict request schema.
- schemas/controller/topology.py: drop the type: ignore made redundant
  by the dict filters field.
- tests: route-level filter validation now expects the controller's 409
  (unknown types, out-of-range values); packet_loss correlation is a
  valid kernel parameter (positive test added); OpenAPI assertions match
  the free-form filters property.
2026-10-09 10:03:50 +08:00
Jeremy Grossmann aab707a48e Merge branch '3.1' into api-issue-10-0eztih 2026-10-08 19:31:48 +02:00
Jeremy Grossmann aeac649c7f Merge branch '3.1' into api-issue-10-0eztih 2026-10-08 19:21:07 +02:00
Jeremy Grossmann 2dd1652e8a Merge branch '3.1' into api-issue-07-x7z6we 2026-10-08 19:10:42 +02:00
Jeremy Grossmann 57e4d1632d Merge branch '3.1' into api-issue-05-2xftmd 2026-10-08 19:04:36 +02:00
Jeremy Grossmann 7a3a34d3fc Merge branch '3.1' into api-issue-06-8s9swu 2026-10-08 19:01:57 +02:00
Jeremy Grossmann 8dd438190d Merge pull request #2960 from markparonyan/api-issue-12-cu0jkp
feat(api): add discriminators to link iface, appliance version and template settings schemas
2026-10-08 17:17:02 +02:00
Jeremy Grossmann cea858acc4 Merge branch '3.1' into api-issue-12-cu0jkp 2026-10-08 16:46:37 +02:00
Jeremy Grossmann 029983260c Merge branch '3.1' into api-issue-15-r12z73 2026-10-08 16:42:57 +02:00
Jeremy Grossmann ddcf661e11 Merge branch '3.1' into api-issue-07-x7z6we 2026-10-08 15:51:11 +02:00
Jeremy Grossmann 3187828325 Merge branch '3.1' into api-issue-06-8s9swu 2026-10-08 15:02:51 +02:00
Jeremy Grossmann a9e5f859c8 Reorder import statements in test_nodes.py 2026-10-08 14:49:12 +02:00
Jeremy Grossmann 71da996362 Merge branch '3.1' into api-issue-05-2xftmd 2026-10-08 14:40:53 +02:00