Commit Graph
1186 Commits
Author SHA1 Message Date
YueGuobin 653ea386a5 fix: use the OCI created label when the daemon reports no build time
Images built for reproducibility, such as the Ubuntu images assembled
with rockcraft, carry no created timestamp in their config and Docker
reports 0, shown as 1970-01-01. Fall back to the
org.opencontainers.image.created label in list_images so the image list
shows the real build time.
2026-11-07 19:27:55 +08:00
YueGuobin 076c949a4e fix: default tagless Docker image pulls to the latest tag
The daemon pulls every tag of a repository when the tag parameter is
empty. The docker CLI never hits this because it adds :latest before
calling the API, so only raw API callers do. Pulling "debian" through
the API downloaded dozens of images until the registry rate limit
stopped the pull.

Pass tag=latest when the reference has no tag and no digest. Only a
':' after the last '/' counts as a tag separator: before it, it is a
registry port (e.g. registry:5000/repo).

This covers both the /docker/images/pull route and the automatic pull
fallback when a node image is missing.
2026-11-07 14:43:13 +08:00
YueGuobin de986e890e feat: name nodes and templates in Docker image delete conflicts
The compute translates the container ids in a daemon delete conflict
into the GNS3 node and project behind each container. Leftover
containers from closed projects and containers created outside GNS3
are named as such.

The controller lists the templates referencing the image in the
conflict error, matched through the canonical image name. After a
successful deletion it emits a docker.image.deleted notification with
those templates: their nodes can no longer start.
2026-11-06 22:05:52 +08:00
YueGuobin 04a8ba06a8 fix: start pull events when the pull begins and reject unsafe image names
The pull started event was emitted by the route before the pull took
its per compute lock, so a request queued behind another pull claimed
to be running while it was still waiting. The pull now reports the
started state itself, once it owns the lock.

Deleting an image interpolates the name into the daemon API path. The
pattern on the path parameter limits it to the characters of an image
reference and validate_docker_image_name also rejects "..", so a name
cannot escape that path. Applied on both the controller and compute
routes.
2026-11-06 15:42:51 +08:00
YueGuobin d5d25eba06 fix: stop reporting a missing image as a network problem
A 404 from the registry means the image name is wrong or the registry
needs authentication, but the pull wrapped every error with "please
check your Internet connection". Say what actually failed and keep the
network advice for the other errors.
2026-11-06 15:23:34 +08:00
YueGuobin 8c98142d26 feat: report Docker pull progress and add an idle timeout
A pull now reports docker.image.pull notifications on the controller
stream: started, throttled downloading events with the layer counts
and the latest daemon message, then completed or failed. An optional
progress_id is echoed in every event so a client can correlate them
with its own request. Image operations belong to a compute, not to a
project, so the events go to the controller stream and reach clients
that have no project open.

The pull also has an idle socket read timeout of 60 seconds. Without
it an unreachable registry left the pull, the compute request and the
client request hanging forever with no feedback at all. The timeout
is idle time, not total time: the daemon streams progress during the
whole pull, so a minute without data means the registry is gone. When
it fires, the daemon connection closes and Docker cancels the pull.

Docker.http_query now accepts an aiohttp ClientTimeout object so a
caller can ask for a policy finer than a total timeout.
2026-11-06 15:03:34 +08:00
YueGuobin 54cfcef6e2 fix: send the docker force query parameter as a string
The aiohttp URL builder only accepts strings, ints and floats as query
values and raises a TypeError on a Python bool, so deleting an image
answered a 500. Send true/false like the controller does.
2026-11-06 14:31:26 +08:00
YueGuobin c1e03194d3 api: list Docker image details and delete images
GET /computes/{compute_id}/docker/images now returns the daemon image
id, size and creation date in addition to the tag, so a UI can show
and compare images instead of bare names. The fields are optional to
keep the response compatible with older computes.

Add DELETE /computes/{compute_id}/docker/images/{image} which removes
the image from the compute Docker daemon. The image name is passed as
a path so names with a slash or a tag separator work. A daemon that
refuses because a container uses the image answers a conflict; the
force query parameter removes it anyway.

The compute also drops the controller digest to local id translation
recorded when the image was synced, since it no longer matches
anything on the daemon.
2026-11-06 14:22:22 +08:00
grossmj 5e11378a03 feat(qemu): improve UEFI boot configuration and add KVM acceleration warning 2026-11-04 13:07:21 +07:00
grossmj 0a88127e33 feat(qemu): enhance aarch64 support with UEFI and improved console options 2026-11-04 00:31:52 +07:00
grossmj 524fff814b feat: add support for running aarch64 Qemu VMs 2026-11-01 18:48:41 +07:00
grossmj af7b70001a fix: hardware acceleration support for ARM architectures in QEMU 2026-10-31 23:38:29 +07:00
grossmj 110638d81a refactor: replace os.path.commonprefix with os.path.commonpath for path comparisons 2026-10-31 19:43:52 +07:00
grossmj c7e085e23c refactor: remove local gns3vm support and remove more platform-specific code 2026-10-31 17:51:37 +07:00
grossmj bad48286a5 refactor: remove platform-specific code for Darwin and Windows. Clean up Qemu options handling 2026-10-31 16:49:23 +07:00
Jeremy Grossmann fa4b763640 Merge pull request #3027 from tikD1337/fix/ubridge-kernel-datapath-version
Require uBridge >= 1.3.1
2026-10-25 02:49:15 +02:00
tikD1337andClaude Opus 5.5 07b0c21fe5 Require uBridge >= 1.3.1 instead of gating the kernel datapath
Review feedback: 3.1 is still in alpha and the GNS3 VM ships a matching
uBridge, so a hard minimum is simpler than a version-based degrade path,
and it also covers Docker endpoints and the builtin Ethernet switch,
which never went through the anchor probes. Raise minimum_required_version
to 1.3.1 and drop supports_kernel_datapath and its uses; the relay
fallback for capability problems (no CAP_NET_ADMIN) is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 01:39:52 +02:00
Jeremy Grossmann dc6a9b3c7e Merge pull request #3028 from tikD1337/fix/switch-stale-bridge
fix(ethernet-switch): release a leftover bridge's ports before the crash-recovery delete
2026-10-24 17:31:11 +02:00
tikD1337andClaude Opus 5.5 156b77000c fix(qemu): open the adapter TAP without a vnet header
The TAP netdev kept QEMU's default vnet_hdr, so virtio-net, e1000e, igb
and vmxnet3 guests offload TCP/UDP checksums (and TSO) to the TAP: their
frames carry only the pseudo-header sum in the checksum field. A kernel
bridge link finishes them, but the uBridge relay (cloud, NAT, hub, ...)
reads the TAP with libpcap and forwards the frames as they are, so DNS
and TCP through a cloud fail while ICMP and DHCP work.

With vnet_hdr=off QEMU stops offering the offloads and the guest (or the
emulated NIC) sends finished frames, as with the 2.x socket netdev.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:28:53 +02:00
tikD1337andClaude Opus 5.5 d440849886 fix(ethernet-switch): release a leftover bridge's ports before the crash-recovery delete
A server killed before close() leaves the switch's kernel bridge behind
with its ports still enslaved: a peer's persistent anchor TAP and cascade
veth ends outlive uBridge. uBridge's brctl delete refuses a bridge that
still has ports (EBUSY), so the best-effort recovery delete in
_ensure_bridge() failed silently and brctl create hit EEXIST: the switch,
and the project holding it, could no longer be opened (HTTP 409 "Could
not create bridge gns3xxxxxx: File exists").

List the leftover bridge's ports from /sys/class/net/<bridge>/brif and
release each with brctl delif before the delete. uBridge's brctl show
reports no ports, and the switch already reads sysfs on the same host.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:10:13 +02:00
tikD1337andClaude Opus 5.5 52fcc83494 Gate the TAP/kernel datapath on uBridge >= 1.3.1
uBridge 1.2.3 already has the tap module, so the persistent-TAP probes
(_probe_persistent_taps on the node, probe_tap_support and siblings for
/capabilities) pass on it. QEMU, Dynamips and VPCS nodes then anchor on
TAPs and the controller builds kernel links that need 1.3.x commands:
kernel markers and the eBPF tc modes (1.3.0) and per-interface kernel
captures (`capture stop_kernel <ifc>`, 1.3.1). On 1.2.3 a MikroTik CHR
node sent frames but never got DHCP past DISCOVER/OFFER, and stopping
a capture failed with "Bad number of parameters".

Add Hypervisor.supports_kernel_datapath (version >= 1.3.1) and refuse
older builds in the node-side probe and the four anchoring capability
probes (tap, iol tap, bridge tap, vxlan). The existing fallback then
applies: nodes keep the legacy relay datapath with the usual warning,
which now names the installed version, and links stay on the relay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 15:00:27 +02:00
Jeremy Grossmann 2c10782635 Merge pull request #3026 from tikD1337/fix/node-names-adapter-type
fix: node names from templates with "+" and custom adapters with a null type
2026-10-23 14:08:47 +02:00
tikD1337andClaude Opus 5.5 b9cf9158ca fix(compute): ignore null settings of custom adapters
When only a port is renamed in a template's custom adapters, the adapter is
saved with "adapter_type": null. The emulators read it with
custom_adapter.get("adapter_type", self._adapter_type), which returns None
for a key set to null: QEMU was then started with "-device None,..." and
died ("'None' is not a valid device model name"), GNS3 only reporting
"Could not start Telnet QEMU console". VirtualBox and VMware read the
adapter type the same way.

Drop the null settings in _get_custom_adapter_settings so the node's own
adapter type (and MAC address) applies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 09:56:48 +02:00
tikD1337andClaude Opus 5.5 53164eccee fix(qemu): turn off kvmclock and TSC deadline timer on Hyper-V hosts without a usable TSC
When the compute is itself a Hyper-V guest without an invariant TSC (e.g.
the GNS3 VM on hosts whose CPU lacks TSC scaling), Linux falls back to the
hyperv_clocksource_tsc_page clocksource and the guest timekeeping KVM
derives from the TSC is broken with recent kernels:

- kvmclock runs thousands of times too fast: Linux guests (MikroTik
  RouterOS, ...) panic with "watchdog: BUG: soft lockup - CPU#0 stuck for
  29604s" and reboot in a loop right after boot;
- TSC deadline timers fire far too late: guests with -cpu host hang.

The only workaround so far was -machine accel=tcg, i.e. no KVM at all.

Add a kvmclock setting to the [Qemu] section (auto, enabled, disabled).
In auto mode, KVM guests started on a host whose clocksource is a Hyper-V
one get kvmclock and tsc-deadline turned off: on the -cpu given in the
additional options (unless it sets these features itself), otherwise on
QEMU's default CPU model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 15:38:10 +02:00
grossmj a82b06d676 feat: add QEMU support documentation and enforce minimum version requirement 2026-10-20 18:01:25 +07:00
grossmj cfba748677 fix: SPICE socket binding failure 2026-10-20 17:16:44 +07:00
grossmj 437f68961b chore: clean up Windows specific code 2026-10-20 11:51:29 +07:00
grossmj 569b46d272 refactor: fix lint issues 2026-10-19 23:28:49 +07:00
Jeremy Grossmann 68b5e89bfa Merge branch '3.1' into 3.1 2026-10-19 18:11:29 +02:00
Cristi 53aa7ab0d9 fix(cloud): keep ports_mapping getter and setter adjacent 2026-10-19 16:02:39 +03:00
Cristi a9f5306f16 feat(cloud): attach TAP interfaces to existing Linux bridges 2026-10-19 15:42:18 +03:00
Cristi 58e5591935 feat(qemu): support SATA CD-ROM interfaces
- Add per-drive IDE/SATA selection with backward-compatible IDE defaults
- Persist interfaces in nodes, templates, and appliance imports
- Add database migration support for versioned and unversioned databases
- Prevent partial updates when rejecting live interface changes
- Validate interfaces and support SATA media changes through the monitor
2026-10-18 22:07:37 +03:00
Cristi 84b4c05b9f feat(qemu): support two CD/DVD images and consistent ISO handling 2026-10-17 23:08:11 +03:00
Mark Paronyan 4c311baabf refactor(gns3server,tests): StrEnum, ASYNC109 and others 2026-10-17 12:02:53 +03:00
Mark Paronyan b7e6c02647 refactor: ruff autofixes 2026-10-17 11:43:36 +03:00
YueGuobin 21b866aeed feat: cross-compute VXLAN kernel datapath
Cross-compute links can ride kernel VXLAN tunnels instead of the uBridge
UDP relay: each endpoint creates one VXLAN device (uBridge's rtnetlink
vxlan module) enslaved into the same per-link Linux bridge the same-host
datapath uses, so filters, markers, capture and suspend keep running on
the anchor unchanged. Opt-in (Server.enable_vxlan_cross_compute, default
off) because the tunnel caps the usable MTU at the underlay minus the
outer headers; the relay stays the default and the fallback.

The controller derives what the pair must agree on: one VNI per link
(upper VNI space, per-link random salt) and deterministic device names
(gvl{link}{node}) that make crashed-run leftovers claimable
(show -> delete -> create). Same-host compute pairs, switch endpoints
and serial/unix-socket containers stay on the relay.

A VNI the kernel refuses self-heals on every path it can surface:
interactive create retries a fresh salt then degrades to the relay;
project open re-dispatches refused batch entries the same way; node
start recognises the refused device, stops the half-started node,
re-wires both ends on a fresh VNI (or degrades the link) and retries
the start. The device MTU is derived from the lower device (dev=) and
the per-link bridge inherits it - no configuration. The capability
probe (ubridge_vxlan in /capabilities) runs one scratch create/delete
on a scratch port that never equals the configured dstport.

The e2e scenario runs two computes on one machine (a network namespace
joined by a veth pair as the fabric) and needs root.
2026-10-17 14:11:00 +08:00
Jeremy Grossmann 344a342924 Merge pull request #3004 from Xeriioo/fix/docker-vnc-display-race
fix(docker): give each VNC node its own X display
2026-10-16 12:57:38 +02:00
YueGuobin 1090a32bbb fix: refuse Docker node updates that recreate a running container
Updating any Docker node property destroys and recreates the container
(the route calls DockerVM.update() as soon as the property diff is
non-empty), so it discards everything written inside it. On a running
node that happened silently, as a side effect of an unrelated edit: the
user only saw the node restart.

Refuse the update while the container is up and say what stopping it is
for, like QEMU, IOU, Dynamips, VPCS, VirtualBox and VMware already do
for their own restart-requiring settings. The check runs before anything
is applied — a refused update leaves the node untouched — and only when
the PUT really changes a property, since Web clients send the whole node
back on every edit.

is_running() is not reusable for the question: it reports a paused
container as not running and stops crashed containers as a side effect,
so DockerVM gains is_up() for "running or paused".
2026-10-16 16:49:11 +08:00
ClemandClaude Opus 5.5 c736d5863b fix(docker): give each VNC node its own X display
When a project is opened, Docker nodes are created in parallel. The free
display was found by checking whether /tmp/.X11-unix/X<n> exists, but that
socket only appears once Xvnc has started, so several nodes got the same
display. Only the first Xvnc started; the others exited with "Server is
already active for display N" ("The vnc process has stopped with return
code 1"), and their containers drew on the screen of another node.

- reserve the displays given to the nodes of this server, and also skip a
  display whose lock file belongs to a live process
- restart a crashed Xvnc on the display the container was created with,
  after removing its stale socket, instead of picking a new display the
  container cannot reach (black screen)
- on close, only remove the X socket of our own Xvnc, never the socket of
  the node that owns the display

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:29:19 +02:00
YueGuobin 3e6a53c4b9 style: apply ruff format to the files this branch touches 2026-10-14 22:13:58 +08:00
YueGuobin 9e0d5050db feat: concurrent kernel captures, one per interface
uBridge's capture module now serves one AF_PACKET capture per interface,
any number per process, and `capture stop_kernel` takes the interface it
stops — this rides that (breaking, paired-binary) protocol change.

The server-side single-slot tracker (_kernel_capture_ifc with its
reserve/release/owned_by helpers) is gone: same-port duplicates were
already refused by the NIO capturing flag, uBridge answers EALREADY on a
same-interface duplicate, and per-interface stops removed the process-
wide stop the ownership gating guarded against.

The docker e2e drives ten concurrent captures on one node's uBridge and
asserts a per-interface stop freezes exactly the stopped half while the
live half keeps recording.
2026-10-14 21:52:34 +08:00
YueGuobin c0568dc25f fix: hide kernel-datapath interfaces from the cloud node
The cloud node offered the kernel datapath's host-side interfaces —
node anchors (gv/gq/gi/gd/gx/gp), switch cascade veth ends (gs) and
cross-compute VXLAN devices (gvl) — in its interface list and even
pre-filled them as ports on a freshly created cloud: none of them is
flagged special and only "gns3*" names were filtered. These interfaces
belong to the link or node that created them for the project's whole
lifetime, so bridging a cloud to one would race its owner.

Add is_kernel_datapath_interface() next to the naming contract in
utils/kernel_anchor.py: exact-shape matching (prefix plus its
fixed-length id-derived tail), so a user interface that merely starts
with "gv" is never swallowed. The cloud keeps managed names out of its
host-interface list and default ports mapping, while saved mappings
keep working (the existence checks still see the full interface list).
2026-10-14 10:11:49 +08:00
Jeremy Grossmann b08441b9ad Merge pull request #2983 from yueguobin/fix/docker-image-digest-translation
fix(docker): accept the pinned image id when the two daemons report different ids for the same content
2026-10-13 16:23:39 +02:00
grossmj e8c641bd08 docs: update comments regarding Ethernet switch replacement with Linux bridge 2026-10-13 20:18:29 +07:00
YueGuobin 90917c4a2c fix(docker): accept the pinned image id when the stores report different ids
Docker ids are not comparable across image stores: for the same content a
classic store reports the config digest while a containerd store (or an
image loaded from an OCI archive) reports the manifest digest. The
controller pins the id of its own daemon, so a compute whose store reports
another id for that content answered ImageMissingError forever: every node
creation streamed the image again (save/load round trip) without ever
converging, then failed with "The image 'x' is missing" after the six
retries. Docker 29 makes this easy to hit, since the id a daemon reports
now depends on how the image entered it.

The load endpoint now takes the tag and the digest pinned by the
controller, and the compute records how its own daemon identifies the
image it just received. Node creation accepts the pin through that pair,
and only while the tag still resolves to the synced content: a tag moved
to some other image, or an image the controller never pushed here, still
counts as missing. The pair lives in the compute process, so a restarted
compute syncs once more.

Computes that predate the new endpoint parameters keep the previous
behaviour, as do controllers that do not send them.
2026-10-13 17:10:17 +08:00
YueGuobin c7ffe919dc feat: VPCS kernel datapath
A VPCS node's single adapter anchors on a persistent TAP the server
creates (tap create + set_owner via uBridge) and the vpcs process
attaches to by name: -e -d <gp{id8}e0p0> -n, the same
server-creates/guest-attaches shape as QEMU's netdev. -n (a vpcs fork
feature) keeps the guest's ip command from ifconfig'ing the anchor on
the host; the vpcs build is probed once per binary identity
(vpcs_tap capability, usage scan) and gates kernel links together
with ubridge_tap — anything else keeps the node on the relay (the
legacy local UDP tunnel on a stock binary, an AF_PACKET endpoint on
the anchor with a capable one).

VPCSVM takes KernelDatapathMixin: _connect_nio dispatches kernel
(enslave into the per-link bridge) / relay-on-anchor / legacy,
captures pick capture start_kernel vs the bridge engine, stop runs
under an execute lock and terminates the process before deleting the
anchor TAPs (a held device refuses deletion), and the duplicate-port
guard backstops the controller's. The NIO routes accept nio_bridge
and write suspend back on update.

_delete_anchor_taps verifies each tap is gone from sysfs and retries
a bounded three times: a holder can release its fd a moment after its
process was reaped (vpcs's daemon parent kills its console child and
exits first), so the unconditional delete raced EBUSY and leaked the
anchor intermittently.

The vpcs side needs IFNAMSIZ (16) instead of its own IFNAMESIZ (12)
for the -d name buffer — anchor names are 14 chars and truncated to a
nonexistent device, which TUNSETIFF then tries to create without
CAP_NET_ADMIN (fixed in the vpcs fork).

e2e (tests/e2e/test_vpcs_kernel_datapath.py): kernel and relay
scenarios green against the real binary — real ICMP both ways, §E.1/§
E.2 on the anchors, delay/suspend/capture, link delete/re-create,
node stop/restart self-heal, zero leftovers; the relay control passes
with both the fork binary and a stock one.
2026-10-12 17:19:57 +08:00
grossmj a437d46138 Revert "fix: enable Layer 1 keepalives by default. Ref #2975"
This reverts commit e4c25549f0.
2026-10-11 16:44:07 +07:00
grossmj e4c25549f0 fix: enable Layer 1 keepalives by default. Ref #2975 2026-10-11 16:34:44 +07:00
YueGuobin 1e0d123f4b Merge branch '3.1' into fix/duplicate-device-configs-merge
Resolve conflicts against the upstream wave (PRs #2952-#2965, incl. the
typed-response/schema hardening and the drop of Python 3.10):

- controller/node.py: keep upstream's _transitioning/_confirm_status
  lifecycle guard; the kernel-anchor link wiring (node_started) runs
  after the transition completes.
- controller/link.py: keep upstream's add_node rollback (release_ports
  on create failure) and delete via release_ports; release_ports now
  resolves ports live (same staleness reason as _bind_members) and
  discards the node link unconditionally.
- schemas/controller/links.py: extend LinkFilterType with the eight
  kernel-datapath filter types and LinkFilterParameter.type with "str"
  so the newly typed /available_filters response validates; Link.filters
  stays a free-form dict — the vocabulary is capability-gated per link
  and validated by the controller (409), not by a strict request schema.
- schemas/controller/topology.py: drop the type: ignore made redundant
  by the dict filters field.
- tests: route-level filter validation now expects the controller's 409
  (unknown types, out-of-range values); packet_loss correlation is a
  valid kernel parameter (positive test added); OpenAPI assertions match
  the free-form filters property.
2026-10-09 10:03:50 +08:00
grossmj 811b45f863 fix: drop Python 3.10 and add Python 3.15 support 2026-10-08 15:14:34 +02:00