115 Commits

Author SHA1 Message Date
YueGuobin
35cf964882
test: cover the kernel-datapath review fixes
Regression coverage added with the fixes:
- capture-slot semantics: second-port refusal, failed-start rollback, and
  a non-owner port's stop not reaching `capture stop_kernel`;
- QEMU stop/start teardown on a mid-body or pre-launch failure, IOU start
  cleanup and filter guards, Dynamips half-attach undo plus retry;
- switch VLAN reconcile skipping a deferred anchor and rolling the
  ports_mapping back on failure;
- Docker/QEMU occupied-port refusal, probe ctime invalidation, probe
  survival of a failing uBridge start;
- env-marker dialects (mid-line mentions, falsy values) and the agent
  merge rejecting entries without a device_name.

Test-body imports of product modules move to the shared header.
2026-10-06 19:55:31 +08:00
YueGuobin
7d04d08dba
fix: repair imports lost in the 3.1 rebase and meet the new lint baseline
The rebase onto upstream 3.1 (mypy + ruff I001/F401 waves) left the tree
short of the newly enforced baselines in three ways:

* contextlib / UbridgeError imports dropped as false duplicates during
  conflict resolution left NameErrors in iou_vm and qemu_vm (F821);
  the Dynamips router grabbed the compute-level NIOTAP instead of the
  hypervisor one, breaking TAP anchoring (test failure).
* unused imports (KERNEL_UNSUPPORTED_FILTERS, UbridgeError, List) and
  unsorted blocks left behind where the stack's later commits moved the
  last usage elsewhere — ruff I001/F401/RUF022 now enforce these.
* mypy, now baseline-free: cache dict annotations in tc_probe, the
  locators candidate list, the resolved-params None branch in the MCP
  runner, and validate_bpf_syntax keeps the upstream Dict[str, Any]
  return (the strict Optional[str] was wrong for the "valid" bool entry).

The e2e raw-frame probe resolves its compiler via shutil.which (S607)
and zips with strict= (B905); the remaining subprocess call is S603 by
nature and gets a per-file ignore like other test files.

ruff check/format clean, mypy at upstream parity (no new errors),
full unit suite 2167 passed.
2026-10-06 00:21:11 +08:00
YueGuobin
a40503e44b
fix(qemu): release the datapath under the execution lock
QEMU's process monitor calls stop() by itself when the process dies, so an
API stop and a process-death stop run concurrently in the normal case — and
the datapath release sat outside _execute_lock, letting both stops sweep the
adapter TAP map at the same time. Whichever finished first cleared
_kernel_taps under the other's iterator, which died with "dictionary changed
size during iteration" (the monitor's task, so it surfaced as "Task exception
was never retrieved") with the rest of its sweep, the orphan-bridge removal
and uBridge's shutdown skipped.

Keep the previous fix's ordering (process side first, then the devices) and
hold the lock across the release: serialized, the second stop finds uBridge
gone and sweeps nothing. Reproduced deterministically with two concurrent
stop() calls, the first parked mid-sweep — it raised at qemu_vm.py:1659
before the change, and every TAP is swept exactly once after it.
2026-10-06 00:21:09 +08:00
YueGuobin
0b78d313eb
fix(qemu): release the datapath after the process is gone
QEMU is the guest side of every adapter TAP, and uBridge's tap delete
refuses a device another process holds open ('Device or resource busy') —
a best-effort delete whose failure is suppressed. QEMU's stop did the
delete first and stopped the process after, so every node stop or project
delete leaked the persistent TAPs: found by inspecting the host after a
project delete (16 taps left behind, carrier down, no holder), and the
held-tap behaviour reproduced against uBridge directly.

Stop the QEMU process first, then remove the TAPs and the per-link kernel
bridges — process side first, then the devices, the same lesson as IOU's
reverse stop order. Verified live: node stop and project delete now leave
no taps and no processes behind, while a restart recreates the TAPs.

The adapter anchor naming also goes through the shared
utils.kernel_anchor helper, like the other node types.
2026-10-06 00:21:09 +08:00
YueGuobin
a1fcd29bce
fix(qemu): build a command line for a headless node
_console_options() fell through the "none" branch and returned None, so
_build_command() blew up on `command.extend(None)`: a QEMU node with
console_type "none" -- offered by the GUI, and what API/MCP-driven labs use
to skip the console entirely -- failed to start with a 500 and a traceback.

Return an empty list, exactly like _aux_options() already does, and cover it
with a test that builds the command line for a headless node. Verified on a
live instance: such a node now starts (204) with `console: null` and no
-serial device in the QEMU command line.
2026-10-06 00:21:08 +08:00
YueGuobin
7d2d8c6747
feat(qemu): anchor QEMU adapters on a persistent TAP
QEMU had no host-side interface at all: every adapter was a local UDP tunnel
into uBridge (-netdev socket plus two UDP ports per adapter), so a link could
only ever be a userspace relay. Each adapter now owns a persistent TAP
(gq{node_id[:8]}e{adapter}p{port}), created by uBridge before QEMU is launched
and handed to this user, and the netdev is that TAP -- the anchor role Docker's
veth host end plays. Links attach to it:

* kernel link (NIOBridge): brctl create/up/addif enslaves the TAP into the
  per-link bridge and capture, markers and tc filters are installed on it,
  verbatim through the shared KernelDatapathMixin -- a TAP and a veth host end
  are interchangeable there (an egress qdisc on either impairs traffic entering
  the node, and PACKET_OUTGOING on either is the node receiving);
* relay link: bridge add_nio_ethernet plus add_nio_udp -- AF_PACKET, never the
  TAP NIO, which would take the fd QEMU itself holds.

The netdev type is fixed at launch, so this is all-or-nothing per VM: on the
TAP datapath every link type attaches to (or detaches from) a running VM,
including switching a link between the two datapaths. Stop un-persists the
TAPs (QEMU's open fd keeps each device alive until it exits) and removes the
per-link bridges; start sweeps leftovers first. A uBridge without the tap
module keeps the previous socket-netdev datapath, so an unupgraded install is
unaffected -- such a node simply cannot carry kernel links.
2026-10-06 00:21:08 +08:00
Mark Paronyan
bee6b3f8b1
refactor: use f-strings in compute tests 2026-10-03 19:44:01 +03:00
Mark Paronyan
306d8fbde4
refactor: remove unused imports 2026-10-03 19:43:58 +03:00
Mark Paronyan
6d16d923c7 style: sort imports in compute and controller tests 2026-10-02 12:43:26 +03:00
Mark Paronyan
06e98f8b55
style: drop redundant utf-8 coding declarations 2026-10-01 19:57:40 +03:00
Mark Paronyan
a0450b6bce
refactor: ruff autofixes 2026-09-27 15:55:10 +03:00
Cristi
da16dc9874 fix(qemu): Fixed missing image replacement for linked-clone nodes 2026-09-18 23:58:49 +03:00
grossmj
2bf3eff143
Remove debug message in tests 2025-05-13 20:47:38 +02:00
grossmj
2b96e54629
Fix tests and require Qemu version >= 2.4 2025-04-30 13:38:52 +07:00
grossmj
d9dcc2738d
Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	CHANGELOG
#	Dockerfile
#	README.md
#	gns3server/appliances/infix.gns3a
#	gns3server/compute/docker/docker_vm.py
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/controller/gns3vm/virtualbox_gns3_vm.py
#	gns3server/crash_report.py
#	gns3server/static/web-ui/index.html
#	gns3server/static/web-ui/main.9bcf455e62558dedfd48.js
#	gns3server/version.py
2025-04-21 20:13:31 +07:00
grossmj
2b7eb91694
Add edk2-stable202502 UEFI firmwares and try to first use firmwares from the ovmf package if installed. Fixes #2494 2025-04-18 17:13:29 +07:00
grossmj
b37db57bb9
Fix tests 2024-11-17 15:00:13 +10:00
grossmj
ec9dbd20b8
Merge branch '2.2' into 3.0
# Conflicts:
#	gns3server/compute/base_node.py
#	gns3server/compute/docker/docker_vm.py
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/crash_report.py
#	gns3server/version.py
2024-11-07 23:18:42 +10:00
grossmj
52a7696618
Fix tests 2024-11-07 14:48:57 +10:00
grossmj
7bf17392dd
Do not use "ide" if there is a disk image and no interface type has been explicitly configured. 2024-11-07 14:37:26 +10:00
grossmj
8c7c17b889
Upgrade dependencies and fix Pydantic warnings 2024-10-31 17:09:52 +10:00
grossmj
6ed18c561f
Merge branch '2.2' into 3.0
# Conflicts:
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/config_samples/gns3_server.conf
#	gns3server/controller/export_project.py
#	gns3server/controller/project.py
#	gns3server/controller/snapshot.py
#	gns3server/handlers/api/controller/project_handler.py
#	tests/controller/test_export_project.py
#	tests/controller/test_import_project.py
#	tests/controller/test_snapshot.py
2024-07-07 19:04:50 +02:00
grossmj
b194e48649
Forbid unsafe Qemu additional options 2024-07-06 12:24:22 +02:00
grossmj
46a9fdb61d
Remove event_loop 2024-05-09 17:53:02 +07:00
grossmj
33499be425
Fix warnings in tests 2024-05-09 17:11:12 +07:00
grossmj
c076275bd0
Fix test create image with not supported characters by filesystem. Fixes #2375 2024-05-09 16:01:55 +07:00
grossmj
d680bbbc77 Merge branch '2.2' into 3.0
# Conflicts:
#	gns3server/schemas/qemu_template.py
#	requirements.txt
#	tests/compute/qemu/test_qemu_vm.py
#	tests/handlers/api/compute/test_qemu.py
2023-10-24 18:31:51 +10:00
grossmj
34ee75e1f1 Fix tests running on Python 3.12 2023-10-23 16:12:12 +10:00
grossmj
425ad845fc Speed up tests 2023-08-19 12:29:25 +10:00
grossmj
3f691b60c9 Merge branch '2.2' into 3.0
# Conflicts:
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/crash_report.py
#	gns3server/schemas/qemu.py
#	gns3server/schemas/qemu_template.py
#	gns3server/static/web-ui/index.html
#	gns3server/static/web-ui/main.1379a5647e8bc6d3e401.js
#	gns3server/version.py
#	tests/compute/qemu/test_qemu_vm.py
2023-07-13 10:09:26 +10:00
grossmj
c181df6935 Only fetch Qemu version once when starting Qemu + only add speed/duplex for virtio-net-pci with Qemu version >= 2.12 2023-07-06 16:29:55 +10:00
grossmj
6c8c5c1787 Ignore uefi test on Windows 2023-07-04 13:31:21 +10:00
grossmj
f541c03b94 Fix uefi test on Windows 2023-07-04 13:22:18 +10:00
grossmj
0a14a08d54 Use recent OVMF firmware (stable-202305) and use flash drives to configure Qemu command line 2023-07-04 13:06:34 +10:00
grossmj
d5cfb85de5 Backport UEFI boot mode support for Qemu VMs 2023-06-23 11:18:25 +09:30
grossmj
40c265152b Add UEFI boot mode option for Qemu VMs 2023-06-21 17:56:34 +09:30
grossmj
1ff23348d3 Merge branch '2.2' into 3.0
# Conflicts:
#	gns3server/compute/base_node.py
#	gns3server/compute/docker/__init__.py
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/controller/compute.py
#	gns3server/controller/gns3vm/virtualbox_gns3_vm.py
#	gns3server/controller/node.py
#	gns3server/controller/project.py
#	gns3server/crash_report.py
#	gns3server/handlers/api/controller/template_handler.py
#	gns3server/static/web-ui/index.html
#	gns3server/static/web-ui/main.8448c96e4facbe79a613.js
#	gns3server/version.py
#	tests/compute/iou/test_iou_vm.py
#	tests/compute/qemu/test_qemu_vm.py
#	tests/handlers/api/controller/test_template.py
2023-06-20 16:06:53 +09:30
grossmj
71d1aefb65 Test Qemu command with virtio-net-pci adapter 2023-06-03 20:57:23 +09:30
grossmj
267c4cbbbb Merge branch '2.2' into 3.0
# Conflicts:
#	README.md
#	gns3server/compute/base_node.py
#	gns3server/compute/dynamips/__init__.py
#	gns3server/compute/dynamips/hypervisor.py
#	gns3server/compute/qemu/__init__.py
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/controller/__init__.py
#	gns3server/crash_report.py
#	gns3server/handlers/api/controller/node_handler.py
#	gns3server/schemas/qemu_template.py
#	gns3server/static/web-ui/index.html
#	gns3server/static/web-ui/main.11410ae4eaf4d4c08cd0.js
#	gns3server/version.py
#	requirements.txt
2023-03-17 17:44:32 +10:00
grossmj
7968ee7ff2 Check swtpm version and start swtpm before qemu 2023-02-26 20:51:24 +10:00
grossmj
af9b883c08 Fix and adjustments after merge 2023-01-10 12:09:36 +08:00
grossmj
f09594ae4c Merge branch '2.2' into 3.0
# Conflicts:
#	README.md
#	README.rst
#	tests/handlers/api/compute/test_nat.py
2023-01-10 11:52:58 +08:00
grossmj
997e0ad6ee Fix more tests 2023-01-10 11:22:12 +08:00
grossmj
eb3fc1bb3a Fix more tests 2023-01-10 11:09:27 +08:00
grossmj
e9c95ff299 Fix tests 2023-01-10 10:07:26 +08:00
grossmj
27d9063e56 Merge 2.2 2023-01-05 12:38:00 +08:00
grossmj
ae200d9add Add Trusted Platform Module (TPM) support for Qemu VMs 2023-01-04 12:13:19 +08:00
grossmj
86c44cd6e7 Upgrade dev dependencies and fix tests to support pytest-asyncio strict mode 2022-08-24 21:03:16 +02:00
grossmj
2535e5508d Remove Qemu binary requirement 2022-04-19 18:21:39 +07:00
grossmj
e50bed5bee Support delete Qemu disk image from API
Return the real disk image name in the 'hdx_disk_image_backed' property for Qemu VMs
2022-04-14 17:01:54 +07:00