624 Commits

Author SHA1 Message Date
YueGuobin
61020d3bbc
feat(docker): traffic-insight markers on kernel-datapath links via AF_PACKET
Markers no longer disqualify the kernel datapath. They ride the NIO like
on the relay path and attach to the veth host end through uBridge's
marker add_kernel/delete_kernel/enable_kernel commands (AF_PACKET taps,
same BPF/direction/signal semantics as the relay mark filter).

- base_node: marker anchors are now polymorphic (relay bridge name or
  kernel veth host interface); the reconcile loop is datapath-agnostic
  and DockerVM overrides the add/delete/enable primitives to translate
  the commands. Marker-name validation extracted for both add paths.
- docker_vm: kernel NIO connect/update apply markers on the veth anchor;
  link deletion tears markers down explicitly (the veth survives the
  link, unlike a relay bridge whose death drops its filters).
- controller: eligibility and start_marker no longer exclude markers or
  project-level marker definitions; the kernel NIO carries the capture
  node's marker specs.
- NIO creation accepts markers on kernel NIOs (filters still rejected).
2026-10-06 00:21:07 +08:00
YueGuobin
f6b869be63
feat(docker): packet capture on kernel-datapath links via AF_PACKET
Replace the capture rejection with wiring to uBridge's capture module:
an AF_PACKET socket bound to the veth host end writes into the same pcap
file the relay path uses (capture start_kernel <if> "<pcap>" /
capture stop_kernel), so the pcap streaming to clients is unchanged.

- capture survives node restarts: _connect_nio re-arms it when the NIO
  is already capturing, mirroring the relay path
- uBridge's kernel capture is a singleton per process: a concurrent
  second capture on the same node surfaces uBridge's EALREADY error
- update the datapath docs: filters and markers remain relay-only,
  capture is now served on both datapaths
2026-10-06 00:21:07 +08:00
YueGuobin
37d7b9121e
feat(docker): wire Docker-to-Docker links through a kernel bridge
Eligible links (both endpoints Docker, same compute, no filters/markers,
stopped nodes, no unix-socket NIO containers) are wired as veth pairs
enslaved into a per-link Linux kernel bridge (gns3 + 11 hex of the link
id, filling IFNAMSIZ) instead of the uBridge UDP relay, so frames flow
entirely in the kernel.

- NIOBridge NIO type: the controller decides the datapath in
  UDPLink._prepare and both computes enslave their deterministic gv*/gc*
  veth pair via uBridge's docker/brctl modules
- carrier/suspend = host-end admin state; both endpoints race-safely
  create/delete the bridge (EEXIST/EBUSY/ENOENT tolerated)
- veth adapters can still attach relay (UDP) links on a running node:
  lazy relay bridge over AF_PACKET on the host end (add_nio_ethernet)
- filters/markers/capture are rejected with explicit errors on this
  datapath (they live in the uBridge relay); vendor unix-socket
  containers are excluded; Server.enable_kernel_datapath is the
  kill-switch (default on)
- host-end veths survive container death and are removed on stop plus
  swept by name on the next start (crash/stale recovery)
2026-10-06 00:21:07 +08:00
Mark Paronyan
72fae7a1ce
refactor: declare new image sync models in the public API 2026-10-03 19:43:59 +03:00
Mark Paronyan
b622ccae7b
refactor: declare explicit public API in package __init__ modules 2026-10-03 19:43:59 +03:00
Mark Paronyan
306d8fbde4
refactor: remove unused imports 2026-10-03 19:43:58 +03:00
Mark Paronyan
b78672a8ca style: sort imports in the new image management modules 2026-10-02 12:43:26 +03:00
Mark Paronyan
3b993b464f style: sort imports in schemas 2026-10-02 12:43:26 +03:00
Jeremy Grossmann
a2cdc14a25
Merge branch '3.1' into image-manager-out-of-sync 2026-10-01 14:44:24 +02:00
Jeremy Grossmann
e90be1ebac
Merge pull request #2930 from markparonyan/mypy-db-templates-images
fix(typing): resolve the remaining mypy errors (18 modules)
2026-09-29 21:37:14 +02:00
Mark Paronyan
0954723ebd
fix(typing): resolve mypy errors in schemas.controller.computes 2026-09-29 22:02:22 +03:00
Mark Paronyan
a3cb73da3d
fix(typing): resolve mypy errors in api.routes.controller.dependencies.authentication 2026-09-29 21:48:33 +03:00
Jeremy Grossmann
1838ad7f05
Merge branch '3.1' into mypy-schemas-plus 2026-09-29 13:41:11 +02:00
Jeremy Grossmann
eace280c5f
Merge branch '3.1' into mypy-schemas 2026-09-29 13:40:00 +02:00
Jeremy Grossmann
e6410ed729
Merge branch '3.1' into mypy-schemas-compute 2026-09-29 13:39:01 +02:00
Jeremy Grossmann
e8f49877fc
Merge branch '3.1' into mypy-api-routes-controller-1 2026-09-29 13:28:33 +02:00
Mark Paronyan
9bad1d92f8
fix(typing): resolve mypy errors in schemas.qemu_disk_image 2026-09-29 09:23:14 +03:00
Mark Paronyan
79862d900a
fix(typing): resolve mypy errors in schemas.controller.templates.vmware_templates 2026-09-29 09:23:14 +03:00
Mark Paronyan
3abed40707
fix(typing): resolve mypy errors in schemas.controller.templates.docker_templates 2026-09-29 09:23:13 +03:00
Mark Paronyan
edaf9a4377
fix(typing): resolve mypy errors in schemas.controller.templates.virtualbox_templates 2026-09-29 09:23:13 +03:00
Mark Paronyan
94a0f462bf
fix(typing): resolve mypy errors in schemas.controller.templates.iou_templates 2026-09-29 09:23:13 +03:00
Mark Paronyan
07dc9b4d7d
fix(typing): resolve mypy errors in schemas.controller.settings 2026-09-29 09:23:12 +03:00
Mark Paronyan
01dd09495c
fix(typing): resolve mypy errors in schemas.controller.projects 2026-09-29 09:23:12 +03:00
Mark Paronyan
a68e57dd2e
fix(typing): resolve mypy errors in schemas.controller.nodes 2026-09-29 09:23:12 +03:00
Mark Paronyan
7439e91f5a
fix(typing): resolve mypy errors in schemas.controller.appliances 2026-09-29 09:23:12 +03:00
Mark Paronyan
515ae8dcfb
fix(typing): resolve mypy errors in schemas.compute.virtualbox_nodes 2026-09-29 09:23:11 +03:00
Mark Paronyan
d027146fb9
fix(typing): resolve mypy errors in schemas.compute.vpcs_nodes 2026-09-29 09:23:11 +03:00
Mark Paronyan
87f209f548
fix(typing): resolve mypy errors in schemas.compute.qemu_nodes 2026-09-29 09:23:11 +03:00
Mark Paronyan
898205a33b
fix(typing): resolve mypy errors in schemas.compute.vmware_nodes 2026-09-29 09:23:11 +03:00
Mark Paronyan
aa7941411f
fix(typing): resolve mypy errors in schemas.compute.iou_nodes 2026-09-29 09:23:10 +03:00
Mark Paronyan
abb564a2fa
fix(typing): resolve mypy errors in schemas.compute.frame_relay_switch_nodes 2026-09-29 09:23:10 +03:00
Mark Paronyan
2b704025e3
fix(typing): resolve mypy errors in schemas.compute.docker_nodes 2026-09-29 09:23:10 +03:00
Mark Paronyan
908a1eb3e6
fix(typing): resolve mypy errors in schemas.compute.cloud_nodes 2026-09-29 09:23:10 +03:00
Mark Paronyan
68d96193ec
fix(typing): resolve mypy errors in api.routes.controller.groups 2026-09-29 09:23:07 +03:00
Mark Paronyan
a3c4c95431
fix(typing): resolve mypy errors in schemas.compute.nat_nodes 2026-09-29 08:52:44 +03:00
Mark Paronyan
672ef2e3ce
fix(typing): resolve mypy errors in schemas.compute.dynamips_nodes 2026-09-29 08:52:43 +03:00
Mark Paronyan
094fcf2747
fix(typing): resolve mypy errors in schemas.compute.atm_switch_nodes 2026-09-29 08:52:43 +03:00
Cristi
42d30651da Merge origin/3.1 into image-manager-out-of-sync
Preserve image reconciliation, locking, and regression coverage while integrating upstream release, dependency, typing, and formatting updates. Align feature code with the new Ruff checks.
2026-09-28 23:16:24 +03:00
Mark Paronyan
6f3e5cfc2b
fix(typing): resolve mypy errors in schemas.controller.templates.dynamips_templates 2026-09-28 21:47:56 +03:00
Cristi
046dff4d08 fix(images): reconcile the image inventory with filesystem changes 2026-09-28 16:01:36 +03:00
Mark Paronyan
a0450b6bce
refactor: ruff autofixes 2026-09-27 15:55:10 +03:00
Cristi
da16dc9874 fix(qemu): Fixed missing image replacement for linked-clone nodes 2026-09-18 23:58:49 +03:00
Cristi
59a367b88d Allow projects containing missing images to open in a degraded state instead of failing to load 2026-09-18 15:06:17 +03:00
YueGuobin
be62e8c022
feat: keep Docker images on computes consistent with the controller host
Only relying on the image name lets a moved tag (e.g. a newer :latest)
silently serve stale content from a compute that already has an image
under the same name. When creating a Docker node, the controller now
pins the image id (Id from the Docker daemon on the controller host)
into the create payload. A compute holding a different image under the
same tag reports the image as missing, which routes it through the
image sync added by the previous commit and re-aligns the tag.

No new template fields or database changes: the controller host daemon
remains the source of truth and the pin is resolved per creation. When
the image is not available on the controller host the pin is omitted
and behavior is unchanged (the compute pulls from the repository).
2026-09-14 00:55:18 +08:00
YueGuobin
8b3aafbbdc
feat: IOU-style startup-config for IOL Docker nodes
Templates reference a config file with the GNS3_IOL_STARTUP_CONFIG
environment knob; the controller materializes the file content into
startup_config_content on node creation (sent once, knob consumed —
the same pattern as the IOU startup_config mapping). The compute builds
the content into the node's nvram_<app id> at the next start using the
IOU nvram_import utility (IOL and IOU share the nvram container format,
verified against iol-xe 17.18.02): valid config at boot, no setup
dialog, %h hostname substitution, hostname rewrite on rename.

Semantics verified against the runner: IOL boots from NVRAM whenever it
holds a config, so a plain stop/start never re-applies the startup
config and 'write memory' survives restarts; an explicit content edit
(PUT) is re-applied on the next start and wins over the saved config,
like IOU.
2026-09-05 21:54:52 +08:00
YueGuobin
0119373690
feat: allocate IOL Docker application IDs from a pool disjoint from IOU
IOL interface MACs derive from the application ID (aabb.cc{app}{iface}),
so ids must be unique across opened projects sharing computes — the same
reason IOU has its allocator. IOL Docker nodes draw from the upper half
(512-1022, netiomux's fixed peer is 1023) so the two node types can
neither collide nor starve each other; IOU behavior is unchanged.

The controller sniffs the same GNS3_IOL_RUNNER environment marker the
compute uses to select IOLDockerVM (both the nested-properties and
template/top-level-kwarg shapes), stores the id in node properties like
IOU does, and passes it through the Docker create payload. Without an
allocation the compute falls back to a stable node-derived id in the
same upper range.
2026-09-05 21:54:52 +08:00
YueGuobin
317f4d2406
docs: clarify the default admin credentials only seed the database
The default_admin_username/password descriptions now state that the
values are applied when the controller database is created and only
take effect again after it is re-created (resetting the account).
2026-08-24 01:12:48 +08:00
YueGuobin
27ffc1e7a6
fix: type optional configuration fields as Optional
secrets_dir, certfile and certkey were typed FilePath/DirectoryPath
with a None default, and several str fields (jwt_secret_key,
iourc_path, resources_path, default_nat_interface, vboxmanage_path,
vmrun_path) had the same mismatch. Serializing the unset values
emitted PydanticSerializationUnexpectedValue warnings on every
GET /v3/settings (two lines per request with SSL disabled).
2026-08-24 00:42:48 +08:00
YueGuobin
b7dbf45a90
docs: add field descriptions to the settings schemas
Document all 70 configuration fields with pydantic Field
descriptions, ported from the config sample comments and verified
against the actual consumers (allow_remote_console and local had
no documentation anywhere). The descriptions flow into the
OpenAPI schema of GET /v3/settings, giving the Web UI tooltips,
defaults and validation bounds from a single source. Response
models re-declare six path fields as plain strings, which drops
the inherited description — restore them explicitly.

Sync the config sample: document local and allow_remote_console,
drop hardware_virtualization_check which no longer exists in the
schema.
2026-08-24 00:35:26 +08:00
YueGuobin
e0962a59ed
feat: add GET/PUT /v3/settings server settings API
GET returns all settings sections except the deprecated
VirtualBox/VMware ones (Server.Audit privilege); secrets are
masked and Controller.jwt_secret_key is excluded entirely.
PUT applies a partial update (Server.Modify privilege): masked
or empty secrets mean unchanged, null removes the option, the
response carries the new values plus restart_required, and a
settings.updated notification is emitted. New
Server.Audit/Server.Modify privileges are seeded into the
Administrator role at table creation.
2026-08-23 18:54:31 +08:00