Commit Graph
624 Commits
Author SHA1 Message Date
YueGuobin 61020d3bbc feat(docker): traffic-insight markers on kernel-datapath links via AF_PACKET
Markers no longer disqualify the kernel datapath. They ride the NIO like
on the relay path and attach to the veth host end through uBridge's
marker add_kernel/delete_kernel/enable_kernel commands (AF_PACKET taps,
same BPF/direction/signal semantics as the relay mark filter).

- base_node: marker anchors are now polymorphic (relay bridge name or
  kernel veth host interface); the reconcile loop is datapath-agnostic
  and DockerVM overrides the add/delete/enable primitives to translate
  the commands. Marker-name validation extracted for both add paths.
- docker_vm: kernel NIO connect/update apply markers on the veth anchor;
  link deletion tears markers down explicitly (the veth survives the
  link, unlike a relay bridge whose death drops its filters).
- controller: eligibility and start_marker no longer exclude markers or
  project-level marker definitions; the kernel NIO carries the capture
  node's marker specs.
- NIO creation accepts markers on kernel NIOs (filters still rejected).
2026-10-06 00:21:07 +08:00
YueGuobin f6b869be63 feat(docker): packet capture on kernel-datapath links via AF_PACKET
Replace the capture rejection with wiring to uBridge's capture module:
an AF_PACKET socket bound to the veth host end writes into the same pcap
file the relay path uses (capture start_kernel <if> "<pcap>" /
capture stop_kernel), so the pcap streaming to clients is unchanged.

- capture survives node restarts: _connect_nio re-arms it when the NIO
  is already capturing, mirroring the relay path
- uBridge's kernel capture is a singleton per process: a concurrent
  second capture on the same node surfaces uBridge's EALREADY error
- update the datapath docs: filters and markers remain relay-only,
  capture is now served on both datapaths
2026-10-06 00:21:07 +08:00
YueGuobin 37d7b9121e feat(docker): wire Docker-to-Docker links through a kernel bridge
Eligible links (both endpoints Docker, same compute, no filters/markers,
stopped nodes, no unix-socket NIO containers) are wired as veth pairs
enslaved into a per-link Linux kernel bridge (gns3 + 11 hex of the link
id, filling IFNAMSIZ) instead of the uBridge UDP relay, so frames flow
entirely in the kernel.

- NIOBridge NIO type: the controller decides the datapath in
  UDPLink._prepare and both computes enslave their deterministic gv*/gc*
  veth pair via uBridge's docker/brctl modules
- carrier/suspend = host-end admin state; both endpoints race-safely
  create/delete the bridge (EEXIST/EBUSY/ENOENT tolerated)
- veth adapters can still attach relay (UDP) links on a running node:
  lazy relay bridge over AF_PACKET on the host end (add_nio_ethernet)
- filters/markers/capture are rejected with explicit errors on this
  datapath (they live in the uBridge relay); vendor unix-socket
  containers are excluded; Server.enable_kernel_datapath is the
  kill-switch (default on)
- host-end veths survive container death and are removed on stop plus
  swept by name on the next start (crash/stale recovery)
2026-10-06 00:21:07 +08:00
Mark Paronyan 72fae7a1ce refactor: declare new image sync models in the public API 2026-10-03 19:43:59 +03:00
Mark Paronyan b622ccae7b refactor: declare explicit public API in package __init__ modules 2026-10-03 19:43:59 +03:00
Mark Paronyan 306d8fbde4 refactor: remove unused imports 2026-10-03 19:43:58 +03:00
Mark Paronyan b78672a8ca style: sort imports in the new image management modules 2026-10-02 12:43:26 +03:00
Mark Paronyan 3b993b464f style: sort imports in schemas 2026-10-02 12:43:26 +03:00
Jeremy Grossmann a2cdc14a25 Merge branch '3.1' into image-manager-out-of-sync 2026-10-01 14:44:24 +02:00
Jeremy Grossmann e90be1ebac Merge pull request #2930 from markparonyan/mypy-db-templates-images
fix(typing): resolve the remaining mypy errors (18 modules)
2026-09-29 21:37:14 +02:00
Mark Paronyan 0954723ebd fix(typing): resolve mypy errors in schemas.controller.computes 2026-09-29 22:02:22 +03:00
Mark Paronyan a3cb73da3d fix(typing): resolve mypy errors in api.routes.controller.dependencies.authentication 2026-09-29 21:48:33 +03:00
Jeremy Grossmann 1838ad7f05 Merge branch '3.1' into mypy-schemas-plus 2026-09-29 13:41:11 +02:00
Jeremy Grossmann eace280c5f Merge branch '3.1' into mypy-schemas 2026-09-29 13:40:00 +02:00
Jeremy Grossmann e6410ed729 Merge branch '3.1' into mypy-schemas-compute 2026-09-29 13:39:01 +02:00
Jeremy Grossmann e8f49877fc Merge branch '3.1' into mypy-api-routes-controller-1 2026-09-29 13:28:33 +02:00
Mark Paronyan 9bad1d92f8 fix(typing): resolve mypy errors in schemas.qemu_disk_image 2026-09-29 09:23:14 +03:00
Mark Paronyan 79862d900a fix(typing): resolve mypy errors in schemas.controller.templates.vmware_templates 2026-09-29 09:23:14 +03:00
Mark Paronyan 3abed40707 fix(typing): resolve mypy errors in schemas.controller.templates.docker_templates 2026-09-29 09:23:13 +03:00
Mark Paronyan edaf9a4377 fix(typing): resolve mypy errors in schemas.controller.templates.virtualbox_templates 2026-09-29 09:23:13 +03:00
Mark Paronyan 94a0f462bf fix(typing): resolve mypy errors in schemas.controller.templates.iou_templates 2026-09-29 09:23:13 +03:00
Mark Paronyan 07dc9b4d7d fix(typing): resolve mypy errors in schemas.controller.settings 2026-09-29 09:23:12 +03:00
Mark Paronyan 01dd09495c fix(typing): resolve mypy errors in schemas.controller.projects 2026-09-29 09:23:12 +03:00
Mark Paronyan a68e57dd2e fix(typing): resolve mypy errors in schemas.controller.nodes 2026-09-29 09:23:12 +03:00
Mark Paronyan 7439e91f5a fix(typing): resolve mypy errors in schemas.controller.appliances 2026-09-29 09:23:12 +03:00
Mark Paronyan 515ae8dcfb fix(typing): resolve mypy errors in schemas.compute.virtualbox_nodes 2026-09-29 09:23:11 +03:00
Mark Paronyan d027146fb9 fix(typing): resolve mypy errors in schemas.compute.vpcs_nodes 2026-09-29 09:23:11 +03:00
Mark Paronyan 87f209f548 fix(typing): resolve mypy errors in schemas.compute.qemu_nodes 2026-09-29 09:23:11 +03:00
Mark Paronyan 898205a33b fix(typing): resolve mypy errors in schemas.compute.vmware_nodes 2026-09-29 09:23:11 +03:00
Mark Paronyan aa7941411f fix(typing): resolve mypy errors in schemas.compute.iou_nodes 2026-09-29 09:23:10 +03:00
Mark Paronyan abb564a2fa fix(typing): resolve mypy errors in schemas.compute.frame_relay_switch_nodes 2026-09-29 09:23:10 +03:00
Mark Paronyan 2b704025e3 fix(typing): resolve mypy errors in schemas.compute.docker_nodes 2026-09-29 09:23:10 +03:00
Mark Paronyan 908a1eb3e6 fix(typing): resolve mypy errors in schemas.compute.cloud_nodes 2026-09-29 09:23:10 +03:00
Mark Paronyan 68d96193ec fix(typing): resolve mypy errors in api.routes.controller.groups 2026-09-29 09:23:07 +03:00
Mark Paronyan a3c4c95431 fix(typing): resolve mypy errors in schemas.compute.nat_nodes 2026-09-29 08:52:44 +03:00
Mark Paronyan 672ef2e3ce fix(typing): resolve mypy errors in schemas.compute.dynamips_nodes 2026-09-29 08:52:43 +03:00
Mark Paronyan 094fcf2747 fix(typing): resolve mypy errors in schemas.compute.atm_switch_nodes 2026-09-29 08:52:43 +03:00
Cristi 42d30651da Merge origin/3.1 into image-manager-out-of-sync
Preserve image reconciliation, locking, and regression coverage while integrating upstream release, dependency, typing, and formatting updates. Align feature code with the new Ruff checks.
2026-09-28 23:16:24 +03:00
Mark Paronyan 6f3e5cfc2b fix(typing): resolve mypy errors in schemas.controller.templates.dynamips_templates 2026-09-28 21:47:56 +03:00
Cristi 046dff4d08 fix(images): reconcile the image inventory with filesystem changes 2026-09-28 16:01:36 +03:00
Mark Paronyan a0450b6bce refactor: ruff autofixes 2026-09-27 15:55:10 +03:00
Cristi da16dc9874 fix(qemu): Fixed missing image replacement for linked-clone nodes 2026-09-18 23:58:49 +03:00
Cristi 59a367b88d Allow projects containing missing images to open in a degraded state instead of failing to load 2026-09-18 15:06:17 +03:00
YueGuobin be62e8c022 feat: keep Docker images on computes consistent with the controller host
Only relying on the image name lets a moved tag (e.g. a newer :latest)
silently serve stale content from a compute that already has an image
under the same name. When creating a Docker node, the controller now
pins the image id (Id from the Docker daemon on the controller host)
into the create payload. A compute holding a different image under the
same tag reports the image as missing, which routes it through the
image sync added by the previous commit and re-aligns the tag.

No new template fields or database changes: the controller host daemon
remains the source of truth and the pin is resolved per creation. When
the image is not available on the controller host the pin is omitted
and behavior is unchanged (the compute pulls from the repository).
2026-09-14 00:55:18 +08:00
YueGuobin 8b3aafbbdc feat: IOU-style startup-config for IOL Docker nodes
Templates reference a config file with the GNS3_IOL_STARTUP_CONFIG
environment knob; the controller materializes the file content into
startup_config_content on node creation (sent once, knob consumed —
the same pattern as the IOU startup_config mapping). The compute builds
the content into the node's nvram_<app id> at the next start using the
IOU nvram_import utility (IOL and IOU share the nvram container format,
verified against iol-xe 17.18.02): valid config at boot, no setup
dialog, %h hostname substitution, hostname rewrite on rename.

Semantics verified against the runner: IOL boots from NVRAM whenever it
holds a config, so a plain stop/start never re-applies the startup
config and 'write memory' survives restarts; an explicit content edit
(PUT) is re-applied on the next start and wins over the saved config,
like IOU.
2026-09-05 21:54:52 +08:00
YueGuobin 0119373690 feat: allocate IOL Docker application IDs from a pool disjoint from IOU
IOL interface MACs derive from the application ID (aabb.cc{app}{iface}),
so ids must be unique across opened projects sharing computes — the same
reason IOU has its allocator. IOL Docker nodes draw from the upper half
(512-1022, netiomux's fixed peer is 1023) so the two node types can
neither collide nor starve each other; IOU behavior is unchanged.

The controller sniffs the same GNS3_IOL_RUNNER environment marker the
compute uses to select IOLDockerVM (both the nested-properties and
template/top-level-kwarg shapes), stores the id in node properties like
IOU does, and passes it through the Docker create payload. Without an
allocation the compute falls back to a stable node-derived id in the
same upper range.
2026-09-05 21:54:52 +08:00
YueGuobin 317f4d2406 docs: clarify the default admin credentials only seed the database
The default_admin_username/password descriptions now state that the
values are applied when the controller database is created and only
take effect again after it is re-created (resetting the account).
2026-08-24 01:12:48 +08:00
YueGuobin 27ffc1e7a6 fix: type optional configuration fields as Optional
secrets_dir, certfile and certkey were typed FilePath/DirectoryPath
with a None default, and several str fields (jwt_secret_key,
iourc_path, resources_path, default_nat_interface, vboxmanage_path,
vmrun_path) had the same mismatch. Serializing the unset values
emitted PydanticSerializationUnexpectedValue warnings on every
GET /v3/settings (two lines per request with SSL disabled).
2026-08-24 00:42:48 +08:00
YueGuobin b7dbf45a90 docs: add field descriptions to the settings schemas
Document all 70 configuration fields with pydantic Field
descriptions, ported from the config sample comments and verified
against the actual consumers (allow_remote_console and local had
no documentation anywhere). The descriptions flow into the
OpenAPI schema of GET /v3/settings, giving the Web UI tooltips,
defaults and validation bounds from a single source. Response
models re-declare six path fields as plain strings, which drops
the inherited description — restore them explicitly.

Sync the config sample: document local and allow_remote_console,
drop hardware_virtualization_check which no longer exists in the
schema.
2026-08-24 00:35:26 +08:00
YueGuobin e0962a59ed feat: add GET/PUT /v3/settings server settings API
GET returns all settings sections except the deprecated
VirtualBox/VMware ones (Server.Audit privilege); secrets are
masked and Controller.jwt_secret_key is excluded entirely.
PUT applies a partial update (Server.Modify privilege): masked
or empty secrets mean unchanged, null removes the option, the
response carries the new values plus restart_required, and a
settings.updated notification is emitted. New
Server.Audit/Server.Modify privileges are seeded into the
Administrator role at table creation.
2026-08-23 18:54:31 +08:00