Commit Graph
45 Commits
Author SHA1 Message Date
YueGuobin de986e890e feat: name nodes and templates in Docker image delete conflicts
The compute translates the container ids in a daemon delete conflict
into the GNS3 node and project behind each container. Leftover
containers from closed projects and containers created outside GNS3
are named as such.

The controller lists the templates referencing the image in the
conflict error, matched through the canonical image name. After a
successful deletion it emits a docker.image.deleted notification with
those templates: their nodes can no longer start.
2026-11-06 22:05:52 +08:00
YueGuobin 04a8ba06a8 fix: start pull events when the pull begins and reject unsafe image names
The pull started event was emitted by the route before the pull took
its per compute lock, so a request queued behind another pull claimed
to be running while it was still waiting. The pull now reports the
started state itself, once it owns the lock.

Deleting an image interpolates the name into the daemon API path. The
pattern on the path parameter limits it to the characters of an image
reference and validate_docker_image_name also rejects "..", so a name
cannot escape that path. Applied on both the controller and compute
routes.
2026-11-06 15:42:51 +08:00
YueGuobin d5d25eba06 fix: stop reporting a missing image as a network problem
A 404 from the registry means the image name is wrong or the registry
needs authentication, but the pull wrapped every error with "please
check your Internet connection". Say what actually failed and keep the
network advice for the other errors.
2026-11-06 15:23:34 +08:00
YueGuobin 8c98142d26 feat: report Docker pull progress and add an idle timeout
A pull now reports docker.image.pull notifications on the controller
stream: started, throttled downloading events with the layer counts
and the latest daemon message, then completed or failed. An optional
progress_id is echoed in every event so a client can correlate them
with its own request. Image operations belong to a compute, not to a
project, so the events go to the controller stream and reach clients
that have no project open.

The pull also has an idle socket read timeout of 60 seconds. Without
it an unreachable registry left the pull, the compute request and the
client request hanging forever with no feedback at all. The timeout
is idle time, not total time: the daemon streams progress during the
whole pull, so a minute without data means the registry is gone. When
it fires, the daemon connection closes and Docker cancels the pull.

Docker.http_query now accepts an aiohttp ClientTimeout object so a
caller can ask for a policy finer than a total timeout.
2026-11-06 15:03:34 +08:00
YueGuobin 54cfcef6e2 fix: send the docker force query parameter as a string
The aiohttp URL builder only accepts strings, ints and floats as query
values and raises a TypeError on a Python bool, so deleting an image
answered a 500. Send true/false like the controller does.
2026-11-06 14:31:26 +08:00
YueGuobin c1e03194d3 api: list Docker image details and delete images
GET /computes/{compute_id}/docker/images now returns the daemon image
id, size and creation date in addition to the tag, so a UI can show
and compare images instead of bare names. The fields are optional to
keep the response compatible with older computes.

Add DELETE /computes/{compute_id}/docker/images/{image} which removes
the image from the compute Docker daemon. The image name is passed as
a path so names with a slash or a tag separator work. A daemon that
refuses because a container uses the image answers a conflict; the
force query parameter removes it anyway.

The compute also drops the controller digest to local id translation
recorded when the image was synced, since it no longer matches
anything on the daemon.
2026-11-06 14:22:22 +08:00
Mark Paronyan b7e6c02647 refactor: ruff autofixes 2026-10-17 11:43:36 +03:00
YueGuobin 90917c4a2c fix(docker): accept the pinned image id when the stores report different ids
Docker ids are not comparable across image stores: for the same content a
classic store reports the config digest while a containerd store (or an
image loaded from an OCI archive) reports the manifest digest. The
controller pins the id of its own daemon, so a compute whose store reports
another id for that content answered ImageMissingError forever: every node
creation streamed the image again (save/load round trip) without ever
converging, then failed with "The image 'x' is missing" after the six
retries. Docker 29 makes this easy to hit, since the id a daemon reports
now depends on how the image entered it.

The load endpoint now takes the tag and the digest pinned by the
controller, and the compute records how its own daemon identifies the
image it just received. Node creation accepts the pin through that pair,
and only while the tag still resolves to the synced content: a tag moved
to some other image, or an image the controller never pushed here, still
counts as missing. The pair lives in the compute process, so a restarted
compute syncs once more.

Computes that predate the new endpoint parameters keep the previous
behaviour, as do controllers that do not send them.
2026-10-13 17:10:17 +08:00
Mark Paronyan bee6b3f8b1 refactor: use f-strings in compute tests 2026-10-03 19:44:01 +03:00
Mark Paronyan 6d16d923c7 style: sort imports in compute and controller tests 2026-10-02 12:43:26 +03:00
Mark Paronyan a0450b6bce refactor: ruff autofixes 2026-09-27 15:55:10 +03:00
YueGuobin c477812332 feat: sync Docker images from the controller to remote computes
When a Docker node is created on a remote compute whose Docker daemon
does not have the image, the compute now raises ImageMissingError
instead of blindly pulling from the Docker repository. The controller
exports the image from the Docker daemon on its host (docker save
stream) and streams it to the compute which loads it, so locally built
or docker-loaded images work across computes. When the image is not
available on the controller host either, the compute is asked to pull
it from the Docker repository as a fallback.

- add a POST /docker/images/load compute endpoint that streams a
  docker save tar into the Docker daemon
- let Docker.http_query pass raw (non-dict) request bodies through so
  the tar can be streamed to the daemon
- drop the inline pull from DockerVM.create() and the now unused
  DockerVM.pull_image wrapper
2026-09-14 00:36:54 +08:00
YueGuobin 9604c85fda docker: harden the shm/devices/extra_configs/masking work (code review)
Nine fixes from a review of the docker-shm-devices diff:

* GNS3_STOP_TIMEOUT >300 s aborted at the manager's default HTTP timeout
  before Docker finished the stop — the stop query now gets a timeout
  with a margin over the grace period.
* Overlapping bind targets (GNS3_MASK_UDEV + GNS3_MASK_SYSTEMD on the
  same unit, a unit named twice, an extra_configs target equal to a
  masked unit) made Docker reject the create with 'Duplicate mount
  point' — Mounts are deduplicated by target.
* ExtraConfig.target now carries a pydantic validator (absolute file
  path, no '..'), so bad targets 422 at template-save time instead of
  failing at node-create time after a multi-GB image pull; directory
  forms ('/', '/etc/') are also rejected by the runtime guard instead
  of raising IsADirectoryError (raw 500).
* _check_host_readiness skipped every remaining check when one
  /proc/sys key was unreadable (mid-loop return) — now continues.
* The base-class GNS3_* env parser strips trailing commas like the
  vendor parser, so 'GNS3_MASK_UDEV=1,' composed from a list still
  activates.
* Vendor env knobs are re-parsed on every create(), so a PUT to the
  node's environment takes effect on the next (re)create.
* The graceful SIGTERM stop is now limited to the explicit user stop
  route; delete/update/close/crash-cleanup keep the immediate kill
  (those paths force-delete or recreate the container right after).
* An extra_configs target beneath a persisted volume is shadowed by the
  volume bind — warn at create time.
2026-08-15 00:52:55 +08:00
YueGuobin 2bee34031c docker: warn about low inotify/file-max and missing FUSE at connect
Add a read-only _check_host_readiness() that runs once after the Docker
daemon connection is established. It reads /proc/sys inotify/file-max
limits and /proc/filesystems (for FUSE), and logs a warning with the exact
commands to fix when they are too low for heavy containers -- XRd wants
~4000 inotify instances per node against a stock default of 128.

The server runs unprivileged (only the setuid ubridge helper has root), so
it can only check, not set; the warning tells the admin exactly what to
raise once. Stays silent when the limits are already sufficient.
2026-08-13 22:39:48 +08:00
Cristi 2abc7e09a3 Adds controller and compute API support for explicitly pulling or updating Docker images. 2026-07-24 16:40:20 +03:00
grossmj c5b9f379fe Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	Dockerfile
#	dev-requirements.txt
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/version.py
#	requirements.txt
2026-02-20 16:28:53 +08:00
grossmj 715b7032bc Fix tests 2026-02-20 15:51:49 +08:00
grossmj 1d45df6613 Upgrade dependencies 2026-01-24 21:38:58 +08:00
grossmj 60c10e7ecc Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	.github/workflows/testing.yml
#	CHANGELOG
#	README.md
#	gns3server/appliances/infix.gns3a
#	gns3server/compute/docker/__init__.py
#	gns3server/compute/docker/docker_vm.py
#	gns3server/crash_report.py
#	gns3server/version.py
#	gns3server/web/web_server.py
#	setup.py
#	tests/controller/test_project.py
2025-11-29 11:42:02 +10:00
grossmj f2aaf5ab38 Docker API version requirements and process to handle older daemons 2025-11-17 22:26:18 +10:00
grossmj d3860ba84b Merge branch '2.2' into 3.0
# Conflicts:
#	CHANGELOG
#	dev-requirements.txt
#	gns3server/compute/qemu/__init__.py
#	gns3server/compute/vmware/__init__.py
#	gns3server/controller/snapshot.py
#	gns3server/crash_report.py
#	gns3server/handlers/api/compute/atm_switch_handler.py
#	gns3server/run.py
#	gns3server/static/web-ui/26.77d4bfd104f37c42e028.js
#	gns3server/static/web-ui/index.html
#	gns3server/static/web-ui/runtime.415291667f70565cd8ef.js
#	gns3server/utils/__init__.py
#	gns3server/utils/images.py
#	gns3server/utils/interfaces.py
#	gns3server/version.py
#	gns3server/web/web_server.py
#	pytest.ini
#	requirements.txt
#	scripts/update-bundled-web-ui.sh
#	setup.py
#	tests/api/routes/compute/test_dynamips_nodes.py
#	tests/compute/builtin/nodes/test_cloud.py
#	tests/compute/docker/test_docker.py
#	tests/compute/docker/test_docker_vm.py
#	tests/compute/dynamips/test_dynamips_manager.py
#	tests/compute/dynamips/test_dynamips_router.py
#	tests/compute/iou/test_iou_vm.py
#	tests/compute/qemu/test_qcow2.py
#	tests/compute/qemu/test_qemu_manager.py
#	tests/compute/qemu/test_qemu_vm.py
#	tests/compute/test_base_node.py
#	tests/compute/test_manager.py
#	tests/compute/test_project.py
#	tests/compute/traceng/test_traceng_vm.py
#	tests/compute/virtualbox/test_virtualbox_manager.py
#	tests/compute/virtualbox/test_virtualbox_vm.py
#	tests/compute/vmware/test_vmware_manager.py
#	tests/compute/vmware/test_vmware_vm.py
#	tests/compute/vpcs/test_vpcs_vm.py
#	tests/conftest.py
#	tests/controller/gns3vm/test_remote_gns3_vm.py
#	tests/controller/gns3vm/test_virtualbox_gns3_vm.py
#	tests/controller/gns3vm/test_vmware_gns3_vm.py
#	tests/controller/test_export_project.py
#	tests/controller/test_gns3vm.py
#	tests/controller/test_import_project.py
#	tests/test_config.py
#	tests/utils/test_asyncio.py
#	tests/utils/test_images.py
#	tests/web/test_response.py
2024-05-16 17:53:39 +07:00
grossmj 46a9fdb61d Remove event_loop 2024-05-09 17:53:02 +07:00
grossmj 33499be425 Fix warnings in tests 2024-05-09 17:11:12 +07:00
grossmj 2609849507 Fix tests after updating error message when busybox is not installed. 2024-04-23 18:00:37 +07:00
grossmj 1a53c9aabf Backport from v3: install Docker resources in a writable location at runtime. 2024-02-14 16:13:45 +08:00
grossmj 09ff807055 Install Docker resources in writable location 2023-11-13 11:23:26 +10:00
grossmj 1ce0c13fc9 Better mocking in Docker tests 2023-08-18 12:20:54 +10:00
grossmj 2f2aabeb5a Fix tests when running Python 3.7 2023-08-11 17:58:00 +10:00
grossmj d9eb61efc4 Fix tests with asyncio_patch 2023-08-11 17:37:11 +10:00
grossmj 1fd8444d22 Add tests for install_busybox() 2023-08-11 17:32:05 +10:00
grossmj 86c44cd6e7 Upgrade dev dependencies and fix tests to support pytest-asyncio strict mode 2022-08-24 21:03:16 +02:00
grossmj eb3cb8a41f Migration to FastAPI 2020-10-02 16:07:50 +09:30
grossmj d3ea67da24 Refactor tests
* Use pytest-aiohttp
* Use the async def / await syntax.
* Fix tests to run with Python 3.8
2020-06-16 13:59:03 +09:30
grossmj 76af98404a Drop Python 3.4 and switch to async / await syntax for asyncio. Fixes #1425 2018-10-15 17:05:49 +07:00
ziajka d9a0ec9ff3 Replace asyncio.async with ensure_future because of deprecation, Fixes: #1269 2018-01-24 11:11:53 +01:00
ziajka e0f0adf3c8 Added preferred Docker API version. Fixes #2136 2017-07-06 11:24:55 +02:00
Julien Duponchelle 2da581139c Lower docker requirements in tests also 2017-04-24 18:49:47 +02:00
Julien Duponchelle effbe59414 Lock docker API to 1.27
Fix https://github.com/GNS3/gns3-gui/issues/1994
2017-04-24 17:51:55 +02:00
Julien Duponchelle 132bc87081 Dissallow parallel pull of docker images
Fix #962
2017-03-27 20:46:25 +02:00
Julien Duponchelle 34f5a6f82c Fix Session is closed when listing docker images
Fix #955
2017-03-20 23:50:31 +01:00
Julien Duponchelle 78c154e376 Cleanup docker source code 2017-03-20 20:30:59 +01:00
Julien Duponchelle a571b1a2f8 Use aiohttp session for docker queries 2017-03-20 19:46:50 +01:00
Julien Duponchelle a62b791c68 Fix TypeError: http_query() got an unexpected keyword argument 'timeout'
Fix #947
2017-03-20 17:06:00 +01:00
Julien Duponchelle a3b243c501 Merge branch '1.5' into 2.0 2016-05-02 17:08:15 +02:00
Julien Duponchelle 58d4a529d4 Rename hypervisor to compute
Fix #487
2016-04-15 17:59:59 +02:00