34 Commits

Author SHA1 Message Date
YueGuobin
c477812332
feat: sync Docker images from the controller to remote computes
When a Docker node is created on a remote compute whose Docker daemon
does not have the image, the compute now raises ImageMissingError
instead of blindly pulling from the Docker repository. The controller
exports the image from the Docker daemon on its host (docker save
stream) and streams it to the compute which loads it, so locally built
or docker-loaded images work across computes. When the image is not
available on the controller host either, the compute is asked to pull
it from the Docker repository as a fallback.

- add a POST /docker/images/load compute endpoint that streams a
  docker save tar into the Docker daemon
- let Docker.http_query pass raw (non-dict) request bodies through so
  the tar can be streamed to the daemon
- drop the inline pull from DockerVM.create() and the now unused
  DockerVM.pull_image wrapper
2026-09-14 00:36:54 +08:00
YueGuobin
9604c85fda
docker: harden the shm/devices/extra_configs/masking work (code review)
Nine fixes from a review of the docker-shm-devices diff:

* GNS3_STOP_TIMEOUT >300 s aborted at the manager's default HTTP timeout
  before Docker finished the stop — the stop query now gets a timeout
  with a margin over the grace period.
* Overlapping bind targets (GNS3_MASK_UDEV + GNS3_MASK_SYSTEMD on the
  same unit, a unit named twice, an extra_configs target equal to a
  masked unit) made Docker reject the create with 'Duplicate mount
  point' — Mounts are deduplicated by target.
* ExtraConfig.target now carries a pydantic validator (absolute file
  path, no '..'), so bad targets 422 at template-save time instead of
  failing at node-create time after a multi-GB image pull; directory
  forms ('/', '/etc/') are also rejected by the runtime guard instead
  of raising IsADirectoryError (raw 500).
* _check_host_readiness skipped every remaining check when one
  /proc/sys key was unreadable (mid-loop return) — now continues.
* The base-class GNS3_* env parser strips trailing commas like the
  vendor parser, so 'GNS3_MASK_UDEV=1,' composed from a list still
  activates.
* Vendor env knobs are re-parsed on every create(), so a PUT to the
  node's environment takes effect on the next (re)create.
* The graceful SIGTERM stop is now limited to the explicit user stop
  route; delete/update/close/crash-cleanup keep the immediate kill
  (those paths force-delete or recreate the container right after).
* An extra_configs target beneath a persisted volume is shadowed by the
  volume bind — warn at create time.
2026-08-15 00:52:55 +08:00
YueGuobin
2bee34031c
docker: warn about low inotify/file-max and missing FUSE at connect
Add a read-only _check_host_readiness() that runs once after the Docker
daemon connection is established. It reads /proc/sys inotify/file-max
limits and /proc/filesystems (for FUSE), and logs a warning with the exact
commands to fix when they are too low for heavy containers -- XRd wants
~4000 inotify instances per node against a stock default of 128.

The server runs unprivileged (only the setuid ubridge helper has root), so
it can only check, not set; the warning tells the admin exactly what to
raise once. Stays silent when the limits are already sufficient.
2026-08-13 22:39:48 +08:00
Cristi
2abc7e09a3 Adds controller and compute API support for explicitly pulling or updating Docker images. 2026-07-24 16:40:20 +03:00
grossmj
c5b9f379fe
Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	Dockerfile
#	dev-requirements.txt
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/version.py
#	requirements.txt
2026-02-20 16:28:53 +08:00
grossmj
715b7032bc
Fix tests 2026-02-20 15:51:49 +08:00
grossmj
1d45df6613
Upgrade dependencies 2026-01-24 21:38:58 +08:00
grossmj
60c10e7ecc
Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	.github/workflows/testing.yml
#	CHANGELOG
#	README.md
#	gns3server/appliances/infix.gns3a
#	gns3server/compute/docker/__init__.py
#	gns3server/compute/docker/docker_vm.py
#	gns3server/crash_report.py
#	gns3server/version.py
#	gns3server/web/web_server.py
#	setup.py
#	tests/controller/test_project.py
2025-11-29 11:42:02 +10:00
grossmj
f2aaf5ab38
Docker API version requirements and process to handle older daemons 2025-11-17 22:26:18 +10:00
grossmj
d3860ba84b
Merge branch '2.2' into 3.0
# Conflicts:
#	CHANGELOG
#	dev-requirements.txt
#	gns3server/compute/qemu/__init__.py
#	gns3server/compute/vmware/__init__.py
#	gns3server/controller/snapshot.py
#	gns3server/crash_report.py
#	gns3server/handlers/api/compute/atm_switch_handler.py
#	gns3server/run.py
#	gns3server/static/web-ui/26.77d4bfd104f37c42e028.js
#	gns3server/static/web-ui/index.html
#	gns3server/static/web-ui/runtime.415291667f70565cd8ef.js
#	gns3server/utils/__init__.py
#	gns3server/utils/images.py
#	gns3server/utils/interfaces.py
#	gns3server/version.py
#	gns3server/web/web_server.py
#	pytest.ini
#	requirements.txt
#	scripts/update-bundled-web-ui.sh
#	setup.py
#	tests/api/routes/compute/test_dynamips_nodes.py
#	tests/compute/builtin/nodes/test_cloud.py
#	tests/compute/docker/test_docker.py
#	tests/compute/docker/test_docker_vm.py
#	tests/compute/dynamips/test_dynamips_manager.py
#	tests/compute/dynamips/test_dynamips_router.py
#	tests/compute/iou/test_iou_vm.py
#	tests/compute/qemu/test_qcow2.py
#	tests/compute/qemu/test_qemu_manager.py
#	tests/compute/qemu/test_qemu_vm.py
#	tests/compute/test_base_node.py
#	tests/compute/test_manager.py
#	tests/compute/test_project.py
#	tests/compute/traceng/test_traceng_vm.py
#	tests/compute/virtualbox/test_virtualbox_manager.py
#	tests/compute/virtualbox/test_virtualbox_vm.py
#	tests/compute/vmware/test_vmware_manager.py
#	tests/compute/vmware/test_vmware_vm.py
#	tests/compute/vpcs/test_vpcs_vm.py
#	tests/conftest.py
#	tests/controller/gns3vm/test_remote_gns3_vm.py
#	tests/controller/gns3vm/test_virtualbox_gns3_vm.py
#	tests/controller/gns3vm/test_vmware_gns3_vm.py
#	tests/controller/test_export_project.py
#	tests/controller/test_gns3vm.py
#	tests/controller/test_import_project.py
#	tests/test_config.py
#	tests/utils/test_asyncio.py
#	tests/utils/test_images.py
#	tests/web/test_response.py
2024-05-16 17:53:39 +07:00
grossmj
46a9fdb61d
Remove event_loop 2024-05-09 17:53:02 +07:00
grossmj
33499be425
Fix warnings in tests 2024-05-09 17:11:12 +07:00
grossmj
2609849507
Fix tests after updating error message when busybox is not installed. 2024-04-23 18:00:37 +07:00
grossmj
1a53c9aabf
Backport from v3: install Docker resources in a writable location at runtime. 2024-02-14 16:13:45 +08:00
grossmj
09ff807055 Install Docker resources in writable location 2023-11-13 11:23:26 +10:00
grossmj
1ce0c13fc9 Better mocking in Docker tests 2023-08-18 12:20:54 +10:00
grossmj
2f2aabeb5a Fix tests when running Python 3.7 2023-08-11 17:58:00 +10:00
grossmj
d9eb61efc4 Fix tests with asyncio_patch 2023-08-11 17:37:11 +10:00
grossmj
1fd8444d22 Add tests for install_busybox() 2023-08-11 17:32:05 +10:00
grossmj
86c44cd6e7 Upgrade dev dependencies and fix tests to support pytest-asyncio strict mode 2022-08-24 21:03:16 +02:00
grossmj
eb3cb8a41f Migration to FastAPI 2020-10-02 16:07:50 +09:30
grossmj
d3ea67da24 Refactor tests
* Use pytest-aiohttp
* Use the async def / await syntax.
* Fix tests to run with Python 3.8
2020-06-16 13:59:03 +09:30
grossmj
76af98404a Drop Python 3.4 and switch to async / await syntax for asyncio. Fixes #1425 2018-10-15 17:05:49 +07:00
ziajka
d9a0ec9ff3 Replace asyncio.async with ensure_future because of deprecation, Fixes: #1269 2018-01-24 11:11:53 +01:00
ziajka
e0f0adf3c8 Added preferred Docker API version. Fixes #2136 2017-07-06 11:24:55 +02:00
Julien Duponchelle
2da581139c
Lower docker requirements in tests also 2017-04-24 18:49:47 +02:00
Julien Duponchelle
effbe59414
Lock docker API to 1.27
Fix https://github.com/GNS3/gns3-gui/issues/1994
2017-04-24 17:51:55 +02:00
Julien Duponchelle
132bc87081
Dissallow parallel pull of docker images
Fix #962
2017-03-27 20:46:25 +02:00
Julien Duponchelle
34f5a6f82c
Fix Session is closed when listing docker images
Fix #955
2017-03-20 23:50:31 +01:00
Julien Duponchelle
78c154e376
Cleanup docker source code 2017-03-20 20:30:59 +01:00
Julien Duponchelle
a571b1a2f8
Use aiohttp session for docker queries 2017-03-20 19:46:50 +01:00
Julien Duponchelle
a62b791c68
Fix TypeError: http_query() got an unexpected keyword argument 'timeout'
Fix #947
2017-03-20 17:06:00 +01:00
Julien Duponchelle
a3b243c501
Merge branch '1.5' into 2.0 2016-05-02 17:08:15 +02:00
Julien Duponchelle
58d4a529d4
Rename hypervisor to compute
Fix #487
2016-04-15 17:59:59 +02:00