82 Commits

Author SHA1 Message Date
YueGuobin
c477812332
feat: sync Docker images from the controller to remote computes
When a Docker node is created on a remote compute whose Docker daemon
does not have the image, the compute now raises ImageMissingError
instead of blindly pulling from the Docker repository. The controller
exports the image from the Docker daemon on its host (docker save
stream) and streams it to the compute which loads it, so locally built
or docker-loaded images work across computes. When the image is not
available on the controller host either, the compute is asked to pull
it from the Docker repository as a fallback.

- add a POST /docker/images/load compute endpoint that streams a
  docker save tar into the Docker daemon
- let Docker.http_query pass raw (non-dict) request bodies through so
  the tar can be streamed to the daemon
- drop the inline pull from DockerVM.create() and the now unused
  DockerVM.pull_image wrapper
2026-09-14 00:36:54 +08:00
YueGuobin
e272ad915b
feat: add IOL (iol-runner) Docker node support
Run Cisco CML containerized IOL images (e.g. iol-xe/iol-xe:17-18-02,
driven by virl.lab/cmd/iol-runner) as GNS3 Docker router nodes:

- VendorDockerVM: generic GNS3_UNIX_SOCKET_NIO/GNS3_UNIX_SOCKET_DIR knobs
  wiring adapters through AF_UNIX datagram socket pairs
  (add_nio_unix cNN.sock sNN.sock) instead of TAP + move_to_ns; node
  creation fails if the socket dir is not a persisted volume.
- IOLDockerVM (GNS3_IOL_RUNNER=1): forces skip-init + unix-socket NIO +
  /config,/tmp volumes, writes iol-config.json on every start (num-eth
  tracks adapters, runner drops to the server uid/gid so the sockets are
  reachable), pre-creates tmp/run, cleans stale sockets after unclean
  kills, and makes reload a graceful stop + full start (NVRAM flush +
  rewiring). Console stays telnet on PID 1 stdio.
- Manager selects the node class from console_type or GNS3_* environment
  markers (create-time, like console_type).
- Image-free tests (25) and feature documentation.
2026-09-05 21:54:12 +08:00
YueGuobin
9604c85fda
docker: harden the shm/devices/extra_configs/masking work (code review)
Nine fixes from a review of the docker-shm-devices diff:

* GNS3_STOP_TIMEOUT >300 s aborted at the manager's default HTTP timeout
  before Docker finished the stop — the stop query now gets a timeout
  with a margin over the grace period.
* Overlapping bind targets (GNS3_MASK_UDEV + GNS3_MASK_SYSTEMD on the
  same unit, a unit named twice, an extra_configs target equal to a
  masked unit) made Docker reject the create with 'Duplicate mount
  point' — Mounts are deduplicated by target.
* ExtraConfig.target now carries a pydantic validator (absolute file
  path, no '..'), so bad targets 422 at template-save time instead of
  failing at node-create time after a multi-GB image pull; directory
  forms ('/', '/etc/') are also rejected by the runtime guard instead
  of raising IsADirectoryError (raw 500).
* _check_host_readiness skipped every remaining check when one
  /proc/sys key was unreadable (mid-loop return) — now continues.
* The base-class GNS3_* env parser strips trailing commas like the
  vendor parser, so 'GNS3_MASK_UDEV=1,' composed from a list still
  activates.
* Vendor env knobs are re-parsed on every create(), so a PUT to the
  node's environment takes effect on the next (re)create.
* The graceful SIGTERM stop is now limited to the explicit user stop
  route; delete/update/close/crash-cleanup keep the immediate kill
  (those paths force-delete or recreate the container right after).
* An extra_configs target beneath a persisted volume is shadowed by the
  volume bind — warn at create time.
2026-08-15 00:52:55 +08:00
YueGuobin
2bee34031c
docker: warn about low inotify/file-max and missing FUSE at connect
Add a read-only _check_host_readiness() that runs once after the Docker
daemon connection is established. It reads /proc/sys inotify/file-max
limits and /proc/filesystems (for FUSE), and logs a warning with the exact
commands to fix when they are too low for heavy containers -- XRd wants
~4000 inotify instances per node against a stock default of 128.

The server runs unprivileged (only the setuid ubridge helper has root), so
it can only check, not set; the warning tells the admin exactly what to
raise once. Stays silent when the limits are already sufficient.
2026-08-13 22:39:48 +08:00
YueGuobin
5388fd3796
refactor: move vendor NOS Docker support into VendorDockerVM subclass
Extract the docker_exec console and GNS3_* prototype knobs (SKIP_INIT,
INTERFACE_NAMES, CONSOLE_CMD) from DockerVM into a VendorDockerVM subclass.
DockerVM is restored to its 3.1 baseline plus four small extension hooks
(_prepare_init_and_interface_env, _start_console_server,
_get_container_ifname, _cleanup_console_resources) that are pure
refactorings with zero behaviour change for existing nodes.

VendorDockerVM additionally replicates init.sh's volume persistence
(bind-mount /gns3volumes over the in-container path) via docker exec for
containers that skip init.sh, so vendor NOS config (e.g. /etc/opt/srlinux)
survives node stop/start.

The Docker manager selects VendorDockerVM when console_type == docker_exec;
all other nodes keep using DockerVM unchanged.
2026-08-13 02:05:17 +08:00
Cristi
2abc7e09a3 Adds controller and compute API support for explicitly pulling or updating Docker images. 2026-07-24 16:40:20 +03:00
YueGuobin
ce90b2b92c fix(docker): handle container name conflict automatically
When a Docker container with the same name already exists (e.g., from a
   previous crashed GNS3 session), Docker returns a 409 Conflict error
   when trying to create a new container with that name. This causes the
   project open operation to fail.

   This fix adds automatic cleanup of stale containers when encountering
   a name conflict:
   - Added DockerHttp409Error exception class
   - Updated http_query to detect 409 status codes
   - Modified create() to remove conflicting containers and retry

   Fixes the issue where opening a project fails with:
   "Docker has returned an error: 409 Conflict. The container name
   '/GNS3.xxx' is already in use by container 'xxx'"
2026-02-27 23:35:11 +08:00
grossmj
c5b9f379fe
Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	Dockerfile
#	dev-requirements.txt
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/version.py
#	requirements.txt
2026-02-20 16:28:53 +08:00
Dale Arbogast
3dd4184193 fix: busybox static link detection on Alpine/musl
On musl-based systems (Alpine), ldd returns exit code 0 for static
binaries, unlike glibc which returns 1. This causes install_busybox()
to reject all busybox binaries as "dynamically linked" on Alpine.

Fix by also accepting binaries whose executable name contains "static"
(i.e. busybox-static, busybox.static), which are the first two
candidates checked by the function. The generic "busybox" fallback
still relies on the ldd return code check.
2026-02-19 16:44:01 -05:00
grossmj
60c10e7ecc
Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	.github/workflows/testing.yml
#	CHANGELOG
#	README.md
#	gns3server/appliances/infix.gns3a
#	gns3server/compute/docker/__init__.py
#	gns3server/compute/docker/docker_vm.py
#	gns3server/crash_report.py
#	gns3server/version.py
#	gns3server/web/web_server.py
#	setup.py
#	tests/controller/test_project.py
2025-11-29 11:42:02 +10:00
grossmj
f2aaf5ab38
Docker API version requirements and process to handle older daemons 2025-11-17 22:26:18 +10:00
grossmj
d9dcc2738d
Merge remote-tracking branch 'origin/2.2' into 3.0
# Conflicts:
#	CHANGELOG
#	Dockerfile
#	README.md
#	gns3server/appliances/infix.gns3a
#	gns3server/compute/docker/docker_vm.py
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/controller/gns3vm/virtualbox_gns3_vm.py
#	gns3server/crash_report.py
#	gns3server/static/web-ui/index.html
#	gns3server/static/web-ui/main.9bcf455e62558dedfd48.js
#	gns3server/version.py
2025-04-21 20:13:31 +07:00
grossmj
ffd628902c
Fix Docker logs decoding. Ref #2522 2025-04-18 13:58:56 +07:00
grossmj
0e8d969cd2
Merge branch '2.2' into 3.0
# Conflicts:
#	gns3server/compute/docker/__init__.py
#	gns3server/compute/docker/docker_vm.py
#	requirements.txt
2025-04-16 17:53:44 +07:00
grossmj
0e89ff56a8
Replace "Docker hub" by "Docker repository" because it is possible to use different repositories 2025-04-16 17:42:40 +07:00
grossmj
4758431c76
Improvements for built-in disks
* Checksum is updated in the database for updated disks.
* It is not possible to prune them.
2025-01-02 23:10:51 +07:00
grossmj
3f7f5a3cda
Fix to access resources_path and install_builtin_appliances settings 2024-07-09 12:28:39 +02:00
grossmj
59ad5c55ec
Merge branch '2.2' into 3.0
# Conflicts:
#	CHANGELOG
#	gns3server/config_samples/gns3_server.conf
#	gns3server/controller/__init__.py
#	gns3server/crash_report.py
#	gns3server/static/web-ui/index.html
#	gns3server/static/web-ui/main.b65e52314df938ebe652.js
#	gns3server/version.py
2024-07-09 12:14:44 +02:00
grossmj
ca1d99b112
Add 'install_builtin_appliances' and 'resources_path' settings in the server config 2024-07-08 18:06:33 +02:00
grossmj
d3860ba84b
Merge branch '2.2' into 3.0
# Conflicts:
#	CHANGELOG
#	dev-requirements.txt
#	gns3server/compute/qemu/__init__.py
#	gns3server/compute/vmware/__init__.py
#	gns3server/controller/snapshot.py
#	gns3server/crash_report.py
#	gns3server/handlers/api/compute/atm_switch_handler.py
#	gns3server/run.py
#	gns3server/static/web-ui/26.77d4bfd104f37c42e028.js
#	gns3server/static/web-ui/index.html
#	gns3server/static/web-ui/runtime.415291667f70565cd8ef.js
#	gns3server/utils/__init__.py
#	gns3server/utils/images.py
#	gns3server/utils/interfaces.py
#	gns3server/version.py
#	gns3server/web/web_server.py
#	pytest.ini
#	requirements.txt
#	scripts/update-bundled-web-ui.sh
#	setup.py
#	tests/api/routes/compute/test_dynamips_nodes.py
#	tests/compute/builtin/nodes/test_cloud.py
#	tests/compute/docker/test_docker.py
#	tests/compute/docker/test_docker_vm.py
#	tests/compute/dynamips/test_dynamips_manager.py
#	tests/compute/dynamips/test_dynamips_router.py
#	tests/compute/iou/test_iou_vm.py
#	tests/compute/qemu/test_qcow2.py
#	tests/compute/qemu/test_qemu_manager.py
#	tests/compute/qemu/test_qemu_vm.py
#	tests/compute/test_base_node.py
#	tests/compute/test_manager.py
#	tests/compute/test_project.py
#	tests/compute/traceng/test_traceng_vm.py
#	tests/compute/virtualbox/test_virtualbox_manager.py
#	tests/compute/virtualbox/test_virtualbox_vm.py
#	tests/compute/vmware/test_vmware_manager.py
#	tests/compute/vmware/test_vmware_vm.py
#	tests/compute/vpcs/test_vpcs_vm.py
#	tests/conftest.py
#	tests/controller/gns3vm/test_remote_gns3_vm.py
#	tests/controller/gns3vm/test_virtualbox_gns3_vm.py
#	tests/controller/gns3vm/test_vmware_gns3_vm.py
#	tests/controller/test_export_project.py
#	tests/controller/test_gns3vm.py
#	tests/controller/test_import_project.py
#	tests/test_config.py
#	tests/utils/test_asyncio.py
#	tests/utils/test_images.py
#	tests/web/test_response.py
2024-05-16 17:53:39 +07:00
grossmj
448b94f701
Add more details to error message when busybox is not installed. Fixes https://github.com/GNS3/gns3-gui/issues/3569 2024-04-23 17:54:06 +07:00
grossmj
800920e3df
Merge branch '2.2' into 3.0
# Conflicts:
#	.github/workflows/testing.yml
#	gns3server/compute/docker/__init__.py
#	gns3server/compute/docker/docker_vm.py
#	gns3server/run.py
#	requirements.txt
#	setup.py
#	tests/compute/docker/test_docker_vm.py
2024-02-21 12:24:30 +08:00
grossmj
1a53c9aabf
Backport from v3: install Docker resources in a writable location at runtime. 2024-02-14 16:13:45 +08:00
grossmj
1f5085608c
Use Docker API v1.24 to get version. 2024-02-14 15:40:19 +08:00
grossmj
09ff807055 Install Docker resources in writable location 2023-11-13 11:23:26 +10:00
grossmj
f3b6825e40 Test if busybox is not dynamically linked 2023-08-11 14:10:25 +10:00
grossmj
1cb433c5bc New packaging relying only pyproject.toml 2023-08-10 22:44:37 +10:00
grossmj
1ff23348d3 Merge branch '2.2' into 3.0
# Conflicts:
#	gns3server/compute/base_node.py
#	gns3server/compute/docker/__init__.py
#	gns3server/compute/qemu/qemu_vm.py
#	gns3server/controller/compute.py
#	gns3server/controller/gns3vm/virtualbox_gns3_vm.py
#	gns3server/controller/node.py
#	gns3server/controller/project.py
#	gns3server/crash_report.py
#	gns3server/handlers/api/controller/template_handler.py
#	gns3server/static/web-ui/index.html
#	gns3server/static/web-ui/main.8448c96e4facbe79a613.js
#	gns3server/version.py
#	tests/compute/iou/test_iou_vm.py
#	tests/compute/qemu/test_qemu_vm.py
#	tests/handlers/api/controller/test_template.py
2023-06-20 16:06:53 +09:30
grossmj
2361e138ec Properly catch aiohttp client exception. Ref #2228 2023-05-30 16:17:12 +09:30
grossmj
296446189f Merge branch '2.2' into 3.0
# Conflicts:
#	gns3server/appliances/Simulator.gns3a
#	gns3server/compute/docker/__init__.py
#	gns3server/compute/docker/docker_vm.py
#	gns3server/version.py
2021-12-02 19:57:04 +10:30
grossmj
b1a62dfdc2 Fix unhandled KeyError exception when starting Docker container. Ref #1991 2021-11-04 16:59:35 +10:30
grossmj
c021e21309 Use black with -l 120 param. 2021-04-13 18:46:50 +09:30
grossmj
f928738bd5 Use pyupgrade with --py36-plus param. 2021-04-13 18:37:58 +09:30
grossmj
a92c47b310 Add HTTP client to reuse the aiohttp session where needed.
Remove unnecessary aiohttp exceptions.
2020-10-22 16:19:44 +10:30
grossmj
af80b0bb6e Use aiofiles where relevant. 2019-03-06 23:00:01 +07:00
grossmj
5754747a90 Merging 2.1 into 2.2 2019-02-22 18:04:49 +07:00
grossmj
d9a9abf845 Add explicit error when trying to pull a Docker image from Docker Hub without Internet access. Fixes #1506. 2019-02-17 19:21:21 +08:00
grossmj
4b52e9313e Fix issue with coroutine not awaited. Fixes #1499 2019-01-17 17:43:09 +07:00
grossmj
86f87aec74 Fix installation with Python 3.7. Fixes #1414.
Fix deprecated use of aiohttp.Timeout. Fixes #1296.
Use "async with" with aiohttp.ClientSession().
Make sure websocket connections are properly closed, see https://docs.aiohttp.org/en/stable/web_advanced.html#graceful-shutdown
Finish to drop Python 3.4.
2018-10-16 15:56:06 +07:00
grossmj
76af98404a Drop Python 3.4 and switch to async / await syntax for asyncio. Fixes #1425 2018-10-15 17:05:49 +07:00
grossmj
33e8710495 Merge branch '2.1' into 2.2
# Conflicts:
#	gns3server/compute/docker/docker_vm.py
#	gns3server/controller/compute.py
#	gns3server/controller/gns3vm/__init__.py
#	gns3server/controller/link.py
#	gns3server/controller/project.py
#	gns3server/handlers/api/controller/project_handler.py
#	gns3server/handlers/api/controller/server_handler.py
#	gns3server/utils/asyncio/__init__.py
#	gns3server/utils/asyncio/telnet_server.py
#	gns3server/version.py
#	gns3server/web/web_server.py
#	tests/compute/qemu/test_qemu_vm.py
2018-08-29 15:57:54 +07:00
grossmj
902de3dd47 Refactor asyncio locking system for Python 3.7 support. Ref https://github.com/GNS3/gns3-gui/issues/2566 Ref https://github.com/GNS3/gns3-gui/issues/2568 2018-08-25 14:10:47 +07:00
grossmj
21a1800edc Merge branch '2.1' into 2.2
# Conflicts:
#	gns3server/controller/__init__.py
#	gns3server/controller/export_project.py
#	gns3server/controller/import_project.py
#	gns3server/controller/project.py
#	gns3server/version.py
2018-04-28 19:41:12 +07:00
grossmj
305fe2e817 Handle asyncio timeouts. Ref #1307. 2018-04-28 17:42:02 +07:00
grossmj
90ce6093d8 Some spring cleaning. 2018-03-15 14:17:39 +07:00
grossmj
658aa4bae9 Merge 2.1 into 2.2 branch 2018-03-12 13:38:50 +07:00
grossmj
07d2d7b340 Update documentation. 2018-01-31 23:37:02 +07:00
Julien Duponchelle
f557ec4bda
Fix regression in docker introduce by 2.0 merge 2017-07-12 17:59:29 +02:00
Julien Duponchelle
699ecb7c89
Merge branch 'master' into 2.1 2017-07-11 18:59:50 +02:00
ziajka
e0f0adf3c8 Added preferred Docker API version. Fixes #2136 2017-07-06 11:24:55 +02:00