Commit Graph
379 Commits
Author SHA1 Message Date
YueGuobin 0a8d430d50 feat(docker): packet filters on kernel-datapath links via tc netem
Impairment filters with a netem equivalent (delay, packet_loss, corrupt)
now run on the kernel datapath as one tc netem qdisc per veth host end,
pushed through uBridge's netlink tc module (tc netem set / tc reset).
This removes the relay delay filter's nanosleep bottleneck (upstream

Both link endpoints carry the filters: each end's qdisc egress covers
the traffic entering that container, so every direction is impaired
exactly once -- the same net effect as the relay, where both directions
cross the single filtered bridge.

- controller: eligibility keys on relay-only types (frequency_drop, bpf)
  instead of any filter; kernel links push the real filters (the
  synthetic suspend frequency_drop no longer flips a suspended link to
  relay on reopen/reset); update_filters 409s only relay-only types;
  available_filters hides them on kernel links
- compute: create_nio whitelists netem-capable filter types; the NIO
  update path keeps its own guard (PUT mutates the NIO in place);
  _ubridge_apply_netem translates the filter dict into one atomic
  netem-set (full re-apply IS the reconcile); node restart restores the
  qdisc, link deletion detaches it (the veth survives the link)
- e2e (27/27): delay 100 -> 200.2 ms RTT, atomic replace 50 -> 100.2 ms,
  clear -> baseline, packet_loss 30 -> 48 % round-trip (expected 51 %),
  suspend/resume and node restart keep the qdisc, link delete leaves no
  residual impairment
2026-10-06 00:21:08 +08:00
Mark Paronyan 3135c8b7e3 refactor: use f-strings in remaining modules, scripts and tests 2026-10-03 19:44:02 +03:00
Mark Paronyan b622ccae7b refactor: declare explicit public API in package __init__ modules 2026-10-03 19:43:59 +03:00
Mark Paronyan 306d8fbde4 refactor: remove unused imports 2026-10-03 19:43:58 +03:00
Mark Paronyan 634189d443 style: sort imports in api and utils modules 2026-10-02 12:43:26 +03:00
Mark Paronyan a77e06b907 Merge branch '3.1' into ruff/trivia-fixups 2026-10-01 19:58:02 +03:00
Mark Paronyan 74943b4c74 refactor: convert remaining percent formatting in log messages 2026-10-01 19:57:47 +03:00
Mark Paronyan 95ada23395 refactor: replace percent formatting with format specifiers 2026-10-01 19:57:47 +03:00
Mark Paronyan 2dc7479d11 style: remove f-strings without placeholders 2026-10-01 19:57:46 +03:00
Mark Paronyan 396407827e style: drop redundant open mode arguments 2026-10-01 19:57:44 +03:00
Mark Paronyan 7a1e79a8a0 style: parenthesize chained boolean operators 2026-10-01 19:57:43 +03:00
Mark Paronyan c5b5091f5e refactor: use explicit conversion in f-strings 2026-10-01 19:57:41 +03:00
grossmj 1456d36bf0 chore(tests): fix mypy errors 2026-10-01 15:22:17 +02:00
Jeremy Grossmann a2cdc14a25 Merge branch '3.1' into image-manager-out-of-sync 2026-10-01 14:44:24 +02:00
Cristi 87df09c00c fix(images): Fix symlinked image imports and add regression tests 2026-09-29 20:16:22 +03:00
Cristi 2e12775911 fix(images): Fix symlink support in image inventory 2026-09-29 15:42:49 +03:00
Jeremy Grossmann 5c87c0e1a4 Merge branch '3.1' into mypy-utils 2026-09-29 13:41:58 +02:00
Jeremy Grossmann 1838ad7f05 Merge branch '3.1' into mypy-schemas-plus 2026-09-29 13:41:11 +02:00
Jeremy Grossmann ec4b04ba54 Merge pull request #2914 from markparonyan/mypy-utils-misc
fix(typing): resolve mypy errors in utils
2026-09-29 13:01:12 +02:00
Mark Paronyan 1284701253 fix(typing): resolve mypy errors in utils.zipfile_zstd._patcher 2026-09-29 09:23:15 +03:00
Mark Paronyan df8b3c2903 fix(typing): resolve mypy errors in utils.asyncio.aiozipstream 2026-09-29 09:23:15 +03:00
Mark Paronyan 1772c7380e fix(typing): resolve mypy errors in utils.websocket_to_websocket 2026-09-29 09:23:15 +03:00
Mark Paronyan 5bf3c8d203 fix(typing): resolve mypy errors in utils.http_client 2026-09-29 09:23:15 +03:00
Mark Paronyan 94bf56040b fix(typing): resolve mypy errors in utils.packet_filter_validation 2026-09-29 08:48:06 +03:00
Mark Paronyan ab5572a0c1 fix(typing): resolve mypy errors in utils.images 2026-09-29 08:48:06 +03:00
Mark Paronyan 943bd6732c fix(typing): resolve mypy errors in utils.encryption 2026-09-29 08:48:06 +03:00
Cristi 42d30651da Merge origin/3.1 into image-manager-out-of-sync
Preserve image reconciliation, locking, and regression coverage while integrating upstream release, dependency, typing, and formatting updates. Align feature code with the new Ruff checks.
2026-09-28 23:16:24 +03:00
Jeremy Grossmann 71965e0e81 Merge pull request #2894 from markparonyan/mypy-utils-zipfile-zstd
fix(typing): resolve mypy errors in utils.zipfile_zstd
2026-09-28 20:50:20 +02:00
Mark Paronyan 3fc1d27937 fix(typing): resolve mypy errors in utils.zipfile_zstd 2026-09-28 21:47:05 +03:00
Cristi 046dff4d08 fix(images): reconcile the image inventory with filesystem changes 2026-09-28 16:01:36 +03:00
Mark Paronyan d1df96e0a6 fix(typing): resolve mypy errors in utils.asyncio.embed_shell 2026-09-28 04:56:18 +00:00
Mark Paronyan a0450b6bce refactor: ruff autofixes 2026-09-27 15:55:10 +03:00
YueGuobin 0119373690 feat: allocate IOL Docker application IDs from a pool disjoint from IOU
IOL interface MACs derive from the application ID (aabb.cc{app}{iface}),
so ids must be unique across opened projects sharing computes — the same
reason IOU has its allocator. IOL Docker nodes draw from the upper half
(512-1022, netiomux's fixed peer is 1023) so the two node types can
neither collide nor starve each other; IOU behavior is unchanged.

The controller sniffs the same GNS3_IOL_RUNNER environment marker the
compute uses to select IOLDockerVM (both the nested-properties and
template/top-level-kwarg shapes), stores the id in node properties like
IOU does, and passes it through the Docker create payload. Without an
allocation the compute falls back to a stable node-derived id in the
same upper range.
2026-09-05 21:54:52 +08:00
YueGuobin 7d3cbf1023 feat: add config read-modify-write update and harden file watcher
Config.update_config() applies submitted options to the main
configuration file via configparser read-modify-write: unknown
options are preserved, null removes an option, the merged view
of all files is validated as ServerConfig before anything is
written (a bad file would kill the FileWatcher polling loop),
and the write is atomic (.tmp + os.replace, mode 0600). Options
whose effective value is owned by a later configuration file
raise ConfigConflictError instead of writing a no-op. The
reload logic is factored into reload_and_notify() and the file
watcher callback is exception-guarded so polling never dies.
2026-08-23 18:54:23 +08:00
YueGuobin e98c51889e fix: notification ping starved under sustained event load
NotificationQueue.get only generated a synthetic ping when the queue
was idle for the full timeout. Under sustained event load (e.g. a
project with markers matching at 15-260 events/s) the queue never
idled, so compute notification streams never carried a ping and the
controller stopped emitting compute.updated: clients lost compute
statistics until the event flow paused or the server restarted.

A ping is now guaranteed at least every timeout seconds regardless of
event flow: when the ping deadline is reached the next get() returns a
ping ahead of queued events (pings only carry statistics, so skipping
ahead of real events is harmless). Both the compute stream (compute
CPU/memory/disk stats -> compute.updated) and the controller stream
(idle keepalive) benefit.
2026-08-22 00:50:49 +08:00
grossmj 52de01523f Merge branch '2.2' into 3.1
# Conflicts:
#	gns3server/compute/builtin/nodes/cloud.py
#	gns3server/utils/interfaces.py
2026-08-18 18:05:56 +02:00
chara 80da7da515 add support basic support for OpenBSD.
there remains some work on ubridge, but this seems to work so far.
2026-08-13 22:24:58 +03:30
YueGuobin 0ed1715fe6 Use typed InterfaceStatus enum and simplify stats fallback in interfaces()
- Change `status: str` to `status: InterfaceStatus` (str Enum) in the
  HostInterface schema, matching the existing IPAddressFamily pattern.
- Use default-then-override pattern in interfaces() to eliminate the
  `else` branch and deduplicate defaults.
2026-07-10 14:55:21 +08:00
YueGuobin 9721660cc3 Expose full interface address list and link state in cloud node API
Each host interface surfaced by the cloud node now reports:
- ip_addresses: every IPv4 and IPv6 address (previously only a single
  IPv4 was collected internally and then dropped before the response)
- status / speed / mtu / flags: operational state and link attributes
  sourced from psutil.net_if_stats(), with flags normalized to a list

The legacy ip_address / netmask / mac_address fields are preserved so
existing callers (compute link detection, GNS3 VM, VMware, has_netmask)
keep working. The new fields travel through the existing interfaces
payload that the controller forwards verbatim, so no controller-side
change is required and the PUT / ports_mapping flow is unaffected.
2026-07-10 13:56:07 +08:00
YueGuobin 292b60efaa Make AI features (AI Copilot + MCP) optional via [ai-features] extra
- Move fastmcp from core requirements.txt to mcp-requirements.txt
- Add MCP_AVAILABLE feature flag in agent/__init__.py (graceful degradation)
- Guard MCP imports/registration in server.py and tasks.py
- Replace ai-copilot/mcp/ai-support extras with single ai-features extra
- Add stub MCP routes returning 501 when MCP is not installed
- Add gns3server-uninstall-ai-features CLI command
- Remove old gns3server-uninstall-ai-copilot command
- Update all error messages and docs to reference ai-features

Closes #2794
2026-06-27 10:14:43 +08:00
YueGuobin 3465c39eaa Reduce md5sum cache write failure log level from error to warning
This is a non-fatal issue that occurs during concurrent project close
when multiple nodes try to write their md5sum cache while the project
directory is being cleaned up.
2026-06-15 13:27:14 +08:00
YueGuobin e659b64bf0 Add async_iterable_to_stream utility to avoid aiohttp compatibility issues
Create async_iterable_to_stream() in gns3server.utils.asyncio that
converts an async iterable to an aiohttp StreamReader via a background
feeder task. This bypasses aiohttp's AsyncIterablePayload which can
cause 'Connection reset by peer' with certain HTTP servers.

Use it in _run_http_query for the __aiter__ data path.
2026-06-09 23:55:59 +08:00
YueGuobin 3958279b8a feat: add client information logging to MCP connection rejection
Add reusable utility for extracting client information from ASGI scope:
- Create gns3server/utils/request_utils.py with extract_client_info()
- Extract client IP, port, path, method, and authenticated username
- Format comprehensive log messages with client context

Benefits:
- Better observability for connection rejection events
- Reusable utility for other modules
- Consistent client logging format across codebase
- Helps diagnose timing issues during server startup

Usage:
  from gns3server.utils.request_utils import extract_client_info
  client_info = extract_client_info(scope, auth_service)
  log.warning(f"Connection rejected - Client: {client_info['host']}:{client_info['port']} ({client_info['user_info']})")
2026-06-06 22:35:43 +08:00
YueGuobin ced73574b4 Fix delay filter validation: ensure delay: [0, X] returns proper error message
This fix addresses the issue where delay: [0, X] configurations were being
silently dropped instead of returning validation errors.

Changes:
- Created new utility function filter_inactive_filters() in packet_filter_validation.py
- Implemented smart filtering logic for delay filter that checks both latency and jitter:
  * delay: [0, 0] → User wants to disable delay, filter out silently
  * delay: [0, X] where X > 0 → Invalid config, keep for validation error
  * delay: [X, X] where X > 0 → Normal configuration, validate normally
- Simplified link.py update_filters() method to use the new utility function
- Added comprehensive tests for the new filtering logic

Before this fix:
- delay: [0, 100] would be silently dropped with no error message
- Users wouldn't know their configuration was invalid

After this fix:
- delay: [0, 100] returns proper error: "delay parameter Latency must be between 1 and 32767 ms, got: 0"
- delay: [0, 0] is correctly handled as intentional disable
- Normal delay configurations continue to work as expected
2026-05-31 22:35:58 +08:00
YueGuobin 0aa0467b02 Fix delay latency minimum: ubridge rejects latency <= 0
Align validation rules with ubridge source: delay latency must be > 0
(packet_filter.c delay_setup line 182). Update FILTERS definition in
link.py and test cases accordingly.
2026-05-30 01:21:00 +08:00
YueGuobin 7bcb96a368 Improve packet filter validation: use tcpdump, handle multi-line BPF, safe project load
Changes:
- Replace tshark BPF validation with tcpdump -d (calls pcap_compile
  internally like ubridge, returns instantly without waiting for traffic)
- Support multi-line BPF expressions: split on newlines and validate
  each line individually
- Always validate, never save invalid filters on error
- Drop invalid filters during project load with warning (prevents
  old topologies with bad filters from failing to open)
- Simplify test cases (no longer depend on tshark availability)
2026-05-30 01:02:21 +08:00
YueGuobin 87f38b1507 Add packet filter parameter validation to prevent ubridge errors
This commit implements comprehensive parameter validation for GNS3 packet
filters at the API layer, preventing invalid parameters from reaching
ubridge and causing technical errors.

## Changes

### New Features
- **Add packet filter validation module** (`gns3server/utils/packet_filter_validation.py`)
  - Validate parameter ranges for all filter types:
    - frequency_drop: -1 to 32767
    - packet_loss: 0 to 100%
    - delay: 0 to 32767ms (latency + jitter)
    - corrupt: 0 to 100%
  - **BPF syntax validation** using tshark (method from gns3_copilot)
  - Parameter count and type validation
  - User-friendly error messages with parameter details

### Integration
- **Integrate validation into Link.update_filters()** (`gns3server/controller/link.py`)
  - Validate parameters before applying filters
  - Raise ControllerError with clear error messages for invalid input
  - Prevent invalid parameters from reaching ubridge

### Testing
- **Add comprehensive test suite** (`tests/utils/test_packet_filter_validation.py`)
  - 15 test cases covering all validation scenarios
  - Tests for valid/invalid parameters, edge cases, and BPF syntax
  - All tests passing

## Benefits
- **Better UX**: Clear, actionable error messages instead of ubridge technical errors
- **Prevents crashes**: Validate parameters at API layer before reaching ubridge
- **Consistent validation**: Centralized validation logic for all filter types
- **BPF safety**: Syntax validation using tshark prevents invalid BPF expressions
2026-05-29 23:39:44 +08:00
YueGuobin 868e317d24 fix: close DockerHTTPClient session to prevent UnixConnector leak in Web Wireshark
The Web Wireshark WebSocket endpoint created a WebWiresharkManager but
never called close(), leaving the DockerHTTPClient's ClientSession with
UnixConnector unclosed when users closed the browser tab.

Also switch asyncio.wait in WebSocket proxy from ALL_COMPLETED to
FIRST_COMPLETED to avoid blocking cleanup when one direction disconnects.
2026-05-18 23:59:56 +08:00
grossmj 6551bf56f8 Tweak and merge changes from 3.0 branch 2026-05-08 15:36:40 +08:00
Cristi 9d451eb652 feat(compute): Updated server host key to use the secure ssh-ed25519 instead of old ssh-rsa algorithm 2026-04-30 17:56:05 +03:00