1256 Commits

Author SHA1 Message Date
YueGuobin
187201a08c
feat(controller): offer QEMU the kernel datapath, gated per compute
Kernel eligibility becomes "both endpoints on one compute, each able to
anchor": Docker always (adapters are born as veth pairs), QEMU when that
compute's uBridge can create the persistent TAPs its adapters anchor on. The
question is asked per compute, because the link needs both ends anchored --
so a mixed or half-upgraded deployment degrades to the relay instead of
failing an attach.

probe_tap_support() creates and deletes one throwaway persistent TAP through a
scratch uBridge, cached by binary identity like the tc report; /capabilities
exposes `ubridge_tap` (absent = unknown, which keeps links on the relay).

The compute route for a QEMU adapter NIO took UDPNIO only, so a kernel link
was rejected with 422: its body and response now accept
Union[UDPNIO, BridgeNIO] exactly like Docker's.

Stale "Docker-to-Docker" wording in the config, link and MCP descriptions now
talks about anchors, and docs/features/qemu-kernel-datapath.md documents the
datapath -- verified end-to-end on a real instance: 32/32 kernel-datapath
checks and 29/29 on the relay variant.
2026-10-06 00:21:08 +08:00
YueGuobin
1db9f171ca
fix(controller): re-arm window_drop and expose kernel_datapath
Three fixes from live-calibrating the kernel impairment filters on an
isolated instance (Docker-to-Docker veth link, per-sequence evidence):

- update_filters skipped an unchanged filters dict as a no-op, so a
  one-shot window_drop outage that had already passed could never be
  re-armed: re-applying the same values produced no datapath command at
  all. A PUT still carrying window_drop now reconciles, restarting the
  schedule from the apply as documented. Measured: identical re-apply
  moves the outage window from seq 50-70 to 70-91 (arm at +8s of the
  new apply).

- gemodel parameter semantics were documented backwards in the
  validation rules, available_filters, the MCP/copilot descriptions and
  the feature docs. Measured against the kernel (per-direction counters,
  offered = sent + dropped): p is the good-to-bad transition chance, r
  the bad-to-good transition, 1-h the loss chance in the bad state (the
  good state loses nothing) - mean one-way loss p/(p+r)*(1-h), e.g.
  [10,30,100] -> 24.8%/25.7% measured vs 25% predicted, [5,45,100] ->
  10.8% both ways vs 10%.

- kernel_datapath was silently stripped from every REST/MCP link payload
  by the response model (LinkBase carried no such field), so clients
  could not tell a kernel link from a relay one.

Also verified on the same setup: the 3-argument window_drop form works
exactly as specified ([3000,5000,100] -> a single seq 31-55 outage,
[20000,6000,100] -> seq 115-144), and both generations of the command
builder emit the parameters unpadded.

Adds an API test for the field and controller tests for the no-op skip
and the window_drop re-arm exemption.
2026-10-06 00:21:08 +08:00
YueGuobin
cecaf298bb
feat(controller): expose kernel_datapath on the link payload
Runtime fact recomputed from the NIO wiring on every load (not
persisted). Without it clients cannot tell a relay link from a kernel
link whose compute lacks a filter capability — both show up as
kernel-only types missing from available_filters.
2026-10-06 00:21:08 +08:00
YueGuobin
b4e4739d52
feat(compute): report uBridge tc capabilities and gate filters on them
Consume uBridge's ebpf_modes contract (feature/tc-window 40c36ae): a
stateful mode is usable iff ebpf=1 and its token is listed; builds
predating the field keep their shipped modes (nth/quota/flow), so the
pre-correction window semantics can never be installed silently.

- gns3server/utils/tc_capabilities.py: shared reply parser, mode
  policy and filter-type mappings (compute and controller gate on the
  same rules)
- gns3server/compute/ubridge/tc_probe.py: standalone probe spawning a
  throwaway uBridge as the server's own user, cached by binary
  identity (path/mtime/size - the version string cannot distinguish
  rebuilds); failures cache as unknown
- GET /v3/compute/capabilities gains ubridge_tc {netem, ebpf,
  ebpf_modes, cbpf}; the controller forwards it (Compute.capabilities)
  and available_filters hides kernel-datapath filter types an endpoint
  compute cannot run (both ends apply their own filters, so one
  incapable compute hides the type); computes without a report keep
  the full list and apply-time 409 stays the guard
- docker: _ubridge_apply_ebpf_drops gates per mode token; the cleanup
  path only sends "off" for modes the build declares
- split DockerVM's kernel-datapath machinery (veth lifecycle, carrier,
  marker primitives, tc filter translation, kernel NIO teardown) into
  compute/docker/docker_kernel_datapath.py (DockerKernelDatapathMixin)
2026-10-06 00:21:08 +08:00
YueGuobin
316e7002c1
feat(docker): expose the eBPF time-window filter on kernel links
uBridge's window mode gained the semantics the B.2 spec asked for —
`tc window_drop <if> <start> <outage> <pct> [<period> [<jitter>]]`: a
single outage that traffic passes before *and* after, recurring flaps
with period >= outage, and a per-cycle jitter — so the kernel-only
`window_drop` filter type is now exposed:

- validation mirrors the tc grammar (3-5 params, ranges, period >=
  outage, jitter cap) and adds window_drop to KERNEL_ONLY_FILTERS;
  filter_inactive_filters keeps start=0 — unlike every other filter a
  zero first parameter means "the outage starts now", not "disabled".
- `_ubridge_apply_ebpf_drops` sends `tc window_drop` (positional
  pass-through) and turns the mode off explicitly when absent, under the
  same ebpf=1 gate.
- FILTERS entry (Start/Outage/Chance/Period/Jitter) + available_filters.

The schedule is relative to the apply: reconcile re-sets the classifier
on every apply, so any filter change, node restart or link reset restarts
the window.

Verified on a two-container kernel link, 15/15: a single outage
[1000, 1500, 100] arrives per-seq as [0..4, 13..19] (passing before the
window, dropped inside, passing after); recurring 1000/2000 ms = 52 %
round trip (duty cycle); period == outage = permanent outage; period <
outage = 409 before uBridge; jittered flap keeps the ~50 % duty cycle;
30 % chance on a permanent window = 53 % round trip against the
per-direction theory 1-0.7^2 = 51 %; coexist with netem; node restart
restores; available_filters kernel vs relay (409 + hidden); teardown
clean. P6b re-run as a regression on the same uBridge build: 13/13.

Worth recording for the next rebuild: the first delivered window program
loaded as root but reported ebpf=0 through gns3-server — the verifier's
non-root path loses the loop counter's bound, treats the catch-up loop as
unbounded and rejects the *whole* program (E2BIG, 8193 pending states),
which also took frequency_drop and quota down, with ebpf=0 as the only
symptom. Smaller trip caps did not help; the loop had to go (uBridge
572ca0c), on top of a Makefile header-dependency fix (53e34bb) that had
been shipping a stale program object.
2026-10-06 00:21:08 +08:00
YueGuobin
0c4ec8c38c
feat(docker): eBPF stateful packet filters on kernel links
Consume the eBPF impairment classifier delivered in uBridge
feature/tc-precision: frequency_drop becomes the exact every-Nth mode
(tc nth_drop; -1 = drop everything maps to every 1st) and the new
kernel-only quota type is the byte-cap mode (tc quota_drop, 100 = hard
cutoff). KERNEL_UNSUPPORTED_FILTERS is gone entirely — kernel/relay
eligibility is now purely topological. Both modes gate on ebpf=1 from
the per-process tc capabilities probe (error names the setcap fix); the
cleanup path (modes off) only talks to the eBPF commands when the probe
reported them. Reconcile is a plain re-set: the netem reset-before-set
already detaches the program and clears uBridge's per-interface
registry, so a mode set after it is a fresh load+attach.

Documented semantic difference: the relay counts both directions
through one bridge counter, the kernel counts per direction — N=3
round-trip loss is 1-(2/3)^2 = 55.6% on kernel vs ~1/N on relay
(measured 57%). window_drop is NOT exposed: the delivered
implementation advances the window with no gap, making 'outside the
window packets pass' unreachable (needs outage/period fields —
recorded in the spec); flow_drop awaits a parameter-shape decision.

Also fixes a pre-existing restart bug the e2e exposed: _bridges (relay
bridge names) was never cleared on stop, so restarting a node with a
relay link skipped 'bridge create' and failed on add_nio_udp.

e2e (isolated instance): 13/13 — pref-1 tc_impair classifier on both
veths, 57% round-trip for N=3, -1 = exact 100%, quota hard cutoff,
delay coexistence, clean teardown, suspend/resume, node restart,
relay-leg frequency_drop regression through an Ethernet switch. Full
suite: 1925 passed.
2026-10-06 00:21:08 +08:00
YueGuobin
d4b57d49fb
feat(docker): expose netem-extension packet filters on kernel links
Surface the netem extensions delivered in uBridge feature/tc-netem-ext
as GNS3 filter types: rate (tc-style bandwidth string), reorder (needs
delay), gemodel (Gilbert-Elliot bursty loss, exclusive with
packet_loss), duplicate, seed and limit, plus the delay jitter
distribution and the loss/dup correlation parameters. All are
kernel-datapath only — the relay's packet-filter registry has no
equivalent, so available_filters hides them on relay links, setting one
on a created relay link returns 409, and the project-load path drops
them with a warning (update_filters runs before the datapath is
decided, so it cannot reject there). Validation mirrors the tc grammar:
reorder requires delay, gemodel and packet_loss are mutually exclusive,
a distribution requires jitter > 0, rate is integer+unit capped at
100gbit.

The translation in DockerVM._netem_command_parts emits the frozen
grammar order and gates extension keywords on the 'tc capabilities'
netem token list (probed once per uBridge process; a plain
delay/loss/corrupt/dup filter never probes, keeping old uBridge builds
untouched; the correl suffix has no token and gates on 'rate' as the
extension-build marker).

Reconcile became reset-before-set: the kernel's netem change merges
optional attributes (rate, correlation, reorder, corrupt, gemodel,
distribution, seed keep their previous value when the new message omits
them), so removing a parameter needs an explicit 'tc reset' — the
bpf_drop filters are re-added right after in the same flow, so the
clsact churn is invisible. The capability check runs before the reset
so a failed update leaves the previous qdisc in place.

e2e (isolated instance, kernel link): 24/24 — rate 512kbit adds exactly
2x the per-packet serialization to the RTT (45.5 ms for 1400-byte
pings), paretonormal jitter + rate-cleared leak check, reorder gap 5,
gemodel 50% bursty loss, duplicate, loss 30% correl 50%, seed 42 +
limit 5000 visible in the qdisc dump, suspend/resume and node restart
keep the filters, relay link rejects rate with 409, teardown leaves no
netem. Full unit suite: 1916 passed.
2026-10-06 00:21:08 +08:00
YueGuobin
0a8d430d50
feat(docker): packet filters on kernel-datapath links via tc netem
Impairment filters with a netem equivalent (delay, packet_loss, corrupt)
now run on the kernel datapath as one tc netem qdisc per veth host end,
pushed through uBridge's netlink tc module (tc netem set / tc reset).
This removes the relay delay filter's nanosleep bottleneck (upstream

Both link endpoints carry the filters: each end's qdisc egress covers
the traffic entering that container, so every direction is impaired
exactly once -- the same net effect as the relay, where both directions
cross the single filtered bridge.

- controller: eligibility keys on relay-only types (frequency_drop, bpf)
  instead of any filter; kernel links push the real filters (the
  synthetic suspend frequency_drop no longer flips a suspended link to
  relay on reopen/reset); update_filters 409s only relay-only types;
  available_filters hides them on kernel links
- compute: create_nio whitelists netem-capable filter types; the NIO
  update path keeps its own guard (PUT mutates the NIO in place);
  _ubridge_apply_netem translates the filter dict into one atomic
  netem-set (full re-apply IS the reconcile); node restart restores the
  qdisc, link deletion detaches it (the veth survives the link)
- e2e (27/27): delay 100 -> 200.2 ms RTT, atomic replace 50 -> 100.2 ms,
  clear -> baseline, packet_loss 30 -> 48 % round-trip (expected 51 %),
  suspend/resume and node restart keep the qdisc, link delete leaves no
  residual impairment
2026-10-06 00:21:08 +08:00
YueGuobin
672efda592
feat(docker): unify adapter interfaces on veth, allow kernel links at runtime
Every Docker adapter is now born as a veth pair at container start
(unified interface) regardless of NIO type: guest end in the container
namespace, host end in the root namespace. Kernel links enslave the host
end into the per-link Linux bridge; relay links attach it to the uBridge
relay via AF_PACKET (bridge add_nio_ethernet) — the TAP path is gone.

This makes the datapath a runtime decision:
- links can be created on running containers (kernel or relay) without
  touching container interfaces — the controller stopped-node
  eligibility check is removed
- project reopen re-prepares relay links and upgrades eligible ones to
  the kernel datapath
- packet capture start/stop now key on the NIO type (not veth presence)
  so relay-over-veth captures use the relay capture, kernel ones the
  AF_PACKET module
- vendor unix-socket containers keep their wiring: no veth is required
  on the relay path
2026-10-06 00:21:08 +08:00
YueGuobin
61020d3bbc
feat(docker): traffic-insight markers on kernel-datapath links via AF_PACKET
Markers no longer disqualify the kernel datapath. They ride the NIO like
on the relay path and attach to the veth host end through uBridge's
marker add_kernel/delete_kernel/enable_kernel commands (AF_PACKET taps,
same BPF/direction/signal semantics as the relay mark filter).

- base_node: marker anchors are now polymorphic (relay bridge name or
  kernel veth host interface); the reconcile loop is datapath-agnostic
  and DockerVM overrides the add/delete/enable primitives to translate
  the commands. Marker-name validation extracted for both add paths.
- docker_vm: kernel NIO connect/update apply markers on the veth anchor;
  link deletion tears markers down explicitly (the veth survives the
  link, unlike a relay bridge whose death drops its filters).
- controller: eligibility and start_marker no longer exclude markers or
  project-level marker definitions; the kernel NIO carries the capture
  node's marker specs.
- NIO creation accepts markers on kernel NIOs (filters still rejected).
2026-10-06 00:21:07 +08:00
YueGuobin
f6b869be63
feat(docker): packet capture on kernel-datapath links via AF_PACKET
Replace the capture rejection with wiring to uBridge's capture module:
an AF_PACKET socket bound to the veth host end writes into the same pcap
file the relay path uses (capture start_kernel <if> "<pcap>" /
capture stop_kernel), so the pcap streaming to clients is unchanged.

- capture survives node restarts: _connect_nio re-arms it when the NIO
  is already capturing, mirroring the relay path
- uBridge's kernel capture is a singleton per process: a concurrent
  second capture on the same node surfaces uBridge's EALREADY error
- update the datapath docs: filters and markers remain relay-only,
  capture is now served on both datapaths
2026-10-06 00:21:07 +08:00
YueGuobin
58b97b4ce5
fix(controller): don't push the synthetic suspend filter to kernel links
update_suspend() piggybacks on the filter system: get_active_filters()
returns {"frequency_drop": [-1]} while a link is suspended so relay-based
node types drop all traffic at the uBridge filter. Kernel-datapath links
implement suspend natively via interface carrier, and pushing the
synthetic filter tripped the compute-side guard, making suspend fail
with 409 on every kernel link (and leaving controller/kernel state torn).
_get_node_filters() now returns empty filters for kernel links.
2026-10-06 00:21:07 +08:00
YueGuobin
37d7b9121e
feat(docker): wire Docker-to-Docker links through a kernel bridge
Eligible links (both endpoints Docker, same compute, no filters/markers,
stopped nodes, no unix-socket NIO containers) are wired as veth pairs
enslaved into a per-link Linux kernel bridge (gns3 + 11 hex of the link
id, filling IFNAMSIZ) instead of the uBridge UDP relay, so frames flow
entirely in the kernel.

- NIOBridge NIO type: the controller decides the datapath in
  UDPLink._prepare and both computes enslave their deterministic gv*/gc*
  veth pair via uBridge's docker/brctl modules
- carrier/suspend = host-end admin state; both endpoints race-safely
  create/delete the bridge (EEXIST/EBUSY/ENOENT tolerated)
- veth adapters can still attach relay (UDP) links on a running node:
  lazy relay bridge over AF_PACKET on the host end (add_nio_ethernet)
- filters/markers/capture are rejected with explicit errors on this
  datapath (they live in the uBridge relay); vendor unix-socket
  containers are excluded; Server.enable_kernel_datapath is the
  kill-switch (default on)
- host-end veths survive container death and are removed on stop plus
  swept by name on the next start (crash/stale recovery)
2026-10-06 00:21:07 +08:00
Mark Paronyan
7a9c600f54
refactor: use f-strings in controller modules 2026-10-03 19:44:01 +03:00
Mark Paronyan
b622ccae7b
refactor: declare explicit public API in package __init__ modules 2026-10-03 19:43:59 +03:00
Mark Paronyan
306d8fbde4
refactor: remove unused imports 2026-10-03 19:43:58 +03:00
Mark Paronyan
55df230864 style: sort imports in controller 2026-10-02 12:43:26 +03:00
Mark Paronyan
a77e06b907
Merge branch '3.1' into ruff/trivia-fixups 2026-10-01 19:58:02 +03:00
Mark Paronyan
2dc7479d11
style: remove f-strings without placeholders 2026-10-01 19:57:46 +03:00
Mark Paronyan
c5b5091f5e
refactor: use explicit conversion in f-strings 2026-10-01 19:57:41 +03:00
Mark Paronyan
06e98f8b55
style: drop redundant utf-8 coding declarations 2026-10-01 19:57:40 +03:00
Jeremy Grossmann
a2cdc14a25
Merge branch '3.1' into image-manager-out-of-sync 2026-10-01 14:44:24 +02:00
Mark Paronyan
472c1b61fd
fix(typing): resolve mypy errors in controller.project 2026-09-29 22:02:23 +03:00
Mark Paronyan
783f243230
fix(typing): resolve mypy errors in controller.appliance_manager 2026-09-29 22:02:23 +03:00
Jeremy Grossmann
76411f1eb0
Merge pull request #2924 from markparonyan/mypy-api-plus
fix(typing): resolve mypy errors in api + compute.dynamips + compute.iou + compute.vpcs + controller
2026-09-29 13:29:08 +02:00
Mark Paronyan
d783ccfc0d
fix(typing): resolve mypy errors in controller.topology 2026-09-29 09:45:36 +03:00
Mark Paronyan
a7b32ca6df
fix(typing): resolve mypy errors in controller.appliance 2026-09-29 09:38:24 +03:00
Jeremy Grossmann
7864a4e567
Merge pull request #2897 from markparonyan/mypy-api-controller-links
fix(typing): resolve mypy errors in api.routes.controller.links
2026-09-28 23:26:48 +02:00
Cristi
42d30651da Merge origin/3.1 into image-manager-out-of-sync
Preserve image reconciliation, locking, and regression coverage while integrating upstream release, dependency, typing, and formatting updates. Align feature code with the new Ruff checks.
2026-09-28 23:16:24 +03:00
Mark Paronyan
ecff4beb65
fix(typing): resolve mypy errors in api.routes.controller.links 2026-09-28 21:47:46 +03:00
Cristi
046dff4d08 fix(images): reconcile the image inventory with filesystem changes 2026-09-28 16:01:36 +03:00
Mark Paronyan
21643fe89d
fix(typing): resolve mypy errors in controller.compute 2026-09-28 04:56:59 +00:00
Mark Paronyan
a0450b6bce
refactor: ruff autofixes 2026-09-27 15:55:10 +03:00
Cristi
da16dc9874 fix(qemu): Fixed missing image replacement for linked-clone nodes 2026-09-18 23:58:49 +03:00
Cristi
59a367b88d Allow projects containing missing images to open in a degraded state instead of failing to load 2026-09-18 15:06:17 +03:00
YueGuobin
be62e8c022
feat: keep Docker images on computes consistent with the controller host
Only relying on the image name lets a moved tag (e.g. a newer :latest)
silently serve stale content from a compute that already has an image
under the same name. When creating a Docker node, the controller now
pins the image id (Id from the Docker daemon on the controller host)
into the create payload. A compute holding a different image under the
same tag reports the image as missing, which routes it through the
image sync added by the previous commit and re-aligns the tag.

No new template fields or database changes: the controller host daemon
remains the source of truth and the pin is resolved per creation. When
the image is not available on the controller host the pin is omitted
and behavior is unchanged (the compute pulls from the repository).
2026-09-14 00:55:18 +08:00
YueGuobin
c477812332
feat: sync Docker images from the controller to remote computes
When a Docker node is created on a remote compute whose Docker daemon
does not have the image, the compute now raises ImageMissingError
instead of blindly pulling from the Docker repository. The controller
exports the image from the Docker daemon on its host (docker save
stream) and streams it to the compute which loads it, so locally built
or docker-loaded images work across computes. When the image is not
available on the controller host either, the compute is asked to pull
it from the Docker repository as a fallback.

- add a POST /docker/images/load compute endpoint that streams a
  docker save tar into the Docker daemon
- let Docker.http_query pass raw (non-dict) request bodies through so
  the tar can be streamed to the daemon
- drop the inline pull from DockerVM.create() and the now unused
  DockerVM.pull_image wrapper
2026-09-14 00:36:54 +08:00
YueGuobin
5d91ca0efc
fix: harden sharkd replay sessions and serve the uncapped frame list
Review-driven session/transport fixes (each reproduced live against
sharkd 4.6.7 before fixing):

- raise the RPC stream limit to 16 MB: a full 1000-row frames page
  measures ~190 KB against the 64 KB StreamReader default, which failed
  the request with a 500 and desynchronized the resident session; a
  line-over-limit ValueError is now treated as a transport failure
- verify JSON-RPC reply ids: a timed-out request's late reply was
  served as the next request's answer; timeouts, dead pipes, malformed
  and stale replies now kill the session for good instead
- make check-spawn atomic under one manager lock: concurrent requests
  for the same pcap double-spawned sharkd and leaked the loser (process
  plus /tmp scratch copy) forever
- refcount sessions and evict idle only (LRU, cap raised 8 -> 16): a
  tag with more sources than the cap respawned every source on every
  request, and concurrent requests could get their session killed
  mid-RPC (spurious 502)
- map FilterError to sharkd's filter rejection (-13002) only; other
  engine failures with a filter set are 502, not a client 400
- detail: accept an optional frame_number to disambiguate
  same-microsecond frames (ts is not unique within a pcap); drop the
  -8003 -> 404 mapping (the range is validated locally, engine errors
  are real faults); a failed hex read is a 404 instead of "hex": null
- a pcap deleted mid-request is a 404, not a 500; the pcap-sized
  scratch copy runs off the event loop; server shutdown kills every
  resident session and drops its scratch directory
- pin the packet-list layout through scratch-HOME Wireshark
  preferences: the column indexes are a contract the server owns
  (protocol-level column negotiation is rejected by sharkd 4.6.x)

Range contract change (WebUI moved to an always-flat list): the merged
frame list is returned in full, deliberately uncapped - truncated and
per-second buckets are removed, frame_count always equals
len(frames), and rendering cost is the client's concern (the window
endpoint remains the incremental path).
2026-09-11 00:55:18 +08:00
YueGuobin
ef5d81498a
feat: accept link=<link_id> on the replay range and frames endpoints
Narrows the merged frame stream to one capture source BEFORE counting,
slicing and bucketing (frame_count / frames | buckets all recomputed on
the narrowed set), AND-composing with the display filter. A pure
identity filter applied before any engine work — only the selected
link's pcap gets a sharkd pass, so link+filter is cheaper than filter
alone.

Two boundaries by contract with the Web UI:
- sources[] stays the tag's stable inventory: every capture source
  listed with engine-free TOTAL counts, unaffected by link/filter — a
  source dropdown must not shrink when the view narrows (this also
  settles sources[].count on total counts rather than post-filter
  matches, which no spec ever required)
- an unknown link_id matches nothing: frame_count 0, start null, empty
  frames/buckets — the same shape as a zero-match display filter,
  deliberately not a 404; an empty link param is treated as absent
2026-09-09 00:53:06 +08:00
YueGuobin
790c423c26
feat: drive marker replay with resident sharkd sessions
sharkd (the Wireshark daemon) is now the single decode engine — the
tshark/PDML path is gone, and without sharkd every replay endpoint
returns 501 (no degraded mode: one engine, one rendering shape for the
Web UI).

- Frame entries gain packet-list columns from sharkd's frames RPC:
  src/dst/proto/info plus the Wireshark coloring hints bg/fg
- range and frames accept ?filter=<display filter>, applied before
  counting and slicing; invalid expressions are 400 carrying sharkd's
  original text; filters travel as single argv-style elements, capped
  at 2000 chars; filtered frames keep their original pcap frame numbers
- frame detail returns sharkd's protocol tree with keys renamed into
  the REST contract (element/label/name/filter_expr/pos+size/expert/
  generated/children): a census-verified closed key set, values
  untouched, unknown keys passed through verbatim, Wireshark-internal
  hf ids dropped. filter_expr gives the UI click-to-filter; pos/size
  drives hex highlighting (hex still read straight from the pcap)
- one resident 'sharkd -' session per source pcap: lazy spawn, /tmp
  scratch copy + scratch HOME (hardened profiles), per-request
  (mtime,size) validation with respawn, LRU bound, per-session lock,
  per-RPC timeout, bounded close

Timeline backbone (gate, record-header scan, merge ordering, canonical
ts strings, hex reads) stays plain Python — identity and ordering never
depend on the engine.
2026-09-08 00:04:04 +08:00
YueGuobin
8b3aafbbdc
feat: IOU-style startup-config for IOL Docker nodes
Templates reference a config file with the GNS3_IOL_STARTUP_CONFIG
environment knob; the controller materializes the file content into
startup_config_content on node creation (sent once, knob consumed —
the same pattern as the IOU startup_config mapping). The compute builds
the content into the node's nvram_<app id> at the next start using the
IOU nvram_import utility (IOL and IOU share the nvram container format,
verified against iol-xe 17.18.02): valid config at boot, no setup
dialog, %h hostname substitution, hostname rewrite on rename.

Semantics verified against the runner: IOL boots from NVRAM whenever it
holds a config, so a plain stop/start never re-applies the startup
config and 'write memory' survives restarts; an explicit content edit
(PUT) is re-applied on the next start and wins over the saved config,
like IOU.
2026-09-05 21:54:52 +08:00
YueGuobin
a8fa529252
feat: add tag-keyed aggregate replay over paused markers' pcaps
Markers on different links sharing a tag form one distributed capture
session. Once every marker under the tag is paused (409 otherwise), three
read-only endpoints replay it:

- GET .../markers/tags/{tag}/replay/range merges the per-marker pcaps by
  scanning 16-byte record headers only (no tshark) into a timestamp-
  ordered frame list (per-second buckets above a 5000-frame cap)
- GET .../replay/frames?ts=&window_ms= returns frames in [T, T+window];
  an empty window is a normal empty array
- GET .../replay/frame/detail lazily decodes one frame the user opened:
  raw bytes for the hex view read straight from the pcap, protocol tree
  from 'tshark -T pdml' mapped isomorphically to JSON (every attribute
  survives, values stay strings). tshark reads a /tmp scratch copy with a
  scratch HOME — hardened profiles deny it the project directory.

Sort key is (ts, source file, frame number): ts is not unique across a
merge. The ts parameter round-trips as the exact string from the frame
list. Round-trip tests pin the PDML→JSON fidelity (element count and
attribute coverage).
2026-09-05 21:54:52 +08:00
YueGuobin
0119373690
feat: allocate IOL Docker application IDs from a pool disjoint from IOU
IOL interface MACs derive from the application ID (aabb.cc{app}{iface}),
so ids must be unique across opened projects sharing computes — the same
reason IOU has its allocator. IOL Docker nodes draw from the upper half
(512-1022, netiomux's fixed peer is 1023) so the two node types can
neither collide nor starve each other; IOU behavior is unchanged.

The controller sniffs the same GNS3_IOL_RUNNER environment marker the
compute uses to select IOLDockerVM (both the nested-properties and
template/top-level-kwarg shapes), stores the id in node properties like
IOU does, and passes it through the Docker create payload. Without an
allocation the compute falls back to a stable node-derived id in the
same upper range.
2026-09-05 21:54:52 +08:00
YueGuobin
0ca9ccc637
feat: model IOL adapters as 4-port units like the IOU node type
IOL interfaces come in 4-port units (Ethernet0/0-3, Ethernet1/0-3, ...),
addressed like IOU as (adapter_number, port_number 0-3):

- IOLDockerVM builds EthernetAdapter(interfaces=4) per adapter and asks
  the runner for adapters x 4 interfaces (num-eth).
- DockerVM threads port_number through the NIO/capture API (the compute
  routes parsed it from the URL but dropped it); single-port adapters
  keep the historical bridge{N} names and command sequence, multi-port
  adapters get one bridge per port (bridge{a}_{p}).
- The unix-socket NIO wiring addresses sockets flat across adapters:
  adapter x ports-per-adapter + port, so single-port images keep their
  exact socket layout.
- The controller port list for GNS3_IOL_RUNNER docker nodes is generated
  by StandardPortFactory with the IOU naming (Ethernet{segment0}/{port0},
  segment size 4); plain docker nodes keep eth{N}.
2026-09-05 21:54:51 +08:00
YueGuobin
823cf37287
fix: list every project in template/image in-use refusal, not just the first
The usage scans broke out of the whole (project, node) iteration at the
first matching node, so deleting a template or image used by several
projects reported only one project name while the error message says
'one or more projects'. Keep scanning and list each project once.

Caught on a live server: an IOU L3 template used by two projects
(one of them closed) was refused citing only one of them.
2026-08-31 23:04:00 +08:00
YueGuobin
13548deae8
feat: forbid deleting templates and images still used by projects
Deleting a template with prune_images, deleting an image, or pruning
orphan images only checked template references — a project node still
pointing at the image (e.g. via hda_disk_image_backing_file) was left
with a dangling reference and the project could no longer be opened.

- Add controller helpers scanning every known project (opened projects
  via in-memory nodes, closed projects via their .gns3 file) for
  template and image usage; unreadable topologies are skipped
- Guard DELETE /templates/{id}, DELETE /templates/{id}?prune_images,
  DELETE /images/{path} and /images/prune with a 409 listing the
  project names
- Run all template-delete checks before any mutation so a refused
  deletion cannot leave the template gone while its images survive
2026-08-31 22:30:42 +08:00
YueGuobin
8a8314ab29
fix: dedupe and report automatic template creation from images
install_appliances_from_image relied on the name+version pair check in
TemplatesService, so the same appliance reached through a second image
(the CSR1000v case) created a second template sharing the name. The auto
path now skips when any template with the same name exists, whatever the
version, and returns a manifest of created and skipped candidates;
POST /images/install replies 200 with that manifest instead of an empty
204, and the image_install MCP tool surfaces it.
2026-08-26 00:04:57 +08:00
YueGuobin
888afdccbd
fix: return the created template from appliance install
POST /appliances/{id}/install replied 204 with an empty body, so the MCP
appliance_install tool crashed with 'Expecting value: line 1 column 1'
while the template had actually been created. The route now returns the
created template (201, response_model=schemas.Template), _create_template
propagates it, and the MCP handler parses the body defensively so an
empty reply degrades to a plain success message.
2026-08-25 23:39:38 +08:00
YueGuobin
97e7a79117
fix: report empty projects as not locked
GET /projects/{id}/locked returned True for a project with no drawings
or nodes: both loops ran zero times and the fallback return won. Locking
and unlocking such a project always succeeded while the state stayed
"locked", so it could never be unlocked.

Report a project with nothing to lock as not locked, and re-check the
state after unlock in the route tests.
2026-08-25 13:45:36 +08:00
YueGuobin
702fc1f6d9
fix: never allow the projects directory to become a project directory
Loading a .gns3 placed directly in the projects root registered the
shared projects directory as the project path (load_project derives
the path from the file's parent directory). Deleting such an entry ran
rmtree on the projects directory itself, wiping every project until a
root-owned file stopped it, and left a zombie entry in the controller.

Three layers of protection:

- Controller.load_project() refuses a .gns3 whose parent directory is
  the projects directory; the normal subdirectory layout is unaffected
- the Project.path setter rejects the projects directory itself and
  its ancestors, closing the same hole for POST/PUT with an explicit
  path
- Project.delete() uses realpath + commonpath instead of commonprefix:
  entries whose path is the projects root are refused, and sibling
  directories sharing a string prefix (/srv/projects-evil vs
  /srv/projects) are no longer treated as inside the projects dir

Also removes the project_load MCP tool: loading by raw server
filesystem path is a footgun for automated clients; projects can still
be opened by project_id via the remaining tools.
2026-08-25 13:17:37 +08:00