feat(docker): eBPF stateful packet filters on kernel links

Consume the eBPF impairment classifier delivered in uBridge
feature/tc-precision: frequency_drop becomes the exact every-Nth mode
(tc nth_drop; -1 = drop everything maps to every 1st) and the new
kernel-only quota type is the byte-cap mode (tc quota_drop, 100 = hard
cutoff). KERNEL_UNSUPPORTED_FILTERS is gone entirely — kernel/relay
eligibility is now purely topological. Both modes gate on ebpf=1 from
the per-process tc capabilities probe (error names the setcap fix); the
cleanup path (modes off) only talks to the eBPF commands when the probe
reported them. Reconcile is a plain re-set: the netem reset-before-set
already detaches the program and clears uBridge's per-interface
registry, so a mode set after it is a fresh load+attach.

Documented semantic difference: the relay counts both directions
through one bridge counter, the kernel counts per direction — N=3
round-trip loss is 1-(2/3)^2 = 55.6% on kernel vs ~1/N on relay
(measured 57%). window_drop is NOT exposed: the delivered
implementation advances the window with no gap, making 'outside the
window packets pass' unreachable (needs outage/period fields —
recorded in the spec); flow_drop awaits a parameter-shape decision.

Also fixes a pre-existing restart bug the e2e exposed: _bridges (relay
bridge names) was never cleared on stop, so restarting a node with a
relay link skipped 'bridge create' and failed on add_nio_udp.

e2e (isolated instance): 13/13 — pref-1 tc_impair classifier on both
veths, 57% round-trip for N=3, -1 = exact 100%, quota hard cutoff,
delay coexistence, clean teardown, suspend/resume, node restart,
relay-leg frequency_drop regression through an Ethernet switch. Full
suite: 1925 passed.
This commit is contained in:
YueGuobin 2026-09-29 23:35:03 +08:00
parent d4b57d49fb
commit 0c4ec8c38c
No known key found for this signature in database
11 changed files with 449 additions and 159 deletions

View File

@ -4,17 +4,16 @@ See LICENSE file for licensing information.
--> -->
> Frozen requirements spec for the **uBridge** project. Delivery status: > Frozen requirements spec for the **uBridge** project. Delivery status:
> parts **A** (netem keyword extensions, uBridge `feature/tc-netem-ext`), **C** > **all parts delivered and integrated.** A (netem keyword extensions,
> (cBPF match-drop) and **D + E** (full-restore idempotent `tc reset`, > uBridge `feature/tc-netem-ext`), C + D + E (cBPF match-drop, full-restore
> `tc capabilities` — uBridge `feature/tc-bpf-drop`) are **delivered and > idempotent `tc reset`, `tc capabilities` — uBridge `feature/tc-bpf-drop`)
> integrated** on gns3-server branches `feat/docker-kernel-bpf-drop` (C/D/E) > and **B (eBPF stateful classifier — uBridge `feature/tc-precision`
> and `feat/docker-kernel-netem-ext` (A: rate/reorder/gemodel/duplicate/ > 59e2b38, including the non-root setcap load fix)** are consumed by
> seed/limit/distributions/correlation exposed as GNS3 filter types). > gns3-server on the branch stack ending at `feat/docker-kernel-ebpf-drops`
> Part **B** (eBPF stateful classifier) is delivered in the uBridge > (B: `frequency_drop` → `tc nth_drop`, the new kernel-only `quota` type →
> `feature/tc-precision` branch but **not yet consumed** by gns3-server. > `tc quota_drop`). The originally shipped netem surface (delay/jitter/
> The originally shipped netem surface (delay/jitter/loss/dup/corrupt) > loss/dup/corrupt) exists in uBridge 1.2.3+ and is verified against
> exists in uBridge 1.2.3+ and is verified against gns3-server branch > gns3-server branch `feat/docker-kernel-filters`.
> `feat/docker-kernel-filters`.
> >
> **Deviation found during integration (part A/D):** the delivered > **Deviation found during integration (part A/D):** the delivered
> `tc netem set` uses NLM_F_REPLACE, but the *kernel's* netem change merges > `tc netem set` uses NLM_F_REPLACE, but the *kernel's* netem change merges
@ -27,6 +26,20 @@ See LICENSE file for licensing information.
> invisible). If uBridge later emits explicit zero/absent clears for every > invisible). If uBridge later emits explicit zero/absent clears for every
> optional attribute, the server-side reset-before-set can be dropped; the > optional attribute, the server-side reset-before-set can be dropped; the
> two behaviours are compatible either way. > two behaviours are compatible either way.
>
> **Deviation found during B integration:** `window_drop`'s implementation
> advances the window start by one length on expiry with no gap, making the
> windows back-to-back — after the first `start_ms` the "outside the window
> packets pass" behaviour described in B.2 is unreachable (inside ≡ always).
> gns3-server therefore does **not** expose window_drop yet; it needs either
> a second length field (outage vs period) or a next-start semantics fix on
> the uBridge side. `flow_drop` is delivered and functional but not yet
> exposed as a GNS3 filter type (its mask/target parameter shape needs a
> UX decision). Additionally, the delivered B.1 program had to fold the
> flow-hash L4 port reads to constant offsets (IHL==20 only) because the
> verifier prohibits variable packet-pointer arithmetic for non-root —
> even with CAP_BPF (Spectre-mitigation gating by uid); documented in
> uBridge's doc/tc.md.
# uBridge kernel impairment: tc netem extensions + eBPF classifiers # uBridge kernel impairment: tc netem extensions + eBPF classifiers
@ -293,11 +306,14 @@ corresponding filter types per capability.
in `DockerVM._ubridge_apply_netem`; extension keywords are gated on the in `DockerVM._ubridge_apply_netem`; extension keywords are gated on the
`tc capabilities` netem token list (old uBridge builds are never probed `tc capabilities` netem token list (old uBridge builds are never probed
for the original surface). for the original surface).
* **Done** (`feat/docker-kernel-bpf-drop`): `bpf` runs as cBPF match-drop * **Done** (`feat/docker-kernel-bpf-drop`): `bpf` runs as cBPF match-drop.
(`KERNEL_UNSUPPORTED_FILTERS` shrank to `frequency_drop`). * **Done** (`feat/docker-kernel-ebpf-drops`): the eBPF classifier (part B)
* `frequency_drop` moves out of `KERNEL_UNSUPPORTED_FILTERS` once the eBPF is consumed — `frequency_drop` → `tc nth_drop` (-1 → every 1st), the new
classifier (part B, uBridge `feature/tc-precision`) is consumed on the kernel-only `quota` type → `tc quota_drop`, both keyed on `ebpf=1` from
server side, keyed on `ebpf=1` per compute. the per-process `tc capabilities` probe. `KERNEL_UNSUPPORTED_FILTERS` is
gone entirely: kernel/relay eligibility is purely topological now.
Not yet exposed: `window_drop` (implementation deviation above),
`flow_drop` (parameter-shape UX decision).
* Every veth end owns one qdisc + its filters: per-direction impairment is * Every veth end owns one qdisc + its filters: per-direction impairment is
an architectural freebie to expose later (API `direction` field), aligned an architectural freebie to expose later (API `direction` field), aligned
with marker `dir` semantics. with marker `dir` semantics.

View File

@ -24,13 +24,14 @@ Frames never leave the kernel.
| Link | uBridge bridge, TAP fd ↔ UDP, userspace copy | per-link kernel bridge `gns3{link_id[:11]}`, zero-copy | | Link | uBridge bridge, TAP fd ↔ UDP, userspace copy | per-link kernel bridge `gns3{link_id[:11]}`, zero-copy |
| Relay latency | ~0.3 ms RTT | ~0.05 ms RTT (measured) | | Relay latency | ~0.3 ms RTT | ~0.05 ms RTT (measured) |
| Link attach at runtime | yes (relay attaches the pre-existing TAP) | yes (brctl addif / add_nio_ethernet — interface untouched) | | Link attach at runtime | yes (relay attaches the pre-existing TAP) | yes (brctl addif / add_nio_ethernet — interface untouched) |
| Packet filters | uBridge userspace filters | tc netem on the veth host end (delay / packet_loss / corrupt / duplicate + the netem extensions rate, reorder, gemodel, seed, limit, jitter distributions, loss correlation); bpf as cls_bpf match-drop classifiers; frequency_drop stays relay-only (eBPF classifier pending) | | Packet filters | uBridge userspace filters | tc netem on the veth host end (delay / packet_loss / corrupt / duplicate + the netem extensions rate, reorder, gemodel, seed, limit, jitter distributions, loss correlation); bpf as cls_bpf match-drop classifiers; frequency_drop and quota as the eBPF stateful classifier — every filter type has a kernel equivalent |
| Capture / markers | relay bridge | uBridge AF_PACKET modules on the veth host end | | Capture / markers | relay bridge | uBridge AF_PACKET modules on the veth host end |
The work landed in seven stages on branch stack `feat/docker-kernel-datapath` → The work landed in eight stages on branch stack `feat/docker-kernel-datapath` →
`feat/docker-kernel-capture` → `feat/docker-kernel-markers` → `feat/docker-kernel-capture` → `feat/docker-kernel-markers` →
`feat/docker-veth-everywhere` → `feat/docker-kernel-filters` → `feat/docker-veth-everywhere` → `feat/docker-kernel-filters` →
`feat/docker-kernel-bpf-drop` → `feat/docker-kernel-netem-ext`: `feat/docker-kernel-bpf-drop` → `feat/docker-kernel-netem-ext` →
`feat/docker-kernel-ebpf-drops`:
1. **Kernel datapath** — NIOBridge NIO type, per-link bridges, carrier-based 1. **Kernel datapath** — NIOBridge NIO type, per-link bridges, carrier-based
suspend, crash-safe reconciliation. suspend, crash-safe reconciliation.
@ -57,6 +58,14 @@ The work landed in seven stages on branch stack `feat/docker-kernel-datapath`
process. Reconcile became reset-before-set: the kernel's netem replace process. Reconcile became reset-before-set: the kernel's netem replace
merges optional attributes, so removing a parameter needs the explicit merges optional attributes, so removing a parameter needs the explicit
reset. reset.
8. **eBPF stateful drops** — `frequency_drop` becomes the eBPF classifier's
exact every-Nth mode (`tc nth_drop`, uBridge `feature/tc-precision`;
-1 = drop everything → every 1st) and the new kernel-only `quota` type
the byte-cap mode (`tc quota_drop`). Needs a uBridge reporting `ebpf=1`
(setcap cap_bpf,cap_net_admin,cap_net_raw=ep — verified working for a
non-root server process, the production credential shape). With this,
no filter type forces the relay anymore: kernel/relay is a purely
topological choice.
## Adapter interface types ## Adapter interface types
@ -89,8 +98,8 @@ Kernel link between two containers on the same compute:
└───────────────────┘ └───────────────────┘
``` ```
Relay link on the same unified veth (fallback for filtered links or when Relay link on the same unified veth (fallback for mixed node types, cross-
`enable_kernel_datapath` is off — e.g. to reach another compute): compute wiring or when `enable_kernel_datapath` is off):
``` ```
┌────────────┐ ┌────────────────────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────────────────────┐ ┌────────────┐
@ -110,10 +119,11 @@ time; the compute side only reacts to the NIO type:
* both endpoints Docker (any class except unix-socket containers), same * both endpoints Docker (any class except unix-socket containers), same
compute compute
* no active packet filters (filters live in the relay; kernelization via tc
netem is planned)
* `Server.enable_kernel_datapath` enabled (default) * `Server.enable_kernel_datapath` enabled (default)
Filters never disqualify the kernel path anymore — every type has a kernel
equivalent (netem / cls_bpf / eBPF classifier).
There is **no stopped-node requirement**. Since every adapter is a veth, links There is **no stopped-node requirement**. Since every adapter is a veth, links
attach to running containers: `brctl addif` (kernel) or attach to running containers: `brctl addif` (kernel) or
`bridge add_nio_ethernet` (relay) are runtime-safe operations on the host end. `bridge add_nio_ethernet` (relay) are runtime-safe operations on the host end.
@ -189,7 +199,8 @@ ends (uBridge `tc netem set`, raw netlink — no `tc` binary needed):
| `seed [u32]` | netem `seed N` | reproducible random draws; kernel-only type | | `seed [u32]` | netem `seed N` | reproducible random draws; kernel-only type |
| `limit [pkts]` | netem `limit N` | queue depth above the 1000 default (rate+delay BDP); kernel-only type | | `limit [pkts]` | netem `limit N` | queue depth above the 1000 default (rate+delay BDP); kernel-only type |
| `bpf` (one line per expression, OR) | `tc bpf_drop add <if> <prio> "<expr>"` — cls_bpf on clsact egress + gact drop | needs uBridge `cbpf=1` (`tc capabilities`, probed once per uBridge process); a line that fails to compile on the compute is skipped with a warning, mirroring the relay | | `bpf` (one line per expression, OR) | `tc bpf_drop add <if> <prio> "<expr>"` — cls_bpf on clsact egress + gact drop | needs uBridge `cbpf=1` (`tc capabilities`, probed once per uBridge process); a line that fails to compile on the compute is skipped with a warning, mirroring the relay |
| `frequency_drop` | — not yet (eBPF stateful classifier spec'd: [ubridge-kernel-impairment-spec](../design/ubridge-kernel-impairment-spec.md), part B pending) | 409 on kernel links; relay fallback | | `frequency_drop [N]` | `tc nth_drop <if> N` — the eBPF stateful classifier's exact every-Nth mode (clsact egress prio 1) | needs uBridge `ebpf=1`; -1 = drop everything → every 1st; per-direction counters (see below) |
| `quota [bytes, %]` | `tc quota_drop <if> <bytes> <pct>` — after the byte quota is consumed, each further packet drops with the given chance (100 = hard cutoff) | kernel-only type; needs `ebpf=1`; per-direction byte counters |
Semantics: Semantics:
@ -213,12 +224,24 @@ Semantics:
`tc capabilities` once per uBridge process and require the tokens; `tc capabilities` once per uBridge process and require the tokens;
a plain delay/loss/corrupt/dup filter never probes — an old uBridge a plain delay/loss/corrupt/dup filter never probes — an old uBridge
serves the original surface untouched. serves the original surface untouched.
* **kernel-only vs relay-only.** The extension types have no uBridge relay * **kernel-only vs relay-available.** The extension types and `quota` have
equivalent: `available_filters` hides them on relay links, setting one on no uBridge relay equivalent: `available_filters` hides them on relay
a created relay link returns 409, and a project loaded with such filters links, setting one on a created relay link returns 409, and a project
on a link that cannot be kernel-wired drops them with a warning (invalid loaded with such filters on a link that cannot be kernel-wired drops them
filters are dropped the same way at load). Symmetrically, with a warning (invalid filters are dropped the same way at load).
`frequency_drop` stays relay-only. `frequency_drop` runs on both datapaths (relay userspace filter / eBPF
every-Nth).
* **frequency_drop counting semantics differ per datapath.** The relay's
single filtered bridge counts packets of BOTH directions through one
counter; the kernel attaches one classifier per veth end, so each
direction drops every Nth independently. For a round-trip measurement
with every-Nth N: relay ≈ 1/N of frames (phase-locked), kernel = 1 −
((N−1)/N)² (e.g. N=3 → 55.6 % round-trip loss, measured 57 %). The
kernel count is exact (atomic counter), unlike netem's stochastic loss.
* **Classifier order** on clsact egress: the eBPF stateful filter runs at
prio 1, `bpf_drop` expressions at 10–99 — a packet dropped by the
stateful modes never reaches the expression drops, and dropped packets
never reach netem.
* **Validation** mirrors the tc grammar: `reorder` requires `delay`, * **Validation** mirrors the tc grammar: `reorder` requires `delay`,
`gemodel` and `packet_loss` are mutually exclusive (both map to the netem `gemodel` and `packet_loss` are mutually exclusive (both map to the netem
loss keyword), `distribution` requires jitter > 0, rate must be integer + loss keyword), `distribution` requires jitter > 0, rate must be integer +
@ -253,7 +276,9 @@ tc netem set <if> [delay <ms>] [jitter <ms>] [loss <%> [correl <%>] | loss gemod
[dup <%> [correl <%>]] [corrupt <%>] [reorder <%> [correl <%>] [gap <n>]] [dup <%> [correl <%>]] [corrupt <%>] [reorder <%> [correl <%>] [gap <n>]]
[rate <bw>] [limit <pkts>] [distribution uniform|normal|pareto|paretonormal] [seed <u32>] [rate <bw>] [limit <pkts>] [distribution uniform|normal|pareto|paretonormal] [seed <u32>]
tc bpf_drop add <if> <prio 10-99> "<expr>" / tc bpf_drop flush <if> tc bpf_drop add <if> <prio 10-99> "<expr>" / tc bpf_drop flush <if>
tc reset <if> # full restore: filters -> clsact -> root qdisc, idempotent tc nth_drop <if> <n | off> # eBPF exact every-Nth (frequency_drop)
tc quota_drop <if> <bytes> <pct> | off # eBPF byte cap (quota)
tc reset <if> # full restore: eBPF filter -> bpf_drops -> clsact -> root qdisc, idempotent
tc capabilities # "netem=<kw,...>;ebpf=0|1;cbpf=0|1", probed once per process tc capabilities # "netem=<kw,...>;ebpf=0|1;cbpf=0|1", probed once per process
bridge add_nio_ethernet / add_nio_udp / start / stop / start_capture / stop_capture bridge add_nio_ethernet / add_nio_udp / start / stop / start_capture / stop_capture
``` ```
@ -280,7 +305,8 @@ pairs), capture freeze on stop, server restart reconciliation.
Filters (two-container kernel link, 27/27 checks): baseline 0.06 ms → Filters (two-container kernel link, 27/27 checks): baseline 0.06 ms →
`delay 100` = 200.2 ms RTT (2× per direction) → `delay 50` = 100.2 ms `delay 100` = 200.2 ms RTT (2× per direction) → `delay 50` = 100.2 ms
(atomic replace) → clear = baseline (qdisc detached); `packet_loss 30` = (atomic replace) → clear = baseline (qdisc detached); `packet_loss 30` =
48 % round-trip loss (expected 51 % = 1 − 0.7²); `frequency_drop` = 409; 48 % round-trip loss (expected 51 % = 1 − 0.7²); `frequency_drop` = 409 (at that
stage — superseded by the eBPF stage below);
suspend with delay active = 100 % loss, resume keeps the qdisc suspend with delay active = 100 % loss, resume keeps the qdisc
(200.1 ms); node restart restores it (200.1 ms); link delete detaches it (200.1 ms); node restart restores it (200.1 ms); link delete detaches it
(`/usr/sbin/tc qdisc show` — no netem left); re-created link has no residual (`/usr/sbin/tc qdisc show` — no netem left); re-created link has no residual
@ -291,8 +317,9 @@ ends' clsact; size-discriminating expression (`greater 150`) = small pings
pass / 300-byte pings dropped (real byte-level BPF match); two-line OR; pass / 300-byte pings dropped (real byte-level BPF match); two-line OR;
coexistence with netem (delay 50 → small pings 100.2 ms while big dropped); coexistence with netem (delay 50 → small pings 100.2 ms while big dropped);
clear removes clsact entirely; suspend/resume keeps the drops; link delete + clear removes clsact entirely; suspend/resume keeps the drops; link delete +
recreate leaves no residual filters; `frequency_drop` still 409; recreate leaves no residual filters; `frequency_drop` still 409 and hidden
`available_filters` shows `bpf` and hides `frequency_drop`. Unit tests: (at that stage — superseded by the eBPF stage below); `available_filters` shows
`bpf`. Unit tests:
`tests/compute/docker/test_docker_kernel_datapath.py`, `tests/compute/docker/test_docker_kernel_datapath.py`,
`tests/controller/test_kernel_datapath_link.py`. `tests/controller/test_kernel_datapath_link.py`.
@ -307,11 +334,27 @@ cleared; `duplicate 50` link alive; `packet_loss 30 correl 50` shows
exactly 2×50 ms; suspend/resume and node restart keep the filters; a exactly 2×50 ms; suspend/resume and node restart keep the filters; a
docker↔ethernet-switch relay link rejects `rate` with 409 and its docker↔ethernet-switch relay link rejects `rate` with 409 and its
`available_filters` hides the kernel-only types while the kernel link shows `available_filters` hides the kernel-only types while the kernel link shows
them all (minus frequency_drop); `reorder` without `delay` and them all (frequency_drop included since the eBPF stage); `reorder` without `delay` and
`gemodel`+`packet_loss` are 409s; teardown leaves no netem on either veth. `gemodel`+`packet_loss` are 409s; teardown leaves no netem on either veth.
Also verified live: a relay attach (`bridge add_nio_ethernet`) on the Also verified live: a relay attach (`bridge add_nio_ethernet`) on the
admin-down veth host end fails in libpcap — the relay path now brings the admin-down veth host end fails in libpcap — the relay path now brings the
interface up before attaching. Unit tests: interface up before attaching.
eBPF stateful drops (same setup, 13/13 checks): `frequency_drop 3` installs
the tc_impair classifier on both veths (clsact egress pref 1, visible in
`tc filter show`); round-trip loss 57 % against the per-direction theory 1 −
(2/3)² = 55.6 % (each direction drops every 3rd independently — the relay's
single shared counter would give ≈ 1/3); `frequency_drop -1` = exactly
100 % loss; `quota 3000B/100 %` = pings pass until the byte cap then hard
cutoff; coexistence with `delay 50` (RTT 100.3 ms with every-4th drops);
clearing filters leaves no residual bpf filter; suspend = 100 % while down
with drops persisting after resume; node restart restores every-Nth on the
fresh veth; `available_filters` shows frequency_drop + quota on kernel
links and hides quota on relay links; a two-leg relay path through an
Ethernet switch still serves frequency_drop from the userspace filter
(56 % round-trip for N=3). The e2e also exposed and fixed a pre-existing
restart bug: the relay bridge-name registry was not cleared on stop, so a
node restart with a relay link skipped `bridge create` and failed. Unit tests:
`tests/utils/test_packet_filter_validation.py` (P6a class), `tests/utils/test_packet_filter_validation.py` (P6a class),
`tests/compute/docker/test_docker_kernel_datapath.py`, `tests/compute/docker/test_docker_kernel_datapath.py`,
`tests/controller/test_kernel_datapath_link.py`. `tests/controller/test_kernel_datapath_link.py`.

View File

@ -375,17 +375,6 @@ class BaseManager:
nio.suspend = nio_settings.get("suspend", False) nio.suspend = nio_settings.get("suspend", False)
elif nio_settings["type"] == "nio_bridge": elif nio_settings["type"] == "nio_bridge":
nio = NIOBridge(nio_settings["bridge"]) nio = NIOBridge(nio_settings["bridge"])
# Impairment filters with a tc netem equivalent run on the veth
# host end; the relay-only types are rejected here (the controller
# already keeps them off kernel links — this is the second guard).
from gns3server.utils.packet_filter_validation import KERNEL_UNSUPPORTED_FILTERS
unsupported = KERNEL_UNSUPPORTED_FILTERS.intersection(nio_settings.get("filters") or {})
if unsupported:
raise ComputeError(
"Packet filter(s) {} cannot run on a kernel-datapath link "
"(no uBridge relay in the forwarding path)".format(", ".join(sorted(unsupported)))
)
nio.filters = nio_settings.get("filters", {}) nio.filters = nio_settings.get("filters", {})
# Markers ride the NIO and attach to the veth host interface via # Markers ride the NIO and attach to the veth host interface via
# uBridge's AF_PACKET marker module (marker add_kernel). # uBridge's AF_PACKET marker module (marker add_kernel).

View File

@ -1438,6 +1438,11 @@ class DockerVM(BaseNode):
# The next start spawns a fresh uBridge process — its capabilities # The next start spawns a fresh uBridge process — its capabilities
# must be re-probed (see _ubridge_tc_capabilities). # must be re-probed (see _ubridge_tc_capabilities).
self._ubridge_tc_caps = None self._ubridge_tc_caps = None
# The uBridge process dies with every bridge it hosted — drop the
# relay bridge names too, or the next start would skip
# "bridge create" for them (the gate checks the name's presence)
# and fail on "bridge add_nio_udp: bridge doesn't exist".
self._bridges.clear()
# veth host ends live in the root namespace and survive container # veth host ends live in the root namespace and survive container
# death — remove them while the uBridge control channel is still up. # death — remove them while the uBridge control channel is still up.
await self._remove_kernel_veths() await self._remove_kernel_veths()
@ -1944,9 +1949,11 @@ class DockerVM(BaseNode):
await self._ubridge_apply_markers(host_ifc, nio) await self._ubridge_apply_markers(host_ifc, nio)
# Impairment filters become one tc netem qdisc on the veth host # Impairment filters become one tc netem qdisc on the veth host
# end (restored here on node restart, like the capture above); # end (restored here on node restart, like the capture above);
# bpf expressions become cls_bpf match-drop classifiers. # bpf expressions become cls_bpf match-drop classifiers and
# frequency_drop/quota become the eBPF stateful classifier.
await self._ubridge_apply_netem(host_ifc, nio.filters) await self._ubridge_apply_netem(host_ifc, nio.filters)
await self._ubridge_apply_bpf_drops(host_ifc, nio.filters) await self._ubridge_apply_bpf_drops(host_ifc, nio.filters)
await self._ubridge_apply_ebpf_drops(host_ifc, nio.filters)
return return
# Relay NIO. On a veth-backed adapter (the unified Docker interface) # Relay NIO. On a veth-backed adapter (the unified Docker interface)
@ -2015,28 +2022,19 @@ class DockerVM(BaseNode):
if self.ubridge: if self.ubridge:
if isinstance(nio, NIOBridge): if isinstance(nio, NIOBridge):
# The controller keeps frequency_drop off kernel links (no
# kernel equivalent until the eBPF classifier lands) and
# create_nio rejects it on the POST path; this PUT path
# mutates the NIO in place, so guard here too. bpf is fine —
# it becomes cls_bpf match-drop classifiers (cbpf-probed).
unsupported = KERNEL_UNSUPPORTED_FILTERS.intersection(nio.filters or {})
if unsupported:
raise DockerError(
"Packet filter(s) {} cannot run on a kernel-datapath link "
"(no uBridge relay in the forwarding path)".format(", ".join(sorted(unsupported)))
)
if self.status == "started": if self.status == "started":
host_ifc = self._kernel_veths.get((adapter_number, port_number)) host_ifc = self._kernel_veths.get((adapter_number, port_number))
if host_ifc is not None: if host_ifc is not None:
# Incremental marker reconcile on the veth anchor — # Incremental marker reconcile on the veth anchor —
# the relay-datapath equivalent of the branch below. # the relay-datapath equivalent of the branch below.
await self._ubridge_apply_markers(host_ifc, nio) await self._ubridge_apply_markers(host_ifc, nio)
# netem set is an atomic replace, so a full re-apply # The netem apply resets the interface first (the
# IS the incremental reconcile for impairment filters; # kernel merges optional netem attrs on replace), so
# bpf drops reconcile as flush + re-add. # everything anchored on clsact must re-apply after
# it: bpf drops flush + re-add, eBPF modes re-set.
await self._ubridge_apply_netem(host_ifc, nio.filters) await self._ubridge_apply_netem(host_ifc, nio.filters)
await self._ubridge_apply_bpf_drops(host_ifc, nio.filters) await self._ubridge_apply_bpf_drops(host_ifc, nio.filters)
await self._ubridge_apply_ebpf_drops(host_ifc, nio.filters)
await self._set_adapter_carrier(adapter_number, not nio.suspend, port_number) await self._set_adapter_carrier(adapter_number, not nio.suspend, port_number)
return return
@ -2376,6 +2374,70 @@ class DockerVM(BaseNode):
log.warning(message) log.warning(message)
self.project.emit("log.warning", {"message": message}) self.project.emit("log.warning", {"message": message})
async def _ubridge_apply_ebpf_drops(self, host_ifc, filters):
"""
Translate the NIO's stateful filters into uBridge's eBPF impairment
classifier on the veth host end's clsact egress (prio 1, below
bpf_drop's 10-99): ``frequency_drop`` becomes the exact every-Nth
mode (``tc nth_drop``; -1 = drop everything maps to every 1st), and
the kernel-only ``quota`` type the byte-cap mode (``tc quota_drop``).
Reconcile is a plain re-set on every apply: the preceding netem
apply resets the interface (which detaches the program and drops
uBridge's per-interface registry entry), so a mode set re-loads the
program from scratch, and a mode left absent is turned off
explicitly (idempotent) so a removed filter stops dropping.
Semantic difference vs the relay, documented server-side: the relay
counts packets of BOTH directions through its single filtered
bridge, while each veth end's classifier counts only the traffic
entering that container — with both endpoints applying theirs, each
direction drops every Nth independently.
:param host_ifc: veth host-end interface name
:param filters: NIO filters dictionary
"""
def values_of(key):
values = filters.get(key)
if isinstance(values, (list, tuple)):
return list(values)
return [values] if values else []
frequency = values_of("frequency_drop")
quota = values_of("quota")
if not frequency and not quota and self._ubridge_tc_caps is None:
# Nothing to set and nothing was ever installed (no probe yet) —
# skip touching an old uBridge entirely.
return
caps = await self._ubridge_tc_capabilities()
if not frequency and not quota:
if caps.get("ebpf") == "1":
# Only when the eBPF module answers can anything be
# installed; "off" is idempotent in uBridge.
await self._ubridge_send(f'tc nth_drop "{host_ifc}" off')
await self._ubridge_send(f'tc quota_drop "{host_ifc}" off')
return
if caps.get("ebpf") != "1":
raise DockerError(
"Packet filter(s) {} on a kernel-datapath link need a uBridge with eBPF support "
"(tc capabilities reports no ebpf; setcap cap_bpf,cap_net_admin,cap_net_raw=ep on "
"the uBridge binary); upgrade uBridge on this compute or keep the link on the "
"relay datapath".format(", ".join(sorted(f for f in ("frequency_drop", "quota") if f in filters)))
)
if frequency:
# relay semantics: -1 = drop everything, N = every Nth packet
nth = 1 if int(frequency[0]) == -1 else int(frequency[0])
await self._ubridge_send(f'tc nth_drop "{host_ifc}" {nth}')
else:
await self._ubridge_send(f'tc nth_drop "{host_ifc}" off')
if quota:
await self._ubridge_send(
'tc quota_drop "{ifc}" {bytes} {pct}'.format(ifc=host_ifc, bytes=int(quota[0]), pct=int(quota[1]))
)
else:
await self._ubridge_send(f'tc quota_drop "{host_ifc}" off')
async def _remove_kernel_markers(self, host_ifc): async def _remove_kernel_markers(self, host_ifc):
""" """
Tear down every kernel marker anchored to *host_ifc*: the veth survives Tear down every kernel marker anchored to *host_ifc*: the veth survives

View File

@ -46,7 +46,8 @@ FILTERS = [
{ {
"type": "frequency_drop", "type": "frequency_drop",
"name": "Frequency drop", "name": "Frequency drop",
"description": "It will drop everything with a -1 frequency, drop every Nth packet with a positive frequency, or drop nothing", "description": "It will drop everything with a -1 frequency, drop every Nth packet with a positive "
"frequency, or drop nothing. On kernel-datapath links this is exact (eBPF every-Nth counter)",
"parameters": [{"name": "Frequency", "minimum": -1, "maximum": 32767, "type": "int", "unit": "th packet"}], "parameters": [{"name": "Frequency", "minimum": -1, "maximum": 32767, "type": "int", "unit": "th packet"}],
}, },
{ {
@ -136,6 +137,16 @@ FILTERS = [
"combining a low rate with a long delay (kernel-datapath links only)", "combining a low rate with a long delay (kernel-datapath links only)",
"parameters": [{"name": "Limit", "minimum": 1, "maximum": 1000000, "type": "int", "unit": "packets"}], "parameters": [{"name": "Limit", "minimum": 1, "maximum": 1000000, "type": "int", "unit": "packets"}],
}, },
{
"type": "quota",
"name": "Byte quota",
"description": "After the byte quota is consumed, each further packet drops with the given chance "
"(100 = hard cutoff) — a data cap, like a mobile plan (kernel-datapath links only)",
"parameters": [
{"name": "Quota", "minimum": 1, "maximum": 1000000000000000, "type": "int", "unit": "bytes"},
{"name": "Chance", "minimum": 0, "maximum": 100, "type": "int", "unit": "%"},
],
},
] ]
@ -296,24 +307,12 @@ class Link:
except FilterValidationError as e: except FilterValidationError as e:
raise ControllerError(f"Invalid packet filter parameters: {e!s}") raise ControllerError(f"Invalid packet filter parameters: {e!s}")
if self.kernel_datapath: if self._created and not self.kernel_datapath:
from gns3server.utils.packet_filter_validation import KERNEL_UNSUPPORTED_FILTERS # Kernel-only filters (the netem extensions, the eBPF quota mode)
# have no relay equivalent. Only enforced on created links — while
unsupported = KERNEL_UNSUPPORTED_FILTERS.intersection(new_filters or {}) # loading a project the datapath is not decided yet (a link that
if unsupported: # will be wired on the kernel datapath must accept them), and the
raise ControllerError( # relay prepare path drops what it cannot run with a warning.
"Packet filter(s) {} cannot run on a kernel-datapath link (no uBridge relay "
"in the forwarding path; delay, packet loss and corrupt are served by tc "
"netem on the veth); delete and recreate the link to use them".format(
", ".join(sorted(unsupported))
)
)
elif self._created:
# The reverse guard: netem-extension filters have no relay
# equivalent. Only enforced on created links — while loading a
# project the datapath is not decided yet (a link that will be
# wired on the kernel datapath must accept them), and the relay
# prepare path drops what it cannot run with a warning.
conflicts = kernel_only_features(new_filters) conflicts = kernel_only_features(new_filters)
if conflicts: if conflicts:
raise ControllerError( raise ControllerError(
@ -722,13 +721,11 @@ class Link:
filter_node = self._get_filter_node() filter_node = self._get_filter_node()
if filter_node: if filter_node:
if self.kernel_datapath: if self.kernel_datapath:
# Kernel-datapath links serve filters via tc netem on the # Kernel-datapath links serve every filter type: netem for
# veth host end; types without a netem equivalent (which # delay/loss/corrupt and the netem extensions, cls_bpf for
# only run in the uBridge relay) are hidden from the picker. # bpf, the eBPF classifier for frequency_drop and quota.
from gns3server.utils.packet_filter_validation import KERNEL_UNSUPPORTED_FILTERS return FILTERS
# Relay links: hide the kernel-only types the relay cannot run
return [f for f in FILTERS if f["type"] not in KERNEL_UNSUPPORTED_FILTERS]
# Relay links: hide the netem-extension types the relay cannot run
# (they would be rejected with 409 on update). # (they would be rejected with 409 on update).
from gns3server.utils.packet_filter_validation import KERNEL_ONLY_FILTERS from gns3server.utils.packet_filter_validation import KERNEL_ONLY_FILTERS

View File

@ -165,11 +165,10 @@ class UDPLink(Link):
""" """
Whether this link can be wired on the kernel datapath (veth pairs Whether this link can be wired on the kernel datapath (veth pairs
enslaved into a per-link Linux bridge) instead of the uBridge UDP enslaved into a per-link Linux bridge) instead of the uBridge UDP
relay. Impairment filters with a tc netem equivalent (delay, relay. Every GNS3 filter type has a kernel equivalent — netem for
packet_loss, corrupt, bpf via cls_bpf, and the netem extensions delay/packet_loss/corrupt and the netem extensions, cls_bpf for bpf,
rate/reorder/gemodel/duplicate/seed/limit) are served on the veth the eBPF stateful classifier for frequency_drop — so filters no
host end; frequency_drop only exists in the uBridge relay and longer disqualify the kernel path. Capture and markers are served by
disqualifies the kernel path. Capture and markers are served by
uBridge's AF_PACKET modules on the veth host end. uBridge's AF_PACKET modules on the veth host end.
Docker adapters are born as veth pairs (unified interface), so the Docker adapters are born as veth pairs (unified interface), so the
@ -185,12 +184,6 @@ class UDPLink(Link):
return False return False
if node1.compute.id != node2.compute.id: if node1.compute.id != node2.compute.id:
return False return False
# Look at the *stored* filters, not get_active_filters(): a suspended
# link reports the synthetic frequency_drop emulation, which is
# relay-only mechanics and must not flip a kernel link to the relay
# on reopen/reset.
if KERNEL_UNSUPPORTED_FILTERS.intersection(self._filters or {}):
return False
if _is_unix_socket_docker(node1) or _is_unix_socket_docker(node2): if _is_unix_socket_docker(node1) or _is_unix_socket_docker(node2):
return False return False
return True return True
@ -229,10 +222,10 @@ class UDPLink(Link):
# Markers ride the NIO like on the relay datapath, routed by # Markers ride the NIO like on the relay datapath, routed by
# capture node; they attach to the veth host end via uBridge's # capture node; they attach to the veth host end via uBridge's
# AF_PACKET marker module instead of a relay `mark` filter. # AF_PACKET marker module instead of a relay `mark` filter.
# Filters (delay/packet_loss/corrupt plus the netem extensions # Filters become one tc netem qdisc per veth host end (the netem
# rate/reorder/gemodel/duplicate/seed/limit — eligibility # surface and its extensions), plus cls_bpf match-drop (bpf) and
# guarantees frequency_drop is absent) become one tc netem qdisc # the eBPF stateful classifier (frequency_drop, quota) — every
# per veth host end, pushed to both endpoints. # GNS3 filter type has a kernel equivalent now.
bridge_name = "gns3" + self._id.replace("-", "")[:11] bridge_name = "gns3" + self._id.replace("-", "")[:11]
node1_filters, node2_filters = self._get_node_filters(node1, node2, kernel=True) node1_filters, node2_filters = self._get_node_filters(node1, node2, kernel=True)
node1_markers, node2_markers = self._get_node_markers(node1, node2) node1_markers, node2_markers = self._get_node_markers(node1, node2)

View File

@ -38,7 +38,7 @@ class UDPNIO(BaseModel):
filters: Optional[dict] = Field( filters: Optional[dict] = Field(
None, None,
description="Packet filters (relay-supported types only: delay/packet_loss/corrupt/frequency_drop/bpf — " description="Packet filters (relay-supported types only: delay/packet_loss/corrupt/frequency_drop/bpf — "
"the netem extensions need a kernel-datapath NIO)", "the netem extensions and the quota mode need a kernel-datapath NIO)",
) )
markers: Optional[dict] = Field(None, description="Traffic-insight markers") markers: Optional[dict] = Field(None, description="Traffic-insight markers")
@ -81,9 +81,9 @@ class BridgeNIO(BaseModel):
delay/packet_loss/corrupt and the netem extensions (rate, reorder, delay/packet_loss/corrupt and the netem extensions (rate, reorder,
gemodel, duplicate, seed, limit, jitter distributions, loss correlation) gemodel, duplicate, seed, limit, jitter distributions, loss correlation)
as one tc netem qdisc, bpf as cls_bpf match-drop classifiers (needs a as one tc netem qdisc, bpf as cls_bpf match-drop classifiers (needs a
uBridge reporting cbpf). frequency_drop still has no kernel equivalent uBridge reporting cbpf), frequency_drop and quota as the eBPF stateful
and is rejected. Markers and packet capture are served by uBridge's classifier (needs ebpf). Markers and packet capture are served by
AF_PACKET modules on the veth host end. uBridge's AF_PACKET modules on the veth host end.
""" """
type: BridgeNIOType type: BridgeNIOType
@ -91,8 +91,8 @@ class BridgeNIO(BaseModel):
suspend: Optional[bool] = Field(None, description="Suspend the NIO") suspend: Optional[bool] = Field(None, description="Suspend the NIO")
filters: Optional[dict] = Field( filters: Optional[dict] = Field(
None, None,
description="Packet filters served on the veth host end (delay/packet_loss/corrupt and the netem extensions " description="Packet filters served on the veth host end (netem surface + extensions via one tc netem qdisc, "
"via one tc netem qdisc, bpf via cls_bpf)", "bpf via cls_bpf, frequency_drop/quota via the eBPF classifier)",
) )
markers: Optional[dict] = Field(None, description="Traffic-insight markers (attached to the veth host end)") markers: Optional[dict] = Field(None, description="Traffic-insight markers (attached to the veth host end)")

View File

@ -16,18 +16,18 @@ class FilterValidationError(Exception):
pass pass
# Packet filters with no kernel-datapath equivalent yet: frequency_drop needs # Every GNS3 filter type now has a kernel-datapath equivalent: delay /
# the eBPF stateful classifier (frozen in # packet_loss / corrupt / the netem extensions run as one tc netem qdisc,
# docs/design/ubridge-kernel-impairment-spec.md, part B — not delivered), # bpf as cBPF match-drop classifiers, and frequency_drop as the eBPF
# so it still only runs in the uBridge userspace relay. bpf is served by the # stateful classifier's exact every-Nth mode (tc nth_drop — needs a uBridge
# cBPF match-drop classifier (tc bpf_drop) when uBridge reports cbpf=1. # reporting ebpf=1). No type is relay-only anymore; the controller-side
KERNEL_UNSUPPORTED_FILTERS = frozenset({"frequency_drop"}) # kernel/relay choice is purely topological (same compute, docker/docker).
#
# Packet filters with no uBridge *relay* equivalent: the tc netem extensions # Packet filters with no uBridge *relay* equivalent: the netem extensions
# (uBridge feature/tc-netem-ext) serve them on the kernel datapath only. The # plus the eBPF quota mode run on the kernel datapath only. The relay's
# relay's packet-filter registry only knows frequency_drop / packet_loss / # packet-filter registry only knows frequency_drop / packet_loss / delay /
# delay / corrupt / bpf / mark. # corrupt / bpf / mark.
KERNEL_ONLY_FILTERS = frozenset({"rate", "reorder", "gemodel", "duplicate", "seed", "limit"}) KERNEL_ONLY_FILTERS = frozenset({"rate", "reorder", "gemodel", "duplicate", "seed", "limit", "quota"})
# Jitter distributions embedded in the netem-extension uBridge (tc_netem_dist). # Jitter distributions embedded in the netem-extension uBridge (tc_netem_dist).
NETEM_DISTRIBUTIONS = ("uniform", "normal", "pareto", "paretonormal") NETEM_DISTRIBUTIONS = ("uniform", "normal", "pareto", "paretonormal")
@ -220,6 +220,15 @@ def validate_filter_parameters(filter_type: str, values: List[Any]) -> None:
}, },
"seed": {"params_count": (1, 1), "ranges": [(0, 4294967295)], "names": ["Seed"], "units": [""]}, "seed": {"params_count": (1, 1), "ranges": [(0, 4294967295)], "names": ["Seed"], "units": [""]},
"limit": {"params_count": (1, 1), "ranges": [(1, 1000000)], "names": ["Limit"], "units": ["packets"]}, "limit": {"params_count": (1, 1), "ranges": [(1, 1000000)], "names": ["Limit"], "units": ["packets"]},
"quota": {
# eBPF stateful classifier (uBridge tc quota_drop): after the byte
# quota is consumed, each further packet drops with the given
# chance (100 = hard cutoff). Kernel-datapath only.
"params_count": (2, 2),
"ranges": [(1, 10**15), (0, 100)],
"names": ["Quota", "Chance"],
"units": ["bytes", "%"],
},
"bpf": {"params_count": (1, 1), "is_text": True, "names": ["Filters"]}, "bpf": {"params_count": (1, 1), "is_text": True, "names": ["Filters"]},
} }

View File

@ -34,7 +34,7 @@ from tests.utils import AsyncioMagicMock
from gns3server.compute.docker import Docker from gns3server.compute.docker import Docker
from gns3server.compute.docker.docker_vm import DockerVM from gns3server.compute.docker.docker_vm import DockerVM
from gns3server.compute.docker.docker_error import DockerError from gns3server.compute.docker.docker_error import DockerError, DockerHttp404Error
from gns3server.compute.compute_error import ComputeError from gns3server.compute.compute_error import ComputeError
from gns3server.compute.nios.nio_udp import NIOUDP from gns3server.compute.nios.nio_udp import NIOUDP
from gns3server.compute.nios.nio_bridge import NIOBridge from gns3server.compute.nios.nio_bridge import NIOBridge
@ -102,16 +102,28 @@ def test_create_nio_bridge_accepts_bpf_filters(vm):
assert nio.filters == {"bpf": ["icmp"]} assert nio.filters == {"bpf": ["icmp"]}
def test_create_nio_bridge_rejects_relay_only_filters(vm): def test_create_nio_bridge_accepts_frequency_drop(vm):
"""
frequency_drop runs on the kernel datapath as the eBPF classifier's
exact every-Nth mode — every filter type is accepted on a kernel NIO.
"""
with pytest.raises(ComputeError, match="kernel-datapath"): nio = vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE, "filters": {"frequency_drop": [10]}})
vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE, "filters": {"frequency_drop": [10]}}) assert isinstance(nio, NIOBridge)
assert nio.filters == {"frequency_drop": [10]}
def test_create_nio_bridge_accepts_quota(vm):
nio = vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE, "filters": {"quota": [1000000, 100]}})
assert isinstance(nio, NIOBridge)
assert nio.filters == {"quota": [1000000, 100]}
def test_create_nio_bridge_accepts_markers(vm): def test_create_nio_bridge_accepts_markers(vm):
""" """
Markers ride the kernel NIO (attached to the veth host end via uBridge's Markers ride the kernel NIO (attached to the veth host end via uBridge's
AF_PACKET marker module); only filters are rejected. AF_PACKET marker module).
""" """
nio = vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE, "markers": {"m": {"bpf": "icmp"}}}) nio = vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE, "markers": {"m": {"bpf": "icmp"}}})
@ -421,26 +433,134 @@ async def test_connect_nio_udp_on_veth_is_idempotent(vm):
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_update_nio_kernel_rejects_relay_only_filters(vm): async def test_update_nio_kernel_applies_ebpf_drops(vm):
""" """
The PUT route mutates the NIO in place (no create_nio), so the update frequency_drop becomes the eBPF every-Nth mode and quota the byte-cap
path carries its own guard against relay-only filter types. mode, applied after the netem/bpf pass on the same NIO update.
"""
vm._ubridge_hypervisor = _caps_hypervisor(ebpf="1")
vm._ubridge_send = AsyncioMagicMock()
vm.status = "started"
host_ifc, _ = vm._veth_names(0, 0)
vm._kernel_veths[(0, 0)] = host_ifc
nio = vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE})
vm._ethernet_adapters[0].add_nio(0, nio)
nio.filters = {"frequency_drop": [3], "quota": [1000000, 50]}
await vm.adapter_update_nio_binding(0, nio)
vm._ubridge_send.assert_any_call(f'tc nth_drop "{host_ifc}" 3')
vm._ubridge_send.assert_any_call(f'tc quota_drop "{host_ifc}" 1000000 50')
@pytest.mark.asyncio
async def test_update_nio_kernel_ebpf_drop_everything(vm):
"""
frequency_drop -1 (drop everything) maps to every 1st packet.
"""
vm._ubridge_hypervisor = _caps_hypervisor(ebpf="1")
vm._ubridge_send = AsyncioMagicMock()
vm.status = "started"
host_ifc, _ = vm._veth_names(0, 0)
vm._kernel_veths[(0, 0)] = host_ifc
nio = vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE})
vm._ethernet_adapters[0].add_nio(0, nio)
nio.filters = {"frequency_drop": [-1]}
await vm.adapter_update_nio_binding(0, nio)
vm._ubridge_send.assert_any_call(f'tc nth_drop "{host_ifc}" 1')
# the quota mode is explicitly off (not left to chance)
vm._ubridge_send.assert_any_call(f'tc quota_drop "{host_ifc}" off')
@pytest.mark.asyncio
async def test_update_nio_kernel_turns_ebpf_modes_off(vm):
"""
A re-apply with the stateful filters removed must turn both modes off
(the netem reset already detached the program; the offs are idempotent).
"""
vm._ubridge_hypervisor = _caps_hypervisor(ebpf="1")
vm._ubridge_send = AsyncioMagicMock()
vm.status = "started"
host_ifc, _ = vm._veth_names(0, 0)
vm._kernel_veths[(0, 0)] = host_ifc
nio = vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE})
vm._ethernet_adapters[0].add_nio(0, nio)
nio.filters = {"frequency_drop": [3]}
await vm.adapter_update_nio_binding(0, nio)
vm._ubridge_send.reset_mock()
nio.filters = {"delay": [50]}
await vm.adapter_update_nio_binding(0, nio)
vm._ubridge_send.assert_any_call(f'tc nth_drop "{host_ifc}" off')
vm._ubridge_send.assert_any_call(f'tc quota_drop "{host_ifc}" off')
for c in vm._ubridge_send.call_args_list:
assert "nth_drop" not in c.args[0] or "off" in c.args[0]
@pytest.mark.asyncio
async def test_apply_ebpf_drops_requires_ebpf_capability(vm):
"""
An old or non-capped uBridge (ebpf=0) gets a clear upgrade error naming
the setcap fix.
"""
hyp = MagicMock()
hyp.is_running.return_value = True
hyp.send = AsyncioMagicMock(return_value=["netem=delay;ebpf=0;cbpf=1"])
vm._ubridge_hypervisor = hyp
vm._ubridge_send = AsyncioMagicMock()
host_ifc, _ = vm._veth_names(0, 0)
with pytest.raises(DockerError, match="setcap cap_bpf"):
await vm._ubridge_apply_ebpf_drops(host_ifc, {"frequency_drop": [3]})
vm._ubridge_send.assert_not_called()
@pytest.mark.asyncio
async def test_apply_ebpf_drops_skips_old_ubridge_without_filters(vm):
"""
No stateful filters and no capability probe yet: nothing was ever
installed on an old uBridge, so it must not be touched at all.
""" """
vm._ubridge_hypervisor = MagicMock() vm._ubridge_hypervisor = MagicMock()
nio = vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE}) vm._ubridge_send = AsyncioMagicMock()
vm._ethernet_adapters[0].add_nio(0, nio) host_ifc, _ = vm._veth_names(0, 0)
nio.filters = {"frequency_drop": [10]}
with pytest.raises(DockerError, match="kernel-datapath"): await vm._ubridge_apply_ebpf_drops(host_ifc, {})
await vm.adapter_update_nio_binding(0, nio) vm._ubridge_send.assert_not_called()
vm._ubridge_hypervisor.send.assert_not_called()
def _caps_hypervisor(cbpf="1"): @pytest.mark.asyncio
async def test_apply_ebpf_drops_cleanup_only_probes_with_ebpf(vm):
"""
The filter-removal path (modes off) only talks to the eBPF commands
when the capability probe reported them — a netem-ext uBridge without
the eBPF module must not see an unknown command.
"""
hyp = MagicMock()
hyp.is_running.return_value = True
hyp.send = AsyncioMagicMock(return_value=["netem=delay,rate;ebpf=0;cbpf=1"])
vm._ubridge_hypervisor = hyp
vm._ubridge_send = AsyncioMagicMock()
host_ifc, _ = vm._veth_names(0, 0)
vm._ubridge_tc_caps = None # force the probe (something was set before)
await vm._ubridge_apply_ebpf_drops(host_ifc, {})
vm._ubridge_send.assert_not_called()
def _caps_hypervisor(cbpf="1", ebpf="0"):
"""A hypervisor mock whose direct send() answers `tc capabilities`.""" """A hypervisor mock whose direct send() answers `tc capabilities`."""
hyp = MagicMock() hyp = MagicMock()
hyp.is_running.return_value = True hyp.is_running.return_value = True
hyp.send = AsyncioMagicMock(return_value=[f"netem=delay;ebpf=0;cbpf={cbpf}"]) hyp.send = AsyncioMagicMock(return_value=[f"netem=delay;ebpf={ebpf};cbpf={cbpf}"])
return hyp return hyp
@ -1027,3 +1147,25 @@ def test_create_nio_udp_rejects_kernel_only_filters(vm):
vm.manager.create_nio( vm.manager.create_nio(
{"type": "nio_udp", "lport": 4242, "rport": 4343, "rhost": "127.0.0.1", "filters": {"gemodel": [100, 0, 30]}} {"type": "nio_udp", "lport": 4242, "rport": 4343, "rhost": "127.0.0.1", "filters": {"gemodel": [100, 0, 30]}}
) )
@pytest.mark.asyncio
async def test_stop_clears_relay_bridge_registry(vm):
"""
The uBridge process dies with every bridge it hosted — stop() must drop
the relay bridge names, or the next start skips "bridge create" (the
gate checks the name's presence) and fails on add_nio_udp.
"""
vm._ubridge_hypervisor = MagicMock()
vm._ubridge_send = AsyncioMagicMock()
vm._bridges.add("bridge1")
vm._ubridge_tc_caps = {"netem": "delay"}
async def query(method, path, *args, **kwargs):
raise DockerHttp404Error("no such container")
vm.manager.query = AsyncioMagicMock(side_effect=query)
await vm.stop()
assert vm._bridges == set()
assert vm._ubridge_tc_caps is None

View File

@ -68,6 +68,21 @@ def _mock_compute_http(compute):
compute.post.side_effect = port_callback compute.post.side_effect = port_callback
async def _relay_link(project):
"""A docker↔qemu link (mixed node types) — never kernel-eligible, so the
relay _prepare path runs (UDP ports reserved through the mocked compute)."""
compute = MagicMock()
compute.id = "compute-1"
_mock_compute_http(compute)
node1 = _node(project, compute, "docker1")
node2 = _node(project, compute, "qemu1", node_type="qemu")
link = UDPLink(project)
await link.add_node(node1, 0, 0, batch=True, dump=False)
await link.add_node(node2, 0, 0, batch=True, dump=False)
return link, node1, node2
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_kernel_datapath_eligible(project): async def test_kernel_datapath_eligible(project):
@ -117,18 +132,16 @@ async def test_kernel_datapath_eligible_with_netem_filters(project):
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_kernel_datapath_not_eligible_with_relay_only_filters(project): async def test_kernel_datapath_eligible_with_frequency_drop(project):
""" """
frequency_drop only exists in the uBridge userspace relay (its eBPF frequency_drop runs on the kernel datapath too (the eBPF classifier's
classifier is not delivered yet) — a link carrying one stays on the exact every-Nth mode) — no filter type forces the relay anymore; only
relay. bpf is kernel-capable (cls_bpf match-drop) and eligible. topology does (mixed node types, cross-compute, unix-socket
containers, config off).
""" """
link, node1, node2 = await _kernel_link(project) link, node1, node2 = await _kernel_link(project)
link._filters = {"delay": [10, 0], "frequency_drop": [7]} link._filters = {"delay": [10, 0], "frequency_drop": [7], "quota": [1000000, 100]}
assert link._kernel_datapath_eligible(node1, node2) is False
link._filters = {"bpf": ["icmp"]}
assert link._kernel_datapath_eligible(node1, node2) is True assert link._kernel_datapath_eligible(node1, node2) is True
@ -277,13 +290,20 @@ async def test_update_netem_filters_accepted_on_kernel_link(project):
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_update_relay_only_filters_rejected_on_kernel_link(project): async def test_update_frequency_drop_accepted_on_kernel_link(project):
"""
frequency_drop rides the kernel NIO to both endpoints like every other
filter type — the compute translates it to the eBPF every-Nth mode.
"""
link, _n1, _n2 = await _kernel_link(project) link, node1, node2 = await _kernel_link(project)
await link._prepare() await link._prepare()
with pytest.raises(ControllerError, match="kernel-datapath"): await link.update_filters({"frequency_drop": [10]})
await link.update_filters({"frequency_drop": [10]}) assert link.filters == {"frequency_drop": [10]}
by_node = {entry[0].id: entry[3] for entry in (await link._prepare())}
assert by_node[node1.id]["filters"] == {"frequency_drop": [10]}
assert by_node[node2.id]["filters"] == {"frequency_drop": [10]}
@pytest.mark.asyncio @pytest.mark.asyncio
@ -449,14 +469,12 @@ async def test_update_netem_extension_filters_accepted_on_kernel_link(project):
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_update_kernel_only_filters_rejected_on_relay_link(project): async def test_update_kernel_only_filters_rejected_on_relay_link(project):
""" """
Netem-extension filters have no relay equivalent: setting them on a Netem-extension / quota filters have no relay equivalent: setting them
created relay link is a 409. (frequency_drop in the stored filters on a created relay link is a 409. (The link is relay-wired because the
forces this link onto the relay datapath.) endpoints are mixed node types.)
""" """
link, _n1, _n2 = await _kernel_link(project) link, _n1, _n2 = await _relay_link(project)
_mock_compute_http(link._nodes[0]["node"].compute)
link._filters = {"frequency_drop": [7]}
await link._prepare() await link._prepare()
assert link.kernel_datapath is False assert link.kernel_datapath is False
link._created = True # the computes hold the UDP NIOs link._created = True # the computes hold the UDP NIOs
@ -464,6 +482,10 @@ async def test_update_kernel_only_filters_rejected_on_relay_link(project):
with pytest.raises(ControllerError, match="kernel-datapath"): with pytest.raises(ControllerError, match="kernel-datapath"):
await link.update_filters({"rate": ["512kbit"]}) await link.update_filters({"rate": ["512kbit"]})
# the stored filters are untouched # the stored filters are untouched
assert link.filters == {}
# frequency_drop still runs on the relay (its uBridge userspace filter)
await link.update_filters({"frequency_drop": [7]})
assert link.filters == {"frequency_drop": [7]} assert link.filters == {"frequency_drop": [7]}
@ -484,13 +506,13 @@ async def test_update_kernel_only_filters_accepted_before_creation(project):
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_prepare_relay_link_strips_kernel_only_filters(project): async def test_prepare_relay_link_strips_kernel_only_filters(project):
""" """
A relay-wired link (here: frequency_drop in the stored filters) built A relay-wired link (mixed node types) built from a topology carrying
from a topology carrying extension filters drops them with a warning extension filters drops them with a warning instead of pushing an
instead of pushing an unknown filter type to uBridge. unknown filter type to uBridge. frequency_drop stays — the relay's
userspace filter still serves it.
""" """
link, node1, node2 = await _kernel_link(project) link, node1, node2 = await _relay_link(project)
_mock_compute_http(link._nodes[0]["node"].compute)
link._filters = {"frequency_drop": [7], "rate": ["512kbit"], "delay": [100, 20, "normal"], "packet_loss": [5, 25]} link._filters = {"frequency_drop": [7], "rate": ["512kbit"], "delay": [100, 20, "normal"], "packet_loss": [5, 25]}
entries = await link._prepare() entries = await link._prepare()
assert link.kernel_datapath is False assert link.kernel_datapath is False

View File

@ -366,3 +366,20 @@ class TestNetemExtensionFilters:
def test_kernel_only_features_empty(self): def test_kernel_only_features_empty(self):
assert split_kernel_only_features({}) == ({}, set()) assert split_kernel_only_features({}) == ({}, set())
assert split_kernel_only_features(None) == ({}, set()) assert split_kernel_only_features(None) == ({}, set())
def test_quota_valid_and_invalid(self):
validate_filter_parameters("quota", [1000000, 100])
validate_filter_parameters("quota", [1, 0])
with pytest.raises(FilterValidationError, match="Quota"):
validate_filter_parameters("quota", [0, 100])
with pytest.raises(FilterValidationError, match="Chance"):
validate_filter_parameters("quota", [1000, 101])
with pytest.raises(FilterValidationError, match="expects 2 parameter"):
validate_filter_parameters("quota", [1000])
def test_quota_is_kernel_only(self):
clean, dropped = split_kernel_only_features({"quota": [1000, 50], "frequency_drop": [7]})
assert dropped == {"quota"}
# frequency_drop runs on both datapaths (relay userspace filter /
# eBPF every-Nth) and stays
assert clean == {"frequency_drop": [7]}