mirror of
https://github.com/GNS3/gns3-server.git
synced 2026-10-12 21:45:41 +03:00
feat(docker): eBPF stateful packet filters on kernel links
Consume the eBPF impairment classifier delivered in uBridge feature/tc-precision: frequency_drop becomes the exact every-Nth mode (tc nth_drop; -1 = drop everything maps to every 1st) and the new kernel-only quota type is the byte-cap mode (tc quota_drop, 100 = hard cutoff). KERNEL_UNSUPPORTED_FILTERS is gone entirely — kernel/relay eligibility is now purely topological. Both modes gate on ebpf=1 from the per-process tc capabilities probe (error names the setcap fix); the cleanup path (modes off) only talks to the eBPF commands when the probe reported them. Reconcile is a plain re-set: the netem reset-before-set already detaches the program and clears uBridge's per-interface registry, so a mode set after it is a fresh load+attach. Documented semantic difference: the relay counts both directions through one bridge counter, the kernel counts per direction — N=3 round-trip loss is 1-(2/3)^2 = 55.6% on kernel vs ~1/N on relay (measured 57%). window_drop is NOT exposed: the delivered implementation advances the window with no gap, making 'outside the window packets pass' unreachable (needs outage/period fields — recorded in the spec); flow_drop awaits a parameter-shape decision. Also fixes a pre-existing restart bug the e2e exposed: _bridges (relay bridge names) was never cleared on stop, so restarting a node with a relay link skipped 'bridge create' and failed on add_nio_udp. e2e (isolated instance): 13/13 — pref-1 tc_impair classifier on both veths, 57% round-trip for N=3, -1 = exact 100%, quota hard cutoff, delay coexistence, clean teardown, suspend/resume, node restart, relay-leg frequency_drop regression through an Ethernet switch. Full suite: 1925 passed.
This commit is contained in:
YueGuobin
parent
d4b57d49fb
commit
0c4ec8c38c
@ -4,17 +4,16 @@ See LICENSE file for licensing information.
|
||||
-->
|
||||
|
||||
> Frozen requirements spec for the **uBridge** project. Delivery status:
|
||||
> parts **A** (netem keyword extensions, uBridge `feature/tc-netem-ext`), **C**
|
||||
> (cBPF match-drop) and **D + E** (full-restore idempotent `tc reset`,
|
||||
> `tc capabilities` — uBridge `feature/tc-bpf-drop`) are **delivered and
|
||||
> integrated** on gns3-server branches `feat/docker-kernel-bpf-drop` (C/D/E)
|
||||
> and `feat/docker-kernel-netem-ext` (A: rate/reorder/gemodel/duplicate/
|
||||
> seed/limit/distributions/correlation exposed as GNS3 filter types).
|
||||
> Part **B** (eBPF stateful classifier) is delivered in the uBridge
|
||||
> `feature/tc-precision` branch but **not yet consumed** by gns3-server.
|
||||
> The originally shipped netem surface (delay/jitter/loss/dup/corrupt)
|
||||
> exists in uBridge 1.2.3+ and is verified against gns3-server branch
|
||||
> `feat/docker-kernel-filters`.
|
||||
> **all parts delivered and integrated.** A (netem keyword extensions,
|
||||
> uBridge `feature/tc-netem-ext`), C + D + E (cBPF match-drop, full-restore
|
||||
> idempotent `tc reset`, `tc capabilities` — uBridge `feature/tc-bpf-drop`)
|
||||
> and **B (eBPF stateful classifier — uBridge `feature/tc-precision`
|
||||
> 59e2b38, including the non-root setcap load fix)** are consumed by
|
||||
> gns3-server on the branch stack ending at `feat/docker-kernel-ebpf-drops`
|
||||
> (B: `frequency_drop` → `tc nth_drop`, the new kernel-only `quota` type →
|
||||
> `tc quota_drop`). The originally shipped netem surface (delay/jitter/
|
||||
> loss/dup/corrupt) exists in uBridge 1.2.3+ and is verified against
|
||||
> gns3-server branch `feat/docker-kernel-filters`.
|
||||
>
|
||||
> **Deviation found during integration (part A/D):** the delivered
|
||||
> `tc netem set` uses NLM_F_REPLACE, but the *kernel's* netem change merges
|
||||
@ -27,6 +26,20 @@ See LICENSE file for licensing information.
|
||||
> invisible). If uBridge later emits explicit zero/absent clears for every
|
||||
> optional attribute, the server-side reset-before-set can be dropped; the
|
||||
> two behaviours are compatible either way.
|
||||
>
|
||||
> **Deviation found during B integration:** `window_drop`'s implementation
|
||||
> advances the window start by one length on expiry with no gap, making the
|
||||
> windows back-to-back — after the first `start_ms` the "outside the window
|
||||
> packets pass" behaviour described in B.2 is unreachable (inside ≡ always).
|
||||
> gns3-server therefore does **not** expose window_drop yet; it needs either
|
||||
> a second length field (outage vs period) or a next-start semantics fix on
|
||||
> the uBridge side. `flow_drop` is delivered and functional but not yet
|
||||
> exposed as a GNS3 filter type (its mask/target parameter shape needs a
|
||||
> UX decision). Additionally, the delivered B.1 program had to fold the
|
||||
> flow-hash L4 port reads to constant offsets (IHL==20 only) because the
|
||||
> verifier prohibits variable packet-pointer arithmetic for non-root —
|
||||
> even with CAP_BPF (Spectre-mitigation gating by uid); documented in
|
||||
> uBridge's doc/tc.md.
|
||||
|
||||
# uBridge kernel impairment: tc netem extensions + eBPF classifiers
|
||||
|
||||
@ -293,11 +306,14 @@ corresponding filter types per capability.
|
||||
in `DockerVM._ubridge_apply_netem`; extension keywords are gated on the
|
||||
`tc capabilities` netem token list (old uBridge builds are never probed
|
||||
for the original surface).
|
||||
* **Done** (`feat/docker-kernel-bpf-drop`): `bpf` runs as cBPF match-drop
|
||||
(`KERNEL_UNSUPPORTED_FILTERS` shrank to `frequency_drop`).
|
||||
* `frequency_drop` moves out of `KERNEL_UNSUPPORTED_FILTERS` once the eBPF
|
||||
classifier (part B, uBridge `feature/tc-precision`) is consumed on the
|
||||
server side, keyed on `ebpf=1` per compute.
|
||||
* **Done** (`feat/docker-kernel-bpf-drop`): `bpf` runs as cBPF match-drop.
|
||||
* **Done** (`feat/docker-kernel-ebpf-drops`): the eBPF classifier (part B)
|
||||
is consumed — `frequency_drop` → `tc nth_drop` (-1 → every 1st), the new
|
||||
kernel-only `quota` type → `tc quota_drop`, both keyed on `ebpf=1` from
|
||||
the per-process `tc capabilities` probe. `KERNEL_UNSUPPORTED_FILTERS` is
|
||||
gone entirely: kernel/relay eligibility is purely topological now.
|
||||
Not yet exposed: `window_drop` (implementation deviation above),
|
||||
`flow_drop` (parameter-shape UX decision).
|
||||
* Every veth end owns one qdisc + its filters: per-direction impairment is
|
||||
an architectural freebie to expose later (API `direction` field), aligned
|
||||
with marker `dir` semantics.
|
||||
|
||||
@ -24,13 +24,14 @@ Frames never leave the kernel.
|
||||
| Link | uBridge bridge, TAP fd ↔ UDP, userspace copy | per-link kernel bridge `gns3{link_id[:11]}`, zero-copy |
|
||||
| Relay latency | ~0.3 ms RTT | ~0.05 ms RTT (measured) |
|
||||
| Link attach at runtime | yes (relay attaches the pre-existing TAP) | yes (brctl addif / add_nio_ethernet — interface untouched) |
|
||||
| Packet filters | uBridge userspace filters | tc netem on the veth host end (delay / packet_loss / corrupt / duplicate + the netem extensions rate, reorder, gemodel, seed, limit, jitter distributions, loss correlation); bpf as cls_bpf match-drop classifiers; frequency_drop stays relay-only (eBPF classifier pending) |
|
||||
| Packet filters | uBridge userspace filters | tc netem on the veth host end (delay / packet_loss / corrupt / duplicate + the netem extensions rate, reorder, gemodel, seed, limit, jitter distributions, loss correlation); bpf as cls_bpf match-drop classifiers; frequency_drop and quota as the eBPF stateful classifier — every filter type has a kernel equivalent |
|
||||
| Capture / markers | relay bridge | uBridge AF_PACKET modules on the veth host end |
|
||||
|
||||
The work landed in seven stages on branch stack `feat/docker-kernel-datapath` →
|
||||
The work landed in eight stages on branch stack `feat/docker-kernel-datapath` →
|
||||
`feat/docker-kernel-capture` → `feat/docker-kernel-markers` →
|
||||
`feat/docker-veth-everywhere` → `feat/docker-kernel-filters` →
|
||||
`feat/docker-kernel-bpf-drop` → `feat/docker-kernel-netem-ext`:
|
||||
`feat/docker-kernel-bpf-drop` → `feat/docker-kernel-netem-ext` →
|
||||
`feat/docker-kernel-ebpf-drops`:
|
||||
|
||||
1. **Kernel datapath** — NIOBridge NIO type, per-link bridges, carrier-based
|
||||
suspend, crash-safe reconciliation.
|
||||
@ -57,6 +58,14 @@ The work landed in seven stages on branch stack `feat/docker-kernel-datapath`
|
||||
process. Reconcile became reset-before-set: the kernel's netem replace
|
||||
merges optional attributes, so removing a parameter needs the explicit
|
||||
reset.
|
||||
8. **eBPF stateful drops** — `frequency_drop` becomes the eBPF classifier's
|
||||
exact every-Nth mode (`tc nth_drop`, uBridge `feature/tc-precision`;
|
||||
-1 = drop everything → every 1st) and the new kernel-only `quota` type
|
||||
the byte-cap mode (`tc quota_drop`). Needs a uBridge reporting `ebpf=1`
|
||||
(setcap cap_bpf,cap_net_admin,cap_net_raw=ep — verified working for a
|
||||
non-root server process, the production credential shape). With this,
|
||||
no filter type forces the relay anymore: kernel/relay is a purely
|
||||
topological choice.
|
||||
|
||||
## Adapter interface types
|
||||
|
||||
@ -89,8 +98,8 @@ Kernel link between two containers on the same compute:
|
||||
└───────────────────┘
|
||||
```
|
||||
|
||||
Relay link on the same unified veth (fallback for filtered links or when
|
||||
`enable_kernel_datapath` is off — e.g. to reach another compute):
|
||||
Relay link on the same unified veth (fallback for mixed node types, cross-
|
||||
compute wiring or when `enable_kernel_datapath` is off):
|
||||
|
||||
```
|
||||
┌────────────┐ ┌────────────────────────────┐ ┌────────────┐
|
||||
@ -110,10 +119,11 @@ time; the compute side only reacts to the NIO type:
|
||||
|
||||
* both endpoints Docker (any class except unix-socket containers), same
|
||||
compute
|
||||
* no active packet filters (filters live in the relay; kernelization via tc
|
||||
netem is planned)
|
||||
* `Server.enable_kernel_datapath` enabled (default)
|
||||
|
||||
Filters never disqualify the kernel path anymore — every type has a kernel
|
||||
equivalent (netem / cls_bpf / eBPF classifier).
|
||||
|
||||
There is **no stopped-node requirement**. Since every adapter is a veth, links
|
||||
attach to running containers: `brctl addif` (kernel) or
|
||||
`bridge add_nio_ethernet` (relay) are runtime-safe operations on the host end.
|
||||
@ -189,7 +199,8 @@ ends (uBridge `tc netem set`, raw netlink — no `tc` binary needed):
|
||||
| `seed [u32]` | netem `seed N` | reproducible random draws; kernel-only type |
|
||||
| `limit [pkts]` | netem `limit N` | queue depth above the 1000 default (rate+delay BDP); kernel-only type |
|
||||
| `bpf` (one line per expression, OR) | `tc bpf_drop add <if> <prio> "<expr>"` — cls_bpf on clsact egress + gact drop | needs uBridge `cbpf=1` (`tc capabilities`, probed once per uBridge process); a line that fails to compile on the compute is skipped with a warning, mirroring the relay |
|
||||
| `frequency_drop` | — not yet (eBPF stateful classifier spec'd: [ubridge-kernel-impairment-spec](../design/ubridge-kernel-impairment-spec.md), part B pending) | 409 on kernel links; relay fallback |
|
||||
| `frequency_drop [N]` | `tc nth_drop <if> N` — the eBPF stateful classifier's exact every-Nth mode (clsact egress prio 1) | needs uBridge `ebpf=1`; -1 = drop everything → every 1st; per-direction counters (see below) |
|
||||
| `quota [bytes, %]` | `tc quota_drop <if> <bytes> <pct>` — after the byte quota is consumed, each further packet drops with the given chance (100 = hard cutoff) | kernel-only type; needs `ebpf=1`; per-direction byte counters |
|
||||
|
||||
Semantics:
|
||||
|
||||
@ -213,12 +224,24 @@ Semantics:
|
||||
`tc capabilities` once per uBridge process and require the tokens;
|
||||
a plain delay/loss/corrupt/dup filter never probes — an old uBridge
|
||||
serves the original surface untouched.
|
||||
* **kernel-only vs relay-only.** The extension types have no uBridge relay
|
||||
equivalent: `available_filters` hides them on relay links, setting one on
|
||||
a created relay link returns 409, and a project loaded with such filters
|
||||
on a link that cannot be kernel-wired drops them with a warning (invalid
|
||||
filters are dropped the same way at load). Symmetrically,
|
||||
`frequency_drop` stays relay-only.
|
||||
* **kernel-only vs relay-available.** The extension types and `quota` have
|
||||
no uBridge relay equivalent: `available_filters` hides them on relay
|
||||
links, setting one on a created relay link returns 409, and a project
|
||||
loaded with such filters on a link that cannot be kernel-wired drops them
|
||||
with a warning (invalid filters are dropped the same way at load).
|
||||
`frequency_drop` runs on both datapaths (relay userspace filter / eBPF
|
||||
every-Nth).
|
||||
* **frequency_drop counting semantics differ per datapath.** The relay's
|
||||
single filtered bridge counts packets of BOTH directions through one
|
||||
counter; the kernel attaches one classifier per veth end, so each
|
||||
direction drops every Nth independently. For a round-trip measurement
|
||||
with every-Nth N: relay ≈ 1/N of frames (phase-locked), kernel = 1 −
|
||||
((N−1)/N)² (e.g. N=3 → 55.6 % round-trip loss, measured 57 %). The
|
||||
kernel count is exact (atomic counter), unlike netem's stochastic loss.
|
||||
* **Classifier order** on clsact egress: the eBPF stateful filter runs at
|
||||
prio 1, `bpf_drop` expressions at 10–99 — a packet dropped by the
|
||||
stateful modes never reaches the expression drops, and dropped packets
|
||||
never reach netem.
|
||||
* **Validation** mirrors the tc grammar: `reorder` requires `delay`,
|
||||
`gemodel` and `packet_loss` are mutually exclusive (both map to the netem
|
||||
loss keyword), `distribution` requires jitter > 0, rate must be integer +
|
||||
@ -253,7 +276,9 @@ tc netem set <if> [delay <ms>] [jitter <ms>] [loss <%> [correl <%>] | loss gemod
|
||||
[dup <%> [correl <%>]] [corrupt <%>] [reorder <%> [correl <%>] [gap <n>]]
|
||||
[rate <bw>] [limit <pkts>] [distribution uniform|normal|pareto|paretonormal] [seed <u32>]
|
||||
tc bpf_drop add <if> <prio 10-99> "<expr>" / tc bpf_drop flush <if>
|
||||
tc reset <if> # full restore: filters -> clsact -> root qdisc, idempotent
|
||||
tc nth_drop <if> <n | off> # eBPF exact every-Nth (frequency_drop)
|
||||
tc quota_drop <if> <bytes> <pct> | off # eBPF byte cap (quota)
|
||||
tc reset <if> # full restore: eBPF filter -> bpf_drops -> clsact -> root qdisc, idempotent
|
||||
tc capabilities # "netem=<kw,...>;ebpf=0|1;cbpf=0|1", probed once per process
|
||||
bridge add_nio_ethernet / add_nio_udp / start / stop / start_capture / stop_capture
|
||||
```
|
||||
@ -280,7 +305,8 @@ pairs), capture freeze on stop, server restart reconciliation.
|
||||
Filters (two-container kernel link, 27/27 checks): baseline 0.06 ms →
|
||||
`delay 100` = 200.2 ms RTT (2× per direction) → `delay 50` = 100.2 ms
|
||||
(atomic replace) → clear = baseline (qdisc detached); `packet_loss 30` =
|
||||
48 % round-trip loss (expected 51 % = 1 − 0.7²); `frequency_drop` = 409;
|
||||
48 % round-trip loss (expected 51 % = 1 − 0.7²); `frequency_drop` = 409 (at that
|
||||
stage — superseded by the eBPF stage below);
|
||||
suspend with delay active = 100 % loss, resume keeps the qdisc
|
||||
(200.1 ms); node restart restores it (200.1 ms); link delete detaches it
|
||||
(`/usr/sbin/tc qdisc show` — no netem left); re-created link has no residual
|
||||
@ -291,8 +317,9 @@ ends' clsact; size-discriminating expression (`greater 150`) = small pings
|
||||
pass / 300-byte pings dropped (real byte-level BPF match); two-line OR;
|
||||
coexistence with netem (delay 50 → small pings 100.2 ms while big dropped);
|
||||
clear removes clsact entirely; suspend/resume keeps the drops; link delete +
|
||||
recreate leaves no residual filters; `frequency_drop` still 409;
|
||||
`available_filters` shows `bpf` and hides `frequency_drop`. Unit tests:
|
||||
recreate leaves no residual filters; `frequency_drop` still 409 and hidden
|
||||
(at that stage — superseded by the eBPF stage below); `available_filters` shows
|
||||
`bpf`. Unit tests:
|
||||
`tests/compute/docker/test_docker_kernel_datapath.py`,
|
||||
`tests/controller/test_kernel_datapath_link.py`.
|
||||
|
||||
@ -307,11 +334,27 @@ cleared; `duplicate 50` link alive; `packet_loss 30 correl 50` shows
|
||||
exactly 2×50 ms; suspend/resume and node restart keep the filters; a
|
||||
docker↔ethernet-switch relay link rejects `rate` with 409 and its
|
||||
`available_filters` hides the kernel-only types while the kernel link shows
|
||||
them all (minus frequency_drop); `reorder` without `delay` and
|
||||
them all (frequency_drop included since the eBPF stage); `reorder` without `delay` and
|
||||
`gemodel`+`packet_loss` are 409s; teardown leaves no netem on either veth.
|
||||
Also verified live: a relay attach (`bridge add_nio_ethernet`) on the
|
||||
admin-down veth host end fails in libpcap — the relay path now brings the
|
||||
interface up before attaching. Unit tests:
|
||||
interface up before attaching.
|
||||
|
||||
eBPF stateful drops (same setup, 13/13 checks): `frequency_drop 3` installs
|
||||
the tc_impair classifier on both veths (clsact egress pref 1, visible in
|
||||
`tc filter show`); round-trip loss 57 % against the per-direction theory 1 −
|
||||
(2/3)² = 55.6 % (each direction drops every 3rd independently — the relay's
|
||||
single shared counter would give ≈ 1/3); `frequency_drop -1` = exactly
|
||||
100 % loss; `quota 3000B/100 %` = pings pass until the byte cap then hard
|
||||
cutoff; coexistence with `delay 50` (RTT 100.3 ms with every-4th drops);
|
||||
clearing filters leaves no residual bpf filter; suspend = 100 % while down
|
||||
with drops persisting after resume; node restart restores every-Nth on the
|
||||
fresh veth; `available_filters` shows frequency_drop + quota on kernel
|
||||
links and hides quota on relay links; a two-leg relay path through an
|
||||
Ethernet switch still serves frequency_drop from the userspace filter
|
||||
(56 % round-trip for N=3). The e2e also exposed and fixed a pre-existing
|
||||
restart bug: the relay bridge-name registry was not cleared on stop, so a
|
||||
node restart with a relay link skipped `bridge create` and failed. Unit tests:
|
||||
`tests/utils/test_packet_filter_validation.py` (P6a class),
|
||||
`tests/compute/docker/test_docker_kernel_datapath.py`,
|
||||
`tests/controller/test_kernel_datapath_link.py`.
|
||||
|
||||
@ -375,17 +375,6 @@ class BaseManager:
|
||||
nio.suspend = nio_settings.get("suspend", False)
|
||||
elif nio_settings["type"] == "nio_bridge":
|
||||
nio = NIOBridge(nio_settings["bridge"])
|
||||
# Impairment filters with a tc netem equivalent run on the veth
|
||||
# host end; the relay-only types are rejected here (the controller
|
||||
# already keeps them off kernel links — this is the second guard).
|
||||
from gns3server.utils.packet_filter_validation import KERNEL_UNSUPPORTED_FILTERS
|
||||
|
||||
unsupported = KERNEL_UNSUPPORTED_FILTERS.intersection(nio_settings.get("filters") or {})
|
||||
if unsupported:
|
||||
raise ComputeError(
|
||||
"Packet filter(s) {} cannot run on a kernel-datapath link "
|
||||
"(no uBridge relay in the forwarding path)".format(", ".join(sorted(unsupported)))
|
||||
)
|
||||
nio.filters = nio_settings.get("filters", {})
|
||||
# Markers ride the NIO and attach to the veth host interface via
|
||||
# uBridge's AF_PACKET marker module (marker add_kernel).
|
||||
|
||||
@ -1438,6 +1438,11 @@ class DockerVM(BaseNode):
|
||||
# The next start spawns a fresh uBridge process — its capabilities
|
||||
# must be re-probed (see _ubridge_tc_capabilities).
|
||||
self._ubridge_tc_caps = None
|
||||
# The uBridge process dies with every bridge it hosted — drop the
|
||||
# relay bridge names too, or the next start would skip
|
||||
# "bridge create" for them (the gate checks the name's presence)
|
||||
# and fail on "bridge add_nio_udp: bridge doesn't exist".
|
||||
self._bridges.clear()
|
||||
# veth host ends live in the root namespace and survive container
|
||||
# death — remove them while the uBridge control channel is still up.
|
||||
await self._remove_kernel_veths()
|
||||
@ -1944,9 +1949,11 @@ class DockerVM(BaseNode):
|
||||
await self._ubridge_apply_markers(host_ifc, nio)
|
||||
# Impairment filters become one tc netem qdisc on the veth host
|
||||
# end (restored here on node restart, like the capture above);
|
||||
# bpf expressions become cls_bpf match-drop classifiers.
|
||||
# bpf expressions become cls_bpf match-drop classifiers and
|
||||
# frequency_drop/quota become the eBPF stateful classifier.
|
||||
await self._ubridge_apply_netem(host_ifc, nio.filters)
|
||||
await self._ubridge_apply_bpf_drops(host_ifc, nio.filters)
|
||||
await self._ubridge_apply_ebpf_drops(host_ifc, nio.filters)
|
||||
return
|
||||
|
||||
# Relay NIO. On a veth-backed adapter (the unified Docker interface)
|
||||
@ -2015,28 +2022,19 @@ class DockerVM(BaseNode):
|
||||
|
||||
if self.ubridge:
|
||||
if isinstance(nio, NIOBridge):
|
||||
# The controller keeps frequency_drop off kernel links (no
|
||||
# kernel equivalent until the eBPF classifier lands) and
|
||||
# create_nio rejects it on the POST path; this PUT path
|
||||
# mutates the NIO in place, so guard here too. bpf is fine —
|
||||
# it becomes cls_bpf match-drop classifiers (cbpf-probed).
|
||||
unsupported = KERNEL_UNSUPPORTED_FILTERS.intersection(nio.filters or {})
|
||||
if unsupported:
|
||||
raise DockerError(
|
||||
"Packet filter(s) {} cannot run on a kernel-datapath link "
|
||||
"(no uBridge relay in the forwarding path)".format(", ".join(sorted(unsupported)))
|
||||
)
|
||||
if self.status == "started":
|
||||
host_ifc = self._kernel_veths.get((adapter_number, port_number))
|
||||
if host_ifc is not None:
|
||||
# Incremental marker reconcile on the veth anchor —
|
||||
# the relay-datapath equivalent of the branch below.
|
||||
await self._ubridge_apply_markers(host_ifc, nio)
|
||||
# netem set is an atomic replace, so a full re-apply
|
||||
# IS the incremental reconcile for impairment filters;
|
||||
# bpf drops reconcile as flush + re-add.
|
||||
# The netem apply resets the interface first (the
|
||||
# kernel merges optional netem attrs on replace), so
|
||||
# everything anchored on clsact must re-apply after
|
||||
# it: bpf drops flush + re-add, eBPF modes re-set.
|
||||
await self._ubridge_apply_netem(host_ifc, nio.filters)
|
||||
await self._ubridge_apply_bpf_drops(host_ifc, nio.filters)
|
||||
await self._ubridge_apply_ebpf_drops(host_ifc, nio.filters)
|
||||
await self._set_adapter_carrier(adapter_number, not nio.suspend, port_number)
|
||||
return
|
||||
|
||||
@ -2376,6 +2374,70 @@ class DockerVM(BaseNode):
|
||||
log.warning(message)
|
||||
self.project.emit("log.warning", {"message": message})
|
||||
|
||||
async def _ubridge_apply_ebpf_drops(self, host_ifc, filters):
|
||||
"""
|
||||
Translate the NIO's stateful filters into uBridge's eBPF impairment
|
||||
classifier on the veth host end's clsact egress (prio 1, below
|
||||
bpf_drop's 10-99): ``frequency_drop`` becomes the exact every-Nth
|
||||
mode (``tc nth_drop``; -1 = drop everything maps to every 1st), and
|
||||
the kernel-only ``quota`` type the byte-cap mode (``tc quota_drop``).
|
||||
|
||||
Reconcile is a plain re-set on every apply: the preceding netem
|
||||
apply resets the interface (which detaches the program and drops
|
||||
uBridge's per-interface registry entry), so a mode set re-loads the
|
||||
program from scratch, and a mode left absent is turned off
|
||||
explicitly (idempotent) so a removed filter stops dropping.
|
||||
|
||||
Semantic difference vs the relay, documented server-side: the relay
|
||||
counts packets of BOTH directions through its single filtered
|
||||
bridge, while each veth end's classifier counts only the traffic
|
||||
entering that container — with both endpoints applying theirs, each
|
||||
direction drops every Nth independently.
|
||||
|
||||
:param host_ifc: veth host-end interface name
|
||||
:param filters: NIO filters dictionary
|
||||
"""
|
||||
|
||||
def values_of(key):
|
||||
values = filters.get(key)
|
||||
if isinstance(values, (list, tuple)):
|
||||
return list(values)
|
||||
return [values] if values else []
|
||||
|
||||
frequency = values_of("frequency_drop")
|
||||
quota = values_of("quota")
|
||||
if not frequency and not quota and self._ubridge_tc_caps is None:
|
||||
# Nothing to set and nothing was ever installed (no probe yet) —
|
||||
# skip touching an old uBridge entirely.
|
||||
return
|
||||
caps = await self._ubridge_tc_capabilities()
|
||||
if not frequency and not quota:
|
||||
if caps.get("ebpf") == "1":
|
||||
# Only when the eBPF module answers can anything be
|
||||
# installed; "off" is idempotent in uBridge.
|
||||
await self._ubridge_send(f'tc nth_drop "{host_ifc}" off')
|
||||
await self._ubridge_send(f'tc quota_drop "{host_ifc}" off')
|
||||
return
|
||||
if caps.get("ebpf") != "1":
|
||||
raise DockerError(
|
||||
"Packet filter(s) {} on a kernel-datapath link need a uBridge with eBPF support "
|
||||
"(tc capabilities reports no ebpf; setcap cap_bpf,cap_net_admin,cap_net_raw=ep on "
|
||||
"the uBridge binary); upgrade uBridge on this compute or keep the link on the "
|
||||
"relay datapath".format(", ".join(sorted(f for f in ("frequency_drop", "quota") if f in filters)))
|
||||
)
|
||||
if frequency:
|
||||
# relay semantics: -1 = drop everything, N = every Nth packet
|
||||
nth = 1 if int(frequency[0]) == -1 else int(frequency[0])
|
||||
await self._ubridge_send(f'tc nth_drop "{host_ifc}" {nth}')
|
||||
else:
|
||||
await self._ubridge_send(f'tc nth_drop "{host_ifc}" off')
|
||||
if quota:
|
||||
await self._ubridge_send(
|
||||
'tc quota_drop "{ifc}" {bytes} {pct}'.format(ifc=host_ifc, bytes=int(quota[0]), pct=int(quota[1]))
|
||||
)
|
||||
else:
|
||||
await self._ubridge_send(f'tc quota_drop "{host_ifc}" off')
|
||||
|
||||
async def _remove_kernel_markers(self, host_ifc):
|
||||
"""
|
||||
Tear down every kernel marker anchored to *host_ifc*: the veth survives
|
||||
|
||||
@ -46,7 +46,8 @@ FILTERS = [
|
||||
{
|
||||
"type": "frequency_drop",
|
||||
"name": "Frequency drop",
|
||||
"description": "It will drop everything with a -1 frequency, drop every Nth packet with a positive frequency, or drop nothing",
|
||||
"description": "It will drop everything with a -1 frequency, drop every Nth packet with a positive "
|
||||
"frequency, or drop nothing. On kernel-datapath links this is exact (eBPF every-Nth counter)",
|
||||
"parameters": [{"name": "Frequency", "minimum": -1, "maximum": 32767, "type": "int", "unit": "th packet"}],
|
||||
},
|
||||
{
|
||||
@ -136,6 +137,16 @@ FILTERS = [
|
||||
"combining a low rate with a long delay (kernel-datapath links only)",
|
||||
"parameters": [{"name": "Limit", "minimum": 1, "maximum": 1000000, "type": "int", "unit": "packets"}],
|
||||
},
|
||||
{
|
||||
"type": "quota",
|
||||
"name": "Byte quota",
|
||||
"description": "After the byte quota is consumed, each further packet drops with the given chance "
|
||||
"(100 = hard cutoff) — a data cap, like a mobile plan (kernel-datapath links only)",
|
||||
"parameters": [
|
||||
{"name": "Quota", "minimum": 1, "maximum": 1000000000000000, "type": "int", "unit": "bytes"},
|
||||
{"name": "Chance", "minimum": 0, "maximum": 100, "type": "int", "unit": "%"},
|
||||
],
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
@ -296,24 +307,12 @@ class Link:
|
||||
except FilterValidationError as e:
|
||||
raise ControllerError(f"Invalid packet filter parameters: {e!s}")
|
||||
|
||||
if self.kernel_datapath:
|
||||
from gns3server.utils.packet_filter_validation import KERNEL_UNSUPPORTED_FILTERS
|
||||
|
||||
unsupported = KERNEL_UNSUPPORTED_FILTERS.intersection(new_filters or {})
|
||||
if unsupported:
|
||||
raise ControllerError(
|
||||
"Packet filter(s) {} cannot run on a kernel-datapath link (no uBridge relay "
|
||||
"in the forwarding path; delay, packet loss and corrupt are served by tc "
|
||||
"netem on the veth); delete and recreate the link to use them".format(
|
||||
", ".join(sorted(unsupported))
|
||||
)
|
||||
)
|
||||
elif self._created:
|
||||
# The reverse guard: netem-extension filters have no relay
|
||||
# equivalent. Only enforced on created links — while loading a
|
||||
# project the datapath is not decided yet (a link that will be
|
||||
# wired on the kernel datapath must accept them), and the relay
|
||||
# prepare path drops what it cannot run with a warning.
|
||||
if self._created and not self.kernel_datapath:
|
||||
# Kernel-only filters (the netem extensions, the eBPF quota mode)
|
||||
# have no relay equivalent. Only enforced on created links — while
|
||||
# loading a project the datapath is not decided yet (a link that
|
||||
# will be wired on the kernel datapath must accept them), and the
|
||||
# relay prepare path drops what it cannot run with a warning.
|
||||
conflicts = kernel_only_features(new_filters)
|
||||
if conflicts:
|
||||
raise ControllerError(
|
||||
@ -722,13 +721,11 @@ class Link:
|
||||
filter_node = self._get_filter_node()
|
||||
if filter_node:
|
||||
if self.kernel_datapath:
|
||||
# Kernel-datapath links serve filters via tc netem on the
|
||||
# veth host end; types without a netem equivalent (which
|
||||
# only run in the uBridge relay) are hidden from the picker.
|
||||
from gns3server.utils.packet_filter_validation import KERNEL_UNSUPPORTED_FILTERS
|
||||
|
||||
return [f for f in FILTERS if f["type"] not in KERNEL_UNSUPPORTED_FILTERS]
|
||||
# Relay links: hide the netem-extension types the relay cannot run
|
||||
# Kernel-datapath links serve every filter type: netem for
|
||||
# delay/loss/corrupt and the netem extensions, cls_bpf for
|
||||
# bpf, the eBPF classifier for frequency_drop and quota.
|
||||
return FILTERS
|
||||
# Relay links: hide the kernel-only types the relay cannot run
|
||||
# (they would be rejected with 409 on update).
|
||||
from gns3server.utils.packet_filter_validation import KERNEL_ONLY_FILTERS
|
||||
|
||||
|
||||
@ -165,11 +165,10 @@ class UDPLink(Link):
|
||||
"""
|
||||
Whether this link can be wired on the kernel datapath (veth pairs
|
||||
enslaved into a per-link Linux bridge) instead of the uBridge UDP
|
||||
relay. Impairment filters with a tc netem equivalent (delay,
|
||||
packet_loss, corrupt, bpf via cls_bpf, and the netem extensions
|
||||
rate/reorder/gemodel/duplicate/seed/limit) are served on the veth
|
||||
host end; frequency_drop only exists in the uBridge relay and
|
||||
disqualifies the kernel path. Capture and markers are served by
|
||||
relay. Every GNS3 filter type has a kernel equivalent — netem for
|
||||
delay/packet_loss/corrupt and the netem extensions, cls_bpf for bpf,
|
||||
the eBPF stateful classifier for frequency_drop — so filters no
|
||||
longer disqualify the kernel path. Capture and markers are served by
|
||||
uBridge's AF_PACKET modules on the veth host end.
|
||||
|
||||
Docker adapters are born as veth pairs (unified interface), so the
|
||||
@ -185,12 +184,6 @@ class UDPLink(Link):
|
||||
return False
|
||||
if node1.compute.id != node2.compute.id:
|
||||
return False
|
||||
# Look at the *stored* filters, not get_active_filters(): a suspended
|
||||
# link reports the synthetic frequency_drop emulation, which is
|
||||
# relay-only mechanics and must not flip a kernel link to the relay
|
||||
# on reopen/reset.
|
||||
if KERNEL_UNSUPPORTED_FILTERS.intersection(self._filters or {}):
|
||||
return False
|
||||
if _is_unix_socket_docker(node1) or _is_unix_socket_docker(node2):
|
||||
return False
|
||||
return True
|
||||
@ -229,10 +222,10 @@ class UDPLink(Link):
|
||||
# Markers ride the NIO like on the relay datapath, routed by
|
||||
# capture node; they attach to the veth host end via uBridge's
|
||||
# AF_PACKET marker module instead of a relay `mark` filter.
|
||||
# Filters (delay/packet_loss/corrupt plus the netem extensions
|
||||
# rate/reorder/gemodel/duplicate/seed/limit — eligibility
|
||||
# guarantees frequency_drop is absent) become one tc netem qdisc
|
||||
# per veth host end, pushed to both endpoints.
|
||||
# Filters become one tc netem qdisc per veth host end (the netem
|
||||
# surface and its extensions), plus cls_bpf match-drop (bpf) and
|
||||
# the eBPF stateful classifier (frequency_drop, quota) — every
|
||||
# GNS3 filter type has a kernel equivalent now.
|
||||
bridge_name = "gns3" + self._id.replace("-", "")[:11]
|
||||
node1_filters, node2_filters = self._get_node_filters(node1, node2, kernel=True)
|
||||
node1_markers, node2_markers = self._get_node_markers(node1, node2)
|
||||
|
||||
@ -38,7 +38,7 @@ class UDPNIO(BaseModel):
|
||||
filters: Optional[dict] = Field(
|
||||
None,
|
||||
description="Packet filters (relay-supported types only: delay/packet_loss/corrupt/frequency_drop/bpf — "
|
||||
"the netem extensions need a kernel-datapath NIO)",
|
||||
"the netem extensions and the quota mode need a kernel-datapath NIO)",
|
||||
)
|
||||
markers: Optional[dict] = Field(None, description="Traffic-insight markers")
|
||||
|
||||
@ -81,9 +81,9 @@ class BridgeNIO(BaseModel):
|
||||
delay/packet_loss/corrupt and the netem extensions (rate, reorder,
|
||||
gemodel, duplicate, seed, limit, jitter distributions, loss correlation)
|
||||
as one tc netem qdisc, bpf as cls_bpf match-drop classifiers (needs a
|
||||
uBridge reporting cbpf). frequency_drop still has no kernel equivalent
|
||||
and is rejected. Markers and packet capture are served by uBridge's
|
||||
AF_PACKET modules on the veth host end.
|
||||
uBridge reporting cbpf), frequency_drop and quota as the eBPF stateful
|
||||
classifier (needs ebpf). Markers and packet capture are served by
|
||||
uBridge's AF_PACKET modules on the veth host end.
|
||||
"""
|
||||
|
||||
type: BridgeNIOType
|
||||
@ -91,8 +91,8 @@ class BridgeNIO(BaseModel):
|
||||
suspend: Optional[bool] = Field(None, description="Suspend the NIO")
|
||||
filters: Optional[dict] = Field(
|
||||
None,
|
||||
description="Packet filters served on the veth host end (delay/packet_loss/corrupt and the netem extensions "
|
||||
"via one tc netem qdisc, bpf via cls_bpf)",
|
||||
description="Packet filters served on the veth host end (netem surface + extensions via one tc netem qdisc, "
|
||||
"bpf via cls_bpf, frequency_drop/quota via the eBPF classifier)",
|
||||
)
|
||||
markers: Optional[dict] = Field(None, description="Traffic-insight markers (attached to the veth host end)")
|
||||
|
||||
|
||||
@ -16,18 +16,18 @@ class FilterValidationError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
# Packet filters with no kernel-datapath equivalent yet: frequency_drop needs
|
||||
# the eBPF stateful classifier (frozen in
|
||||
# docs/design/ubridge-kernel-impairment-spec.md, part B — not delivered),
|
||||
# so it still only runs in the uBridge userspace relay. bpf is served by the
|
||||
# cBPF match-drop classifier (tc bpf_drop) when uBridge reports cbpf=1.
|
||||
KERNEL_UNSUPPORTED_FILTERS = frozenset({"frequency_drop"})
|
||||
|
||||
# Packet filters with no uBridge *relay* equivalent: the tc netem extensions
|
||||
# (uBridge feature/tc-netem-ext) serve them on the kernel datapath only. The
|
||||
# relay's packet-filter registry only knows frequency_drop / packet_loss /
|
||||
# delay / corrupt / bpf / mark.
|
||||
KERNEL_ONLY_FILTERS = frozenset({"rate", "reorder", "gemodel", "duplicate", "seed", "limit"})
|
||||
# Every GNS3 filter type now has a kernel-datapath equivalent: delay /
|
||||
# packet_loss / corrupt / the netem extensions run as one tc netem qdisc,
|
||||
# bpf as cBPF match-drop classifiers, and frequency_drop as the eBPF
|
||||
# stateful classifier's exact every-Nth mode (tc nth_drop — needs a uBridge
|
||||
# reporting ebpf=1). No type is relay-only anymore; the controller-side
|
||||
# kernel/relay choice is purely topological (same compute, docker/docker).
|
||||
#
|
||||
# Packet filters with no uBridge *relay* equivalent: the netem extensions
|
||||
# plus the eBPF quota mode run on the kernel datapath only. The relay's
|
||||
# packet-filter registry only knows frequency_drop / packet_loss / delay /
|
||||
# corrupt / bpf / mark.
|
||||
KERNEL_ONLY_FILTERS = frozenset({"rate", "reorder", "gemodel", "duplicate", "seed", "limit", "quota"})
|
||||
|
||||
# Jitter distributions embedded in the netem-extension uBridge (tc_netem_dist).
|
||||
NETEM_DISTRIBUTIONS = ("uniform", "normal", "pareto", "paretonormal")
|
||||
@ -220,6 +220,15 @@ def validate_filter_parameters(filter_type: str, values: List[Any]) -> None:
|
||||
},
|
||||
"seed": {"params_count": (1, 1), "ranges": [(0, 4294967295)], "names": ["Seed"], "units": [""]},
|
||||
"limit": {"params_count": (1, 1), "ranges": [(1, 1000000)], "names": ["Limit"], "units": ["packets"]},
|
||||
"quota": {
|
||||
# eBPF stateful classifier (uBridge tc quota_drop): after the byte
|
||||
# quota is consumed, each further packet drops with the given
|
||||
# chance (100 = hard cutoff). Kernel-datapath only.
|
||||
"params_count": (2, 2),
|
||||
"ranges": [(1, 10**15), (0, 100)],
|
||||
"names": ["Quota", "Chance"],
|
||||
"units": ["bytes", "%"],
|
||||
},
|
||||
"bpf": {"params_count": (1, 1), "is_text": True, "names": ["Filters"]},
|
||||
}
|
||||
|
||||
|
||||
@ -34,7 +34,7 @@ from tests.utils import AsyncioMagicMock
|
||||
|
||||
from gns3server.compute.docker import Docker
|
||||
from gns3server.compute.docker.docker_vm import DockerVM
|
||||
from gns3server.compute.docker.docker_error import DockerError
|
||||
from gns3server.compute.docker.docker_error import DockerError, DockerHttp404Error
|
||||
from gns3server.compute.compute_error import ComputeError
|
||||
from gns3server.compute.nios.nio_udp import NIOUDP
|
||||
from gns3server.compute.nios.nio_bridge import NIOBridge
|
||||
@ -102,16 +102,28 @@ def test_create_nio_bridge_accepts_bpf_filters(vm):
|
||||
assert nio.filters == {"bpf": ["icmp"]}
|
||||
|
||||
|
||||
def test_create_nio_bridge_rejects_relay_only_filters(vm):
|
||||
def test_create_nio_bridge_accepts_frequency_drop(vm):
|
||||
"""
|
||||
frequency_drop runs on the kernel datapath as the eBPF classifier's
|
||||
exact every-Nth mode — every filter type is accepted on a kernel NIO.
|
||||
"""
|
||||
|
||||
with pytest.raises(ComputeError, match="kernel-datapath"):
|
||||
vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE, "filters": {"frequency_drop": [10]}})
|
||||
nio = vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE, "filters": {"frequency_drop": [10]}})
|
||||
assert isinstance(nio, NIOBridge)
|
||||
assert nio.filters == {"frequency_drop": [10]}
|
||||
|
||||
|
||||
def test_create_nio_bridge_accepts_quota(vm):
|
||||
|
||||
nio = vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE, "filters": {"quota": [1000000, 100]}})
|
||||
assert isinstance(nio, NIOBridge)
|
||||
assert nio.filters == {"quota": [1000000, 100]}
|
||||
|
||||
|
||||
def test_create_nio_bridge_accepts_markers(vm):
|
||||
"""
|
||||
Markers ride the kernel NIO (attached to the veth host end via uBridge's
|
||||
AF_PACKET marker module); only filters are rejected.
|
||||
AF_PACKET marker module).
|
||||
"""
|
||||
|
||||
nio = vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE, "markers": {"m": {"bpf": "icmp"}}})
|
||||
@ -421,26 +433,134 @@ async def test_connect_nio_udp_on_veth_is_idempotent(vm):
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_update_nio_kernel_rejects_relay_only_filters(vm):
|
||||
async def test_update_nio_kernel_applies_ebpf_drops(vm):
|
||||
"""
|
||||
The PUT route mutates the NIO in place (no create_nio), so the update
|
||||
path carries its own guard against relay-only filter types.
|
||||
frequency_drop becomes the eBPF every-Nth mode and quota the byte-cap
|
||||
mode, applied after the netem/bpf pass on the same NIO update.
|
||||
"""
|
||||
|
||||
vm._ubridge_hypervisor = _caps_hypervisor(ebpf="1")
|
||||
vm._ubridge_send = AsyncioMagicMock()
|
||||
vm.status = "started"
|
||||
host_ifc, _ = vm._veth_names(0, 0)
|
||||
vm._kernel_veths[(0, 0)] = host_ifc
|
||||
nio = vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE})
|
||||
vm._ethernet_adapters[0].add_nio(0, nio)
|
||||
nio.filters = {"frequency_drop": [3], "quota": [1000000, 50]}
|
||||
|
||||
await vm.adapter_update_nio_binding(0, nio)
|
||||
vm._ubridge_send.assert_any_call(f'tc nth_drop "{host_ifc}" 3')
|
||||
vm._ubridge_send.assert_any_call(f'tc quota_drop "{host_ifc}" 1000000 50')
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_update_nio_kernel_ebpf_drop_everything(vm):
|
||||
"""
|
||||
frequency_drop -1 (drop everything) maps to every 1st packet.
|
||||
"""
|
||||
|
||||
vm._ubridge_hypervisor = _caps_hypervisor(ebpf="1")
|
||||
vm._ubridge_send = AsyncioMagicMock()
|
||||
vm.status = "started"
|
||||
host_ifc, _ = vm._veth_names(0, 0)
|
||||
vm._kernel_veths[(0, 0)] = host_ifc
|
||||
nio = vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE})
|
||||
vm._ethernet_adapters[0].add_nio(0, nio)
|
||||
nio.filters = {"frequency_drop": [-1]}
|
||||
|
||||
await vm.adapter_update_nio_binding(0, nio)
|
||||
vm._ubridge_send.assert_any_call(f'tc nth_drop "{host_ifc}" 1')
|
||||
# the quota mode is explicitly off (not left to chance)
|
||||
vm._ubridge_send.assert_any_call(f'tc quota_drop "{host_ifc}" off')
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_update_nio_kernel_turns_ebpf_modes_off(vm):
|
||||
"""
|
||||
A re-apply with the stateful filters removed must turn both modes off
|
||||
(the netem reset already detached the program; the offs are idempotent).
|
||||
"""
|
||||
|
||||
vm._ubridge_hypervisor = _caps_hypervisor(ebpf="1")
|
||||
vm._ubridge_send = AsyncioMagicMock()
|
||||
vm.status = "started"
|
||||
host_ifc, _ = vm._veth_names(0, 0)
|
||||
vm._kernel_veths[(0, 0)] = host_ifc
|
||||
nio = vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE})
|
||||
vm._ethernet_adapters[0].add_nio(0, nio)
|
||||
nio.filters = {"frequency_drop": [3]}
|
||||
|
||||
await vm.adapter_update_nio_binding(0, nio)
|
||||
vm._ubridge_send.reset_mock()
|
||||
nio.filters = {"delay": [50]}
|
||||
await vm.adapter_update_nio_binding(0, nio)
|
||||
|
||||
vm._ubridge_send.assert_any_call(f'tc nth_drop "{host_ifc}" off')
|
||||
vm._ubridge_send.assert_any_call(f'tc quota_drop "{host_ifc}" off')
|
||||
for c in vm._ubridge_send.call_args_list:
|
||||
assert "nth_drop" not in c.args[0] or "off" in c.args[0]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_apply_ebpf_drops_requires_ebpf_capability(vm):
|
||||
"""
|
||||
An old or non-capped uBridge (ebpf=0) gets a clear upgrade error naming
|
||||
the setcap fix.
|
||||
"""
|
||||
|
||||
hyp = MagicMock()
|
||||
hyp.is_running.return_value = True
|
||||
hyp.send = AsyncioMagicMock(return_value=["netem=delay;ebpf=0;cbpf=1"])
|
||||
vm._ubridge_hypervisor = hyp
|
||||
vm._ubridge_send = AsyncioMagicMock()
|
||||
host_ifc, _ = vm._veth_names(0, 0)
|
||||
|
||||
with pytest.raises(DockerError, match="setcap cap_bpf"):
|
||||
await vm._ubridge_apply_ebpf_drops(host_ifc, {"frequency_drop": [3]})
|
||||
vm._ubridge_send.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_apply_ebpf_drops_skips_old_ubridge_without_filters(vm):
|
||||
"""
|
||||
No stateful filters and no capability probe yet: nothing was ever
|
||||
installed on an old uBridge, so it must not be touched at all.
|
||||
"""
|
||||
|
||||
vm._ubridge_hypervisor = MagicMock()
|
||||
nio = vm.manager.create_nio({"type": "nio_bridge", "bridge": BRIDGE})
|
||||
vm._ethernet_adapters[0].add_nio(0, nio)
|
||||
nio.filters = {"frequency_drop": [10]}
|
||||
vm._ubridge_send = AsyncioMagicMock()
|
||||
host_ifc, _ = vm._veth_names(0, 0)
|
||||
|
||||
with pytest.raises(DockerError, match="kernel-datapath"):
|
||||
await vm.adapter_update_nio_binding(0, nio)
|
||||
await vm._ubridge_apply_ebpf_drops(host_ifc, {})
|
||||
vm._ubridge_send.assert_not_called()
|
||||
vm._ubridge_hypervisor.send.assert_not_called()
|
||||
|
||||
|
||||
def _caps_hypervisor(cbpf="1"):
|
||||
@pytest.mark.asyncio
|
||||
async def test_apply_ebpf_drops_cleanup_only_probes_with_ebpf(vm):
|
||||
"""
|
||||
The filter-removal path (modes off) only talks to the eBPF commands
|
||||
when the capability probe reported them — a netem-ext uBridge without
|
||||
the eBPF module must not see an unknown command.
|
||||
"""
|
||||
|
||||
hyp = MagicMock()
|
||||
hyp.is_running.return_value = True
|
||||
hyp.send = AsyncioMagicMock(return_value=["netem=delay,rate;ebpf=0;cbpf=1"])
|
||||
vm._ubridge_hypervisor = hyp
|
||||
vm._ubridge_send = AsyncioMagicMock()
|
||||
host_ifc, _ = vm._veth_names(0, 0)
|
||||
vm._ubridge_tc_caps = None # force the probe (something was set before)
|
||||
|
||||
await vm._ubridge_apply_ebpf_drops(host_ifc, {})
|
||||
vm._ubridge_send.assert_not_called()
|
||||
|
||||
|
||||
def _caps_hypervisor(cbpf="1", ebpf="0"):
|
||||
"""A hypervisor mock whose direct send() answers `tc capabilities`."""
|
||||
hyp = MagicMock()
|
||||
hyp.is_running.return_value = True
|
||||
hyp.send = AsyncioMagicMock(return_value=[f"netem=delay;ebpf=0;cbpf={cbpf}"])
|
||||
hyp.send = AsyncioMagicMock(return_value=[f"netem=delay;ebpf={ebpf};cbpf={cbpf}"])
|
||||
return hyp
|
||||
|
||||
|
||||
@ -1027,3 +1147,25 @@ def test_create_nio_udp_rejects_kernel_only_filters(vm):
|
||||
vm.manager.create_nio(
|
||||
{"type": "nio_udp", "lport": 4242, "rport": 4343, "rhost": "127.0.0.1", "filters": {"gemodel": [100, 0, 30]}}
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_stop_clears_relay_bridge_registry(vm):
|
||||
"""
|
||||
The uBridge process dies with every bridge it hosted — stop() must drop
|
||||
the relay bridge names, or the next start skips "bridge create" (the
|
||||
gate checks the name's presence) and fails on add_nio_udp.
|
||||
"""
|
||||
|
||||
vm._ubridge_hypervisor = MagicMock()
|
||||
vm._ubridge_send = AsyncioMagicMock()
|
||||
vm._bridges.add("bridge1")
|
||||
vm._ubridge_tc_caps = {"netem": "delay"}
|
||||
|
||||
async def query(method, path, *args, **kwargs):
|
||||
raise DockerHttp404Error("no such container")
|
||||
|
||||
vm.manager.query = AsyncioMagicMock(side_effect=query)
|
||||
await vm.stop()
|
||||
assert vm._bridges == set()
|
||||
assert vm._ubridge_tc_caps is None
|
||||
|
||||
@ -68,6 +68,21 @@ def _mock_compute_http(compute):
|
||||
compute.post.side_effect = port_callback
|
||||
|
||||
|
||||
async def _relay_link(project):
|
||||
"""A docker↔qemu link (mixed node types) — never kernel-eligible, so the
|
||||
relay _prepare path runs (UDP ports reserved through the mocked compute)."""
|
||||
|
||||
compute = MagicMock()
|
||||
compute.id = "compute-1"
|
||||
_mock_compute_http(compute)
|
||||
node1 = _node(project, compute, "docker1")
|
||||
node2 = _node(project, compute, "qemu1", node_type="qemu")
|
||||
link = UDPLink(project)
|
||||
await link.add_node(node1, 0, 0, batch=True, dump=False)
|
||||
await link.add_node(node2, 0, 0, batch=True, dump=False)
|
||||
return link, node1, node2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_kernel_datapath_eligible(project):
|
||||
|
||||
@ -117,18 +132,16 @@ async def test_kernel_datapath_eligible_with_netem_filters(project):
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_kernel_datapath_not_eligible_with_relay_only_filters(project):
|
||||
async def test_kernel_datapath_eligible_with_frequency_drop(project):
|
||||
"""
|
||||
frequency_drop only exists in the uBridge userspace relay (its eBPF
|
||||
classifier is not delivered yet) — a link carrying one stays on the
|
||||
relay. bpf is kernel-capable (cls_bpf match-drop) and eligible.
|
||||
frequency_drop runs on the kernel datapath too (the eBPF classifier's
|
||||
exact every-Nth mode) — no filter type forces the relay anymore; only
|
||||
topology does (mixed node types, cross-compute, unix-socket
|
||||
containers, config off).
|
||||
"""
|
||||
|
||||
link, node1, node2 = await _kernel_link(project)
|
||||
link._filters = {"delay": [10, 0], "frequency_drop": [7]}
|
||||
assert link._kernel_datapath_eligible(node1, node2) is False
|
||||
|
||||
link._filters = {"bpf": ["icmp"]}
|
||||
link._filters = {"delay": [10, 0], "frequency_drop": [7], "quota": [1000000, 100]}
|
||||
assert link._kernel_datapath_eligible(node1, node2) is True
|
||||
|
||||
|
||||
@ -277,13 +290,20 @@ async def test_update_netem_filters_accepted_on_kernel_link(project):
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_update_relay_only_filters_rejected_on_kernel_link(project):
|
||||
async def test_update_frequency_drop_accepted_on_kernel_link(project):
|
||||
"""
|
||||
frequency_drop rides the kernel NIO to both endpoints like every other
|
||||
filter type — the compute translates it to the eBPF every-Nth mode.
|
||||
"""
|
||||
|
||||
link, _n1, _n2 = await _kernel_link(project)
|
||||
link, node1, node2 = await _kernel_link(project)
|
||||
await link._prepare()
|
||||
|
||||
with pytest.raises(ControllerError, match="kernel-datapath"):
|
||||
await link.update_filters({"frequency_drop": [10]})
|
||||
await link.update_filters({"frequency_drop": [10]})
|
||||
assert link.filters == {"frequency_drop": [10]}
|
||||
by_node = {entry[0].id: entry[3] for entry in (await link._prepare())}
|
||||
assert by_node[node1.id]["filters"] == {"frequency_drop": [10]}
|
||||
assert by_node[node2.id]["filters"] == {"frequency_drop": [10]}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@ -449,14 +469,12 @@ async def test_update_netem_extension_filters_accepted_on_kernel_link(project):
|
||||
@pytest.mark.asyncio
|
||||
async def test_update_kernel_only_filters_rejected_on_relay_link(project):
|
||||
"""
|
||||
Netem-extension filters have no relay equivalent: setting them on a
|
||||
created relay link is a 409. (frequency_drop in the stored filters
|
||||
forces this link onto the relay datapath.)
|
||||
Netem-extension / quota filters have no relay equivalent: setting them
|
||||
on a created relay link is a 409. (The link is relay-wired because the
|
||||
endpoints are mixed node types.)
|
||||
"""
|
||||
|
||||
link, _n1, _n2 = await _kernel_link(project)
|
||||
_mock_compute_http(link._nodes[0]["node"].compute)
|
||||
link._filters = {"frequency_drop": [7]}
|
||||
link, _n1, _n2 = await _relay_link(project)
|
||||
await link._prepare()
|
||||
assert link.kernel_datapath is False
|
||||
link._created = True # the computes hold the UDP NIOs
|
||||
@ -464,6 +482,10 @@ async def test_update_kernel_only_filters_rejected_on_relay_link(project):
|
||||
with pytest.raises(ControllerError, match="kernel-datapath"):
|
||||
await link.update_filters({"rate": ["512kbit"]})
|
||||
# the stored filters are untouched
|
||||
assert link.filters == {}
|
||||
|
||||
# frequency_drop still runs on the relay (its uBridge userspace filter)
|
||||
await link.update_filters({"frequency_drop": [7]})
|
||||
assert link.filters == {"frequency_drop": [7]}
|
||||
|
||||
|
||||
@ -484,13 +506,13 @@ async def test_update_kernel_only_filters_accepted_before_creation(project):
|
||||
@pytest.mark.asyncio
|
||||
async def test_prepare_relay_link_strips_kernel_only_filters(project):
|
||||
"""
|
||||
A relay-wired link (here: frequency_drop in the stored filters) built
|
||||
from a topology carrying extension filters drops them with a warning
|
||||
instead of pushing an unknown filter type to uBridge.
|
||||
A relay-wired link (mixed node types) built from a topology carrying
|
||||
extension filters drops them with a warning instead of pushing an
|
||||
unknown filter type to uBridge. frequency_drop stays — the relay's
|
||||
userspace filter still serves it.
|
||||
"""
|
||||
|
||||
link, node1, node2 = await _kernel_link(project)
|
||||
_mock_compute_http(link._nodes[0]["node"].compute)
|
||||
link, node1, node2 = await _relay_link(project)
|
||||
link._filters = {"frequency_drop": [7], "rate": ["512kbit"], "delay": [100, 20, "normal"], "packet_loss": [5, 25]}
|
||||
entries = await link._prepare()
|
||||
assert link.kernel_datapath is False
|
||||
|
||||
@ -366,3 +366,20 @@ class TestNetemExtensionFilters:
|
||||
def test_kernel_only_features_empty(self):
|
||||
assert split_kernel_only_features({}) == ({}, set())
|
||||
assert split_kernel_only_features(None) == ({}, set())
|
||||
|
||||
def test_quota_valid_and_invalid(self):
|
||||
validate_filter_parameters("quota", [1000000, 100])
|
||||
validate_filter_parameters("quota", [1, 0])
|
||||
with pytest.raises(FilterValidationError, match="Quota"):
|
||||
validate_filter_parameters("quota", [0, 100])
|
||||
with pytest.raises(FilterValidationError, match="Chance"):
|
||||
validate_filter_parameters("quota", [1000, 101])
|
||||
with pytest.raises(FilterValidationError, match="expects 2 parameter"):
|
||||
validate_filter_parameters("quota", [1000])
|
||||
|
||||
def test_quota_is_kernel_only(self):
|
||||
clean, dropped = split_kernel_only_features({"quota": [1000, 50], "frequency_drop": [7]})
|
||||
assert dropped == {"quota"}
|
||||
# frequency_drop runs on both datapaths (relay userspace filter /
|
||||
# eBPF every-Nth) and stays
|
||||
assert clean == {"frequency_drop": [7]}
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user