From 1cddeb8c3fe05151f56a123346f8f038053ee775 Mon Sep 17 00:00:00 2001 From: YueGuobin Date: Fri, 14 Aug 2026 19:50:58 +0800 Subject: [PATCH] docs: record the link UDP self-loop bug; mark XRd datapath validated Add bugs/link-udp-self-loop.md: the intermittent one-way docker link observed during XRd validation (one end's nio_udp rport pointing at its own lport after a uBridge restart), with the evidence table, the uBridge console capture diagnostics, the delete/recreate workaround, and the narrowed suspects (batch port preallocation / link re-creation race). Also note that a docker node's in-container ethN is a TAP device held by uBridge (no veth host end exists). Update the XRd feature doc: datapath validated end-to-end (XRd brings its own interfaces up, ARP/ICMP bidirectional), add a troubleshooting row pointing at the bug doc. Index the bug in docs/README.md. --- docs/README.md | 1 + docs/bugs/link-udp-self-loop.md | 77 +++++++++++++++++++++++++++++++++ docs/features/vendor-nos-xrd.md | 2 + 3 files changed, 80 insertions(+) create mode 100644 docs/bugs/link-udp-self-loop.md diff --git a/docs/README.md b/docs/README.md index 6c5498c50..79908e27c 100644 --- a/docs/README.md +++ b/docs/README.md @@ -109,6 +109,7 @@ Cisco XRd as a GNS3 Docker router: vendor path + shm/device injection (`GNS3_SHM ## Known Issues (`bugs/`) - [Telnet Server Connection Race Condition](bugs/telnet-server-connection-race-condition.md) — `getpeername()` error when client disconnects during connection setup (High severity, Open) +- [Docker Link UDP Self-Loop](bugs/link-udp-self-loop.md) — intermittent one-way link (one end's NIO points at itself); delete/recreate the link as workaround (Medium severity, Open) --- diff --git a/docs/bugs/link-udp-self-loop.md b/docs/bugs/link-udp-self-loop.md new file mode 100644 index 000000000..98c4ba1cc --- /dev/null +++ b/docs/bugs/link-udp-self-loop.md @@ -0,0 +1,77 @@ + + +> This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt. + + +# Docker Link UDP Self-Loop Bug (One-Way Link) + +## Bug Report + +**Date**: 2026-08-14 +**Severity**: Medium (one-way connectivity, CPU burn from packet duplication; intermittent) +**Status**: Open — root cause not yet pinpointed; workaround reliable +**Component**: Link wiring — `gns3server/controller/udp_link.py` (`_prepare` / +`pop_preallocated_udp_port` in `gns3server/controller/project.py`) interacting with +node/uBridge restarts + +## Symptoms + +Two Docker nodes (observed with Cisco XRd; likely node-type agnostic) linked on the +same compute cannot ping each other. Packet capture on the link shows only **one** +side sending ARP. The other side's traffic never appears on the link at all. + +## Evidence (from a live occurrence) + +Captured simultaneously on both nodes' uBridge `bridge1` (see diagnostics below): + +| Direction | Result | +|---|---| +| A → B | works — A's ARP requests arrive at B's bridge, B replies | +| B → A | dead — B's replies/ICMP appear only on **B's own bridge** (duplicated ×2–×3), nothing arrives at A | +| B's `ethN` counters | RX ≈ TX ≈ 5000+ — B receives its own transmissions back | +| A's `ethN` counters | TX > 0, RX = 0 — never receives anything | + +Conclusion: **B's `nio_udp` rport pointed at B's own lport** — a UDP self-loop. B's +uBridge had restarted ~77 s after A's (node stop/start during the session), +i.e. the link was re-wired at least once. + +Deleting and re-creating the link fixed it immediately (fresh port allocation). + +## Root Cause Analysis (narrowed, not final) + +- `UDPLink._prepare()` builds mirrored NIO data correctly + (`node1: lport=P1/rport=P2`, `node2: lport=P2/rport=P1`) — the logic itself is sound. +- Suspects for the corrupted runtime state: + 1. `Project.pop_preallocated_udp_port()` — the batch project-open preallocation + (link-create performance work) racing with link re-creation; + 2. link re-creation racing a node/uBridge restart (commit NIOs to a uBridge that is + being torn down/rebuilt), leaving a stale/self-pointing NIO on one side. +- A deterministic reproduction is still needed: create two Docker nodes + link, + restart one node, then verify the UDP wiring (see diagnostics). + +## Diagnostics (uBridge console is the fast path) + +1. **uBridge console** — each node's uBridge listens on a Unix socket + `/run/user/1000/gns3/ubridge-.sock`; connect and send: + `bridge list` (NIO count per bridge), and + `bridge start_capture bridge "/tmp/ub-.pcap"` / + `bridge stop_capture bridge` to capture what the bridge actually forwards. + Comparing the two ends' pcaps localizes the break immediately. +2. **Container counters** — `docker exec ip -s link show ethN`: + TX>0/RX=0 → peer never returns; RX≈TX huge with µs-scale duplicates → self-loop. +3. **UDP sockets** — `ss -uln` (no `-p`; uBridge runs setuid-root so process names + are hidden, ports are visible): a two-node link owns exactly two "orphan" UDP ports. +4. **Workaround** — delete and re-create the link (or stop/start both nodes). + +Note: a Docker node's in-container `ethN` is a **TAP device** whose file descriptor +lives inside uBridge (the interface is created host-side, then moved into the +container namespace and renamed). There is no veth host end to look for — do not +waste time hunting for one in the host namespace. + +## Related + +- `docs/features/vendor-nos-xrd.md` — troubleshooting table entry pointing here. +- Link-create batch optimization (`controller/udp_link.py`, project-open bulk path). diff --git a/docs/features/vendor-nos-xrd.md b/docs/features/vendor-nos-xrd.md index e586e3f3d..633a4b580 100644 --- a/docs/features/vendor-nos-xrd.md +++ b/docs/features/vendor-nos-xrd.md @@ -158,6 +158,7 @@ sequenceDiagram | `Invalid interface entries ... XR_MGMT_INTERFACES` | use `xr_name=Mg0/RP0/CPU0/0` | | Host audio muted / USB reconnects when the node starts | set `GNS3_MASK_UDEV=1` | | Config lost across stop/start | `extra_volumes` must be `/xr-storage-shadow` | +| Two nodes can't ping, only one side ARPs | GNS3 link wiring bug (UDP self-loop on one end), not XRd — delete and re-create the link; see [link-udp-self-loop](../bugs/link-udp-self-loop.md) | | Compute log: busybox coredump storm | fixed by the container-chown change; verify gns3-server is current | ## Notes @@ -190,4 +191,5 @@ sequenceDiagram | Version | Date | Changes | |---------|------|---------| +| 1.1 | 2026-08-14 | Datapath validated end-to-end (XRd brings its own interfaces up; ARP/ICMP bidirectional). Add troubleshooting entry for the one-way-link symptom (GNS3 link UDP self-loop bug, see bugs/link-udp-self-loop.md). | | 1.0 | 2026-08-14 | Initial documentation of the XRd control-plane adaptation: vendor path requirement, shm/devices/extra_configs/udev-mask mechanisms, host-disturbance root causes, appliance recipe. |