gns3-server/docs/features/marker-traffic-insight.md

This documentation is organized by AI with reference to actual code. AI can make mistakes — please verify against the source code when in doubt.

Marker (Traffic Insight)

Overview

A marker is a passive traffic-insight tap attached to a link. It runs a libpcap BPF expression inside uBridge; on every match uBridge emits a real-time MARK signal and appends the matching packet to a per-marker pcap file. Markers exist at two layers that coexist on the same link: per-link private markers and project-level definitions that are inherited by every capable link.

Architecture

graph TB
    UI["Web UI"]

    subgraph Controller["Controller"]
        DEF["Project definitions<br/>(inheritance templates)"]
        LNK["Per-link markers"]
    end

    Compute["Compute Node"]
    UB["uBridge<br/>mark filter"]
    PCAP[("pcap file")]
    LSTN["Marker listener<br/>(UDP, per compute)"]

    UI -->|"REST + notifications ws"| Controller
    DEF -.->|"fan-out: global-{name}"| LNK
    LNK -->|"node.post /markers"| Compute
    Compute --> UB
    UB -->|"BPF match"| PCAP
    UB -->|"UDP MARK signal"| LSTN
    LSTN -->|"marker.match"| UI

Inheritance is a controller-only fan-out: a definition CRUD loops over links and reuses the existing per-link marker operations, so the compute side sees an ordinary marker and is unchanged. Each compute process runs one UDP listener serving every uBridge on that host; the node and link fields in each signal together identify the source link (see Per-link attribution).

Business Process

sequenceDiagram
    participant UI as Web UI
    participant C as Controller
    participant L as Capable Link
    participant N as Compute / uBridge

    UI->>C: POST /marker-definitions {name, bpf, ...}
    C->>C: store definition
    loop every capable link
        C->>L: start_marker("global-{name}")
        L->>N: install mark filter (BPF + pcap)
    end
    C-->>UI: 201 + link_ids

    Note over N: later: a packet matches the BPF
    N->>N: emit MARK signal + append pcap
    N-->>UI: marker.match notification (per-project ws)

Updating a definition syncs bpf / tag / color / highlight_duration to every inherited copy; deleting a definition removes every inherited copy. A newly created link inherits all existing definitions automatically.

Per-link attribution

A uBridge MARK signal carries node, filter, link, tag, and len — but no bridge name. When one node is the capture side for several links — the common case for a project-level global-{name} marker on a multi-interface router — node + filter alone are identical across those links, so they cannot tell the signals (or pcap files) apart. The link field resolves this:

1. At install time the controller stamps each filter with its link id (mark <bpf> [tag <id>] link <link_id> [pcap <path>]).
2. uBridge treats link as opaque and echoes it verbatim in the signal (link=<link_id>).
3. The listener takes the signal's link= as the authoritative link_id of the marker.match event, falling back to its registry only for legacy signals that carry no link=.

This is also why the pcap path is keyed on link — <project>/markers/<node_id>_<link_id>_<filter>.pcap, not on bridge+filter: a single uBridge bridge can serve several links, and only the link id keeps their captures distinct.

IOU: one bridge, many interfaces

IOU runs a single IOL-BRIDGE per node shared by every interface, so bridge+filter are identical across that node's links. uBridge keeps a separate filter list per port (bay/unit) within the bridge, so each interface gets its own global-{name} filter, its own pcap file, and its own link=. The shared bridge name is irrelevant to attribution. Other capable node types (qemu, docker, vpcs, cloud, ethernet_switch) already use one bridge per link (the switch's per-port relay); link applies uniformly to all of them.

Direction

A MARK signal optionally carries dir=<tx|rx> — the matched packet's travel direction relative to the capture node (the node=<id> in the same signal, i.e. the node whose uBridge hosts the marker):

dir	Ingress NIO	Meaning
tx	device side (source_nio on a generic bridge; the IOL instance on an IOU IOL-BRIDGE)	capture node is sending
rx	link side (destination_nio on a generic bridge; the NIO side on an IOU IOL-BRIDGE)	capture node is receiving

A marker is single-sided: only the chosen capture node's uBridge installs the mark filter, yet both directions of the link transit that one bridge (it carries exactly two NIOs — the device side and the link side), so that single uBridge observes and classifies both directions. The marker.match event forwards dir through unchanged; the Web UI combines it with the link's two endpoints and the capture node_id to draw an arrow:

• dir=tx → capture_node → far_node
• dir=rx → far_node → capture_node
• dir absent (older uBridge) → undirected highlight (current behaviour)

Because the listener ignores unknown keys, dir is additive: an older server silently drops it and an older uBridge simply omits it — either way the system falls back to undirected rendering with no error.

Choosing the capture node

Since dir is relative to the capture node, which endpoint is the observer decides what tx/rx mean. By default the server auto-picks (first started marker-capable endpoint, in link-endpoint order). To pin it — e.g. so dir=tx unambiguously means "vpcs1 is sending" — pass capture_node_id on marker create:

{ "bpf": "icmp", "direction": "tx", "capture_node_id": "<vpcs1 node uuid>" }

The value must be one of the link's two endpoints and a marker-capable type (vpcs, qemu, docker, iou, dynamips, cloud, ethernet_switch); any other id is rejected with 409. Omit it to keep the auto-pick. The chosen id is echoed back as capture_node_id in the marker entry and in each MARK signal's node=<id>, so the Web UI always knows the observer regardless of who picked it.

capture_node_id is create-only: it is fixed once the marker exists (changing the observer would silently flip the meaning of stored direction, so recreate the marker instead). It is not accepted on project-level definitions — a definition is link-agnostic and has no endpoints to choose from, so inherited markers always auto-pick per link.

For the same reason, a definition rejects direction: tx|rx (HTTP 409): each inherited copy auto-picks its capture node, so a fixed tx/rx would denote different session directions on different links. A definition is both only; encode the direction you want in the BPF instead — e.g. icmp and icmp[icmptype]==8 for echo requests, a packet-intrinsic property that is consistent on every link regardless of capture node. tx/rx remains available on per-link markers, where the capture node is fixed.

Pause & resume

Two levels of silencing, both instant (no NIO rebuild, no pcap flush):

• Per-marker (private) — PUT /v3/projects/{pid}/links/{lid}/markers/{name} with {"enabled": false} flips that one filter off in place (uBridge enable_packet_filter … off): no signal, no pcap, but traffic still relays — a paused mark is a no-op tap, not a drop. {"enabled": true} flips it back. A change to enabled alone is a single command (the pcap identity and emitted counter are preserved). Changing bpf, tag, or direction rebuilds just that one filter (delete_packet_filter + add) — only that marker's own pcap reopens (a new capture session for the new BPF); changing color/highlight_duration is UI-only, nothing is pushed to uBridge.
• Per-definition (inherited) — POST /v3/projects/{pid}/marker-definitions/{name}/pause and /resume toggle every inherited global-{name} copy across all links at once (same enable_packet_filter on|off, fanned out per copy). Use to pause or resume a whole rule independently of the others. The definition's paused flag is persisted to the .gns3 and echoed on the definition object, so links created later inherit it already paused, and the Web UI renders the per-rule button from server truth.

Action	signal	pcap	sink
per-marker enabled: false	stop	stop	n/a
per-def pause (all global-{name} copies)	stop	stop	n/a
per-def resume	resume	resume	n/a

Capture files

Each marker appends matches to <project>/project-files/markers/<node_id>_<link_id>_<filter>.pcap. Removing a marker — per-link DELETE .../markers/{name} or deleting a definition (which removes every inherited copy) — deletes that marker's pcap too, even with the capture node stopped (the filter is removed with delete_packet_filter, the file is unlinked). uBridge's reset_packet_filters (run on NIO/filter changes) preserves mark filters, so unrelated changes no longer close/reopen any marker's pcap.

API Endpoints

All endpoints require a JWT bearer token (POST /v3/access/users/authenticate). The Auth column lists the required privilege.

Per-link markers

Method	Path	Description	Auth
GET	/v3/projects/{pid}/links/{lid}/markers	List markers on a link	Link.Audit
POST	/v3/projects/{pid}/links/{lid}/markers	Attach a marker	Link.Modify
PUT	/v3/projects/{pid}/links/{lid}/markers/{name}	Update a marker	Link.Modify
DELETE	/v3/projects/{pid}/links/{lid}/markers/{name}	Remove a marker	Link.Modify

Project-level definitions

Method	Path	Description	Auth
GET	/v3/projects/{pid}/marker-definitions	List definitions + bound link_ids	Project.Audit
POST	/v3/projects/{pid}/marker-definitions	Create definition (fans out to every link)	Project.Modify
PUT	/v3/projects/{pid}/marker-definitions/{name}	Update definition (syncs all copies)	Project.Modify
DELETE	/v3/projects/{pid}/marker-definitions/{name}	Delete definition (clears all copies)	Project.Modify
POST	/v3/projects/{pid}/marker-definitions/{name}/pause	Pause every inherited copy (instant, persisted)	Project.Modify
POST	/v3/projects/{pid}/marker-definitions/{name}/resume	Resume every inherited copy	Project.Modify

Aggregation

Method	Path	Description	Auth
GET	/v3/projects/{pid}/markers	All markers across links, flat	Project.Audit

The link object returned by GET /v3/projects/{pid}/links[/{lid}] also carries a markers field (including inherited markers), so the Web UI can render a link's markers without an extra request.

Request / Response

Marker create body (MarkerCreate, shared by per-link POST and PUT):

{
  "name": "icmp",
  "bpf": "icmp",
  "tag": 1,
  "direction": "tx",
  "capture_node_id": "a37e2235-e21f-46c9-a2ab-ba0f8c5465e6",
  "color": "#ff5722",
  "highlight_duration": 800,
  "enabled": true
}

direction and capture_node_id are both optional and create-only (see Direction).

Definition create body (MarkerDefinitionCreate, shared by POST and PUT):

{
  "name": "arp",
  "bpf": "arp",
  "tag": 5,
  "color": "#ff5722",
  "highlight_duration": 1200
}

Marker entry (returned by GET/POST/PUT, and the value of each link's markers[name]):

{
  "bpf": "icmp",
  "tag": 1,
  "enabled": true,
  "color": "#ff5722",
  "highlight_duration": 800,
  "capture_node_id": "a37e2235-e21f-46c9-a2ab-ba0f8c5465e6",
  "inherited_from": null
}

Definition GET response (adds link_ids):

{
  "arp": {
    "bpf": "arp",
    "tag": 5,
    "color": null,
    "highlight_duration": 1200,
    "direction": null,
    "paused": false,
    "link_ids": ["656ed826-...", "6bd9d156-..."]
  }
}

Field Reference

Marker entry

Field	Type	Description
bpf	string	libpcap BPF expression (required)
tag	int | null	Correlation id echoed in MARK signals
enabled	bool	Whether the marker is active. Toggle is instant: false flips the uBridge filter off in place (no signal/pcap), true back on — no NIO rebuild (see Pause & resume)
color	string | null	Hex color render hint, e.g. #ff5722
highlight_duration	int | null	UI highlight duration in ms after a match; null = UI default
direction	string | null	tx / rx filter relative to the capture node; null = both
capture_node_id	string	Node whose uBridge hosts the marker — caller-set on create, else auto-picked
inherited_from	string	Source definition name — present on inherited markers only

Definition

Field	Type	Description
bpf	string	libpcap BPF expression (required)
tag	int | null	Correlation id
color	string | null	Hex color render hint
highlight_duration	int | null	UI highlight duration in ms; null = UI default
direction	string | null	tx / rx filter relative to the capture node; null = both
paused	bool	Per-definition mute flag — true mutes every inherited copy (persisted)
link_ids	string[]	Links currently carrying an inherited copy (GET only)

Notifications

Event	Payload	Delivered to
link.updated	Link object (its markers field is the source of truth)	Project notification ws
marker.match	project_id, node_id, link_id, filter, tag, ts, len, dir	Project notification ws only

The marker.match link_id is taken from the signal's link= field (authoritative); see Per-link attribution. The dir field is the matched packet's travel direction relative to the capture node; see Direction.

Error Responses

Status	Description
401	Not authenticated
404	Link / marker / definition not found
409	Per-link edit or delete of an inherited marker; reserved (global) name or duplicate name on create
422	Validation failure (name format, highlight_duration < 1, missing bpf)

Notes

• Marker name is immutable. It is the identifier across the controller, the uBridge filter, the pcap filename, and MARK signal routing — so rename is a delete + recreate, not a field update. PUT ignores the body name; the {name} path parameter identifies the target, and only bpf / tag / color / enabled / highlight_duration are changeable. Names are 1–32 chars ([A-Za-z0-9][A-Za-z0-9_.-]*); inherited copies carry a global- prefix, so their filter names reach ~39.
• global prefix reserved. User-chosen names may not start with global; inherited markers are stored as global-{definition_name} so the two namespaces cannot collide. Omitting name on create yields an auto-generated, prefix-free name.
• Inherited markers are read-only per-link. PUT/DELETE on an inherited marker returns 409 — edit them through the definitions API.
• Render hints are not enforced. color and highlight_duration (milliseconds, >= 1) are stored on the link and never sent to uBridge; null lets the UI apply its own default. A partial PUT (e.g. changing only bpf) leaves them untouched.
• BPF is validated once per source. A private per-link marker validates its BPF inline on create/update. A definition validates its BPF once at create/update (and once per definition on project load, dropping any whose BPF has gone invalid); the inherited fan-out to every link then skips re-validation, so creating a definition over N links runs one tcpdump -d rather than N. (uBridge still runs pcap_compile itself at install time, so an invalid expression can never slip through.)
• Supported node types. A marker needs a uBridge bridge: vpcs, qemu, docker, iou, dynamips, cloud, ethernet_switch (one capable endpoint suffices). The ethernet_switch hosts markers on its per-port uBridge relays (brctl backend); the ethernet_hub is still Dynamips-hosted and has no uBridge. Types without a uBridge are silently skipped by the inheritance fan-out. IOU uses one shared IOL-BRIDGE per node but keeps filters, pcap files, and link= ids per port, so multi-interface nodes are handled (see Per-link attribution).
• Shared capture-side node. When one node hosts markers for several links (typical for global-* definitions on a router), each filter is stamped with its link_id so signals and pcap files stay link-distinct; the controller never collapses them to a single link.
• Persistence. Definitions and private markers persist in the topology; inherited markers are re-created from definitions on project load, so reopening a project restores the same configuration and stale inherited copies cannot survive on disk.
• Log interpretation across node types. Each node type logs its startup and link operations differently — do not mistake sparse logs from one type for inactivity. QEMU prints set_link gns3-<N> on via its QEMU monitor, which is the most visible startup log among all types. VPCS, Docker, IOU, Dynamips, and Cloud each have their own startup paths (fork + ubridge, container veth, iouyap, Dynamips hypervisor, and TAP device respectively) and none of them emit QEMU-monitor-style logs. To verify marker operations (toggle, pause, resume) on non-QEMU types, either inspect uBridge's own log for enable_packet_filter / marker pause / marker resume commands, or watch the gns3server log for the corresponding compute-route calls at INFO level.