Document all 70 configuration fields with pydantic Field
descriptions, ported from the config sample comments and verified
against the actual consumers (allow_remote_console and local had
no documentation anywhere). The descriptions flow into the
OpenAPI schema of GET /v3/settings, giving the Web UI tooltips,
defaults and validation bounds from a single source. Response
models re-declare six path fields as plain strings, which drops
the inherited description — restore them explicitly.
Sync the config sample: document local and allow_remote_console,
drop hardware_virtualization_check which no longer exists in the
schema.
GET returns all settings sections except the deprecated
VirtualBox/VMware ones (Server.Audit privilege); secrets are
masked and Controller.jwt_secret_key is excluded entirely.
PUT applies a partial update (Server.Modify privilege): masked
or empty secrets mean unchanged, null removes the option, the
response carries the new values plus restart_required, and a
settings.updated notification is emitted. New
Server.Audit/Server.Modify privileges are seeded into the
Administrator role at table creation.
New GET /v3/netmiko/device_types endpoint returns the device types
supported by the netmiko library installed on the server, including
the gns3-copilot custom drivers, so the web UI can populate the
netmiko_device_type dropdown on templates and nodes.
The list is read at runtime from netmiko's ssh_dispatcher.CLASS_MAPPER
registry (the same table ConnectHandler dispatches on), filtered to
drop the '_ssh' aliases and the 'autodetect' pseudo type, and cached
for the process lifetime. Returns 501 when netmiko is not installed
(ai-features extra).
Add default_username/default_password as controller-only node properties
(the netmiko_device_type pattern): they are not sent to the compute,
persist with the project topology and can be updated or cleared per
node. Creating a node from a template seeds them from the template
appliance metadata, and the metadata itself is dropped there so it
never leaks into the node properties.
Appliance fields that describe the appliance (vendor information, default
credentials, installation instructions...) were dropped when installing a
template. Keep them in a new appliance_metadata JSON column on the
templates table, filled by the appliance-to-template conversion for both
registry v1-6 and v8 (version level values override the appliance level
ones). The nested schema allows extra fields so future registry fields
persist without a migration.
- install: resolve the image directory from the version's settings type and
skip image handling for docker appliances; guard appliance.images
- appliance schema: validate template_properties against template_type,
align cpu_throttling with the qemu template, add kvm and version idlepc
- conversion: map IOU image to path, kvm disable to accel=tcg, inherit only
same-type default settings, symbol fallback from the effective category,
template_properties cannot override structural fields
- allow clearing netmiko_device_type with an empty string
- download the template symbol regardless of the level it is defined at and
give qemu guests a default symbol
Both the v1-6 and v8 appliance models accept an optional top-level
netmiko_device_type, and ApplianceToTemplate copies it into the created
template so installed appliances carry the automation hint end to end.
netmiko_device_type follows the CONTROLLER_ONLY_PROPERTIES pattern
(like console_auto_start): a node created from a template inherits the
template value, PUT /nodes can override it inside a topology, updates
never round-trip to the compute, and the value persists in the project
topology file.
Common template field (schema + templates table column + Alembic
migration) holding the Netmiko device type (e.g. 'cisco_xr', 'nokia_srl')
so Netmiko/Nornir based tooling can look up how to reach a node's CLI
without hard-coded vendor mappings. Free-form lowercase string on
purpose: Netmiko's platform list evolves independently of GNS3.
DockerPropertiesV8 now accepts custom_adapters (already available to
v1-6 top-level appliances and to Qemu v8 properties), so port-named
Docker appliances (XRd, SR Linux) can move to the v8 format without
losing their interface naming.
Appliance.type resolves the node type from the v8 settings template_type
(default set first) instead of misclassifying every v8 appliance as
qemu, and _get_default_symbol applies the docker guest symbol to v8
Docker guest appliances.
Add an `extra_configs` field (list of {target, content}) to the docker
node/template/appliance schemas. For each entry GNS3 writes `content` to a
file in the node working directory and bind-mounts it read-only at `target`
inside the container.
This lets a NOS appliance seed its startup config without rebuilding the
image: XRd points XR_FIRST_BOOT_CONFIG at an injected /firstboot.cfg, FRR at
/etc/frr/frr.conf, etc. The bind is a single-file mount applied at create
time, so it works for both the generic init.sh path and vendor nodes that
skip init.sh (console_type=docker_exec). Entries are only injected when
present, so ordinary nodes are unaffected.
The content can't go through `environment` (it is line-delimited, one var per
line), hence a dedicated field -- the same plumbing shape as extra_volumes.
The top-level ApplianceV1_6 model declared first_port_name /
port_name_format / port_segment_size but not custom_adapters, so the
GET /appliances endpoint (response_model=schemas.Appliance) stripped
custom_adapters from the response — the frontend never saw per-adapter
port names even though the appliance file and the server-side template
conversion (appliance_to_template reads it from the raw dict) handled it.
Add custom_adapters: Optional[List[CustomAdapterItem]] to ApplianceV1_6
so the field round-trips through the API. (ApplianceV8 already models it
inside its TemplateSetting.)
The Docker appliance Pydantic model (DockerConsoleType) rejected
console_type='docker_exec', so an appliance file using the vendor NOS
docker_exec console could not be loaded (validation error at import).
Add docker_exec to the enum — it is already a valid ConsoleType
(schemas/common.py) and is handled by VendorDockerVM.
Markers now work on serial links (Cisco HDLC / PPP / Frame Relay / ATM), not just Ethernet. A marker carries a data_link_type (default DLT_EN10MB); at the uBridge boundary it becomes the 'mark ... linktype <dlt>' keyword so the BPF compiles and the pcap is written with the matching link-layer.
- MarkerCreate / MarkerDefinitionCreate gain data_link_type (default DLT_EN10MB). Per-link it is create-only; definitions are updatable (a change re-fans-out).
- base_node._marker_linktype() normalizes the GNS3 DLT name (strip DLT_, uppercase, None for EN10MB). Single source is SerialPort.data_link_types, so Cisco PPP -> PPP_SERIAL (50), matching the capture path -- no second mapping table.
- _ubridge_add_marker_filter (generic) and the IOU marker loop append 'linktype <dlt>'.
- Definition fan-out branches on link_type in inherit_marker: Ethernet is always EN10MB; a serial link uses the definition's WAN encapsulation, or is SKIPPED when none was chosen (an EN10MB pcap on serial is undecodable). One definition covers a mixed topology.
- MCP marker_definition exposes data_link_type (None = not forwarded).
- No uBridge rebuild on a data_link_type change -- only that one marker's filter is swapped (delete + re-add), mirroring a BPF change; reset_packet_filters preserves sibling mark filters.
Requires the uBridge build with 'mark ... linktype' support.
128 characters was far beyond any realistic marker label (icmp, arp, tcp-syn)
and would have collided with the pcap filename budget once a tag prefix is
added later. Cap the user-facing name at 32 in both MarkerCreate and
MarkerDefinitionCreate; the compute-side name guard now also rejects names
longer than 48, which covers the `global-{def_name}` inherited form (≤ 39).
update_marker reused MarkerCreate, whose bpf is required, so a partial PUT like
{"enabled": false} was rejected with 422 "bpf field required". Add a MarkerUpdate
schema with every field optional (bpf included; capture_node_id and name are
create-only/path-driven and omitted) and use it for the PUT route — partial
updates now validate cleanly.
A marker is single-sided — only the chosen capture node's uBridge installs the
mark filter — and dir=tx|rx is interpreted from that node's perspective. Until
now the observer was always auto-picked (_choose_marker_side), so dir=tx meant
"the auto-chosen endpoint is sending", which is unpredictable and makes the
direction filter hard to render meaningfully in the Web UI.
Add an optional create-only capture_node_id to MarkerCreate: when set, the
marker is pinned to that endpoint's uBridge (validated as a link endpoint and a
marker-capable type); when omitted, behavior is unchanged (auto-pick). The
chosen id is already echoed back as capture_node_id and in MARK signals, so the
UI can always render the observer regardless of who picked it.
capture_node_id is create-only (changing it would silently flip the meaning of
stored direction; recreate instead) and is not accepted on project-level
definitions — they are link-agnostic and have no endpoints, so inherited
markers keep auto-picking per link.
Plumbed through REST create_marker, the MCP link_marker tool, and base
Link.start_marker. update_marker does not forward it.
Direction field in marker.match events
=======================================
Read ubridge dir=<tx|rx> from MARK signal datagrams and forward it
as a "dir" key in the marker.match notification event. The field is
additive -- older ubridge builds omit it and the parser leaves it null,
so consumers fall back to undirected rendering with no version coupling.
Semantics are relative to the capture node (the signal's node=<id>):
tx = capture node is sending (ingressed device-side NIO), rx = it is
receiving (ingressed link-side NIO).
Per-marker direction filter (opt-in, server-side pipeline)
==========================================================
Add a direction field to MarkerCreate and MarkerDefinitionCreate
schemas ("tx" | "rx" | null). Plumb it through the full pipeline:
Schema -> controller (start_marker/update_marker, marker_entry,
_markers_for_node, update_marker_definition sync)
-> REST/MCP handlers -> compute _ubridge_add_marker_filter +
IOU _ubridge_apply_markers -> bridge add_packet_filter dir <tx|rx>
When set, ubridge only fires the mark handler (signal + pcap) for
packets matching the chosen direction. null (default/legacy) = both
directions -- zero behavioural change for existing markers.
Docs and tests
==============
- docs/features/marker-traffic-insight.md: signal format updated,
new Direction section with NIO mapping, arrow mapping, and additive
compatibility note.
- tests/compute/marker/test_marker_manager.py: 3 new parser tests
(dir tx/rx/absent) plus existing test extended to assert dir=None.
13 files, +123/-22, 72 tests pass (zero breakage)
The global-prefix reservation belongs on the create path, where it keeps
user-chosen names disjoint from inherited global-{name} markers. It was
implemented as a field_validator on MarkerCreate / MarkerDefinitionCreate,
which also back the PUT update bodies — so updating an inherited marker
while echoing its name in the body tripped the validator with a generic
422 before the controller could return the actionable 409 ("inherited,
use the marker-definitions API").
Drop the schema validators and enforce the prefix in the two create routes
(ControllerError -> 409). PUT no longer validates the body name (ignored
anyway — the target is the {name} path param), so editing an inherited
marker now reaches the controller's inheritance guard and returns the
clear 409. The name-format regex stays on the schema (still 422).
Per-marker UI hint (milliseconds, ge=1) for how long the Web UI keeps a
marker highlighted after a match. Mirrors color: stored on the link,
persisted in the topology, inherited via project-level definitions, and
never sent to uBridge. Omitted = null = frontend uses its own default.
The default is intentionally NOT set in the schema: MarkerCreate also
backs PUT updates, so a schema default would make every partial update
silently reset the value. None means "not provided" (keep existing on
update / frontend decides on create).
pydantic_core (Rust engine) does not support regex look-around.
Keep the character-class pattern for basic safety and enforce the
"global" prefix reservation via a field_validator instead.
Project-level marker definitions fan out to every link (existing and new).
A definition is stored once on Project._marker_definitions; when applied to
a link the marker is named "global-{def_name}" — the "global" prefix was
pre-reserved in the schema, so inherited and per-link markers can never
collide, nor will their registry keys.
Key behavior:
- POST /projects/{pid}/marker-definitions → fan out to all existing links
- PUT /projects/{pid}/marker-definitions/{name} → sync all inherited copies
- DELETE → remove every inherited copy from every link
- New links auto-inherit all active defs (hook in UDPLink.create)
- Per-link DELETE/PUT of a "global-*" marker is rejected (409)
- Inherited markers are NOT persisted in the topology; they are re-created
from _marker_definitions on project load
- Compute side is untouched — the marker reaches uBridge via the existing
start_marker→update→NIO→ubridge pipeline
Files:
- controller/project.py — _marker_definitions + CRUD + fanout + topology load
- controller/link.py — Link.inherit_marker() + asdict() filter
- controller/udp_link.py — guards on stop/update + create() inheritance hook
- controller/topology.py — persist marker_definitions in project topology
- schemas/controller/links.py — MarkerDefinitionCreate schema
- api/routes/controller/projects.py — REST endpoints (marker-definitions)
Add a name pattern constraint to MarkerCreate schema (alphanumeric +
_.-) to prevent injection into uBridge commands, with a matching
defense-in-depth check in _ubridge_add_marker_filter for hand-edited
topology files. Also add the missing "enabled" field to MarkerCreate
and pass it through the PUT route so the documented toggle actually
works.
Markers now follow exactly the same apply pattern as packet filters:
state lives in Link._markers, application goes through NIO
(update() -> PUT /nio -> _ubridge_apply_markers). The former
immediate-apply REST endpoints (/markers/start, /markers/stop on
the compute side) and the per-node start_marker/stop_marker methods
are removed — they were a legacy of the original capture-inspired
design and have been superseded by the NIO flow.
Changes:
- controller/udp_link: start_marker/stop_marker/update_marker now
set _markers state + call self.update() (mirrors update_filters).
Removed _marker_capture_nodes runtime dict and its helpers.
- controller/project: _create_link_from_topology_data restores
_markers directly from persisted data (with BPF validation,
like filter reload). No long calls start_marker during load.
- compute: _ubridge_apply_markers swallows BPF compile errors
(warn+skip), matching _ubridge_apply_filters behaviour so a
single bad expression cannot break link creation / node restart.
- Removed: /markers/start,stop endpoints (6 handlers across
vpcs/qemu/docker route files), node start_marker/stop_marker
methods (3 VM files), _ubridge_delete_marker_filter,
_marker_capture_nodes, MarkerDelete schema.
Net: ~280 lines of dead code removed; marker and packet filter now
share a single, unified apply path via the NIO.
Add a user-chosen hex color (e.g. '#ff5722') to each marker, stored in
Link._markers alongside bpf/tag/enabled. The color:
- Is serialized with the link (asdict) and persisted in the topology, so
it survives project reload and is consistent across devices.
- Never reaches uBridge (compute ignores it) — it is purely a Web UI
concern.
- Can be updated independently of the BPF (color-only changes skip the
ubridge round-trip).
MarkerCreate schema gains color: Optional[str]; start_marker and
update_marker carry it through; REST create/update endpoints pass it.
The marker.match WebSocket event stays lean (no color) — the Web UI maps
filter -> color from the link state it already holds.
- New config: Controller.jwt_refresh_token_expire_minutes (default 30 days)
- New endpoint: POST /v3/access/users/refresh (public, unauthenticated)
- Login/authenticate responses now include refresh_token
- AuthService: _create_token helper, create_refresh_token, get_token_data
now parses type claim (token_use) for token classification
- Security: refresh tokens rejected on HTTP + WebSocket access paths;
/refresh strictly requires type=='refresh'
- Logout works for free via existing token_version mechanism
- Tests: 9 new TestRefreshToken cases, all passing; 34 existing tests
still pass (no regressions)
- New db model: api_keys table with bcrypt-hashed keys
- New API: POST/GET/DELETE /v3/access/api-keys endpoints
- MCP _resolve_token: validates API keys, resolves to 5-min JWT
- API keys inherit the creating user's RBAC permissions
- MCP auth supports both JWT (24h) and API key (permanent) tokens
- Stream file GET/POST through controller without buffering in memory
- Add recursive and subdirectory filtering to node file listing
- Replace file extension with magic-based file type detection
- Add DELETE endpoint for node and project files
- Include directories in listing response
- Add params and stream support to http_query
- Fix lambda closures, streamer exception scope, and delete error codes
This commit adds a new property to the Link class, allowing users to control whether filter icons are displayed in the Web UI at the individual link level.
**Changes:**
- Added attribute to Link class (default: True)
- Added property getter
- Added method for updating the property
- Updated to include the new field with backward compatibility
- Added field to LinkBase schema using Optional[bool] = Field(True, ...) pattern
- Updated API routes to handle the new field in create and update operations
- Added loading logic for show_filters_icon in project.open() to preserve settings when reopening projects
**Schema Definition:**
Uses the same pattern as the field:
**API Impact:**
- POST /v3/projects/{project_id}/links - accepts in request body
- PUT /v3/projects/{project_id}/links/{link_id} - can update
- GET /v3/projects/{project_id}/links/{link_id} - returns field
**Future Applications:**
This feature provides granular control for future AI fault injection modules to manage link-level protocol failures while maintaining clean UI presentation.
Set default value for show_filters_icon in LinkBase schema to ensure the field is always included in API responses, even when response_model_exclude_unset=True is used.
This commit adds a new `show_filters_icon` property to the Link class, allowing users to control whether filter icons are displayed in the Web UI at the individual link level.
**Changes:**
- Added `_show_filters_icon` attribute to Link class (default: True)
- Added `show_filters_icon` property getter
- Added `update_show_filters_icon()` method for updating the property
- Updated `asdict()` to include the new field in both topology and regular dumps
- Added `show_filters_icon` field to LinkBase schema
- Updated API routes to handle the new field in create and update operations
**API Impact:**
- POST /v3/projects/{project_id}/links - accepts `show_filters_icon` in request body
- PUT /v3/projects/{project_id}/links/{link_id} - can update `show_filters_icon`
- GET /v3/projects/{project_id}/links/{link_id} - returns `show_filters_icon` field
**Future Applications:**
This feature provides granular control for future AI fault injection modules to manage link-level protocol failures while maintaining clean UI presentation.
Remove MD5 checksum calculation from the node files API since
disk image files are dynamic and change frequently. MD5 calculation
was also causing significant performance overhead.
Changes:
- Remove md5sum field from NodeFile schema
- Remove MD5 calculation from list_node_files method
- Improve error handling for timestamp conversion
- Simplify code by removing lambda functions
Performance improvement:
- Response time reduced from ~0.5-1s to ~0.017s (30-60x faster)
- Especially beneficial for large files and multiple files
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
