create_batch_nios bound NIOs in a serial for-loop, so during project open
every builtin L2 node (ethernet_switch/hub/cloud/nat) started its uBridge
one at a time (~0.5s each for fork + AF_UNIX connect). Group entries by
node and bind in parallel with asyncio.gather — mirroring update_batch_nios
— so independent uBridge processes start concurrently. Within a node,
entries stay serial to respect the per-node uBridge command lock.
The update_batch_nios handler looped serially across all entries (5010
for a 2505-link topology). On started nodes each entry does uBridge I/O,
so the serial loop added orders of magnitude to the fan-out wall time.
Group entries by node_id before dispatching. Different nodes talk to
their own uBridge process (AF_UNIX socket) and are fully independent, so
their updates run in parallel via asyncio.gather. Per-node entries are
still serial (respecting the per-node uBridge command lock).
When a marker definition is created (or re-applied on data_link_type
change), the fan-out used to call inherit_marker -> update() on every
link, issuing one PUT /nio per link end (5000+ round-trips on a 2500-link
project). On started nodes each round-trip also reconfigured uBridge.
Two-phase fan-out:
- inherit_marker/start_marker gain memory_only: writes the marker into
link._markers and refreshes _link_data without any HTTP/emit/dump.
- _apply_def_to_all_links applies memory-only to every link, then
_batch_update_link_nios groups the updated NIO specs by compute and
sends a single PUT /projects/{id}/nios/batch per compute.
compute: new PUT /projects/{id}/nios/batch endpoint with _get_existing_nio
+ _update_nio_binding dispatch (mirrors create_batch_nios), re-applies
filters+markers to uBridge on started nodes.
Precise per-marker operations (update_marker bpf change, stop_marker on
def delete) are untouched — they deliberately avoid a full reapply to
preserve sibling marker pcaps.
Dynamips.create_nio is async def while BaseManager.create_nio is a sync
def. The previous fix only added the extra 'node' argument but did not
await the resulting coroutine, causing 'was never awaited' warnings and
passing a coroutine object instead of an NIO instance to the binding
dispatch. Add 'await' on the Dynamips branch. Test updated to verify
both the async nature and the parameter count.
The inspect check tested unbound function signatures (3 params unbound vs
2 unbound) but node.manager.create_nio is a bound method — inspect
excludes 'self'. Dynamips bound = 2 (node + nio_settings), standard
bound = 1 (nio_settings). The old '== 3' never matched, so the extra
'node' arg was never passed. Switch to '>= 2' and rewrite the test to
exercise the actual bound-method scenario.
Dynamips.create_nio requires the node as first positional argument
(unlike every other manager which takes only nio_settings). The batch
handler now detects this via parameter-count inspection (3 vs 2) and
passes node when needed.
Also add Dynamips to _add_nio_binding dispatch: routers use
slot_add_nio_binding(slot, port, nio), switches/hubs fall back to
add_nio(nio, port_number).
Add tests covering Dynamips router dispatch, switch dispatch, and the
create_nio signature detection to prevent regression.
Project open used to create each link by issuing two NIO POSTs from the
controller to the compute — ~5000 HTTP round-trips for a 2500-link
topology, all funnelling through the single shared controller/compute
event loop and capping throughput near 12 links/s.
Replace it with a bulk path:
- UDPLink split into _prepare() (local: ports, peer addrs, link_data)
and _commit_nios() (dispatch). create() = prepare + commit (interactive).
- Link.add_node gains batch=True: attach both nodes without triggering
per-link NIO HTTP.
- compute: new POST /projects/{id}/nios/batch endpoint with a unified
_add_nio_binding dispatch across node types (docker/qemu/iou/vpcs/
builtin differ in signature).
- project.open: prepare all links locally, group NIO entries by compute,
send each compute a single /nios/batch, then finalise (wire node/port
refs, mark created, notify, apply marker defs) in parallel.
Cuts controller->compute HTTP from O(links) to O(computes). Test added
for the batch endpoint.
Both is_safe_path rejection and PermissionError were returning 403
without a detail message, making them indistinguishable in logs.
Add specific detail strings to each:
- is_safe_path: 'Path is outside the project directory'
- PermissionError (write): 'Permission denied writing to ...'
- PermissionError (delete): 'Permission denied deleting ...'
The inner try-except caught OSError/UnicodeEncodeError with 'pass',
silently swallowing all write failures and returning HTTP 204 as if
the file was written successfully.
Remove the nested try-except and let errors propagate properly:
- OSError → 500 with error detail
- PermissionError → 403 (already handled)
- FileNotFoundError → 404 (already handled)
- Stream file GET/POST through controller without buffering in memory
- Add recursive and subdirectory filtering to node file listing
- Replace file extension with magic-based file type detection
- Add DELETE endpoint for node and project files
- Include directories in listing response
- Add params and stream support to http_query
- Fix lambda closures, streamer exception scope, and delete error codes
Enhance the node files API to include comprehensive file metadata:
- File size in bytes
- File creation time (ISO 8601 format)
- File modification time (ISO 8601 format)
- File extension
Create new NodeFile schema to support these additional fields
while keeping the existing ProjectFile schema for backward compatibility.
This provides users with better information to manage and identify
files in the Web UI.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implement issue #2719 - Add API endpoint to list project files
- Add GET /v3/projects/{project_id}/nodes/{node_id}/files endpoint
- Add list_node_files() method to Project class
- Add security checks to prevent path traversal
- Filter out .ghost temporary files
- Return file paths with MD5 checksums
- Require Node.Audit privilege
This allows users to discover dynamically created files
such as QEMU disk images created via the disk image API.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
