183 Commits

Author SHA1 Message Date
YueGuobin
82015dcb5c
fix(controller): stop flooding clients with transitional node statuses
Every node lifecycle call (start/stop) emitted a "starting"/"stopping"
node.updated notification on entry, and for the whole duration of the
call asdict() masked the real status with the transitional one — every
compute event forwarded in that window was re-emitted as a duplicate
transitional status.

Stop-all/start-all on a 13-node project pushed two transitional events
per node before the final state. Clients that do not render these
transitional statuses (the Web UI) showed nodes flickering through an
unexpected suspended-looking state before reaching started/stopped.

Remove the transitional notification and the status masking: clients
see the real statuses again (started/stopped), from the forwarded
compute events plus the controller-side confirmation. The wait option
on the lifecycle routes, the strict stop error handling and
wait_for_status() are unchanged.
2026-10-10 00:24:20 +08:00
Guobin Yue
7df7ce1c96
Merge branch '3.1' into fix/duplicate-device-configs-merge 2026-10-09 21:23:38 +08:00
YueGuobin
1e0d123f4b
Merge branch '3.1' into fix/duplicate-device-configs-merge
Resolve conflicts against the upstream wave (PRs #2952-#2965, incl. the
typed-response/schema hardening and the drop of Python 3.10):

- controller/node.py: keep upstream's _transitioning/_confirm_status
  lifecycle guard; the kernel-anchor link wiring (node_started) runs
  after the transition completes.
- controller/link.py: keep upstream's add_node rollback (release_ports
  on create failure) and delete via release_ports; release_ports now
  resolves ports live (same staleness reason as _bind_members) and
  discards the node link unconditionally.
- schemas/controller/links.py: extend LinkFilterType with the eight
  kernel-datapath filter types and LinkFilterParameter.type with "str"
  so the newly typed /available_filters response validates; Link.filters
  stays a free-form dict — the vocabulary is capability-gated per link
  and validated by the controller (409), not by a strict request schema.
- schemas/controller/topology.py: drop the type: ignore made redundant
  by the dict filters field.
- tests: route-level filter validation now expects the controller's 409
  (unknown types, out-of-range values); packet_loss correlation is a
  valid kernel parameter (positive test added); OpenAPI assertions match
  the free-form filters property.
2026-10-09 10:03:50 +08:00
Jeremy Grossmann
57e4d1632d
Merge branch '3.1' into api-issue-05-2xftmd 2026-10-08 19:04:36 +02:00
Jeremy Grossmann
cea858acc4
Merge branch '3.1' into api-issue-12-cu0jkp 2026-10-08 16:46:37 +02:00
Jeremy Grossmann
a9e5f859c8
Reorder import statements in test_nodes.py 2026-10-08 14:49:12 +02:00
Jeremy Grossmann
71da996362
Merge branch '3.1' into api-issue-05-2xftmd 2026-10-08 14:40:53 +02:00
Jeremy Grossmann
a32f466183
Merge branch '3.1' into api-issue-04-burx2g 2026-10-08 13:59:00 +02:00
Jeremy Grossmann
48b6257410
Merge pull request #2952 from markparonyan/api-issue-02-rp8mx3
feat(api): return node and add wait option on node lifecycle calls
2026-10-08 11:05:28 +02:00
Jeremy Grossmann
5055664456
Merge pull request #2957 from markparonyan/api-issue-08-67pip0
feat(api): add machine-readable error codes and align OpenAPI error schemas
2026-10-08 09:30:13 +02:00
Jeremy Grossmann
ed8b0c3e20
Merge pull request #2958 from markparonyan/api-issue-09-aef9as
feat(api): return token lifetimes and document API keys as bearer tokens
2026-10-08 04:35:19 +02:00
Jeremy Grossmann
dc57c95f1c
Merge pull request #2963 from markparonyan/api-issue-16-0gmylm
feat(api): add Snapshot.created and type NodeFile timestamps as date-time
2026-10-08 04:15:01 +02:00
Jeremy Grossmann
d797069921
Merge pull request #2964 from markparonyan/api-issue-17-asr039
feat: add optional ETag/If-Match concurrency for projects, nodes and links
2026-10-08 04:14:04 +02:00
Jeremy Grossmann
0a78a2bf12
Merge pull request #2961 from markparonyan/api-issue-13-bfbm9z
feat(api): add Notification schema and document notification streams
2026-10-08 04:11:14 +02:00
Jeremy Grossmann
bcf38d74f8
Merge pull request #2951 from markparonyan/api-issue-11-gqdty1
feat(api): type link filters and available_filters response
2026-10-07 18:25:30 +02:00
Guobin Yue
5238bd3b39
Merge branch '3.1' into fix/duplicate-device-configs-merge 2026-10-07 23:59:30 +08:00
Mark Paronyan
241c2b65e7
feat: add optional ETag/If-Match concurrency for projects, nodes and links 2026-10-07 16:53:11 +03:00
Mark Paronyan
11a310dd9b
feat(api): type link filters and available_filters response 2026-10-07 15:48:52 +03:00
Mark Paronyan
a9e87df644
fix(links): reserve ports on add, accept port name and auto, add port_in_use code 2026-10-07 15:41:04 +03:00
Mark Paronyan
e1a27dadcf
feat(api): add strict_names and batch node and link creation 2026-10-07 15:35:45 +03:00
Mark Paronyan
6a416d1a2a
feat(api): return node and add wait option on node lifecycle calls 2026-10-07 15:35:11 +03:00
Mark Paronyan
ead44787f3
feat(api): add Snapshot.created and type NodeFile timestamps as date-time 2026-10-07 15:25:15 +03:00
Mark Paronyan
86a279330d
feat(api): add Notification schema and document notification streams 2026-10-07 15:14:50 +03:00
Jeremy Grossmann
0836e01525
Merge pull request #2950 from markparonyan/api-issue-14-yywwso
fix(api): return 200 instead of 201 for ACL endpoints, IOU license up…
2026-10-07 13:50:53 +02:00
Mark Paronyan
a4f20e0966
feat(api): add discriminators to link iface, appliance version and template settings schemas 2026-10-07 14:22:18 +03:00
Mark Paronyan
fcfa968cc5
feat(api): return token lifetimes and document API keys as bearer tokens 2026-10-07 14:18:18 +03:00
Mark Paronyan
289f9394b6
feat(api): add machine-readable error codes and align OpenAPI error schemas 2026-10-07 14:15:41 +03:00
Mark Paronyan
97d5480241
fix(api): return 200 instead of 201 for ACL endpoints, IOU license update 2026-10-07 13:56:08 +03:00
Mark Paronyan
852dc5be0f
feat(api): add exact-match name filter to list endpoints 2026-10-07 13:40:34 +03:00
Jeremy Grossmann
49f047d07a
Merge branch '3.1' into fix/duplicate-device-configs-merge 2026-10-06 17:09:32 +02:00
Jeremy Grossmann
f184883aa7
Merge branch '3.1' into images-nested-folders-support 2026-10-06 16:50:40 +02:00
Jeremy Grossmann
0170d2c8ce
Merge branch '3.1' into fix/duplicate-device-configs-merge 2026-10-06 16:10:23 +02:00
Mark Paronyan
d9678ce98c
refactor: derive update schemas with PartialUpdateModel 2026-10-06 16:46:22 +03:00
Cristi
a7f0c7174b fix(templates): Fixed template test formatting to pass CI 2026-10-06 00:04:26 +03:00
YueGuobin
4fa7f158fd
fix(schemas): one definition of the capabilities payload
GET /computes/{id} answered with only part of the capabilities the compute
reports: the response schema kept a second, private Capabilities class of
its own (it knew ubridge_tc and nothing else), and FastAPI's response model
silently drops fields it does not define — so ubridge_tap and
ubridge_iol_tap never reached clients even though the compute had sent
them and links were already being built on those datapaths.

Import the one definition instead of duplicating it, so the next capability
cannot drift the same way.
2026-10-06 00:21:09 +08:00
YueGuobin
1db9f171ca
fix(controller): re-arm window_drop and expose kernel_datapath
Three fixes from live-calibrating the kernel impairment filters on an
isolated instance (Docker-to-Docker veth link, per-sequence evidence):

- update_filters skipped an unchanged filters dict as a no-op, so a
  one-shot window_drop outage that had already passed could never be
  re-armed: re-applying the same values produced no datapath command at
  all. A PUT still carrying window_drop now reconciles, restarting the
  schedule from the apply as documented. Measured: identical re-apply
  moves the outage window from seq 50-70 to 70-91 (arm at +8s of the
  new apply).

- gemodel parameter semantics were documented backwards in the
  validation rules, available_filters, the MCP/copilot descriptions and
  the feature docs. Measured against the kernel (per-direction counters,
  offered = sent + dropped): p is the good-to-bad transition chance, r
  the bad-to-good transition, 1-h the loss chance in the bad state (the
  good state loses nothing) - mean one-way loss p/(p+r)*(1-h), e.g.
  [10,30,100] -> 24.8%/25.7% measured vs 25% predicted, [5,45,100] ->
  10.8% both ways vs 10%.

- kernel_datapath was silently stripped from every REST/MCP link payload
  by the response model (LinkBase carried no such field), so clients
  could not tell a kernel link from a relay one.

Also verified on the same setup: the 3-argument window_drop form works
exactly as specified ([3000,5000,100] -> a single seq 31-55 outage,
[20000,6000,100] -> seq 115-144), and both generations of the command
builder emit the parameters unpadded.

Adds an API test for the field and controller tests for the no-op skip
and the window_drop re-arm exemption.
2026-10-06 00:21:08 +08:00
Cristi
207db770c6 feat(image-manager): Add support for nested image folders and appliance compatibility checks 2026-10-05 18:49:21 +03:00
Mark Paronyan
3135c8b7e3
refactor: use f-strings in remaining modules, scripts and tests 2026-10-03 19:44:02 +03:00
Mark Paronyan
306d8fbde4
refactor: remove unused imports 2026-10-03 19:43:58 +03:00
Mark Paronyan
b78672a8ca style: sort imports in the new image management modules 2026-10-02 12:43:26 +03:00
Mark Paronyan
b46b9a1d26 style: sort imports in api tests 2026-10-02 12:43:26 +03:00
Mark Paronyan
a77e06b907
Merge branch '3.1' into ruff/trivia-fixups 2026-10-01 19:58:02 +03:00
Mark Paronyan
2dc7479d11
style: remove f-strings without placeholders 2026-10-01 19:57:46 +03:00
Mark Paronyan
06e98f8b55
style: drop redundant utf-8 coding declarations 2026-10-01 19:57:40 +03:00
Cristi
42d30651da Merge origin/3.1 into image-manager-out-of-sync
Preserve image reconciliation, locking, and regression coverage while integrating upstream release, dependency, typing, and formatting updates. Align feature code with the new Ruff checks.
2026-09-28 23:16:24 +03:00
Cristi
046dff4d08 fix(images): reconcile the image inventory with filesystem changes 2026-09-28 16:01:36 +03:00
Mark Paronyan
a0450b6bce
refactor: ruff autofixes 2026-09-27 15:55:10 +03:00
Cristi
59a367b88d Allow projects containing missing images to open in a degraded state instead of failing to load 2026-09-18 15:06:17 +03:00
YueGuobin
5d91ca0efc
fix: harden sharkd replay sessions and serve the uncapped frame list
Review-driven session/transport fixes (each reproduced live against
sharkd 4.6.7 before fixing):

- raise the RPC stream limit to 16 MB: a full 1000-row frames page
  measures ~190 KB against the 64 KB StreamReader default, which failed
  the request with a 500 and desynchronized the resident session; a
  line-over-limit ValueError is now treated as a transport failure
- verify JSON-RPC reply ids: a timed-out request's late reply was
  served as the next request's answer; timeouts, dead pipes, malformed
  and stale replies now kill the session for good instead
- make check-spawn atomic under one manager lock: concurrent requests
  for the same pcap double-spawned sharkd and leaked the loser (process
  plus /tmp scratch copy) forever
- refcount sessions and evict idle only (LRU, cap raised 8 -> 16): a
  tag with more sources than the cap respawned every source on every
  request, and concurrent requests could get their session killed
  mid-RPC (spurious 502)
- map FilterError to sharkd's filter rejection (-13002) only; other
  engine failures with a filter set are 502, not a client 400
- detail: accept an optional frame_number to disambiguate
  same-microsecond frames (ts is not unique within a pcap); drop the
  -8003 -> 404 mapping (the range is validated locally, engine errors
  are real faults); a failed hex read is a 404 instead of "hex": null
- a pcap deleted mid-request is a 404, not a 500; the pcap-sized
  scratch copy runs off the event loop; server shutdown kills every
  resident session and drops its scratch directory
- pin the packet-list layout through scratch-HOME Wireshark
  preferences: the column indexes are a contract the server owns
  (protocol-level column negotiation is rejected by sharkd 4.6.x)

Range contract change (WebUI moved to an always-flat list): the merged
frame list is returned in full, deliberately uncapped - truncated and
per-second buckets are removed, frame_count always equals
len(frames), and rendering cost is the client's concern (the window
endpoint remains the incremental path).
2026-09-11 00:55:18 +08:00
YueGuobin
ef5d81498a
feat: accept link=<link_id> on the replay range and frames endpoints
Narrows the merged frame stream to one capture source BEFORE counting,
slicing and bucketing (frame_count / frames | buckets all recomputed on
the narrowed set), AND-composing with the display filter. A pure
identity filter applied before any engine work — only the selected
link's pcap gets a sharkd pass, so link+filter is cheaper than filter
alone.

Two boundaries by contract with the Web UI:
- sources[] stays the tag's stable inventory: every capture source
  listed with engine-free TOTAL counts, unaffected by link/filter — a
  source dropdown must not shrink when the view narrows (this also
  settles sources[].count on total counts rather than post-filter
  matches, which no spec ever required)
- an unknown link_id matches nothing: frame_count 0, start null, empty
  frames/buckets — the same shape as a zero-match display filter,
  deliberately not a 404; an empty link param is treated as absent
2026-09-09 00:53:06 +08:00