feat: add auto_connect flag and disconnect endpoint for computes

Computes can now be flagged auto_connect in the database. The controller
connects to flagged computes in the background at startup (reusing the
one-second delayed fire-and-forget mechanism), and enabling the flag via
PUT /computes/{id} triggers a background connection without the request
ever waiting for it. Disabling the flag does not drop an existing
connection.

Add POST /computes/{id}/disconnect to tear a compute down manually: the
notification stream and HTTP session are closed and every reconnection
path (retry backoff, notification stream reconnect, lazy connection in
http_query) is blocked until connect() is called again or the controller
restarts with auto_connect enabled. Disconnecting a compute used by an
opened project is rejected with 409 unless force=true; forcing only
drops the controller-side control channel, nodes keep running unmanaged.
The local compute cannot be disconnected.

Also make test_compute_list create its own compute (it relied on rows
from earlier tests and failed under reordering), and stop scheduling the
delayed add_compute connection during tests: the timer leaked real
connection attempts into other tests' event loops.
This commit is contained in:
YueGuobin committed 2026-10-15 22:50:22 +08:00
1 parent 716edc3a6a
commit 6734d2d666
10 files changed
+433 -14

No files matched your search

+22 -9
View File
@@ -201,10 +201,12 @@ class Controller:
if computes:
for c in computes:
try:
# FIXME: Task exception was never retrieved
await self.add_compute(
compute_id=str(c.compute_id),
connect=False,
# computes flagged auto_connect connect in the
# background, one second after the controller started
connect=c.auto_connect,
wait_connection=False,
**c.dict(exclude_unset=True, exclude={"compute_id", "created_at", "updated_at"}),
ssl_context=self._ssl_context,
)
@@ -625,9 +627,12 @@ class Controller:
if connect:
if wait_connection:
await compute.connect()
else:
# call compute.connect() later to give time to the controller to be fully started
asyncio.get_event_loop().call_later(1, lambda: asyncio.ensure_future(compute.connect()))
elif not hasattr(sys, "_called_from_test"):
# connect in the background to give time to the controller to be fully
# started; goes through _try_reconnect so connection errors don't die
# as unretrieved task exceptions. Skipped during tests: the timer would
# leak the connection attempt into another test's event loop.
asyncio.get_event_loop().call_later(1, lambda: asyncio.ensure_future(compute._try_reconnect()))
self.notification.controller_emit("compute.created", compute.asdict())
return compute
else:
@@ -645,6 +650,17 @@ class Controller:
if compute in project.computes:
await project.close()
def get_compute_open_projects(self, compute):
"""
Return the opened projects using a compute.
:returns: List of opened projects
"""
return [
project for project in self._projects.values() if compute in project.computes and project.status == "opened"
]
def compute_has_open_project(self, compute):
"""
Check is compute has a project opened.
@@ -652,10 +668,7 @@ class Controller:
:returns: True if a project is open
"""
for project in self._projects.values():
if compute in project.computes and project.status == "opened":
return True
return False
return bool(self.get_compute_open_projects(compute))
async def delete_compute(self, compute_id):
"""