YueGuobin 5d91ca0efc
fix: harden sharkd replay sessions and serve the uncapped frame list
Review-driven session/transport fixes (each reproduced live against
sharkd 4.6.7 before fixing):

- raise the RPC stream limit to 16 MB: a full 1000-row frames page
  measures ~190 KB against the 64 KB StreamReader default, which failed
  the request with a 500 and desynchronized the resident session; a
  line-over-limit ValueError is now treated as a transport failure
- verify JSON-RPC reply ids: a timed-out request's late reply was
  served as the next request's answer; timeouts, dead pipes, malformed
  and stale replies now kill the session for good instead
- make check-spawn atomic under one manager lock: concurrent requests
  for the same pcap double-spawned sharkd and leaked the loser (process
  plus /tmp scratch copy) forever
- refcount sessions and evict idle only (LRU, cap raised 8 -> 16): a
  tag with more sources than the cap respawned every source on every
  request, and concurrent requests could get their session killed
  mid-RPC (spurious 502)
- map FilterError to sharkd's filter rejection (-13002) only; other
  engine failures with a filter set are 502, not a client 400
- detail: accept an optional frame_number to disambiguate
  same-microsecond frames (ts is not unique within a pcap); drop the
  -8003 -> 404 mapping (the range is validated locally, engine errors
  are real faults); a failed hex read is a 404 instead of "hex": null
- a pcap deleted mid-request is a 404, not a 500; the pcap-sized
  scratch copy runs off the event loop; server shutdown kills every
  resident session and drops its scratch directory
- pin the packet-list layout through scratch-HOME Wireshark
  preferences: the column indexes are a contract the server owns
  (protocol-level column negotiation is rejected by sharkd 4.6.x)

Range contract change (WebUI moved to an always-flat list): the merged
frame list is returned in full, deliberately uncapped - truncated and
per-second buckets are removed, frame_count always equals
len(frames), and rendering cost is the client's concern (the window
endpoint remains the incremental path).
2026-09-11 00:55:18 +08:00

133 lines
4.4 KiB
Python

#!/usr/bin/env python
#
# Copyright (C) 2020 GNS3 Technologies Inc.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import asyncio
from fastapi import FastAPI
from contextlib import asynccontextmanager
from gns3server.controller import Controller
from gns3server.config import Config
from gns3server.compute import MODULES
from gns3server.compute.port_manager import PortManager
from gns3server.compute.marker.marker_manager import MarkerManager
from gns3server.utils.http_client import HTTPClient
from gns3server.db.tasks import connect_to_db, get_computes, disconnect_from_db, discover_images_on_filesystem
import logging
log = logging.getLogger(__name__)
auto_discover_images_task_handle = None
@asynccontextmanager
async def lifespan(app: FastAPI):
await startup(app)
yield
await shutdown(app)
async def startup(app: FastAPI) -> None:
"""
Tasks to be performed when the server is starting.
"""
loop = asyncio.get_event_loop()
logger = logging.getLogger("asyncio")
logger.setLevel(logging.ERROR)
if log.getEffectiveLevel() == logging.DEBUG:
# On debug version we enable info that
# coroutine is not called in a way await/await
loop.set_debug(True)
# connect to the database
await connect_to_db(app)
# retrieve the computes from the database
computes = await get_computes(app)
await Controller.instance().start(computes)
# Because with a large image collection
# without md5sum already computed we start the
# computing with server start
from gns3server.compute.qemu import Qemu
if Config.instance().settings.Server.auto_discover_images is True:
# Start the discovering new images on file system 5 seconds after the server has started
# to give it a chance to process API requests
global auto_discover_images_task_handle
auto_discover_images_task_handle = asyncio.get_event_loop().call_later(
5,
lambda: asyncio.create_task(discover_images_on_filesystem(app))
)
for module in MODULES:
log.debug(f"Loading module {module.__name__}")
m = module.instance()
m.port_manager = PortManager.instance()
# Start the marker (traffic-insight) UDP sink. One listener per compute
# process receives ubridge MARK signals; ubridges are told its host/port at
# startup (see BaseNode._start_ubridge).
server_settings = Config.instance().settings.Server
await MarkerManager.instance().start(
host=server_settings.marker_listen_host, port=server_settings.marker_listen_port
)
# Mark MCP server as ready to accept connections (if MCP is available)
from gns3server.agent import MCP_AVAILABLE
if MCP_AVAILABLE:
from gns3server.agent.mcp import set_mcp_server_ready
set_mcp_server_ready(True)
log.info("GNS3 server startup completed")
async def shutdown(app: FastAPI) -> None:
"""
Tasks to be performed when the server is exiting.
"""
if auto_discover_images_task_handle is not None and not auto_discover_images_task_handle.cancelled():
auto_discover_images_task_handle.cancel()
await HTTPClient.close_session()
await MarkerManager.instance().stop()
# Kill resident sharkd sessions (marker replay) and drop their /tmp
# scratch copies before the process exits.
from gns3server.controller import marker_replay
await marker_replay.close_sessions()
await Controller.instance().stop()
for module in MODULES:
log.debug(f"Unloading module {module.__name__}")
m = module.instance()
await m.unload()
if PortManager.instance().tcp_ports:
log.warning(f"TCP ports are still used {PortManager.instance().tcp_ports}")
if PortManager.instance().udp_ports:
log.warning(f"UDP ports are still used {PortManager.instance().udp_ports}")
await disconnect_from_db(app)