diff options
| author | Stefan Boberg <[email protected]> | 2026-03-23 14:19:57 +0100 |
|---|---|---|
| committer | GitHub Enterprise <[email protected]> | 2026-03-23 14:19:57 +0100 |
| commit | 2a445406e09328cb4cf320300f2678997d6775b7 (patch) | |
| tree | a92f02d94c92144cb6ae32160397298533e4c822 /src/zenserver/sessions/sessions.cpp | |
| parent | add hub instance crash recovery (#885) (diff) | |
| download | zen-2a445406e09328cb4cf320300f2678997d6775b7.tar.xz zen-2a445406e09328cb4cf320300f2678997d6775b7.zip | |
Dashboard refresh (logs, storage, network, object store, docs) (#835)
## Summary
This PR adds a session management service, several new dashboard pages, and a number of infrastructure improvements.
### Sessions Service
- `SessionsServiceClient` in `zenutil` announces sessions to a remote zenserver with a 15s heartbeat (POST/PUT/DELETE lifecycle)
- Storage server registers itself with its own local sessions service on startup
- Session mode attribute coupled to server mode (Compute, Proxy, Hub, etc.)
- Ended sessions tracked with `ended_at` timestamp; status filtering (Active/Ended/All)
- `--sessions-url` config option for remote session announcement
- In-process log sink (`InProcSessionLogSink`) forwards server log output to the server's own session, visible in the dashboard
### Session Log Viewer
- POST/GET endpoints for session logs (`/sessions/{id}/log`) supporting raw text and structured JSON/CbObject with batch `entries` array
- In-memory log storage per session (capped at 10k entries) with cursor-based pagination for efficient incremental fetching
- Log panel in the sessions dashboard with incremental DOM updates, auto-scroll (Follow toggle), newest-first toggle, text filter, and log-level coloring
- Auto-selects the server's own session on page load
### TCP Log Streaming
- `LogStreamListener` and `TcpLogStreamSink` for log delivery over TCP
- Sequence numbers on each message with drop detection and synthetic "dropped" notice on gaps
- Gathered buffer writes to reduce syscall overhead when flushing batches
- Tests covering basic delivery, multi-line splitting, drop detection, and sequencing
### New Dashboard Pages
- **Sessions**: master-detail layout with selectable rows, metadata panel, live WebSocket updates, paging, abbreviated date formatting, and "this" pill for the local session
- **Object Store**: summary stats tiles and bucket table with click-to-expand inline object listing (`GET /obj/`)
- **Storage**: per-volume disk usage breakdown (`GET /admin/storage`), Garbage Collection status section (next-run countdown, last-run stats), and GC History table with paginated rows and expandable detail panels
- **Network**: overview tiles, per-service request table, proxy connections, and live WebSocket updates; distinct client IPs and session counts via HyperLogLog
### Documentation Page
- In-dashboard Docs page with sidebar navigation, markdown rendering (via `marked`), Mermaid diagram support (theme-aware), collapsible sections, text filtering with highlighting, and cross-document linking
- New user-facing docs: `overview.md` (with architecture and per-mode diagrams), `sessions.md`, `cache.md`, `projects.md`; updated `compute.md`
- Dev docs moved to `docs/dev/`
### Infrastructure & Bug Fixes
- **Deflate compression** for the embedded frontend zip (~3.4MB → ~950KB); zlib inflate support added to `ZipFs` with cached decompressed buffers
- **Local IP addresses**: `GetLocalIpAddresses()` (Windows via `GetAdaptersAddresses`, Linux/Mac via `getifaddrs`); surfaced in `/status/status`, `/health/info`, and the dashboard banner
- **Dashboard nav**: unified into `zen-nav` web component with `MutationObserver` for dynamically added links, CSS `::part()` to merge banner/nav border radii, and prefix-based active link detection
- Stats broadcast refactored from manual JSON string concatenation to `CbObjectWriter`; `CbObject`-to-JS conversion improved for `TimeSpan`, `DateTime`, and large integers
- Stats WebSocket boilerplate consolidated into `ZenPage.connect_stats_ws()`
Diffstat (limited to 'src/zenserver/sessions/sessions.cpp')
| -rw-r--r-- | src/zenserver/sessions/sessions.cpp | 141 |
1 files changed, 119 insertions, 22 deletions
diff --git a/src/zenserver/sessions/sessions.cpp b/src/zenserver/sessions/sessions.cpp index d919db6e9..1212ba5d8 100644 --- a/src/zenserver/sessions/sessions.cpp +++ b/src/zenserver/sessions/sessions.cpp @@ -46,6 +46,78 @@ SessionsService::Session::Session(const SessionInfo& Info) : m_Info(Info) } SessionsService::Session::~Session() = default; +void +SessionsService::Session::AppendLog(LogEntry Entry) +{ + RwLock::ExclusiveLockScope Lock(m_LogLock); + m_LogEntries.push_back(std::move(Entry)); + ++m_TotalAppended; + while (m_LogEntries.size() > MaxLogEntries) + { + m_LogEntries.pop_front(); + } +} + +std::vector<SessionsService::LogEntry> +SessionsService::Session::GetLogEntries(uint32_t Limit, uint32_t Offset) const +{ + RwLock::SharedLockScope Lock(m_LogLock); + + const uint32_t Total = uint32_t(m_LogEntries.size()); + if (Offset >= Total) + { + return {}; + } + + const uint32_t Available = Total - Offset; + const uint32_t Count = (Limit > 0) ? std::min(Limit, Available) : Available; + + std::vector<LogEntry> Result; + Result.reserve(Count); + for (uint32_t i = Offset; i < Offset + Count; i++) + { + Result.push_back(m_LogEntries[i]); + } + return Result; +} + +uint64_t +SessionsService::Session::GetLogCount() const +{ + RwLock::SharedLockScope Lock(m_LogLock); + return m_LogEntries.size(); +} + +SessionsService::Session::CursorResult +SessionsService::Session::GetLogEntriesAfter(uint64_t AfterCursor) const +{ + RwLock::SharedLockScope Lock(m_LogLock); + + const uint64_t DequeSize = m_LogEntries.size(); + + // Cursor 0 means "give me everything currently in the deque". + // Otherwise, compute how many new entries were appended since the cursor. + uint64_t NewCount = (AfterCursor == 0) ? DequeSize : (m_TotalAppended > AfterCursor ? m_TotalAppended - AfterCursor : 0); + + // Clamp to what's actually available in the deque (entries may have been evicted). + NewCount = std::min(NewCount, DequeSize); + + std::vector<LogEntry> Result; + Result.reserve(NewCount); + + const uint64_t StartIndex = DequeSize - NewCount; + for (uint64_t i = StartIndex; i < DequeSize; i++) + { + Result.push_back(m_LogEntries[i]); + } + + return CursorResult{ + .Entries = std::move(Result), + .Cursor = m_TotalAppended, + .Count = DequeSize, + }; +} + ////////////////////////////////////////////////////////////////////////// SessionsService::SessionsService() : m_Log(logging::Get("sessions")) @@ -55,25 +127,31 @@ SessionsService::SessionsService() : m_Log(logging::Get("sessions")) SessionsService::~SessionsService() = default; bool -SessionsService::RegisterSession(const Oid& SessionId, std::string AppName, const Oid& JobId, CbObjectView Metadata) +SessionsService::RegisterSession(const Oid& SessionId, std::string AppName, std::string Mode, const Oid& JobId, CbObjectView Metadata) { - RwLock::ExclusiveLockScope Lock(m_Lock); - - if (m_Sessions.contains(SessionId)) + // Log outside the lock scope — InProcSessionLogSink calls back into + // GetSession() which acquires m_Lock shared, so logging while holding + // m_Lock exclusively would deadlock. { - return false; + RwLock::ExclusiveLockScope Lock(m_Lock); + + if (m_Sessions.contains(SessionId)) + { + return false; + } + + const DateTime Now = DateTime::Now(); + m_Sessions.emplace(SessionId, + Ref(new Session(SessionInfo{.Id = SessionId, + .AppName = AppName, + .Mode = Mode, + .JobId = JobId, + .Metadata = CbObject::Clone(Metadata), + .CreatedAt = Now, + .UpdatedAt = Now}))); } - ZEN_INFO("Session {} registered (AppName: {}, JobId: {})", SessionId, AppName, JobId); - - const DateTime Now = DateTime::Now(); - m_Sessions.emplace(SessionId, - Ref(new Session(SessionInfo{.Id = SessionId, - .AppName = std::move(AppName), - .JobId = JobId, - .Metadata = CbObject::Clone(Metadata), - .CreatedAt = Now, - .UpdatedAt = Now}))); + ZEN_INFO("Session {} registered (AppName: {}, Mode: {}, JobId: {})", SessionId, AppName, Mode, JobId); return true; } @@ -126,20 +204,39 @@ SessionsService::GetSessions() const bool SessionsService::RemoveSession(const Oid& SessionId) { - RwLock::ExclusiveLockScope Lock(m_Lock); + std::string RemovedAppName; + Oid RemovedJobId; - auto It = m_Sessions.find(SessionId); - if (It == m_Sessions.end()) { - return false; - } + RwLock::ExclusiveLockScope Lock(m_Lock); + + auto It = m_Sessions.find(SessionId); + if (It == m_Sessions.end()) + { + return false; + } + + RemovedAppName = It.value()->Info().AppName; + RemovedJobId = It.value()->Info().JobId; + + Ref<Session> Ended = It.value(); + Ended->SetEndedAt(DateTime::Now()); + m_EndedSessions.push_back(std::move(Ended)); - ZEN_INFO("Session {} removed (AppName: {}, JobId: {})", SessionId, It.value()->Info().AppName, It.value()->Info().JobId); + m_Sessions.erase(It); + } - m_Sessions.erase(It); + ZEN_INFO("Session {} removed (AppName: {}, JobId: {})", SessionId, RemovedAppName, RemovedJobId); return true; } +std::vector<Ref<SessionsService::Session>> +SessionsService::GetEndedSessions() const +{ + RwLock::SharedLockScope Lock(m_Lock); + return m_EndedSessions; +} + uint64_t SessionsService::GetSessionCount() const { |