diff options
| author | Stefan Boberg <[email protected]> | 2026-05-05 15:47:48 +0200 |
|---|---|---|
| committer | GitHub Enterprise <[email protected]> | 2026-05-05 15:47:48 +0200 |
| commit | 01286c6233347d561064fc9e6cf9deaf2087ceb7 (patch) | |
| tree | bdbfdf01725baa2d2dd3d73727e6506b41421dff /src/zen/frontend/html/counters.js | |
| parent | hub async s3 client (#1024) (diff) | |
| download | archived-zen-main.tar.xz archived-zen-main.zip | |
Branch started as a sessions-service overhaul (persistence, client liveness, UE_LOGFMT intake) and grew to pick up adjacent infrastructure work: an early-startup log backlog, a hardened `MemoryArena`, the `zen trace serve` viewer gaining a counter view + compact timeline + tabbed callsite panel, defensive fixes in the third-party `tourist` trace parser, a series of allocation reductions across the HTTP and compact-binary hot paths, and a new `zen sessions` CLI command tree.
## Sessions service
**Persistence.** Each session lives on disk under `<DataRoot>/sessions/<id>/` as `info.cb` (metadata) plus `log.bin` (length-prefixed CbObject log records). On startup the service scans that directory and loads prior sessions as ended sessions, preloading the tail of each log so historical views work after a restart. `SessionLog` is noexcept-constructed and falls back to a disabled state on disk errors, so a bad disk can't take down `RegisterSession`. `GetSession` falls back to the ended-sessions list (fixes historical log fetches over HTTP). `LoadTail` counts only successfully-parsed records.
**Pruning.** Periodic cleanup task drops ended sessions once any of three caps is exceeded: age (default 1 year), count (default 1000), or total on-disk footprint (default 50 MiB). Runs 30 s after startup, hourly thereafter. Active sessions never pruned; disk removal and directory stat happen outside the exclusive lock so a slow filesystem can't stall lookups.
**Client liveness.** Sessions carry a `ProcessHandle` for the client-reported pid, captured at registration time so Windows pid recycling can't produce false positives. A 30 s asio timer probes liveness and ends dead sessions through the normal remove path, producing a synthetic `Session ended: process exited (...)` line persisted to `log.bin`. Windows decodes common NTSTATUS exit codes to human names (Ctrl-C, access violation, stack overflow, ...); POSIX stays at plain `process exited`. Clients auto-fill `ClientPid` only for local targets (unix socket / loopback); the server defensively accepts pids only from `IsLocalMachineRequest()` peers. zenserver also reports its own pid when registering its self-session, so it shows up with a real pid in the dashboard and `zen sessions ls`.
**Synthetic end-of-session line.** `RemoveSession` takes an optional reason; before the session moves to the ended list it appends an Info-level `Session ended[: reason]` entry through the normal log path (released outside `m_Lock`). Current reasons: `client request` (HTTP DELETE), `server shutdown` (self-session), `process exited (...)` (liveness).
**UE_LOGFMT structured entries.** `POST /sessions/{id}/log` now accepts `{level, logger, format, fields}` alongside the existing `{level, logger, message}` shape. New `logtemplate.{h,cpp}` implements UE's `StructuredLog.cpp` template grammar (field paths with `.name` / `[N]`, `{{`/`}}` escapes, `$text` / `$format` / `$locformat` object conventions, bounded recursion). Renders to a displayable message at intake while persisting raw format + fields so a future UI can drill into fields without another schema bump. Hot path is zero-alloc — renders into `ExtendableStringBuilder<256>` using stack-buffered `Oid::ToString` / `IoHash::ToHexString` overloads. UI shows a `{…}` marker with the raw template + JSON-pretty fields on hover.
**Parent sessions.** `SessionInfo` gains `parent_session_id`; hub-managed storage server child processes inherit the hub's session id via `--parent-session=<id>`. `ZEN_SESSIONS_URL` env var becomes a fallback for `--sessions-url` / config when neither is provided. The in-process session log sink is disabled when a remote sessions target is configured (logs flow through `SessionsServiceClient` instead). The sessions UI groups child sessions under their parent (collapsible/expandable, sorts as a unit, supports nesting).
**Platform reporting.** `SessionInfo` gains `Platform`, flowed end-to-end: client auto-fills via `GetRuntimePlatformName()`, server persists in `info.cb` (`plat`) and emits on GET. UI renders as a SimpleIcons-style inline SVG (windows / macOS / iOS / linux / wine / android / playstation / xbox / nintendo) with case-insensitive alias resolution (Win32/Win64, PS4/PS5, XSX/XSS, NintendoSwitch, iPhone/iPad, Darwin/OSX). Unknown values fall back to text; sorting runs on the underlying string.
**WebSocket log streaming.** Sessions UI moves from 2 s polling to a WebSocket push model. New `WsSubscriber` has a stable id + helper methods. UI caps the log-line DOM at 5 000 entries with a shared cursor-regression helper, factored out of two call sites. Per-broadcast allocations trimmed on the push path; fixed a stack overrun in the WS log broadcast hex-id buffer.
**Log memory.** `LogEntry::Level` is now `logging::LogLevel` (1 byte) instead of `std::string` (~32 B) — saves ~310 KB per full 10 k-entry deque and eliminates a per-message allocation in the in-proc sink. On-disk format writes an int32 and accepts either int or legacy string on read. `LogEntry` strings now live in a `MemoryArena`; logger names are interned across the deque. `SessionLog::Append` and `WriteSessionInfoFile` drop their `UniqueBuffer` round-trip and write `CbObject::GetView()` straight through `BasicFile` / `SafeWriteFile`. Multi-entry `POST /log` batched under one lock + one push.
**In-proc log timestamps.** `InProcSessionLogSink::TimePointToDateTime` previously preserved only whole seconds, so every in-proc entry rendered at `.000` ms in the dashboard and `zen sessions tail`. It now adds the sub-second part (nanoseconds → 100 ns ticks) to keep ms precision end-to-end.
**UI.** Side "Session Details" panel is gone — its info is inline in the table (appname, mode, platform, id, timestamps, this/log pills, active dot). Bottom panel is a tabbed `Log | Metadata` view with a right-side "Session Information" panel beside metadata; log-only controls (filter, newest-first, follow, log-level filter, expand/collapse) hide when Metadata is active, polling keeps running across tab switches. Wide-mode toggle fills the viewport edge-to-edge. Log lines show the logger category; timestamps render in 24 h with zero-padded fields regardless of locale. Sessions list defaults to All / 10 per page / created-desc, gains click-to-sort headers on the full dataset, a header filter box, and a pager aligned to the table's right edge. Duplicate auto-injected `<h1>Sessions</h1>` removed.
## `zen sessions` CLI
New command tree on the `zen` client for inspecting the sessions service from the terminal:
- **`zen sessions ls`** — lists sessions (active first, ended next; newest-first within each group) with id, status, app/mode, pid, created, duration, and log count. Supports `--status active|ended|all` (default `all`).
- **`zen sessions status`** — prints the sessions service summary: self id, active / ended counts, and the read/write/delete/list/request/bad-request counters from `/stats/sessions`.
- **`zen sessions tail [session]`** — tails a session's log. With no argument it tails zenserver's own session (resolved via `/sessions/list`'s `self_id`); an explicit 24-hex id targets any session, including ended ones (historical replay). `--lines N` (default 50, 0 = all buffered) trims the initial dump client-side. `--follow` prefers a WebSocket push subscription on `/sessions/ws` for sub-second latency; on upgrade failure (older server, blocked port, unix-socket transport) it falls back to HTTP cursor polling at `--interval-ms` (default 500), with sleeps chunked to 50 ms so Ctrl-C reacts quickly. Output matches `zen::logging::FullFormatter` (`[YY-MM-DD HH:MM:SS.mmm] [lvl] [logger] message`); on a TTY the level is colored and the logger is bold, with continuation lines indented under the message column using the *visible* prefix width. 404 surfaces as `(session ended)` and connection errors as `(server gone)` — both clean exits, so stopping the server mid-tail no longer prints a stack trace.
- **`zen sessions ui`** — opens `<host>/dashboard/?page=sessions` in the user's default browser. Rejects unix-socket hosts.
A small `ZenServiceClient::IsUnixSocket()` helper now wraps the unix-socket check used by `ui`, `sessions tail` (WS path), and `sessions ui`.
## Logging
`BacklogSink` captures early-startup log entries in a fixed-capacity ring so late-attached sinks (session sink, file sink) can replay them. Detaches from the broadcast list when disabled; backed by destructor-only cleanup (no `unique_ptr` indirection per entry). Tuned defaults so the backlog covers typical bring-up without unbounded growth.
## `zen trace serve` viewer
- Compact timeline mode for high-density views.
- New `TRACE_INT_VALUE` / `TRACE_FLOAT_VALUE` counter trace points + a counters page in the viewer.
- Callsite tables collapsed into a single tabbed panel.
- Lossless `Oid <-> Guid` bridge for trace session ids; trace `SessionId` plumbed through.
- `tourist` parser hardening: bounds-check `BufferStream::read`, validate `Type::info_size` before `patch()`, convert `parse_important_aux` to a loop (avoids deep recursion), widen `ParserPool` index to `uint32`, bounds-check field offsets in the dispatcher, pin `Types::parse` buffer up-front.
## `MemoryArena`
Configurable chunk size, inline chunk list, oversize requests routed to truly-dedicated chunks (no slack waste, no fragmentation when one allocation is much larger than the chunk).
## Allocation cleanups across hot paths
- `zenhttp::HttpRequestRouter::HandleRequest` and `FormatPackageMessageInternal`: drop heap allocations.
- Compact-binary validation: `eastl::fixed_vector` + `eastl::sort`; eliminate `std::vector` churn.
- `zenserverprocess`: trim transient allocations in spawn paths.
- Sessions HTTP intake / broadcast: drop transient `std::string` allocs.
Diffstat (limited to 'src/zen/frontend/html/counters.js')
| -rw-r--r-- | src/zen/frontend/html/counters.js | 404 |
1 files changed, 404 insertions, 0 deletions
diff --git a/src/zen/frontend/html/counters.js b/src/zen/frontend/html/counters.js new file mode 100644 index 000000000..fe3e4d338 --- /dev/null +++ b/src/zen/frontend/html/counters.js @@ -0,0 +1,404 @@ +// Copyright Epic Games, Inc. All Rights Reserved. +// Counters viewer — list registered TRACE_INT_VALUE / TRACE_FLOAT_VALUE +// counters and chart selected ones. +// +// Mirrors csvstats.js layout (tree on the left, line chart on the right), +// adapted to the simpler counter-id keyed series model. + +import { getCounters, getCounterSeries } from "./api.js"; +import { escapeHtml } from "./util.js"; + +function formatTime(us) { + if (us < 1000) return `${us} µs`; + if (us < 1_000_000) return `${(us / 1000).toFixed(2)} ms`; + return `${(us / 1_000_000).toFixed(2)} s`; +} + +const BYTE_UNITS = ["B", "KiB", "MiB", "GiB", "TiB"]; +function formatBytes(value) { + let v = value; + let i = 0; + while (Math.abs(v) >= 1024 && i + 1 < BYTE_UNITS.length) { + v /= 1024; + ++i; + } + return `${v.toFixed(i === 0 ? 0 : 2)} ${BYTE_UNITS[i]}`; +} + +function formatCounterValue(value, def) { + if (def && def.display_hint === 1) { + return formatBytes(value); + } + if (def && def.type === 0) { + // Integer — render without fractional digits. + return Number(value).toLocaleString(); + } + return value.toFixed(3); +} + +const LINE_COLORS = [ + "#4fc3f7", "#81c784", "#ffb74d", "#e57373", "#ba68c8", + "#4db6ac", "#fff176", "#f06292", "#7986cb", "#a1887f", +]; + +export class CountersView { + constructor(model, containerEl) { + this.model = model; + this.container = containerEl; + this.loaded = false; + this.defs = []; + this.defById = new Map(); + + this.selected = new Set(); + this.seriesData = new Map(); + this.colorIndex = 0; + this.colorById = new Map(); + + this.viewStartUs = 0; + this.viewEndUs = (model.session && model.session.trace_end_us) || 1; + + this.buildLayout(); + } + + buildLayout() { + this.container.innerHTML = + `<div class="csv-layout">` + + `<div class="csv-tree-panel">` + + `<div class="sidebar-label">Counters</div>` + + `<div class="csv-tree counters-tree"></div>` + + `</div>` + + `<div class="csv-chart-panel">` + + `<canvas class="csv-chart-canvas"></canvas>` + + `<div class="csv-chart-tooltip" hidden></div>` + + `</div>` + + `</div>`; + + this.treeEl = this.container.querySelector(".counters-tree"); + this.canvas = this.container.querySelector(".csv-chart-canvas"); + this.tooltipEl = this.container.querySelector(".csv-chart-tooltip"); + this.ctx = this.canvas.getContext("2d"); + this.dpr = Math.max(1, window.devicePixelRatio || 1); + + this.resizeObserver = new ResizeObserver(() => this.drawChart()); + this.resizeObserver.observe(this.canvas); + this.canvas.addEventListener("mousemove", (e) => this.onChartHover(e)); + this.canvas.addEventListener("mouseleave", () => { this.tooltipEl.hidden = true; }); + + this.panning = false; + this.canvas.addEventListener("mousedown", (e) => this.onPanStart(e)); + window.addEventListener("mousemove", (e) => this.onPanMove(e)); + window.addEventListener("mouseup", () => this.onPanEnd()); + this.canvas.addEventListener("wheel", (e) => this.onWheel(e), { passive: false }); + } + + async ensureLoaded() { + if (this.loaded) { + this.drawChart(); + return; + } + try { + this.defs = await getCounters(); + for (const d of this.defs) { + this.defById.set(d.id, d); + } + this.renderTree(); + } catch (e) { + this.treeEl.innerHTML = `<div class="csv-empty">Failed to load counters: ${escapeHtml(e.message)}</div>`; + console.error(e); + } + this.loaded = true; + this.drawChart(); + } + + renderTree() { + // Group by leading path component (everything before the first "/"). + const groups = new Map(); + for (const d of this.defs) { + if (!d.sample_count) continue; + const slash = d.name.indexOf("/"); + const group = slash > 0 ? d.name.substring(0, slash) : ""; + if (!groups.has(group)) groups.set(group, []); + groups.get(group).push(d); + } + + if (groups.size === 0) { + this.treeEl.innerHTML = `<div class="csv-empty">No counters in this trace.</div>`; + return; + } + + const groupNames = Array.from(groups.keys()).sort((a, b) => a.localeCompare(b)); + const parts = []; + for (const g of groupNames) { + const list = groups.get(g); + parts.push(`<div class="csv-cat-header">${escapeHtml(g || "(ungrouped)")}</div>`); + for (const d of list) { + parts.push( + `<label class="csv-stat-row">` + + `<input type="checkbox" data-counter-id="${d.id}">` + + `<span class="csv-stat-name"></span>` + + `<span class="thread-count">${Number(d.sample_count).toLocaleString()}</span>` + + `</label>` + ); + } + } + this.treeEl.innerHTML = parts.join(""); + + // Set names via DOM (XSS safe). + const rows = this.treeEl.querySelectorAll(".csv-stat-row"); + let idx = 0; + for (const g of groupNames) { + for (const d of groups.get(g)) { + if (idx < rows.length) { + const slash = d.name.indexOf("/"); + const display = slash > 0 ? d.name.substring(slash + 1) : d.name; + rows[idx].querySelector(".csv-stat-name").textContent = display; + rows[idx].title = d.name; + } + ++idx; + } + } + + // Wire checkboxes. + for (const cb of this.treeEl.querySelectorAll("input[type=checkbox]")) { + cb.addEventListener("change", () => { + const id = Number(cb.dataset.counterId); + if (cb.checked) { + this.selected.add(id); + if (!this.colorById.has(id)) { + this.colorById.set(id, LINE_COLORS[this.colorIndex++ % LINE_COLORS.length]); + } + this.fetchSeries(id); + } else { + this.selected.delete(id); + this.drawChart(); + } + }); + } + } + + async fetchSeries(id) { + if (this.seriesData.has(id)) { + this.drawChart(); + return; + } + try { + const result = await getCounterSeries(id); + const samples = (result.samples || []).map(([t, v]) => ({ timeUs: t, value: v })); + this.seriesData.set(id, samples); + this.drawChart(); + } catch (e) { + console.error(`Failed to fetch counter series for ${id}: ${e.message}`); + } + } + + resizeCanvas() { + const rect = this.canvas.getBoundingClientRect(); + this.width = Math.floor(rect.width); + this.height = Math.floor(rect.height); + const bw = Math.floor(rect.width * this.dpr); + const bh = Math.floor(rect.height * this.dpr); + if (this.canvas.width !== bw || this.canvas.height !== bh) { + this.canvas.width = bw; + this.canvas.height = bh; + } + this.ctx.setTransform(this.dpr, 0, 0, this.dpr, 0, 0); + } + + drawChart() { + this.resizeCanvas(); + const ctx = this.ctx; + const W = this.width; + const H = this.height; + + const bg = getComputedStyle(document.body).getPropertyValue("--bg0") || "#0d1117"; + const fg2 = getComputedStyle(document.body).getPropertyValue("--fg2") || "#8b949e"; + const border = getComputedStyle(document.body).getPropertyValue("--border") || "#30363d"; + ctx.fillStyle = bg; + ctx.fillRect(0, 0, W, H); + + if (this.selected.size === 0) { + ctx.fillStyle = fg2; + ctx.font = "12px -apple-system, Segoe UI, sans-serif"; + ctx.textAlign = "center"; + ctx.textBaseline = "middle"; + ctx.fillText("Select counters from the tree to chart them", W / 2, H / 2); + return; + } + + const PAD_L = 70, PAD_R = 12, PAD_T = 12, PAD_B = 28; + const chartW = W - PAD_L - PAD_R; + const chartH = H - PAD_T - PAD_B; + if (chartW <= 0 || chartH <= 0) return; + + const startUs = this.viewStartUs; + const endUs = this.viewEndUs; + const rangeUs = Math.max(1, endUs - startUs); + + let minVal = Infinity, maxVal = -Infinity; + for (const id of this.selected) { + const samples = this.seriesData.get(id); + if (!samples) continue; + for (const s of samples) { + if (s.timeUs < startUs || s.timeUs > endUs) continue; + if (s.value < minVal) minVal = s.value; + if (s.value > maxVal) maxVal = s.value; + } + } + if (!isFinite(minVal)) { minVal = 0; maxVal = 1; } + if (minVal === maxVal) { minVal -= 0.5; maxVal += 0.5; } + const valRange = maxVal - minVal; + const valPad = valRange * 0.05; + minVal -= valPad; + maxVal += valPad; + + const xAt = (us) => PAD_L + (us - startUs) / rangeUs * chartW; + const yAt = (v) => PAD_T + (1 - (v - minVal) / (maxVal - minVal)) * chartH; + + ctx.strokeStyle = border; + ctx.lineWidth = 0.5; + for (let i = 0; i <= 4; i++) { + const y = PAD_T + chartH * i / 4; + ctx.beginPath(); ctx.moveTo(PAD_L, y); ctx.lineTo(PAD_L + chartW, y); ctx.stroke(); + } + + // Format Y-axis labels using the display hint of the first selected counter. + const firstSelected = this.selected.values().next().value; + const firstDef = this.defById.get(firstSelected); + + ctx.fillStyle = fg2; + ctx.font = "10px -apple-system, Segoe UI, sans-serif"; + ctx.textAlign = "right"; + ctx.textBaseline = "middle"; + for (let i = 0; i <= 4; i++) { + const v = minVal + (maxVal - minVal) * (1 - i / 4); + const y = PAD_T + chartH * i / 4; + ctx.fillText(formatCounterValue(v, firstDef), PAD_L - 4, y); + } + + ctx.textAlign = "center"; + ctx.textBaseline = "top"; + const tickCount = Math.max(2, Math.min(8, Math.floor(chartW / 80))); + for (let i = 0; i <= tickCount; i++) { + const us = startUs + rangeUs * i / tickCount; + const x = xAt(us); + ctx.fillText(formatTime(us), x, PAD_T + chartH + 4); + } + + // Draw step-style lines (counters change at discrete events). + for (const id of this.selected) { + const samples = this.seriesData.get(id); + if (!samples || samples.length === 0) continue; + const color = this.colorById.get(id) || "#fff"; + + ctx.strokeStyle = color; + ctx.lineWidth = 1.5; + ctx.beginPath(); + let started = false; + let prevY = 0; + for (const s of samples) { + if (s.timeUs < startUs || s.timeUs > endUs) continue; + const x = xAt(s.timeUs); + const y = yAt(s.value); + if (!started) { ctx.moveTo(x, y); started = true; } + else { ctx.lineTo(x, prevY); ctx.lineTo(x, y); } + prevY = y; + } + ctx.stroke(); + } + + ctx.strokeStyle = border; + ctx.lineWidth = 1; + ctx.strokeRect(PAD_L, PAD_T, chartW, chartH); + + ctx.font = "10px -apple-system, Segoe UI, sans-serif"; + ctx.textAlign = "left"; + ctx.textBaseline = "top"; + let legendX = PAD_L + 6; + for (const id of this.selected) { + const def = this.defById.get(id); + const name = def ? def.name : `counter ${id}`; + const color = this.colorById.get(id) || "#fff"; + ctx.fillStyle = color; + ctx.fillRect(legendX, PAD_T + 4, 10, 10); + ctx.fillStyle = "#ccc"; + ctx.fillText(name, legendX + 14, PAD_T + 4); + legendX += ctx.measureText(name).width + 24; + } + + this._chartLayout = { PAD_L, PAD_R, PAD_T, PAD_B, chartW, chartH, startUs, endUs, rangeUs, minVal, maxVal, xAt, yAt, firstDef }; + } + + onChartHover(e) { + if (!this._chartLayout || this.selected.size === 0) { + this.tooltipEl.hidden = true; + return; + } + const rect = this.canvas.getBoundingClientRect(); + const mx = e.clientX - rect.left; + const my = e.clientY - rect.top; + const { PAD_L, PAD_T, chartW, chartH, startUs, rangeUs } = this._chartLayout; + + if (mx < PAD_L || mx > PAD_L + chartW || my < PAD_T || my > PAD_T + chartH) { + this.tooltipEl.hidden = true; + return; + } + + const cursorUs = startUs + (mx - PAD_L) / chartW * rangeUs; + const lines = []; + for (const id of this.selected) { + const samples = this.seriesData.get(id); + if (!samples || samples.length === 0) continue; + let best = null, bestDist = Infinity; + for (const s of samples) { + const d = Math.abs(s.timeUs - cursorUs); + if (d < bestDist) { bestDist = d; best = s; } + } + if (best) { + const def = this.defById.get(id); + const name = def ? def.name : `counter ${id}`; + const color = this.colorById.get(id) || "#fff"; + lines.push(`<span style="color:${color}">${escapeHtml(name)}</span>: ${formatCounterValue(best.value, def)}`); + } + } + if (lines.length === 0) { this.tooltipEl.hidden = true; return; } + this.tooltipEl.innerHTML = `<div style="margin-bottom:2px">${formatTime(cursorUs)}</div>` + lines.join("<br>"); + this.tooltipEl.style.left = `${mx + 12}px`; + this.tooltipEl.style.top = `${my + 12}px`; + this.tooltipEl.hidden = false; + } + + onPanStart(e) { + if (e.button !== 0) return; + this.panning = true; + this.panStartX = e.clientX; + this.panStartViewStart = this.viewStartUs; + this.panStartViewEnd = this.viewEndUs; + } + + onPanMove(e) { + if (!this.panning || !this._chartLayout) return; + const dx = e.clientX - this.panStartX; + const usPerPx = (this.panStartViewEnd - this.panStartViewStart) / this._chartLayout.chartW; + const shift = -dx * usPerPx; + this.viewStartUs = this.panStartViewStart + shift; + this.viewEndUs = this.panStartViewEnd + shift; + this.drawChart(); + } + + onPanEnd() { this.panning = false; } + + onWheel(e) { + e.preventDefault(); + if (!this._chartLayout) return; + const rect = this.canvas.getBoundingClientRect(); + const mx = e.clientX - rect.left; + const { PAD_L, chartW, startUs, rangeUs } = this._chartLayout; + const cursorUs = startUs + (mx - PAD_L) / chartW * rangeUs; + const factor = e.deltaY > 0 ? 1.25 : 0.8; + const newRange = Math.max(10, (this.viewEndUs - this.viewStartUs) * factor); + const ratio = (mx - PAD_L) / chartW; + this.viewStartUs = cursorUs - ratio * newRange; + this.viewEndUs = this.viewStartUs + newRange; + this.drawChart(); + } +} |