aboutsummaryrefslogtreecommitdiff
path: root/src/zen/trace
diff options
context:
space:
mode:
authorStefan Boberg <[email protected]>2026-05-05 15:47:48 +0200
committerGitHub Enterprise <[email protected]>2026-05-05 15:47:48 +0200
commit01286c6233347d561064fc9e6cf9deaf2087ceb7 (patch)
treebdbfdf01725baa2d2dd3d73727e6506b41421dff /src/zen/trace
parenthub async s3 client (#1024) (diff)
downloadarchived-zen-main.tar.xz
archived-zen-main.zip
sessions: persist to disk, prune, track client liveness, accept UE_LOGFMT (#1014)HEADmain
Branch started as a sessions-service overhaul (persistence, client liveness, UE_LOGFMT intake) and grew to pick up adjacent infrastructure work: an early-startup log backlog, a hardened `MemoryArena`, the `zen trace serve` viewer gaining a counter view + compact timeline + tabbed callsite panel, defensive fixes in the third-party `tourist` trace parser, a series of allocation reductions across the HTTP and compact-binary hot paths, and a new `zen sessions` CLI command tree. ## Sessions service **Persistence.** Each session lives on disk under `<DataRoot>/sessions/<id>/` as `info.cb` (metadata) plus `log.bin` (length-prefixed CbObject log records). On startup the service scans that directory and loads prior sessions as ended sessions, preloading the tail of each log so historical views work after a restart. `SessionLog` is noexcept-constructed and falls back to a disabled state on disk errors, so a bad disk can't take down `RegisterSession`. `GetSession` falls back to the ended-sessions list (fixes historical log fetches over HTTP). `LoadTail` counts only successfully-parsed records. **Pruning.** Periodic cleanup task drops ended sessions once any of three caps is exceeded: age (default 1 year), count (default 1000), or total on-disk footprint (default 50 MiB). Runs 30 s after startup, hourly thereafter. Active sessions never pruned; disk removal and directory stat happen outside the exclusive lock so a slow filesystem can't stall lookups. **Client liveness.** Sessions carry a `ProcessHandle` for the client-reported pid, captured at registration time so Windows pid recycling can't produce false positives. A 30 s asio timer probes liveness and ends dead sessions through the normal remove path, producing a synthetic `Session ended: process exited (...)` line persisted to `log.bin`. Windows decodes common NTSTATUS exit codes to human names (Ctrl-C, access violation, stack overflow, ...); POSIX stays at plain `process exited`. Clients auto-fill `ClientPid` only for local targets (unix socket / loopback); the server defensively accepts pids only from `IsLocalMachineRequest()` peers. zenserver also reports its own pid when registering its self-session, so it shows up with a real pid in the dashboard and `zen sessions ls`. **Synthetic end-of-session line.** `RemoveSession` takes an optional reason; before the session moves to the ended list it appends an Info-level `Session ended[: reason]` entry through the normal log path (released outside `m_Lock`). Current reasons: `client request` (HTTP DELETE), `server shutdown` (self-session), `process exited (...)` (liveness). **UE_LOGFMT structured entries.** `POST /sessions/{id}/log` now accepts `{level, logger, format, fields}` alongside the existing `{level, logger, message}` shape. New `logtemplate.{h,cpp}` implements UE's `StructuredLog.cpp` template grammar (field paths with `.name` / `[N]`, `{{`/`}}` escapes, `$text` / `$format` / `$locformat` object conventions, bounded recursion). Renders to a displayable message at intake while persisting raw format + fields so a future UI can drill into fields without another schema bump. Hot path is zero-alloc — renders into `ExtendableStringBuilder<256>` using stack-buffered `Oid::ToString` / `IoHash::ToHexString` overloads. UI shows a `{…}` marker with the raw template + JSON-pretty fields on hover. **Parent sessions.** `SessionInfo` gains `parent_session_id`; hub-managed storage server child processes inherit the hub's session id via `--parent-session=<id>`. `ZEN_SESSIONS_URL` env var becomes a fallback for `--sessions-url` / config when neither is provided. The in-process session log sink is disabled when a remote sessions target is configured (logs flow through `SessionsServiceClient` instead). The sessions UI groups child sessions under their parent (collapsible/expandable, sorts as a unit, supports nesting). **Platform reporting.** `SessionInfo` gains `Platform`, flowed end-to-end: client auto-fills via `GetRuntimePlatformName()`, server persists in `info.cb` (`plat`) and emits on GET. UI renders as a SimpleIcons-style inline SVG (windows / macOS / iOS / linux / wine / android / playstation / xbox / nintendo) with case-insensitive alias resolution (Win32/Win64, PS4/PS5, XSX/XSS, NintendoSwitch, iPhone/iPad, Darwin/OSX). Unknown values fall back to text; sorting runs on the underlying string. **WebSocket log streaming.** Sessions UI moves from 2 s polling to a WebSocket push model. New `WsSubscriber` has a stable id + helper methods. UI caps the log-line DOM at 5 000 entries with a shared cursor-regression helper, factored out of two call sites. Per-broadcast allocations trimmed on the push path; fixed a stack overrun in the WS log broadcast hex-id buffer. **Log memory.** `LogEntry::Level` is now `logging::LogLevel` (1 byte) instead of `std::string` (~32 B) — saves ~310 KB per full 10 k-entry deque and eliminates a per-message allocation in the in-proc sink. On-disk format writes an int32 and accepts either int or legacy string on read. `LogEntry` strings now live in a `MemoryArena`; logger names are interned across the deque. `SessionLog::Append` and `WriteSessionInfoFile` drop their `UniqueBuffer` round-trip and write `CbObject::GetView()` straight through `BasicFile` / `SafeWriteFile`. Multi-entry `POST /log` batched under one lock + one push. **In-proc log timestamps.** `InProcSessionLogSink::TimePointToDateTime` previously preserved only whole seconds, so every in-proc entry rendered at `.000` ms in the dashboard and `zen sessions tail`. It now adds the sub-second part (nanoseconds → 100 ns ticks) to keep ms precision end-to-end. **UI.** Side "Session Details" panel is gone — its info is inline in the table (appname, mode, platform, id, timestamps, this/log pills, active dot). Bottom panel is a tabbed `Log | Metadata` view with a right-side "Session Information" panel beside metadata; log-only controls (filter, newest-first, follow, log-level filter, expand/collapse) hide when Metadata is active, polling keeps running across tab switches. Wide-mode toggle fills the viewport edge-to-edge. Log lines show the logger category; timestamps render in 24 h with zero-padded fields regardless of locale. Sessions list defaults to All / 10 per page / created-desc, gains click-to-sort headers on the full dataset, a header filter box, and a pager aligned to the table's right edge. Duplicate auto-injected `<h1>Sessions</h1>` removed. ## `zen sessions` CLI New command tree on the `zen` client for inspecting the sessions service from the terminal: - **`zen sessions ls`** — lists sessions (active first, ended next; newest-first within each group) with id, status, app/mode, pid, created, duration, and log count. Supports `--status active|ended|all` (default `all`). - **`zen sessions status`** — prints the sessions service summary: self id, active / ended counts, and the read/write/delete/list/request/bad-request counters from `/stats/sessions`. - **`zen sessions tail [session]`** — tails a session's log. With no argument it tails zenserver's own session (resolved via `/sessions/list`'s `self_id`); an explicit 24-hex id targets any session, including ended ones (historical replay). `--lines N` (default 50, 0 = all buffered) trims the initial dump client-side. `--follow` prefers a WebSocket push subscription on `/sessions/ws` for sub-second latency; on upgrade failure (older server, blocked port, unix-socket transport) it falls back to HTTP cursor polling at `--interval-ms` (default 500), with sleeps chunked to 50 ms so Ctrl-C reacts quickly. Output matches `zen::logging::FullFormatter` (`[YY-MM-DD HH:MM:SS.mmm] [lvl] [logger] message`); on a TTY the level is colored and the logger is bold, with continuation lines indented under the message column using the *visible* prefix width. 404 surfaces as `(session ended)` and connection errors as `(server gone)` — both clean exits, so stopping the server mid-tail no longer prints a stack trace. - **`zen sessions ui`** — opens `<host>/dashboard/?page=sessions` in the user's default browser. Rejects unix-socket hosts. A small `ZenServiceClient::IsUnixSocket()` helper now wraps the unix-socket check used by `ui`, `sessions tail` (WS path), and `sessions ui`. ## Logging `BacklogSink` captures early-startup log entries in a fixed-capacity ring so late-attached sinks (session sink, file sink) can replay them. Detaches from the broadcast list when disabled; backed by destructor-only cleanup (no `unique_ptr` indirection per entry). Tuned defaults so the backlog covers typical bring-up without unbounded growth. ## `zen trace serve` viewer - Compact timeline mode for high-density views. - New `TRACE_INT_VALUE` / `TRACE_FLOAT_VALUE` counter trace points + a counters page in the viewer. - Callsite tables collapsed into a single tabbed panel. - Lossless `Oid <-> Guid` bridge for trace session ids; trace `SessionId` plumbed through. - `tourist` parser hardening: bounds-check `BufferStream::read`, validate `Type::info_size` before `patch()`, convert `parse_important_aux` to a loop (avoids deep recursion), widen `ParserPool` index to `uint32`, bounds-check field offsets in the dispatcher, pin `Types::parse` buffer up-front. ## `MemoryArena` Configurable chunk size, inline chunk list, oversize requests routed to truly-dedicated chunks (no slack waste, no fragmentation when one allocation is much larger than the chunk). ## Allocation cleanups across hot paths - `zenhttp::HttpRequestRouter::HandleRequest` and `FormatPackageMessageInternal`: drop heap allocations. - Compact-binary validation: `eastl::fixed_vector` + `eastl::sort`; eliminate `std::vector` churn. - `zenserverprocess`: trim transient allocations in spawn paths. - Sessions HTTP intake / broadcast: drop transient `std::string` allocs.
Diffstat (limited to 'src/zen/trace')
-rw-r--r--src/zen/trace/trace_analyze.cpp54
-rw-r--r--src/zen/trace/trace_cache.cpp120
-rw-r--r--src/zen/trace/trace_cache.h31
-rw-r--r--src/zen/trace/trace_model.cpp209
-rw-r--r--src/zen/trace/trace_model.h33
-rw-r--r--src/zen/trace/trace_viewer_service.cpp91
-rw-r--r--src/zen/trace/trace_viewer_service.h2
7 files changed, 539 insertions, 1 deletions
diff --git a/src/zen/trace/trace_analyze.cpp b/src/zen/trace/trace_analyze.cpp
index ff168cd9c..3538c074d 100644
--- a/src/zen/trace/trace_analyze.cpp
+++ b/src/zen/trace/trace_analyze.cpp
@@ -174,6 +174,7 @@ public:
AppendThreads();
AppendChannels();
AppendCpuScopeStats();
+ AppendCounters();
AppendMemorySummary();
AppendLiveAllocationCallstacks();
AppendChurnCallstacks();
@@ -336,6 +337,59 @@ private:
ZEN_CONSOLE("");
}
+ void AppendCounters() const
+ {
+ if (m_Model.CounterDefs.empty() && m_Model.CounterTimeSeries.empty())
+ {
+ return;
+ }
+
+ ZEN_CONSOLE("Counters:");
+ ZEN_CONSOLE("");
+ ZEN_CONSOLE("{:<48} {:>5} {:>10} {:>14} {:>14} {:>10}", "Counter", "Type", "Samples", "Min", "Max", "Last");
+ ZEN_CONSOLE("{:-<{}}", "", 48 + 5 + 10 + 14 + 14 + 10 + 5);
+
+ eastl::hash_map<uint16_t, const TraceModel::CounterSeries*> SeriesById;
+ SeriesById.reserve(m_Model.CounterTimeSeries.size());
+ for (const TraceModel::CounterSeries& S : m_Model.CounterTimeSeries)
+ {
+ SeriesById[S.Id] = &S;
+ }
+
+ auto FormatValue = [](double Value, uint8_t DisplayHint, uint8_t Type) -> std::string {
+ if (DisplayHint == 1 /* Memory */)
+ {
+ return fmt::format("{}", zen::NiceBytes(uint64_t(Value < 0 ? 0 : Value)));
+ }
+ if (Type == 0 /* Int */)
+ {
+ return fmt::format("{}", zen::ThousandsNum(int64_t(Value)));
+ }
+ return fmt::format("{:.3f}", Value);
+ };
+
+ for (const TraceModel::CounterDef& Def : m_Model.CounterDefs)
+ {
+ auto It = SeriesById.find(Def.Id);
+ if (It == SeriesById.end())
+ {
+ continue;
+ }
+ const TraceModel::CounterSeries& S = *It->second;
+ const char* TypeStr = (Def.Type == 0) ? "int" : "flt";
+ double Last = S.Samples.empty() ? 0.0 : S.Samples.back().Value;
+
+ ZEN_CONSOLE("{:<48.48} {:>5} {:>10} {:>14} {:>14} {:>10}",
+ Def.Name,
+ TypeStr,
+ zen::ThousandsNum(S.Count),
+ FormatValue(S.Min, Def.DisplayHint, Def.Type),
+ FormatValue(S.Max, Def.DisplayHint, Def.Type),
+ FormatValue(Last, Def.DisplayHint, Def.Type));
+ }
+ ZEN_CONSOLE("");
+ }
+
void AppendMemorySummary() const
{
const AllocationSummary& AllocSummary = m_Model.AllocSummary;
diff --git a/src/zen/trace/trace_cache.cpp b/src/zen/trace/trace_cache.cpp
index 165c1eecf..954df14e0 100644
--- a/src/zen/trace/trace_cache.cpp
+++ b/src/zen/trace/trace_cache.cpp
@@ -448,6 +448,53 @@ namespace {
return ToSharedBuffer(W);
}
+ // -- Counters section --
+
+ SharedBuffer WriteCountersSection(const TraceModel& Model, StringTableBuilder& Strings)
+ {
+ BinaryWriter W;
+
+ uint32_t DefCount = uint32_t(Model.CounterDefs.size());
+ WritePod(W, DefCount);
+ for (const TraceModel::CounterDef& D : Model.CounterDefs)
+ {
+ CounterDefPod P = {};
+ P.Id = D.Id;
+ P.Type = D.Type;
+ P.DisplayHint = D.DisplayHint;
+ P.Name = Strings.Intern(D.Name);
+ WritePod(W, P);
+ }
+
+ uint32_t SeriesCount = uint32_t(Model.CounterTimeSeries.size());
+ WritePod(W, SeriesCount);
+ // First pass: headers (so the reader can size each series before it
+ // reads samples).
+ for (const TraceModel::CounterSeries& S : Model.CounterTimeSeries)
+ {
+ CounterHeaderPod H = {};
+ H.Id = S.Id;
+ H.Type = S.Type;
+ H.SampleCount = uint32_t(S.Samples.size());
+ H.Min = S.Min;
+ H.Max = S.Max;
+ WritePod(W, H);
+ }
+ // Second pass: contiguous sample blob in the same order as headers.
+ for (const TraceModel::CounterSeries& S : Model.CounterTimeSeries)
+ {
+ for (const TraceModel::CounterSample& Sample : S.Samples)
+ {
+ CounterSamplePod SP = {};
+ SP.TimeUs = Sample.TimeUs;
+ SP.Value = Sample.Value;
+ WritePod(W, SP);
+ }
+ }
+
+ return ToSharedBuffer(W);
+ }
+
// -- Symbols section --
SharedBuffer WriteSymbolsSection(const eastl::hash_map<uint64_t, std::string>& ResolvedSymbols, StringTableBuilder& Strings)
@@ -860,6 +907,67 @@ namespace {
return true;
}
+ bool ReadCountersSection(const SharedBuffer& Data, const StringTableReader& Strings, TraceModel& Model)
+ {
+ BinaryReader R(Data.GetData(), Data.GetSize());
+
+ uint32_t DefCount = 0;
+ if (!ReadUint32(R, DefCount))
+ {
+ return false;
+ }
+ Model.CounterDefs.resize(DefCount);
+ for (uint32_t I = 0; I < DefCount; ++I)
+ {
+ CounterDefPod P;
+ if (!ReadPod(R, P))
+ {
+ return false;
+ }
+ Model.CounterDefs[I].Id = P.Id;
+ Model.CounterDefs[I].Type = P.Type;
+ Model.CounterDefs[I].DisplayHint = P.DisplayHint;
+ Model.CounterDefs[I].Name = std::string(Strings.Get(P.Name));
+ }
+
+ uint32_t SeriesCount = 0;
+ if (!ReadUint32(R, SeriesCount))
+ {
+ return false;
+ }
+ eastl::vector<CounterHeaderPod> Headers(SeriesCount);
+ for (uint32_t I = 0; I < SeriesCount; ++I)
+ {
+ if (!ReadPod(R, Headers[I]))
+ {
+ return false;
+ }
+ }
+ Model.CounterTimeSeries.resize(SeriesCount);
+ for (uint32_t I = 0; I < SeriesCount; ++I)
+ {
+ const CounterHeaderPod& H = Headers[I];
+ TraceModel::CounterSeries& Out = Model.CounterTimeSeries[I];
+ Out.Id = H.Id;
+ Out.Type = H.Type;
+ Out.Count = H.SampleCount;
+ Out.Min = H.Min;
+ Out.Max = H.Max;
+ Out.Samples.resize(H.SampleCount);
+ for (uint32_t J = 0; J < H.SampleCount; ++J)
+ {
+ CounterSamplePod SP;
+ if (!ReadPod(R, SP))
+ {
+ return false;
+ }
+ Out.Samples[J].TimeUs = SP.TimeUs;
+ Out.Samples[J].Value = SP.Value;
+ }
+ }
+ return true;
+ }
+
// ===========================================================================
// File-level helpers
// ===========================================================================
@@ -913,6 +1021,7 @@ WriteAnalyzeCache(const std::filesystem::path& CachePath,
SharedBuffer MemoryRaw = WriteMemorySection(Model, Strings);
SharedBuffer CallstacksRaw = WriteCallstacksSection(Model);
SharedBuffer SymbolsRaw = WriteSymbolsSection(ResolvedSymbols, Strings);
+ SharedBuffer CountersRaw = WriteCountersSection(Model, Strings);
SharedBuffer StringTableRaw = Strings.Serialize();
// Compress each section
@@ -922,6 +1031,7 @@ WriteAnalyzeCache(const std::filesystem::path& CachePath,
Sections[uint32_t(CacheSectionId::Memory)] = CompressSection(MemoryRaw);
Sections[uint32_t(CacheSectionId::Callstacks)] = CompressSection(CallstacksRaw);
Sections[uint32_t(CacheSectionId::Symbols)] = CompressSection(SymbolsRaw);
+ Sections[uint32_t(CacheSectionId::Counters)] = CompressSection(CountersRaw);
// Build file header
CacheFileHeader Header = {};
@@ -1091,6 +1201,16 @@ TryLoadAnalyzeCache(const std::filesystem::path& CachePath, const std::filesyste
}
}
+ SharedBuffer CountersData = DecompressSection(Base, Directory[uint32_t(CacheSectionId::Counters)]);
+ if (!CountersData.IsNull())
+ {
+ if (!ReadCountersSection(CountersData, Strings, Result.Model))
+ {
+ ZEN_DEBUG("Analysis cache: failed to read counters section");
+ // Counters are optional -- continue without them.
+ }
+ }
+
ZEN_INFO("Loaded analysis from cache ({})", zen::NiceBytes(FileData.Size()));
return Result;
}
diff --git a/src/zen/trace/trace_cache.h b/src/zen/trace/trace_cache.h
index 88778a020..6b5bd1da9 100644
--- a/src/zen/trace/trace_cache.h
+++ b/src/zen/trace/trace_cache.h
@@ -24,7 +24,7 @@ namespace zen::trace_detail {
// ---------------------------------------------------------------------------
static constexpr uint32_t kCacheMagic = 0x005A4355; // "UCZ\0"
-static constexpr uint32_t kCacheVersion = 1;
+static constexpr uint32_t kCacheVersion = 2; // bump on any layout change
enum class CacheSectionId : uint32_t
{
@@ -33,6 +33,7 @@ enum class CacheSectionId : uint32_t
Memory = 2,
Callstacks = 3,
Symbols = 4,
+ Counters = 5,
Count
};
@@ -211,6 +212,31 @@ struct SymbolEntryPod
uint32_t Pad;
};
+struct CounterDefPod
+{
+ uint16_t Id;
+ uint8_t Type;
+ uint8_t DisplayHint;
+ uint32_t Name; // string index
+};
+
+struct CounterHeaderPod
+{
+ uint16_t Id;
+ uint8_t Type;
+ uint8_t Pad0;
+ uint32_t SampleCount;
+ double Min;
+ double Max;
+};
+
+struct CounterSamplePod
+{
+ uint32_t TimeUs;
+ uint32_t Pad;
+ double Value;
+};
+
// Pin the on-disk layout. Any change here is a cache format change and must
// bump kCacheVersion.
static_assert(sizeof(CacheFileHeader) == 32);
@@ -229,6 +255,9 @@ static_assert(sizeof(CallstackChurnStatPod) == 48);
static_assert(sizeof(CallstackHeaderPod) == 16);
static_assert(sizeof(ResolvedFramePod) == 24);
static_assert(sizeof(SymbolEntryPod) == 16);
+static_assert(sizeof(CounterDefPod) == 8);
+static_assert(sizeof(CounterHeaderPod) == 24);
+static_assert(sizeof(CounterSamplePod) == 16);
// ---------------------------------------------------------------------------
// Cache read / write API
diff --git a/src/zen/trace/trace_model.cpp b/src/zen/trace/trace_model.cpp
index ac81161a1..c11e2c47c 100644
--- a/src/zen/trace/trace_model.cpp
+++ b/src/zen/trace/trace_model.cpp
@@ -387,6 +387,26 @@ begin_outline(CsvProfiler, Metadata)
field(uint8[], Key)
field(uint8[], Value)
end_outline()
+
+// Counters trace events (UE CountersTrace / zen ZEN_TRACE_INT_VALUE).
+begin_outline(Counters, Spec)
+ field(uint16, Id)
+ field(uint8, Type)
+ field(uint8, DisplayHint)
+ field(uint8[], Name)
+end_outline()
+
+begin_outline(Counters, SetValueInt)
+ field(uint64, Cycle)
+ field(int64, Value)
+ field(uint16, CounterId)
+end_outline()
+
+begin_outline(Counters, SetValueFloat)
+ field(uint64, Cycle)
+ field(double, Value)
+ field(uint16, CounterId)
+end_outline()
// clang-format on
//////////////////////////////////////////////////////////////////////////////
@@ -1471,6 +1491,138 @@ private:
};
//////////////////////////////////////////////////////////////////////////////
+// Counters analyzer -- consumes Counters.Spec / SetValueInt / SetValueFloat
+// (UE TRACE_INT_VALUE / TRACE_FLOAT_VALUE / zen ZEN_TRACE_INT_VALUE etc.).
+// Spec events register a counter id; SetValue events emit a sample. We keep
+// per-counter time series for the viewer / report to render.
+
+class CountersAnalyzer : public Analyzer
+{
+public:
+ explicit CountersAnalyzer(const TraceTiming* Timing = nullptr) : m_Timing(Timing) {}
+
+ void subscribe(Vector<Subscription>& Subs) override
+ {
+ Subs.emplace_back(this, &CountersAnalyzer::OnSpec);
+ Subs.emplace_back(this, &CountersAnalyzer::OnSetValueInt);
+ Subs.emplace_back(this, &CountersAnalyzer::OnSetValueFloat);
+ }
+
+ struct EditableSeries
+ {
+ zen::trace_detail::TraceModel::CounterSeries Series;
+ bool HasMin = false;
+ };
+
+ const eastl::hash_map<uint16_t, zen::trace_detail::TraceModel::CounterDef>& Defs() const { return m_Defs; }
+ eastl::hash_map<uint16_t, EditableSeries>& MutableSeriesMap() { return m_Series; }
+
+private:
+ uint32_t CycleToTimeUs(uint64_t Cycle) const
+ {
+ if (!m_Timing || m_Timing->Freq == 0)
+ {
+ return 0;
+ }
+ uint64_t Elapsed = (Cycle >= m_Timing->Base) ? (Cycle - m_Timing->Base) : 0;
+ return uint32_t(Elapsed * 1'000'000 / m_Timing->Freq);
+ }
+
+ static std::string DecodeAnsiName(const Array<uint8[]>& Data)
+ {
+ const uint8_t* P = Data.get();
+ size_t Size = Data.get_size();
+ if (!P || Size == 0)
+ {
+ return {};
+ }
+ return std::string(reinterpret_cast<const char*>(P), Size);
+ }
+
+ void OnSpec(const Counters_Spec& Ev)
+ {
+ uint16_t Id = uint16_t(Ev.Id());
+ if (Id == 0)
+ {
+ return;
+ }
+ zen::trace_detail::TraceModel::CounterDef Def;
+ Def.Id = Id;
+ Def.Type = uint8_t(Ev.Type());
+ Def.DisplayHint = uint8_t(Ev.DisplayHint());
+ Def.Name = DecodeAnsiName(Ev.Name());
+ if (Def.Name.empty())
+ {
+ Def.Name = fmt::format("counter_{}", Id);
+ }
+ m_Defs[Id] = std::move(Def);
+ }
+
+ EditableSeries& EnsureSeries(uint16_t Id, uint8_t Type)
+ {
+ auto It = m_Series.find(Id);
+ if (It == m_Series.end())
+ {
+ EditableSeries E;
+ E.Series.Id = Id;
+ E.Series.Type = Type;
+ It = m_Series.emplace(Id, std::move(E)).first;
+ }
+ return It->second;
+ }
+
+ void OnSetValueInt(const Counters_SetValueInt& Ev)
+ {
+ uint16_t Id = uint16_t(Ev.CounterId());
+ if (Id == 0)
+ {
+ return;
+ }
+ EditableSeries& E = EnsureSeries(Id, /*Int*/ 0);
+ double Value = double(int64_t(Ev.Value()));
+ uint32_t TimeUs = CycleToTimeUs(Ev.Cycle());
+ E.Series.Samples.push_back({TimeUs, Value});
+ ++E.Series.Count;
+ if (!E.HasMin || Value < E.Series.Min)
+ {
+ E.Series.Min = Value;
+ E.HasMin = true;
+ }
+ if (Value > E.Series.Max)
+ {
+ E.Series.Max = Value;
+ }
+ }
+
+ void OnSetValueFloat(const Counters_SetValueFloat& Ev)
+ {
+ uint16_t Id = uint16_t(Ev.CounterId());
+ if (Id == 0)
+ {
+ return;
+ }
+ EditableSeries& E = EnsureSeries(Id, /*Float*/ 1);
+ double Value = double(Ev.Value());
+ uint32_t TimeUs = CycleToTimeUs(Ev.Cycle());
+ E.Series.Samples.push_back({TimeUs, Value});
+ ++E.Series.Count;
+ if (!E.HasMin || Value < E.Series.Min)
+ {
+ E.Series.Min = Value;
+ E.HasMin = true;
+ }
+ if (Value > E.Series.Max)
+ {
+ E.Series.Max = Value;
+ }
+ }
+
+ const TraceTiming* m_Timing = nullptr;
+ eastl::hash_map<uint16_t, zen::trace_detail::TraceModel::CounterDef> m_Defs;
+ eastl::hash_map<uint16_t, EditableSeries> m_Series;
+};
+
+//////////////////////////////////////////////////////////////////////////////
// Analyzers
class CpuAnalyzer : public Analyzer
@@ -3165,6 +3317,7 @@ BuildTraceModel(const std::filesystem::path& FilePath, WorkerThreadPool& ThreadP
LogAnalyzer LogAn(&Timing);
BookmarksAnalyzer BookmarkAn(&Timing);
CsvProfilerAnalyzer CsvAn(&Timing);
+ CountersAnalyzer CountersAn(&Timing);
AllocationAnalyzer AllocAn(&Timing);
CallstackAnalyzer CallstackAn;
@@ -3182,6 +3335,7 @@ BuildTraceModel(const std::filesystem::path& FilePath, WorkerThreadPool& ThreadP
Dispatch.add_analyzer(LogAn);
Dispatch.add_analyzer(BookmarkAn);
Dispatch.add_analyzer(CsvAn);
+ Dispatch.add_analyzer(CountersAn);
Dispatch.add_analyzer(AllocAn);
Dispatch.add_analyzer(CallstackAn);
@@ -3434,6 +3588,61 @@ BuildTraceModel(const std::filesystem::path& FilePath, WorkerThreadPool& ThreadP
Model.CsvEvents.size());
}
+ // Counters (TRACE_INT_VALUE / TRACE_FLOAT_VALUE)
+ {
+ using CounterDefT = zen::trace_detail::TraceModel::CounterDef;
+ using CounterSeriesT = zen::trace_detail::TraceModel::CounterSeries;
+ using CounterSampleT = zen::trace_detail::TraceModel::CounterSample;
+
+ Model.CounterDefs.reserve(CountersAn.Defs().size());
+ for (const auto& [Id, Def] : CountersAn.Defs())
+ {
+ Model.CounterDefs.push_back(Def);
+ }
+
+ auto& SeriesMap = CountersAn.MutableSeriesMap();
+ Model.CounterTimeSeries.reserve(SeriesMap.size());
+ for (auto& [Id, Editable] : SeriesMap)
+ {
+ // Each counter's samples were appended in stream order. Tourist
+ // guarantees per-thread monotonicity but counters can be set from
+ // any thread, so a final sort by TimeUs is required.
+ eastl::sort(Editable.Series.Samples.begin(),
+ Editable.Series.Samples.end(),
+ [](const CounterSampleT& A, const CounterSampleT& B) { return A.TimeUs < B.TimeUs; });
+ // If the counter never produced a Spec event we still want the
+ // series visible. Synthesize a default def so the viewer has a name.
+ if (CountersAn.Defs().find(Id) == CountersAn.Defs().end())
+ {
+ CounterDefT Synth;
+ Synth.Id = Id;
+ Synth.Type = Editable.Series.Type;
+ Synth.Name = fmt::format("counter_{}", Id);
+ Model.CounterDefs.push_back(std::move(Synth));
+ }
+ Model.CounterTimeSeries.push_back(std::move(Editable.Series));
+ }
+ eastl::sort(Model.CounterDefs.begin(), Model.CounterDefs.end(), [](const CounterDefT& A, const CounterDefT& B) {
+ return A.Id < B.Id;
+ });
+ eastl::sort(Model.CounterTimeSeries.begin(), Model.CounterTimeSeries.end(), [](const CounterSeriesT& A, const CounterSeriesT& B) {
+ return A.Id < B.Id;
+ });
+
+ size_t TotalSamples = 0;
+ for (const CounterSeriesT& S : Model.CounterTimeSeries)
+ {
+ TotalSamples += S.Samples.size();
+ }
+ if (!Model.CounterDefs.empty() || !Model.CounterTimeSeries.empty())
+ {
+ ZEN_INFO("Counters: {} defined, {} with samples ({} samples total)",
+ Model.CounterDefs.size(),
+ Model.CounterTimeSeries.size(),
+ zen::ThousandsNum(TotalSamples));
+ }
+ }
+
// Memory allocation data
{
AllocAn.EmitFinalSample(Model.TraceEndUs);
diff --git a/src/zen/trace/trace_model.h b/src/zen/trace/trace_model.h
index bd6dcc674..3ac4c0cce 100644
--- a/src/zen/trace/trace_model.h
+++ b/src/zen/trace/trace_model.h
@@ -260,6 +260,39 @@ struct TraceModel
eastl::vector<CsvEvent> CsvEvents; // sorted by TimeUs
eastl::vector<CsvMeta> CsvMetadata;
+ // -- Counters (TRACE_INT_VALUE / TRACE_FLOAT_VALUE / TRACE_MEMORY_VALUE) --
+ // One CounterDef per registered counter (Counters.Spec event), and one
+ // CounterSeries per counter that produced any samples (Counters.SetValueInt
+ // / SetValueFloat events).
+ struct CounterDef
+ {
+ uint16_t Id = 0;
+ uint8_t Type = 0; // 0 = Int, 1 = Float
+ uint8_t DisplayHint = 0; // 0 = None, 1 = Memory
+ std::string Name;
+ };
+
+ struct CounterSample
+ {
+ uint32_t TimeUs;
+ double Value; // int counters are widened to double for transport;
+ // exact int values up to 2^53 round-trip losslessly.
+ };
+
+ struct CounterSeries
+ {
+ uint16_t Id = 0;
+ uint8_t Type = 0; // mirrors CounterDef::Type
+ uint8_t Pad = 0;
+ uint32_t Count = 0;
+ double Min = 0.0;
+ double Max = 0.0;
+ eastl::vector<CounterSample> Samples; // sorted by TimeUs
+ };
+
+ eastl::vector<CounterDef> CounterDefs; // sorted by Id
+ eastl::vector<CounterSeries> CounterTimeSeries; // one per counter that produced samples, sorted by Id
+
// -- Event type counts (sorted by count descending) --
struct EventTypeCount
{
diff --git a/src/zen/trace/trace_viewer_service.cpp b/src/zen/trace/trace_viewer_service.cpp
index 7d8301ae2..cd5517613 100644
--- a/src/zen/trace/trace_viewer_service.cpp
+++ b/src/zen/trace/trace_viewer_service.cpp
@@ -382,6 +382,14 @@ TraceViewerService::HandleApiRequest(HttpServerRequest& Request, std::string_vie
{
HandleCsvMetadataApi(Request);
}
+ else if (Path == "counters"sv)
+ {
+ HandleCountersApi(Request);
+ }
+ else if (Path == "counter-series"sv)
+ {
+ HandleCounterSeriesApi(Request);
+ }
else if (Path == "alloc-summary"sv)
{
HandleAllocSummaryApi(Request);
@@ -887,6 +895,89 @@ TraceViewerService::HandleCsvMetadataApi(HttpServerRequest& Request)
Request.WriteResponse(HttpResponseCode::OK, Writer.Save().AsArray());
}
+void
+TraceViewerService::HandleCountersApi(HttpServerRequest& Request)
+{
+ // Map id -> sample count so the front-end can hide empty defs without a
+ // second round-trip.
+ eastl::hash_map<uint16_t, uint32_t> SeriesByCounterId;
+ SeriesByCounterId.reserve(m_Model.CounterTimeSeries.size());
+ for (const trace_detail::TraceModel::CounterSeries& S : m_Model.CounterTimeSeries)
+ {
+ SeriesByCounterId[S.Id] = S.Count;
+ }
+
+ CbWriter Writer;
+ Writer.BeginArray();
+ for (const trace_detail::TraceModel::CounterDef& D : m_Model.CounterDefs)
+ {
+ auto It = SeriesByCounterId.find(D.Id);
+ Writer.BeginObject();
+ Writer << "id" << D.Id;
+ Writer << "type" << D.Type;
+ Writer << "display_hint" << D.DisplayHint;
+ Writer << "name" << D.Name;
+ Writer << "sample_count" << (It != SeriesByCounterId.end() ? It->second : 0u);
+ Writer.EndObject();
+ }
+ Writer.EndArray();
+
+ Request.WriteResponse(HttpResponseCode::OK, Writer.Save().AsArray());
+}
+
+void
+TraceViewerService::HandleCounterSeriesApi(HttpServerRequest& Request)
+{
+ HttpServerRequest::QueryParams Params = Request.GetQueryParams();
+ std::string_view IdStr = Params.GetValue("id");
+ if (IdStr.empty())
+ {
+ Request.WriteResponse(HttpResponseCode::BadRequest, HttpContentType::kText, "missing required query parameter 'id'");
+ return;
+ }
+ uint32_t WantedId = ParseUintParam(IdStr, 0);
+ if (WantedId == 0 || WantedId > 0xFFFF)
+ {
+ Request.WriteResponse(HttpResponseCode::BadRequest, HttpContentType::kText, "invalid 'id' (must be 1..65535)");
+ return;
+ }
+
+ const trace_detail::TraceModel::CounterSeries* Found = nullptr;
+ for (const trace_detail::TraceModel::CounterSeries& S : m_Model.CounterTimeSeries)
+ {
+ if (S.Id == uint16_t(WantedId))
+ {
+ Found = &S;
+ break;
+ }
+ }
+
+ CbObjectWriter Obj;
+ Obj << "id" << uint32_t(WantedId);
+ if (Found != nullptr)
+ {
+ Obj << "type" << uint32_t(Found->Type);
+ Obj << "count" << Found->Count;
+ Obj << "min" << Found->Min;
+ Obj << "max" << Found->Max;
+ }
+ Obj.BeginArray("samples");
+ if (Found != nullptr)
+ {
+ // [time_us, value] tuples to keep the payload tight for large series.
+ for (const trace_detail::TraceModel::CounterSample& S : Found->Samples)
+ {
+ Obj.BeginArray();
+ Obj.AddInteger(uint32_t(S.TimeUs));
+ Obj.AddFloat(S.Value);
+ Obj.EndArray();
+ }
+ }
+ Obj.EndArray();
+
+ Request.WriteResponse(HttpResponseCode::OK, Obj.Save());
+}
+
//////////////////////////////////////////////////////////////////////////////
// Memory allocation endpoints
diff --git a/src/zen/trace/trace_viewer_service.h b/src/zen/trace/trace_viewer_service.h
index f7bc51499..af74134b1 100644
--- a/src/zen/trace/trace_viewer_service.h
+++ b/src/zen/trace/trace_viewer_service.h
@@ -49,6 +49,8 @@ private:
void HandleCsvSeriesApi(HttpServerRequest& Request);
void HandleCsvEventsApi(HttpServerRequest& Request);
void HandleCsvMetadataApi(HttpServerRequest& Request);
+ void HandleCountersApi(HttpServerRequest& Request);
+ void HandleCounterSeriesApi(HttpServerRequest& Request);
void HandleAllocSummaryApi(HttpServerRequest& Request);
void HandleHeapsApi(HttpServerRequest& Request);
void HandleAllocTagsApi(HttpServerRequest& Request);