sessions: persist to disk, prune, track client liveness, accept UE_LOGFMT (#1014)HEADmain

Branch started as a sessions-service overhaul (persistence, client liveness, UE_LOGFMT intake) and grew to pick up adjacent infrastructure work: an early-startup log backlog, a hardened `MemoryArena`, the `zen trace serve` viewer gaining a counter view + compact timeline + tabbed callsite panel, defensive fixes in the third-party `tourist` trace parser, a series of allocation reductions across the HTTP and compact-binary hot paths, and a new `zen sessions` CLI command tree.

## Sessions service

**Persistence.** Each session lives on disk under `<DataRoot>/sessions/<id>/` as `info.cb` (metadata) plus `log.bin` (length-prefixed CbObject log records). On startup the service scans that directory and loads prior sessions as ended sessions, preloading the tail of each log so historical views work after a restart. `SessionLog` is noexcept-constructed and falls back to a disabled state on disk errors, so a bad disk can't take down `RegisterSession`. `GetSession` falls back to the ended-sessions list (fixes historical log fetches over HTTP). `LoadTail` counts only successfully-parsed records.

**Pruning.** Periodic cleanup task drops ended sessions once any of three caps is exceeded: age (default 1 year), count (default 1000), or total on-disk footprint (default 50 MiB). Runs 30 s after startup, hourly thereafter. Active sessions never pruned; disk removal and directory stat happen outside the exclusive lock so a slow filesystem can't stall lookups.

**Client liveness.** Sessions carry a `ProcessHandle` for the client-reported pid, captured at registration time so Windows pid recycling can't produce false positives. A 30 s asio timer probes liveness and ends dead sessions through the normal remove path, producing a synthetic `Session ended: process exited (...)` line persisted to `log.bin`. Windows decodes common NTSTATUS exit codes to human names (Ctrl-C, access violation, stack overflow, ...); POSIX stays at plain `process exited`. Clients auto-fill `ClientPid` only for local targets (unix socket / loopback); the server defensively accepts pids only from `IsLocalMachineRequest()` peers. zenserver also reports its own pid when registering its self-session, so it shows up with a real pid in the dashboard and `zen sessions ls`.

**Synthetic end-of-session line.** `RemoveSession` takes an optional reason; before the session moves to the ended list it appends an Info-level `Session ended[: reason]` entry through the normal log path (released outside `m_Lock`). Current reasons: `client request` (HTTP DELETE), `server shutdown` (self-session), `process exited (...)` (liveness).

**UE_LOGFMT structured entries.** `POST /sessions/{id}/log` now accepts `{level, logger, format, fields}` alongside the existing `{level, logger, message}` shape. New `logtemplate.{h,cpp}` implements UE's `StructuredLog.cpp` template grammar (field paths with `.name` / `[N]`, `{{`/`}}` escapes, `$text` / `$format` / `$locformat` object conventions, bounded recursion). Renders to a displayable message at intake while persisting raw format + fields so a future UI can drill into fields without another schema bump. Hot path is zero-alloc — renders into `ExtendableStringBuilder<256>` using stack-buffered `Oid::ToString` / `IoHash::ToHexString` overloads. UI shows a `{…}` marker with the raw template + JSON-pretty fields on hover.

**Parent sessions.** `SessionInfo` gains `parent_session_id`; hub-managed storage server child processes inherit the hub's session id via `--parent-session=<id>`. `ZEN_SESSIONS_URL` env var becomes a fallback for `--sessions-url` / config when neither is provided. The in-process session log sink is disabled when a remote sessions target is configured (logs flow through `SessionsServiceClient` instead). The sessions UI groups child sessions under their parent (collapsible/expandable, sorts as a unit, supports nesting).

**Platform reporting.** `SessionInfo` gains `Platform`, flowed end-to-end: client auto-fills via `GetRuntimePlatformName()`, server persists in `info.cb` (`plat`) and emits on GET. UI renders as a SimpleIcons-style inline SVG (windows / macOS / iOS / linux / wine / android / playstation / xbox / nintendo) with case-insensitive alias resolution (Win32/Win64, PS4/PS5, XSX/XSS, NintendoSwitch, iPhone/iPad, Darwin/OSX). Unknown values fall back to text; sorting runs on the underlying string.

**WebSocket log streaming.** Sessions UI moves from 2 s polling to a WebSocket push model. New `WsSubscriber` has a stable id + helper methods. UI caps the log-line DOM at 5 000 entries with a shared cursor-regression helper, factored out of two call sites. Per-broadcast allocations trimmed on the push path; fixed a stack overrun in the WS log broadcast hex-id buffer.

**Log memory.** `LogEntry::Level` is now `logging::LogLevel` (1 byte) instead of `std::string` (~32 B) — saves ~310 KB per full 10 k-entry deque and eliminates a per-message allocation in the in-proc sink. On-disk format writes an int32 and accepts either int or legacy string on read. `LogEntry` strings now live in a `MemoryArena`; logger names are interned across the deque. `SessionLog::Append` and `WriteSessionInfoFile` drop their `UniqueBuffer` round-trip and write `CbObject::GetView()` straight through `BasicFile` / `SafeWriteFile`. Multi-entry `POST /log` batched under one lock + one push.

**In-proc log timestamps.** `InProcSessionLogSink::TimePointToDateTime` previously preserved only whole seconds, so every in-proc entry rendered at `.000` ms in the dashboard and `zen sessions tail`. It now adds the sub-second part (nanoseconds → 100 ns ticks) to keep ms precision end-to-end.

**UI.** Side "Session Details" panel is gone — its info is inline in the table (appname, mode, platform, id, timestamps, this/log pills, active dot). Bottom panel is a tabbed `Log | Metadata` view with a right-side "Session Information" panel beside metadata; log-only controls (filter, newest-first, follow, log-level filter, expand/collapse) hide when Metadata is active, polling keeps running across tab switches. Wide-mode toggle fills the viewport edge-to-edge. Log lines show the logger category; timestamps render in 24 h with zero-padded fields regardless of locale. Sessions list defaults to All / 10 per page / created-desc, gains click-to-sort headers on the full dataset, a header filter box, and a pager aligned to the table's right edge. Duplicate auto-injected `<h1>Sessions</h1>` removed.

## `zen sessions` CLI

New command tree on the `zen` client for inspecting the sessions service from the terminal:

- **`zen sessions ls`** — lists sessions (active first, ended next; newest-first within each group) with id, status, app/mode, pid, created, duration, and log count. Supports `--status active|ended|all` (default `all`).
- **`zen sessions status`** — prints the sessions service summary: self id, active / ended counts, and the read/write/delete/list/request/bad-request counters from `/stats/sessions`.
- **`zen sessions tail [session]`** — tails a session's log. With no argument it tails zenserver's own session (resolved via `/sessions/list`'s `self_id`); an explicit 24-hex id targets any session, including ended ones (historical replay). `--lines N` (default 50, 0 = all buffered) trims the initial dump client-side. `--follow` prefers a WebSocket push subscription on `/sessions/ws` for sub-second latency; on upgrade failure (older server, blocked port, unix-socket transport) it falls back to HTTP cursor polling at `--interval-ms` (default 500), with sleeps chunked to 50 ms so Ctrl-C reacts quickly. Output matches `zen::logging::FullFormatter` (`[YY-MM-DD HH:MM:SS.mmm] [lvl] [logger] message`); on a TTY the level is colored and the logger is bold, with continuation lines indented under the message column using the *visible* prefix width. 404 surfaces as `(session ended)` and connection errors as `(server gone)` — both clean exits, so stopping the server mid-tail no longer prints a stack trace.
- **`zen sessions ui`** — opens `<host>/dashboard/?page=sessions` in the user's default browser. Rejects unix-socket hosts.

A small `ZenServiceClient::IsUnixSocket()` helper now wraps the unix-socket check used by `ui`, `sessions tail` (WS path), and `sessions ui`.

## Logging

`BacklogSink` captures early-startup log entries in a fixed-capacity ring so late-attached sinks (session sink, file sink) can replay them. Detaches from the broadcast list when disabled; backed by destructor-only cleanup (no `unique_ptr` indirection per entry). Tuned defaults so the backlog covers typical bring-up without unbounded growth.

## `zen trace serve` viewer

- Compact timeline mode for high-density views.
- New `TRACE_INT_VALUE` / `TRACE_FLOAT_VALUE` counter trace points + a counters page in the viewer.
- Callsite tables collapsed into a single tabbed panel.
- Lossless `Oid <-> Guid` bridge for trace session ids; trace `SessionId` plumbed through.
- `tourist` parser hardening: bounds-check `BufferStream::read`, validate `Type::info_size` before `patch()`, convert `parse_important_aux` to a loop (avoids deep recursion), widen `ParserPool` index to `uint32`, bounds-check field offsets in the dispatcher, pin `Types::parse` buffer up-front.

## `MemoryArena`

Configurable chunk size, inline chunk list, oversize requests routed to truly-dedicated chunks (no slack waste, no fragmentation when one allocation is much larger than the chunk).

## Allocation cleanups across hot paths

- `zenhttp::HttpRequestRouter::HandleRequest` and `FormatPackageMessageInternal`: drop heap allocations.
- Compact-binary validation: `eastl::fixed_vector` + `eastl::sort`; eliminate `std::vector` churn.
- `zenserverprocess`: trim transient allocations in spawn paths.
- Sessions HTTP intake / broadcast: drop transient `std::string` allocs.

5 files changed, 197 insertions, 13 deletions

diff --git a/src/zenutil/include/zenutil/logging.h b/src/zenutil/include/zenutil/logging.h
index 6abf6a96f..94a45e46f 100644
--- a/src/zenutil/include/zenutil/logging.h
+++ b/src/zenutil/include/zenutil/logging.h
@@ -19,8 +19,9 @@
 //
 
 namespace zen::logging {
+class BacklogSink;
 class BroadcastSink;
-}
+}  // namespace zen::logging
 
 namespace zen {
 
@@ -46,4 +47,25 @@ void ShutdownLogging();
 logging::SinkPtr			GetFileSink();
 Ref<logging::BroadcastSink> GetDefaultBroadcastSink();
 
+/// The default backlog sink, installed alongside the broadcast sink at
+/// BeginInitializeLogging. Captures every log line until DisableLogBacklog()
+/// is called. Use AttachSinkWithBacklogReplay() to add a sink that should
+/// receive the backlog, or call Replay() directly on this sink for ad-hoc
+/// replay into a target. May return nullptr before logging is initialized
+/// or after the backlog is disabled.
+Ref<logging::BacklogSink> GetLogBacklogSink();
+
+/// Add a sink to the default broadcast and replay the captured backlog
+/// into it before any new messages reach it. Use this for any sink that
+/// should see the early-startup window — e.g. an in-proc session log
+/// sink, a Sentry breadcrumb sink, an OTLP forwarder. After the backlog
+/// has been disabled this is equivalent to a plain AddSink.
+void AttachSinkWithBacklogReplay(logging::SinkPtr Sink);
+
+/// Stop capturing into the backlog and free its buffer. Idempotent.
+/// Call this once the bootstrap window has closed — typically right
+/// before the server enters its run loop or the CLI dispatches its
+/// command. Subsequent log calls bypass the backlog cheaply.
+void DisableLogBacklog();
+
 }  // namespace zen
diff --git a/src/zenutil/include/zenutil/sessionsclient.h b/src/zenutil/include/zenutil/sessionsclient.h
index c144a9baa..ae2364279 100644
--- a/src/zenutil/include/zenutil/sessionsclient.h
+++ b/src/zenutil/include/zenutil/sessionsclient.h
@@ -24,12 +24,21 @@ class SessionsServiceClient
 public:
 	struct Options
 	{
-		std::string		   TargetUrl;			   // Base URL of the target zenserver (e.g. "http://localhost:8558")
-		std::string		   AppName;				   // Application name to register
-		std::string		   Mode;				   // Server mode (e.g. "Server", "Compute", "Proxy")
-		Oid				   SessionId = Oid::Zero;  // Session ID to register under
-		Oid				   JobId	 = Oid::Zero;  // Optional job ID
-		HttpClientSettings ClientSettings;		   // Optional; timeouts are overridden internally (e.g. for unix sockets)
+		std::string TargetUrl;	// Base URL of the target zenserver (e.g. "http://localhost:8558")
+		std::string AppName;	// Application name to register
+		std::string Mode;		// Server mode (e.g. "Server", "Compute", "Proxy")
+		std::string Platform;	// Client platform; empty = auto-detect via GetRuntimePlatformName()
+		// PID the server uses to track this client's liveness. 0 = don't
+		// report. Auto-filled to GetCurrentProcessId() when TargetUrl looks
+		// local (unix socket, localhost, 127.0.0.1); set explicitly to
+		// override or suppress.
+		uint32_t ClientPid		 = 0;
+		Oid		 SessionId		 = Oid::Zero;  // Session ID to register under
+		Oid		 ParentSessionId = Oid::Zero;  // Optional parent session ID
+		// Optional task/action identifier. Use this to associate the session with
+		// a specific unit of work while ParentSessionId links process/session ancestry.
+		Oid				   JobId = Oid::Zero;
+		HttpClientSettings ClientSettings;	// Optional; timeouts are overridden internally (e.g. for unix sockets)
 	};
 
 	/// Command sent to the background worker thread.
diff --git a/src/zenutil/logging/logging.cpp b/src/zenutil/logging/logging.cpp
index 936e3c4fd..06e8f920e 100644
--- a/src/zenutil/logging/logging.cpp
+++ b/src/zenutil/logging/logging.cpp
@@ -8,6 +8,7 @@
 #include <zencore/logging.h>
 #include <zencore/logging/ansicolorsink.h>
 #include <zencore/logging/asyncsink.h>
+#include <zencore/logging/backlogsink.h>
 #include <zencore/logging/broadcastsink.h>
 #include <zencore/logging/logger.h>
 #include <zencore/logging/msvcsink.h>
@@ -27,6 +28,7 @@ namespace zen {
 static bool					g_IsLoggingInitialized;
 logging::SinkPtr			g_FileSink;
 Ref<logging::BroadcastSink> g_BroadcastSink;
+Ref<logging::BacklogSink>	g_BacklogSink;
 
 logging::SinkPtr
 GetFileSink()
@@ -40,6 +42,83 @@ GetDefaultBroadcastSink()
 	return g_BroadcastSink;
 }
 
+Ref<logging::BacklogSink>
+GetLogBacklogSink()
+{
+	return g_BacklogSink;
+}
+
+void
+AttachSinkWithBacklogReplay(logging::SinkPtr Sink)
+{
+	if (!Sink)
+	{
+		return;
+	}
+	// Drain any AsyncSink queue first so messages emitted just before this
+	// call have a chance to land in the backlog before we snapshot its
+	// cursor. Without this we'd still be correct (AsyncSink would deliver
+	// the queued lines to the new sink directly via the broadcast after
+	// AddSink runs), but the new sink would observe them out of timestamp
+	// order — replayed history first, then late drains interleaved with
+	// fresh post-attach messages.
+	zen::logging::FlushLogging();
+
+	if (!g_BroadcastSink)
+	{
+		return;
+	}
+
+	// Subscribe Sink to the broadcast and snapshot the backlog cursor
+	// atomically. The broadcast's exclusive lock blocks fanout while it's
+	// held, so during the callback no thread can be inside backlog.Log()
+	// and the cursor we capture is exact w.r.t. the moment Sink became
+	// visible to fanout.
+	//
+	// After the callback returns and the lock is released, two things are
+	// true:
+	//  - Any backlog entry at index < Cursor was added before Sink became
+	//    a fanout target, so Sink did NOT receive it via Log() — Replay
+	//    must deliver it.
+	//  - Any backlog entry at index >= Cursor was added after Sink became
+	//    a fanout target, so Sink already received it via Log() — Replay
+	//    must skip it.
+	// This closes the prior race where a message arriving between Replay
+	// and AddSink would land in the backlog only and be lost on
+	// DisableLogBacklog.
+	size_t Cursor = 0;
+	g_BroadcastSink->AddSinkAtomic(Sink, [&]() {
+		if (g_BacklogSink && g_BacklogSink->IsEnabled())
+		{
+			Cursor = g_BacklogSink->Size();
+		}
+	});
+
+	if (Cursor > 0)
+	{
+		g_BacklogSink->Replay(*Sink, Cursor);
+	}
+}
+
+void
+DisableLogBacklog()
+{
+	if (!g_BacklogSink)
+	{
+		return;
+	}
+	// Remove from the broadcast first so subsequent log lines don't even
+	// fan out to a now-no-op sink — one fewer per-log overhead for the
+	// rest of the process lifetime. Disable() then frees the arena, and
+	// dropping our own Ref releases the object.
+	if (g_BroadcastSink)
+	{
+		g_BroadcastSink->RemoveSink(logging::SinkPtr(g_BacklogSink.Get()));
+	}
+	g_BacklogSink->Disable();
+	g_BacklogSink = nullptr;
+}
+
 void
 InitializeLogging(const LoggingOptions& LogOptions)
 {
@@ -129,6 +208,14 @@ BeginInitializeLogging(const LoggingOptions& LogOptions)
 	// a child sink later is immediately visible to every logger.
 	std::vector<logging::SinkPtr> BroadcastChildren;
 
+	// Install the backlog sink as the first broadcast child so it sees
+	// every line emitted from this point until the bootstrap window
+	// closes (DisableLogBacklog at server-run-loop / CLI-dispatch). Sinks
+	// attached later via AttachSinkWithBacklogReplay can replay the
+	// captured window into themselves so they don't miss the early logs.
+	g_BacklogSink = Ref<logging::BacklogSink>(new logging::BacklogSink());
+	BroadcastChildren.push_back(logging::SinkPtr(g_BacklogSink.Get()));
+
 	if (LogOptions.NoConsoleOutput)
 	{
 		zen::logging::SuppressConsoleLog();
@@ -274,6 +361,7 @@ ShutdownLogging()
 
 	g_FileSink		= nullptr;
 	g_BroadcastSink = nullptr;
+	g_BacklogSink	= nullptr;
 }
 
 }  // namespace zen
diff --git a/src/zenutil/sessionsclient.cpp b/src/zenutil/sessionsclient.cpp
index 6ba997a62..f8dab4fb9 100644
--- a/src/zenutil/sessionsclient.cpp
+++ b/src/zenutil/sessionsclient.cpp
@@ -6,6 +6,8 @@
 #include <zencore/fmtutils.h>
 #include <zencore/iobuffer.h>
 #include <zencore/logging/logmsg.h>
+#include <zencore/process.h>
+#include <zencore/system.h>
 #include <zencore/thread.h>
 
 #include <vector>
@@ -70,6 +72,27 @@ SessionsServiceClient::SessionsServiceClient(Options Opts)
 		m_Options.TargetUrl.pop_back();
 	}
 
+	// Auto-detect the platform if the caller didn't set one explicitly.
+	if (m_Options.Platform.empty())
+	{
+		m_Options.Platform = std::string(GetRuntimePlatformName());
+	}
+
+	// Auto-fill ClientPid when we can reasonably assume the target is on the
+	// same machine. The server ALSO defensively gates pid acceptance on
+	// IsLocalMachineRequest(), so sending a pid for a non-local URL doesn't
+	// cause false positives — this heuristic just avoids the redundant send.
+	if (m_Options.ClientPid == 0)
+	{
+		const bool IsUnixSocket = !m_Options.ClientSettings.UnixSocketPath.empty();
+		const bool LooksLocal	= IsUnixSocket || m_Options.TargetUrl.find("localhost") != std::string::npos ||
+								m_Options.TargetUrl.find("127.0.0.1") != std::string::npos;
+		if (LooksLocal)
+		{
+			m_Options.ClientPid = static_cast<uint32_t>(GetCurrentProcessId());
+		}
+	}
+
 	m_WorkerThread = std::thread([this]() {
 		zen::SetCurrentThreadName("SessionIO");
 		WorkerLoop();
@@ -98,6 +121,18 @@ SessionsServiceClient::BuildRequestBody(CbObjectView Metadata) const
 	{
 		Writer << "mode" << m_Options.Mode;
 	}
+	if (!m_Options.Platform.empty())
+	{
+		Writer << "platform" << m_Options.Platform;
+	}
+	if (m_Options.ClientPid != 0)
+	{
+		Writer << "pid" << m_Options.ClientPid;
+	}
+	if (m_Options.ParentSessionId != Oid::Zero)
+	{
+		Writer << "parent_session_id" << m_Options.ParentSessionId;
+	}
 	if (m_Options.JobId != Oid::Zero)
 	{
 		Writer << "jobid" << m_Options.JobId;
diff --git a/src/zenutil/zenserverprocess.cpp b/src/zenutil/zenserverprocess.cpp
index 2d4334ffa..16232333f 100644
--- a/src/zenutil/zenserverprocess.cpp
+++ b/src/zenutil/zenserverprocess.cpp
@@ -15,6 +15,7 @@
 #include <zencore/timer.h>
 
 #include <atomic>
+#include <cctype>
 #include <string>
 
 #include <gsl/gsl-lite.hpp>
@@ -555,6 +556,28 @@ ZenServerState::ZenServerEntry::AddSponsorProcess(uint32_t PidToAdd, uint64_t Ti
 
 static constexpr size_t kInstanceInfoSize = 4096;
 
+// Token-aware search for a CLI flag (e.g. "--parent-session") within an
+// argument string. Avoids false positives like "--parent-session-foo" by
+// requiring the match to start at the beginning or after whitespace, and to
+// end at the end of the string, at '=', or at whitespace.
+static bool
+HasCliFlag(std::string_view Args, std::string_view Flag)
+{
+	size_t Pos = 0;
+	while ((Pos = Args.find(Flag, Pos)) != std::string_view::npos)
+	{
+		const bool	 LeftOk	 = (Pos == 0) || std::isspace(static_cast<unsigned char>(Args[Pos - 1]));
+		const size_t End	 = Pos + Flag.size();
+		const bool	 RightOk = (End == Args.size()) || (Args[End] == '=') || std::isspace(static_cast<unsigned char>(Args[End]));
+		if (LeftOk && RightOk)
+		{
+			return true;
+		}
+		Pos = End;
+	}
+	return false;
+}
+
 ZenServerInstanceInfo::ZenServerInstanceInfo() = default;
 
 ZenServerInstanceInfo::~ZenServerInstanceInfo()
@@ -1119,9 +1142,10 @@ ZenServerInstance::SpawnServerInternal(int ChildId, std::string_view ServerArgs,
 
 	ExtendableStringBuilder<512> CommandLine;
 	{
-		const std::string  ExeUtf8	  = PathToUtf8(Executable);
+		ExtendableStringBuilder<260> ExeUtf8;
+		PathToUtf8(Executable, ExeUtf8);
 		constexpr AsciiSet QuoteChars = " \t\"";
-		if (AsciiSet::HasAny(ExeUtf8.c_str(), QuoteChars))
+		if (AsciiSet::HasAny(ExeUtf8.ToView(), QuoteChars))
 		{
 			CommandLine << '"' << ExeUtf8 << '"';
 		}
@@ -1147,6 +1171,11 @@ ZenServerInstance::SpawnServerInternal(int ChildId, std::string_view ServerArgs,
 		CommandLine << " --enable-execution-history=false";
 	}
 
+	if (!HasCliFlag(ServerArgs, "--parent-session"))
+	{
+		CommandLine << " --parent-session " << GetSessionIdString();
+	}
+
 	if (!ServerArgs.empty())
 	{
 		CommandLine << " " << ServerArgs;
@@ -1246,7 +1275,7 @@ ZenServerInstance::SpawnServer(int BasePort, std::string_view AdditionalServerAr
 		CommandLine << " --test --log-id " << m_Name;
 		CommandLine << " --no-sentry";
 
-		if (AdditionalServerArgs.find("--system-dir") == std::string_view::npos)
+		if (!HasCliFlag(AdditionalServerArgs, "--system-dir"))
 		{
 			CommandLine << " --system-dir ";
 			PathToUtf8((m_Env.CreateNewTestDir() / "system-dir").c_str(), CommandLine);
@@ -1665,10 +1694,11 @@ StartupZenServer(LoggerRef LogRef, const StartupZenServerOptions& Options)
 	ZenServerInstance Server(ServerEnvironment, Options.Mode);
 	Server.SetEnableExecutionHistory(Options.EnableExecutionHistory);
 
-	std::string ServerArguments(Options.ExtraArgs);
-	if ((Options.Port != 0) && (ServerArguments.find("--port") == std::string::npos))
+	ExtendableStringBuilder<256> ServerArguments;
+	ServerArguments << Options.ExtraArgs;
+	if (Options.Port != 0 && !HasCliFlag(ServerArguments, "--port"))
 	{
-		ServerArguments.append(fmt::format(" --port {}", Options.Port));
+		ServerArguments << " --port " << Options.Port;
 	}
 	Server.SpawnServer(ServerArguments, Options.OpenConsole, /*WaitTimeoutMs*/ 0);