aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* Add sequence numbers to log stream protocol and tests for drop detectionsb/sessionizeStefan Boberg21 hours5-13/+179
| | | | | | | | TcpLogStreamSink now stamps each message with a monotonic sequence number. LogStreamListener tracks the expected sequence per session and emits a synthetic "dropped" notice when gaps appear. Includes tests covering basic delivery, multi-line splitting, drop detection, and contiguous sequencing. Also simplifies LogMessage::s_DefaultPoint to a single entry.
* Use gathered buffer sequence in TcpLogStreamSink for batched TCP writesStefan Boberg21 hours1-8/+15
| | | | | | Replace per-message asio::write loop with a single gathered write using a fixed_vector of const_buffers, reducing syscall overhead when flushing batches of log messages.
* Add LogStreamListener and TcpLogStreamSink from compute-scheduler branchStefan Boberg34 hours3-0/+500
| | | | | | Brings over TCP log streaming infrastructure for use in session management. LogStreamListener accepts connections and delivers log lines via the LogStreamTarget interface; TcpLogStreamSink is the sending counterpart.
* Merge main into sb/sessionizeStefan Boberg46 hours42-902/+1158
|\ | | | | | | Resolve html.zip conflict by accepting deletion from main.
| * Made CPR optional, html generated at build time (#840)Stefan Boberg2 days8-22/+85
| | | | | | | | | | | | | | - Fix potential crash on startup caused by logging macros being invoked before the logging system is initialized (null logger dereference in `ZenServerState::Sweep()`). `LoggerRef::ShouldLog` now guards against a null logger pointer. - Make CPR an optional dependency (`--zencpr` build option, enabled by default) so builds can proceed without it - Make zenvfs Windows-only (platform-specific target) - Generate the frontend zip at build time from source HTML files instead of checking in a binary blob which would accumulate with every single update
| * Add clang-cl build supportStefan Boberg3 days5-74/+82
| | | | | | | | | | | | | | | | | | | | - Add clang-cl warning suppressions in xmake.lua matching Linux/macOS set - Guard /experimental:c11atomics with {tools="cl"} for MSVC-only - Fix long long / int64_t redefinition in string.h for clang-cl - Fix unclosed namespace in callstacktrace.cpp #else branch - Fix missing override in httpplugin.cpp - Reorder WorkerPool fields to match designated initializer order - Use INVALID_SOCKET instead of SOCKET_ERROR for SOCKET comparisons
| * Unix Domain Socket auto discovery (#833)Stefan Boberg3 days28-105/+538
| | | | | | | | | | | | | | | | This PR adds end-to-end Unix domain socket (UDS) support, allowing zen CLI to discover and connect to UDS-only servers automatically. - **`unix://` URI scheme in zen CLI**: The `-u` / `--hosturl` option now accepts `unix:///path/to/socket` to connect to a zenserver via a Unix domain socket instead of TCP. - **Per-instance shared memory for extended server info**: Each zenserver instance now publishes a small shared memory section (keyed by SessionId) containing per-instance data that doesn't fit in the fixed-size ZenServerEntry -- starting with the UDS socket path. This is a 4KB pagefile-backed section on Windows (`Global\ZenInstance_{sessionid}`) and a POSIX shared memory object on Linux/Mac (`/UnrealEngineZen_{sessionid}`). - **Client-side auto-discovery of UDS servers**: `zen info`, `zen status`, etc. now automatically discover and prefer UDS connections when a server publishes a socket path. Servers running with `--no-network` (UDS-only) are no longer invisible to the CLI. - **`kNoNetwork` flag in ZenServerEntry**: Servers started with `--no-network` advertise this in their shared state entry. Clients skip TCP fallback for these servers, and display commands (`ps`, `status`, `top`) show `-` instead of a port number to indicate TCP is not available.
| * Switch httpclient default back-end over to libcurl (#832)Stefan Boberg3 days6-702/+454
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Switches the default HTTP client to the libcurl-based backend and follows up with a series of correctness fixes and code quality improvements to `CurlHttpClient`. **Backend switch & build fixes:** - Switch default HTTP client to libcurl-based backend - Suppress `[[nodiscard]]` warning when building fmt - Miscellaneous bugfixes in HttpClient/libcurl - Pass `-y` to `xmake config` in `xmake test` task **Boilerplate reduction:** - Add `Session::SetHeaders()` for RAII ownership of `curl_slist`, eliminating manual `curl_slist_free_all` calls from every verb method - Add `Session::PerformWithResponseCallbacks()` to absorb the repeated 12-line write+header callback setup block - Extract `ParseHeaderLine()` shared helper, replacing 4 duplicate header-parsing implementations - Extract `BuildHeaderMap()` and `ApplyContentTypeFromHeaders()` helpers to deduplicate header-to-map conversion and Content-Type scanning - Unify the two `DoWithRetry` overloads (PayloadFile variant now delegates to the Validate variant) **Correctness fixes:** - `TransactPackage`: both phases now use `PerformWithResponseCallbacks()`, fixing missing abort support and a dead header collection loop - `TransactPackage`: error path now routes through `CommonResponse`, preserving curl error codes and messages for the caller - `ValidatePayload`: merged 3 separate header-scan loops into a single pass **Performance improvements:** - Replace `fmt::format` with `ExtendableStringBuilder` in `BuildHeaderList` and `BuildUrlWithParameters`, eliminating heap allocations in the common case - Replace `curl_easy_escape`/`curl_free` with inline URL percent-encoding using `AsciiSet` - Remove wasteful `CommonResponse(...)` construction in retry logging, formatting directly from `CurlResult` fields
* | Add session log ingestion, dashboard visualization, and log forwarding sinkStefan Boberg3 days11-3/+689
| | | | | | | | | | | | | | | | | | | | - Add POST/GET endpoints for session logs (/sessions/{id}/log) supporting raw text (newline-split) and structured JSON/CbObject with batch "entries" array - Add in-memory log storage per session (capped at 10k entries) with pagination - Add log panel to sessions dashboard with polling, auto-scroll, and level coloring - Add SessionLogSink that batches log messages on a background thread and forwards them to the sessions endpoint via HTTP, integrated into ZenStorageServer when a remote sessions URL is configured
* | Add object store dashboard pageStefan Boberg3 days6-0/+361
| | | | | | | | | | | | | | | | - Add ListBuckets endpoint (GET /obj/) returning bucket names, object counts, sizes, and total bytes served - New objectstore.js dashboard page with summary stats tiles and buckets table with click-to-expand inline object listing - Conditionally visible in nav when object store service is enabled
* | Add SessionsServiceClient and enhanced sessions dashboardStefan Boberg3 days14-68/+827
|/ | | | | | | | | | | | | - Add SessionsServiceClient in zenutil for announcing sessions to a remote zenserver (POST/PUT/DELETE lifecycle with 15s heartbeat timer) - Storage server registers itself into its own local sessions service - Add session mode attribute coupled to server mode (Compute, Proxy, etc) - Track ended sessions with ended_at timestamp and status filtering - Sessions dashboard: master-detail layout with selectable rows, Active/Ended/All tabs, metadata panel, live WebSocket updates, paging, abbreviated date formatting, and "this" pill for self session - Accept JSON payloads on sessions POST endpoint via ReadPayloadObject - Add --sessions-url config option for remote session announcement
* Add --no-network option (#831)Stefan Boberg4 days11-29/+101
| | | | - Add `--no-network` CLI option which disables all TCP/HTTPS listeners, restricting zenserver to Unix domain socket communication only. - Also fixes asio upgrade breakage on main
* upgrade asio from 1.29.0 to 1.38.0 (#827)Stefan Boberg4 days6-41/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Migrate removed deprecated APIs: - io_service -> io_context - io_service::work -> executor_work_guard - resolver::query/iterator -> resolver::resolve() with results_type - address::from_string() -> make_address() --- Breaking Changes (1.33.0) - deferred as default completion token — can omit token in coroutines: co_await socket.async_read_some(buf) - cancel_after / cancel_at — timeout any async operation: co_await sock.async_read_some(buf, cancel_after(5s)) - Partial completion token adapters — as_tuple, redirect_error, bind_executor etc. composable via pipe: co_await (async_write(sock, buf) | as_tuple | cancel_after(10s)) - composed — simpler alternative to async_compose for stateful operation implementations - co_composed moved out of experimental 1.35.0 — Allocator & Resolver - Allocator constructors for io_context and thread_pool — control memory allocation for services, I/O objects, strands - Configurable resolver thread pool ("resolver"/"threads") - Timer heap pre-allocation ("timer"/"heap_reserve") 1.37.0 — Inline Executors & Reactor Tuning - inline_executor — always executes inline (useful as completion executor) - inline_or_executor<> — tries inline first, falls back to wrapped executor - New dispatch/post/defer overloads that run a function on one executor and deliver result to a handler on another - redirect_disposition — captures disposition into a variable (like redirect_error but generic) - Reactor config: reset_edge_on_partial_read, use_eventfd, use_timerfd Notable Fixes - Resource leak in awaitable move assignment (1.37.0) - Memory leak in SSL stream move assignment (1.37.0) - Thread sanitizer issue in kqueue reactor (1.37.0) - co_spawn non-reentrant completion handler fix (1.36.0) - Windows file append mode fix (1.32.0) - SSL engine move assignment leak (1.33.0)
* Transparent proxy mode (#823)Stefan Boberg4 days36-186/+3191
| | | | | | | | | | | | | | | | | Adds a **transparent TCP proxy mode** to zenserver (activated via `zenserver proxy`), allowing it to sit between clients and upstream Zen servers to inspect and monitor HTTP/1.x traffic in real time. Primarily useful during development, to be able to observe multi-server/client interactions in one place. - **Dedicated proxy port** -- Proxy mode defaults to port 8118 with its own data directory to avoid collisions with a normal zenserver instance. - **TCP proxy core** (`src/zenserver/proxy/`) -- A new transparent TCP proxy that forwards connections to upstream targets, with support for both TCP/IP and Unix socket listeners. Multi-threaded I/O for connection handling. Supports Unix domain sockets for both upstream/downstream. - **HTTP traffic inspection** -- Parses HTTP/1.x request/response streams inline to extract method, path, status, content length, and WebSocket upgrades without breaking the proxied data. - **Proxy dashboard** -- A web UI showing live connection stats, per-target request counts, active connections, bytes transferred, and client IP/session ID rollups. - **Server mode display** -- Dashboard banner now shows the running server mode (Zen Proxy, Zen Compute, etc.). Supporting changes included in this branch: - **Wildcard log level matching** -- Log levels can now be set per-category using wildcard patterns (e.g. `proxy.*=debug`). - **`zen down --all`** -- New flag to shut down all running zenserver instances; also used by the new `xmake kill` task. - Minor test stability fixes (flaky hash collisions, per-thread RNG seeds). - Support ZEN_MALLOC environment variable for default allocator selection and switch default to rpmalloc - Fixed sentry-native build to allow LTO on Windows
* show ETA in plain and log output style (#829)Dan Engelbrecht4 days1-10/+11
| | | * show ETA in plain and log output style
* added streaming download of payloads http client Post (#824)Dan Engelbrecht5 days10-76/+379
| | | | | | * added streaming download of payloads in cpr client ::Post * curlclient Post streaming download * case sensitivity fixes for http headers * move over missing functionality from crpclient to httpclient
* improved oplog import progress reporting (#825)Dan Engelbrecht5 days7-158/+251
|
* block scavenge of other downloads that uses an older state file (#822)Dan Engelbrecht5 days3-3/+22
|
* hub consul integration (#820)Dan Engelbrecht5 days12-44/+1200
| | | | | | | | - Feature: Basic consul integration for zenserver hub mode, restricted to host local consul agent and register/deregister of services - Feature: Added new options to zenserver hub mode - `consul-endpoint` - Consul endpoint URL for service registration (empty = disabled) - `hub-base-port-number` - Base port number for provisioned instances - `hub-instance-limit` - Maximum number of provisioned instances for this hub - `hub-use-job-object` - Enable the use of a Windows Job Object for child process management (Windows only)
* minor zenstore/blockstore fixes (#821)Stefan Boberg6 days5-83/+63
| | | | | | | | | | | - Fix clang-format error accidentally introduced by recent PR - Fix `FileSize()` CAS race that repeatedly invalidated the cache when concurrent callers both missed; remove `store(0)` on CAS failure - Fix `WriteChunks` not accounting for initial alignment padding in `m_TotalSize`, causing drift vs `WriteChunk`'s correct accounting - Fix Create retry sleep computing negative values (100 - N*100 instead of 100 + N*100), matching the Open retry pattern - Fix `~BlockStore` error log missing format placeholder for `Ex.what()` - Fix `GetFreeBlockIndex` infinite loop when all indexes have orphan files on disk but aren't in `m_ChunkBlocks`; bound probe to `m_MaxBlockCount` - Fix `IterateBlock` ignoring `SmallSizeCallback` return value for single out-of-bounds chunks, preventing early termination - Fix `BlockStoreCompactState::IterateBlocks` iterating map by value instead of const reference
* HttpClient using libcurl, Unix Sockets for HTTP. HTTPS support (#770)Stefan Boberg6 days43-532/+4881
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The main goal of this change is to eliminate the cpr back-end altogether and replace it with the curl implementation. I would expect to drop cpr as soon as we feel happy with the libcurl back-end. That would leave us with a direct dependency on libcurl only, and cpr can be eliminated as a dependency. ### HttpClient Backend Overhaul - Implemented a new **libcurl-based HttpClient** backend (`httpclientcurl.cpp`, ~2000 lines) as an alternative to the cpr-based one - Made HttpClient backend **configurable at runtime** via constructor arguments and `-httpclient=...` CLI option (for zen, zenserver, and tests) - Extended HttpClient test suite to cover multipart/content-range scenarios ### Unix Domain Socket Support - Added Unix domain socket support to **httpasio** (server side) - Added Unix domain socket support to **HttpClient** - Added Unix domain socket support to **HttpWsClient** (WebSocket client) - Templatized `HttpServerConnectionT<SocketType>` and `WsAsioConnectionT<SocketType>` to handle TCP, Unix, and SSL sockets uniformly via `if constexpr` dispatch ### HTTPS Support - Added **preliminary HTTPS support to httpasio** (for Mac/Linux via OpenSSL) - Added **basic HTTPS support for http.sys** (Windows) - Implemented HTTPS test for httpasio - Split `InitializeServer` into smaller sub-functions for http.sys ### Other Notable Changes - Improved **zenhttp-test stability** with dynamic port allocation - Enhanced port retry logic in http.sys (handles ERROR_ACCESS_DENIED) - Fatal signal/exception handlers for backtrace generation in tests - Added `zen bench http` subcommand to exercise network + HTTP client/server communication stack
* hubservice refactor (#819)Dan Engelbrecht6 days11-737/+822
| | | | | * move Hub to separate class * move StorageServerInstance to separate files * refactor HttpHubService to not own Hub instance
* Merge branch 'main' into lm/restrict-content-typeLiam Mitchell6 days10-72/+157
|\
| * Merge branch 'main' into lm/oidctoken-exe-pathLiam Mitchell6 days430-5286/+54399
| |\
| * \ Merge branch 'main' into lm/oidctoken-exe-pathLiam Mitchell6 days127-3068/+9008
| |\ \
| * | | Allow external OidcToken executable to be specified unless disabled via ↵Liam Mitchell11 days5-8/+58
| | | | | | | | | | | | | | | | command line or config
| * | | Pass command-line OidcToken option through config rather than env variables, ↵Liam Mitchell2026-01-1510-64/+59
| | | | | | | | | | | | | | | | and add lua option
| * | | Use well-known OidcToken paths or command line arguments to determine ↵Liam Mitchell2026-01-1410-73/+119
| | | | | | | | | | | | | | | | OidcToken executable path
* | | | Merge branch 'main' into lm/restrict-content-typeLiam Mitchell6 days429-5283/+54397
|\ \ \ \ | | |_|/ | |/| |
| * | | updated chunk–block analyser (#818)Dan Engelbrecht6 days5-495/+1228
| | | | | | | | | | | | | | | | | | | | * create oplogs as they are imported * Improved logic for partial block analisys * unit tests for ChunkBlockAnalyser
| * | | Dashboard overhaul, compute integration (#814)Stefan Boberg7 days80-1032/+6027
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - **Frontend dashboard overhaul**: Unified compute/main dashboards into a single shared UI. Added new pages for cache, projects, metrics, sessions, info (build/runtime config, system stats). Added live-update via WebSockets with pause control, sortable detail tables, themed styling. Refactored compute/hub/orchestrator pages into modular JS. - **HTTP server fixes and stats**: Fixed http.sys local-only fallback when default port is in use, implemented root endpoint redirect for http.sys, fixed Linux/Mac port reuse. Added /stats endpoint exposing HTTP server metrics (bytes transferred, request rates). Added WebSocket stats tracking. - **OTEL/diagnostics hardening**: Improved OTLP HTTP exporter with better error handling and resilience. Extended diagnostics services configuration. - **Session management**: Added new sessions service with HTTP endpoints for registering, updating, querying, and removing sessions. Includes session log file support. This is still WIP. - **CLI subcommand support**: Added support for commands with subcommands in the zen CLI tool, with improved command dispatch. - **Misc**: Exposed CPU usage/hostname to frontend, fixed JS compact binary float32/float64 decoding, limited projects displayed on front page to 25 sorted by last access, added vscode:// link support. Also contains some fixes from TSAN analysis.
| * | | add fallback for zencache multirange (#816)Dan Engelbrecht7 days23-1437/+2426
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * clean up BuildStorageResolveResult to allow capabilities * add check for multirange request capability * add MaxRangeCountPerRequest capabilities * project export tests * add InMemoryBuildStorageCache * progress and logging improvements * fix ElapsedSeconds calculations in fileremoteprojectstore.cpp * oplogs/builds test script
| * | | added auto-detection logic for console colour output (#817)Stefan Boberg7 days2-5/+107
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add auto-detection of colour support to `AnsicolourStdoutSink`. **New `colorMode` enum** (`On`, `Off`, `Auto`) added to the header, accepted by the `AnsicolorStdoutSink` constructor. Defaults to `Auto`, so all existing call sites are unaffected. **`Auto` mode detection logic** (in `IscolourTerminal()`): 1. **TTY check** -- if stdout is not a terminal, colour is disabled. 2. **`NO_COLOR`** -- respects the no-colour.org convention. If set, colour is disabled. 3. **`COLORTERM`** -- if set (e.g. `truecolour`, `24bit`), colour is enabled. 4. **`TERM`** -- rejects `dumb`; accepts known colour-capable terminals via substring match: `alacritty`, `ansi`, `colour`, `console`, `cygwin`, `gnome`, `konsole`, `kterm`, `linux`, `msys`, `putty`, `rxvt`, `screen`, `tmux`, `vt100`, `vt102`, `xterm`. Substring matching covers variants like `xterm-256color` and `rxvt-unicode`. 5. **Fallback** -- Windows defaults to colour enabled (modern console supports ANSI natively); other platforms default to disabled. When colour is disabled, ANSI escape sequences are omitted entirely from the output. NOTE: this doesn't currently apply to all paths which do logging in zen as they may be determining their colour output mode separately from `AnsicolorStdoutSink`.
| * | | Eliminate spdlog dependency (#773)Stefan Boberg7 days61-1025/+2297
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Removes the vendored spdlog library (~12,000 lines) and replaces it with a purpose-built logging system in zencore (~1,800 lines). The new implementation provides the same functionality with fewer abstractions, no shared_ptr overhead, and full control over the logging pipeline. ### What changed **New logging core in zencore/logging/:** - LogMessage, Formatter, Sink, Logger, Registry - core abstractions matching spdlog's model but simplified - AnsiColorStdoutSink - ANSI color console output (replaces spdlog stdout_color_sink) - MsvcSink - OutputDebugString on Windows (replaces spdlog msvc_sink) - AsyncSink - async logging via BlockingQueue worker thread (replaces spdlog async_logger) - NullSink, MessageOnlyFormatter - utility types - Thread-safe timestamp caching in formatters using RwLock **Moved to zenutil/logging/:** - FullFormatter - full log formatting with timestamp, logger name, level, source location, multiline alignment - JsonFormatter - structured JSON log output - RotatingFileSink - rotating file sink with atomic size tracking **API changes:** - Log levels are now an enum (LogLevel) instead of int, eliminating the zen::logging::level namespace - LoggerRef no longer wraps shared_ptr - it holds a raw pointer with the registry owning lifetime - Logger error handler is wired through Registry and propagated to all loggers on registration - Logger::Log() now populates ThreadId on every message **Cleanup:** - Deleted thirdparty/spdlog/ entirely (110+ files) - Deleted full_test_formatter (was ~80% duplicate of FullFormatter) - Renamed snake_case classes to PascalCase (full_formatter -> FullFormatter, json_formatter -> JsonFormatter, sentry_sink -> SentrySink) - Removed spdlog from xmake dependency graph ### Build / test impact - zencore no longer depends on spdlog - zenutil and zenvfs xmake.lua updated to drop spdlog dep - zentelemetry xmake.lua updated to drop spdlog dep - All existing tests pass, no test changes required beyond formatter class renames
| * | | zenstore bug-fixes from static analysis pass (#815)Stefan Boberg10 days25-138/+144
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | **Bug fixes across zenstore, zenremotestore, and related subsystems, primarily surfaced by static analysis.** ## Cache subsystem (cachedisklayer.cpp) - Fixed tombstone scoping bug: tombstone flag and missing entry were recorded outside the block where data was removed, causing non-missing entries to be incorrectly tombstoned - Fixed use-after-overwrite: `RemoveMemCachedData`/`RemoveMetaData` were called after `Payload` was overwritten on cache put, leaking stale data - Fixed incorrect retry sleep formula (`100 - (3 - RetriesLeft) * 100` always produced the same or negative value; corrected to `(3 - RetriesLeft) * 100`) - Fixed broken `break` missing from sidecar file read loop, causing reads past valid data - Fixed missing format argument in three `ZEN_WARN`/`ZEN_ERROR` log calls (format string had `{}` placeholders with no corresponding argument, or vice versa) - Fixed elapsed timer being accumulated inside the wrong scope in `HandleRpcGetCacheRecords` - Fixed test asserting against unserialized `RecordPolicy` instead of the deserialized `Loaded` copy - Initialized `AbortFlag`/`PauseFlag` atomics at declaration (UB if read before first write) ## Build store (buildstore.cpp / buildstore.h) - Fixed wrong variable used in warning log: used loop index `ResultIndex` instead of `Index`/`MetaLocationResultIndexes[Index]`, logging wrong hash values - Fixed `sizeof(AccessTimesHeader)` used instead of `sizeof(AccessTimeRecord)` when advancing write offset, corrupting the access times file if the sizes differ - Initialized `m_LastAccessTimeUpdateCount` atomic member (was uninitialized) - Changed map iteration loops to use `const auto&` to avoid unnecessary copies ## Project store (projectstore.cpp / projectstore.h) - Fixed wrong iterator dereferenced in `IterateChunks`: used `ChunkIt->second` (from a different map lookup) instead of `MetaIt->second` - Fixed wrong assert variable: `Sizes[Index]` should be `RawSizes[Index]` - Fixed `MakeTombstone`/`IsTombstone` inconsistency: `MakeTombstone` was zeroing `OpLsn` but `IsTombstone` checks `OpLsn.Number != 0`; tombstone creation now preserves `OpLsn` - Fixed uninitialized `InvalidEntries` counter - Fixed format string mismatch in warning log - Initialized `AbortFlag`/`PauseFlag` atomics; changed map iteration to `const auto&` ## Workspaces (workspaces.cpp) - Fixed missing alias registration when a workspace share is updated: alias was deleted but never re-inserted - Fixed integer overflow in range clamping: `(RequestedOffset + RequestedSize) > Size` could wrap; corrected to `RequestedSize > Size - RequestedOffset` - Changed map iteration loops to `const auto&` ## CAS subsystem (cas.cpp, caslog.cpp, compactcas.cpp, filecas.cpp) - Fixed `IterateChunks` passing original `Payload` buffer instead of the modified `Chunk` buffer (content type was set on the copy but the original was sent to the callback) - Fixed invalid `std::future::get()` call on default-constructed futures - Fixed sign-comparison in `CasLogFile::Replay` loop (`int i` vs `size_t`) - Changed `CasLogFile::IsValid` and `Open` to take `const std::filesystem::path&` instead of by value - Fixed format string in `~CasContainerStrategy` error log ## Remote store (zenremotestore) - Fixed `FolderContent::operator==` always returning true: loop variable `PathCount` was initialized to 0 instead of `Paths.size()` - Fixed `GetChunkIndexForRawHash` looking up from wrong map (`RawHashToSequenceIndex` instead of `ChunkHashToChunkIndex`) - Fixed double-counted `UniqueSequencesFound` stat (incremented in both branches of an if/else) - Fixed `RawSize` sentinel value truncation: `(uint32_t)-1` assigned to a `uint64_t` field; corrected to `(uint64_t)-1` - Initialized uninitialized atomic and struct members across `buildstorageoperations.h`, `chunkblock.h`, and `remoteprojectstore.h`
| * | | Claude config, some bug fixes (#813)Stefan Boberg10 days35-128/+208
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Claude config updates * Bug fixes and hardening across `zencore` and `zenhttp`, identified via static analysis. ### zencore - **`ZEN_ASSERT` macro** -- extended to accept an optional string message literal; added `ZEN_ASSERT_MSG_` helper for message formatting. Callers needing runtime fmt-style formatting should use `ZEN_ASSERT_FORMAT`. - **`MpscQueue`** -- fixed `TypeCompatibleStorage` to use a properly-sized `char Storage[sizeof(T)]` array instead of a single `char`; corrected `Data()` to cast `&Storage` rather than `this`; switched cache-line alignment to a fixed constant to avoid GCC's `-Winterference-size` warning. Enabled previously-disabled tests. - **`StringBuilderImpl`** -- initialized `m_Base`/`m_CurPos`/`m_End` to `nullptr`. Fixed `StringCompare` return type (`bool` -> `int`). Fixed `ParseInt` to reject strings with trailing non-numeric characters. Removed deprecated `<codecvt>` include. - **`NiceNumGeneral`** -- replaced `powl()` with integer `IntPow()` to avoid floating-point precision issues. - **`RwLock::ExclusiveLockScope`** -- added move constructor/assignment; initialized `m_Lock` to `nullptr`. - **`Latch::AddCount`** -- fixed variable type (`std::atomic_ptrdiff_t` -> `std::ptrdiff_t` for the return value of `fetch_add`). - **`thread.cpp`** -- fixed Linux `pthread_setname_np` 16-byte name truncation; added null check before dereferencing in `Event::Close()`; fixed `NamedEvent::Close()` to call `close(Fd)` outside the lock region; added null guard in `NamedMutex` destructor; `Sleep()` now returns early for non-positive durations. - **`MD5Stream`** -- was entirely stubbed out (no-op); now correctly calls `MD5Init`/`MD5Update`/`MD5Final`. Fixed `ToHexString` to use the correct string length. Fixed forward declarations. Fixed tests to compare `compare() == 0`. - **`sentryintegration.cpp`** -- guard against null `filename`/`funcname` in spdlog message handler to prevent a crash in `fmt::format`. - **`jobqueue.cpp`** -- fixed lost job ID when `IdGenerator` wraps around zero; fixed raw `Job*` in `RunningJobs` map (potential use-after-free) to `RefPtr<Job>`; fixed range-loop copies; fixed format string typo. - **`trace.cpp`** -- suppress GCC false-positive warnings in third-party `trace.h` include. ### zenhttp - **WebSocket close race** (`wsasio`, `wshttpsys`, `httpwsclient`) -- `m_CloseSent` promoted from `bool` to `std::atomic<bool>`; close check changed to `exchange(true)` to eliminate the check-then-set data race. - **`wsframecodec.cpp`** -- reject WebSocket frames with payload > 256 MB to prevent OOM from malformed/malicious frames. - **`oidc.cpp`** -- URL-encode refresh token and client ID in token requests (`FormUrlEncode`); parse `end_session_endpoint` and `device_authorization_endpoint` from OIDC discovery document. - **`httpclientcommon.cpp`** -- propagate error code from `AppendData` when flushing the cache buffer. - **`httpclient.h`** -- initialize all uninitialized members (`ErrorCode`, `UploadedBytes`, `DownloadedBytes`, `ElapsedSeconds`, `MultipartBoundary` fields). - **`httpserver.h`** -- fix `operator=` return type for `HttpRpcHandler` (missing `&`). - **`packageformat.h`** -- fix `~0u` (32-bit truncation) to `~uint64_t(0)` for a `uint64_t` field. - **`httpparser`** -- initialize `m_RequestVerb` in both declaration and `ResetState()`. - **`httpplugin.cpp`** -- initialize `m_BasePort`; fix format string missing quotes around connection name. - **`httptracer.h`** -- move `#pragma once` before includes. - **`websocket.h`** -- initialize `WebSocketMessage::Opcode`. ### zenserver - **`hubservice.cpp`** -- fix two `ZEN_ASSERT` calls that incorrectly used fmt-style format args; converted to `ZEN_ASSERT_FORMAT`.
| * | | fix oidctoken exe lookup check (#811)Dan Engelbrecht10 days1-1/+1
| | | |
| * | | oidctoken tool package (#810)Stefan Boberg11 days1-0/+11
| | | | | | | | | | | | | | | | | | | | * added OidcToken binary to the build process. The binary is mirrored from p4 and is placed next to the output of the build process. It is also placed in the release zip archives. * also fixed issue with Linux symbol stripping which was introduced in toolchain changes yesterday
| * | | added TEST_SUITE_BEGIN/END around some TEST_CASEs which didn't have them (#809)Stefan Boberg11 days9-57/+180
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * added TEST_SUITE_BEGIN/END around some TEST_CASEs which didn't have them * fixed some stats issues * ScopedSpan should Initialize * annotated classes in stats.h with some documentation comments
| * | | Fixing various compiler issues (#807)Stefan Boberg12 days4-6/+19
| | | | | | | | | | | | Compile fixes for various versions of gcc,clang (non-UE)
| * | | more feedback during auth option parsing (#806)Dan Engelbrecht12 days4-41/+99
| | | | | | | | | | | | | | | | | | | | * remove stray std::unique_ptr<AuthMgr> Auth; causing crashes * add more feedback during parsing of auth options
| * | | compute orchestration (#763)Stefan Boberg12 days118-2886/+24246
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Added local process runners for Linux/Wine, Mac with some sandboxing support - Horde & Nomad provisioning for development and testing - Client session queues with lifecycle management (active/draining/cancelled), automatic retry with configurable limits, and manual reschedule API - Improved web UI for orchestrator, compute, and hub dashboards with WebSocket push updates - Some security hardening - Improved scalability and `zen exec` command Still experimental - compute support is disabled by default
| * | | IterateChunks callback is multithreaded - make sure AttachmentsSize can ↵Dan Engelbrecht12 days1-2/+2
| | | | | | | | | | | | | | | | handle it (#804)
| * | | HTTP improvements (#803)Stefan Boberg12 days8-19/+99
| | | | | | | | | | | | | | | | | | | | - Add GetTotalBytesReceived/GetTotalBytesSent to HttpServer with implementations in ASIO and http.sys backends - Add ExpectedErrorCodes to HttpClientSettings to suppress warn/info logs for anticipated HTTP error codes - Also fixes minor issues in `CprHttpClient::Download`
| * | | unity build fixes (#802)Stefan Boberg12 days13-19/+55
| | | | | | | | | | | | | | | | | | | | Various fixes to make cpp files build in unity build mode as an aside using Unity build doesn't really seem to work on Linux, unsure why but it leads to link-time issues
| * | | use multi range requests (#800)Dan Engelbrecht13 days27-722/+1582
| | | | | | | | | | | | | | | | | | | | | | | | | | | | - Improvement: `zen builds download` now uses multi-range requests for blocks to reduce download size - Improvement: `zen oplog-import` now uses partial block with multi-range requests for blocks to reduce download size - Improvement: Improved feedback in log/console during `zen oplog-import` - Improvement: `--allow-partial-block-requests` now defaults to `true` for `zen builds download` and `zen oplog-import` (was `mixed`) - Improvement: Improved range merging analysis when downloading partial blocks
| * | | fix objectstore uri path parsing (#801)Dan Engelbrecht13 days2-14/+147
| | | | | | | | | | | | | | | | * add objectstore tests * in http router, for last matcher, test if it matches the remaining part of the uri
| * | | Add test suites (#799)Stefan Boberg14 days71-23/+311
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Makes all test cases part of a test suite. Test suites are named after the module and the name of the file containing the implementation of the test. * This allows for better and more predictable filtering of which test cases to run which should also be able to reduce the time CI spends in tests since it can filter on the tests for that particular module. Also improves `xmake test` behaviour: * instead of an explicit list of projects just enumerate the test projects which are available based on build system state * also introduces logic to avoid running `xmake config` unnecessarily which would invalidate the existing build and do lots of unnecessary work since dependencies were invalidated by the updated config * also invokes build only for the chosen test targets As a bonus, also adds `xmake sln --open` which allows opening IDE after generation of solution/xmake project is done.
| * | | added `--verbose` option to zenserver-test and `xmake test` (#798)Stefan Boberg2026-03-0118-270/+228
| | | | | | | | | | | | | | | | | | | | | | | | * when `--verbose` is specified to zenserver-test, all child process output (typically, zenserver instances) is piped through to stdout. you can also pass `--verbose` to `xmake test` to accomplish the same thing. * this PR also consolidates all test runner `main` function logic (such as from zencore-test, zenhttp-test etc) into central implementation in zencore for consistency and ease of maintenance * also added extended utf8-tests including a fix to `Utf8ToWide()`
| * | | subprocess tracking using Jobs on Windows/hub (#796)Stefan Boberg2026-02-286-8/+199
| | | | | | | | | | | | | | | | | | | | This change introduces job object support on Windows to be able to more accurately track and limit resource usage on storage instances created by the hub service. It also ensures that all child instances can be torn down reliably on exit. Also made it so hub tests no longer pop up console windows while running.
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-16 08:30:28 +0000