aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* Propagate errors from FunctionRunner::RegisterWorkersb/compute-batchStefan Boberg5 hours7-19/+36
| | | | | | | | | | | | | | Change RegisterWorker return type from void to [[nodiscard]] bool across the FunctionRunner hierarchy so callers can detect registration failures at the point of registration rather than at action execution time. - FunctionRunner pure virtual: void -> [[nodiscard]] bool - LocalProcessRunner: always returns true (debug dump only) - RemoteHttpRunner: returns false on the three HTTP error paths (removes the TODO comments), true on success/already-known - BaseRunnerGroup: returns true only if all runners succeeded - Callers in computeservice.cpp log warnings on failure but continue best-effort
* updated zencompute CLAUDE.mdStefan Boberg5 hours1-5/+34
|
* Subscribe compute session to orchestrator WebSocket for instant worker discoveryStefan Boberg5 hours7-42/+184
| | | | | | | | | | | | | | | | Instead of relying solely on the 5-second polling throttle in UpdateCoordinatorState(), the compute session now connects a WebSocket client to /orch/ws. When the orchestrator broadcasts a state change the handler calls NotifyOrchestratorChanged() which bypasses the throttle and wakes the scheduler thread immediately. The MockOrchestratorService in tests gains IWebSocketHandler support and broadcasts a poke on SetWorkerList(), and all nine Sleep(7'000) calls are replaced with NotifyOrchestratorChanged() + Sleep(2'000), saving ~45s from the remote-function test suite. Also extracts the repeated HTTP→WS scheme-conversion pattern into a shared HttpToWsUrl() helper in zenhttp.
* Allow using websocket messaging for faster remote completion updatesStefan Boberg6 hours7-18/+366
|
* Add convenience helpers to ComputeServiceSessionStefan Boberg7 hours3-31/+33
| | | | | Add Ready(), Abandon(), and SetOrchestrator() inline helpers to reduce verbosity at call sites, especially in tests.
* Implement cancellation in RemoteHttpRunner::Shutdown()Stefan Boberg7 hours3-3/+180
| | | | | | | | | | | | | | | Previously Shutdown() only stopped the monitor thread without cleaning up remote state. Now it: 1. Stops accepting new work (m_AcceptNewActions flag guards QueryCapacity, SubmitAction, and SubmitActionBatch) 2. Cancels all known remote queues via HTTP DELETE 3. Stops the monitor thread 4. Drains m_RemoteRunningMap and marks remaining actions as Failed so the scheduler can reschedule or finalize them Adds two integration tests exercising the new behavior: - shutdown_cancels_queues: verifies remote queues are cancelled on shutdown - shutdown_rejects_new_work: verifies submissions are rejected after shutdown
* Merge branch 'main' into sb/compute-batchStefan Boberg10 hours3-9/+24
|\
| * add sanitizer options to xmake (#847)v5.7.23-pre1v5.7.23-pre0Dan Engelbrecht15 hours3-9/+24
| | | | | | | | | | | | - Improvement: Add easy access options for sanitizers with `xmake config` and `xmake test` as options - `--msan=[y|n]` Enable MemorySanitizer (Linux only, requires all deps instrumented) - `--asan=[y|n]` Enable AddressSanitizer (disables mimalloc and sentry) - `--tsan=[y|n]` Enable ThreadSanitizer (Linux/Mac only)
* | Fix two potential SIGSEGV paths during early startupStefan Boberg11 hours2-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | - sentryintegration.cpp: sentry_options_new() can return nullptr on OOM; all subsequent sentry_options_set_* calls would immediately SIGSEGV. Guard with an early return that sets m_SentryErrorCode = -1 so Close() correctly skips sentry_close(). - zenserver.cpp: ZenServerState::Register() returns nullptr if the shared memory entry table is full. The subsequent Entry->SignalHasInstanceInfo(), Entry->SignalNoNetwork(), Entry->AddSponsorProcess(), and DoRun(Entry) calls would all SIGSEGV. Throw a descriptive runtime_error instead, which is caught by the existing catch handlers in Run().
* | Fix silent exit when second zenserver instance detects port conflictStefan Boberg11 hours2-42/+50
| | | | | | | | | | | | | | | | | | ZEN_WARN/INFO/CRITICAL calls before InitializeLogging() are silently discarded. Switch all log calls in ZenServerMain::Run() that precede InitializeLogging() — the ZenServerState early-exit block, the lock file retry block, and the catch handlers — to their ZEN_CONSOLE_* equivalents. Also fix the same issue in ZenServerState::Sweep() and Snapshot() which had been inconsistently converted to fprintf+std::format.
* | Allow parent options to appear after subcommand nameStefan Boberg14 hours4-41/+123
| | | | | | | | | | | | | | | | Parse subcommand args permissively with allow_unrecognised_options(), then forward unmatched tokens back to the parent parser which remains strict. This lets commands like `zen exec http --path d:\recording` work where --path is a parent (exec) option placed after the subcommand name.
* | Add test suite banners to test listener outputStefan Boberg16 hours1-2/+12
| |
* | Fix --test-suite filter being ignored due to accumulation with defaultStefan Boberg16 hours3-16/+28
| | | | | | | | | | | | | | | | | | | | The default suite filter (e.g. "core.*") was set via setOption before applyCommandLine, but doctest appends filters rather than replacing, so an explicit --test-suite=core.xxhash would match against both "core.*" and "core.xxhash", with the wildcard winning. Fold the default into ApplyCommandLine so it is only applied when no explicit --test-suite/--ts is present on the command line.
* | Fix multiline log message alignment when ANSI colors are enabledStefan Boberg17 hours2-3/+41
| | | | | | | | | | | | The line-prefix width calculation included ANSI color escape bytes, causing continuation lines to be over-indented by the size of the color sequences. Subtract ANSI byte count to get the visual width.
* | Fail on unrecognized zenserver mode instead of silently defaulting to storeStefan Boberg17 hours1-0/+5
| |
* | Add Retracted state to RunnerAction for retry-free reschedulingStefan Boberg17 hours6-13/+325
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Retracted is an explicit, instigator-initiated request to pull an action back and reschedule it on a different runner (e.g. capacity opened up elsewhere). Unlike Failed/Abandoned auto-retry, rescheduling from Retracted does not increment RetryCount since nothing went wrong. - Add Retracted enum value after Cancelled with static_assert guarding ordinal placement so runner-side transitions cannot override it - Implement idempotent RetractAction() CAS method on RunnerAction - Extend ResetActionStateToPending() to accept Retracted without incrementing RetryCount - Add RetractAction() to ComputeServiceSession with pending/running map lookup and runner cancellation for running actions - Handle Retracted in HandleActionUpdates() scheduler loop (remove from active maps, reset to Pending, no history/results entry) - Add POST jobs/{lsn}/retract and queues/{queueref}/jobs/{lsn}/retract HTTP endpoints - Bump ActionHistoryEntry::Timestamps array from [8] to [9] - Add state machine documentation and per-state comments to RunnerAction - Add tests: retract_pending, retract_not_terminal, retract_http
* | Fix queue ActiveCount race in HandleActionUpdates terminal pathStefan Boberg18 hours1-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | NotifyQueueActionComplete (which decrements ActiveCount) was called after releasing m_ResultsLock. GetActionResult acquires m_ResultsLock to consume the result, so a caller could observe ActiveCount still at 1 immediately after GetActionResult returned OK if the scheduler thread was preempted between releasing m_ResultsLock and reaching NotifyQueueActionComplete. Fix by calling NotifyQueueActionComplete before the m_ResultsLock block that publishes the result into m_ResultsMap. This guarantees that by the time GetActionResult can return OK, the queue counters are already updated.
* | Clarify StripAnsiSgrSequences comment to note CSI non-SGR sequences are not ↵Stefan Boberg18 hours1-0/+2
| | | | | | | | stripped
* | Fix race in HandleActionUpdates causing abandon_pending test to flakeStefan Boberg18 hours1-1/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | AbandonAllActions() scans m_PendingActions to mark actions as Abandoned, but EnqueueAction posts actions to m_UpdatedActions first — the scheduler inserts them into m_PendingActions on its next tick. If the session transitions to Abandoned in that window, AbandonAllActions() sees an empty m_PendingActions and the actions are later inserted as Pending with no one left to abandon them, causing GetActionResult to return 202 indefinitely. Fix: in HandleActionUpdates, when processing a Pending-state action, check if the session is already Abandoned and if so call SetActionState(Abandoned) immediately rather than inserting into the pending map. SetActionState calls PostUpdate internally, so the action re-enters m_UpdatedActions as Abandoned and flows into m_ResultsMap on the next scheduler pass.
* | made function.session.abandon_pending more robustStefan Boberg31 hours1-5/+15
| |
* | Extract ExecSessionRunner class from ExecUsingSessionStefan Boberg31 hours4-936/+1131
| | | | | | | | | | | | | | | | | | | | | | | | | | - Move ~920-line ExecUsingSession method into a dedicated ExecSessionRunner class with explicit per-run state and focused methods (DrainCompletedJobs, SaveResultOutput, SaveActionOutput, WriteSummaryFiles, etc.) - Bundle read-only config into ExecSessionConfig struct, both defined in the .cpp since no external clients exist - Promote FunctionDefinition to namespace-scope ExecFunctionDefinition - Move EscapeHtml/EscapeJson from lambdas to static functions in anonymous namespace - Replace monolithic ExecCommand with subcommand-based architecture using ZenCmdWithSubCommands (http, inproc, beacon, dump, buildlog) - Subcommands call ExecCommand::RunSession() which constructs config + runner
* | Move RotatingFileSink and JsonFormatter to pimpl with .cpp filesStefan Boberg32 hours6-364/+463
| | | | | | | | | | | | | | - Convert RotatingFileSink from header-only to pimpl with rotatingfilesink.cpp - Convert JsonFormatter from header-only to pimpl with jsonformatter.cpp - Strip ANSI SGR sequences in JsonFormatter's WriteEscapedString - Move fullformatter.cpp and logging.cpp into src/zenutil/logging/ subdirectory
* | Refactor logging color handling and formatter architectureStefan Boberg33 hours9-346/+424
| | | | | | | | | | | | | | | | | | | | - Move ANSI color emission from sink into formatters via Formatter::SetColorEnabled() - Remove ColorRangeStart/End from LogMessage — no longer needed - Extract color helpers (AnsiColorForLevel, StripAnsiSgrSequences) into helpers.h - Extract IsColorTerminal() and ResolveColorMode() as public free functions - Strip ANSI SGR sequences in RotatingFileSink to keep log files clean - Move FullFormatter from header-only to pimpl with fullformatter.cpp - Make AnsiColorStdoutSink::m_Dirty atomic for lock-free Flush() fast path
* | Strip upstream ANSI color escapes in non-color output modeStefan Boberg33 hours1-5/+43
| | | | | | | | | | | | When color is disabled, strip any SGR escape sequences (\033[...m) from the formatted log line before writing. This prevents upstream application code from leaking raw escape sequences into non-terminal output.
* | Improve AnsiColorStdoutSink write efficiencyStefan Boberg33 hours1-21/+44
| | | | | | | | | | | | | | | | Build the complete log line (with ANSI escapes) in memory and emit it in a single write call instead of multiple fwrite calls. On Windows, use WriteFile directly to bypass C runtime buffering. Remove per-message fflush and add a dirty flag so Flush() is a no-op when nothing was written. Replace std::mutex with RwLock for codebase consistency.
* | Show host details when discovering new compute workersStefan Boberg34 hours1-1/+18
| | | | | | | | | | Log hostname, platform, CPU count and total memory alongside the worker URI so operators can identify machines at a glance.
* | Move frontend html.zip from source tree into build directoryStefan Boberg34 hours1-2/+4
| | | | | | | | | | | | The generated zip is a build artifact, not a source file. Generate it into $(buildir)/frontend/ instead of src/zenserver/frontend/ so it no longer pollutes the source tree.
* | suppress frontend zip stdout spamStefan Boberg34 hours1-2/+2
| |
* | Extended ConsoleTui with some more functionalityStefan Boberg36 hours2-0/+93
| |
* | clean up code which triggers gcc false positiveStefan Boberg36 hours1-2/+4
| |
* | Batch action submission for compute serviceStefan Boberg36 hours7-450/+503
|/ | | | | | | - Consolidate duplicated action submission logic in httpcomputeservice into a single HandleSubmitAction method supporting both single-action and batch (actions array) payloads - Group actions by queue in RemoteHttpRunner and submit as batches with configurable chunk size, falling back to individual submission on failure - Extract shared helpers in computeservice: MakeErrorResult, ValidateQueueForEnqueue, ActivateActionInQueue, RemoveActionFromActiveMaps - Add WriteCompactBinaryObject to zencore
* revise oplog block arrangement (#842)Dan Engelbrecht37 hours7-2473/+4918
| | | | | - Improvement: Fixed issue where oplog upload could create blocks larger than the max limit (64Mb) Refactored remoteprojectstore.cpp to use ParallelWork and exceptions for error handling.
* Restructure zen builds using subcommands (#834)Stefan Boberg42 hours2-2284/+2647
| | | Refactored builds_cmd to split subcommands into dedicated classes, in an effort to reduce surface area and complexity to improve maintainability.
* Enable cross compilation of Windows targets on Linux (#839)Stefan Boberg43 hours18-27/+57
| | | | | | | This PR makes it *possible* to do a Windows build on Linux via `clang-cl`. It doesn't actually change any build process. No policy change, just mechanics and some code fixes to clear clang compilation. The code fixes are mainly related to #include file name casing, to match the on-disk casing of the SDK files (via xwin).
* URI decoding, process env, compiler info, httpasio strands, regex route ↵Stefan Boberg43 hours17-381/+393
| | | | | | | | | | | | | | | | | removal (#841) - Percent-decode URIs in ASIO HTTP server to match http.sys CookedUrl behavior, ensuring consistent decoded paths across backends - Add Environment field to CreateProcOptions for passing extra env vars to child processes (Windows: merged into Unicode environment block; Unix: setenv in fork) - Add GetCompilerName() and include it in build options startup logging - Suppress Windows CRT error dialogs in test harness for headless/CI runs - Fix mimalloc package: pass CMAKE_BUILD_TYPE, skip cfuncs test for cross-compile - Add virtual destructor to SentryAssertImpl to fix debug-mode warning - Simplify object store path handling now that URIs arrive pre-decoded - Add URI decoding test coverage for percent-encoded paths and query params - Simplify httpasio request handling by using strands (guarantees no parallel handlers per connection) - Removed deprecated regex-based route matching support - Fix full GC never triggering after cross-toolchain builds: The `gc_state` file stores `system_clock` ticks, but the tick resolution differs between toolchains (nanoseconds on GCC/standard clang, microseconds on UE clang). A nanosecond timestamp misinterpreted as microseconds appears far in the future (~year 58,000), bypassing the staleness check and preventing time-based full GC from ever running. Fixed by also resetting when the stored timestamp is in the future. - Clamp GC countdown display to configured interval: Prevents nonsensical log output (e.g. "Full GC in 492128002h") caused by the above or any other clock anomaly. The clamp applies to both the scheduler log and the status API.
* block/file cloning support for macOS / Linux (#786)Stefan Boberg43 hours2-72/+523
| | | | | | | | - Add block cloning (copy-on-write) support for Linux and macOS to complement the existing Windows (ReFS) implementation - **Linux**: `TryCloneFile` via `FICLONE` ioctl, `CloneQueryInterface` with range cloning via `FICLONERANGE` (Btrfs/XFS) - **macOS**: `TryCloneFile` via `clonefile()` syscall (APFS), `SupportsBlockRefCounting` via `VOL_CAP_INT_CLONE`. `CloneQueryInterface` is not implemented as macOS lacks a sub-file range clone API - Promote `ScopedFd` to file scope for broader use in filesystem code - Add test scripts for block cloning validation on Linux (Btrfs via loopback) and macOS (APFS) - Also added test script for testing on Windows (ReFS)
* Made CPR optional, html generated at build time (#840)Stefan Boberg4 days8-22/+85
| | | | | | | - Fix potential crash on startup caused by logging macros being invoked before the logging system is initialized (null logger dereference in `ZenServerState::Sweep()`). `LoggerRef::ShouldLog` now guards against a null logger pointer. - Make CPR an optional dependency (`--zencpr` build option, enabled by default) so builds can proceed without it - Make zenvfs Windows-only (platform-specific target) - Generate the frontend zip at build time from source HTML files instead of checking in a binary blob which would accumulate with every single update
* Add clang-cl build supportStefan Boberg5 days5-74/+82
| | | | | | | | | | - Add clang-cl warning suppressions in xmake.lua matching Linux/macOS set - Guard /experimental:c11atomics with {tools="cl"} for MSVC-only - Fix long long / int64_t redefinition in string.h for clang-cl - Fix unclosed namespace in callstacktrace.cpp #else branch - Fix missing override in httpplugin.cpp - Reorder WorkerPool fields to match designated initializer order - Use INVALID_SOCKET instead of SOCKET_ERROR for SOCKET comparisons
* Unix Domain Socket auto discovery (#833)Stefan Boberg5 days28-105/+538
| | | | | | | | This PR adds end-to-end Unix domain socket (UDS) support, allowing zen CLI to discover and connect to UDS-only servers automatically. - **`unix://` URI scheme in zen CLI**: The `-u` / `--hosturl` option now accepts `unix:///path/to/socket` to connect to a zenserver via a Unix domain socket instead of TCP. - **Per-instance shared memory for extended server info**: Each zenserver instance now publishes a small shared memory section (keyed by SessionId) containing per-instance data that doesn't fit in the fixed-size ZenServerEntry -- starting with the UDS socket path. This is a 4KB pagefile-backed section on Windows (`Global\ZenInstance_{sessionid}`) and a POSIX shared memory object on Linux/Mac (`/UnrealEngineZen_{sessionid}`). - **Client-side auto-discovery of UDS servers**: `zen info`, `zen status`, etc. now automatically discover and prefer UDS connections when a server publishes a socket path. Servers running with `--no-network` (UDS-only) are no longer invisible to the CLI. - **`kNoNetwork` flag in ZenServerEntry**: Servers started with `--no-network` advertise this in their shared state entry. Clients skip TCP fallback for these servers, and display commands (`ps`, `status`, `top`) show `-` instead of a port number to indicate TCP is not available.
* Switch httpclient default back-end over to libcurl (#832)Stefan Boberg5 days6-702/+454
| | | | | | | | | | | | | | | | | | | | | | | | | | | Switches the default HTTP client to the libcurl-based backend and follows up with a series of correctness fixes and code quality improvements to `CurlHttpClient`. **Backend switch & build fixes:** - Switch default HTTP client to libcurl-based backend - Suppress `[[nodiscard]]` warning when building fmt - Miscellaneous bugfixes in HttpClient/libcurl - Pass `-y` to `xmake config` in `xmake test` task **Boilerplate reduction:** - Add `Session::SetHeaders()` for RAII ownership of `curl_slist`, eliminating manual `curl_slist_free_all` calls from every verb method - Add `Session::PerformWithResponseCallbacks()` to absorb the repeated 12-line write+header callback setup block - Extract `ParseHeaderLine()` shared helper, replacing 4 duplicate header-parsing implementations - Extract `BuildHeaderMap()` and `ApplyContentTypeFromHeaders()` helpers to deduplicate header-to-map conversion and Content-Type scanning - Unify the two `DoWithRetry` overloads (PayloadFile variant now delegates to the Validate variant) **Correctness fixes:** - `TransactPackage`: both phases now use `PerformWithResponseCallbacks()`, fixing missing abort support and a dead header collection loop - `TransactPackage`: error path now routes through `CommonResponse`, preserving curl error codes and messages for the caller - `ValidatePayload`: merged 3 separate header-scan loops into a single pass **Performance improvements:** - Replace `fmt::format` with `ExtendableStringBuilder` in `BuildHeaderList` and `BuildUrlWithParameters`, eliminating heap allocations in the common case - Replace `curl_easy_escape`/`curl_free` with inline URL percent-encoding using `AsciiSet` - Remove wasteful `CommonResponse(...)` construction in retry logging, formatting directly from `CurlResult` fields
* Add --no-network option (#831)Stefan Boberg6 days11-29/+101
| | | | - Add `--no-network` CLI option which disables all TCP/HTTPS listeners, restricting zenserver to Unix domain socket communication only. - Also fixes asio upgrade breakage on main
* upgrade asio from 1.29.0 to 1.38.0 (#827)Stefan Boberg6 days6-41/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Migrate removed deprecated APIs: - io_service -> io_context - io_service::work -> executor_work_guard - resolver::query/iterator -> resolver::resolve() with results_type - address::from_string() -> make_address() --- Breaking Changes (1.33.0) - deferred as default completion token — can omit token in coroutines: co_await socket.async_read_some(buf) - cancel_after / cancel_at — timeout any async operation: co_await sock.async_read_some(buf, cancel_after(5s)) - Partial completion token adapters — as_tuple, redirect_error, bind_executor etc. composable via pipe: co_await (async_write(sock, buf) | as_tuple | cancel_after(10s)) - composed — simpler alternative to async_compose for stateful operation implementations - co_composed moved out of experimental 1.35.0 — Allocator & Resolver - Allocator constructors for io_context and thread_pool — control memory allocation for services, I/O objects, strands - Configurable resolver thread pool ("resolver"/"threads") - Timer heap pre-allocation ("timer"/"heap_reserve") 1.37.0 — Inline Executors & Reactor Tuning - inline_executor — always executes inline (useful as completion executor) - inline_or_executor<> — tries inline first, falls back to wrapped executor - New dispatch/post/defer overloads that run a function on one executor and deliver result to a handler on another - redirect_disposition — captures disposition into a variable (like redirect_error but generic) - Reactor config: reset_edge_on_partial_read, use_eventfd, use_timerfd Notable Fixes - Resource leak in awaitable move assignment (1.37.0) - Memory leak in SSL stream move assignment (1.37.0) - Thread sanitizer issue in kqueue reactor (1.37.0) - co_spawn non-reentrant completion handler fix (1.36.0) - Windows file append mode fix (1.32.0) - SSL engine move assignment leak (1.33.0)
* Transparent proxy mode (#823)Stefan Boberg6 days36-186/+3191
| | | | | | | | | | | | | | | | | Adds a **transparent TCP proxy mode** to zenserver (activated via `zenserver proxy`), allowing it to sit between clients and upstream Zen servers to inspect and monitor HTTP/1.x traffic in real time. Primarily useful during development, to be able to observe multi-server/client interactions in one place. - **Dedicated proxy port** -- Proxy mode defaults to port 8118 with its own data directory to avoid collisions with a normal zenserver instance. - **TCP proxy core** (`src/zenserver/proxy/`) -- A new transparent TCP proxy that forwards connections to upstream targets, with support for both TCP/IP and Unix socket listeners. Multi-threaded I/O for connection handling. Supports Unix domain sockets for both upstream/downstream. - **HTTP traffic inspection** -- Parses HTTP/1.x request/response streams inline to extract method, path, status, content length, and WebSocket upgrades without breaking the proxied data. - **Proxy dashboard** -- A web UI showing live connection stats, per-target request counts, active connections, bytes transferred, and client IP/session ID rollups. - **Server mode display** -- Dashboard banner now shows the running server mode (Zen Proxy, Zen Compute, etc.). Supporting changes included in this branch: - **Wildcard log level matching** -- Log levels can now be set per-category using wildcard patterns (e.g. `proxy.*=debug`). - **`zen down --all`** -- New flag to shut down all running zenserver instances; also used by the new `xmake kill` task. - Minor test stability fixes (flaky hash collisions, per-thread RNG seeds). - Support ZEN_MALLOC environment variable for default allocator selection and switch default to rpmalloc - Fixed sentry-native build to allow LTO on Windows
* show ETA in plain and log output style (#829)Dan Engelbrecht6 days1-10/+11
| | | * show ETA in plain and log output style
* added streaming download of payloads http client Post (#824)Dan Engelbrecht7 days10-76/+379
| | | | | | * added streaming download of payloads in cpr client ::Post * curlclient Post streaming download * case sensitivity fixes for http headers * move over missing functionality from crpclient to httpclient
* improved oplog import progress reporting (#825)Dan Engelbrecht7 days7-158/+251
|
* block scavenge of other downloads that uses an older state file (#822)Dan Engelbrecht7 days3-3/+22
|
* hub consul integration (#820)Dan Engelbrecht7 days12-44/+1200
| | | | | | | | - Feature: Basic consul integration for zenserver hub mode, restricted to host local consul agent and register/deregister of services - Feature: Added new options to zenserver hub mode - `consul-endpoint` - Consul endpoint URL for service registration (empty = disabled) - `hub-base-port-number` - Base port number for provisioned instances - `hub-instance-limit` - Maximum number of provisioned instances for this hub - `hub-use-job-object` - Enable the use of a Windows Job Object for child process management (Windows only)
* minor zenstore/blockstore fixes (#821)Stefan Boberg7 days5-83/+63
| | | | | | | | | | | - Fix clang-format error accidentally introduced by recent PR - Fix `FileSize()` CAS race that repeatedly invalidated the cache when concurrent callers both missed; remove `store(0)` on CAS failure - Fix `WriteChunks` not accounting for initial alignment padding in `m_TotalSize`, causing drift vs `WriteChunk`'s correct accounting - Fix Create retry sleep computing negative values (100 - N*100 instead of 100 + N*100), matching the Open retry pattern - Fix `~BlockStore` error log missing format placeholder for `Ex.what()` - Fix `GetFreeBlockIndex` infinite loop when all indexes have orphan files on disk but aren't in `m_ChunkBlocks`; bound probe to `m_MaxBlockCount` - Fix `IterateBlock` ignoring `SmallSizeCallback` return value for single out-of-bounds chunks, preventing early termination - Fix `BlockStoreCompactState::IterateBlocks` iterating map by value instead of const reference
* HttpClient using libcurl, Unix Sockets for HTTP. HTTPS support (#770)Stefan Boberg7 days43-532/+4881
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The main goal of this change is to eliminate the cpr back-end altogether and replace it with the curl implementation. I would expect to drop cpr as soon as we feel happy with the libcurl back-end. That would leave us with a direct dependency on libcurl only, and cpr can be eliminated as a dependency. ### HttpClient Backend Overhaul - Implemented a new **libcurl-based HttpClient** backend (`httpclientcurl.cpp`, ~2000 lines) as an alternative to the cpr-based one - Made HttpClient backend **configurable at runtime** via constructor arguments and `-httpclient=...` CLI option (for zen, zenserver, and tests) - Extended HttpClient test suite to cover multipart/content-range scenarios ### Unix Domain Socket Support - Added Unix domain socket support to **httpasio** (server side) - Added Unix domain socket support to **HttpClient** - Added Unix domain socket support to **HttpWsClient** (WebSocket client) - Templatized `HttpServerConnectionT<SocketType>` and `WsAsioConnectionT<SocketType>` to handle TCP, Unix, and SSL sockets uniformly via `if constexpr` dispatch ### HTTPS Support - Added **preliminary HTTPS support to httpasio** (for Mac/Linux via OpenSSL) - Added **basic HTTPS support for http.sys** (Windows) - Implemented HTTPS test for httpasio - Split `InitializeServer` into smaller sub-functions for http.sys ### Other Notable Changes - Improved **zenhttp-test stability** with dynamic port allocation - Enhanced port retry logic in http.sys (handles ERROR_ACCESS_DENIED) - Fatal signal/exception handlers for backtrace generation in tests - Added `zen bench http` subcommand to exercise network + HTTP client/server communication stack