| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
| |
- **Replace crashpad static-libc++ patch file with `io.replace()` in `on_install`** — The old `.patch` file was fragile (trailing-whitespace stripping on Windows would silently break it). Using `io.replace()` in the xmake build script is more robust and easier to maintain.
- **Skip sentry-native `on_test` link check on Linux** — The link test requires `-lc++abi` when building with the UE clang toolchain but adding it unconditionally breaks GCC/libstdc++ builds. The zenserver build itself validates that the library is usable.
- **Add `crashpad-test.sh`** — A test script that launches a release zenserver, waits for the health endpoint, then verifies that `crashpad_handler` is running, no `sentry_init` failure was logged, and the handler has no dynamic `libc++.so.1` dependency.
- **Add Crashpad Check step to Linux release CI** — Runs `crashpad-test.sh` in the `validate` workflow for release builds to catch crashpad regressions before merge.
|
| |
|
|
|
|
|
|
|
|
|
| |
This PR improves process lifecycle handling and resilience across several areas:
- **Reclaim stale shared-memory entries instead of exiting** (`zenserver.cpp`): When a zenserver instance fails to attach as a sponsor to an existing process (e.g. because the PID was reused by an unrelated process), the server now clears the stale shared-memory entry and proceeds with normal startup instead of calling `std::exit(1)`.
- **Wait for child process exit in `Kill()` and `Terminate()` on Unix** (`process.cpp`): After sending `SIGTERM` in `Kill()`, the code now waits up to 5s for graceful shutdown (escalating to `SIGKILL` on timeout), matching the Windows behavior. `Terminate()` also waits after `SIGKILL` so the child is properly reaped and doesn't linger as a zombie clogging up the process table.
- **Fix sysctl buffer race in macOS `FindProcess`** (`process.cpp`): The macOS process enumeration now retries the `sysctl` call (up to 3 attempts with 25% buffer padding) to handle the race where the process list changes between the sizing call and the data-fetching call. Also flattens the nesting and fixes the guard/free scoping.
- **Terminate stale processes before integration tests** (`zenserver-test.cpp`, `test.lua`): The integration test runner now accepts a `--kill-stale-processes` flag (passed automatically by `test.lua`) that scans for and terminates any leftover `zenserver`, `zenserver-test`, and `zentest-appstub` processes from previous test runs, logging the executable name and PID of each. This addresses flaky test failures caused by stale processes from prior runs holding ports or other resources.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- **RAII pipe handles for child process stdout/stderr capture**: `StdoutPipeHandles` is now a proper RAII type with automatic cleanup, move semantics, and partial close support. This makes it safe to use pipes for capturing child process output without risking handle/fd leaks.
- **Optional separate stderr pipe**: `CreateProcOptions` now accepts a `StderrPipe` field so callers can capture stdout and stderr independently. When null (default), stderr shares the stdout pipe as before.
- **LogStreamListener with pluggable handler**: The TCP log stream listener accepts connections from remote processes and delivers parsed log lines through a `LogStreamHandler` interface, set dynamically via `SetHandler()`. This allows any client to receive log messages without depending on a specific console implementation.
- **TcpLogStreamSink for zen::logging**: A logging sink that forwards log messages to a `LogStreamListener` over TCP, using the native `zen::logging::Sink` infrastructure with proper thread-safe synchronization.
- **Reliable child process exit codes on Linux**: `waitpid` result handling is fixed so `ProcessHandle::GetExitCode()` returns the real exit code. `ProcessHandle::Reset()` reaps zombies directly, replacing the global `IgnoreChildSignals()` which prevented exit code collection entirely. Also fixes a TOCTOU race in `ProcessHandle::Wait()` on Linux/Mac.
- **Pipe capture test suite**: Tests covering stdout/stderr capture via pipes (both shared and separate modes), RAII cleanup, move semantics, and exit code propagation using `zentest-appstub` as the child process.
- **Service command integration tests**: Shell-based integration tests for `zen service` covering the full lifecycle (install, status, start, stop, uninstall) on all three platforms — Linux (systemd), macOS (launchd), and Windows (SCM via PowerShell).
- **Test script reorganization**: Platform-specific test scripts moved from `scripts/test_scripts/` into `scripts/test_linux/`, `test_mac/`, and `test_windows/`.
|
| |
|
|
|
|
|
|
|
|
| |
- Install a crash handler at the very top of main() in both zenserver and zen
- On Windows, uses SetUnhandledExceptionFilter with StackWalk64 for accurate
crash-site backtraces with DbgHelp symbol resolution
- On Linux/Mac, uses sigaction with async-signal-safe backtrace output
- Automatically superseded when Sentry/crashpad installs its own handlers
- Stays active for the full process lifetime if Sentry is disabled or absent
- Include .sym debug symbol files in Linux release bundle
|
| | |
|
| |
|
|
|
|
| |
- Improvement: Add easy access options for sanitizers with `xmake config` and `xmake test` as options
- `--msan=[y|n]` Enable MemorySanitizer (Linux only, requires all deps instrumented)
- `--asan=[y|n]` Enable AddressSanitizer (disables mimalloc and sentry)
- `--tsan=[y|n]` Enable ThreadSanitizer (Linux/Mac only)
|
| |
|
| |
Update Linux toochain to v26_clang-20.1.8-rockylinux8, which matches what UE is currently using.
|
| |
|
|
|
|
|
| |
This PR makes it *possible* to do a Windows build on Linux via `clang-cl`.
It doesn't actually change any build process. No policy change, just mechanics and some code fixes to clear clang compilation.
The code fixes are mainly related to #include file name casing, to match the on-disk casing of the SDK files (via xwin).
|
| |
|
|
|
|
|
|
| |
- **Sentry crashpad patch**: Make static libc++ linking conditional on `-stdlib=libc++` being active, so the patch doesn't break gcc or system clang builds that use libstdc++
- **GCC warning fix**: Suppress `-Wunused-but-set-variable` for gcc (false positive with `constinit` static locals passed by reference)
- **ASIO typo fix**: `ASIO_STANDLONE` → `ASIO_STANDALONE`
- **Toolchain verification script**: `scripts/ue_build_linux/verify_linux_toolchains.sh` for testing builds across gcc, ue-clang, clang-19 and clang-20
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
removal (#841)
- Percent-decode URIs in ASIO HTTP server to match http.sys CookedUrl behavior, ensuring consistent decoded paths across backends
- Add Environment field to CreateProcOptions for passing extra env vars to child processes (Windows: merged into Unicode environment block; Unix: setenv in fork)
- Add GetCompilerName() and include it in build options startup logging
- Suppress Windows CRT error dialogs in test harness for headless/CI runs
- Fix mimalloc package: pass CMAKE_BUILD_TYPE, skip cfuncs test for cross-compile
- Add virtual destructor to SentryAssertImpl to fix debug-mode warning
- Simplify object store path handling now that URIs arrive pre-decoded
- Add URI decoding test coverage for percent-encoded paths and query params
- Simplify httpasio request handling by using strands (guarantees no parallel handlers per connection)
- Removed deprecated regex-based route matching support
- Fix full GC never triggering after cross-toolchain builds: The `gc_state` file stores `system_clock` ticks, but the tick resolution differs between toolchains (nanoseconds on GCC/standard clang, microseconds on UE clang). A nanosecond timestamp misinterpreted as microseconds appears far in the future (~year 58,000), bypassing the staleness check and preventing time-based full GC from ever running. Fixed by also resetting when the stored timestamp is in the future.
- Clamp GC countdown display to configured interval: Prevents nonsensical log output (e.g. "Full GC in 492128002h") caused by the above or any other clock anomaly. The clamp applies to both the scheduler log and the status API.
|
| |
|
| |
Adds a Dockerfile (Ubuntu 24.04 + WineHQ) and an `xmake docker` task to build and optionally push a zenserver-compute Docker image, enabling Linux deployment of compute workers that run Windows executables via Wine.
|
| |
|
|
|
|
|
|
| |
- Add block cloning (copy-on-write) support for Linux and macOS to complement the existing Windows (ReFS) implementation
- **Linux**: `TryCloneFile` via `FICLONE` ioctl, `CloneQueryInterface` with range cloning via `FICLONERANGE` (Btrfs/XFS)
- **macOS**: `TryCloneFile` via `clonefile()` syscall (APFS), `SupportsBlockRefCounting` via `VOL_CAP_INT_CLONE`. `CloneQueryInterface` is not implemented as macOS lacks a sub-file range clone API
- Promote `ScopedFd` to file scope for broader use in filesystem code
- Add test scripts for block cloning validation on Linux (Btrfs via loopback) and macOS (APFS)
- Also added test script for testing on Windows (ReFS)
|
| |
|
|
| |
* updated test scripts
* remove some hardcoded assumptions
|
| |
|
|
|
|
|
| |
- Fix potential crash on startup caused by logging macros being invoked before the logging system is initialized (null logger dereference in `ZenServerState::Sweep()`). `LoggerRef::ShouldLog` now guards against a null logger pointer.
- Make CPR an optional dependency (`--zencpr` build option, enabled by default) so builds can proceed without it
- Make zenvfs Windows-only (platform-specific target)
- Generate the frontend zip at build time from source HTML files instead of checking in a binary blob which would accumulate with every single update
|
| | |
|
| |
|
| |
fixes issue with zip not finding the OidcToken file
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The main goal of this change is to eliminate the cpr back-end altogether and replace it with the curl implementation. I would expect to drop cpr as soon as we feel happy with the libcurl back-end. That would leave us with a direct dependency on libcurl only, and cpr can be eliminated as a dependency.
### HttpClient Backend Overhaul
- Implemented a new **libcurl-based HttpClient** backend (`httpclientcurl.cpp`, ~2000 lines)
as an alternative to the cpr-based one
- Made HttpClient backend **configurable at runtime** via constructor arguments
and `-httpclient=...` CLI option (for zen, zenserver, and tests)
- Extended HttpClient test suite to cover multipart/content-range scenarios
### Unix Domain Socket Support
- Added Unix domain socket support to **httpasio** (server side)
- Added Unix domain socket support to **HttpClient**
- Added Unix domain socket support to **HttpWsClient** (WebSocket client)
- Templatized `HttpServerConnectionT<SocketType>` and `WsAsioConnectionT<SocketType>`
to handle TCP, Unix, and SSL sockets uniformly via `if constexpr` dispatch
### HTTPS Support
- Added **preliminary HTTPS support to httpasio** (for Mac/Linux via OpenSSL)
- Added **basic HTTPS support for http.sys** (Windows)
- Implemented HTTPS test for httpasio
- Split `InitializeServer` into smaller sub-functions for http.sys
### Other Notable Changes
- Improved **zenhttp-test stability** with dynamic port allocation
- Enhanced port retry logic in http.sys (handles ERROR_ACCESS_DENIED)
- Fatal signal/exception handlers for backtrace generation in tests
- Added `zen bench http` subcommand to exercise network + HTTP client/server communication stack
|
| |
|
|
|
| |
* create oplogs as they are imported
* Improved logic for partial block analisys
* unit tests for ChunkBlockAnalyser
|
| |
|
|
|
|
|
|
|
|
|
| |
* clean up BuildStorageResolveResult to allow capabilities
* add check for multirange request capability
* add MaxRangeCountPerRequest capabilities
* project export tests
* add InMemoryBuildStorageCache
* progress and logging improvements
* fix ElapsedSeconds calculations in fileremoteprojectstore.cpp
* oplogs/builds test script
|
| |
|
|
|
| |
* added OidcToken binary to the build process. The binary is mirrored from p4 and is placed next to the output of the build process. It is also placed in the release zip archives.
* also fixed issue with Linux symbol stripping which was introduced in toolchain changes yesterday
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change is meant to provide a smoother experience when working on Linux. After this change, the toolchain setup process is now simply
```bash
$ scripts/ue_build_linux/get_ue_toolchain.sh
```
and then at config time the toolchain is automatically detected if you downloaded it to the default location or have the `UE_TOOLCHAIN_DIR` environment variable set
```bash
xmake config --mode=debug
```
Compared to the old script-based approach this configures the toolchain more precisely, avoiding leakage into unrelated build processes such as when a package manager decides to build something like Ninja locally etc.
|
| | |
|
| |
|
|
|
| |
* add docs/Deploy.md
* added ability to specify a version on the command line, via `--version`
|
| | |
|
| |
|
|
| |
Upgrade libcurl to 8.17.0 and enable native Mac CA validation via Apple SecTrust over the file-based approach which was in place previously. This should be more robust and more closely matches the behaviour of Apple's curl build and the rest of the OS.
|
| |
|
|
|
| |
* Automated more of the decisions around which options to set when using ASAN
* Also disabled Sentry by default as it's a bit annoying to have it upload crashes during development. Sentry is still automatically enabled and integrated as part of the `xmake bundle` step however so released builds will still have it.
|
| |
|
|
|
|
|
| |
* switch back to openssl for Mac (fixes cross-compilation config)
* add openssl 3.6.0 and change make target to install_dev for quicker install
* disable LTO on Mac to reduce build time on openssl3
* add mbedTLS 3.6.5 (but this is not used anywhere right now)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change removes our dependency on vcpkg for package management, in favour of bringing some code in-tree in the `thirdparty` folder as well as using the xmake build-in package management feature. For the latter, all the package definitions are maintained in the zen repo itself, in the `repo` folder.
It should now also be easier to build the project as it will no longer depend on having the right version of vcpkg installed, which has been a common problem for new people coming in to the codebase. Now you should only need xmake to build.
* Bumps xmake requirement on github runners to 2.9.9 to resolve an issue where xmake on Windows invokes cmake with `v144` toolchain which does not exist
* BLAKE3 is now in-tree at `thirdparty/blake3`
* cpr is now in-tree at `thirdparty/cpr`
* cxxopts is now in-tree at `thirdparty/cxxopts`
* fmt is now in-tree at `thirdparty/fmt`
* robin-map is now in-tree at `thirdparty/robin-map`
* ryml is now in-tree at `thirdparty/ryml`
* sol2 is now in-tree at `thirdparty/sol2`
* spdlog is now in-tree at `thirdparty/spdlog`
* utfcpp is now in-tree at `thirdparty/utfcpp`
* xmake package repo definitions is in `repo`
* implemented support for sanitizers. ASAN is supported on windows, TSAN, UBSAN, MSAN etc are supported on Linux/MacOS though I have not yet tested it extensively on MacOS
* the zencore encryption implementation also now supports using mbedTLS which is used on MacOS, though for now we still use openssl on Linux
* crashpad
* bumps libcurl to 8.11.0 (from 8.8.0) which should address a rare build upload bug
|
| |
|
|
|
| |
* remove --deep option for codesign as it is considered harmful
* sign crashpadhandler
* add verbosity and strict to codesign
|
| |
|
|
|
| |
Instead of forcing user to specify a suitable directory for the UE toolchain, we offer a default which matches what the Github runners use and which is also covered by the root `.gitignore` file
|
| |
|
|
| |
and libc++abi
|
| |
|
| |
* Restore Mac minver to 12.5
|
| |
|
|
|
| |
* remove xcode 12.1 workaround
* bump min macos version to 14.0
|
| |
|
|
| |
* upload mac/linux executables to sentry using `debug-files bundle-sources` on all platforms
* update sentry-cli to latest on windows
|
| |
|
| |
* Enabled signing on windows agents again
|
| |
|
| |
fixes rare race condition when using RPC recording for long periods of time
|
| |
|
| |
Windows executables are now signed with official cert when creating a release
|
| |
|
|
|
|
| |
- Feature: Added xmake task `updatefrontend` which updates the zip file containing the frontend html (`/src/zenserver/frontend/html.zip`)
- Improvement: The frontend html content is no longer appended at the end of the executable which prevented signing, instead it is compiled in from the `/src/zenserver/frontend/html.zip` archive
- Improvement: MacOS now does ad-hoc code signing by default when issuing `xmake bundle`, signing with proper cert is done on CI builds
|
| | |
|
| |
|
|
|
|
|
|
|
| |
* add comment about 7z usage in xmake bundle
* add ue-libcxx lib/headers and licence + tps
* update get_ue_toolchain.sh to use embedded libc++ and decouple from p4
* clearer output when falling back to zip from 7z
* update ci scripts to use ue toolchain on linux
* updated linux build README.md
* changelog
|
| |
|
|
|
| |
previously, an attempt was made to locate any installed binary but the
located path was not actually used in practice
|
| |
|
|
|
|
| |
* moved source directories into `/src`
* updated bundle.lua for new `src` path
* moved some docs, icon
* removed old test trees
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|