aboutsummaryrefslogtreecommitdiff
path: root/scripts
Commit message (Collapse)AuthorAgeFilesLines
* add sanitizer options to xmake (#847)v5.7.23-pre1v5.7.23-pre0Dan Engelbrecht10 hours1-3/+14
| | | | | | - Improvement: Add easy access options for sanitizers with `xmake config` and `xmake test` as options - `--msan=[y|n]` Enable MemorySanitizer (Linux only, requires all deps instrumented) - `--asan=[y|n]` Enable AddressSanitizer (disables mimalloc and sentry) - `--tsan=[y|n]` Enable ThreadSanitizer (Linux/Mac only)
* bump toolchain to match what's in use for UE (#846)Stefan Boberg31 hours1-1/+1
| | | Update Linux toochain to v26_clang-20.1.8-rockylinux8, which matches what UE is currently using.
* Enable cross compilation of Windows targets on Linux (#839)Stefan Boberg37 hours1-0/+305
| | | | | | | This PR makes it *possible* to do a Windows build on Linux via `clang-cl`. It doesn't actually change any build process. No policy change, just mechanics and some code fixes to clear clang compilation. The code fixes are mainly related to #include file name casing, to match the on-disk casing of the SDK files (via xwin).
* Linux build improvements (#843)Stefan Boberg37 hours1-0/+121
| | | | | | | | - **Sentry crashpad patch**: Make static libc++ linking conditional on `-stdlib=libc++` being active, so the patch doesn't break gcc or system clang builds that use libstdc++ - **GCC warning fix**: Suppress `-Wunused-but-set-variable` for gcc (false positive with `constinit` static locals passed by reference) - **ASIO typo fix**: `ASIO_STANDLONE` → `ASIO_STANDALONE` - **Toolchain verification script**: `scripts/ue_build_linux/verify_linux_toolchains.sh` for testing builds across gcc, ue-clang, clang-19 and clang-20
* URI decoding, process env, compiler info, httpasio strands, regex route ↵Stefan Boberg38 hours1-0/+401
| | | | | | | | | | | | | | | | | removal (#841) - Percent-decode URIs in ASIO HTTP server to match http.sys CookedUrl behavior, ensuring consistent decoded paths across backends - Add Environment field to CreateProcOptions for passing extra env vars to child processes (Windows: merged into Unicode environment block; Unix: setenv in fork) - Add GetCompilerName() and include it in build options startup logging - Suppress Windows CRT error dialogs in test harness for headless/CI runs - Fix mimalloc package: pass CMAKE_BUILD_TYPE, skip cfuncs test for cross-compile - Add virtual destructor to SentryAssertImpl to fix debug-mode warning - Simplify object store path handling now that URIs arrive pre-decoded - Add URI decoding test coverage for percent-encoded paths and query params - Simplify httpasio request handling by using strands (guarantees no parallel handlers per connection) - Removed deprecated regex-based route matching support - Fix full GC never triggering after cross-toolchain builds: The `gc_state` file stores `system_clock` ticks, but the tick resolution differs between toolchains (nanoseconds on GCC/standard clang, microseconds on UE clang). A nanosecond timestamp misinterpreted as microseconds appears far in the future (~year 58,000), bypassing the staleness check and preventing time-based full GC from ever running. Fixed by also resetting when the stored timestamp is in the future. - Clamp GC countdown display to configured interval: Prevents nonsensical log output (e.g. "Full GC in 492128002h") caused by the above or any other clock anomaly. The clamp applies to both the scheduler log and the status API.
* Add Docker image build for compute workers (#837)Stefan Boberg38 hours1-0/+88
| | | Adds a Dockerfile (Ubuntu 24.04 + WineHQ) and an `xmake docker` task to build and optionally push a zenserver-compute Docker image, enabling Linux deployment of compute workers that run Windows executables via Wine.
* block/file cloning support for macOS / Linux (#786)Stefan Boberg38 hours3-0/+331
| | | | | | | | - Add block cloning (copy-on-write) support for Linux and macOS to complement the existing Windows (ReFS) implementation - **Linux**: `TryCloneFile` via `FICLONE` ioctl, `CloneQueryInterface` with range cloning via `FICLONERANGE` (Btrfs/XFS) - **macOS**: `TryCloneFile` via `clonefile()` syscall (APFS), `SupportsBlockRefCounting` via `VOL_CAP_INT_CLONE`. `CloneQueryInterface` is not implemented as macOS lacks a sub-file range clone API - Promote `ScopedFd` to file scope for broader use in filesystem code - Add test scripts for block cloning validation on Linux (Btrfs via loopback) and macOS (APFS) - Also added test script for testing on Windows (ReFS)
* add buildid updates to oplog and builds test scripts (#838)Dan Engelbrecht3 days4-74/+478
| | | | * updated test scripts * remove some hardcoded assumptions
* Made CPR optional, html generated at build time (#840)Stefan Boberg4 days1-111/+0
| | | | | | | - Fix potential crash on startup caused by logging macros being invoked before the logging system is initialized (null logger dereference in `ZenServerState::Sweep()`). `LoggerRef::ShouldLog` now guards against a null logger pointer. - Make CPR an optional dependency (`--zencpr` build option, enabled by default) so builds can proceed without it - Make zenvfs Windows-only (platform-specific target) - Generate the frontend zip at build time from source HTML files instead of checking in a binary blob which would accumulate with every single update
* fix casing of windows OidcToken.exe (#830)Dan Engelbrecht5 days1-1/+1
|
* fix OidcToken casing (#826)Stefan Boberg6 days1-2/+2
| | | fixes issue with zip not finding the OidcToken file
* HttpClient using libcurl, Unix Sockets for HTTP. HTTPS support (#770)Stefan Boberg7 days1-22/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The main goal of this change is to eliminate the cpr back-end altogether and replace it with the curl implementation. I would expect to drop cpr as soon as we feel happy with the libcurl back-end. That would leave us with a direct dependency on libcurl only, and cpr can be eliminated as a dependency. ### HttpClient Backend Overhaul - Implemented a new **libcurl-based HttpClient** backend (`httpclientcurl.cpp`, ~2000 lines) as an alternative to the cpr-based one - Made HttpClient backend **configurable at runtime** via constructor arguments and `-httpclient=...` CLI option (for zen, zenserver, and tests) - Extended HttpClient test suite to cover multipart/content-range scenarios ### Unix Domain Socket Support - Added Unix domain socket support to **httpasio** (server side) - Added Unix domain socket support to **HttpClient** - Added Unix domain socket support to **HttpWsClient** (WebSocket client) - Templatized `HttpServerConnectionT<SocketType>` and `WsAsioConnectionT<SocketType>` to handle TCP, Unix, and SSL sockets uniformly via `if constexpr` dispatch ### HTTPS Support - Added **preliminary HTTPS support to httpasio** (for Mac/Linux via OpenSSL) - Added **basic HTTPS support for http.sys** (Windows) - Implemented HTTPS test for httpasio - Split `InitializeServer` into smaller sub-functions for http.sys ### Other Notable Changes - Improved **zenhttp-test stability** with dynamic port allocation - Enhanced port retry logic in http.sys (handles ERROR_ACCESS_DENIED) - Fatal signal/exception handlers for backtrace generation in tests - Added `zen bench http` subcommand to exercise network + HTTP client/server communication stack
* updated chunk–block analyser (#818)Dan Engelbrecht8 days1-5/+11
| | | | | * create oplogs as they are imported * Improved logic for partial block analisys * unit tests for ChunkBlockAnalyser
* add fallback for zencache multirange (#816)Dan Engelbrecht8 days10-0/+496
| | | | | | | | | | | * clean up BuildStorageResolveResult to allow capabilities * add check for multirange request capability * add MaxRangeCountPerRequest capabilities * project export tests * add InMemoryBuildStorageCache * progress and logging improvements * fix ElapsedSeconds calculations in fileremoteprojectstore.cpp * oplogs/builds test script
* oidctoken tool package (#810)Stefan Boberg12 days1-4/+7
| | | | | * added OidcToken binary to the build process. The binary is mirrored from p4 and is placed next to the output of the build process. It is also placed in the release zip archives. * also fixed issue with Linux symbol stripping which was introduced in toolchain changes yesterday
* native xmake toolchain definition for UE-clang (#805)Stefan Boberg13 days6-59/+32
| | | | | | | | | | | | | | | This change is meant to provide a smoother experience when working on Linux. After this change, the toolchain setup process is now simply ```bash $ scripts/ue_build_linux/get_ue_toolchain.sh ``` and then at config time the toolchain is automatically detected if you downloaded it to the default location or have the `UE_TOOLCHAIN_DIR` environment variable set ```bash xmake config --mode=debug ``` Compared to the old script-based approach this configures the toolchain more precisely, avoiding leakage into unrelated build processes such as when a package manager decides to build something like Ninja locally etc.
* bump sentry to 0.12.1 (#721)Stefan Boberg2026-02-121-1/+2
|
* add Deploy.md and ability to specify a version via --version (#663)Stefan Boberg2025-11-241-22/+23
| | | | | * add docs/Deploy.md * added ability to specify a version on the command line, via `--version`
* renamed deployment scriptStefan Boberg2025-11-191-0/+0
|
* Update to curl 8.17.0 (from 8.11.0) (#648)Stefan Boberg2025-11-131-0/+344
| | | | Upgrade libcurl to 8.17.0 and enable native Mac CA validation via Apple SecTrust over the file-based approach which was in place previously. This should be more robust and more closely matches the behaviour of Apple's curl build and the rest of the OS.
* sentry/asan configuration tweaks (#649)v5.7.10-pre0Stefan Boberg2025-11-131-0/+1
| | | | | * Automated more of the decisions around which options to set when using ASAN * Also disabled Sentry by default as it's a bit annoying to have it upload crashes during development. Sentry is still automatically enabled and integrated as part of the `xmake bundle` step however so released builds will still have it.
* switch back to openssl on Mac (#641)Stefan Boberg2025-11-101-1/+1
| | | | | | | * switch back to openssl for Mac (fixes cross-compilation config) * add openssl 3.6.0 and change make target to install_dev for quicker install * disable LTO on Mac to reduce build time on openssl3 * add mbedTLS 3.6.5 (but this is not used anywhere right now)
* switch to xmake for package management (#611)Stefan Boberg2025-11-071-42/+15
| | | | | | | | | | | | | | | | | | | | | | This change removes our dependency on vcpkg for package management, in favour of bringing some code in-tree in the `thirdparty` folder as well as using the xmake build-in package management feature. For the latter, all the package definitions are maintained in the zen repo itself, in the `repo` folder. It should now also be easier to build the project as it will no longer depend on having the right version of vcpkg installed, which has been a common problem for new people coming in to the codebase. Now you should only need xmake to build. * Bumps xmake requirement on github runners to 2.9.9 to resolve an issue where xmake on Windows invokes cmake with `v144` toolchain which does not exist * BLAKE3 is now in-tree at `thirdparty/blake3` * cpr is now in-tree at `thirdparty/cpr` * cxxopts is now in-tree at `thirdparty/cxxopts` * fmt is now in-tree at `thirdparty/fmt` * robin-map is now in-tree at `thirdparty/robin-map` * ryml is now in-tree at `thirdparty/ryml` * sol2 is now in-tree at `thirdparty/sol2` * spdlog is now in-tree at `thirdparty/spdlog` * utfcpp is now in-tree at `thirdparty/utfcpp` * xmake package repo definitions is in `repo` * implemented support for sanitizers. ASAN is supported on windows, TSAN, UBSAN, MSAN etc are supported on Linux/MacOS though I have not yet tested it extensively on MacOS * the zencore encryption implementation also now supports using mbedTLS which is used on MacOS, though for now we still use openssl on Linux * crashpad * bumps libcurl to 8.11.0 (from 8.8.0) which should address a rare build upload bug
* remove --deep option for codesign as it is considered harmful (#629)v5.7.8-pre9Dan Engelbrecht2025-11-041-4/+15
| | | | | * remove --deep option for codesign as it is considered harmful * sign crashpadhandler * add verbosity and strict to codesign
* added default Linux UE toolchain directory (#558)Stefan Boberg2025-10-062-14/+22
| | | | | Instead of forcing user to specify a suitable directory for the UE toolchain, we offer a default which matches what the Github runners use and which is also covered by the root `.gitignore` file
* Update to recent UE toolchain, and link statically against toolchain libc++ ↵Liam Mitchell2025-06-134-14/+4
| | | | and libc++abi
* Zs/mac restore minver 12.5 (#344)Zousar Shaker2025-04-041-2/+2
| | | * Restore Mac minver to 12.5
* remove xcode 12.1 workaround (#339)Dan Engelbrecht2025-04-021-2/+2
| | | | | * remove xcode 12.1 workaround * bump min macos version to 14.0
* upload linux mac exe to sentry (#196)Dan Engelbrecht2024-10-161-0/+0
| | | | * upload mac/linux executables to sentry using `debug-files bundle-sources` on all platforms * update sentry-cli to latest on windows
* Enabled signing on windows agents again (#12)Joakim Lindqvist2024-03-201-0/+4
| | | * Enabled signing on windows agents again
* changed RPC recording to MPSC setup (#638)Stefan Boberg2024-01-311-2/+2
| | | fixes rare race condition when using RPC recording for long periods of time
* windows executable signing (#566)Dan Engelbrecht2023-12-151-2/+28
| | | Windows executables are now signed with official cert when creating a release
* embed html frontend content as binary compiled data (#559)Dan Engelbrecht2023-11-232-27/+145
| | | | | | - Feature: Added xmake task `updatefrontend` which updates the zip file containing the frontend html (`/src/zenserver/frontend/html.zip`) - Improvement: The frontend html content is no longer appended at the end of the executable which prevented signing, instead it is compiled in from the `/src/zenserver/frontend/html.zip` archive - Improvement: MacOS now does ad-hoc code signing by default when issuing `xmake bundle`, signing with proper cert is done on CI builds
* fixed paths in ue_build_linux/README.mdStefan Boberg2023-11-211-3/+3
|
* revive UE toolchain build (#343)Dan Engelbrecht2023-08-223-49/+43
| | | | | | | | | * add comment about 7z usage in xmake bundle * add ue-libcxx lib/headers and licence + tps * update get_ue_toolchain.sh to use embedded libc++ and decouple from p4 * clearer output when falling back to zip from 7z * update ci scripts to use ue toolchain on linux * updated linux build README.md * changelog
* updated logic for 7z/zipStefan Boberg2023-05-191-6/+6
| | | | | previously, an attempt was made to locate any installed binary but the located path was not actually used in practice
* moved source directories into `/src` (#264)Stefan Boberg2023-05-021-1/+1
| | | | | | * moved source directories into `/src` * updated bundle.lua for new `src` path * moved some docs, icon * removed old test trees
* fix: bundle shouldn't append content zip to zenStefan Boberg2023-05-011-4/+0
|
* Added setup_windows.bat for use in provisioning test machinesStefan Boberg2023-04-121-0/+5
|
* removed some deprecated scriptsStefan Boberg2023-02-088-1007/+0
|
* Removed unused scriptMartin Ridgers2023-02-071-277/+0
|
* Fixed wrong if-statement when checking for the presence of clang++Martin Ridgers2023-02-061-1/+1
|
* Added note about UE toolchain meeting VFX reference platform versionsMartin Ridgers2023-02-061-3/+4
|
* Fixed two typos in ReadmeMartin Ridgers2023-02-061-2/+2
|
* Removed the AppImage-based bundling step on LinuxMartin Ridgers2023-02-022-40/+1
|
* Scripts to acquire and use UE's Linux toolchainMartin Ridgers2023-02-025-0/+126
|
* Include zen cmd in release (#188)Dan Engelbrecht2022-11-091-3/+32
| | | | | * make sure we build the zen command line as well in bundle * include zen command line executable * changelog
* Don't use -r option for 7z, it is not needed and picks up more than you ↵Dan Engelbrecht2022-09-051-1/+1
| | | | expect (#156)
* tweak compression settings to streamline buildStefan Boberg2022-06-031-2/+2
| | | | previously the bundles used maximum compression, now it uses the fastest mode. With the old setting bundling (release) took 55s on my machine, with these settings it took 28s
* Commented out key file conditioningMartin Ridgers2022-04-121-1/+2
|
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-17 22:58:27 +0000