aboutsummaryrefslogtreecommitdiff
path: root/src/zenserver-test
Commit message (Collapse)AuthorAgeFilesLines
* test running / reporting improvements (#797)Stefan Boberg11 days5-36/+30
| | | | | | | | | | | | | | | | | | | **CI/CD improvements (validate.yml):** - Add test reporter (`ue-foundation/test-reporter@v2`) for all three platforms, rendering JUnit test results directly in PR check runs - Add "Trust workspace" step on Windows to fix git safe.directory ownership issue with self-hosted runners - Clean stale report files before each test run to prevent false failures from leftover XML - Broaden `paths-ignore` to skip builds for non-code changes (`*.md`, `LICENSE`, `.gitignore`, `docs/**`) **Test improvements:** - Convert `CHECK` to `REQUIRE` in several test suites (projectstore, integration, http) for fail-fast behavior - Mark some tests with `doctest::skip()` for selective execution - Skip httpclient transport tests pending investigation - Add `--noskip` option to `xmake test` task - Add `--repeat=<N>` option to `xmake test` task, to run tests repeatedly N times or until there is a failure **xmake test output improvements:** - Add totals row to test summary table - Right-justify numeric columns in summary table
* add multirange requests to blob store (#795)Dan Engelbrecht12 days1-1/+199
| | | * add multirange requests to blob store
* structured compute basics (#714)Stefan Boberg2026-02-181-0/+34
| | | | | | | | | this change adds the `zencompute` component, which can be used to distribute work dispatched from UE using the DDB (Derived Data Build) APIs via zenserver this change also adds a distinct zenserver compute mode (`zenserver compute`) which is intended to be used for leaf compute nodes to exercise the compute functionality without directly involving UE, a `zen exec` subcommand is also added, which can be used to feed replays through the system all new functionality is considered *experimental* and disabled by default at this time, behind the `zencompute` option in xmake config
* logging config move to zenutil (#754)Stefan Boberg2026-02-131-1/+1
| | | made logging config options from zenserver available in zen CLI
* hotfix 5.7.18 (#730)Dan Engelbrecht2026-01-221-0/+3
| | | | * make sure we properly convert command line args for zenserver as well * make sure we *add* wildcards/excludes in addition to defaults
* zen hub (#574)Stefan Boberg2026-01-211-0/+252
| | | | | Initial implementation of zenserver "hub" mode. This is an experimental feature. zenserver can be started in hub mode by specifying `hub` as the first argument to zenserver
* ZenServerProcess API changes (#719)Stefan Boberg2026-01-195-28/+28
| | | | | | | This refactor aims to improve the `ZenServerProcess` classes by making them useful for managing child zenserver instances in more scenarios than just automated tests. This involves changing some functions to not talk about "test directory" and instead use "data directory" etc As a consequence of the API changes, some tests have changed accordingly. The code includes som reference to the "hub" mode but there is not yet any other code using this mode, it's just included in this PR to simplify future merges.
* use current locale when converting wide char command line arguments to ↵Dan Engelbrecht2026-01-151-0/+4
| | | | | | string (#712) * set utf8 locale, only set LC_CTYPE as we don't want to affect how decimal point or sorting orders are handled * set language/region explicitly so we can use LC_ALL
* Ensure upstream put propagation includes overwritezousar2025-12-191-1/+1
| | | | When changing the default limit-overwrite behavior, a unit test surfaced a bug where an put of data with overwrite cache policy would not get propagated via zen's built-in upstream mechanism with a matching overwrite cache policy to the upstream. This change ensures that it does and leaves the unit test configured to exercise this scenario.
* Change default limit-overwrite behavior to truezousar2025-12-171-3/+4
|
* add otel instrumentation (#581)Stefan Boberg2025-12-111-1/+1
| | | | | | | | this change adds OTEL tracing to a few places * Top-level application lifecycle (config/init/cleanup, main loop) * http.sys requests it also brings some otlptrace optimizations and dynamic configuration of tracing. OTLP tracing is currently always disabled
* HTTP server API changes for improved extensibility (#684)Stefan Boberg2025-12-111-12/+12
| | | | * refactored `HttpServer` so all subclass member functions are proctected, to make it easier to extend base functionality * added API service, can be used to enumerate registered endpoints (at `/api`). Currently only very basic information is provided
* batch op not in destructor (#676)Dan Engelbrecht2025-12-041-0/+125
| | | | | * use fixed vectors for batch requests * refactor cache batch value put/get to not execute code that can throw execeptions in destructor * extend test with multi-bucket requests
* RawOffset can be anything and we expect an empty buffer to be returned along ↵Dan Engelbrecht2025-11-261-0/+182
| | | | with RawSize = 0 if the offset was out of bounds for the value. (#666)
* switch to xmake for package management (#611)Stefan Boberg2025-11-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | This change removes our dependency on vcpkg for package management, in favour of bringing some code in-tree in the `thirdparty` folder as well as using the xmake build-in package management feature. For the latter, all the package definitions are maintained in the zen repo itself, in the `repo` folder. It should now also be easier to build the project as it will no longer depend on having the right version of vcpkg installed, which has been a common problem for new people coming in to the codebase. Now you should only need xmake to build. * Bumps xmake requirement on github runners to 2.9.9 to resolve an issue where xmake on Windows invokes cmake with `v144` toolchain which does not exist * BLAKE3 is now in-tree at `thirdparty/blake3` * cpr is now in-tree at `thirdparty/cpr` * cxxopts is now in-tree at `thirdparty/cxxopts` * fmt is now in-tree at `thirdparty/fmt` * robin-map is now in-tree at `thirdparty/robin-map` * ryml is now in-tree at `thirdparty/ryml` * sol2 is now in-tree at `thirdparty/sol2` * spdlog is now in-tree at `thirdparty/spdlog` * utfcpp is now in-tree at `thirdparty/utfcpp` * xmake package repo definitions is in `repo` * implemented support for sanitizers. ASAN is supported on windows, TSAN, UBSAN, MSAN etc are supported on Linux/MacOS though I have not yet tested it extensively on MacOS * the zencore encryption implementation also now supports using mbedTLS which is used on MacOS, though for now we still use openssl on Linux * crashpad * bumps libcurl to 8.11.0 (from 8.8.0) which should address a rare build upload bug
* move cpr in-tree (#605)Stefan Boberg2025-10-241-1/+1
| | | | | | * added cpr 1.10.5 in-tree to allow updates to vcpkg without breaking the build * added asio 1.29.0 in-tree to remove one more vcpkg dependency * bumped vcpkg to 2024.06.15 to address failure to build due to use of deprecated binaries in vcpkg (404 error: `https://mirror.msys2.org/mingw/mingw64/mingw-w64-x86_64-pkgconf-1~2.1.0-1-any.pkg.tar.zst` during build)
* speed up tests (#555)Dan Engelbrecht2025-10-062-10/+10
| | | | | | | | | | | | * faster FileSystemTraversal test * faster jobqueue test * faster NamedEvent test * faster cache tests * faster basic http tests * faster blockstore test * faster cache store tests * faster compactcas tests * more responsive zenserver launch * tweak worker pool sizes in tests
* cacherequests helpers test only (#551)Dan Engelbrecht2025-10-034-3/+1730
| | | | * don't use cacherequests utils in cache_cmd.cpp * make zenutil/cacherequests code into test code helpers only
* remove zenutil dependency in zenremotestore (#547)Dan Engelbrecht2025-10-031-5/+9
| | | | | | | | | * remove dependency to zenutil/workerpools.h from remoteprojectstore.cpp * remove dependency to zenutil/workerpools.h from buildstoragecache.cpp * remove unneded include * move jupiter helpers to zenremotestore * move parallelwork to zencore * remove zenutil dependency from zenremotestore * clean up test project dependencies - use indirect dependencies
* move zenutil builds code to zenremotestore (#543)Dan Engelbrecht2025-10-033-3/+9
| | | | * move buildstorage implementations to zenremotestore lib * move builds storage to zenremotelib
* fix for RPC replay issue (wrong content-type) (#536)Stefan Boberg2025-10-021-1/+1
| | | | | likely fall-out from HttpClient refactor. The content type used to be explicit via headers but is now taken from the `IoBuffer` also fixed issue which meant the original request session ID would also not be propagated as intended
* split zenserver-test monolith into multiple source files (#528)Stefan Boberg2025-09-296-4602/+4701
|
* make cpr a HttpClient implementation detail (#517)Stefan Boberg2025-09-293-685/+451
| | | | | these changes remove cpr from anything which is not `HttpClient` internals. The goal is to eventually replace cpr with a more direct curl interface to eliminate cpr since it's proven problematic due to their development practices which frequently breaks APIs and prevents us from updating vcpkg. But this PR is limited to refactoring existing cpr code to use `HttpClient` instead.
* make sure trace is configured for all test drivers (#523)Stefan Boberg2025-09-291-0/+8
|
* new append op rpc method (#511)Dan Engelbrecht2025-09-261-0/+270
| | | | | | - Feature: New `/prj/{project}/{oplog}/rpc` endpoint method `appendops` to send an array of oplog ops and receiving a list of `need` for attachments not present - Feature: Added `usingtmpfiles` boolean field to `/prj/{project}/{oplog}/rpc` method `putchunks` to be explicit about allowing move of temp attachment files - Improvement: Added additional validation of compact binary objects when reading from disk/receiving from client - Improvement: Windows: Added fallback code to use standard `MoveFile` api when rename via `SetFileInformationByHandle` fails in `MoveToFile` (used by filecas)
* Adjust the responses from PUT commandszousar2025-09-231-1/+1
| | | | | - Ensure that text responses are in a field named "Message" - Change the record response to be named "Record" instead of "Object"
* Change batch put responses for client reportingzousar2025-09-191-1/+13
| | | | Conflicts are now treated as successes, and we optionally return a Details array instead of an ErrorMessages array. Details are returned for all requests in a batch, or no requests in a batch depending on whether there are any details to be shared about any of the put requests. The details for a conflict include the raw hash and raw size of the item. If the item is a record, we also include the record as an object.
* add EMode to WorkerTheadPool to avoid thread starvation (#492)Dan Engelbrecht2025-09-101-45/+50
| | | - Improvement: Add a new mode to worker thread pools to avoid starvation of workers which could cause long stalls due to other work begin queued up. UE-305498
* Merge remote-tracking branch 'origin/main' into de/zen-service-commandLiam Mitchell2025-08-211-5/+823
|\
| * add limitoverwrites option per bucket (#466)Dan Engelbrecht2025-08-121-6/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Feature: Added global zenserver option `--cache-bucket-limit-overwrites` controlling Whether to require policy flag pattern before allowing overwrites or not. Default `false` = overwrites always allowed - Feature: Add per bucket cache configuration option `limitoverwrites` (Lua options file only) cache = { bucket = { -- This is the default for all namespaces limitoverwrites = true }, buckets = { -- Here you can add matching per bucket name (matches accross namespaces) iostorecompression = { limitoverwrites = false }, }, }
| * Merge branch 'main' into zs/put-overwrite-policyZousar Shaker2025-08-081-0/+475
| |\
| | * refactor blobstore (#458)Dan Engelbrecht2025-08-061-0/+475
| | | | | | | | | | | | | | | | | | - Improvement: Refactored build store cache to use existing CidStore implementation instead of implementation specific blob storage - **CAUTION** This will clear any existing cache when updating as the manifest version and storage strategy has changed - Bugfix: BuildStorage cache return "true" for metadata existance for all blobs that had payloads regardless of actual existance for metadata
| * | Merge branch 'main' into zs/put-overwrite-policyzousar2025-06-241-11/+12
| |\|
| * | Establish TODOs and unit test for rejected PUT propagationzousar2025-03-241-2/+74
| | |
| * | Control overwrite enforcement with a config settingzousar2025-03-021-45/+140
| | |
| * | Expand and fix unit tests for overwrite behaviorzousar2025-02-261-7/+82
| | |
| * | Add unit test for overwrite behaviorzousar2025-02-181-0/+102
| | |
* | | Merge branch 'main' into de/zen-service-commandLiam Mitchell2025-07-291-11/+12
|\ \ \ | | |/ | |/|
| * | make sure tests initialize trace so we don't end up allocating tons of ↵Stefan Boberg2025-05-151-0/+1
| | | | | | | | | | | | memory for no reason (#397)
| * | long filename support (#330)Dan Engelbrecht2025-03-311-9/+9
| | | | | | | | | - Bugfix: Long file paths now works correctly on Windows
| * | build list filters (#313)Dan Engelbrecht2025-03-191-2/+2
| |/ | | | | | | | | - Feature: `zen builds list` command has new options - `--query-path` - path to a .json (json format) or .cbo (compact binary object format) with the search query to use - `--result-path` - path to a .json (json format) or .cbo (compact binary object format) to write output result to, if omitted json format will be output to console
* | Merge remote-tracking branch 'origin/main' into de/zen-service-commandDan Engelbrecht2025-03-141-2/+2
|\|
* | Implementation of service commands for Linux.Liam Mitchell2025-02-271-0/+4
|/
* move basicfile.h/cpp -> zencore (#273)Dan Engelbrecht2025-01-161-1/+1
| | | | | | move jupiter.h/cpp -> zenutil move packageformat.h/.cpp -> zenhttp zenutil now depends on zenhttp instead of the inverse
* fixed XXH3_128Stream so it initializes the state properlyStefan Boberg2024-12-131-1/+1
| | | | the old version is still present for now, with a _deprecated suffix
* added ComputeOpKey so all instances of mapping key -> Oid is in a single placeStefan Boberg2024-12-131-4/+14
|
* snapshot crash (#234)Dan Engelbrecht2024-11-251-1/+96
| | | | * snapshot test * use proper IoBuffer constructor to take handle ownership
* split zencore/memory.h -> memoryview.h, memcmp.h (#228)Stefan Boberg2024-11-251-1/+1
| | | | | | | minor clean-up `zencore/memory.h` used to contain a variety of things including `Malloc` support along with `MemoryView` etc since the memory allocator stuff moved into `zencore/memory/memory.h` there was basically only `MemoryView` and `MemCmp` in there which seemed better to split out into separate headers to avoid overloading `memory.h`
* Insights-compatible memory tracking (#214)Stefan Boberg2024-11-251-8/+1
| | | | | | | | | | | | | This change introduces support for tracing of memory allocation activity. The code is ported from UE5, and Unreal Insights can be used to analyze the output. This is currently only fully supported on Windows, but will be extended to Mac/Linux in the near future. To activate full memory tracking, pass `--trace=memory` on the commandline alongside `--tracehost=<ip>` or `-tracefile=<path>`. For more control over how much detail is traced you can instead pass some combination of `callstack`, `memtag`, `memalloc` instead. In practice, `--trace=memory` is an alias for `--trace=callstack,memtag,memalloc`). For convenience we also support `--trace=memory_light` which omits call stacks. This change also introduces multiple memory allocators, which may be selected via command-line option `--malloc=<allocator>`: * `mimalloc` - mimalloc (default, same as before) * `rpmalloc` - rpmalloc is another high performance allocator for multithreaded applications which may be a better option than mimalloc (to be evaluated). Due to toolchain limitations this is currently only supported on Windows. * `stomp` - an allocator intended to be used during development/debugging to help track down memory issues such as use-after-free or out-of-bounds access. Currently only supported on Windows. * `ansi` - fallback to default system allocator
* workspace share security (#192)Dan Engelbrecht2024-10-231-18/+147
| | | | | | | - Improvement: Reworked workspace shares to be more secure. Workspaces and workspace shares can only be created using the `zen workspace` command, the http endpoint is disabled unless zenserver is started with the `--workspaces-allow-changes` option enabled. - Each workspace are now configured via a `zenworkspaceconfig.json` file in the root of each workspace - A workspace can allow shares to be created via the http interface if the workspace is created with the `--allow-share-create-from-http` option enabled - A new http endpoint at `/ws` - issuing a `Get` operation will get you a list of workspaces - A new http endpoint at `/ws/refresh` - issuing a `Get` will make zenserver scan for edits in workspaces and workspace shares
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-11 21:42:17 +0000