aboutsummaryrefslogtreecommitdiff
path: root/src/zenserver
Commit message (Collapse)AuthorAgeFilesLines
* reduced WaitForThreads time to see how it behaves with explicit thread poolssb/threadpoolStefan Boberg2026-02-231-1/+1
|
* added new I/O thread pool implementationStefan Boberg2026-02-201-0/+10
| | | | | | the new variant manages a dynamically growing/shrinking set of threads manually instead of relying on the built-in Windows thread pool the benefit of this is that we're in charge of setup and teardown so can make better guarantees about lifetimes of threads which can help with shutdown issues
* GC - fix handling of attachment ranges, http access token expiration, lock ↵Stefan Boberg2026-02-203-4/+6
| | | | | | | | file retry logic (#766) * GC - fix handling of attachment ranges * fix trace/log strings * fix HTTP access token expiration time logic * added missing lock retry in zenserver startup
* Merge branch 'main' into zs/web-ui-improvementszousar2026-02-1813-19/+1673
|\
| * structured compute basics (#714)Stefan Boberg2026-02-1813-19/+1673
| | | | | | | | | | | | | | | | | | this change adds the `zencompute` component, which can be used to distribute work dispatched from UE using the DDB (Derived Data Build) APIs via zenserver this change also adds a distinct zenserver compute mode (`zenserver compute`) which is intended to be used for leaf compute nodes to exercise the compute functionality without directly involving UE, a `zen exec` subcommand is also added, which can be used to feed replays through the system all new functionality is considered *experimental* and disabled by default at this time, behind the `zencompute` option in xmake config
* | updatefrontendzousar2026-02-181-0/+0
| |
* | entry.js handles missing/native items more gracefullyzousar2026-02-182-4/+32
| |
* | updatefrontendzousar2026-02-171-0/+0
| |
* | Merge branch 'main' into zs/web-ui-improvementszousar2026-02-174-16/+75
|\|
| * add http server root password protection (#757)Dan Engelbrecht2026-02-174-16/+75
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Feature: Added `--security-config-path` option to zenserver to configure security settings - Expects a path to a .json file - Default is an empty path resulting in no extra security settings and legacy behavior - Current support is a top level filter of incoming http requests restricted to the `password` type - `password` type will check the `Authorization` header and match it to the selected authorization strategy - Currently the security settings is very basic and configured to a fixed username+password at startup { "http" { "root": { "filter": { "type": "password", "config": { "password": { "username": "<username>", "password": "<password>" }, "protect-machine-local-requests": false, "unprotected-uris": [ "/health/", "/health/info", "/health/version" ] } } } } }
* | Dependencies table doesn't reflow the entries pagezousar2026-02-171-5/+19
| |
* | Rename the cache section in the web uizousar2026-02-171-1/+1
| |
* | Make files table in entry.js paginated and searchablezousar2026-02-171-40/+170
| |
* | Added custom page for cook.artifactszousar2026-02-164-4/+428
| |
* | Change breadcrumbs for oplogs to be more descriptivezousar2026-02-152-6/+12
| |
* | Add support for listing files on oplog entrieszousar2026-02-151-9/+110
| |
* | Restore handling for hard/soft name prefixeszousar2026-02-151-4/+16
| |
* | Enhance dependencies to include soft and hard depszousar2026-02-141-11/+13
|/
* logging config move to zenutil (#754)Stefan Boberg2026-02-137-87/+47
| | | made logging config options from zenserver available in zen CLI
* Fix formatting of stat pages (#748)Liam Mitchell2026-02-094-3/+10
| | | * Fix formatting of stat pages
* use matcher over regex (#744)Dan Engelbrecht2026-02-045-16/+56
| | | | * replace http router AddPattern with AddMatcher * fix scrub logging
* Merge pull request #724 from ue-foundation/lm/restrict-reads-to-projectLiam Mitchell2026-02-031-4/+32
|\ | | | | Restrict filesystem reads in snapshot to paths under project root
| * Disallow external file reads from project with unset project rootsLiam Mitchell2026-01-291-5/+8
| |
| * Revert to previous behaviour for path checking in the presence of an empty ↵Liam Mitchell2026-01-281-9/+13
| | | | | | | | project root
| * Use std::mismatch to compare paths and hoist project root canonicalization ↵Liam Mitchell2026-01-211-13/+18
| | | | | | | | out of loop
| * Restrict filesystem reads in snapshot to paths under project rootLiam Mitchell2026-01-191-2/+18
| |
* | add command line option for scrub timeslice (#742)Dan Engelbrecht2026-02-031-1/+9
| |
* | restore missing healthinfo call (#737)Dan Engelbrecht2026-01-281-0/+5
| |
* | Zs/oplog navigation fix (#731)Zousar Shaker2026-01-232-6/+6
| | | | | | * Fix incorrect oplog navigation symbols
* | hotfix 5.7.18 (#730)Dan Engelbrecht2026-01-221-2/+5
| | | | | | | | * make sure we properly convert command line args for zenserver as well * make sure we *add* wildcards/excludes in addition to defaults
* | zen hub (#574)Stefan Boberg2026-01-219-10/+1546
|/ | | | | Initial implementation of zenserver "hub" mode. This is an experimental feature. zenserver can be started in hub mode by specifying `hub` as the first argument to zenserver
* zenserver API changes, some other minor changes (#720)Stefan Boberg2026-01-195-25/+129
| | | | | | | * add system metrics output to top command * removed unnecessary xmake directives * file system API/comment tweaks * fixed out-of-range access in httpserver test * updated ZenServer base API to allow customization by mode
* OTLP/trace improvements (#717)Stefan Boberg2026-01-193-15/+19
| | | | | | | This PR brings over some changes made to avoid performing setup for otel instrumentation if we are not sending otel information anywhere anyway. It also adds the ability to configure an OTLP endpoint on the command line using `--otlp-endpoint=<URI>`. Bear in mind that OTLP support is still not officially supported so this should not be used in production at this stage.
* use current locale when converting wide char command line arguments to ↵Dan Engelbrecht2026-01-151-0/+8
| | | | | | string (#712) * set utf8 locale, only set LC_CTYPE as we don't want to affect how decimal point or sorting orders are handled * set language/region explicitly so we can use LC_ALL
* asio/http optimizations (#449)Stefan Boberg2026-01-141-27/+2
| | | | | | | This change primarily introduces improved logic for dealing with sending data from file references. This is intended to reduce the amount of memory-mapping we end up doing when sending data from files. Windows now uses `TransmitFile` to send file data more efficiently using kernel-side I/O, but Linux/Mac basically behaves as before since they don't offer any true async file I/O support via asio. This should be implemented separately using a background I/O thread pool. This PR also includes improved memory management for http/asio with reduced allocation counts, and a fix for a potential use-after-free in very high load scenarios.
* if we fail to create the server mutex, gracefully report error without ↵Dan Engelbrecht2026-01-121-1/+3
| | | | sending error to Sentry (#705)
* Avoid rendering user text input as HTML (#700)Liam Mitchell2026-01-092-1/+1
| | | * Avoid rendering user text input as HTML
* Ensure upstream put propagation includes overwritezousar2025-12-194-5/+39
| | | | When changing the default limit-overwrite behavior, a unit test surfaced a bug where an put of data with overwrite cache policy would not get propagated via zen's built-in upstream mechanism with a matching overwrite cache policy to the upstream. This change ensures that it does and leaves the unit test configured to exercise this scenario.
* Change default limit-overwrite behavior to truezousar2025-12-172-2/+2
|
* remove error warning in output (#695)Dan Engelbrecht2025-12-172-2/+2
| | | * changed some logging string so they don't get caught in CI logging
* add boost-worker oplog import export options (#693)Dan Engelbrecht2025-12-162-22/+73
| | | | | | | | | | - Feature: `zen oplog-export`, `zen oplog-import` and `zen oplog-download` now has options to boost workers - `--boost-worker-count` - Increase the number of worker threads - may cause computer to be less responsive - `--boost-worker-memory` - Increase the limit where we write downloaded data to temporary storage to conserve space - may cause computer to be less responsive due to high memory usage - `--boost-workers` - Enables both 'boost-worker-count' and 'boost-worker-memory' - may cause computer to be less responsive - Improvement: Refactored boost options for `zen builds` operations `upload`, `download`, `diff`, `prime-cache`, `fetch-blob` and `validate-part` - `--boost-worker-count` - Increase the number of worker threads - may cause computer to be less responsive - `--boost-worker-memory` - Increase the limit where we write downloaded data to temporary storage to conserve space - may cause computer to be less responsive due to high memory usage - `--boost-workers` - Enables both 'boost-worker-count' and 'boost-worker-memory' - may cause computer to be less responsive
* add otel instrumentation (#581)Stefan Boberg2025-12-113-0/+19
| | | | | | | | this change adds OTEL tracing to a few places * Top-level application lifecycle (config/init/cleanup, main loop) * http.sys requests it also brings some otlptrace optimizations and dynamic configuration of tracing. OTLP tracing is currently always disabled
* upload to zen cache from oplog import now disabled by default (#680)Dan Engelbrecht2025-12-091-1/+1
|
* add missing patterns to objectstore service http routing (#674)Dan Engelbrecht2025-12-031-3/+7
|
* automatic scrub on startup (#667)Dan Engelbrecht2025-11-273-32/+46
| | | | | - Improvement: Deeper validation of data when scrub is activated (cas/cache/project) - Improvement: Enabled more multi threading when running scrub operations - Improvement: Added means to force a scrub operation at startup with a new release using ZEN_DATA_FORCE_SCRUB_VERSION variable in xmake.lua
* logging improvements (#664)Stefan Boberg2025-11-243-21/+24
| | | | | * Eliminate spurious `FlushFileBuffers` calls on Windows by tracking writes * Make cache log file creation optional
* Added actions to drop all projects or all z$ namespaces. (#655)Martin Ridgers2025-11-183-2/+54
| | | | | | | | | | | * Save references to the project and zcache tables * Add an attribute to table rows with the actionable project/namespace id * Drop-all option for projects and cache namespaces * Updated frontend .zip archive * Edited changelog
* Include version string on the dashboard's start page. (#651)Martin Ridgers2025-11-174-2/+26
| | | | | | | | | | | * Method to get plain text from an async request * Include server's version on the dashboard start page * Same paragraph style as the rest of the method * Updated changelog * Update frontend archive
* control cache upload (#646)Dan Engelbrecht2025-11-121-0/+2
| | | * add option to enable/disable upload to builds cache
* switch to xmake for package management (#611)Stefan Boberg2025-11-074-10/+55
| | | | | | | | | | | | | | | | | | | | | | This change removes our dependency on vcpkg for package management, in favour of bringing some code in-tree in the `thirdparty` folder as well as using the xmake build-in package management feature. For the latter, all the package definitions are maintained in the zen repo itself, in the `repo` folder. It should now also be easier to build the project as it will no longer depend on having the right version of vcpkg installed, which has been a common problem for new people coming in to the codebase. Now you should only need xmake to build. * Bumps xmake requirement on github runners to 2.9.9 to resolve an issue where xmake on Windows invokes cmake with `v144` toolchain which does not exist * BLAKE3 is now in-tree at `thirdparty/blake3` * cpr is now in-tree at `thirdparty/cpr` * cxxopts is now in-tree at `thirdparty/cxxopts` * fmt is now in-tree at `thirdparty/fmt` * robin-map is now in-tree at `thirdparty/robin-map` * ryml is now in-tree at `thirdparty/ryml` * sol2 is now in-tree at `thirdparty/sol2` * spdlog is now in-tree at `thirdparty/spdlog` * utfcpp is now in-tree at `thirdparty/utfcpp` * xmake package repo definitions is in `repo` * implemented support for sanitizers. ASAN is supported on windows, TSAN, UBSAN, MSAN etc are supported on Linux/MacOS though I have not yet tested it extensively on MacOS * the zencore encryption implementation also now supports using mbedTLS which is used on MacOS, though for now we still use openssl on Linux * crashpad * bumps libcurl to 8.11.0 (from 8.8.0) which should address a rare build upload bug
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-11 19:43:34 +0000