aboutsummaryrefslogtreecommitdiff
path: root/src/zenserver
Commit message (Collapse)AuthorAgeFilesLines
* subprocess tracking using Jobs on Windows/hub (#796)HEADmainStefan Boberg11 days2-3/+53
| | | | | This change introduces job object support on Windows to be able to more accurately track and limit resource usage on storage instances created by the hub service. It also ensures that all child instances can be torn down reliably on exit. Also made it so hub tests no longer pop up console windows while running.
* add multirange requests to blob store (#795)Dan Engelbrecht12 days1-19/+95
| | | * add multirange requests to blob store
* updatefrontendzousar13 days1-0/+0
|
* Merge branch 'main' into zs/web-ui-improvementszousar13 days13-38/+128
|\
| * work around doctest shutdown issues with static CRT (#784)Stefan Boberg2026-02-251-0/+9
| | | | | | | | | | | | * tweaked doctest.h to avoid shutdown issues due to thread_local variables running destructors after the main thread has torn down everything including the heap * disabled zenserver exit thread waiting since doctest should hopefully not be causing issues during shutdown anymore after my workaround This should help reduce the duration of tests spawning lots of server instances
| * HttpService/Frontend improvements (#782)Stefan Boberg2026-02-253-15/+51
| | | | | | | | | | | | | | - zenhttp: added `GetServiceUri()`/`GetExternalHost()` - enables code to quickly generate an externally reachable URI for a given service - frontend: improved Uri handling (better defaults) - added support for 404 page (to make it easier to find a good URL)
| * use partial blocks for oplog import (#780)Dan Engelbrecht2026-02-241-6/+27
| | | | | | | | | | Feature: Add --allow-partial-block-requests to zen oplog-import Improvement: zen oplog-import now uses partial block requests to reduce download size Improvement: Use latency to Cloud Storage host and Zen Cache host when calculating partial block requests
| * Various bug fixes (#778)Stefan Boberg2026-02-246-18/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | zencore fixes: - filesystem.cpp: ReadFile error reporting logic - compactbinaryvalue.h: CbValue::As*String error reporting logic zenhttp fixes: - httpasio BindAcceptor would `return 0;` in a function returning `std::string` (UB) - httpsys async workpool initialization race zenstore fixes: - cas.cpp: GetFileCasResults Results param passed by value instead of reference (large chunk results were silently lost) - structuredcachestore.cpp: MissCount unconditionally incremented (counted hits as misses) - cacherpc.cpp: Wrong boolean in Incomplete response array (all entries marked incomplete) - cachedisklayer.cpp: sizeof(sizeof(...)) in two validation checks computed sizeof(size_t) instead of struct size - buildstore.cpp: Wrong hash tracked in GC key list (BlobHash pushed twice instead of MetadataHash) - buildstore.cpp: Removed duplicate m_LastAccessTimeUpdateCount increment in PutBlob zenserver fixes: - httpbuildstore.cpp: Reversed subtraction in HTTP range calculation (unsigned underflow) - hubservice.cpp: Deadlock in Provision() calling Wake() while holding m_Lock (extracted WakeLocked helper) - zipfs.cpp: Data race in GetFile() lazy initialization (added RwLock with shared/exclusive paths)
| * Revert "Fix correctness and concurrency bugs found during code review"Stefan Boberg2026-02-244-28/+8
| | | | | | | | This reverts commit 3c89c486338890ce39ddebe5be4722a09e85701a.
| * Fix correctness and concurrency bugs found during code reviewStefan Boberg2026-02-244-8/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | zenstore fixes: - cas.cpp: GetFileCasResults Results param passed by value instead of reference (large chunk results were silently lost) - structuredcachestore.cpp: MissCount unconditionally incremented (counted hits as misses) - cacherpc.cpp: Wrong boolean in Incomplete response array (all entries marked incomplete) - cachedisklayer.cpp: sizeof(sizeof(...)) in two validation checks computed sizeof(size_t) instead of struct size - buildstore.cpp: Wrong hash tracked in GC key list (BlobHash pushed twice instead of MetadataHash) - buildstore.cpp: Removed duplicate m_LastAccessTimeUpdateCount increment in PutBlob zenserver fixes: - httpbuildstore.cpp: Reversed subtraction in HTTP range calculation (unsigned underflow) - hubservice.cpp: Deadlock in Provision() calling Wake() while holding m_Lock (extracted WakeLocked helper) - zipfs.cpp: Data race in GetFile() lazy initialization (added RwLock with shared/exclusive paths) Co-Authored-By: Claude Opus 4.6 <[email protected]>
| * GC - fix handling of attachment ranges, http access token expiration, lock ↵Stefan Boberg2026-02-203-4/+6
| | | | | | | | | | | | | | | | file retry logic (#766) * GC - fix handling of attachment ranges * fix trace/log strings * fix HTTP access token expiration time logic * added missing lock retry in zenserver startup
* | updatefrontendzousar13 days1-0/+0
| |
* | icon and header logo changeszousar2026-02-195-20/+22
|/
* Merge branch 'main' into zs/web-ui-improvementszousar2026-02-1813-19/+1673
|\
| * structured compute basics (#714)Stefan Boberg2026-02-1813-19/+1673
| | | | | | | | | | | | | | | | | | this change adds the `zencompute` component, which can be used to distribute work dispatched from UE using the DDB (Derived Data Build) APIs via zenserver this change also adds a distinct zenserver compute mode (`zenserver compute`) which is intended to be used for leaf compute nodes to exercise the compute functionality without directly involving UE, a `zen exec` subcommand is also added, which can be used to feed replays through the system all new functionality is considered *experimental* and disabled by default at this time, behind the `zencompute` option in xmake config
* | updatefrontendzousar2026-02-181-0/+0
| |
* | entry.js handles missing/native items more gracefullyzousar2026-02-182-4/+32
| |
* | updatefrontendzousar2026-02-171-0/+0
| |
* | Merge branch 'main' into zs/web-ui-improvementszousar2026-02-174-16/+75
|\|
| * add http server root password protection (#757)Dan Engelbrecht2026-02-174-16/+75
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Feature: Added `--security-config-path` option to zenserver to configure security settings - Expects a path to a .json file - Default is an empty path resulting in no extra security settings and legacy behavior - Current support is a top level filter of incoming http requests restricted to the `password` type - `password` type will check the `Authorization` header and match it to the selected authorization strategy - Currently the security settings is very basic and configured to a fixed username+password at startup { "http" { "root": { "filter": { "type": "password", "config": { "password": { "username": "<username>", "password": "<password>" }, "protect-machine-local-requests": false, "unprotected-uris": [ "/health/", "/health/info", "/health/version" ] } } } } }
* | Dependencies table doesn't reflow the entries pagezousar2026-02-171-5/+19
| |
* | Rename the cache section in the web uizousar2026-02-171-1/+1
| |
* | Make files table in entry.js paginated and searchablezousar2026-02-171-40/+170
| |
* | Added custom page for cook.artifactszousar2026-02-164-4/+428
| |
* | Change breadcrumbs for oplogs to be more descriptivezousar2026-02-152-6/+12
| |
* | Add support for listing files on oplog entrieszousar2026-02-151-9/+110
| |
* | Restore handling for hard/soft name prefixeszousar2026-02-151-4/+16
| |
* | Enhance dependencies to include soft and hard depszousar2026-02-141-11/+13
|/
* logging config move to zenutil (#754)Stefan Boberg2026-02-137-87/+47
| | | made logging config options from zenserver available in zen CLI
* Fix formatting of stat pages (#748)Liam Mitchell2026-02-094-3/+10
| | | * Fix formatting of stat pages
* use matcher over regex (#744)Dan Engelbrecht2026-02-045-16/+56
| | | | * replace http router AddPattern with AddMatcher * fix scrub logging
* Merge pull request #724 from ue-foundation/lm/restrict-reads-to-projectLiam Mitchell2026-02-031-4/+32
|\ | | | | Restrict filesystem reads in snapshot to paths under project root
| * Disallow external file reads from project with unset project rootsLiam Mitchell2026-01-291-5/+8
| |
| * Revert to previous behaviour for path checking in the presence of an empty ↵Liam Mitchell2026-01-281-9/+13
| | | | | | | | project root
| * Use std::mismatch to compare paths and hoist project root canonicalization ↵Liam Mitchell2026-01-211-13/+18
| | | | | | | | out of loop
| * Restrict filesystem reads in snapshot to paths under project rootLiam Mitchell2026-01-191-2/+18
| |
* | add command line option for scrub timeslice (#742)Dan Engelbrecht2026-02-031-1/+9
| |
* | restore missing healthinfo call (#737)Dan Engelbrecht2026-01-281-0/+5
| |
* | Zs/oplog navigation fix (#731)Zousar Shaker2026-01-232-6/+6
| | | | | | * Fix incorrect oplog navigation symbols
* | hotfix 5.7.18 (#730)Dan Engelbrecht2026-01-221-2/+5
| | | | | | | | * make sure we properly convert command line args for zenserver as well * make sure we *add* wildcards/excludes in addition to defaults
* | zen hub (#574)Stefan Boberg2026-01-219-10/+1546
|/ | | | | Initial implementation of zenserver "hub" mode. This is an experimental feature. zenserver can be started in hub mode by specifying `hub` as the first argument to zenserver
* zenserver API changes, some other minor changes (#720)Stefan Boberg2026-01-195-25/+129
| | | | | | | * add system metrics output to top command * removed unnecessary xmake directives * file system API/comment tweaks * fixed out-of-range access in httpserver test * updated ZenServer base API to allow customization by mode
* OTLP/trace improvements (#717)Stefan Boberg2026-01-193-15/+19
| | | | | | | This PR brings over some changes made to avoid performing setup for otel instrumentation if we are not sending otel information anywhere anyway. It also adds the ability to configure an OTLP endpoint on the command line using `--otlp-endpoint=<URI>`. Bear in mind that OTLP support is still not officially supported so this should not be used in production at this stage.
* use current locale when converting wide char command line arguments to ↵Dan Engelbrecht2026-01-151-0/+8
| | | | | | string (#712) * set utf8 locale, only set LC_CTYPE as we don't want to affect how decimal point or sorting orders are handled * set language/region explicitly so we can use LC_ALL
* asio/http optimizations (#449)Stefan Boberg2026-01-141-27/+2
| | | | | | | This change primarily introduces improved logic for dealing with sending data from file references. This is intended to reduce the amount of memory-mapping we end up doing when sending data from files. Windows now uses `TransmitFile` to send file data more efficiently using kernel-side I/O, but Linux/Mac basically behaves as before since they don't offer any true async file I/O support via asio. This should be implemented separately using a background I/O thread pool. This PR also includes improved memory management for http/asio with reduced allocation counts, and a fix for a potential use-after-free in very high load scenarios.
* if we fail to create the server mutex, gracefully report error without ↵Dan Engelbrecht2026-01-121-1/+3
| | | | sending error to Sentry (#705)
* Avoid rendering user text input as HTML (#700)Liam Mitchell2026-01-092-1/+1
| | | * Avoid rendering user text input as HTML
* Ensure upstream put propagation includes overwritezousar2025-12-194-5/+39
| | | | When changing the default limit-overwrite behavior, a unit test surfaced a bug where an put of data with overwrite cache policy would not get propagated via zen's built-in upstream mechanism with a matching overwrite cache policy to the upstream. This change ensures that it does and leaves the unit test configured to exercise this scenario.
* Change default limit-overwrite behavior to truezousar2025-12-172-2/+2
|
* remove error warning in output (#695)Dan Engelbrecht2025-12-172-2/+2
| | | * changed some logging string so they don't get caught in CI logging
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-11 21:25:56 +0000