aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* promoted ScopeFd to file scope and applied it to tests and other codesb/block-cloneStefan Boberg2026-02-251-92/+94
| | | | fixes potential file handle leak in ScanFile (if callback throws) and generally makes code more maintainable and aligned with how handles are typically managed on Windows
* add tentative block cloning support for macOS / LinuxStefan Boberg2026-02-251-20/+461
| | | | needs extensive testing
* work around doctest shutdown issues with static CRT (#784)Stefan Boberg2026-02-251-0/+9
| | | | | | * tweaked doctest.h to avoid shutdown issues due to thread_local variables running destructors after the main thread has torn down everything including the heap * disabled zenserver exit thread waiting since doctest should hopefully not be causing issues during shutdown anymore after my workaround This should help reduce the duration of tests spawning lots of server instances
* HttpService/Frontend improvements (#782)Stefan Boberg2026-02-259-35/+191
| | | | | | | - zenhttp: added `GetServiceUri()`/`GetExternalHost()` - enables code to quickly generate an externally reachable URI for a given service - frontend: improved Uri handling (better defaults) - added support for 404 page (to make it easier to find a good URL)
* Fix zencore bugs and propagate content type through IoBufferBuilder (#783)Stefan Boberg2026-02-246-27/+65
| | | | | | | - Add missing includes in hashutils.h (`<cstddef>`, `<type_traits>`) - Add `ZenContentType` parameter to all `IoBufferBuilder` factory methods so content type is set at buffer creation time - Fix null dereference in `SharedBuffer::GetFileReference()` when buffer is null - Fix out-of-bounds read in trace command-line argument parsing when arg length exactly matches option length - Add unit tests for 32-bit `CountLeadingZeros`
* use partial blocks for oplog import (#780)Dan Engelbrecht2026-02-2427-362/+1243
| | | | | Feature: Add --allow-partial-block-requests to zen oplog-import Improvement: zen oplog-import now uses partial block requests to reduce download size Improvement: Use latency to Cloud Storage host and Zen Cache host when calculating partial block requests
* Add `zen ui` command (#779)Stefan Boberg2026-02-2410-54/+1054
| | | | | Allows user to automate launching of zenserver dashboard, including when multiple instances are running. If multiple instances are running you can open all dashboards with `--all`, and also using the in-terminal chooser which also allows you to open a specific instance. Also includes a fix to `zen exec` when using offset/stride/limit
* Various bug fixes (#778)Stefan Boberg2026-02-2420-69/+93
| | | | | | | | | | | | | | | | | | | | | | zencore fixes: - filesystem.cpp: ReadFile error reporting logic - compactbinaryvalue.h: CbValue::As*String error reporting logic zenhttp fixes: - httpasio BindAcceptor would `return 0;` in a function returning `std::string` (UB) - httpsys async workpool initialization race zenstore fixes: - cas.cpp: GetFileCasResults Results param passed by value instead of reference (large chunk results were silently lost) - structuredcachestore.cpp: MissCount unconditionally incremented (counted hits as misses) - cacherpc.cpp: Wrong boolean in Incomplete response array (all entries marked incomplete) - cachedisklayer.cpp: sizeof(sizeof(...)) in two validation checks computed sizeof(size_t) instead of struct size - buildstore.cpp: Wrong hash tracked in GC key list (BlobHash pushed twice instead of MetadataHash) - buildstore.cpp: Removed duplicate m_LastAccessTimeUpdateCount increment in PutBlob zenserver fixes: - httpbuildstore.cpp: Reversed subtraction in HTTP range calculation (unsigned underflow) - hubservice.cpp: Deadlock in Provision() calling Wake() while holding m_Lock (extracted WakeLocked helper) - zipfs.cpp: Data race in GetFile() lazy initialization (added RwLock with shared/exclusive paths)
* Revert "Fix correctness and concurrency bugs found during code review"Stefan Boberg2026-02-249-42/+20
| | | | This reverts commit 3c89c486338890ce39ddebe5be4722a09e85701a.
* Fix correctness and concurrency bugs found during code reviewStefan Boberg2026-02-249-20/+42
| | | | | | | | | | | | | | | | | zenstore fixes: - cas.cpp: GetFileCasResults Results param passed by value instead of reference (large chunk results were silently lost) - structuredcachestore.cpp: MissCount unconditionally incremented (counted hits as misses) - cacherpc.cpp: Wrong boolean in Incomplete response array (all entries marked incomplete) - cachedisklayer.cpp: sizeof(sizeof(...)) in two validation checks computed sizeof(size_t) instead of struct size - buildstore.cpp: Wrong hash tracked in GC key list (BlobHash pushed twice instead of MetadataHash) - buildstore.cpp: Removed duplicate m_LastAccessTimeUpdateCount increment in PutBlob zenserver fixes: - httpbuildstore.cpp: Reversed subtraction in HTTP range calculation (unsigned underflow) - hubservice.cpp: Deadlock in Provision() calling Wake() while holding m_Lock (extracted WakeLocked helper) - zipfs.cpp: Data race in GetFile() lazy initialization (added RwLock with shared/exclusive paths) Co-Authored-By: Claude Opus 4.6 <[email protected]>
* implement yaml generation (#774)Stefan Boberg2026-02-232-127/+302
| | | | | this implements a yaml generation strategy similar to the JSON generation where we just build a string instead of building a ryml tree. This also removes the dependency on ryml for reduced binary/build times.
* changed command names and descriptions to use class members instead of ↵Stefan Boberg2026-02-2317-90/+211
| | | | string literals in zen.cpp (#776)
* implemented base64 decoding (#777)Stefan Boberg2026-02-232-2/+194
| | | Co-authored-by: Stefan Boberg <[email protected]>
* move partial chunk block anailsys to chunkblock.h/cpp (#767)Dan Engelbrecht2026-02-204-679/+887
|
* fix builds download indexing timer (#769)Dan Engelbrecht2026-02-202-7/+7
| | | * fix build download indexing timer log
* fix plain progress bar (#768)Dan Engelbrecht2026-02-201-0/+1
| | | * fix plain progress not updating current state
* fix MakeSafeAbsolutePathInPlace mis-spelling (#765)Stefan Boberg2026-02-208-49/+49
| | | | | (was MakeSafeAbsolutePathÍnPlace - note accent) Also fixed misleading comments on multiple functions in filesystem.h
* GC - fix handling of attachment ranges, http access token expiration, lock ↵Stefan Boberg2026-02-207-12/+18
| | | | | | | | file retry logic (#766) * GC - fix handling of attachment ranges * fix trace/log strings * fix HTTP access token expiration time logic * added missing lock retry in zenserver startup
* Merge branch 'main' into zs/web-ui-improvementszousar2026-02-1850-99/+7853
|\
| * structured compute basics (#714)Stefan Boberg2026-02-1845-47/+7639
| | | | | | | | | | | | | | | | | | this change adds the `zencompute` component, which can be used to distribute work dispatched from UE using the DDB (Derived Data Build) APIs via zenserver this change also adds a distinct zenserver compute mode (`zenserver compute`) which is intended to be used for leaf compute nodes to exercise the compute functionality without directly involving UE, a `zen exec` subcommand is also added, which can be used to feed replays through the system all new functionality is considered *experimental* and disabled by default at this time, behind the `zencompute` option in xmake config
| * add selective request logging support to http.sys (#762)Stefan Boberg2026-02-183-39/+156
| | | | | | | | | | * implemented selective request logging for http.sys for consistency with asio * fixed traversal of GetLogicalProcessorInformationEx to account for variable-sized records * also adds CPU usage metrics
| * convert ZEN_ASSERTs to exception to handle corrupt data gracefully (#760)Dan Engelbrecht2026-02-182-13/+58
| | | | | | * convert ZEN_ASSERTs to exception to handle corrupt data gracefully
* | updatefrontendzousar2026-02-181-0/+0
| |
* | entry.js handles missing/native items more gracefullyzousar2026-02-182-4/+32
| |
* | updatefrontendzousar2026-02-171-0/+0
| |
* | Merge branch 'main' into zs/web-ui-improvementszousar2026-02-1725-197/+478
|\|
| * add http server root password protection (#757)Dan Engelbrecht2026-02-1719-166/+415
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Feature: Added `--security-config-path` option to zenserver to configure security settings - Expects a path to a .json file - Default is an empty path resulting in no extra security settings and legacy behavior - Current support is a top level filter of incoming http requests restricted to the `password` type - `password` type will check the `Authorization` header and match it to the selected authorization strategy - Currently the security settings is very basic and configured to a fixed username+password at startup { "http" { "root": { "filter": { "type": "password", "config": { "password": { "username": "<username>", "password": "<password>" }, "protect-machine-local-requests": false, "unprotected-uris": [ "/health/", "/health/info", "/health/version" ] } } } } }
| * misc fixes brought over from sb/proto (#759)Stefan Boberg2026-02-177-31/+49
| | | | | | | | | | | | | | | | * `RwLock::WithSharedLock` and `RwLock::WithExclusiveLock` can now return a value (which is returned by the passed function) * Comma-separated logger specification now correctly deals with commas * `GetSystemMetrics` properly accounts for cores * cpr response formatter passes arguments in the right order * `HttpServerRequest::SetLogRequest` can be used to selectively log HTTP requests
| * added ResetConsoleLog (#758)Stefan Boberg2026-02-163-0/+14
| | | | | | also made sure log initialization calls it to ensure the console output format is retained even if the console logger was set up before logging is initialized
* | Dependencies table doesn't reflow the entries pagezousar2026-02-171-5/+19
| |
* | Rename the cache section in the web uizousar2026-02-171-1/+1
| |
* | Make files table in entry.js paginated and searchablezousar2026-02-171-40/+170
| |
* | Added custom page for cook.artifactszousar2026-02-164-4/+428
| |
* | Change breadcrumbs for oplogs to be more descriptivezousar2026-02-152-6/+12
| |
* | Add support for listing files on oplog entrieszousar2026-02-151-9/+110
| |
* | Restore handling for hard/soft name prefixeszousar2026-02-151-4/+16
| |
* | Enhance dependencies to include soft and hard depszousar2026-02-141-11/+13
|/
* add foundation for http password protection (#756)Dan Engelbrecht2026-02-133-0/+275
|
* spelling fixes (#755)Dan Engelbrecht2026-02-131-2/+2
|
* logging config move to zenutil (#754)Stefan Boberg2026-02-1317-99/+187
| | | made logging config options from zenserver available in zen CLI
* add IHttpRequestFilter to allow server implementation to filter/reject ↵Dan Engelbrecht2026-02-1310-96/+353
| | | | | requests (#753) * add IHttpRequestFilter to allow server implementation to filter/reject requests
* add IsLocalMachineRequest to HttpServerRequest (#749)Dan Engelbrecht2026-02-126-7/+78
| | | * add IsLocalMachineRequest to HttpServerRequest
* add simple http client tests (#751)Dan Engelbrecht2026-02-128-27/+202
| | | * add simple http client tests and fix run loop of http server to not rely on application quit
* reduce lock time for project store gc precache and gc validate (#750)Dan Engelbrecht2026-02-112-46/+261
| | | | | * add oplog snapshot function to allow reduction of held oplog locks * release project lock when precaching each oplog
* Fix formatting of stat pages (#748)Liam Mitchell2026-02-094-3/+10
| | | * Fix formatting of stat pages
* ported optimizations of MeasureVarUInt (#747)Dan Engelbrecht2026-02-061-2/+12
|
* fix target folder cleanup in builds download (#746)Dan Engelbrecht2026-02-061-12/+9
|
* only check file size for requested range in asio TransmitFileAsync in debug ↵Dan Engelbrecht2026-02-061-41/+63
| | | | | | mode (#745) * only check file size for requested range in asio TransmitFileAsync in debug mode * only do Transmit file for whole files
* Merge pull request #723 from ue-foundation/lm/service-interactive-sessionLiam Mitchell2026-02-044-40/+16
|\ | | | | Run Windows service as an interactive session, so we correctly respond to RequestApplicationExit when requested to stop by the service manager
| * Revert "Run Windows service as an interactive session, so we correctly ↵Liam Mitchell2026-02-031-2/+2
| | | | | | | | | | | | respond to RequestApplicationExit when requested to stop by the service manager" This reverts commit b380c1bac4b738f49cd412697e0a051402e6d171.