aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* zen hub (#574)Stefan Boberg2026-01-2110-10/+1798
| | | | | Initial implementation of zenserver "hub" mode. This is an experimental feature. zenserver can be started in hub mode by specifying `hub` as the first argument to zenserver
* builds multipart upload (#722)Dan Engelbrecht2026-01-207-1437/+2219
| | | | | | | | | | | | | | | | | | | | | - Feature: `zen builds upload` now support structure manifest input for `--manifest-path` when the path has a `.json` extension - The structured manifest supports splitting a build into multiple parts { "parts": { "default" : { "partId": "f939f3939939fff3f3202", # optional - used to control the id of each part "files": [ "foo/bar", "baz.exe" ] }, "symbols": { "files": [ "baz.pdb" ] } } }
* zenserver API changes, some other minor changes (#720)Stefan Boberg2026-01-199-29/+186
| | | | | | | * add system metrics output to top command * removed unnecessary xmake directives * file system API/comment tweaks * fixed out-of-range access in httpserver test * updated ZenServer base API to allow customization by mode
* ZenServerProcess API changes (#719)Stefan Boberg2026-01-198-48/+162
| | | | | | | This refactor aims to improve the `ZenServerProcess` classes by making them useful for managing child zenserver instances in more scenarios than just automated tests. This involves changing some functions to not talk about "test directory" and instead use "data directory" etc As a consequence of the API changes, some tests have changed accordingly. The code includes som reference to the "hub" mode but there is not yet any other code using this mode, it's just included in this PR to simplify future merges.
* remove ZENCORE_API completely (#718)Stefan Boberg2026-01-1922-423/+404
| | | initially we had ZENCORE_API macros to potentially allow for DLL linkage. It turns out that this is not useful and the macros just contribute noise, so this change removes them completely.
* OTLP/trace improvements (#717)Stefan Boberg2026-01-196-35/+152
| | | | | | | This PR brings over some changes made to avoid performing setup for otel instrumentation if we are not sending otel information anywhere anyway. It also adds the ability to configure an OTLP endpoint on the command line using `--otlp-endpoint=<URI>`. Bear in mind that OTLP support is still not officially supported so this should not be used in production at this stage.
* consul package and basic client added (#716)Stefan Boberg2026-01-198-20/+426
| | | | | | | | * this adds a consul package which can be used to fetch a consul binary * it also adds a `ConsulProcess` helper which can be used to spawn and manage a consul service instance * zencore dependencies brought across: - `except_fmt.h` for easer generation of formatted exception messages - `process.h/cpp` changes (adds `Kill` operation and process group support on Windows) - `string.h` changes to allow generic use of `WideToUtf8()`
* Merge pull request #701 from ue-foundation/lm/mac-daemon-modeLiam Mitchell2026-01-152-41/+35
|\ | | | | Implement final changes required for daemon mode on Mac
| * Run clang-format on service.cppLiam Mitchell2026-01-151-3/+2
| |
| * Rename IsElevated to RequiresElevationLiam Mitchell2026-01-151-9/+9
| |
| * Implement final changes required for daemon mode on MacLiam Mitchell2026-01-072-33/+28
| |
* | throw error on bad oidctoken-exe-path (#713)Dan Engelbrecht2026-01-152-10/+24
| | | | | | | | * throw error on bad oidctoken-exe-path
* | use current locale when converting wide char command line arguments to ↵Dan Engelbrecht2026-01-1511-1/+49
| | | | | | | | | | | | string (#712) * set utf8 locale, only set LC_CTYPE as we don't want to affect how decimal point or sorting orders are handled * set language/region explicitly so we can use LC_ALL
* | fix init of buildpart id in zen oplog-download (#711)Dan Engelbrecht2026-01-151-1/+1
| |
* | structured output for builds ls (#709)Dan Engelbrecht2026-01-143-70/+188
| | | | | | | | * make ResolveBuildStore respect Verbose flag * add structured output to zen builds ls command
* | asio/http optimizations (#449)Stefan Boberg2026-01-142-131/+640
| | | | | | | | | | | | | | This change primarily introduces improved logic for dealing with sending data from file references. This is intended to reduce the amount of memory-mapping we end up doing when sending data from files. Windows now uses `TransmitFile` to send file data more efficiently using kernel-side I/O, but Linux/Mac basically behaves as before since they don't offer any true async file I/O support via asio. This should be implemented separately using a background I/O thread pool. This PR also includes improved memory management for http/asio with reduced allocation counts, and a fix for a potential use-after-free in very high load scenarios.
* | minor fixes (#708)Stefan Boberg2026-01-132-3/+2
| | | | | | | | | | * remove unreferenced local in projectstore_cmd * fix minor atomic memory issue in RotatingFileSink
* | added options to configure exclude folders and extensions to zen build ↵Dan Engelbrecht2026-01-1310-223/+468
| | | | | | | | | | | | | | | | commands (#706) * added `--exclude-folders` to `zen upload`, `zen download` and `zen diff` added `--exclude-extensions` to `zen upload` and `zen diff` excluded folder names are now matched by folder name in subfolders in addition to root level folders * allow multiple token separators
* | fixed block generation after bug introduced in PR #704 (#707)Dan Engelbrecht2026-01-131-12/+15
| |
* | if we fail to create the server mutex, gracefully report error without ↵Dan Engelbrecht2026-01-121-1/+3
| | | | | | | | sending error to Sentry (#705)
* | various optimizations (#704)Dan Engelbrecht2026-01-095-33/+151
| | | | | | | | | | | | | | | | | | - Improvement: Validate chunk hashes when dechunking files in oplog import - Improvement: Use stream decompression when dechunking files - Improvement: When assembling blocks for oplog export, make sure we keep under/at block size limit - Improvement: Make cancelling of oplog import more responsive - Improvement: Use decompress to composite to avoid allocating a new memory buffer for uncompressed chunks during oplog import - Improvement: Reduce memory buffer size and allocate it on demand when writing multiple chunks to block store - Improvement: Reduce lock contention when fetching/checking existence of chunks in block store
* | CprHttpClient cleanup (#703)Dan Engelbrecht2026-01-092-7/+3
| |
* | Avoid rendering user text input as HTML (#700)Liam Mitchell2026-01-092-1/+1
| | | | | | * Avoid rendering user text input as HTML
* | read build state files into memory instead of relying on memory mapping (#702)Dan Engelbrecht2026-01-081-2/+7
| |
* | optimize scavenge part 2 (#699)Dan Engelbrecht2026-01-082-148/+109
|/ | | | * optimize scavenge discovery by iterate over chunks only instead of iterating through all chunks in the scavenged files * refactor scavenge lookup
* added early-out check in GcManager::ScrubStorage(ScrubContext& GcCtx) (#698)Stefan Boberg2026-01-071-1/+7
| | | | | minimises time spent doing setup work after the deadline has expired also added log output with deadline/timeout information
* Add base port getter and set the dll directory for plugin dependencies (#692)Tomasz Obrębski2025-12-193-7/+21
| | | | | | | * Add base port getter and set the dll directory for plugin dependencies * Use UTF8 for dll paths * Rename BasePort to m_BasePort for consistency
* Merge branch 'main' into zs/limit-overwrite-defaultZousar Shaker2025-12-195-254/+260
|\
| * optimize scavenge (#697)Dan Engelbrecht2025-12-195-254/+260
| | | | | | | | * optimize FindScavengeContent * optimize GetValidFolderContent
* | Fix unit test that relies on being able to overwritezousar2025-12-191-2/+2
| |
* | Ensure upstream put propagation includes overwritezousar2025-12-1910-17/+73
| | | | | | | | When changing the default limit-overwrite behavior, a unit test surfaced a bug where an put of data with overwrite cache policy would not get propagated via zen's built-in upstream mechanism with a matching overwrite cache policy to the upstream. This change ensures that it does and leaves the unit test configured to exercise this scenario.
* | Change default limit-overwrite behavior to truezousar2025-12-174-6/+7
|/
* remove error warning in output (#695)Dan Engelbrecht2025-12-177-18/+20
| | | * changed some logging string so they don't get caught in CI logging
* fix scavenge source path match (again) (#694)Dan Engelbrecht2025-12-171-11/+12
| | | | * fix check for scavenged path matching target path
* add boost-worker oplog import export options (#693)Dan Engelbrecht2025-12-1611-116/+311
| | | | | | | | | | - Feature: `zen oplog-export`, `zen oplog-import` and `zen oplog-download` now has options to boost workers - `--boost-worker-count` - Increase the number of worker threads - may cause computer to be less responsive - `--boost-worker-memory` - Increase the limit where we write downloaded data to temporary storage to conserve space - may cause computer to be less responsive due to high memory usage - `--boost-workers` - Enables both 'boost-worker-count' and 'boost-worker-memory' - may cause computer to be less responsive - Improvement: Refactored boost options for `zen builds` operations `upload`, `download`, `diff`, `prime-cache`, `fetch-blob` and `validate-part` - `--boost-worker-count` - Increase the number of worker threads - may cause computer to be less responsive - `--boost-worker-memory` - Increase the limit where we write downloaded data to temporary storage to conserve space - may cause computer to be less responsive due to high memory usage - `--boost-workers` - Enables both 'boost-worker-count' and 'boost-worker-memory' - may cause computer to be less responsive
* remove found chunks as they are found in blocks (#691)Dan Engelbrecht2025-12-151-7/+18
|
* oplog download size (#690)Dan Engelbrecht2025-12-1511-320/+655
| | | | - Bugfix: Upload of oplogs could reference multiple blocks for the same chunk causing redundant downloads of blocks - Improvement: Use the improved block reuse selection function from zen builds upload in zen oplog-export to reduce oplog download size
* show download source data (#689)Dan Engelbrecht2025-12-1212-11/+223
| | | * show source stats for jupiter/cache
* add otel instrumentation (#581)Stefan Boberg2025-12-1113-55/+237
| | | | | | | | this change adds OTEL tracing to a few places * Top-level application lifecycle (config/init/cleanup, main loop) * http.sys requests it also brings some otlptrace optimizations and dynamic configuration of tracing. OTLP tracing is currently always disabled
* remove direct console output from code that is used from service mode (#688)Dan Engelbrecht2025-12-118-107/+244
|
* HTTP server API changes for improved extensibility (#684)Stefan Boberg2025-12-1112-67/+189
| | | | * refactored `HttpServer` so all subclass member functions are proctected, to make it easier to extend base functionality * added API service, can be used to enumerate registered endpoints (at `/api`). Currently only very basic information is provided
* catch all exceptions during projectstore scrub (#686)Dan Engelbrecht2025-12-111-0/+6
|
* fix zero length stream decompress (#685)Dan Engelbrecht2025-12-101-1/+1
| | | * fix validation during stream decompress of zero-size rawsize
* remove catching of exceptions in batch operations now that they are not ↵Dan Engelbrecht2025-12-102-101/+61
| | | | | executed in the destructor (#683) don't call WriteChunks in batch operation if no chunks needs to be written
* fix buffer memory in builds cache (#682)Dan Engelbrecht2025-12-102-18/+74
| | | | * add --zen-cache-upload option to zen oplog-import command * fix buildstoragecache to not hold on to possibly materialized buffers
* fix reading of stale iterator when doing download-resume (#681)Dan Engelbrecht2025-12-091-9/+9
|
* upload to zen cache from oplog import now disabled by default (#680)Dan Engelbrecht2025-12-092-2/+2
|
* safer lambda captures (#633)Dan Engelbrecht2025-12-081-8/+10
|
* implement --dedicated option on asio http server (#679)Dan Engelbrecht2025-12-054-45/+101
| | | * implement --dedicated option on asio http server
* catch exception leaks in windows thread pool (#677)Dan Engelbrecht2025-12-051-4/+18
| | | | * catch exception leaks in windows thread pool * make non-assert exception an error, assert is kept as warning as it is already reported
generated by cgit v1.2.3 (git 2.25.1) at 2026-04-22 08:55:42 +0000