aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
...
* | asio/http optimizations (#449)Stefan Boberg2026-01-142-131/+640
| | | | | | | | | | | | | | This change primarily introduces improved logic for dealing with sending data from file references. This is intended to reduce the amount of memory-mapping we end up doing when sending data from files. Windows now uses `TransmitFile` to send file data more efficiently using kernel-side I/O, but Linux/Mac basically behaves as before since they don't offer any true async file I/O support via asio. This should be implemented separately using a background I/O thread pool. This PR also includes improved memory management for http/asio with reduced allocation counts, and a fix for a potential use-after-free in very high load scenarios.
* | minor fixes (#708)Stefan Boberg2026-01-132-3/+2
| | | | | | | | | | * remove unreferenced local in projectstore_cmd * fix minor atomic memory issue in RotatingFileSink
* | added options to configure exclude folders and extensions to zen build ↵Dan Engelbrecht2026-01-1310-223/+468
| | | | | | | | | | | | | | | | commands (#706) * added `--exclude-folders` to `zen upload`, `zen download` and `zen diff` added `--exclude-extensions` to `zen upload` and `zen diff` excluded folder names are now matched by folder name in subfolders in addition to root level folders * allow multiple token separators
* | fixed block generation after bug introduced in PR #704 (#707)Dan Engelbrecht2026-01-131-12/+15
| |
* | if we fail to create the server mutex, gracefully report error without ↵Dan Engelbrecht2026-01-121-1/+3
| | | | | | | | sending error to Sentry (#705)
* | various optimizations (#704)Dan Engelbrecht2026-01-095-33/+151
| | | | | | | | | | | | | | | | | | - Improvement: Validate chunk hashes when dechunking files in oplog import - Improvement: Use stream decompression when dechunking files - Improvement: When assembling blocks for oplog export, make sure we keep under/at block size limit - Improvement: Make cancelling of oplog import more responsive - Improvement: Use decompress to composite to avoid allocating a new memory buffer for uncompressed chunks during oplog import - Improvement: Reduce memory buffer size and allocate it on demand when writing multiple chunks to block store - Improvement: Reduce lock contention when fetching/checking existence of chunks in block store
* | CprHttpClient cleanup (#703)Dan Engelbrecht2026-01-092-7/+3
| |
* | Avoid rendering user text input as HTML (#700)Liam Mitchell2026-01-092-1/+1
| | | | | | * Avoid rendering user text input as HTML
* | read build state files into memory instead of relying on memory mapping (#702)Dan Engelbrecht2026-01-081-2/+7
| |
* | optimize scavenge part 2 (#699)Dan Engelbrecht2026-01-082-148/+109
|/ | | | * optimize scavenge discovery by iterate over chunks only instead of iterating through all chunks in the scavenged files * refactor scavenge lookup
* added early-out check in GcManager::ScrubStorage(ScrubContext& GcCtx) (#698)Stefan Boberg2026-01-071-1/+7
| | | | | minimises time spent doing setup work after the deadline has expired also added log output with deadline/timeout information
* Add base port getter and set the dll directory for plugin dependencies (#692)Tomasz Obrębski2025-12-193-7/+21
| | | | | | | * Add base port getter and set the dll directory for plugin dependencies * Use UTF8 for dll paths * Rename BasePort to m_BasePort for consistency
* Merge branch 'main' into zs/limit-overwrite-defaultZousar Shaker2025-12-195-254/+260
|\
| * optimize scavenge (#697)Dan Engelbrecht2025-12-195-254/+260
| | | | | | | | * optimize FindScavengeContent * optimize GetValidFolderContent
* | Fix unit test that relies on being able to overwritezousar2025-12-191-2/+2
| |
* | Ensure upstream put propagation includes overwritezousar2025-12-1910-17/+73
| | | | | | | | When changing the default limit-overwrite behavior, a unit test surfaced a bug where an put of data with overwrite cache policy would not get propagated via zen's built-in upstream mechanism with a matching overwrite cache policy to the upstream. This change ensures that it does and leaves the unit test configured to exercise this scenario.
* | Change default limit-overwrite behavior to truezousar2025-12-174-6/+7
|/
* remove error warning in output (#695)Dan Engelbrecht2025-12-177-18/+20
| | | * changed some logging string so they don't get caught in CI logging
* fix scavenge source path match (again) (#694)Dan Engelbrecht2025-12-171-11/+12
| | | | * fix check for scavenged path matching target path
* add boost-worker oplog import export options (#693)Dan Engelbrecht2025-12-1611-116/+311
| | | | | | | | | | - Feature: `zen oplog-export`, `zen oplog-import` and `zen oplog-download` now has options to boost workers - `--boost-worker-count` - Increase the number of worker threads - may cause computer to be less responsive - `--boost-worker-memory` - Increase the limit where we write downloaded data to temporary storage to conserve space - may cause computer to be less responsive due to high memory usage - `--boost-workers` - Enables both 'boost-worker-count' and 'boost-worker-memory' - may cause computer to be less responsive - Improvement: Refactored boost options for `zen builds` operations `upload`, `download`, `diff`, `prime-cache`, `fetch-blob` and `validate-part` - `--boost-worker-count` - Increase the number of worker threads - may cause computer to be less responsive - `--boost-worker-memory` - Increase the limit where we write downloaded data to temporary storage to conserve space - may cause computer to be less responsive due to high memory usage - `--boost-workers` - Enables both 'boost-worker-count' and 'boost-worker-memory' - may cause computer to be less responsive
* remove found chunks as they are found in blocks (#691)Dan Engelbrecht2025-12-151-7/+18
|
* oplog download size (#690)Dan Engelbrecht2025-12-1511-320/+655
| | | | - Bugfix: Upload of oplogs could reference multiple blocks for the same chunk causing redundant downloads of blocks - Improvement: Use the improved block reuse selection function from zen builds upload in zen oplog-export to reduce oplog download size
* show download source data (#689)Dan Engelbrecht2025-12-1212-11/+223
| | | * show source stats for jupiter/cache
* add otel instrumentation (#581)Stefan Boberg2025-12-1113-55/+237
| | | | | | | | this change adds OTEL tracing to a few places * Top-level application lifecycle (config/init/cleanup, main loop) * http.sys requests it also brings some otlptrace optimizations and dynamic configuration of tracing. OTLP tracing is currently always disabled
* remove direct console output from code that is used from service mode (#688)Dan Engelbrecht2025-12-118-107/+244
|
* HTTP server API changes for improved extensibility (#684)Stefan Boberg2025-12-1112-67/+189
| | | | * refactored `HttpServer` so all subclass member functions are proctected, to make it easier to extend base functionality * added API service, can be used to enumerate registered endpoints (at `/api`). Currently only very basic information is provided
* catch all exceptions during projectstore scrub (#686)Dan Engelbrecht2025-12-111-0/+6
|
* fix zero length stream decompress (#685)Dan Engelbrecht2025-12-101-1/+1
| | | * fix validation during stream decompress of zero-size rawsize
* remove catching of exceptions in batch operations now that they are not ↵Dan Engelbrecht2025-12-102-101/+61
| | | | | executed in the destructor (#683) don't call WriteChunks in batch operation if no chunks needs to be written
* fix buffer memory in builds cache (#682)Dan Engelbrecht2025-12-102-18/+74
| | | | * add --zen-cache-upload option to zen oplog-import command * fix buildstoragecache to not hold on to possibly materialized buffers
* fix reading of stale iterator when doing download-resume (#681)Dan Engelbrecht2025-12-091-9/+9
|
* upload to zen cache from oplog import now disabled by default (#680)Dan Engelbrecht2025-12-092-2/+2
|
* safer lambda captures (#633)Dan Engelbrecht2025-12-081-8/+10
|
* implement --dedicated option on asio http server (#679)Dan Engelbrecht2025-12-054-45/+101
| | | * implement --dedicated option on asio http server
* catch exception leaks in windows thread pool (#677)Dan Engelbrecht2025-12-051-4/+18
| | | | * catch exception leaks in windows thread pool * make non-assert exception an error, assert is kept as warning as it is already reported
* exists must be called before equivalent (#678)Dan Engelbrecht2025-12-051-1/+1
|
* batch op not in destructor (#676)Dan Engelbrecht2025-12-046-343/+553
| | | | | * use fixed vectors for batch requests * refactor cache batch value put/get to not execute code that can throw execeptions in destructor * extend test with multi-bucket requests
* add checks to protect against access violation due to failed disk read (#675)Dan Engelbrecht2025-12-042-0/+17
| | | * add checkes to protect against access violation due to failed disk read
* add missing patterns to objectstore service http routing (#674)Dan Engelbrecht2025-12-031-3/+7
|
* make sure we use exclusive lock in projectstore when flushing/writing ↵Dan Engelbrecht2025-12-012-12/+37
| | | | snapshot (#673)
* use scope guards to secure that BasicFile::Detach is called even on ↵Dan Engelbrecht2025-12-012-11/+6
| | | | exception (#672)
* fix crash when parsing empty key in httpstats service (#671)Dan Engelbrecht2025-12-011-2/+1
|
* automatic scrub on startup (#667)Dan Engelbrecht2025-11-2726-467/+748
| | | | | - Improvement: Deeper validation of data when scrub is activated (cas/cache/project) - Improvement: Enabled more multi threading when running scrub operations - Improvement: Added means to force a scrub operation at startup with a new release using ZEN_DATA_FORCE_SCRUB_VERSION variable in xmake.lua
* remove bad assert (#670)Dan Engelbrecht2025-11-271-4/+0
|
* fix stream decompress (#668)Dan Engelbrecht2025-11-271-10/+14
| | | * fix source file offset in DecompressToStream implementations
* RawOffset can be anything and we expect an empty buffer to be returned along ↵Dan Engelbrecht2025-11-263-21/+224
| | | | with RawSize = 0 if the offset was out of bounds for the value. (#666)
* remove 'auto' option for zen builds download (#665)Dan Engelbrecht2025-11-252-52/+4
|
* logging improvements (#664)Stefan Boberg2025-11-244-21/+34
| | | | | * Eliminate spurious `FlushFileBuffers` calls on Windows by tracking writes * Make cache log file creation optional
* Add regex-free route matching support (#662)Stefan Boberg2025-11-242-50/+474
| | | | | This change adds support for non-regex matching of routes. Instead of using regex patterns you can associate matcher functions with pattern names and string literal components are identified and matched directly. Also implemented tests for `HttpRequestRouter` class.
* update state when wildcard (#657)Dan Engelbrecht2025-11-2413-1028/+2037
| | | * add --append option and improve state handling when using downloads for `zen builds download`
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-13 17:38:57 +0000