aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
...
* projectstore refactor (#531)Dan Engelbrecht2025-09-304-721/+667
| | | | | | | | | | | | | | | | | * convert ProjectStore::GetProjectFiles to not use http return codes * convert ProjectStore::GetProjectChunkInfos to not use http return codes * convert ProjectStore::GetChunkInfo to not use http return codes * convert ProjectStore::GetChunkRange to not use http return codes * convert ProjectStore::GetChunk to not use http return codes * convert ProjectStore::PutChunk to not use http return codes * convert ProjectStore::WriteOplog to not use http return codes * convert ProjectStore::ReadOplog to not use http return codes
* fix bounds check when finalizing build state (#533)Dan Engelbrecht2025-09-301-1/+2
|
* HttpClient support for pluggable back-ends (#532)Stefan Boberg2025-09-306-1634/+1942
| | | refactored HttpClient to separate out cpr implementation into separate classes, with an abstract base class to allow plugging in multiple implementations in the future
* parsing of `zen builds` `--log-progress` option fixed (#530)Dan Engelbrecht2025-09-291-9/+13
|
* more cbobject validations (#527)Dan Engelbrecht2025-09-291-5/+19
| | | - Improvement: Add additional validations when reading disk cache records to get references in GC
* split zenserver-test monolith into multiple source files (#528)Stefan Boberg2025-09-296-4602/+4701
|
* gracefully handle missing chunks when exporting an oplog (#526)Dan Engelbrecht2025-09-292-28/+66
| | | | - Improvement: Gracefully handle missing chunks when exporting an oplog - Bugfix: Add object validation when replaying oplog to avoid crash on malformed data
* builds multi wildcard (#524)Dan Engelbrecht2025-09-291-50/+83
| | | * allow multiple include/exclude wildcards in zen builds command
* fix race condition in BlockStoreFile::Flush (#525)Dan Engelbrecht2025-09-291-2/+2
| | | Bugfix: Flush of blockstore file could sometimes cause an error due to a race condition
* GetCacheChunk value request respects RawHash (#518)Zousar Shaker2025-09-291-1/+3
| | | When requesting a value via the GetCacheChunks rpc method, if the request specified a raw hash, only return a hit if the raw hash matches what was in the cache.
* make cpr a HttpClient implementation detail (#517)Stefan Boberg2025-09-2919-1094/+854
| | | | | these changes remove cpr from anything which is not `HttpClient` internals. The goal is to eventually replace cpr with a more direct curl interface to eliminate cpr since it's proven problematic due to their development practices which frequently breaks APIs and prevents us from updating vcpkg. But this PR is limited to refactoring existing cpr code to use `HttpClient` instead.
* fixed race condition in zen::logging::Get (#519)Stefan Boberg2025-09-291-3/+12
| | | if two requests for the same logger came in from different threads they could end up creating the same logger twice which causes an exception on registration
* remove spurious cpr.h includes (#520)Stefan Boberg2025-09-296-15/+1
|
* fixed ParsePackageMessage error message when mismatched header magic is ↵Stefan Boberg2025-09-291-1/+1
| | | | detected (#521)
* some bug fixes (#522)Stefan Boberg2025-09-292-3/+4
| | | | | | * fix for invalid regex in HttpBuildStoreService - triggers with most recent MSVC version * in GcScheduler don't wait for exit signal if exit has already been requested. this caused extended waits for shutdown in some automated tests on very fast machines, possibly also due to some behaviour change in condition_variable * speculative fix/workaround for issue with TLS teardown on secondary thread while main was tearing down trace
* make sure trace is configured for all test drivers (#523)Stefan Boberg2025-09-296-1/+48
|
* more iterate chunk logging (#516)Dan Engelbrecht2025-09-263-13/+57
| | | | | * add log warnings when we can't read payloads in cas when we thing we should have them * fix misleading option help
* Make sure we call the previous terminate handle if present when we intercept ↵Dan Engelbrecht2025-09-266-23/+123
| | | | | | terminate calls (#514) Improvement: Make sure we call the previous terminate handle if present when we intercept terminate calls Improvement: Avoid allocating memory for call stack in terminate handle and assert callback
* fix for C4244 truncation warning (#515)Stefan Boberg2025-09-261-1/+3
| | | Fixes warning/error introduced with msvc cl 19.38.33145
* list missing block hashes when --verbose is enabled (#512)Dan Engelbrecht2025-09-261-0/+14
|
* new append op rpc method (#511)Dan Engelbrecht2025-09-266-229/+676
| | | | | | - Feature: New `/prj/{project}/{oplog}/rpc` endpoint method `appendops` to send an array of oplog ops and receiving a list of `need` for attachments not present - Feature: Added `usingtmpfiles` boolean field to `/prj/{project}/{oplog}/rpc` method `putchunks` to be explicit about allowing move of temp attachment files - Improvement: Added additional validation of compact binary objects when reading from disk/receiving from client - Improvement: Windows: Added fallback code to use standard `MoveFile` api when rename via `SetFileInformationByHandle` fails in `MoveToFile` (used by filecas)
* Merge pull request #509 from ue-foundation/zs/put-overwrite-policy-responseZousar Shaker2025-09-259-49/+157
|\ | | | | Zs/put overwrite policy response
| * Improvement to Incomplete Result Iterationzousar2025-09-251-2/+1
| | | | | | | | From review feedback
| * Report Incomplete Records To Clientzousar2025-09-242-8/+44
| | | | | | | | When requesting partial records, report back when a record is incomplete via an "Incomplete" array of bools that is a sibling to the "Result" array for batch/rpc operations, or via the HttpResponseCode::PartialContent status code for individual record requests.
| * Adjust the responses from PUT commandszousar2025-09-234-15/+10
| | | | | | | | | | - Ensure that text responses are in a field named "Message" - Change the record response to be named "Record" instead of "Object"
| * Merge branch 'main' into zs/put-overwrite-policy-responsezousar2025-09-2226-842/+1647
| |\
| * | Change batch put responses for client reportingzousar2025-09-199-41/+119
| | | | | | | | | | | | Conflicts are now treated as successes, and we optionally return a Details array instead of an ErrorMessages array. Details are returned for all requests in a batch, or no requests in a batch depending on whether there are any details to be shared about any of the put requests. The details for a conflict include the raw hash and raw size of the item. If the item is a record, we also include the record as an object.
* | | limit number of chunks per block (#508)Dan Engelbrecht2025-09-237-61/+202
| |/ |/| | | | | | | - Improvement: Chunk block generation for `zen oplog-export` and `zen builds upload` command are now limited to max 4000 chunks per block zen builds list-block testing function to check results for recently used blocks in a context
* | dont set Size or RawSize when fetching chunkinfos if the payload can't be ↵Dan Engelbrecht2025-09-221-2/+7
| | | | | | | | found/validated (#507)
* | more responsive cancel during oplog import (#505)Dan Engelbrecht2025-09-223-12/+31
| | | | | | | | - Improvement: Faster oplog import due to chunk existance check improvement - Improvement: Cancelling oplog import is now more responsive during initial phase
* | Added `--oidctoken-exe-unattended` to`zen builds` and `zen oplog-download` ↵Dan Engelbrecht2025-09-229-12/+29
| | | | | | | | command to use unattended mode when launching oidc-token.exe (#506)
* | change default sentry dsn to one listed on Sentry setup page (#504)Dan Engelbrecht2025-09-221-1/+1
| |
* | issue error on second retry, not first attempt (#503)Dan Engelbrecht2025-09-221-2/+2
| |
* | fetch cloud oplog (#502)Dan Engelbrecht2025-09-2214-440/+909
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Feature: Added `zen oplog-download` command to download the oplog body of a cooked output stored in Cloud DDC - Oplog source is specified using one of the following options - `--cloud-url` Cloud artifact URL for oplog - `--host` Base host to resolve download host from - `--override-host` Specific host to use without resolve - `--assume-http2` assume that the builds endpoint is a HTTP/2 endpoint skipping HTTP/1.1 upgrade handshake - `--namespace` Builds Storage namespace - `--bucket` Builds Storage bucket - `--build-id` an Oid in hex form for the source identifier to use - `--yes` suppress conformation query when doing output of a very large oplog to console - `--quiet` suppress all non-essential console output - `--output-path` path to oplog output, extension .json or .cb (compact binary). Default is output to console - `--system-dir` override default system root path - Authentication options - Auth token - `--access-token` http auth Cloud Storage access token - `--access-token-env` name of environment variable that holds the Http auth Cloud Storage access token - `--access-token-path` path to json file that holds the Http auth Cloud Storage access token - `--oidctoken-exe-path` path to OidcToken executable - OpenId authentication - `--openid-provider-name` Open ID provider name - `--openid-provider-url` Open ID provider url - `--openid-client-id`Open ID client id - `--openid-refresh-token` Open ID refresh token - `--encryption-aes-key` 256 bit AES encryption key for storing OpenID credentials - `--encryption-aes-iv` 128 bit AES encryption initialization vector for storing OpenID credentials - OAuth authentication - `--oauth-url` OAuth provier url - `--oauth-clientid` OAuth client id - `--oauth-clientsecret` OAuth client secret - Bugfix: `zen print` command now properly outputs very large compact binary objects as json to console
* | improve builds download partial logic (#501)Dan Engelbrecht2025-09-223-376/+671
|/ | | | | | | | | | - Improvement: `zen command` help now uses available horizontal space for output - Improvement: Partial block request analisys has been improved with reduced download size at lower number of requests - Improvement: Validate that `--zen-cache-host` exists and is responsive before begin used - Feature: Options `--allow-partial-block-requests` for `zen builds download` command has been augmented with two new modes, `zencacheonly` and `mixed`. Defaults to `mixed`. - `false` only full block requests allowed - `mixed` multiple partial block ranges requests per block allowed to zen cache, single partial block range request per block to host - `zencacheonly` multiple partial block ranges requests per block allowed to zen cache, only full block requests allowed to host - `true` multiple partial block ranges requests per block allowed to zen cache and host
* fix quoted wildcard options (#500)Dan Engelbrecht2025-09-172-34/+24
| | | * improve command line parsing to automatically remove quotes
* add builds download force option (#498)Dan Engelbrecht2025-09-172-270/+297
| | | * add --force option to builds download command
* Sorting oplog tree view by size would raise an error. (#497)Martin Ridgers2025-09-162-1/+1
| | | | | | | * Sorting oplog tree view by size would raise an error * Updated frontend .zip archive * Updated changelog
* new in-memory storage strategy for oplogs (#490)Dan Engelbrecht2025-09-154-518/+748
| | | - Improvement: Revised project oplog in-memory representation which reduces load times and memory usage
* revise exception vs error (#495)Dan Engelbrecht2025-09-154-26/+34
| | | | | - Change BadAlloc exceptions in GC to warnings - Add explict ASSERT exception catch in http plugin request processing - Make exceptions handled in http request processing to warnings
* initialize all services before healthy (#493)Dan Engelbrecht2025-09-121-35/+58
| | | * make sure all services are initialized and registered before we respond OK to health check
* add EMode to WorkerTheadPool to avoid thread starvation (#492)Dan Engelbrecht2025-09-1030-1421/+1602
| | | - Improvement: Add a new mode to worker thread pools to avoid starvation of workers which could cause long stalls due to other work begin queued up. UE-305498
* faster oplog entries with referenceset (#488)Dan Engelbrecht2025-09-087-57/+88
| | | | - Improvement: Faster project store `/entries` endpoint, 10-15% faster when using a reference set to limit entries fix missing space after task name in pretty progressbar
* Set ownership of service executables to the service user on Linux (#489)Liam Mitchell2025-09-081-0/+44
| | | - Bugfix: Linux only, set ownership of installed files to specified user when using `zen service install --full --user`
* refactor zen command return value handling (#487)Dan Engelbrecht2025-09-0547-1083/+972
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Improvement: Use consistent language for command line argument parsing errors - Improvement: Changed zen command parsing errors to output help first and error last to make it easier to spot the error - Improvement: Refactor zen command return codes to conform to valid Linux range (0-255) kSuccess = 0, kOtherError = 1, kBadInput = 2, kOutOfMemory = 16, kOutOfDisk = 17, kAssertError = 70, kHttpOtherClientError = 80, kHttpCantConnectError = 81, kHttpNotFound = 66, // NotFound(404) kHttpUnauthorized = 77, // Unauthorized(401), kHttpSLLError = 82, kHttpForbidden = 83, // Forbidden(403) kHttpTimeout = 84, // RequestTimeout(408) kHttpConflict = 85, // Conflict(409) kHttpNoHost = 86, kHttpOtherServerError = 90, kHttpInternalServerError = 91, // InternalServerError(500) kHttpServiceUnavailable = 69, // ServiceUnavailable(503) kHttpBadGateway = 92, // BadGateway(502) kHttpGatewayTimeout = 93, // GatewayTimeout(504)
* Avoid mutating executable paths when copying files during full service ↵Liam Mitchell2025-09-041-5/+7
| | | | install (#486)
* asio large header support (#484)Dan Engelbrecht2025-09-042-123/+128
| | | - Improvement: Allow large headers support in asio server implementation
* add validation of compact binary payloads before reading them (#483)Dan Engelbrecht2025-09-0420-289/+536
| | | * add validation of compact binary payloads before reading them
* oplog memory usage reduction (#482)Dan Engelbrecht2025-09-044-99/+197
| | | | | | - Improvement: For projectstore oplog GET operation, only read basic information and release it if the oplog is not already open to reduce memory usage when listing oplogs in web UI - Improvement: Reduce memory usage for oplog Op address lookup Refactored Oplog::EState -> Oplog ::EMode and make sure we open data files in read-only mode when EMode::kBasicReadOnly is used.
* projectstore lifetime improvements (#481)Dan Engelbrecht2025-08-294-368/+552
| | | | | - Improvement: Oplogs that have not been touched for 15 min are unloaded from memory during GC pass, oplogs are reloaded on demand - Improvement: Oplogs read for GC/Validation operations are loaded using a lightweight mode reducing memory usage and load times - Improvement: Cleaned up logging for oplogs/projects to reduce clutter
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-27 01:40:45 +0000