aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
* make sure trace is configured for all test drivers (#523)Stefan Boberg2025-09-296-1/+48
|
* 5.7.2v5.7.2Dan Engelbrecht2025-09-261-1/+1
|
* 5.7.2-pre3v5.7.2-pre3Dan Engelbrecht2025-09-261-1/+1
|
* more iterate chunk logging (#516)Dan Engelbrecht2025-09-264-13/+58
| | | | | * add log warnings when we can't read payloads in cas when we thing we should have them * fix misleading option help
* Make sure we call the previous terminate handle if present when we intercept ↵Dan Engelbrecht2025-09-267-23/+125
| | | | | | terminate calls (#514) Improvement: Make sure we call the previous terminate handle if present when we intercept terminate calls Improvement: Avoid allocating memory for call stack in terminate handle and assert callback
* fix for C4244 truncation warning (#515)Stefan Boberg2025-09-261-1/+3
| | | Fixes warning/error introduced with msvc cl 19.38.33145
* list missing block hashes when --verbose is enabled (#512)Dan Engelbrecht2025-09-262-0/+15
|
* new append op rpc method (#511)Dan Engelbrecht2025-09-267-229/+680
| | | | | | - Feature: New `/prj/{project}/{oplog}/rpc` endpoint method `appendops` to send an array of oplog ops and receiving a list of `need` for attachments not present - Feature: Added `usingtmpfiles` boolean field to `/prj/{project}/{oplog}/rpc` method `putchunks` to be explicit about allowing move of temp attachment files - Improvement: Added additional validation of compact binary objects when reading from disk/receiving from client - Improvement: Windows: Added fallback code to use standard `MoveFile` api when rename via `SetFileInformationByHandle` fails in `MoveToFile` (used by filecas)
* Merge pull request #509 from ue-foundation/zs/put-overwrite-policy-responseZousar Shaker2025-09-2510-49/+159
|\ | | | | Zs/put overwrite policy response
| * Improvement to Incomplete Result Iterationzousar2025-09-251-2/+1
| | | | | | | | From review feedback
| * Changelogzousar2025-09-241-1/+2
| |
| * Report Incomplete Records To Clientzousar2025-09-242-8/+44
| | | | | | | | When requesting partial records, report back when a record is incomplete via an "Incomplete" array of bools that is a sibling to the "Result" array for batch/rpc operations, or via the HttpResponseCode::PartialContent status code for individual record requests.
| * Adjust the responses from PUT commandszousar2025-09-234-15/+10
| | | | | | | | | | - Ensure that text responses are in a field named "Message" - Change the record response to be named "Record" instead of "Object"
| * Merge branch 'main' into zs/put-overwrite-policy-responsezousar2025-09-2228-843/+1690
| |\
| * | Change batch put responses for client reportingzousar2025-09-1910-41/+120
| | | | | | | | | | | | Conflicts are now treated as successes, and we optionally return a Details array instead of an ErrorMessages array. Details are returned for all requests in a batch, or no requests in a batch depending on whether there are any details to be shared about any of the put requests. The details for a conflict include the raw hash and raw size of the item. If the item is a record, we also include the record as an object.
* | | 5.7.2-pre2v5.7.2-pre2Dan Engelbrecht2025-09-231-1/+1
| | |
* | | limit number of chunks per block (#508)Dan Engelbrecht2025-09-238-61/+203
| |/ |/| | | | | | | - Improvement: Chunk block generation for `zen oplog-export` and `zen builds upload` command are now limited to max 4000 chunks per block zen builds list-block testing function to check results for recently used blocks in a context
* | 5.7.2-pre1v5.7.2-pre1Dan Engelbrecht2025-09-221-1/+1
| |
* | dont set Size or RawSize when fetching chunkinfos if the payload can't be ↵Dan Engelbrecht2025-09-221-2/+7
| | | | | | | | found/validated (#507)
* | more responsive cancel during oplog import (#505)Dan Engelbrecht2025-09-224-12/+33
| | | | | | | | - Improvement: Faster oplog import due to chunk existance check improvement - Improvement: Cancelling oplog import is now more responsive during initial phase
* | Added `--oidctoken-exe-unattended` to`zen builds` and `zen oplog-download` ↵Dan Engelbrecht2025-09-2210-12/+30
| | | | | | | | command to use unattended mode when launching oidc-token.exe (#506)
* | change default sentry dsn to one listed on Sentry setup page (#504)Dan Engelbrecht2025-09-221-1/+1
| |
* | issue error on second retry, not first attempt (#503)Dan Engelbrecht2025-09-221-2/+2
| |
* | fetch cloud oplog (#502)Dan Engelbrecht2025-09-2215-440/+940
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Feature: Added `zen oplog-download` command to download the oplog body of a cooked output stored in Cloud DDC - Oplog source is specified using one of the following options - `--cloud-url` Cloud artifact URL for oplog - `--host` Base host to resolve download host from - `--override-host` Specific host to use without resolve - `--assume-http2` assume that the builds endpoint is a HTTP/2 endpoint skipping HTTP/1.1 upgrade handshake - `--namespace` Builds Storage namespace - `--bucket` Builds Storage bucket - `--build-id` an Oid in hex form for the source identifier to use - `--yes` suppress conformation query when doing output of a very large oplog to console - `--quiet` suppress all non-essential console output - `--output-path` path to oplog output, extension .json or .cb (compact binary). Default is output to console - `--system-dir` override default system root path - Authentication options - Auth token - `--access-token` http auth Cloud Storage access token - `--access-token-env` name of environment variable that holds the Http auth Cloud Storage access token - `--access-token-path` path to json file that holds the Http auth Cloud Storage access token - `--oidctoken-exe-path` path to OidcToken executable - OpenId authentication - `--openid-provider-name` Open ID provider name - `--openid-provider-url` Open ID provider url - `--openid-client-id`Open ID client id - `--openid-refresh-token` Open ID refresh token - `--encryption-aes-key` 256 bit AES encryption key for storing OpenID credentials - `--encryption-aes-iv` 128 bit AES encryption initialization vector for storing OpenID credentials - OAuth authentication - `--oauth-url` OAuth provier url - `--oauth-clientid` OAuth client id - `--oauth-clientsecret` OAuth client secret - Bugfix: `zen print` command now properly outputs very large compact binary objects as json to console
* | improve builds download partial logic (#501)Dan Engelbrecht2025-09-224-376/+679
|/ | | | | | | | | | - Improvement: `zen command` help now uses available horizontal space for output - Improvement: Partial block request analisys has been improved with reduced download size at lower number of requests - Improvement: Validate that `--zen-cache-host` exists and is responsive before begin used - Feature: Options `--allow-partial-block-requests` for `zen builds download` command has been augmented with two new modes, `zencacheonly` and `mixed`. Defaults to `mixed`. - `false` only full block requests allowed - `mixed` multiple partial block ranges requests per block allowed to zen cache, single partial block range request per block to host - `zencacheonly` multiple partial block ranges requests per block allowed to zen cache, only full block requests allowed to host - `true` multiple partial block ranges requests per block allowed to zen cache and host
* fix quoted wildcard options (#500)Dan Engelbrecht2025-09-173-34/+25
| | | * improve command line parsing to automatically remove quotes
* add builds download force option (#498)Dan Engelbrecht2025-09-173-270/+298
| | | * add --force option to builds download command
* rpmalloc fixes (#499)Stefan Boberg2025-09-174-52/+69
| | | | | | | | * fixed rpmalloc build on Linux and Mac * updated rpmalloc from develop branch on the advice of mjansson * enabled rpmalloc on all platforms note that this does not change any behaviour unless `--malloc=rpmalloc` is passed in on the command line. The default is still `mimalloc`.
* Sorting oplog tree view by size would raise an error. (#497)Martin Ridgers2025-09-163-1/+2
| | | | | | | * Sorting oplog tree view by size would raise an error * Updated frontend .zip archive * Updated changelog
* 5.7.2-pre0v5.7.2-pre0Dan Engelbrecht2025-09-151-1/+1
|
* update readme with required vcpkg version (#496)Dan Engelbrecht2025-09-154-10/+20
| | | - Improvement: Updated README.md to state the required version vcpkg
* new in-memory storage strategy for oplogs (#490)Dan Engelbrecht2025-09-155-518/+749
| | | - Improvement: Revised project oplog in-memory representation which reduces load times and memory usage
* revise exception vs error (#495)Dan Engelbrecht2025-09-155-26/+39
| | | | | - Change BadAlloc exceptions in GC to warnings - Add explict ASSERT exception catch in http plugin request processing - Make exceptions handled in http request processing to warnings
* 5.7.1v5.7.1Dan Engelbrecht2025-09-121-1/+1
|
* 5.7.1-pre3v5.7.1-pre3Dan Engelbrecht2025-09-121-1/+1
|
* initialize all services before healthy (#493)Dan Engelbrecht2025-09-122-35/+59
| | | * make sure all services are initialized and registered before we respond OK to health check
* 5.7.1-pre2v5.7.1-pre2Dan Engelbrecht2025-09-101-1/+1
|
* add EMode to WorkerTheadPool to avoid thread starvation (#492)Dan Engelbrecht2025-09-1031-1421/+1603
| | | - Improvement: Add a new mode to worker thread pools to avoid starvation of workers which could cause long stalls due to other work begin queued up. UE-305498
* faster oplog entries with referenceset (#488)Dan Engelbrecht2025-09-088-57/+90
| | | | - Improvement: Faster project store `/entries` endpoint, 10-15% faster when using a reference set to limit entries fix missing space after task name in pretty progressbar
* 5.7.1-pre1v5.7.1-pre1Dan Engelbrecht2025-09-081-1/+1
|
* Set ownership of service executables to the service user on Linux (#489)Liam Mitchell2025-09-082-1/+46
| | | - Bugfix: Linux only, set ownership of installed files to specified user when using `zen service install --full --user`
* 5.7.1-pre0v5.7.1-pre0Dan Engelbrecht2025-09-051-1/+1
|
* refactor zen command return value handling (#487)Dan Engelbrecht2025-09-0548-1084/+1001
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Improvement: Use consistent language for command line argument parsing errors - Improvement: Changed zen command parsing errors to output help first and error last to make it easier to spot the error - Improvement: Refactor zen command return codes to conform to valid Linux range (0-255) kSuccess = 0, kOtherError = 1, kBadInput = 2, kOutOfMemory = 16, kOutOfDisk = 17, kAssertError = 70, kHttpOtherClientError = 80, kHttpCantConnectError = 81, kHttpNotFound = 66, // NotFound(404) kHttpUnauthorized = 77, // Unauthorized(401), kHttpSLLError = 82, kHttpForbidden = 83, // Forbidden(403) kHttpTimeout = 84, // RequestTimeout(408) kHttpConflict = 85, // Conflict(409) kHttpNoHost = 86, kHttpOtherServerError = 90, kHttpInternalServerError = 91, // InternalServerError(500) kHttpServiceUnavailable = 69, // ServiceUnavailable(503) kHttpBadGateway = 92, // BadGateway(502) kHttpGatewayTimeout = 93, // GatewayTimeout(504)
* Avoid mutating executable paths when copying files during full service ↵Liam Mitchell2025-09-041-5/+7
| | | | install (#486)
* asio large header support (#484)Dan Engelbrecht2025-09-043-123/+129
| | | - Improvement: Allow large headers support in asio server implementation
* add validation of compact binary payloads before reading them (#483)Dan Engelbrecht2025-09-0420-289/+536
| | | * add validation of compact binary payloads before reading them
* oplog memory usage reduction (#482)Dan Engelbrecht2025-09-045-100/+200
| | | | | | - Improvement: For projectstore oplog GET operation, only read basic information and release it if the oplog is not already open to reduce memory usage when listing oplogs in web UI - Improvement: Reduce memory usage for oplog Op address lookup Refactored Oplog::EState -> Oplog ::EMode and make sure we open data files in read-only mode when EMode::kBasicReadOnly is used.
* Merge pull request #485 from ue-foundation/zs/cfg-enable-overwrite-preventionZousar Shaker2025-09-031-0/+3
|\ | | | | Enable overwrite limits by default on all cache buckets
| * Enable overwrite limits by default on all cache bucketszousar2025-09-021-0/+3
|/
* projectstore lifetime improvements (#481)Dan Engelbrecht2025-08-295-368/+557
| | | | | - Improvement: Oplogs that have not been touched for 15 min are unloaded from memory during GC pass, oplogs are reloaded on demand - Improvement: Oplogs read for GC/Validation operations are loaded using a lightweight mode reducing memory usage and load times - Improvement: Cleaned up logging for oplogs/projects to reduce clutter