aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* prime temp file sizes and order request based on first write positionde/disk-io-optimizationsDan Engelbrecht2025-04-103-8/+143
|
* only validate hash if we have a complete payload in http clientDan Engelbrecht2025-04-101-5/+5
|
* fall back to FindFirstFile if access prevents us from using GetFileAttributesDan Engelbrecht2025-04-101-34/+67
|
* add more forgiving retries with filesystemDan Engelbrecht2025-04-101-20/+31
|
* multpart download crash (#353)Dan Engelbrecht2025-04-101-1/+2
| | | * fix lambda capture during multipart-download
* scavenge builds (#352)Dan Engelbrecht2025-04-083-187/+669
| | | | - Improvement: `zen builds` now scavenges previous download locations for data to reduce download size, enabled by default, disable with `--enable-scavenge=false` - Bugfix: Failing to rename a file during download sometimes reported an error when it succeeded when retrying
* tweaked verbose output (#351)Dan Engelbrecht2025-04-071-23/+18
|
* fixed host resolving if both host and and override-host (url) was given (#350)Dan Engelbrecht2025-04-071-17/+40
|
* save global download info file for scavenging (#349)Dan Engelbrecht2025-04-071-105/+189
| | | | * save global download info file for scavenging * don't let test code write to official state folder
* improved layout of end of run stats output (#348)Dan Engelbrecht2025-04-071-10/+15
|
* "unlimited" line length when using plain progress (#347)Dan Engelbrecht2025-04-041-7/+19
|
* xmake updatefrontendzousar2025-04-041-0/+0
|
* Merge pull request #343 from ue-foundation/zs/web-ui-oplog-searchZousar Shaker2025-04-042-3/+7
|\ | | | | Oplog search improvements
| * Merge branch 'main' into zs/web-ui-oplog-searchZousar Shaker2025-04-047-35/+384
| |\
| * | Oplog search improvementszousar2025-04-032-3/+7
| | | | | | | | | | | | | | | | | | - Case insensitive search - Allow search of 1 or 2 character strings - Reset table when doing a null search
* | | Merge branch 'main' into zs/web-ui-blank-import-name-fixZousar Shaker2025-04-047-35/+384
|\ \ \ | | |/ | |/|
| * | progress bar improvements (#346)Dan Engelbrecht2025-04-041-18/+82
| | | | | | | | | | | | * hide ETA until at least 5% is complete * dynamically adjust progres output length
| * | blobstore size limit (#342)Dan Engelbrecht2025-04-046-17/+302
| |/ | | | | - Feature: zenserver option `--buildstore-disksizelimit` to set an soft upper limit for build storage data. Defaults to 1TB.
* | Alternate fix by explicitly initializing pkg_idzousar2025-04-041-2/+1
| |
* | Bump db versionzousar2025-04-031-1/+1
| | | | | | | | Required to refresh db contents after ID fix.
* | Don't duplicate ID bytes when more than one pkg_datazousar2025-04-031-0/+1
|/ | | | | | ID was getting extended and left shifted if we encountered multiple package data items in a single entry. So instead of the ID being 0x0c6500b7fb8dbe2e, it was 0x0C6500B7FB8DBE2E0C6500B7FB8DBE2E. When we went to look up an imported package by ID, it would not be found and the import would be presented as a blank string. Addressing this by making the first package data the only referenceable one. Second package datas are currently used for optional data blobs, and will not be imported or referenced. They are sidecar data.
* build store save access times (#341)v5.6.3-pre0Dan Engelbrecht2025-04-034-35/+309
| | | | | * save payload size in log for buildstore * read/write access times and manifest for buldstore * use retry when removing temporary files
* `zen oplog-export`, `zen oplog-import` for `--url` (cloud) and `--builds` ↵Dan Engelbrecht2025-04-037-140/+159
| | | | (builds) option now has `--oidctoken-exe-path` to let zen run the OidcToken executable to get and refresh authentication token (#340)
* use oidctoken executable to generate auth (#336)Dan Engelbrecht2025-04-027-18/+161
| | | - Feature: `zen builds` auth option `--oidctoken-exe-path` to let zen run the OidcToken executable to get and refresh authentication token
* Merge branch 'main' of https://github.ol.epicgames.net/ue-foundation/zenzousar2025-04-022-11/+18
|\
| * upload fail mac linux (#338)Dan Engelbrecht2025-04-021-11/+17
| | | | | | * fix macos/linux path resolving
| * Merge branch 'main' into fd-fix-binary-jsonFlorent Devillechabrol2025-04-0281-1991/+7345
| |\
| * | Fixed missing trailing quote when converting binary data from compact binary ↵Florent Devillechabrol2025-03-241-0/+1
| | | | | | | | | | | | to json
* | | Merge branch 'main' of https://github.ol.epicgames.net/ue-foundation/zenzousar2025-04-028-13/+33
|\ \ \ | | |/ | |/|
| * | added --find-max-block-count option to builds upload (#337)Dan Engelbrecht2025-04-028-13/+33
| | |
* | | Updating frontend archivezousar2025-04-011-0/+0
|/ /
* | builds url discovery (#334)Dan Engelbrecht2025-04-0115-77/+345
| | | | | | | | | | - Feature: Added `--host` option to use Jupiters list of cloud host and zen servers to resolve best hosts - Feature: Use local zenserver as builds cache if it has the `builds` service enabled and `--cloud-discovery-host` is provided and no remote zenserver cache hosts can be found - Improvement: Added `--override-host` option as a replacement for `--url` (`--url` still works, but `--override-host` is preferred)
* | reduce disk io during gc (#335)Dan Engelbrecht2025-04-011-20/+8
| | | | | | * do cache bucket flush/write snapshot as part of compact to reduce disk I/O
* | verify that we can read input files that are only hashed (#333)Dan Engelbrecht2025-04-012-7/+122
| | | | | | | | | | * output build and part details by default * output executable and version at start of builds command * verify that we can read files we do not chunk
* | logging improvement (#332)Dan Engelbrecht2025-03-311-17/+20
| | | | | | | | * Cache -> Download cache * cleaned up info regarding local cache/state and remote cache
* | multithreaded clean (#331)Dan Engelbrecht2025-03-313-92/+393
| | | | | | | | - Improvement: Faster cleaning of directories - Improvement: Faster initial scanning of local state
* | long filename support (#330)Dan Engelbrecht2025-03-3141-519/+1270
| | | | | | - Bugfix: Long file paths now works correctly on Windows
* | check file from local track state during download (#329)Dan Engelbrecht2025-03-301-118/+147
| |
* | temp path options and reduced scanning of target folder (#328)Dan Engelbrecht2025-03-283-252/+580
| | | | | | | | | | | | - Feature: zen: `--zen-folder-path` added to `builds` command, `list`, `upload`, `download`, `fetch-blob`, `validate-part` to control where `.zen` folder is placed and named - Improvement: Only check known files from remote state when downloading to a target folder with no local state file - Improvement: Don't move existing local to cache and back if they are untouched
* | build cache prime (#327)Dan Engelbrecht2025-03-276-454/+722
| | | | | | | | - Feature: zen `--boost-workers` option to builds `upload`, `download` and `validate-part` that will increase the number of worker threads, may cause computer to be less responsive - Feature: zen `--cache-prime-only` that uploads referenced data from a part to `--zen-cache-host` if it is not already present. Target folder will be untouched.
* | Merge branch 'main' into zs/ui-show-cook-artifactsZousar Shaker2025-03-277-66/+216
|\ \
| * | optional compress of block chunks (#326)Dan Engelbrecht2025-03-277-66/+216
| | | | | | | | | | | | | | | | | | - Feature: zenserver: Add command line option `--gc-buildstore-duration-seconds` to control GC life time of build store data - Improvement: ELF and MachO executable files are no longer chunked - Improvement: Compress chunks in blocks that encloses a full file (such as small executables) - Bugfix: Strip path delimiter at end of string in StringToPath
* | | Merge branch 'main' into zs/ui-show-cook-artifactsZousar Shaker2025-03-2633-947/+3984
|\| |
| * | zen build cache service (#318)Dan Engelbrecht2025-03-2633-947/+3984
| |/ | | | | | | | | | | | | | | | | - **EXPERIMENTAL** `zen builds` - Feature: `--zen-cache-host` option for `upload` and `download` operations to use a zenserver host `/builds` endpoint for storing build blob and blob metadata - Feature: New `/builds` endpoint for caching build blobs and blob metadata - `/builds/{namespace}/{bucket}/{buildid}/blobs/{hash}` `GET` and `PUT` method for storing and fetching blobs - `/builds/{namespace}/{bucket}/{buildid}/blobs/putBlobMetadata` `POST` method for storing metadata about blobs - `/builds/{namespace}/{bucket}/{buildid}/blobs/getBlobMetadata` `POST` method for fetching metadata about blobs - `/builds/{namespace}/{bucket}/{buildid}/blobs/exists` `POST` method for checking existance of blobs
* | Descriptive type conversion messageszousar2025-03-261-15/+39
| | | | | | | | Handling decompression or validation errors with more descriptive messages
* | Removed do_nothing from entry.jszousar2025-03-251-2/+0
| |
* | Moved AcceptType Handling From ProjectStore To HttpProjectStorezousar2025-03-253-61/+35
| |
* | Add CookPackageArtifacts attachment to web uizousar2025-03-212-6/+106
|/
* don't let auth env argument block other auth options (#316)Dan Engelbrecht2025-03-191-8/+12
| | | * prioritize actual options over implicit env variable for auth token
* jupiter builds stats upload (#315)Dan Engelbrecht2025-03-196-15/+97
| | | | | | | | | | | | | - Improvement: At end of build upload we post statistics to the Jupiter build stats endpoint: - `totalSize` - `reusedRatio` - `reusedBlockCount` - `reusedBlockByteCount` - `newBlockCount` - `newBlockByteCount` - `uploadedCount` - `uploadedByteCount` - `elapsedTimeSec` - `uploadedBytesPerSec`