aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* error fix in ReadFileRangede/httpclient-send-refactorDan Engelbrecht2024-04-261-2/+3
|
* add generic read file streaming in http clientDan Engelbrecht2024-04-263-47/+141
|
* use direct file access for large file hash (#63)Dan Engelbrecht2024-04-2615-69/+157
| | | - Improvement: Refactor `IoHash::HashBuffer` and `BLAKE3::HashBuffer` to not use memory mapped files. Performs better and saves ~10% of oplog export time on CI
* fix oplog import during gcv2 (#62)v5.5.0-pre3v5.5.0-pre2Dan Engelbrecht2024-04-254-114/+140
| | | | - Bugfix: Always pre-cache oplog when creating project store GCv2 referencer - Bugfix: Correctly capture attachments imported with oplog to void them being GCd before oplog is written
* zenserverprocess hardening (#61)Dan Engelbrecht2024-04-254-25/+53
| | | | * verify running process before creating event * make sure we don't signal/wait for a zenserver instance that we did not wait for to get ready
* iterate cas chunks (#59)Dan Engelbrecht2024-04-2414-140/+299
| | | - Improvement: Reworked GetChunkInfos in oplog store to reduce disk thrashing and improve performance
* remove obsolete codeDan Engelbrecht2024-04-241-4/+0
|
* safer gcv2 on error (#60)Dan Engelbrecht2024-04-247-6/+42
| | | - Bugfix: Harden GCv2 when errors occur and gracefully abort GC operation on error
* bump vcpkg and xmake to latest (#40)Dan Engelbrecht2024-04-245-14/+18
| | | - Improvement: Bumped xmake to 2.9.1 and vcpkg version to 2024.03.25
* Bugfix: Only disable oplog update capture if we have started it (#58)Dan Engelbrecht2024-04-242-1/+7
|
* 5.5.0-pre0v5.5.0-pre0Dan Engelbrecht2024-04-221-1/+1
|
* set gc-v2 as default (#56)Dan Engelbrecht2024-04-222-1/+2
| | | - Change: GCv2 is now the default option, use `--gc-v2=false` to fall back to GCv1
* InsertChunks for CAS store (#55)Dan Engelbrecht2024-04-2215-177/+654
| | | - Improvement: Add batching when writing multiple small chunks to block store - decreases I/O load significantly on oplog import
* fix LogRemoteStoreStatsDetails (#53)Dan Engelbrecht2024-04-201-2/+3
|
* import oplog improvements (#54)Dan Engelbrecht2024-04-2011-205/+387
| | | | | | | | | | | * report down/up transfer speed during progress * add disk buffering in http client * offload block decoding and chunk writing form network worker pool threads add block hash verification for blocks recevied at oplog import * separate download-latch from write-latch to get more accurate download speed * check headers when downloading with http client to go directly to file writing for large payloads * we must clear write callback even if we only provide it as an argument to the Download() call * make timeout optional in AddSponsorProcess * check return codes when creating windows threadpool
* 5.4.5-pre0v5.4.5-pre0Dan Engelbrecht2024-04-181-1/+1
|
* capture zenserver output on error (#51)Dan Engelbrecht2024-04-187-30/+112
| | | | * capture spawned server output and output on launch error * fix logging and launch validation in tests
* safer oplog import (#52)de/safer-oplog-importDan Engelbrecht2024-04-186-76/+202
| | | | * reference cache gc update capture * When importing oplogs we now import all attachments first and (optionally clean) write the oplog on success
* improved lock file handling (#50)Dan Engelbrecht2024-04-1811-70/+324
| | | | | | | | | | | | - Feature: `zen down` - --`data-dir` to specify a data directory to deduce which zen instance to bring down - Feature: `zen attach` - --`data-dir` to specify a data directory to deduce which zen instance to attach to222 - Feature: `zen status` - --`port` filter running zen instances based on port - --`data-dir` filter running zen instances based on information in the data directory - Improvement: Trying to load a compact binary object from an empty file no longer causes access violation
* zen startup hardening (#49)Dan Engelbrecht2024-04-1712-83/+506
| | | | | | | | | | | | | - Feature: `zen up` command improvements - --`port` allows you to specify a base port when starting an instance - --`base-dir` allows you to specify a base directory for the zenserver executable if it is not located next to the zen.exe executable - Feature: `zen down` - --`port` allows you to specify a base port when shutting down an instance - --`base-dir` allows you to specify a base directory for the zenserver executable if it is not located next to the zen.exe executable - --`force` if regular shutdown fails it tries to find a running zenserver.exe process and terminate it - If it fails to attach to the running server it now waits for it to exit when setting the RequestExit shared memory flag - Improvement: zenserver now checks the RequestExit flag in the shared memory and exist gracefully if it is set - Improvement: When adding a sponsor process to a running zenserver instance, we wait for it to be picked up from the shared memory section to determine success/fail
* gc v2 disk freed space fix and oplog stats report improvement (#45)Dan Engelbrecht2024-04-153-87/+149
| | | | | - Bugfix: Correctly calculate size freed/data moved from blocks in GCv2 - Improvement: Reduced details in remote store stats for oplog export/import to user - Improvement: Transfer speed for oplog export/import is now an overall number rather than average of speed per single request
* Validate input buffer size when trying to parse package message (#47)Dan Engelbrecht2024-04-133-22/+41
| | | | * add validation of input buffer size when trying to parse package message * avoid doing memcopy when parsing package message
* typo fix in gc.lightweightintervalsecondsStefan Boberg2024-04-111-1/+1
|
* 5.4.4v5.4.4Dan Engelbrecht2024-04-101-1/+1
|
* 5.4.4-pre2v5.4.4-pre2Dan Engelbrecht2024-04-101-1/+1
|
* remote project store stats (#44)Dan Engelbrecht2024-04-1012-104/+339
| | | | | * add remote oplog store statistics * block chunking when uploading oplog to zenserver (mirroring) * make sure we can move temporary dechunked file into cas store
* changelogDan Engelbrecht2024-04-091-1/+0
|
* 5.4.4-pre1v5.4.4-pre1Dan Engelbrecht2024-04-091-1/+1
|
* Use proper API when checking oplog export blob existance in Jupiter (#42)Dan Engelbrecht2024-04-092-3/+5
|
* Merge pull request #41 from ue-foundation/zs/import-oplog-cleanZousar Shaker2024-04-053-5/+19
|\ | | | | Zs/import oplog clean
| * Update changelogzousar2024-04-051-0/+1
| |
| * precommitzousar2024-04-041-1/+6
| |
| * Add optional gcpath param to import-oplog cmdzousar2024-04-042-2/+10
| |
| * Categorize the clean argument on oplog-import as a general argzousar2024-04-042-3/+3
|/
* 5.4.4-pre0v5.4.4-pre0Dan Engelbrecht2024-04-041-1/+1
|
* fix changelogDan Engelbrecht2024-04-041-3/+13
|
* improved assert (#37)Dan Engelbrecht2024-04-0446-167/+594
| | | | - Improvement: Add file and line to ASSERT exceptions - Improvement: Catch call stack when throwing assert exceptions and log/output call stack at important places to provide more context to caller
* hardening parsepackagemessage (#38)Dan Engelbrecht2024-04-043-37/+75
| | | * hardening of ParsePackageMessage and extended details when malformed attachments are detected
* validate rpc chunk responses (#36)Dan Engelbrecht2024-04-035-20/+35
| | | * Validate size of found chunks in cas/cache
* zenremoteprojectstore with httpclient (#35)Dan Engelbrecht2024-04-038-211/+117
| | | | | | - Bugfix: Fix log of Success/Failure for oplog import - Improvement: Use HttpClient when doing oplog export/import with a zenserver as a remote target. Includes retry logic - Improvement: Increase the retry count to 4 (5 attempts in total) when talking to Jupiter for oplog export/import
* compressed header reading opt (#33)Dan Engelbrecht2024-04-032-53/+88
| | | * refactor so we don't have to re-read data from source to get block sizes
* 5.4.3v5.4.3Dan Engelbrecht2024-03-281-1/+1
|
* 5.4.3-pre0v5.4.3-pre0Dan Engelbrecht2024-03-281-1/+1
|
* Fix sentry using wrong folder path when data path contains non-anscii ↵Dan Engelbrecht2024-03-282-3/+4
| | | | | characters (#32) * Fix sentry using wrong folder path when data path contains non-ascii characters
* Use multithreading to fetch size/rawsize of entries in ↵Dan Engelbrecht2024-03-289-68/+200
| | | | | | `/prj/{project}/oplog/{log}/chunkinfos` and `/prj/{project}/oplog/{log}/files` (#30) - Improvement: Use multithreading to fetch size/rawsize of entries in `/prj/{project}/oplog/{log}/chunkinfos` and `/prj/{project}/oplog/{log}/files` - Improvement: Add `GetMediumWorkerPool()` in addition to `LargeWorkerPool()` and `SmallWorkerPool()`
* add "fieldnames" query param for GetProjectFiles/GetProjectChunkInfos (#29)Dan Engelbrecht2024-03-284-23/+143
| | | | | | | | - Improvement: It is now possible to control which fields to include in `/prj/{project}/oplog/{log}/chunkinfos` request by adding a comma delimited list of filed names for `fieldnames` parameter - Default fields are: `id`, `rawhash` and `rawsize` (translates to `?fieldnames=id,rawhash,rawsize`) - Use `?fieldnames=*` to get all the fields - Improvement: It is now possible to control which fields to include in `/prj/{project}/oplog/{log}/files` request by adding a comma delimited list of filed names for `fieldnames` parameter - Default fields are: `id`, `clientpath` and `serverpath` (translates to `?fieldnames=id,clientpath,serverpath`), `filter=client` only applies if `fieldnames` is not given as a parameter - Use `?fieldnames=*` to get all the fields
* Faster reading of compressed buffer headers by not materializing entire ↵Dan Engelbrecht2024-03-282-22/+53
| | | | source buffer (#28)
* Get raw size for compressed chunks correctly for ↵Dan Engelbrecht2024-03-272-1/+10
| | | | `/prj/{project}/oplog/{log}/chunkinfos` (#27)
* 5.4.2v5.4.2Dan Engelbrecht2024-03-261-1/+1
|
* make existing block log clearerDan Engelbrecht2024-03-261-1/+1
|
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-12 11:45:55 +0000