aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* fix oplog import during gcv2 (#62)v5.5.0-pre3v5.5.0-pre2Dan Engelbrecht2024-04-254-114/+140
| | | | - Bugfix: Always pre-cache oplog when creating project store GCv2 referencer - Bugfix: Correctly capture attachments imported with oplog to void them being GCd before oplog is written
* zenserverprocess hardening (#61)Dan Engelbrecht2024-04-254-25/+53
| | | | * verify running process before creating event * make sure we don't signal/wait for a zenserver instance that we did not wait for to get ready
* iterate cas chunks (#59)Dan Engelbrecht2024-04-2414-140/+299
| | | - Improvement: Reworked GetChunkInfos in oplog store to reduce disk thrashing and improve performance
* remove obsolete codeDan Engelbrecht2024-04-241-4/+0
|
* safer gcv2 on error (#60)Dan Engelbrecht2024-04-247-6/+42
| | | - Bugfix: Harden GCv2 when errors occur and gracefully abort GC operation on error
* bump vcpkg and xmake to latest (#40)Dan Engelbrecht2024-04-245-14/+18
| | | - Improvement: Bumped xmake to 2.9.1 and vcpkg version to 2024.03.25
* Bugfix: Only disable oplog update capture if we have started it (#58)Dan Engelbrecht2024-04-242-1/+7
|
* 5.5.0-pre0v5.5.0-pre0Dan Engelbrecht2024-04-221-1/+1
|
* set gc-v2 as default (#56)Dan Engelbrecht2024-04-222-1/+2
| | | - Change: GCv2 is now the default option, use `--gc-v2=false` to fall back to GCv1
* InsertChunks for CAS store (#55)Dan Engelbrecht2024-04-2215-177/+654
| | | - Improvement: Add batching when writing multiple small chunks to block store - decreases I/O load significantly on oplog import
* fix LogRemoteStoreStatsDetails (#53)Dan Engelbrecht2024-04-201-2/+3
|
* import oplog improvements (#54)Dan Engelbrecht2024-04-2011-205/+387
| | | | | | | | | | | * report down/up transfer speed during progress * add disk buffering in http client * offload block decoding and chunk writing form network worker pool threads add block hash verification for blocks recevied at oplog import * separate download-latch from write-latch to get more accurate download speed * check headers when downloading with http client to go directly to file writing for large payloads * we must clear write callback even if we only provide it as an argument to the Download() call * make timeout optional in AddSponsorProcess * check return codes when creating windows threadpool
* 5.4.5-pre0v5.4.5-pre0Dan Engelbrecht2024-04-181-1/+1
|
* capture zenserver output on error (#51)Dan Engelbrecht2024-04-187-30/+112
| | | | * capture spawned server output and output on launch error * fix logging and launch validation in tests
* safer oplog import (#52)de/safer-oplog-importDan Engelbrecht2024-04-186-76/+202
| | | | * reference cache gc update capture * When importing oplogs we now import all attachments first and (optionally clean) write the oplog on success
* improved lock file handling (#50)Dan Engelbrecht2024-04-1811-70/+324
| | | | | | | | | | | | - Feature: `zen down` - --`data-dir` to specify a data directory to deduce which zen instance to bring down - Feature: `zen attach` - --`data-dir` to specify a data directory to deduce which zen instance to attach to222 - Feature: `zen status` - --`port` filter running zen instances based on port - --`data-dir` filter running zen instances based on information in the data directory - Improvement: Trying to load a compact binary object from an empty file no longer causes access violation
* zen startup hardening (#49)Dan Engelbrecht2024-04-1712-83/+506
| | | | | | | | | | | | | - Feature: `zen up` command improvements - --`port` allows you to specify a base port when starting an instance - --`base-dir` allows you to specify a base directory for the zenserver executable if it is not located next to the zen.exe executable - Feature: `zen down` - --`port` allows you to specify a base port when shutting down an instance - --`base-dir` allows you to specify a base directory for the zenserver executable if it is not located next to the zen.exe executable - --`force` if regular shutdown fails it tries to find a running zenserver.exe process and terminate it - If it fails to attach to the running server it now waits for it to exit when setting the RequestExit shared memory flag - Improvement: zenserver now checks the RequestExit flag in the shared memory and exist gracefully if it is set - Improvement: When adding a sponsor process to a running zenserver instance, we wait for it to be picked up from the shared memory section to determine success/fail
* gc v2 disk freed space fix and oplog stats report improvement (#45)Dan Engelbrecht2024-04-153-87/+149
| | | | | - Bugfix: Correctly calculate size freed/data moved from blocks in GCv2 - Improvement: Reduced details in remote store stats for oplog export/import to user - Improvement: Transfer speed for oplog export/import is now an overall number rather than average of speed per single request
* Validate input buffer size when trying to parse package message (#47)Dan Engelbrecht2024-04-133-22/+41
| | | | * add validation of input buffer size when trying to parse package message * avoid doing memcopy when parsing package message
* typo fix in gc.lightweightintervalsecondsStefan Boberg2024-04-111-1/+1
|
* 5.4.4v5.4.4Dan Engelbrecht2024-04-101-1/+1
|
* 5.4.4-pre2v5.4.4-pre2Dan Engelbrecht2024-04-101-1/+1
|
* remote project store stats (#44)Dan Engelbrecht2024-04-1012-104/+339
| | | | | * add remote oplog store statistics * block chunking when uploading oplog to zenserver (mirroring) * make sure we can move temporary dechunked file into cas store
* changelogDan Engelbrecht2024-04-091-1/+0
|
* 5.4.4-pre1v5.4.4-pre1Dan Engelbrecht2024-04-091-1/+1
|
* Use proper API when checking oplog export blob existance in Jupiter (#42)Dan Engelbrecht2024-04-092-3/+5
|
* Merge pull request #41 from ue-foundation/zs/import-oplog-cleanZousar Shaker2024-04-053-5/+19
|\ | | | | Zs/import oplog clean
| * Update changelogzousar2024-04-051-0/+1
| |
| * precommitzousar2024-04-041-1/+6
| |
| * Add optional gcpath param to import-oplog cmdzousar2024-04-042-2/+10
| |
| * Categorize the clean argument on oplog-import as a general argzousar2024-04-042-3/+3
|/
* 5.4.4-pre0v5.4.4-pre0Dan Engelbrecht2024-04-041-1/+1
|
* fix changelogDan Engelbrecht2024-04-041-3/+13
|
* improved assert (#37)Dan Engelbrecht2024-04-0446-167/+594
| | | | - Improvement: Add file and line to ASSERT exceptions - Improvement: Catch call stack when throwing assert exceptions and log/output call stack at important places to provide more context to caller
* hardening parsepackagemessage (#38)Dan Engelbrecht2024-04-043-37/+75
| | | * hardening of ParsePackageMessage and extended details when malformed attachments are detected
* validate rpc chunk responses (#36)Dan Engelbrecht2024-04-035-20/+35
| | | * Validate size of found chunks in cas/cache
* zenremoteprojectstore with httpclient (#35)Dan Engelbrecht2024-04-038-211/+117
| | | | | | - Bugfix: Fix log of Success/Failure for oplog import - Improvement: Use HttpClient when doing oplog export/import with a zenserver as a remote target. Includes retry logic - Improvement: Increase the retry count to 4 (5 attempts in total) when talking to Jupiter for oplog export/import
* compressed header reading opt (#33)Dan Engelbrecht2024-04-032-53/+88
| | | * refactor so we don't have to re-read data from source to get block sizes
* 5.4.3v5.4.3Dan Engelbrecht2024-03-281-1/+1
|
* 5.4.3-pre0v5.4.3-pre0Dan Engelbrecht2024-03-281-1/+1
|
* Fix sentry using wrong folder path when data path contains non-anscii ↵Dan Engelbrecht2024-03-282-3/+4
| | | | | characters (#32) * Fix sentry using wrong folder path when data path contains non-ascii characters
* Use multithreading to fetch size/rawsize of entries in ↵Dan Engelbrecht2024-03-289-68/+200
| | | | | | `/prj/{project}/oplog/{log}/chunkinfos` and `/prj/{project}/oplog/{log}/files` (#30) - Improvement: Use multithreading to fetch size/rawsize of entries in `/prj/{project}/oplog/{log}/chunkinfos` and `/prj/{project}/oplog/{log}/files` - Improvement: Add `GetMediumWorkerPool()` in addition to `LargeWorkerPool()` and `SmallWorkerPool()`
* add "fieldnames" query param for GetProjectFiles/GetProjectChunkInfos (#29)Dan Engelbrecht2024-03-284-23/+143
| | | | | | | | - Improvement: It is now possible to control which fields to include in `/prj/{project}/oplog/{log}/chunkinfos` request by adding a comma delimited list of filed names for `fieldnames` parameter - Default fields are: `id`, `rawhash` and `rawsize` (translates to `?fieldnames=id,rawhash,rawsize`) - Use `?fieldnames=*` to get all the fields - Improvement: It is now possible to control which fields to include in `/prj/{project}/oplog/{log}/files` request by adding a comma delimited list of filed names for `fieldnames` parameter - Default fields are: `id`, `clientpath` and `serverpath` (translates to `?fieldnames=id,clientpath,serverpath`), `filter=client` only applies if `fieldnames` is not given as a parameter - Use `?fieldnames=*` to get all the fields
* Faster reading of compressed buffer headers by not materializing entire ↵Dan Engelbrecht2024-03-282-22/+53
| | | | source buffer (#28)
* Get raw size for compressed chunks correctly for ↵Dan Engelbrecht2024-03-272-1/+10
| | | | `/prj/{project}/oplog/{log}/chunkinfos` (#27)
* 5.4.2v5.4.2Dan Engelbrecht2024-03-261-1/+1
|
* make existing block log clearerDan Engelbrecht2024-03-261-1/+1
|
* 5.4.2-pre17v5.4.2-pre17Dan Engelbrecht2024-03-261-1/+1
|
* fix order of parameters in block check result message (#26)Dan Engelbrecht2024-03-261-1/+1
| | | * fix order of parametes in block check result message
* add yaml serialization support (#3)Stefan Boberg2024-03-2611-783/+1250
| | | | | | | this change adds serialization of payloads as YAML, but not parsing. The implementation is somewhat based on the JSON path, and may be collapsed eventually as it is possible to serialize JSON format using the same code it also separates out the JSON serialization into a separate file for ease of maintenance any HTTP request response may be formatted as yaml by using a `.yaml` suffix or an `Accept: text/yaml` header
generated by cgit v1.2.3 (git 2.25.1) at 2026-04-24 20:06:09 +0000