aboutsummaryrefslogtreecommitdiff
path: root/src/zencore/compress.cpp
Commit message (Collapse)AuthorAgeFilesLines
* fix zero length stream decompress (#685)Dan Engelbrecht2025-12-101-1/+1
| | | * fix validation during stream decompress of zero-size rawsize
* use scope guards to secure that BasicFile::Detach is called even on ↵Dan Engelbrecht2025-12-011-10/+5
| | | | exception (#672)
* automatic scrub on startup (#667)Dan Engelbrecht2025-11-271-15/+46
| | | | | - Improvement: Deeper validation of data when scrub is activated (cas/cache/project) - Improvement: Enabled more multi threading when running scrub operations - Improvement: Added means to force a scrub operation at startup with a new release using ZEN_DATA_FORCE_SCRUB_VERSION variable in xmake.lua
* fix stream decompress (#668)Dan Engelbrecht2025-11-271-10/+14
| | | * fix source file offset in DecompressToStream implementations
* RawOffset can be anything and we expect an empty buffer to be returned along ↵Dan Engelbrecht2025-11-261-4/+19
| | | | with RawSize = 0 if the offset was out of bounds for the value. (#666)
* fixed size chunking for encrypted files (#410)Dan Engelbrecht2025-06-031-14/+36
| | | | | - Improvement: Use fixed size block chunking for know encrypted/compressed file types - Improvement: Skip trying to compress chunks that are sourced from files that are known to be encrypted/compressed - Improvement: Add global open file cache for written files increasing throughput during download by reducing overhead of open/close of file by 80%
* streaming none compressor (#414)Dan Engelbrecht2025-06-021-3/+36
| | | * add proper streaming to none compressor type
* improved post upload/download summary (#308)Dan Engelbrecht2025-03-181-50/+80
| | | | | | | | | * added ValidateStatistics and improved post upload summary * improved download statistics * smoother stats update when compressing * better feedback during stream compresss/decompress * don't capture TotalPartWriteCount by reference * disk stats cleanup * multi-test-download overall timer
* pick up existing cache (#299)Dan Engelbrecht2025-03-101-16/+35
| | | | | | - Improvement: Scavenge .zen temp folders for existing data (downloaded, decompressed or written) from previous failed run - Improvement: Faster abort during stream compression - Improvement: Try to move downloaded blobs with rename if possible avoiding an extra disk write - Improvement: Only clean temp folders on successful or cancelled build - keep it if download fails
* streaming compress (#295)Dan Engelbrecht2025-03-051-2/+169
| | | | - Improvement: Validate hash of decompressed data inline with streaming decompression - Improvement: Do streaming compression of large blobs to improve memory and I/O performance
* stream decompress (#293)Dan Engelbrecht2025-03-041-0/+186
| | | | * clean up latency parameters and slow down rate updates * add DecompressToStream
* improvements and infrastructure for upcoming builds api command line (#284)Dan Engelbrecht2025-02-251-53/+161
| | | | | | | | | | | | | | * add modification tick to filesystem traversal * add ShowDetails option to ProgressBar * log callstack if we terminate process * handle chunking if MaxSize > 1MB * BasicFile write helpers and WriteToTempFile simplifications * bugfix for CompositeBuffer::IterateRange when using DecompressToComposite for actually comrpessed data revert of earlier optimization * faster compress/decompress for large disk-based files * enable progress feedback in IoHash::HashBuffer * add payload validation in HttpClient::Get * fix range requests (range is including end byte) * remove BuildPartId for blob/block related operations in builds api
* safer partial requests (#82)Dan Engelbrecht2024-05-171-34/+46
| | | | * fix partial cache miss logging * Handle zero size partial requests safer
* compressed header reading opt (#33)Dan Engelbrecht2024-04-031-52/+86
| | | * refactor so we don't have to re-read data from source to get block sizes
* non memory copy compressed range (#13)Dan Engelbrecht2024-03-201-0/+192
| | | | | * Add CompressedBuffer::GetRange that references source data rather than make a memory copy * Use Compressed.CopyRange in project store GetChunkRange * docs for CompressedBuffer::CopyRange and CompressedBuffer::GetRange
* special treatment large oplog attachments v2 (#5)Dan Engelbrecht2024-03-141-9/+21
| | | | | - Bugfix: Install Ctrl+C handler earlier when doing `zen oplog-export` and `zen oplog-export` to properly cancel jobs - Improvement: Add ability to block a set of CAS entries from GC in project store - Improvement: Large attachments and loose files are now split into smaller chunks and stored in blocks during oplog export
* header only compressed buffers are valid (#604)Dan Engelbrecht2023-12-121-2/+2
| | | - Bugfix: Allow attachments that contains a raw size of zero
* streaming decompression support (#142)Stefan Boberg2023-05-231-20/+786
| | | Added CompressedBufferReader support from UE. This provides some streaming decompression support which can be employed to reduce memory and other resource usage.
* cleaned up how Oodle is incorporatedStefan Boberg2023-05-021-4/+1
| | | | changed win64 to be more in line with other platforms wrt xmake config
* moved source directories into `/src` (#264)Stefan Boberg2023-05-021-0/+1353
* moved source directories into `/src` * updated bundle.lua for new `src` path * moved some docs, icon * removed old test trees
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-11 12:56:08 +0000