aboutsummaryrefslogtreecommitdiff
path: root/src/zencore/iobuffer.cpp
Commit message (Collapse)AuthorAgeFilesLines
* special treatment large oplog attachments v2 (#5)Dan Engelbrecht2024-03-141-3/+4
| | | | | - Bugfix: Install Ctrl+C handler earlier when doing `zen oplog-export` and `zen oplog-export` to properly cancel jobs - Improvement: Add ability to block a set of CAS entries from GC in project store - Improvement: Large attachments and loose files are now split into smaller chunks and stored in blocks during oplog export
* hashing fixes (#657)Dan Engelbrecht2024-02-261-2/+19
| | | | | * move structuredcachestore tests to zenstore-test * Don't materialize entire files when hashing if it is a large files * rewrite CompositeBuffer::Mid to never materialize buffers
* exception on basicfile read error (#642)Dan Engelbrecht2024-01-291-2/+2
| | | | | * Change BasicFile::Read to throw exception like IoBuffer - Don't ASSERT on dwNumberOfBytesRead == NumberOfBytesToRead - throw exception with details instead - Use proper return type for pread()
* fix leak in IoBuffer for manifested small chunk (#618)Stefan Boberg2023-12-191-2/+2
| | | | * fix leak in IoBuffer for manifested small chunk. previously it would null out the `m_DataPtr` member on every path from `IoBufferExtendedCore::~IoBufferExtendedCore()` but it only really makes sense to null it out when the buffer has been memory mapped
* keep a "null" iobuffer core to reduce redundant memory allocations (#507)Dan Engelbrecht2023-11-061-0/+2
|
* added missing includes (#504)Stefan Boberg2023-10-271-0/+2
| | | | | this change adds some includes to files which "inherit" includes from elsewhere this was exposed on another branch when removing some heavy dependencies from central headers
* ZenCacheMemoryLayer should always store values using memory buffers (#451)Stefan Boberg2023-10-051-2/+5
| | | | | | | | this change fixes a problem where the memory cache layer can inadvertently prevent underlying block store files from being deleted * ensure we get memory buffers on all paths * added more context to error in IoBufferBuilder::ReadFromFileMaybe * fixed problematic pread call success check in IoBufferBuilder::ReadFromFileMaybe which would always report failure on Linux/MacOS
* fixed issue where IoBufferBuilder::ReadFromFileMaybe loses content type (#450)Stefan Boberg2023-10-051-1/+3
|
* Handle OOM and OOD more gracefully to not spam Sentry with error reports (#434)Dan Engelbrecht2023-10-021-13/+13
| | | | | | - Improvement: Catch Out Of Memory and Out Of Disk exceptions and report back to reqeuster without reporting an error to Sentry - Improvement: If creating bucket fails when storing and item in the structured cache, log a warning and propagate error to requester without reporting an error to Sentry - Improvement: Make an explicit flush of the active block written to in blockstore flush - Improvement: Make sure cache and cas MakeIndexSnapshot does not throw exception on failure which would cause and abnormal termniation at exit
* add more trace scopes (#362)Dan Engelbrecht2023-09-151-0/+14
| | | | | * more trace scopes * Make sure ReplayLogEntries uses the correct size for oplog buffer * changelog
* Make sure error logging or destructors don't throw exception when trying to ↵Dan Engelbrecht2023-09-121-7/+10
| | | | | get file name from handle (#393) - Bugfix: Make sure error logging or destructors don't throw exception when trying to get file name from handle
* stream oplog attachments from jupiter (#384)Dan Engelbrecht2023-09-061-13/+16
| | | | | | | | | | * stream large downloads from jupiter to temporary file * rework DeleteOnClose - top level marks file for delete and if lower level parts wants to keep it it clears that flag * changelog * log number of attachments to download * add delay on jupiter request failure when retrying * make sure we upload all attachments even if Needs are empty when ForceUpload is true release TempAttachment as soon as it is used * sort attachments so we get predictable blocks for the same oplog
* use better hash function for better distribution in IoBuffer g_MappingLocks ↵Dan Engelbrecht2023-08-211-1/+10
| | | | | | (#370) * use better hash function for better distribution in IoBuffer g_MappingLocks * changelog
* use exception when allocations fail rather than asserts (#319)Dan Engelbrecht2023-05-231-4/+10
| | | | * use exception when allocations fail rather than asserts * changelog
* brought over minor IoBuffer changes from sb/scrubStefan Boberg2023-05-151-0/+1
| | | | no actual code changes, just comments
* added IoBuffer.mmap test caseStefan Boberg2023-05-151-0/+27
| | | | | | this verifies that attempting to map a range outside the underlying file fails. Posix appears to offer different semantics so it silently accepts it still however.
* Remove ATL header usage (#306)Stefan Boberg2023-05-151-3/+3
| | | ATL has been used here and there as a convenience. Given that this is a legacy component and not always something which gets installed along with the compiler we hereby remove the dependency altogether in favour of our own simple wrappers
* better logging/exception when reading file in IoBuffer::Materialize fails (#294)Dan Engelbrecht2023-05-121-8/+37
| | | | * better logging/exception when reading file in IoBuffer::Materialize fails * changelog
* add context to MapViewOfFile errors (#282)Stefan Boberg2023-05-091-3/+6
| | | | * added FileSizeFromHandle function * added file size to error message when MapViewOfFile fails
* moved source directories into `/src` (#264)Stefan Boberg2023-05-021-0/+653
* moved source directories into `/src` * updated bundle.lua for new `src` path * moved some docs, icon * removed old test trees