diff options
| author | Dan Engelbrecht <[email protected]> | 2024-11-25 14:49:04 +0100 |
|---|---|---|
| committer | GitHub Enterprise <[email protected]> | 2024-11-25 14:49:04 +0100 |
| commit | bcb81b326a373aa86d7e6a046febc8ba74f21c04 (patch) | |
| tree | b20c6d59cefd299b4daac0754c8fab7ec7019b9c /src/zenstore/cas.cpp | |
| parent | stronger validation of payload existance (#229) (diff) | |
| download | zen-bcb81b326a373aa86d7e6a046febc8ba74f21c04.tar.xz zen-bcb81b326a373aa86d7e6a046febc8ba74f21c04.zip | |
caller controls threshold for bulk-loading chunks in IterateChunks (#222)
* Allow caller to control threshold for bulk-loading chunks in IterateChunks
* use smaller batch chunk reading for /fileinfos and /chunkinfos as we do not intend to read the payload
* use smaller batch read buffer when just querying for size of attachments
Diffstat (limited to 'src/zenstore/cas.cpp')
| -rw-r--r-- | src/zenstore/cas.cpp | 12 |
1 files changed, 8 insertions, 4 deletions
diff --git a/src/zenstore/cas.cpp b/src/zenstore/cas.cpp index 60dc9a505..4c1836cf7 100644 --- a/src/zenstore/cas.cpp +++ b/src/zenstore/cas.cpp @@ -59,7 +59,8 @@ public: virtual void FilterChunks(HashKeySet& InOutChunks) override; virtual bool IterateChunks(std::span<IoHash> DecompressedIds, const std::function<bool(size_t Index, const IoBuffer& Payload)>& AsyncCallback, - WorkerThreadPool* OptionalWorkerPool) override; + WorkerThreadPool* OptionalWorkerPool, + uint64_t LargeSizeLimit) override; virtual void Flush() override; virtual void ScrubStorage(ScrubContext& Ctx) override; virtual CidStoreSize TotalSize() const override; @@ -392,7 +393,8 @@ CasImpl::FilterChunks(HashKeySet& InOutChunks) bool CasImpl::IterateChunks(std::span<IoHash> DecompressedIds, const std::function<bool(size_t Index, const IoBuffer& Payload)>& AsyncCallback, - WorkerThreadPool* OptionalWorkerPool) + WorkerThreadPool* OptionalWorkerPool, + uint64_t LargeSizeLimit) { ZEN_TRACE_CPU("CAS::IterateChunks"); if (!m_SmallStrategy.IterateChunks( @@ -402,7 +404,8 @@ CasImpl::IterateChunks(std::span<IoHash> DecompressedIds, Chunk.SetContentType(ZenContentType::kCompressedBinary); return AsyncCallback(Index, Payload); }, - OptionalWorkerPool)) + OptionalWorkerPool, + LargeSizeLimit)) { return false; } @@ -413,7 +416,8 @@ CasImpl::IterateChunks(std::span<IoHash> DecompressedIds, Chunk.SetContentType(ZenContentType::kCompressedBinary); return AsyncCallback(Index, Payload); }, - OptionalWorkerPool)) + OptionalWorkerPool, + LargeSizeLimit)) { return false; } |