diff options
| author | Dan Engelbrecht <[email protected]> | 2023-02-23 14:54:22 +0100 |
|---|---|---|
| committer | GitHub <[email protected]> | 2023-02-23 05:54:22 -0800 |
| commit | d361aa896e2e74ae4a790c4668c78c830f9b5d1c (patch) | |
| tree | c76518eaab8d4b6b0ba185bdec0fe07639729ea8 /zenserver/cache/structuredcachestore.h | |
| parent | junit test reporting (#239) (diff) | |
| download | zen-d361aa896e2e74ae4a790c4668c78c830f9b5d1c.tar.xz zen-d361aa896e2e74ae4a790c4668c78c830f9b5d1c.zip | |
store cache rawhash and rawsize for unstructured cache values (#234)
* refactored MemoryCacheBucket to allow for storing RawHash/RawSize.
* remove redundant conversions in AccessTime
* reduce max count for memory cache bucket to 32-bit value
* refactored DiskCacheBucket to allow for storing RawHash/RawSize.
* Use CompressedBuffer::ValidateCompressedHeader when applicable
* Make sure we rewrite the snapshot if we read an legacy existing index/log
* changelog
Diffstat (limited to 'zenserver/cache/structuredcachestore.h')
| -rw-r--r-- | zenserver/cache/structuredcachestore.h | 21 |
1 files changed, 13 insertions, 8 deletions
diff --git a/zenserver/cache/structuredcachestore.h b/zenserver/cache/structuredcachestore.h index 3d2896ff8..fe83f3c97 100644 --- a/zenserver/cache/structuredcachestore.h +++ b/zenserver/cache/structuredcachestore.h @@ -63,7 +63,8 @@ namespace access_tracking { struct ZenCacheValue { IoBuffer Value; - CbObject IndexData; + uint64_t RawSize = 0; + IoHash RawHash = IoHash::Zero; }; ////////////////////////////////////////////////////////////////////////// @@ -125,12 +126,15 @@ struct DiskLocation struct DiskIndexEntry { - IoHash Key; // 20 bytes + uint64_t RawSize; // 8 bytes DiskLocation Location; // 12 bytes + IoHash RawHash; // 20 bytes + IoHash Key; // 20 bytes }; + #pragma pack(pop) -static_assert(sizeof(DiskIndexEntry) == 32); +static_assert(sizeof(DiskIndexEntry) == 60); // This store the access time as seconds since epoch internally in a 32-bit value giving is a range of 136 years since epoch struct AccessTime @@ -358,13 +362,14 @@ private: void BuildPath(PathBuilderBase& Path, const IoHash& HashKey); void PutStandaloneCacheValue(const IoHash& HashKey, const ZenCacheValue& Value); - bool GetStandaloneCacheValue(const DiskLocation& Loc, const IoHash& HashKey, ZenCacheValue& OutValue); + IoBuffer GetStandaloneCacheValue(const DiskLocation& Loc, const IoHash& HashKey); void PutInlineCacheValue(const IoHash& HashKey, const ZenCacheValue& Value); - bool GetInlineCacheValue(const DiskLocation& Loc, ZenCacheValue& OutValue); + IoBuffer GetInlineCacheValue(const DiskLocation& Loc); void MakeIndexSnapshot(); - uint64_t ReadIndexFile(); - uint64_t ReadLog(uint64_t LogPosition); - void OpenLog(const std::filesystem::path& BucketDir, const bool IsNew); + uint64_t ReadIndexFile(const std::filesystem::path& IndexPath, uint32_t& OutVersion); + uint64_t ReadLogV2(const std::filesystem::path& LogPath, uint64_t LogPosition); + uint64_t ReadLog(const std::filesystem::path& LogPath, uint64_t LogPosition); + void OpenLog(const bool IsNew); void SaveManifest(); // These locks are here to avoid contention on file creation, therefore it's sufficient |