aboutsummaryrefslogtreecommitdiff
path: root/src/zenstore/cache/structuredcachestore.cpp
Commit message (Collapse)AuthorAgeFilesLines
* speed up tests (#555)Dan Engelbrecht2025-10-061-32/+56
| | | | | | | | | | | | * faster FileSystemTraversal test * faster jobqueue test * faster NamedEvent test * faster cache tests * faster basic http tests * faster blockstore test * faster cache store tests * faster compactcas tests * more responsive zenserver launch * tweak worker pool sizes in tests
* cacherequests helpers test only (#551)Dan Engelbrecht2025-10-031-2/+1
| | | | * don't use cacherequests utils in cache_cmd.cpp * make zenutil/cacherequests code into test code helpers only
* Adjust the responses from PUT commandszousar2025-09-231-2/+2
| | | | | - Ensure that text responses are in a field named "Message" - Change the record response to be named "Record" instead of "Object"
* Change batch put responses for client reportingzousar2025-09-191-2/+7
| | | | Conflicts are now treated as successes, and we optionally return a Details array instead of an ErrorMessages array. Details are returned for all requests in a batch, or no requests in a batch depending on whether there are any details to be shared about any of the put requests. The details for a conflict include the raw hash and raw size of the item. If the item is a record, we also include the record as an object.
* add EMode to WorkerTheadPool to avoid thread starvation (#492)Dan Engelbrecht2025-09-101-45/+59
| | | - Improvement: Add a new mode to worker thread pools to avoid starvation of workers which could cause long stalls due to other work begin queued up. UE-305498
* per namespace/project cas prep refactor (#470)Dan Engelbrecht2025-08-201-39/+29
| | | | | | | - Refactor so we can have more than one cas store for project store and cache. - Refactor `UpstreamCacheClient` so it is not tied to a specific CidStore - Refactor scrub to keep the GC interface ScrubStorage function separate from scrub accessor functions (renamed to Scrub). - Refactor storage size to keep GC interface StorageSize function separate from size accessor functions (renamed to TotalSize) - Refactor cache storage so `ZenCacheDiskLayer::CacheBucket` implements GcStorage interface rather than `ZenCacheNamespace`
* Merge branch 'main' into zs/put-overwrite-policyzousar2025-06-241-18/+33
|\
| * make sure we unregister from GC before we drop bucket/namespaces (#443)Dan Engelbrecht2025-06-191-0/+1
| |
| * unblock cache bucket drop (#406)Dan Engelbrecht2025-05-261-14/+28
| | | | | | | | * don't hold exclusive locks while deleting files from a dropped bucket/namespace * cleaner detection of missing namespace when issuing a drop
| * reduced memory churn using fixed_xxx containers (#236)Stefan Boberg2025-03-061-4/+4
| | | | | | | | | | | | * Added EASTL to help with eliminating memory allocations * Applied EASTL to eliminate memory allocations, primarily by using `fixed_vector` et al to use stack allocations / inline struct allocations Reduces memory events in traces by close to a factor of 10 in test scenario (starting editor for project F)
* | Change to PutResult structurezousar2025-06-241-10/+10
| | | | | | | | Result structure contains status and a string message (may be empty)
* | Enforce Overwrite Prevention According To Cache Policyzousar2025-02-261-23/+39
|/ | | | Overwrite with differing value should be denied if QueryLocal is not present and StoreLocal is present. Overwrite with equal value should succeed regardless of policy flags.
* Add multithreading directory scanning in core/filesystem (#277)Dan Engelbrecht2025-01-221-1/+1
| | | | | | add DirectoryContent::IncludeFileSizes add DirectoryContent::IncludeAttributes add multithreaded GetDirectoryContent use multithreaded GetDirectoryContent in workspace folder scanning
* added support for dynamic LLM tags (#245)Stefan Boberg2024-12-021-0/+27
| | | | | * added FLLMTag which can be used to register memory tags outside of core * changed `UE_MEMSCOPE` -> `ZEN_MEMSCOPE` for consistency * instrumented some subsystems with dynamic tags
* oplog prep gc fix (#216)Dan Engelbrecht2024-11-151-18/+35
| | | | | | - Added option gc-validation to zenserver that does a check for missing references in all oplog post full GC. Enabled by default. - Feature: Added option gc-validation to zen gc command to control reference validation. Enabled by default. - Added more details in post GC log. - Fixed race condition in oplog writes which could cause used attachments to be incorrectly removed by GC
* bucket size queries (#203)Dan Engelbrecht2024-10-211-1/+23
| | | - Feature: Added options --bucketsize and --bucketsizes to zen cache-info to get data sizes in cache buckets and attachments
* remove gc v1 (#121)Dan Engelbrecht2024-10-031-283/+17
| | | | | * kill gc v1 * block use of gc v1 from zen command line * warn and flip to gcv2 if --gc-v2=false is specified for zenserver
* Add `gc-attachment-passes` option to zenserver (#167)Dan Engelbrecht2024-09-251-1/+1
| | | | | Added option `gc-attachment-passes` to zenserver Cleaned up GCv2 start and stop logs and added identifier to easily find matching start and end of a GC pass in log file Fixed project store not properly sorting references found during lock phase
* gc unused refactor (#165)Dan Engelbrecht2024-09-231-14/+10
| | | | | * optimize IoHash and OId comparisions * refactor filtering of unused references * add attachment filtering to gc
* unblock PreCache (#164)Dan Engelbrecht2024-09-201-1/+1
| | | Don't lock disk cache buckets from writing when scanning records for attachment references
* trace scopes improvementsDan Engelbrecht2024-09-101-2/+2
|
* move gc logs to gc logger (#142)Dan Engelbrecht2024-09-041-1/+9
| | | - Improvement: Move GC logging in callback functions into "gc" context
* oplog index snapshots (#140)Dan Engelbrecht2024-09-031-0/+2
| | | - Feature: Added project store oplog index snapshots for faster opening of oplog - opening oplogs are roughly 10x faster
* Make sure `noexcept` functions does not leak exceptions (#136)Dan Engelbrecht2024-08-231-6/+6
|
* Make sure we monitor for new project, oplogs, namespaces and buckets during ↵Dan Engelbrecht2024-06-131-0/+235
| | | | | | GCv2 (#93) - Bugfix: Make sure we monitor and include new project/oplogs created during GCv2 - Bugfix: Make sure we monitor and include new namespaces/cache buckets created during GCv2
* add batching of CacheStore requests for GetCacheValues/GetCacheChunks (#90)Dan Engelbrecht2024-06-041-10/+135
| | | | | | * cache file size of block on open * add ability to control size limit for small chunk callback when iterating block * Add batch fetch of cache values in the GetCacheValues request
* batch cache put (#67)Dan Engelbrecht2024-05-021-12/+69
| | | - Improvement: Batch scope for put of cache values
* use direct file access for large file hash (#63)Dan Engelbrecht2024-04-261-4/+4
| | | - Improvement: Refactor `IoHash::HashBuffer` and `BLAKE3::HashBuffer` to not use memory mapped files. Performs better and saves ~10% of oplog export time on CI
* improved assert (#37)Dan Engelbrecht2024-04-041-2/+2
| | | | - Improvement: Add file and line to ASSERT exceptions - Improvement: Catch call stack when throwing assert exceptions and log/output call stack at important places to provide more context to caller
* clean up test linking (#4)Dan Engelbrecht2024-03-141-10/+10
| | | | | | | - Improvement: Add zenhttp-test and zenutil-test - Improvement: Moved cachepolicy test to cachepolicy.cpp - Improvement: Renamed cachestore tests from z$ to cachestore - Improvement: Moved test linking so test for a lib is linked by <lib>-test - Improvement: Removed HttpRequestParseRelativeUri in httpstructuredcache.cpp and use the one in cacherequests.h instead
* improved block store logging and more gcv2 tests (#659)Dan Engelbrecht2024-02-271-10/+1
| | | | * improved gc/blockstore logging * more gcv2 tests
* remove reference caching (#658)Dan Engelbrecht2024-02-271-135/+84
| | | * remove reference caching
* hashing fixes (#657)Dan Engelbrecht2024-02-261-1/+72
| | | | | * move structuredcachestore tests to zenstore-test * Don't materialize entire files when hashing if it is a large files * rewrite CompositeBuffer::Mid to never materialize buffers
* separate RPC processing from HTTP processing (#626)Stefan Boberg2023-12-201-1/+1
| | | | | | * moved all RPC processing from HttpStructuredCacheService into separate CacheRpcHandler class in zenstore * move package marshaling to zenutil. was previously in zenhttp/httpshared but it's useful in other contexts as well where we don't want to depend on zenhttp * introduced UpstreamCacheClient, this provides a subset of functions on UpstreamCache and lives in zenstore
* move cachedisklayer and structuredcachestore into zenstore (#624)Stefan Boberg2023-12-191-0/+2440
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-12 23:04:47 +0000