aboutsummaryrefslogtreecommitdiff
path: root/zenserver/cache/structuredcache.h
Commit message (Collapse)AuthorAgeFilesLines
* moved source directories into `/src` (#264)Stefan Boberg2023-05-021-187/+0
| | | | | | * moved source directories into `/src` * updated bundle.lua for new `src` path * moved some docs, icon * removed old test trees
* oplog and cache stats (#244)Dan Engelbrecht2023-04-211-0/+1
| | | | | | | | | | | | | | | | * basic oplog stats * add GetValueStats to cache store * RwLock::ExclusiveLockScope -> RwLock::SharedLockScope * add rawhash and attachment count to CacheValueStats * added cache-stats and project-stats commands * add cast to make Mac overload detection happy * fix accept type in cache-stats command * Add options to project-stats command * use resource paths for stats in project store * use resource paths for stats in cache * fix cache-info and project-info url discriminator * more control over details$ output * cleanup * changelog
* send payloads as duplicated handles (#240)Dan Engelbrecht2023-03-141-1/+2
| | | | | | | | | * send payloads as duplicated handles if requestor provides process id and allows local file references. * linux/macos fixes * tests * fix access rights when duplicating handle * fix closing of duplicated handles on error * cleanup * changelog
* Enhanced rpc request recording (#229)Dan Engelbrecht2023-02-171-28/+10
| | | | | | | | | | | | | * rpc replay zen command * fix replay sessions for thread * recording start/stop as zen commands * move rpcrecording code to zenutil to remove code duplication * simplify recording http request threading * added more data logging to rpc replay * NotFound is an acceptable response for an rpc request * fix rpc replay command line parsing * rpc replay stats * Allow spawning of sub-process workers when replaying rpc recording * changelog
* Add info (GET) endpoints for structured cache (#211)Dan Engelbrecht2023-01-111-12/+13
| | | | | | | * Add GET requests on cache/namespace/bucket level * Add root route for project store requests (same as /list) * Add markerpath to oplog info object * Add totalsize, opcount and expired to oplog info * Changelog
* Cache request record/replay (#198)Dan Engelbrecht2022-12-071-20/+47
| | | | | | | | | This adds recording and playback of cache request with full data - both get and put operations can be replayed. Invoke via web request. `<host>/z$/exec$/start-recording?<disk-storage-path>` `<host>/z$/exec$/stop-recording` `<host>/z$/exec$/replay-recording?<thread-count>&<disk-storage-path>`
* Support file reference in package message (#184)Dan Engelbrecht2022-11-071-0/+2
| | | | | | | | * Fix packed message parsing for absolute path * Always enable are sharing when opening files as IoBuffers. * Allow control over sending partial files as localfile ref * Check "AcceptFlags" field in RPC message for allowing localfile ref in reply * make oplog entry add operations ZEN_DEBUG level logs * changelog
* Add "Accept" field in RPC request to gracefully handle requests from older ↵Dan Engelbrecht2022-10-131-2/+4
| | | | instances (#180)
* rename URI chunk requests from value -> chunk (#166)Dan Engelbrecht2022-09-191-3/+3
|
* removed unnecessary CasStore forward declarationsStefan Boberg2022-06-131-1/+0
|
* Keep Namespace out of CacheKey and store it on request levelDan Engelbrecht2022-05-191-5/+12
| | | | | RPC requests now has a Namespace field under Params instead of one Namespace per cache key Fall back to legacy upstream HTTP URI format if default namespace is requested
* review feedbackDan Engelbrecht2022-05-161-5/+5
|
* Add validation to namespace names that follows Jupiters rules.Dan Engelbrecht2022-05-121-18/+17
| | | | | Add unified validation of Namespace, Bucket and Hash for rpc requests. cleanup
* Tests for HttpRequestParseRelativeUriDan Engelbrecht2022-05-121-0/+2
|
* Add support for /api/v2/ URI requests with namespace supportDan Engelbrecht2022-05-121-2/+3
|
* revert Bucket - Key parameter changeDan Engelbrecht2022-05-121-1/+1
|
* cleanupDan Engelbrecht2022-05-111-1/+1
|
* parameterize namespace for upstream (first hack)Dan Engelbrecht2022-05-111-1/+2
|
* Fixed "Record* Record;" non-conformant compile errors from GCCMartin Ridgers2022-02-111-3/+3
|
* Simplify HandleRpcGetCacheChunks (#53)mattpetersepic2022-02-091-10/+27
| | | Refactor HandleRpcGetCacheChunks to reduce complexity. Port CacheStore tests from Unreal.
* Remove the backwards compatibility for the Zen CachePolicy changes no… (#49)mattpetersepic2022-02-081-20/+13
| | | Remove the backwards compatibility for the Zen CachePolicy changes now that there has been enough time for all internal users of the old protocol to update.
* Cache policy support (#47)mattpetersepic2022-02-011-6/+36
| | | | | | | | Add HandleRpc methods for the remaining ICacheStore requests from unreal: PutCacheValues/GetCacheValues. We now have batched versions for PutCacheRecords,GetCacheRecords,PutCacheValues,GetCacheValues,GetCacheChunks. Add support for CachePolicy flags to all of these batched methods. * Add Batched PutCacheValues/GetCacheValues. Rename old GetCacheValues to GetCacheChunks. * HandleRpcGetCacheRecords: Receive a CacheRecordPolicy with each key, and skipdata on attachments we already have. * Changes to CachePolicy copied from Release-5.0 depot. Change serialization to use the key BasePolicy instead of DefaultValuePolicy. * GetChunks: Read CacheRecords from remote if necessary to find ContentId. Implement QueryLocal, StoreLocal, and SkipData.
* Rename Paylod to Value to match the client side. Rename PayloadId to ↵mattpetersepic2022-01-271-5/+5
| | | | ValueContentId where its a hash instead of an oid.
* Add batched CacheRecord put rpc (#38)mattpetersepic2022-01-271-0/+9
| | | * Add batched CacheRecord put rpc
* Cachepolicy (#36)mattpetersepic2022-01-251-7/+7
| | | | | | | | | | | | | | | | | * Copy CachePolicy implementation from UE5/Release-5.0. Add backwards compatability for clients and upstreams that are using the old protocol. * Add RefPtr templated move operator and constructor, so that RefPtr<const Foo*> A = std::move(RefPtr<Foo*>()) will do a move. * Fix broken CachePolicy tests and add tests for new Save/Load. * Remove TODO comments * CachePolicy Save/Load Fixes from codereview * Fix comment to match code change. * Remove backwards compatibility for CachePolicy change. Convert policy string tokens to PascalCase. Fix tests for new policy text. Change ParseCachePolicy to assert string is non-empty and always succeed. * Fix release build: use ZEN_WITH_TESTS define
* Refactored upstream cache to better handle different states in prep for ↵Per Larsson2022-01-241-16/+16
| | | | dynamic auth tokens.
* Fixed up some merge falloutStefan Boberg2021-11-181-0/+1
|
* merge from mainStefan Boberg2021-11-181-1/+4
|\
| * Updated cache policy according to UE.Per Larsson2021-11-151-1/+1
| |
| * Changed from batch to RPC.Per Larsson2021-11-111-3/+3
| |
| * Handle cache record policy.Per Larsson2021-11-101-3/+3
| |
| * Sort cache keys when resolving payload ID's.Per Larsson2021-11-091-1/+1
| |
| * Added batched get chunk(s).Per Larsson2021-11-081-0/+2
| |
| * First pass batch request.Per Larsson2021-10-291-0/+1
| |
* | Add /vsxmake* to .gitignoreStefan Boberg2021-11-031-0/+1
| |
* | z$: Removed CasStoreStefan Boberg2021-10-211-2/+0
|/
* Support for asynchronous HTTP response processing (#19)Stefan Boberg2021-10-061-6/+6
| | | | | | | | This change introduces WriteResponseAsync which can be used to move potentially slow request handler code (like upstream lookups) off the I/O service thread to ensure we are always able to serve as many HTTP requests as possible. The current implementation defaults to 16 async worker threads and there is currently no back-pressure. - Added RequestStats - Metrics for network requests. Aggregates tracking of duration, payload sizes into a single class for ease of use - Added some metrics on upstream communication Co-authored-by: Per Larsson <[email protected]>
* Merged from upstreamStefan Boberg2021-10-051-7/+15
|
* Added cache HIT/MISS stat counters.Per Larsson2021-10-011-0/+8
|
* structured cache: Rewrote metrics tracking using OperationTimingStefan Boberg2021-09-301-3/+1
|
* structured cache: added request meter and stats reportingStefan Boberg2021-09-301-0/+1
|
* structured cache: Added stats test code (needs additional work / metrics)Stefan Boberg2021-09-301-0/+4
|
* Refactored get/set cache pyload.Per Larsson2021-09-211-1/+3
|
* Merge branch 'main' of https://github.com/EpicGames/zenPer Larsson2021-09-211-0/+1
|\
| * Wired up scrubbing to more higher level servicesStefan Boberg2021-09-211-0/+1
| | | | | | | | Also moved sharding logic for filecas into a function to redduce cut/pasta
* | Refactored out get/set cache record.Per Larsson2021-09-211-1/+3
|/
* Added more scrub stubs in higher level servicesStefan Boberg2021-09-201-0/+1
|
* Moved more code into zen namespace, for consistencyStefan Boberg2021-09-201-3/+2
| | | | Also removed snapshot_manifest (remnants of vfs prototype)
* Initial support for cache policies.Per Larsson2021-09-171-2/+3
|
* Fixed logging in structured cache (did not end up getting routed right)Stefan Boberg2021-09-161-0/+1
|
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-12 14:33:02 +0000