aboutsummaryrefslogtreecommitdiff
path: root/zenserver/cache/structuredcachestore.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Support file reference in package message (#184)Dan Engelbrecht2022-11-071-1/+1
| | | | | | | | * Fix packed message parsing for absolute path * Always enable are sharing when opening files as IoBuffers. * Allow control over sending partial files as localfile ref * Check "AcceptFlags" field in RPC message for allowing localfile ref in reply * make oplog entry add operations ZEN_DEBUG level logs * changelog
* Adjust errors vs warnings messages (#160)Dan Engelbrecht2022-09-081-1/+1
| | | | * demote a number of ZEN_ERROR to ZEN_WARN * changelog
* Use "\\?\" prefixed paths and fix hardcoded path delimiters (#149)Dan Engelbrecht2022-08-261-1/+3
| | | | | | * use "\\?\" prefix for windows paths * fix path delimiters * disable vcpkg caching * Workaround for spdlog not being able to create directories prefixed with `\\?\`
* Make cas storage an hidden implementation detail of CidStore (#130)v0.1.4-pre6v0.1.4-pre5Dan Engelbrecht2022-06-171-549/+179
| | | | | | | | | | | | | | | | | | - Bumped ZEN_SCHEMA_VERSION - CasStore no longer a public API, it is hidden behind CidStore - Moved cas.h from public header folder - CidStore no longer maps from Cid -> Cas, we store entries in Cas under RawHash - CasStore now decompresses data to validate content (matching against RawHash) - CasChunkSet renames to HashKeySet and put in separate header/cpp file - Disabled "Chunk" command for now as it relied on CAS being exposed as a service - Changed CAS http service to Cid http server - Moved "Run" command completely inside ZEN_WITH_EXEC_SERVICES define - Removed "cas.basic" test - Uncommented ".exec.basic" test and added return-skip at start of test - Moved ScrubContext to separate header file - Renamed CasGC to GcManager - Cleaned up configuration passing in cas store classes - Removed CAS stuff from GcContext and clarified naming in class - Remove migration code
* merged from mainStefan Boberg2022-06-161-24/+116
|
* fixed mac build ("unused" variable)Stefan Boberg2022-06-111-0/+2
|
* Use m_BucketDir for cache key contextDan Engelbrecht2022-06-081-2/+2
| | | | Easier to debug and no conflicts
* We need to make each Cache Key contribution per bucket unique even across ↵Dan Engelbrecht2022-06-071-2/+2
| | | | namespaces
* Don't hold index lock while reading standalone valuesDan Engelbrecht2022-06-031-0/+2
| | | | Can cause deadlock
* hardening of ZenCacheDiskLayer::CacheBucket::PutStandaloneCacheValueDan Engelbrecht2022-06-031-13/+36
|
* add comment about removing filesDan Engelbrecht2022-05-301-0/+1
|
* Enable FILE_SHARE_DELETE on standalone files in disk bucketsDan Engelbrecht2022-05-281-107/+77
| | | | | | | | This allows us to delete the file even if it is open for read. We do a delete, the rename since we are not allowed to do a rename-overwrite, only delete. As we have the shard lock for the file we want to replace we can safely do a delete+rename. In the rare case that we fail to rename the file into place the old data is lost. As this is a *cache* and it should be very rare this is OK.
* bugfixes and test for namespace dropDan Engelbrecht2022-05-251-15/+92
|
* clean up namespace foldersDan Engelbrecht2022-05-251-36/+42
|
* namespace dropDan Engelbrecht2022-05-251-1/+24
|
* Make sure ZenCacheMemoryLayer handles dropped buckets correctly (just like ↵Dan Engelbrecht2022-05-251-32/+87
| | | | ZenCacheDiskLayer)
* dropIndex -> DropIndexDan Engelbrecht2022-05-251-3/+3
|
* If a bucket is in m_BucketMap it is OK, no need for separate flagDan Engelbrecht2022-05-251-56/+21
|
* drop bucket testDan Engelbrecht2022-05-241-0/+73
|
* Use rename/delete and keep pointer for dropped bucketsDan Engelbrecht2022-05-241-53/+104
|
* Make sure to hold exclusive lock over index and all shard locks.Dan Engelbrecht2022-05-241-1/+9
| | | | Clear index on drop.
* Merge pull request #102 from EpicGames/de/auto-create-namespacesDan Engelbrecht2022-05-241-19/+52
|\ | | | | Automatically create namespaces on requests (if enabled via configuration)
| * Automatically create namespaces on requests (if enabled via configuration)Dan Engelbrecht2022-05-201-19/+52
| |
* | De/fix namespace folder scanning (#103)Dan Engelbrecht2022-05-231-1/+1
|/
* Merge pull request #99 from EpicGames/de/move-namespace-fieldDan Engelbrecht2022-05-191-5/+19
|\ | | | | Keep Namespace out of CacheKey and store it on request level
| * migrate legacy cache folders to ue4.ddc namespaceDan Engelbrecht2022-05-191-5/+19
| | | | | | | | | | map default namespace to at runtime ue4.ddc use a non-valid name for the default namespace so we avoid any collision or accidental creation of folder for that
* | Fix and retry count and add an extra iteration to give more time for successDan Engelbrecht2022-05-191-5/+5
|/
* fix release build, misplaced namespace bracketsDan Engelbrecht2022-05-171-4/+0
|
* use ns_ prefix on disk onlyDan Engelbrecht2022-05-161-4/+6
|
* Tests for HttpRequestParseRelativeUriDan Engelbrecht2022-05-121-6/+5
|
* Add namespace testDan Engelbrecht2022-05-111-0/+53
|
* Make sure we clean up temp file in all scenariosDan Engelbrecht2022-05-101-8/+20
|
* happy path should be minimal workDan Engelbrecht2022-05-091-23/+23
|
* make test run on more platformsDan Engelbrecht2022-05-091-7/+7
|
* Restore logic where we accept failed overwrite if resulting size is the sameDan Engelbrecht2022-05-091-3/+44
| | | | Correctly calculate the m_TotalSize difference when overwriting file
* Make sure CacheBucket::PutStandaloneCacheValue cleans up the temp file if we ↵Dan Engelbrecht2022-05-091-0/+48
| | | | fail to move the it into place
* Merge pull request #89 from EpicGames/de/namespacesv1.0.1.5Dan Engelbrecht2022-05-091-65/+195
|\ | | | | Add namespacecachestore layer to allow multiple structured cache namespaces
| * remove use of Ref<> in ZenCacheStoreDan Engelbrecht2022-05-061-20/+20
| | | | | | | | naming cleanup
| * review feedback and cleanupDan Engelbrecht2022-05-061-45/+36
| |
| * Added GetDirectoryContent utilityDan Engelbrecht2022-05-061-65/+13
| |
| * revert back constructor order for ZenCacheStoreDan Engelbrecht2022-05-051-1/+1
| |
| * mac/linux build fixDan Engelbrecht2022-05-051-2/+6
| |
| * cleanup and review feedbackDan Engelbrecht2022-05-051-20/+13
| |
| * cleanupDan Engelbrecht2022-05-041-31/+225
| |
* | Merge pull request #90 from EpicGames/de/simplify-cache-bucket-put-standalonev1.0.1.4Dan Engelbrecht2022-05-091-64/+72
|\ \ | | | | | | Fix standalone file lock in CacheBucket
| * | fix exception message/loggingDan Engelbrecht2022-05-091-3/+6
| | |
| * | restore write using rename in PutStandaloneCacheValueDan Engelbrecht2022-05-061-89/+61
| | |
| * | clean up file on failed writeDan Engelbrecht2022-05-061-32/+41
| | |
| * | Fix standalone file lock in CacheBucketDan Engelbrecht2022-05-061-73/+97
| |/ | | | | | | | | | | Grab sharding lock when deleting files during GC Don't hold sharding lock when sleeping in back-off due to file contention Remove unneeded renaming logic when writing standalone cache values
* / Make sure blockstore owner and block store state does not get out of sync ↵Dan Engelbrecht2022-05-081-26/+22
|/ | | | | | when fetching a chunk Move MarkAsDeleteOnClose() to IoBuffer(ExtendedCore) and set it on close, SetFileInformationByHandle sometimes fails if done in parallel with FileMapping