aboutsummaryrefslogtreecommitdiff
path: root/src/zenstore
Commit message (Collapse)AuthorAgeFilesLines
* add more trace scopes (#362)Dan Engelbrecht2023-09-156-114/+150
| | | | | * more trace scopes * Make sure ReplayLogEntries uses the correct size for oplog buffer * changelog
* More statistics for Cache, Project Store and Cid Store (#405)Dan Engelbrecht2023-09-142-10/+45
| | | | | Cache: requestcount, badrequestcount, writes Project Store: requestcount Cid Store: cidhits, cidmisses, cidwrites
* stream oplog attachments from jupiter (#384)Dan Engelbrecht2023-09-062-75/+27
| | | | | | | | | | * stream large downloads from jupiter to temporary file * rework DeleteOnClose - top level marks file for delete and if lower level parts wants to keep it it clears that flag * changelog * log number of attachments to download * add delay on jupiter request failure when retrying * make sure we upload all attachments even if Needs are empty when ForceUpload is true release TempAttachment as soon as it is used * sort attachments so we get predictable blocks for the same oplog
* retry file create (#383)Dan Engelbrecht2023-09-042-3/+27
| | | | | * add retry logic when creating files * only write disk usage log if disk writes are allowed * changelog
* safer gc on low disk (#373)Dan Engelbrecht2023-08-222-37/+46
| | | * - Improvement: Make sure we have disk space available to do GC and use reserve up front if need be
* use robinmap in compact cas (#368)Dan Engelbrecht2023-08-213-10/+19
| | | | | * Use robin-map in compactcas for 30% faster CasContainerStrategy::CollectGarbage * use robin_set in ProjectStore::Oplog::GatherReferences and BlockStore::ReclaimSpace * changelog
* update vcpkg dependencies (#356)Dan Engelbrecht2023-08-111-1/+1
| | | | | | * bump vcpkg version * fmt lib 10 fixes * xmake dependencies (with linux workarounds) * changelog
* catch exceptions when scheduling GC and when writing GC scheduling state (#339)Dan Engelbrecht2023-08-011-136/+149
| | | * catch exceptions when scheduling GC and when writing GC scheduling state
* CidStore now implements the ChunkResolver interfaceStefan Boberg2023-06-304-13/+21
| | | | | this allows client code to use the ChunkResolver interface instead of CidStore, which can help with testing scenarios
* added zen::ChunkResolverStefan Boberg2023-06-301-0/+9
| | | | cherry-picked from sb/proto to reduce delta
* drive-by constStefan Boberg2023-06-121-1/+1
|
* Content scrubbing (#271)Stefan Boberg2023-05-167-103/+316
| | | Added zen scrub command which may be triggered via the zen CLI helper. This traverses storage and validates contents either by content hash and/or by structure. If unexpected data is encountered it is invalidated.
* Additional trace instrumentation (#312)Stefan Boberg2023-05-165-1/+65
| | | | | | | | | * added trace instrumentation to upstreamcache * added asio trace instrumentation * added trace annotations for project store * added trace annotations for BlockStore * added trace annotations for HttpClient * added trace annotations for CAS/GC
* Add `--gc-projectstore-duration-seconds` option (#281)Dan Engelbrecht2023-05-164-45/+83
| | | | | | * Add `--gc-projectstore-duration-seconds` option * Cleanup lua gc options parsing * Remove dead configuration values * changelog
* removed remnants of ZEN_USE_REF_TRACKINGStefan Boberg2023-05-152-28/+0
| | | | this code was originally meant to be used for GC but is no longer needed
* added error reporting for bad BlockStore chunksStefan Boberg2023-05-151-5/+22
| | | | also eliminated some copy-paste
* removed unnecessary #include directives from cas.hStefan Boberg2023-05-151-10/+3
|
* added ScrubStorage to GcStorage base classStefan Boberg2023-05-155-15/+38
|
* added static_assert for BlockStoreDiskLocationStefan Boberg2023-05-151-0/+2
|
* corrected CidStore commentStefan Boberg2023-05-151-4/+0
|
* CidStore::Impl explicit constructor, fixed #ifndef NDEBUGStefan Boberg2023-05-151-3/+4
|
* minor GC API cleanupStefan Boberg2023-05-1510-35/+35
| | | | | Scrub -> ScrubStorage Trigger -> TriggerGc (to make relationship to TriggerScrub clearer)
* Remove ATL header usage (#306)Stefan Boberg2023-05-151-4/+4
| | | ATL has been used here and there as a convenience. Given that this is a legacy component and not always something which gets installed along with the compiler we hereby remove the dependency altogether in favour of our own simple wrappers
* fix logic for old blocks in blockstore gc (#295)Dan Engelbrecht2023-05-121-21/+24
| | | | | | | * fix logic for old blocks in blockstore gc If we will remove all entries in a block and keep nothing we can't expect for the block to exist. If we want to keep entries in a block, the block must exist, if not error and move entries to delete list. Don't reset output block between blocks we are reading from, keep using it until it exceeds the max limit. * changelog
* clang-format (sorry)Stefan Boberg2023-05-111-1/+1
|
* build fix (accidental commit on the wrong branch)Stefan Boberg2023-05-112-1/+4
|
* clang-formatStefan Boberg2023-05-111-1/+3
|
* Merge branch 'main' of https://github.com/EpicGames/zenStefan Boberg2023-05-112-24/+33
|\
| * flush file cas on exit (#291)Dan Engelbrecht2023-05-112-24/+33
| | | | | | | | | | | | | | | | * flush caslog and index snapshot on flush * fix save reading of index/logfile with cleanup write snapshot at flush * don't validate entries we just scanned/created * fix total size found when scanning for cas files * changelog
* | added scrubcontext.cppStefan Boberg2023-05-111-0/+15
|/
* if a block is missing during gc, log an error but still continue (#289)Dan Engelbrecht2023-05-111-4/+8
|
* WARN level log if we can't write snapshot/manifest/access times (#288)Dan Engelbrecht2023-05-111-1/+1
|
* clean up log/index reading and fix incorrect logging about bad log files (#286)Dan Engelbrecht2023-05-102-84/+98
|
* make sure we create gc root directory before checking disk spaceDan Engelbrecht2023-05-101-7/+7
|
* Low disk space detector (#277)Dan Engelbrecht2023-05-093-14/+83
| | | | * - Feature: Disk writes are now blocked early and return an insufficient storage error if free disk space falls below the `--low-diskspace-threshold` value * Never keep an entry in m_ChunkBlocks that points to a nullptr
* Validate that entries points inside valid blocks at startup (#280)Dan Engelbrecht2023-05-093-42/+93
| | | | | * Separate initialization of block store from pruning of unknown blocks * Validate that entries points inside valid blocks
* Treat reading outside of block store file as a not found error. We may ↵Dan Engelbrecht2023-05-021-0/+4
| | | | encounter truncated blocks due to earlier abnormal termination of zenserver or disk failures. (#268)
* moved source directories into `/src` (#264)Stefan Boberg2023-05-0220-0/+7371
* moved source directories into `/src` * updated bundle.lua for new `src` path * moved some docs, icon * removed old test trees
generated by cgit v1.2.3 (git 2.25.1) at 2026-04-22 11:14:02 +0000