aboutsummaryrefslogtreecommitdiff
path: root/src/zenstore/filecas.cpp
Commit message (Collapse)AuthorAgeFilesLines
* faster accesstime save restore (#439)Dan Engelbrecht2023-10-031-5/+18
| | | | | | | | | | - Improvement: Reduce time a cache bucket is locked for write when flushing/garbage collecting - Change format for faster read/write and reduced size on disk - Don't lock index while writing manifest to disk - Skip garbage collect if we are currently in a Flush operation - BlockStore::Flush no longer terminates currently writing block - Garbage collect references to currently writing block but keep the block as new data may be added - Fix BlockStore::Prune used disk space calculation - Don't materialize data in filecas when we just need the size
* Handle OOM and OOD more gracefully to not spam Sentry with error reports (#434)Dan Engelbrecht2023-10-021-9/+25
| | | | | | - Improvement: Catch Out Of Memory and Out Of Disk exceptions and report back to reqeuster without reporting an error to Sentry - Improvement: If creating bucket fails when storing and item in the structured cache, log a warning and propagate error to requester without reporting an error to Sentry - Improvement: Make an explicit flush of the active block written to in blockstore flush - Improvement: Make sure cache and cas MakeIndexSnapshot does not throw exception on failure which would cause and abnormal termniation at exit
* lightweight gc (#431)Dan Engelbrecht2023-10-021-0/+5
| | | | | | - Feature: Add lightweight GC that only removes items from cache/project store without cleaning up data referenced in Cid store - Add `skipcid` parameter to http endpoint `admin/gc`, defaults to "false" - Add `--skipcid` option to `zen gc` command, defaults to false - Add `--gc-lightweight-interval-seconds` option to zenserver
* add more trace scopes (#362)Dan Engelbrecht2023-09-151-23/+32
| | | | | * more trace scopes * Make sure ReplayLogEntries uses the correct size for oplog buffer * changelog
* stream oplog attachments from jupiter (#384)Dan Engelbrecht2023-09-061-72/+24
| | | | | | | | | | * stream large downloads from jupiter to temporary file * rework DeleteOnClose - top level marks file for delete and if lower level parts wants to keep it it clears that flag * changelog * log number of attachments to download * add delay on jupiter request failure when retrying * make sure we upload all attachments even if Needs are empty when ForceUpload is true release TempAttachment as soon as it is used * sort attachments so we get predictable blocks for the same oplog
* update vcpkg dependencies (#356)Dan Engelbrecht2023-08-111-1/+1
| | | | | | * bump vcpkg version * fmt lib 10 fixes * xmake dependencies (with linux workarounds) * changelog
* drive-by constStefan Boberg2023-06-121-1/+1
|
* Additional trace instrumentation (#312)Stefan Boberg2023-05-161-1/+27
| | | | | | | | | * added trace instrumentation to upstreamcache * added asio trace instrumentation * added trace annotations for project store * added trace annotations for BlockStore * added trace annotations for HttpClient * added trace annotations for CAS/GC
* Add `--gc-projectstore-duration-seconds` option (#281)Dan Engelbrecht2023-05-161-2/+2
| | | | | | * Add `--gc-projectstore-duration-seconds` option * Cleanup lua gc options parsing * Remove dead configuration values * changelog
* added ScrubStorage to GcStorage base classStefan Boberg2023-05-151-0/+8
|
* minor GC API cleanupStefan Boberg2023-05-151-1/+1
| | | | | Scrub -> ScrubStorage Trigger -> TriggerGc (to make relationship to TriggerScrub clearer)
* Remove ATL header usage (#306)Stefan Boberg2023-05-151-4/+4
| | | ATL has been used here and there as a convenience. Given that this is a legacy component and not always something which gets installed along with the compiler we hereby remove the dependency altogether in favour of our own simple wrappers
* flush file cas on exit (#291)Dan Engelbrecht2023-05-111-22/+31
| | | | | | | | * flush caslog and index snapshot on flush * fix save reading of index/logfile with cleanup write snapshot at flush * don't validate entries we just scanned/created * fix total size found when scanning for cas files * changelog
* moved source directories into `/src` (#264)Stefan Boberg2023-05-021-0/+1452
* moved source directories into `/src` * updated bundle.lua for new `src` path * moved some docs, icon * removed old test trees