aboutsummaryrefslogtreecommitdiff
path: root/src/zenstore/gc.cpp
Commit message (Collapse)AuthorAgeFilesLines
* added missing includes (#504)Stefan Boberg2023-10-271-0/+1
| | | | | this change adds some includes to files which "inherit" includes from elsewhere this was exposed on another branch when removing some heavy dependencies from central headers
* fix m_LastFullGcDuration, m_LastFullGCDiff, m_LastFullGcDuration and ↵Dan Engelbrecht2023-10-231-18/+13
| | | | m_LastLightweightGcDuration stats (#494)
* clean up GcContributor and GcStorage to be pure interfaces (#485)Dan Engelbrecht2023-10-201-24/+0
|
* Add --skip-delete option to gc command (#484)Dan Engelbrecht2023-10-201-0/+4
| | | | - Feature: Add `--skip-delete` option to gc command - Bugfix: Fix implementation when claiming GC reserve during GC
* add `flush` command and more gc status info (#483)Dan Engelbrecht2023-10-181-23/+89
| | | | | | - Feature: New endpoint `/admin/flush ` to flush all storage - CAS, Cache and ProjectStore - Feature: New command `zen flush` to flush all storage - CAS, Cache and ProjectStore - Improved: Command `zen gc-status` now gives details about storage, when last GC occured, how long until next GC etc - Changed: Cache access and write log are disabled by default
* skip lightweight GC if full GC is due soon (#467)Stefan Boberg2023-10-121-20/+30
| | | | | | GC will now skip a lightweight GC if a full GC is due to run within the next lightweight GC interval also fixed some minor typos
* fixed GC logging output stats (#458)Stefan Boberg2023-10-101-1/+1
| | | disk usage stats are now properly reported in log messages
* fix gc infinite loop (#453)Dan Engelbrecht2023-10-061-1/+9
| | | | * make sure we update last gc time even if gc fails * If we can't check if an oplog/project markerfile exists, assume it is not expired
* trivial: log output typo in GCStefan Boberg2023-10-051-1/+1
|
* clean up date formatting (#440)Stefan Boberg2023-10-021-4/+4
| | | | * clean up date formatting (previous code would include a newline)
* fix formatting of gc start messagev0.2.26-pre0Dan Engelbrecht2023-10-021-1/+1
|
* Handle OOM and OOD more gracefully to not spam Sentry with error reports (#434)Dan Engelbrecht2023-10-021-1/+39
| | | | | | - Improvement: Catch Out Of Memory and Out Of Disk exceptions and report back to reqeuster without reporting an error to Sentry - Improvement: If creating bucket fails when storing and item in the structured cache, log a warning and propagate error to requester without reporting an error to Sentry - Improvement: Make an explicit flush of the active block written to in blockstore flush - Improvement: Make sure cache and cas MakeIndexSnapshot does not throw exception on failure which would cause and abnormal termniation at exit
* lightweight gc (#431)Dan Engelbrecht2023-10-021-47/+141
| | | | | | - Feature: Add lightweight GC that only removes items from cache/project store without cleaning up data referenced in Cid store - Add `skipcid` parameter to http endpoint `admin/gc`, defaults to "false" - Add `--skipcid` option to `zen gc` command, defaults to false - Add `--gc-lightweight-interval-seconds` option to zenserver
* add more trace scopes (#362)Dan Engelbrecht2023-09-151-0/+5
| | | | | * more trace scopes * Make sure ReplayLogEntries uses the correct size for oplog buffer * changelog
* retry file create (#383)Dan Engelbrecht2023-09-041-1/+4
| | | | | * add retry logic when creating files * only write disk usage log if disk writes are allowed * changelog
* safer gc on low disk (#373)Dan Engelbrecht2023-08-221-36/+45
| | | * - Improvement: Make sure we have disk space available to do GC and use reserve up front if need be
* catch exceptions when scheduling GC and when writing GC scheduling state (#339)Dan Engelbrecht2023-08-011-136/+149
| | | * catch exceptions when scheduling GC and when writing GC scheduling state
* Content scrubbing (#271)Stefan Boberg2023-05-161-7/+135
| | | Added zen scrub command which may be triggered via the zen CLI helper. This traverses storage and validates contents either by content hash and/or by structure. If unexpected data is encountered it is invalidated.
* Additional trace instrumentation (#312)Stefan Boberg2023-05-161-0/+5
| | | | | | | | | * added trace instrumentation to upstreamcache * added asio trace instrumentation * added trace annotations for project store * added trace annotations for BlockStore * added trace annotations for HttpClient * added trace annotations for CAS/GC
* Add `--gc-projectstore-duration-seconds` option (#281)Dan Engelbrecht2023-05-161-26/+51
| | | | | | * Add `--gc-projectstore-duration-seconds` option * Cleanup lua gc options parsing * Remove dead configuration values * changelog
* removed remnants of ZEN_USE_REF_TRACKINGStefan Boberg2023-05-151-20/+0
| | | | this code was originally meant to be used for GC but is no longer needed
* minor GC API cleanupStefan Boberg2023-05-151-5/+5
| | | | | Scrub -> ScrubStorage Trigger -> TriggerGc (to make relationship to TriggerScrub clearer)
* make sure we create gc root directory before checking disk spaceDan Engelbrecht2023-05-101-7/+7
|
* Low disk space detector (#277)Dan Engelbrecht2023-05-091-3/+45
| | | | * - Feature: Disk writes are now blocked early and return an insufficient storage error if free disk space falls below the `--low-diskspace-threshold` value * Never keep an entry in m_ChunkBlocks that points to a nullptr
* moved source directories into `/src` (#264)Stefan Boberg2023-05-021-0/+1312
* moved source directories into `/src` * updated bundle.lua for new `src` path * moved some docs, icon * removed old test trees