| Commit message (Collapse) | Author | Age | Files | Lines | |
|---|---|---|---|---|---|
| * | add cmake build config to be able to use cliondi/add_cmake_dev | Dmytro Ivanov | 2024-10-02 | 1 | -2/+2 |
| | | |||||
| * | gc command attachment options (#176) | Dan Engelbrecht | 2024-09-30 | 2 | -9/+8 |
| | | | | * zen command - add options to control meta data cache when triggering gc | ||||
| * | Add `gc-attachment-passes` option to zenserver (#167) | Dan Engelbrecht | 2024-09-25 | 1 | -6/+9 |
| | | | | | | Added option `gc-attachment-passes` to zenserver Cleaned up GCv2 start and stop logs and added identifier to easily find matching start and end of a GC pass in log file Fixed project store not properly sorting references found during lock phase | ||||
| * | gc unused refactor (#165) | Dan Engelbrecht | 2024-09-23 | 1 | -28/+21 |
| | | | | | | * optimize IoHash and OId comparisions * refactor filtering of unused references * add attachment filtering to gc | ||||
| * | End was clamped to the wrong side if Count was defaulted | Martin Ridgers | 2024-09-19 | 1 | -1/+1 |
| | | |||||
| * | Reinstate member initialisation that was there originally | Martin Ridgers | 2024-09-19 | 1 | -2/+2 |
| | | |||||
| * | Poorly cherry-pick | Martin Ridgers | 2024-09-19 | 1 | -2/+2 |
| | | |||||
| * | Surfaced oplog paging to HTTP handler | Martin Ridgers | 2024-09-19 | 1 | -3/+18 |
| | | |||||
| * | clang-format on branch's changed files | Martin Ridgers | 2024-09-19 | 2 | -106/+113 |
| | | |||||
| * | Mandatory IterateOplog() paging argument | Martin Ridgers | 2024-09-19 | 3 | -16/+13 |
| | | |||||
| * | Removed redundant initialisation | Martin Ridgers | 2024-09-19 | 1 | -2/+2 |
| | | |||||
| * | A clang-format pass | Martin Ridgers | 2024-09-19 | 1 | -3/+3 |
| | | |||||
| * | Appease Apple Clang's historical lack of std::span constructors | Martin Ridgers | 2024-09-19 | 1 | -6/+2 |
| | | |||||
| * | Forgot there was a std::clamp | Martin Ridgers | 2024-09-19 | 1 | -7/+4 |
| | | |||||
| * | Placate clang | Martin Ridgers | 2024-09-19 | 2 | -6/+6 |
| | | |||||
| * | Optional paged results when iterating oplog entries | Martin Ridgers | 2024-09-19 | 2 | -6/+28 |
| | | |||||
| * | fix oplog state check and wiping (#154) | Dan Engelbrecht | 2024-09-11 | 1 | -2/+17 |
| | | | | | | fix oplog state check fix wipe of oplog storage + meta only on invalid oplog handle vanishing oplog during gc | ||||
| * | validate oplog before opening - if invalid, warn and wipe oplog (#153) | Dan Engelbrecht | 2024-09-10 | 1 | -2/+12 |
| | | |||||
| * | cleanup oplog logging (#147) | Dan Engelbrecht | 2024-09-05 | 1 | -9/+29 |
| | | | | * clean up logging | ||||
| * | fix oplog index snapshot reading (#146) | Dan Engelbrecht | 2024-09-05 | 1 | -5/+29 |
| | | | | * fixed parsing and added extra validation of oplog index snapshot | ||||
| * | fix oplog instance leak in project store gc (#145) | Dan Engelbrecht | 2024-09-05 | 1 | -3/+7 |
| | | |||||
| * | move gc logs to gc logger (#142) | Dan Engelbrecht | 2024-09-04 | 1 | -10/+48 |
| | | | | - Improvement: Move GC logging in callback functions into "gc" context | ||||
| * | delay oplog read (#141) | Dan Engelbrecht | 2024-09-03 | 2 | -145/+213 |
| | | | | - Improvement: Don't keep all oplogs open after GC, close them when references are fetched unless they are open by client | ||||
| * | oplog index snapshots (#140) | Dan Engelbrecht | 2024-09-03 | 2 | -203/+732 |
| | | | | - Feature: Added project store oplog index snapshots for faster opening of oplog - opening oplogs are roughly 10x faster | ||||
| * | meta info store (#75) | Dan Engelbrecht | 2024-08-30 | 2 | -20/+138 |
| | | | | | - Feature: Added option `--gc-cache-attachment-store` which caches referenced attachments in cache records on disk for faster GC - default is `false` - Feature: Added option `--gc-projectstore-attachment-store` which caches referenced attachments in project store oplogs on disk for faster GC - default is `false` | ||||
| * | Make sure `noexcept` functions does not leak exceptions (#136) | Dan Engelbrecht | 2024-08-23 | 1 | -1/+1 |
| | | |||||
| * | separate worker pools into burst/background to avoid background jobs ↵ | Dan Engelbrecht | 2024-08-22 | 2 | -7/+7 |
| | | | | | blocking client requests (#134) | ||||
| * | demote project store access time read fail to warning (#127) | Dan Engelbrecht | 2024-08-20 | 1 | -1/+1 |
| | | |||||
| * | verify that project oplog dir exists before trying to iterate it (#123) | Dan Engelbrecht | 2024-08-19 | 1 | -5/+11 |
| | | |||||
| * | fix oplog compact size log | Dan Engelbrecht | 2024-08-15 | 1 | -1/+1 |
| | | |||||
| * | adds more information in log when oplog rename/delete fails (#117) | Dan Engelbrecht | 2024-08-15 | 1 | -6/+25 |
| | | | | | | | * adds more information in log when oplog rename/delete fails * warn if we fail to delete dropped directory * close oplog storage file inside try/catch * don't try to flush closed files | ||||
| * | add compacting of oplogs as part of GC (#106) | Dan Engelbrecht | 2024-08-12 | 3 | -122/+481 |
| | | | | | * add compacting of oplogs as part of GC * force retain of LSN unless we have less than 16 miln entries left | ||||
| * | project store chunk requests that are out of range will be treated as not ↵ | Dan Engelbrecht | 2024-08-12 | 1 | -0/+26 |
| | | | | | | | found (#108) * project store chunk requests that are out of range will be treaded as not found * add chunkid to log | ||||
| * | make oplog lsn unsigned (#107) | Dan Engelbrecht | 2024-08-12 | 3 | -45/+34 |
| | | | | * change oplog lsn to uint32 | ||||
| * | project/oplog delete improvements (#105) | Dan Engelbrecht | 2024-08-12 | 3 | -53/+110 |
| | | | | | | * make oplog/project folder removeal more robust * report back error to http caller if removal fails | ||||
| * | stop exceptions from leaking on threaded work (#102) | Dan Engelbrecht | 2024-08-06 | 2 | -39/+69 |
| | | | | | * catch exceptions in threaded work * don't abort all project file/chunk info fetch for single failure | ||||
| * | hardening read of corrupt oplog (#98) | Dan Engelbrecht | 2024-08-06 | 1 | -24/+50 |
| | | | | * Add extra validation of oplog entries when reading oplog | ||||
| * | Make sure we monitor for new project, oplogs, namespaces and buckets during ↵ | Dan Engelbrecht | 2024-06-13 | 2 | -117/+415 |
| | | | | | | | GCv2 (#93) - Bugfix: Make sure we monitor and include new project/oplogs created during GCv2 - Bugfix: Make sure we monitor and include new namespaces/cache buckets created during GCv2 | ||||
| * | Use a smaller thread pool for network operations when doing oplog import to ↵ | Dan Engelbrecht | 2024-05-30 | 1 | -1/+1 |
| | | | | | | reduce risk NIC/router failure (#89) Medium worker pool now uses a minimum of 2 threads (up from 1) | ||||
| * | cache optimizations (#88) | Dan Engelbrecht | 2024-05-30 | 1 | -4/+4 |
| | | | | | | * message formatting optimizations * bump iostorecompression small value threshold to 1MB | ||||
| * | fix iterate chunks crash (#86) | Dan Engelbrecht | 2024-05-27 | 1 | -0/+1 |
| | | | | * fix worklatch count in Oplog::IterateChunks | ||||
| * | fix zero size attachment replies (#69) | Dan Engelbrecht | 2024-05-02 | 2 | -3/+11 |
| | | | | | - Bugfix: Don't try to respond with zero size partial cache value when partial size is zero - Improvement: Added more validation of data read from cache / cas | ||||
| * | use write and move in place for safer writing of files (#70) | Dan Engelbrecht | 2024-05-02 | 1 | -3/+3 |
| | | | | * use write and move in place for safer writing of files | ||||
| * | fix get project files loop (#68) | Dan Engelbrecht | 2024-04-30 | 1 | -23/+20 |
| | | | | | | - Bugfix: Remove extra loop causing GetProjectFiles for project store to find all chunks once for each chunk found - Bugfix: Don't capture ChunkIndex variable in CasImpl::IterateChunks by reference as it causes crash - Improvement: Make FileCasStrategy::IterateChunks (optionally) multithreaded (improves GetProjectFiles performance) | ||||
| * | oplog iterate chunks content type (#65) | Dan Engelbrecht | 2024-04-26 | 3 | -51/+120 |
| | | | | | - Bugfix: Properly set content type of chunks fetch from CidStore - Improvement: Add IterateChunks(std::span<Oid>) for better performance in get oplog | ||||
| * | use direct file access for large file hash (#63) | Dan Engelbrecht | 2024-04-26 | 1 | -4/+20 |
| | | | | - Improvement: Refactor `IoHash::HashBuffer` and `BLAKE3::HashBuffer` to not use memory mapped files. Performs better and saves ~10% of oplog export time on CI | ||||
| * | fix oplog import during gcv2 (#62)v5.5.0-pre3v5.5.0-pre2 | Dan Engelbrecht | 2024-04-25 | 3 | -114/+138 |
| | | | | | - Bugfix: Always pre-cache oplog when creating project store GCv2 referencer - Bugfix: Correctly capture attachments imported with oplog to void them being GCd before oplog is written | ||||
| * | iterate cas chunks (#59) | Dan Engelbrecht | 2024-04-24 | 2 | -28/+30 |
| | | | | - Improvement: Reworked GetChunkInfos in oplog store to reduce disk thrashing and improve performance | ||||
| * | safer gcv2 on error (#60) | Dan Engelbrecht | 2024-04-24 | 1 | -4/+13 |
| | | | | - Bugfix: Harden GCv2 when errors occur and gracefully abort GC operation on error | ||||
| * | Bugfix: Only disable oplog update capture if we have started it (#58) | Dan Engelbrecht | 2024-04-24 | 1 | -1/+6 |
| | | |||||