| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
| |
- Improvement: Cleaned up snapshot writing for CompactCAS/FileCas/Cache/Project stores
- Improvement: Safer recovery when failing to delete log for CompactCAS/FileCas/Cache/Project stores
- Improvement: Added log file reset when writing snapshot at startup for FileCas
|
| |
|
|
|
|
| |
- Don't add RawSize and Size in ProjectStore::GetProjectFiles response if we can't get the payload
- Use validation of payload size/existance in all chunk fetch operations in file cas
- In project store oplog validate, make sure we can reach all the payloads
- Add threading to oplog validate request
|
| |
|
|
|
|
|
|
| |
GC (#224)
make sure we rewrite filecas entries if chunk size changes (due to compression changes)
hardening of move/write files in filecas
if we encounter a filecas entry with mismatching size (due to pre-existing bug) we validate the file and update the index
if we find a bad filecas file on disk we now attempt to remove it
|
| |
|
|
|
| |
* kill gc v1
* block use of gc v1 from zen command line
* warn and flip to gcv2 if --gc-v2=false is specified for zenserver
|
| |
|
|
|
| |
- Bugfix: Remove extra loop causing GetProjectFiles for project store to find all chunks once for each chunk found
- Bugfix: Don't capture ChunkIndex variable in CasImpl::IterateChunks by reference as it causes crash
- Improvement: Make FileCasStrategy::IterateChunks (optionally) multithreaded (improves GetProjectFiles performance)
|
| |
|
| |
- Improvement: Reworked GetChunkInfos in oplog store to reduce disk thrashing and improve performance
|
| |
|
|
|
| |
* reserve vectors in gcv2 upfront
* set max load factor for robin_map indexes to reduce memory usage
* set min load factor for robin_map indexes to allow them to shrink
|
| |
|
|
|
|
|
|
|
| |
this change aims to hide logging internals from client code, in order to make it easier to extend and take more control over the logging process in the future.
As a bonus side effect, the generated code is much tighter (net delta around 2.5% on the resulting executable which includes lots of thirdparty code) and should take less time to compile and link.
Client usage via macros is pretty much unchanged. The main exposure client code had to spdlog internals before was the use of custom loggers per subsystem, where it would be common to have `spdlog::logger` references to keep a reference to a logger within a class. This is now replaced by `zen::LoggerRef` which currently simply encapsulates an actual `spdlog::logger` instance, but this is intended to be an implementation detail which will change in the future.
The way the change works is that we now handle any formatting of log messages in the zencore logging subsystem instead of relying on `spdlog` to manage this. We use the `fmt` library to do the formatting which means the client usage is identical to using `spdlog`. The formatted message is then forwarded onto any sinks etc which are still implememted via `spdlog`.
|
| |
|
|
|
| |
- Feature: New parameter for endpoint `admin/gc` (GET) `details=true` which gives details stats on GC operation when using GC V2
- Feature: New options for zen command `gc-status`
- `--details` that enables the detailed output from the last GC operation when using GC V2
|
| |
|
| |
- Feature: New garbage collection implementation, still in evaluation mode. Enabled by `--gc-v2` command line option
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
- Improvement: Reduce time a cache bucket is locked for write when flushing/garbage collecting
- Change format for faster read/write and reduced size on disk
- Don't lock index while writing manifest to disk
- Skip garbage collect if we are currently in a Flush operation
- BlockStore::Flush no longer terminates currently writing block
- Garbage collect references to currently writing block but keep the block as new data may be added
- Fix BlockStore::Prune used disk space calculation
- Don't materialize data in filecas when we just need the size
|
| |
|
|
|
| |
* more trace scopes
* Make sure ReplayLogEntries uses the correct size for oplog buffer
* changelog
|
| | |
|
| |
|
|
|
| |
Scrub -> ScrubStorage
Trigger -> TriggerGc (to make relationship to TriggerScrub clearer)
|
| |
|
|
|
|
|
|
| |
* flush caslog and index snapshot on flush
* fix save reading of index/logfile with cleanup
write snapshot at flush
* don't validate entries we just scanned/created
* fix total size found when scanning for cas files
* changelog
|
|
|
* moved source directories into `/src`
* updated bundle.lua for new `src` path
* moved some docs, icon
* removed old test trees
|