| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| |
|
|
|
|
|
| |
- Refactor so we can have more than one cas store for project store and cache.
- Refactor `UpstreamCacheClient` so it is not tied to a specific CidStore
- Refactor scrub to keep the GC interface ScrubStorage function separate from scrub accessor functions (renamed to Scrub).
- Refactor storage size to keep GC interface StorageSize function separate from size accessor functions (renamed to TotalSize)
- Refactor cache storage so `ZenCacheDiskLayer::CacheBucket` implements GcStorage interface rather than `ZenCacheNamespace`
|
| |
|
|
|
| |
- Improvement: Cleaned up snapshot writing for CompactCAS/FileCas/Cache/Project stores
- Improvement: Safer recovery when failing to delete log for CompactCAS/FileCas/Cache/Project stores
- Improvement: Added log file reset when writing snapshot at startup for FileCas
|
| |
|
| |
* make sure we remove the cas log file when writing full index at startup
|
| |
|
|
|
|
|
|
|
| |
- **EXPERIMENTAL** `zen builds`
- Feature: `--zen-cache-host` option for `upload` and `download` operations to use a zenserver host `/builds` endpoint for storing build blob and blob metadata
- Feature: New `/builds` endpoint for caching build blobs and blob metadata
- `/builds/{namespace}/{bucket}/{buildid}/blobs/{hash}` `GET` and `PUT` method for storing and fetching blobs
- `/builds/{namespace}/{bucket}/{buildid}/blobs/putBlobMetadata` `POST` method for storing metadata about blobs
- `/builds/{namespace}/{bucket}/{buildid}/blobs/getBlobMetadata` `POST` method for fetching metadata about blobs
- `/builds/{namespace}/{bucket}/{buildid}/blobs/exists` `POST` method for checking existance of blobs
|
| |
|
|
|
|
| |
* Allow caller to control threshold for bulk-loading chunks in IterateChunks
* use smaller batch chunk reading for /fileinfos and /chunkinfos as we do not intend to read the payload
* use smaller batch read buffer when just querying for size of attachments
|
| |
|
|
|
| |
* kill gc v1
* block use of gc v1 from zen command line
* warn and flip to gcv2 if --gc-v2=false is specified for zenserver
|
| |
|
| |
- Improvement: Reworked GetChunkInfos in oplog store to reduce disk thrashing and improve performance
|
| |
|
| |
- Improvement: Add batching when writing multiple small chunks to block store - decreases I/O load significantly on oplog import
|
| |
|
| |
- Improvement: Reduce memory usage in GC and diskbucket flush
|
| |
|
|
|
|
|
|
|
| |
this change aims to hide logging internals from client code, in order to make it easier to extend and take more control over the logging process in the future.
As a bonus side effect, the generated code is much tighter (net delta around 2.5% on the resulting executable which includes lots of thirdparty code) and should take less time to compile and link.
Client usage via macros is pretty much unchanged. The main exposure client code had to spdlog internals before was the use of custom loggers per subsystem, where it would be common to have `spdlog::logger` references to keep a reference to a logger within a class. This is now replaced by `zen::LoggerRef` which currently simply encapsulates an actual `spdlog::logger` instance, but this is intended to be an implementation detail which will change in the future.
The way the change works is that we now handle any formatting of log messages in the zencore logging subsystem instead of relying on `spdlog` to manage this. We use the `fmt` library to do the formatting which means the client usage is identical to using `spdlog`. The formatted message is then forwarded onto any sinks etc which are still implememted via `spdlog`.
|
| |
|
|
|
| |
- Feature: New parameter for endpoint `admin/gc` (GET) `details=true` which gives details stats on GC operation when using GC V2
- Feature: New options for zen command `gc-status`
- `--details` that enables the detailed output from the last GC operation when using GC V2
|
| |
|
| |
- Feature: New garbage collection implementation, still in evaluation mode. Enabled by `--gc-v2` command line option
|
| | |
|
| |
|
|
|
| |
- Bugfix: Fix scrub messing up payload and access time in disk cache bucket when compacting index
- Improvement: Split up disk cache bucket index into hash lookup and payload array to improve performance
- Improvement: Reserve space up front for compact binary output when saving cache bucket manifest to improve performance
|
| |
|
|
|
| |
* Use robin-map in compactcas for 30% faster CasContainerStrategy::CollectGarbage
* use robin_set in ProjectStore::Oplog::GatherReferences and BlockStore::ReclaimSpace
* changelog
|
| | |
|
| |
|
|
|
| |
Scrub -> ScrubStorage
Trigger -> TriggerGc (to make relationship to TriggerScrub clearer)
|
| | |
|
|
|
* moved source directories into `/src`
* updated bundle.lua for new `src` path
* moved some docs, icon
* removed old test trees
|