| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
| |
- Improvement: Multithreaded scavenge pass for zen builds download
- Improvement: Optimized check for modified files when verifying state of scavenged paths
|
| | |
|
| | |
|
| |
|
| |
- Bugfix: Long file paths now works correctly on Windows
|
| |
|
| |
Handling of quotes and quotes with leading backslash for command line parsing - UE-231677
|
| |
|
|
|
|
| |
add DirectoryContent::IncludeFileSizes
add DirectoryContent::IncludeAttributes
add multithreaded GetDirectoryContent
use multithreaded GetDirectoryContent in workspace folder scanning
|
| |
|
|
|
|
| |
move jupiter.h/cpp -> zenutil
move packageformat.h/.cpp -> zenhttp
zenutil now depends on zenhttp instead of the inverse
|
| |
|
|
|
|
| |
- Added option gc-validation to zenserver that does a check for missing references in all oplog post full GC. Enabled by default.
- Feature: Added option gc-validation to zen gc command to control reference validation. Enabled by default.
- Added more details in post GC log.
- Fixed race condition in oplog writes which could cause used attachments to be incorrectly removed by GC
|
| |
|
|
|
| |
* kill gc v1
* block use of gc v1 from zen command line
* warn and flip to gcv2 if --gc-v2=false is specified for zenserver
|
| |
|
| |
* zen command - add options to control meta data cache when triggering gc
|
| |
|
|
|
| |
Added option `gc-attachment-passes` to zenserver
Cleaned up GCv2 start and stop logs and added identifier to easily find matching start and end of a GC pass in log file
Fixed project store not properly sorting references found during lock phase
|
| |
|
|
|
| |
* optimize IoHash and OId comparisions
* refactor filtering of unused references
* add attachment filtering to gc
|
| |
|
| |
* add option to force gcv2 to run single threaded
|
| |
|
|
|
|
| |
command (#651)
* be safe and use ToText() rather than AsText() when getting http responses
* Added `--copy-log`, `--copy-cache-log` and `copy-http-log` option to zen logs command to copy logs from a local running zen server instance
|
| |
|
|
|
|
| |
- Improvement: Scrub command now validates compressed buffer hashes in filecas storage (used for large chunks)
- Improvement: Added --dry, --no-gc and --no-cas options to zen scrub command
- Improvement: Implemented oplog scrubbing (previously was a no-op)
- Improvement: Implemented support for running scrubbint at startup with --scrub=<options>
|
| |
|
|
|
|
| |
- Improvement: Added more trace scopes for GCv2
- Bugfix: Make sure we can override flags to "false" when running `zen gc` commmand
- `smallobjects`, `skipcid`, `skipdelete`, `verbose`
|
| |
|
|
|
| |
the previous implementation of in-memory index snapshots serialise data to memory before writing to disk and vice versa when reading. This leads to some memory spikes which end up pushing useful data out of system cache and also cause stalls on I/O operations.
this change moves more code to a streaming serialisation approach which scales better from a memory usage perspective and also performs much better
|
| |
|
|
|
| |
- Feature: New endpoint `/admin/gc-stop` to cancel a running garbage collect operation
- Feature: Added `zen gc-stop` command to cancel a running garbage collect operation
- Bugfix: GCv2 - make sure to discover all projects and oplogs before checking for expired data
|
| |
|
|
|
|
|
|
|
|
|
| |
- Feature: Added new options to zenserver for GC V2
- `--gc-compactblock-threshold` GCV2 - how much of a compact block should be used to skip compacting the block, default is 90%
- `--gc-verbose` GCV2 - enable more verbose output when running a GC pass
- Feature: Added new options to `zen gc` command for GC V2
- `--compactblockthreshold` GCV2 - how much of a compact block should be used to skip compacting the block, default is 90%
- `--verbose` GCV2 - enable more verbose output when running a GC pass
- Feature: Added new parameters for endpoint `admin/gc` (PUT)
- `compactblockthreshold` GCV2 - how much of a compact block should be used to skip compacting the block, default is 90%
- `verbose` GCV2 - enable more verbose output when running a GC pass
|
| |
|
|
|
| |
data (#510)
* zen copy-state command to copy a zenserver data directory without the bulk data
|
| |
|
|
|
| |
- Feature: New parameter for endpoint `admin/gc` (GET) `details=true` which gives details stats on GC operation when using GC V2
- Feature: New options for zen command `gc-status`
- `--details` that enables the detailed output from the last GC operation when using GC V2
|
| |
|
| |
- Feature: New garbage collection implementation, still in evaluation mode. Enabled by `--gc-v2` command line option
|
| |
|
|
| |
- Feature: Add `--skip-delete` option to gc command
- Bugfix: Fix implementation when claiming GC reserve during GC
|
| |
|
|
|
|
| |
- Feature: New endpoint `/admin/flush ` to flush all storage - CAS, Cache and ProjectStore
- Feature: New command `zen flush` to flush all storage - CAS, Cache and ProjectStore
- Improved: Command `zen gc-status` now gives details about storage, when last GC occured, how long until next GC etc
- Changed: Cache access and write log are disabled by default
|
| |
|
|
|
|
| |
- Feature: Add lightweight GC that only removes items from cache/project store without cleaning up data referenced in Cid store
- Add `skipcid` parameter to http endpoint `admin/gc`, defaults to "false"
- Add `--skipcid` option to `zen gc` command, defaults to false
- Add `--gc-lightweight-interval-seconds` option to zenserver
|
| |
|
|
|
|
|
|
|
|
|
| |
- Feature: New endpoint `/admin/logs` to query status of logging and log file locations and cache logging
- `enablewritelog`=`true`/`false` parameter to control cache write logging
- `enableaccesslog`=`true`/`false` parameter to control cache access logging
- `loglevel` = `trace`/`debug`/`info`/`warning`/`error`
- Feature: New zen command `logs` to query/control zen logging
- No arguments gives status of logging and paths to log files
- `--cache-write-log` `enable`/`disable` to control cache write logging
- `--cache-access-log` `enable`/`disable` to control cache access logging
- `--loglevel` `trace`/`debug`/`info`/`warning`/`error` to set debug level
|
|
|
* move commands in scrub.h/cpp to admin_cmd.h/cpp
* move job command into admin_cmd.h/.cpp
* admin -> admin_cmd
* bench -> bench_cmd
* cache -> cache_cmd
* copy -> copy_cmd
* dedup -> dedup_cmd
* hash -> hash_cmd
* print -> print_cmd
* projectstore -> projectstore_cmd
* rpcreplay -> rpcreplay_cmd
* serve -> serve_cmd
* status -> status_cmd
* top -> top_cmd
* trace -> trace_cmd
* up -> up_cmd
* version -> version_cmd
|