aboutsummaryrefslogtreecommitdiff
path: root/src/zen/cmds/projectstore_cmd.h
Commit message (Collapse)AuthorAgeFilesLines
* zen CLI: project-* commands → 'project <sub>' subcommands (#1026)Stefan Boberg2026-05-041-61/+144
| | | | | - Refactors the five `project-*` top-level commands into a `project <sub>` subcommand structure, mirroring the existing `cache <sub>` pattern. New surface: `project create | drop | info | op-details | stats`. - Legacy `project-create`, `project-drop`, `project-info`, `project-op-details`, `project-stats` remain functional as hidden deprecated shims that forward through `project_legacy_shim::RunAs`, so existing scripts (e.g. `scripts/test_scripts/oplog-import-export-test.py`) keep working unchanged.
* Oplog commands -> oplog subcommands (#1025)Stefan Boberg2026-05-041-137/+253
| | | | | - Consolidates the seven `oplog-*` top-level commands into a single `zen oplog <sub>` command tree, mirroring the cache refactor and PR #1026's `project <sub>` work. New surface: `oplog create | export | import | snapshot | mirror | validate | download`. - Legacy `oplog-create`, `oplog-export`, `oplog-import`, `oplog-snapshot`, `oplog-mirror`, `oplog-validate`, `oplog-download` remain functional as hidden deprecated aliases that forward through `oplog_legacy_shim::RunAs`, so existing scripts keep working.
* Update description of oplog snapshot (#964)Matt Peters2026-04-151-1/+1
| | | Update the description of oplog-snapshot to be more descriptive.
* use multi range requests (#800)Dan Engelbrecht2026-03-031-1/+1
| | | | | | | - Improvement: `zen builds download` now uses multi-range requests for blocks to reduce download size - Improvement: `zen oplog-import` now uses partial block with multi-range requests for blocks to reduce download size - Improvement: Improved feedback in log/console during `zen oplog-import` - Improvement: `--allow-partial-block-requests` now defaults to `true` for `zen builds download` and `zen oplog-import` (was `mixed`) - Improvement: Improved range merging analysis when downloading partial blocks
* use partial blocks for oplog import (#780)Dan Engelbrecht2026-02-241-0/+2
| | | | | Feature: Add --allow-partial-block-requests to zen oplog-import Improvement: zen oplog-import now uses partial block requests to reduce download size Improvement: Use latency to Cloud Storage host and Zen Cache host when calculating partial block requests
* changed command names and descriptions to use class members instead of ↵Stefan Boberg2026-02-231-14/+44
| | | | string literals in zen.cpp (#776)
* add boost-worker oplog import export options (#693)Dan Engelbrecht2025-12-161-3/+12
| | | | | | | | | | - Feature: `zen oplog-export`, `zen oplog-import` and `zen oplog-download` now has options to boost workers - `--boost-worker-count` - Increase the number of worker threads - may cause computer to be less responsive - `--boost-worker-memory` - Increase the limit where we write downloaded data to temporary storage to conserve space - may cause computer to be less responsive due to high memory usage - `--boost-workers` - Enables both 'boost-worker-count' and 'boost-worker-memory' - may cause computer to be less responsive - Improvement: Refactored boost options for `zen builds` operations `upload`, `download`, `diff`, `prime-cache`, `fetch-blob` and `validate-part` - `--boost-worker-count` - Increase the number of worker threads - may cause computer to be less responsive - `--boost-worker-memory` - Increase the limit where we write downloaded data to temporary storage to conserve space - may cause computer to be less responsive due to high memory usage - `--boost-workers` - Enables both 'boost-worker-count' and 'boost-worker-memory' - may cause computer to be less responsive
* upload to zen cache from oplog import now disabled by default (#680)Dan Engelbrecht2025-12-091-1/+1
|
* control cache upload (#646)Dan Engelbrecht2025-11-121-0/+2
| | | * add option to enable/disable upload to builds cache
* get oplog attachments (#622)Dan Engelbrecht2025-11-071-3/+13
| | | * add support for downloading individual attachments from an oplog
* add host discovery and zen cache support for oplog import (#601)Dan Engelbrecht2025-10-231-1/+3
| | | * add host discovery and zen cache support for oplog import
* limit number of chunks per block (#508)Dan Engelbrecht2025-09-231-2/+1
| | | | | - Improvement: Chunk block generation for `zen oplog-export` and `zen builds upload` command are now limited to max 4000 chunks per block zen builds list-block testing function to check results for recently used blocks in a context
* fetch cloud oplog (#502)Dan Engelbrecht2025-09-221-0/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Feature: Added `zen oplog-download` command to download the oplog body of a cooked output stored in Cloud DDC - Oplog source is specified using one of the following options - `--cloud-url` Cloud artifact URL for oplog - `--host` Base host to resolve download host from - `--override-host` Specific host to use without resolve - `--assume-http2` assume that the builds endpoint is a HTTP/2 endpoint skipping HTTP/1.1 upgrade handshake - `--namespace` Builds Storage namespace - `--bucket` Builds Storage bucket - `--build-id` an Oid in hex form for the source identifier to use - `--yes` suppress conformation query when doing output of a very large oplog to console - `--quiet` suppress all non-essential console output - `--output-path` path to oplog output, extension .json or .cb (compact binary). Default is output to console - `--system-dir` override default system root path - Authentication options - Auth token - `--access-token` http auth Cloud Storage access token - `--access-token-env` name of environment variable that holds the Http auth Cloud Storage access token - `--access-token-path` path to json file that holds the Http auth Cloud Storage access token - `--oidctoken-exe-path` path to OidcToken executable - OpenId authentication - `--openid-provider-name` Open ID provider name - `--openid-provider-url` Open ID provider url - `--openid-client-id`Open ID client id - `--openid-refresh-token` Open ID refresh token - `--encryption-aes-key` 256 bit AES encryption key for storing OpenID credentials - `--encryption-aes-iv` 128 bit AES encryption initialization vector for storing OpenID credentials - OAuth authentication - `--oauth-url` OAuth provier url - `--oauth-clientid` OAuth client id - `--oauth-clientsecret` OAuth client secret - Bugfix: `zen print` command now properly outputs very large compact binary objects as json to console
* refactor zen command return value handling (#487)Dan Engelbrecht2025-09-051-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Improvement: Use consistent language for command line argument parsing errors - Improvement: Changed zen command parsing errors to output help first and error last to make it easier to spot the error - Improvement: Refactor zen command return codes to conform to valid Linux range (0-255) kSuccess = 0, kOtherError = 1, kBadInput = 2, kOutOfMemory = 16, kOutOfDisk = 17, kAssertError = 70, kHttpOtherClientError = 80, kHttpCantConnectError = 81, kHttpNotFound = 66, // NotFound(404) kHttpUnauthorized = 77, // Unauthorized(401), kHttpSLLError = 82, kHttpForbidden = 83, // Forbidden(403) kHttpTimeout = 84, // RequestTimeout(408) kHttpConflict = 85, // Conflict(409) kHttpNoHost = 86, kHttpOtherServerError = 90, kHttpInternalServerError = 91, // InternalServerError(500) kHttpServiceUnavailable = 69, // ServiceUnavailable(503) kHttpBadGateway = 92, // BadGateway(502) kHttpGatewayTimeout = 93, // GatewayTimeout(504)
* `zen oplog-export`, `zen oplog-import` for `--url` (cloud) and `--builds` ↵Dan Engelbrecht2025-04-031-0/+2
| | | | (builds) option now has `--oidctoken-exe-path` to let zen run the OidcToken executable to get and refresh authentication token (#340)
* Builds API remote project store (#258)Dan Engelbrecht2024-12-121-16/+27
| | | Feature: zen command oplog-export and oplog-import now supports --builds remote target using the Jupiter builds API
* oplog prep gc fix (#216)Dan Engelbrecht2024-11-151-0/+15
| | | | | | - Added option gc-validation to zenserver that does a check for missing references in all oplog post full GC. Enabled by default. - Feature: Added option gc-validation to zen gc command to control reference validation. Enabled by default. - Added more details in post GC log. - Fixed race condition in oplog writes which could cause used attachments to be incorrectly removed by GC
* project details cmd resolve prj oplog (#209)Dan Engelbrecht2024-11-061-4/+4
| | | | | * auto-resolve project name if only one project exist * always try to resolve project and oplog name in project-details command * project-details -> project-op-details
* Improved oplog import/export progress indicator at commandline (#206)Dan Engelbrecht2024-11-061-0/+2
| | | | Nicer progress bar during oplog import/export Verify that oplog has not been deleted from disk behind our back
* Add ability to read the oplog's ReferencedSet, as written by the cook… (#190)v5.5.9-pre7Matt Peters2024-10-111-1/+2
| | | | Add ability to read the oplog's ReferencedSet, as written by the cooker, from the ReferencedSet op. Filter oplog entries requests by the ReferencedSet, if trim_by_referencedset parameter is present.. Add -trim=true/false parameter to oplog-mirror command, default to true, to request the trimmed/not trimmed oplog. Helper functions: Add paging to IterateOpLogWithKey. Add unit tests for IterateOpLog functions. Move OpKeyStringAsOid from httpprojectstore into projectstore.
* Mrp/referencedset (#174)Matt Peters2024-10-041-5/+5
| | | | * zen.exe commands: Add initializers for all POD types; this fixes incorrect behavior for flags in debug * README.md: Fix typo in --no-dry-run line
* add automatic decompression to cache-get (default on) and oplog-mirror ↵Dan Engelbrecht2024-10-041-0/+1
| | | | (default off) (#186)
* oplog cmd improvements (#152)Dan Engelbrecht2024-09-111-31/+4
| | | | | | - Improvement: Removed redundant commands `project-delete` and `oplog-delete`. Use already existing `project-drop` instead. - Improvement: zen oplog commands `project-drop`, `project-info`, `oplog-create`, `oplog-import`, `oplog-mirror` can now help resolve partial project and oplog identifiers - Improvement: zen `oplog-mirror` command now has new filter options to control which files are realized to disk: `--key` for op key, `--file` for file path matching and `--chunk` for chunk id matching - Improvement: `project-drop` command defaults to `--dry-run=true` and will only delete the target if `--dry-run=false` is added to the command line to avoid accidental delete
* Add optional gcpath param to import-oplog cmdzousar2024-04-041-0/+1
|
* Categorize the clean argument on oplog-import as a general argzousar2024-04-041-1/+1
|
* read jupiter token from file (#10)Dan Engelbrecht2024-03-181-0/+2
| | | | | * Add ability to specify a json file for cloud access token OidcToken.exe in UE can be asked to produce such a file with the -OutFile option * avoid division by zero when reporting progress
* add ignore-missing-attachments option to oplog export (debugging tool) (#641)Dan Engelbrecht2024-01-251-2/+3
| | | | | | | * add ignore-missing-attachments option to oplog export (debugging tool) * add more status codes to do retry for in http client * add missing X-Jupiter-IoHash header for jupiter PutRef * reduce oplog block size to reduce amount of redundant chunks to download * improved logging
* add --ignore-missing-attachments to oplog-import command (#637)Dan Engelbrecht2024-01-221-4/+5
|
* zen run command (#552)Stefan Boberg2023-11-211-12/+17
| | | | | | | | | | initial version -- this is primarily intended to be used for running stress tests and/or benchmarks example usage: `zen run -n 10 -- zenserver-test` `zen run -n 10 -- zenserver-test --ts=core.assert` run zenserver-test 10 times (testing only the `core.assert` test suite) `zen run --time 600 --basepath=d:\test_dir\test1 -- zenserver-test` keeps spawning new instances for 10 minutes (600 seconds)
* Collect all zen admin-related commands into admin.h/.cpp (#418)Dan Engelbrecht2023-09-221-0/+256
* move commands in scrub.h/cpp to admin_cmd.h/cpp * move job command into admin_cmd.h/.cpp * admin -> admin_cmd * bench -> bench_cmd * cache -> cache_cmd * copy -> copy_cmd * dedup -> dedup_cmd * hash -> hash_cmd * print -> print_cmd * projectstore -> projectstore_cmd * rpcreplay -> rpcreplay_cmd * serve -> serve_cmd * status -> status_cmd * top -> top_cmd * trace -> trace_cmd * up -> up_cmd * version -> version_cmd
generated by cgit v1.2.3 (git 2.25.1) at 2026-06-12 09:12:57 +0000