aboutsummaryrefslogtreecommitdiff
path: root/src/zenremotestore/builds/buildstorageutil.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Change builds ls command to default to all parts (#1019)Zousar Shaker2026-04-291-20/+14
| | | For backwards compatibility, `builds ls` retains past behavior of listing all parts, but allow both `builds download` and `builds prime-cache` to use the new standard of only operating on the "default" part.
* improve zen folder consistency and creation/cleanup (#1007)Dan Engelbrecht2026-04-221-1/+1
| | | | | | | - Improvement: `zen builds` zen-folder handling is now consistent per subcommand - `list-namespaces`, `list`, `list-blocks`, `ls`: no local scratch folder is created; responses stay in memory - `upload`, `fetch-blob`, `prime-cache`, `validate-part`: default to `<cwd>/.zen` (no change) - `download`: default to `<local-path>/.zen` (no change) - Bugfix: `zen builds ls` no longer fails against cloud build storage (`--host`/`--url`) when `--storage-path` is not supplied
* builds download "default" part if nothing is specified (#994)Dan Engelbrecht2026-04-211-0/+75
|
* Zen CLI common server interface (#920)Stefan Boberg2026-04-201-0/+31
| | | | | | | | | | | | | | | | | | Introduces a common `ZenServiceClient` RAII wrapper for zen CLI commands that interact with a zenserver instance. CLI operations (admin, builds, cache, exec, hub, info, projectstore, trace, ui, version, vfs, workspaces) automatically register sessions so they become visible in the server's session list, and forward log output to the server's session log endpoint. All session HTTP I/O (announce, remove, log batches) runs on a single background worker thread, so CLI startup and shutdown never block on server availability. ### Key changes - **`ZenServiceClient`** — new RAII class that wraps host resolution, HTTP client creation, and session lifecycle (register on connect, remove on exit). Replaces ad-hoc boilerplate across all command files that talk to a server, including the new `trace` subcommands (`start`, `stop`, `status`). - **Async session I/O** — `SessionsServiceClient` now owns a single worker thread and command queue. `Announce()`, `Remove()`, and `UpdateMetadata()` enqueue commands and return immediately. The worker creates one `HttpClient` with a 5-second total timeout, bounding any individual request. Eliminates main-thread stalls when the server is unreachable. - **Session log forwarding** — `SessionLogSink` is a thin enqueuer that posts log messages to the same worker queue (no separate thread or HTTP client). Log levels are serialized as integers; the server-side ingest handles both string and integer formats for backwards compatibility, with bounds checking on integer values. - **Build & projectstore session registration** — Long-running `builds` and projectstore cache (oplog-download) connections register sessions too, making them visible alongside regular CLI command sessions. ### Cleanup - Extract `SetupCacheSession` helper on `StorageInstance` to reduce duplication. - Remove unused `HttpClient` reference in ui command.
* builds cmd refactor (#975)Dan Engelbrecht2026-04-201-238/+1420
| | | | | | | | | - Bugfix: `builds download` partial-block fetch decisions now account for build storage host latency - Bugfix: Transfer rate displays in `builds` commands now smooth correctly - Split `buildstorageoperations.cpp` (8.5k lines) into per-operation TUs: buildinspect, buildprimecache, buildstorageresolve, buildupdatefolder, builduploadfolder, buildvalidatebuildpart; stats moved to buildstoragestats.h. - FilteredRate extracted to zenutil. - BuildsCommand shared state consolidated into a BuildsConfiguration struct; subcommands inherit from BuildsSubCmdBase holding a `const BuildsConfiguration&` instead of a `BuildsCommand&`. - `ProgressBar` renamed to `ConsoleProgressBar`; mode enum (`ConsoleProgressMode`) lifted to namespace scope; `PushLogOperation`/`PopLogOperation`/`ForceLinebreak` promoted to virtuals on `ProgressBase`. - Free-function wrappers (`UploadFolder`, `DownloadFolder`, `ValidateBuildPart`) added around the existing operation classes so callers stop reimplementing setup + stats logging.
* log cleanup (#969)Dan Engelbrecht2026-04-171-2/+2
| | | | - Improvement: New `ZEN_SCOPED_LOG(Expr)` macro routes `ZEN_INFO`/`ZEN_WARN`/`ZEN_DEBUG` in the enclosing block through the given logger expression instead of the default - Improvement: `BuildContainer`, `SaveOplog`, and `LoadOplogContext` now take a caller-provided `LoggerRef` so diagnostic messages route through the caller's logger
* operationlogoutput refactor (#967)Dan Engelbrecht2026-04-171-27/+23
| | | - Improvement: Replaced `OperationLogOutput` with `ProgressBase` in `zenutil`; logging and progress reporting are now separate concerns. Operation classes receive a `LoggerRef` for logging and a `ProgressBase&` for progress bars
* add fallback for zencache multirange (#816)Dan Engelbrecht2026-03-091-41/+56
| | | | | | | | | | | * clean up BuildStorageResolveResult to allow capabilities * add check for multirange request capability * add MaxRangeCountPerRequest capabilities * project export tests * add InMemoryBuildStorageCache * progress and logging improvements * fix ElapsedSeconds calculations in fileremoteprojectstore.cpp * oplogs/builds test script
* use multi range requests (#800)Dan Engelbrecht2026-03-031-17/+0
| | | | | | | - Improvement: `zen builds download` now uses multi-range requests for blocks to reduce download size - Improvement: `zen oplog-import` now uses partial block with multi-range requests for blocks to reduce download size - Improvement: Improved feedback in log/console during `zen oplog-import` - Improvement: `--allow-partial-block-requests` now defaults to `true` for `zen builds download` and `zen oplog-import` (was `mixed`) - Improvement: Improved range merging analysis when downloading partial blocks
* use partial blocks for oplog import (#780)Dan Engelbrecht2026-02-241-5/+14
| | | | | Feature: Add --allow-partial-block-requests to zen oplog-import Improvement: zen oplog-import now uses partial block requests to reduce download size Improvement: Use latency to Cloud Storage host and Zen Cache host when calculating partial block requests
* structured output for builds ls (#709)Dan Engelbrecht2026-01-141-3/+12
| | | | * make ResolveBuildStore respect Verbose flag * add structured output to zen builds ls command
* remove direct console output from code that is used from service mode (#688)Dan Engelbrecht2025-12-111-17/+24
|
* Change curl defaults on MacOS (#645)Stefan Boberg2025-11-111-11/+75
| | | | | | | * changed curl config to match the default from vcpkg (i.e `CURL_CA_FALLBACK=ON`) * disables use of Secure Transport for Mac since it's deprecated * Also worked around an issue (with `CURL_CA_BUNDLE`) where cross compiling curl on Mac would not configure curl in the same way as when compiling natively. This meant builds would not download on ARM macs when the CI build machine architecture was x86. The workaround should be redundant if we upgrade to 8.17 and use Apple SecTrust for cert validation. This should happen soon. * Also added various verbose logging to facilitate trouble shooting
* remotestore op refactorings (#637)Dan Engelbrecht2025-11-061-0/+210
| | | | | | * broke out BuildLogOutput to separate file * refactored out GetBlockDescriptions * log progress improvements * refactorings to accomodate oplog download operations
* add host discovery and zen cache support for oplog import (#601)Dan Engelbrecht2025-10-231-0/+157
* add host discovery and zen cache support for oplog import
generated by cgit v1.2.3 (git 2.25.1) at 2026-06-12 16:36:14 +0000