aboutsummaryrefslogtreecommitdiff
path: root/src/zenserver
Commit message (Collapse)AuthorAgeFilesLines
* per namespace/project cas prep refactor (#470)Dan Engelbrecht2025-08-2012-344/+538
| | | | | | | - Refactor so we can have more than one cas store for project store and cache. - Refactor `UpstreamCacheClient` so it is not tied to a specific CidStore - Refactor scrub to keep the GC interface ScrubStorage function separate from scrub accessor functions (renamed to Scrub). - Refactor storage size to keep GC interface StorageSize function separate from size accessor functions (renamed to TotalSize) - Refactor cache storage so `ZenCacheDiskLayer::CacheBucket` implements GcStorage interface rather than `ZenCacheNamespace`
* zen print fixes/improvements (#469)Dan Engelbrecht2025-08-193-6/+5
| | | | | - Improvement: `zen print` now allows output of compact binary content even if they are in non-optimal format (Unifom vs Non-Uniform arrays and objects) - Feature: `zen print` now has a `--show-type-info` option to add type information to output of compact binary content - Bugfix: Stats information for Build Store (Zen Store Cache) no longer throws exception and outputs invalid state information
* Ensure that FinalizeContainer finalizes both part and buildzousar2025-08-141-0/+31
|
* New frontend zip.Florent Devillechabrol2025-08-141-0/+0
|
* If the oplog has no packagestoreentry then show the raw json.Florent Devillechabrol2025-08-141-2/+5
|
* New frontend zip.Florent Devillechabrol2025-08-141-0/+0
|
* Do not skip oplog without package data.Florent Devillechabrol2025-08-142-36/+39
|
* add limitoverwrites option per bucket (#466)Dan Engelbrecht2025-08-123-11/+17
| | | | | | | | | | | | | | | - Feature: Added global zenserver option `--cache-bucket-limit-overwrites` controlling Whether to require policy flag pattern before allowing overwrites or not. Default `false` = overwrites always allowed - Feature: Add per bucket cache configuration option `limitoverwrites` (Lua options file only) cache = { bucket = { -- This is the default for all namespaces limitoverwrites = true }, buckets = { -- Here you can add matching per bucket name (matches accross namespaces) iostorecompression = { limitoverwrites = false }, }, }
* use new builds api for oplogs (#464)Dan Engelbrecht2025-08-124-309/+295
| | | | - Improvement: Refactored jupiter oplog export code to reuse builds jupiter wrapper classes - Improvement: If `zen builds`, `zen oplog-import` or `zen oplog-import` command fails due to a http error, the return code for the program will be set to the error/status code
* list build part content (#462)Dan Engelbrecht2025-08-115-8/+10
| | | | | | | | | | | - Feature: Added `zen build ls` option to list the content of a build part(s) - Build source is specified using one of the following options - `--cloud-url` cloud artifact URL to build - `--host` or `--override-host`, `--namespace`, `--bucket` and `--buildid` - `--filestorage`, `--namespace`, `--bucket` and `--buildid` - `--build-part-name` to specify a particular build part(s) in the build - `--wildcard` windows style wildcard (using * and ?) to match file paths to include - `--exclude-wildcard` windows style wildcard (using * and ?) to match file paths to exclude. Applied after --wildcard include filter - Improvement: Added `--quiet` option to zen `builds` commands to suppress non-essential output
* Merge branch 'main' into zs/put-overwrite-policyZousar Shaker2025-08-083-20/+29
|\
| * refactor blobstore (#458)Dan Engelbrecht2025-08-063-20/+29
| | | | | | | | | | | | - Improvement: Refactored build store cache to use existing CidStore implementation instead of implementation specific blob storage - **CAUTION** This will clear any existing cache when updating as the manifest version and storage strategy has changed - Bugfix: BuildStorage cache return "true" for metadata existance for all blobs that had payloads regardless of actual existance for metadata
* | Merge branch 'main' into zs/put-overwrite-policyzousar2025-08-051-6/+10
|\|
| * Set ready event when reusing an existing server instance. (#448)Martin Ridgers2025-08-041-6/+10
| | | | | | | | | | * Moved firing of child-id event into a lambda * Fire notify event to parent after attaching to an existing instance * Updated changelog
* | Merge branch 'main' into zs/put-overwrite-policyzousar2025-06-2447-1252/+2048
|\|
| * Surfaced basic z$ information to self-hosted dashboard (#441)Martin Ridgers2025-06-183-0/+124
| | | | | | | | | | | | - Namespaces are listed on the start page. - Namespaces can be dropped. - New page to show details of a namespace and list its buckets. - Buckets can be dropped.
| * graceful wait in parallelwork destructor (#438)Dan Engelbrecht2025-06-162-68/+114
| | | | | | | | | | * exception safety when issuing ParallelWork * add asserts to Latch usage to catch usage errors * extended error messaging and recovery handling in ParallelWork destructor to help find issues
| * fix build store range check (#437)Dan Engelbrecht2025-06-161-1/+1
| | | | | | | | | | * fix range check for blob store fetch * don't try to parse blockdesriptions if empty result is returned * add range to log when fetching blob range fails
| * sentry config (#430)Dan Engelbrecht2025-06-124-37/+78
| | | | | | | | | | | | | | | | | | | | - Feature: Added `--sentry-environment` to `zen` and `zenserver` - Feature: Added `--sentry-debug` to `zen` and `zenserver` - Feature: Added environment variable parsing for the following options: - `UE_ZEN_SENTRY_ENABLED`: `--no-sentry` (inverted) - `UE_ZEN_SENTRY_DEBUG`: `--sentry-debug` - `UE_ZEN_SENTRY_ALLOWPERSONALINFO`: `--sentry-allow-personal-info` - `UE_ZEN_SENTRY_DSN`: `--sentry-dsn` - `UE_ZEN_SENTRY_ENVIRONMENT`: `--sentry-environment`
| * revert 61b4a88f and cadaad63Dan Engelbrecht2025-06-104-78/+37
| |
| * add sentry configurations options for debug/environmentDan Engelbrecht2025-06-104-37/+78
| | | | | | | | | | add env-variable parsing for sentry option
| * `--sentry-dsn` option for zen command line and zenserver to control Sentry ↵Dan Engelbrecht2025-06-093-26/+33
| | | | | | | | | | reporting endpoint (#427) moved sentry database path to temporary directory for zen commandline
| * missing chunks bugfix (#424)Dan Engelbrecht2025-06-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | * make sure to close log file when resetting log * drop entries that refers to missing blocks * Don't scrub keys that has been rewritten * currectly count added bytes / m_TotalSize * fix negative sleep time in BlockStoreFile::Open() * be defensive when fetching log position * append to log files *after* we updated all state successfully * explicitly close stuff in destructors with exception catching * clean up empty size block store files
| * pause, resume and abort running builds cmd (#421)Dan Engelbrecht2025-06-052-6/+10
| | | | | | | | | | - Feature: `zen builds pause`, `zen builds resume` and `zen builds abort` commands to control a running `zen builds` command - `--process-id` the process id to control, if omitted it tries to find a running process using the same executable as itself - Improvement: Process report now indicates if it is pausing or aborting
| * faster oplog validate (#408)Dan Engelbrecht2025-05-301-9/+16
| | | | | | Improvement: Faster oplog validate to reduce GC wall time and disk I/O pressure
| * handle exception with batch work (#401)Dan Engelbrecht2025-05-192-31/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * use ParallelWork in rpc playback * use ParallelWork in projectstore * use ParallelWork in buildstore * use ParallelWork in cachedisklayer * use ParallelWork in compactcas * use ParallelWork in filecas * don't set abort flag in ParallelWork destructor * add PrepareFileForScatteredWrite for temp files in httpclient * Use PrepareFileForScatteredWrite when stream-decompressing files * be more relaxed when deleting temp files * allow explicit zen-cache when using direct host url without resolving * fix lambda capture when writing loose chunks * no delay when attempting to remove temp files
| * keep snapshot on log delete fail (#391)Dan Engelbrecht2025-05-121-54/+7
| | | | | | | | | | - Improvement: Cleaned up snapshot writing for CompactCAS/FileCas/Cache/Project stores - Improvement: Safer recovery when failing to delete log for CompactCAS/FileCas/Cache/Project stores - Improvement: Added log file reset when writing snapshot at startup for FileCas
| * enable per bucket config (#388)Dan Engelbrecht2025-05-123-14/+186
| | | | | | | | Feature: Add per bucket cache configuration (Lua options file only) Improvement: --cache-memlayer-sizethreshold is now deprecated and has a new name: --cache-bucket-memlayer-sizethreshold to line up with per cache bucket configuration
| * Change plugin config parsing to warn instead of throwzousar2025-05-071-8/+8
| |
| * optimize block store CompactBlocks (#384)Dan Engelbrecht2025-05-071-16/+8
| | | | | | | | | | - Improvement: Optimize block compact reducing memcpy operations - Improvement: Handle padding of block store blocks when compacting to avoid excessive flusing of write buffer - Improvement: Handle padding when writing oplog index snapshot to avoid unnecessary flushing of write buffer
| * add sentry for zen command (#373)Dan Engelbrecht2025-05-065-392/+2
| | | | | | | | * refactor sentry integration and add to zen command line tool * move add_ldflags("-framework Security")
| * builds allow redirect option (#379)Dan Engelbrecht2025-05-053-40/+31
| | | | | | * add --allow-redirect to zen builds upload/download
| * iterate chunks crash fix (#376)Dan Engelbrecht2025-05-021-1/+1
| | | | | | * Bugfix: Add explicit lambda capture in CasContainer::IterateChunks to avoid accessing state data references
| * fix buildstore disksizelimit lua config name (#372)Dan Engelbrecht2025-04-241-1/+1
| |
| * Added config, versioning and logging for pluginsDmytro Ivanov2025-04-222-0/+73
| |
| * add cxxopts overload for parsing file paths from command line (#362)Dan Engelbrecht2025-04-223-54/+12
| |
| * xmake updatefrontendzousar2025-04-161-0/+0
| |
| * Make metadata presentation more genericzousar2025-04-151-23/+42
| |
| * xmake updatefrontendzousar2025-04-151-0/+0
| |
| * Fix for BigInt conversion bugzousar2025-04-151-1/+1
| |
| * xmake updatefrontendzousar2025-04-111-0/+0
| |
| * Avoid signed overflow using BigIntzousar2025-04-115-13/+16
| | | | | | | | Bias for use of BigInt when consuming integer fields in compact binary to avoid values showing up as negative due to overflow on the Number type.
| * xmake updatefrontendzousar2025-04-041-0/+0
| |
| * Merge pull request #343 from ue-foundation/zs/web-ui-oplog-searchZousar Shaker2025-04-042-3/+7
| |\ | | | | | | Oplog search improvements
| | * Merge branch 'main' into zs/web-ui-oplog-searchZousar Shaker2025-04-044-4/+47
| | |\
| | * | Oplog search improvementszousar2025-04-032-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | - Case insensitive search - Allow search of 1 or 2 character strings - Reset table when doing a null search
| * | | Merge branch 'main' into zs/web-ui-blank-import-name-fixZousar Shaker2025-04-044-4/+47
| |\ \ \ | | | |/ | | |/|
| | * | blobstore size limit (#342)Dan Engelbrecht2025-04-044-4/+47
| | |/ | | | | | | - Feature: zenserver option `--buildstore-disksizelimit` to set an soft upper limit for build storage data. Defaults to 1TB.
| * | Alternate fix by explicitly initializing pkg_idzousar2025-04-041-2/+1
| | |
| * | Bump db versionzousar2025-04-031-1/+1
| | | | | | | | | | | | Required to refresh db contents after ID fix.
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-11 14:21:45 +0000