aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
...
| * | | | | | fix build store range check (#437)Dan Engelbrecht2025-06-162-3/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * fix range check for blob store fetch * don't try to parse blockdesriptions if empty result is returned * add range to log when fetching blob range fails
| * | | | | | sentry config (#430)Dan Engelbrecht2025-06-129-56/+311
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Feature: Added `--sentry-environment` to `zen` and `zenserver` - Feature: Added `--sentry-debug` to `zen` and `zenserver` - Feature: Added environment variable parsing for the following options: - `UE_ZEN_SENTRY_ENABLED`: `--no-sentry` (inverted) - `UE_ZEN_SENTRY_DEBUG`: `--sentry-debug` - `UE_ZEN_SENTRY_ALLOWPERSONALINFO`: `--sentry-allow-personal-info` - `UE_ZEN_SENTRY_DSN`: `--sentry-dsn` - `UE_ZEN_SENTRY_ENVIRONMENT`: `--sentry-environment`
| * | | | | | make sure we finish progress at 100% if not done (#432)Dan Engelbrecht2025-06-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * make sure we finish progress at 100% if not done * fix condition
| * | | | | | crash in composite buffer stream (#431)Dan Engelbrecht2025-06-101-1/+1
| | | | | | | | | | | | | | | | | | | | | * fix BufferedReadFileStream calculating buffer end wrong
| * | | | | | revert 61b4a88f and cadaad63Dan Engelbrecht2025-06-109-311/+56
| | | | | | |
| * | | | | | add sentry configurations options for debug/environmentDan Engelbrecht2025-06-107-56/+135
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | add env-variable parsing for sentry option
| * | | | | | add EnvironmentOptions helperDan Engelbrecht2025-06-102-0/+176
| | | | | | |
| * | | | | | Don't require `--namespace` option when using `zen list-namespaces` command ↵Dan Engelbrecht2025-06-101-19/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | (#429)
| * | | | | | restore legacy --url option for builds download (#428)Dan Engelbrecht2025-06-101-1/+7
| | | | | | |
| * | | | | | `--sentry-dsn` option for zen command line and zenserver to control Sentry ↵Dan Engelbrecht2025-06-096-37/+59
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | reporting endpoint (#427) moved sentry database path to temporary directory for zen commandline
| * | | | | | use unique tmp name for auth token file (#426)Dan Engelbrecht2025-06-091-3/+3
| | | | | | | | | | | | | | | | | | | | | * use Oid to generate unique name since std::tmpnam is not good practice
| * | | | | | fix namespace picking up wrong argument (#425)Dan Engelbrecht2025-06-091-1/+1
| | | | | | |
| * | | | | | missing chunks bugfix (#424)Dan Engelbrecht2025-06-098-85/+480
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * make sure to close log file when resetting log * drop entries that refers to missing blocks * Don't scrub keys that has been rewritten * currectly count added bytes / m_TotalSize * fix negative sleep time in BlockStoreFile::Open() * be defensive when fetching log position * append to log files *after* we updated all state successfully * explicitly close stuff in destructors with exception catching * clean up empty size block store files
| * | | | | | silence oom ood errors for sentry (#423)Dan Engelbrecht2025-06-051-0/+18
| | | | | | | | | | | | | | | | | | | | | - Improvement: Don't report OOD or OOM errors to Sentry when running `zen builds` commands
| * | | | | | pause, resume and abort running builds cmd (#421)Dan Engelbrecht2025-06-0521-161/+799
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Feature: `zen builds pause`, `zen builds resume` and `zen builds abort` commands to control a running `zen builds` command - `--process-id` the process id to control, if omitted it tries to find a running process using the same executable as itself - Improvement: Process report now indicates if it is pausing or aborting
| * | | | | | revert system temp dir for builds upload (#422)Dan Engelbrecht2025-06-051-29/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * revert temp upload folder to be inside source directory to avoid filling up system disk * use selected subcommand when displaying help for failed command line options in zen builds
| * | | | | | Initialize sentry after command line parsing is done (#420)Stefan Boberg2025-06-041-24/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously the option variables were used before options parsing had a chance to set them according to command line options
| * | | | | | builds download url (#419)Dan Engelbrecht2025-06-044-64/+154
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * RemoveQuotes helper * `--url` option for `zen builds` command has been reworked to accept a "Cloud Artifact URL", removing the need to specify "host", "namespace" and "bucket" separately
| * | | | | | new builds search (#418)Dan Engelbrecht2025-06-042-19/+32
| | | | | | | | | | | | | | | | | | | | | * don't require bucket for search operations to allow multi-bucket search
| * | | | | | faster scavenge (#417)Dan Engelbrecht2025-06-044-101/+251
| | | | | | | | | | | | | | | | | | | | | | | | | | | | - Improvement: Multithreaded scavenge pass for zen builds download - Improvement: Optimized check for modified files when verifying state of scavenged paths
| * | | | | | fixed size chunking for encrypted files (#410)Dan Engelbrecht2025-06-0314-429/+873
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Improvement: Use fixed size block chunking for know encrypted/compressed file types - Improvement: Skip trying to compress chunks that are sourced from files that are known to be encrypted/compressed - Improvement: Add global open file cache for written files increasing throughput during download by reducing overhead of open/close of file by 80%
| * | | | | | minor: fix unused variable warning on some compilersStefan Boberg2025-06-031-4/+4
| | | | | | |
| * | | | | | http client streaming upload (#413)Dan Engelbrecht2025-06-025-200/+438
| | | | | | | | | | | | | | | | | | | | | - Improvement: Add streaming upload from HttpClient to reduce I/O caused by excessive MMap usage
| * | | | | | fix cachbucket mem hit count (#415)Dan Engelbrecht2025-06-021-4/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Don't count a miss twice for memory stats if the entry can't be found * changelog
| * | | | | | streaming none compressor (#414)Dan Engelbrecht2025-06-021-3/+36
| | | | | | | | | | | | | | | | | | | | | * add proper streaming to none compressor type
| * | | | | | use system temp dir (#412)Dan Engelbrecht2025-06-021-21/+24
| | | | | | | | | | | | | | | | | | | | | * use system temp dir when uploading builds
| * | | | | | add missing flush inblockstore compact (#411)Dan Engelbrecht2025-05-305-17/+130
| | | | | | | | | | | | | | | | | | | | | | | | | | | | - Bugfix: Flush the last block before closing the last new block written to during blockstore compact. UE-291196 - Feature: Drop unreachable CAS data during GC pass. UE-291196
| * | | | | | faster oplog validate (#408)Dan Engelbrecht2025-05-306-20/+86
| | | | | | | | | | | | | | | | | | | | | Improvement: Faster oplog validate to reduce GC wall time and disk I/O pressure
| * | | | | | frequent disk space check (#407)Dan Engelbrecht2025-05-272-26/+89
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * check low disk space condition more frequently and trigger GC when low water mark is reached * show waited time when waiting for zenserver instance to exit
| * | | | | | made fmt use of format_context more consistent (#409)Stefan Boberg2025-05-262-4/+8
| | | | | | | | | | | | | | | | | | | | | fixes compilation issues on Linux in some cases
| * | | | | | unblock cache bucket drop (#406)Dan Engelbrecht2025-05-264-54/+124
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * don't hold exclusive locks while deleting files from a dropped bucket/namespace * cleaner detection of missing namespace when issuing a drop
| * | | | | | use explicit capture for lambdas (#404)Dan Engelbrecht2025-05-202-896/+1254
| | | | | | |
| * | | | | | Merge pull request #402 from ue-foundation/zs/fix-oplog-import-oplog-creationZousar Shaker2025-05-201-6/+0
| |\ \ \ \ \ \ | | | | | | | | | | | | | | | | Zs/fix oplog import oplog creation
| | * | | | | | Fix oplog creation during oplog-importzousar2025-05-171-6/+0
| | | | | | | |
| * | | | | | | replace copy file (#403)Dan Engelbrecht2025-05-201-6/+34
| | | | | | | | | | | | | | | | | | | | | | | | Custom CopyFile in zen builds command increasing throughput by 50% on Windows and give better progress update
| * | | | | | | handle exception with batch work (#401)Dan Engelbrecht2025-05-199-424/+463
| |/ / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * use ParallelWork in rpc playback * use ParallelWork in projectstore * use ParallelWork in buildstore * use ParallelWork in cachedisklayer * use ParallelWork in compactcas * use ParallelWork in filecas * don't set abort flag in ParallelWork destructor * add PrepareFileForScatteredWrite for temp files in httpclient * Use PrepareFileForScatteredWrite when stream-decompressing files * be more relaxed when deleting temp files * allow explicit zen-cache when using direct host url without resolving * fix lambda capture when writing loose chunks * no delay when attempting to remove temp files
| * | | | | | parallel work handle dispatch exception (#400)Dan Engelbrecht2025-05-1611-1372/+1444
| | | | | | | | | | | | | | | | | | | | | - Bugfix: Wait for async threads if dispatching of work using ParallellWork throws exception
| * | | | | | validate custom fields (#399)Stefan Boberg2025-05-161-1/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * implemented validation of compact binary custom fields
| * | | | | | sparse win file write (#398)Dan Engelbrecht2025-05-164-1/+51
| | | | | | | | | | | | | | | | | | | | | * Added `--use-sparse-files` option to `zen builds` command improving write performance of large files. Enabled by default.
| * | | | | | make sure tests initialize trace so we don't end up allocating tons of ↵Stefan Boberg2025-05-157-2/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | memory for no reason (#397)
| * | | | | | `zen oplog-import` and `zen oplog-export` now supports ↵Dan Engelbrecht2025-05-141-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | `--oidctoken-exe-path` option (#395)
| * | | | | | extend log on failed httpsys response (#394)Dan Engelbrecht2025-05-132-8/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * extend log on failed httpsys response * fix formatting for "Desired port is in use, retrying" * add warning log if port is remapped
| * | | | | | skip empty or single-space command line arguments (#393)Dan Engelbrecht2025-05-131-1/+5
| | | | | | |
| * | | | | | keep snapshot on log delete fail (#391)Dan Engelbrecht2025-05-127-158/+88
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Improvement: Cleaned up snapshot writing for CompactCAS/FileCas/Cache/Project stores - Improvement: Safer recovery when failing to delete log for CompactCAS/FileCas/Cache/Project stores - Improvement: Added log file reset when writing snapshot at startup for FileCas
| * | | | | | enable per bucket config (#388)Dan Engelbrecht2025-05-125-16/+208
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Feature: Add per bucket cache configuration (Lua options file only) Improvement: --cache-memlayer-sizethreshold is now deprecated and has a new name: --cache-bucket-memlayer-sizethreshold to line up with per cache bucket configuration
| * | | | | | tweak iterate block parameters (#390)Dan Engelbrecht2025-05-122-34/+50
| | | | | | | | | | | | | | | | | | | | | * tweak block iteration chunk sizes
| * | | | | | handle exception in oplog mirror (#389)Dan Engelbrecht2025-05-121-46/+68
| | | | | | | | | | | | | | | | | | | | | * gracefully handle errors in threaded part of oplog-mirror
| * | | | | | flush cas log file (#387)Dan Engelbrecht2025-05-092-46/+31
| | | | | | | | | | | | | | | | | | | | | * make sure we remove the cas log file when writing full index at startup
| * | | | | | Change plugin config parsing to warn instead of throwzousar2025-05-071-8/+8
| | | | | | |
| * | | | | | make RemoveExpiredData and PreCache serial to reduce CPU overhead / lock ↵Dan Engelbrecht2025-05-071-85/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | contention (#385) * make RemoveExpiredData and PreCache serial to reduce CPU overhead / lock contention