aboutsummaryrefslogtreecommitdiff
path: root/src/zencore
Commit message (Collapse)AuthorAgeFilesLines
* remove zenutil dependency in zenremotestore (#547)Dan Engelbrecht2025-10-033-0/+346
| | | | | | | | | * remove dependency to zenutil/workerpools.h from remoteprojectstore.cpp * remove dependency to zenutil/workerpools.h from buildstoragecache.cpp * remove unneded include * move jupiter helpers to zenremotestore * move parallelwork to zencore * remove zenutil dependency from zenremotestore * clean up test project dependencies - use indirect dependencies
* make cpr a HttpClient implementation detail (#517)Stefan Boberg2025-09-292-0/+58
| | | | | these changes remove cpr from anything which is not `HttpClient` internals. The goal is to eventually replace cpr with a more direct curl interface to eliminate cpr since it's proven problematic due to their development practices which frequently breaks APIs and prevents us from updating vcpkg. But this PR is limited to refactoring existing cpr code to use `HttpClient` instead.
* fixed race condition in zen::logging::Get (#519)Stefan Boberg2025-09-291-3/+12
| | | if two requests for the same logger came in from different threads they could end up creating the same logger twice which causes an exception on registration
* Make sure we call the previous terminate handle if present when we intercept ↵Dan Engelbrecht2025-09-265-17/+84
| | | | | | terminate calls (#514) Improvement: Make sure we call the previous terminate handle if present when we intercept terminate calls Improvement: Avoid allocating memory for call stack in terminate handle and assert callback
* new append op rpc method (#511)Dan Engelbrecht2025-09-263-1/+29
| | | | | | - Feature: New `/prj/{project}/{oplog}/rpc` endpoint method `appendops` to send an array of oplog ops and receiving a list of `need` for attachments not present - Feature: Added `usingtmpfiles` boolean field to `/prj/{project}/{oplog}/rpc` method `putchunks` to be explicit about allowing move of temp attachment files - Improvement: Added additional validation of compact binary objects when reading from disk/receiving from client - Improvement: Windows: Added fallback code to use standard `MoveFile` api when rename via `SetFileInformationByHandle` fails in `MoveToFile` (used by filecas)
* change default sentry dsn to one listed on Sentry setup page (#504)Dan Engelbrecht2025-09-221-1/+1
|
* add EMode to WorkerTheadPool to avoid thread starvation (#492)Dan Engelbrecht2025-09-105-120/+171
| | | - Improvement: Add a new mode to worker thread pools to avoid starvation of workers which could cause long stalls due to other work begin queued up. UE-305498
* refactor zen command return value handling (#487)Dan Engelbrecht2025-09-052-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Improvement: Use consistent language for command line argument parsing errors - Improvement: Changed zen command parsing errors to output help first and error last to make it easier to spot the error - Improvement: Refactor zen command return codes to conform to valid Linux range (0-255) kSuccess = 0, kOtherError = 1, kBadInput = 2, kOutOfMemory = 16, kOutOfDisk = 17, kAssertError = 70, kHttpOtherClientError = 80, kHttpCantConnectError = 81, kHttpNotFound = 66, // NotFound(404) kHttpUnauthorized = 77, // Unauthorized(401), kHttpSLLError = 82, kHttpForbidden = 83, // Forbidden(403) kHttpTimeout = 84, // RequestTimeout(408) kHttpConflict = 85, // Conflict(409) kHttpNoHost = 86, kHttpOtherServerError = 90, kHttpInternalServerError = 91, // InternalServerError(500) kHttpServiceUnavailable = 69, // ServiceUnavailable(503) kHttpBadGateway = 92, // BadGateway(502) kHttpGatewayTimeout = 93, // GatewayTimeout(504)
* add validation of compact binary payloads before reading them (#483)Dan Engelbrecht2025-09-043-5/+47
| | | * add validation of compact binary payloads before reading them
* improve console output (#476)Dan Engelbrecht2025-08-262-1/+23
| | | | * add color coded logging level to console output (for warn/err/crit levels) * clean up console output
* Merge pull request #139 from ue-foundation/de/zen-service-commandLiam Mitchell2025-08-226-5/+35
|\ | | | | zen service command
| * Merge remote-tracking branch 'origin/main' into de/zen-service-commandLiam Mitchell2025-08-2110-27/+213
| |\
| * | Use correct sysctl headers on MacLiam Mitchell2025-08-211-1/+1
| | |
| * | Add missing headers to Mac process.cppLiam Mitchell2025-08-211-0/+2
| | |
| * | Merge pull request #452 from ue-foundation/lm/full-service-installLiam Mitchell2025-08-061-0/+2
| |\ \ | | | | | | | | Add --full option to service install, which will handle stop/uninstall if necessary, and copy binaries to install location
| | * | Fix permissions and ownership issues with service binary copy and remove ↵Liam Mitchell2025-07-251-0/+2
| | | | | | | | | | | | | | | | unnecessary alias from unit file
| * | | Merge branch 'main' into de/zen-service-commandLiam Mitchell2025-07-2922-563/+2113
| |\ \ \ | | |/ / | |/| |
| * | | Merge remote-tracking branch 'origin/main' into de/zen-service-commandDan Engelbrecht2025-03-1426-140/+1257
| |\ \ \
| * \ \ \ Merge remote-tracking branch 'origin/de/zen-service-command' into ↵Dan Engelbrecht2025-03-144-4/+12
| |\ \ \ \ | | | | | | | | | | | | | | | | | | de/zen-service-command
| | * | | | Implementation of service commands for Linux.Liam Mitchell2025-02-272-3/+8
| | | | | |
| | * | | | Linux compilation fixesLiam Mitchell2025-02-193-1/+4
| | | | | |
| * | | | | Merge remote-tracking branch 'origin/main' into de/zen-service-commandDan Engelbrecht2025-01-233-29/+195
| |\ \ \ \ \ | | |/ / / / | |/| | | |
| * | | | | Merge remote-tracking branch 'origin/main' into de/zen-service-commandDan Engelbrecht2025-01-173-0/+1220
| |\ \ \ \ \
| * | | | | | clang formatDan Engelbrecht2025-01-151-1/+1
| | | | | | |
| * | | | | | systemd unit file, incompleteDan Engelbrecht2025-01-152-1/+5
| | | | | | |
| * | | | | | if we can't open a process handle, assume it is running under other user idDan Engelbrecht2025-01-101-0/+5
| | | | | | |
| * | | | | | remove ServiceLevelDan Engelbrecht2025-01-102-9/+0
| | | | | | |
| * | | | | | get home folderDan Engelbrecht2025-01-102-0/+9
| | | | | | |
| * | | | | | add ServiceLevel for service processes: User, AllUsers and ServiceDan Engelbrecht2025-01-082-0/+9
| | | | | | |
* | | | | | | clean up trace options parsing (#473)Dan Engelbrecht2025-08-225-66/+97
| |_|_|_|_|/ |/| | | | | | | | | | | | | | | | | | | | | | | * clean up trace command line options explicitly shut down worker pools * some additional startup trace scopes
* | | | | | avoid new in static IoBuffer (#472)Dan Engelbrecht2025-08-211-1/+19
| | | | | |
* | | | | | zen print fixes/improvements (#469)Dan Engelbrecht2025-08-194-11/+132
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Improvement: `zen print` now allows output of compact binary content even if they are in non-optimal format (Unifom vs Non-Uniform arrays and objects) - Feature: `zen print` now has a `--show-type-info` option to add type information to output of compact binary content - Bugfix: Stats information for Build Store (Zen Store Cache) no longer throws exception and outputs invalid state information
* | | | | | use new builds api for oplogs (#464)Dan Engelbrecht2025-08-122-1/+32
| | | | | | | | | | | | | | | | | | | | | | | | - Improvement: Refactored jupiter oplog export code to reuse builds jupiter wrapper classes - Improvement: If `zen builds`, `zen oplog-import` or `zen oplog-import` command fails due to a http error, the return code for the program will be set to the error/status code
* | | | | | de/stringbuilder safety (#456)Dan Engelbrecht2025-08-053-14/+30
| |_|_|_|/ |/| | | | | | | | | | | | | | | | | | | | | | | | - Improvement: Safeguard FormatCallstack to not throw exceptions when building the callstack string - Improvement: Limit thread name length when setting it for debugger use - Improvemnet: Don't allow assert callbacks to throw exception - Improvement: When formatting log output for malformed attachments in a package message, allow the string buffer to grow instead of throwing exception
* | | | | graceful wait in parallelwork destructor (#438)Dan Engelbrecht2025-06-161-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | * exception safety when issuing ParallelWork * add asserts to Latch usage to catch usage errors * extended error messaging and recovery handling in ParallelWork destructor to help find issues
* | | | | sentry config (#430)Dan Engelbrecht2025-06-122-13/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Feature: Added `--sentry-environment` to `zen` and `zenserver` - Feature: Added `--sentry-debug` to `zen` and `zenserver` - Feature: Added environment variable parsing for the following options: - `UE_ZEN_SENTRY_ENABLED`: `--no-sentry` (inverted) - `UE_ZEN_SENTRY_DEBUG`: `--sentry-debug` - `UE_ZEN_SENTRY_ALLOWPERSONALINFO`: `--sentry-allow-personal-info` - `UE_ZEN_SENTRY_DSN`: `--sentry-dsn` - `UE_ZEN_SENTRY_ENVIRONMENT`: `--sentry-environment`
* | | | | revert 61b4a88f and cadaad63Dan Engelbrecht2025-06-102-23/+13
| | | | |
* | | | | add sentry configurations options for debug/environmentDan Engelbrecht2025-06-102-13/+23
| | | | | | | | | | | | | | | | | | | | | | | | | add env-variable parsing for sentry option
* | | | | `--sentry-dsn` option for zen command line and zenserver to control Sentry ↵Dan Engelbrecht2025-06-092-3/+12
| | | | | | | | | | | | | | | | | | | | | | | | | reporting endpoint (#427) moved sentry database path to temporary directory for zen commandline
* | | | | pause, resume and abort running builds cmd (#421)Dan Engelbrecht2025-06-054-18/+288
| | | | | | | | | | | | | | | | | | | | | | | | | - Feature: `zen builds pause`, `zen builds resume` and `zen builds abort` commands to control a running `zen builds` command - `--process-id` the process id to control, if omitted it tries to find a running process using the same executable as itself - Improvement: Process report now indicates if it is pausing or aborting
* | | | | fixed size chunking for encrypted files (#410)Dan Engelbrecht2025-06-036-26/+86
| | | | | | | | | | | | | | | | | | | | | | | | | - Improvement: Use fixed size block chunking for know encrypted/compressed file types - Improvement: Skip trying to compress chunks that are sourced from files that are known to be encrypted/compressed - Improvement: Add global open file cache for written files increasing throughput during download by reducing overhead of open/close of file by 80%
* | | | | http client streaming upload (#413)Dan Engelbrecht2025-06-024-173/+155
| | | | | | | | | | | | | | | - Improvement: Add streaming upload from HttpClient to reduce I/O caused by excessive MMap usage
* | | | | streaming none compressor (#414)Dan Engelbrecht2025-06-021-3/+36
| | | | | | | | | | | | | | | * add proper streaming to none compressor type
* | | | | made fmt use of format_context more consistent (#409)Stefan Boberg2025-05-262-4/+8
| | | | | | | | | | | | | | | fixes compilation issues on Linux in some cases
* | | | | parallel work handle dispatch exception (#400)Dan Engelbrecht2025-05-161-0/+5
| | | | | | | | | | | | | | | - Bugfix: Wait for async threads if dispatching of work using ParallellWork throws exception
* | | | | validate custom fields (#399)Stefan Boberg2025-05-161-1/+44
| | | | | | | | | | | | | | | | | | | | * implemented validation of compact binary custom fields
* | | | | sparse win file write (#398)Dan Engelbrecht2025-05-162-0/+30
| | | | | | | | | | | | | | | * Added `--use-sparse-files` option to `zen builds` command improving write performance of large files. Enabled by default.
* | | | | optimize block store CompactBlocks (#384)Dan Engelbrecht2025-05-072-3/+34
| | | | | | | | | | | | | | | | | | | | | | | | | - Improvement: Optimize block compact reducing memcpy operations - Improvement: Handle padding of block store blocks when compacting to avoid excessive flusing of write buffer - Improvement: Handle padding when writing oplog index snapshot to avoid unnecessary flushing of write buffer
* | | | | add sentry for zen command (#373)Dan Engelbrecht2025-05-063-0/+398
| | | | | | | | | | | | | | | | | | | | * refactor sentry integration and add to zen command line tool * move add_ldflags("-framework Security")
* | | | | add cxxopts overload for parsing file paths from command line (#362)Dan Engelbrecht2025-04-224-163/+0
| | | | |