| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
| |
* Moved firing of child-id event into a lambda
* Fire notify event to parent after attaching to an existing instance
* Updated changelog
|
| |
|
|
|
|
|
|
|
|
| |
- Feature: Added `--sentry-environment` to `zen` and `zenserver`
- Feature: Added `--sentry-debug` to `zen` and `zenserver`
- Feature: Added environment variable parsing for the following options:
- `UE_ZEN_SENTRY_ENABLED`: `--no-sentry` (inverted)
- `UE_ZEN_SENTRY_DEBUG`: `--sentry-debug`
- `UE_ZEN_SENTRY_ALLOWPERSONALINFO`: `--sentry-allow-personal-info`
- `UE_ZEN_SENTRY_DSN`: `--sentry-dsn`
- `UE_ZEN_SENTRY_ENVIRONMENT`: `--sentry-environment`
|
| | |
|
| |
|
|
|
| |
add env-variable parsing for sentry option
|
| |
|
|
|
| |
reporting endpoint (#427)
moved sentry database path to temporary directory for zen commandline
|
| |
|
|
| |
* refactor sentry integration and add to zen command line tool
* move add_ldflags("-framework Security")
|
| |
|
| |
- Bugfix: Long file paths now works correctly on Windows
|
| |
|
|
|
| |
* added FLLMTag which can be used to register memory tags outside of core
* changed `UE_MEMSCOPE` -> `ZEN_MEMSCOPE` for consistency
* instrumented some subsystems with dynamic tags
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This change introduces support for tracing of memory allocation activity. The code is ported from UE5, and Unreal Insights can be used to analyze the output. This is currently only fully supported on Windows, but will be extended to Mac/Linux in the near future.
To activate full memory tracking, pass `--trace=memory` on the commandline alongside `--tracehost=<ip>` or `-tracefile=<path>`. For more control over how much detail is traced you can instead pass some combination of `callstack`, `memtag`, `memalloc` instead. In practice, `--trace=memory` is an alias for `--trace=callstack,memtag,memalloc`). For convenience we also support `--trace=memory_light` which omits call stacks.
This change also introduces multiple memory allocators, which may be selected via command-line option `--malloc=<allocator>`:
* `mimalloc` - mimalloc (default, same as before)
* `rpmalloc` - rpmalloc is another high performance allocator for multithreaded applications which may be a better option than mimalloc (to be evaluated). Due to toolchain limitations this is currently only supported on Windows.
* `stomp` - an allocator intended to be used during development/debugging to help track down memory issues such as use-after-free or out-of-bounds access. Currently only supported on Windows.
* `ansi` - fallback to default system allocator
|
| |
|
| |
* make sure to clear sponsor slot if pickup does not happen
|
| |
|
| |
* safer calls to IsProcessRunning to handle cases where we can't check status of processes
|
| |
|
|
| |
instead of reporting error to Sentry (#132)
|
| |
|
|
|
| |
added `--detach` option to zenserver. When this is passed in with a false value, we do not create a new process group in order to behave more as expected when running with `xmake run zenserver`. Without this change the zenserver process does not receive any signals and won't exit when xmake does, causing processes to linger in the background.
The default behaviour (when run from UE) is unchanged.
|
| |
|
|
|
|
| |
* add support for indenting callstack output
* Explicitly catch option-parse error and reduce log spam on bad parameters
* add command line to sentry error reports
* log command line at startup
|
| |
|
| |
Improvement: Don't use "error:" in log messages unless there is an error as Horde CI will pick up that log line and interpret it as an error
|
| |
|
|
|
|
|
|
|
|
|
| |
* report down/up transfer speed during progress
* add disk buffering in http client
* offload block decoding and chunk writing form network worker pool threads
add block hash verification for blocks recevied at oplog import
* separate download-latch from write-latch to get more accurate download speed
* check headers when downloading with http client to go directly to file writing for large payloads
* we must clear write callback even if we only provide it as an argument to the Download() call
* make timeout optional in AddSponsorProcess
* check return codes when creating windows threadpool
|
| |
|
|
|
|
|
|
|
|
|
|
| |
- Feature: `zen down`
- --`data-dir` to specify a data directory to deduce which zen instance to bring down
- Feature: `zen attach`
- --`data-dir` to specify a data directory to deduce which zen instance to attach to222
- Feature: `zen status`
- --`port` filter running zen instances based on port
- --`data-dir` filter running zen instances based on information in the data directory
- Improvement: Trying to load a compact binary object from an empty file no longer causes access violation
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
- Feature: `zen up` command improvements
- --`port` allows you to specify a base port when starting an instance
- --`base-dir` allows you to specify a base directory for the zenserver executable if it is not located next to the zen.exe executable
- Feature: `zen down`
- --`port` allows you to specify a base port when shutting down an instance
- --`base-dir` allows you to specify a base directory for the zenserver executable if it is not located next to the zen.exe executable
- --`force` if regular shutdown fails it tries to find a running zenserver.exe process and terminate it
- If it fails to attach to the running server it now waits for it to exit when setting the RequestExit shared memory flag
- Improvement: zenserver now checks the RequestExit flag in the shared memory and exist gracefully if it is set
- Improvement: When adding a sponsor process to a running zenserver instance, we wait for it to be picked up from the shared memory section to determine success/fail
|
| |
|
|
| |
- Improvement: Add file and line to ASSERT exceptions
- Improvement: Catch call stack when throwing assert exceptions and log/output call stack at important places to provide more context to caller
|
| |
|
|
|
| |
characters (#32)
* Fix sentry using wrong folder path when data path contains non-ascii characters
|
| |
|
|
|
| |
- Improvement: Delay exiting due to no sponsor processes by one second to handle race conditions
- Improvement: Safer IsProcessRunning check
- Improvement: make sure we can RequestApplicationExit safely from any thread
|
| |
|
| |
* add a retry when trying to create the lock file if UE is trying to read it at the same time
|
| |
|
|
|
| |
* make sure zenserver reacts and exist on SIGTERM signal
* add zen tag to all runners
* temp disable mac codesigning
|
| |
|
|
|
|
|
| |
- Improvement: Add zenhttp-test and zenutil-test
- Improvement: Moved cachepolicy test to cachepolicy.cpp
- Improvement: Renamed cachestore tests from z$ to cachestore
- Improvement: Moved test linking so test for a lib is linked by <lib>-test
- Improvement: Removed HttpRequestParseRelativeUri in httpstructuredcache.cpp and use the one in cacherequests.h instead
|
| |
|
|
|
| |
* move structuredcachestore tests to zenstore-test
* Don't materialize entire files when hashing if it is a large files
* rewrite CompositeBuffer::Mid to never materialize buffers
|
| |
|
| |
this adds information on program name and command line to trace initialization
|
| |
|
|
|
|
| |
- Improvement: Scrub command now validates compressed buffer hashes in filecas storage (used for large chunks)
- Improvement: Added --dry, --no-gc and --no-cas options to zen scrub command
- Improvement: Implemented oplog scrubbing (previously was a no-op)
- Improvement: Implemented support for running scrubbint at startup with --scrub=<options>
|
| |
|
|
|
| |
the previous implementation of in-memory index snapshots serialise data to memory before writing to disk and vice versa when reading. This leads to some memory spikes which end up pushing useful data out of system cache and also cause stalls on I/O operations.
this change moves more code to a streaming serialisation approach which scales better from a memory usage perspective and also performs much better
|
| | |
|
| |
|
|
|
| |
this introduces a --snapshot-dir command line option to zenserver which specifies a directory which will be propagated to the persistence root directory on start-up.
This is most powerful with file systems which support block cloning, such as ReFS on Windows. This allows even very large state snapshots to be used repeatedly without having to worry about mutating the original dataset on disk. When using ReFS the state copy for even large state directories can be very fast since the duration is primarily proportional to the number of files in the tree rather than the size of the files being cloned. The storage requirements are also minimal as all data will be handled in a copy-on-write manner.
|
| |
|
|
| |
completion (#539)
|
| |
|
|
|
|
|
|
|
| |
this change aims to hide logging internals from client code, in order to make it easier to extend and take more control over the logging process in the future.
As a bonus side effect, the generated code is much tighter (net delta around 2.5% on the resulting executable which includes lots of thirdparty code) and should take less time to compile and link.
Client usage via macros is pretty much unchanged. The main exposure client code had to spdlog internals before was the use of custom loggers per subsystem, where it would be common to have `spdlog::logger` references to keep a reference to a logger within a class. This is now replaced by `zen::LoggerRef` which currently simply encapsulates an actual `spdlog::logger` instance, but this is intended to be an implementation detail which will change in the future.
The way the change works is that we now handle any formatting of log messages in the zencore logging subsystem instead of relying on `spdlog` to manage this. We use the `fmt` library to do the formatting which means the client usage is identical to using `spdlog`. The formatted message is then forwarded onto any sinks etc which are still implememted via `spdlog`.
|
| |
|
|
|
|
|
|
|
| |
zenutil and zenserver both contain very similar logging setup code and this change aims to make them have most code in common.
* fullformatter/jsonformatter/RotatingFileSink are moved into dedicated header files in zenutil
* zenserver `InitializeLogging`/`ShutdownLogging` are renamed `InitializeServerLogging`/`InitializeServerLogging`
* these now call into the common zenutil `BeginInitializeLogging`/`FinishInitializeLogging` in addition to setting up server custom logging
* `std::filesystem::path` is now logged after stripping any `\\\\?\\` prefix for readability
|
| |
|
| |
when specified with `--clean`, the data directory will be wiped clean at startup
|
| | |
|
| |
|
|
| |
this was there before but lost in a merge recently
|
| |
|