| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
| |
These overloads are Windows-only but were missing the preprocessor guard,
causing a linker error on Linux.
|
| |
|
|
|
|
|
|
|
|
|
| |
- Add explicit constructors from int (pid) and void* (native handle)
- Add move constructor and move assignment operator
- Fix subprocess manager tests: add ZEN_EXE_SUFFIX_LITERAL to appstub
path so CreateProcessW finds the executable on Windows
- Fix zentest-appstub writing a spurious Versions file to cwd by
defaulting VersionPath to empty
- Simplify test code to use new constructors instead of two-step
default-construct + Initialize pattern
|
| |
|
|
|
|
|
|
|
|
|
| |
Introduces ProcessGroup as a first-class object for managing related
processes together. On Windows, groups are backed by a JobObject with
kill-on-close guarantees. On Linux/macOS, groups use setpgid() for bulk
signal delivery via kill(-pgid).
Groups provide per-group metrics aggregation, bulk KillAll, and
independent Spawn/Adopt/Remove. The SubprocessManager metrics timer
samples across both grouped and ungrouped processes via a unified index.
|
| |
|
|
|
|
|
| |
- **Cross-platform `GetProcessMetrics`**: Implement Linux (`/proc/{pid}/stat`, `/proc/{pid}/statm`, `/proc/{pid}/status`) and macOS (`proc_pidinfo(PROC_PIDTASKINFO)`) support for CPU times and memory metrics. Fix Windows to populate the `MemoryBytes` field (was always 0). All platforms now set `MemoryBytes = WorkingSetSize`.
- **`ProcessMetricsTracker`**: Experimental utility class (`zenutil`) that periodically samples resource usage for a set of tracked child processes. Supports both a dedicated background thread and an ASIO steady_timer mode. Computes delta-based CPU usage percentage across samples, with batched sampling (8 processes per tick) to limit per-cycle overhead.
- **`ProcessHandle` documentation**: Add Doxygen comments to all public methods describing platform-specific behavior.
- **Cleanup**: Remove unused `ZEN_RUN_TESTS` macro (inlined at its single call site in `zenserver/main.cpp`), remove dead `#if 0` thread-shutdown workaround block.
- **Minor fixes**: Use `HttpClientAccessToken` constructor in hordeclient instead of setting private members directly. Log ASIO version at startup and include it in the server settings list.
|
| |
|
|
|
|
|
|
|
|
|
| |
This PR improves process lifecycle handling and resilience across several areas:
- **Reclaim stale shared-memory entries instead of exiting** (`zenserver.cpp`): When a zenserver instance fails to attach as a sponsor to an existing process (e.g. because the PID was reused by an unrelated process), the server now clears the stale shared-memory entry and proceeds with normal startup instead of calling `std::exit(1)`.
- **Wait for child process exit in `Kill()` and `Terminate()` on Unix** (`process.cpp`): After sending `SIGTERM` in `Kill()`, the code now waits up to 5s for graceful shutdown (escalating to `SIGKILL` on timeout), matching the Windows behavior. `Terminate()` also waits after `SIGKILL` so the child is properly reaped and doesn't linger as a zombie clogging up the process table.
- **Fix sysctl buffer race in macOS `FindProcess`** (`process.cpp`): The macOS process enumeration now retries the `sysctl` call (up to 3 attempts with 25% buffer padding) to handle the race where the process list changes between the sizing call and the data-fetching call. Also flattens the nesting and fixes the guard/free scoping.
- **Terminate stale processes before integration tests** (`zenserver-test.cpp`, `test.lua`): The integration test runner now accepts a `--kill-stale-processes` flag (passed automatically by `test.lua`) that scans for and terminates any leftover `zenserver`, `zenserver-test`, and `zentest-appstub` processes from previous test runs, logging the executable name and PID of each. This addresses flaky test failures caused by stale processes from prior runs holding ports or other resources.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- **RAII pipe handles for child process stdout/stderr capture**: `StdoutPipeHandles` is now a proper RAII type with automatic cleanup, move semantics, and partial close support. This makes it safe to use pipes for capturing child process output without risking handle/fd leaks.
- **Optional separate stderr pipe**: `CreateProcOptions` now accepts a `StderrPipe` field so callers can capture stdout and stderr independently. When null (default), stderr shares the stdout pipe as before.
- **LogStreamListener with pluggable handler**: The TCP log stream listener accepts connections from remote processes and delivers parsed log lines through a `LogStreamHandler` interface, set dynamically via `SetHandler()`. This allows any client to receive log messages without depending on a specific console implementation.
- **TcpLogStreamSink for zen::logging**: A logging sink that forwards log messages to a `LogStreamListener` over TCP, using the native `zen::logging::Sink` infrastructure with proper thread-safe synchronization.
- **Reliable child process exit codes on Linux**: `waitpid` result handling is fixed so `ProcessHandle::GetExitCode()` returns the real exit code. `ProcessHandle::Reset()` reaps zombies directly, replacing the global `IgnoreChildSignals()` which prevented exit code collection entirely. Also fixes a TOCTOU race in `ProcessHandle::Wait()` on Linux/Mac.
- **Pipe capture test suite**: Tests covering stdout/stderr capture via pipes (both shared and separate modes), RAII cleanup, move semantics, and exit code propagation using `zentest-appstub` as the child process.
- **Service command integration tests**: Shell-based integration tests for `zen service` covering the full lifecycle (install, status, start, stop, uninstall) on all three platforms — Linux (systemd), macOS (launchd), and Windows (SCM via PowerShell).
- **Test script reorganization**: Platform-specific test scripts moved from `scripts/test_scripts/` into `scripts/test_linux/`, `test_mac/`, and `test_windows/`.
|
| |
|
|
|
|
|
| |
- Improvement: Hub module listing now includes per-instance process metrics (memory, CPU time, working set, pagefile usage)
- Improvement: Hub now monitors provisioned instance health in the background and refreshes process metrics periodically
- Improvement: Hub no longer exposes raw `StorageServerInstance` pointers to callers; instance state is returned as value snapshots (`Hub::InstanceInfo`)
- Improvement: Hub instance access is now guarded by RAII per-instance locks (`SharedLockedPtr`/`ExclusiveLockedPtr`), preventing concurrent modifications during provisioning and deprovisioning
- Improvement: Hub instance lifecycle is now tracked as a `HubInstanceState` enum covering transitional states (Provisioning, Deprovisioning, Hibernating, Waking); exposed as a string in the HTTP API and dashboard
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This functionality is intended to be used to manage datasets for test cases, but may be useful elsewhere in the future.
- **Add S3 client with AWS Signature V4 (SigV4) signing** — new `S3Client` in `zenutil/cloud/` supporting `GetObject`, `PutObject`, `DeleteObject`, `HeadObject`, and `ListObjects` operations
- **Add EC2 IMDS credential provider** — automatically fetches and refreshes temporary AWS credentials from the EC2 Instance Metadata Service (IMDSv2) for use by the S3 client
- **Add SigV4 signing library** — standalone implementation of AWS Signature Version 4 request signing (headers and query-string presigning)
- **Add path-style addressing support** — enables compatibility with S3-compatible stores like MinIO (in addition to virtual-hosted style)
- **Add S3 integration tests** — includes a `MinioProcess` test helper that spins up a local MinIO server, plus integration tests exercising the S3 client end-to-end
- **Add S3-backed `HttpObjectStoreService` tests** — integration tests verifying the zenserver object store works against an S3 backend
- **Refactor mock IMDS into `zenutil/cloud/`** — moved and generalized the mock IMDS server from `zencompute` so it can be reused by both compute and S3 credential tests
|
| |
|
|
|
|
|
| |
This PR makes it *possible* to do a Windows build on Linux via `clang-cl`.
It doesn't actually change any build process. No policy change, just mechanics and some code fixes to clear clang compilation.
The code fixes are mainly related to #include file name casing, to match the on-disk casing of the SDK files (via xwin).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
removal (#841)
- Percent-decode URIs in ASIO HTTP server to match http.sys CookedUrl behavior, ensuring consistent decoded paths across backends
- Add Environment field to CreateProcOptions for passing extra env vars to child processes (Windows: merged into Unicode environment block; Unix: setenv in fork)
- Add GetCompilerName() and include it in build options startup logging
- Suppress Windows CRT error dialogs in test harness for headless/CI runs
- Fix mimalloc package: pass CMAKE_BUILD_TYPE, skip cfuncs test for cross-compile
- Add virtual destructor to SentryAssertImpl to fix debug-mode warning
- Simplify object store path handling now that URIs arrive pre-decoded
- Add URI decoding test coverage for percent-encoded paths and query params
- Simplify httpasio request handling by using strands (guarantees no parallel handlers per connection)
- Removed deprecated regex-based route matching support
- Fix full GC never triggering after cross-toolchain builds: The `gc_state` file stores `system_clock` ticks, but the tick resolution differs between toolchains (nanoseconds on GCC/standard clang, microseconds on UE clang). A nanosecond timestamp misinterpreted as microseconds appears far in the future (~year 58,000), bypassing the staleness check and preventing time-based full GC from ever running. Fixed by also resetting when the stored timestamp is in the future.
- Clamp GC countdown display to configured interval: Prevents nonsensical log output (e.g. "Full GC in 492128002h") caused by the above or any other clock anomaly. The clamp applies to both the scheduler log and the status API.
|
| |
|
|
|
| |
- Add GetTotalBytesReceived/GetTotalBytesSent to HttpServer with implementations in ASIO and http.sys backends
- Add ExpectedErrorCodes to HttpClientSettings to suppress warn/info logs for anticipated HTTP error codes
- Also fixes minor issues in `CprHttpClient::Download`
|
| |
|
|
|
| |
This change introduces job object support on Windows to be able to more accurately track and limit resource usage on storage instances created by the hub service. It also ensures that all child instances can be torn down reliably on exit.
Also made it so hub tests no longer pop up console windows while running.
|
| |
|
|
|
| |
Allows user to automate launching of zenserver dashboard, including when multiple instances are running. If multiple instances are running you can open all dashboards with `--all`, and also using the in-terminal chooser which also allows you to open a specific instance.
Also includes a fix to `zen exec` when using offset/stride/limit
|
| |
|
|
|
|
|
|
| |
* this adds a consul package which can be used to fetch a consul binary
* it also adds a `ConsulProcess` helper which can be used to spawn and manage a consul service instance
* zencore dependencies brought across:
- `except_fmt.h` for easer generation of formatted exception messages
- `process.h/cpp` changes (adds `Kill` operation and process group support on Windows)
- `string.h` changes to allow generic use of `WideToUtf8()`
|
| | |
|
| | |
|
| |\ |
|
| | |
| |
| |
| |
| | |
- Feature: `zen builds pause`, `zen builds resume` and `zen builds abort` commands to control a running `zen builds` command
- `--process-id` the process id to control, if omitted it tries to find a running process using the same executable as itself
- Improvement: Process report now indicates if it is pausing or aborting
|
| | | |
|
| | |
| |
| | |
- Bugfix: Long file paths now works correctly on Windows
|
| |\| |
|
| | |
| |
| | |
Handling of quotes and quotes with leading backslash for command line parsing - UE-231677
|
| |\ \
| | |
| | |
| | | |
de/zen-service-command
|
| | | | |
|
| | | | |
|
| |\ \ \
| |/ /
|/| /
| |/ |
|
| | |
| |
| |
| |
| |
| | |
add DirectoryContent::IncludeFileSizes
add DirectoryContent::IncludeAttributes
add multithreaded GetDirectoryContent
use multithreaded GetDirectoryContent in workspace folder scanning
|
| | | |
|
| | | |
|
| | | |
|
| |/ |
|
| |
|
|
| |
(#226)
|
| |
|
|
|
|
| |
* zenserver process launch/termination improvements
* fix GetPidStatus to return error code on Linux
* fix linux FindProcess()
* cleanup IsZombieProcess
|
| |
|
|
| |
* capture spawned server output and output on launch error
* fix logging and launch validation in tests
|
| |
|
|
|
|
|
|
|
|
|
|
| |
- Feature: `zen down`
- --`data-dir` to specify a data directory to deduce which zen instance to bring down
- Feature: `zen attach`
- --`data-dir` to specify a data directory to deduce which zen instance to attach to222
- Feature: `zen status`
- --`port` filter running zen instances based on port
- --`data-dir` filter running zen instances based on information in the data directory
- Improvement: Trying to load a compact binary object from an empty file no longer causes access violation
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
- Feature: `zen up` command improvements
- --`port` allows you to specify a base port when starting an instance
- --`base-dir` allows you to specify a base directory for the zenserver executable if it is not located next to the zen.exe executable
- Feature: `zen down`
- --`port` allows you to specify a base port when shutting down an instance
- --`base-dir` allows you to specify a base directory for the zenserver executable if it is not located next to the zen.exe executable
- --`force` if regular shutdown fails it tries to find a running zenserver.exe process and terminate it
- If it fails to attach to the running server it now waits for it to exit when setting the RequestExit shared memory flag
- Improvement: zenserver now checks the RequestExit flag in the shared memory and exist gracefully if it is set
- Improvement: When adding a sponsor process to a running zenserver instance, we wait for it to be picked up from the shared memory section to determine success/fail
|
| |
|
|
|
| |
- Improvement: Delay exiting due to no sponsor processes by one second to handle race conditions
- Improvement: Safer IsProcessRunning check
- Improvement: make sure we can RequestApplicationExit safely from any thread
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
initial version -- this is primarily intended to be used for running stress tests and/or benchmarks
example usage:
`zen run -n 10 -- zenserver-test`
`zen run -n 10 -- zenserver-test --ts=core.assert` run zenserver-test 10 times (testing only the `core.assert` test suite)
`zen run --time 600 --basepath=d:\test_dir\test1 -- zenserver-test` keeps spawning new instances for 10 minutes (600 seconds)
|
| |
|