diff options
| author | Stefan Boberg <[email protected]> | 2026-03-18 11:19:10 +0100 |
|---|---|---|
| committer | GitHub Enterprise <[email protected]> | 2026-03-18 11:19:10 +0100 |
| commit | eba410c4168e23d7908827eb34b7cf0c58a5dc48 (patch) | |
| tree | 3cda8e8f3f81941d3bb5b84a8155350c5bb2068c /src/zencore/include | |
| parent | bugfix release - v5.7.23 (#851) (diff) | |
| download | zen-eba410c4168e23d7908827eb34b7cf0c58a5dc48.tar.xz zen-eba410c4168e23d7908827eb34b7cf0c58a5dc48.zip | |
Compute batching (#849)
### Compute Batch Submission
- Consolidate duplicated action submission logic in `httpcomputeservice` into a single `HandleSubmitAction` supporting both single-action and batch (actions array) payloads
- Group actions by queue in `RemoteHttpRunner` and submit as batches with configurable chunk size, falling back to individual submission on failure
- Extract shared helpers: `MakeErrorResult`, `ValidateQueueForEnqueue`, `ActivateActionInQueue`, `RemoveActionFromActiveMaps`
### Retracted Action State
- Add `Retracted` state to `RunnerAction` for retry-free rescheduling — an explicit request to pull an action back and reschedule it on a different runner without incrementing `RetryCount`
- Implement idempotent `RetractAction()` on `RunnerAction` and `ComputeServiceSession`
- Add `POST jobs/{lsn}/retract` and `queues/{queueref}/jobs/{lsn}/retract` HTTP endpoints
- Add state machine documentation and per-state comments to `RunnerAction`
### Compute Race Fixes
- Fix race in `HandleActionUpdates` where actions enqueued between session abandon and scheduler tick were never abandoned, causing `GetActionResult` to return 202 indefinitely
- Fix queue `ActiveCount` race where `NotifyQueueActionComplete` was called after releasing `m_ResultsLock`, allowing callers to observe stale counters immediately after `GetActionResult` returned OK
### Logging Optimization and ANSI improvements
- Improve `AnsiColorStdoutSink` write efficiency — single write call, dirty-flag flush, `RwLock` instead of `std::mutex`
- Move ANSI color emission from sink into formatters via `Formatter::SetColorEnabled()`; remove `ColorRangeStart`/`End` from `LogMessage`
- Extract color helpers (`AnsiColorForLevel`, `StripAnsiSgrSequences`) into `helpers.h`
- Strip upstream ANSI SGR escapes in non-color output mode. This enables colour in log messages without polluting log files with ANSI control sequences
- Move `RotatingFileSink`, `JsonFormatter`, and `FullFormatter` from header-only to pimpl with `.cpp` files
### CLI / Exec Refactoring
- Extract `ExecSessionRunner` class from ~920-line `ExecUsingSession` into focused methods and a `ExecSessionConfig` struct
- Replace monolithic `ExecCommand` with subcommand-based architecture (`http`, `inproc`, `beacon`, `dump`, `buildlog`)
- Allow parent options to appear after subcommand name by parsing subcommand args permissively and forwarding unmatched tokens to the parent parser
### Testing Improvements
- Fix `--test-suite` filter being ignored due to accumulation with default wildcard filter
- Add test suite banners to test listener output
- Made `function.session.abandon_pending` test more robust
### Startup / Reliability Fixes
- Fix silent exit when a second zenserver instance detects a port conflict — use `ZEN_CONSOLE_*` for log calls that precede `InitializeLogging()`
- Fix two potential SIGSEGV paths during early startup: guard `sentry_options_new()` returning nullptr, and throw on `ZenServerState::Register()` returning nullptr instead of dereferencing
- Fail on unrecognized zenserver `--mode` instead of silently defaulting to store
### Other
- Show host details (hostname, platform, CPU count, memory) when discovering new compute workers
- Move frontend `html.zip` from source tree into build directory
- Add format specifications for Compact Binary and Compressed Buffer wire formats
- Add `WriteCompactBinaryObject` to zencore
- Extended `ConsoleTui` with additional functionality
- Add `--vscode` option to `xmake sln` for clangd / `compile_commands.json` support
- Disable compute/horde/nomad in release builds (not yet production-ready)
- Disable unintended `ASIO_HAS_IO_URING` enablement
- Fix crashpad patch missing leading whitespace
- Clean up code triggering gcc false positives
Diffstat (limited to 'src/zencore/include')
| -rw-r--r-- | src/zencore/include/zencore/compactbinaryfile.h | 1 | ||||
| -rw-r--r-- | src/zencore/include/zencore/logging/ansicolorsink.h | 3 | ||||
| -rw-r--r-- | src/zencore/include/zencore/logging/formatter.h | 6 | ||||
| -rw-r--r-- | src/zencore/include/zencore/logging/helpers.h | 77 | ||||
| -rw-r--r-- | src/zencore/include/zencore/logging/logmsg.h | 3 | ||||
| -rw-r--r-- | src/zencore/include/zencore/testing.h | 5 |
6 files changed, 89 insertions, 6 deletions
diff --git a/src/zencore/include/zencore/compactbinaryfile.h b/src/zencore/include/zencore/compactbinaryfile.h index 33f3e7bea..a06524549 100644 --- a/src/zencore/include/zencore/compactbinaryfile.h +++ b/src/zencore/include/zencore/compactbinaryfile.h @@ -15,5 +15,6 @@ struct CbObjectFromFile }; CbObjectFromFile LoadCompactBinaryObject(const std::filesystem::path& FilePath); +void WriteCompactBinaryObject(const std::filesystem::path& Path, const CbObject& Object); } // namespace zen diff --git a/src/zencore/include/zencore/logging/ansicolorsink.h b/src/zencore/include/zencore/logging/ansicolorsink.h index 5060a8393..939c70d12 100644 --- a/src/zencore/include/zencore/logging/ansicolorsink.h +++ b/src/zencore/include/zencore/logging/ansicolorsink.h @@ -15,6 +15,9 @@ enum class ColorMode Auto }; +bool IsColorTerminal(); +bool ResolveColorMode(ColorMode Mode); + class AnsiColorStdoutSink : public Sink { public: diff --git a/src/zencore/include/zencore/logging/formatter.h b/src/zencore/include/zencore/logging/formatter.h index 11904d71d..e605b22b8 100644 --- a/src/zencore/include/zencore/logging/formatter.h +++ b/src/zencore/include/zencore/logging/formatter.h @@ -15,6 +15,12 @@ public: virtual ~Formatter() = default; virtual void Format(const LogMessage& Msg, MemoryBuffer& Dest) = 0; virtual std::unique_ptr<Formatter> Clone() const = 0; + + void SetColorEnabled(bool Enabled) { m_UseColor = Enabled; } + bool IsColorEnabled() const { return m_UseColor; } + +private: + bool m_UseColor = false; }; } // namespace zen::logging diff --git a/src/zencore/include/zencore/logging/helpers.h b/src/zencore/include/zencore/logging/helpers.h index ce021e1a5..765aa59e3 100644 --- a/src/zencore/include/zencore/logging/helpers.h +++ b/src/zencore/include/zencore/logging/helpers.h @@ -119,4 +119,81 @@ LevelToShortString(LogLevel Level) return ToStringView(Level); } +inline std::string_view +AnsiColorForLevel(LogLevel Level) +{ + using namespace std::string_view_literals; + switch (Level) + { + case Trace: + return "\033[37m"sv; // white + case Debug: + return "\033[36m"sv; // cyan + case Info: + return "\033[32m"sv; // green + case Warn: + return "\033[33m\033[1m"sv; // bold yellow + case Err: + return "\033[31m\033[1m"sv; // bold red + case Critical: + return "\033[1m\033[41m"sv; // bold on red background + default: + return "\033[m"sv; + } +} + +inline constexpr std::string_view kAnsiReset = "\033[m"; + +inline void +AppendAnsiColor(LogLevel Level, MemoryBuffer& Dest) +{ + std::string_view Color = AnsiColorForLevel(Level); + Dest.append(Color.data(), Color.data() + Color.size()); +} + +inline void +AppendAnsiReset(MemoryBuffer& Dest) +{ + Dest.append(kAnsiReset.data(), kAnsiReset.data() + kAnsiReset.size()); +} + +// Strip ANSI SGR escape sequences (\033[...m) from the buffer in-place. +// Only sequences terminated by 'm' are removed (colors, bold, underline, etc.). +// Other CSI sequences (cursor movement, erase, etc.) are left intact. +inline void +StripAnsiSgrSequences(MemoryBuffer& Buf) +{ + const char* Src = Buf.data(); + const char* End = Src + Buf.size(); + char* Dst = Buf.data(); + + while (Src < End) + { + if (Src[0] == '\033' && (Src + 1) < End && Src[1] == '[') + { + const char* Seq = Src + 2; + while (Seq < End && *Seq != 'm') + { + ++Seq; + } + if (Seq < End) + { + ++Seq; // skip 'm' + } + Src = Seq; + } + else + { + if (Dst != Src) + { + *Dst = *Src; + } + ++Dst; + ++Src; + } + } + + Buf.resize(static_cast<size_t>(Dst - Buf.data())); +} + } // namespace zen::logging::helpers diff --git a/src/zencore/include/zencore/logging/logmsg.h b/src/zencore/include/zencore/logging/logmsg.h index 1d8b6b1b7..a1acb503b 100644 --- a/src/zencore/include/zencore/logging/logmsg.h +++ b/src/zencore/include/zencore/logging/logmsg.h @@ -40,9 +40,6 @@ struct LogMessage void SetTime(LogClock::time_point InTime) { m_Time = InTime; } void SetSource(const SourceLocation& InSource) { m_Source = InSource; } - mutable size_t ColorRangeStart = 0; - mutable size_t ColorRangeEnd = 0; - private: static constexpr LogPoint s_DefaultPoints[LogLevelCount] = { {{}, Trace, {}}, diff --git a/src/zencore/include/zencore/testing.h b/src/zencore/include/zencore/testing.h index 8410216c4..01356fa00 100644 --- a/src/zencore/include/zencore/testing.h +++ b/src/zencore/include/zencore/testing.h @@ -43,9 +43,8 @@ public: TestRunner(); ~TestRunner(); - void SetDefaultSuiteFilter(const char* Pattern); - int ApplyCommandLine(int Argc, char const* const* Argv); - int Run(); + int ApplyCommandLine(int Argc, char const* const* Argv, const char* DefaultSuiteFilter = nullptr); + int Run(); private: struct Impl; |