| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
## Summary
This PR adds a session management service, several new dashboard pages, and a number of infrastructure improvements.
### Sessions Service
- `SessionsServiceClient` in `zenutil` announces sessions to a remote zenserver with a 15s heartbeat (POST/PUT/DELETE lifecycle)
- Storage server registers itself with its own local sessions service on startup
- Session mode attribute coupled to server mode (Compute, Proxy, Hub, etc.)
- Ended sessions tracked with `ended_at` timestamp; status filtering (Active/Ended/All)
- `--sessions-url` config option for remote session announcement
- In-process log sink (`InProcSessionLogSink`) forwards server log output to the server's own session, visible in the dashboard
### Session Log Viewer
- POST/GET endpoints for session logs (`/sessions/{id}/log`) supporting raw text and structured JSON/CbObject with batch `entries` array
- In-memory log storage per session (capped at 10k entries) with cursor-based pagination for efficient incremental fetching
- Log panel in the sessions dashboard with incremental DOM updates, auto-scroll (Follow toggle), newest-first toggle, text filter, and log-level coloring
- Auto-selects the server's own session on page load
### TCP Log Streaming
- `LogStreamListener` and `TcpLogStreamSink` for log delivery over TCP
- Sequence numbers on each message with drop detection and synthetic "dropped" notice on gaps
- Gathered buffer writes to reduce syscall overhead when flushing batches
- Tests covering basic delivery, multi-line splitting, drop detection, and sequencing
### New Dashboard Pages
- **Sessions**: master-detail layout with selectable rows, metadata panel, live WebSocket updates, paging, abbreviated date formatting, and "this" pill for the local session
- **Object Store**: summary stats tiles and bucket table with click-to-expand inline object listing (`GET /obj/`)
- **Storage**: per-volume disk usage breakdown (`GET /admin/storage`), Garbage Collection status section (next-run countdown, last-run stats), and GC History table with paginated rows and expandable detail panels
- **Network**: overview tiles, per-service request table, proxy connections, and live WebSocket updates; distinct client IPs and session counts via HyperLogLog
### Documentation Page
- In-dashboard Docs page with sidebar navigation, markdown rendering (via `marked`), Mermaid diagram support (theme-aware), collapsible sections, text filtering with highlighting, and cross-document linking
- New user-facing docs: `overview.md` (with architecture and per-mode diagrams), `sessions.md`, `cache.md`, `projects.md`; updated `compute.md`
- Dev docs moved to `docs/dev/`
### Infrastructure & Bug Fixes
- **Deflate compression** for the embedded frontend zip (~3.4MB → ~950KB); zlib inflate support added to `ZipFs` with cached decompressed buffers
- **Local IP addresses**: `GetLocalIpAddresses()` (Windows via `GetAdaptersAddresses`, Linux/Mac via `getifaddrs`); surfaced in `/status/status`, `/health/info`, and the dashboard banner
- **Dashboard nav**: unified into `zen-nav` web component with `MutationObserver` for dynamically added links, CSS `::part()` to merge banner/nav border radii, and prefix-based active link detection
- Stats broadcast refactored from manual JSON string concatenation to `CbObjectWriter`; `CbObject`-to-JS conversion improved for `TimeSpan`, `DateTime`, and large integers
- Stats WebSocket boilerplate consolidated into `ZenPage.connect_stats_ws()`
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
### Compute Batch Submission
- Consolidate duplicated action submission logic in `httpcomputeservice` into a single `HandleSubmitAction` supporting both single-action and batch (actions array) payloads
- Group actions by queue in `RemoteHttpRunner` and submit as batches with configurable chunk size, falling back to individual submission on failure
- Extract shared helpers: `MakeErrorResult`, `ValidateQueueForEnqueue`, `ActivateActionInQueue`, `RemoveActionFromActiveMaps`
### Retracted Action State
- Add `Retracted` state to `RunnerAction` for retry-free rescheduling — an explicit request to pull an action back and reschedule it on a different runner without incrementing `RetryCount`
- Implement idempotent `RetractAction()` on `RunnerAction` and `ComputeServiceSession`
- Add `POST jobs/{lsn}/retract` and `queues/{queueref}/jobs/{lsn}/retract` HTTP endpoints
- Add state machine documentation and per-state comments to `RunnerAction`
### Compute Race Fixes
- Fix race in `HandleActionUpdates` where actions enqueued between session abandon and scheduler tick were never abandoned, causing `GetActionResult` to return 202 indefinitely
- Fix queue `ActiveCount` race where `NotifyQueueActionComplete` was called after releasing `m_ResultsLock`, allowing callers to observe stale counters immediately after `GetActionResult` returned OK
### Logging Optimization and ANSI improvements
- Improve `AnsiColorStdoutSink` write efficiency — single write call, dirty-flag flush, `RwLock` instead of `std::mutex`
- Move ANSI color emission from sink into formatters via `Formatter::SetColorEnabled()`; remove `ColorRangeStart`/`End` from `LogMessage`
- Extract color helpers (`AnsiColorForLevel`, `StripAnsiSgrSequences`) into `helpers.h`
- Strip upstream ANSI SGR escapes in non-color output mode. This enables colour in log messages without polluting log files with ANSI control sequences
- Move `RotatingFileSink`, `JsonFormatter`, and `FullFormatter` from header-only to pimpl with `.cpp` files
### CLI / Exec Refactoring
- Extract `ExecSessionRunner` class from ~920-line `ExecUsingSession` into focused methods and a `ExecSessionConfig` struct
- Replace monolithic `ExecCommand` with subcommand-based architecture (`http`, `inproc`, `beacon`, `dump`, `buildlog`)
- Allow parent options to appear after subcommand name by parsing subcommand args permissively and forwarding unmatched tokens to the parent parser
### Testing Improvements
- Fix `--test-suite` filter being ignored due to accumulation with default wildcard filter
- Add test suite banners to test listener output
- Made `function.session.abandon_pending` test more robust
### Startup / Reliability Fixes
- Fix silent exit when a second zenserver instance detects a port conflict — use `ZEN_CONSOLE_*` for log calls that precede `InitializeLogging()`
- Fix two potential SIGSEGV paths during early startup: guard `sentry_options_new()` returning nullptr, and throw on `ZenServerState::Register()` returning nullptr instead of dereferencing
- Fail on unrecognized zenserver `--mode` instead of silently defaulting to store
### Other
- Show host details (hostname, platform, CPU count, memory) when discovering new compute workers
- Move frontend `html.zip` from source tree into build directory
- Add format specifications for Compact Binary and Compressed Buffer wire formats
- Add `WriteCompactBinaryObject` to zencore
- Extended `ConsoleTui` with additional functionality
- Add `--vscode` option to `xmake sln` for clangd / `compile_commands.json` support
- Disable compute/horde/nomad in release builds (not yet production-ready)
- Disable unintended `ASIO_HAS_IO_URING` enablement
- Fix crashpad patch missing leading whitespace
- Clean up code triggering gcc false positives
|
| |
|
|
|
|
|
|
|
|
| |
- **Frontend dashboard overhaul**: Unified compute/main dashboards into a single shared UI. Added new pages for cache, projects, metrics, sessions, info (build/runtime config, system stats). Added live-update via WebSockets with pause control, sortable detail tables, themed styling. Refactored compute/hub/orchestrator pages into modular JS.
- **HTTP server fixes and stats**: Fixed http.sys local-only fallback when default port is in use, implemented root endpoint redirect for http.sys, fixed Linux/Mac port reuse. Added /stats endpoint exposing HTTP server metrics (bytes transferred, request rates). Added WebSocket stats tracking.
- **OTEL/diagnostics hardening**: Improved OTLP HTTP exporter with better error handling and resilience. Extended diagnostics services configuration.
- **Session management**: Added new sessions service with HTTP endpoints for registering, updating, querying, and removing sessions. Includes session log file support. This is still WIP.
- **CLI subcommand support**: Added support for commands with subcommands in the zen CLI tool, with improved command dispatch.
- **Misc**: Exposed CPU usage/hostname to frontend, fixed JS compact binary float32/float64 decoding, limited projects displayed on front page to 25 sorted by last access, added vscode:// link support.
Also contains some fixes from TSAN analysis.
|
| |
|
| |
* Fix formatting of stat pages
|
| |
|
| |
* Avoid rendering user text input as HTML
|
| |
|
|
|
|
|
|
|
|
|
| |
* Method to get plain text from an async request
* Include server's version on the dashboard start page
* Same paragraph style as the rest of the method
* Updated changelog
* Update frontend archive
|
| |
|
|
| |
Bias for use of BigInt when consuming integer fields in compact binary to avoid values showing up as negative due to overflow on the Number type.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
downloads (#264)
* Single-column tables could overflow their maximum width
* Suffix oplog entry data's file name when downloading
* Renamed "view" link to "list"
* Ensure all undesirable characters are removed from page name
* Updated embedded frontend Zip archive
* Wrote some entries into the changelog
|
| |
|
|
|
|
|
|
|
|
|
| |
* Input boxes' text was unreadable when using the dark theme
* Change from margins to padding top/bottom - easier to reason about vertical styling.
* A treemap. Not used anywhere and not finished. Submitting so it isn't lost
* Prevent tables' first content columns from collapsing
* Dashboardk .zip archive update
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Wrong divisor for friendly giga-values
* We want Explorer style for kilo/kibi units; round up
* var -> const - zero idea if this matters
* Include sum of an entry's package data sizes in index
* Method to enurate all properties of a loaded oplog index
* Include bulkdata size in an oplog index
* Found a space that was missing
* Show package data sizes when viewing an oplog entry
* Navigating a component tree would error out at the end of the chain
* Parameterise friendly rounding
* Added size and rawsize columns to oplog tree view
* Sort of parameterised indexer's worker count and page size
* Right-align size columns on entry view page
* Updated frontend .zip archive
* A changelog update
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Turned tables and progress bars and friends into "widgets!"
* A step to abstracting away a page's the internal DOM structure
* Folded sector creation into Page and pivoted it to a widget host
* Try and keep start/count as numbers regardless of input
* No need for the entry table to be defined up front now
* Add op count and log sixe to oplog list page
* Cache left side toolbar object
* Bounds count page start when building list of oplog entrie
* Start/end navigation tools
* Build rest of entry page while waiting for indexer to load
* Consistent naming with other pages
* Spacially consolidate fetching code
* Hide fetch latency to speed up index generation workers
* Extract dashboard structure from zen.js monolith
* Fix breadcrumbs after restructuring
* Add view link to actions cell of oplogs list
* Generator to enumerate names of entries in indexer
* Methods for simple traversal of component relations
* is() to check if a component is of a certain type
* Extend attr() to get and unset a component's attributes
* Unsetting all styles of anchor tags was underisrable
* Restore page name as id of container element
* A tree view of an oplog
* Move helper class out to private module scope
* Small tweak to use left var that already exists
* Changelog update
* Updated frontend .zip archive
|
|
|
dependencies (#213)
* Consistent use of semicolons
* Added fallback if oplog entry assumptions do not hold
* 'marker' and 'expired' cells were incorrectly friendly
* Two spaces when there should only be one
* Robustness against .text(undefined) calls
* A single step into JavaScript modules
* Turned Fetcher into a module
* Friendly into a module
* Specialise Cbo field name comparison as TextDecoder() is very slow
* Prefer is_named() over get_name()
* Incorrect logic checking if a server reply was okay
* Try and make sure it's always numbers that flow through Friendly
* Added a progress bar component
* Swap key and package hash columns
* CbObject cloning
* Dark and light themes depending on browser settings
* Adjust styling of input boxes
* Add theme swatches to test page
* Turns out one can nest CSS selectors
* Separate swatch for links/actions
* Generate theme by lerping intermediate colours
* Clearer progress bar
* Chromium was complaining about label-less input elements
* Promise-based cache using an IndexedDb
* WebWorker for generating map of package ids to names
* Indexer class for building, loading, and saving map of ids to names
* Added links to oplog entries of an entry's dependencies
* This doesn't need to be decorated as async any longer
* Implemented oplog searching
* View and drop make no sense on package data payloads
* Rudimentary search result truncation
* Updated changelog
* Updated HTML zip archive
|