| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
| |
- Feature: Hub dashboard now shows a Resources tile with disk and memory usage against configured limits
- Feature: Hub module listing now shows state-change timestamps and duration for each instance
- Improvement: Hub provisioning rejects new instances when disk or memory usage exceeds configurable thresholds; limits are disabled by default (0 = no limit)
- `--hub-provision-disk-limit-bytes` - Reject provisioning when used disk exceeds this many bytes
- `--hub-provision-disk-limit-percent` - Reject provisioning when used disk exceeds this percentage of total disk
- `--hub-provision-memory-limit-bytes` - Reject provisioning when used memory exceeds this many bytes
- `--hub-provision-memory-limit-percent` - Reject provisioning when used memory exceeds this percentage of total RAM
- Improvement: Hub process metrics are now tracked atomically per active instance slot, eliminating per-query process handle lookups
- Improvement: Hub, Build Store, and Workspaces service stats sections in the dashboard are now collapsible
- Bugfix: Hub watchdog loop did not check `m_ShutdownFlag`, causing it to spin indefinitely on shutdown
|
| |
|
|
|
|
| |
- Feature: Added Workspaces dashboard page with HTTP request stats and per-workspace metrics
- Feature: Added Build Storage dashboard page with service-specific HTTP request stats
- Improvement: Front page now shows Hub and Object Store activity tiles; HTTP panel is fixed above the tiles grid
- Improvement: HTTP stats tiles now include 5m/15m rates and p999/max latency across all service pages
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
## Summary
This PR adds a session management service, several new dashboard pages, and a number of infrastructure improvements.
### Sessions Service
- `SessionsServiceClient` in `zenutil` announces sessions to a remote zenserver with a 15s heartbeat (POST/PUT/DELETE lifecycle)
- Storage server registers itself with its own local sessions service on startup
- Session mode attribute coupled to server mode (Compute, Proxy, Hub, etc.)
- Ended sessions tracked with `ended_at` timestamp; status filtering (Active/Ended/All)
- `--sessions-url` config option for remote session announcement
- In-process log sink (`InProcSessionLogSink`) forwards server log output to the server's own session, visible in the dashboard
### Session Log Viewer
- POST/GET endpoints for session logs (`/sessions/{id}/log`) supporting raw text and structured JSON/CbObject with batch `entries` array
- In-memory log storage per session (capped at 10k entries) with cursor-based pagination for efficient incremental fetching
- Log panel in the sessions dashboard with incremental DOM updates, auto-scroll (Follow toggle), newest-first toggle, text filter, and log-level coloring
- Auto-selects the server's own session on page load
### TCP Log Streaming
- `LogStreamListener` and `TcpLogStreamSink` for log delivery over TCP
- Sequence numbers on each message with drop detection and synthetic "dropped" notice on gaps
- Gathered buffer writes to reduce syscall overhead when flushing batches
- Tests covering basic delivery, multi-line splitting, drop detection, and sequencing
### New Dashboard Pages
- **Sessions**: master-detail layout with selectable rows, metadata panel, live WebSocket updates, paging, abbreviated date formatting, and "this" pill for the local session
- **Object Store**: summary stats tiles and bucket table with click-to-expand inline object listing (`GET /obj/`)
- **Storage**: per-volume disk usage breakdown (`GET /admin/storage`), Garbage Collection status section (next-run countdown, last-run stats), and GC History table with paginated rows and expandable detail panels
- **Network**: overview tiles, per-service request table, proxy connections, and live WebSocket updates; distinct client IPs and session counts via HyperLogLog
### Documentation Page
- In-dashboard Docs page with sidebar navigation, markdown rendering (via `marked`), Mermaid diagram support (theme-aware), collapsible sections, text filtering with highlighting, and cross-document linking
- New user-facing docs: `overview.md` (with architecture and per-mode diagrams), `sessions.md`, `cache.md`, `projects.md`; updated `compute.md`
- Dev docs moved to `docs/dev/`
### Infrastructure & Bug Fixes
- **Deflate compression** for the embedded frontend zip (~3.4MB → ~950KB); zlib inflate support added to `ZipFs` with cached decompressed buffers
- **Local IP addresses**: `GetLocalIpAddresses()` (Windows via `GetAdaptersAddresses`, Linux/Mac via `getifaddrs`); surfaced in `/status/status`, `/health/info`, and the dashboard banner
- **Dashboard nav**: unified into `zen-nav` web component with `MutationObserver` for dynamically added links, CSS `::part()` to merge banner/nav border radii, and prefix-based active link detection
- Stats broadcast refactored from manual JSON string concatenation to `CbObjectWriter`; `CbObject`-to-JS conversion improved for `TimeSpan`, `DateTime`, and large integers
- Stats WebSocket boilerplate consolidated into `ZenPage.connect_stats_ws()`
|
| |
|
| |
* add hub instance crash recovery
|
| |
|
|
|
|
|
|
|
|
| |
- Improvement: Hub dashboard module list improved
- Animated state dots, with flashing transitions for hibernating, waking, provisioning, and deprovisioning
- Per-row port used by the provisioned instance
- Per-row hibernate, wake, and deprovision actions with confirmation for destructive operations
- Per-row button to open the instance dashboard in a new window
- Multi-select with bulk hibernate/wake/deprovision and select-all
- Pagination at 50 lines per page
- Inline fold-out panel per row with human-readable process metrics
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a **transparent TCP proxy mode** to zenserver (activated via `zenserver proxy`), allowing it to sit between clients and upstream Zen servers to inspect and monitor HTTP/1.x traffic in real time. Primarily useful during development, to be able to observe multi-server/client interactions in one place.
- **Dedicated proxy port** -- Proxy mode defaults to port 8118 with its own data directory to avoid collisions with a normal zenserver instance.
- **TCP proxy core** (`src/zenserver/proxy/`) -- A new transparent TCP proxy that forwards connections to upstream targets, with support for both TCP/IP and Unix socket listeners. Multi-threaded I/O for connection handling. Supports Unix domain sockets for both upstream/downstream.
- **HTTP traffic inspection** -- Parses HTTP/1.x request/response streams inline to extract method, path, status, content length, and WebSocket upgrades without breaking the proxied data.
- **Proxy dashboard** -- A web UI showing live connection stats, per-target request counts, active connections, bytes transferred, and client IP/session ID rollups.
- **Server mode display** -- Dashboard banner now shows the running server mode (Zen Proxy, Zen Compute, etc.).
Supporting changes included in this branch:
- **Wildcard log level matching** -- Log levels can now be set per-category using wildcard patterns (e.g. `proxy.*=debug`).
- **`zen down --all`** -- New flag to shut down all running zenserver instances; also used by the new `xmake kill` task.
- Minor test stability fixes (flaky hash collisions, per-thread RNG seeds).
- Support ZEN_MALLOC environment variable for default allocator selection and switch default to rpmalloc
- Fixed sentry-native build to allow LTO on Windows
|
| |
|
|
|
|
|
|
|
|
| |
- **Frontend dashboard overhaul**: Unified compute/main dashboards into a single shared UI. Added new pages for cache, projects, metrics, sessions, info (build/runtime config, system stats). Added live-update via WebSockets with pause control, sortable detail tables, themed styling. Refactored compute/hub/orchestrator pages into modular JS.
- **HTTP server fixes and stats**: Fixed http.sys local-only fallback when default port is in use, implemented root endpoint redirect for http.sys, fixed Linux/Mac port reuse. Added /stats endpoint exposing HTTP server metrics (bytes transferred, request rates). Added WebSocket stats tracking.
- **OTEL/diagnostics hardening**: Improved OTLP HTTP exporter with better error handling and resilience. Extended diagnostics services configuration.
- **Session management**: Added new sessions service with HTTP endpoints for registering, updating, querying, and removing sessions. Includes session log file support. This is still WIP.
- **CLI subcommand support**: Added support for commands with subcommands in the zen CLI tool, with improved command dispatch.
- **Misc**: Exposed CPU usage/hostname to frontend, fixed JS compact binary float32/float64 decoding, limited projects displayed on front page to 25 sorted by last access, added vscode:// link support.
Also contains some fixes from TSAN analysis.
|
| |
|
|
|
|
|
|
|
|
| |
- Added local process runners for Linux/Wine, Mac with some sandboxing support
- Horde & Nomad provisioning for development and testing
- Client session queues with lifecycle management (active/draining/cancelled), automatic retry with configurable limits, and manual reschedule API
- Improved web UI for orchestrator, compute, and hub dashboards with WebSocket push updates
- Some security hardening
- Improved scalability and `zen exec` command
Still experimental - compute support is disabled by default
|
| | |
|
| | |
|
| |
|
| |
* Fix formatting of stat pages
|
| |
|
|
|
|
|
|
|
|
|
| |
* Save references to the project and zcache tables
* Add an attribute to table rows with the actionable project/namespace id
* Drop-all option for projects and cache namespaces
* Updated frontend .zip archive
* Edited changelog
|
| |
|
|
|
|
|
|
|
|
|
| |
* Method to get plain text from an async request
* Include server's version on the dashboard start page
* Same paragraph style as the rest of the method
* Updated changelog
* Update frontend archive
|
| |
|
|
|
|
|
|
|
|
|
| |
* Input boxes' text was unreadable when using the dark theme
* Change from margins to padding top/bottom - easier to reason about vertical styling.
* A treemap. Not used anywhere and not finished. Submitting so it isn't lost
* Prevent tables' first content columns from collapsing
* Dashboardk .zip archive update
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Wrong divisor for friendly giga-values
* We want Explorer style for kilo/kibi units; round up
* var -> const - zero idea if this matters
* Include sum of an entry's package data sizes in index
* Method to enurate all properties of a loaded oplog index
* Include bulkdata size in an oplog index
* Found a space that was missing
* Show package data sizes when viewing an oplog entry
* Navigating a component tree would error out at the end of the chain
* Parameterise friendly rounding
* Added size and rawsize columns to oplog tree view
* Sort of parameterised indexer's worker count and page size
* Right-align size columns on entry view page
* Updated frontend .zip archive
* A changelog update
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Turned tables and progress bars and friends into "widgets!"
* A step to abstracting away a page's the internal DOM structure
* Folded sector creation into Page and pivoted it to a widget host
* Try and keep start/count as numbers regardless of input
* No need for the entry table to be defined up front now
* Add op count and log sixe to oplog list page
* Cache left side toolbar object
* Bounds count page start when building list of oplog entrie
* Start/end navigation tools
* Build rest of entry page while waiting for indexer to load
* Consistent naming with other pages
* Spacially consolidate fetching code
* Hide fetch latency to speed up index generation workers
* Extract dashboard structure from zen.js monolith
* Fix breadcrumbs after restructuring
* Add view link to actions cell of oplogs list
* Generator to enumerate names of entries in indexer
* Methods for simple traversal of component relations
* is() to check if a component is of a certain type
* Extend attr() to get and unset a component's attributes
* Unsetting all styles of anchor tags was underisrable
* Restore page name as id of container element
* A tree view of an oplog
* Move helper class out to private module scope
* Small tweak to use left var that already exists
* Changelog update
* Updated frontend .zip archive
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
dependencies (#213)
* Consistent use of semicolons
* Added fallback if oplog entry assumptions do not hold
* 'marker' and 'expired' cells were incorrectly friendly
* Two spaces when there should only be one
* Robustness against .text(undefined) calls
* A single step into JavaScript modules
* Turned Fetcher into a module
* Friendly into a module
* Specialise Cbo field name comparison as TextDecoder() is very slow
* Prefer is_named() over get_name()
* Incorrect logic checking if a server reply was okay
* Try and make sure it's always numbers that flow through Friendly
* Added a progress bar component
* Swap key and package hash columns
* CbObject cloning
* Dark and light themes depending on browser settings
* Adjust styling of input boxes
* Add theme swatches to test page
* Turns out one can nest CSS selectors
* Separate swatch for links/actions
* Generate theme by lerping intermediate colours
* Clearer progress bar
* Chromium was complaining about label-less input elements
* Promise-based cache using an IndexedDb
* WebWorker for generating map of package ids to names
* Indexer class for building, loading, and saving map of ids to names
* Added links to oplog entries of an entry's dependencies
* This doesn't need to be decorated as async any longer
* Implemented oplog searching
* View and drop make no sense on package data payloads
* Rudimentary search result truncation
* Updated changelog
* Updated HTML zip archive
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* MSVC's std::fs::path doesn't like appending '/' separated path components
* Redirect '/dashboard' to '/dashboard/'
* Missed a few copyright headers
* Moved unescaped logo somewhere more suitable
* More robust catching and displaying of errors
* No need for the guard now or for waiting
* Formal access to a component's style
* Style tables explicitly without instead of via a CSS variable
* Highlight a row under the cursor to guide user's eye
* Not using css_var() so it was removed
* Add more detail to a project's list of oplogs
* Disabled test page's CbObject testing
* Consider all fields ending in "importedpackageids" as dependencies
* Don't wrap sector headers
* Package ids were derived with endianess back to front
* Moved oplog marker column further left
* Adopt a vararg-style to Table.add_row() for cell contents
* List and hotlink oplog entries' package data
* Modest control over how a table's columns are arranged
* Added tables to test column spacing
* Keep stat filter input box correctly up to date
* A clang-format run
* Updated html.zip
* validate found oplog directories when doing discover pass
---------
Co-authored-by: Dan Engelbrecht <[email protected]>
|
| |
|
|
|
|
|
| |
* Unselectable logo text
* Revisit alignment of complex request/bytes stats
* ZenStore logo takes you home
|
| |
|
| |
Self-hosted dashboard gets oplog entry view and a stats browser
|
| |
|
|
|
|
|
| |
* Separate modal dialog's background into its own div with own opacity
* Use :root and CSS vars for colours
* An html.zip refresh
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|