diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/WindowsOnLinux.md | 109 | ||||
| -rw-r--r-- | docs/compute.md | 79 |
2 files changed, 174 insertions, 14 deletions
diff --git a/docs/WindowsOnLinux.md b/docs/WindowsOnLinux.md new file mode 100644 index 000000000..540447cb2 --- /dev/null +++ b/docs/WindowsOnLinux.md @@ -0,0 +1,109 @@ +# Cross-Compiling Windows Binaries on Linux + +This document describes how to build `zenserver.exe` and `zen.exe` for Windows +from a Linux host using clang-cl, lld-link, and the Windows SDK fetched via +[xwin](https://github.com/Jake-Shadle/xwin). + +## Prerequisites + +- **LLVM/Clang 17+** with the following tools (typically in `/usr/lib/llvm-<ver>/bin/`): + - `clang` (used as `clang-cl` via `--driver-mode=cl`) + - `lld-link` + - `llvm-lib` + - `llvm-rc` + - `llvm-ml` + - `llvm-mt` + + On Ubuntu/Debian: + ```bash + sudo apt install clang-18 lld-18 llvm-18 + ``` + +- **xmake** (build system) — see [xmake.io](https://xmake.io) for installation. + +## Step 1: Fetch the Windows SDK + +Run the provided script to download the Windows SDK and MSVC CRT headers/libs: + +```bash +./scripts/win_cross/get_win_sdk.sh +``` + +This downloads the SDK to `~/.xwin-sdk` by default. Override the location with +an argument or the `$XWIN_SDK_DIR` environment variable: + +```bash +./scripts/win_cross/get_win_sdk.sh /path/to/sdk +``` + +The script also creates: +- Tool wrapper scripts (clang-cl, lld-link, llvm-lib, etc.) in the SDK `bin/` directory +- An MSVC-compatible directory layout that xmake's built-in clang-cl toolchain can discover +- Debug CRT lib symlinks so cmake package builds succeed + +## Step 2: Configure and Build + +```bash +xmake config -y -p windows -a x64 \ + --toolchain=clang-cl \ + --sdk=$HOME/.xwin-sdk \ + --mrc=$HOME/.xwin-sdk/bin/x64/rc.exe \ + -m release \ + --zensentry=n \ + --zenmimalloc=n + +xmake +``` + +### Notes + +- **`--sdk` must use an absolute path** — `$HOME/.xwin-sdk` works, but + `~/.xwin-sdk` does not (xmake doesn't expand `~`). +- **`--mrc`** points to the resource compiler wrapper. Without it, xmake can't + auto-detect `llvm-rc` (it doesn't support `--version`). +- **`--zensentry=no`** is recommended — crashpad (used by sentry-native) is + difficult to cross-compile. +- **`--zenmimalloc=no`** is recommended initially to reduce the package surface. +- **LTO is automatically disabled** for cross-compilation builds. + +## Step 3: Verify + +```bash +file build/windows/x64/release/zenserver.exe +# PE32+ executable (console) x86-64, for MS Windows, 12 sections + +file build/windows/x64/release/zen.exe +# PE32+ executable (console) x86-64, for MS Windows, 12 sections +``` + +## Running with Wine (optional) + +The resulting binaries can be run under Wine for testing: + +```bash +wine build/windows/x64/release/zenserver.exe --help +``` + +## Troubleshooting + +### Library not found (case sensitivity) + +The Windows SDK ships headers and libs with specific casing (e.g. `DbgHelp.h`, +`ws2_32.lib`). Linux filesystems are case-sensitive, so `#include <Dbghelp.h>` +won't find `DbgHelp.h`. The codebase uses lowercase includes where possible. If +you encounter a missing header/lib, check the actual casing in the SDK directory. + +### `add_ldflags(...) is ignored` + +xmake's auto flag checker may reject MSVC linker flags when using clang-cl. Add +`{force = true}` to the `add_ldflags()` call. + +### Stale build state + +If you hit unexpected errors after changing toolchain settings, clean everything: + +```bash +rm -rf .xmake build ~/.xmake +``` + +Then reconfigure and rebuild. diff --git a/docs/compute.md b/docs/compute.md index 417622f94..df8a22870 100644 --- a/docs/compute.md +++ b/docs/compute.md @@ -122,31 +122,82 @@ functions: version: '83027356-2cf7-41ca-aba5-c81ab0ff2129' ``` -## API (WIP not final) +## API -The compute interfaces are currently exposed on the `/apply` endpoint but this -will be subject to change as we adapt the interfaces during development. The LSN +The compute interfaces are exposed on the `/compute` endpoint. The LSN APIs below are intended to replace the action ID oriented APIs. The POST APIs typically involve a two-step dance where a descriptor is POSTed and -the service responds with a list of `needs` chunks (identified via `IoHash`) which -it does not have yet. The client can then follow up with a POST of a Compact Binary +the service responds with a list of `needs` chunks (identified via `IoHash`) which +it does not have yet. The client can then follow up with a POST of a Compact Binary Package containing the descriptor along with the needed chunks. -`/apply/ready` - health check endpoint returns HTTP 200 OK or HTTP 503 +`/compute/ready` - health check endpoint returns HTTP 200 OK or HTTP 503 -`/apply/sysinfo` - system information endpoint +`/compute/sysinfo` - system information endpoint -`/apply/record/start`, `/apply/record/stop` - start/stop action recording +`/compute/record/start`, `/compute/record/stop` - start/stop action recording -`/apply/workers/{worker}` - GET/POST worker descriptors and payloads +`/compute/workers/{worker}` - GET/POST worker descriptors and payloads -`/apply/jobs/completed` - GET list of completed actions +`/compute/jobs/completed` - GET list of completed actions -`/apply/jobs/{lsn}` - GET completed action results from LSN, POST action cancellation by LSN, priority changes by LSN +`/compute/jobs/{lsn}` - GET completed action results from LSN, POST action cancellation by LSN, priority changes by LSN -`/apply/jobs/{worker}/{action}` - GET completed action (job) results by action ID +`/compute/jobs/{worker}/{action}` - GET completed action (job) results by action ID -`/apply/jobs/{worker}` - GET pending/running jobs for worker, POST requests to schedule action as a job +`/compute/jobs/{worker}` - GET pending/running jobs for worker, POST requests to schedule action as a job -`/apply/jobs` - POST request to schedule action as a job +`/compute/jobs` - POST request to schedule action as a job + +### Queues + +Queues provide a way to logically group actions submitted by a client session. This enables +per-session cancellation and completion polling without affecting actions submitted by other +sessions. + +#### Local access (integer ID routes) + +These routes use sequential integer queue IDs and are restricted to local (loopback) +connections only. Remote requests receive HTTP 403 Forbidden. + +`/compute/queues` - POST to create a new queue. Returns a `queue_id` which is used to +reference the queue in subsequent requests. + +`/compute/queues/{queue}` - GET queue status (active, completed, failed, and cancelled +action counts, plus `is_complete` flag indicating all actions have finished). DELETE to +cancel all pending and running actions in the queue. + +`/compute/queues/{queue}/completed` - GET list of completed action LSNs for this queue +whose results have not yet been retired. A queue-scoped alternative to `/compute/jobs/completed`. + +`/compute/queues/{queue}/jobs` - POST to submit an action to a queue with automatic worker +resolution. Accepts an optional `priority` query parameter. + +`/compute/queues/{queue}/jobs/{worker}` - POST to submit an action to a queue targeting a +specific worker. Accepts an optional `priority` query parameter. + +`/compute/queues/{queue}/jobs/{lsn}` - GET action result by LSN, scoped to the queue + +#### Remote access (OID token routes) + +These routes use cryptographically generated 24-character hex tokens (OIDs) instead of +integer queue IDs. Tokens are unguessable and safe to use over the network. The token +mapping lives entirely in the HTTP service layer; the underlying compute service only +knows about integer queue IDs. + +`/compute/queues/remote` - POST to create a new queue with token-based access. Returns +`queue_token` (24-char hex string) and `queue_id` (integer, for internal visibility). + +`/compute/queues/{oidtoken}` - GET queue status or DELETE to cancel, same semantics as +the integer ID variant but using the OID token for identification. + +`/compute/queues/{oidtoken}/completed` - GET list of completed action LSNs for this queue. + +`/compute/queues/{oidtoken}/jobs` - POST to submit an action to a queue with automatic +worker resolution. + +`/compute/queues/{oidtoken}/jobs/{worker}` - POST to submit an action targeting a specific +worker. + +`/compute/queues/{oidtoken}/jobs/{lsn}` - GET action result by LSN, scoped to the queue |