diff options
| author | Stefan Boberg <[email protected]> | 2026-04-20 14:10:10 +0200 |
|---|---|---|
| committer | GitHub Enterprise <[email protected]> | 2026-04-20 14:10:10 +0200 |
| commit | eb677209c0b837c3797c951ec1eb937965e40418 (patch) | |
| tree | ca48648423d621a5909888abcbb37ba86aa8ad73 /src/zencore/testing.cpp | |
| parent | add --pid och --executable till zen down command (#988) (diff) | |
| download | archived-zen-eb677209c0b837c3797c951ec1eb937965e40418.tar.xz archived-zen-eb677209c0b837c3797c951ec1eb937965e40418.zip | |
zen-test: add CLI integration harness + TestArtifactProvider + CI host stats (#985)
Establishes a new end-to-end integration test harness for the `zen` CLI, the shared fetcher it uses to pull test artifacts, and the CI plumbing that feeds both. Also lowers the default test-harness log level and broadens the artifact fetcher's credential resolution.
### `zen-test` executable (`src/zen-test/`)
- New binary modeled on `zenserver-test`, built only in debug.
- `zen-test.{h,cpp}` harness: spawns `zen.exe` via `CreateProc` and captures combined stdout/stderr into a `ZenCommandResult` for assertion.
- Registered with `scripts/test.lua` under the short name `zen` (`xmake test --run=zen`) and enabled for `--kill-stale-processes`.
- Prints a clear console message when invoked from a release build (tests disabled), so misconfiguration is easy to spot.
- Documented in `CLAUDE.md` (test-suite naming table + test projects section) and `README.md`.
- Test cases in the `zen.artifactprovider` suite:
- `probe.lyra_cook_rpc_recording` — probe against a canonical Lyra cook RPC recording that skips with a diagnostic `MESSAGE` when no artifact source is configured.
- `probe.s3_readme` — probes the configured S3 bucket for `README.md` using a fresh temp cache to force the request through to S3; skips on macOS without static creds (no EC2 Mac runners in our fleet).
- `zen.utility-cmd` suite: new integration tests exercising `zen print`, `zen wipe`, and `zen copy`.
### `TestArtifactProvider` (`src/zenutil/testartifactprovider.{h,cpp}`)
- `Ref<TestArtifactProvider>` factory returning a local-only or S3-backed provider, selected from env vars:
- `ZEN_TEST_ARTIFACTS_PATH` — local directory to serve from (write-through cache for remote fetches).
- `ZEN_TEST_ARTIFACTS_S3` — S3 URL to fetch from.
- `AWS_DEFAULT_REGION` / `AWS_REGION`, `AWS_ENDPOINT_URL`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN` — standard AWS config.
- `Exists(path)` / `Fetch(path)` API with a `TestArtifactFetchResult` return carrying the content buffer and a diagnostic error string. Content is cached on disk across test runs.
- **IMDS credential fallback**: when no static `AWS_ACCESS_KEY_ID` is present, attaches an `ImdsCredentialProvider` so self-hosted EC2 runners with an attached IAM role can sign S3 requests without static credentials (mirrors the pattern in `zenserver/hub/hydration.cpp`).
- **IMDS opt-out**: honors the standard `AWS_EC2_METADATA_DISABLED=true` env var, and skips IMDS by default on macOS where the link-local probe would just emit noise.
### Test harness log level (`src/zencore/testing.cpp`)
- `TestRunner::ApplyCommandLine` now defaults the global log level to `Info` (was effectively `Trace`), cutting the noise from `xmake test --run=all` now that the suite has grown. Applies uniformly to `zencore-test`, `zenhttp-test`, `zenstore-test`, `zenutil-test`, `zenserver-test`, `zen-test`, etc. `--debug` (Debug) and `--verbose` (Trace) still opt back in when chasing failures.
### CI (`.github/workflows/validate.yml`)
- **Runner info step** on all three platforms (Windows/Linux/macOS): prints host, CPU topology, memory, and disk usage before the build/test step, so flakes that correlate with a particular runner or low disk space are easy to spot.
- **Artifact env wiring**: passes `ZEN_TEST_ARTIFACTS_S3` and `AWS_DEFAULT_REGION` into the debug Build & Test step on all three platforms so the probe can reach its source when the repo variable is configured. The probe skips cleanly when unset.
Diffstat (limited to 'src/zencore/testing.cpp')
| -rw-r--r-- | src/zencore/testing.cpp | 4 |
1 files changed, 4 insertions, 0 deletions
diff --git a/src/zencore/testing.cpp b/src/zencore/testing.cpp index 67285dcf1..20a53bff3 100644 --- a/src/zencore/testing.cpp +++ b/src/zencore/testing.cpp @@ -279,6 +279,10 @@ TestRunner::ApplyCommandLine(int Argc, char const* const* Argv, const char* Defa m_Impl->Session.applyCommandLine(Argc, Argv); + // Tests default to Info so that common runs aren't buried in debug/trace output. + // Use --debug or --verbose to opt back in when investigating a failure. + zen::logging::SetLogLevel(zen::logging::Info); + for (int i = 1; i < Argc; ++i) { if (Argv[i] == "--debug"sv) |