hub async s3 client (#1024)

- Feature: `AsyncHttpClient` adds cancellable request tokens, streaming GET to a file (`AsyncDownload`), zero-copy chunk-callback GET (`AsyncStream`), pull-mode body source for streaming `AsyncPut`, retry layer mirroring the synchronous client, and a submit-side in-flight cap (`HttpClientSettings::MaxConcurrentRequests`) so hub-scale fanout against a single host cannot stall queued handles into curl's connect-timeout window
- Feature: Hub hydration can route S3 transfers through a non-blocking `AsyncHttpClient` (curl_multi + asio) backed by a single io thread; hydrate and dehydrate now pipeline requests instead of blocking worker threads
  - `--hub-hydration-async-enabled` (Lua: `hub.hydration.async.enabled`, default true)
  - `--hub-hydration-async-max-concurrent-requests` (Lua: `hub.hydration.async.maxconcurrentrequests`, default `clamp(cpu*4, 128, 512)`)
- Feature: Hub provision/deprovision/obliterate now run as two phases on separate worker pools so per-module hydration cannot starve child-process spawn/despawn (and vice versa)
  - New `--hub-instance-spawn-threads` (Lua: `hub.instance.spawnthreads`, default `clamp(cpu/8, 4, 16)`) drives child-process spawn/despawn
  - `--hub-instance-provision-threads` (Lua: `hub.instance.provisionthreads`) now drives per-module hydrate/dehydrate scheduling only; default changed from `max(cpu/4, 2)` to `clamp(cpu/8, 4, 12)`
  - `--hub-hydration-threads` (Lua: `hub.hydration.threads`) now controls per-file workers inside a single hydrate/dehydrate; default changed from `max(cpu/4, 2)` to `clamp(cpu/8, 4, 12)`
- Feature: `AsyncHttpClient` owns its `asio::io_context` and one io thread by default; the `(BaseUri, io_context&)` constructor is preserved for callers that want to share an externally-driven `io_context` across clients (caller MUST keep the loop running until the client destructs)
- Feature: `Hub::Configuration` C++ struct fields renamed (`OptionalProvisionWorkerPool`/`OptionalHydrationWorkerPool` -> `OptionalProvisionPool`/`OptionalSpawnPool`/`OptionalHydrationPool`). Embedders constructing `Hub` directly must update field names; provision and spawn pools must both be set or both null (asserted at construction).
- Bugfix: `S3Client` signing-key cache no longer returns stale signatures after IMDS-rotated credentials change `AccessKeyId`; cache is now keyed on `(DateStamp, AccessKeyId)`

1 files changed, 43 insertions, 0 deletions

diff --git a/src/zenhttp/httpclient_test.cpp b/src/zenhttp/httpclient_test.cpp
index b0e097a54..ea73ff7a3 100644
--- a/src/zenhttp/httpclient_test.cpp
+++ b/src/zenhttp/httpclient_test.cpp
@@ -300,6 +300,49 @@ struct TestServerFixture
 
 TEST_SUITE_BEGIN("http.httpclient");
 
+TEST_CASE("httpclient.response.findheader.arena")
+{
+	// Async client populates HeaderArena (raw "Key: Value\r\n" bytes); FindHeader
+	// scans it lazily without building the full KeyValueMap. Exercise arena scan
+	// path independent of any live HTTP transfer.
+	HttpClient::Response Resp;
+	Resp.HeaderArena =
+		"Content-Type: application/json\r\n"
+		"ETag: \"abc-123\"\r\n"
+		"Content-Length: 42\r\n"
+		"X-Amz-Request-Id: deadbeef\r\n"
+		"\r\n";
+
+	CHECK_EQ(Resp.FindHeader("Content-Type"), "application/json");
+	CHECK_EQ(Resp.FindHeader("etag"), "\"abc-123\"");	// case-insensitive
+	CHECK_EQ(Resp.FindHeader("CONTENT-LENGTH"), "42");	// case-insensitive
+	CHECK_EQ(Resp.FindHeader("x-amz-request-id"), "deadbeef");
+	CHECK_EQ(Resp.FindHeader("missing"), "");  // absent
+	CHECK_EQ(Resp.FindHeader(""), "");		   // empty name
+}
+
+TEST_CASE("httpclient.response.findheader.map_fallback")
+{
+	// Sync client populates Header KeyValueMap; FindHeader falls back to it
+	// when HeaderArena is empty.
+	HttpClient::Response Resp;
+	Resp.Header->insert_or_assign("Content-Type", "text/plain");
+	Resp.Header->insert_or_assign("ETag", "\"xyz\"");
+
+	CHECK_EQ(Resp.FindHeader("content-type"), "text/plain");
+	CHECK_EQ(Resp.FindHeader("ETag"), "\"xyz\"");
+	CHECK_EQ(Resp.FindHeader("missing"), "");
+}
+
+TEST_CASE("httpclient.response.findheader.arena_takes_priority")
+{
+	// If both populated (unusual), arena is scanned first.
+	HttpClient::Response Resp;
+	Resp.HeaderArena = "ETag: from-arena\r\n";
+	Resp.Header->insert_or_assign("ETag", "from-map");
+	CHECK_EQ(Resp.FindHeader("ETag"), "from-arena");
+}
+
 TEST_CASE("httpclient.verbs")
 {
 	TestServerFixture Fixture;