initial CLAUDE.md (#764)

1 files changed, 271 insertions, 0 deletions

diff --git a/CLAUDE.md b/CLAUDE.md
new file mode 100644
index 000000000..757ce5d2a
--- /dev/null
+++ b/CLAUDE.md
@@ -0,0 +1,271 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+Unreal Zen Storage Service (zenserver) is a high-performance local storage service for UE5, managing secondary data such as cooker output and local caches (DDC - Derived Data Cache). Can be deployed as a daemon on user machines or as a shared cache instance for build farms.
+
+## Build System
+
+This project uses **xmake** as its build system. xmake is stateful - run configuration before building.
+
+### Common Build Commands
+
+```bash
+# Configure for debug (includes tests)
+xmake config -m debug
+
+# Configure for release
+xmake config -m release
+
+# Build everything
+xmake
+
+# Build specific target
+xmake build zenserver
+xmake build zen
+
+# Run targets
+xmake run zenserver
+xmake run zen
+
+# Generate IDE project files
+xmake sln                    # Windows: Visual Studio 2022
+xmake project -k xcode       # macOS: Xcode
+```
+
+### Build Configuration Options
+
+- `--zensentry=yes|no` - Enable/disable Sentry crash reporting (default: yes)
+- `--zenmimalloc=yes|no` - Use mimalloc for memory management (default: yes, disabled with ASAN)
+- `--zenrpmalloc=yes|no` - Use rpmalloc for memory management (default: yes)
+- `--httpsys=yes|no` - Enable http.sys server (Windows only, default: yes)
+- `--zencompute=yes|no` - Enable compute services endpoint (default: yes)
+- `--zenmemtrack=yes|no` - Enable UE Memory Trace support (Windows only, default: yes)
+- `--zentrace=yes|no` - Enable UE Trace support (default: yes)
+
+### Testing
+
+```bash
+# Tests are only built in debug mode
+xmake config -m debug
+xmake
+
+# Run all tests
+xmake test --run=all
+
+# Run specific test suites
+xmake test --run=core          # zencore-test
+xmake test --run=store         # zenstore-test
+xmake test --run=server        # zenserver test command
+xmake test --run=integration   # zenserver-test
+xmake test --run=http          # zenhttp-test
+xmake test --run=util          # zenutil-test
+xmake test --run=remotestore   # zenremotestore-test
+
+# Run tests with JUnit reporting
+xmake test --run=all --junit
+```
+
+Tests use the **doctest** framework. To run server tests: `xmake run zenserver test`
+
+### Code Formatting
+
+```bash
+# Format all files (requires pre-commit)
+xmake precommit
+
+# Or use pre-commit directly
+pre-commit run --all-files
+```
+
+Install pre-commit hooks with `pre-commit install` to auto-format on commit.
+
+### Cleaning Build State
+
+When encountering build issues, clean all build state:
+
+**Windows:**
+```cmd
+rmdir /s /q .xmake build %LOCALAPPDATA%\.xmake %TEMP%\.xmake
+```
+
+**Linux/macOS:**
+```bash
+rm -rf .xmake build ~/.xmake
+```
+
+## Architecture
+
+### Module Structure
+
+The codebase is organized into layered modules with clear dependencies:
+
+**Core Libraries (Foundation Layer):**
+- `zenbase` - Platform abstraction primitives
+- `zencore` - Core utilities, memory management, logging, filesystem, crypto, threading
+- `zenutil` - Higher-level utilities and common patterns
+
+**Protocol & Network Layer:**
+- `zenhttp` - HTTP server implementations (http.sys on Windows, ASIO-based on Linux/Mac)
+- `zennet` - Network utilities and protocols
+- `transports` - Transport SDK and implementations
+
+**Storage Layer:**
+- `zenstore` - Core storage engine and key-value store
+- `zenvfs` - Virtual filesystem implementation
+- `zenremotestore` - Remote storage client
+
+**Service Layer:**
+- `zenserver` - Main server binary integrating all services
+- `zencompute` - Compute services endpoint
+- `zentelemetry` - Telemetry and metrics collection
+
+**Client Tools:**
+- `zen` - CLI client utility for interacting with zenserver
+  - Commands are in `src/zen/cmds/` (e.g., `cache_cmd.h`, `status_cmd.h`, `ui_cmd.h`)
+  - Each command inherits from base command classes and registers with the CLI dispatcher
+
+**Test Projects:**
+- `*-test` modules contain tests for their corresponding module (only built in debug mode)
+- `zenserver-test` contains integration tests
+
+### Key Architectural Patterns
+
+**HTTP Server Implementations:**
+- Windows: Uses http.sys kernel driver for highest throughput (requires elevation or URL reservation)
+- Linux/macOS: ASIO-based HTTP server (also available on Windows)
+- Server selection is compile-time based on platform but can be overridden on the command line via `--http=asio/httpsys/etc`
+
+**Storage Architecture:**
+- zenserver manages multiple storage services (build store, cache, project store, object store)
+- Content-addressable storage with deduplication on the lowest storage layer but this is mostly an internal implementation detail. 
+  - Clients generally do not request content via content address. Instead they interface with the higher level storage services
+  - CAS content is garbage collected based on what is referenced from of higher level services
+
+**Frontend:**
+- Web UI bundled as ZIP in `src/zenserver/frontend/*.zip`
+- Dashboards for hub, orchestrator, and compute services are located in `src/zenserver/frontent/html/`
+  - These are the files which end up being bundled into the front-end zip mentioned above
+- Update with `xmake updatefrontend` after modifying HTML/JS, and check in the resulting zip
+
+**Memory Management:**
+- Can use mimalloc or rpmalloc for performance
+- UE-style LLM (Low-Level Memory) tracking available on Windows
+- Memory tracing support integrated with UE trace system
+
+**Tracing:**
+- UE Trace integration for profiling and diagnostics
+- Trace files use .utrace format
+- Trace analysis tools in zentrace module (if available)
+- Traces can be visualized using Unreal Insights, available in the UE5 code base
+
+## Coding Standards
+
+**Naming Conventions (UE-inspired with modifications):**
+- Classes/Structs: `PascalCase`
+- Functions/Methods: `PascalCase()`
+- Member variables: `m_PascalCase`
+- Global variables: `g_PascalCase`
+- Static variables: `s_PascalCase`
+- Thread-local variables: `t_PascalCase`
+
+**Note:** Unlike UE, no `F` prefix on structs/classes is required or encouraged. Also, no `b` prefix on booleans.
+
+**Code Style:**
+- C++20 standard
+- clang-format enforced via pre-commit hooks
+- Use `std` containers; `eastl::fixed_vector` etc. for performance-critical paths
+- Exceptions used only for unexpected errors, not flow control
+- Logging is done via `ZEN_DEBUG(...)`, `ZEN_INFO(...)`, `ZEN_WARN`, `ZEN_ERROR` macros
+  - Logging macros use `fmt::format` for message formatting. The first argument is the format string, which
+    must be a string literal (turned into `std::string_view` in the macros)
+  - Logging channels can be overridden on a scope-by-scope basis (per-class or even per-function) by implementing 
+    a `LoggerRef Log()` function
+
+**Includes:**
+- Wrap third-party includes with `ZEN_THIRD_PARTY_INCLUDES_START` / `ZEN_THIRD_PARTY_INCLUDES_END`
+  - This helps avoid spurious compiler warnings due to the use of strict warnings-as-error policy
+
+**Platform Macros:**
+- `ZEN_PLATFORM_WINDOWS`, `ZEN_PLATFORM_LINUX`, `ZEN_PLATFORM_MAC`
+- Use `is_plat("windows")` etc. in xmake.lua files
+
+## Security Considerations
+
+**XSS in Web UI:**
+Sanitize all dynamic data before inserting into innerHTML. Use `escapeHtml()` helper functions in JavaScript.
+
+**http.sys URL Reservation:**
+On Windows, zenserver requires elevation or a URL reservation to bind http.sys to network interfaces:
+```cmd
+netsh http add urlacl url=http://*:8558/ user=<username>
+```
+
+## Platform-Specific Notes
+
+**Windows:**
+- Primary development platform
+- Visual Studio 2022+ required (C++20 features)
+- Static CRT linking by default (`/MT` in release, `/MTd` in debug)
+- Recommended profilers: Superluminal Performance, Visual Studio profiler
+
+**Linux:**
+- Requires GCC-11+ or Clang-12+ with libstdc++-11+
+- Set `CXX=g++-11` before running xmake if needed
+- UE cross-compile toolchain support via `UE_TOOLCHAIN_DIR` environment variable
+
+**macOS:**
+- Requires Xcode or Xcode command-line tools
+- Supports both x86_64 and arm64 architectures
+
+## Dependencies
+
+Key third-party libraries (managed by xmake):
+- EASTL - High-performance containers
+- ASIO - Async I/O
+- spdlog/fmt - Logging and formatting
+- doctest - Testing framework
+- cxxopts - Command-line parsing
+- json11 - JSON parsing
+- lz4 - Compression
+- xxhash - Hashing
+- OpenSSL (Linux/Mac) - Crypto operations
+- Sentry - Crash reporting (optional)
+
+## Adding New Commands to `zen` CLI
+
+1. Create `src/zen/cmds/mycommand_cmd.h` and `src/zen/cmds/mycommand_cmd.cpp`
+2. Inherit from appropriate base command class
+3. Implement `Execute()` method
+4. Add `#include "cmds/mycommand_cmd.h"` to `src/zen/zen.cpp`
+5. Register command in `zen.cpp` command dispatcher. These should be ordered in alphabetical identifier order for consistent merging
+
+## Adding New Modules
+
+1. Create `src/mymodule/` directory structure
+2. Add `include/` subdirectory for public headers
+3. Create `xmake.lua` defining the target
+4. Add `includes("src/mymodule")` to root `xmake.lua`
+5. Add dependencies via `add_deps()` in the module's xmake.lua
+
+## Debugging
+
+**Visual Studio:**
+- Use Debug configuration (includes test code)
+- For http.sys debugging, run Visual Studio as Administrator or add URL reservation
+- Enable child process debugging with Microsoft Child Process Debugging Power Tool
+
+**Multi-process Scenarios:**
+When debugging zenserver-test or other multi-process scenarios, use child process debugging tools to attach to spawned processes automatically.
+
+## Bundle Creation
+
+```bash
+# Create deployable ZIP bundle
+xmake bundle
+
+# Update frontend ZIP after HTML changes
+xmake updatefrontend
+```