path: root/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

Unreal Zen Storage Service (zenserver) is a high-performance local storage service for UE5, managing secondary data such as cooker output and local caches (DDC - Derived Data Cache). Can be deployed as a daemon on user machines or as a shared cache instance for build farms.

## Build System

This project uses **xmake** as its build system. xmake is stateful - run configuration before building.

### Common Build Commands

```bash
# Configure for debug (includes tests)
xmake config -m debug

# Configure for release
xmake config -m release

# Build everything
xmake

# Build specific target
xmake build zenserver
xmake build zen

# Run targets
xmake run zenserver
xmake run zen

# Generate IDE project files
xmake sln                    # Windows: Visual Studio 2022
xmake project -k xcode       # macOS: Xcode
```

### Build Configuration Options

- `--zensentry=yes|no` - Enable/disable Sentry crash reporting (default: yes)
- `--zenmimalloc=yes|no` - Use mimalloc for memory management (default: yes, disabled with ASAN)
- `--zenrpmalloc=yes|no` - Use rpmalloc for memory management (default: yes)
- `--httpsys=yes|no` - Enable http.sys server (Windows only, default: yes)
- `--zencompute=yes|no` - Enable compute services endpoint (default: yes)
- `--zenmemtrack=yes|no` - Enable UE Memory Trace support (Windows only, default: yes)
- `--zentrace=yes|no` - Enable UE Trace support (default: yes)

### Testing

```bash
# Tests are only built in debug mode
xmake config -m debug
xmake

# Run all tests
xmake test --run=all

# Run specific test suites
xmake test --run=core          # zencore-test
xmake test --run=store         # zenstore-test
xmake test --run=server        # zenserver test command
xmake test --run=integration   # zenserver-test
xmake test --run=http          # zenhttp-test
xmake test --run=util          # zenutil-test
xmake test --run=remotestore   # zenremotestore-test

# Run tests with JUnit reporting
xmake test --run=all --junit
```

Tests use the **doctest** framework. To run server tests: `xmake run zenserver test`

### Code Formatting

```bash
# Format all files (requires pre-commit)
xmake precommit

# Or use pre-commit directly
pre-commit run --all-files
```

Install pre-commit hooks with `pre-commit install` to auto-format on commit.

### Cleaning Build State

When encountering build issues, clean all build state:

**Windows:**
```cmd
rmdir /s /q .xmake build %LOCALAPPDATA%\.xmake %TEMP%\.xmake
```

**Linux/macOS:**
```bash
rm -rf .xmake build ~/.xmake
```

## Architecture

### Module Structure

The codebase is organized into layered modules with clear dependencies:

**Core Libraries (Foundation Layer):**
- `zenbase` - Platform abstraction primitives
- `zencore` - Core utilities, memory management, logging, filesystem, crypto, threading
- `zenutil` - Higher-level utilities and common patterns

**Protocol & Network Layer:**
- `zenhttp` - HTTP server implementations (http.sys on Windows, ASIO-based on Linux/Mac)
- `zennet` - Network utilities and protocols
- `transports` - Transport SDK and implementations

**Storage Layer:**
- `zenstore` - Core storage engine and key-value store
- `zenvfs` - Virtual filesystem implementation
- `zenremotestore` - Remote storage client

**Service Layer:**
- `zenserver` - Main server binary integrating all services
- `zencompute` - Compute services endpoint
- `zentelemetry` - Telemetry and metrics collection

**Client Tools:**
- `zen` - CLI client utility for interacting with zenserver
  - Commands are in `src/zen/cmds/` (e.g., `cache_cmd.h`, `status_cmd.h`, `ui_cmd.h`)
  - Each command inherits from base command classes and registers with the CLI dispatcher

**Test Projects:**
- `*-test` modules contain tests for their corresponding module (only built in debug mode)
- `zenserver-test` contains integration tests

### Key Architectural Patterns

**HTTP Server Implementations:**
- Windows: Uses http.sys kernel driver for highest throughput (requires elevation or URL reservation)
- Linux/macOS: ASIO-based HTTP server (also available on Windows)
- Server selection is compile-time based on platform but can be overridden on the command line via `--http=asio/httpsys/etc`

**Storage Architecture:**
- zenserver manages multiple storage services (build store, cache, project store, object store)
- Content-addressable storage with deduplication on the lowest storage layer but this is mostly an internal implementation detail. 
  - Clients generally do not request content via content address. Instead they interface with the higher level storage services
  - CAS content is garbage collected based on what is referenced from of higher level services

**Frontend:**
- Web UI bundled as ZIP in `src/zenserver/frontend/*.zip`
- Dashboards for hub, orchestrator, and compute services are located in `src/zenserver/frontent/html/`
  - These are the files which end up being bundled into the front-end zip mentioned above
- Update with `xmake updatefrontend` after modifying HTML/JS, and check in the resulting zip

**Memory Management:**
- Can use mimalloc or rpmalloc for performance
- UE-style LLM (Low-Level Memory) tracking available on Windows
- Memory tracing support integrated with UE trace system

**Tracing:**
- UE Trace integration for profiling and diagnostics
- Trace files use .utrace format
- Trace analysis tools in zentrace module (if available)
- Traces can be visualized using Unreal Insights, available in the UE5 code base

## Coding Standards

**Naming Conventions (UE-inspired with modifications):**
- Classes/Structs: `PascalCase`
- Functions/Methods: `PascalCase()`
- Member variables: `m_PascalCase`
- Global variables: `g_PascalCase`
- Static variables: `s_PascalCase`
- Thread-local variables: `t_PascalCase`

**Note:** Unlike UE, no `F` prefix on structs/classes is required or encouraged. Also, no `b` prefix on booleans.

**Code Style:**
- C++20 standard
- clang-format enforced via pre-commit hooks
- Use `std` containers; `eastl::fixed_vector` etc. for performance-critical paths
- Exceptions used only for unexpected errors, not flow control
- Logging is done via `ZEN_DEBUG(...)`, `ZEN_INFO(...)`, `ZEN_WARN`, `ZEN_ERROR` macros
  - Logging macros use `fmt::format` for message formatting. The first argument is the format string, which
    must be a string literal (turned into `std::string_view` in the macros)
  - Logging channels can be overridden on a scope-by-scope basis (per-class or even per-function) by implementing 
    a `LoggerRef Log()` function

**Includes:**
- Wrap third-party includes with `ZEN_THIRD_PARTY_INCLUDES_START` / `ZEN_THIRD_PARTY_INCLUDES_END`
  - This helps avoid spurious compiler warnings due to the use of strict warnings-as-error policy

**Platform Macros:**
- `ZEN_PLATFORM_WINDOWS`, `ZEN_PLATFORM_LINUX`, `ZEN_PLATFORM_MAC`
- Use `is_plat("windows")` etc. in xmake.lua files

## Security Considerations

**XSS in Web UI:**
Sanitize all dynamic data before inserting into innerHTML. Use `escapeHtml()` helper functions in JavaScript.

**http.sys URL Reservation:**
On Windows, zenserver requires elevation or a URL reservation to bind http.sys to network interfaces:
```cmd
netsh http add urlacl url=http://*:8558/ user=<username>
```

## Platform-Specific Notes

**Windows:**
- Primary development platform
- Visual Studio 2022+ required (C++20 features)
- Static CRT linking by default (`/MT` in release, `/MTd` in debug)
- Recommended profilers: Superluminal Performance, Visual Studio profiler

**Linux:**
- Requires GCC-11+ or Clang-12+ with libstdc++-11+
- Set `CXX=g++-11` before running xmake if needed
- UE cross-compile toolchain support via `UE_TOOLCHAIN_DIR` environment variable

**macOS:**
- Requires Xcode or Xcode command-line tools
- Supports both x86_64 and arm64 architectures

## Dependencies

Key third-party libraries (managed by xmake):
- EASTL - High-performance containers
- ASIO - Async I/O
- spdlog/fmt - Logging and formatting
- doctest - Testing framework
- cxxopts - Command-line parsing
- json11 - JSON parsing
- lz4 - Compression
- xxhash - Hashing
- OpenSSL (Linux/Mac) - Crypto operations
- Sentry - Crash reporting (optional)

## Adding New Commands to `zen` CLI

1. Create `src/zen/cmds/mycommand_cmd.h` and `src/zen/cmds/mycommand_cmd.cpp`
2. Inherit from appropriate base command class
3. Implement `Execute()` method
4. Add `#include "cmds/mycommand_cmd.h"` to `src/zen/zen.cpp`
5. Register command in `zen.cpp` command dispatcher. These should be ordered in alphabetical identifier order for consistent merging

## Adding New Modules

1. Create `src/mymodule/` directory structure
2. Add `include/` subdirectory for public headers
3. Create `xmake.lua` defining the target
4. Add `includes("src/mymodule")` to root `xmake.lua`
5. Add dependencies via `add_deps()` in the module's xmake.lua

## Debugging

**Visual Studio:**
- Use Debug configuration (includes test code)
- For http.sys debugging, run Visual Studio as Administrator or add URL reservation
- Enable child process debugging with Microsoft Child Process Debugging Power Tool

**Multi-process Scenarios:**
When debugging zenserver-test or other multi-process scenarios, use child process debugging tools to attach to spawned processes automatically.

## Bundle Creation

```bash
# Create deployable ZIP bundle
xmake bundle

# Update frontend ZIP after HTML changes
xmake updatefrontend
```