# Structured Cache (DDC)

The Cache service is the primary storage interface for the Derived Data Cache
(DDC). It stores and retrieves derived data produced by the Unreal Engine
cooker — cooked assets, shader bytecode, texture compilations, and other build
artifacts.

The cache is designed to allow the client to avoid doing computationally intensive
work by using the output produced by another machine/user or a previous local run.

## Concepts

### Namespace

A logical partition within the cache. Each namespace (e.g. `ns_ue.ddc`) has its
own set of buckets and independent storage accounting. Namespaces allow different
types of cached data to be managed separately.

### Bucket

A subdivision within a namespace. Buckets group cache entries by the type of
derived data they contain (e.g. `bulkdatalist`, `animationsequence`,
`shaderbytecode`). Bucket names are lowercase alphanumeric strings.

### Key

Cache entries are identified by a 40-character hex hash within a bucket. The
full key path is `{bucket}/{keyhash}`. The key itself has no prescribed
derivation, but typically the client summarises all inputs to a transformation
and feeds the result into a hash function such as **IoHash** (BLAKE3/160) to
produce it.

The inputs conceptually include the code performing the transformation, though
not in a literal sense — that would be impractical. Instead, one or more version
numbers are typically included as a proxy for the code itself.

It is important that all inputs influencing the outcome of the transformation
are accounted for. Missing inputs can lead to non-determinism in the cooker and
editor, where stale cached results are used instead of recomputing.

### Cache Record

A cache record is a structured entry stored in Compact Binary format. Records
contain metadata describing the cached result along with references to one or
more attachments that hold the actual payload data. When a client performs a
cache lookup, the record is returned first, and the client can then fetch the
referenced attachments as needed.

### Cache Value

A cache value is an unstructured (opaque) entry stored as a raw binary blob.
Unlike records, values do not carry structured metadata or separate attachment
references — the entire payload is stored and retrieved as a single unit. Values
are used for simpler caching scenarios where the overhead of structured records
is unnecessary.

### Attachment

A binary payload (chunk) associated with a cache entry, identified by the
content hash of the data. Attachments are stored in CAS and referenced by
the cache entry. The full attachment path is `{bucket}/{keyhash}/{valuehash}`.

## API

**Base URI:** `/z$/`

### Service and Namespace Info

```
GET /z$/                            # Service info (namespaces, storage size)
GET /z$/{namespace}                 # Namespace info (buckets, sizes, config)
GET /z$/{namespace}?bucketsizes=*   # Include per-bucket storage sizes
```

### Bucket Operations

```
GET    /z$/{namespace}/{bucket}     # Bucket info (size, entry count)
DELETE /z$/{namespace}/{bucket}     # Drop all entries in a bucket
DELETE /z$/{namespace}              # Drop entire namespace
```

### Record Operations

Records are structured cache entries (Compact Binary format).

```
GET /z$/{namespace}/{bucket}/{keyhash}      # Get cached record
PUT /z$/{namespace}/{bucket}/{keyhash}      # Store cached record
```

### Attachment Operations

Attachments are binary chunks referenced by cache entries.

```
GET /z$/{namespace}/{bucket}/{keyhash}/{valuehash}  # Get attachment
PUT /z$/{namespace}/{bucket}/{keyhash}/{valuehash}  # Store attachment
```

### Batch Operations (RPC)

For high-throughput scenarios, the RPC endpoint supports batch get/put
operations using the Compact Binary protocol:

```
POST /z$/{namespace}/$rpc           # Batch RPC operations
```

Supported RPC operations:

- `PutCacheRecords` — batch store of structured records
- `GetCacheRecords` — batch retrieve of structured records
- `PutCacheValues` — batch store of unstructured values
- `GetCacheValues` — batch retrieve of unstructured values
- `GetCacheChunks` — batch retrieve of chunks with upstream fallback

### Detailed Inventory

```
GET /z$/details$                                    # All cache contents
GET /z$/details$/{namespace}                        # Namespace inventory
GET /z$/details$/{namespace}/{bucket}               # Bucket inventory
GET /z$/details$/{namespace}/{bucket}/{key}         # Single key details
```

Add `?csv=true` for CSV output, `?details=true` for size information, or
`?attachmentdetails=true` for full attachment listings.

## Upstream Caching

The cache service supports tiered caching through upstream connections. When a
cache miss occurs locally, the request can be forwarded to an upstream storage
instance (e.g. a shared build farm cache). Successfully retrieved content is
stored locally for future requests.

## Web Dashboard

The Cache page in the dashboard shows:

- Per-namespace breakdown with disk and memory usage
- Per-bucket entry counts and storage sizes
- Cache hit/miss statistics