diff options
Diffstat (limited to 'docs/glossary.md')
| -rw-r--r-- | docs/glossary.md | 120 |
1 files changed, 120 insertions, 0 deletions
diff --git a/docs/glossary.md b/docs/glossary.md new file mode 100644 index 000000000..6973f7222 --- /dev/null +++ b/docs/glossary.md @@ -0,0 +1,120 @@ +# Storage + +## DDC (Derived Data Cache) +A cache for derived data produced by the Unreal Engine cooker. DDC entries are +keyed by a hash (typically generated from the input data combined with code +versions and parameters) and typically contain cooked assets, shader bytecode, or +other build artifacts. DDC is one of the primary workloads served by zen. + +### Cache Namespace +A logical partition within the cache service. Each namespace (e.g. +`ns_ue.ddc`) has its own set of buckets and storage accounting. Namespaces +allow different types of cached data to be managed independently. + +### Cache Bucket +A subdivision within a cache namespace. Buckets provide fine-grained grouping of +cache entries, typically corresponding to a specific type of derived data (e.g. +`bulkdatalist`, `animationsequence`). + +## Project Store +A storage service for per-project data such as asset metadata and editor state. +Like the build store, project store entries have configurable expiration and are +garbage collected accordingly. + +## CAS (Content-Addressable Storage) +The lowest storage layer in zen. Data is stored and retrieved by its content hash +(IoHash) also known as Content Id (aka 'cid'). CAS provides automatic deduplication +since identical content always produces the same hash. CAS content is garbage +collected based on what is referenced from higher-level services. + +Content is immutable once written to CAS. Note that zenserver always stores data +in `CompressedBuffer` format, and the Content Id is derived from the *decompressed* +data. + +### IoHash +A 20-byte (160-bit) content hash used to identify data in CAS. IoHash is the +fundamental addressing unit for content-addressable operations. + +### Attachment +A reference from a higher-level store (build store, project store, or cache) to +a CAS object. Attachments are what keep CAS objects alive during garbage +collection — unreferenced CAS content is eligible for removal. + +## Build Store +A storage service for build artifacts. Build store entries have configurable +expiration and are subject to garbage collection when they exceed their maximum +retention duration. + +## Object Store +A general-purpose object storage service for arbitrary key-value data. Objects +are organized by namespace and can have associated metadata. + +# Garbage Collection + +## Full GC +A complete garbage collection pass that scans all referencers and reference +stores, removes expired data, prunes unreferenced CAS content, and optionally +compacts storage blocks. Full GC runs at a configured interval (typically hours). + +## Lightweight GC +A faster, less thorough garbage collection pass focused on removing expired +entries without performing full reference scanning or compaction. Runs more +frequently than full GC. + +## Referencer +A component that holds references to CAS content (e.g. the cache service, +project store, or build store). During GC, each referencer is scanned to +determine which CAS objects are still in use. + +## Reference Store +The CAS-side counterpart to a referencer. During GC, reference stores are scanned +to identify and remove CAS objects that are no longer referenced by any +referencer. + +## Compaction +A GC phase that reclaims fragmented disk space by rewriting storage blocks. A +block is compacted when its usage falls below the configured threshold +percentage. Compaction frees disk space but is more expensive than simple +deletion. + +## Write Block +A period during GC where write operations to the affected stores are temporarily +blocked to ensure consistency. The write block duration is tracked as a GC +performance metric. + +# Network + +## http.sys +The Windows kernel-mode HTTP server driver used by zen on Windows for maximum +throughput. Requires either administrator privileges or a URL reservation to bind +to network interfaces. Can be overridden with `--http=asio` to use the +user-mode ASIO server instead. + +## ASIO Server +An asynchronous I/O based HTTP server implementation available on all platforms. +This is the default on Linux and macOS, and can be used on Windows as an +alternative to http.sys. + +# Sessions + +## Session +A logical connection from a client application (e.g. Unreal Editor, a build +agent) to the zen server. Each session has an ID, application name, mode, and +tracks activity timestamps. Sessions can carry metadata and log entries. + +## Session Log +A per-session log stream visible in the sessions browser UI. The server's own +session log captures zen's internal log output, while external sessions can +post log entries via the API. + +# General + +## zen CLI +The command-line client utility (`zen`) for interacting with a running +zenserver. Provides commands for cache management, server status, diagnostics, +and more. + +## zenserver +The main server binary that integrates all zen storage services, the HTTP +server, telemetry, and the web dashboard. It can be launched in a number +of modes (storage, compute, proxy, hub) depending on the desired functionality. |