path: root/CHANGELOG.md

##
- Bugfix: Verify that there are blocks to GC for block store garbage collect (void division by zero)
- Bugfix: Write log error and flush log before reporting error to Sentry/error logger
- Bugfix: Log ERROR in scope guard if function throws exception, throwing exception causes application abort

## 0.2.7
- Bugfix: Safely handle missing blocks when doing garbage collection in block store data
- Bugfix: Only strip uri accept type suffix if it can be parsed to a known type
- Bugfix: Keep system error code on Windows when file mapping fails and propagate to log/exception
- Bugfix: Catch any errors throw in HttpAsioServer() destructor and log error.

## 0.2.6
- Strip __FILE__ macro names in logging to only include the file name as to not expose file paths of the machine building the executable
- Bugfix: Reporting the correct callstack to sentry on ERROR/CRITICAL failure.
- Extend sentry message with triggering file/line/function

## 0.2.5
- Feature: Zen command line tool `rpc-record-start` to record all RPC requests to the structured cache
  - `--path` Recording file path where the rpc requests will be stored
- Feature: Zen command line tool `rpc-record-stop` stop the currently active RPC request recording started with `rpc-record-start`
- Feature: Zen command line tool `rpc-record-replay` replacy a RPC request recording created with `rpc-record-start`
  - `--path` Recording file path where the rpc requests are stored
  - `--numthreads` Number of worker threads to use while replaying the RPC requests
  - `--numproc` Number of worker processes to run, if more than one new processes will be spawn with `<numthreads>` workers each
  - `--offset` Offset into request playback to start at
  - `--stride` The stride to use when selecting requests to playback
  - `--onhost` Replay the recording inside the zenserver bypassing http overhead
  - `--showmethodstats` Show statistics of which RPC methods are used
  - `--forceallowlocalrefs` Force the requests to allow local references (file path/file handle)
  - `--disablelocalrefs` Force disable local references in request (file path/file handle)
  - `--forceallowlocalhandlerefs` Force the requests to allow local references via duplicated file handles for requests that allow local refs
  - `--disablelocalhandlerefs` Force disable local references via duplicated file handles in requests
  - `--forceallowpartiallocalref` Force the requests to allow local references for files that are not saved as whole files for requests that allow local refs
  - `--disablepartiallocalrefs` Force disable local references for files that are not saved as whole files for requests that allow local refs
- Feature: Zen command line tool `cache-stats` to give stats result about the zen cache
- Feature: Zen command line tool `project-stats` to give stats result about the zen project store
- Feature: Zen command line tool `cache-details` to give detail result about the zen cache, defaults to overview information about the cache
  - `--namespace` Get information about cache values in a namespace
  - `--bucket` Get information about cache values limited to a specific bucket in a namespace
  - `--valuekey` Get information about a cache value in a specific bucket in a namespace, valuekey is specified as IoHash hex string
  - `--details` Get detailed information about each cache record
  - `--attachmentdetails` Get detailed information about each attachments for each cache record
  - `--csv` Format the output as a comma delimited CSV file. If not specified it defaults to JSon style response.
- Feature: Zen command line tool `project-details` to give detail result about the zen project store, defaults to overview information about the project store
  - `--project` The project id to get information about
  - `--oplog` The oplog id to get information about
  - `--opid` The op Oid to get information about
  - `--details` Get detailed information about the op
  - `--opdetails` Extract the entire op information (not available in CSV output)
  - `--attachmentdetails` Get detailed information about each attachments for each op
  - `--csv` Format the output as a comma delimited CSV file. If not specified it defaults to JSon style response.
- Feature: New project store stats endpoint `/stats/prj` to get stats info for zen project store
- Feature: New project store details endpoints `/prj/details$`, `/prj/details$/{project}`, `/prj/details$/{project}/{oplog}`, `/prj/details$/{project}/{oplog}/{op}` to give detail result about the zen project store, defaults to overview information about the project store items
  - `details=true` Get detailed information about the op
  - `opdetails=true` Extract the entire op information
  - `attachmentdetails=true` Get detailed information about each attachments for each op
  - `csv=true` Format the output as a comma delimited CSV file. If not specified it defaults to JSon style response.
- Feature: New cache detail endpoints `/z$/details$`,  `/z$/details$/{namespace}`,  `/z$/details$/{namespace}/{bucket}`,  `/z$/details$/{namespace}/{bucket}/{key}` has been added
  - `details=true` Get detailed information about each cache record
  - `attachmentdetails=true` Get detailed information about each attachments for each cache record
  - `csv=true` Format the response as a comma delimited CSV file. If not specified it defaults to CbObject but can auto-format to json
- Feature: `--junit` switch to `xmake test` to generate junit style reports of tests.
- Feature: CI build on GitHub now uploads junit test reports as artifact to the check for PR validation and mainline validation
- Feature: Payloads from zenserver can now be sent using duplicated file handles if caller requests provides client ProcessId (Windows only).
- Feature: Add `--port` option to zen down command to shut down servers on different base ports
- Bugfix: Make sure async responses are sent async correctly in httpsys
- Bugfix: Don't delete manifest file in cas root when initializing a new filecas folder
- Bugfix: Sentry does not like UNC paths, so strip the prefix before passing them to sentry
- Bugfix: Make sure zen down command uses the correct port for shutdown event
- Improvement: FileCas now keeps an up to date index of all the entries improving performance when getting cache misses on large payloads
- Improvement: Structured cache now keeps RawHash and RawSize in memory avoiding materialization of cache values before sending response
- Changed: Exit with failure code on port conflict rather than reporting crash to Sentry
- Changed: removed catch2 support for now since it does not handle multithreaded tests
- Bugfix: fixed bug in dashboard content serving (see PR #255)

## 0.2.4
- Bugfix: Don't send empty http responses with content type set to Text. Fixes UE-177895

## 0.2.3
- Feature: Add support for "packagedata" mapping in oplog entries
- Feature: Zen command line tool `project-create` to create a project store project
  - `--project` Project name (id)
  - `--rootdir` Absolute path to root directory (optional)
  - `--enginedir` Absolute path to engine root directory (optional)
  - `--projectdir` Absolute path to project directory (optional)
  - `--projectfile` Absolute path to .uproject file (optional)
- Feature: Zen command line tool `oplog-create` to create a project store oplog
  - `--project` Project name (id)
  - `--oplog` Oplog name (id)
  - `--gcpath` Absolute path to oplog lifetime marker file (optional)
- Feature: Build scripts and tooling to build zen compliant with VFX reference platform CY2022/2021 matching UE linux builds
- Feature: added `xmake sln` task which replaces `generate_projects.bat`
- Feature: Zen server endpoint `prj/{project}/oplog/{log}/chunks` to post multiple attachments in one request.
- Feature: Zen server endpoint `prj/{project}/oplog/{log}/save` to save an oplog container. Accepts `CbObject` containing a compressed oplog and attachment references organized in blocks. 
- Feature: Zen server endpoint `prj/{project}/oplog/{log}/load` to request an oplog container. Responds with an `CbObject` containing a compressed oplog and attachment references organized in blocks.
- Feature: Zen server endpoint `{project}/oplog/{log}/rpc` to initiate an import to or export from an external location and other operations. Use either JSon or CbPackage as payload.
  - CbObject/JSon RPC format for `import` and `export` methods:
      ```json
        {
          "method" : "<method>",
          "params" : {
            "maxblocksize": "<maxblocksize>",
            "maxchunkembedsize": "<maxchunkembedsize>",
            "file" : {
              "path" : "<file-system-folder-path>",
              "name" : "<oplog-file-name>"
            },
            "cloud" : {
              "url" : "<serviceurl>",
              "namespace" : "<namespace>",
              "bucket" : "<bucket>",
              "key" : "<iohash>",
              "openid-provider" : "<provider-id>",
              "access-token" : "<access-token>",
			  "access-token-env": <envvariablename>",
              "disableblocks" : "<disableblocks>",
              "disabletempblocks" : "<disabletempblocks>"
            },
            "zen" : {
              "url" : "<url>",
              "project" : "<projectid>",
              "oplog" : "<oplogid>"
            }
          }
        }
      ```
    - `"method"`supported methods are `"export"` and `"import"` to import/export an oplog
    - `"params"` container for parameters
      - `"maxblocksize"` - Optional. The maximum size of a block of attachments, default 134217728 (128 Mb) (export only)
      - `"maxchunkembedsize"` - Optional. The maximum size of an attachment to be put in a block, larger attachments will be stored as usual attachments, default 1048576 (1Mb) (export only)
      - `"force"` - Optional. Boolean flag to indicate weather attachments should be uploaded/downloaded disregarding prior existance
      - External location types are "file" (File system), "cloud" (UE Cloud Storage service) or "zen" (Zen server instance), provide one of those as remote location.
        - `"file"` - Optional. Indicates remote location is the local file system
          - `"path"` - File system path folder to export to / import from
          - `"name"` - File name of oplog output, written into <file-system-folder-path>
        - `"cloud"` - Optional. Indicates remote location is UE Cloud Storage service
          - `"url"` - Jupiter service endpoint url
          - `"namespace"` - Name of namespace to store data to
          - `"bucket"` - Name of bucket to store data to
          - `"key"` - IoHash key to the stored oplog container
          - `"openid-provider"` - Optional. Name of openid provider used to authenticate with, requires that the zen server instance has been provided with a oids refresh token for <provider-id>
          - `"access-token"` - Optional. JWT access token to authenticate with
          - `"access-token-env"` - Optional. Name of environment variable that holds an JWT access token to authenticate with
          - `"disableblocks"` - Optional. Disable creation of attachments blocks - "true"/"false" (export only)
          - `"disabletempblocks"` - Optional. Disable creation of attachments temp blocks forcing upload before oplog container - "true"/"false" (export only)
        - `"zen"` - Optional. Indicates remote location is a Zen server instance
          - `"url"` - Zen server instance url
          - `"project"` - The remote project name (id)
          - `"oplog" - The remote oplog name (id)
  - CbObject RPC format for `getchunks` method, returns CbPackage with the found chunks, if all chunks are found the number of attachments matches number of chunks requested.
      ```json
        {
          "method" : "getchunks",
          "chunks" : [
            "<rawhash>",
          ]
        }
      ```
  - CbPackage RPC format for `putchunks` method, attachments are stored in CidStore
      ```json
        {
          "method" : "putchunks",
        }
      ```
- Feature: Zen server `{project}/oplog/{log}/{hash}` now accepts `HttpVerb::kPost` as well as `HttpVerb::kGet`.
- Feature: Zen command line tool `oplog-export` to export an oplog to an external target using the zenserver oplog export endpoint.
  - `--project` Project name (id)
  - `--oplog` Project name (id)
  - `--maxblocksize` The maximum size of a block of attachments (optional)
  - `--maxchunkembedsize` The maximum size of an attachment to be put in a block, larger attachments will be stored as usual attachments (optional)
  - `--force` Force upload/download of attachments even if they already exist.
  - `--file` File system path folder to export to / import from
    - `--name` File name of oplog output, written into `--file` path
    - `--disableblocks` Disable block creation and save all attachments individually
    - `--forcetempblocks` Force creation of temp attachment blocks
  - `--cloud` Jupiter service endpoint to export to / import from
    - `namespace` Name of namespace to store data to
    - `bucket` Name of bucket to store data to
    - `key` Key to the stored oplog container (If omitted a default key will be generated based on project/oplog/namespace/bucket)
    - `openid-provider` Optional name of openid provider used to authenticate with, requires that the zen server instance has been provided with a oids refresh token for the provider name
    - `access-token` Optional JWT access token to authenticate with
    - `access-token-env` - Optional name of environment variable that holds an JWT access token to authenticate with
    - `disableblocks` Disable block creation and save all attachments individually
    - `disabletempblocks` Disable temp block creation and upload blocks without waiting for oplog container to be uploaded
  - `--zen` Zen server instance url to export to / import from
    - `--target-project` The remote project name (id) (optional, defaults to same as `project`)
    - `--taret-oplog` The remote oplog name (id) (optional, defaults to same as `olplog`)
    - `--clean` Delete and create a new oplog before starting export
- Feature: Zen command line tool `oplog-import` to import an oplog from an external source using the zenserver oplog import endpoint.
  - `--project` Project name (id)
  - `--oplog` Project name (id)
  - `--force` Force upload/download of attachments even if they already exist.
  - `--file` File system path folder to export to / import from
    - `--name` File name of oplog output, written into `--file` path
  - `--cloud` Jupiter service endpoint to export to / import from
    - `namespace` Name of namespace to store data to
    - `bucket` Name of bucket to store data to
    - `key` Key to the stored oplog container (If omitted a default key will be generated based on project/oplog/namespace/bucket)
    - `openid-provider` Optional name of openid provider used to authenticate with, requires that the zen server instance has been provided with a oids refresh token for the provider name
    - `access-token` Optional JWT access token to authenticate with
    - `access-token-env` - Optional name of environment variable that holds an JWT access token to authenticate with
  - `--zen` Zen server instance url to export to / import from
    - `--source-project` The remote project name (id) (optional, defaults to same as `project`)
    - `--source-oplog` The remote oplog name (id) (optional, defaults to same as `olplog`)
    - `--clean` Delete and create a new oplog before starting import
- Improvement: Faster oplog replay - reduces time to open an existing oplog
- Improvement: Clearer error messages and logging when requests to project store fails
- Changed: Removed remnants of old mesh experiment
- Changed: Remove obsolete export-project command
- Changed: Removed remnants import-project command
- Changed: Removed unused remote build scripts
- Changed: Removed very old and invalid TODO.md
- Changed: Removed some deprecated scripts

## 0.2.2
- Feature: Added info (GET) endpoints for structured cache
 - `/z$` - get a list of namespaces and global info
 - `/z$/{namespace}` - get list of buckets in a namespace and namespace related info
 - `/z$/{namespace}/{bucket}` - get bucket info
- Feature: Added project store oplog info: `markerpath`,  `totalsize`, `opcount`, `expired` on GET requests for oplog
- Feature: Added project store project info: `expired` on GET requests for project
- Feature :Added project store root route `/prj` which is identical to `/prj/list`
- Feature: Zen command line tool `cache-info` to show cache, namespace or bucket info
- Feature: Zen command line tool `project-info` to show store, project or oplog info
- Feature: Zen command line tool `project-drop` to drop project or oplog
- Feature: Zen command line tool `gc` to trigger a GC run
- Feature: Zen command line tool `gc-info` to check status of GC
- Feature: Added version information to zenserver and zen command line tool executables
- Bugfix: Don't log "time to next GC" if time to next GC is not set
- Improvement: Don't wait for GC monitor interval before doing first GC check
- Improvement: Zen command line tool now fails on any unrecognized arguments
- Improvement: Zen command line tool now displays extra help for all sub-commands
- Improvement: Host address can now be configured for zen command line tool `drop` command
- Improvement: Added precommit xmake task `xmake precommit` to run precommit checks
- Changed: Default GC interval set to 1 hour
- Changed: Default GC cache duration set to 2 weeks
- Changed: Removed HttpLaunchService and related code
- Changed: Removed dead/experimental file system related code
- Changed: Removed faux vfs config option

## 0.2.1
- Feature: Oplog level GC in project store. If gc marker file path is given by UE, oplogs will be GC:d when marker file is deleted (and GC is triggered)
- Bugfix: Index handling for cache large object store was broken resulting in log always being played back
- Bugfix: Make sure to flush cache store on call to flush on service and exit
- Improvement: Don't write index snapshots if no new entries has been added to log

## 0.2.0
- Feature: Recording and playback of cache request with full data - both get and put operations can be replayed. Invoke via web request
  - `<host>/z$/exec$/start-recording?<disk-storage-path>`
  - `<host>/z$/exec$/stop-recording`
  - `<host>/z$/exec$/replay-recording?<thread-count>&<disk-storage-path>`
- Feature: Disk size triggered GC, a soft disk usage limit for cache data
- Feature: New option `--gc-disk-size-soft-limit` (command line), `gc.cache.disksizesoftlimit` (lua config) controlling limit for soft disk usage limit. Defaults to zero which disables soft disk usage limit
- Improvement: Disk write pressure in GC log and cleaned up clutter in GC logging
- Improvement: Much improved performance, between 2x to 9x improvement under heavy load (excluding http service overhead). See https://github.com/EpicGames/zen/pull/200 for details
- Bugfix: Always store records or oplog entries before storing attachments to avoid GC finding unreferenced chunks i CidStore
- Bugfix: Updated Zen `drop` command to support namespaces
- Bugfix: Use ZEN_CONSOLE for output to console in Zen commands
- Bugfix: Zen `status` command now shows info about found Zen instances
- Bugfix: Zen `top` command now shows session id string
- Bugfix: On Windows platforms explicitly set the special `SO_EXCLUSIVEADDRUSE` flag as `SO_REUSEADDR` still allows shared use of sockets
- Bugfix: Fix logging of number of entries read from caslog at startup
- Bugfix: Fix asio http handling of very large/malformed headers and handle mismatching content size
- Changed: Reduced GC `INFO` spam by converting to `DEBUG` log messages
- Changed: Use Iso8601 format for logging start and end message

## 0.1.9
- Feature: Adds two command to Zen command tool to export/import project store oplogs with attachments
  - `export-project <target-folder> <project> [oplogs...]`
  - `import-project <target-folder> <project> [oplogs...]`
- Feature: Adds command to query Zen version, specify `host-name` url to query running service version, otherwise you get zen command version. `detailed` option gives you long form version.
  - `version [host-name] [detailed]`
- Feature: New service endpoint to query Zen server version, add `?detailed=true` to get long form version
  - `<host>/health/version`
- Feature: Configure OpenID providers from cmd line and Lua cfg
- Feature: Added zen command line executable to release distribution
- Bugfix: Fix double reporting of disk usage for namespaces
- Bugfix: Fix double garbage collection analisys and garbage collection execution of namespaces
- Improvement: Improve tracking of used disk space for filecas and compactcas
- Improvement: Add tracking of used disk space for project store
- Improvement: Bumped limit for storing cache values as separate files to reduce number of loose files
- Improvement: Optimizations when handling compressed buffer (less materialization and reading of headers)
- Improvement: Send attachments as file references if the IoBuffer we find represents a complete file and `AcceptFlags` in RPC request allows it.
- Improvement: Don't reserve full block size for block store files at creation

## v0.1.8
- Change: Responding with new wire format for RPC requests requires the requestor to add a `Accept` field in the request. This is to allow compatability with older clients for shared instances.
- Improvement: Fixed concurrency issues in project store - project and oplog lifetime issues.
- Improvement: Don't open oplogs until we require use of them.
- Cleanup: Remove rocksdb experimental code.
- Feature: Add GC to projects store. Checks path to project file in UE side to determine when a project may be GCd on the Zen side.

## v0.1.7
- Change: All RPC responses are now formatted using dedicated wire format, Zen server has fallback to enable compatability with legacy upstreams
- Feature: Adding a `.json` extension to the `--abslog` option will make zenserver log in json format to file
- Feature: Create release in Sentry and use `sentry_options_set_release` to associate the executable
- Bugfix: CompactBinary: Fixed LoadCompactBinary to gracefully handle read failures and sizes larger than the archive. From http1s://p4-swarm.epicgames.net/changes/21983905
- Bugfix: Use bucket/key to get inline value in upstream for chunks without a chunkid
- Bugfix: Handle edge case when trying to materialize a IoBuffer of zero size via memory mapping
- Improvement: Logging: don't do formatting of messages the will not be logged
- Improvement: Logging: Timing and upstream source information in upstream logging when debug level logging is enabled
- Improvement: Reduce buffer creation and copying in ParsePackageMessage
- Improvement: Don't read attachments for oplogs we already have when parsing oplog message

## v0.1.6
- Bugfix: Bugfix: Use bucket/key to get inline value in upstream for chunks without a chunkid (UE-164966)

## v0.1.5
- Bugfix: Don't fail entire request if GetCacheValue from Horde fails for a single value

## v0.1.4
- Change: Bumped ZEN_SCHEMA_VERSION - this will invalidate entire local cache when deployed
- Change: Make CAS storage an hidden implementation detail of CidStore, we no longer hash and do mapping to compressed hash when storing cache values
- Feature: Extended zen print command to also handle CbPackage and CompressedBuffer format payloads
- Feature: Added /prj/{project}/oplog/{log}/{op} endpoint to allow retrieval of an op entry by LSN. Supports returning CbObject or CbPackage format payloads
- Improvement: asio: added some context to error reporting
- Improvement: namespace/bucket validation now uses AsciiSet for more efficient validation
- Improvement: Frontend: simplified content-type logic
- Improvement: Improved message indicating no GC is scheduled
- Improvement: Implement proper GetCacheValues upstream path
- Improvement: Demote a number of ZEN_ERROR log calls for problems that are recoverable and handled
- Bugfix: Use bucket/key to get inline value in upstream for chunks without a chunkid
- Bugfix: Fixed issue in CbPackage marshaling of local reference
- Bugfix: Fix crash when switching Zen upstream configured via DNS when one endpoint becomes unresposive
- Bugfix: Fixed issue where projects would not be discovered via DiscoverProjects due to use of stem() vs filename()
- Bugfix: Use "\\\\?\\" prefixed paths on Windows and fix hardcoded path delimiters (UE-141222)
- Bugfix: Safer detection of html folder when running non-bundled executable
- Bugfix: Use "application/x-jupiter-inline" to fetch GetCacheValues from Horde (UE-162151)
- Sentry: Added logging of sentry_init error code
- Sentry: Attach log file to Sentry error reports
- Sentry: Capture capture error/critical log statements as errors in Sentry
- Update: Bump VCPKG version to 2022.08.15
- CI: MacOS build enable again in GitHub CI
- CI: Upload debug info and source files to Sentry when release is created

## v0.1.3
-- Enable adding namespace to URI based upstream requests
-- Add logging of namespace name and bucket name if we get invalid names in requests
-- Updated README.md with Linux dev prerequisites
-- asio: added some logging to indicate concurrency
-- Fixed expired cache keys overwriting between namespaces when bucket names were the same in multiple namespaces

## v0.1.2
- Tweak bundle compression settings to streamline build
- ZenCacheDiskLayer::CacheBucket::GatherReferences: Don't hold index lock while reading standalone values
- hardening of ZenCacheDiskLayer::CacheBucket::PutStandaloneCacheValue
- GitHub Actions: Move release job to in-house linux agent

## v0.1.1
- BlockStore (small object store) Always block GC of current write block
- Make it possible to configure GC monitoring interval using `--gc-monitor-interval-seconds`
- Keep "reason" from upstream response so we can present it even if the request fails without outright error
- New GitHub Actions release flow - Add release flow in GitHub actions on pushed tag `v0.1.2` gives full release, `v0.1.2-pre0` gives pre-release

## 0d08450
- Fixes issue with broken Zen instances for legacy requests

## 63f50b5
- Enable FILE_SHARE_DELETE on standalone files in disk buckets  - fixes Jira UE-154234
- Make sure we can properly create the block file before assigning it for use - fixes Jira UE-154438
- Horde execute compressed input blobs
- Drop namespace support
- Safer delete of cache buckets

## dba8b36
- Namespaces: This introduces namespaces to the zenserver but only the default ue4.ddc is supported. Clients that don't send a namespace in the request will keep old behviour, new clients that sends namespace is required to use ue4.ddc (which they currently do)
- Aligned bucket naming rules with UE code base
- Fix retry counter and add an extra iteration to give more time for success during contention for standalone files in cache
- Make sure CacheBucket::PutStandaloneCacheValue cleans up the temp file
- Restore logic where we accept failed overwrite if resulting size is the same for standlone file in cache
- Correctly calculate the m_TotalSize difference when overwriting file for standalone files in cache
- Fix namespace folder scanning