# Zen Storage Service (zenserver)

This is the implementation of the local storage service for UE5. It is intended to be deployed on
user machines either as a daemon or launched ad hoc as required during  of editor/cooker/game startup

Zen can also be deployed as a shared instance for use as a shared cache. It also supports upstream 
connectivity to cloud storage services as well as other Zen server instances.

We currently only support building and running the server on Windows. Linux and Mac support is in progress

All platforms require [xmake](xmake.io)
Download the latest release [here](https://github.com/xmake-io/xmake/releases)

## Building on Windows

### Windows Setup

To build the code you will need Visual Studio 2019 (we use c++20 features), git and vcpkg.

* Install Visual Studio 2019 Version 16.10 or later
* Install [git](https://git-scm.com/download/win)

We use vcpkg to manage some libraries. Right now it's not set up on a project local
basis and requires manual bootstrap so you will need to do the following at least once:

* open up a command line window
  * create a `git`/`github` directory somewhere for you to clone repos into
  * issue `git clone https://github.com/microsoft/vcpkg.git` and build it using the `bootstrap-vcpkg.bat` script
* optional: add the `vcpkg` directory you cloned to your PATH to allow invoking vcpkg on the command line
* issue `vcpkg integrate install` to make sure you can build from Visual Studio using package manifests

Now you are ready to start building!

### Building with Visual Studio

We currently require Visual Studio 2019 Version 16.10 or later. Visual Studio 2022 can also be used, but
currently building with the VS2022 toolchain has not been tested (please leave the sln/vcxproj un-upgraded).

* clone the `zen` repository if you haven't already
  * run `git clone https://github.com/EpicGames/zen.git`
* run `xmake project -k vsxmake2019 -a x64 -y`
* open the `vsxmake2019\zen.sln` VS solution (NOTE: you currently need to run Visual Studio in ADMIN mode since 
  http.sys requires elevation)
  * you can now build and run `zenserver` as usual from Visual Studio
  * third-party dependencies will be built the first time via the `vcpkg` integration. This is not as
    fast as it could be (it does not go wide) but should only happen on the first build and will leverage
    a local build cache.

### Building with xmake

* configure xmake `xmake config -m debug|release -a x64`
* build zenserver `xmake build zenserver`

Binaries are located in the `build` directory

## Building on Linux

The following instructions have been collated using Ubuntu 20.04.

At the time of writing only GCC v11 supports the C++20 features used by Zen. As
this version is not available in the default package repositories we need to
get GCC from one of Ubuntu's toolchain repositories;

```
sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test
sudo apt install -y gcc-11
gcc-11 --version
```

Next we need the `xmake` build system. For this we will download and install
`xmake` as a `.deb` package. Check the project's release page for more up to
date `.deb` files;

```
wget https://github.com/xmake-io/xmake/releases/download/v2.5.7/xmake-v2.5.7.amd64.deb
sudo dpkg -i xmake-v2.5.7.amd64.deb
xmake --version
```

For some of Zen's third party dependencies are provided by Microsoft's `vcpkg`
C++ library manager. After cloning the project there is a initialisation step;

```
git clone https://github.com/microsoft/vcpkg.git ~/zen/vcpkg
~/zen/vcpkg/bootstrap-vcpkg.sh
```

`xmake` uses an environment variable to find `vcpkg`. Alternatively this can be
done by including `VCPKG_ROOT=...` on the command line when invoking `xmake`;

```
export VCPKG_ROOT=~/zen/vcpkg
```

Clone the Zen project and tell `xmake` to use the correct GCC version;

```
git clone https://github.com/EpicGames/zen.git ~/zen/main
cd ~/zen/main
xmake config --plat=linux --cxx=g++-11 --cc=gcc-11 [--mode=debug]
```

The `--mode=debug` is optionally required to change the build variant. This
cannot be done when executing the build and the setting of configuration options
can not be composed across multiple `xmake config` calls.

Now we are ready to build Zen.  The `-y` skips `xmake` from prompting about
updating `vcpkg` packages;

```
xmake build -y
```

The `xmake` flags `-vD` can be useful to diagnose `xmake` issues.

## Building on Mac

... coming soon

# Implementation Notes

* The implementation currently depends only on a few libraries including the C++ standard library
* It uses exceptions for errors
* It is currently not portable as it uses Windows APIs directly. But as we all know, there is no 
  portable code, just code that has been ported many times. The plan is to implement support for 
  MacOS and Linux soon, and some research to enable it has been done
* `zenservice.exe` currently requires elevated access to enable `http.sys` access. This will be relaxed 
  in the future by offering to use a portable server interface without elevation
* The service endpoints are currently open on all NICs and will respond to requests from any host. This
  will be tightened up in the future to require some degree of authentication to satisfy security 
  requirements

# Contributing Code

To run the pre-commit scripts you'll need a few things:

* We rely on clang-format for consistent code formatting. You can install version 12 or later from https://llvm.org/builds/
* The helper scripts also depend on Python 3.x, which you may install from https://www.python.org/downloads/windows/ (I am presently using 3.9.5 which works). NOTE: *do* check the option to add Python to your PATH!

Once you have those dependencies, you can simply run `prepare_commit.bat` to ensure the code is properly formatted and has the Epic copyright header comment. I'm sure there's a better way to integrate this into the git submit flow but my git-fu is not strong enough yet to know how to do that best.

# Debugging

When debugging multi-process scenarios such as when running `zenserver-test`, the [Microsoft Child Process Debugging Power Tool](https://marketplace.visualstudio.com/items?itemName=vsdbgplat.MicrosoftChildProcessDebuggingPowerTool) 
is incredibly handy. When that is installed you may enable auto-attach to child processes via the Debug -> Other Debug Targets menu in Visual Studio.

# Testing

* There are some test projects
  * `zencore-test` exercises unit tests in the zencore project
  * `zenserver-test` exercises the zen server itself (functional tests)

The tests are implemented using [doctest](https://github.com/onqtam/doctest), which is similar to Catch in usage.

# Coding Standards

See [Coding.md](Coding.md)

Run `prepare_commit.bat` before committing code. It ensures all source files are formatted with
clang-format which you will need to install.

(More helpful instructions needed here :)