| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
| |
Feature: Project store "getchunks" rpc call /prj/{project}/oplog/{log}/rpc extended to accept both CAS (RawHash) and Id (Oid) identifiers as well as partial ranges
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This change introduces support for tracing of memory allocation activity. The code is ported from UE5, and Unreal Insights can be used to analyze the output. This is currently only fully supported on Windows, but will be extended to Mac/Linux in the near future.
To activate full memory tracking, pass `--trace=memory` on the commandline alongside `--tracehost=<ip>` or `-tracefile=<path>`. For more control over how much detail is traced you can instead pass some combination of `callstack`, `memtag`, `memalloc` instead. In practice, `--trace=memory` is an alias for `--trace=callstack,memtag,memalloc`). For convenience we also support `--trace=memory_light` which omits call stacks.
This change also introduces multiple memory allocators, which may be selected via command-line option `--malloc=<allocator>`:
* `mimalloc` - mimalloc (default, same as before)
* `rpmalloc` - rpmalloc is another high performance allocator for multithreaded applications which may be a better option than mimalloc (to be evaluated). Due to toolchain limitations this is currently only supported on Windows.
* `stomp` - an allocator intended to be used during development/debugging to help track down memory issues such as use-after-free or out-of-bounds access. Currently only supported on Windows.
* `ansi` - fallback to default system allocator
|
| |
|
|
|
|
|
|
| |
GC (#224)
make sure we rewrite filecas entries if chunk size changes (due to compression changes)
hardening of move/write files in filecas
if we encounter a filecas entry with mismatching size (due to pre-existing bug) we validate the file and update the index
if we find a bad filecas file on disk we now attempt to remove it
|
| |
|
|
|
|
|
| |
- Improvement: Reworked workspace shares to be more secure. Workspaces and workspace shares can only be created using the `zen workspace` command, the http endpoint is disabled unless zenserver is started with the `--workspaces-allow-changes` option enabled.
- Each workspace are now configured via a `zenworkspaceconfig.json` file in the root of each workspace
- A workspace can allow shares to be created via the http interface if the workspace is created with the `--allow-share-create-from-http` option enabled
- A new http endpoint at `/ws` - issuing a `Get` operation will get you a list of workspaces
- A new http endpoint at `/ws/refresh` - issuing a `Get` will make zenserver scan for edits in workspaces and workspace shares
|
| |
|
| |
* remove PathFromHandle that throws to give better context on failures
|
| |
|
| |
we use the std implementation for all normal cases now. The Windows-only path still exists for VFS cleanup
|
| |
|
|
| |
windows (#129)
|
| |
|
| |
- Improvement: Refactor `IoHash::HashBuffer` and `BLAKE3::HashBuffer` to not use memory mapped files. Performs better and saves ~10% of oplog export time on CI
|
| |
|
|
| |
- Improvement: Add file and line to ASSERT exceptions
- Improvement: Catch call stack when throwing assert exceptions and log/output call stack at important places to provide more context to caller
|
| |
|
|
| |
* Make sure WriteFile() does not leave incomplete files
* use TemporaryFile and MoveTemporaryIntoPlace to avoid leaving partial files on error
|
| |
|
|
|
|
|
|
|
| |
(#600)
* Make sure that PathFromHandle don't hide true error when throwing exceptions
* changelog
* return error info in PathFromHandle if path fails to resolve
|
| |
|
|
|
| |
the previous implementation of in-memory index snapshots serialise data to memory before writing to disk and vice versa when reading. This leads to some memory spikes which end up pushing useful data out of system cache and also cause stalls on I/O operations.
this change moves more code to a streaming serialisation approach which scales better from a memory usage perspective and also performs much better
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
initial version -- this is primarily intended to be used for running stress tests and/or benchmarks
example usage:
`zen run -n 10 -- zenserver-test`
`zen run -n 10 -- zenserver-test --ts=core.assert` run zenserver-test 10 times (testing only the `core.assert` test suite)
`zen run --time 600 --basepath=d:\test_dir\test1 -- zenserver-test` keeps spawning new instances for 10 minutes (600 seconds)
|
| | |
|
| |
|
|
|
| |
this introduces a --snapshot-dir command line option to zenserver which specifies a directory which will be propagated to the persistence root directory on start-up.
This is most powerful with file systems which support block cloning, such as ReFS on Windows. This allows even very large state snapshots to be used repeatedly without having to worry about mutating the original dataset on disk. When using ReFS the state copy for even large state directories can be very fast since the duration is primarily proportional to the number of files in the tree rather than the size of the files being cloned. The storage requirements are also minimal as all data will be handled in a copy-on-write manner.
|
| |
|
|
|
| |
- Feature: Writes a `gc.log` with settings and detailed result after each GC execution (version 2 only)
- Break out file name rotate to allow access for gclog
- CompactBinaryToJson(MemoryView Data, StringBuilderBase& InBuilder)
|
| |
|
|
| |
(#528)
|
| |
|
| |
currently, only Windows (using Projected File System) is supported
|
| |
|
|
|
| |
get file name from handle (#393)
- Bugfix: Make sure error logging or destructors don't throw exception when trying to get file name from handle
|
| |
|
| |
* extend http client with configuration, headers, parameters and disk streaming upload/download
|
| |
|
|
|
|
|
|
|
|
| |
* stream large downloads from jupiter to temporary file
* rework DeleteOnClose - top level marks file for delete and if lower level parts wants to keep it it clears that flag
* changelog
* log number of attachments to download
* add delay on jupiter request failure when retrying
* make sure we upload all attachments even if Needs are empty when ForceUpload is true
release TempAttachment as soon as it is used
* sort attachments so we get predictable blocks for the same oplog
|
| |
|
|
|
|
| |
- Bugfix: All options given on command line now overrides lua config file settings
- Improvement: All options available from command line can now be configured in the lua config file (with a few exceptions such as `owner-pid`, `install` and `uninstall`)
We can now also generate a Lua config (just string so far) that includes all options given on command line together with any optional Lua input config file. The idea here is to be able to save config options in the future.
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
* Added LoadCompactBinaryObject from file to compactbinaryfile.cpp/h
* Added SaveCompactBinary(BinaryWriter& Ar, ...) functions
* Added ZEN_PLATFORM_NAME define
* Added SystemMetrics functionality to query system properties (see zencore/system.h)
|
| |
|
|
| |
characters (#332)
|
| |
|
| |
ATL has been used here and there as a convenience. Given that this is a legacy component and not always something which gets installed along with the compiler we hereby remove the dependency altogether in favour of our own simple wrappers
|
| |
|
|
| |
* added FileSizeFromHandle function
* added file size to error message when MapViewOfFile fails
|
|
|
* moved source directories into `/src`
* updated bundle.lua for new `src` path
* moved some docs, icon
* removed old test trees
|