| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Feature: Added global zenserver option `--cache-bucket-limit-overwrites` controlling Whether to require policy flag pattern before allowing overwrites or not. Default `false` = overwrites always allowed
- Feature: Add per bucket cache configuration option `limitoverwrites` (Lua options file only)
cache = {
bucket = { -- This is the default for all namespaces
limitoverwrites = true
},
buckets = { -- Here you can add matching per bucket name (matches accross namespaces)
iostorecompression = {
limitoverwrites = false
},
},
}
|
| |\ |
|
| | |
| |
| |
| |
| |
| | |
- Improvement: Refactored build store cache to use existing CidStore implementation instead of implementation specific blob storage
- **CAUTION** This will clear any existing cache when updating as the manifest version and storage strategy has changed
- Bugfix: BuildStorage cache return "true" for metadata existance for all blobs that had payloads regardless of actual existance for metadata
|
| |\| |
|
| | |
| |
| |
| | |
memory for no reason (#397)
|
| | |
| |
| | |
- Bugfix: Long file paths now works correctly on Windows
|
| | |
| |
| |
| |
| | |
- Feature: `zen builds list` command has new options
- `--query-path` - path to a .json (json format) or .cbo (compact binary object format) with the search query to use
- `--result-path` - path to a .json (json format) or .cbo (compact binary object format) to write output result to, if omitted json format will be output to console
|
| | | |
|
| | | |
|
| | | |
|
| |/ |
|
| |
|
|
|
|
| |
move jupiter.h/cpp -> zenutil
move packageformat.h/.cpp -> zenhttp
zenutil now depends on zenhttp instead of the inverse
|
| |
|
|
| |
the old version is still present for now, with a _deprecated suffix
|
| | |
|
| |
|
|
| |
* snapshot test
* use proper IoBuffer constructor to take handle ownership
|
| |
|
|
|
|
|
| |
minor clean-up
`zencore/memory.h` used to contain a variety of things including `Malloc` support along with `MemoryView` etc
since the memory allocator stuff moved into `zencore/memory/memory.h` there was basically only `MemoryView` and `MemCmp` in there which seemed better to split out into separate headers to avoid overloading `memory.h`
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This change introduces support for tracing of memory allocation activity. The code is ported from UE5, and Unreal Insights can be used to analyze the output. This is currently only fully supported on Windows, but will be extended to Mac/Linux in the near future.
To activate full memory tracking, pass `--trace=memory` on the commandline alongside `--tracehost=<ip>` or `-tracefile=<path>`. For more control over how much detail is traced you can instead pass some combination of `callstack`, `memtag`, `memalloc` instead. In practice, `--trace=memory` is an alias for `--trace=callstack,memtag,memalloc`). For convenience we also support `--trace=memory_light` which omits call stacks.
This change also introduces multiple memory allocators, which may be selected via command-line option `--malloc=<allocator>`:
* `mimalloc` - mimalloc (default, same as before)
* `rpmalloc` - rpmalloc is another high performance allocator for multithreaded applications which may be a better option than mimalloc (to be evaluated). Due to toolchain limitations this is currently only supported on Windows.
* `stomp` - an allocator intended to be used during development/debugging to help track down memory issues such as use-after-free or out-of-bounds access. Currently only supported on Windows.
* `ansi` - fallback to default system allocator
|
| |
|
|
|
|
|
| |
- Improvement: Reworked workspace shares to be more secure. Workspaces and workspace shares can only be created using the `zen workspace` command, the http endpoint is disabled unless zenserver is started with the `--workspaces-allow-changes` option enabled.
- Each workspace are now configured via a `zenworkspaceconfig.json` file in the root of each workspace
- A workspace can allow shares to be created via the http interface if the workspace is created with the `--allow-share-create-from-http` option enabled
- A new http endpoint at `/ws` - issuing a `Get` operation will get you a list of workspaces
- A new http endpoint at `/ws/refresh` - issuing a `Get` will make zenserver scan for edits in workspaces and workspace shares
|
| |
|
|
|
|
| |
* resolve relative paths for root path
* block share paths that go outside of root path
* fix test using invalid share_path
* validate that root path is absolute
|
| |
|
| |
Feature: New 'workspaces' service which allows a user to share a local folder via zenserver. A workspace can have mulitple workspace shares and they provie an HTTP API that is compatible with the project oplog HTTP API. Workspaces and shares are preserved between runs. Workspaces feature is disabled by default - enable with --workspaces-enabled option when launching zenserver.
|
| |
|
|
| |
* verify running process before creating event
* make sure we don't signal/wait for a zenserver instance that we did not wait for to get ready
|
| | |
|
| |
|
| |
- Improvement: Add batching when writing multiple small chunks to block store - decreases I/O load significantly on oplog import
|
| |
|
|
| |
* capture spawned server output and output on launch error
* fix logging and launch validation in tests
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
- Feature: `zen up` command improvements
- --`port` allows you to specify a base port when starting an instance
- --`base-dir` allows you to specify a base directory for the zenserver executable if it is not located next to the zen.exe executable
- Feature: `zen down`
- --`port` allows you to specify a base port when shutting down an instance
- --`base-dir` allows you to specify a base directory for the zenserver executable if it is not located next to the zen.exe executable
- --`force` if regular shutdown fails it tries to find a running zenserver.exe process and terminate it
- If it fails to attach to the running server it now waits for it to exit when setting the RequestExit shared memory flag
- Improvement: zenserver now checks the RequestExit flag in the shared memory and exist gracefully if it is set
- Improvement: When adding a sponsor process to a running zenserver instance, we wait for it to be picked up from the shared memory section to determine success/fail
|
| |
|
|
|
| |
* add remote oplog store statistics
* block chunking when uploading oplog to zenserver (mirroring)
* make sure we can move temporary dechunked file into cas store
|
| |
|
| |
* hardening of ParsePackageMessage and extended details when malformed attachments are detected
|
| |
|
|
| |
* Separate chunk raw hash from section hash (how to find the fragment attachment)
* fix partial get cache value tests
|
| |
|
| |
* add support for responding with partial cache chunks
|
| |
|
|
|
|
|
| |
- Improvement: Add zenhttp-test and zenutil-test
- Improvement: Moved cachepolicy test to cachepolicy.cpp
- Improvement: Renamed cachestore tests from z$ to cachestore
- Improvement: Moved test linking so test for a lib is linked by <lib>-test
- Improvement: Removed HttpRequestParseRelativeUri in httpstructuredcache.cpp and use the one in cacherequests.h instead
|
| |
|
| |
- Bugfix: Implement two listening sockets in ASIO (ipv4+ipv6) when either we start with `--http-forceloopback` or we resort to that mode because of a failure to bind to the "any" address
|
| |
|
|
|
|
| |
* moved all RPC processing from HttpStructuredCacheService into separate CacheRpcHandler class in zenstore
* move package marshaling to zenutil. was previously in zenhttp/httpshared but it's useful in other contexts as well where we don't want to depend on zenhttp
* introduced UpstreamCacheClient, this provides a subset of functions on UpstreamCache and lives in zenstore
|
| |
|
|
|
|
|
|
|
|
| |
initial version -- this is primarily intended to be used for running stress tests and/or benchmarks
example usage:
`zen run -n 10 -- zenserver-test`
`zen run -n 10 -- zenserver-test --ts=core.assert` run zenserver-test 10 times (testing only the `core.assert` test suite)
`zen run --time 600 --basepath=d:\test_dir\test1 -- zenserver-test` keeps spawning new instances for 10 minutes (600 seconds)
|
| |
|
| |
this change replaces hard-coded port numbers in tests with dynamically assigned ports, to avoid potential issues around socket lifetimes and re-use policies
|
| |
|
|
|
|
|
|
|
|
| |
this change just cleans up dependency declarations in xmake.lua files, discovered while exploring a more to xrepo which catches dependency problems since it will not just place all includes in a single directory, unlike vcpkg.
* removed spurious asio dependency from zenserver-test
* removed rocksdb reference
* add missing asio package dependency
* removed catch2 reference (no longer available)
* added explicit cpr reference
* made some zencore package dependencies public (this is necessary because some public zencore headers pull in package headers. If you use a more strict package manager than vcpkg then you get compilation errors whenever these includes are pulled in unless you declare the dependency explicitly)
|
| |
|
|
|
| |
this is a header-only library which mostly contains definitions to support different platforms and compilers.
It is part of the zen codebase but is intended to be consumable separately to zenbase etc to support standalone transport plug-ins and similar.
|
| |
|
|
|
|
|
|
|
| |
this change aims to hide logging internals from client code, in order to make it easier to extend and take more control over the logging process in the future.
As a bonus side effect, the generated code is much tighter (net delta around 2.5% on the resulting executable which includes lots of thirdparty code) and should take less time to compile and link.
Client usage via macros is pretty much unchanged. The main exposure client code had to spdlog internals before was the use of custom loggers per subsystem, where it would be common to have `spdlog::logger` references to keep a reference to a logger within a class. This is now replaced by `zen::LoggerRef` which currently simply encapsulates an actual `spdlog::logger` instance, but this is intended to be an implementation detail which will change in the future.
The way the change works is that we now handle any formatting of log messages in the zencore logging subsystem instead of relying on `spdlog` to manage this. We use the `fmt` library to do the formatting which means the client usage is identical to using `spdlog`. The formatted message is then forwarded onto any sinks etc which are still implememted via `spdlog`.
|
| |
|
|
|
|
|
|
|
|
| |
* set MaxBlockCount at init
* properly calculate total size
* basic blockstore compact blocks test
* correct detection of block swap
* Use one implementation for CreateRandomBlob
* reduce some data sets to increase speed of tests
* reduce test time
* rename BlockStoreCompactState::AddBlock -> BlockStoreCompactState::IncludeBlock
|
| |
|
| |
removed websocket support since it is not used right now and is unlikely to be used in the future
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Feature: New http endpoint for background jobs `/admin/jobs/status` which will return a response listing the currently active background jobs and their status
- Feature: New http endpoint for background jobs information `/admin/jobs/status/{jobid}` which will return a response detailing status, pending messages and progress status
- GET will return a response detailing status, pending messages and progress status
- DELETE will mark the job for cancelling and return without waiting for completion
- If status returned is "Complete" or "Aborted" the jobid will be removed from the server and can not be queried again
- Feature: New zen command `jobs` to list, get info about and cancel background jobs
- If no options are given it will display a list of active background jobs
- `--jobid` accepts an id (returned from for example `oplog-export` with `--async`) and will return a response detailing status, pending messages and progress status for that job
- `--cancel` can be added when `--jobid` is given which will request zenserver to cancel the background job
- Feature: oplog import and export http rpc requests are now async operations that will run in the background
- Feature: `oplog-export` and `oplog-import` now reports progress to the console as work progress by default
- Feature: `oplog-export` and `oplog-import` can now be cancelled using Ctrl+C
- Feature: `oplog-export` and `oplog-import` has a new option `--async` which will only trigger the work and report a background job id back
|
| |
|
|
|
|
| |
* bump vcpkg version
* fmt lib 10 fixes
* xmake dependencies (with linux workarounds)
* changelog
|
| | |
|
| |
|
| |
ATL has been used here and there as a convenience. Given that this is a legacy component and not always something which gets installed along with the compiler we hereby remove the dependency altogether in favour of our own simple wrappers
|
| |
|
|
| |
This reverts commit b8a305713892cb23907ca61e07a70bdfb73ac2f5.
|
| | |
|
|
|
* moved source directories into `/src`
* updated bundle.lua for new `src` path
* moved some docs, icon
* removed old test trees
|