aboutsummaryrefslogtreecommitdiff
path: root/src/zenhttp
Commit message (Collapse)AuthorAgeFilesLines
* improved assert (#37)Dan Engelbrecht2024-04-048-25/+56
| | | | - Improvement: Add file and line to ASSERT exceptions - Improvement: Catch call stack when throwing assert exceptions and log/output call stack at important places to provide more context to caller
* zenremoteprojectstore with httpclient (#35)Dan Engelbrecht2024-04-032-18/+40
| | | | | | - Bugfix: Fix log of Success/Failure for oplog import - Improvement: Use HttpClient when doing oplog export/import with a zenserver as a remote target. Includes retry logic - Improvement: Increase the retry count to 4 (5 attempts in total) when talking to Jupiter for oplog export/import
* add yaml serialization support (#3)Stefan Boberg2024-03-261-0/+10
| | | | | | | this change adds serialization of payloads as YAML, but not parsing. The implementation is somewhat based on the JSON path, and may be collapsed eventually as it is possible to serialize JSON format using the same code it also separates out the JSON serialization into a separate file for ease of maintenance any HTTP request response may be formatted as yaml by using a `.yaml` suffix or an `Accept: text/yaml` header
* add filter to projectstore entries request (#25)Dan Engelbrecht2024-03-262-0/+36
| | | | | | | * Add HttpServerRequest::Decode to decode http request parameters * Add support to filter projectstore `entries` request using the `fieldfilter` where the wanted fields are comma (,) delimited * Add support for responding with compressed payloads for projectstore `entries` requests by adding AcceptType `compressed-binary` to the request header * Add support for responding with compressed payloads for projectstore `files` requests by adding AcceptType `compressed-binary` to the request header * Add support for responding with compressed payloads for projectstore `chunkinfo` requests by adding AcceptType `compressed-binary` to the request header
* HTTP request logging (#6)Stefan Boberg2024-03-144-31/+108
| | | this change adds support for tracing http payloads when using the asio path. This was already supported when using the `--http=plugin` path and this change moves some code into a shared class for reuse.
* http request parser safety (#664)Dan Engelbrecht2024-03-121-1/+7
| | | | * make sure we don't add more headers than we support * don't capture for loop variables by reference for async work
* fix zenserver state macos (#669)Dan Engelbrecht2024-03-101-5/+24
| | | | | | - Shared memory for zenserver state may hang around after all zenserver processes exit - make sure we find a valid entry in `zen up` before bailing - Httpasio add retry for desired port - Httpasio only call listen() once - Httpasio explicitly close acceptor sockets
* some asio HTTP server tweaks for correctness (#663)Stefan Boberg2024-03-051-9/+11
|
* adding context to http.sys error messageStefan Boberg2024-02-201-5/+15
| | | | added some context to http.sys API call error reporting
* Add testcase and fix for ASIO ipv4 localhost support (#649)zousar2024-02-061-14/+59
| | | - Bugfix: Implement two listening sockets in ASIO (ipv4+ipv6) when either we start with `--http-forceloopback` or we resort to that mode because of a failure to bind to the "any" address
* Actually thow exception if we can't parse the jobid when starting async job ↵Dan Engelbrecht2024-01-311-2/+7
| | | | | | (#645) Return system error code on exception throw in zen command Clean up HttpClient::Response::ErrorMessage to remove redundant ": "
* improve oplog export logging (#644)Dan Engelbrecht2024-01-311-9/+9
| | | | | | - Improvement: More details in oplog import/export logs - Improvement: Switch from Download to Get when fetching Refs from Jupiter as they can't be resumed anyway and streaming to disk is redundant - Bugfix: Make sure we clear read callback when doing Put in HttpClient to avoid timeout due to not sending data when reusing sessions - Bugfix: Respect `--ignore-missing-attachments` in `oplog-export` command when loose file is missing on disk
* fix response error conversion (#643)Dan Engelbrecht2024-01-291-1/+1
| | | | * make sure we properly convert compact-binary results to text when receiving errors * log fix
* add ignore-missing-attachments option to oplog export (debugging tool) (#641)Dan Engelbrecht2024-01-251-1/+4
| | | | | | | * add ignore-missing-attachments option to oplog export (debugging tool) * add more status codes to do retry for in http client * add missing X-Jupiter-IoHash header for jupiter PutRef * reduce oplog block size to reduce amount of redundant chunks to download * improved logging
* Use proper format for range request header (#640)Dan Engelbrecht2024-01-241-4/+10
| | | | Clear header callback after use Use separate temp-vector for headers
* Add retry with optional resume logic to HttpClient::Download (#639)Dan Engelbrecht2024-01-242-141/+424
| | | | | | | - Improvement: Refactored Jupiter upstream to use HttpClient - Improvement: Added retry and resume logic to HttpClient - Improvement: Added authentication support to HttpClient - Improvement: Clearer logging in GCV2 compact of FileCas/BlockStore - Improvement: Size details in oplog import logging
* improved errors from jupiter upstream (#636)Dan Engelbrecht2024-01-221-1/+10
| | | * get more detailed error messages from jupiter upstream
* separate RPC processing from HTTP processing (#626)Stefan Boberg2023-12-207-996/+5
| | | | | | * moved all RPC processing from HttpStructuredCacheService into separate CacheRpcHandler class in zenstore * move package marshaling to zenutil. was previously in zenhttp/httpshared but it's useful in other contexts as well where we don't want to depend on zenhttp * introduced UpstreamCacheClient, this provides a subset of functions on UpstreamCache and lives in zenstore
* ensure we can build without trace (#619)Stefan Boberg2023-12-191-0/+4
| | | | `xmake config -zentrace=n` would previously not build cleanly
* various TSAN/ASAN/LeakAnalyzer fixes (#622)Stefan Boberg2023-12-191-1/+3
| | | | | | | | | * fix JobQueue test threading issue. The inner job queued with `QueueJob` would reference `I` from inside the captured closure which would subsequently disappear * made sure application exit is thread safe * don't try to access string data out of bounds * keep-alive flag is accessed from multiple threads * fix memory leaks in Zen upstream client code * TSAN fixes for Event
* http plugin dependency fixes (#590)Stefan Boberg2023-12-051-3/+11
| | | these changes clean up module dependencies and allow the transports subtree to be built standalone (in the UE tree for instance)
* HTTP plugin request debug logging (#587)Stefan Boberg2023-12-0511-128/+207
| | | | | | * added log level control/query to LoggerRef * added debug logging to http plugin implementation * added GetDebugName() to transport plugin interfaces * added debug name to log output
* WinIoThreadPool teardown cleaned up (#580)Stefan Boberg2023-11-301-0/+5
| | | previously would not shut down the threadpool properly which would leave threads around. This was not a problem in practice for our usage since we keep the thread pool alive for the duration of the process but it's better to clean up properly.
* minor: updated transport plugin default ports to match the new defaultsStefan Boberg2023-11-201-3/+3
|
* Make object store endpoint S3 compatible. (#535)Per Larsson2023-11-151-0/+17
| | | | | * Make object store endpoint S3 compatible. * Removed XML pretty printing and set object store endpoint disabled by default.
* package dependency clean-ups (#531)Stefan Boberg2023-11-131-2/+2
| | | | | | | | | | this change just cleans up dependency declarations in xmake.lua files, discovered while exploring a more to xrepo which catches dependency problems since it will not just place all includes in a single directory, unlike vcpkg. * removed spurious asio dependency from zenserver-test * removed rocksdb reference * add missing asio package dependency * removed catch2 reference (no longer available) * added explicit cpr reference * made some zencore package dependencies public (this is necessary because some public zencore headers pull in package headers. If you use a more strict package manager than vcpkg then you get compilation errors whenever these includes are pulled in unless you declare the dependency explicitly)
* option for zenserver - `--http-forceloopback` (#516)Dan Engelbrecht2023-11-096-30/+55
| | | | * New option for zenserver - `--http-forceloopback` which forces opening of the server http server using loopback (local) connection (UE-199776) * add fallback to local connection for asio if we get access denied on public port
* removed duplicate and unused constantsStefan Boberg2023-11-091-8/+0
|
* disk layer gc and error/warnings cleanup (#515)Dan Engelbrecht2023-11-081-1/+1
| | | | | | | - Improvement: Use GC reserve when writing index/manifest for a disk cache bucket when disk is low when available - Improvement: Demote errors to warning for issues that are not critical and we handle gracefully - Improvement: Treat more out of memory errors from windows as Out Of Memory errors Fixed wrong sizeof() statement for compactcas index (luckily the two structs are of same size)
* factored out some compiler definitions etc into zenbase (#517)Stefan Boberg2023-11-073-3/+3
| | | | | this is a header-only library which mostly contains definitions to support different platforms and compilers. It is part of the zen codebase but is intended to be consumable separately to zenbase etc to support standalone transport plug-ins and similar.
* spdlog implementation hiding (#498)Stefan Boberg2023-11-0610-40/+39
| | | | | | | | | this change aims to hide logging internals from client code, in order to make it easier to extend and take more control over the logging process in the future. As a bonus side effect, the generated code is much tighter (net delta around 2.5% on the resulting executable which includes lots of thirdparty code) and should take less time to compile and link. Client usage via macros is pretty much unchanged. The main exposure client code had to spdlog internals before was the use of custom loggers per subsystem, where it would be common to have `spdlog::logger` references to keep a reference to a logger within a class. This is now replaced by `zen::LoggerRef` which currently simply encapsulates an actual `spdlog::logger` instance, but this is intended to be an implementation detail which will change in the future. The way the change works is that we now handle any formatting of log messages in the zencore logging subsystem instead of relying on `spdlog` to manage this. We use the `fmt` library to do the formatting which means the client usage is identical to using `spdlog`. The formatted message is then forwarded onto any sinks etc which are still implememted via `spdlog`.
* added missing includes (#504)Stefan Boberg2023-10-273-0/+7
| | | | | this change adds some includes to files which "inherit" includes from elsewhere this was exposed on another branch when removing some heavy dependencies from central headers
* improve error handling when parsing http requests (#489)Dan Engelbrecht2023-10-201-6/+36
| | | * catch exceptions in HttpRequestParser::OnMessageComplete() callback
* added HttpClient::GetBaseUriStefan Boberg2023-10-201-1/+2
| | | | mostly useful when reporting errors
* Cache (rpc) activitity recording improvements (#482)Stefan Boberg2023-10-201-1/+1
| | | | | | | this adds a new RPC recording path aimed at more continuous recording and analysis of recorded sessions the new strategy is implemented alongside the original in order to retain the ability to read the older format the main difference between v2 and v1 is that the new strategy splits the recording into segments which are independent from each other. This is done to enable long running sessions with automatic disk cleanup (not implemented yet), appending to an existing recording (not implemented) and/or partial analysis and processing. The recorder will start a new segment when some criteria is fulfilled, including the number of files in the segment directory, disk footprint etc
* minor: fixed some typosStefan Boberg2023-10-171-3/+2
|
* support for multiple http servers (#473)Stefan Boberg2023-10-1311-361/+484
| | | | | | * added support for having multiple http servers active in one session * added configuration API to pluggable transports * removed pimpl pattern from some pluggable transports implementations
* improved http.sys initialization diagnostics and amended logic for dedicated ↵Stefan Boberg2023-10-131-40/+71
| | | | servers (#475)
* restructured zenhttp (#472)Stefan Boberg2023-10-1315-259/+4
| | | separating the http server implementations into a directory and moved diagsvcs into zenserver since it's somewhat hard-coded for it
* Merge remote-tracking branch 'origin/main' into zs/default-port-changeStefan Boberg2023-10-127-27/+83
|\
| * restructured transports SDK for easier UE integration (#470)Stefan Boberg2023-10-121-1/+1
| |
| * adjust resource usage for dedicated servers (#466)Stefan Boberg2023-10-126-26/+82
| | | | | | | | | | | | | | | | | | when dedicated mode is enabled via `--dedicated` or `server.dedicated` then we tune http.sys server settings to be more suitable for a shared server initially we tune two things * the thread pool used to service I/O requests allows a larger number of threads to be created when needed. The minimum thread count is unchanged but in dedicated server mode we double the maximum number of threads allowed * the http.sys request queue length (`HttpServerQueueLengthProperty`) is increased to 50,000 in dedicated mode. The regular default is 1,000
* | Change default port to 8558zousar2023-10-112-4/+4
|/ | | | Changes the default port without altering config for shared instances.
* pluggable asio transport (#460)Stefan Boberg2023-10-1113-188/+562
| | | | | | | | | added pluggable transport based on asio. This is in an experimental state and is not yet a replacement for httpasio even though that is the ultimate goal also moved plugin API header into dedicated part of the tree to clarify that it is meant to be usable in isolation, without any dependency on zencore et al moved transport implementations into dedicated source directory in zenhttp note that this adds code to the build but nothing should change at runtime since the instantiation of the new code is conditional and is inactive by default
* hide HttpAsioServer interface behind factory function (#463)Stefan Boberg2023-10-113-24/+29
|
* updated plugin API class names (#462)Stefan Boberg2023-10-118-79/+78
| | | | | | | | | * renamed some interfaces to improve pluggable transport API TransportConnectionInterface -> TransportConnection TransportPluginInterface -> TransportPlugin TransportServerConnectionHandler -> TransportServerConnection TransportServerInterface -> TransportServer
* suppress no-op auth debug loggingStefan Boberg2023-10-101-1/+6
|
* experimental pluggable transport support (#436)Stefan Boberg2023-10-1012-41/+1670
| | | this change adds a `--http=plugin` mode where we support pluggable transports. Currently this defaults to a barebones blocking winsock implementation but there is also support for dynamic loading of transport plugins, which will be further developed in the near future.
* removed websocket protocol support(#445)Stefan Boberg2023-10-044-1906/+1
| | | removed websocket support since it is not used right now and is unlikely to be used in the future
* factored out http parser from asio into separate files (#444)Stefan Boberg2023-10-033-450/+495
| | | | factored out http request parsing from httpasio into separate files to enable code to be reused for different transports
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-11 12:21:26 +0000