aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
...
* HTTP request logging (#6)Stefan Boberg2024-03-144-31/+108
| | | this change adds support for tracing http payloads when using the asio path. This was already supported when using the `--http=plugin` path and this change moves some code into a shared class for reuse.
* fix potential partially written files (#2)Dan Engelbrecht2024-03-135-17/+47
| | | | * Make sure WriteFile() does not leave incomplete files * use TemporaryFile and MoveTemporaryIntoPlace to avoid leaving partial files on error
* http request parser safety (#664)Dan Engelbrecht2024-03-122-3/+9
| | | | * make sure we don't add more headers than we support * don't capture for loop variables by reference for async work
* fix zenserver state macos (#669)Dan Engelbrecht2024-03-103-8/+48
| | | | | | - Shared memory for zenserver state may hang around after all zenserver processes exit - make sure we find a valid entry in `zen up` before bailing - Httpasio add retry for desired port - Httpasio only call listen() once - Httpasio explicitly close acceptor sockets
* Add WriteMeasuredVarUInt to avoid measuring ints twice before writing (#665)Dan Engelbrecht2024-03-062-26/+53
| | | | Avoid double resize of buffer in CbWriter::SetName and CbWriter::AddBinary Add WriteMeasuredVarUInt to avoid measuring ints twice before writing
* some asio HTTP server tweaks for correctness (#663)Stefan Boberg2024-03-051-9/+11
|
* Make sure we wait for all scheduled tasks to complete before throwing ↵Dan Engelbrecht2024-02-287-62/+145
| | | | | exceptions further (#662) Bugfix: We must not throw exceptions to calling function until all async work we spawned has returned
* add disk caching to block move (#661)Dan Engelbrecht2024-02-271-23/+36
| | | * add disk caching to block move
* improved block store logging and more gcv2 tests (#659)Dan Engelbrecht2024-02-277-114/+215
| | | | * improved gc/blockstore logging * more gcv2 tests
* remove reference caching (#658)Dan Engelbrecht2024-02-277-698/+197
| | | * remove reference caching
* hashing fixes (#657)Dan Engelbrecht2024-02-2611-97/+173
| | | | | * move structuredcachestore tests to zenstore-test * Don't materialize entire files when hashing if it is a large files * rewrite CompositeBuffer::Mid to never materialize buffers
* adding context to http.sys error messageStefan Boberg2024-02-201-5/+15
| | | | added some context to http.sys API call error reporting
* Keep track of added ops during GCV2 instead of rescanning full oplog when ↵Dan Engelbrecht2024-02-132-8/+25
| | | | added ops are detected (#652)
* Added --copy-log, --copy-cache-log and copy-http-log option to zen logs ↵Dan Engelbrecht2024-02-125-20/+81
| | | | | | command (#651) * be safe and use ToText() rather than AsText() when getting http responses * Added `--copy-log`, `--copy-cache-log` and `copy-http-log` option to zen logs command to copy logs from a local running zen server instance
* Save compressed large attachments to temporary files on disk (#650)Dan Engelbrecht2024-02-123-189/+219
| | | | | | | | | * Save large compressed large attachments to temporary files on disk * bump oplog block max size up to 64Mb again * Make sure CompositeBuffer::AppendBuffers actually moves inputs when it should * removed parallell execution of fetching payload for block assembly it was not actually helping and added complexity * make sure we move/release payload buffers as soon as possible * make sure we don't read in full large attachments to memory when computing hash
* Add testcase and fix for ASIO ipv4 localhost support (#649)zousar2024-02-062-14/+86
| | | - Bugfix: Implement two listening sockets in ASIO (ipv4+ipv6) when either we start with `--http-forceloopback` or we resort to that mode because of a failure to bind to the "any" address
* compress large attachments on demand (#647)Dan Engelbrecht2024-02-056-390/+561
| | | | | | | - Improvement: Speed up oplog export by fetching/compressing big attachments on demand - Improvement: Speed up oplog export by batch-fetcing small attachments - Improvement: Speed up oplog import by batching writes of oplog ops - Improvement: Tweak oplog export default block size and embed size limit - Improvement: Add more messaging and progress during oplog import/export
* respond with BadRequest result instead of throwing exception on bad request ↵Dan Engelbrecht2024-02-053-17/+45
| | | | input (#648)
* added robin-map dependency to zenutilStefan Boberg2024-02-011-1/+1
|
* changed RPC recording to MPSC setup (#638)Stefan Boberg2024-01-314-92/+176
| | | fixes rare race condition when using RPC recording for long periods of time
* only try to traverse an objectstore bucket if it really exists (#646)Dan Engelbrecht2024-01-311-0/+1
| | | * Only try to traverse a objectstore bucket if it really exists
* Actually thow exception if we can't parse the jobid when starting async job ↵Dan Engelbrecht2024-01-313-4/+15
| | | | | | (#645) Return system error code on exception throw in zen command Clean up HttpClient::Response::ErrorMessage to remove redundant ": "
* improve oplog export logging (#644)Dan Engelbrecht2024-01-319-126/+232
| | | | | | - Improvement: More details in oplog import/export logs - Improvement: Switch from Download to Get when fetching Refs from Jupiter as they can't be resumed anyway and streaming to disk is redundant - Bugfix: Make sure we clear read callback when doing Put in HttpClient to avoid timeout due to not sending data when reusing sessions - Bugfix: Respect `--ignore-missing-attachments` in `oplog-export` command when loose file is missing on disk
* fix response error conversion (#643)Dan Engelbrecht2024-01-293-6/+7
| | | | * make sure we properly convert compact-binary results to text when receiving errors * log fix
* exception on basicfile read error (#642)Dan Engelbrecht2024-01-292-10/+30
| | | | | * Change BasicFile::Read to throw exception like IoBuffer - Don't ASSERT on dwNumberOfBytesRead == NumberOfBytesToRead - throw exception with details instead - Use proper return type for pread()
* add ignore-missing-attachments option to oplog export (debugging tool) (#641)Dan Engelbrecht2024-01-257-39/+68
| | | | | | | * add ignore-missing-attachments option to oplog export (debugging tool) * add more status codes to do retry for in http client * add missing X-Jupiter-IoHash header for jupiter PutRef * reduce oplog block size to reduce amount of redundant chunks to download * improved logging
* Use proper format for range request header (#640)Dan Engelbrecht2024-01-241-4/+10
| | | | Clear header callback after use Use separate temp-vector for headers
* Add retry with optional resume logic to HttpClient::Download (#639)Dan Engelbrecht2024-01-249-1258/+688
| | | | | | | - Improvement: Refactored Jupiter upstream to use HttpClient - Improvement: Added retry and resume logic to HttpClient - Improvement: Added authentication support to HttpClient - Improvement: Clearer logging in GCV2 compact of FileCas/BlockStore - Improvement: Size details in oplog import logging
* oplog import/export improvements (#634)Dan Engelbrecht2024-01-234-226/+371
| | | | * improve feedback from oplog import/export * improve oplog save performance
* add --ignore-missing-attachments to oplog-import command (#637)Dan Engelbrecht2024-01-225-53/+98
|
* improved errors from jupiter upstream (#636)Dan Engelbrecht2024-01-223-6/+82
| | | * get more detailed error messages from jupiter upstream
* jobqueue - allow multiple threads to report progress/messages (#635)Dan Engelbrecht2024-01-224-19/+37
| | | jobqueue - add AbortReason and properly propagate error when running async command
* make sure to advance read buffer pointer in BasicFileWriter::Write (#633)Dan Engelbrecht2024-01-221-1/+3
|
* Use correct HTTP range values. (#632)Per Larsson2024-01-171-1/+1
|
* reinstate formatter settings for file sink (#631)Stefan Boberg2024-01-171-0/+9
| | | fixes issue introduced in #615
* remove use of <random> in stats (#628)Stefan Boberg2024-01-102-6/+45
| | | | the previous implementation was quite slow due to use of mt and uniform_distribution.
* release RPC recording memory early (#627)Stefan Boberg2023-12-201-2/+7
| | | previously the segment would call `clear()` on the `m_Entries` vector but this does not release the backing memory so we have to do the `swap` dance
* separate RPC processing from HTTP processing (#626)Stefan Boberg2023-12-2031-1789/+2002
| | | | | | * moved all RPC processing from HttpStructuredCacheService into separate CacheRpcHandler class in zenstore * move package marshaling to zenutil. was previously in zenhttp/httpshared but it's useful in other contexts as well where we don't want to depend on zenhttp * introduced UpstreamCacheClient, this provides a subset of functions on UpstreamCache and lives in zenstore
* move cachedisklayer and structuredcachestore into zenstore (#624)Stefan Boberg2023-12-1912-28/+27
|
* Don't capture local variables in loop by reference (#623)Dan Engelbrecht2023-12-191-27/+27
| | | * Don't capture local variables in loop by reference
* ensure we can build without trace (#619)Stefan Boberg2023-12-195-2/+13
| | | | `xmake config -zentrace=n` would previously not build cleanly
* various TSAN/ASAN/LeakAnalyzer fixes (#622)Stefan Boberg2023-12-196-10/+32
| | | | | | | | | * fix JobQueue test threading issue. The inner job queued with `QueueJob` would reference `I` from inside the captured closure which would subsequently disappear * made sure application exit is thread safe * don't try to access string data out of bounds * keep-alive flag is accessed from multiple threads * fix memory leaks in Zen upstream client code * TSAN fixes for Event
* fix ChunkIndexToChunkHash indexing (#621)Stefan Boberg2023-12-192-2/+2
| | | would previously index into a reserved-but-not-sized vector which is bad but not crash-inducing bad
* fix leak in IoBuffer for manifested small chunk (#618)Stefan Boberg2023-12-191-2/+2
| | | | * fix leak in IoBuffer for manifested small chunk. previously it would null out the `m_DataPtr` member on every path from `IoBufferExtendedCore::~IoBufferExtendedCore()` but it only really makes sense to null it out when the buffer has been memory mapped
* cache RPC recorder threading fixes (#617)Stefan Boberg2023-12-193-54/+110
| | | | | * ensure all access to m_Entries is done while holding lock * RPC recorder concurrency fixes - setup/teardown of recorder needs to be done while holding an exclusive lock. Calls into recorder should be done while holding a shared lock.
* Fix crash bug when trying to inspect non-open block file in GC (#614)Dan Engelbrecht2023-12-182-7/+20
|
* Make sure we initialize the pattern of FileSink before it is added as a ↵Dan Engelbrecht2023-12-181-12/+8
| | | | usable logger (#615)
* log each block that is compacted (#613)Dan Engelbrecht2023-12-151-0/+5
|
* fixed v2 rpc recording issue with >4GB data per segment (#612)Stefan Boberg2023-12-154-146/+323
| | | | | | | | | * fixed v2 rpc recording issue with >4GB data per segment * implemented recovery logic to deal with partial RPC recordings * added check for invalid/null requests in RPC replay * also made sure at least one worker thread is configured * fix problem where "null" requests would cause infinite loop! * added basic RPC recorder tests
* implement cache recording segment split by age (#611)Stefan Boberg2023-12-143-17/+110
| | | - also fixes weird DateTime/TimeSpan comparison operator
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-13 22:10:03 +0000