aboutsummaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
...
| * handle error in error log (#422)Dan Engelbrecht2023-09-251-15/+46
| | | | | | * do not allow exceptions to leak from Sentry error reporting or SentryAssertImpl::OnAssert
| * Add runtime status/control of logging (#419)Dan Engelbrecht2023-09-2211-29/+277
| | | | | | | | | | | | | | | | | | | | | | - Feature: New endpoint `/admin/logs` to query status of logging and log file locations and cache logging - `enablewritelog`=`true`/`false` parameter to control cache write logging - `enableaccesslog`=`true`/`false` parameter to control cache access logging - `loglevel` = `trace`/`debug`/`info`/`warning`/`error` - Feature: New zen command `logs` to query/control zen logging - No arguments gives status of logging and paths to log files - `--cache-write-log` `enable`/`disable` to control cache write logging - `--cache-access-log` `enable`/`disable` to control cache access logging - `--loglevel` `trace`/`debug`/`info`/`warning`/`error` to set debug level
| * added name to http.sys request queue (#417)Stefan Boberg2023-09-221-1/+33
| | | | | | | | | | | | * added name to http.sys request queue * changed so that rejected requests are answered with a 403 response instead of just dropping the connection
| * Collect all zen admin-related commands into admin.h/.cpp (#418)Dan Engelbrecht2023-09-2233-140/+117
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * move commands in scrub.h/cpp to admin_cmd.h/cpp * move job command into admin_cmd.h/.cpp * admin -> admin_cmd * bench -> bench_cmd * cache -> cache_cmd * copy -> copy_cmd * dedup -> dedup_cmd * hash -> hash_cmd * print -> print_cmd * projectstore -> projectstore_cmd * rpcreplay -> rpcreplay_cmd * serve -> serve_cmd * status -> status_cmd * top -> top_cmd * trace -> trace_cmd * up -> up_cmd * version -> version_cmd
| * add trace command to enable/disable tracing at runtime (#416)Dan Engelbrecht2023-09-227-38/+248
| | | | | | | | | | * add trace command to enable/disable tracing at runtime * rework tracing init/start/stop * changelog
* | sort commands for cleaner mergesStefan Boberg2023-09-261-2/+2
|/
* Improvement: Add names to background jobs for easier debugging (#412)Dan Engelbrecht2023-09-206-113/+176
| | | | Improvement: Background jobs now temporarily sets thread name to background job name while executing Improvement: Background jobs tracks worker thread id used while executing
* controlled zenserver shutdown (#413)Dan Engelbrecht2023-09-201-20/+36
| | | * Do controlled shutdown of zenserver catching any exception thrown during shutdown
* added info verb to vfs command help textStefan Boberg2023-09-201-1/+1
|
* VFS implementation for local storage service (#396)Stefan Boberg2023-09-2031-27/+2553
| | | currently, only Windows (using Projected File System) is supported
* increase jupiter oplog import/export connection timeout to 30 min (#411)Dan Engelbrecht2023-09-191-1/+1
| | | | * increase jupiter oplog import/export connection timeout to 30 min * changelog
* Add retry if FinalizeRef responds with non-empty "Needs" attachments (#409)Dan Engelbrecht2023-09-195-261/+377
| | | | | * Add retry if FinalizeRef responds with non-empty "Needs" attachments * better logging/progress report * changelog
* handle errors in spdlog gracefully (#410)Dan Engelbrecht2023-09-191-0/+29
| | | | * handle errors in spdlog gracefully - try to report and avoid termination * changelog
* add DiskWriteBlocker to structured cache store log writer (#408)Dan Engelbrecht2023-09-153-32/+45
| | | | | * add DiskWriteBlocker to structured cache store log writer * changelog
* add more trace scopes (#362)Dan Engelbrecht2023-09-1514-282/+438
| | | | | * more trace scopes * Make sure ReplayLogEntries uses the correct size for oplog buffer * changelog
* bugfix: http.sys async response handling race (#406)Stefan Boberg2023-09-151-1/+14
| | | | under heavy load, an async response could end up deleting the HTTP transaction object before the issuing call had completed
* More statistics for Cache, Project Store and Cid Store (#405)Dan Engelbrecht2023-09-146-15/+105
| | | | | Cache: requestcount, badrequestcount, writes Project Store: requestcount Cid Store: cidhits, cidmisses, cidwrites
* http and httpsys config options (#401)Dan Engelbrecht2023-09-149-31/+90
| | | | * Added `--http-threads`, `--httpsys-async-work-threads`, `--httpsys-enable-request-logging` and `--httpsys-enable-async-response` command line options to zenserver * remove unused CreateHttpSysServer
* job queue and async oplog-import/export (#395)Dan Engelbrecht2023-09-1322-153/+1369
| | | | | | | | | | | | | | | | - Feature: New http endpoint for background jobs `/admin/jobs/status` which will return a response listing the currently active background jobs and their status - Feature: New http endpoint for background jobs information `/admin/jobs/status/{jobid}` which will return a response detailing status, pending messages and progress status - GET will return a response detailing status, pending messages and progress status - DELETE will mark the job for cancelling and return without waiting for completion - If status returned is "Complete" or "Aborted" the jobid will be removed from the server and can not be queried again - Feature: New zen command `jobs` to list, get info about and cancel background jobs - If no options are given it will display a list of active background jobs - `--jobid` accepts an id (returned from for example `oplog-export` with `--async`) and will return a response detailing status, pending messages and progress status for that job - `--cancel` can be added when `--jobid` is given which will request zenserver to cancel the background job - Feature: oplog import and export http rpc requests are now async operations that will run in the background - Feature: `oplog-export` and `oplog-import` now reports progress to the console as work progress by default - Feature: `oplog-export` and `oplog-import` can now be cancelled using Ctrl+C - Feature: `oplog-export` and `oplog-import` has a new option `--async` which will only trigger the work and report a background job id back
* scan oplog object for fields (#397)Dan Engelbrecht2023-09-131-89/+130
| | | | * scan oplog object for fields * Read all oplog entries but only read op data and get mapping for latest op of each key
* issue warning instead of assert on bad data in cid store (#400)Dan Engelbrecht2023-09-131-10/+14
| | | * issue warning instead of assert on bad data in cid store
* fix url parsing crash (#399)Dan Engelbrecht2023-09-131-1/+1
| | | * Don't index out of string_view range when parsing URI in httpsys
* minor: log fixStefan Boberg2023-09-131-1/+1
|
* Hash utility functions (#385)Stefan Boberg2023-09-133-0/+51
| | | | * added std::hash implementation for Guid and Oid types * Added CombineHashes implementation
* ZenCacheStore is now reference counted (#398)Stefan Boberg2023-09-134-9/+20
| | | this change also adds a GetNamespaces function which may be used to enumerate all currently known cache namespaces
* incremental oplog upload for block-based targets (#392)Dan Engelbrecht2023-09-1210-74/+258
| | | | | | | * add option for base container for oplog export read base oplog and fetch known blocks * reuse blocks if a known block has 80+ % usage * changelog * better logging and added base to remotestore descriptions
* Make sure error logging or destructors don't throw exception when trying to ↵Dan Engelbrecht2023-09-127-66/+125
| | | | | get file name from handle (#393) - Bugfix: Make sure error logging or destructors don't throw exception when trying to get file name from handle
* gracefully handle errors when writing cache log (#391)Dan Engelbrecht2023-09-111-29/+50
| | | | | | | * gracefully handle errors when writing cache log * changelog * fix log message
* add `cache-write-log` and `cache-access-log´ configuration options (#394)Dan Engelbrecht2023-09-114-10/+33
| | | * add `cache-write-log` and `cache-access-log´ configuration options
* better sorting of attachments in oplog blocks (#390)Dan Engelbrecht2023-09-111-13/+54
| | | | | - sort attachments based on (first) associated op - sort attachments based on key of op to get a more consistent order rather than order of ops written - keep attachments from same op in same block
* add console logging to zen command (#389)Dan Engelbrecht2023-09-083-5/+33
| | | | properly set trace log level if IsVerbose add log category to http client
* multithread file realization in oplog-mirror (#388)Dan Engelbrecht2023-09-083-458/+464
| | | | convert project store commands to use http client use MakeCbObjectPayload everywhere in project store commands
* Extend http client (#387)Dan Engelbrecht2023-09-085-93/+623
| | | * extend http client with configuration, headers, parameters and disk streaming upload/download
* stream oplog attachments from jupiter (#384)Dan Engelbrecht2023-09-0613-215/+297
| | | | | | | | | | * stream large downloads from jupiter to temporary file * rework DeleteOnClose - top level marks file for delete and if lower level parts wants to keep it it clears that flag * changelog * log number of attachments to download * add delay on jupiter request failure when retrying * make sure we upload all attachments even if Needs are empty when ForceUpload is true release TempAttachment as soon as it is used * sort attachments so we get predictable blocks for the same oplog
* retry file create (#383)Dan Engelbrecht2023-09-045-5/+73
| | | | | * add retry logic when creating files * only write disk usage log if disk writes are allowed * changelog
* add `--write-config` to zenserver options (#382)Dan Engelbrecht2023-09-014-11/+115
| | | | | | | | | | - Feature: `zen up` command has two new command line options - `--config <file_path>` tells zenserver to start with a specific config file - `--owner-pid <pid>` tells zenserver to start with a owning process id - Feature: `zen attach` command to add additional owning processes to a running zenserver instance - `--owner-pid <pid>` adds pid to running zenserver instance list of owning processes - Feature: `--write-config` command line option for zenserver - `--write-config <file_path>` path to a file which will contain a lua config file for zenserver combining all command line options and optional lua config files - Improvement: `zen up` command will check if zenserver is currently running before starting up a new instance
* lua config file improvements (#379)Dan Engelbrecht2023-08-312-300/+795
| | | | | | - Bugfix: All options given on command line now overrides lua config file settings - Improvement: All options available from command line can now be configured in the lua config file (with a few exceptions such as `owner-pid`, `install` and `uninstall`) We can now also generate a Lua config (just string so far) that includes all options given on command line together with any optional Lua input config file. The idea here is to be able to save config options in the future.
* project store gc deadlock (#381)Dan Engelbrecht2023-08-313-40/+57
| | | | * Trying to recursively take a shared RWLock while another thread is waiting for a exclusive lock results in a deadlock. Forward the shared lock so we don't have to grab it recursively. * changelog
* fix zen server state mac mmap (#380)Dan Engelbrecht2023-08-311-1/+1
| | | | * use MAP_SHARED when initializing read only access of shared memory * changelog
* add helper functions in HttpProjectService to help with readability (#378)Dan Engelbrecht2023-08-292-1195/+1326
|
* Minor: Make sure to reset cache logging worker thread event to avoid ↵Dan Engelbrecht2023-08-241-0/+1
| | | | busy-looping looking for more work
* Add `--embedloosefiles` option to `oplog-export` (#376)Dan Engelbrecht2023-08-245-73/+253
| | | | * Add `--embedloosefiles` option to `oplog-export` which adds loose files to the export, removing need to call `oplog-snapshot` * Retain `ServerPath` in oplog when performing `oplog-snapshot`. This is a short-term fix for current incompatability with the UE cooker.
* crash in process cache (#375)Dan Engelbrecht2023-08-231-36/+70
| | | | * - Bugfix: Fix OpenProcessCache state error causing assert/error * changelog
* safer gc on low disk (#373)Dan Engelbrecht2023-08-222-37/+46
| | | * - Improvement: Make sure we have disk space available to do GC and use reserve up front if need be
* Fix construction order in OpenProcessCache (#374)Dan Engelbrecht2023-08-221-1/+1
|
* make sure trace::ThreadRegister is inside ZEN_WITH_TRACEDan Engelbrecht2023-08-221-1/+2
|
* use better hash function for better distribution in IoBuffer g_MappingLocks ↵Dan Engelbrecht2023-08-211-1/+10
| | | | | | (#370) * use better hash function for better distribution in IoBuffer g_MappingLocks * changelog
* use robinmap in compact cas (#368)Dan Engelbrecht2023-08-214-14/+24
| | | | | * Use robin-map in compactcas for 30% faster CasContainerStrategy::CollectGarbage * use robin_set in ProjectStore::Oplog::GatherReferences and BlockStore::ReclaimSpace * changelog
* use atexit hook to shut down tracing (#369)Dan Engelbrecht2023-08-211-1/+9
|
* buffered file reading for oplog (#366)Dan Engelbrecht2023-08-213-5/+233
| | | | | * add BasicFileBuffer for buffered read of BasicFile * Use BasicFileBuffer when reading oplog * changelog
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-14 10:26:30 +0000