aboutsummaryrefslogtreecommitdiff
path: root/src/zencore/jobqueue.cpp
Commit message (Collapse)AuthorAgeFilesLines
* speed up tests (#555)Dan Engelbrecht2025-10-061-4/+4
| | | | | | | | | | | | * faster FileSystemTraversal test * faster jobqueue test * faster NamedEvent test * faster cache tests * faster basic http tests * faster blockstore test * faster cache store tests * faster compactcas tests * more responsive zenserver launch * tweak worker pool sizes in tests
* add EMode to WorkerTheadPool to avoid thread starvation (#492)Dan Engelbrecht2025-09-101-32/+36
| | | - Improvement: Add a new mode to worker thread pools to avoid starvation of workers which could cause long stalls due to other work begin queued up. UE-305498
* use new builds api for oplogs (#464)Dan Engelbrecht2025-08-121-1/+21
| | | | - Improvement: Refactored jupiter oplog export code to reuse builds jupiter wrapper classes - Improvement: If `zen builds`, `zen oplog-import` or `zen oplog-import` command fails due to a http error, the return code for the program will be set to the error/status code
* Improved oplog import/export progress indicator at commandline (#206)Dan Engelbrecht2024-11-061-23/+41
| | | | Nicer progress bar during oplog import/export Verify that oplog has not been deleted from disk behind our back
* improved assert (#37)Dan Engelbrecht2024-04-041-3/+14
| | | | - Improvement: Add file and line to ASSERT exceptions - Improvement: Catch call stack when throwing assert exceptions and log/output call stack at important places to provide more context to caller
* add a limit to the number of times we attempt to finalize (#22)Dan Engelbrecht2024-03-251-1/+0
| | | | | | - Improvement: Add limit to the number of times we attempt to finalize and exported oplog - Improvement: Switch to large thread pool when executing oplog export/import - Improvement: Clean up reporting of missing attachments in oplog export/import - Improvement: Remove double-reporting of abort reason for oplog export/import
* jobqueue - allow multiple threads to report progress/messages (#635)Dan Engelbrecht2024-01-221-6/+8
| | | jobqueue - add AbortReason and properly propagate error when running async command
* various TSAN/ASAN/LeakAnalyzer fixes (#622)Stefan Boberg2023-12-191-3/+4
| | | | | | | | | * fix JobQueue test threading issue. The inner job queued with `QueueJob` would reference `I` from inside the captured closure which would subsequently disappear * made sure application exit is thread safe * don't try to access string data out of bounds * keep-alive flag is accessed from multiple threads * fix memory leaks in Zen upstream client code * TSAN fixes for Event
* Improvement: Add names to background jobs for easier debugging (#412)Dan Engelbrecht2023-09-201-56/+99
| | | | Improvement: Background jobs now temporarily sets thread name to background job name while executing Improvement: Background jobs tracks worker thread id used while executing
* job queue and async oplog-import/export (#395)Dan Engelbrecht2023-09-131-0/+486
- Feature: New http endpoint for background jobs `/admin/jobs/status` which will return a response listing the currently active background jobs and their status - Feature: New http endpoint for background jobs information `/admin/jobs/status/{jobid}` which will return a response detailing status, pending messages and progress status - GET will return a response detailing status, pending messages and progress status - DELETE will mark the job for cancelling and return without waiting for completion - If status returned is "Complete" or "Aborted" the jobid will be removed from the server and can not be queried again - Feature: New zen command `jobs` to list, get info about and cancel background jobs - If no options are given it will display a list of active background jobs - `--jobid` accepts an id (returned from for example `oplog-export` with `--async`) and will return a response detailing status, pending messages and progress status for that job - `--cancel` can be added when `--jobid` is given which will request zenserver to cancel the background job - Feature: oplog import and export http rpc requests are now async operations that will run in the background - Feature: `oplog-export` and `oplog-import` now reports progress to the console as work progress by default - Feature: `oplog-export` and `oplog-import` can now be cancelled using Ctrl+C - Feature: `oplog-export` and `oplog-import` has a new option `--async` which will only trigger the work and report a background job id back