aboutsummaryrefslogtreecommitdiff
path: root/zenserver/projectstore.h
Commit message (Collapse)AuthorAgeFilesLines
* Clean up project store file structure (#218)Dan Engelbrecht2023-02-011-313/+0
| | | | * move project store to separate folder * moved import/export project commands into projectstore cmd files
* Better error responses/logging in project store request (#217)Dan Engelbrecht2023-01-261-17/+21
| | | | * Better error responses/logging in project store request * changelog
* Add info (GET) endpoints for structured cache (#211)Dan Engelbrecht2023-01-111-0/+1
| | | | | | | * Add GET requests on cache/namespace/bucket level * Add root route for project store requests (same as /list) * Add markerpath to oplog info object * Add totalsize, opcount and expired to oplog info * Changelog
* oplog level GC (#209)Dan Engelbrecht2022-12-141-14/+26
| | | | | | | | Adds check for marker file supplied by UE to see if an oplog is expired (user has deleted the corresponding cooked folder). Fixed concurrency vulnerabilities is project store related to oplogs. * fix concurrency vulnerabilities * propagate lifetime file path * oplog level gc * changelog
* Fix disk usage stats (#194)Dan Engelbrecht2022-11-241-1/+3
| | | | | | | | * Improve tracking of used disk space for filecas and compactcas Add tracking of used disk space for project store Remove ZenCacheStore as GcStorage/GcContributor - underlying ZenCacheNamespace instances register themselves directly - removing this also fixes double reporting of GcStorageSize for namespaces * changelog
* Add `import-project` and `export-project` (#183)Dan Engelbrecht2022-11-181-0/+20
| | | * Add `import-project` and `export-project` command line parsing
* fix concurrency issues in projectstore and enable GC (#181)Dan Engelbrecht2022-10-171-39/+50
| | | | | | | | | | | | | | * Fix concurreny issues when deleting projects/oplogs * remove rocksdb test code * project store unit tests * safer deletion of oplogs/projects * reference count ProjectStore::Project to handle lifetime during GC * Don't open all project oplogs unless we need them * Don't scrub expired projects * Don't gather references from expired projects * added logging details for GC * release lock as soon as folder is moved * more tests for project store * changelog
* De/oplog gc lifetime (#178)Dan Engelbrecht2022-10-101-4/+10
| | | | | | | * Make sure we don't use invalidated iterators in projectstore.cpp * project store keeps track of project file and will garbage collect data for a project if the project file no longer exist * Implement GC of projects in project store - still need to fix lifetime issues for Project instances * Add INFO log if project file path is empty in projectstore * changelog
* Make cas storage an hidden implementation detail of CidStore (#130)v0.1.4-pre6v0.1.4-pre5Dan Engelbrecht2022-06-171-3/+1
| | | | | | | | | | | | | | | | | | - Bumped ZEN_SCHEMA_VERSION - CasStore no longer a public API, it is hidden behind CidStore - Moved cas.h from public header folder - CidStore no longer maps from Cid -> Cas, we store entries in Cas under RawHash - CasStore now decompresses data to validate content (matching against RawHash) - CasChunkSet renames to HashKeySet and put in separate header/cpp file - Disabled "Chunk" command for now as it relied on CAS being exposed as a service - Changed CAS http service to Cid http server - Moved "Run" command completely inside ZEN_WITH_EXEC_SERVICES define - Removed "cas.basic" test - Uncommented ".exec.basic" test and added return-skip at start of test - Moved ScrubContext to separate header file - Renamed CasGC to GcManager - Cleaned up configuration passing in cas store classes - Removed CAS stuff from GcContext and clarified naming in class - Remove migration code
* clang formatDan Engelbrecht2022-06-141-1/+1
|
* removed LocalProjectService prototypeStefan Boberg2022-06-131-25/+10
|
* fixed issue where projects would not be discovered via DiscoverProjects due ↵Stefan Boberg2022-06-101-1/+2
| | | | | | to use of stem() vs filename() added /prj/{project}/oplog/{log}/{op} endpoint to allow retrieval of an op entry by LSN. Supports returning CbObject or CbPackage format payloads
* Added route '/prj/list' for retrieving project info.Per Larsson2022-03-231-0/+1
|
* Merged main.Per Larsson2021-12-141-1/+0
|\
| * Fixed up some assumptions that satd::fs::path uses wchar_tMartin Ridgers2021-11-011-1/+0
| |
* | projectstore: Implemented project/oplog discovery so we can take any ↵Stefan Boberg2021-11-011-0/+2
| | | | | | | | persistent oplog into account even if it has not been opened in the current session
* | projectstore: added validation of oplog data during initializationStefan Boberg2021-11-011-1/+4
| | | | | | | | also added validation during writing
* | gc: Implemented initial root gathering for projects/oplogsStefan Boberg2021-10-281-4/+9
| |
* | general: Fixed up some vcxproj files which had a warning level setStefan Boberg2021-10-241-2/+5
|/ | | | | | Thus they did not get the same settings as all other projects which made it easy to introduce warnings which would trigger in xmake builds Cleaned up some #include usage to use third-party include brackets
* Compressed oplog attachmentsPer Larsson2021-10-211-11/+12
|
* Added more scrub stubs in higher level servicesStefan Boberg2021-09-201-0/+3
|
* Corrected some issues in the HttpProjectService comments to eliminate some ↵Stefan Boberg2021-09-201-4/+4
| | | | potential confusion around namespaces
* Changed logging implementationStefan Boberg2021-09-151-0/+2
| | | | | | * Code should no longer directly `#include spdlog/spdlog.h`, instead use `#include <zencore/logging.h>` * Instead of explicit calls to `spdlog::info(...)` and such please use the logging macros defined in `zencore/logging.h`. I.e `ZEN_INFO`, `ZEN_DEBUG`, `ZEN_TRACE`, `ZEN_ERROR`, `ZEN_CRITITCAL` * The macros will pick up the "most local" logger via a `Log()` call to retrieve a logger instance. To override the default logger in a class please implement your own `Log()` function
* Changed direct includes of spdlog/spdlog.h into zencore/logging.h to make it ↵Stefan Boberg2021-09-151-1/+1
| | | | easier to tweak implementation
* Changed so more loggers go via the zen::loggers interfaceStefan Boberg2021-09-151-2/+2
|
* clang-formatStefan Boberg2021-09-091-1/+1
|
* Factored out http server related code into zenhttp module since it feels out ↵Stefan Boberg2021-09-091-1/+1
| | | | of place in zencore
* Added {project}/oplog/{log}/{hash} endpoint (implemented by Matt Peters)Stefan Boberg2021-08-121-2/+4
| | | | Added project store flush implementation
* Support iterative cooks (#3)Per Larsson2021-06-231-7/+12
| | | | | | | | | | | | | | * Added new route to get all chunk IDs and chunk hashes. Changed to always update chunk mapping to support iterative cooks. * Replay latest oplog entries. * Include server path when fetching file(s) and support for fetching single oplog entry. * Removed get chunks route. * Removed iterate chunk map. * Take read lock when iterating oplog. * Take read lock when reading oplog entry. * Take ownership of buffer reading oplog entry. * Fixed incorrect oplog key when fetching single entry. * Changed map updates to use insert_or_assign for efficiency Co-authored-by: Stefan Boberg <[email protected]>
* Moved lock closer to the members it protects for clarity and maybe cache ↵Stefan Boberg2021-05-271-1/+1
| | | | locality
* Added support for package additional files. Added io hash chunk id to… (#2)Per Larsson2021-05-211-8/+13
| | | | Added support for attaching additional files to oplog entries in project store
* Made SharedBuffer/UniqueBuffer share guts with IoBufferStefan Boberg2021-05-131-1/+1
| | | | This enables way more efficient marshaling of compact binary objects and attachments
* Implemented basic support for marshaling attachments out-of-band with the ↵Stefan Boberg2021-05-111-0/+1
| | | | | | | | package payload Currently supported in project store but will also be used for the structured cache Currently, cleanup is missing. Ultimately the intent is that the file used for marshaling will simply be moved into place.
* Adding zenservice codeStefan Boberg2021-05-111-0/+241
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-23 14:46:40 +0000