mem cache perf improvements (#592)

- Improvement: Refactor memory cache for faster trimming and correct trim reporting
- Improvement: Added trace scopes for memory cache trimming

Adding a link back to the cache item payload on the memory cache item allows us to iterate over only the items cached in memory instead of over the entire index.
This also allows us to do efficient compact of the memory cache array when trimming.
It adds 4 bytes of overhead to each item cached in memory.

0 files changed, 0 insertions, 0 deletions