diff options
| author | Stefan Boberg <[email protected]> | 2026-04-20 21:50:41 +0200 |
|---|---|---|
| committer | GitHub Enterprise <[email protected]> | 2026-04-20 21:50:41 +0200 |
| commit | 2dfb5da16b97a6c12e01977af5b5188522178a4e (patch) | |
| tree | 428aa0aa8e6079c64438931e0fd4f828c613c94d /thirdparty/raw_pdb/src/PDB_CoalescedMSFStream.cpp | |
| parent | Add CompactString utility type (#990) (diff) | |
| download | archived-zen-2dfb5da16b97a6c12e01977af5b5188522178a4e.tar.xz archived-zen-2dfb5da16b97a6c12e01977af5b5188522178a4e.zip | |
zen trace analysis support (#945)
Integrates the **tourist** trace analysis library and builds a full `zen trace` command suite for working with Unreal Engine `.utrace` files.
### Trace analysis library (`thirdparty/tourist/`)
- Adds the tourist library as a third-party dependency with three modules: **foundation** (platform primitives, memory, scheduling), **trace** (UE Trace protocol decoding), and **analysis** (event dispatching and analyzer framework).
- Cross-platform support for Windows, Linux, and macOS.
### `zen trace` CLI commands (`src/zen/cmds/`, `src/zen/trace/`)
- **`zen trace analyze`** — Summarize a `.utrace` file: session metadata, thread inventory, command line + build configuration, CPU profiling scopes, timing, event rates, log messages, and (with symbols) memory allocation metrics including live-allocs dumps, callstack-keyed aggregation, and allocation churn. Optional HTML output for memory reports.
- **`zen trace inspect`** — Dump the event schema (declared types, fields, sizes) from a trace file.
- **`zen trace trim`** — Extract a time-window from a trace into a new `.utrace` file.
- **`zen trace serve`** — Launch a local HTTP server hosting an interactive trace viewer; opens in the default browser.
### Symbolication (`src/zen/trace/symbol_resolver.*`, `thirdparty/raw_pdb/`)
- Pluggable resolver with multiple backends: `pdb` (in-tree raw_pdb), `dbghelp` (Windows), `llvm-symbolizer` (all platforms), `atos` (macOS). An `auto` backend picks the best available tool per platform.
- Microsoft Symbol Server support: downloads PDBs on demand using a redirect-aware HTTP client.
- Local PDB cache keyed by image GUID preserves symbols across binary recompilation.
- Callstack trimming heuristic strips UE internal noise from reports.
- Binary analysis cache (`.ucache_z`) avoids re-resolving the same trace.
### Interactive trace viewer (`src/zen/frontend/html/`, `src/zen/trace/trace_viewer_service.*`)
- Timeline: scope-level detail, horizontal zoom/pan, vertical scrolling, viewport-driven loading with pre-computed LOD for responsive navigation of large traces.
- Thread grouping (collapsible sidebar sections) synthesized from name suffixes, natural sort order, visual distinction between lane threads and OS threads.
- Bookmark and region annotations; region categories with per-category toggles; bookmark marker toggle in the toolbar.
- Filterable Logs tab showing captured `UE_LOG` output.
- Stats tab with per-scope aggregate statistics.
- Memory tab with interactive allocation analysis and an allocation size histogram.
- CsvProfiler event parsing and chart UI.
### Other in-branch supporting changes
- **Cross-platform browser launcher** (`browser_launcher.{h,cpp}`) used by `trace serve`.
- **`ReciprocalU64`** fast 64-bit integer division (zencore/intmath) for trace analyzers.
- **`parallelsort`** cross-platform parallel sort helper (zenutil).
- Frontend zip build rule so the viewer's HTML assets are bundled into `zen.exe`.
- `/Zo` flag for better optimized debug info on Windows release builds.
- `trace-tests.cpp` in the `zen-test` harness (harness itself landed on main via #985).
Diffstat (limited to 'thirdparty/raw_pdb/src/PDB_CoalescedMSFStream.cpp')
| -rw-r--r-- | thirdparty/raw_pdb/src/PDB_CoalescedMSFStream.cpp | 169 |
1 files changed, 169 insertions, 0 deletions
diff --git a/thirdparty/raw_pdb/src/PDB_CoalescedMSFStream.cpp b/thirdparty/raw_pdb/src/PDB_CoalescedMSFStream.cpp new file mode 100644 index 000000000..fe544e4e9 --- /dev/null +++ b/thirdparty/raw_pdb/src/PDB_CoalescedMSFStream.cpp @@ -0,0 +1,169 @@ +// Copyright 2011-2022, Molecular Matters GmbH <[email protected]> +// See LICENSE.txt for licensing details (2-clause BSD License: https://opensource.org/licenses/BSD-2-Clause) + +#include "PDB_PCH.h" +#include "PDB_CoalescedMSFStream.h" +#include "PDB_Util.h" +#include "PDB_DirectMSFStream.h" +#include "Foundation/PDB_PointerUtil.h" +#include "Foundation/PDB_Memory.h" +#include "Foundation/PDB_CRT.h" + + +namespace +{ + // ------------------------------------------------------------------------------------------------ + // ------------------------------------------------------------------------------------------------ + PDB_NO_DISCARD static bool AreBlockIndicesContiguous(const uint32_t* blockIndices, uint32_t blockSize, uint32_t streamSize) PDB_NO_EXCEPT + { + const uint32_t blockCount = PDB::ConvertSizeToBlockCount(streamSize, blockSize); + + // start with the first index, checking if all following indices are contiguous (N, N+1, N+2, ...) + uint32_t expectedIndex = blockIndices[0]; + for (uint32_t i = 1u; i < blockCount; ++i) + { + ++expectedIndex; + if (blockIndices[i] != expectedIndex) + { + return false; + } + } + + return true; + } +} + + +// ------------------------------------------------------------------------------------------------ +// ------------------------------------------------------------------------------------------------ +PDB::CoalescedMSFStream::CoalescedMSFStream(void) PDB_NO_EXCEPT + : m_ownedData(nullptr) + , m_data(nullptr) + , m_size(0u) +{ +} + + +// ------------------------------------------------------------------------------------------------ +// ------------------------------------------------------------------------------------------------ +PDB::CoalescedMSFStream::CoalescedMSFStream(CoalescedMSFStream&& other) PDB_NO_EXCEPT + : m_ownedData(PDB_MOVE(other.m_ownedData)) + , m_data(PDB_MOVE(other.m_data)) + , m_size(PDB_MOVE(other.m_size)) +{ + other.m_ownedData = nullptr; + other.m_data = nullptr; + other.m_size = 0u; +} + + +// ------------------------------------------------------------------------------------------------ +// ------------------------------------------------------------------------------------------------ +PDB::CoalescedMSFStream& PDB::CoalescedMSFStream::operator=(CoalescedMSFStream&& other) PDB_NO_EXCEPT +{ + if (this != &other) + { + PDB_DELETE_ARRAY(m_ownedData); + + m_ownedData = PDB_MOVE(other.m_ownedData); + m_data = PDB_MOVE(other.m_data); + m_size = PDB_MOVE(other.m_size); + + other.m_ownedData = nullptr; + other.m_data = nullptr; + other.m_size = 0u; + } + + return *this; +} + + +// ------------------------------------------------------------------------------------------------ +// ------------------------------------------------------------------------------------------------ +PDB::CoalescedMSFStream::CoalescedMSFStream(const void* data, uint32_t blockSize, const uint32_t* blockIndices, uint32_t streamSize) PDB_NO_EXCEPT + : m_ownedData(nullptr) + , m_data(nullptr) + , m_size(streamSize) +{ + if (AreBlockIndicesContiguous(blockIndices, blockSize, streamSize)) + { + // fast path, all block indices are contiguous, so we don't have to copy any data at all. + // instead, we directly point into the memory-mapped file at the correct offset. + const uint32_t index = blockIndices[0]; + const size_t fileOffset = PDB::ConvertBlockIndexToFileOffset(index, blockSize); + m_data = Pointer::Offset<const Byte*>(data, fileOffset); + } + else + { + // slower path, we need to copy disjunct blocks into our own data array, block by block + m_ownedData = PDB_NEW_ARRAY(Byte, streamSize); + m_data = m_ownedData; + + Byte* destination = m_ownedData; + + // copy full blocks first + const uint32_t fullBlockCount = streamSize / blockSize; + for (uint32_t i = 0u; i < fullBlockCount; ++i) + { + const uint32_t index = blockIndices[i]; + + // read one single block at the correct offset in the stream + const size_t fileOffset = PDB::ConvertBlockIndexToFileOffset(index, blockSize); + const void* sourceData = Pointer::Offset<const void*>(data, fileOffset); + memcpy(destination, sourceData, blockSize); + + destination += blockSize; + } + + // account for non-full blocks + const uint32_t remainingBytes = streamSize - (fullBlockCount * blockSize); + if (remainingBytes != 0u) + { + const uint32_t index = blockIndices[fullBlockCount]; + + // read remaining bytes at correct offset in the stream + const size_t fileOffset = PDB::ConvertBlockIndexToFileOffset(index, blockSize); + const void* sourceData = Pointer::Offset<const void*>(data, fileOffset); + memcpy(destination, sourceData, remainingBytes); + } + } +} + + +// ------------------------------------------------------------------------------------------------ +// ------------------------------------------------------------------------------------------------ +PDB::CoalescedMSFStream::CoalescedMSFStream(const DirectMSFStream& directStream, uint32_t size, uint32_t offset) PDB_NO_EXCEPT + : m_ownedData(nullptr) + , m_data(nullptr) + , m_size(size) +{ + const DirectMSFStream::IndexAndOffset indexAndOffset = directStream.GetBlockIndexForOffset(offset); + + // Note: we need to add the offset within the block to the size of the stream to determine if the block + // indices are contiguous. This is needed to deal with the case where reading the requested number of bytes + // from the specified offset would cross a block boundary. For example, if the offset within the block is + // 64 and we want to read 4096 bytes with a block size of 4096, we need to consider *two* block indices, + // not *one*, even though 4096 / 4096 = 1. + if (AreBlockIndicesContiguous(directStream.GetBlockIndices() + indexAndOffset.index, directStream.GetBlockSize(), indexAndOffset.offsetWithinBlock + size)) + { + // fast path, all block indices inside the direct stream from (data + offset) to (data + offset + size) are contiguous + const size_t offsetWithinData = directStream.GetDataOffsetForIndexAndOffset(indexAndOffset); + m_data = Pointer::Offset<const Byte*>(directStream.GetData(), offsetWithinData); + } + else + { + // slower path, we need to copy from disjunct blocks, which is performed by the direct stream + m_ownedData = PDB_NEW_ARRAY(Byte, size); + m_data = m_ownedData; + + directStream.ReadAtOffset(m_ownedData, size, offset); + } +} + + +// ------------------------------------------------------------------------------------------------ +// ------------------------------------------------------------------------------------------------ +PDB::CoalescedMSFStream::~CoalescedMSFStream(void) PDB_NO_EXCEPT +{ + PDB_DELETE_ARRAY(m_ownedData); +} |