diff options
| author | Stefan Boberg <[email protected]> | 2026-04-20 21:50:41 +0200 |
|---|---|---|
| committer | GitHub Enterprise <[email protected]> | 2026-04-20 21:50:41 +0200 |
| commit | 2dfb5da16b97a6c12e01977af5b5188522178a4e (patch) | |
| tree | 428aa0aa8e6079c64438931e0fd4f828c613c94d /thirdparty/raw_pdb/src/Examples/ExampleSymbols.cpp | |
| parent | Add CompactString utility type (#990) (diff) | |
| download | archived-zen-2dfb5da16b97a6c12e01977af5b5188522178a4e.tar.xz archived-zen-2dfb5da16b97a6c12e01977af5b5188522178a4e.zip | |
zen trace analysis support (#945)
Integrates the **tourist** trace analysis library and builds a full `zen trace` command suite for working with Unreal Engine `.utrace` files.
### Trace analysis library (`thirdparty/tourist/`)
- Adds the tourist library as a third-party dependency with three modules: **foundation** (platform primitives, memory, scheduling), **trace** (UE Trace protocol decoding), and **analysis** (event dispatching and analyzer framework).
- Cross-platform support for Windows, Linux, and macOS.
### `zen trace` CLI commands (`src/zen/cmds/`, `src/zen/trace/`)
- **`zen trace analyze`** — Summarize a `.utrace` file: session metadata, thread inventory, command line + build configuration, CPU profiling scopes, timing, event rates, log messages, and (with symbols) memory allocation metrics including live-allocs dumps, callstack-keyed aggregation, and allocation churn. Optional HTML output for memory reports.
- **`zen trace inspect`** — Dump the event schema (declared types, fields, sizes) from a trace file.
- **`zen trace trim`** — Extract a time-window from a trace into a new `.utrace` file.
- **`zen trace serve`** — Launch a local HTTP server hosting an interactive trace viewer; opens in the default browser.
### Symbolication (`src/zen/trace/symbol_resolver.*`, `thirdparty/raw_pdb/`)
- Pluggable resolver with multiple backends: `pdb` (in-tree raw_pdb), `dbghelp` (Windows), `llvm-symbolizer` (all platforms), `atos` (macOS). An `auto` backend picks the best available tool per platform.
- Microsoft Symbol Server support: downloads PDBs on demand using a redirect-aware HTTP client.
- Local PDB cache keyed by image GUID preserves symbols across binary recompilation.
- Callstack trimming heuristic strips UE internal noise from reports.
- Binary analysis cache (`.ucache_z`) avoids re-resolving the same trace.
### Interactive trace viewer (`src/zen/frontend/html/`, `src/zen/trace/trace_viewer_service.*`)
- Timeline: scope-level detail, horizontal zoom/pan, vertical scrolling, viewport-driven loading with pre-computed LOD for responsive navigation of large traces.
- Thread grouping (collapsible sidebar sections) synthesized from name suffixes, natural sort order, visual distinction between lane threads and OS threads.
- Bookmark and region annotations; region categories with per-category toggles; bookmark marker toggle in the toolbar.
- Filterable Logs tab showing captured `UE_LOG` output.
- Stats tab with per-scope aggregate statistics.
- Memory tab with interactive allocation analysis and an allocation size histogram.
- CsvProfiler event parsing and chart UI.
### Other in-branch supporting changes
- **Cross-platform browser launcher** (`browser_launcher.{h,cpp}`) used by `trace serve`.
- **`ReciprocalU64`** fast 64-bit integer division (zencore/intmath) for trace analyzers.
- **`parallelsort`** cross-platform parallel sort helper (zenutil).
- Frontend zip build rule so the viewer's HTML assets are bundled into `zen.exe`.
- `/Zo` flag for better optimized debug info on Windows release builds.
- `trace-tests.cpp` in the `zen-test` harness (harness itself landed on main via #985).
Diffstat (limited to 'thirdparty/raw_pdb/src/Examples/ExampleSymbols.cpp')
| -rw-r--r-- | thirdparty/raw_pdb/src/Examples/ExampleSymbols.cpp | 238 |
1 files changed, 238 insertions, 0 deletions
diff --git a/thirdparty/raw_pdb/src/Examples/ExampleSymbols.cpp b/thirdparty/raw_pdb/src/Examples/ExampleSymbols.cpp new file mode 100644 index 000000000..c1b2ef8ac --- /dev/null +++ b/thirdparty/raw_pdb/src/Examples/ExampleSymbols.cpp @@ -0,0 +1,238 @@ +// Copyright 2011-2022, Molecular Matters GmbH <[email protected]> +// See LICENSE.txt for licensing details (2-clause BSD License: https://opensource.org/licenses/BSD-2-Clause) + +#include "Examples_PCH.h" +#include "ExampleTimedScope.h" +#include "PDB_RawFile.h" +#include "PDB_DBIStream.h" + + +namespace +{ + // we don't have to store std::string in the symbols, since all the data is memory-mapped anyway. + // we do it in this example to ensure that we don't "cheat" when reading the PDB file. memory-mapped data will only + // be faulted into the process once it's touched, so actually copying the string data makes us touch the needed data, + // giving us a real performance measurement. + struct Symbol + { + std::string name; + uint32_t rva; + }; +} + + +void ExampleSymbols(const PDB::RawFile& rawPdbFile, const PDB::DBIStream& dbiStream); +void ExampleSymbols(const PDB::RawFile& rawPdbFile, const PDB::DBIStream& dbiStream) +{ + TimedScope total("\nRunning example \"Symbols\""); + + // in order to keep the example easy to understand, we load the PDB data serially. + // note that this can be improved a lot by reading streams concurrently. + + // prepare the image section stream first. it is needed for converting section + offset into an RVA + TimedScope sectionScope("Reading image section stream"); + const PDB::ImageSectionStream imageSectionStream = dbiStream.CreateImageSectionStream(rawPdbFile); + sectionScope.Done(); + + + // prepare the module info stream for matching contributions against files + TimedScope moduleScope("Reading module info stream"); + const PDB::ModuleInfoStream moduleInfoStream = dbiStream.CreateModuleInfoStream(rawPdbFile); + moduleScope.Done(); + + + // prepare symbol record stream needed by both public and global streams + TimedScope symbolStreamScope("Reading symbol record stream"); + const PDB::CoalescedMSFStream symbolRecordStream = dbiStream.CreateSymbolRecordStream(rawPdbFile); + symbolStreamScope.Done(); + + std::vector<Symbol> symbols; + + // read public symbols + TimedScope publicScope("Reading public symbol stream"); + const PDB::PublicSymbolStream publicSymbolStream = dbiStream.CreatePublicSymbolStream(rawPdbFile); + publicScope.Done(); + { + TimedScope scope("Storing public symbols"); + + const PDB::ArrayView<PDB::HashRecord> hashRecords = publicSymbolStream.GetRecords(); + const size_t count = hashRecords.GetLength(); + + symbols.reserve(count); + + for (const PDB::HashRecord& hashRecord : hashRecords) + { + const PDB::CodeView::DBI::Record* record = publicSymbolStream.GetRecord(symbolRecordStream, hashRecord); + if (record->header.kind != PDB::CodeView::DBI::SymbolRecordKind::S_PUB32) + { + // normally, a PDB only contains S_PUB32 symbols in the public symbol stream, but we have seen PDBs that also store S_CONSTANT as public symbols. + // ignore these. + continue; + } + + const uint32_t rva = imageSectionStream.ConvertSectionOffsetToRVA(record->data.S_PUB32.section, record->data.S_PUB32.offset); + if (rva == 0u) + { + // certain symbols (e.g. control-flow guard symbols) don't have a valid RVA, ignore those + continue; + } + + symbols.push_back(Symbol { record->data.S_PUB32.name, rva }); + } + + scope.Done(count); + } + + + // read global symbols + TimedScope globalScope("Reading global symbol stream"); + const PDB::GlobalSymbolStream globalSymbolStream = dbiStream.CreateGlobalSymbolStream(rawPdbFile); + globalScope.Done(); + { + TimedScope scope("Storing global symbols"); + + const PDB::ArrayView<PDB::HashRecord> hashRecords = globalSymbolStream.GetRecords(); + const size_t count = hashRecords.GetLength(); + + symbols.reserve(symbols.size() + count); + + for (const PDB::HashRecord& hashRecord : hashRecords) + { + const PDB::CodeView::DBI::Record* record = globalSymbolStream.GetRecord(symbolRecordStream, hashRecord); + + const char* name = nullptr; + uint32_t rva = 0u; + if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_GDATA32) + { + name = record->data.S_GDATA32.name; + rva = imageSectionStream.ConvertSectionOffsetToRVA(record->data.S_GDATA32.section, record->data.S_GDATA32.offset); + } + else if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_GTHREAD32) + { + name = record->data.S_GTHREAD32.name; + rva = imageSectionStream.ConvertSectionOffsetToRVA(record->data.S_GTHREAD32.section, record->data.S_GTHREAD32.offset); + } + else if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_LDATA32) + { + name = record->data.S_LDATA32.name; + rva = imageSectionStream.ConvertSectionOffsetToRVA(record->data.S_LDATA32.section, record->data.S_LDATA32.offset); + } + else if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_LTHREAD32) + { + name = record->data.S_LTHREAD32.name; + rva = imageSectionStream.ConvertSectionOffsetToRVA(record->data.S_LTHREAD32.section, record->data.S_LTHREAD32.offset); + } + else if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_UDT) + { + name = record->data.S_UDT.name; + } + else if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_UDT_ST) + { + name = record->data.S_UDT_ST.name; + } + + if (rva == 0u) + { + // certain symbols (e.g. control-flow guard symbols) don't have a valid RVA, ignore those + continue; + } + + symbols.push_back(Symbol { name, rva }); + } + + scope.Done(count); + } + + + // read module symbols + { + TimedScope scope("Storing symbols from modules"); + + const PDB::ArrayView<PDB::ModuleInfoStream::Module> modules = moduleInfoStream.GetModules(); + + for (const PDB::ModuleInfoStream::Module& module : modules) + { + if (!module.HasSymbolStream()) + { + continue; + } + + const PDB::ModuleSymbolStream moduleSymbolStream = module.CreateSymbolStream(rawPdbFile); + moduleSymbolStream.ForEachSymbol([&symbols, &imageSectionStream](const PDB::CodeView::DBI::Record* record) + { + const char* name = nullptr; + uint32_t rva = 0u; + if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_THUNK32) + { + if (record->data.S_THUNK32.thunk == PDB::CodeView::DBI::ThunkOrdinal::TrampolineIncremental) + { + // we have never seen incremental linking thunks stored inside a S_THUNK32 symbol, but better be safe than sorry + name = "ILT"; + rva = imageSectionStream.ConvertSectionOffsetToRVA(record->data.S_THUNK32.section, record->data.S_THUNK32.offset); + } + } + else if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_TRAMPOLINE) + { + // incremental linking thunks are stored in the linker module + name = "ILT"; + rva = imageSectionStream.ConvertSectionOffsetToRVA(record->data.S_TRAMPOLINE.thunkSection, record->data.S_TRAMPOLINE.thunkOffset); + } + else if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_BLOCK32) + { + // blocks never store a name and are only stored for indicating whether other symbols are children of this block + } + else if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_LABEL32) + { + // labels don't have a name + } + else if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_LPROC32) + { + name = record->data.S_LPROC32.name; + rva = imageSectionStream.ConvertSectionOffsetToRVA(record->data.S_LPROC32.section, record->data.S_LPROC32.offset); + } + else if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_GPROC32) + { + name = record->data.S_GPROC32.name; + rva = imageSectionStream.ConvertSectionOffsetToRVA(record->data.S_GPROC32.section, record->data.S_GPROC32.offset); + } + else if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_LPROC32_ID) + { + name = record->data.S_LPROC32_ID.name; + rva = imageSectionStream.ConvertSectionOffsetToRVA(record->data.S_LPROC32_ID.section, record->data.S_LPROC32_ID.offset); + } + else if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_GPROC32_ID) + { + name = record->data.S_GPROC32_ID.name; + rva = imageSectionStream.ConvertSectionOffsetToRVA(record->data.S_GPROC32_ID.section, record->data.S_GPROC32_ID.offset); + } + else if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_REGREL32) + { + name = record->data.S_REGREL32.name; + // You can only get the address while running the program by checking the register value and adding the offset + } + else if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_LDATA32) + { + name = record->data.S_LDATA32.name; + rva = imageSectionStream.ConvertSectionOffsetToRVA(record->data.S_LDATA32.section, record->data.S_LDATA32.offset); + } + else if (record->header.kind == PDB::CodeView::DBI::SymbolRecordKind::S_LTHREAD32) + { + name = record->data.S_LTHREAD32.name; + rva = imageSectionStream.ConvertSectionOffsetToRVA(record->data.S_LTHREAD32.section, record->data.S_LTHREAD32.offset); + } + + if (rva == 0u) + { + // certain symbols (e.g. control-flow guard symbols) don't have a valid RVA, ignore those + return; + } + + symbols.push_back(Symbol { name, rva }); + }); + } + + scope.Done(modules.GetLength()); + } + + total.Done(symbols.size()); +} |