Orbit C++ → Rust · Post 05

LLVM Is Gone

The thing the whole port was for. Post 04 said PDB was blocked; it wasn't, and the correction is the most useful part of this one.

0LLVM dependents
−34%Cold build
−2,191ObjectUtils lines
16.3MSymbols verified
71Rust tests
0C++ tests edited

Post 04 ended by saying stage 2g was blocked on reimplementing LLVM's CodeView type-name renderer, and that this made the whole goal unreachable. That was wrong. The renderer is about two hundred lines, and once it existed, line info fell too, and then there was nothing keeping LLVM in the tree.

The assessment that was wrong

I had measured the right things. The pdb crate read Orbit's test file exactly — GUID, age, 5,459 procedures, every address and size. Only the names were missing their argument lists, because PdbFileLlvm appends LazyRandomTypeCollection::getTypeName of the function's argument list, and that is LLVM's CodeView type-name renderer.

What I got wrong was the size of that. I wrote "a bounded, nameable project, but it is a project". It is TypeNameComputer plus TypeIndex::simpleTypeName: a switch over a dozen record kinds and a forty-entry table of primitive names.

And PdbFileTest turns out to be a specification of it. Whoever wrote it put one construct per symbol, deliberately:

PrintString(const char*)
TakesVolatileInt(volatile int)
TakesFooReference(Foo&)
TakesFooRValueReference(Foo&&)
TakesConstPtrToInt(int* const)
TakesReferenceToIntPtr(int*&)
TakesVolatilePointerToConstUnsignedChar(const unsigned char* volatile)
TakesVoidFunctionPointer(void (int)*)        // the test notes LLVM gets these wrong
TakesMemberFunctionPointer(const char* Foo::(int) Foo::*, Foo)

All of them pass. Matching LLVM means reproducing the two the test itself flags as wrong, which the renderer does.

Worth being precise about

I did not discover new information and change my mind. The measurement in post 04 was correct and complete; I drew a conclusion from it that was too pessimistic, and only found out because someone said "try again". An estimate stated as a blocker becomes one.

Four more rounds of the corpus

The renderer got the unit tests right immediately. The corpus, as usual, did not agree.

16 · pdb

An array renders as its name, which is empty

TypeNameComputer renders an array by the record's name field, which MSVC leaves blank — so a reference to char[261] is &, not char&. Matching LLVM meant rendering nothing.

17 · pdb

nullptr_t, wrong in both directions

TypeIndex::NullptrT() is Void with SimpleTypeMode::NearPointer — 0x0103. Plain void* on x64 is NearPointer64, 0x0603. I guessed 0x0603 first, which rendered nullptr_t as void*; corrected it too far and rendered every void pointer as std::nullptr_t. Two consecutive corpus runs, one wrong answer each, each naming the exact symbol.

The special indices also have to be checked before the type stream, or a nullptr_t that also exists as a record renders through the table anyway.

18 · pdb

PDB names are MSVC-mangled

Obvious in hindsight. llvm::demangle dispatches ?-prefixed names to microsoftDemangle, and abi::__cxa_demangle knows nothing about them. msvc-demangler is a port of LLVM's own MicrosoftDemangle.cpp, which is why it matches — on all but one symbol in 12,420, a vtordisp thunk whose $4… encoding it rejects.

And then line info, which post 03 had given up on

With the same technique — reproduce LLVM's rules rather than trust a library's — the thing that stopped stage 2d came apart. The fix was to stop using addr2line's pre-joined path and walk the line table with gimli directly, applying getFileNameByIndex's rules.

Two findings, and they are a matched pair worth putting side by side:

one address, inside an inlined call · two adjacent methods · opposite answers DIE tree compile_unit subprogram main decl_line 67 inlined_subroutine call_line 13 abstract_origin → decl_line 410 ← the address is here line-table row says line 10 GetLineInfo outermost frame → the call site line 13 the row says 10; symbolizeInlinedCode takes the *last* frame, so the caller wins GetDeclarationLocationOfFunction innermost DIE → the callee's declaration line 410 getSubroutineForAddress indexes inlined subroutines too, and the deepest wins I got each of these backwards once. Both wrong answers are plausible line numbers from a real function — which is why only a value comparison finds them.
The same tree, read in opposite directions, one method apart. Neither mistake produces anything that looks wrong: you get a real line number from a real function, just not the one LLVM would have given. Nothing short of running both implementations over the same binaries distinguishes them.

There was a third, smaller one in between: gimli parses DW_AT_call_file as AttributeValue::FileIndex, which udata_value() does not unwrap. Every inlined call was silently falling through to the callee's line because one attribute read returned None.

Stage 2h

With every method on Rust and the corpus clean, the deletion was mechanical:

MeasureBaselineNow
//bazel/deps:llvm dependents10deleted
Files including an llvm/ header110−11
bazel/patches/31both LLVM patches
bazel_dep count1413−1
ObjectUtils implementation lines3,6801,489−2,191
deps(//src/Service:OrbitService)22,22021,237−983
Cold build of ObjectUtils158.7s104.6s−34%
  critical path68.3s48.3s−29%
  actions3,3242,531−793
deps(//src/ObjectUtils)5,8437,173+1,330
  external repositories1799+82

Those last two rows are the honest cost. ObjectUtils gained an entire Rust toolchain it did not have before, and at the module level that is a bigger graph than LLVM was. At the binary level the trade is already positive, because the WebAssembly viewer had brought the toolchain in regardless — OrbitService's dependency graph is smaller by 983 targets.

The build-time number is the one that will be felt. A cold ObjectUtils build was two and a half minutes and is now under two, and the critical path — what a developer actually waits for — dropped by a third.

What it cost

Three rendering differences are now production behaviour. They were measured before the switch was flipped, not after:

DifferenceRateWhat it looks like
Demangled names3,426 / 17,152,735  0.020%llvm writes 'lambda'(), libstdc++ writes {lambda()#1}
Source paths2 / 1,235  0.162%sigaction.c versus ../sysdeps/unix/sysv/linux/x86_64/sigaction.c
PDB names1 / 12,420  0.008%one vtordisp thunk that stays mangled

Everything structural — addresses, sizes, hotpatchability, symbol counts, symbol ordering, and every line number — is identical across 16.3 million symbols from every binary on the machine.

The other cost is the one that is easy to miss. Deleting the C++ deletes the comparison. ORBIT_OBJECT_BACKEND=both is what found essentially every bug in this port — eighteen of them across five posts, most unreachable from the curated test data — and it only worked because there were two implementations. The differential tool is now a smoke test: it reads 4,709 files and reports what it read, and the counts must not move without explanation. To compare against LLVM again you check out c7c4e6566.

The whole thing, in numbers

Goal, as the plan stated itResult
Correctness — the existing suite in three modes, a differential corpus, fuzz parity, no test file edited 16.3M symbols, 12,420 PDB symbols and 1,235 line lookups compared against LLVM with zero structural disagreements. Not one C++ test edited, across 21 commits.
Performance — measured, no regression permitted ParseMaps 1.75×. ELF metadata parse 6.5× to 50×, growing with file size.
Ease of build — LLVM's compile off the critical path, patches gone Cold build −34%, critical path −29%, both LLVM patches deleted, the version pin that existed only because the registry had nothing newer is gone.
Shrinking dependencies — //bazel/deps:llvm to zero dependents Zero. OrbitService's graph −983 targets. ObjectUtils's own graph is larger, and that is stated in every post.

Five external Rust crates do the work that six LLVM libraries did: object, gimli, pdb, msvc-demangler and flate2. Two things deliberately stayed C++ because porting them would have removed nothing — abi::__cxa_demangle and Orbit's own protobuf post-processing.

What is left

ObjectUtils and ModuleUtils are done. Phase 3 — the seven pure-logic classes in LinuxTracing, 1,355 implementation lines against 3,053 lines of test — has not started, and it is independent of everything here.

The one thing worth carrying forward is not a technique, it is a number: eighteen divergences, of which the repository's own test suite would have caught none. Every one came from running two implementations over binaries that happened to be on the machine. That is ninety seconds of compute per run, and it is the only reason any of the figures above are worth printing.