Orbit on Bazel

Orbit builds with Bazel on Linux and Windows. git clone, git submodule update --init third_party/py-spy, bazel build //... — that is the whole setup. No Conan, no CMake, no apt install, no Rust toolchain. CMake and Conan are gone from the tree entirely. This is the reasoning behind each decision, including the two that were wrong the first time.

What was there before

81 module directories under src/, about 1,220 C++ source and header files, one Rust crate, and a Qt 5 desktop UI. CMake drove the build; Conan supplied Abseil, protobuf, gRPC, GoogleTest, LLVM, capstone, OpenSSL, libssh2 and Outcome; Qt came from the system via apt install qtbase5-dev; and eleven more libraries were vendored directly into third_party/, each of which now has its own BUILD file.

The cmake/Find*.cmake directory is the interesting part of the old setup — twenty-four hand-written find modules, several with fallback chains (try the Conan package, then pkg-config, then build it from third_party/). That is the machinery the port had to replace, and it is also a good map of where the sharp edges were.

The one principle

You asked for Bazel modules instead of Conan, and for the first build to be fast. Those two pull in opposite directions: a Bazel module means source, and source means compiling. So the rule for every dependency was:

That last clause is what keeps the build honest. "Prebuilt" here does not mean "whatever apt put in /usr"; it means a specific file, at a specific URL, with a specific hash, recorded in the repository. A different machine gets the same bytes.

Where each dependency comes from

Four paths, in descending order of preference.

The one thing the host still provides is the low-level shared libraries Qt itself loads — ICU, glib, fontconfig, X11. Those are present on any desktop install and are the same libraries the CMake build linked against.

Qt: prebuilt, from pinned .debs

Qt is where the "affordable compile" test fails hardest. Building Qt 5 from source would have dwarfed everything else in a first build, and there is no Qt 5 Bazel module — rules_qt in the registry is Qt 6 only. The old build sidestepped this by requiring apt install qtbase5-dev, which means the build depends on machine state and on having root.

Bazel can extract the outer ar container of a .deb natively. That turns out to be the whole trick: a repository rule downloads the package, lets Bazel unpack the container, picks data.tar.zst back out, and unpacks that over a shared tree. No host tools, no dpkg, no root.

Three small libraries had to be added to that set — libdouble-conversion3, libpcre2-16-0, libmd4c0 — because libQt5Core and libQt5Gui need them and Ubuntu 26.04 does not ship them in a default install. To bump Qt, edit the package list in bazel/deps/resolve_debs.py and re-run it; it rewrites debs.bzl from apt metadata.

The plugin problem

libQt5Core has the Ubuntu package's install prefix compiled into it and looks for platform plugins underneath it. Since Qt was never installed, that path does not exist and QApplication refuses to start. Qt reads a qt.conf next to the executable before falling back to the built-in prefix, so qt_application() writes one and mirrors the plugin tree into the same output directory. bazel-bin/src/Orbit/Orbit now starts standalone.

LLVM: built, and patched twice

LLVM was the decision most likely to blow the build-time budget, so it got measured first. Orbit links six libraries — Object, Symbolize, DebugInfoDWARF, DebugInfoPDB, DebugInfoCodeView, Demangle — and Bazel builds only what is reachable. That subset compiles in about 40 seconds, not the hours a full LLVM build takes, so the registry module won and no prebuilt tarball was needed.

It needed two patches, both in bazel/patches/.

1 · The missing include

SmallVector.h names uint64_t without including <cstdint>. Older libstdc++ headers dragged the declaration in transitively; GCC 15's do not, so every translation unit that reaches SmallVector.h — LLVM's own and Orbit's alike — failed to compile. Upstream fixed this after 17.0.4; the patch backports the one-line include.

2 · The missing zlib

LLVM's Bazel overlay has the zlib dependency commented out, which makes every attempt to read a SHF_COMPRESSED debug section fail with "LLVM was not built with LLVM_ENABLE_ZLIB". Orbit reads compressed debug info from real binaries, and ObjectUtilsTests caught it immediately. Restoring it means adding LLVM_ENABLE_ZLIB=1 and a dependency edge — but the edge needs zlib to be visible inside llvm-project, which means adding a bazel_dep to its MODULE.bazel.

The SSL problem

This one took three attempts and is the most interesting thing in the port.

gRPC links BoringSSL by default. libssh2 links OpenSSL. Both export the OpenSSL C API, and the Orbit UI needs SSH and gRPC — so it failed to link with hundreds of duplicate definitions: CONF_modules_load_file, X509V3_EXT_nconf, i2v_GENERAL_NAMES, on and on.

The obvious fix is to make gRPC use OpenSSL too; it has a flag for exactly that. The UI then linked. But UserSpaceInstrumentationTests started failing in a way that took a while to pin down: the library Orbit injects into a target process would load, initialise, and produce no threads at all. Reproduced outside Bazel with a twelve-line dlmopen harness, it was blunter than that — corrupted double-linked list, then SIGABRT. OpenSSL's initialisation does not survive being loaded into the fresh linker namespace dlmopen(LM_ID_NEWLM) creates. The same harness against a BoringSSL build printed init returned and 21 threads.

So: BoringSSL breaks the UI, OpenSSL breaks injection. The third option is to notice that Orbit never uses TLS at all. Every channel and every server in the tree is built with InsecureChannelCredentials or InsecureServerCredentials — ten call sites, no exceptions — because the transport is either a Unix socket or a tunnel SSH has already secured. That is also what the pkg-config fallback in cmake/FindgRPC.cmake selected: grpc++_unsecure.

gRPC builds grpc++_unsecure but keeps it behind an empty package group, so the module carries a one-line patch changing that target's visibility to public. That is the third patch in bazel/patches/.

To be explicit: this drops no security

The target's name invites the opposite reading, so it is worth stating plainly. Orbit has never used gRPC's TLS. All ten channel and server call sites use InsecureChannelCredentials or InsecureServerCredentials; grpc::SslCredentials has never appeared anywhere in this repository's history; there is no certificate, key or PEM handling in the tree. Linking grpc++ compiled a TLS stack that no code path could reach, and removing it changes no runtime behaviour.

What protects each link, and always did:

TLS on a loopback socket would add nothing to the first row: anything that can open that port can also complete a handshake with it.

The one place TLS is real

Debug symbols are fetched over HTTPS from the Microsoft symbol server, through QNetworkAccessManager. Qt resolves its TLS backend by dlopening libssl at runtime, which means a Qt without a usable one fails every https:// request at runtime and nowhere else — no link error, no warning. Since this build supplies Qt from pinned packages rather than from the machine, that property is now asserted rather than assumed:

[ RUN      ] Tls.QtCanDoHttps
[       OK ] Tls.QtCanDoHttps

The Qt shipped here is built against OpenSSL 3.5.3 and resolves 3.5.5 at runtime.

Every version that moved

Most of these jumps were not chosen so much as forced. Bazel 9 removed the native cc_library from BUILD files, and older registry modules — and the rule sets they pull in transitively — have not caught up. Pinning gRPC near its old 1.67 was tried and abandoned: 1.72 loads, but its BUILD files want @protobuf//bazel:upb_proto_library.bzl, which protobuf 30 deleted, and minimal version selection pushes protobuf to 35 regardless because envoy_api and rules_go ask for it.

Two things about that table are worth calling out. OpenSSL is no longer a direct dependency — it arrives only under libssh2, and the UI is the only binary that gets it. And every one of these is the newest release available, the rule sets included (rules_cc 0.2.22, rules_python 2.3.1, rules_shell 0.8.0, rules_rust 0.73.0, bazel_skylib 1.9.2, platforms 1.1.0). Two are pinned by availability rather than by choice: LLVM, because 17.0.4 is the only version llvm-project publishes to the registry, and capstone, whose 6.0 line has only alpha releases.

Writing 101 BUILD files

The CMakeLists.txt files were strikingly regular — a library, its sources, its dependencies, a test binary, register_test — so they were translated mechanically rather than by hand. bazel/tools/cmake2bazel.py parsed the CMake command stream, evaluated the conditionals per platform, and emitted BUILD.bazel. It was checked in while CMake was still there so the port could be re-derived and diffed against it; once CMake went, so did the translator. Its last version is at commit aeda7f74b.

It handles a few things that are easy to get wrong by hand:

• Headers CMake never had to list. CMake compiles fine without naming private headers; Bazel needs every included header declared. The translator adds any header in the module that the CMakeLists did not mention.
• if(WIN32) branches. Sources inside them are still collected, purely so their headers can be kept out of the Linux target rather than swept up by the rule above.
• PUBLIC vs PRIVATE include directories. Public ones become includes and propagate; private ones become -I flags in copts and do not.
• Qt macros. Headers are scanned for Q_OBJECT, Q_GADGET and Q_NAMESPACE to decide what moc has to run on, and .ui files are globbed the way AUTOUIC finds them.

Everything it could not model is either hand-written (the two proto packages, the version file, the gtest entry points, the API header's C compile test) or listed in small tables at the top of the script — extra runtime data, the tests that need a binary copied next to them, the one test marked flaky.

Qt code generation

CMake's AUTOMOC and AUTOUIC scan sources at build time and generate whatever they find. Bazel needs the generated files named before the build starts, so qt_cc_library takes them as attributes.

One detail worth recording, because it is invisible until it breaks: moc writes an #include of the header it processed into the generated file, and the path it writes has to resolve from the generated file's own directory. The rule passes -f with the header's path relative to whichever include directory exposes it — falling back to the workspace-relative path for private headers, which resolves through the -iquote entry Bazel adds anyway.

Protos

Orbit's .proto files import each other by bare filename (import "capture.proto";) while the generated C++ is included with a module prefix ("GrpcProtos/capture.pb.h"). proto_library cannot express both — strip_import_prefix satisfies one or the other. Rather than rewrite thirteen proto files, bazel/proto/rules.bzl drives protoc directly, the same way cmake/grpc_helper.cmake did. Both proto packages are self-contained, so one generation step per package is exactly the right granularity.

Two include-path quirks

Both come from the same place: Conan exposed every package's headers through one include directory, so Orbit spells third-party includes with angle brackets — #include <absl/strings/str_cat.h>. Bazel publishes a module's headers through -iquote, which only satisfies quoted includes. Rewriting 1,159 absl includes across the tree was one option; the other was to add the repository root as a system include directory. Label(…).workspace_root resolves to the canonical repository path at load time, so bazel/copts.bzl computes the flag rather than hard-coding external/abseil-cpp+.

The second quirk is warnings. Orbit compiles with -Werror=old-style-cast, among others, and third-party headers do not survive that. Bazel's external_include_paths feature switches external repositories to -isystem, which handles the registry modules — but not the code vendored inside this repository, which is not "external" as far as Bazel is concerned. The directories cmake/Find*.cmake marked SYSTEM are therefore listed explicitly in copts.bzl; without it, gte's headers alone produce about 4,000 errors.

Source changes, and why

Fourteen files, +132/−62, each change in its own commit. None of it is Bazel-specific — it is what the newer dependency versions require, plus two genuine bugs that the newer versions exposed.

Forced by protobuf 36

Arena::CreateMessage was removed in protobuf 31. Arena::Create has been the spelling for messages far longer and works with both, so the change is backwards-compatible with the Conan build.

Forced by LLVM 17

Elf_Note_Impl::getDesc gained a required alignment argument, which the section header carries. This is the one change that is not backwards-compatible: LLVM 13, which Conan pinned, had the no-argument form.

Forced by GCC 15

Vendored procinfo/process.h names uint64_t without including anything that declares it.

A real hash bug

Abseil's newer hardened builds assert that eq(a, b) implies hash(a) == hash(b), and ClientModelTests aborted on it. The transparent hash functor in SamplingDataPostProcessor hashed a CallstackInfo through its AbslHashValue, which combines (type, frames) — but hashed the heterogeneous lookup key as a std::pair, which combines (frames, type). Keys that compare equal hashed differently, so the heterogeneous lookup the functor exists for could never find anything, and every callstack was inserted afresh. Both overloads now go through one expression.

Thread exclusion that outlived its assumptions

When Orbit injects liborbituserspaceinstrumentation.so into a target, that library starts threads of its own. The instrumentation payload must skip them, or it recurses into the code reporting the events — so OrbitService finds them after injecting and hands their ids back.

It did that by waiting for exactly five threads with hard-coded names, two of which came from gRPC's internal executor. gRPC deleted the executor in 1.73 in favour of an event engine whose pool is sized by core count. The set never matched again, so instrumentation failed outright — and the count is no longer fixed, which the six-argument SetOrbitThreads could not express.

Two tests that asserted CMake's output layout

ExecutablePathTest asserted the test binary sits in a directory named bin — which says nothing about GetExecutablePath and everything about where CMake puts binaries. The copybara note directly above it says as much: Google's internal build skips the test because its directory is named differently. GetTestLibLibraryPath looked for its library only in ../lib for the same reason, and now checks next to the executable first, which is where every other library lookup in the tree starts.

Where the 6m35s goes

Measured on this machine — 32 cores, empty output base, empty disk cache, nothing downloaded:

$ bazel build //...
INFO: Elapsed time: 394.796s, Critical Path: 101.22s
INFO: Build completed successfully, 9520 total actions

Three things keep that number where it is.

Qt is not compiled. It is unpacked. That alone is the difference between six minutes and something closer to an hour.

The TLS stack is not built. Choosing grpc++_unsecure removed roughly 2,500 targets — BoringSSL, and everything in gRPC that only exists to feed it.

A shared disk cache. .bazelrc points --disk_cache at ~/.cache/bazel/orbit, which is keyed by action rather than by output base. Deleting bazel-out, switching branches, or making a second clone re-uses it, so the expensive dependency compile happens once per machine rather than once per checkout. The 6m35s above is the pessimistic case where even that is empty.

A second cold run, on the same machine but with other work in flight, measured 412.6s for the same 9,520 actions — so the figure is stable to within a few percent rather than a lucky run.

Two figures for scale: LLVM's six libraries take about 40 seconds, and py-spy with the 213 crates in its lockfile takes 43. A no-op rebuild is under a second. Put machine-specific settings — --jobs, a remote cache, a different cache location — in .bazelrc.user, which is git-ignored and imported automatically.

Test results

bazel test //... matches 51 targets on Linux, two of which are the Windows test suites and skip cleanly; of the 49 that run, 48 pass. Getting there needed the tracing surface switched on, which is what turned two of the defects in this port up:

$ sudo sysctl -w kernel.perf_event_paranoid=-1 kernel.yama.ptrace_scope=0

Before that, LinuxTracingIntegrationTests reported success while skipping its own body — it gates on IsRunningAsRoot() || perf_event_paranoid <= N. Once it actually ran, it found that cc_test links dynamically where cc_binary does not, so the test binary was split across a shared object per module and a test that reads its own symbol table found nothing there.

The one failure

InstrumentProcessTest.Instrument fails on about four runs in five when run on its own, and two full-suite runs in three; one full bazel test //... has come back 49 of 49. It is a defect in the injection, not in the build — and it is a race, with the timing as the evidence.

Adding two ORBIT_LOG calls around the body of InitializeInstrumentation — nothing else — makes the test pass consistently. The failure rate inverts with machine load. Waiting five times longer on the OrbitService side does not help, because the producer's threads are never created at all rather than created late.

The shape of it: OrbitService clones a thread into the target with CLONE_VM | CLONE_THREAD and, deliberately, no thread-local storage of its own. MachineCodeForCloneCall says as much: "we do not create a new data structure for thread local storage but use the one of the thread we halted." That thread calls InitializeInstrumentation, which starts the producer's connect thread through std::thread, then exits immediately through a raw SYS_exit. Anything that delays that exit makes the problem disappear.

A standalone harness reproducing the borrowed-TLS thread model segfaults every time, which shows the model is fragile. It is not proof of the cause here, though: that harness leaves the TLS owner running concurrently, where ptrace has it halted, so it is harsher than the real path.

Two fixes were tried, and both failing is itself informative:

So it is not enough for the producer's own thread to exist before the injected thread exits. What never appears is gRPC's event-engine pool, which is created later, on first use of the channel. Both attempts were reverted rather than left in shared code.

A fix likely has to keep the injected thread alive until the whole set of Orbit-owned threads exists — a completion signal the injection does not currently have. That is a change to the product rather than to the build. It is the same area the upstream GTEST_SKIP in that test cites (b/237251106).

What still needs actual root

Seven integration tests call geteuid() == 0 directly and skip otherwise — SchedulingSlices, FunctionCalls, two callstack-with-uprobes tests, ThreadStateSlices, ThreadNames, and GpuJobs (which also wants an AMD GPU). No sysctl or permission change unskips them: they read kernel tracepoints under /sys/kernel/debug/tracing, which is 0700 root. Running the test binary under sudo is the only way.

Timing, not correctness

Several tests carry Bazel tags instead of changed assertions. The integration tests are exclusive: they assert that no perf records were lost and no events were discarded as out of order, which is only true on a machine that is not saturated — run alongside 48 other tests, the ring buffer legitimately overflows and the tracer correctly reports it. SessionSetupTests and CaptureEventProducerTests are flaky: one waits 500 ms for a 250 ms timer, the other sleeps 25 ms and expects a gRPC round trip inside it.

Two more tags exist so CI can be precise about what gates a change. The integration tests are also integration, which excludes them on a hosted runner that cannot grant perf_event_open or ptrace — the same set ctest -E IntegrationTest excluded. UserSpaceInstrumentationTests is known_flaky, for the race above: it still runs in CI, in a step marked continue-on-error, so the failure stays visible without blocking unrelated work.

Windows, in one BUILD file per module

The five src/Windows* modules — ETW tracing, the process and process-launcher services, the capture service, and the utility layer underneath them — sat behind if(WIN32) in CMake, which meant a Linux checkout never even parsed them. Bazel has no equivalent of "this directory does not exist on your platform", and that turns out to be the better arrangement: one BUILD.bazel per module describes both platforms, and the graph is complete no matter where you are standing.

Two mechanisms do all of it.

select() picks sources and dependencies inside a target that exists on both platforms. src/Service is the clearest case: the same OrbitService binary links LinuxCaptureService or WindowsCaptureService, and seven modules carry a platform branch: OrbitBase, ObjectUtils, ModuleUtils, ApiLoader, OrbitSsh, ProducerSideService and Service.

target_compatible_with handles the targets that exist on one platform only. The five Windows modules and the four Windows-only vendored libraries declare ["@platforms//os:windows"], which makes bazel build //... on Linux skip them rather than fail — the same wildcard build works on both machines, and asking for one explicitly on the wrong platform gives you a clear message instead of a compiler error.

The four Windows-only libraries

Each was already vendored in third_party/ and each got a BUILD file mirroring what its CMake find-module declared:

The DIA SDK

The PDB reader needs Microsoft's Debug Interface Access SDK, which ships inside a Visual Studio installation rather than as a package. bazel/deps/dia_sdk.bzl is a repository rule that locates it through VSINSTALLDIR — the same search cmake/FindDiaSdk.cmake did — with DIA_SDK_DIR as an override. On Linux it generates an empty stub whose only source is an #error, so every BUILD file that references it still loads on a platform where the SDK cannot exist.

What is verified, and what is not

This was developed on Linux, so none of it has been through MSVC. What is checked is that the whole Windows dependency graph resolves — bazel query walks 6,555 nodes across the five modules without an unresolved label — and that bazel build //... on Linux skips them cleanly rather than failing. Compiling them is the first thing the MSVC leg of CI does.

C++20, on Windows only

The Windows modules use coroutines and <span>. Rather than move the whole tree, .bazelrc sets the standard per platform — --cxxopt=-std=gnu++17 under build:linux, --cxxopt=/std:c++20 under build:windows — which is what the CMake build did too.

Removing CMake

With Windows building, nothing in the tree needed CMake any more, so it went: 146 files, about 7,400 lines. The root and per-module CMakeLists.txt, the twenty-four cmake/Find*.cmake modules and the rest of cmake/, conanfile.py, the Conan profiles under conan/ and contrib/conan/, the MSVC and ARM cross toolchain files, and the two CMakeLists.txt files Orbit had written for vendored code. The genuinely upstream ones — libprotobuf-mutator, MinHook, xxHash — stay, because they are part of those libraries' own sources.

bazel/tools/cmake2bazel.py went with them. A translator whose input no longer exists is a trap, not documentation; it and everything it read are at commit aeda7f74b if the derivation ever needs to be re-run.

Everything downstream of the build system moved with it: the VS Code build task, the license-header check (which walked CMakeLists.txt and now walks .bzl and .bazel), .gitignore, .bazelignore, and the three CI workflows.

CI

build-and-test.yml used to apt-install nineteen packages, configure CMake and run ctest. It now checks out, restores two Bazel caches and runs two commands, across four configurations including MSVC on Windows:

bazel build //...
bazel test --test_tag_filters=-integration,-known_flaky //...

integration is the set that drives perf_event_open and ptrace against a live process, which a hosted runner does not permit — the same set ctest -E IntegrationTest excluded. known_flaky is the one suite with the open race; it still runs, in a step marked continue-on-error, so its output stays visible without blocking unrelated changes. The CodeQL workflow builds through Bazel with --spawn_strategy=local so its tracer can observe the compiler, and the nightly release builds //src/Orbit and //src/Service:OrbitService and publishes tarballs.

What is still not built, on any platform

Two groups, and they are the two CMake also left out: src/OrbitVulkanLayer, src/OrbitTriggerCaptureVulkanLayer and src/VulkanTutorial, which sat behind a WITH_VULKAN option the root CMakeLists.txt left commented out; and src/FuzzingUtils, whose add_subdirectory was commented out.

Where to look