# BazelCon 2024 - Day 2
## Queries
`bazel query --output=build ...` shows "fictitious" attributes like `generator_{name,function,location}` which indicate what produced a target. Useful to understand what macros and globs are doing. These can also be used in filters.
Can bind variables in query. `bazel query 'let foo = kind(...) in $foo` (need to research for a real example). Useful to avoid repetition inside the query.
`--noimplicit_deps` flag to remove implicit deps from rules. Usually what you want.
`--output=graph | dot -Tsvg -o graph.svg` Must filter output or the graph is useless.
cquery supports `--output=starlark --starlark_expr=...` which allows running a piece of Starlark on every target selected by the query. E.g. to query transitive runtime jars. Explore the API with `dir(target)`, `dir(target.actions)`, etc.
`--transitions=full` can be used to debug transitions (but no details in the talk).
`bazel aquery 'mnemonic(CppCompiler, //...)'` shows all inputs, outputs, and command line for every action. Useful to really understand what's happening.
`outputs` and `inputs` filters: e.g. find actions that produce a particular file `outputs(".*path/to/h", deps(//...)))`. (The path filter is an anchored regexp.)
Running many queries in a loop doesn't scale. Consider building a bigger query if possible with post-processing, or using an aspect to dump all info in a single run.
## Creating C++ toolchains easily
Processing a `cc_library`: first, convert the target to a `CcInfo` provider (by the rule). Providers mapped to action templates. Toolchain configurations convert the templates to actions.
Args are unconditionally added to the command line. Features are optional and can also add args.
Proposal: replace toolchain definitions with definitions in build files. E.g. replace strings with labels for features. And more. `cc_toolchain` rule depends on `cc_tool_map` (which is a flat map of tool "names" specified as labels to tool binaries). Then register features to specify flags for these tools. There is also an `args` list, which also points to `cc_args` rules.
There is a living (and tested) example in `rules_cc`. Use as a starting point.
Can combine this with flags defined in build files to e.g. select features or tweak args. The "native" features support in Bazel is not necessary and should be removed.
Would like to remove the concept of a default C++ toolchain so that you have to define a hermetic toolchain in the rules that are in use. (Cheering.)
## Bazel: fast, correct, secure (Engflow)
SBOMS: Some formats are standardized, like CycloneDX and SPDX, but there are more. Heterogeneous file formats.
SBOMS need to be mergable so that, when you import 3P deps, you can pull those in.
In the past, only had `license` which was package-level, and `rules_license` didn't exist. EngFlow added a new `license` rule. We had a `license_collection` rule to list all licenses in a project, but this was error prone. Switched to an aspect to collect licenses transitively. The aspect is complicated because e.g. not all deps end up in a binary (if a dep says `neverlink=1` it should be skipped), or because some are compile-time dependencies only, or because they are compilers, etc.
Want to switch to `rules_license` at EngFlow.
. Different levels of assurance on what went into producing a binary.
SLSA 2 - Requiring writes to the cache to be restricted by CI is sufficient.
SLSA 3 - Would require verifying that anything downloaded from the cache was actually produced from a trusted environment.
RE doesn't provide a way to verify that the output was not produced by a malicious actor. Needs fixing but it hasn't been done yet. Some talks to do this in V3, maybe in V2, but no plans.
## Symbolic macros
Definition is very similar to `rule()` definitions, separating macro definition from implementation. The impl function must take name and visibility parameters (which are defined by default by `macro()`). Attributes have the standard types as rules. Attributes can specify `configurable=False` to disallow selects.
`**kwargs` doesn't work as it used to. Must specify all attributes. Coming soon: attribute inheritance to make it easier to wrap rules.
Macros also restrict the naming of the targets they create. Either take the same name, or take it as a prefix followed by well-defined separators.
Also coming soon: lazy macro expansion so that only macros brought into the build are expanded -- not all the macros that are present in the build file.
Visibility rules work at the macro level almost like they do with rules (but not exactly). A macro that defines a target doesn't need to make it public to be accessible from everywhere, as happened before.
Legacy macros used to call `native.existing_rules()`. "Epilog macro" idiom, found in large monorepos. These macros defines extra stuff for the build files, and therefore must appear at the end of the build file. But what happens if a build file calls more than one epilog macro? Solution: rule finalizers.
Rule finalizers are special symbolic macros with `finalizer=True`. `native.existing_rules()` becomes available and exposes all targets declared not by any finalizer. Only expanded after everything else has been expanded. Consequently, these macros can appear anywhere in the file. Not compatible with lazy macro evaluation.
If the targets from rule finalizers are not part of a build, why should they be? In CI, you probably want to expand all finalizers, but in the IDE you may not... so we'll have another flag to control this behavior.
Landing in Bazel 8.0 without laziness. 8.1 or later will include opt-in laziness and inheritance.
Q: No plans to remove legacy macros. Must use them to remain compatible with older Bazel versions.
Q: Macro expansions usually pollute query results. Hadn't thought about pruning targets expanded by macros from query results. Feature request needs to be filed.
## Reducing Bazel's resource consumption
Were able to reduce 95% of heap memory *after* the build, but that's not very useful. Overall 25-30% improvements. However... people care about peak memory.
* Skyfocus in Bazel 8, experimental. GC based on working set. Question usability of this by users. Footgun? Useful for incremental builds but UX not yet well defined.
* Skeletal analysis: changing evaluation model to reduce peak by 20%. Trades wall time for memory, so not useful for incremental builds. Use in CI.
* (Remote) Analysis caching: sharing state by different server instances. Cache configured targets, aspects, and action execution results. Use remote caching, but inputs to the cache key are "different" (e.g. the Bazel binary itself is part of the key). ~70% heap and runtime reduction for analysis from cold start. Could potentially be used to prune entire subgraphs during the execution phase (so this is more about caching Bazel's internal state, not just the analysis graph). Especially useful for IDEs because IDEs mostly rely on analysis.
* Distributed analysis: shard analysis across Bazel workers. (Ugh. Great for monorepos, but the overhead of Bazel on small machines is still too high even for small repos.) Still in very early stages of discussion.
## Glob handling
Globs are expensive, especially with network file systems, and one glob may depend on another so there is sequencing.
Can speculate and evaluate all globs eagerly, or can prewarm the file system.
Each part of the glob (`**/a/**/b/*.c`) is expressed as individual nodes in Skyframe as directories on disk are being discovered.
Currently evaluating each glob twice, which is inefficient. Would like to avoid that with maybe green threads. Alternatively, encourage globs to be simple and simplify their handling (postpone until the end or similar).
## rules_nixpkgs
nix requires absolute paths (`/nix/store`) and Bazel doesn't like this. Also, data stored twice (nix cache and bazel).
Using external repos to contact the nix remote scheduler, which builds the requested packages, and places them in NFS. This has "side-effects" but we know things are safe thanks to nix. Then, the remote build workers mount the same NFS share (and we know the packages must be present because they were previously built by the external repo rules).
Hard to avoid file copies by Bazel, but possible. One example: instead of specifying a dependency on a path to nix, use a "flat string" that represents the path. We know the path contents are stable so Bazel shouldn't be bothering to look into the content.
## rules_lint
Announcing version 1.0. Usable through the BCR.
Two rulesets in one: formatting and linting.
Goal is to support formatting for all languages at once, without worrying about the details. "Because while your code might be art... it isn't ASCII art".
Must be deterministic and fast, so that they fit in pre-commit hooks. Should integrate with IDE extensions.
Using `bazel run`: if it was an action, all input files would need to be part of rules, but that's not desirable all the time (e.g. for Markdown files).
`rules_multitool`: Fetches tools and creates a lock file for them.
`rules_multirun`: Assemble multiple binaries to run together.
`.git-blame-ignore-revs` -- if doing one-off formatting operations, make sure to list the commits here to not show up as the author of the whole repo.
For linting, supporting fewer languages because they are harder to wire up. Very different from formatting: need to support multiple linters per language; the linter changes may modify the behavior of the code; there might be more than one fix per issue; can suppress violations; may need inspecting more than one file; is slow.
Want a unified interface, like for formatting, but don't want rules or changes to build files. Want regular actions for these. Modeling effects as "outputs" (reports, patches, and exit code).
Using aspects to do linting. Invoking Bazel is difficult: we need to pass the aspects to the CLI, define the output groups, and ask for certain files to be downloaded if BwoB is enabled.
The "Aspect CLI" is a Bazel fork which modifies the client to add extra commands like "lint". Does not fork the server portion.
Q: What about validation actions from last year's conference? Using these is hard because you have to modify the ruleset of the languages to add validation, whereas with aspects you don't have to.
## Integration with other build systems (MongoDB)
scons generates ninja file. Using ninja, not scons, for efficiency. ninja has very little overhead.
Building takes 2 hours in an 8-core machine. With Icecream (what is this?), it takes 15 minutes. Using remote caching, but not remote execution. Very fragile.
Goal is to give scons repos the tools to migrate.
Initial approach: parallel build systems. Can you quantify the cost of the "no value being delivered" gap while you do a migration? Is the team OK with the old legacy build stagnating for a long time?
Proposed approach: interoperable build systems. Set up basic Bazel and then start integrating both so that they work in unison. The benefits of Bazel increase over time, instead of being a big-bang change. Ensures that you can't get a new feature added to only one of the builds by a "cowboy developer". All interop work is throw-away, but that's part of the deal.
Bottom-up Bazel integration: using scons as the driver and then trying to replace the bottom leaf nodes with Bazel builds. Similar to what Salesforce did with Maven (?).
Tried to use aquery to generate the mapping of scons labels to Bazel targets/output files, but it was too slow because we had to run 100s of queries and each query is 100-500ms. Instead, wrote a little rule that consumes the linker provider and extracted the details from there.
## Scala
Scala bad parts in 2015: binary incompatibility, macros everywhere, slow compilation, too much abstractions, complex code...
Scala in 2024: not much better regarding versions (still 4 active versions), but 3.x offers backwards compatibility and resolves the previous issues.
Scala is "the biggest FP language". Not owned by any one company.
## Misc
Interesting: QR codes for URLs in presentations. Look at sharing link from Chrome.
## Monolithic targets
Monoliths under active development are bad for the org. Devs more likely to contribute to the monolith than not and reflect the culture of the team.
How to tackle the monolith? Make sure there are tests. Ensure leadership is onboard, and document decisions. Make sure scope is well-defined.
Work towards automation to fix common errors after moving a file. (We don't have any of this.) Visualize the build graph to identify sets of classes that could be pulled out. Work from the leaves towards the core.
## Toolchain debugging
`--toolchain_resolution_debug`'s help claims that the flag is only for experts.
Messages have been cleaned up. If you know the resolution algorithm, the output is now understandable.
## Test execution output in safety-critical industries
As part of the automotive industry, we have to provide reports when testing the code (code coverage, unit test execution reports). The reports come from the tests, but we can't easily get them with a `bazel test` invocation.
We want to collect all reports into a tarball, so we have to depend on the test execution output, and we want to validate the report (e.g. to ensure there is 100% line/branch coverage for the PR).
Compromise solution: run the tests during the build (under a specific config). Added a macro that wraps `cc_test` and then also instantiates a custom rule to generate the reports.
## Compilation database in Bazel
JSON file that contains the commands to compile all translation units in a project.
CMake has a built-in feature to generate these. With Bazel, we need one of the many open source solutions that are available. Options:
* Intercept builds: Use `bear` as a wrapper to run Bazel. Requires full build but doesn't integrate with Bazel because of the background server process.
* Extra actions: Add a "listener" extra action that listens for `CppCompile`. Also requires full build and is deprecated by aspects. Example: `kythe`.
* Action graph query: No need for a full build, just a warmed up analysis cache. Doesn't support tree artifacts. Examples: `bazel-compile-commands-extractor`, `bazel-compile-commands` (two forks).
* Aspects: No need for a full build, but the generated commands may not be identical to the ones that are actually used.
E.g. have a code generator that produces a tree artifact and then is fed as an input to a `cc_library`. Tree artifacts are represented as a `CppCompileActionTemplate`: expands to an action at execution time. These cannot work with `aquery`, but with an aspect, we have to delay processing until execution.