How to leverage an old dirty trick in your Bazel wrapper for great effect

As powerful as Bazel is, sometimes it’s not featureful enough. When using this build system, it’s common practice to wrap it in a launcher script—and in fact, this is natively supported by Bazelisk, Bazel’s native dispatcher that stands for the bazel binary in the user’s PATH. Bazelisk will first download the version of Bazel requested by the project, and then, if tools/bazel exists, invoke it instead of the downloaded binary. tools/bazel is what’s known as a Bazel wrapper and is the point of today’s article.

Well, not quite. The actual point of today’s article is to demonstrate a simple trick I learned from the GNU Autoconf and Automake days to implement full-blown conditionals in an ad-hoc template system. But because such trick is trivial once you see it, I have to present it in the context of a modern real-world scenario. So what I’m going to do is guide you through the creation of your very own Bazel wrapper to customize Bazel’s .bazelrc configuration file in ways that the native Bazel tool doesn’t support.

Let’s get started.

The context

Template systems are everywhere. Take any static blog generation system and you’ll find some. Take system management tools like Ansible and you’ll find others. Take a cloud orchestration service like Kubernetes and you’ll find Helm. Heck, even Go’s standard library provides a full blown text template system out of the box.

There is clear benefit and appetite for these and, surely enough, it’s tempting to use any pre-existing such system in your own project… but if all you need are a bunch of variable replacements, some of which may be only conditionally applied, you can go a long way by not taking any dependencies. A call to sed or the substring function of your language of choice is all you need.

To put this in context, let’s say you have Bazel’s .bazelrc configuration file, which is not very flexible, and that you need to set some arguments based on dynamic values that depend on the environment. E.g. something like this:

# Developer builds
startup --host_jvm_args=-Xmx4g
build --jobs=200

# CI builds
startup --host_jvm_args=-Xmx16g
build --jobs=1000

If you know a little bit about .bazelrc, however, you may squint at that and say: “That’s silly! Make those flags conditional on a configuration and you’re set!” So you try something like this:

# Developer builds
startup:dev --host_jvm_args=-Xmx4g
build:dev --jobs=200

# CI builds
startup:ci --host_jvm_args=-Xmx16g
build:ci --jobs=1000

# Enable dev as default configuration
common --config=dev
# ... but CI can pass --config=ci on the CLI to override.

And… this does not work. Gotcha! Startup flags cannot be placed behind a configuration so there is no way for you to parameterize the JVM’s max heap value passed in -Xmx. And having to remember to pass --config=ci from CI all the time is fragile, because you might forget and not get the desired configuration in place.

Basic string replacements

Solving the above is not difficult if we could parameterize the configuration. We might want to write something like this instead:

startup --host_jvm_args=-Xmx@MAX_HEAP@
build --jobs=@JOBS@

… and then have @MAX_HEAP@ and @JOBS@ be replaced dynamically depending on some runtime arbitrary logic. We can do that via the Bazel wrapper, and this sort of dynamic configuration is a common thing to do from it. So let’s do this.

Let’s start with the template logic:

declare -A SUBSTS

subst() {
    local var="${1}"; shift
    local value="${1}"; shift

    SUBSTS["${var}"]="${value}"
}

generate() {
    local in_file="${1}"; shift
    local out_file="${1}"; shift

    local args=()
    for var in "${!SUBSTS[@]}"; do
        local value="${SUBSTS[$var]}"
        args+=( -e "s|@${var}@|${value}|g" )
    done
    sed "${args[@]}" "${in_file}" >"${out_file}"
}

Ugly(?) bash syntax but nothing too complicated:

• The global SUBSTS hashmap tracks variable names and their replacement values.
• The subst function inserts a new key/value pair into SUBSTS. (It’s important that the values given to subst don’t contain sed-special characters like &, backslashes, or the | separator we chose—but we control the generation of those values so we are good.)
• The generate function transforms the hashmap into a set of sed arguments of the form -e s|@VAR@|VALUE|g and then calls sed to process the given input file into the given output file.

Then, we can plug everything together into a minimal Bazel wrapper:

#! /bin/bash

set -euo pipefail

WORKSPACE="$(cd $(dirname "${0}")/.. && pwd -P)"
readonly WORKSPACE

# ... insert template logic from above.

run() {
    local bazelrc
    bazelrc="$(mktemp -p "${TMPDIR:-/tmp}" bazelrc.XXXXXXXX)"
    trap "rm -f '${bazelrc}'" EXIT HUP INT QUIT TERM
    generate "${WORKSPACE}/.bazelrc.in" "${bazelrc}"
    "${REAL_BAZEL}" --noworkspace_rc --bazelrc="${bazelrc}" "${@}"
}

main() {
    if [[ "${CI:-false}" == true ]]; then
        subst MAX_HEAP 16g
        subst JOBS 1000
    else
        subst MAX_HEAP 4g
        subst JOBS 200
    fi

    run "${@}"
}

main "${@}"

In this new snippet, the run function instantiates the .bazelrc file from the contents of SUBSTS via generate and then calls the actual Bazel binary provided by Bazelisk in REAL_BAZEL. The complexity here may seem overkill, but it’s necessary: while it’s pointless to invoke Bazel in parallel due to its global lock, users will run Bazel in parallel and you must make sure that the wrapper is reentrant. Otherwise, you’ll definitely run into races.

The rest of the script in main does the actual work to compute key/value pairs to substitute in your now-templated .bazelrc and then delegates to Bazel via run.

That’s it. This is a barebones implementation of a text template system using bash—and I had to use bash, not sh, to get the niceties of a hashmap—that serves as a tools/bazel launcher. Go try it.

By the way, the @VAR@ and subst nomenclature are inherited from GNU Autoconf’s AC_SUBST primitive.

Conditionals

“Great!” I hear you say in a sarcastic tone. “You have just applied string replacements! But what about conditionals, huh? You CaNnOt Do ThAt So EaSiLy!!11!one!”

Ah, but you can, and showing you that trick is the whole point of this short article, remember?

The necessary insight is that we can use string replacements to comment out lines in the original file. What if we did this:

startup --host_jvm_args=-Xmx@MAX_HEAP@

build:dev --jobs=400
build:ci --jobs=1000

# Pick default build mode.
@CI_FALSE@build --config=dev
@CI_TRUE@build --config=ci

In here, we are defining different configurations for developer workstations and for CI, like we did earlier, but then we are auto-magically picking the default configuration depending on @CI_TRUE@ and @CI_FALSE@. How? Well: @CI_TRUE@ will expand to the empty string when running on CI and @CI_FALSE@ will expand to #, so the corresponding lines will be enabled and disabled. And the opposite replacement values will appear when not running on CI. Ta-da! Conditionals.

We can make things nicer with a helper function and meta-programming:

subst_if() {
    local var="${1}"; shift

    if eval "${*}"; then
        subst "${var}_TRUE" ""
        subst "${var}_FALSE" "#"
    else
        subst "${var}_TRUE" "#"
        subst "${var}_FALSE" ""
    fi
}

subst_if CI '[[ "${CI:-false}" == true ]]'

Don’t panic about that eval. Just as with the sed invocation above where we could have issues with special characters appearing in values, we control the arguments to subst_if so the eval is safe.

And note that we can even nest conditionals arbitrarily. There is nothing preventing you from doing:

@CONDITION_1_TRUE@@CONDITION_2_TRUE@build --flag1
@CONDITION_1_TRUE@@CONDITION_2_FALSE@build --flag2
@CONDITION_1_FALSE@@CONDITION_2_TRUE@build --flag3
@CONDITION_1_FALSE@@CONDITION_2_FALSE@build --flag4

Which corresponds to the conceptual equivalent of:

if CONDITION_1
    if CONDITION_2
        build --flag1
    else
        build --flag2
    endif
else
    if CONDITION_2
        build --flag3
    else
        build --flag4
    endif
endif

Loops

Let’s do loops? Sorry no, can’t do!

Well akshually… we could do loops. Not by using simple tricks like above, but we could definitely sketch something like this:

while read line; do
    case "${line}" in
        for*)
            # Handle the loop logic here...
            ;;

        *)
            echo "${line}"
            ;;
    esac
done <"${input}" >"${output}"

However, this is starting to look a lot like a high-level parser, not scripting where you glue simpler components together. And if you are headed that way, you are better off transitioning to a proper programming language and a well-known template system.

So…

What do you think? Do you hate this already? You can, but note that the whole world runs on this stuff. All of that foundational code behind Linux systems ends up using GNU Automake and GNU Autoconf, and those packages are full of stuff like this in their configure and Makefile.in files.

And you can get very far with just the above constructs if you treat the shell like a real language. The Bazel wrapper that I maintain at work these days grew to almost 1000 lines of code before I pruned a lot of features that had become unnecessary, but it’s still pretty large. We are now transitioning it to a Go-based wrapper for better readability and maintainability… but as we do this, I’m reminded that well-groomed shell scripts give you some flexibility that no other language can match in just a few lines.

So, keep things simple. You can do a lot with just a few primitives.