Waiting for process groups, introduction

Process groups are a feature of Unix systems to group related processes under a common identifier, known as the PGID. Using the PGID, one can look for these related process and send signals in unison to them. This is typically used by shell interpreters to manage processes. For example, let’s launch a shell command that puts two sleep invocations in the background (those with the 10- and 20-second delays) and then sleeps the direct child (with a 5-second delay)—while also putting the whole invocation in the background so that we can inspect what’s going on:

November 12, 2019 · Continue reading (about 6 minutes) · Comments

Bazel's process-wrapper helper tool

As strange as it may sound, a very important job of any build tool is to orchestrate the execution of lots of other programs—and Bazel is no exception. Once Bazel has finished loading and analyzing the build graph, Bazel enters the execution phase. In this phase, the primary thing that Bazel does is walk the graph looking for actions to execute. Then, for each action, Bazel invokes its commands—things like compiler and linker invocations—as subprocesses.

November 8, 2019 · Continue reading (about 6 minutes) · Comments

A quick glance at macOS' sandbox-exec

macOS includes a sandboxing mechanism to closely control what processes can do on the system. Sandboxing can restrict file system accesses on a path level, control which host/port pairs can be reached over the network, limit which binaries can be executed, and much more. All applications installed via the App Store are subject to sandboxing. This sandboxing functionality is exposed via the sandbox-exec(1) command-line utility, which unfortunately has been listed as deprecated for at least the last two major versions of macOS.

November 1, 2019 · Continue reading (about 3 minutes) · Comments

Resurrected ONLamp.com articles

Quite some time ago, I wrote a handful of guest posts for O’Reilly’s ONLamp.com online publication. Unfortunately, that site seems now gone from the interwebs and I only noticed by chance upon realizing that some of my links to those articles were now broken. Fortunately, I was able to find copies of those articles via archive.org’s WayBack Machine. So I have now taken the chance to import those articles into this site.

October 11, 2019 · Continue reading (about 1 minute) · Comments

Sample REST API in Rust and Go

Over the summer, I prototyped a bunch of web apps whose ideas had been floating in my mind for a long time. I spent quite a bit of time learning about REST APIs and, as part of these exercises, implemented skeletons of REST servers in both Go and Rust. (Just for context, the last time I wrote a web app was in high school… and it involved PHP, MySQL, and I think IE6?

September 27, 2019 · Continue reading (about 13 minutes) · Comments

Safely restoring the previous working directory

The current working directory, or CWD for short, is a process-wide property. It is good practice to treat the CWD as read-only because it is essentially global state: if you change the CWD of your process at any point, any relative paths you might have stored in memory1 will stop working. I learned this first many years ago when using the Boost.Filesystem library: I could not find a function to change the CWD and that was very much intentional for this reason.

September 21, 2019 · Continue reading (about 3 minutes) · Comments

Optimizing tree deletions in Bazel

Bazel likes creating very deep and large trees on disk during a build. One example is the output tree, which naturally contains all the artifacts of your build. Another, more problematic example is the symlink forest trees created for every action when sandboxing is enabled. As garbage gets created, it must be deleted. It turns out, however, that deleting file system trees can be very expensive—and especially so on macOS. In fact, calls to our deleteTree algorithm routinely showed up in my profiling runs when trying to diagnose slowdowns using the dynamic scheduler.

March 22, 2019 · Continue reading (about 4 minutes) · Comments