Open files limit, macOS, and the JVM

Bazel’s original raison d’etre was to support Google’s monorepo. A consequence of using a monorepo is that some builds will become very large. And large builds can be very resource hungry, especially when using a tool like Bazel that tries to parallelize as many actions as possible for efficiency reasons. There are many resource types in a system, but today I’d like to focus on the number of open files at any given time (nofiles).

January 29, 2019 · Tags: bazel, jvm, macos, monorepo, portability
Continue reading (about 3 minutes)

#! /usr/bin/env considered harmful

Many programming guides recommend to begin scripts with the #! /usr/bin/env shebang in order to to automatically locate the necessary interpreter. For example, for a Python script you would use #! /usr/bin/env python, and then the saying goes, the script would “just work” on any machine with Python installed. The reason for this recommendation is that /usr/bin/env python will search the PATH for a program called python and execute the first one found… and that usually works fine on one’s own machine.

September 14, 2016 · Tags: featured, portability, programming, scripts, unix
Continue reading (about 5 minutes)

Using va_copy to safely pass ap arguments around

Update (2014-12-19): The advice provided in this blog post is questionable and, in fact, probably incorrect. The bug described below must have happened for some unrelated reason (like, maybe, reuse of ap), but at this point (three years later!) I do not really remember what was going on here nor have much interest in retrying. A long time ago, while I was preparing an ATF release, I faced many failing tests and crashes in one of the platforms under test.

September 12, 2011 · Tags: c, portability
Continue reading (about 3 minutes)

Forget about test(1)'s == operator

Some implementations of test(1), in an attempt to be smart, provide non-standard operators such as ==. Please forget about those: they make your scripts non-portable and a pain to use in other systems. Why? Because, due to the way the shell works, failures in calls to test(1) will often just result in an error message (which may not be seen due to other output) and the script will happily continue running even if it missed to perform some important operation.

April 24, 2010 · Tags: portability
Continue reading (about 1 minute)

Always define an else clause for portability #ifdefs

If you use #ifdef conditionals in your code to check for portability features, be sure to always define a catch-all else clause that actually does something, even if this something is to error out. Consider the following code snippet, quoted from gamin's tests/testing.c file:if (arg != NULL) { #ifdef HAVE_SETENV setenv("GAM_CLIENT_ID", arg, 1); #elif HAVE_PUTENV char *client_id = malloc (strlen (arg) + sizeof "GAM_CLIENT_ID="); if (client_id) { strcpy (client_id, "GAM_CLIENT_ID=");

April 22, 2010 · Tags: portability
Continue reading (about 3 minutes)

unlink(2) can actually remove directories

I have always thought that unlink(2) was meant to remove files only but, yesterday, SunOS (SXDE 200709) proved my wrong. I was sanity-checking the source tree for the imminent ATF 0.4 release under this platform, which is always scary, and the tests for the atf::fs::remove function were failing — only when run as root. The failure happened in the cleanup phase of the test case, in which ATF attempts to recursively remove the temporary work directory.

February 3, 2008 · Tags: atf, portability, sunos
Continue reading (about 2 minutes)

How to kill a tree of processes

Yesterday I mentioned the need for a way to kill a tree of processes in order to effectively implement timeouts for test cases. Let's see how the current algorithm in ATF works: The root process is stopped by sending a SIGSTOP to it so that it cannot spawn any new children while being processed.Get the whole list of active processes and filter them to only get those that are direct children of the root process.

January 16, 2008 · Tags: atf, portability, process
Continue reading (about 3 minutes)

SoC: Prototypes for basename and dirname

Today, I've attempted to build atf on a NetBSD 4.0_BETA2 system I've been setting up in a spare box I had around, as opposed to the Mac OS X system I'm using for daily development. The build failed due to some well-understood problems, but there was an annoying one with respect to some calls to the standard XPG basename(3) and dirname(3) functions. According to the Mac OS X manual pages for those functions, they are supposed to take a const char * argument.

June 28, 2007 · Tags: atf, portability, soc
Continue reading (about 2 minutes)