Showing 26 posts
The point of this post is simple and I’ll spoil it from the get go: every time you make an assumption in a piece of code, make such assumption explicit in the form of an assertion or error check. If you cannot do that (are you sure?), then write a detailed comment. In fact, I’m exceedingly convinced that the amount of assertion-like checks in a piece of code is a good indicator of the programmer’s expertise.
I am pleased to announce that the first release of sandboxfs, 0.1.0, is finally here! You can download the sources and prebuilt binaries from the 0.1.0 release page and you can read the installation instructions for more details. The journey to this first release has been a long one. sandboxfs was first conceived over two years ago, was first announced in August 2017, showed its first promising results in April 2018, and has been undergoing a rewrite from Go to Rust.
Blaze—the variant of Bazel used internally at Google—was originally designed to build the Google monorepo. One of the beauties of sticking to a monorepo is code reuse, but this has the unfortunate side-effect of dependency bloat. As a result, Bazel and Blaze have evolved to support ever-increasingly-bigger pieces of software. The growth of the projects built by Bazel and Blaze has had the unsurprising consequence that our engineers all now have high-end workstations with access to massive amounts of distributed resources.
During the summer of last year, I hosted an intern who implemented sandboxfs: a FUSE-based file system that exposes an arbitrary view of the host’s file system under the mount point. At the end of his internship, we had a functional sandboxfs implementation and some draft patches for integration in Bazel. The goal of sandboxfs in the context of Bazel is to improve the performance of builds when action sandboxing is enabled.
This post is a short, generalized summary of the preceeding two. I believe those two posts put readers off due to their massive length and the fact that they were seemingly tied to Bazel and Java, thus failing to communicate the larger point I wanted to make. Let’s try to distill their key points here in a language- and project-agnostic manner.
In part 1 of this series, I made the case that you should run away from the shell when writing integration tests for your software and that you should embrace the primary language of your project to write those. Depending on the language you are using, doing this will mean significant more work upfront to lay out the foundations for your tests, but this work will pay off. You may also feel that the tests could be more verbose than if they were in shell, though that’s not necessarily the case.
My latest developer productivity rant thesis is that integration tests should be written in the exact same language as the thing they test. Specifically, not shell. This theory applies mostly to tests that verify infrastructure software like servers or command line tools. It is too easy to fall into the trap of using the shell because it feels like the natural choice to interact with tools. But I argue that this is a big mistake that hurts the long-term health of the project, and once trapped, it’s hard to escape.
In a version control system, a rollback is a type of change that undoes the effects of a previous commit. In essence, a rollback is a commit that applies the inverse diff of another commit. At Google, our tools make it trivial to create rollbacks for a given changelist or CL. (A CL is similar to a commit but can be either pending—in review—or submitted.) Making it trivial to create rollback CLs is important in a culture where the standard upon encountering a problem is “rollback first, ask questions later” because it removes friction from the process of backing out problematic changes.
Since the announcement of sandboxfs a few weeks ago, I've been stabilizing its integration with Bazel as a new sandboxing technique. As part of this work, I encountered issues when macOS was immediately killing signed binaries executed through the sandbox. Read on for the long troubleshooting process and the surprising trivial solution.
sandboxfs is a FUSE-based file system that exposes an arbitrary view of the host’s file system under the mount point, and offers access controls that differ from those of the host. You can think of sandboxfs as an advanced version of bindfs (or mount --bind or mount_null(8) depending on your system) in which you can combine and nest directories under an arbitrary layout. The primary use case for this project is to provide a better file system sandboxing technique for the Bazel build system.
This is a tutorial to guide you through the shiny new pkg_comp 2.0 on macOS using the macOS-specific self-installer. Goals: to use pkg_comp 2.0 to build a binary repository of all the packages you are interested in; to keep the repository fresh on a daily basis; and to use that repository with pkgin to maintain your macOS system up-to-date and secure.
This is a tutorial to guide you through the shiny new pkg_comp 2.0 on NetBSD. Goals: to use pkg_comp 2.0 to build a binary repository of all the packages you are interested in; to keep the repository fresh on a daily basis; and to use that repository with pkgin to maintain your NetBSD system up-to-date and secure.
On April 14th, 2016, Microsoft announced the 1.0 release of their open-source Visual Studio Code (VSCode) editor. I’ve been drive-testing it for a few months and have been quite pleased with it, so here go my impressions. How did I get here? Let’s backtrack a bit first. I’ve been a Vim and Emacs user for many years. Yes, I use both regularly depending on what I have to achieve. For me, Vim shines in doing quick single-file changes and repetitive edits through many files, while Emacs shines in long-lived coding sessions that involve numerous open buffers.
I confess I am late to the game: the Go programming language came out in 2009 and I had not had the chance to go all in for a real project until two weeks ago. Here is a summary of my experience. Spoiler alert: I’m truly pleased. The project What I set out to build is a read-only caching file system to try to solve the problems I presented in my previous analysis of large builds on SSHFS.
As a software developer, you have probably disregarded the build system of your project—those pesky Makefiles—as unimportant. You have probably “chosen” to use the de-facto build tool make(1). And you have probably hacked your way around until things “seemingly worked”. But hang on a second. Those build files are way more important than you may think and deserve a wee bit more attention.
Last week, I spent some time looking at the feasibility of using SSHFS on OS X to access Google’s centralized source tree for the purpose of issuing local builds. My goals were two-fold: first, to avoid having to “clone” the large source code of the apps I wanted to build; and, second, to avoid having to port the source file system (a FUSE module) to the Mac. What I found highlights that SSHFS is not the right choice for locally building a remote source tree. That said, the overall study process was interesting, fun, and I am now tempted to make SSHFS viable for this use case. Read on for the details.
For the last two years, I had been meaning to write an Android app just for the sake of it. I had attempted to do so in short chunks of “free time”, but that never played out well: I had to force myself to sit down for a few hours straight to fight Android Studio and overcome the initial difficulties in coding for an unknown platform. That’s why, during the last Thanksgiving week, I took three days off of work to focus on writing my first Android app. The goal was to get a basic app that could later be built on iteratively as open source. The specifics of the app did not matter much for this exercise, but I had a simple idea in mind.
A commonly held axiom in the BSD community is that the C compiler belongs in the base system. “This is how things have been since the beginning of time and they define the way BSD systems are”, the proposition goes. But why is that? What makes “having a compiler in base” a BSD system? Why is the compiler a necessary part of the base system? Hold on, is it? Could we take it out?
You are the developer in charge to resolve a problem and have prepared a changelist to fix the bug. You need the changelist to be reviewed by someone else before checkin. Your changelist is an ugly hack. What kind of response are you gonna get from your reviewer? Well as with everything: it depends! (Cover image courtesy of http://www.startupstockphotos.com/.) If you have: clearly stated upfront that the changelist is a hack, explained how it is a hack, justified that the hack is the right thing to do at this moment, and outlined what the real solution to get rid of the hack would be then your reviewer will most likely just accept the change without fuss (!
This is a rare post because I don’t usually talk about Google stuff here, and this post is about Bazel: a tool recently published by Google. Why? Because I love its internal counterpart, Blaze, and believe that Bazel has the potential to be one of the best build tools if it is not already. However, Bazel currently has some shortcomings to cater to a certain kind of important projects in the open source ecosystem: the projects that form the foundation of open source operating systems.
The project I'm currently working on at university uses Subversion as its version control system. Unfortunately, the project itself has no mailing list to receive notifications on every commit, and the managers refuse to set this up. They do not see the value of such a list and they are scared of it because they probably assume that everyone ought to be subscribed to it. Having worked on projects that have a commit notification mailing list available, I strongly advise to have such a list anytime you have more than one developer working on a project.
A long while ago — just before buying the MacBook Pro — I already complained about software bloat. A year and two months later, it is time to complain again. I am thinking on renewing my MacBook Pro assuming I can sell this one for a good price. The reasons for this are to get slightly better hardware (more disk, better GPU and maybe 4GB of RAM) and software updates. The problem is: if I am able to find a buyer, I will be left without a computer for some days, and that's not a good scenario.
A bit more than three years ago, I renewed my main machine and bought an Athlon XP 2600+ with 512MB of RAM and a 80GB hard disk. The speed boost I noticed in games, builds and the overall system usage was incredible — I was coming from a Pentium II 233 with 384MB of RAM. With the change, I was finally able to switch from plain window managers to desktop environments (alternating KDE and GNOME from time to time) and still keep a usable machine.