Sandboxfs - Julio Merino (jmmv.dev)

What is Rust's Into<T> for?

Rust provides a bunch of traits that you may use or implement in your code, but unless you have experienced them first-hand, it can be hard to imagine what their real utility is. For example, if you go read Into’s documentation, all you find is: Trait std::convert::Into A value-to-value conversion that consumes the input value. The opposite of From. […] Yay, very useful. This text tells me what this trait does, which is fine for a reference manual, but not when I could find it useful. Mind you, during the initial code reviews for sandboxfs, my reviewer pointed out a few times that I should be using into() or into_iter() in a few places but I never quite figured out why.

April 27, 2020 · Tags: <a href="/tags/programming">programming</a>, <a href="/tags/rust">rust</a>, <a href="/tags/sandboxfs">sandboxfs</a>
Continue reading (about 4 minutes)

FOSDEM navigation 101

FOSDEM 2020 is over. As I type this, I’m on my way back home from the conference in Brussels. And it has been nice. In the end. I must confess I was frustrated by the middle of the first day, though things got better after that. Here is the thing: FOSDEM is not your usual conference. There are lots of things going on at once and all of them are crowded. Really, really crowded—to the point where the situation doesn’t make any sense unless you know how to work around it.

February 3, 2020 · Tags: <a href="/tags/conference">conference</a>, <a href="/tags/sandboxfs">sandboxfs</a>
Continue reading (about 6 minutes)

The OSXFUSE, hard links, and dladdr puzzle

Hello everyone and welcome to this new decade! It’s already 2020 and I’m only 17 days late in writing a first post. I was planning to start with an opinion article, but as its draft is taking longer than I wanted… I’ll present you the story of a recent crazy bug that has kept me busy for the last couple of days. Java crashes with Bazel and sandboxfs On a machine running macOS Catalina, install sandboxfs and build Bazel with sandboxfs enabled, like this:

January 17, 2020 · Tags: <a href="/tags/bazel">bazel</a>, <a href="/tags/fuse">fuse</a>, <a href="/tags/internals">internals</a>, <a href="/tags/programming">programming</a>, <a href="/tags/sandboxfs">sandboxfs</a>
Continue reading (about 13 minutes)

Output conflicts and dynamic execution

When the dynamic scheduler is active, Bazel runs the same spawn (aka command line) remotely and locally at the same time via two separate strategies. These two strategies want to write to the same output files (e.g. object files, archives, or final binaries) on the local disk. In computing, two things trying to affect the same thing require some kind of coördination. You might think, however, that because we assume that both strategies are equivalent and will write the same contents to disk1, this is not problematic. But, in fact, it can be, because file creations/writes are not atomic. So we need some form of mutual exclusion in place to avoid races.

December 27, 2019 · Tags: <a href="/tags/bazel">bazel</a>, <a href="/tags/sandboxfs">sandboxfs</a>
Continue reading (about 4 minutes)

What are Bazel's strategies?

“Strategies? Will you talk about Bazel’s strategy for world domination 🙀?” No… not exactly that. Dynamic execution has been quite a hot topic in my work over the last few months and I am getting ready to publish a series of posts on it soon. But before I do that, I need to first review Bazel’s execution strategies because they play a big role in understanding what dynamic execution is and how it’s implemented.

December 14, 2019 · Tags: <a href="/tags/bazel">bazel</a>, <a href="/tags/sandboxfs">sandboxfs</a>
Continue reading (about 6 minutes)

Hello, sandboxfs 0.1.0

I am pleased to announce that the first release of sandboxfs, 0.1.0, is finally here! You can download the sources and prebuilt binaries from the 0.1.0 release page and you can read the installation instructions for more details. The journey to this first release has been a long one. sandboxfs was first conceived over two years ago, was first announced in August 2017, showed its first promising results in April 2018, and has been undergoing a rewrite from Go to Rust. (And by the way, this has been my 20% project at Google so rest assured that they are still possible!)

February 5, 2019 · Tags: <a href="/tags/bazel">bazel</a>, <a href="/tags/featured">featured</a>, <a href="/tags/pkg_comp">pkg_comp</a>, <a href="/tags/sandboxctl">sandboxctl</a>, <a href="/tags/sandboxfs">sandboxfs</a>, <a href="/tags/software">software</a>
Continue reading (about 7 minutes)

Rust vs. Go

There is no good answer to this question: people tend to put Go and Rust in the same bucket because they were released at around the same time, because Rust’s release felt like a response to Go’s, and because they are marketed to similar audiences. They are, however, vastly different. So let’s give in and compare them anyway.

July 13, 2018 · Tags: <a href="/tags/featured">featured</a>, <a href="/tags/go">go</a>, <a href="/tags/programming">programming</a>, <a href="/tags/rust">rust</a>, <a href="/tags/sandboxfs">sandboxfs</a>
Continue reading (about 13 minutes)

Rust review: Protect the data

The one thing that blew my mind about Rust is its approach to data sharing in concurrent situations. I had always thought of mutexes as something that is easy to get wrong and was convinced that the use of a RAII pattern to prevent lock leaks never happen (like with Abseil’s MutexLock) was the panacea. (I’m a fan of RAII in C++ by the way, in case you haven’t noticed.) As Rust has taught me, that’s far from the truth: in Rust, you protect the data, not the code. What this means is that, e.g. a mutex is not an object to control access to a piece of data: a mutex is a container for a piece of data. It is impossible to access the data without going through the mutex.

June 5, 2018 · Tags: <a href="/tags/programming">programming</a>, <a href="/tags/rust">rust</a>, <a href="/tags/sandboxfs">sandboxfs</a>
Continue reading (about 5 minutes)

Rust review: The borrow checker

Aaaah, the borrow checker: the dreaded enemy lurking within the Rust compiler, ready to make its move to bring pain to your life by preventing your code from compiling. Or that’s what everyone seems to say, which is one of the reasons I put off learning Rust for so long. In reality… the borrow checker is a blessing, but it is true that getting past its gates is difficult at first.

June 1, 2018 · Tags: <a href="/tags/programming">programming</a>, <a href="/tags/rust">rust</a>, <a href="/tags/sandboxfs">sandboxfs</a>
Continue reading (about 3 minutes)

Rust review: Introduction

I had been meaning to learn Rust since I first toyed with Go a couple of years ago. During this period, I’ve written a non-trivial amount of Go code both inside and outside Google, but never found the chance to sit back and learn Rust. This changed a month ago during my yearly family trip to Korea. This time around, I decided upfront that I would not work on any personal or work projects for the 2-week long vacation. Instead, I would focus all spare time in reading. And I would read “The Rust Programming Language”, second edition. The plan worked: getting through the book took the two weeks and I barely wrote any code.

May 25, 2018 · Tags: <a href="/tags/programming">programming</a>, <a href="/tags/rust">rust</a>, <a href="/tags/sandboxfs">sandboxfs</a>
Continue reading (about 3 minutes)

Preliminary sandboxfs support in Bazel

During the summer of last year, I hosted an intern who implemented sandboxfs: a FUSE-based file system that exposes an arbitrary view of the host’s file system under the mount point. At the end of his internship, we had a functional sandboxfs implementation and some draft patches for integration in Bazel. The goal of sandboxfs in the context of Bazel is to improve the performance of builds when action sandboxing is enabled. The way in which we try to do so is by replacing the costly process of setting up the file system for each action using symlinks with a file system that does so “instantaneously”.

April 13, 2018 · Tags: <a href="/tags/bazel">bazel</a>, <a href="/tags/google">google</a>, <a href="/tags/sandboxfs">sandboxfs</a>, <a href="/tags/software">software</a>
Continue reading (about 2 minutes)

A case for writing Bazel's integration tests in Java, part 2

In part 1 of this series, I made the case that you should run away from the shell when writing integration tests for your software and that you should embrace the primary language of your project to write those. Depending on the language you are using, doing this will mean significant more work upfront to lay out the foundations for your tests, but this work will pay off. You may also feel that the tests could be more verbose than if they were in shell, though that’s not necessarily the case.

March 19, 2018 · Tags: <a href="/tags/bazel">bazel</a>, <a href="/tags/google">google</a>, <a href="/tags/sandboxfs">sandboxfs</a>, <a href="/tags/software">software</a>
Continue reading (about 12 minutes)

Fighting execs via sandboxfs on macOS

Since the announcement of sandboxfs a few weeks ago, I’ve been stabilizing its integration with Bazel as a new sandboxing technique. As part of this work, I encountered issues when macOS was immediately killing signed binaries executed through the sandbox. Read on for the long troubleshooting process and the surprising trivial solution.

October 6, 2017 · Tags: <a href="/tags/sandboxfs">sandboxfs</a>, <a href="/tags/software">software</a>
Continue reading (about 10 minutes)

Introducing sandboxfs

sandboxfs is a FUSE-based file system that exposes an arbitrary view of the host’s file system under the mount point, and offers access controls that differ from those of the host. You can think of sandboxfs as an advanced version of bindfs (or mount --bind or mount_null(8) depending on your system) in which you can combine and nest directories under an arbitrary layout. The primary use case for this project is to provide a better file system sandboxing technique for the Bazel build system. The goal here is to run each build action (think compiler invocation) in a sandbox so that its inputs and outputs are tightly controlled, and sandboxfs attempts to do this in a more efficient manner than the current symlinks-based implementation.

August 25, 2017 · Tags: <a href="/tags/bazel">bazel</a>, <a href="/tags/pkg_comp">pkg_comp</a>, <a href="/tags/sandboxfs">sandboxfs</a>, <a href="/tags/software">software</a>, <a href="/tags/sourcachefs">sourcachefs</a>
Continue reading (about 2 minutes)