Showing 20 posts
During my 11 years at Google, I can confidently count the number of times I had to do a “clean build” with one hand: their build system is so robust that incremental builds always work. Phrases like “clean everything and try building from scratch” are unheard of. So… you can color me skeptical when someone says that incremental build problems are due to bugs in the build files and not due to a suboptimal build system. The answer lies in having a robust build system, and in this post I’ll examine the common causes behind incremental build breakages, what the build system can do to avoid them, and how Bazel accomplishes most of them.
Now that you know about my recent departure from Google, it is time to look into how the whole interviewing process went with all of Facebook, Twitter, and Microsoft. Read on for details on engineering levels, the timeline of all three processes, and some key observations of each.
After a little over 11 years, it’s time for a much longed change: I’m leaving Google and I’m joining Microsoft as a Principal Software Engineer for Azure. These job changes are effective as of this week, but my family and I already moved from New York City to Redmond, WA about three weeks ago. Read on for a recap on my tenure at Google, the whys behind my departure, and how I ended up choosing the position in Microsoft Azure after mulling over offers from Facebook, Twitter, and Microsoft.
[This post in Spanish to target the Spanish student audience.] La semana pasada nos despedimos de un estudiante de tercer año de carrera que pasó el verano con nosotros haciendo una internship en el equipo de Bazel. Este hecho me hizo pensar en que es el mejor momento de repasar qué son los internships. Mi objetivo es intentar convencerte, si eres un estudiante de carrera, máster, o doctorado, de que son una opción muy interesante y asequible para mejorar tus conocimientos y crecer en el mundo laboral.
Blaze—the variant of Bazel used internally at Google—was originally designed to build the Google monorepo. One of the beauties of sticking to a monorepo is code reuse, but this has the unfortunate side-effect of dependency bloat. As a result, Bazel and Blaze have evolved to support ever-increasingly-bigger pieces of software. The growth of the projects built by Bazel and Blaze has had the unsurprising consequence that our engineers all now have high-end workstations with access to massive amounts of distributed resources.
During the summer of last year, I hosted an intern who implemented sandboxfs: a FUSE-based file system that exposes an arbitrary view of the host’s file system under the mount point. At the end of his internship, we had a functional sandboxfs implementation and some draft patches for integration in Bazel. The goal of sandboxfs in the context of Bazel is to improve the performance of builds when action sandboxing is enabled.
This post is a short, generalized summary of the preceeding two. I believe those two posts put readers off due to their massive length and the fact that they were seemingly tied to Bazel and Java, thus failing to communicate the larger point I wanted to make. Let’s try to distill their key points here in a language- and project-agnostic manner.
In part 1 of this series, I made the case that you should run away from the shell when writing integration tests for your software and that you should embrace the primary language of your project to write those. Depending on the language you are using, doing this will mean significant more work upfront to lay out the foundations for your tests, but this work will pay off. You may also feel that the tests could be more verbose than if they were in shell, though that’s not necessarily the case.
My latest developer productivity rant thesis is that integration tests should be written in the exact same language as the thing they test. Specifically, not shell. This theory applies mostly to tests that verify infrastructure software like servers or command line tools. It is too easy to fall into the trap of using the shell because it feels like the natural choice to interact with tools. But I argue that this is a big mistake that hurts the long-term health of the project, and once trapped, it’s hard to escape.
In a version control system, a rollback is a type of change that undoes the effects of a previous commit. In essence, a rollback is a commit that applies the inverse diff of another commit. At Google, our tools make it trivial to create rollbacks for a given changelist or CL. (A CL is similar to a commit but can be either pending—in review—or submitted.) Making it trivial to create rollback CLs is important in a culture where the standard upon encountering a problem is “rollback first, ask questions later” because it removes friction from the process of backing out problematic changes.
It has been over 6 years since I joined Google and throughout this time I have been in the Storage SRE family: first with GFS, then with Colossus, and last with Persistent Disk. Even though this counts as 3 different teams, the reality is that I have been doing mostly the same type of work all around. I had pondered the idea of switching to a pure Software Engineer (SWE) role for all these years and never taken any action. Until now. Things change, and the time has come for me to make a move and pursue that thought in an effort to grow in a different direction. And why now, you ask? Well, simply because I have found a role in the NYC office for a project that I am personally passionate about.
Mission: Site Reliability Engineer for the Storage Infrastructure at Google D-Day: May 25th, 2009 Location: Dublin, Ireland Duration: Unspecified Six years have passed. Six years since I dropped out of a Ph.D. program, left home, and took a plane to Dublin, Ireland, to start my work life adventure by joining Google. Two years later, I moved to New York City and I am still here without any specific plans to leave.
This is a rare post because I don’t usually talk about Google stuff here, and this post is about Bazel: a tool recently published by Google. Why? Because I love its internal counterpart, Blaze, and believe that Bazel has the potential to be one of the best build tools if it is not already. However, Bazel currently has some shortcomings to cater to a certain kind of important projects in the open source ecosystem: the projects that form the foundation of open source operating systems.
One of the things that often shocks new engineers at Google is the fact that every change to the source tree must be reviewed before commit. It is hard to internalize such a workflow if you have never been exposed to it, but given enough time —O(weeks) is my estimation—, the formal pre-commit code review process becomes a habit and, soon after, something you take for granted. To me, code reviews have become invaluable and, actually, I feel “naked” when I work on open source projects where this process is not standard practice.
If you use a native IM client to connect to Google Talk and also have a Google+ account, you probably have noticed by now that your contacts list is polluted by tons of addresses of the form firstname.lastname@example.org. Attempting to talk to these people from the native IM client does not work (maybe it does in some specific situations? I don't know.), so these contacts only add noise and annoyance.
I've been holding back this announcement until all affected parties knew in advance. They do know now, so I'm happy to announce that I'll be joining Google Dublin on May 25th as a Google.com Software Engineer! Thanks to everyone who made that possible.
The Google SoC 2008 Mentor Summit is now officially over. The summit has taken place during the whole weekend and has been pretty intensive. The organization of the whole event has been excellent thanks to the hard work of Leslie Hawthorn among others; sorry, can't remember your names... I'm very bad at this. We have had multiple sessions, ranging from technical ones such as distributed version control systems to more political ones such as how to deal with assholes in open source projects.
I've landed this morning in San Francisco at 9.00am (which means I left NYC at 6.00am!) and went straight down to the Google Headquarters in Mountain View. No sleep at all except for a little bit of pseudo-sleep in the plane. The Google campus is really nice. It puts the NYC offices in an inferior level than I thought :P But the only problem is that the area surrounding the campus is basically empty.
Don't know why but I finally succumbed to Twitter today, as if I did not have enough things to waste time. You can follow me with the jmmv nick. I just noticed this post comes after more than a month since the last one; my apologies. I do not have any free time these days to think about writing decent posts nor doing anything else. My current work is basically attending class, writing reports, reading papers and going to the gym (this last thing only when possible).