Blog

Hands-on graphics without X11

If you have been following the development of EndBASIC, you know its console can display both text and graphics at once. What you may not know is that, now, it can also achieve this feat on the NetBSD console without using X11 at all. This is done by directly rendering to the wsdisplay framebuffer, and this article presents a crash course on direct graphics and keyboard access via NetBSD’s wscons framework.

January 17, 2025 · Tags: blogsystem5, endbasic, netbsd
Continue reading (about 13 minutes)

Self-documenting Makefiles

Make, as arcane as a build tool can be, may still be a good first fit for certain scenarios. “Heresy!”, you say, as you hear a so-called “Bazel expert” utter these words.

The specific problem I’m facing is that I need to glue together the NetBSD build system, a quilt patch set, EndBASIC’s Cargo-based Rust build, and a couple of QEMU invocations to produce a Frankenstein disk image for a Raspberry Pi. And the thing is: Make allows doing this sort of stitching with relative ease. Sure, Make is not the best option because the overall build performance is “meh” and because incremental builds are almost-impossible to get right… but adopting Bazel for this project would be an almost-infinite time sink.

Anyway. When using Make in this manner, you often end up with what’s essentially a “command dispatcher” and, over time, the number of commands grows and it’s hard to make sense of which one to use for what. Sure, you can write a README.md with instructions, but I guarantee you that the text will get out of sync faster than you can read this article. There is a better way, though.

January 10, 2025 · Tags: blogsystem5, unix
Continue reading (about 7 minutes)

Revisiting the NetBSD build system

I recently picked up an embedded project in which I needed to build a highly customized full system image with minimal boot times. As I explored my options, I came to the conclusion that NetBSD, the often-forgotten BSD variant, was the best viable choice for my project.

One reason for this choice is NetBSD’s build system. Once you look and get past the fact that it feels frozen in time since 2002, you realize it is still one of the most advanced build systems you can find for an OS. And it shows: the NetBSD build system allows you to build the full OS from scratch, on pretty much any host POSIX platform, while targeting any hardware architecture supported by NetBSD. All without root privileges.

Another reason for this choice is that NetBSD was my daily workhorse for many years and I’m quite familiar with its internals, which is useful knowledge to quickly achieve the goals I have in mind. In fact, I was a NetBSD Developer with capital D: I had commit access to the project from about 2002 through 2012 or so, and I have just revived my account in service of this project. jmmv@ is back!

So, strap onto your seats and let’s see how today’s NetBSD build system looks like and what makes it special. I’ll add my own critique at the end, because it ain’t perfect, but overall it continues to deliver on its design goals set in the late 1990s.

December 28, 2024 · Tags: blogsystem5, netbsd
Continue reading (about 16 minutes)

Synology DS923+ vs. FreeBSD w/ZFS

My interest in storage is longstanding—I loved playing with different file systems in my early Unix days and then I worked on Google’s and Microsoft’s distributed storage solutions—and, about four years ago, I started running a home-grown NAS leveraging FreeBSD and its excellent ZFS support. I first hosted the server on a PowerMac G5 and then upgraded it to an overkill 72-core ThinkStation that I snapped second-hand for a great price.

But as stable and low maintenance as FreeBSD is, running day-to-day services myself is not my idea of “fun”. This drove me to replace this machine’s routing functionality with a dedicated pfSense box a year ago and, for similar reasons, I have been curious about dedicated NAS solutions.

I was pretty close to buying a second-hand NAS from the work classifieds channel when a Synology marketing person (hi Kyle!) contacted me to offer a partnership: they’d ship me one of their devices for free in exchange for me publishing a few articles about it. Given my interest to drive-test one of these appliances without committing to buying one (they ain’t cheap and I wasn’t convinced I wanted to get rid of my FreeBSD-based solution), I was game.

And you guessed right: this article is one of those I promised to write but, before you stop reading, the answer is no. This post is not sponsored by Synology and has not been reviewed nor approved by them. The content here, including any opinions, are purely my own. And what I want do do here is compare how the Synology appliance stacks against my home-built FreeBSD server.

December 13, 2024 · Tags: blogsystem5, hardware
Continue reading (about 17 minutes)

Demystifying secure NFS

I recently got a Synology DS923+ for evaluation purposes which led me to setting up NFSv4 with Kerberos. I had done this about a year ago with FreeBSD as the host, and going through this process once again reminded me of how painful it is to secure an NFS connection.

You see, Samba is much easier to set up, but because NFS is the native file sharing protocol of Unix systems, I felt compelled to use it instead. However, if you opt for NFSv3 (the “easy default”), you are left with a system that has zero security: traffic travels unencrypted and unsigned, and the server trusts the client when the client asserts who is who. Madness for today’s standards. Yet, when you look around, people say “oh, but NFSv3 is fine if you trust the network!” But seriously, who trusts the network in this day and age?

You have to turn to NFSv4 and combine it with Kerberos for a secure file sharing option. And let me tell you: the experience of setting these up and getting things to work is horrible, and the documentation out there is terrible. Most documents are operating-system specific so they only tell you what works when a specific server and a specific client talk to each other. Other documents just assume, and thus omit, various important details of the configuration.

So. This article is my recollection of “lab notes” on how to set this whole thing up along with the necessary background to understand NFSv4 and Kerberos. My specific setup involes the Synology DS923+ as the NFSv4 server; Fedora, Debian, and FreeBSD clients; and the supporting KDC on a pfSense (or FreeBSD) box.

November 3, 2024 · Tags: blogsystem5, unix
Continue reading (about 24 minutes)

BazelCon 2024 recap

Just like that, BazelCon 2024 came and went. So… it’s obviously time to summarize the two events of last week: BazelCon 2024 and the adjacent Build Meetup. There is A LOT to cover, but everything is here in just one article!

October 22, 2024 · Tags: bazel, blogsystem5, snowflake
Continue reading (about 42 minutes)

The costs of the i386 to x86-64 upgrade

If you read my previous article on DOS memory models, you may have dismissed everything I wrote as “legacy cruft from the 1990s that nobody cares about any longer”. After all, computers have evolved from sporting 8-bit processors to 64-bit processors and, on the way, the amount of memory that these computers can leverage has grown orders of magnitude: the 8086, a 16-bit machine with a 20-bit address space, could only use 1MB of memory while today’s 64-bit machines can theoretically access 16EB.

All of this growth has been in service of ever-growing programs. But… even if programs are now more sophisticated than they were before, do they all really require access to a 64-bit address space? Has the growth from 8 to 64 bits been a net positive in performance terms?

Let’s try to answer those questions to find some very surprising answers. But first, some theory.

October 7, 2024 · Tags: blogsystem5, hardware, unix
Continue reading (about 18 minutes)

Revisiting the DOS memory models

At the beginning of the year, I wrote a bunch of articles on the various tricks DOS played to overcome the tight memory limits of x86's real mode. There was one question that came up and remained unanswered: what were the various models that the compilers of the day offered?

Tiny, small, medium, compact, large, huge... What did these options mean? What were their effects? And, more importantly... is any of that legacy relevant today in the world of 64-bit machines and gigabytes of RAM? To answer those questions, we must start with a brief review of the 8086 architecture and the binary formats supported by DOS.

September 30, 2024 · Tags: blogsystem5, dos
Continue reading (about 10 minutes)

Windows NT vs. Unix: A design comparison

Over the years, I’ve repeatedly heard that Windows NT is a very advanced operating system and, being a Unix person myself, it has bothered me to not know why. I’ve been meaning to answer this question for years and I can do so now, which means I want to present you my findings.

My desire to know about NT’s internals started in 2006 when I applied to the Google Summer of Code program to develop Boost.Process. I needed such a library for ATF, but I also saw the project as a chance to learn something about the Win32 API. This journey then continued in 2020 with me choosing to join Microsoft after a long stint at Google and me buying the Windows Internals 5th edition book in 2021 (which I never fully read due to its incredible detail and length). None of these made me learn what I wanted though: the ways in which NT fundamentally differs from Unix, if at all.

September 9, 2024 · Tags: blogsystem5, unix, windows
Continue reading (about 23 minutes)

Picking glibc versions at runtime

In a recent work discussion, I came across an argument that didn’t sound quite right. The claim was that we needed to set up containers in our developer machines in order to run tests against a modern glibc. The justifications were that using LD_LIBRARY_PATH to load a different glibc didn’t work and statically linking glibc wasn’t possible either.

But… running a program against a version of glibc that’s different from the one installed on the system seems like a pretty standard requirement, doesn’t it? Consider this: how do the developers of glibc test their changes? glibc has existed for much longer than containers have. And before containers existed, they surely weren’t testing glibc changes by installing modified versions of the library over the system-wide one and YOLOing it.

August 11, 2024 · Tags: blogsystem5, programming, unix
Continue reading (about 11 minutes)