There are, however, a few things that would deserve automation in this process, but that are not in build.sh's domain to solve. These are:
- Fetching and keeping the source tree up to date: interacting with CVS is still the responsibility of the user.
- Simplifying the call to build.sh: this script takes a ton of arguments, and running it by hand quickly becomes "annoying". It would be nice if all the desired values to its arguments where stored elsewhere and picked up automatically. (mk.conf may not always be the right choice.)
- Plugging into your crontab(5) for easy periodic rebuilds of NetBSD, with proper reporting of failures. Upgrading a live system from source is a major supported mechanism (particularly for NetBSD-current), so having a way to keep fresh release files around is welcome.
- Performing such daily rebuilds of NetBSD as a dedicated unprivileged user.
To anybody familiar with NetBSD, it is obvious that all the above items are "simple enough" to solve one by one; however, doing them by hand gets old very quickly — machines exist to make our life easier, don't they? I know I am not the only one that has custom local wrapper scripts around build.sh, and I also know that my scripts were ad-hoc, ugly and non-reusable. (Also, of course, anybody new to NetBSD will certainly not find any of the above trivial, but that's secondary.)
To this end, I have written a script that automates all the aforementioned steps in a single tool, driven by configuration files that specify what to do. Enter sysbuild.
My setup
These days, my main NetBSD development box is a virtual machine. For a couple of years, I have been carrying around this virtual machine across machines and slowly tuning its configuration to suit my needs. The way I work is the following:
I keep a read-write copy of the sources under my home directory, which is the one I use for my development work. This copy is accompanied by a script in my home directory that, given a machine type, updates the sources and rebuilds NetBSD for that machine type, using a known directory layout within my home directory.
I also keep a read-only copy of the sources under /usr/src, checked out from anoncvs. This copy is used by the dedicated "builder" system user to perform dedicated rebuilds of NetBSD. The "builder" user also has a custom script (but different than the other one!) that updates /usr/src, rebuilds NetBSD for the machine types I am interested in, records full logs to files and places the build results into a shared network drive. Lastly, the dedicated "builder" user has a cron job that runs the previous script overnight.
The purpose of this dual setup is to always have a fresh release build somewhere in the system, built from pristine sources, while at the same time allowing me to do my development work in a tree that I could break at any time.
Unfortunately, the size of the virtual disk of my development box has become too small for my current needs, and in order to fix this I prefer to start afresh. The thought of having to set up this whole scheme all over again, by hand, is what triggered the creation of sysbuild.
Getting started
sysbuild lives in pkgsrc under the pkgsrc/sysutils/sysbuild directory. This package provides the main script, tests, sample configuration files and the extensive manual page. Installing the script is as easy as installing any other package, and I would recommend reading its documentation now.
Once installed, sysbuild should be ready to use out of the box assuming that you have write access to /usr/src. Simply typing sysbuild build from anywhere in the system will cause the source tree to be updated and a release for your current platform to be built.
If you wish to customize the behavior of sysbuild, copy /usr/pkg/etc/sysbuild/default.conf to ~/.sysbuild/default.conf and edit the latter. The tool will pick it up on the next execution and use your new configuration settings. You can, of course, manage a variety of configurations in case you want to do different builds (say -current and 6.x).
Setting up a daily cron job
The pkgsrc/sysutils/sysbuild-user package is a convenience bundle that manages a "sysbuild" unprivileged user in the system and installs a sample crontab to perform the desired periodic rebuilds of NetBSD. This crontab uses the sysbuild4cron script, which is a very simple utility to run sysbuild, store its output in a file, and send an error report any time the execution fails.
Simply installing the package will cause the creation of the "sysbuild" user and the configuration of this sample cron job. Follow the instructions provided by the package in its MESSAGE file to tune the behavior of any of these.
Concluding trivia
Yes, these scripts are extremely tied to my particular workflow and use cases, although I don't think my needs are that special. However, I have attempted to come up with a generic-enough script and configuration file to allow the addition of new features.
You may notice that sysbuild's version is 2.0 and not 1.0. While preparing the script for addition to pkgsrc, cvs add barfed that I was re-adding previously-deleted files. Uh hum, what a surprise! Upon reading the CVS log of the package's Makefile, I found that 10 years ago I already wrote this exact same script and that two years after it got deleted because it stopped working. I had completely forgotten about this! However, seeing pkg_comp's messy code (which was written around the same time), I imagine that the previous implementation of this idea was messy; I don't dare to look at the previous code.
Enjoy!
Featured software
Featured posts
- Fast machines, slow machines
- EndBASIC 0.10: Core language, evolved
- Farewell, Microsoft; hello, Snowflake!
- Rust is hard, yes, but does it matter?
- Rust traits and dependency injection
- A year on Windows: Introduction
- Always be quitting
- How does Google keep build times low?
- How does Google avoid clean builds?
- Unit-testing a console app (a text editor)
- More...
