Pkg_comp - Julio Merino (jmmv.dev)

Hello, sandboxfs 0.1.0

I am pleased to announce that the first release of sandboxfs, 0.1.0, is finally here! You can download the sources and prebuilt binaries from the 0.1.0 release page and you can read the installation instructions for more details. The journey to this first release has been a long one. sandboxfs was first conceived over two years ago, was first announced in August 2017, showed its first promising results in April 2018, and has been undergoing a rewrite from Go to Rust. (And by the way, this has been my 20% project at Google so rest assured that they are still possible!)

February 5, 2019 · Tags: <a href="/tags/bazel">bazel</a>, <a href="/tags/featured">featured</a>, <a href="/tags/pkg_comp">pkg_comp</a>, <a href="/tags/sandboxctl">sandboxctl</a>, <a href="/tags/sandboxfs">sandboxfs</a>, <a href="/tags/software">software</a>
Continue reading (about 7 minutes)

Introducing sandboxfs

sandboxfs is a FUSE-based file system that exposes an arbitrary view of the host’s file system under the mount point, and offers access controls that differ from those of the host. You can think of sandboxfs as an advanced version of bindfs (or mount --bind or mount_null(8) depending on your system) in which you can combine and nest directories under an arbitrary layout. The primary use case for this project is to provide a better file system sandboxing technique for the Bazel build system. The goal here is to run each build action (think compiler invocation) in a sandbox so that its inputs and outputs are tightly controlled, and sandboxfs attempts to do this in a more efficient manner than the current symlinks-based implementation.

August 25, 2017 · Tags: <a href="/tags/bazel">bazel</a>, <a href="/tags/pkg_comp">pkg_comp</a>, <a href="/tags/sandboxfs">sandboxfs</a>, <a href="/tags/software">software</a>, <a href="/tags/sourcachefs">sourcachefs</a>
Continue reading (about 2 minutes)

Easy pkgsrc on macOS with pkg_comp 2.0

This is a tutorial to guide you through the shiny new pkg_comp 2.0 on macOS using the macOS-specific self-installer. Goals: to use pkg_comp 2.0 to build a binary repository of all the packages you are interested in; to keep the repository fresh on a daily basis; and to use that repository with pkgin to maintain your macOS system up-to-date and secure.

February 23, 2017 · Tags: <a href="/tags/macos">macos</a>, <a href="/tags/pkg_comp">pkg_comp</a>, <a href="/tags/sandboxctl">sandboxctl</a>, <a href="/tags/software">software</a>, <a href="/tags/tutorial">tutorial</a>
Continue reading (about 6 minutes)

Keeping NetBSD up-to-date with pkg_comp 2.0

This is a tutorial to guide you through the shiny new pkg_comp 2.0 on NetBSD. Goals: to use pkg_comp 2.0 to build a binary repository of all the packages you are interested in; to keep the repository fresh on a daily basis; and to use that repository with pkgin to maintain your NetBSD system up-to-date and secure.

February 18, 2017 · Tags: <a href="/tags/netbsd">netbsd</a>, <a href="/tags/pkg_comp">pkg_comp</a>, <a href="/tags/sandboxctl">sandboxctl</a>, <a href="/tags/software">software</a>, <a href="/tags/tutorial">tutorial</a>
Continue reading (about 6 minutes)

Introducing pkg_comp 2.0 (and sandboxctl 1.0)

Announcing the launch of pkg_comp 2.0, how this differs from the 1.x series, why there was a rewrite, what sandboxctl 1.0 is, and more.

February 17, 2017 · Tags: <a href="/tags/featured">featured</a>, <a href="/tags/pkg_comp">pkg_comp</a>, <a href="/tags/sandboxctl">sandboxctl</a>, <a href="/tags/software">software</a>
Continue reading (about 7 minutes)

Keeping pkgsrc packages up to date

drio asks in the suggestion box which is the best way to keep all the packages installed from pkgsrc up to date. I must confess that pkgsrc is quite weak in the updating area when compared to systems such as apt-get or yum. The problem comes from the fact that pkgsrc is a source-based packaging system, meaning that the end user builds packages by himself most of the times. Doing updates from such a system is hard because rebuilds take a long time and have high chances of breaking, leaving your system in an unusable status. Of course there is support for binary packages in pkgsrc, but we are not doing a good job in providing good sets of prebuilt binaries. Furthermore, and as drio stated, there is few documentation on the subject. The thing is there are several ways of updating all your installed packages. All of them are quite tedious and not "official", but with some work you can configure some scripts and cron jobs to automate the process as much as possible. Before doing an update, I usually start by running pkg_chk -u -n; this tells me which packages are out of date and which are their new versions. If the resulting list is short, I tend to follow the make replace procedure. This only works if the new versions of the packages are binary compatible with the old ones, something that you cannot guarantee by looking at the version numbers. For example, you can assume that if you have version X.Y.A from the libfoo library, the newer X.Y.B will be compatible to the old one. This is generally true but not always. Plus you need to have some knowledge of the dependency graph of your installed packages. Anyway, if you want to take the risk, simply go to the pkgsrc's directory for the outdated packages and run make replace in them. In most cases this works and is the fastest way to do minor updates. Things get worse when you have to update lots of stuff. The first and most obvious approach resorts to doing a clean reinstall. Start by issuing pkg_delete -r "*" followed by wiping /usr/pkg and /var/db/pkg. Then rebuild your packages. The problems with this approach are that it introduces a huge downtime in the system — until you have rebuilt everything (which can take a long time), the tools won't be available — and that any build failure can prevent you from reconfiguring your system soon enough. Another approach involves using different installation prefixes for the old and new installations. I used to do that when working on major GNOME updates. To do this set LOCALBASE to something like /usr/pkg-YYYYMMDD (similarly for PKG_SYSCONFBASE, VARBASE and PKG_DBDIR) where YYYYMMDD is the date when you started the installation of that specific set of packages. Then install your packages as usual and at last create a /usr/pkg symlink to point to the real directory. Do not change the date until you need to do major updates. When that time comes, change the date in your configuration to the current day; after that, pkgsrc will think that you don't have any packages installed so you can cleanly reinstall everything. Once you have finished installing all your packages again, update the /usr/pkg symlink to point to the new directory and remove the old one. Voila, minimum downtime and build failures cannot bother you. (However, you will need to migrate configuration files to the new tree, for example.) The last approach I can think of involves using pkg_comp. Use this tool to configure a sandbox in which you build binary packages for all the stuff you are interested in. You can even set up a cron job to do this rebuild weekly, for example, which is trivial using the tool's auto target. Once pkg_comp has generated a clean set of binary packages for you, you can proceed to update your real system with those packages. The way you proceed is up to you though. You can remove everything and do a clean reinstall (which should be a quick process anyway because you needn't rebuild anything!) or use pkg_add -u for the outdated packages. I think this is the safest way to proceed. Oh, I now notice that there is a pkg_rolling-replace utility that can also be used for updates. Dunno how it works though. Hope this makes any sense! Edit (22:15): Peter Bex refers us to the How to upgrade packages page in the unofficial NetBSD Wiki. It contains all these tricks plus many more, so it is worth to link it from here for completeness.

May 27, 2007 · Tags: <a href="/tags/pkg_comp">pkg_comp</a>, <a href="/tags/pkgsrc">pkgsrc</a>, <a href="/tags/update">update</a>
Continue reading (about 4 minutes)

Posts: Pkg_comp