Pkgsrc - Julio Merino (jmmv.dev)

pkgdb belongs in libdata, not var

The pkgsrc package database, which by default lives in /var/db/pkg/, should not be there. Instead, it should be under /usr/pkg/libdata/pkgdb/. The same applies to FreeBSD’s and OpenBSD’s ports and also Debian’s dpkg, but I’ll focus on pkgsrc because it’s the system I know best. Let’s see why the current default is suboptimal and why libdata is a good alternative.

August 26, 2020 · Tags: <a href="/tags/netbsd">netbsd</a>, <a href="/tags/opinion">opinion</a>, <a href="/tags/pkgsrc">pkgsrc</a>
Continue reading (about 7 minutes)

Announcing etcutils 0.1

During past week, I worked on a new package called etcutils. It provides a (reduced) tool-set to programmatically manage files in /etc and is specially designed to allow pkgsrc to update /etc/shells and /etc/services in a more consistent way. I'm happy to say that the 0.1 release is now ready! Go to the etcutils web page for details. (I know that if you are a Linux user, you probably don't care about this because your distribution most likely already provides something similar... albeit more complex.) I'll now proceed to import this new package into pkgsrc as sysutils/etcutils. Later on (most likely not today), the following should happen: rework pkginstall to use the new shells(8) utility to update /etc/shells, add a new feature to pkgsrc to abstract the updates to /etc/services, and swipe through pkgsrc to make all packages touching this file use the new frameworky option. (Hey FAM, that includes you!)

May 7, 2010 · Tags: <a href="/tags/etcutils">etcutils</a>, <a href="/tags/pkgsrc">pkgsrc</a>
Continue reading (about 1 minute)

New gio-fam package

As briefly outlined in the previous post, new versions of Glib provide GIO, a library that intends to be a low-level file system API on top of the POSIX interface. This library provides an interface to asynchronously wait for file system change notifications including the creation, deletion and modification of files. The monitoring functionality in GIO is modular: it is backed by different loadable plugins that implement OS-specific functionality. In particular, GIO uses an inotify module in Linux and a FAM module everywhere else. Up until now, the devel/glib2 package in pkgsrc provided a build-time option to specify whether to build the GIO FAM plugin or not. Given that this plugin is built as a shared object that is loaded dynamically at run-time, having a build-time option for this is clearly wrong: it gives no choice to those relying on binary packages (e.g. end/new users). Furthermore, it adds a dependency on the ugly-FAM at the very bottom of the huge Gnome dependency chain. (As already stated, FAM is outdated and hard to set up.) So, based on this, I've just removed all FAM support from devel/glib2 altogether and packaged its loadable module as sysutils/gio-fam. Now waiting for a clean rebuild of the Gnome packages to see if the desktop now works on my machine by avoiding FAM/Gamin.

April 20, 2010 · Tags: <a href="/tags/gnome">gnome</a>, <a href="/tags/netbsd">netbsd</a>, <a href="/tags/pkgsrc">pkgsrc</a>
Continue reading (about 2 minutes)

NetBSD in Google Summer of Code 2010

For the 6th year in a row, NetBSD is a mentoring organization for Google Summer of Code 2010! If you are a bright student willing to develop full-time for an open source project during this coming summer, consider applying with us! You will have a chance to work with very smart people and, most likely, in the area that you are most passionate about. NetBSD, being an operating system project, has offers for project ideas at all levels: from the kernel to the packaging system, passing by drivers, networking tools, user-space utilities, the system installer, automation tools and more! I would like to point you at the 3 project proposals I'm willing to directly mentor: Optimize and speed-up ATF: Make the testing framework blazing fast so that running the NetBSD automated tests does not take ages on slow platforms. Reorganize ATF to improve modularity: Refactor pieces of the testing framework so that it is easier to redistribute, has cleaner interfaces and is easier to depend on from third-party projects. Rewrite pkg_comp with portability as a major goal: Use Python to create a tool to automatically build binary packages from within a sandbox.If you find any of the above projects interesting, or if you have any other project proposal that you think I could mentor, do not hesitate to contact me. Feel free to send me a draft of your application, together with a bit of information about you, so that we can discuss your proposal and make sure it gets selected! Or, if none of the projects above interests you, please do check out the full list of NetBSD project proposals. I'm sure you will find something that suits your interests :-)

March 19, 2010 · Tags: <a href="/tags/atf">atf</a>, <a href="/tags/netbsd">netbsd</a>, <a href="/tags/pkgsrc">pkgsrc</a>, <a href="/tags/soc">soc</a>
Continue reading (about 2 minutes)

New version of the monotone-server package in pkgsrc

Wow, it has been a long time... 5 years ago, I created the monotone-server package in pkgsrc, a package that provided an interactive script to set up a monotone server from scratch with, what I though, minimal hassle. My package did the job just fine, but past year I was blown away by the simplicity of the same package in Fedora: their init.d script provides a set of extra commands to initialize the server before starting it up, and that is it. No need to mess with a separate interactive script; no need to create and memorize passphrases that you will never use; and, what's more, all integrated in the only single place that makes sense: in the init.d "service management" script. It has been a while since I became jealous of their approach, but I've finally got to it: I've spent the last few days rewriting the monotone-server package in pkgsrc and came up with a similar scheme. And this new package just made its way to pkgsrc-HEAD! The new package comes with what I think is a detailed manual page that explains how to configure the server from scratch. Take a look and, if you find any mistakes, inconsistencies or improvements to be done, let me know! In the meantime, I will log into my home server, rebuild the updated package and put it in production :-)

March 12, 2010 · Tags: <a href="/tags/monotone">monotone</a>, <a href="/tags/netbsd">netbsd</a>, <a href="/tags/pkgsrc">pkgsrc</a>
Continue reading (about 2 minutes)

Extending sudo credentials

If you use sudo for, e.g. pkgsrc's just-in-time su, you may have often bitten by the problem that some compilations are slow and the build process stops right in the middle to ask you for a root password. If you go away while the system compiles, you'll be frustrated when you come back, as the process may still well be at the very beginning. This happens because, unless disabled by the system administrator, your sudo credentials last for 5 minutes. If you hadn't used sudo for those 5 minutes, it will ask you for your password again. A simple workaround for the problem is to automatically renew your credentials, say, every 2 minutes. You can do this by running the following command (from the same console you are using later on!) right before starting a pkgsrc build:$ ( while :; do sudo -v; sleep 120; done ) &

September 23, 2009 · Tags: <a href="/tags/pkgsrc">pkgsrc</a>
Continue reading (about 1 minute)

Keeping pkgsrc packages up to date

drio asks in the suggestion box which is the best way to keep all the packages installed from pkgsrc up to date. I must confess that pkgsrc is quite weak in the updating area when compared to systems such as apt-get or yum. The problem comes from the fact that pkgsrc is a source-based packaging system, meaning that the end user builds packages by himself most of the times. Doing updates from such a system is hard because rebuilds take a long time and have high chances of breaking, leaving your system in an unusable status. Of course there is support for binary packages in pkgsrc, but we are not doing a good job in providing good sets of prebuilt binaries. Furthermore, and as drio stated, there is few documentation on the subject. The thing is there are several ways of updating all your installed packages. All of them are quite tedious and not "official", but with some work you can configure some scripts and cron jobs to automate the process as much as possible. Before doing an update, I usually start by running pkg_chk -u -n; this tells me which packages are out of date and which are their new versions. If the resulting list is short, I tend to follow the make replace procedure. This only works if the new versions of the packages are binary compatible with the old ones, something that you cannot guarantee by looking at the version numbers. For example, you can assume that if you have version X.Y.A from the libfoo library, the newer X.Y.B will be compatible to the old one. This is generally true but not always. Plus you need to have some knowledge of the dependency graph of your installed packages. Anyway, if you want to take the risk, simply go to the pkgsrc's directory for the outdated packages and run make replace in them. In most cases this works and is the fastest way to do minor updates. Things get worse when you have to update lots of stuff. The first and most obvious approach resorts to doing a clean reinstall. Start by issuing pkg_delete -r "*" followed by wiping /usr/pkg and /var/db/pkg. Then rebuild your packages. The problems with this approach are that it introduces a huge downtime in the system — until you have rebuilt everything (which can take a long time), the tools won't be available — and that any build failure can prevent you from reconfiguring your system soon enough. Another approach involves using different installation prefixes for the old and new installations. I used to do that when working on major GNOME updates. To do this set LOCALBASE to something like /usr/pkg-YYYYMMDD (similarly for PKG_SYSCONFBASE, VARBASE and PKG_DBDIR) where YYYYMMDD is the date when you started the installation of that specific set of packages. Then install your packages as usual and at last create a /usr/pkg symlink to point to the real directory. Do not change the date until you need to do major updates. When that time comes, change the date in your configuration to the current day; after that, pkgsrc will think that you don't have any packages installed so you can cleanly reinstall everything. Once you have finished installing all your packages again, update the /usr/pkg symlink to point to the new directory and remove the old one. Voila, minimum downtime and build failures cannot bother you. (However, you will need to migrate configuration files to the new tree, for example.) The last approach I can think of involves using pkg_comp. Use this tool to configure a sandbox in which you build binary packages for all the stuff you are interested in. You can even set up a cron job to do this rebuild weekly, for example, which is trivial using the tool's auto target. Once pkg_comp has generated a clean set of binary packages for you, you can proceed to update your real system with those packages. The way you proceed is up to you though. You can remove everything and do a clean reinstall (which should be a quick process anyway because you needn't rebuild anything!) or use pkg_add -u for the outdated packages. I think this is the safest way to proceed. Oh, I now notice that there is a pkg_rolling-replace utility that can also be used for updates. Dunno how it works though. Hope this makes any sense! Edit (22:15): Peter Bex refers us to the How to upgrade packages page in the unofficial NetBSD Wiki. It contains all these tricks plus many more, so it is worth to link it from here for completeness.

May 27, 2007 · Tags: <a href="/tags/pkg_comp">pkg_comp</a>, <a href="/tags/pkgsrc">pkgsrc</a>, <a href="/tags/update">update</a>
Continue reading (about 4 minutes)

pkgsrcCon 2007 report

pkgsrcCon 2007 is over. The conference started around 1:00pm on the 27th and has lasted until today^Wyesterday (the 29th) at around 7:00pm. There have been 10 different talks as planned, although we weren't able to follow the proposed schedule. Most of the presentations were delayed and some were shifted because the speakers could not arrive on time. Not a big deal though. We have been, more or less, around 20-25 people. There were 30 registered, but a couple had to withdraw and some others did not come (for unknown reasons to us). Maybe the conferences were too technical or the schedule did not meet their expectations. The thing is I originally intended to write a short report after each day, but I have had literally no life outside pkgsrcCon since Thursday night. Taking care of the organization has been time consuming (and I have done almost nothing!), not to mention that recording all the presentations was exhausting; more on this below. Anyway, it has been an excellent experience. Meeting other pkgsrc developers in person has been very nice, plus they are all also very nice people too. Of course, their presentations were also interesting, and they showed extremely interesting ideas from their creators. I'm specially looking forward to seeing the preliminary results of one of the detailed projects (won't tell you which ;-). So, what did we do? We started with a pre-pkgsrcCon dinner on Thursday night, which was the first encounter among all developers. The talks started the day afterwards in the afternoon, and when they were over we had some beers and dinner together. Saturday was more of the same, even though it had more talks. We then had dinner on a nicer restaurant; great, great time. The last talks happened on Sunday, after which we had some more beers and the last dinner all together. I now have to compress all the video recordings and we have yet to decide if we will make them public and how. Stay tuned! (And don't hesitate to join for pkgsrcCon 2008; I'm sure you will enjoy it, at the very least as much as I did!)

April 29, 2007 · Tags: <a href="/tags/netbsd">netbsd</a>, <a href="/tags/pkgsrc">pkgsrc</a>, <a href="/tags/pkgsrccon">pkgsrccon</a>
Continue reading (about 2 minutes)

A letter to NetBSD

Dear NetBSD, It is almost five years since we first met and I still remember how much I liked you at that time. Despite your 1.5 release had slow disk performance when compared to the other BSDs, I found in you an operating system that just felt right. You focused on clean and well designed code among many other goals; sincerely, I didn’t come to you looking for portability because I never had anything else than i386 machines. All these feelings turned into love after installing and experimenting with you: the system was minimal, well documented and made sense. As you know, I soon left FreeBSD and migrated my machines to you.

September 23, 2006 · Tags: <a href="/tags/gnome">gnome</a>, <a href="/tags/netbsd">netbsd</a>, <a href="/tags/pkgsrc">pkgsrc</a>
Continue reading (about 5 minutes)

GNOME on NetBSD needs YOU!

A few pkgsrc developers and I have been working hard for years to bring the GNOME Desktop to this packaging system and make it work under NetBSD. We are quite happy with the current results because the packages are updated very frequently and everything works. Well, almost. There are still several missing details that really hurt the end user experience and need fixing. If things continue as have gone until now, we will always be one step (or more!) behind other operating systems such as Linux and FreeBSD. Linux, of course, gets full support from the GNOME developers because they do their daily work on it. FreeBSD, on the other hand, has more people working on the port and therefore have more manpower to resolve all the portability problems; they are doing a nice work. In our case, it is clear that we do not have enough manpower to keep up with the huge task that porting GNOME is — we are very few people working on it. Just consider that GNOME is composed of around 100 packages and that there are new major releases every 6 months (minor ones coming much faster). This updating task imposes a lot of stress on us that prevents us from working on the remaining pending items. If we were able to work on all these issues, we could have a fully-functional GNOME Desktop on top of NetBSD. I belive this is a key area to improve NetBSD's visibility: if we had a complete desktop evnrionment, more users could come and use NetBSD for their daily tasks. Eventually, this could attract more developers who would start contributing to the system itself. So... I've prepared a list of GNOME-related projects that details the major items that need to be addressed to have a complete GNOME Desktop installation on top of NetBSD and pkgsrc. I've tried to detail each project as much as possible, explaining the current problem, why solving it could benefit NetBSD and how to get started. Should you need more information, I've also written some generic guidelines about GNOME packaging and porting. And, of course, you can contact me to get more details and take one of the projects! I'm willing to mentor you to make them success. You can certainly make a difference to the current status of things. Let me add that I've have learned a lot about many different areas from my contributions to pkgsrc and NetBSD. You can seize the opportunity to learn new exciting stuff too; don't be shy! Oh... and if GNOME on NetBSD is not of your preference, please see our complete list of proposed projects; I bet you will find something of your interest!

September 1, 2006 · Tags: <a href="/tags/gnome">gnome</a>, <a href="/tags/netbsd">netbsd</a>, <a href="/tags/pkgsrc">pkgsrc</a>
Continue reading (about 3 minutes)

Posts: Pkgsrc