pkgsrc Is Cool

Published: 2021-12-23 22:17:43 -0800 PST

Last updated: 2022-04-24 21:06:24 -0700 PDT

I think I saw something about pkgsrc pop up randomly in my
Mastodon feed a few months ago. Before that, I had never heard of
it. And now…well now I’m currently writing these words in a version
of Emacs that I built from source with pkgsrc.

Pkgsrc is NetBSD’s package manager, but thanks to NetBSD’s portability
philosophy, pkgsrc is fully supported as a package manager on other
Unix-like operating systems like Linux, Mac, and Solaris. It supports
installing binary packages, but the reason I was interested in it is
that pkgsrc supports building packages from source as well.

For some context on other similar projects, pkgsrc was originally
forked from FreeBSD’s Ports collection decages ago, so for people
familiar with the other BSD’s, this is a ports collection that works
on other platforms. Even better, pkgsrc and NetBSD get the terminology
right and correctly call these things packages instead of ports (and
leave “port” as the term for cpu architecture support). Other
similar-ish projects include MacPorts, Gentoo’s emerge, GNU Guix, and
Ravenports. I probably missed some, but you get the point. There are
multiple options out there if you’re interested. I’m not going to
compare/contrast them because honestly the only other one I’ve ever
used was MacPorts like 10 years ago when I didn’t know what I was
doing.

Why did I use it?

I’ve always disliked needing to install a new package to get some
program to run. The main reason why is that the new package could pull
in other packages and they all end up polluting my system folders like
/bin and /lib with extra stuff. Half the time I delete the program
I was testing out anyway, but then I’m stuck with this extra stuff in
my base system. I never really even stopped to think that there might
be way to separate out the “base system” from the other crap I’ve
installed along the way.

But this is exactly what NetBSD does. NetBSD’s base system for /bin
and such is really only for the core NetBSD system code like the stuff
you can build from their source code repo. Anything else is supposed
to be installed somewhere else. By default, pkgsrc will install stuff
into /usr/pkg where it created its own subdirectory tree of
/usr/pkg/bin and /usr/pkg/lib and so on. This way that specfic
pkg directory is like a self-contained userland separate from the
base operating system. I’m actually using pkgsrc on Debian rather than
NetBSD itself, but it works the same way.

Even better, pkgsrc lets you use it in unprivileged mode and install
stuff locally in your user’s home directory like ~/pkg. This can
give your user their own separate custom userland from other
users. Depending on your use case this may not be what you want, but I
found it particularly cool that I could run almost a completely
separate system just for my user. Because I didn’t want to need
sudo/doas to install packages, I am actually using pkgsrc in
unprivileged mode with ~/pkg as my base directory.

So that was a long explanation to this answer. Why did I start using
pkgsrc? Because I can install packages locally in my home directory, I
can build those packages from source, and I know this system will work
on any operating system I will ever run into (and I’ve used some weird
ones considering how relatively young I am).

How it works

Pkgsrc isn’t really a package repository itself. It is simply a
collection of Makefiles that define how to build each package as well
as the dependencies needed for a package. Each package’s Makefile
defines where to download the source code for the package, how to
apply any necessary patches, and then how to build it. This actually
answered a long-standing question for me: why do GitHub “releases”
build a tarball/zip snapshot of the source code? Well, systems like
pkgsrc will use that tarball to download the source code for a
specific version without needing to clone a full repository (or even
needing the source control software installed at all).

Pkgsrc reaches out to various websites like GitHub and Sourceforge (or
wherever else a project’s source code is hosted) to download the
specified version of a package to build it. This is highly
decentralized because pkgsrc does not rely on a single repository
server, instead relying on each project’s own servers. (As a random
side note, this is similar to how the Go programming language handles
dependencies). This could lead to the potential problem where a
single project’s website is down so you can’t build their
software. However, the Makefiles also specify alternative distfile
mirrors for where to download the source code. New specific
terminology for me here was a “distfile” which is that source code
tarball snapshot for a specific version of the package.

Tinfoil Hat Time

One potentially “irrational” requirement of mine was that I wanted to
be able to install software from my own network in case the internet
ever goes down. Not just my internet, but THE internet. And I know
that the internet as whole will probably never really go down, but I
do think it may get to the point where the internet is fragmented into
completely different silos because of government censorship.

In a more practical manner, I also wanted to be able to install
software from my local network to save bandwidth when updating
multiple computers. This is why I host some mirrors on my server.

One of those mirrors is the pkgsrc distfile mirror. I was able to use
pkgsrc with the MASTER_SITE_OVERRIDE option to tell it to use my own
distfile mirror. This allows me to build all of the supported pkgsrc
packages offline because the distfile mirror has already reached out
and downloaded all of the source code for every project (or rather,
I’m mirroring from a mirror that has already done that).

I realize this is a weird almost “prepper’ doomsday mentality, but I
really wanted to be able to install software locally on my own
network. Pkgsrc gave me the solution to be able to build the software
from source for any operating system on any cpu architecture that I
could find and do it all from my own server.

More Articles

After searching for more information on pkgsrc, I found these other cool links:

Followup

To keep this from ballooning into one gigantic article, I’ll keep this
one as simply why I think pkgsrc is cool and why I wanted to use
it. I’ll follow up with another article about how exactly I’m using
it.

I should probably also write another article about using Clang as the
default pkgsrc compiler since I messed around with that until I
finally got it to work. Their documentation about it is a bit
scattered and misleading.

I did end up writing more posts about pkgsrc!

Git – Mastodon

www.wisellama.rocks by Wisellama is marked with CC0 1.0