How I learned to stop worrying and love the powderkeg. #FreeBSD


FreeBSD has grown up a lot in this release cycle.  The most useful tool from the 10.0/11.0 world in a long time, poudriere (powder keg in French) has made my ports usage almost trivial now.

More or less, poudriere is a tool that allows you to build ports packages compatible with the new PKGNG format without contaminating your working system.  It uses a series of jails and build environments to do what a lot of more savvy FreeBSD developers and engineers have been doing for years.

Even using portmaster to maintain my systems seems archaic in comparison, not to mention error prone.  More or less, my 3 or 4 systems have been converted to use themselves as a repository for packages and they build their own packages.  This is a bit redundant to be honest, and it makes the most sense to use one host as a repository and have your other machines pull in packages from it.  My implementation is due to running 11-current and being having machines I control on very different and restrictive networks.

poudriere setup for 11-current (head builds)

Start by install poudriere from ports or a package that you can get your hands on.  Then command poudriere to setup its basejail on FreeBSD SVN HEAD:

poudriere jail -c -j 11-amd64 -v head -a amd64 -m svn

This will create a jail on your local machine based on SVN head at the time of execution (yes, its going to compile everything from source and will take a while, get a cup of coffee, perhaps a sandwich).  The thing is, your machine is still available for other things while this is going on.  You are not going to crash X or other applications while this is happening.  Its building a separate jail for the purpose of creating packages.

Once its build, you can update your jail world trivially via:

poudriere jail -u -j 11-amd64

Now, grab the ports tree via:

poudriere ports -c

Updates to your ports tree via portsnap are easy with a :

poudriere ports -u

At this point, you are ready to configure poudriere to build your package via the “bulk” command.  I copied /usr/local/etc/poudriere.conf.sample to /usr/local/etc/poudriere.conf and made exactly one change to the default settings.  I use ZFS ( which I highly recommend, see my post on the Bacon of Filesystems ) and my ZPOOL is a different name than the default.

Creating your list of ports for your builds is a trial and error endeavor to be honest.  I suspect, there are easier ways to do it, but I determined my list below based on the list I had installed already and some questions to various mailing lists.  I created a /usr/local/etc/myports file with the following in it as a list of ports that I want built.  Poudriere will build all required dependencies for me, build-time and run-time and create nice little packages for me.

x11/xorg
x11/xdm
x11/xsm
x11-wm/xfce4
x11/xfce4-screenshooter-plugin
x11/xscreensaver
shells/bash
www/firefox
www/linux-f10-flashplugin11
www/nspluginwrapper
graphics/evince
net-im/finch
editors/vim
sysutils/tmux
comms/amtterm
ports-mgmt/dialog4ports
ports-mgmt/pkg
ports-mgmt/poudriere
java/openjdk7
editors/vim-lite
sysutils/synergy-devel
devel/git
emulators/qemu-devel

At this point, I was read to do the build run via:

poudriere bulk -f /usr/local/etc/myports -j 11-amd64

This builds all the things for me, caching packages when needed for reuse.  Very handy for me to be honest.

Setting up the pkg repo couldn’t be simpler either.  I copied /usr/local/etc/pkg.conf.sample to /usr/local/etc/pkg.conf and made a single change to point the system to use the locally build packages in a locally generated repo:

PACKAGESITE        : file:///usr/local/poudriere/data/packages/11-amd64-default

The final step was to initialize my repository via:

pkg repo /usr/local/poudriere/data/packages/11-amd64-default

I then updated my system via the newly built packages:

pkg update

pkg upgrade -f

This refreshed all the packages on my system with ones that are cleanly built by poudriere.  This allowed me to now audit what I had installed and to see what I could remove or what else I needed to have built:

pkg version -R

Anything with a “=” means that it comes from the repository and is up to date.  Anything with a “?” means it comes from an unknown source.   I learned I had a lot of dependencies installed for builds that I didn’t need for runtime cases:

pkg autoremove

Many, many, many thanks to the FreeBSD portmgr team (portmgr@freebsd.org), Baptiste Daroussin, Bryan Drewery and the others who have deadlifted the FreeBSD ports system into the future. Now I can look at whats left and I have never been more content with FreeBSD ports.  *boom*

*edit* reference to poudriere official docs and such:

https://fossil.etoilebsd.net/poudriere/doc/trunk/doc/index.wiki

*edit* after pkg-1.2.1 release.

The pkg.conf config and locations have moved around and become incompatible with this blog post.  You’ll want to do two things if you are using this as a guide for updates:

1.  Disable the FreeBSD repo configuration in /etc/pkg/FreeBSD.conf

2. Move your local repo config to /etc/pkg/my_repo.conf and give it the following syntax:

me: {
url: file:///usr/local/poudriere/data/packages/11-amd64-default,
signature_type: none,
enabled: yes
}


5 responses to “How I learned to stop worrying and love the powderkeg. #FreeBSD”

  1. One thing you have either missed off for brevity or else might like to know about is options.
    For me one of the reasons for building my own packages is using non default options to ports, this is handled by
    poudriere options -j 11-amd64 www/firefox
    which effectively run a “make config-recursive” and will store the options under /usr/local/etc/poudriere.d/11-amd64-options (different jailscan have different option directories or I symlink them so i have consistent options)
    If you already have port options saved you can pre-populate this directory from /var/db/ports

    • Please keep in mind that populating the options as Vince detailed above is dangerous. Poudriere will never attempt to tell you if the default options have changed or if options you have set have been renamed or removed. The right thing to do is carefully extract which options you have changed and put them in your poudriere make.conf for easy review and adjustment later.

  2. two corrections to my previous post:

    1. don’t edit /usr/local/poudriere/jails/92amd64/etc/make.conf directly, instead edit /usr/local/etc/poudriere.d/92amd64-make.conf

    2. the syntax for the options is editors_libreoffice_UNSET+=CUPS not editors_libreoffice_OPTIONS_FILE_UNSET+=CUPS

  3. BTW; with regards to finding your initial list of ports to build based on a working system, one (relatively) easy way is with portmaster itself.

    # portmaster –list-origins | sort

    Fair warning: you probably want to do a
    # pkg autoremove
    .. first to get rid of the build time dependencies that may be laying around. If you don’t do this, your –list-origins output will be huge as each not-needed build tool will be a “leaf”.