{"id":247,"date":"2017-02-08T00:01:02","date_gmt":"2017-02-08T00:01:02","guid":{"rendered":"http:\/\/blog.ignoranthack.me\/?p=247"},"modified":"2017-02-08T00:01:02","modified_gmt":"2017-02-08T00:01:02","slug":"one-pxeboot-to-find-them-and-in-the-darkness-bind-them","status":"publish","type":"post","link":"http:\/\/blog.ignoranthack.me\/?p=247","title":{"rendered":"One pxeboot to find them and in the darkness bind them"},"content":{"rendered":"

Irritatingly, I couldn’t find a decent cut-n-paste how to on the Internet for using a much loved and mostly overlooked FreeBSD installation and boot up process using a fully cooked diskless image fetched via TFTP instead of the really well documented and fantastic howto on NFS booting<\/a>.<\/p>\n

Primarily, I wanted to switch off of NFS booting as it tends to be really troublesome if one is doing development or debugging of the FreeBSD network stack or Ethernet drivers. I couldn’t reliably screw around with the network interfaces without disturbing the NFS root mount and inflicting relatively long timeouts on my tests. I decided to try and switch to MFS root based testing so that I didn’t run into these types of problems, however I didn’t realize that the boot loader had gone through some overhauls recently and the documentation that *is* floating around on the Internet, is a bit out of date. So, here is the beginning of what I hope will be the inception of a new FreeBSD handbook entry before version 12 is released.<\/p>\n

I want to cover the actual construction of the entire process, so I will not be hand copying binaries from a release and all files will be constructed from source where possible.\u00a0 This means that I can fully automate a build and regression from source to ensure that this document stays relatively up to date.\u00a0 I’m an active FreeBSD contributor, so I have many checkouts of FreeBSD lying around.\u00a0 If you’re a sysadmin, you probably use \/usr\/src for most of the projects like this, which is fine for our purposes.<\/p>\n

Assuming that you have a fresh checkout of FreeBSD somewhere, lets start by crafting a “src.conf” and a “make.conf” to minimize a bit of build time and a bit of space.<\/p>\n

I turn a lot of stuff off in the build here.\u00a0 Since I’m running the same major revision of FreeBSD that I’ll be building, I can get away with some shenanigans regarding not building toolchains.\u00a0 If you are trying to build 12-CURRENT on 11-STABLE, you may have to play around with the knobs a bit in src.conf:<\/p>\n

WITHOUT_BLUETOOTH=”YES”
\nWITHOUT_BSNMP=”YES”
\nWITHOUT_CAPSICUM=”YES”
\nWITHOUT_CDDL=”YES”
\nWITHOUT_COMPAT32=”YES”
\nWITHOUT_CTM=”YES”
\nWITHOUT_DEBUG_FILES=”YES”
\nWITHOUT_EXAMPLES=”YES
\nWITHOUT_FLOPPY=”YES”
\nWITHOUT_FREEBSD_UPDATE=”YES”
\nWITHOUT_GAMES=”YES”
\nWITHOUT_ICONV=”YES”
\nWITHOUT_JAIL=”YES”
\nWITHOUT_LIB32=”YES”
\nWITHOUT_LOCALES=”YES”
\nWITHOUT_LPR=”YES”
\nWITHOUT_MAKE=”YES”
\nWITHOUT_MAN=”YES”
\nWITHOUT_PORTSNAP=”YES”
\nWITHOUT_TESTS=”YES”
\nWITHOUT_PC_SYSINSTALL=”YES”
\nWITHOUT_RESCUE=”YES”
\nWITHOUT_SVNLITE=”YES”
\nWITHOUT_TEXTPROC=”YES”
\nWITHOUT_TOOLCHAIN=”YES”
\nWITHOUT_WIRELESS=”YES”
\nWITHOUT_KERBEROS=”YES”
\nWITHOUT_MAIL=”YES”
\nWTTHOUT_EFI=”YES”
\nWITHOUT_NTP=”YES”<\/p>\n

My make.conf is much shorter, and only set’s a few things to make life easier:<\/p>\n

BOOT_PXELDR_ALWAYS_SERIAL=y
\nNO_MODULES=y
\nMALLOC_PRODUCTION=y<\/p>\n

Set\/export a couple of variables to your local shell to use these:<\/p>\n

setenv SRCCONF \/path\/to\/src.conf
\nsetenv __MAKE_CONF \/path\/to\/make.conf<\/p>\n

Since I’m not building as root here, I set one more variable:<\/p>\n

setenv MAKEOBJDIRPREFIX \/var\/tmp\/mfsroot<\/p>\n

Execute a buildworld and turn some electrons into heat for a while.\u00a0 You’ll probably note a big difference in build times here.\u00a0 Also note that you’re building a much smaller FreeBSD system with a lot of missing pieces.\u00a0 No local mail agent, no ntp services, no compilers\/debuggers, no wireless bits … etc.\u00a0 Your mileage may vary here, but I needed a small world as I’m debugging\/testing kernel\/network drivers, not userland.<\/p>\n

You can make a choice here of kernels to build.\u00a0 Most folks will want GENERIC-NODEBUG, but if you want, you can base a customized kernel configuration from MINIMAL that will do the trick.\u00a0 You’ll need to include drivers specific to your systems and this will void all your warantees if you were given one.\u00a0 Here is an example that I’m using to build and test various bits and things.\u00a0 Note that I’ve added em(4) to my configuration as that’s the network interface I’m using.<\/p>\n

include MINIMAL
\nident MFS_NETBOOT<\/p>\n

nomakeoptions DEBUG # Build kernel with gdb(1) debug symbols
\nnomakeoptions WITH_CTF # Run ctfconvert(1) for DTrace support<\/p>\n

nooptions COMPAT_FREEBSD32
\nnooptions COMPAT_FREEBSD4
\nnooptions COMPAT_FREEBSD5
\nnooptions COMPAT_FREEBSD6
\nnooptions COMPAT_FREEBSD7
\nnooptions COMPAT_FREEBSD9
\nnooptions COMPAT_FREEBSD10
\nnooptions COMPAT_FREEBSD11
\nnooptions KTRACE
\nnooptions AUDIT
\nnooptions CAPABILITY_MODE
\nnooptions CAPABILITIES
\nnooptions MAC
\nnooptions KDTRACE_FRAME
\nnooptions KDTRACE_HOOKS
\nnooptions DDB_CTF<\/p>\n

# Debugging support. Always need this:
\nnooptions KDB
\nnooptions KDB_TRACE
\n# For full debugger support use (turn off in stable branch):
\nnooptions DDB
\nnooptions GDB
\nnooptions DEADLKRES
\nnooptions INVARIANTS
\nnooptions INVARIANT_SUPPORT
\nnooptions WITNESS
\nnooptions WITNESS_SKIPSPIN
\n# Xen HVM Guest Optimizations
\n# NOTE: XENHVM depends on xenpci. They must be added or removed together.
\nnooptions XENHVM # Xen HVM kernel infrastructure
\nnodevice xenpci # Xen HVM Hypervisor services driver<\/p>\n

options GEOM_PART_GPT
\noptions GEOM_RAID
\noptions GEOM_LABEL
\noptions GEOM_UZIP<\/p>\n

options IFLIB
\ndevice em<\/p>\n

# Serial (COM) ports
\ndevice uart # Generic UART driver<\/p>\n

# Needed to actually boot from an MFS root
\ndevice md # Memory “disks”<\/p>\n

This will do the bare minimum and give you a functioning system, at least it does for me on my 2core Atom box.\u00a0 There’s little to no debugging enabled here, so if you chose to go this route remember that all of the facilities to debug are deactivated.<\/p>\n

Allow me a short digression here.\u00a0 One should NEVER create their own kernel configs from scratch and not derive them from one of the base kernel configs (GENERIC for example).\u00a0 When you hand craft a kernel config, you allow yourself to get into situations where new options and devices can be added or removed.\u00a0 This can lead to scenarios where your systems can be unusable and cost you many hours of hair pulling and gnashing of teeth.\u00a0 It is far simpler to include a base configuration and unset the devices and options you don’t want.\u00a0 Judicious use of noptions, nodevice and nomakeoptions can be used to remove things that you don’t want.\u00a0 Use them, please.<\/p>\n

Anywho, back to the task at hand.<\/p>\n

FreeBSD has shipped a directory to disk image creation tool for quite some time.\u00a0 makefs(8) does a great job of turning a directory tree into a disk image without all the the magical hand waving of creating a file backed md(4) device.\u00a0 Its a shortcut and you should use it.\u00a0 Embedded folks have been using it to boot up their MIPS and ARM devices for a while now.<\/p>\n

We’ll need to install the build somewhere that we can then access with makefs.\u00a0 I tend to just put it in a throw-away directory in \/var\/tmp, e.g. \/var\/tmp\/netboot_mfs<\/p>\n

So, something like ‘make installworld DESTDIR=\/var\/tmp\/netboot_mfs’ should do the trick here.\u00a0 At this point, we can modify this installed directory with whatever configuration changes needed for your system boot up.\u00a0 At a minimum, I modify \/boot\/loader.conf and \/boot.config.\u00a0 These may not be needed for your environment, but I make the following changes:<\/p>\n

loader.conf:
\nconsole=”comconsole”<\/p>\n

boot.config:
\n-D<\/p>\n

You can now build your disk image:
\nmakefs -M 192m \/var\/tmp\/mfs_root.img \/var\/tmp\/netboot_mfs
\nCalculated size of `mfs_root.img’: 201326592 bytes, 3800 inodes
\nExtent size set to 8192
\nmfs_root.img: 192.0MB (393216 sectors) block size 8192, fragment size 1024
\nusing 4 cylinder groups of 54.38MB, 6960 blks, 1152 inodes.
\nsuper-block backups (for fsck -b #) at:
\n32, 111392, 222752, 334112,
\nPopulating `mfs_root.img’
\nImage `mfs_root.img’ complete<\/p>\n

I add the “-M 192m” to the command line here to give my image a bit of space to do stuff and things on the running system without running out of space.\u00a0 makefs(8) will create an image big enough to fit the files regardless of the command line arguments, but the -M will “pad” the image to 192MB if its smaller.<\/p>\n

Go ahead and compress your diskimage with either bzip2 or gzip. I won’t judge.<\/p>\n

-rw-r–r– 1 root sbruno 35235255 Feb 7 16:14 mfs_root.img.bz2
\n-rw-r–r– 1 root sbruno 39516593 Feb 7 16:17 mfs_root.img.gz
\n(use bzip2).<\/p>\n

Don’t think that I haven’t forgotten about the kernel either.\u00a0 We’ll compress that beastie directly.\u00a0 One does not install it however, one does not need it, one does not simply install a kernel into Mordor.\u00a0 Recall that we set a MAKEOBJDIRPREFIX when doing out kernel and world builds.<\/p>\n

\/var\/tmp\/sbruno % find . -name kernel
\n.\/home\/sbruno\/bsd\/fbsd_head\/sys\/GENERIC-NODEBUG\/kernel
\n.\/home\/sbruno\/bsd\/fbsd_head\/sys\/NETBOOT\/kernel<\/p>\n

In this example, I’ve built both so you kind of have an idea of what you’re looking for, it can be a bit strange to *not* install a kernel, but as we are fond of saying these days, EVERYTHING IS FINE.\u00a0 <\/a><\/p>\n

You’re also going to need the pxeboot binary.\u00a0 On 12-CURRENT nowadays you don’t have to recompile it for TFTP installations as it’ll do the right thing depending on DHCP options.<\/p>\n

\/var\/tmp\/sbruno % find . -name pxeboot
\n.\/home\/sbruno\/bsd\/fbsd_head\/sys\/boot\/i386\/pxeldr\/pxeboot<\/p>\n

Speaking of DHCP and TFTP, we should probably set those up now.\u00a0 Remember when I linked you to The Outstanding FreeBSD Handbook<\/a>?\u00a0 It has the correct setting for the setup of inetd(8) and tftp(8) on your sever.\u00a0 Ignore all the NFS bits, those don’t apply.<\/p>\n

Install your dhcp server, you can use the same one from The Outstanding FreeBSD Handbook<\/a> as I did.<\/p>\n

The dhcpd.conf settings are a little different here.\u00a0 We don’t need to pass in a root-path option, but need a custom dhcp option code 150:<\/p>\n

option tftp-server code 150 = { ip-address };<\/p>\n

subnet 192.168.100.0 netmask 255.255.255.0 {
\noption routers 192.168.100.1;
\noption domain-name-servers 192.168.0.1;
\nhost router {
\nhardware ethernet xx:xx:xx:xx:xx:xx;
\nfixed-address 192.168.100.52;
\noption tftp-server 192.168.100.1;
\nnext-server 192.168.100.1;
\nfilename “\/boot\/pxeboot”;
\n}<\/p>\n

Now we can populate \/tftpboot with the files we’ve created and add *one* last configuration file, this time with FORTH to ensure we’ve done the right thing.<\/p>\n

\/tftpboot\/boot # ls -l \/tftpboot\/boot
\ntotal 34844
\ndrwxr-xr-x 2 root wheel 3 Feb 7 16:45 kernel
\n-rw-r–r– 1 root wheel 21 Feb 7 12:45 loader.conf
\n-r–r–r– 1 root wheel 174 Feb 7 09:47 loader.rc
\n-rw-r–r– 1 root wheel 35234122 Feb 7 12:04 netboot.img.bz2
\n-r–r–r– 1 root wheel 337920 Feb 7 08:34 pxeboot<\/p>\n

\/tftpboot\/boot # ls -l \/tftpboot\/boot\/kernel
\ntotal 2564
\n-rwxr-xr-x 1 root wheel 2565979 Feb 7 15:36 kernel.bz2<\/p>\n

loader.rc:
\necho Loading kernel
\nload \/boot\/kernel\/kernel
\necho Loading md_image …
\nload -t md_image \/boot\/netboot.img
\nset vfs.root.mountfrom=”ufs:\/dev\/md0″
\nset console=”comconsole”
\nboot<\/p>\n

Here dear readers, is where I leave you.\u00a0 If you’ve done all the right things … you should end up booting up into your newly built MFS root.\u00a0 If you haven’t done all the right things or I’ve omitted something, then the Nazgul will find you.<\/p>\n","protected":false},"excerpt":{"rendered":"

Irritatingly, I couldn’t find a decent cut-n-paste how to on the Internet for using a much loved and mostly overlooked FreeBSD installation and boot up process using a fully cooked diskless image fetched via TFTP instead of the really well documented and fantastic howto on NFS booting. Primarily, I wanted to switch off of NFS […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"_share_on_mastodon":"0"},"categories":[1],"tags":[],"share_on_mastodon":{"url":"","error":""},"_links":{"self":[{"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=\/wp\/v2\/posts\/247"}],"collection":[{"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=247"}],"version-history":[{"count":2,"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=\/wp\/v2\/posts\/247\/revisions"}],"predecessor-version":[{"id":251,"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=\/wp\/v2\/posts\/247\/revisions\/251"}],"wp:attachment":[{"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=247"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=247"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=247"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}