I think I was successful, and it wasn’t that hard. I did have to teach myself a bit about how sgdisk(8) works and what not to do, but in the end I came up with the following bit of notes that seemed to work for me. A lot of my previous ZFS experience was applicable here, but I have spent decades being oblivious to the magic of Grub and boot loaders on x86 Linux ports.

Thank you to the fine folks over at OpenZFS for having a VERY easy to follow 22.04 install guide, which I definitely stole things from.

The following is just my notes and if you see some things that will bite me, let me know on Mastodon, you can find me over at infosec.exchange

# sgdisk -p /dev/sda
Disk /dev/sda: 781422768 sectors, 372.6 GiB
Model: MK0400GCTZA     
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): DFE9708A-6076-4E24-B683-64A286A1E751
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 781422734
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048         1050623   512.0 MiB   EF00  EFI system partition
   2         1050624         5244927   2.0 GiB     8200  Linux swap
   3         5244928         9439231   2.0 GiB     BE00  Solaris boot
   4         9439232       781422734   368.1 GiB   BF00  Solaris root
# sgdisk -p /dev/sdb
Disk /dev/sdb: 537234768 sectors, 256.2 GiB
Model: Crucial_CT275MX3
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): AC04D23F-E1FD-4B88-9880-B9679DCF4BBD
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 537234734
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048         1050623   512.0 MiB   EF00  EFI System Partition
   2         1050624         5244927   2.0 GiB     8200  
   3         5244928         9439231   2.0 GiB     BE00  
   4         9439232       537234734   251.7 GiB   BF00
---------------------------------------------------------------------------------
   
# sgdisk -d 1 /dev/nvme0n1
# sgdisk -n 1:2048:1050623 -t 1:ef00 -c1:"EFI system partition" /dev/nvme0n1
# sgdisk -n 2:1050624:5244927 -t 2:8200 -c2:"Linux swap" /dev/nvme0n1
# sgdisk -n 3:5244928:9439231 -t 3:be00 -c3:"Solaris boot" /dev/nvme0n1
# sgdisk -N 4 -t 4:bf00 -c4:"Solaris root" /dev/nvme0n1

# sgdisk -p /dev/nvme0n1
Disk /dev/nvme0n1: 500118192 sectors, 238.5 GiB
Model: WDC WDS256G1X0C-00ENX0                  
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): F2E3AFF0-5C0C-4A03-BE8C-83EC2DEDB914
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 500118158
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048         1050623   512.0 MiB   EF00  EFI system partition
   2         1050624         5244927   2.0 GiB     8200  Linux swap
   3         5244928         9439231   2.0 GiB     BE00  Solaris boot
   4         9439232       500118158   234.0 GiB   BF00  Solaris root
--------------------------------------------------------------------------------
# cat /proc/mdstat 
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10] 
md127 : active raid1 sdb2[0] sda2[1]
      2094080 blocks super 1.2 [2/2] [UU]
      
unused devices: <none>
root@hattie:~# mdadm --manage /dev/md127 --fail /dev/sda2
mdadm: set /dev/sda2 faulty in /dev/md127
root@hattie:~# mdadm --manage /dev/md127 --remove /dev/sda2
mdadm: hot removed /dev/sda2 from /dev/md127
root@hattie:~# cat /proc/mdstat 
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10] 
md127 : active raid1 sdb2[0]
ex      2094080 blocks super 1.2 [2/1] [U_]
      
unused devices: <none>
# mdadm --manage /dev/md127 --add /dev/nvme0n1p2
mdadm: added /dev/nvme0n1p2
root@hattie:~# cat /proc/mdstat 
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10] 
md127 : active raid1 nvme0n1p2[2] sdb2[0]
      2094080 blocks super 1.2 [2/1] [U_]
      [===>.................]  recovery = 18.7% (392832/2094080) finish=0.2min speed=130944K/sec
      
unused devices: <none>
-------------------------------------------------------------------------------------
# zpool status bpool
  pool: bpool
 state: ONLINE
  scan: scrub repaired 0B in 00:00:01 with 0 errors on Sun Jul  2 00:00:02 2023
config:

	NAME                                      STATE     READ WRITE CKSUM
	bpool                                     ONLINE       0     0     0
	  mirror-0                                ONLINE       0     0     0
	    2ae2089b-40b0-1944-9c6d-a436c8ba3295  ONLINE       0     0     0
	    b2bbd1e6-7b68-4d90-bf9c-f6d6bb13daac  ONLINE       0     0     0

# ls -l /dev/disk/by-partuuid/
total 0
lrwxrwxrwx 1 root root 10 Jul  4 12:59 0240b02b-5a83-45f7-b209-de850d66cc48 -> ../../sda4
lrwxrwxrwx 1 root root 10 Jul  4 13:01 26050a01-81fa-df4c-bb64-b825072a4cb5 -> ../../sdb2
lrwxrwxrwx 1 root root 15 Jul  4 13:04 277066fd-6087-45d3-bc21-36e2be379a7a -> ../../nvme0n1p2
lrwxrwxrwx 1 root root 10 Jul  4 13:01 2ae2089b-40b0-1944-9c6d-a436c8ba3295 -> ../../sdb3
lrwxrwxrwx 1 root root 15 Jul  4 13:00 76e8a1b2-3877-4732-8b6b-2c4f47615733 -> ../../nvme0n1p4
lrwxrwxrwx 1 root root 10 Jul  4 12:59 83437572-9020-4945-8354-47d99f2de483 -> ../../sda2
lrwxrwxrwx 1 root root 15 Jul  4 13:00 86c776d9-9aa6-47de-8d45-26dc8833cea8 -> ../../nvme0n1p3
lrwxrwxrwx 1 root root 10 Jul  4 13:01 8a0ac60b-20ed-4bde-a27d-dd4f4160fa29 -> ../../sdb1
lrwxrwxrwx 1 root root 15 Jul  4 13:00 a963a8f4-3765-477c-bc08-6ea53f71b22b -> ../../nvme0n1p1
lrwxrwxrwx 1 root root 10 Jul  4 12:59 b2bbd1e6-7b68-4d90-bf9c-f6d6bb13daac -> ../../sda3
lrwxrwxrwx 1 root root 10 Jul  4 12:59 d8132cf7-b5f7-4721-8340-3ce42d4b918e -> ../../sda1
lrwxrwxrwx 1 root root 10 Jul  4 13:01 e70d10dd-f1bd-f345-b5a9-e285acd26d8a -> ../../sdb4

# zpool replace bpool /dev/disk/by-partuuid/b2bbd1e6-7b68-4d90-bf9c-f6d6bb13daac /dev/disk/by-partuuid/86c776d9-9aa6-47de-8d45-26dc8833cea8
# zpool status
  pool: bpool
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
	continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Tue Jul  4 13:08:52 2023
	305M scanned at 102M/s, 125M issued at 41.6M/s, 305M total
	121M resilvered, 40.89% done, 00:00:04 to go
config:

	NAME                                        STATE     READ WRITE CKSUM
	bpool                                       ONLINE       0     0     0
	  mirror-0                                  ONLINE       0     0     0
	    2ae2089b-40b0-1944-9c6d-a436c8ba3295    ONLINE       0     0     0
	    replacing-1                             ONLINE       0     0     0
	      b2bbd1e6-7b68-4d90-bf9c-f6d6bb13daac  ONLINE       0     0     0
	      86c776d9-9aa6-47de-8d45-26dc8833cea8  ONLINE       0     0     0  (resilvering)

errors: No known data errors
-------------------------------------------------------------------------------------------------

~# update-initramfs -c -k all
update-initramfs: Generating /boot/initrd.img-5.19.0-45-generic
cp: cannot stat '/lib/firmware/updates/amdgpu': No such file or directory
W: Possible missing firmware /lib/firmware/amdgpu/ip_discovery.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/vega10_cap.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/sienna_cichlid_cap.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/navi12_cap.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/psp_13_0_10_sos.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/aldebaran_cap.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/aldebaran_sjt_mec2.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/aldebaran_sjt_mec.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/gc_11_0_3_imu.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/gc_11_0_3_rlc.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/gc_11_0_3_mec.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/gc_11_0_3_me.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/gc_11_0_3_pfp.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/gc_11_0_0_toc.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/sdma_6_0_3.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/sienna_cichlid_mes1.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/sienna_cichlid_mes.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/navi10_mes.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/gc_11_0_3_mes1.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/gc_11_0_3_mes.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/smu_13_0_10.bin for module amdgpu
I: The initramfs will attempt to resume from /dev/md127
I: (UUID=514a7d66-c1d7-41c2-86e1-b072f30e2dd6)
I: Set the RESUME variable to override this.
update-initramfs: Generating /boot/initrd.img-5.19.0-46-generic
cp: cannot stat '/lib/firmware/updates/amdgpu': No such file or directory
W: Possible missing firmware /lib/firmware/amdgpu/ip_discovery.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/vega10_cap.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/sienna_cichlid_cap.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/navi12_cap.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/psp_13_0_10_sos.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/aldebaran_cap.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/aldebaran_sjt_mec2.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/aldebaran_sjt_mec.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/gc_11_0_3_imu.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/gc_11_0_3_rlc.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/gc_11_0_3_mec.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/gc_11_0_3_me.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/gc_11_0_3_pfp.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/gc_11_0_0_toc.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/sdma_6_0_3.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/sienna_cichlid_mes1.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/sienna_cichlid_mes.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/navi10_mes.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/gc_11_0_3_mes1.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/gc_11_0_3_mes.bin for module amdgpu
W: Possible missing firmware /lib/firmware/amdgpu/smu_13_0_10.bin for module amdgpu
I: The initramfs will attempt to resume from /dev/md127
I: (UUID=514a7d66-c1d7-41c2-86e1-b072f30e2dd6)
I: Set the RESUME variable to override this.

--------------------------------------------------------------------------------------
# grub-install --target=x86_64-efi --efi-directory=/boot/efi \
    --bootloader-id=ubuntu --recheck --no-floppy
Installing for x86_64-efi platform.
Installation finished. No error reported.

/etc/rc.conf:
powerd_enable="YES"
economy_cx_lowest="LOW"
economy_cpu_freq="LOW"

These will slow your laptop down when running of battery and should give you the needed juice to take notes and update your remote projects. I wouldn’t recommend them for a machine you need to do buildworlds upon.

/boot/loader.conf
acpi_video_load="YES"

This one is a module that I forgot about for some time. If you load it, you’ll get a couple of new tunables that can be used to turn down the backlight, which is useful even if all you want is to not disturb your better-half when you are reading at night.

sysctl hw.acpi.video
hw.acpi.video.lcd0.economy: 50
hw.acpi.video.lcd0.fullpower: 80

You can throw these into your /etc/sysctl.conf at whatever value you like. See the man page, man 4 acpi_video for more information.

% acpiconf -i0|less
Design capacity:        3900 mAh
Last full capacity:     3827 mAh
Technology:             secondary (rechargeable)
Design voltage:         11100 mV
Capacity (warn):        390 mAh
Capacity (low):         118 mAh
Low/warn granularity:   39 mAh
Warn/full granularity:  39 mAh
Model number:           DELL 9C26T33
Serial number:          48476
Type:                   LION
OEM info:               SMP
State:                  discharging
Remaining capacity:     52%
Remaining time:         3:07
Present rate:           640 mA (6935 mW)
Present voltage:        10837 mV

Not too shabby for power savings. Not sure if .6 Amp is really enough to make this solid. I think this gives me a solid 4 hours of note taking and walking around.

# newfs_msdos -F 32 -c32 -C 2G /dev/da0

As of today’s FreeBSD 14CURRENT, you have to massage the numbers around even though this should have just worked right out of the box. I suspect the math inside of sector and cluster calculation may not be correct.

Because I was trying to do things via wireless networking, I couldn’t use the most common and easy method for networking to just use dhcp and a bridge(4) with tap(4) on my laptop. I asked a few people in the FreeBSD Discord, trolled FuzzyKaren’s great Kubernetes blog (https://productionwithscissors.run/) and browsed a few FreeBSD forums to come up with something so simple.

Use the UEFI install method for your linux VM, I used the FreeBSD Handbook and a ZFS zvol for it storage. https://www.cyberciti.biz/faq/how-to-install-linux-vm-on-freebsd-using-bhyve-and-zfs/

I reconfigured the networking in the Ubuntu VM to be statically assigned to 10.0.0.2/24 with a gateway of 10.0.0.1 (and using my home dns server at 192.168.1.1). I setup the tap0 interface being used with:

gateway_enable="YES"
# tap interfaces for ubuntu VM
cloned_interfaces="tap0"
ifconfig_tap0="inet 10.0.0.1 netmask 255.255.255.0"

This bit will let your laptop and your Bhyve VM see each other, but you’ll need a small amount of PF to get to the Internet:

wlan_if="wlan0"

# create all VM interfaces in this subnet
vm_net="10.0.0.0/24"
vm_ubuntu="10.0.0.2"

nat on $wlan_if from $vm_ubuntu to !$vm_net -> $wlan_if

pass out all
pass in all

I’ve started tested and repairing things to get them up and working again.

Started sorting things around, and there’s definitely two IIe enhanced units and two IIe standard models in here. However, I quickly realized that there was a ton of missing screws. Figures.

I looked around the Internet and asked a few vendors. Nobody seems to have the actual screw types and sizes that are needed for the Apple IIe. None of the various Apple service manuals nor the Sams versions indicated what they were. If you were repairing a machine, it most likely came with the machine.

I’ve made an attempt here to list the type and sizes of screws that I was able to use on my reassembly projects.

Apple IIe “Enhanced”

The keyboard for this unit shares screws between its mounting brackets to the upper case and the secure screws that hold the keyboard together. These are fairly common 3/8 inch 6-32 machine screws. Easily obtainable from your any American hardware store. Sorry metric-land, these are definitely American spec screws. You’ll need 4 to mount the keyboard in the case and another 6 if you are missing the screws that hold the keyboard “sandwich” together.

In order to mount the upper case plastic to the lower case metal sheet that the motherboard is mounted on, you’ll need another set of sheet metal screws and lock washers. This is assuming that the mounting clips are still on your upper case plastics. I see from the Internet that these have a tendency to evaporate over the 40 years or so that these have existed. I was able to find a compatible set of 6×1/2 sheet metal screws that worked great with some #8 simple external grasping lock washers.

Now that you can mount the keyboard to the case and you can mount the bottom to the top of the case, we can look at how to mount he power supply to the bottom of the case.

For this, you’ll need 4, 4-40 1/4″ screw with 4, 1/8″ #6 lock washers. This will give you a secure set to hold the power supply in place.

For simplicity sake, here is a small table with all the relevant screws. I can’t believe how hard it was to find this information and I hope it helps whomever finds it once again.

Screw Type and Name, Part Number, Quantity
Base Case Machine Screw, 44705-A, 9
Base Case Lock Washer, 1387-L, 9
Case Motherboard Machine Screw, H90120, 4
Flat Motherboard Washer, 1235-A, 4
Case Keyboard Machine Screw, H90120, 4
Keyboard Machine Screws, H90120, 6
Power Supply Case Machine Screw, 1962-A, 4
Power Supply Case Lock Washer, 1386-K, 4

Bah. Good luck with that.

Error #1: WordPress detonates when I try to post and becomes unresponsive. /var/log/nginx/error.log shows:

PHP message: PHP Fatal error: Uncaught Error: Call to undefined function ctype_digit()

Bah, why did the WP->Twitter plugin explode on this today? I’m assuming that PHP 7.2 has changed its packaging on FreeBSD so … build and install php72-ctype. Let’s try this again.

Error #2: WordPress detonates when I try to post and becomes unresponsive. /var/log/nginx/error.log shows:

PHP message: PHP Fatal error: Uncaught Error: Call to undefined function filter_var()

Same kind of deal. More explosions in the WP->Twitter plugin. Build and install php72-filter. Excellent. I can now annoy everyone with my ignorance again.

1. You have to know that the cell phone app wants the URL of the PHP xml RPC web address. It doesn’t assume anything when you enter in the address of your personal, self-hosted WordPress instance.
2. If you’re like me, you also probably need to up the limit on file uploads that is set by default in nginx.

server {
    listen 80;
    server_name blog.ignoranthack.me;
    location / {
            proxy_pass http://10.0.0.2:80;
            proxy_read_timeout 40;
            client_max_body_size 300m;
    }
}

Primarily, I wanted to switch off of NFS booting as it tends to be really troublesome if one is doing development or debugging of the FreeBSD network stack or Ethernet drivers. I couldn’t reliably screw around with the network interfaces without disturbing the NFS root mount and inflicting relatively long timeouts on my tests. I decided to try and switch to MFS root based testing so that I didn’t run into these types of problems, however I didn’t realize that the boot loader had gone through some overhauls recently and the documentation that *is* floating around on the Internet, is a bit out of date. So, here is the beginning of what I hope will be the inception of a new FreeBSD handbook entry before version 12 is released.

I want to cover the actual construction of the entire process, so I will not be hand copying binaries from a release and all files will be constructed from source where possible.  This means that I can fully automate a build and regression from source to ensure that this document stays relatively up to date.  I’m an active FreeBSD contributor, so I have many checkouts of FreeBSD lying around.  If you’re a sysadmin, you probably use /usr/src for most of the projects like this, which is fine for our purposes.

Assuming that you have a fresh checkout of FreeBSD somewhere, lets start by crafting a “src.conf” and a “make.conf” to minimize a bit of build time and a bit of space.

I turn a lot of stuff off in the build here.  Since I’m running the same major revision of FreeBSD that I’ll be building, I can get away with some shenanigans regarding not building toolchains.  If you are trying to build 12-CURRENT on 11-STABLE, you may have to play around with the knobs a bit in src.conf:

WITHOUT_BLUETOOTH=”YES”
WITHOUT_BSNMP=”YES”
WITHOUT_CAPSICUM=”YES”
WITHOUT_CDDL=”YES”
WITHOUT_COMPAT32=”YES”
WITHOUT_CTM=”YES”
WITHOUT_DEBUG_FILES=”YES”
WITHOUT_EXAMPLES=”YES
WITHOUT_FLOPPY=”YES”
WITHOUT_FREEBSD_UPDATE=”YES”
WITHOUT_GAMES=”YES”
WITHOUT_ICONV=”YES”
WITHOUT_JAIL=”YES”
WITHOUT_LIB32=”YES”
WITHOUT_LOCALES=”YES”
WITHOUT_LPR=”YES”
WITHOUT_MAKE=”YES”
WITHOUT_MAN=”YES”
WITHOUT_PORTSNAP=”YES”
WITHOUT_TESTS=”YES”
WITHOUT_PC_SYSINSTALL=”YES”
WITHOUT_RESCUE=”YES”
WITHOUT_SVNLITE=”YES”
WITHOUT_TEXTPROC=”YES”
WITHOUT_TOOLCHAIN=”YES”
WITHOUT_WIRELESS=”YES”
WITHOUT_KERBEROS=”YES”
WITHOUT_MAIL=”YES”
WTTHOUT_EFI=”YES”
WITHOUT_NTP=”YES”

My make.conf is much shorter, and only set’s a few things to make life easier:

BOOT_PXELDR_ALWAYS_SERIAL=y
NO_MODULES=y
MALLOC_PRODUCTION=y

Set/export a couple of variables to your local shell to use these:

setenv SRCCONF /path/to/src.conf
setenv __MAKE_CONF /path/to/make.conf

Since I’m not building as root here, I set one more variable:

setenv MAKEOBJDIRPREFIX /var/tmp/mfsroot

Execute a buildworld and turn some electrons into heat for a while.  You’ll probably note a big difference in build times here.  Also note that you’re building a much smaller FreeBSD system with a lot of missing pieces.  No local mail agent, no ntp services, no compilers/debuggers, no wireless bits … etc.  Your mileage may vary here, but I needed a small world as I’m debugging/testing kernel/network drivers, not userland.

You can make a choice here of kernels to build.  Most folks will want GENERIC-NODEBUG, but if you want, you can base a customized kernel configuration from MINIMAL that will do the trick.  You’ll need to include drivers specific to your systems and this will void all your warantees if you were given one.  Here is an example that I’m using to build and test various bits and things.  Note that I’ve added em(4) to my configuration as that’s the network interface I’m using.

include MINIMAL
ident MFS_NETBOOT

nomakeoptions DEBUG # Build kernel with gdb(1) debug symbols
nomakeoptions WITH_CTF # Run ctfconvert(1) for DTrace support

nooptions COMPAT_FREEBSD32
nooptions COMPAT_FREEBSD4
nooptions COMPAT_FREEBSD5
nooptions COMPAT_FREEBSD6
nooptions COMPAT_FREEBSD7
nooptions COMPAT_FREEBSD9
nooptions COMPAT_FREEBSD10
nooptions COMPAT_FREEBSD11
nooptions KTRACE
nooptions AUDIT
nooptions CAPABILITY_MODE
nooptions CAPABILITIES
nooptions MAC
nooptions KDTRACE_FRAME
nooptions KDTRACE_HOOKS
nooptions DDB_CTF

# Debugging support. Always need this:
nooptions KDB
nooptions KDB_TRACE
# For full debugger support use (turn off in stable branch):
nooptions DDB
nooptions GDB
nooptions DEADLKRES
nooptions INVARIANTS
nooptions INVARIANT_SUPPORT
nooptions WITNESS
nooptions WITNESS_SKIPSPIN
# Xen HVM Guest Optimizations
# NOTE: XENHVM depends on xenpci. They must be added or removed together.
nooptions XENHVM # Xen HVM kernel infrastructure
nodevice xenpci # Xen HVM Hypervisor services driver

options GEOM_PART_GPT
options GEOM_RAID
options GEOM_LABEL
options GEOM_UZIP

options IFLIB
device em

# Serial (COM) ports
device uart # Generic UART driver

# Needed to actually boot from an MFS root
device md # Memory “disks”

This will do the bare minimum and give you a functioning system, at least it does for me on my 2core Atom box.  There’s little to no debugging enabled here, so if you chose to go this route remember that all of the facilities to debug are deactivated.

Allow me a short digression here.  One should NEVER create their own kernel configs from scratch and not derive them from one of the base kernel configs (GENERIC for example).  When you hand craft a kernel config, you allow yourself to get into situations where new options and devices can be added or removed.  This can lead to scenarios where your systems can be unusable and cost you many hours of hair pulling and gnashing of teeth.  It is far simpler to include a base configuration and unset the devices and options you don’t want.  Judicious use of noptions, nodevice and nomakeoptions can be used to remove things that you don’t want.  Use them, please.

Anywho, back to the task at hand.

FreeBSD has shipped a directory to disk image creation tool for quite some time.  makefs(8) does a great job of turning a directory tree into a disk image without all the the magical hand waving of creating a file backed md(4) device.  Its a shortcut and you should use it.  Embedded folks have been using it to boot up their MIPS and ARM devices for a while now.

We’ll need to install the build somewhere that we can then access with makefs.  I tend to just put it in a throw-away directory in /var/tmp, e.g. /var/tmp/netboot_mfs

So, something like ‘make installworld DESTDIR=/var/tmp/netboot_mfs’ should do the trick here.  At this point, we can modify this installed directory with whatever configuration changes needed for your system boot up.  At a minimum, I modify /boot/loader.conf and /boot.config.  These may not be needed for your environment, but I make the following changes:

loader.conf:
console=”comconsole”

boot.config:
-D

You can now build your disk image:
makefs -M 192m /var/tmp/mfs_root.img /var/tmp/netboot_mfs
Calculated size of `mfs_root.img’: 201326592 bytes, 3800 inodes
Extent size set to 8192
mfs_root.img: 192.0MB (393216 sectors) block size 8192, fragment size 1024
using 4 cylinder groups of 54.38MB, 6960 blks, 1152 inodes.
super-block backups (for fsck -b #) at:
32, 111392, 222752, 334112,
Populating `mfs_root.img’
Image `mfs_root.img’ complete

I add the “-M 192m” to the command line here to give my image a bit of space to do stuff and things on the running system without running out of space.  makefs(8) will create an image big enough to fit the files regardless of the command line arguments, but the -M will “pad” the image to 192MB if its smaller.

Go ahead and compress your diskimage with either bzip2 or gzip. I won’t judge.

-rw-r–r– 1 root sbruno 35235255 Feb 7 16:14 mfs_root.img.bz2
-rw-r–r– 1 root sbruno 39516593 Feb 7 16:17 mfs_root.img.gz
(use bzip2).

Don’t think that I haven’t forgotten about the kernel either.  We’ll compress that beastie directly.  One does not install it however, one does not need it, one does not simply install a kernel into Mordor.  Recall that we set a MAKEOBJDIRPREFIX when doing out kernel and world builds.

/var/tmp/sbruno % find . -name kernel
./home/sbruno/bsd/fbsd_head/sys/GENERIC-NODEBUG/kernel
./home/sbruno/bsd/fbsd_head/sys/NETBOOT/kernel

In this example, I’ve built both so you kind of have an idea of what you’re looking for, it can be a bit strange to *not* install a kernel, but as we are fond of saying these days, EVERYTHING IS FINE. 

You’re also going to need the pxeboot binary.  On 12-CURRENT nowadays you don’t have to recompile it for TFTP installations as it’ll do the right thing depending on DHCP options.

/var/tmp/sbruno % find . -name pxeboot
./home/sbruno/bsd/fbsd_head/sys/boot/i386/pxeldr/pxeboot

Speaking of DHCP and TFTP, we should probably set those up now.  Remember when I linked you to The Outstanding FreeBSD Handbook?  It has the correct setting for the setup of inetd(8) and tftp(8) on your sever.  Ignore all the NFS bits, those don’t apply.

Install your dhcp server, you can use the same one from The Outstanding FreeBSD Handbook as I did.

The dhcpd.conf settings are a little different here.  We don’t need to pass in a root-path option, but need a custom dhcp option code 150:

option tftp-server code 150 = { ip-address };

subnet 192.168.100.0 netmask 255.255.255.0 {
option routers 192.168.100.1;
option domain-name-servers 192.168.0.1;
host router {
hardware ethernet xx:xx:xx:xx:xx:xx;
fixed-address 192.168.100.52;
option tftp-server 192.168.100.1;
next-server 192.168.100.1;
filename “/boot/pxeboot”;
}

Now we can populate /tftpboot with the files we’ve created and add *one* last configuration file, this time with FORTH to ensure we’ve done the right thing.

/tftpboot/boot # ls -l /tftpboot/boot
total 34844
drwxr-xr-x 2 root wheel 3 Feb 7 16:45 kernel
-rw-r–r– 1 root wheel 21 Feb 7 12:45 loader.conf
-r–r–r– 1 root wheel 174 Feb 7 09:47 loader.rc
-rw-r–r– 1 root wheel 35234122 Feb 7 12:04 netboot.img.bz2
-r–r–r– 1 root wheel 337920 Feb 7 08:34 pxeboot

/tftpboot/boot # ls -l /tftpboot/boot/kernel
total 2564
-rwxr-xr-x 1 root wheel 2565979 Feb 7 15:36 kernel.bz2

loader.rc:
echo Loading kernel
load /boot/kernel/kernel
echo Loading md_image …
load -t md_image /boot/netboot.img
set vfs.root.mountfrom=”ufs:/dev/md0″
set console=”comconsole”
boot

Here dear readers, is where I leave you.  If you’ve done all the right things … you should end up booting up into your newly built MFS root.  If you haven’t done all the right things or I’ve omitted something, then the Nazgul will find you.

In order to make vim usable for what I do, I asked the friendly folks in #bsdports on EFNet for suggestions. They suggested the following for my use, but your mileage may vary:

.vimrc
set bg=dark
set nohlsearch
set mouse=v

This seemed to turn off some of the new search features that people like in Vim 8 that caught me off guard. Hope this helps folks if you’re like me and were caught off guard by the whole thing.

The FreeBSD ports tree has 20k+ individual software packages maintained in it, and I was not in the mood to checkout all of that onto a host for the purpose of validating a build of QEMU on FreeBSD 10.

Turns out, if you pay attention, you can indeed do some individual checkouts of things in a sparse fashion and get results that are useful.  Your mileage may vary here depending on the needed dependencies, but this should be a useful hint to getting started.

I will only address how to do this with Subversion, but I’m sure that the folks who use git will have no trouble seeing how to duplicate this scenario.

• svn co –depth files svn+ssh://repo.freebsd.org/ports/head fbsd_ports
• cd fbsd_ports
• svn co svn+ssh://repo.freebsd.org/ports/head/Mk
• svn co svn+ssh://repo.freebsd.org/ports/head/Templates
• svn co –depth files svn+ssh://repo.freebsd.org/ports/head/lang
• cd lang
• svn co svn+ssh://repo.freebsd.org/ports/head/lang/perl5.20
• svn co svn+ssh://repo.freebsd.org/ports/head/lang/python27
• cd ..
• svn co –depth files svn+ssh://repo.freebsd.org/ports/head/emulators
• cd emulators
• svn co svn+ssh://repo.freebsd.org/ports/head/emulators/qemu-user-static
• svn co svn+ssh://repo.freebsd.org/ports/head/emulators/qemu-sbruno

My goal, was to build qemu-user-static for FreeBSD 10.  There’s a bunch of dependencies here, so I’ll try and unwind them a bit.

In order to build any port, you need the files contained in the top of tree.  The support Makefiles in Mk/ and Templates/ are required to do a lot of things and may or may not be used as short cuts/helpers in the Makefiles of individual ports.

The Makefile of any directory that you traverse to get to your port will need to exist (emulators/Makefile for example)

The lang/perl and lang/python ports are needed here as there is some pollution in the Makefiles that requires something to be parsed during builds.  This probably shouldn’t be there, but is beyond the scope of what I wanted to get done this morning.

Checking out the individual ports into their proper locations and setting PORTSDIR in your environment to your new directory structure (/home/sbruno/fbsd_ports) will now allow you to use your new checkout in some fashion to build.