I had an opportunity at work to build up a new machine to do our FreeBSD builds at work this quarter and wanted to see how far I could take ZFS on high end OEM hardware.<\/p>\n
After evaluating HP and Dell gear, I settled on the Dell r720xd as my platform to move forward.\u00a0 Primarily, this was due to the lack of *real* JBOD support on the HP line of SAS controllers.\u00a0 The Dell H310 has a “SYSPD” option in mfi(4) that allows one to use the raw disks and ignore the RAID capabilites.\u00a0 I went ahead and modified the FreeBSD mfiutil(4) tool to allow run time configuration into this mode.\u00a0 http:\/\/svnweb.freebsd.org\/base?view=revision&revision=254906<\/p>\n
I ended up with 64G of RAM and 2x CPU: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz (2300.05-MHz K8-class CPU).\u00a0 I stacked 12x3TB SAS drives (really just SATA drives with SAS firmware, but hey, they cost WAY MORE).<\/p>\n
Setup the zpool with 2x raidz1 vdevs on this go around.\u00a0 There was some debate between myself and other colleagues if I should have gone with 1 raidz2 pool.\u00a0 It theoretically would have some better failure handling since I would have 2x parity disks in the same pool, but it seemed that I should go with 2x vdevs, each with 1 parity drive in this case because of how much write activity building 7 different FreeBSD distributions simultaneously would generate.<\/p>\n
I ended up with a zpool that looks like this:<\/p>\n
pool: zroot
\nstate: ONLINE
\nscan: scrub repaired 0 in 0h0m with 0 errors on Wed Aug 21 15:55:09 2013
\nconfig:<\/p>\n
NAME\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 STATE\u00a0\u00a0\u00a0\u00a0 READ WRITE CKSUM
\nzroot\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nraidz1-0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nmfisyspd1p3\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nmfisyspd2p3\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nmfisyspd3p3\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nmfisyspd4p3\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nmfisyspd5p3\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nmfisyspd0p3\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nraidz1-1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nmfisyspd6p3\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nmfisyspd7p3\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nmfisyspd8p3\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nmfisyspd9p3\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nmfisyspd10p3\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nmfisyspd11p3\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0<\/p>\n
Sexy.<\/p>\n
Performance wise, this machine now spits out our production images in about 95 minutes as opposed to the 255 minutes from before.\u00a0 Its a complete dead lift of hardware, new cpus, disks, more ram, different F\/S, etc.\u00a0 I’m pretty happy with it, but of course, its Cadillac prices, so your mileage will vary.<\/p>\n","protected":false},"excerpt":{"rendered":"
I had an opportunity at work to build up a new machine to do our FreeBSD builds at work this quarter and wanted to see how far I could take ZFS on high end OEM hardware. After evaluating HP and Dell gear, I settled on the Dell r720xd as my platform to move forward.\u00a0 Primarily, […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"_share_on_mastodon":"0"},"categories":[2],"tags":[],"share_on_mastodon":{"url":"","error":""},"_links":{"self":[{"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=\/wp\/v2\/posts\/122"}],"collection":[{"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=122"}],"version-history":[{"count":1,"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=\/wp\/v2\/posts\/122\/revisions"}],"predecessor-version":[{"id":125,"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=\/wp\/v2\/posts\/122\/revisions\/125"}],"wp:attachment":[{"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=122"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=122"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/blog.ignoranthack.me\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=122"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}