Running OpenBSD on SPARC64 (HTTPd, packages, patching, X11, …)

In my previous post I described the process of installing OpenBSD 6.0 on a SPARC64 machine and updating it all the way to 6.5. Now it’s time to actually do something with it to get an idea of how well OpenBSD works on this architecture!

OpenBSD’s base system

The OpenBSD team takes pride in providing an ultra-secure operating system. It’s a well-known fact that the project’s extremely high standards only apply to the base system. Every now and then critics pop up and claim that this basically defeats the whole idea and even accuse the project of “keeping their base system so small that it’s useless by itself” to keep up their defined goals.

There’s some truth to it: The base system is kept (relatively) small if you compare it to some of the fatter operating systems out there. But that’s about it because actually these allegation could not be further from the truth. The base system includes doas, a simpler sudo replacement. It comes with tmux. OpenBSD even maintains it’s own fork of X.org, called Xenocara (not even FreeBSD comes with an X11 server by default) and there’s in fact a lot that you can achieve with the base system alone! Let’s look at one such possibility.

HTTPd

Since the OpenBSD developers are convinced that a webserver is something to keep around all the time, there’s one in base. Originally they used the Apache HTTPd for this. The problem was that at some point, the Apache Foundation decided to give up their Apache 1.0 license and replace it with version 2.0 (they had been criticized a lot for being incompatible with the GPL and the new version solved that problem). The newer version also made the license less simple and permissive than it had been before and OpenBSD did not like the new license. For that reason they basically stayed with the old Apache 1.3 webserver for a long time. They maintained and patched it all that time, but the software really begun to show it’s age.

So for version 5.6, OpenBSD finally removed the old Apache webserver in base and replaced it with Nginx. One release later, they did away with that, too, because they felt that it was starting to become too bloated for their needs. They imported OpenBSD HTTPd into base instead: A home-grown, very simple webserver. It evolved over time, but even though it having gotten more features implemented and becoming a fine little webserver, it strives to keep it simple.

The developers resist the temptation to add new features just because they could and have even made a list of things that some people might want which however will never be implemented because they would raise complexity to an unacceptable level. OpenBSD HTTPd does not want to be a webserver for everyone. It wants to be a ultra-secure webserver that does enough to be useful to many people. If you have any needs above what it offers – get another one.

Simple static website configuration of OpenHTTPd

The simplicity of HTTPd adds a lot to its beauty. I’ve written some HTML for a test page (see screenshot). All of the configuration that I need to do for HTTPd is as follows:

server spaffy.local {
    listen on egress port 80
}

Yes, that’s all that is required: I basically define a vHost (“Server” in HTTPd lingo) and have the application listen on the HTTP default port 80 on egress (a keyword which means whatever interface has the default route). Let’s check if that configuration really is valid by issuing

httpd -n

And it is! Impossible? No. Remember that OpenBSD comes with sane defaults. For that reason there’s usually pretty little that you need to configure. You could, of course. And we’ll be doing that a little later.

Now let’s force-start httpd (we need -f since the service is not enabled, yet, and we want to manually start it once):

rcctl -f start httpd

I’ve edited the /etc/hosts file on my laptop to be able to use the spaffy.local name. So now I can just type that into the address bar of my browser and reach the test page that the SPARC64 machine hosts. OK, a static page probably doesn’t impress you so much. Fortunately that’s not all that we can do in just relying on what base offers!

Static test page displayed in browser

CGI

OpenBSD also comes with Perl as part of the default install. I got that Lama book several years ago, read through about 2/3 of it and then decided that I didn’t like Perl too much. For that reason I never really did anything with it, but here I want to do something with what OpenBSD provides me with, so Perl is a logical choice and I might finally do something with it. Here’s what I came up with:

#!/usr/bin/perl
use strict;
use warnings;

my $osname = `uname -s`;
my $osver = `uname -r`;
my $osarch = `uname -m`;
chomp($osname, $osver, $osarch);

my @months = qw( Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec );
my @days = qw( Sun Mon Tue Wed Thu Fri Sat Sun );
my ( $sec, $min, $hour, $mday, $mon, $year, $wday, $yday ) = gmtime();

print "Content-type: text/html\n\n";
print "<html><head><title>Greetings!</title></head>";
print "<body>Hello from <strong>$osname $osver</strong> on <strong>$osarch</strong>!";
print "<br><br>This page was created by Perl $^V on $days[$wday], $months[$mon] $mday";

if (length($mday < 2)) {
  if (substr($mday, -1) == "1") {
    print "st"; }
  elsif (substr($mday, -1) == "2") {
    print "nd"; }
  elsif (substr($mday, -1) == "3") {
    print "rd"; }
  else {
    print "th"; }
} else {
  if ((substr($mday, 0, 1) ne "1") and (substr($mday, -1) == "1")) {
    print "st"; }
  if ((substr($mday, 0, 1) ne "1") and (substr($mday, -1) == "2")) {
    print "nd"; }
  if ((substr($mday, 0, 1) ne "1") and (substr($mday, -1) == "3")) {
    print "rd"; }
  else {
    print "th"; }
}

print ", $hour:";
if (length($min) == 1) {
  print "0";
}
print "$min (UTC)</body></html>";

Nothing too fancy, but for a first attempt at writing Perl it’s probably OK. After making the script executable, I can run it on the system and get the expected output. Things get a little more complex, though. HTTPd runs in a chroot for security reasons. And just copying the script into the chroot and trying it to execute in a chrooted environment fails with “no such file or directory”.

Huh? I just copied it there, didn’t I? I sure did. The reason for this happening is that the Perl interpreter is not available in the chroot. So let’s copy that one over as well and try again. Abort trap! How do they say? Getting a different error can be considered progress…

Perl CGI script failing due to chroot

Ok, now Perl is there, but it’s not functional. It requires some system libraries not present in the chroot. Using ldd on the Perl executable, I learn which libraries it needs. And after providing them, I can run the script in the chroot! There is a new problem, though: Perl is complaining about missing modules. The simplest solution in our case is to just remove them from the demo script as they are not strictly (haha!) necessary.

Providing Perl dependencies in the web chroot

On to the next step. Here’s a little addition to the HTTPd configuration:

    location "/cgi-bin/*" {
        root "/"
        fastcgi
    }

It basically adds different rules for the case that anything below /cgi-bin is being requested. It changes the document root for this and enables fastcgi. Now I only need to start the slowcgi service (OpenBSD’s shrewdly named fastcgi implementation) and restart HTTPd. My Perl program makes uses of the system’s uname command, so that should be made accessible in the chroot, too, of course.

Finishing the dynamic webpage setup

And that’s it. The script is executed in the webserver and the expected resulting page generated ,which is then served properly:

Dynamically created webpage displayed in beowser

I think this is pretty cool. Try to do that with just the default install of other operating systems! BTW: Want to make HTTPd and slowcgi start automatically after boot? No problem, just put the following into /etc/rc.conf.local:

httpd_flags=""
slowcgi_flags=""

This makes the init system start both daemons by default (and you can of course drop the “-f” flag to rcctl if you need to interact with them).

Binary packages

For OpenBSD 6.5, pre-built packages are offered for 8 of the 13 supported architectures – among them SPARC64. There’s just a couple short of 9,500 packages available (on amd64 it’s 10,600 – so in fact most packages are there)!

Things like GCC 8.3 and even GNAT 4.9 (the Ada part of GCC which is interesting because it’s written in Ada and thus needs to be bootstrapped to every new architecture by means of cross-compiling) are among the packages, as is LLVM 7.0. When it comes to desktop environments, you can choose e.g. between recent versions of Xfce, MATE and Gnome.

Actually, SPARC64 is one of only 4 architectures (the others being the popular ones amd64, i386 and arm64) that are receiving updates to the packages via the packages-stable repository. In there you’ll find newer versions of e.g. PHP, Exim (which had some pretty bad remote exploits fixed), etc.

Basic OpenBSD package management

I choose to install the sysclean package. Remember when I said that I skipped deleting the obsolete files when updating the OS in my last post? This program helps in finding files that should be deleted. However it’s not too intelligent – it just compares a list for a fresh system to the actual system on disk. For that reason it also lists a lot of files that I wouldn’t want to delete. Still it’s helpful to find out obsolete files that you might have forgot to remove.

Sysclean shows a lot of possible remove candidates

Errata patches

While OpenBSD tries it’s very best at providing a safe to use operating system, there really is nothing both useful and free from errors in the IT. If problems with some component of the system are found later, an erratum is published for it. If you are using OpenBSD in production, you are supposed to keep an eye on errata as they are released. Usually they consist of a patch or set of patches for system source code as well as instructions on how to apply it and recompile the needed parts.

Since version 6.1, OpenBSD comes with a handy utility called syspatch(8), which can e.g. be used to fetch binary patches for all known errata that have not been applied to the OS on the respective machine. This is nice – but it’s only available for amd64, i386 and arm64. So on SPARC64 we still have to deal with the old manual way keeping the system secure. However errata patches are also applied to the -STABLE branch and we can use that to get all the fixes.

No syspatch on SPARC64 – tracking -STABLE manually as it used to be

To upgrade our installation to 6.5-STABLE, the first step is to get the operating source of the current release (the sys tarball contains the kernel and src the rest of the base system). After extracting those, CVS is used to update the code to the latest 6.5-STABLE.

Done getting the stable changes from CVS

Once that’s done, it’s time to build the new (non-SMP) kernel:

# cd /sys/arch/$(machine)/compile/GENERIC
# make obj
# make config
# make && make install
# reboot

Building a 6.5-STABLE kernel

On my SunFire v100 the kernel build took 1h 20m. I was curious enough to build the userland as well, just to see how long it would take… The answer is: 85h 17m! I think that LLVM alone took about three days. The rest of the system wasn’t much of a problem for this old machine, but LLVM certainly was.

BTW, I had problems with “permission denied” when trying to “make obj”. After reading the manpage for release(8), I found out that /usr/obj should be owned by build:wobj with 770 permissions which had not been the case on my system.

Kernel build complete

Having done that, I thought that I might build Xenocara as well, to compare how long it takes to build. So I got the sources for that, too, updated them via CVS and built it. It took 9h 26m to build and install.

Xenocara built from (-STABLE) source

X11 on SPARC64

I had left out all X11-related distribution sets when installing OpenBSD. But after having installed Xenocara from source, I had it all available. So I decided to just do something with it. Since the server does not have a graphics card, I cannot run any X program on it directly, because the xserver won’t run. I decided to get a graphical application that is not part of Xenocara installed first. After browsing through the list, I settled on Midori, a WebKitGTK-based webbrowser.

Installing the Midori browser via packages

It took a moment to install all the dependencies, but everything worked. As the next step I enabled SSH X11 forwarding and restarted SSH.

Midori is installed, allowing X11 forwarding for SSH

After connecting to the SPARC64 machine via SSH and checking that the DISPLAY environment variable was set, I could just launch Midori and have it sent over to my laptop that I used to SSH into the other box. So the browser is being executed on the SPARC64 server but displayed on my other machine.

SSHing into the SPARC64 machine and forwarding Midori to my amd64 laptop

Everything worked well, I could even visit the OpenBSD homepage and it was rendered correctly.

The webkit-based browser works well on SPARC64!

Conclusion

OpenBSD is a fine operating system for people who value quality. The SPARC64 port of it seems to be in pretty good shape: Most packages and even stable-package updates are available. What is missing, is syspatch support – but only three architectures have that right now. Also the system compiler is still the ancient GCC version 4.2 which was the last one before the project switched the license to GPLv3.

OpenBSD 6.6 has been released one day after I finished compiling 6.5-STABLE. On amd64 I could now use sysupgrade(8) to upgrade to the new release even easier than before. This is also not supported on SPARC64. But these two little shortcommings just mean a little extra work that all OpenBSD users on any platform had to do anyway until not that long ago.

For 6.6 there are even more packages available for SPARC64. E.g. the Rust compiler has been bootstrapped on this architecture which definitely is great news. Maybe the system compiler will change to LLVM/Clang one day, too. Right now the SPARC64 backend for Clang is incomplete upstream at the LLVM project, if I understood things right. But we’ll see. Maybe it’ll become available in the future. I guess I’ll really have to get a newer SPARC64-based machine with a faster processor. Luckily OpenBSD supports quite a few of them.

OpenBSD on SPARC64 (6.0 to 6.5)

Earlier this year I came by an old SunFire v100 that I wrote about in my previous article. After taking a look at the hardware and the LOM, it’s time to actually do something with it! And that of course means to install an operating system first.

OpenBSD

OpenBSD, huh? Yes, I usually write about FreeBSD and that’s in fact what I tried installing on the machine first. But I ran into problems with it very early on (never even reached single user mode) and put it aside for later. Since I powered up the SunFire again last month, I needed an OS now and chose OpenBSD for the simple reason that I have it available.

First I wanted to call this article simply “OpenBSD on SPARC” – but that would have been misleading since OpenBSD used to support 32-bit SPARC processors, too. The platform was just put to rest after the 5.9 release.

OpenBSD 6.0 CD set

Version 6.0 was the last release of OpenBSD that came on CD-ROM. When I bought it, I thought that I’d never use the SPARC CD. But here was the chance! While it is an obsolete release, it comes with the cryptographic signatures to verify the next release. So the plan is to start at 6.0 as I can trust the original CDs and then update to the latest release. This will also be an opportunity to recap on some of the things that changed over the various versions.

Preparations

I had already prepared the machine for installation previously, so I only had to make a serial connection and everything was good to go. If you’re in need of doing this and don’t feel like reading the whole previous article, here’s the important steps:

  1. Attach power to go to the lom prompt
  2. Issue boot forth and then poweron to go to the loader
  3. At the ok prompt use setenv boot-device cdrom disk to set the boot order
  4. Set an alias for the CD-ROM device with nvalias cdrom /pci@1f,0/ide@d/cdrom@3,0:f
  5. Reset the machine with reset-all or powerdown and then poweron again

Booting up the OpenBSD 6.0 sparc64 CD

Insert the OpenBSD installation CD for SPARC64 and after just a moment you should be in the installation program.

Installing 6.0

OpenBSD’s installation program is very simple. It’s basically an installation script that asks the user several questions and then goes ahead and does the things required for the desired options. In the Linux world e.g. Alpine Linux does the same, and I’ve always liked that approach.

OpenBSD 6.0 installer started

On a casual installation, the script would ask for the keyboard layout. But since we’re installing over serial here, that doesn’t matter. It asks for the kind of terminal instead. Since our CPU architecture is SPARC64, OpenBSD assumes we’re using a Sun Terminal. Well, I don’t, so I choose Xterm.

Of course we need a hostname for the new system. Since it’s Puffy (the OpenBSD mascot) on SPARC here, I settled on spaffy. 😉

Choosing the root password

Next is network configuration. DHCP is fine for this test machine. Then the root password is being set.

Of course I want to access the box over SSH later, so that I don’t need the serial connection anymore and can put the machine in a different room. Compared to many x86 servers it’s not as loud as those, but still quite a bit louder than you would want a machine sitting directly next to you to be. Allowing root over SSH is very bad practice, so I create a user next and disallow remote root logins.

Selecting the partitioning

Then I choose my timezone. Next is deciding on the partitioning. There I noticed a difference compared to i386/amd64 installations. I have a habit of creating partition B first (to put the swap space on the beginning of the drive). When I tried to do this, the installer told me that this architecture didn’t allow doing that. I assume that limitation is due to Sun’s partitioning scheme VTOC that is being used on the SPARC machines. So I created them in order.

What you can see on the screenshot is OpenBSD’s default partitioning. It’s more complex than many people may be used to, but for a good reason. Remember that you can mount filesystems with different options? That way you can e.g. have /tmp mounted noexec. OpenBSD makes good use of this, e.g. enabling or disabling W^X protection on a filesystem-wide base. This is not a production machine, though, and the drive is fairly small for today’s needs. So in the end I went with a much simpler way of dividing the drive.

Selecting the distribution sets to install

Finally I need to choose what to install. OpenBSD offers so-called “sets” for various parts of the full operating system. Since I’m only installing 6.0 as a starting point, I go with the minimum required options: The kernel (bsd) and the base system.

I have no use for the install (ramdisk) kernel (bsd.rd) or the SMP-enabled multi processor kernel (bsd.mp). Also I don’t need the system compiler (comp), manpages (man) or small games (game). Of course I also don’t need the X11-related sets.

Installation finished!

Then the installer goes off and prepares everything. When it has finished, the only thing that is left is rebooting the system (and removing the CD). Now we can also change the boot order in the ok prompt, to set it to booting from disk only, speeding up the boot time minimally:

ok> setenv boot-device disk

And that’s it! Now I have an old but known good version of OpenBSD on my SunFire box.

Freshly installed OpenBSD 6.0 booted up

Updating to 6.1

Alright. What’s next? Running a 3 years old version of OpenBSD is probably not that good an idea if newer versions are available for this architecture – and they are.

So the first thing to do is fetching the ramdisk kernel of version 6.1 and the signature for it. Then I check the integrity of the kernel with signify(1). Everything is fine, so I go on and replace the standard kernel with the install kernel for the newer version. There’s probably a better way to do this, but the SPARC bootcode seems to have “bsd” as the kernel file name hard-coded and I admittedly didn’t dig very deep to figure out a different way of booting alternate kernels.

Getting 6.1 ramdisk kernel and verifying signature

After restarting, the systems boots into the install kernel. This time I select upgrade instead of install, of course. The installer then checks the existing operating system (or at least the root partition).

I then select http for the location of the sets and point the installer to a mirror that still holds the old releases.

Installer started in upgrade mode

Next is selecting the distribution sets to be installed. Again I choose only the bare minimum, since the upgrade is just an intermediary step to upgrading all the way to a current release.

In earlier versions of OpenBSD, etc was a separate set. Since the files required to check newer releases are in /etc, I’d have chosen a different installation strategy if they were still available separately. However the etc set has been included in the big base set for a while now.

Necessary sets updated

After the sets have been downloaded and extracted the upgrade is mostly complete. The remaining things are done in the live system. So it’s time to complete this step and reboot.

Configuration files get updated on first boot after the OS upgrade

OpenBSD automatically updates various configuration files for the new release. If you pay attention, you’ll see that there is one case where the changes could not be merged automatically. So we will I need to see to that myself.

The system also looked if newer firmware was available. However this was not the case (which really is no wonder on this old machine).

Merging OpenSSH config and adding installurl

After doing the manual merge of the OpenSSH configuration, it’s time to do the final tasks to complete the upgrade. OpenBSD keeps a detailed upgrade guide for each version that lists the required manual steps. In fact you should read it before doing the upgrade, since it can involve steps that need to be done prior to booting the install kernel and updating the base system! I skipped them, because they didn’t apply in my case – e.g. I hadn’t installed the manpages anyway.

I chose to only set the installurl since that one is really convenient. Actually I should remove some obsolete files from the filesystem, too. But I decided to leave this for later as there is another method to do so.

Updating to 6.2

Getting the system updated to 6.2 means repeating what I did for the 6.1 update: Get the ramdisk kernel for the new release as well as the signature and verify it. Once that’s done, another reboot is in order.

Downloading and preparing OpenBSD 6.2 install kernel

One thing that’s different is that the installer now defaults to fetching from the web and not from CD. And thanks to setting the installurl before I rebooted, it also knows the default mirror to get the sets from. Which makes the process of upgrading even more straight-forward and convenient.

OpenBSD 6.2 installer: Now knows the URL to fetch from

Finishing the upgrade after the actual unpacking of the new files takes a bit longer for this version. After making all known device nodes, the installer re-links the kernel! This is due to a new feature called KARL (Kernel Address Randomized Link). The idea here is that the objects that make up the kernel are linked in random order for each reboot, essentially creating a new and unique kernel every time. This makes it much harder or even impossible to use parts of the kernel otherwise known to be in certain memory regions for sophisticated attacks.

OpenBSD 6.2 introduced Kernel re-linking (“KARL”)

Oh, and did you notice that the bsd.mp set is gone? This machine only has a single-core CPU and therefore the SMP kernel doesn’t make much sense. The installer detected the CPU and did not offer to install the SMP kernel (even though it of course is still available for machines with multiple cores).

As always, the system needs to rebooted after the upgrade is complete. Just a moment later I’m greeted by my new OpenBSD 6.2! Again I’m skipping the manual steps to be taken afterwards.

OpenBSD 6.2 booted up

Updating to 6.3

Preparing and doing the upgrade for 6.3 is just like you’ve seen twice now, so I’m not going to repeat it. There’s one new feature in the installer that could be mentioned, though: After the upgrade is complete, the reboot option is now the default thing that the installer offers instead of just dropping you to a shell. This means you can save another 6 keystrokes when updating! Yay! 😉

OpenBSD 6.3 install kernel: Rebooting after completion is now the default choice

Updating to 6.5

The upgrade to 6.4 is simply more of the same. Of course I did that step, but I’m cutting it out here. 6.5 is the most recent release as I’m writing this (though 6.6 is already around the corner). This means I’m going to do one more upgrade, following the process that we know pretty well by now: Get and verify bsd.rd, boot it and select “Upgrade”.

Choosing all the sets except for X11-related ones for 6.5

This time I decide to install all the sets except for anything X11-related. The SunFire v100 is a server-class machine which does not even have a graphics card! For that reason there’s no VGA port to connect a monitor to, either. And while X11 could still be of some use, it’s simply not needed at all.

Upgrade to OpenBSD 6.5 complete

Again the upgrade process takes a bit longer, but that’s only thanks to the additional sets (as well as the base distribution getting a little bigger and bigger with each release). After just a little while everything is done and there’s one more reboot to make.

OpenBSD 6.5 booted up and ready

All done! I now have a fine OpenBSD 6.5 system up and running on my old SPARC64 box. And even better: Everything has been cryptographically verified to be the data that I want and no bad person has tempered with it. Sure, the system has not been cleaned up, yet – and it’s just 6.5-RELEASE with no errata fixes applied. Still I’d say: We’re off to a good start! Aren’t we?

What’s next?

In the next post I intend to explore the system a little and find out where there are differences from a common amd64 installation of OpenBSD.

How to choose your *BSD OS to begin with?

After publishing my previous post about how FreeBSD’s documentation is way better than that of Linux, I was asked which BSD I’d recommend to somebody who knows Linux but is new to the world of *BSD. This is a good question – and definitely one that cannot be answered in just a comment reply. So here’s a whole article about that topic (and in fact a rather lengthy one).

The question has been asked many times on the net and a lot of people have given different answers. Many of those recommendations are rather old now and the points that they made may no longer apply. Also it’s often BSD veterans who – of course – tend to advocate their BSD of choice. I’m a newcomer myself and so far I have not settled on one particular BSD but still try to explore all of them, their strong points and their shortcomings. If you ask me about Linux, I’m tempted to recommend the distro that I’ve been using for years now. When it comes to the BSDs this is not the case, yet, and while I do not make the claim that I’m perfectly neutral, I think that I can write something that could be of some value to people thinking about giving some BSD a try.

Dear Linux user!

Long-time Linux users will probably remember the famous “Welcome letter to a Windows user” from 2006. It’s a real classic which is outdated in some regards but the main points are still very much valid. The most important one (which applies in our case as well): If you’re interested in another operating system, it is imperative to be open-minded and to give this OS’s way of doing things a chance! It’s not a good idea at all to judge from the perspective of your old OS. Quite some things will be different. And as a matter of fact, you won’t be able to judge if certain decisions were sensible – they may seem totally weird to you but then actually make perfect sense if you know the background (which you don’t until you gain much more knowledge about the design of the system).

All BSDs are direct descendents of the original AT&T UNIX. Compared to Linux they have a much longer history and there are lots and lots of things that can probably only be explained if you go back in time far enough. While all BSDs share a common heritage they have of course been modernized over time. That is why they also have diverged quite a bit over the last two decades (and why it makes sense to write an article like this!).

One of the most important differences between the Linux world and the BSD one is that there are no “BSD distributions”. Sure, you will find some people use that term on the net but it’s most likely wrong. Most likely? Yes, because there arguably are BSD distributions… But that’s a very special case. You definitely know Debian and you probably know Gentoo. These are two Linux distros, right? Sure – but they could also be called “BSD distributions”. Why? Because there’s not only Debian GNU/Linux. The Debian project also offers Debian GNU/Hurd. And there’s Debian GNU/kFreeBSD, too! The latter is a typical Debian distribution (GNU userland) running on top of the FreeBSD kernel. For Gentoo it’s the same story. That’s where there’s something that you may call a “BSD distribution”. But that’s a rather Linuxy point of view!

While Linux is just a kernel and it is the distributions that create a full operating system by combining it with a userland (usually GNU), all BSDs follow a “whole system“ approach. There’s no such thing as “the BSD kernel”. Each BSD has its own. Fedora and Arch Linux may use different versions and configure their kernels differently, but it’s the same Linux kernel from the same sources that they use. The FreeBSD kernel and the OpenBSD kernel for example are entirely different kernels (even though they share common ancestry). It’s due to this “whole system” approach that calling e.g. NetBSD a “distribution” is plain wrong (and for the same reason BSD people will probably call the idea of a “BSD distribution” an absurdity in the first place).

What else should you know? Well, there’s a lot of little things that will be different from what you are used to. Drives and network interfaces have different names (like ada0p1 instead of sda1, em0 instead of eth0 or enp0s3). Some commands have different options and/or switches. Filesystems are different (unless you’re going to try FreeBSD and already used ZFSonLinux). The configuration works differently and you’re going to meet a different init system. You’ll probably come across a lot of things that turn out not to work like you intended. But that’s nothing horrible. You’re familiar with one unix-like system (Linux) and the BSDs are all unix-likes themselves after all!

A potential problem is drivers. If you always use the absolutely newest hardware (or pretty exotic stuff), chances are that Linux may fit you better. All of the BSD projects are much smaller than Linux is and so it is impossible for them to support as much hardware. However they are doing a fantastic job for the manpower that they have. Just try it out and see!

FreeBSD

Why should you try FreeBSD? FreeBSD is the BSD operating system with the largest community. This can be an important point for deciding which BSD to give a try. Thanks to its relatively large community it offers by far the widest range of software available. FreeBSD is the base of a lot of commercial products and appliances and for that reason it receives quite a bit of funding from the industry. That money can then be spent to fund projects that nobody volunteers for.

FreeBSD hat good support for technology like EFI. Nvidia officially supports FreeBSD and provides a closed-source driver that is meant to be on-par with the Windows driver (this can be interesting if you need good graphics performance). If you are into virtualization, FreeBSD is an excellent choice. Virtualbox 5 has been ported to FreeBSD in a community effort (Oracle does not support FreeBSD hosts). Qemu and Xen are available. And with Bhyve, FreeBSD even has its own modern hypervisor which is a really promising piece of tech! A lot of effort has been put into making the ARM platform a first class citizen with FreeBSD 11 (which is just around the corner now).

The project strives to create a general purpose operating system that is free, open source and under a permissive license. FreeBSD has put a lot of effort in getting rid of GPL’ed software in the base system (you can of course always install it as a package, though!): GCC has been replaced with LLVM/Clang (on major platforms) and a lot of the GNU tools for which no alternative implementation existed have been substituted by newly created, BSD licensed replacements. This is still work in progress, but the plan is to be able to offer a full operating system that is all permissive licensed by the time FreeBSD 12 is released.

FreeBSD’s support for technology like ZFS and DTrace has been mature for quite some time now. Other things like jails have been around like forever and are a stable part of the operating systems literally far more than a decade before “containers” became the thing on Linux. On FreeBSD you can choose between three Firewalls: A modified (e.g. multi-core optimized) variant of the mighty PF, IPFW (FreeBSD’s native firewall) and the old IPF. Also FreeBSD’s Linux emulation is a pretty sophisticated system. Don’t expect it to support all Linux syscalls up to the latest 4.7 kernels – but even though it’s limited to emulating older Linux kernels it still enables you to run some closed source software that is only available for Linux if you need that.

In general FreeBSD is often characterized as “extremely stable” and there’s definitely something to it. However this does not just mean that it’s a rock solid system, mind you! Stability is more than that. E.g. the project follows a design principle called the “POLA”: Policy of least astonishment. You won’t reboot your machine after a major upgrade just to find that your init system has been replaced with something else! You also won’t find that your firewall is inactive because the config format changed and the newer one cannot load your previous rules! It’s not too hard to see the benefit of introducing changes gently over time and with loud heads-up messages instead of quietly. When it comes to supported architectures, FreeBSD concentrates on the more popular ones.

While it’s mostly used with servers, FreeBSD makes a great desktop, too. If you want a desktop FreeBSD, you can of course install vanilla FreeBSD and then add a lot of packages for your favorite desktop environment. However there are two projects which provide a FreeBSD based desktop system (with a graphical installer, packages tuned for desktop usage, and so on): PC-BSD / TrueOS and GhostBSD. The former focuses on a modern OS with a lot of nifty tools – at the cost that it’s available for 64-bit PCs only and forces you to use ZFS. It uses the Qt toolkit and (by default) its own desktop called Lumina. If you prefer the GTK+ side of things, GhostBSD offers the MATE or Xfce desktop environments (and also supports 32-bit systems as well as the older UFS filesystem if e.g. your machine has little RAM). Both are excellent choices if you prefer to start with a complete desktop OS and explore things from there.

OpenBSD

If you choose OpenBSD, there’s a lot of really cool and modern features… that you will have to do without! Oh, don’t get me wrong: That can be a great thing!

The OpenBSD project focuses on security and correctness and for that reason take high code quality extremely serious. They reject the phenomena that is sometimes called featuritis and resist the temptation to implement dozens of “neat” or “nice to have” things at the cost of neglecting the essentials. The OpenBSD developers put quite some effort into maintaining a tidy system. Run across some ancient looking code? Time to ask on the mailing list if somebody does even use that at all. If nobody claims to do so, disable it and wait for someone screaming that it is needed after all. And if there’s any doubt in it still being useful after some time: Bye bye, feature!

Just to give you some examples which prove that those guys really mean it: Not too long Linux emulation has been completely removed because it was old and considered to be in a bad state. Just recently tmpfs was disabled due to a lack of maintenance and may very well be removed in the future as well. And of course it’s security first. Believe it or not: OpenBSD is not afraid to throw things away that pretty much every user of a modern OS takes for granted. Loadable kernel modules can always be a security problem. OpenBSD ripped them out. Usermount turned out to have security issues and – due to its nature – not to be really fixable. It’s gone now.

The project also takes a very tough stance on license matters / politics. There’s GPL(v2) code in the base system but GPLv3 is deemed completely unacceptable. As you would expect from hardliners living up to their creed, OpenBSD still ships with GCC 4.2.1 (the last GPLv2 version) in the base system (you can of course install GPLv3 software as packages)! Even though they heavily patch their compiler, it’s pretty old now and probably is a hassle to continue to maintain. Binary blobs are not accepted even if they’d make something work. Since that would not be debugable or auditable, using blobs would mean a half-assed solution. And OpenBSD follows the principle: “Do it right or don’t do it at all”. They also refuse non-disclosure contracts as a matter of principle. If some vendors choose to be pigheads, OpenBSD usually chooses not to support their hardware. Period.

While most people would probably agree that doing things right is a commendable thing, this may sound a bit too harsh for you. But think again: It’s exactly this seemingly “extremist” stance that brought great benefit to all of us, even if we don’t use OpenBSD. How’s that? The OpenBSD developers have discarded a whole lot of tools because they didn’t live up to their high standards and then took the effort to replace them with a new tool done the OpenBSD way. Tools like OpenSSH, OpenNTPD, OpenSMPD, relayd, spamd, tmux and many more originated in OpenBSD. They even created their own secure webserver (simply called httpd) since they felt that Nginx was getting too bloated over time. As the project has a much smaller community compared to FreeBSD, this really is quite an achievement.

They are also not afraid of taking quite radical steps even if that means a lot of software breaks for them. In 2014 they solved the “2038 issue” that all *nix systems have. Previously OpenBSD pioneered system wide stack protection, ASLR, W^X, etc. All these things are major changes that break a lot of applications. OpenBSD ports maintainers who noticed that some application broke due to changes like these always tried to at least notify the upstream projects or upstream patches if they already resolved the issue. This makes it much easier for other systems to also make these changes since the biggest fallout of doing so was already solved (thanks to OpenBSD).

OpenBSD supports quite a lot of architectures and purposely does so. Even the packages for the most exotic platforms are built on these machines instead of cross-compiling them on faster metal. The idea behind this is that doing so will in some cases expose subtle bugs that would have gone unnoticed otherwise.

While it maintains a rather small base system, OpenBSD still comes with a surprisingly complete set of tools for server duty. Some people say that OpenBSD makes the perfect router and thanks to the PF firewall this may very well be the case. Just don’t be too quick dismissing it for desktop usage! OpenBSD developers are known to be devotedly dogfooding their system on their laptops, servers, desktops – on their everything. If you want to use it with a graphical system, there’s a distribution set called Xenocara on the installation cd. Xenocara is a hardened version of Xorg. Should you ever need help, have a look at the manpages. They are of breathtaking quality.

NetBSD

Alright, NetBSD. This BSD does not have a big community and that community is unfortunately not particularly good at communicating the cool things that they are doing. I’ve used it much less that the two “big” BSDs mentioned before, so I might easily miss quite some information deemed relevant by a regular NetBSD user.

First and foremost NetBSD is known for its portability. Their slogan is “Of course it runs NetBSD!” and if you take a look at the quite impressive list of supported hardware, the project lives up to that. If you get into the greater *BSD community a bit, you’ll sooner or later hit the reference that NetBSD runs on just about every computer and on toasters. Don’t you think this is a joke! 😉

Perhaps even more impressive is that NetBSD is just so portable that you can actually compile NetBSD on a Linux system if you want to! And things don’t stop there. Probably the best known project that originated from NetBSD is pkgsrc (pronounced “package source”). If you already have an idea what the ports system on FreeBSD or OpenBSD is, think that plus – of course – a lot of effort added to make it highly portable. Pkgsrc can bootstrap itself on all the BSDs, Linux, Solaris, OSX, Minix, commercial UNICES (AIX, HP-UX, etc) and even Windows (using Cygwin)! What does it do? It provides three things: 1) Portable package tools 2) A portable package creation framework 3) Ports (think package “recipes”).

If you use multiple operating systems you may be tired of having to use different versions of your applications (since all operating systems or – in case of Linux – even all distributions package applications at their own pace). Pkgsrc may be the solution for you since you can use it to build and install the same version of your applications on all of your platforms! Truth be told, not every port builds on all supported operating systems. But still even having 60 – 90% (depending on which software you need and which OSes you run) is quite nice!

But there’s more to it than just portability. NetBSD has been experimenting with a few interesting things over the last few years. For example they’ve imported the small embedded programming language Lua into their kernel. In case you do not know it: Lua is used in many places since it’s very light-weight and easy to use. A lot of games embed it as their scripting language for example. Having this in the kernel could allow for a whole lot of interesting things. However I have to admit that I haven’t looked into this. Same thing for the new firewall, npf, that is part of NetBSD. It’s meant to be very high-performance but I haven’t used it and didn’t hear much about it.

The most interesting concept that I associate with NetBSD is that of the rump kernel, NetBSD’s take on the anykernel concept. If you’re not familiar with that please do some reading yourself. This topic alone would easily deserve its own article – and I’d not be the one to write it since I’m not very knowledgeable in this field.

Dragonfly BSD

Who should try out Dragonfly BSD? Well, you should have a heart for underdogs if you want to try out dfly first. It has by far the smallest community of the four main BSDs and so the chance that you’ll ever run into it in the wild is pretty low. If you don’t mind that (or this even sounds great for you) then there’s nothing wrong with giving Dragonfly BSD a try. In fact from the technical side there are very good reasons to do so!

Dragonfly BSD is the youngest of the major BSDs. It was forked from FreeBSD because its lead developer did not agree with the direction that the FreeBSD project took. Since he was interested in clustering and ultra-high performance computing, these are some of the main aspects that drive the development of Dragonfly.

The Dragonfly BSD project seems to be more interested in creating the best performing OS possible then in things like license matters. While they are looking into making LLVM/Clang their system compiler now, they had no problem importing GCC 5 into their system. For quite some time dfly used NetBSD’s pkgsrc as their official packaging tools. After FreeBSD introduced pkg-ng they switched to using that and a customized ports tree called dports.

Dfly has achieved high performance especially in networking and is the leading BSD when it comes to porting graphics drivers over from Linux. It has dropped 32 bit support and now supports 64 bit machines only. This decision was made due to the little manpower that the project has. A tool called DMA (Dragonfly Mail Agent) has originated from Dragonfly; it is a simple replacement for sendmail on servers that only need to send mail on the local machine.

However the most important reason to give Dragonfly BSD a chance is because it offers HAMMER. HAMMER is a BSD licensed next generation filesystem a bit like ZFS in some regards but different in others. While it lacks some of ZFS’s advanced features it works well even with much less memory and it can even do things that ZFS can’t (fine-grained file history)! HAMMER has been considered stable and production quality on Dragonfly for a while now so this is something to take a look at if you are into filesystems.

Well, and then there’s HAMMER2, version 2 of the HAMMER filesystem with even more features. It is not ready yet and the Dragonfly developers openly admit that it won’t be too soon, but it is interesting enough to keep an eye on (especially since version 1 of HAMMER is proof enough that the team can implement a filesystem that rocks!).

Your choice!

So let me welcome you to the rich and fascinating world of *BSD. You now have a first and very basic impression on what the four main projects “work” like. Try out either one. Try out all of them. Virtualize or use real hardware. The source will always be with you. The choice is, too.

(And please give me feedback about this article and/or tell me what your first encounter felt like after you tried out a BSD!)

OpenBSD/FreeBSD (ZFS) dual-boot & thoughts about GPT/EFI

In the previous post I wrote about how to get a computer up and running with a dual-boot of FreeBSD and OpenBSD while using full disk encryption. This worked quite well but a bit later I decided that it would be a good time to do the FreeBSD installation again – this time going the modern way and using ZFS on root. This has led to a surprisingly large amount of problems. In the end I got a working system that uses ZFS, so that I’ve actually got an alternative howto for FreeBSD (the OpenBSD part remains the same as before and can be looked up in the previous post). But it’s all a bit different from what I first thought it would be.

Doing things today’s way: GPT

GPT (GUID Partition Table) is a more modern partitioning scheme meant to replace the old MBR style partitioning, redeeming us from limitations we had to live with in the past. It can deal with drives up to 8 Zettabyte (10²¹ Bytes) instead of being limited to 2 Terabyte (10¹² Bytes). That limitation used to be no problem and it still isn’t too bad for most home users since drives bigger than 2 TB are not that cheap, yet. But it obviously won’t be too long before this will be a common issue.

OpenBSD bootloader chainloaded by boot0

A more serious limitation is that MBR only supports 4 partitions. Yes, we all used to call these “primary partitions” and made use of “extended partitions” nested inside one of them to get around that limit. BSD users created disklabels instead to embed more partitions inside disk slices (“MBR partitions”). With GPT this is trouble of the past and you are free to create just about as many partitions as you think you need. There’s no need for embedding BSD disklabels or doing any MBR trickery – which is nice.

Also GPT supports naming partitions. So by using them we can do away with the old glabel mechanism that FreeBSD offers and use native GPT labeling instead. That advantage comes with a little disadvantage, though. By default FreeBSD’s gpart shows such partitions multiple times: Once by their label and once by GPT-id. This can be a bit confusing as you first come to the GPT world. Fortunately there’s a simple solution: Setting a sysctl you can simply disable GTP-ids if you opt for labels (which you should since they are much more meaningful).

GPT is a requirement for machines that use EFI booting. The good news is that all recent x86 hardware comes with EFI (and thus GPT) support. The bad news is that while some machines do support GPT when booting from EFI, they don’t if you choose to boot from legacy BIOS emulation. So if you want to stick to BIOS you may want to check if it is capable of booting a system on a GPT partition. My EliteBook 8470p is fine with GPT partitions in BIOS mode. So I was good to give GPT a try.

Problems with GPT

FreeBSD works great with GPT (and has done so for quite some time now) so there’s nothing wrong installing it using GPT partitions. The first obstacle that I encountered was the boot manager. FreeBSD comes with the nice and simple boot0 tool that I used in my previous howto. Too bad that it’s MBR only! So to continue down the path with GPT, it’s using another boot manager. Usually GRUB is used for that purpose. And while I would certainly prefer to go without a boot manager that has “grand” in its name, I think that using it is acceptable.

The FreeBSD bootloader

A bigger problem awaits, however. OpenBSD didn’t support GPT prior to the current version 5.9. Since it does now, everything should be fine, right? Wrong. If I didn’t miss anything, OpenBSD supports GPT only in conjunction with EFI booting! I did not find a way to use GPT partitions with OpenBSD in BIOS mode. Should anybody have more information on this, I’d very much like to know if it either is possible or if support for this is planned for the future.

Choosing EFI?

This means that not too far down the road there’s already a solid blocker. I had next to no knowledge about the EFI complex and I successfully avoided that topic in the past. So my two options to go on were to either give up on GPT and simply stick with MBR or to make the bold move and go for EFI. I would prefer to try out things one after another but meh… I decided to read a bit about the basics of EFI and then give it a try. To be honest, I don’t like what I read too much. Sure, there are quite a few interesting things that EFI can do. But then again I do not believe in unneccessary complexity. And above all: I don’t like the security implications of it. Not a bit. Trust is not something that I give away for free. Trust has to be earned. Unfortunately closed source vendors have done very little in the past that makes me think I want to trust them. Anyway… EFI is the (near?) future and it will not be easy to avoid it altogether in the next few years.

Alright. So I turned off legacy BIOS emulation on my machine and booted FreeBSD (there are separate EFI images for FreeBSD 10.3 – make sure you pick one of those if you want to go with EFI!) into the installer. Everything worked smoothly and after a couple of minutes my new FreeBSD system had booted. It was so simple and dull that there’s not too much I could write about (which is a good thing since everything really just worked).

Next step: Installing OpenBSD. I read that the memstick image supports EFI booting and I can confirm that installing works just as well as with FreeBSD. Again the new system works like a charm – the OpenBSD people have obviously done a good job for 5.9. OpenBSD was able to cope with GPT just like you would expect.

FreeBSD desktop: EDE, pcmanfm, terminator, smplayer

So far everything looked good. Final step: Make the computer offer a means of booting either system! Honestly I did not expect that step to be a show-stopper. It turned out that it is. Most people seem to use the EFI boot manager rEFInd. You can install it from Windows, OS X and Linux. Yes, that’s it. Now, I was not surprised that it was not ported to OpenBSD. But it did surprise me that it’s not available for FreeBSD!

It may be a quite simple thing to toss in a Linux CD and install an EFI bootloader – or maybe not. I have no clue if there are any other obstacles waiting. But this was the point where I stopped. I wanted a dual-boot BSD system and I don’t want to have to use Linux to do that. I’ve got my pride, too, you know. :p No seriously, at that point I decided to give up EFI for now and continue another time. Maybe that will be worth its own blog post. Who knows.

Back to BIOS / MBR… for now

Here we are, back to using BIOS / MBR. I wanted to use a purely ZFS setup for FreeBSD but another problem showed up: The bootcode to load a kernel from ZFS is quite a bit more complicated than its UFS equivalent. For that reason it doesn’t fit into the boot sector of a partition but needs its own small partition instead. However with gpart the type freebsd-boot seems to be only supported for GPT…

To get anything up and running again (for the time being my on-call laptop was broken after all and my next duty was around the corner!) I settled with a UFS boot partition for the unencrypted /boot partition.

Manual install: FreeBSD with ZFS, GELI

The auto_ashift adjustment is needed for ZFS to adhere the 4k alignment. And to be able to set that sysctl, the ZFS kernel module has to be loaded. That’s a little detail but it took me a moment to figure out what’s going on. Everybody advices to set the sysctl but the installer kept on telling me that there was no such sysctl… Which makes sense once you know that ZFS is not loaded into the kernel by default.

For the layout of the datasets I followed the defaults as used by the automated root-on-ZFS installation (zpool history is a very interesting command! If you don’t know it – try it!). I just changed the order so that similar datasets follow each other which avoids a bit of typing by bringing back the previous command and editing it.

It’s more likely than not that this is not the best way to do things. But it does work. Any suggestions or other comments are welcome of course!


In the partitioning shell:
tcsh
dd if=/dev/zero of=/dev/ada0 bs=1m
gpart create -s mbr ada0
gpart add -a 4k -t freebsd -s 98G ada0
gpart add -a 4k -t freebsd ada0
gpart create -s bsd ada0s1
gpart bootcode -b /boot/boot0 ada0
gpart bootcode -b /boot/boot ada0s1
gpart set -a active -i 1 ada0
gpart add -t freebsd-ufs -s 2G ada0s1
gpart add -t freebsd-swap -s 4G ada0s1
gpart add -t freebsd-zfs ada0s1
glabel label clear /dev/ada0s1a
glabel label swap /dev/ada0s1b
glabel label system /dev/ada0s1d
newfs /dev/label/clear
dd if=/dev/random of=/dev/label/system bs=1m
geli init -b -s 4096 -l 256 /dev/label/system
geli attach /dev/label/system
kldload zfs
sysctl vfs.zfs.min_auto_ashift=12
zpool create -o altroot=/mnt -O compress=lz4 -O \
atime=off -m none -f zroot /dev/label/system.eli
zfs create -o mountpoint=none zroot/ROOT
zfs create -o mountpoint=/ zroot/ROOT/default
zfs create -o mountpoint=/usr -o canmount=off zroot/usr
zfs create -o mountpoint=/var -o canmount=off zroot/var
zfs create -o mountpoint=/tmp -o exec=on -o setuid=off \
zroot/tmp
zfs create zroot/usr/home
zfs create zroot/usr/src
zfs create -o setuid=off zroot/usr/ports
zfs create -o setuid=off zroot/var/tmp
zfs create -o exec=off -o setuid=off zroot/var/audit
zfs create -o exec=off -o setuid=off zroot/var/crash
zfs create -o exec=off -o setuid=off zroot/var/log
zfs create -o atime=on zroot/var/mail
zfs set mountpoint=/zroot zroot
exit
exit

In the "final modifications" chroot:
mkdir /realboot
mount /dev/label/clear /realboot
mv /boot /realboot
ln -s /realboot/boot /boot
echo 'geom_eli_load="YES"' >> /boot/loader.conf
echo 'zfs_load="YES"' >> /boot/loader.conf
echo 'vfs.root.mountfrom="zfs:zroot/ROOT/default"' > \
/boot/loader.conf
echo '/dev/label/swap.eli none swap sw 0 0' >> \
/etc/fstab
echo '/dev/label/clear /realboot ufs rw 1 1' >> \
/etc/fstab
sysrc zfs_enable="YES"

Setting up a FreeBSD/OpenBSD dual-boot with full disk encryption

A bit over a month ago, I bought my first refurbished laptop. Previously I used a ThinkPad (owned by the company I work for) for on-call duty. It’s running a Linux distro which would not be my first choice at all, it has a small screen and – it’s not my property. I wanted my own laptop and since we’re allowed to use whatever distro we prefer, I thought that I’d be going with Arch.

(I you’re just interested in the commands to enter, have a look at the end of this post where I put a list of them.)

*BSD in production

On a second thought: Why not use *BSD? For me it would mean going to use a *BSD desktop “in production” after only running it privately. Thanks to the great BSDNow! show I feel confident enough now to give it a try. The company that I work for is running some FreeBSD servers, too, so it’s not something entirely strange and unknown. I went with asking if using BSD for on-call was ok. The answer was what I expected: If I thought that it would work ok I should well try it. The only requirement was that I’d encrypt the disk (the same rule would apply to Linux, too, of course).

Next question: Which BSD to use? Since I’m just getting into *BSD, I’m not really familiar with all of them now. Net and Dragonfly would certainly be interesting, but since I need that box for work that’s not an option. I need something that I know enough to be able to work with. Of course it would be best if I could learn something at the same time… So, what’s the best way to learn more? Probably tracking -CURRENT! But what if something breaks? I cannot afford that. And which BSD to use anyway? I work with some FreeBSD servers, so more in-depth FreeBSD knowledge would make sense. Then again I’ve really come to like Puffy and all he stands for…

That would be a hard decision! Finally I decided not to decide – and to just install both instead. This also has the advantage of having a second system if either CURRENT should ever break!

Hardware: HP EliteBook

I bought an HP EliteBook 8470p. Why didn’t I go with Lenovo even though those are known to work best with *BSD and I obviously need something that seriously works? Well, there’s one reason for me: With the ThinkPads keyboards just totally suck. I have no idea who came up with that sad story of “Hey, let’s just put the Fn key where Ctrl belongs and vice versa!”. No idea whatsoever. But I know for sure that it drives me insane. No fun at all when you’re working on-call at four AM, barely awake, and nothing happens when you have to CTRL-C something quickly. I could never get used to it ever!

So for that very reason it had to be some other hardware. I had this older HP laptop that a friend sold me for a few bucks a while ago. I can’t remember which model exactly and cannot look it up since I don’t have it anymore. (When my mother’s old computer died as I was over on a visit, my father thought about replacing it with a Windows box since that’s the only thing that he knows. To avoid that, I set up said old HP laptop that I had with me as a replacement and gave it to her. She’s been using it happily since.) That laptop had been a pleasant experience when I had OpenBSD on it and so I decided to give that EliteBook a try.

It works fairly well for most things. On FreeBSD there was the problem with the Intel video driver but since I’m running 11-CURRENT video is all working great even when I quit X11. WiFi is detected according to dmesg but for some reason no iwn0 shows up if I run ifconfig. I didn’t have time to look into that further, however. On OpenBSD backlight gets turned off if I quit X and thus the screen is a bit dark then. Since I usually quit X to shut down the computer afterwards, anyway, that’s only a minor issue. WiFi is correctly detected and I confirmed it to work. Suspend works when I close the laptop but when it wakes up the keyboard does not work anymore. These are the only issues that I ran into so far.

What is the exact use case?

FreeBSD can use ZFS while OpenBSD cannot. I’m not sure if FreeBSD’s and OpenBSD’s UFS/FFS filesystems are compatible (I think OpenBSD’s implementation misses quite some of the newer features). The encryption methods used by the systems however are definitely not compatible. So it doesn’t matter anyway in this case and I’m free to choose whichever filesystem I want.

Since I’ll be compiling FreeBSD-CURRENT now and then (and in general plan to do some stuff that likes much memory to be available), I decided to go with UFS. Yes, there are scenarios where ZFS is simply overkill! There’s only one drive in the laptop, it’s not extremely big and it won’t hold any important data. I have no need any particular ZFS feature on that system, so going with UFS should be fine. (That plus the fact that I’m still reading Lucas’ and Jude’s excellent book on ZFS and intend to play with that filesystem on another machine)

Prior to version 5.9 (released after I originally wrote this), OpenBSD only really supported the MBR partitioning scheme so going with that was an easy choice. I’ll stick to it for now because I need some time to play with it first. I’m going to do everything again in a VM so I can take screenshots for this article.

Installing FreeBSD

The installation begins just like an ordinary FreeBSD install: Boot up the installer media and make your way through the setup questions. When the installer asks about the partitioning however, we’re going to do that by hand.

Choosing to partition by hand

The pure bourne shell is not very comfortable for interactive use, so it generally makes sense to use a more advanced shell (like tcsh) for convenience features like auto-completion. Should you not know which drives your machine has, camcontrol can help you. If you want to start with a clean drive, you can zero out everything with dd (when I bought my laptop it had Windows 7 on it that I wanted to get rid of).

Zeroing out the disk

If you’re not familiar with what partitions and slices are, you may want to have a look at an older post where I wrote up a little excursion about that topic.

First an MBR is created and then two slices are added to it. The first one gets 100 gigabytes, the other the rest (which is also 100 GB in my case). Both slices are created aligned to 4k sector size of the hard drive. Then a BSD disklabel is added in the first slice. After that, boot0 (a simple boot manager) is written on the drive and the standard bootcode into the first slice. Finally the first slice is marked as active for booting.

Slicing the disk

Now three partitions are created inside the BSD label: One for boot (which will hold the kernel and cannot be encrypted), one for swap and one for the system (which will be encrypted). Glabel is used to give these partitions a more meaningful name than ada0s1a and the like. Since the system partition will be encrypted, it makes sense to write some garbage all across it so that it is impossible to see which part holds data and which does not. This takes quite a while and you could of course skip this. As long as your patience lives up to paranoia, that little bit of extra security is worth the wait!

Creating and labeling BSD partitions

Next the system partition is initialized with GELI, one of FreeBSD’s two military grade encryption methods. I only use a passphrase to unlock but you can also use a key (or both) if you wish. After attaching the new GELI partition, a new GEOM provider, system.eli is available with the clear data for you (and your programs) to use.

Creating and attaching the GELI partition

Now it’s time to format the two data partitions (the swap partition does not need any formating). You could also use journaling UFS for the boot partition but it’s usually not necessary.

Creating filesystems

Copy over the boot directory and add two lines to loader.conf so that you’ll have the chance to unlock your GELI partition during system startup. What remains is writing a fstab. Notice that for some reason I’ve forgotten to put swap.eli in there on my screenshot (even though that’s what I have in my script). What this does is using a one-time key for your swap on each boot, thus making sure that any data that remains on the swap partition is useless once the system was powered down once. You do not have to initialize GELI for this. FreeBSD knows what to do when it sees swap.eli.

Mount the decrypted system partition on /mnt as that’s where the installer expects it. And don’t forget to create the clear directory as we demand that in fstab and the system would not boot up correctly if it was missing. Then exit the shell and continue with the installer.

Copying over /boot and writing loader.conf and fstab

Once the installation has finished, the installer will ask you if you wish to make any final modifications. Answer yes and it will drop you into a shell in a chroot of your new system. Delete /boot (that directory lives on the encrypted system partition and the bootloader could not find the kernel there anyway) and make it a symlink pointing to /clear/boot instead. This step is not actually required. But if you don’t do it, you won’t be able to update your system the normal way. If you want to only mount the real /boot by hand whenever you upgrade, that’s fine, too, of course.

Chosing to make final modifications

Exit the shell, reboot and remove the boot media. Then reboot. Your boot manager (boot0) will offer you two FreeBSD systems. Hit F1 to boot up FreeBSD. Don’t hit F2. There’s no system there, yet.

Installing OpenBSD

The OpenBSD installer is neither pretty nor does it offer any kind of menu system. However it is simple, effective and straight-forward. Choose to install OpenBSD, set your keymap, enter a hostname, configure the net and set a root password.

Hostname, network and password configuration

Choose to run an SSH server by default, whether to prepare the system for X11, if you want the display manager XDM to be started automatically. Create a user now or do so later. When asked for the timezone, give a ! instead to drop into a shell.

Going to a shell

If you don’t know your disks, look inside the dmesg for the name. Now use fdisk to change the type of the second partition from A5 (FreeBSD) to A6 (OpenBSD). Then use disklabel to create a swap partition and a main partition. Make absolutely sure that the later has the type RAID!

Partitioning for OpenBSD

Encrypt the new softraid with bioctl then exit the shell. Now enter the correct timezone and choose the newly created softraid for the installation! Dedicate the whole softraid disk to OpenBSD but edit the partitions to fit your need. You do not need a swap partition on the softraid because we created a separat one on the real disk, remember? For that reason, after OpenBSD formated the partitions you created, the installer will ask you if you want to add any other disks before you start the actual installation. You DO because there’s the swap area.

Preparing crypto softraid

Once the installer has finished, reboot the machine. Now the boot manager says “F1 – FreeBSD” and “F2 – BSD”. The second one is your OpenBSD. The manager knows only the partition type and has no clue which system is on there.

Plain text summary

Here’s what you could type in for the shell parts of both installers:

FreeBSD


In the partitioning shell:
tcsh
dd if=/dev/zero of=/dev/ada0 bs=1m
gpart create -s mbr ada0
gpart add -a 4k -t freebsd -s 98G ada0
gpart add -a 4k -t freebsd ada0
gpart create -s bsd ada0s1
gpart bootcode -b /boot/boot0 ada0
gpart bootcode -b /boot/boot ada0s1
gpart set -a active -i 1 ada0
gpart add -t freebsd-ufs -s 2G ada0s1
gpart add -t freebsd-swap -s 4G ada0s1
gpart add -t freebsd-ufs ada0s1
glabel label clear /dev/ada0s1a
glabel label swap /dev/ada0s1b
glabel label system /dev/ada0s1d
dd if=/dev/random of=/dev/label/system bs=1m
geli init -b -s 4096 -l 256 /dev/label/system
geli attach /dev/label/system
newfs /dev/label/clear
newfs -j /dev/label/system.eli
mount /dev/label/clear /media
cp -Rp /boot /media
echo 'vfs.root.mountfrom="ufs:/dev/label/system.eli"' >> /media/boot/loader.conf
echo 'geom_eli_load="YES"' >> /media/boot/loader.conf
echo '/dev/label/system.eli / ufs rw 1 1' >> /tmp/bsdinstall_etc/fstab
echo '/dev/label/swap.eli none swap sw 0 0' >> /tmp/bsdinstall_etc/fstab
echo '/dev/label/clear /clear ufs rw 1 1' >> /tmp/bsdinstall_etc/fstab
mount /dev/label/system.eli /mnt
mkdir /mnt/clear
exit
exit

In the "final modifications" chroot:

rm -r /boot
ln -s /clear/boot /mnt/boot

OpenBSD


i
de
puffy
em0
dhcp
none
done
password
no
yes
no
no
!
dmesg | grep [ws]d0
fdisk -e sd0
setpid 1
A6
quit
disklabel -E sd0
a b
ENTER
4G
swap
a a
ENTER
ENTER
RAID
w
q
bioctl -c C -l /dev/sd0a softraid0
exit
Europe/Berlin
sd1
whole
e
Your layout here
w
q
sd0
OpenBSD
w
q
done
http
none
openbsd.cs.fau.de
pub/OpenBSD/5.9/amd64
done
done