Building a BSD home router (pt. 5): Installing OPNsense

Part 1 of this article series was about why you want to build your own router, and how to assemble the APU2 that I chose as the hardware to build this on. Part 2 gave some Unix history and explained what a serial console is. Part 3 demonstrated serial access to the APU and showed how to update its firmware. The previous article detailed installing pfSense.

This post will show how to install OPNsense, a great alternative to pfSense.

Preparation

OPNsense was forked from pfSense (more on than in the next post) and thus you will find lots of similarities if you have read the post on installing pfSense. The OPNsense team decided to move forward more quickly and did lots of interesting but invasive changes. One strong point for example is that it is already based on FreeBSD 11.0. There is one drawback to this, however: a problem with the XHCI (USB3) driver can lead to the installation media not being able to mount the filesystem and boot up. This makes installing OPNsense a little bit more complicated since the APU2 only has UBS3 ports.

Well, the board does have an internal USB2 controller, too. Therefore I suggest getting a cable that allows connecting USB devices to it. If this is not for you, take a look at the end of the post, I’ve prepared a section “alternative installation methods” there.

First download an image (select amd64 + serial). Then dd it onto an unused memstick and prepare the serial connection (take a look at the previous posts if you need help with dd’ing or attaching the serial console).

Open APU2 box with serial connection and memstick attached to the internal USB2 controller

As you can see, I’ve attached a memstick with OPNsense via USB2 and made a serial connection. That way the installation works just fine.

Step 1: Installation

Hit F10 to go to the boot menu as soon as SeaBIOS offers it.

Boot menu to select which device to boot off of

Since we’ve attached the memstick over USB2, the internal drive would take precedence over it in the default boot order. So in this case I have to select 2 to boot off of the memstick.

The OPNsense boot loader

The OPNsense boot loader looks fine. If you’re installing 17.1 using USB2 you don’t need to do anything here.

Nice feature: Early configuration importer

One notable difference from pfSense is the early configuration importer. If you have a saved configuration XML file, you can put e.g. a UFS2 filesystem on a memstick, create a directory conf on it and copy config.xml there. That makes it available in the importer.

Interface assignment

Then you have the option to assign roles to your interfaces (like WAN and LAN).

Logging into the installer

OPNsense gives you the choice to start the installer or to use a live system. Log in as user installer to perform an installation or as root in the other case. The password for both users is opnsense.

Greeting screen of the installer on the serial console

The OPNsense installer is black and white only when using the console. But that’s fine. The installer greets you with the welcome message.

Console configuration menu

The next screen lets you customize the console. You probably don’t need to do that.

Selecting the installation type

Then you need to select the installation type. You could do advanced partitioning here or setup a softraid (gmirror). We’re going with the simple installation for this post.

Choosing the drive to install on

Now you need to choose where to install to. The mSATA drive is ada0 whereas the memstick is da0.

Selecting the partition scheme to use

OPNsense also lets you choose which partition scheme to use. In case of our router this is not terribly important, especially not with our sample installation that puts everything in one partition. But since stone age is over, you might as well choose GPT anyway.

Progress bar for the installation

While the progress meter was broken with pfSense, this has obviously been fixed for OPNsense. Not that you should reinstall all that often, but still…

Installation done: Reboot!

Once the installation is finished, you of course want to reboot to your new system.

Displaying some information before rebooting

Before rebooting, OPNsense tells you how to access the Web GUI. However the IP address that it uses by default is already taken by my ISP’s modem/router box. We’re going to change that next.

Step 2: Text mode configuration

When the system has started up, you are prompted to log in. This is the default behavior which can be changed to allow unprotected login over the console like with pfSense. But in general I like that bit of extra security.

OPNsense’s text-mode configuration menu

The text-mode configuration menu looks much like that of pfSense.

Configuring the LAN interface

And the interface configuration works right the same.

Setting up DHCP on the LAN interface

As does the DHCP configuration.

Logging out and disconnecting the serial console

Since OPNsense required a login, you can also log out when you’re done. Now disconnect the serial console – we’re done with it.

Step 3: Web GUI configuration

Just like pfSense, OPNsense offers a nice Web GUI to configure all the settings. Fire up your browser on a PC that is in the same subnet (or got its IP address via DHCP from the new router) and enter the router’s LAN IP address in the URL bar.

Self-signed certificate warning

OPNsense uses https to create a secure connection, too. Of course a self-signed certificate is used which is not trusted by my Firefox. Therefore a permanent exception needs to be made.

OPNsense Web GUI login screen

Once you have confirmed the exception, you will see the login screen. Log in as root with the password opnsense.

The configuration wizard

On the first login you will be greeted by the configuration wizard. It will present you about the same choices as pfSense does (without the advertizing of the commercial version, of course).

Configuring general settings

First it’s some general information like hostname and DNS. What OPNsense offers over pfSense is i18n options: Chances are that you can configure the Web GUI to speak your language! That’s pretty nice.

Configuring time-server settings

Time server settings are just like those from pfSense.

Configuring the WAN interface

WAN configuration offers you a lot of options. Take a close look at those. Fortunately you very likely don’t need most of what is there.

Configuring the LAN interface

Same thing for the LAN configuration: You know that from pfSense.

Setting a new password for the Web GUI

Also with the password changing part there’s no surprise here.

All done. Reload the config!

That’s it. Reload the config now and you’re done with the wizard. OPNsense now has a basic configuration and is ready to be used.

Alternative installation methods

OK, you don’t have a cable to connect to the USB2 pins but you want OPNsense? There are several things that you can try. I’ve documented my attempts (including several solutions) on the OPNsense forums in case anybody needs them.

Here are a few things that you can try:

  • Install from SD card (I didn’t try that but it should indeed work)
  • Install 16.7 from USB3 with increase boot_delay and then update
  • Install 17.1 using a USB cdrom, manually enabling the console and importing a pre-made configuration

Should you install 16.7 using a USB3 port, press ESC before the loader countdown runs out. This will drop you to the loader prompt. Then enter the following:

set kern.cam.boot_delay=10000
boot

That did the trick and made the system boot up for me. The actual installation is quite similar to what I covered above.

You could also use a USB cdrom to boot the installation – of course use the OPNsense cdrom ISO in this case! However the cdrom image does not have the serial console enabled by default. So escape to the loader prompt, set some variables to enable the serial console and boot:

set boot_multicons=YES
set boot_serial=YES
set comconsole_speed=115200
set console=comconsole,vidconsole

This will work, too. But there’s one little problem with that: The TTYs are configured on their own using a configuration file – and they are not ready for serial connection! Since this is a CD, we cannot really do much about that. What we can do, however, is using the configuration importer. I will upload a basic configuration xml and add it to this post when I next install a clean OPNsense.

What’s next?

The next post will be pfSense vs. OPNsense! It will discuss some of the notable differences and when to use which one.

Advertisements

Building a BSD home router (pt. 4): Installing pfSense

Part 1 of this article series was about why you want to build your own router, and how to assemble the APU2 that I chose as the hardware to build this on. Part 2 gave some Unix history and explained what a serial console is. Part 3 demonstrated serial access to the APU and showed how to update its firmware.

This post is about the serial installation of pfSense, one of two FreeBSD-based router/firewall operating systems that we’re going to explore in this series (the other being OPNsense). As pfSense is the older and more established product, we’re beginning with that one.

Preparation

We’re just doing the installation here. A closer look at using pfSense or a comparison with OPNsense will be another post. Getting pfSense up and running is really easy, even when you’re using the serial console. The first step is the actual installation. In a second step you need to configure the LAN interface and then you can use the WebGUI to do the final setup.

The first thing to do, however, is getting preparing an installation medium. Head over to pfSense’s Download site. What you want is an install image for amd64. Then select USB Memstick Installer which let’s you choose the console type – obviously get the serial one!

Then get a USB stick that you can spare and dd the image on it. Once you have that ready, plug it into the APU. Next attach the serial cable to your APU and to another computer. Then connect to the console (how to do that was described in the previous post). Now power on the APU.

Step 1: Installation

Even if there’s already an OS installed on your mSATA drive, the memstick should take precedence when it comes to boot order. So you can probably just wait until the installer comes up.

pfSense’s loader menu: screwed up over the serial console…

Don’t be scared when you see garbage displayed on the screen. This is just the bootloader that’s screwed up badly when used over a serial connection (they’ve already fixed that in the beta version for the upcoming pfSense 2.4). Either just wait 10 seconds for it to boot automatically or press enter to boot right now (if you need any other options, you might want to get an ISO for pfSense, too, and test it in a VM or get a VGA image, put that on a stick and try it out on hardware that provides a local console over a screen and keyboard).

…but once the kernel loads, text is fine

As you can see, it’s only the loader. As soon as the kernel takes over, the text is displayed correctly. That means you can actually read the messages in case anything goes wrong here. If you don’t do anything, the installer will eventually come up automatically.

First screen of the installer

In the first screen of the installer you can configure the console. Most likely the defaults will be fine, though.

Selecting the installation method

Then you need to choose the installation method. We will do a quick installation but you could also do a custom installation or setup gmirror (mirrored software RAID).

The usual “this will erase your data” warning

Since installing pfSense means destroying any data that might currently be on the drive, the installer warns you that it will erase it.

Installation progress bar

If you confirmed the warning, the actual installation starts (but the progress meter is kind of useless as it seems… It remained at 5% for a while and then jumped to 100% for me).

Kernel selection

The next thing to do is to select the right kernel. Since our APU2 is a headless device, make sure that you select the embedded kernel! Otherwise you won’t be able to use the serial console with it.

Another progress bar

After the kernel is installed, the installer runs a script to do some final tasks.

Reboot message

When all is done, it’s time to reboot the system.

pfSense rebooting after installation

Just before it reboots, pfSense prints some important information on the screen, telling how to log into the WebGUI. Remove the memstick now or the APU will boot off of it once more an you’ll just see the installer again.

Step 2: Text mode configuration

The OS has been successfully installed, but leave your serial console attached for now.

pfSense’s text mode management menu

Once the system has booted, you will see the management menu. It offers a lot of tools including going to a shell (option 8) and doing everything you like. We want to configure the IP address for our LAN interface (option 2):

Configuring the LAN interface

I’m assigning 192.168.2.1 since my modem/router (yes, I’m not replacing it just yet and will operate the new router between that box and my actual network for now) has already taken 192.168.1.1. It’s not like I need a full /24 subnet for my network, but I go with that subnet mask for now.

Configuring DHCP for the LAN interface

Since I intend to use DHCP for my network, I enable a DHCP server for the LAN interface. The range of DHCP addresses that I use here is just an example for this test installation. I will cut it down to about 10 when I do my final setup. The reserved addresses before the DHCP range serve a purpose, though – more on that in a separate future post.

Back at the menu

As soon as everything is ready, you can now end the serial connection and remove the cable. We have a valid IP address on the LAN interface now after all.

Step 3: WebGUI configuration

So now we can access the WebGUI simply by entering the IP address in the URL bar of any browser. Of course the computer that runs the browser have an IP address that is on the same subnet. So you might want to change your address if that is not the case – or fire up the dhclient, it should get an address in the range that you specified (or simply reboot if your computer is configured for DHCP).

Self-signed certification warning

It’s a good thing that pfSense uses TLS so you can access the router securely via https. However the certificate it uses is self-signed and thus unknown to your browser which will display a warning. That doesn’t mean that it’s useless. In our case it’s just necessary to create an exception to accept that cert permanently.

Logging into pfSense’s WebGUI

You’ll then see the login screen. Use the username admin and the password pfsense to log in.

Running the configuration wizard

Once you’re logged in, pfSense suggests that you run the configuration wizard – and that makes sense.

A little advertising for pfSense Gold

The first screen of the wizard is an advertisement for the commercial version of pfSense called pfSense gold. If you are a company looking for more than the free “Community Edition” of pfSense will give you, have a look at this service. Maybe it’s for you.

General information configuration

First you configure some general settings like the hostname, domain, etc.

Time Server configuration

Next is the configuration of the time zone and NTP daemon.

WAN configuration

Then the WAN interface needs to be configured. There are a lot of settings there and very likely you don’t need all of them.

LAN configuration

After that comes the LAN interface. Here you can only configure the IP address and subnet mask (which we already did in text mode).

Changing the password for the WebGUI

Finally we’re prompted to change the password which is a good idea of course. Even if the WebGUI is only accessible from the LAN interface by default, it’s a matter of principle.

Configuration done: Reload!

That’s it, the wizard is finished. Time to reload the configuration.

All done, pfSense is ready

We’re done here, pfSense is installed and the basic configuration has been applied. There’s another little advertising here which is legit for a free product, I guess. We’re going to take a look at the main WebGUI and its many, many options in another post.

What’s next?

The next blog post will detail the installation of OPNsense, another excellent option for your router.

Building a BSD home router (pt. 3): Serial access and flashing the firmware

Part 1 of this article series was about why you want to build your own router, and how to assemble the APU2 that I chose as the hardware to build this on. Part 2 gave some Unix history and explained what a serial console is.

In this post we will prepare a USB memstick to update the BIOS and connect to the APU2 using the serial console. Then we’ll flash the latest firmware on there.

Cables and serial connections

In the ol’ days you would simply connect the COM port on one machine to the COM port on the other. Today a lot of newer laptops don’t even have a serial port (if yours still has one of those funny devices that you’d access through /dev/fd0, chances are pretty high though, that it also has a COM port!). Fortunately USB to serial adapter cables exist, solving that problem.

The APU2 has a male DB9 (9 pins) serial port. RS-232 is the common standard for serial communication. According to it, some pins are used to transfer information while others are used to receive information. Now if you connect two machines with a straight serial cable, both will talk on the same pins and listen on the same pins. So both will send data over pins that nobody listens on and never receive anything on the other pins. This is not really useful. To make the connection work, you need a crossed-over cable (a so-called null modem cable or adapter). This means that the receiving pins on one end are paired with transmitting pins on the other end and vice versa.

I thought that I would never need a nullmodem cable at home. I still don’t think that I might ever need a straight serial cable. And I could in fact take one home from work and return it the other day. However I could already see what would have happened in that case: When I get some time to tinker with my APU it will be on the weekend and I won’t have the cable in reach when I need it. So I got my own. And while I was at it, I decided to not only get a USB to RS-232 DB9 serial adapter cable (look for the PL2303 chipset translating USB to serial: It’s well supported across a wide range of operating systems including FreeBSD). I also bought a null modem adapter and a gender changer. So now I’m completely flexible with my gear. However you probably just want to get a null modem cable USB/female DB9 (or ask somebody who has one if you could borrow it).

Another thing that you have to know is the baud rate (modulation rate) for your connection. A higher baud rate means a faster connection. As long as both connected machines agree on the baud rate, everything is fine. If they disagree, this can lead to displaying garbage instead of the actual console or in seemingly nothing happening at all.

To flash or not to flash

At the time of this writing, PC Engines have released 5 updates for the APU2’s firmware and if you like the improvements, it makes sense to put the newest version on there. They recommend booting TinyCore Linux and then using flashrom to flash the BIOS.

Flashrom is available for FreeBSD, too. I would imagine that it works as well. However I have no experience with that and flashing stuff always bears the risk of bricking your device (for which case PC Engines offers a small rescue device). If I had thought about this right at the beginning, I would probably have tried it out. But my APU2 is already updated and since this post is public and not just for me… Well, let’s just do this by the book and use Linux for that.

If you’re a little anxious and don’t feel well about flashing at all, leave it be; in general the old BIOS will do, too. Flashing according to a guide does not have a high risk of bricking your device but even a small risk is a risk. Disclaimer: You decide. And it’s all your responsibility, of course.

Alright, we need to prepare a USB stick with TinyCore and the ROM on it. PC Engines even offer a howto guide showing how to create this using FreeBSD. That guide works, but it clearly shows that those guys know Linux a lot better than (modern) FreeBSD. For that reason I’m going to modify it slightly here and use today’s tools.

Preparing the BIOS updater memstick

First we’re going to download the TinyCore tarball and the zipped ROM (you might want to take a look if a newer version than shown here is out) and install the syslinux package (containing the that Linux bootloader):

% su -
# mkdir -p /tmp/apu2 && cd /tmp/apu2
# fetch http://pcengines.ch/file/apu_tinycore.tar.bz2 http://pcengines.ch/file/apu2_v4.0.7.rom.zip
# pkg install syslinux

Now attach your USB memstick to the system and after a couple of seconds look at the dmesg if you don’t know what device it will be attached to:

# dmesg | tail
[...]
da0 at umass-sim0 bus 0 scbus2 target 0 lun 0
da0: < USB DISK 2.0 PMAP> Removable Direct Access SPC-4 SCSI device
[...]

So for me it’s da0 in this case and I need to supplement the “X” that I use in the following commands with “0”. You should use whatever number you figured out. PC Engine’s howto suggests zeroing out the stick and if it isn’t a new and unused one that makes sense. In my case this leads to an error:

# dd if=/dev/zero of=/dev/daX bs=1m
dd: /dev/da0: Operation not supported

# mount
[...]
/dev/da0s1 on /media/disk (ext2fs, local, nosuid)

Looks like this is because GhostBSD automatically mounted da0s1 when it found an EXT2 filesystem from a previous task in there. So let’s unmount that and try again:

# umount /media/disk
# dd if=/dev/zero of=/dev/daX bs=1m

Depending on the speed and size of your memstick (and the generation of your USB port) this can take quite some time to finish. And since dd normally is working quietly, you might want to know how far it came so far. Fortunately FreeBSD implements the SIGINFO signal. Just press CTRL+T while it is running and it print some status information like this:

load: 1.52  cmd: dd 92084 [physwr] 379.15r 0.00u 0.56s 0% 3188k
2512+0 records in
2512+0 records out
2634022912 bytes transferred in 379.208179 secs (6946113 bytes/sec)

When it’s done, we can create an MBR partitioning scheme on the now empty stick as well as an MBR partition and write the bootcode on the stick so that we can boot off of it at all:

# gpart create -s mbr daX
da0 created
# gpart add -t fat32 daX
da0s1 added
# gpart bootcode -b /boot/boot0 daX
bootcode written to da0

The next thing to do is putting a FAT32 filesystem on the partition and force install the Syslinux bootloader there that will be chainloaded by the bootcode that we wrote into the MBR.

# newfs_msdos /dev/daXs1
newfs_msdos: trim 40 sectors to adjust to a multiple of 63
[...]
# syslinux -if /dev/daXs1

Now we need to mount the new filesystem and put the OS on there:

# mount -t msdosfs /dev/daXs1 /mnt
# tar xjf apu_tinycore.tar.bz2 --no-same-owner -C /mnt

We’re writing to a FAT filesystem – and as the primary filesystem from a once popular single user OS it does simply not support concepts like file ownership. That’s why we need “–no-same-owner” here (otherwise we’d see harmless but unnecessary warnings). As the next step we’ll add the ROM image and check its integrity – we don’t want to flash garbage on our APU and brick it, do we?

# unzip -d /mnt apu2_v4.0.7.rom.zip
# grep -c `md5 -q /mnt/apu2_v4.0.7.rom` /mnt/apu2_v4.0.7.rom.md5
1

Make sure that the last command outputs 1 (in which case the calculated md5 hash matches)! If it does not, delete the zip file, download it again and extract it over the corrupt ROM file. Finally sync the filesystem, unmount it and remove the USB stick:

# sync
# umount /mnt

The memstick is ready. If you want to test it, boot some other PC or laptop from it. If you can read the line “Booting the kernel” and then nothing seems to happen anymore, it means that it’s working. TinyCore is configured to use the serial console, that’s why you don’t see anything on your screen and your keyboard doesn’t do anything after that. Just turn your computer off and plug the USB stick out.

Attaching the serial console and flashing the BIOS

Alright, back to the APU2 (finally). Put the memstick into one of the USB ports and attach your serial cable to the COM port. Now connect the other end of the null modem cable with another computer running FreeBSD (or Linux or whatever – this should all work, you’ll just have to figure out how to connect to a serial console on your OS).

Open a terminal, become root and attach the serial console (cuaU0 is the USB to serial adapter, 115,200 the baud rate):

% su -
# cu -l /dev/cuaU0 -s 115200
Connected

Now connect the APU2 to power and see what happens! If you don’t do anything, the BIOS should load TinyCore from the memstick after a couple of seconds:

Connected to the serial console – and booting Linux

If nothing happens, you might have the wrong cable (is it really crossed-over?). Or maybe you’ve mistyped the baud rate? The “Connected” by the way only means that your host system has attached to the serial cable. You’ll get that message even when the other end is not connected to anything or the machine at the other end is turned off.

Once Linux was loaded, use flashrom to update the APU’s firmware like PC Engines show in their howto. Then reboot.

Preparing to flash the BIOS

The APU2’s BIOS supports the serial console. That means that even before the machine has booted an operating system with serial console capability, you can access and configure the BIOS of the headless machine:

Serial access to the BIOS settings

Another nice thing that comes with the APU is the Memtest program. If you want to know whether your new hardware is actually good or might probably have bad ram, put it to the test for a couple of hours or over night:

The firmware comes with Memtest

If you’re using cu as in my example, you can close the serial connection using the character sequence ~. (tilde and dot).

What’s next?

You now know how to access your box using the serial console. Next time we’ll make use of that skill again to put pfSense as the first of two options on the APU. The other option is OPNsense which will be covered in a later post. Both are FreeBSD-based router/firewall operating systems.

Building a BSD home router (pt. 2): The serial console (excursion)

The previous post touched on the topic of why you might want to build your own router, discussed some hardware considerations and showed how to assemble an APU2 bundle. Now that the box is ready for use, we have to do something with it – but there’s a small problem.

People who want to play with inexpensive hardware love their Raspberry Pis. Those make a great little toy: Just attach a keyboard and a monitor, insert an SD card and you’re done. Power on the device and you can start tinkering. With the APU2 it’s not that much harder actually, but it works quite differently. Why? Take a closer look at its backc and you’ll see that the outer left port is not VGA as you might have thought when you saw it from a different angle. It’s a COM port. There are two USB ports which would allow to attach a keyboard (the board doesn’t have a keyboard controller, though, which means that wouldn’t be of much use). But how to attach a screen to it? The RPi offers HDMI output, the APU2 does not. No VGA, no DVI, no HDMI and no DisplayPort either!

So how to access this thing? Typing blindly? Of course not. This post is about the serial console and since they call the COM port a serial port, too, it’s not too hard to figure out what we’re going to use. If you were born after 1980 (like me), chances are that you’ve never used a serial console before. Or rather you might not know exactly what it actually is. Heck, the younger ones might not even have heard about it! So if you’re either old enough or for whatever reason have experience with the serial console you could skip a lot of the following text. If you know what the real TTYs were, what a terminal and a system console is, this whole post probably isn’t going to teach you anything new. But since there are more and more younger people around for whom this is entirely new territory, I’m going to explain not only what we have to do in case of the APU2 but also a bit about how this all came to be.

Once upon a time… in 2017

Huh, 2017? That’s this year! Exactly. But then this headline doesn’t make sense, does it? Trust me, it does. Consider this:

% uname -a
FreeBSD pc0.local 11.1-PRERELEASE FreeBSD 11.1-PRERELEASE #0 r319447: Fri Jun  2 03:47:52 CEST 2017     root@pc0.local:/usr/obj/usr/src/sys/GENERIC  amd64

% ls /dev/tty*
/dev/ttyu2      /dev/ttyv0      /dev/ttyv3      /dev/ttyv6      /dev/ttyv9
/dev/ttyu2.init /dev/ttyv1      /dev/ttyv4      /dev/ttyv7      /dev/ttyva
/dev/ttyu2.lock /dev/ttyv2      /dev/ttyv5      /dev/ttyv8      /dev/ttyvb

Here’s an up-to-date FreeBSD system and it has quite some device nodes that bear “tty” in their name. The abbreviation TTY means Teletypewriter (or “teletype” for short) and with that we’re speaking of a technology that has its roots far back in the first half of the 19th (!) century. As I stated in my first post about FreeBSD 4.11 and legacy systems: Once some technology is established and in use, it’s extremely hard to get rid of it again. And while it’s hard to believe that concepts from almost two hundred years ago still influence today’s Unix machines, there’s some truth to it.

So what is a teletypewriter? Think a classical typewriter first. One of those mechanical machines that allow to type text instead of writing it: You press a key and that makes a small metal arm swing and its end hammer against the paper. That end bears an embossed letter. Between this head with the letter (or rather: character) and the paper there’s an ink ribbon and thus your key press results in a readable imprint on the paper.

Man has long striven to be able to transfer information across a long distance. Morse code made it possible to encode text and then send a message e.g. via optical means like blinking light. The telegraph (from Greek tele + graphein = far + write) achieves the same goal, however by sending signals over a cable.

A simple teletypewriter combines both: Imagine a typewriter with all the usual keys but no paper that is connected to a remote device which has the paper feed, ink ribbon and so on but no keys. This second device is called a teleprinter. The big limitation to this is that the information flow is one-way only.

Where computers fit into this

Sounds familiar? Sure thing: The keyboard that you use essentially does exactly that! Now consider a more advanced approach where two teletypewriters that both have input and output capabilities are connected to each other. In this bi-directional use you can type something that the person on the other end can read as well as write an answer which then you can read (and write again).

It’s obvious how this supports distributing information across large distances. But what does all this have to do with the COM port? Please bear with me. The first type of computers were big and extremely expensive mainframe machines. It makes sense that not everybody would be allowed physical access to them. Thus you wouldn’t sit down before them and attach a screen and keyboard! Instead you’d use a teletype in another room to interact with it. You input any commands and get a response in form of the printed output. More sophisticated teletypes allowed saving input or output on punch cards or read them in again and send the data.

Whatever combination of gear that allows sending and receiving data to/from a computer is called a terminal. Bi-directional typewriters are sometimes referenced as “hard-copy terminals” (because they printed the output on paper). While it’s hard to believe, this procedure has been state of the art at some point in time. Obviously this is not ideal, wasting a lot of paper. Fortunately there’s a solution for this: Enter the electronic terminal! Those replaced the printing on paper with displaying on a screen.

Now this is much more of what we’re familiar with, just in a special machine that is connected to the actual computer in a different place. The great thing about those terminals was that they allowed for adding more and more exciting new features. The bad thing was that they added more and more exciting new features. Usually there was no gain in having those features on the client side only. The server side had to know about it, too. Before long there were a lot of terminals out there, all with nice new features but all incompatible with each other…

This sucked. If you had this expensive new terminal that was unknown to your computer however, you couldn’t use its additional features. Good thing that we don’t use terminals today! That makes this an issue of the past. Right, that is a good thing. But no, it’s not a historic issue. Yes, we’ve replaced terminals quite a while ago. But what did we replace them with? Today we use terminal emulators. Your *nix operating system comes with programs that (by default) use the keyboard for input and the screen for output and thus simulate… a terminal.

Terminal emulators

Those terminal emulators can emulate various types of physical terminals and have additional features of their own. So the problems are back (or rather: They were never really gone). How do programs know what features are available? That used to be really tricky (and often lead to discarding features that were not commonly available on all popular terminals…). Let’s take a look at my shell’s environment:

% env
TMUX_PANE=%2
TMUX=/tmp/tmux-1001/default,1164,0
TERM=screen
[...]

My shell is running inside the terminal multiplexer tmux which set the TERM variable to “screen”. GNU screen was a somewhat similar program and since tmux is compatible with it, it announces that terminal-wise it works like screen. What does this look like if I quit tmux?

% env
OSTYPE=FreeBSD
MM_CHARSET=UTF-8
LANG=de_DE.UTF-8
DISPLAY=unix:0.0
[...]
SHELL=/bin/tcsh
WINDOWID=58720259
TERM=xterm
[...]

Here my terminal emulator claims to support the feature set that xterm is known for. I’m not using xterm, but my terminal emulator (terminator) is compatible with it. You probably know htop. I’ve just installed it to make two screenshots of it. This is what it looks like if I run it in my terminal emulator:

htop running with $TERM=xterm

Now let’s claim that we’re using another terminal type, shall we? I’m going to pretend that my terminal is an old, crufty vt100:

% setenv TERM vt100
% htop

Afterwards the application looks like this:

htop running with $TERM=vt100

The difference is pretty obvious – the vt100 did not support color output (among other things). Let’s be even more radical and simply not announce our terminal type to any applications:

% unsetenv TERM
% tmux
open terminal failed: terminal does not support clear

Tmux won’t even run (and htop would screw it all up badly since it obviously doesn’t handle that case properly)! If the terminal type (and thus its capabilities) are unknown, programs need to fallback to the absolute baseline of functionality that each and every terminal is known to support. And that’s disturbingly little… In this case tmux would be unusable because the terminal might not even be able to clear the screen! Yes, not even the seemingly most basic things were implemented in all terminals.

Imagine you’re the author of a simple *nix program. You were supposed to know what terminals existed and what capabilities they had – and to adopt your program to whatever terminal it ran on! Fortunately Bill Joy wrote termcap, a library that helped dealing with various terminals. This extremely important functionality (which originated in BSD Unix, BTW!) evolved over time. Today we have ncurses which does just about everything to make life easy when programming console applications.

(Serial) console

Which leads to the question: What is the console? The system console (or just console) is a device used to imput and output text. A PC’s BIOS offers a console, the boot loader does and the OS kernel does, too: It uses it to print out messages and to take commands. It’s present even when you don’t see it. A system might for example work in so-called headless mode which means that there is nothing attached to the console. Plug in a keyboard and connect a monitor and you can make use of the console.

Unix-like operating systems often configure multiple virtual consoles (also called: virtual terminals). We saw those when we listed the tty* nodes in /dev at the beginning of this blog post. Together those form your computer’s system console to which usually keyboard and screen are attached.

But you need not attach those two devices to the console. You can also attach a serial device to it which can then be used to transfer commands from and messages to another computer that is connected to the other end of the serial cable. Then you can use that other computer’s keyboard to type commands and its screen to read the output. A serial console is simply a console that can make use of a serial connection to redirect the console. If the BIOS of your machine supports it, you can use the serial console to control everything on that machine using another one over a serial connection.

If you’re completely new to it, this probably was quite a bit of information. However we’ve only talked about the very basics here! If you like to know more about how TTYs, pseudo TTYs, processes, jobs and even signals work, I recommend Linus Åkesson’s article The TTY demystified which is an excellent read. He explains much more in-depth how Linux does all this.

What’s next?

In the next post we’ll prepare a USB memstick and attach a serial console to the APU2 to update the firmware.

Building a BSD home router (pt. 1): Hardware (PC Engines APU2)

Pretty much everybody in the western world has internet access at home these days. It’s not a big deal: You conduct a contract with some ISP. They send you a modem/router combo box that you plug in, do some simple setup and you’re done. Those boxes are pretty much ubiquitous pieces of hardware, silently doing their work most of the time. Some of them even come with wireless access and all that other convenient stuff. Basically you configure them once and then forget that you even have one!

Why would you want your own router?

The paragraph above sounds great, doesn’t it? It sure does. And it actually is. Kind of. There’s only one little problem with it, really. Those boxes are more often than not meant for sunny, green meadows where unicorns graze peacefully. Unfortunately… the internet is a less friendly place: Predators, scavengers and all kinds of poltergeists are out there and after your hide, luring in the shadows ready to rip your guts out when they catch you off guard!

Let’s put it straight: It’s not much of a secret that standard off-the-shelf type home routers suck. Big time. They do little to protect the unsuspecting user. They get in the way of people who know a bit more than the average user and try to tighten security at least a little. They have proven to often come with serious bugs. Or worse: They might even contain holes that were introduced on purpose…

If you’re running Linux, *BSD or some other OS that puts you in control, you might not be too comfortable with that potential spy in your house. (In times where people choose to put known spies like “Alexa” in their homes, your router is probably not the first thing to take care of, though. But maybe you live in a shared apartment together with someone who thinks that this devil’s work is “cool”. You know that a move is in order, don’t you? But hey, it could be worse: You could have married someone who thinks so – in which case of course you’re hosed.)

Hardware considerations

The first thing to think about – at least for me – was which software to use. There are various possibilities to choose from. Since I’ve come to appreciate BSD operating systems, I wanted something BSD based. This basic thing decided on, I needed to find hardware that was supported by my OS of choice.

However there was more to think about. While a PC is usually powered down and turned off when it’s not needed, you probably don’t want to do the same thing with your router. And while an old PC could technically do the job, for something that’s basically an “always on” device, it makes sense to use something that doesn’t draw as much power. For that reason some of those small embedded boards crossed my mind. However those are often ARM or MIPS based and sometimes don’t even have proper gigabit LAN. This doesn’t mean they are not up to the task but going with a less common architecture wouldn’t exactly make things easier. Therefore I decided that x86 was the way to go, at least for now. There’s enough new things to learn and if everything works out very well I can always play the “build your own router” game again, choosing a higher difficulty level.

Ok, x86 and (preferably) low power consumption. There are various products which fit into this category, and they often are passively cooled as well which is nice, too. I read good things about PC Engine’s APU2 boards. There are reports out there indicating that they run FreeBSD and OpenBSD really well. They also seem to be pretty popular among people building their own routers – and it’s not hard to see why.

The APU2c4

The APU2 is a system board that comes in a couple of variants. I opted for the APU2c4 which has among other features the following specs:

  • a quad-core (Jaguar) 64 bit AMD CPU
  • 4 GB DDR3 ECC RAM
  • 3 Gigabit Ethernet jacks

It makes use of Coreboot which is great and the CPU features the AES-NI instruction set that enables AES crypto acceleration which is useful as well. While the RAM is ECC technically, the firmware does not support error correction, yet, unfortunately. But that feature may be enabled with a future firmware update.

In the end I ordered a bundle that consists of the board, an external power supply, an indoor case and an mSATA 16 GB drive. The APU can make use of an SD card, too, but I definitely prefer the mSATA option. And I only paid about 180 Euros for it. Sure you can get off-the-shelf routers for a lot less, but… yeah.

Assembling my router

When I received my shipment, I opened it up and took a look at the parts. Luckily a colleague who is much more experienced with hardware than I am, offered to help me. At my first attempt I wasn’t even able to get the board into the case – it didn’t seem to fit! Of course it does fit… You just have to remove the screws for the COM port first.

APU2c4 and case

My bundle came with a small metal heat transfer plate and two stripes of double-sided sticky tape. You’re supposed to use one to use the latter to fix the plate so that it connects the plate to both the processor and the case. However my colleague asked me if I’d rather do things right and I agreed.

So he put tape on one side of the plate to stick it to the case. Then he put the board into the case. The board has two holes for screws:

Holes for screws in the board

With a marker pen he wrote marks on the on the plate to indicate where the holes on the board are. Then he removed the board again and drilled two holes through the plate and the bottom of the case. Using a screw tap he then cut two screw threads into the material.

The heat transfer plate with markers and double-sided sticky tape

The next step was to properly clean both the plate and the CPU, put thermal conductance paste on there and put the board back in place. Now the screws could be used to correctly fix the whole thing. I also plugged in the mSATA drive.

The board is fixed properly and mSATA drive plugged in

My colleague used plastic screws for the simple reason that it’s easy to cut off the overlapping parts that went through the bottom of the case. Not a bad idea!

Bottom view: Those white spots are the plastic screws

Done! I thought about painting the plastic screws’ ends black but then again that’s only the bottom of the case. I’ve been using my router for a couple of weeks now and I’m pretty happy with it (and have a lot to play with!).

UPDATE: My colleague also got a similar APU2 and was curious enough to test how much it makes a difference to use the heat plate the way the bundle suggests or to use thermal conductance paste. He put his machine under various load situations once with the sticky tape and once with the paste applied. The difference proved to be ranging between 2° to 5° C! That really makes the extra effort worth it.

What’s next?

The APU does not have any VGA port, you have to attach a serial console to work with it. So that’s what the next post will be about.

FreeBSD jails (2/2): 4.11 sentenced to jail

Ok, after introducing FreeBSD’s jails and jail frameworks in the previous post, it’s time to actually put that 4.11 system that was built in February behind bars. Before turning towards that special case, though, we’ll have a look at how to use iocage (one of the jail management frameworks).

Creating and starting an 11.0 jail

We’ve built iocage from ports last time, so let’s just jump right into it and try out one of the most important commands:

# iocage list

Iocage initialized

Not too surprisingly, the jail list is empty. But the important thing here is that iocage automatically created its infrastructure when run for the first time. This is generally fine; if you happen to have more than one ZFS pool, though, you may want to manually activate one for iocage usage.

The next thing to do is to fetch the distribution tarballs for the FreeBSD version that shall populate the jails you want to create:

# iocage fetch -r 11.0-RELEASE

Fetching the 11.0 release

This process will also update the kernel and userland to the latest available version of the selected release. I chose 11.0 because that’s the latest right now and obviously 4.11 is not available!

You can actually skip fetching the release as iocage will do it automatically if you create the jail, but doing it beforehand doesn’t hurt. Now let’s create our first jail:

# iocage create tag=fbsd11 ip4_addr="em0|192.168.2.10/24" -r 11.0-RELEASE

Jail ‘fbsd11’ created!

Each jail has a UUID. This one’s UUID begins with 99a9941a. You can interact with jails by using that UUID. But if you’ve got a lot of jails it would be pretty hard to keep track of which one is which. For that reason you can assign a tag to a jail that you can use to identify it. We’ve effectively given our jail a name: fbsd11. In general it’s a good idea to name it in a way that helps you (or your co-workers) to understand what the jail is used for. You may choose something like “postgres-jail” that includes the technology used, “loadbalancer” to describe its function or whatever fits your needs.

We’ve also assigned the jail an ipv4 address, 192.168.2.10 in this case, noted the subnet in CIDR notation and specified which NIC to use it on. Listing jails shows all that information at a glance.

The jail is ready to be started:

# iocage start fbsd11

Starting the jail

As you can see here from the ifconfig output, the ip address was not yet assigned to the NIC. According to iocage the jail has been started successfully. Let’s take a look at it:

# iocage list

The jail is running

The state for our jail has changed to “up”. Also the ip has been automatically assigned to our network interface when iocage started the jail. When the jail is stopped, it is also automatically removed again.

Using the jail

Iocage offers some means to interact jails without actually “entering” them, like e.g. installing packages, etc. The easiest way to use a jail however, is to get a shell prompt inside the jail (think entering a chroot environment). That’s what we’re going to do next:

# iocage console fbsd11

Entering the jail console

Looks pretty much like we just logged into a FreeBSD system, huh? Well, we basically did just that. With the big difference of not being a full OS (which would be the case with a VM for example) but a jail with just a FreeBSD 11.0 userland (that is running on the host machine’s kernel).

Ok, I bootstrapped pkg and then installed nginx:

Installing the nginx webserver

This also proves that the network is working since we’re receiving the packages to install from. There’s also nothing special in doing this, just use pkg the way you’re used to. For the most part it’s behaving just like a regular FreeBSD system. There are a few exceptions. One of the most notable ones is the fact that by default you won’t be able to use the ping command. You can enable jails to use ping using a sysctl but this affects the jails security and therefore is disabled by default.

Let’s start nginx next:

Nginx is running

Again it’s nothing special. Nginx is running and bound to port 80. Does it do what you’d expect it to?

Testing and destroying the jail

Nginx’s default page

Yes, the webserver is happily serving the default page when accessed from other computers on the LAN! It doesn’t make much of a difference from running nginx on the host system – unless somebody manages to use an exploit. Nginx is privilege separated and thus probably not a good example. But think about a case where an intruder manages to get into your system and get root privileges. That’s pretty bad, yes. But if the application was running in a jail, the damage is at least contained and the other jails as well as the host system are unaffected. (That and it’s much easier to tear that jail down and create a new one compared to having to reinstall the whole host system and setup several applications again!)

Alright. Let’s take a quick look at where the jail data is kept:

Taking a look at iocage’s directory structure

Iocage created the /iocage dataset. The most important path is /iocage/jail. All the jails are in there (named after their UUID). To make accessing them easier, there’s /iocage/tags: Here you’ll find symlinks named after the jail’s tags which are pointing to the actual directories with the data.

Each jail consists of a configuration file, an fstab and a “root” directory (which holds all the files).

This jail has served its purpose and is no longer needed. So let’s get rid of it! First exit the jail like you would exit a chroot (e.g. CTRL-D out of it) before you proceed:

# iocage stop fbsd11
# iocage destroy fbsd11

Stopping and destroying the jail

You cannot destroy a running jail. Therefor it must be stopped first and can then be destroyed.

Creating the 4.11 jail

That has been pretty easy and straight forward, right? And here’s the good news: Jailing the 4.11 system isn’t that difficult, either. I created a tarball of the complete filesystem on the FreeBSD 4.11 server, xz compressed it and scp’d that onto the jailhost. Now we’ll create another 11.0 jail, delete the filesystem and extract the old OS data into the same place:

# iocage create tag=fbsd4.11 ip4_addr="em0|192.168.2.10/24" -r 11.0-RELEASE
# cd /iocage/tags/fbsd4.11/root
# rm -rf * .*
# tar xvJf /path/to/tarball.xz

Creating a new jail and putting the 4.11 OS into it

The old device nodes won’t work on a current kernel and have to be removed. The same goes for the old proc filesystem. We also don’t need any kernels or kernel modules. And /boot is obsolete in a jail, too. So it’s a good idea to get rid of all that:

# rm dev/*
# rm kernel*
# rm -rf boot
# rm -rf modules*
# rm -rf proc/*

Deleting unneeded parts of 4.11

The old userland and the additional data remains. Let’s try to start the jail now:

# iocage stop fbsd11

Starting the 4.11 jail

Uhh… This doesn’t look good. Iocage tries to do things that won’t work with a 4.11 userland (anything older than 9.3 is unsupported with iocage, anyways!). But fortunately that looks worse than it actually is:

It errors out… but it’s running!

Despite those errors the jail is up and running! Can we get console access?

Inside the 4.11 jail

# iocage console fbsd4.11

Entering the console works, too!

Yes, we can! Let’s make 4.11 great again! Time to install a webserver in this jail, too. Since we still have Pkgsrc in place, that’s an easy thing to do:

# cd /usr/pkgsrc/16/www/nginx
# bmake install clean clean-depends

Building nginx on 4.11 through Pkgsrc

I decided to modify nginx’s default starting page so it looks a little different from the stock one. Nginx can be started simply by executing the daemon. However all the system utilities regarding process management don’t work! And sockstat is not working either.

Starting nginx, but tools like ps don’t work!

Does nginx actually run? There’s an easy way to test that. Let’s just open up a browser on another pc and see if we can get the modified webpage.

Nginx is able to serve the webpage from the 4.11 jail!

It works! Yes, it was actually that easy. Jails are not necessarily a complicated thing at all (they can be more complicated than that of course, if you’re doing more advanced stuff). But… No ps, no top, no kill… How are you supposed to work with a 4.11 jail?

Peter Wemm suggested using statically compiled binaries from the host system like those in /rescue. And that’s actually a great idea! If you copy /rescue over into the jail, it at least provides some tools to work with.

Thanks to /rescue process management is possible

Things like top would be nice, too, of course. But to be honest, I don’t know how to statically build them. If anybody knows, please comment and share your wisdom!

More about jails?

This article merely gave a very basic introduction to the topic (plus a little “stunt”). You can do a lot more with jails. I might write about it again in the future but I haven’t decided, yet (feel free to comment and tell me what you’re interested in!). But I wanted to at least give you an idea what other things there are to explore.

If you have a server with only one ip you can still use jails. The solution to this is to give the jails internal network addresses (like 192.168.x.x) and then use Pf to do network address translation (NAT). It took me a bit to figure out how this works (there are things on the net that are better documented) but in the end it’s not that complicated.

With VIMAGE/VNET you can give your jails their own virtual network stack. That way you can even use a firewall inside the jail and assign multiple ip addresses!

Jails support quotas of various kinds. You can limit the amount of CPU that one jail may occupy, limit file system IO, etc. Lots of possibilities with this.

And of course iocage has many nice features. You can e.g. create jail templates with some software already installed and quickly create jails from those. There are multiple jail types like e.g. thin-provisioned jails. You can clone and snapshot jail. And you can have jails started automatically when your server boots.

In the previous article I said “think containers” when you’re completely new to jails. Now you’ve seen a bit of what they can do and have a better idea of what they are. So stop “thinking containers” now. These are two things that are somewhat related but they are not the same thing. Linux’s containers can do things that jails were never designed for (e.g. sharing namespaces). But jails have one huge advantage over containers: While containers are only a means for logical separation, jails were designed for security right from the start. Sure, people are working on making Linux’s containers more secure. But if there’s one thing that we should have learned by now, it is this: Retrofitting security into something is not really going to work… And jails are pretty flexible, too. You might not have missed them when you didn’t know that they exist. But get used to using them once and you’ll think differently about a lot of things and make use of them for more and more things. Jails are a great feature of FreeBSD.

FreeBSD jails (1/2): Introduction and frameworks

This is the first part of a followup to the 4.11 mini-series of posts that I published here on my blog ([1], [2], [3] & [4]). Those posts covered installing an old FreeBSD 4.11 system and using pkgsrc to update some parts of the system from versions released in the last millennium (!) to those of today (or rather from December 2016).

Now we’re going one step further: We’ll migrate a 4.11 system into a jail on a fresh FreeBSD 11.0 host system! This will be dealt with in the next post. In this part I’ll introduce the jail concept for people who are not familiar with it, discuss some jail management frameworks and do a bit of preparation work.

Motivation

Why to send that old system to a jail? We’ve already done a bit of work on a 4.11 system. But that next step tightens security much more since we have a recent kernel, can make use of a state-of-the-art firewall, etc. This is not actually my main concern however. Those remaining machines are running internal processes, so security is nice but not critical. What is critical and really frightens me, is the age of the hard drives that they use… When you do a df and see that the total capacity of a drive is way below 100 GB, you can guess that drives like that have not been sold for… quite some years now! And knowing that drives are rather perfidious little things, I’d rather get rid of those as soon as possible.

What makes things worse: This is 4.11. There’s no GEOM available and thus no gmirror or anything. Yes, there are backups. But I’d rather not have to mess with something like that just to restore a system that should have been decommissioned years ago!

What are FreeBSD jails?

If you’re new to FreeBSD, you may have heard about jails without knowing what they actually are. If you’ve got a Solaris background, think zones. If you’re a Linux user, think containers. FreeBSD people usually cannot understand today’s container hype for the very simple reason that this OS has had jails for ages now (they were first available on development versions at the very end of the 20th century!). You are probably familiar with docker, Linux’s cgroups and all those facilities. FreeBSD basically does the same thing: Provide much enhanced but still very light-weight chroot environment.

The system in that jail is isolated from the actual host operating system so that escaping it isn’t trivial and requires special (insecure) settings. In general jails are meant to be secure after all, aren’t they? You can also give jails their own virtual network stack, you can cap their resource usage, and so on. It’s a mature, stable and secure way to do containerization of applications – or whole operating system installations.

Since FreeBSD is pretty good at keeping compatibility with old releases (for the next major release they are discussing whether to keep compatibility shims that allow executing binaries from commercial UNIX releases of the 80’s!!), it’s very much possible to stuff a 4.11 userland into a jail running on a recent version of the operating system. And that’s what we’re going to look at after getting some of the basics out of the way.

Jail frameworks

Jails in FreeBSD can be created and managed manually. We’re talking *nix here, after all! However there are several jail management frameworks available that promise to make things even easier or at least more convenient. If you want to get started with jails, I suggest that you pick a framework that sounds good to you feature-wise and simply begin playing with it. The whole thing is not that complicated actually.

If you’re just getting started, don’t bother with sysutils/warden. It was a popular jail manager created for PC-BSD but it is considered obsolete now.

One true classic among the jail frameworks is sysutils/ezjail. It still does the job and a lot of people continue to use it simply because they are already familiar with it. It allows for things like thin-provisioned jails that use a “base jail” which holds a complete FreeBSD userland, and uses nullmount to make that accessible in other jails. This makes updating very quick and convenient. It also leaves the base system read-only which may be another nice security feature at the cost of decreased flexibility. On the plus side Ezjail is independent on filesystems and thus works on UFS, too.

Then there was sysutils/iocage, a modern approach that made use of ZFS properties to store certain values. For that reason it demands at least one zpool on the system to be usable. It had been implemented in shell code which made it easy to read for admins who don’t have a lot of coding experience. It has however been rewritten and the old shell version is declared obsolete. If you like the idea of having it written in shell, take a look at sysutils/iocell, which is a fork of the last shell version and is actively maintained.

The new version of iocage is implemented in Python (initially it was announced to be rewritten in Go but that decision was revoked). Another warning: sysutils/py-iocage is the wrong version, too! Do not use it. The rewrite initially supported Python 2.7 – which is what this port is all about – but newer versions dropped support for that and now require Python 3.6. So the actual port you should use is sysutils/py3-iocage! That version requirement makes sense but unfortunately it leads to iocage not being available from packages (as FreeBSD’s default version of Python 3 is still 3.3 currently). So you probably need to build it from ports.

There’s one more framework that I want to mention: sysutils/cbsd. It is an advanced system that not only does jails but also allows to manage bhyve (FreeBSD’s modern hypervisor). If you’re looking for something more comprehensive you may well give it a try.

Choosing my framework

When I first wanted to get into jails, I tried out ezjail. It worked well for me and I’d still use it on a system with no ZFS available. When I have ZFS, though, I liked the iocage approach. Cbsd looks quite good but I didn’t do too much with it because I require things to be scriptable (like using the framework through SaltStack states – there’s even a Salt Formula for iocage, but that’s for the old implementation) and that doesn’t seem to be what they focus on. So when the shell based iocage was deprecated, I moved to iocell and then I was torn to stick with that or to return to the new iocage. In the end I opted for at least giving the rewrite a chance. Since it does its thing quite nicely, I’ve adopted that and will use it here.

You can of course use any other framework if you prefer a different one. Or you could setup your jail by hand – if you’re thinking about that, definitely take a look at this article by M. W. Lucas whom I actually have to thank not only for his post but kind of for this one as well. The thought of jailing the old systems crossed my mind once or twice, but if he hadn’t written about it in the past, I don’t think I’d ever have tried it (no, I read his post back in the day when he published it; yes, I wanted to try this for over three years now before I finally found some time to give it a shot!).

Preparations

Let’s install iocage first. Like stated before, we will have to build it from ports. Assuming a system that does not have a ports tree installed, this is how you can do it (there are better (cleaner) ways to build packages from ports, I planned to do a post about that topic since December!) but this old method still works:

# portsnap fetch extract
# cd /usr/ports/sysutils/py3-iocage
# make config-recursive
# make install clean

The new iocage needs a UTF-8 locale set on the system or it won’t run. Unfortunately this is not the standard in FreeBSD (this is something that I really hope for in FreeBSD 12!). There are multiple ways to do this, my preferred one to set it in /etc/login.conf:

Look for default:\ – it is followed by a block of indented lines, the last one being :umask=022:. Add another backslash at the end to make the block continue and add two more lines to it:

:charset=UTF-8:\
:lang=en_US.UTF-8:

Login configuration is one of the few things where the actual values are in fact stored in a database. The changes that you made to the text file won’t take effect even if you log out and back in. First run another command to update the DB according to the changes in the text file:

cap_mkdb /etc/login.conf

Now log out and back in. Try echoing $LANG – that should print the UTF locale’s name. If it does, you are set.

What’s next?

The next post will demonstrate how to use iocage to manage jails and show how to jail a FreeBSD 4.11 system so it runs on a FreeBSD 11 host.