FreeBSD jails (2/2): 4.11 sentenced to jail

Ok, after introducing FreeBSD’s jails and jail frameworks in the previous post, it’s time to actually put that 4.11 system that was built in February behind bars. Before turning towards that special case, though, we’ll have a look at how to use iocage (one of the jail management frameworks).

Creating and starting an 11.0 jail

We’ve built iocage from ports last time, so let’s just jump right into it and try out one of the most important commands:

# iocage list

Iocage initialized

Not too surprisingly, the jail list is empty. But the important thing here is that iocage automatically created its infrastructure when run for the first time. This is generally fine; if you happen to have more than one ZFS pool, though, you may want to manually activate one for iocage usage.

The next thing to do is to fetch the distribution tarballs for the FreeBSD version that shall populate the jails you want to create:

# iocage fetch -r 11.0-RELEASE

Fetching the 11.0 release

This process will also update the kernel and userland to the latest available version of the selected release. I chose 11.0 because that’s the latest right now and obviously 4.11 is not available!

You can actually skip fetching the release as iocage will do it automatically if you create the jail, but doing it beforehand doesn’t hurt. Now let’s create our first jail:

# iocage create tag=fbsd11 ip4_addr="em0|192.168.2.10/24" -r 11.0-RELEASE

Jail ‘fbsd11’ created!

Each jail has a UUID. This one’s UUID begins with 99a9941a. You can interact with jails by using that UUID. But if you’ve got a lot of jails it would be pretty hard to keep track of which one is which. For that reason you can assign a tag to a jail that you can use to identify it. We’ve effectively given our jail a name: fbsd11. In general it’s a good idea to name it in a way that helps you (or your co-workers) to understand what the jail is used for. You may choose something like “postgres-jail” that includes the technology used, “loadbalancer” to describe its function or whatever fits your needs.

We’ve also assigned the jail an ipv4 address, 192.168.2.10 in this case, noted the subnet in CIDR notation and specified which NIC to use it on. Listing jails shows all that information at a glance.

The jail is ready to be started:

# iocage start fbsd11

Starting the jail

As you can see here from the ifconfig output, the ip address was not yet assigned to the NIC. According to iocage the jail has been started successfully. Let’s take a look at it:

# iocage list

The jail is running

The state for our jail has changed to “up”. Also the ip has been automatically assigned to our network interface when iocage started the jail. When the jail is stopped, it is also automatically removed again.

Using the jail

Iocage offers some means to interact jails without actually “entering” them, like e.g. installing packages, etc. The easiest way to use a jail however, is to get a shell prompt inside the jail (think entering a chroot environment). That’s what we’re going to do next:

# iocage console fbsd11

Entering the jail console

Looks pretty much like we just logged into a FreeBSD system, huh? Well, we basically did just that. With the big difference of not being a full OS (which would be the case with a VM for example) but a jail with just a FreeBSD 11.0 userland (that is running on the host machine’s kernel).

Ok, I bootstrapped pkg and then installed nginx:

Installing the nginx webserver

This also proves that the network is working since we’re receiving the packages to install from. There’s also nothing special in doing this, just use pkg the way you’re used to. For the most part it’s behaving just like a regular FreeBSD system. There are a few exceptions. One of the most notable ones is the fact that by default you won’t be able to use the ping command. You can enable jails to use ping using a sysctl but this affects the jails security and therefore is disabled by default.

Let’s start nginx next:

Nginx is running

Again it’s nothing special. Nginx is running and bound to port 80. Does it do what you’d expect it to?

Testing and destroying the jail

Nginx’s default page

Yes, the webserver is happily serving the default page when accessed from other computers on the LAN! It doesn’t make much of a difference from running nginx on the host system – unless somebody manages to use an exploit. Nginx is privilege separated and thus probably not a good example. But think about a case where an intruder manages to get into your system and get root privileges. That’s pretty bad, yes. But if the application was running in a jail, the damage is at least contained and the other jails as well as the host system are unaffected. (That and it’s much easier to tear that jail down and create a new one compared to having to reinstall the whole host system and setup several applications again!)

Alright. Let’s take a quick look at where the jail data is kept:

Taking a look at iocage’s directory structure

Iocage created the /iocage dataset. The most important path is /iocage/jail. All the jails are in there (named after their UUID). To make accessing them easier, there’s /iocage/tags: Here you’ll find symlinks named after the jail’s tags which are pointing to the actual directories with the data.

Each jail consists of a configuration file, an fstab and a “root” directory (which holds all the files).

This jail has served its purpose and is no longer needed. So let’s get rid of it! First exit the jail like you would exit a chroot (e.g. CTRL-D out of it) before you proceed:

# iocage stop fbsd11
# iocage destroy fbsd11

Stopping and destroying the jail

You cannot destroy a running jail. Therefor it must be stopped first and can then be destroyed.

Creating the 4.11 jail

That has been pretty easy and straight forward, right? And here’s the good news: Jailing the 4.11 system isn’t that difficult, either. I created a tarball of the complete filesystem on the FreeBSD 4.11 server, xz compressed it and scp’d that onto the jailhost. Now we’ll create another 11.0 jail, delete the filesystem and extract the old OS data into the same place:

# iocage create tag=fbsd4.11 ip4_addr="em0|192.168.2.10/24" -r 11.0-RELEASE
# cd /iocage/tags/fbsd4.11/root
# rm -rf * .*
# tar xvJf /path/to/tarball.xz

Creating a new jail and putting the 4.11 OS into it

The old device nodes won’t work on a current kernel and have to be removed. The same goes for the old proc filesystem. We also don’t need any kernels or kernel modules. And /boot is obsolete in a jail, too. So it’s a good idea to get rid of all that:

# rm dev/*
# rm kernel*
# rm -rf boot
# rm -rf modules*
# rm -rf proc/*

Deleting unneeded parts of 4.11

The old userland and the additional data remains. Let’s try to start the jail now:

# iocage stop fbsd11

Starting the 4.11 jail

Uhh… This doesn’t look good. Iocage tries to do things that won’t work with a 4.11 userland (anything older than 9.3 is unsupported with iocage, anyways!). But fortunately that looks worse than it actually is:

It errors out… but it’s running!

Despite those errors the jail is up and running! Can we get console access?

Inside the 4.11 jail

# iocage console fbsd4.11

Entering the console works, too!

Yes, we can! Let’s make 4.11 great again! Time to install a webserver in this jail, too. Since we still have Pkgsrc in place, that’s an easy thing to do:

# cd /usr/pkgsrc/16/www/nginx
# bmake install clean clean-depends

Building nginx on 4.11 through Pkgsrc

I decided to modify nginx’s default starting page so it looks a little different from the stock one. Nginx can be started simply by executing the daemon. However all the system utilities regarding process management don’t work! And sockstat is not working either.

Starting nginx, but tools like ps don’t work!

Does nginx actually run? There’s an easy way to test that. Let’s just open up a browser on another pc and see if we can get the modified webpage.

Nginx is able to serve the webpage from the 4.11 jail!

It works! Yes, it was actually that easy. Jails are not necessarily a complicated thing at all (they can be more complicated than that of course, if you’re doing more advanced stuff). But… No ps, no top, no kill… How are you supposed to work with a 4.11 jail?

Peter Wemm suggested using statically compiled binaries from the host system like those in /rescue. And that’s actually a great idea! If you copy /rescue over into the jail, it at least provides some tools to work with.

Thanks to /rescue process management is possible

Things like top would be nice, too, of course. But to be honest, I don’t know how to statically build them. If anybody knows, please comment and share your wisdom!

More about jails?

This article merely gave a very basic introduction to the topic (plus a little “stunt”). You can do a lot more with jails. I might write about it again in the future but I haven’t decided, yet (feel free to comment and tell me!). But I wanted to at least give you an idea what other things there are to explore.

If you have a server with only one ip you can still use jails. The solution to this is to give the jails internal network addresses (like 192.168.x.x) and then use Pf to do network address translation (NAT). It took me a bit to figure out how this works (there are things on the net that are better documented) but in the end it’s not that complicated.

With VIMAGE/VNET you can give your jails their own virtual network stack. That way you can even use a firewall inside the jail and assign multiple ip addresses!

Jails support quotas of various kinds. You can limit the amount of CPU that one jail may occupy, limit file system IO, etc. Lots of possibilities with this.

And of course iocage has many nice features. You can e.g. create jail templates with some software already installed and quickly create jails from those. There are multiple jail types like e.g. thin-provisioned jails. You can clone and snapshot jail. And you can have jails started automatically when your server boots.

In the previous article I said “think containers” when you’re completely new to jails. Now you’ve seen a bit of what they can do and have a better idea of what they are. So stop “thinking containers” now. These are two things that are somewhat related but they are not the same thing. Linux’s containers can do things that jails were never designed for (e.g. sharing namespaces). But jails have one huge advantage over containers: While containers are only a means for logical separation, jails were designed for security right from the start. Sure, people are working on making Linux’s containers more secure. But if there’s one thing that we should have learned by now, it is this: Retrofitting security into something is not really going to work… And jails are pretty flexible, too. You might not have missed them when you didn’t know that they exist. But get used to using them once and you’ll think differently about a lot of things and make use of them for more and more things. Jails are a great feature of FreeBSD.

FreeBSD jails (1/2): Introduction and frameworks

This is the first part of a followup to the 4.11 mini-series of posts that I published here on my blog ([1], [2], [3] & [4]). Those posts covered installing an old FreeBSD 4.11 system and using pkgsrc to update some parts of the system from versions released in the last millennium (!) to those of today (or rather from December 2016).

Now we’re going one step further: We’ll migrate a 4.11 system into a jail on a fresh FreeBSD 11.0 host system! This will be dealt with in the next post. In this part I’ll introduce the jail concept for people who are not familiar with it, discuss some jail management frameworks and do a bit of preparation work.

Motivation

Why to send that old system to a jail? We’ve already done a bit of work on a 4.11 system. But that next step tightens security much more since we have a recent kernel, can make use of a state-of-the-art firewall, etc. This is not actually my main concern however. Those remaining machines are running internal processes, so security is nice but not critical. What is critical and really frightens me, is the age of the hard drives that they use… When you do a df and see that the total capacity of a drive is way below 100 GB, you can guess that drives like that have not been sold for… quite some years now! And knowing that drives are rather perfidious little things, I’d rather get rid of those as soon as possible.

What makes things worse: This is 4.11. There’s no GEOM available and thus no gmirror or anything. Yes, there are backups. But I’d rather not have to mess with something like that just to restore a system that should have been decommissioned years ago!

What are FreeBSD jails?

If you’re new to FreeBSD, you may have heard about jails without knowing what they actually are. If you’ve got a Solaris background, think zones. If you’re a Linux user, think containers. FreeBSD people usually cannot understand today’s container hype for the very simple reason that this OS has had jails for ages now (they were first available on development versions at the very end of the 20th century!). You are probably familiar with docker, Linux’s cgroups and all those facilities. FreeBSD basically does the same thing: Provide much enhanced but still very light-weight chroot environment.

The system in that jail is isolated from the actual host operating system so that escaping it isn’t trivial and requires special (insecure) settings. In general jails are meant to be secure after all, aren’t they? You can also give jails their own virtual network stack, you can cap their resource usage, and so on. It’s a mature, stable and secure way to do containerization of applications – or whole operating system installations.

Since FreeBSD is pretty good at keeping compatibility with old releases (for the next major release they are discussing whether to keep compatibility shims that allow executing binaries from commercial UNIX releases of the 80’s!!), it’s very much possible to stuff a 4.11 userland into a jail running on a recent version of the operating system. And that’s what we’re going to look at after getting some of the basics out of the way.

Jail frameworks

Jails in FreeBSD can be created and managed manually. We’re talking *nix here, after all! However there are several jail management frameworks available that promise to make things even easier or at least more convenient. If you want to get started with jails, I suggest that you pick a framework that sounds good to you feature-wise and simply begin playing with it. The whole thing is not that complicated actually.

If you’re just getting started, don’t bother with sysutils/warden. It was a popular jail manager created for PC-BSD but it is considered obsolete now.

One true classic among the jail frameworks is sysutils/ezjail. It still does the job and a lot of people continue to use it simply because they are already familiar with it. It allows for things like thin-provisioned jails that use a “base jail” which holds a complete FreeBSD userland, and uses nullmount to make that accessible in other jails. This makes updating very quick and convenient. It also leaves the base system read-only which may be another nice security feature at the cost of decreased flexibility. On the plus side Ezjail is independent on filesystems and thus works on UFS, too.

Then there was sysutils/iocage, a modern approach that made use of ZFS properties to store certain values. For that reason it demands at least one zpool on the system to be usable. It had been implemented in shell code which made it easy to read for admins who don’t have a lot of coding experience. It has however been rewritten and the old shell version is declared obsolete. If you like the idea of having it written in shell, take a look at sysutils/iocell, which is a fork of the last shell version and is actively maintained.

The new version of iocage is implemented in Python (initially it was announced to be rewritten in Go but that decision was revoked). Another warning: sysutils/py-iocage is the wrong version, too! Do not use it. The rewrite initially supported Python 2.7 – which is what this port is all about – but newer versions dropped support for that and now require Python 3.6. So the actual port you should use is sysutils/py3-iocage! That version requirement makes sense but unfortunately it leads to iocage not being available from packages (as FreeBSD’s default version of Python 3 is still 3.3 currently). So you probably need to build it from ports.

There’s one more framework that I want to mention: sysutils/cbsd. It is an advanced system that not only does jails but also allows to manage bhyve (FreeBSD’s modern hypervisor). If you’re looking for something more comprehensive you may well give it a try.

Choosing my framework

When I first wanted to get into jails, I tried out ezjail. It worked well for me and I’d still use it on a system with no ZFS available. When I have ZFS, though, I liked the iocage approach. Cbsd looks quite good but I didn’t do too much with it because I require things to be scriptable (like using the framework through SaltStack states – there’s even a Salt Formula for iocage, but that’s for the old implementation) and that doesn’t seem to be what they focus on. So when the shell based iocage was deprecated, I moved to iocell and then I was torn to stick with that or to return to the new iocage. In the end I opted for at least giving the rewrite a chance. Since it does its thing quite nicely, I’ve adopted that and will use it here.

You can of course use any other framework if you prefer a different one. Or you could setup your jail by hand – if you’re thinking about that, definitely take a look at this article by M. W. Lucas whom I actually have to thank not only for his post but kind of for this one as well. The thought of jailing the old systems crossed my mind once or twice, but if he hadn’t written about it in the past, I don’t think I’d ever have tried it (no, I read his post back in the day when he published it; yes, I wanted to try this for over three years now before I finally found some time to give it a shot!).

Preparations

Let’s install iocage first. Like stated before, we will have to build it from ports. Assuming a system that does not have a ports tree installed, this is how you can do it (there are better (cleaner) ways to build packages from ports, I planned to do a post about that topic since December!) but this old method still works:

# portsnap fetch extract
# cd /usr/ports/sysutils/py3-iocage
# make config-recursive
# make install clean

The new iocage needs a UTF-8 locale set on the system or it won’t run. Unfortunately this is not the standard in FreeBSD (this is something that I really hope for in FreeBSD 12!). There are multiple ways to do this, my preferred one to set it in /etc/login.conf:

Look for default:\ – it is followed by a block of indented lines, the last one being :umask=022:. Add another backslash at the end to make the block continue and add two more lines to it:

:charset=UTF-8:\
:lang=en_US.UTF-8:

Login configuration is one of the few things where the actual values are in fact stored in a database. The changes that you made to the text file won’t take effect even if you log out and back in. First run another command to update the DB according to the changes in the text file:

cap_mkdb /etc/login.conf

Now log out and back in. Try echoing $LANG – that should print the UTF locale’s name. If it does, you are set.

What’s next?

The next post will demonstrate how to use iocage to manage jails and show how to jail a FreeBSD 4.11 system so it runs on a FreeBSD 11 host.

Updating FreeBSD 4.11 (4/4) – Reflecting radical resurrection

In the first post of this mini series I wrote about legacy systems and installing FreeBSD 4.11. The second one shows how to configure the fresh system for remote access, bootstrap Pkgsrc, install Subversion to checkout FreeBSD code and update the system to the stable branch. And part three mainly deals with upgrading OpenSSH and the compilers. This post details some more updates until we reach the final state that’s possible with such an old system (without resorting to extreme means).

Planting a new tree

So far we’ve built some packages from 2013 and before. Using a current pkgsrc tree won’t work – the various pkgsrc tools that our system has are too old. It might not be too big a step but we can use a tree from the second half of 2014. Of course the newer SSH that we built before is not currently in the path so we need to create a temporary symlink before we can use CVS again:

# ln -s /usr/local/temp/bin/ssh /usr/local/pkgsrc/bin/ssh
# rehash

# cd /usr/pkgsrc
# cvs -danoncvs@anoncvs.netbsd.org:/cvsroot get -rpkgsrc-2014Q3 -P pkgsrc
# mv pkgsrc 14
# rm /usr/local/pkgsrc/bin/ssh

What the system looks like package-wise at the beginning of part 4

Most of pkgsrc’s tools make use of NetBSD’s compatibility library. Unfortunately the version that comes with the new pkgsrc tree won’t build anymore on an OS as old as FreeBSD 4.11. Same thing for libfetch. But the newer tools will work with older versions of that libs, too. So let’s prepare those two – libfetch need’s some more love to build:

# cd /usr/pkgsrc/13/pkgtools/libnbcompat
# bmake

# cd /usr/pkgsrc/13/net/libfetch
# cp Makefile Makefile.bak
# sed '14i\\
CFLAGS=         -Wno-error' Makefile.bak > Makefile
# bmake

As a next step we’re going to do two updates. Yes, in theory we could use “bmake update” to update packages. We will not do that. The reason is that we needed to abuse pkgsrc quite a bit so far by mixing package versions from various trees. Since “bmake update” is a destructive command (it will happily uninstall programs as well as packages depending on them!) this can lead to all sort of fun things like unresolvable dependencies and such.

If you like pain, go ahead. I’ve been there and I can confirm that it does work for some packages. For a lot of them actually. But in those cases where it doesn’t, it tends to do so much damage that you’re better off starting over than trying to fix things… That’s why I’ll show you a safer method instead: Build a package and update via pkg_add! Also it really starts to show how old the system is that we’re trying to build rather new packages on. More and more of them require some trickery to persuade them to build – but hey, we’re doing a gross thing here, anyway. So there’s no real reason to complain!

# cd /usr/pkgsrc/14/pkgtools/pkg_install
# bmake extract
# rm -r work/libnbcompat/*
# rm -r work/libfetch/*
# cp -R /usr/pkgsrc/13/pkgtools/libnbcompat/work/libnbcompat-20120702/* /usr/pkgsrc/14/pkgtools/pkg_install/work/libnbcompat/
# cp -R /usr/pkgsrc/13/net/libfetch/work/libfetch-2.34/* /usr/pkgsrc/14/pkgtools/pkg_install/work/libfetch/
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/14/packages/All/pkg_install-20130902nb1.tgz

# cd /usr/pkgsrc/14/pkgtools/bootstrap-mk-files
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/14/packages/All/bootstrap-mk-files-20140516.tgz

We made it so far, now let’s make a daring move and just download the latest stable pkgsrc tree – released in January 2017:

# cd /usr/pkgsrc
# fetch http://cdn.netbsd.org/pub/pkgsrc/stable/pkgsrc-2016Q4.tar.bz2
# tar xvjf pkgsrc-2016Q4.tar.bz2
# rm pkgsrc-2016Q4.tar.bz2
# mv pkgsrc 16

Updating pkgsrc tools

Since mid 2014, pkgsrc makes use of a new package, cwrappers. During my test run I somehow managed to just get this package built. Despite taking notes I have no idea what I did to just make it work! It must have been something that looked like a dead end (which is why I didn’t include it in my notes) but somehow provided “getline”… I tried to get it working again for almost one whole Sunday but for the life of me couldn’t find out what I previously did… In the end I gave up and tried to find another solution. I found one but while it is way more complex it at least means that I got rid of that nasty blocker again:

# cd /usr/pkgsrc/16/pkgtools/cwrappers
# bmake extract
# rm -r work/libnbcompat/*
# cp -R /usr/pkgsrc/13/pkgtools/libnbcompat/work/libnbcompat-20120702/* /usr/pkgsrc/16/pkgtools/cwrappers/work/libnbcompat/
# cp work/cwrappers-20161125/mi_vector_hash.c work/cwrappers-20161125/mi_vector_hash.c.bak
# cp work/cwrappers-20161125/fixup-libtool.c work/cwrappers-20161125/fixup-libtool.c.bak
# sed 's/stdint.h/inttypes.h/' work/cwrappers-20161125/mi_vector_hash.c.bak > work/cwrappers-20161125/mi_vector_hash.c
# sed 's/stdint.h/inttypes.h/' work/cwrappers-20161125/fixup-libtool.c.bak > work/cwrappers-20161125/fixup-libtool.c
# cp /usr/pkgsrc/14/pkgtools/cwrappers/files/bin/getline.c /usr/pkgsrc/16/pkgtools/cwrappers/work/cwrappers-20161125/getline.c.bak
# sed 's/ssize_t/size_t/' work/cwrappers-20161125/getline.c.bak > work/cwrappers-20161125/getline.c
# cp work/cwrappers-20161125/common.h work/cwrappers-20161125/common.h.bak
# sed '107i\\
size_t  getline(char **, size_t *, FILE *);' work/cwrappers-20161125/common.h.bak > work/cwrappers-20161125/common.h
# cp work/cwrappers-20161125/Makefile work/cwrappers-20161125/Makefile.bak
# sed '14i\\
LIB_SRCS+=      getline.c' work/cwrappers-20161125/Makefile.bak > work/cwrappers-20161125/Makefile
# bmake install clean clean-depends

Phew! Fortunately the next few updates are straight forward:

# cd /usr/pkgsrc/16/pkgtools/bootstrap-mk-files
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/bootstrap-mk-files-20160908.tgz

# cd /usr/pkgsrc/16/devel/bmake
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/bmake-20150505.tgz

# cd /usr/pkgsrc/16/net/tnftp
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/tnftp-20151004nb1.tgz

Next is another one that requires some patching:

# cd /usr/pkgsrc/16/pkgtools/digest/
# bmake extract
# cp work/digest-20160304/sha3.h work/digest-20160304/sha3.h.bak
# cp work/digest-20160304/keccak.c work/digest-20160304/keccak.c.bak
# cp work/digest-20160304/keccak.h work/digest-20160304/keccak.h.bak
# cp work/digest-20160304/sha3.c work/digest-20160304/sha3.c.bak
# sed 's/stdint.h/inttypes.h/' work/digest-20160304/sha3.h.bak > work/digest-20160304/sha3.h
# sed 's/stdint.h/inttypes.h/' work/digest-20160304/keccak.c.bak > work/digest-20160304/keccak.c
# sed 's/stdint.h/inttypes.h/' work/digest-20160304/keccak.h.bak > work/digest-20160304/keccak.h
# sed 's/stdint.h/inttypes.h/' work/digest-20160304/sha3.c.bak > work/digest-20160304/sha3.c
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/digest-20160304.tgz

Updating installed packages

Let’s update gettext first as a lot of packages need that one; xz is one of the packages that is linked against the old one and since libintl received a soname bump, it needs to be rebuilt. Since we want to update it anyway that’s not too bad. But there are other packages that we cannot update which depend on the old lib. So we’ll have to create a symlink to satisfy their need, too:

# cd /usr/pkgsrc/16/devel/gettext-lib
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/gettext-lib-0.19.8.1.tgz

# cd /usr/pkgsrc/16/archivers/xz
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/xz-5.2.2.tgz

# ln -s /usr/local/pkgsrc/lib/libintl.so.9 /usr/local/pkgsrc/lib/libintl.so.7
# cd /usr/pkgsrc/16/devel/gettext-tools
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/gettext-tools-0.19.8.1.tgz

Next in line is some more typical build dependencies:

# cd /usr/pkgsrc/16/devel/libtool-base
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/libtool-base-2.4.2nb13.tgz

# cd /usr/pkgsrc/16/devel/m4
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/m4-1.4.17.tgz

# cd /usr/pkgsrc/16/devel/bison
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/bison-3.0.4nb3.tgz

Just a few more packages and we’ll have updated most packages that can be updated (a few like zip and nbpatch can’t):

# cd /usr/pkgsrc/16/shells/bash
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/bash-4.4.005.tgz

# cd /usr/pkgsrc/16/lang/perl5
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/perl-5.24.0.tgz

# cd /usr/pkgsrc/16/devel/autoconf
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/autoconf-2.69nb7.tgz

# cd /usr/pkgsrc/16/devel/gmake
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/gmake-4.1nb3.tgz

Rebuilding the compiler

First we need to update the two math libraries (and create another symlink so we can go on compiling):

# cd /usr/pkgsrc/16/devel/gmp
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/gmp-6.1.2.tgz
# ln -s /usr/local/pkgsrc/lib/libgmp.so.13 /usr/local/pkgsrc/lib/libgmp.so.11

# cd /usr/pkgsrc/16/math/mpfr
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/mpfr-3.1.5.tgz

This unfortunately breaks the compiler. But we can still resort to the old GCC3 to build GCC4 again, right? Right:

# cp /usr/pkgsrc/13/distfiles/gcc-4.4.7.tar.bz2 /usr/pkgsrc/16/distfiles
# cp /root/.cshrc /root/.cshrc.bak
# sed 's:pkgsrc/gcc44:temp/gcc34:' /root/.cshrc.bak > /root/.cshrc
# source /root/.cshrc
# cc -v

While we probably still don’t need Object-C or Java we could in fact build GCC with them this time. Java requires Python2.7 installed but that can actually be built from the 2014 tree! The problem is that building Java requires more RAM than is available on 32 bit machines and will for that reason fail. However Java is deactivated by default for GCC 4.4 in the 2016 tree. So let’s just get rid of our custom options, build the default package and set the correct path again:

# cp /usr/local/pkgsrc/etc/mk.conf /usr/local/pkgsrc/etc/mk.conf.bak
# sed '/PKG_OPTIONS.gcc44/d' /usr/local/pkgsrc/etc/mk.conf.bak > /usr/local/pkgsrc/etc/mk.conf
# cd /usr/pkgsrc/16/lang/gcc44
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/gcc44-4.4.7nb7.tgz

Almost everything updated!

Now we only have to restore the correct path and then we have the GCC4 back (with a newer patch level):

# cp /root/.cshrc.bak /root/.cshrc
# source /root/.cshrc
# cc -v
gcc version 4.4.7 (GCC)

Modern OpenSSH

There’s one more package to build that needs a bit of care: Pkgconf. It’s a simpler replacement for the older pkg-config but it won’t work out of the box for us:

# cd /usr/pkgsrc/16/devel/pkgconf
# bmake extract
# cp work/pkgconf-1.0.1/libpkgconf/stdinc.h work/pkgconf-1.0.1/libpkgconf/stdinc.h.bak
# cp work/pkgconf-1.0.1/getopt_long.h work/pkgconf-1.0.1/getopt_long.h.bak
# sed 's/stdint.h/inttypes.h/' work/pkgconf-1.0.1/libpkgconf/stdinc.h.bak > work/pkgconf-1.0.1/libpkgconf/stdinc.h
# sed 's/stdint.h/inttypes.h/' work/pkgconf-1.0.1/getopt_long.h.bak > work/pkgconf-1.0.1/getopt_long.h
# bmake install clean clean-depends

Finally the time has come to do what I wanted to do in the first place, provide a recent version of OpenSSH! Of course it’s also necessary to generate new host keys once more. And then, just to prove everything works when the machine boots, let’s just restart the machine after adjusting the sshd path:

# cd /usr/pkgsrc/16/security/openssh
# bmake install clean clean-depends
# rehash
# ssh -V

# ssh-keygen -f /usr/local/pkgsrc/etc/ssh/ssh_host_rsa_key -N '' -t rsa
# ssh-keygen -f /usr/local/pkgsrc/etc/ssh/ssh_host_dsa_key -N '' -t dsa
# mkdir -p /usr/local/pkgsrc/run

# cp /etc/rc.conf /etc/rc.conf.bak
# sed 's:temp/sbin:pkgsrc/sbin:' /etc/rc.conf.bak > /etc/rc.conf
# shutdown -r now

Generating new host keys for OpenSSH

Now we need to remove the vierelf entry in ~/.ssh/known_hosts before we connect again. Doing so in verbose mode even shows that the 4.11 box now has a newer version of OpenSSH installed that the FreeBSD 11 workstation that I use to connect to it! 😀

FreeBSD 4.11 running a newer OpenSSH than my FreeBSD 11.0 workstation!

Conclusion

FreeBSD 4.11 is really, really, really old now. But you can get surprisingly far in running somewhat modern software on it – more recent software at least than I initially thought would be possible! And you? What was your bet? Would you have guessed that I’d make it up to the 2016Q4 pkgsrc tree and even install the latest version of OpenSSL and OpenSSH?

Here’s a little summary of some important program updates:

binutils 2.12.1 (2002) -> binutils 2.17 (2006)
perl 5.005 (1998) -> perl 5.8 (2002) -> perl 5.18 (2013) -> perl 5.24 (2016)
GCC 2.95.4 (2001) -> GCC 3.4.6 (2006) -> GCC 4.4.7 (2012)
OpenSSH 3.5 (2002) -> OpenSSH 4.6 (2007) -> OpenSSH 7.3 (2016)

Not too bad, eh? The notable exception here is binutils. Newer versions would probably be possible but there’s a gap in pkgsrc – which stuck with 2.17 for a long time and then directly moved to 2.22 which no longer builds on FreeBSD 4.11. GCC 4.5.3 does build BTW but something goes sideways and the comparison of stage 2 and 3 fails for quite some files.

I’ve met my initial goal to provide a newer version of OpenSSH, surpassing all expectations that I had. There’s room for more of course but that’s not worth another post. I’m going to add sudo and since Python 2.7 can be built it might even be possible to manage the 4.11 servers using salt-ssh (the ordinary SaltStack doesn’t work as it requires ZeroMQ which looks like it cannot be built)! We have a recent version of bash and can thus do some pretty nifty things with the right .bashrc.

This whole adventure took far longer than I had anticipated – a bit over a month instead of the intended two weekends! But that was mostly because I decided to start over with a clean system several times to ensure that everything works as I wrote it down here (and because GCC4 simply takes so long to build on the only spare machine that I had for this…). But it has been an interesting ride and I don’t regret spending some time on the legendary FreeBSD 4.11!

Oh, and my special thanks to everybody involved with Pkgsrc! I usually don’t have much use for NetBSD but Pkgsrc is extremely useful. I might use it in the future on other systems (like Linux), too. And thanks to you for reading. I hope that you enjoyed it as well!

Updating FreeBSD 4.11 (3/4) – Neophyte’s notorious necromancy

The first post of this mini series was about legacy systems in general and about what installing the old FreeBSD 4.11 is like. In the second one I showed the initial configuration of the system, how to SSH into it despite the obsolete DSA host key and how to bootstrap pkgsrc, NetBSD’s portable ports tree. I also covered the installation of SVN, checking out of the 4.11-STABELE code and updating the system. This post will cover installing newer software.

Any bets?

So far we have a pkgsrc tree from mid 2007 and things seem to be working. However that’s pretty close to 4.11’s release in 2005 and thus not too amazing. Working with such an old system there are plenty of cases which mean “game over”. Here are just three errors of that kind which you can encounter trying to build more modern software:

/usr/libexec/elf/ld: cannot find -lpthread

There’s no modern pthreads available on 4.11. Game over.

/usr/include/sys/resource.h:58: error: field 'ru_utime' has incomplete type

We’ll have to do with very old system headers missing a lot of what we take for granted today. Game over again.

fileio.o(.text+0x354): undefined reference to `towupper'
collect2: ld returned 1 exit status

Sorry, that ancient libc that we have on our system doesn’t provide that symbol. Game over yet again.

How far do you think can we take it in building and installing more recent software? Make a guess now and see if you were right! To be honest I was not expecting the end result. Not at all. So let’s get back to work!

In for a screening

We’re going to compile a lot of stuff this time – building SVN and dependencies was just a warm up. And what do you do when you’re building stuff remotely over SSH? You’re doing so in a screen or tmux session of course. Neither is part of the base system so we’ve got to build one. Tmux was not yet available in 2007 so it’s not too hard a choice:

# cd /usr/pkgsrc/07/misc/screen
# bmake install clean clean-depends
# rehash
# screen

GNU screen started up and ready

If you don’t know screen do some reading because you will want to start using it (or rather the superior tmux). It basically allows you to detach from a session and reconnect later – and your programs will continue running on the remote system even while you’re logged out. You can also resume the session from another terminal or computer, share sessions, etc. And that’s just one of the things that it does. There are other features like allowing you to have multiple shell instances in just one terminal between which you can switch back and forth (think tabs of a browser) and a lot more. Should you not like this (what’s wrong with you?!), fine. Don’t install screen. It’s optional.

Replacing the front door lock

Now it’s time to take care of the main problem of our system: That darned version 3.5 of OpenSSH! Let’s build whatever our pkgsrc tree has to offer:

# cd /usr/pkgsrc/07/security/openssh
# bmake install clean clean-depends
# rehash
# ssh -V
OpenSSH_4.6p1, OpenSSL 0.9.7d-p1 17 Mar 2004

Still far from a modern version of OpenSSH but also a lot better. And the best thing: It supports RSA keys. Let’s generate host keys with this newer SSH and make it the version that FreeBSD launches during startup:

# ssh-keygen -f /usr/local/temp/etc/ssh/ssh_host_key -N '' -t rsa1
# ssh-keygen -f /usr/local/temp/etc/ssh/ssh_host_rsa_key -N '' -t rsa
# ssh-keygen -f /usr/local/temp/etc/ssh/ssh_host_dsa_key -N '' -t dsa
# mkdir -p /usr/local/temp/run
# echo 'sshd_program="/usr/local/temp/sbin/sshd"' >> /etc/rc.conf

Ok, everything is in place. We could reboot now – or just kill off the old daemon and launch the new one. Let’s first look for SSHD and see which PID it has (this of course varies from system to system!):

# ps aux | grep sshd

Replacing SSHD

Got it? Great, let’s kill it (your SSH connection is maintained by a child and it’s generally save to kill the parent. You won’t lose your SSH connection!), start the new one and ensure that it’s running:

# kill [PID on your system]
# /usr/local/temp/sbin/sshd
# ps aux | grep sshd

What’s this? It looks like it’s not running! Yes, it looks like it but actually it should be running… Let’s grep again:

# ps aux | grep local

This does return one process – and trust me it’s actually our new sshd. What’s happening here is this: The output of ps is truncated because more wouldn’t fit on the screen. And only that data is handed to grep! So the process with the name /usr/local/temp that we found (see the screenshot above) is actually /usr/local/temp/sbin/sshd with the last part of it cut off… This is why grep doesn’t find “sshd”. There’s a funny way to fix this, though: Maximize your terminal emulator so that more space is available. Then grep will find sshd!

Now we can quite the old SSH session so we can make one with the new server. We can even keep our screen session open, but we need to detach from it by pressing CTRL-A and then D before we logout from vierelf:

[detached]

# logout
> exit
Connection to 192.168.1.5 closed.

Time to edit your known hosts and get rid of the former host key for vierelf or else you’ll see that scary SSH warning when you try to login again. Oh, and you can leave out that compatibility option from now on – which is a major step ahead! When you’re back in, you can resume the screen session:

% ssh kraileth@192.168.1.5
> su -
# screen -r

Connecting to the new SSH server (debug mode)

Compiler: from antediluvian to ancient

Alright. Currently we have the last version of the second generation of GCC on our system. We totally need to get our hands on something newer. How about updating the last version of generation three? Let’s try that! We only want the C and C++ compilers. Fortran is deactivated by default for this version (it would need GMP installed and the version of GMP that’s in the tree requires GCC3. It’s a good idea to avoid that potential circular dependency). However Java and Object-C are activated. There’s no need to waste time on them, they should be deactivated as well. The following sed command may look a bit complex, but it’s not that bad. Just copy all three lines that make up that single command and you’re good to go:

# cd /usr/pkgsrc/07/lang/gcc34
# cp Makefile Makefile.bak
# sed -e '64,65d' -e '63a\\
BUILD_JAVA?=    NO' -e '63a\\
BUILD_OBJC?=    NO' Makefile.bak > Makefile
# bmake install clean clean-depends

After installing that newer GCC, the path needs to be changed again so that the system picks it up instead of the older system compiler:

# vi /root/.cshrc

Prepend the following path to the PATH variable:

/usr/local/temp/gcc34/bin

Now let’s log out and in again and see if the new compiler is available:

# exit
[screen is terminating]
# logout
> su -
# screen
# cc -v
[...]
gcc version 3.4.6

Updating pkgsrc

Since we also have a more recent OpenSSH now, we can checkout a newer copy of pkgsrc from CVS! That takes a while, be patient. Even after it is finished downloading (and you see no new lines on the screen) it will still take some time to clean things up. This is normal and you have to wait a little longer. Don’t CTRL+C it as that would leave your tree in bad shape!

# cd /usr/pkgsrc
# cvs -danoncvs@anoncvs.netbsd.org:/cvsroot get -rpkgsrc-2009Q4 -P pkgsrc
# mv pkgsrc 09

Thanks to the newer SSH: CVS works now, too!

We’ll need some ports from there later. But since we have GCC 3 available now we can also grab an even newer copy and primarily use that one:

# cvs -danoncvs@anoncvs.netbsd.org:/cvsroot get -rpkgsrc-2013Q2 -P pkgsrc
# mv pkgsrc 13

We’re going to start a fresh environment, using only GCC (and sshd) from the old one. To do so we first bootstrap the pkgsrc from 2013 into a new directory:

# mkdir /usr/local/pkgsrc
# cd /usr/pkgsrc/13/bootstrap
# ./bootstrap --prefix=/usr/local/pkgsrc --varbase=/usr/local/pkgsrc

The next step is to adjust the path variable so that the binaries from the new location are being used. To do so we need to replace /usr/local/temp with /usr/local/pkgsrc for both sbin and bin. Don’t change the compiler path, though! GCC 3 will remain in temp. After logging out and back in, screen is no longer in PATH so we need to execute it with the absolute path:

# cp /root/.cshrc /root/.cshrc.bak
# sed -e 's:temp/bin:pkgsrc/bin:' -e 's:temp/sbin:pkgsrc/sbin:' /root/.cshrc.bak > /root/.cshrc
# exit
# logout
> su -
# /usr/local/temp/bin/screen

Cherry-picking dependencies

This gives us a way to easily build software from 2013. Let’s continue on by fetching some source tarballs by hand that are no longer available on the mirrors that pkgsrc knew for them:

# cd /usr/pkgsrc/09/distfiles
# fetch http://ftp.cc.uoc.gr/mirrors/NetBSD/packages/distfiles/binutils-2.17.tar.gz
# fetch http://ftp.cc.uoc.gr/mirrors/NetBSD/packages/distfiles/pkg-config-0.23.tar.gz

The following part is not too interesting: We’re going to build the dependencies in preparation for the next big step. In general we try to build the newest version possible (2013) but resort to old (2009) or even older (2007) where necessary if newer versions don’t build for various reasons:

# cd /usr/pkgsrc/13/converters/libiconv
# bmake install clean clean-depends

Zip from 2009 and onwards is incompatible with FreeBSD 4.11’s libc. And the 2007 version expects tar in a location where there’s none on our system. Instead of building tar we can safely symlink it:

# ln -s /usr/bin/tar /usr/local/pkgsrc/bin/tar
# cd /usr/pkgsrc/07/archivers/zip
# bmake install clean clean-depends

The binutils are a special case. The port normally builds the programs of which it consists with a prefix so they don’t get in the way of the system binaries. Since we actually want to use them instead of the old stuff from the base system, we need to get rid of that prefix:

# cd /usr/pkgsrc/09/devel/binutils
# bmake GNU_PROGRAM_PREFIX='' install clean clean-depends
# rehash
# ld -v
GNU ld version 2.17

The next few are trivial:

# cd /usr/pkgsrc/09/devel/gettext-tools
# bmake install clean clean-depends

# cd /usr/pkgsrc/13/devel/m4
# bmake install clean clean-depends

# cd /usr/pkgsrc/09/devel/bison
# bmake install clean clean-depends

The bash port from 2013 would draw in a newer version of gettext which would not build. But bash can actually be built with the old one, too. So we have to make a simple change in the buildlink file for gettext in 2013’s pkgsrc tree:

# cd /usr/pkgsrc/13/devel/gettext-lib
# cp buildlink3.mk buildlink3.mk.bak
# sed 's/0.18/0.14/g' buildlink3.mk.bak > buildlink3.mk

With that change the next port can be built:

# cd /usr/pkgsrc/13/shells/bash
# bmake install clean clean-depends

Next in line is perl. The 2013 port would however build with dtrace support by default – which was of course not available on 4.11. Therefore it needs to be switched off by making an addition to the pkgsrc config file:

# vi /usr/local/pkgsrc/etc/mk.conf

Add the following line at the end of the file (but above .endif):

PKG_OPTIONS.perl=       -dtrace

Now let’s build the last few dependencies:

# cd /usr/pkgsrc/13/lang/perl5
# bmake install clean clean-depends

# cd /usr/pkgsrc/13/archivers/xz
# bmake install clean clean-depends

# cd /usr/pkgsrc/09/devel/autoconf
# bmake install clean clean-depends

Compiler: from ancient to old

With this all dependencies from earlier than 2013 in place we are good to go for the biggest update. We’re still not interested in Java and Object-C, so let’s edit pkgsrc’s configuration again:

# vi /usr/local/pkgsrc/etc/mk.conf

and add one more line (e.g. after the perl one):

PKG_OPTIONS.gcc44=      -gcc-java -gcc-objc

Building the newer version of GCC means building two more dependencies as well, one of which is libgmp. GMP is the first package so far that uses C++ and in fact our C++ compiler has been broken the whole time. Luckily a symlink can heal it and another one will make GCC happy so that we can finally build it – which takes quite a bit of time (I’ve seen the compilation stop at one point and I’m not sure what happens there. But just calling bmake again will eventually complete the build process!):

# ln -s /usr/local/pkgsrc/lib/libiconv.so.7 /usr/lib/libiconv.so.7
# ln -s /usr/local/temp/gcc34/lib/libgcc_s.so.1 /usr/lib/libgcc_s.so.1
# cd /usr/pkgsrc/13/lang/gcc44/
# bmake install clean clean-depends

Once it’s build, we need to change our PATH so that the newer GCC is the primary compiler:

# mv /root/.cshrc /root/.cshrc.bak
# sed 's:temp/gcc34:pkgsrc/gcc44:' /root/.cshrc.bak > /root/.cshrc

Now all that we have to do is log out and back in:

# exit
# logout
> su -
# /usr/local/temp/bin/screen

Let’s take a look if the new compiler responds to cc (and fix c++ support along the way):

# ln -sf /usr/local/pkgsrc/gcc44/lib/libgcc_s.so.1 /usr/lib/libgcc_s.so.1
# cc -v
[...]
gcc version 4.4.7 (GCC)

GCC 4.4.7 running on FreeBSD 4.11

Yes, we really have GCC 4.4 running on FreeBSD 4.11! While it’s certainly not a modern compiler, it’s recent enough to build a lot of software. The latest release of OpenBSD, version 6.0 released on September 2016, still comes with GCC 4.2, BTW! Yes, OpenBSD maintained that all the time and heavily patch it. Still we now actually have a compiler available on FreeBSD 4.11 from 2005 which is two major versions newer!

With this we’re kind of back in business. But this post is already becoming quite long and for that reason I’m putting the “grand finale” off to one more post. See you there for the final outcome of this “little” experiment (which I hadn’t intended to write more than three posts for, but there you have it).

Updating FreeBSD 4.11 (2/4) – Digging up old graves

In part one I wrote about Legacy systems in general and showed a FreeBSD 4.11 installation for those of my readers who are interested in software history.

This post is about the first part of updating this fresh 4.11 system to a state that’s a bit less catastrophic. Remember: FreeBSD 4.11 was released in 2005 – however the ABI of each release is carved in stone with a .0 release. Which means that the software in the base system is from 4.0 and thus we venture back into the last millenium: 1999!

Initial state

To give you an idea what this means, here are a few program versions:

Various program’s versions in 4.11’s base system

So we have these programs among others:

GCC 2.95.4
Binutils 2.12.1
Perl 5.0
OpenSSH 3.5

To make matters worse, the ports tree for FreeBSD 4.11 is pretty dead, too. It’s important to get newer compilers running, but around 2005 FreeBSD used special releases to build GCC from (“gcc-core”) and I was not able to find a single mirror on the net that still holds those old and exotic files! Out of luck here. We’ll have to do without those ports.

“Modernizing” this is going to be interesting… Considering how fast the IT world moves, all of this is just as dead as it gets. So let’s prepare for the (code) smell and start digging up an old grave, shall we?

Allowing remote connection

After a passwordless login as root it makes sense to set up the right keymap (if you don’t use the default, that is). I had no idea how to do it on 4.11 and so I just gave the usual way of doing it a try – and was met with success:

# kbdmap

Looks like the keymap selection has not changed in all the time! Let’s try to make it persistent:

# echo keymap=german.iso.kbd >> /etc/rc.conf

I tried it out and it did just what I wanted. Time to try to add a regular user:

# adduser

The script is a bit different from what we’re used to today but in the end it does the same thing: Allow us to create a user. It’s important to add this user to the wheel group so that we’re able to su to root. However we need to give root a password first:

The useradd script in FreeBSD 4.11

# passwd

Ok, sshd should be running. Let’s check just to be sure:

# ps aux | grep sshd
[...]
root        75  0.0  0.1  2600 2040  ??  Is    6:26AM   0:00.11 /usr/sbin/sshd

Looks good. What about connectivity? Let’s see:

# ifconfig
vr0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        ether 00:05:5d:96:fa:f9
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
[...]

Nope, no connection. Luckily I paid attention when I the installer started from the CD and somewhere the strange-looking path of /stand caught my eye. Doing a little research, I found out that this directory used to be part of the OS and was removed in early FreeBSD 5 as it was mostly redundant with /rescue. What was in there, you ask? Have a look yourself:

# ls /stand
-sh             etc             minigzip        rm              tunefs
[               find            mount_mfs       route           usbd
arp             fsck            mount_nfs       rtsol           usbdevs
boot_crunch     gunzip          newfs           sed             zcat
camcontrol      gzip            pccardc         sh
cpio            help            pccardd         slattach
dhclient        hostname        ppp             sysinstall
dhclient-script ifconfig        pwd             test

Network configuration

There’s our friend sysinstall. I already said that it does more than just install the system. So let’s bring it up now:

# /stand/sysinstall

Sysinstall to configure the network

There we choose Networking -> Interfaces -> the appropriate NIC. No, I don’t want IPv6 and yes, DHCP is the thing.

Interface configuration using sysinstall

I’ve called the system vierelf which would be “foureleven” in English because I couldn’t think of anything better. And it’s just a test system anyway. Does the connection work now?

# ping elderlinux.org
PING elderlinux.org (212.77.232.71): 56 data bytes
64 bytes from 212.77.232.71: icmp_seq=0 ttl=54 time=24.993 ms

A little housekeeping

Alright! Let’s just take a look what services are listening right now:

# sockstat -4 -l
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS      
root     dhclient   281    7 udp4   *:68                  *:*                  
root     sendmail    84    4 tcp4   *:25                  *:*                  
root     sendmail    84    6 tcp4   *:587                 *:*                  
root     sshd        75    4 tcp4   *:22                  *:*                  
root     syslogd     61    5 udp4   *:514                 *:*

Ugh! Time to make a few changes in /etc/rc.conf to deactivate any daemons except for SSH (which we need). This may not be strictly necessary but we want to improve the security of this system, right? And I wouldn’t trust those crusty old daemons at all. And whatever is not running won’t cause us any problems. So let’s get rid of them with extreme prejudice!

# vi /etc/rc.conf

To do so, add daemonname_enable=”NO” for each daemon and in case of sendmail use sendmail_enable=”NONE”.

FreeBSD 4.11 rc.conf with most daemons disabled

If you reboot now, sendmail and syslogd as well as cron, usbd and inetd will be disabled. That’s a starting point in securing the system. Let’s move on.

Replacing a dead tree

I’ll connect to the 4.11 box remotely over SSH because it’s much more convenient to have my trusty terminal at hand and to be able to copy and paste stuff:

% ssh kraileth@192.168.1.5
Unable to negotiate with 192.168.1.5 port 22: no matching host key type found. Their offer: ssh-dss

Ouch! I cannot even SSH into the system because its version of OpenSSH is so old that it only offers ssh-dss keys which have been deprecated for quite a while and disabled by default in OpenSSH >=7.0. So to connect to that old server, I have to tell my SSH client to accept ssh-dss for this connection:

% ssh kraileth@192.168.1.5 -oHostKeyAlgorithms=+ssh-dss

SSH login using “-oHostKeyAlgorithms=+ssh-dss”

Ok, I’m in. But what now? We cannot use FreeBSD’s ports tree but I strongly prefer some means of package management over installing stuff using make install. So how do we accomplish this? Enter pkgsrc. Pkgsrc is basically NetBSD’s fork of the FreeBSD ports tree. Being a NetBSD project however, it’s not limited to just NetBSD. It’s a truely portable way of building and managing software (I might write a separate post about it some time).

There’s just one problem: Downloading and decompressing the latest pkgsrc release (currently 2016Q4) won’t complete the bootstrapping process. Obviously FreeBSD 4.11 is no longer supported – which is not so much of a surprise. Time to try out older releases! After doing so I found out that 2009Q4 seems to be the last release to bootstrap successfully.

But here’s another problem: Pkgsrc doesn’t seem to keep older releases around and I also haven’t found them mirrored anywhere on the net. Pkgsrc uses CVS, however. So it’s possible to checkout older versions. FreeBSD comes with CVS as part of the base system. Unfortunately CVS works over SSH. And NetBSD’s CVS server won’t accept ssh-dss (which totally makes sense)! Since we don’t control the server, there’s also no way to just add a parameter or something to make it work. It simply doesn’t work that way.

Time to get CVS on my slightly more modern FreeBSD 11, do the checkout there and tar it all up to copy it over via scp! We’re going to get 2007Q2 instead, though, since we need things that won’t work on FreeBSD 4.11 with later versions. Oh, and if you’re not familiar with CVS, don’t worry. You don’t need to know what modules or tags are. Just copy the commands that I prepared for you and you’re good to go:

% sudo pkg install cvs
% cvs -danoncvs@anoncvs.netbsd.org:/cvsroot get -rpkgsrc-2007Q2 -P pkgsrc
% tar cvjf pkgsrc2007Q2.tbz2 pkgsrc/*
% scp pkgsrc2007Q2.tbz2 kraileth@192.168.1.5:/usr/home/kraileth

Then back on vierelf the next step is to prepare some directories and extract the pkgsrc tarball:

# mkdir -p /usr/local/temp /usr/pkgsrc
# cd /usr/pkgsrc
# mv /usr/home/kraileth/pkgsrc2007Q2.tbz2 .
# tar xvjf pkgsrc2007Q2
# rm pkgsrc2007Q2.tbz2
# mv pkgsrc 07

Bridgehead in hostile territory

Now we can bootstrap pkgsrc:

# cd /usr/pkgsrc/07/bootstrap
# ./bootstrap --prefix=/usr/local/temp --varbase=/usr/local/temp --pkgdbdir=/usr/local/temp/db

Pkgsrc 2007Q2 bootstrap complete!

Pkgsrc has been bootstrapped successfully! We just need to adjust the path variable so that the system picks up binaries from the new paths (and make those take precedence over the old system binaries). We could just change the PATH variable but it’s better to make the changes persistent. So let’s add the two new paths in the shell’s rc file in front of the others:

# vi /root/.cshrc

This is what needs to be prepended:
/usr/local/temp/sbin /usr/local/temp/bin

Root’s .cshrc file for the first phase pkgsrc

Now simply log out and become root again to have the new environment:

# logout
> su -

As a last step check if everything is right and we can access binaries in both paths:

# which bmake
/usr/local/temp/bin/bmake
# pkg_info 
bootstrap-mk-files-20061111 *.mk files for the bootstrap bmake utility
bmake-20051105nb3   Portable (autoconf) version of NetBSD 'make' utility
tnftp-20050625nb1   The enhanced FTP client in NetBSD
mtree-20070710      Utility for mapping and checking directory hierarchies
pax-20060202nb1     POSIX standard archiver with many extensions
pkg_install-20070710 Package management and administration tools for pkgsrc

Excellent! Now we have a working replacement for FreeBSD’s dead ports tree. This is definitely something that we can build upon.

Bring on some stability!

Lame pun, I know. Nevertheless it makes sense to… err… update the OS to the latest version. This is what we currently have:

# uname -a
FreeBSD vierelf.localdomain 4.11-RELEASE FreeBSD 4.11-RELEASE #0: Fri Jan 21 17:21:22 GMT 2005     root@perseus.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386

We’re on 4.11-RELEASE. The latest code for each release cycle is always in the -STABLE branch. Believe it or not: The latest code change was in April of 2014! The traditional way of getting FreeBSD code was over CVS. No, this time the problem is not ssh-dss. FreeBSD migrated from CVS to SVN (subversion) in 2008. FreeBSD CVS servers have been removed years ago. Therefore the old tools like cvsup are useless. Subversion is needed to checkout the source.

Luckily we have our pkgsrc ready. This old release has a very old port for subversion but that’s fair enough. There are a few source tarballs that are no longer available from the mirrors that pkgsrc knew for them. Not a big problem, we can download those manually:

# cd /usr/pkgsrc/07/distfiles
# fetch http://archive.apache.org/dist/httpd/httpd-2.0.61.tar.bz2
# fetch http://repository.timesys.com/buildsources/e/expat/expat-2.0.1/expat-2.0.1.tar.gz
# fetch http://download.nust.na/pub2/openpkg1/sources/DST/pkgconfig/pkg-config-0.21.tar.gz

Now we can build subversion:

# cd /usr/pkgsrc/07/devel/subversion-base
# bmake install clean clean-depends

Various dependencies will be downloaded, built and installed. Eventually subversion will be installed and available on the system. Time to tell the shell to look for new binaries and then checkout the stable source code:

# rehash
# svn co svn://svn.freebsd.org/base/stable/4 /usr/src

SVN checkout of the 4.11-STABLE code

The FreeBSD 4 base system never knew anything beyond CVS and cannot cope with the .svn directories that the svn checkout creates. World builds but the installation fails like this:

install: /usr/libdata/perl/5.00503/./.svn/text-base/Cwd.pm.svn-base: No such file or directory

Therefore it’s necessary to get rid of those disruptive directories:

# cd /usr/src
# find . -iname '.svn' -exec rm -rf {} \;

Now we can build world and kernel, install both and reboot the system:

# make buildworld
# make buildkernel
# make installkernel
# make installworld
# shutdown -r now

When the system comes back up we can SSH into it again. And there we can see that we’re on -STABLE now!

The newly built FreeBSD 4.11-STABLE

# uname -a
FreeBSD vierelf.localdomain 4.11-STABLE FreeBSD 4.11-STABLE #0: Sat Jan 28 11:53:06 GMT 2017     root@vierelf.localdomain:/usr/obj/usr/src/sys/GENERIC  i386

That’s it for today. We’re not quite there yet, but we’ve laid the groundwork for many more updates to come. Those will be described in the coming two posts of this mini series.

Updating FreeBSD 4.11 (1/4) – Blast from the past

Ah, weekend. And it’s a nice day, too: The sun shines on the snowy landscape! A perfect day to go out with the children – or to embark on a vastly different kind of adventure… Yes, you read that title right. This is not about four FreeBSD 11 machines. It’s about one FreeBSD 4.11 machine in 2017!

Legacy systems in general

Why even bother with FreeBSD 4.11 today? Well, most people would probably respect historical interest and in fact I was keen on tinkering with a legendary old 4.x release and see if you can have some fun with it (spoiler: Oh yes, you can!). But this would be a task for times when I have absolutely nothing else to do. So it’s fairly obvious that there’s another reason.

“Old habits die hard” they say and there’s definitely truth to it. Another truth, however, is that in the IT world things often don’t go away easily. There’s still a considerable number of machines out there that run DOS. In production. Doing important stuff. Sometimes just because it works sufficiently well. But often they have special tasks that for some reason or another cannot be migrated as nobody has any idea clue how you’d do this.

You want examples? Sure. The first example is “Game of Thrones” author George R.R. Martin writing his books using WordStar 4.0 on a DOS machine. A lot of companies use DOS, too, but you usually don’t hear about that. And less than a month ago a new version of FreeDOS (v. 1.2) was released.

Or do you remember the French airport that was having trouble due to using Windows 3.11? That was in 2015 and they announced that they were planning to migrate their system officially by 2017 (but expectedly by 2019 or even 2021)!

FreeBSD 4.11

One such case of a formerly very popular operating system that just won’t go away is FreeBSD 4.11. From the version number alone you can see that this is a very special one: Usually FreeBSD has three to four point releases in a release’s life cycle. FreeBSD 4 obviously was different. The main reason for that was that version 5 kind of was to FreeBSD what version 4 was for MS-DOS: Innovative in its day but disappointing especially in terms of stability (and performance) among other things.

I have to take up the cudgels for FreeBSD 5, though. It blazed a trail for important technology that we rely on today like the GEOM framework, porting the system to amd64, and so on. Implementing things like that meant digging deep into the system and doing pretty invasive changes. It totally paid off in the long run but back in the day people tended to avoid the early 5.x releases or the whole release cycle altogether.

There are many reasons why some companies still have FreeBSD 4.11 running: Critical systems that cannot be migrated, installed programs to which the source code was lost, etc. Things like that. Most of these systems are probably used internally only but I’m pretty sure that there’s also quite some 4.11 machines still serving data on the net…

Now what others do (and what their reasons for it might be) are not my primary concern – I can’t do anything about it anyway. However the company that I work for has its share of legacy systems, too. Especially younger admins hate them. Actually they are impressively stable and rarely if ever have problems. In fact they keep running, silently doing their job. But now and then you have to do some changes to them and that’s where the trouble starts: They simply cannot be compared to the modern-day Linux systems that we’re used to administer daily!

Preparations

I kept wondering if that situation could be improved. It wouldn’t be worth the effort to do this during work time but I was curious enough about the legendary 4.11 release to give it a try. There’d surely be enough to learn. And should I succeed in bringing a test system into a more modern state there would also be a benefit for us at work.

We still have all those old CDs at work but I made my decision impulsively on Saturday. Fortunately ISOs of the old Walnut Creek CD-ROMs are available for free download. Thanks, Archive.org! I burned it to a CD and was able to boot from it.

I actually installed the OS on real hardware. First on an old HP compaq nx9010 laptop that I got for free a while ago. Things worked pretty well, but since what I’m trying to do is almost all about compiling software, the compilation times proved to be a bit long. Therefore I decided to redo the previous steps on faster hardware – and since amd64 PCs are capable of running i386 programs, all was well. I just had to disable AHCI in the BIOS since FreeBSD 4.11 doesn’t know what to do with that.

The actual installation was interesting enough (for people like me who enjoy history) and therefore I decided to redo it again in VirtualBox to take screenshots and describe the installation process for the remainder of this post.

Installation

Quite some time has passed since FreeBSD 4.11 was released in 2005. More than a decade is a lot in a man’s life but it’s a hell of a lot when it comes to software and operating system development! I had never before installed any version of FreeBSD older than 7.4 and even thinking about the differences between 7.x and 11, many things changed. But what would 4.11 look like? Would I be able to install it without difficulties (I tried a very old version of OpenBSD once and quickly gave up because I didn’t really like the idea of manual partitioning of the drives…)?

FreeBSD 4.11’s Kernel Configuration Menu

I was surprised to run into something that I had never seen even before the installer started! There’s this Kernel Configuration Menu where you can choose to either boot the standard kernel or to customize it. Of course I was curious and so I decided to take a look at it.

FreeBSD 4.11 kernel drivers selection

This brings up a friendly little tool that allows for easy navigation through the driver tree. The screen is basically split into two parts: Active drivers and inactive drivers. Each one consists of several categories which are collapsed and can be expanded for a list of drivers.

Browsing FreeBSD 4.11’s additional network drivers

The standard kernel worked on both of my machines and so I didn’t play around with this after exploring it briefly. When the kernel had booted, I found myself in the good old Sysinstall utility. Yes, it’s pretty ugly and it looks a bit complicated – which is due to the fact that despite it’s name it does a lot more than just handle the installation. Yet in fact it’s really easy to navigate through.

I like the Express option that lets you perform an installation as quickly as possible by asking you only the absolute minimum of necessary questions. This goes as far as not asking for a root password and leaving it blank on your first login! Fair enough, it’s easy to set one yourself. Current versions of FreeBSD do not have this option. That may be due to the fact that it’s not terribly helpful for setting up a serious system. And the use case that I have for it here is definitely more of an exception!

FreeBSD 4.11’s Sysinstall: Main Menu

The first thing that’s essential is of course disk partitioning. If you don’t require any special setup, you can simply use the whole disk. I was glad to see that this can be done by simply pressing “A”!

Installing FreeBSD 4.11: Disk partitioning

Just like today FreeBSD 4.11 allows you to choose if you want to install a boot manager, just write the boot loader or don’t do anything. The first option can be used for dual-boot systems where you want to load either FreeBSD or another OS. The last option won’t work by itself; if you choose that, you need to work with an external boot manager that supports FreeBSD. In my case just installing the standard loader makes most sense.

Installing FreeBSD 4.11: Choosing boot options

Then there’s creating the disklabels. This is another thing where I’m happy to find an auto defaults option (simply pressing “A” again) because I just wanted a quick test system anyways and didn’t want to think too much about the disklabel layout!

Installing FreeBSD 4.11: Editing disk labels

Finally we have to select what to install. To make things easy, sysinstall offers several distribution sets to choose from. Minimal is all that I need for my experiments, but back in the day a lot of the other options were definitely quite handy. Two things stand out: You can choose to install FreeBSD + X11 for example. Current versions of FreeBSD don’t let you do that. You’ll always get a console only system and have to install X11 yourself (which IMO is something that leaves the barrier unnecessarily high for new users who don’t know FreeBSD nor its packaging system, yet!).

Sysinstall also allows for custom distribution selection. This gives you access menu that allows for really fine-grained choices. Again, FreeBSD’s installer cannot do that today. While it’s probably nice to have that option, it’s not a must-have. People who have use cases for that will probably know what they do and install FreeBSD manually, anyway. Still I like having it in the installer!

Installing FreeBSD 4.11: Choosing distribution sets

The only thing left is selecting what medium to install from – the CD in my case. Formatting a large drive takes quite a moment because we’re talking UFS1 here! Don’t start looking for a way to use UFS2, it’s not available in FreeBSD 4. The actual “minimal” installation is quite quick and just a moment later I can reboot and the system comes up just like you’d expect it to. Now that was the easy part of my adventure. Next stop: Brushing the dust away and updating that beast(ie)!

3+ months on TrueOS – a critical write-up

My journey into the *nix world has not been a completely straight forward one. I’ve used Linux (various distributions) exclusively for quite some time before I felt that more and more things were heading in the wrong direction. Sure, it’s all open source and I could do things my own way. In fact I did roll my own distribution for a short period of time but this was more because I wanted to experiment with things. And while there are examples (like Void Linux) that prove that a single person can keep a serious distro running, I’m far from having the knowledge nor the time (or even the urge) to do so. But even if I had all that there’s something wrong with parts of the Linux ecosystem and community that I don’t feel is fixable at the moment.

For those reasons I was pretty open to new things when I encountered actual FreeBSD servers at work. I came to love the *BSD way of doing things and used FreeBSD and OpenBSD systems on laptops and in VMs to play around with. In January I decided to put PC-BSD on my main machine but had to leave it for Linux again pretty soon, for various reasons. Over the time I really wanted my BSD back and get rid of the Linux trouble (that used not to bug me as I knew nothing better). I’ve written about my experience with PC-BSD and Linux again in my previous post in some detail. The time to give a FreeBSD based desktop another try came when TrueOS was released. So that’s what this article is about: Using TrueOS as your daily driver for some months!

TrueOS in general

When I found out about TrueOS sometime in July, I was very curious how different it would be from PC-BSD. So I downloaded the ISO and installed the OS on my primary machine (no risk no fun, right?). The installation went as smooth as it did with PC-BSD. My hardware was supported. It seemed like a good start and I thought that I’d probably only need a few weeks to adapt to Lumina and be a happy TrueOS user. That was a bit too optimistic. My experience with this system really is a mixed bag. It’s not all bad nor is it all great. There’s light and shadow. And to be honest: While it’s pretty much ok as a daily driver (it’s mostly stable and most features are there), I think it feels quite “beta” currently.

TrueOS installation: Simple but shiny

I tried to post problems that I had with it on the PC-BSD forums but didn’t have too much luck with that. One post has not had a single answer (after several month now). Another was hijacked by somebody else and then closed by a moderator even though my issues (being the thread starter!) had not been resolved. While I originally wanted to open bug reports after getting into the community on the forums, that experience didn’t really help holding up my motivation. Yes, I know: I should just have reported things in the proper place. But I’m a person who wants to have a deeper relationship to his OS. I don’t just want to use it. I want to be part of it! For that reason it’s important to me to get into the community around an OS early on. For me this did not work out on TrueOS. I didn’t feel like I was getting anywhere on the forums and I didn’t succeed in feeling at home on the OS level, either.

But that’s personal stuff. Let’s start with going into a bit of detail on some areas of the system, shall we?

Bad: Graphics acceleration

Most of the bad points that I mentioned for PC-BSD (see the previous post) were still valid: Startup time was a bit long, the system is rather heavy on the battery, I didn’t find a way to log in multiple users graphically, etc. But there’s more. I think that the single most important problem that TrueOS suffers from is the fact that graphics acceleration had been broken for quite some time now. It worked when I first installed the system. Up to the update on 9/11 things were fine, too, but the next one broke it and while I hoped that it’d be fixed soon, I’ve waited for that ever since.

A freshly installed Lumina desktop

So if I wanted to watch a video I had to reboot and select the boot environment from 9/11. This also affected my work with VirtualBox. Probably thanks to that issue I could no longer switch my VMs to full-screen mode which was rather annoying. So I often had to reboot and use an old BE to be able to actually use my PC as intended. The fact that after a Lumina update the old BE would no longer be able to show the panel didn’t help either… 😦 I love BSD and want it to succeed. Only for that reason I’ve been tolerating things like that (which mean major inconvenience, honestly) for quite some time. But certainly you don’t attract many new BSD users with a system that has such issues! I really, really hope that this one gets fixed soon…

Good AND bad: Display settings

Lumina comes with a monitor configuration utility which is an excellent program – at least in theory. What makes it great is that it not only allows you to set the resolution. It also offers a simple and elegant GUI solution to manage multiple monitors! Great stuff, I love that. My primary PC is a laptop that I mostly use at home with an external monitor instead of the laptop screen. The monitor configuration allows me to add the external monitor and deactivate the other one. I’ve played with this a bit and it all seems very nice and mature.

Adding a second screen in the ‘Lumina Monitor Configuration’

However… It simply won’t keep the settings! That means was always 8 clicks after each start before my monitor setup was the way that I needed it and I could begin to do some work. I have no idea why it won’t save the settings and remember them next time. This is completely unnecessary.

Screen resolution configuration

I’ve also noticed a little consistency issue is that most of the Lumina desktop is localized properly (for the screenshots here I’ve used an English system, though!). The monitor control for example isn’t. I speak English and thus I have no problems using it but it makes the tool feel a bit out of place.

Neutral: Insight

I’ve tried to become friends with Lumina’s file manager, Insight. It didn’t feel right at the beginning and sometimes it decided to dump core when I accessed a new directory. But I finally kind of got used to it after some time.

The old Insight file manager with the side bar

Then another update happened… Yeah, the new git functionality may be nice. But guys, really… What did you do to that sidebar with the icons for copy, cut, paste, etc? The one thing that made me accept Insight was gone all of the sudden and I have been unable to get it back… I must admit that it’s probably not that bad of a loss as shortcuts were finally implemented. Still it didn’t take that much space and for people who don’t use shortcuts (I’ve seen a lot of such people!) it still was a very nice feature. No idea why it was dropped entirely instead of making it opt-in. Same thing about the “action buttons” and the “show thumbnails” option. It’s simply gone.

The new Insight with dual column mode

Another update added the dual column mode. That’s pretty useful IMO. But it happened after I stopped using TrueOS as my main operating system.

Good: SysAdm

This is one aspect where TrueOS really shines. Initially it felt quite empty and unpopulated but same of the updates added more and more options to it. SysAdm is a middleware that exposes an API to manage FreeBSD based machines locally or remotely.

The ‘Sysadm’ client for the local machine

I’ll be keeping an eye on this and look forward to install it on a FreeBSD server and try out remote management! But even the local client comes with a very nice GUI that has a lot of functionality now. Thanks and great work on that one TrueOS team!

Bad: Window manager

TrueOS uses the Fluxbox window manager with Lumina. Some people like it, some people don’t. I’m on the side of those who don’t but that’s not my main problem. People who’ve used *nix systems for any period of time probably know more than one wm and simply switch over to one that they like. The trouble is: It looks like Fluxbox is not meant to be replaced when you run Lumina! There’s no easy way to configure a different wm and in fact I didn’t find anything at all.

Lumina desktop settings

Worse: There are standards (the ICCCM in case of window managers). Following standards makes sense. Fluxbox doesn’t follow them. Window managers are meant to let other wms “take over” if you run your-favorite-vm –replace on the console. Fluxbox won’t cooperate which is very unfortunate. To replace it with sawfish (my wm of choice) I need to kill fluxbox first and then start the other wm… That’s not cool.

However I can fully understand that the small team that brings us TrueOS concentrates on supporting only one wm. Using sawfish I’ve experienced repeatable crashes (especially with Insight) where the system proved to be stable otherwise. And there’s another reason not to take this point too seriously: Fluxbox is not here to stay. Ken Moore has stated that he’ll write his own window manager to work perfectly with Lumina. So at some point Fluxbox will be replaced. I’m looking forward to this and hope that it’ll be a better replacement.

Neutral to good: Lumina DE

One of the core features of TrueOS is its native Lumina desktop. It was written from scratch, is BSD licensed and one of its design goals is being light-weight. Sounds excellent, doesn’t it? You bet it does! But does it live up to the high expectations? Like the whole TrueOS project its a bit of a mixed bag… First: A permissively licensed “BSD first” desktop is a dream come true. And I’m all for it being light-weight! The only problem here is… it isn’t.

It probably depends on what you compare it to. Sure thing: Compare it to KDE and you will find Lumina pretty light-weight. Then again – good luck finding any *nix DE that’s more heavy-weight than KDE is! If you compare Lumina to other desktops that state that they’re light-weight (be it Xfce, LXDE or even EDE), it clearly is quite a bit on the heavy side. In fact that’s no surprise due to the choice of toolkit that was made. Qt is the fattest toolkit out there. It does have it’s good parts, but being light-weight is nowhere near its strong points. However KDE (which uses Qt, too) has been the default DE of PC-BSD before and so Qt is what the TrueOS team knows best and that makes this toolkit a sensible choice despite the downsides.

Lumina panel configuration: Nice and flexible!

What Lumina does very well is being flexible. You can configure the menu the panels and just about everything to your liking. And you can do so using the GUI instead of having to edit config files. Even better: It’s pretty easy to do and after playing around with it a little you soon know how things work and where they are configured. Two thumbs up for that! I just miss the ability to right-click the panel to configure it. That’s probably the first thing newcomers try as everybody expects it to work.

Lumina is a desktop environment with lots of potential that already works quite well. It isn’t my favorite desktop and it does have some issues right now. However it works well and there are definitely people who prefer it over any other DE. And I’m pretty confident that it will continue to improve.

Good: Bootloader with BEs

PC-BSD used a modified version of GRUB2. While that program certainly works it’s not exactly my cup of tea. As stated above, I like light-weight software. And a bootloader that bears Grand in its name (and rightfully so) is not really my first choice. It’s alright if you need to dual-boot Linux or something but for just booting FreeBSD its more than I want.

TrueOS’s Bootloader – it supports BEs!

For TrueOS the team has decided to migrate back to the default FreeBSD bootloader after years of using GRUB. Excellent choice! Especially since it can now also use Boot Environments.

Good: PCDM

When it comes to display managers, I’ve come to like LXDM on Linux. Unfortunately it uses some linuxism in its code and for that reason could not be ported to *BSD. TrueOS offers another gem in this regard: PCDM. It’s a program that let’s you log in conveniently, providing all the features that you expect from a decent display manager – and more.

TrueOS’s display manager: PCDM

In PC-BSD I remember that I was initially unable to log in due to using a character that’s on the German keyboard but not on the US one. With TrueOS I no longer experienced such problems. On the contrary: I learned an alternative keyboard layout a while ago that offers better ergonomics and lets me type lots of foreign characters, too (e.g. the whole Greek alphabet). With PCDM I can use it to log in (allowing for nice, strong passwords, yay!).

Changing the keyboard layout in PCDM

I don’t have a 4k monitor, but it’s nice to see that PCDM is prepared for 4k already. The display manager lets you select various DPI options so that you don’t have the feeling of sluggish mouse movement when you use high resolutions.

PCDM’s DPI options: Ready for 4k

The only thing that bugged me quite a bit: The display manager was only displayed on the first screen, forcing me to open my laptop again to log in. I suggest to just show the login manager on every screen; this would be much more convenient.

Neutral to good: Update

The update mechanism of TrueOS has some advantages over the common desktop update methods. If you begin an update, it only fetches all packages and lets you continue to work in your current session. When you are about to shut down the system, it then asks if it should install the updates. Accept that and the system will shutdown the desktop and graphical mode and start updating. While it is doing that it’ll tell you not to turn off the computer and to change to the second TTY to see the details of the update process. When it’s done, the machine is powered down (or restarted if you chose that) like normal.

System update on TrueOS

But the really special thing is how the update is performed. A new Boot Environment is created, all packages are deinstalled and then reinstalled in their newest version. This has two advantages: It is the cleanest possible approach and it means that you can go back to any previous state by just selecting the respective BE! If you plan to do that, be sure to configure how many BEs the TrueOS update system keeps. Otherwise it will trash old ones (which may be a sensible default for space reasons).

What’s the downside? TrueOS is not a tiny operating system. Downloading all the packages with every update (can anybody say Noto fonts?) will require quite a few bits going over the wire. If you’re in a place with a slow connection, or worse, you have a monthly limit of how much you can download, then TrueOS’s way of doing things might not be for you.

Good: OpenRC

Ah, the bliss of a new init system! I’ve waited quite a while for that to happen. And when does it happen? About a week after I replaced TrueOS on my computer with another BSD operating system, Kris Moore (founder of PC-BSD/TrueOS) announces that they’re in the middle of switching over to OpenRC!

Truth be told: I didn’t expect this. When I heard Kris talk about nosh on the BSDNow show, I suspected that they might build that into PC-BSD. Then I had the impression that the team favored relaunchd (now renamed to jobd). And now it looks like OpenRC has landed!

TrueOS starting using OpenRC!

I’m fine with that as I already know it from Gentoo Linux. While I think it would have been interesting to see one of the other options get some attention, it now looks like OpenRC becomes the most significant alternative for people who don’t want to use Systemd! Maybe that’s not a bad thing at all.

While I don’t think that FreeBSD is going to adopt this change in the forseeable future (the BSD init system works well for servers after all) it totally makes sense to speed up the starting process for desktop machines. Thank you, TrueOS team! This takes care of another issue where FreeBSD just couldn’t compete with Linux.

Major differences from PC-BSD

The applications that come with TrueOS are pretty standard now. Initially the team provided some Qt5 alternatives like a browser I had never heard of and such. It’s a good decision to provide applications that people know but I also like the spirit of trying out new things and see if they work out. TrueOS in general is even more open to experiments than PC-BSD was – and even that was quite the opposite of a boring OS!

Obviously TrueOS has a new name. There are people who don’t like it. Some claim that it can be mistaken for another OS. But let’s be honest: Does “the average user” even know that there’s Tru64 UNIX? Most likely not. And people who know that it exists probably have enough *nix knowledge to tell the two apart. Other people criticise that it’s a bit of a big-mouthed name. Maybe it is but I don’t really care so much about that. In times of PC-BSD the server variant was already called TrueOS and for their new system the team wanted to rebrand the project without using a completely unknown name and so TrueOS actually was a sensible choice.

The biggest difference between the two is that PC-BSD was built from FreeBSD releases originally and ultimately headed towards the -STABLE branch. TrueOS takes this one step further and builds upon -CURRENT! This means that you always get the latest drivers and newest stuff but you may have to live with problems like broken graphics acceleration…

Another difference is that while PC-BSD begun its life supporting KDE and later leaving the user a choice between several desktop environments, TrueOS concentrates on the Lumina desktop. Some other DEs can be installed if you wish, but Lumina definitely is the standard.

And then there’s the newest addition to the project is TrueOS Pico, a variant meant to build ARM based thin-clients as well as a “Pico Server”. A very intriguing concept!

Conclusion

Looking back on more than three months of daily TrueOS usage, I must say that I went through highs and lows. There’s the painful moments where I had to grind my teeth and force myself to carry on. And then the opposite happened and I come across something that’s just amazing. All of that makes it not too easy to draw a clear conclusion. TrueOS is evolving at a very fast pace and at the present time my conclusion is that it is a unique OS that might work for you. There are operating systems where that’s more likely than with TrueOS but I’d bet that they are also far less innovative.

The TrueOS project is relatively young. I wouldn’t bet on even the team leader to know exactly where it’s heading. This is a good thing for us all, even for people who do not plan to use TrueOS. Why? Because it is not afraid to try new things and by doing so will continue to push FreeBSD forward in the desktop field!

PC-BSD’s goal was to provide a FreeBSD powered desktop that’s both easy and convenient to use for seasoned and new users alike. The rolling release character of TrueOS may not fit the former audience completely. It will be very interesting to see where the project will eventually find its place. Long time FreeBSD users who want the newest features on their desktop? BSD enthusiasts who want enjoy a permissively licensed desktop OS? Who knows! Time will tell (those who keep an eye on it). I may have switched to another OS for my daily work but TrueOS is far too exciting a project to just abandon completely. And if you love BSD you may want to give it a shot now and then. If you haven’t already tried it, correct your ignorance and download an ISO now!