Updating FreeBSD 4.11 (2/4) – Digging up old graves

In part one I wrote about Legacy systems in general and showed a FreeBSD 4.11 installation for those of my readers who are interested in software history.

This post is about the first part of updating this fresh 4.11 system to a state that’s a bit less catastrophic. Remember: FreeBSD 4.11 was released in 2005 – however the ABI of each release is carved in stone with a .0 release. Which means that the software in the base system is from 4.0 and thus we venture back into the last millenium: 1999!

Initial state

To give you an idea what this means, here are a few program versions:

Various program’s versions in 4.11’s base system

So we have these programs among others:

GCC 2.95.4
Binutils 2.12.1
Perl 5.0
OpenSSH 3.5

To make matters worse, the ports tree for FreeBSD 4.11 is pretty dead, too. It’s important to get newer compilers running, but around 2005 FreeBSD used special releases to build GCC from (“gcc-core”) and I was not able to find a single mirror on the net that still holds those old and exotic files! Out of luck here. We’ll have to do without those ports.

“Modernizing” this is going to be interesting… Considering how fast the IT world moves, all of this is just as dead as it gets. So let’s prepare for the (code) smell and start digging up an old grave, shall we?

Allowing remote connection

After a passwordless login as root it makes sense to set up the right keymap (if you don’t use the default, that is). I had no idea how to do it on 4.11 and so I just gave the usual way of doing it a try – and was met with success:

# kbdmap

Looks like the keymap selection has not changed in all the time! Let’s try to make it persistent:

# echo keymap=german.iso.kbd >> /etc/rc.conf

I tried it out and it did just what I wanted. Time to try to add a regular user:

# adduser

The script is a bit different from what we’re used to today but in the end it does the same thing: Allow us to create a user. It’s important to add this user to the wheel group so that we’re able to su to root. However we need to give root a password first:

The useradd script in FreeBSD 4.11

# passwd

Ok, sshd should be running. Let’s check just to be sure:

# ps aux | grep sshd
[...]
root        75  0.0  0.1  2600 2040  ??  Is    6:26AM   0:00.11 /usr/sbin/sshd

Looks good. What about connectivity? Let’s see:

# ifconfig
vr0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        ether 00:05:5d:96:fa:f9
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
[...]

Nope, no connection. Luckily I paid attention when I the installer started from the CD and somewhere the strange-looking path of /stand caught my eye. Doing a little research, I found out that this directory used to be part of the OS and was removed in early FreeBSD 5 as it was mostly redundant with /rescue. What was in there, you ask? Have a look yourself:

# ls /stand
-sh             etc             minigzip        rm              tunefs
[               find            mount_mfs       route           usbd
arp             fsck            mount_nfs       rtsol           usbdevs
boot_crunch     gunzip          newfs           sed             zcat
camcontrol      gzip            pccardc         sh
cpio            help            pccardd         slattach
dhclient        hostname        ppp             sysinstall
dhclient-script ifconfig        pwd             test

Network configuration

There’s our friend sysinstall. I already said that it does more than just install the system. So let’s bring it up now:

# /stand/sysinstall

Sysinstall to configure the network

There we choose Networking -> Interfaces -> the appropriate NIC. No, I don’t want IPv6 and yes, DHCP is the thing.

Interface configuration using sysinstall

I’ve called the system vierelf which would be “foureleven” in English because I couldn’t think of anything better. And it’s just a test system anyway. Does the connection work now?

# ping elderlinux.org
PING elderlinux.org (212.77.232.71): 56 data bytes
64 bytes from 212.77.232.71: icmp_seq=0 ttl=54 time=24.993 ms

A little housekeeping

Alright! Let’s just take a look what services are listening right now:

# sockstat -4 -l
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS      
root     dhclient   281    7 udp4   *:68                  *:*                  
root     sendmail    84    4 tcp4   *:25                  *:*                  
root     sendmail    84    6 tcp4   *:587                 *:*                  
root     sshd        75    4 tcp4   *:22                  *:*                  
root     syslogd     61    5 udp4   *:514                 *:*

Ugh! Time to make a few changes in /etc/rc.conf to deactivate any daemons except for SSH (which we need). This may not be strictly necessary but we want to improve the security of this system, right? And I wouldn’t trust those crusty old daemons at all. And whatever is not running won’t cause us any problems. So let’s get rid of them with extreme prejudice!

# vi /etc/rc.conf

To do so, add daemonname_enable=”NO” for each daemon and in case of sendmail use sendmail_enable=”NONE”.

FreeBSD 4.11 rc.conf with most daemons disabled

If you reboot now, sendmail and syslogd as well as cron, usbd and inetd will be disabled. That’s a starting point in securing the system. Let’s move on.

Replacing a dead tree

I’ll connect to the 4.11 box remotely over SSH because it’s much more convenient to have my trusty terminal at hand and to be able to copy and paste stuff:

% ssh kraileth@192.168.1.5
Unable to negotiate with 192.168.1.5 port 22: no matching host key type found. Their offer: ssh-dss

Ouch! I cannot even SSH into the system because its version of OpenSSH is so old that it only offers ssh-dss keys which have been deprecated for quite a while and disabled by default in OpenSSH >=7.0. So to connect to that old server, I have to tell my SSH client to accept ssh-dss for this connection:

% ssh kraileth@192.168.1.5 -oHostKeyAlgorithms=+ssh-dss

SSH login using “-oHostKeyAlgorithms=+ssh-dss”

Ok, I’m in. But what now? We cannot use FreeBSD’s ports tree but I strongly prefer some means of package management over installing stuff using make install. So how do we accomplish this? Enter pkgsrc. Pkgsrc is basically NetBSD’s fork of the FreeBSD ports tree. Being a NetBSD project however, it’s not limited to just NetBSD. It’s a truely portable way of building and managing software (I might write a separate post about it some time).

There’s just one problem: Downloading and decompressing the latest pkgsrc release (currently 2016Q4) won’t complete the bootstrapping process. Obviously FreeBSD 4.11 is no longer supported – which is not so much of a surprise. Time to try out older releases! After doing so I found out that 2009Q4 seems to be the last release to bootstrap successfully.

But here’s another problem: Pkgsrc doesn’t seem to keep older releases around and I also haven’t found them mirrored anywhere on the net. Pkgsrc uses CVS, however. So it’s possible to checkout older versions. FreeBSD comes with CVS as part of the base system. Unfortunately CVS works over SSH. And NetBSD’s CVS server won’t accept ssh-dss (which totally makes sense)! Since we don’t control the server, there’s also no way to just add a parameter or something to make it work. It simply doesn’t work that way.

Time to get CVS on my slightly more modern FreeBSD 11, do the checkout there and tar it all up to copy it over via scp! We’re going to get 2007Q2 instead, though, since we need things that won’t work on FreeBSD 4.11 with later versions. Oh, and if you’re not familiar with CVS, don’t worry. You don’t need to know what modules or tags are. Just copy the commands that I prepared for you and you’re good to go:

% sudo pkg install cvs
% cvs -danoncvs@anoncvs.netbsd.org:/cvsroot get -rpkgsrc-2007Q2 -P pkgsrc
% tar cvjf pkgsrc2007Q2.tbz2 pkgsrc/*
% scp pkgsrc2007Q2.tbz2 kraileth@192.168.1.5:/usr/home/kraileth

Then back on vierelf the next step is to prepare some directories and extract the pkgsrc tarball:

# mkdir -p /usr/local/temp /usr/pkgsrc
# cd /usr/pkgsrc
# mv /usr/home/kraileth/pkgsrc2007Q2.tbz2 .
# tar xvjf pkgsrc2007Q2
# rm pkgsrc2007Q2.tbz2
# mv pkgsrc 07

Bridgehead in hostile territory

Now we can bootstrap pkgsrc:

# cd /usr/pkgsrc/07/bootstrap
# ./bootstrap --prefix=/usr/local/temp --varbase=/usr/local/temp --pkgdbdir=/usr/local/temp/db

Pkgsrc 2007Q2 bootstrap complete!

Pkgsrc has been bootstrapped successfully! We just need to adjust the path variable so that the system picks up binaries from the new paths (and make those take precedence over the old system binaries). We could just change the PATH variable but it’s better to make the changes persistent. So let’s add the two new paths in the shell’s rc file in front of the others:

# vi /root/.cshrc

This is what needs to be prepended:
/usr/local/temp/sbin /usr/local/temp/bin

Root’s .cshrc file for the first phase pkgsrc

Now simply log out and become root again to have the new environment:

# logout
> su -

As a last step check if everything is right and we can access binaries in both paths:

# which bmake
/usr/local/temp/bin/bmake
# pkg_info 
bootstrap-mk-files-20061111 *.mk files for the bootstrap bmake utility
bmake-20051105nb3   Portable (autoconf) version of NetBSD 'make' utility
tnftp-20050625nb1   The enhanced FTP client in NetBSD
mtree-20070710      Utility for mapping and checking directory hierarchies
pax-20060202nb1     POSIX standard archiver with many extensions
pkg_install-20070710 Package management and administration tools for pkgsrc

Excellent! Now we have a working replacement for FreeBSD’s dead ports tree. This is definitely something that we can build upon.

Bring on some stability!

Lame pun, I know. Nevertheless it makes sense to… err… update the OS to the latest version. This is what we currently have:

# uname -a
FreeBSD vierelf.localdomain 4.11-RELEASE FreeBSD 4.11-RELEASE #0: Fri Jan 21 17:21:22 GMT 2005     root@perseus.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386

We’re on 4.11-RELEASE. The latest code for each release cycle is always in the -STABLE branch. Believe it or not: The latest code change was in April of 2014! The traditional way of getting FreeBSD code was over CVS. No, this time the problem is not ssh-dss. FreeBSD migrated from CVS to SVN (subversion) in 2008. FreeBSD CVS servers have been removed years ago. Therefore the old tools like cvsup are useless. Subversion is needed to checkout the source.

Luckily we have our pkgsrc ready. This old release has a very old port for subversion but that’s fair enough. There are a few source tarballs that are no longer available from the mirrors that pkgsrc knew for them. Not a big problem, we can download those manually:

# cd /usr/pkgsrc/07/distfiles
# fetch http://archive.apache.org/dist/httpd/httpd-2.0.61.tar.bz2
# fetch http://repository.timesys.com/buildsources/e/expat/expat-2.0.1/expat-2.0.1.tar.gz
# fetch http://download.nust.na/pub2/openpkg1/sources/DST/pkgconfig/pkg-config-0.21.tar.gz

Now we can build subversion:

# cd /usr/pkgsrc/07/devel/subversion-base
# bmake install clean clean-depends

Various dependencies will be downloaded, built and installed. Eventually subversion will be installed and available on the system. Time to tell the shell to look for new binaries and then checkout the stable source code:

# rehash
# svn co svn://svn.freebsd.org/base/stable/4 /usr/src

SVN checkout of the 4.11-STABLE code

The FreeBSD 4 base system never knew anything beyond CVS and cannot cope with the .svn directories that the svn checkout creates. World builds but the installation fails like this:

install: /usr/libdata/perl/5.00503/./.svn/text-base/Cwd.pm.svn-base: No such file or directory

Therefore it’s necessary to get rid of those disruptive directories:

# cd /usr/src
# find . -iname '.svn' -exec rm -rf {} \;

Now we can build world and kernel, install both and reboot the system:

# make buildworld
# make buildkernel
# make installkernel
# make installworld
# shutdown -r now

When the system comes back up we can SSH into it again. And there we can see that we’re on -STABLE now!

The newly built FreeBSD 4.11-STABLE

# uname -a
FreeBSD vierelf.localdomain 4.11-STABLE FreeBSD 4.11-STABLE #0: Sat Jan 28 11:53:06 GMT 2017     root@vierelf.localdomain:/usr/obj/usr/src/sys/GENERIC  i386

That’s it for today. We’re not quite there yet, but we’ve laid the groundwork for many more updates to come. Those will be described in the coming two posts of this mini series.

Advertisements

Updating FreeBSD 4.11 (1/4) – Blast from the past

Ah, weekend. And it’s a nice day, too: The sun shines on the snowy landscape! A perfect day to go out with the children – or to embark on a vastly different kind of adventure… Yes, you read that title right. This is not about four FreeBSD 11 machines. It’s about one FreeBSD 4.11 machine in 2017!

Legacy systems in general

Why even bother with FreeBSD 4.11 today? Well, most people would probably respect historical interest and in fact I was keen on tinkering with a legendary old 4.x release and see if you can have some fun with it (spoiler: Oh yes, you can!). But this would be a task for times when I have absolutely nothing else to do. So it’s fairly obvious that there’s another reason.

“Old habits die hard” they say and there’s definitely truth to it. Another truth, however, is that in the IT world things often don’t go away easily. There’s still a considerable number of machines out there that run DOS. In production. Doing important stuff. Sometimes just because it works sufficiently well. But often they have special tasks that for some reason or another cannot be migrated as nobody has any idea clue how you’d do this.

You want examples? Sure. The first example is “Game of Thrones” author George R.R. Martin writing his books using WordStar 4.0 on a DOS machine. A lot of companies use DOS, too, but you usually don’t hear about that. And less than a month ago a new version of FreeDOS (v. 1.2) was released.

Or do you remember the French airport that was having trouble due to using Windows 3.11? That was in 2015 and they announced that they were planning to migrate their system officially by 2017 (but expectedly by 2019 or even 2021)!

FreeBSD 4.11

One such case of a formerly very popular operating system that just won’t go away is FreeBSD 4.11. From the version number alone you can see that this is a very special one: Usually FreeBSD has three to four point releases in a release’s life cycle. FreeBSD 4 obviously was different. The main reason for that was that version 5 kind of was to FreeBSD what version 4 was for MS-DOS: Innovative in its day but disappointing especially in terms of stability (and performance) among other things.

I have to take up the cudgels for FreeBSD 5, though. It blazed a trail for important technology that we rely on today like the GEOM framework, porting the system to amd64, and so on. Implementing things like that meant digging deep into the system and doing pretty invasive changes. It totally paid off in the long run but back in the day people tended to avoid the early 5.x releases or the whole release cycle altogether.

There are many reasons why some companies still have FreeBSD 4.11 running: Critical systems that cannot be migrated, installed programs to which the source code was lost, etc. Things like that. Most of these systems are probably used internally only but I’m pretty sure that there’s also quite some 4.11 machines still serving data on the net…

Now what others do (and what their reasons for it might be) are not my primary concern – I can’t do anything about it anyway. However the company that I work for has its share of legacy systems, too. Especially younger admins hate them. Actually they are impressively stable and rarely if ever have problems. In fact they keep running, silently doing their job. But now and then you have to do some changes to them and that’s where the trouble starts: They simply cannot be compared to the modern-day Linux systems that we’re used to administer daily!

Preparations

I kept wondering if that situation could be improved. It wouldn’t be worth the effort to do this during work time but I was curious enough about the legendary 4.11 release to give it a try. There’d surely be enough to learn. And should I succeed in bringing a test system into a more modern state there would also be a benefit for us at work.

We still have all those old CDs at work but I made my decision impulsively on Saturday. Fortunately ISOs of the old Walnut Creek CD-ROMs are available for free download. Thanks, Archive.org! I burned it to a CD and was able to boot from it.

I actually installed the OS on real hardware. First on an old HP compaq nx9010 laptop that I got for free a while ago. Things worked pretty well, but since what I’m trying to do is almost all about compiling software, the compilation times proved to be a bit long. Therefore I decided to redo the previous steps on faster hardware – and since amd64 PCs are capable of running i386 programs, all was well. I just had to disable AHCI in the BIOS since FreeBSD 4.11 doesn’t know what to do with that.

The actual installation was interesting enough (for people like me who enjoy history) and therefore I decided to redo it again in VirtualBox to take screenshots and describe the installation process for the remainder of this post.

Installation

Quite some time has passed since FreeBSD 4.11 was released in 2005. More than a decade is a lot in a man’s life but it’s a hell of a lot when it comes to software and operating system development! I had never before installed any version of FreeBSD older than 7.4 and even thinking about the differences between 7.x and 11, many things changed. But what would 4.11 look like? Would I be able to install it without difficulties (I tried a very old version of OpenBSD once and quickly gave up because I didn’t really like the idea of manual partitioning of the drives…)?

FreeBSD 4.11’s Kernel Configuration Menu

I was surprised to run into something that I had never seen even before the installer started! There’s this Kernel Configuration Menu where you can choose to either boot the standard kernel or to customize it. Of course I was curious and so I decided to take a look at it.

FreeBSD 4.11 kernel drivers selection

This brings up a friendly little tool that allows for easy navigation through the driver tree. The screen is basically split into two parts: Active drivers and inactive drivers. Each one consists of several categories which are collapsed and can be expanded for a list of drivers.

Browsing FreeBSD 4.11’s additional network drivers

The standard kernel worked on both of my machines and so I didn’t play around with this after exploring it briefly. When the kernel had booted, I found myself in the good old Sysinstall utility. Yes, it’s pretty ugly and it looks a bit complicated – which is due to the fact that despite it’s name it does a lot more than just handle the installation. Yet in fact it’s really easy to navigate through.

I like the Express option that lets you perform an installation as quickly as possible by asking you only the absolute minimum of necessary questions. This goes as far as not asking for a root password and leaving it blank on your first login! Fair enough, it’s easy to set one yourself. Current versions of FreeBSD do not have this option. That may be due to the fact that it’s not terribly helpful for setting up a serious system. And the use case that I have for it here is definitely more of an exception!

FreeBSD 4.11’s Sysinstall: Main Menu

The first thing that’s essential is of course disk partitioning. If you don’t require any special setup, you can simply use the whole disk. I was glad to see that this can be done by simply pressing “A”!

Installing FreeBSD 4.11: Disk partitioning

Just like today FreeBSD 4.11 allows you to choose if you want to install a boot manager, just write the boot loader or don’t do anything. The first option can be used for dual-boot systems where you want to load either FreeBSD or another OS. The last option won’t work by itself; if you choose that, you need to work with an external boot manager that supports FreeBSD. In my case just installing the standard loader makes most sense.

Installing FreeBSD 4.11: Choosing boot options

Then there’s creating the disklabels. This is another thing where I’m happy to find an auto defaults option (simply pressing “A” again) because I just wanted a quick test system anyways and didn’t want to think too much about the disklabel layout!

Installing FreeBSD 4.11: Editing disk labels

Finally we have to select what to install. To make things easy, sysinstall offers several distribution sets to choose from. Minimal is all that I need for my experiments, but back in the day a lot of the other options were definitely quite handy. Two things stand out: You can choose to install FreeBSD + X11 for example. Current versions of FreeBSD don’t let you do that. You’ll always get a console only system and have to install X11 yourself (which IMO is something that leaves the barrier unnecessarily high for new users who don’t know FreeBSD nor its packaging system, yet!).

Sysinstall also allows for custom distribution selection. This gives you access menu that allows for really fine-grained choices. Again, FreeBSD’s installer cannot do that today. While it’s probably nice to have that option, it’s not a must-have. People who have use cases for that will probably know what they do and install FreeBSD manually, anyway. Still I like having it in the installer!

Installing FreeBSD 4.11: Choosing distribution sets

The only thing left is selecting what medium to install from – the CD in my case. Formatting a large drive takes quite a moment because we’re talking UFS1 here! Don’t start looking for a way to use UFS2, it’s not available in FreeBSD 4. The actual “minimal” installation is quite quick and just a moment later I can reboot and the system comes up just like you’d expect it to. Now that was the easy part of my adventure. Next stop: Brushing the dust away and updating that beast(ie)!