School, exams and… BSD!

Alright, January is already almost over, so there’s not much use in wishing my readers a happy new year, right? I wanted to have this month’s blog post out much earlier and in fact wanted to write about a completely different topic. But after January 27th it was pretty obvious for me what I’d have to write about – On that day I passed my final exam and now I’m a Computer Science Expert by profession. Time to take a look back at the apprenticeship and the status of *nix in German IT training today.

Spoiler: It’s Microsoft, Microsoft and again Microsoft. Only then there’s one drop of Linux in the ocean. I had left the (overly colorful) world of Windows in 2008. When I started the apprenticeship I was determined not to eat humble pie and come crawling back to that. While it was at times a rather tough fight, it was possible to do. And I’m documenting it here because I want to encourage other people to also take this path. The more people take the challenge the easier it will become for everyone. Besides: It is absolutely necessary to blaze the trail for better technology to actually arrive in mainstream business. This is of great importance if we do not want to totally fall behind.

Detours

I didn’t take the straight way into IT. While I had been hooked with computers since I was a little child, I also found that I had a passion to explain things to others. I gave private lessons after school for many years and after passing the Abitur (think of the British A levels) I chose to go to the university to become a teacher.

It took me a very long time of struggle to accept that I could not actually do that for a living. I am in fundamental opposition to how the German school system is being ruined and I could not spend all my work life faithfully serving an employer that I have not even the least bit of respect for.

The situation is as follows: We once had a school system in Germany that aimed at educating young people to be fit for whatever their life holds. The result was people who could stand on their own feet. Today the opposite is true: A lot of people who leave school have no idea how to find their way in life. Playing computer games is the only thing that a lot of young men (and an increasing number of women) actually do. They have not developed any character, they have no passion for anything (and thus no goals in life) and they often haven’t learned no empathy at all (and thus keep hurting other people – not even because of bad will but because of total ignorance).

At the same time things taught in school aim purely at making people available as workmen as soon as possible. Sounds contradictory? Sure thing. At the university I enjoyed the benefits of the old system where there was relatively large academic freedom and you were encouraged to take your time to learn things properly, to do some research if you hit topics of interest to you and to take courses from other faculties, etc. And this is pure insanity: All that is largely gone. New students are forced to hasten through their studies thanks to tight requirements (which semester to take which course in – very schoolish, no freedom at all)… In the name of “comparability” we did away with our own academic degrees only to adopt the inferior “master” (as well as the even more inferior “bachelor”).

Secondary schools are lowering their standards further and further so that almost anybody can get their A levels and flood the universities. At the same time there are not enough people remaining for other paths of education – and those who are far too often are completely useless to the companies: People who can be described as unreliable at best are of no use at all. I did not want to be part of that madness and so I finally decided to get out and do what I probably should have done right from the start.

Vocational school: Windows

The German vocational school system is a bit special: You only go to school one or two days (this varies among semesters). What about the other days? You spend them in a company you apply at before you can start the apprenticeship. That way you get to know the daily work routine right from the start (which is a really good thing). School is meant to teach some general skills and at work you learn practical things.

On the first day I went to vocational school, I kind of felt… displaced. Why? Well, coming back to school to teach children is something that takes a moment to adjust to. I enjoyed teaching in general (even though there are always horrible classes as well ;)) but becoming a student again afterwards is really strange. At least for a while.

Subject matter was extremely easy for me. But being almost 30 years old when I started the apprenticeship of course meant that I had a lot more of knowledge and experience than the typical 18 or 20 years old student. However this was a good thing for me since I also have a wife, two children and had to drive about 1.5 hours to school and the same distance back. Which meant that I had far less time for homework or learning than the others. In fact I only found a few hours to learn for the preliminary exam as well as for the final exam. But that’s it.

We had PCs with Windows XP and were required to work with that. Most of my classmates protested because they were used to Windows 7. I simply installed Cygwin, changed tho panel position to top and things were pretty much ok for me (it’s only for a few hours, right?). A while later we got new PCs with Windows 8(.1?) and new policies. The later made it impossible for me to use Cygwin. Since I had never touched anything after Windows XP, I took my time to take a look at that system. In fact I tried to be open for new things and since a lot of time passed since I left Windows, I no longer had any strong feelings towards it. Still Win 8 managed to surprise me: It was even worse than I had thought possible…

The UI was just plain laughable. I have no idea how anybody could do some actual work with it using the mouse. Now, I’m a console guy and I need no mouse to do stuff (if I at least have Cygwin that is). But that must have been a joke, right?

Then I found out that Windows still was not capable of even reading an EXT2 file system. Oh my. So I decided to format one USB key to FAT32 for school. But guess what? When I attached it, Windows made some message pop up that it was installing drivers – which then failed… I removed the USB key and inserted it again. Same story. A classmate told me to try another USB connector. I thought that he was fooling me but he insisted on it so I did it (expecting him to laugh at me any second). To my big surprise this time the driver could be installed! But the story does not end here. No drive icon appeared in the explorer. I removed the USB key again and reattached it once more. Nothing. My classmate took it out yet again and plugged it into the former connector (the one from which installing the driver failed). And this time the drive appeared in the explorer! It was that moment that I realized not too much had changed since XP – despite the even uglier looks. Bluescreens, program crashes and cryptic error messages that I had not seen in years all were back.

I decided that I could not work like that and decided to bring a laptop each school day. Just about all my classmates were fine with Windows however. But speaking of classmates: We lost five of them in the first two years. Two simply never showed up again, two more were fired by their companies (due to various misbehavings) and thus could not continue their apprenticeship and the other one had a serious problem with alcohol (being just 17 years old) and was also fired.

BYOD: Linux desktop

My laptop was running Linux Mint. When I bought it, it came with Mint pre-installed. My wife got used to that system and did not like my idea to install a different system (I mainly use Arch Linux as a desktop at work and on other PCs at home) and so Linux Mint stayed on there.

There were a few classmates interested in Linux in general. These quickly became the ones that I spend most of my time in school with. Three already had some experience with it but that’s it. One of them decided that it was time to switch to Linux about a year ago. I introduced him to Arch and he’s a happy Antergos (an Arch-based distro) user since then. Another classmate was also unhappy with Windows at home. I answered a few questions and helped with the usual little problems and she successfully made the switch and runs Mint now.

Some teachers couldn’t quite understand how one could be such a weirdo and not even have one single Windows PC. We were supposed to finish some project planning using some Microsoft software (forgot the name of it). I told the teacher that the required software wouldn’t run on any of my operating systems. Anything not Windows obviously wasn’t thinkable for him and he replied that in that case I’d really have to update! I explained to him that this was not the case since I ran a rolling-release distro which was not just up to date but in fact bleeding edge.

When he understood that I only had Linux at home, he asked me to install Windows in that case. Now I told him that I didn’t own any current version of Windows. He rolled his eyes and replied that I could sign up for some Microsoft service (“dream spark” or something?) where each student or apprentice could get it all for free. Then I objected that this would be of no use since I could not install Windows even if I had a license because I did not agree to Microsoft’s EULA. For a moment he did not know what to say. Then he asked me to please do it at work then. “Sorry”, I replied, “we don’t use Windows in the office either.” After that he just walked away saying nothing.

We were required to learn some basics about object-orientated programming – using C#. So I got mono as well as monodevelop and initially followed the course.

Another Laptop: Puffy for fun!

I got an older laptop for a really cheap price from a classmate and put OpenBSD on there. After having played a bit with that OS in virtual machines I wanted to run it on real hardware and so that seemed to be the perfect chance to do it. OpenBSD with full disk encryption and everything worked really nice and I even got monodevelop on there (even though it was an ancient version). So after a week I decided to use that laptop in school because it was much smaller and lighter (14″ instead of 18.3″!) – and also cheaper. ;)

After upgrading to OpenBSD 5.6 however, I realized that the mono package had been updated from 2.10.9p3 to 3.4.0p1 which broke the ancient (2.4.2p3 – from 2011!) version of monodevelop. Now I had the option of bringing that big Linux laptop again or downgrade OpenBSD to 5.5 again. I decided to go with option 3 and complain about .NET instead. By now the programming course teacher already knew me and I received permission to do the exercises with C++ instead! He just warned me that I’d be mostly on my own in that case and that I’d of course have to write the classroom tests on C# just like everyone else. I could live with that and it worked out really well. Later when we started little GUI programs with winforms I would have been out of luck even on Linux and mono anyway. So I did these with C++ and the FLTK toolkit.

Around christmas I visited my parents for some days. My mother’s computer (a Linux machine I had set up for her) stopped working. As my father decided that he’d replace it with a new Windows box (as that’s what he knows), I gave up my OpenBSD laptop. I installed Linux on it again and gave it to my mother as a replacement to prevent her having to re-learn everything on a Windows computer…

Beastie’s turn

So for the last couple of weeks I was back on Linux. However the final exam consists of two parts: A written exam and an oral one. The later is mostly a presentation of a 35 hour project that we had to do last year. I took the chance and chose a project involving FreeBSD (comparing configuration management tools for use on that particular OS). We also had to hand in a documentation of that project.

Six days before the presentation was to be held, I decided that it would suck to present a FreeBSD project using Linux. So I announced to my wife that I’d install a different OS on it now, did a full backup, inserted a PC-BSD 10.2 cd and rebooted. What then happened is a story of its own… With FreeBSD 10.3 just around the corner I’ll wait until that is released and write about my experiences with PC-BSD in a future blog post. Just so much for now: I have PC-BSD installed on the laptop – and that’s what I use to write this post.

The presentation also succeeded more or less (had a problem with Libre Office). But the big issue was that I obviously chose a topic that was too much for my examiners. My documentation was “too technical” (!) for them and they would have liked to see “a comparison with other operating systems, like Windows (!)” – which simply was far beyond the scope of my project… I ended up with a medicore mark for the project which is in complete contrast to the final grade of the vocational school (where I missed a perfect average by 0.1).

Ok, I cannot say that this came completely unexpected. I had been warned. Just a few years earlier, another apprentice chose a Linux topic and even failed the final exam! He took action against the examiners and court decided in his favor. His work was reviewed by people with Linux knowledge – and all of a sudden he was no longer failing but in fact got a 1 (German equivalent to an A)! I won’t sue anybody since I have passed. Still my conclusion here is that we need more people who dare to bring *nix topics on the list. I would do it again anytime. If you’re in the same situation: Please consider it.

Oh, and for another small success: The former classmate who runs Antergos also tried out FreeBSD on his server after I recommended it. He has come to like jails, the ports system and package audit among other things. One new happy *BSD user may not be much. But it’s certainly a good thing! Also all of my former classmates now at least know that *BSD exists. I’ve held presentations about that and mentioned it in many cases. Awareness for *nix systems and what they can do may lead to giving it a try some time in the future.

Top things that I missed in 2015

Another year of blogging comes to an end. It has been quite full of *BSD stuff so that I’d even say: Regarding this blog it has been a BSD year. This was not actually planned but isn’t a real surprise, either. I’ve not given up on Linux (which I use on a daily basis as my primary desktop OS) but it’s clear that I’m fascinated with the BSDs and will try to get into them further in 2016.

Despite being a busy year, there were quite a few things that I would have liked to do and blog about that never happened. I hope to be able to do some of these things next year.

Desktops, toolkits, live DVD

One of the most “successful” (in case of hits) article series was the desktop comparison that I did in 2012. Now in that field a lot has happened since then and I really wanted to do this again. Some desktops are no longer alive others have become available since then and it is a sure thing that the amount of memory needed has changed as well… ;)

Also I’ve never been able to finish the toolkit comparison which I stopped in the middle of writing about GTK-based applications. This has been started in 2013 so it would also be about time. However my focus has shifted away from the original intend of finding tools for a light-weight Linux desktop. I’ve become involved with the EDE project (“Equinox Desktop Environment”) that uses the FLTK toolkit and so people could argue that I’m not really unbiased anymore. Then again… I chose to become involved because that was the winner of my last test series – and chances are that the reasons for it are still valid.

And then there’s the “Desktop Demo DVD” subproject that never really took off. I had an Arch-based image with quite some desktops to choose from but there were a few problems: Trinity could not be installed alongside KDE, Unity for Arch was not exactly in good shape, etc. But the biggest issue was the fact that I did not have webspace available to store a big iso file.

My traffic statistics show that there has been a constant interest in the article about creating an Arch Linux live-CD. Unfortunately it is completely obsolete since the tool that creates it has changed substantially. I’d really like to write an updated version somewhen.

In fact I wanted to start over with the desktop tests this summer and had started with this. However Virtual Box hardware acceleration for graphics was broken on Arch, and since this is a real blocker I could not continue (has this been resolved since?).

OSes

I wrote an article about HURD in 2013, too, and wanted to re-visit a HURD-based system to see what happened in the mean time. ArchHURD has been in coma for quite some time. Just recently there was a vital sign however. I wish the new developer best luck and will surely do another blog post about it once there’s something usable to show off!

The experiments with Arch and an alternative libc (musl) were stopped due to a lack of time and could be taken further. This has been an interesting project that I’d like to continue some time in some form. I also had some reviews of interesting but lesser known Linux distros in mind. Not sure if I find time for that, though.

There has been a whole lot going about both FreeBSD and OpenBSD. Still I would have liked to do more in that field (exploring jails, ZFS, etc.). But that’s things I’ll do in 2016 for sure.

Hardware

I’ve played a bit with a Raspberry 2 and built a little router with it using a security orientated Linux distro. It was a fun project to do and maybe it is of any use to somebody.

One highlight that I’m looking forward to mess with is the RISC-V platform, a very promising effort to finally give us a CPU that is actually open hardware!

Other things

There are a few other things that I want to write about and hope to find time for soon. I messed with some version control tools a while back and this would make a nice series of articles, I think. Also I have something about devops in mind and want to do a brief comparison of some configuration management tools (Puppet, Chef, Salt Stack, Ansible – and perhaps some more). If there is interest in that I might pick it up and document some examples on FreeBSD or OpenBSD (there’s more than enough material for Linux around but *BSD is often a rather weak spot). We’ll see.

Well, and I still have one article about GPL vs. BSD license(s) in store that will surely happen next year. That and a few topics about programming that I’ve been thinking about writing for a while now.

So – goodbye 2015 and welcome 2016!

Happy new year everyone! As you can see, I have not run out of ideas. :)

Thea: The gain of giving away for free

This post is inspired by the game Thea: The Awakening. No, Eerie Linux has not mutated into a games blog. Yes, I will give a short description of the game. But what this post is really about is some thoughts about software development in the past, today and what could be a more open future.

Why Thea? Because the developers did something very uncommon: They decided to give the game away for free – if you’re a Linux user that is!

Thea: The Awakening

The game in question is a turn-based strategy game with a strong focus on survival. There’s a nice background story: The world had turned to darkness (playing the game you will discover why) and is haunted by creatures and spirits of the dark. Now the sun is rising again and the gods have returned but both are very weak and darkness will not give up without a fierce fight. Slavic mythology makes for a very nice and rather uncommon setting.

In case you want to give it a try, you can find a download link here. And yes, it is really completely free. You don’t need to buy the Windows version first or something.

I’ve successfully run the game on the Mint laptop that I share with my wife and can confirm that it works well. No luck on a 32-bit machine that I installed Arch on to give the 32-bit version of the game a try. It won’t start and the console messages give no clues why this may be. So if you’re still stuck with 32-bit only systems, you’re probably out of luck. ;)

The developers stated that they have not even tested the Linux version themselves! So what works and what doesn’t? Most things seem to work surprisingly well in fact. Sound, graphics, even the intro video. I’ve experienced graphical glitches with some white pixels appearing for a second (nope, no AMD video card – it’s Intel!). But this happens just rarely and is a fairly minor issue. Far more annoying is the fact that you cannot really use the keyboard: A key press works but the release event doesn’t… This is a known issue with the version of the Unity engine that Thea uses. It may or may not be addressed in a future release. You can however get the keys released by ALT-TABbing out of the game and back in. That way you can at least always access the menu.

You choose one of the gods when starting a game. I’ve played scenarios for multiple gods now. The main story (“Cosmic Tree”) gets pretty repetitive soon since it’s always the same. This is also true for a lot of the other quests. However the game has options to skip a lot of the text in case you already know it which certainly was a good idea. Some of the quests are different depending of which god you chose which keeps things interesting story-wise. Maps, resources, encounters, etc. are randomly generated for each game. This together with a challenging survival, plenty of combinations to try for crafting items and interesting gameplay, Thea might still cause a rather high motivation to replay the game often.

Software development models

I’d like to separate some development approaches here and sum them up by giving their model as I see it a name. These are no official models (I’m not a game developer) but an attempt to sum up the whole thing in one heading.

The shareware model

There was once a time when software was developed in a purely closed manner. It was developed internally and when it was ready, a release was done and advertised. The good thing was that games were often cut into “episodes” and the first one given away as shareware so people could try out the game for free and might decide to buy the full product.

The public relations model

Advertising grew bigger and bigger as well as more and more aggressive. Top titles games were often announced as development begun and some material was released along the development process to keep people hooked. This worked in some cases and failed in others (say Duke Nukem forever announced in 1996).

It was a reasonable move to try to build up an audience interested in a certain title early. The problem with that is mainly two things: You cannot keep people hooked for an arbitrary amount of time and such a continuing advertising campaign costs a whole lot of money way before you start earning anything from sales.

These problems lead to a new one, however. It puts very high pressure on the developers to meet deadlines to stay on schedule. And sometimes people in charge may even decide to release a half-baked product which almost always is a very bad idea… (what was the latest example? That Batman game perhaps?)

The community-aware model

It’s not a new insight that it is rather helpful for any title to have a large community. Some studios provide forums in an attempt to simplify building up of a community. And it’s also common knowledge today that feedback from that community is extremely valuable: Knowing your audience better helps a lot to provide the perfect product after all!

The most important point of this model is that interacting with the players is now bidirectional: There’s advertising targeting them but you certainly want to have (and honor) feedback provided by them. And it also makes sense to think about designing the game and/or providing the tools to easily modify the game and thus make it as easy as possible to create mods for the game. This can also be a huge plus when it leads to a bigger, more active and longer living community!

Independent of a single title, there is a possibility for a studio to get itself a good name by opening the source code for older games. This may require some cleaning up work first but some studios have also released code as-is (which can be rather terrible). But usually the community figures out what to do with it and before long the game is ported to new platforms, receives technical updates and enhancements. This has totally made some titles immortal: There are still new episodes, mods and total conversions for Wolfenstein being released. Yes, for a game from 1992 with extremely “poor graphics” (320×240, 8bit) by today’s standards! And there’s not one week without new maps for the mighty DooM (1993).

The community-supported model

There’s this interesting trend of “early access” games: Players are given the opportunity to playtest games before they are ready for release. People know they have to expect bugs but they can try out a game of their interest early and if they are very committed to it, they can report bugs as they encounter them.

This is a classical win-win situation: The developers get a broad testing done for free and the players can have a peak into the game early. Oh, and any form of interaction is of course always a good thing.

The community-backed model

That’s a rather new thing and basically means that some developers try to get their game crowd-funded. This can succeed and this can fail. There are examples for both cases. But while this is clearly a development model since it has a lot of impact on it, I’d say that it’s also more of a special case than a general model.

The future?

MuHa Games have made one clever step ahead with Thea as the gain of giving the title away for free on Linux is really considerable. How’s that? Well, if there was no Linux version, Linux people wouldn’t have bought the game, either. So giving it away is no actual loss: The number of people of the “hey, I would have bought it for Windows but why should I since I can play it for free on Linux!” kind are most likely extremely rare – if they exist at all.

No loss is fine, but where’s the actual gain? Well, there’s the “Just bought the Windows version. Besides: I don’t run Windows at all” type of guy. These people alone should suffice to cover the costs of the additional efforts to package a Linux release and upload it somewhere. But that’s not the main point at all: Can you say “Free advertising”? People talk about the game and people write about the game, many of which would not have done it if it had just been an ordinary game! Now with the free Linux release the game, MuHa managed to make it stand out (and that is not too easy today).

For these reasons giving it away proves to be a very sensible PR action! I do not mind if that was intended or not. That doesn’t change the facts.

Community-assisted model?

So what could the future hold? I can imagine that making the community engage even more would be a big benefit. From a studio’s perspective, fans do unpaid work because they love the product. And from the fan’s perspective it’s just cool to be part of one of your favorite games and help improve it.

What could this look like? My vision is to sort of blend closed source development with what we learned from open source development. It’s cool that people playtesting a game can report bugs via forum or email. But when will the first project set up a public bugtracker along with a tutorial on how to use that for bug reports and maybe (sensible) feature requests?

Then: What about translation? Open source achieved made very, very good results using translation frameworks like Transifex. Now Thea is only available in English. My native language is German and I would not have minded at all to dedicate some time translating a few strings (I got a nice game for free after all!). There’s a lot of potential in this.

And along that it would totally make sense to avoid using proprietary containers for files. I did not bother to try to extract text out of whatever format it is that MuHa uses for Thea. In 1999 ID Software did a clever thing for Quake III Arena: They used container files called “.pk3” – which were simply renamed, uncompressed Zip files. The benefit is obvious: Everybody can extract the resources, modify them and put things back together. Great! I noticed a lot of spelling mistakes in Thea. If I had had access to the game text you’d have received a series of patches from me (and by applying they you’d instantly see which ones are still valid and fixing mistakes). Wouldn’t that be a great way to improve the game?

Licensed Open Source model?

Can open source work for a commercial game? Well, why not? Open source alone does mean just that: The source is open. It does not say under which license and it does not say that it’s free. Now I generally support as much freedom as possible – but that last word there is important. A more open development is a nice improvement IMO. There’s no reason to demand more than that.

In this model the customers pay for the game data without which you obviously cannot play the games but the program source is open (or perhaps semi-open where it is included with the copy of the game you get when you buy it and you’re free to distribute a series of patches but not the source itself). I’m pretty sure that this can work. One potential problem here may be deadlines. Often the code in commercial games must be horrible – not because the programmers suck but because unrealistic deadlines blow. A lot of studios may hesitate to open up their code just for that very reason…

Addressing the problem could however also be easy: You sell games in early access? Buyers get the code and know that it’s early and may not be in perfect shape (and can actually help improving it). Again both sides win: The studio gets code review and maybe some patches plus some people may even attempt to port the game to platforms unsupported by the studio. The players get better games they can help to improve, take modding to the next level and even a chance to see what coding is like and get yourself some reference work if you intent to work in that industry!

There’s one other issue, though. In many cases studios will want to hide some things from competitors. That may be old (and at some point hopefully obsolete) thinking but we have to accept it as a present fact. So what about this? Well, those things could be put into libraries… It’s far better to have the program code open and make it use closed libraries than having nothing open at all!

Time for change

Who’s stepping forward making the next step in game development? I’m really curious if something in the direction of what I wrote here happens any time in the future. For each step there’s good press to catch for free again, you know? ;) Perhaps some small studio dares to make the move.

Update: I wrote this in a hurry on 11/30 to rush out my November post. And then I once again forgot to make it public. But now it is…

Exploring FreeBSD (3/3) – a tutorial from the Linux user’s perspective

This is the third and last post of a series of introducing FreeBSD to Linux users. You might want to take a look at the first post (talking about some things different from Linux) and the second one (about binary updating and package, user and service management) if you have not done so already.

If you’re all new to FreeBSD (or the BSDs in general) I tried to sum up the most important things to know about this OS family in another post. And if you want to know how to install FreeBSD (and what disklabels are as well as some other *BSD specific stuff), there’s yet another post dealing with it.

So what are we up to this time? There are a few topics left that I want to write about (and quite some more that I would like to touch on, too – but it doesn’t make sense to try and put too much into too little space): Updating binary packages, the ports system and updating “world” (the OS itself) from source.

Updating packages

In the last post we installed bash via FreeBSD’s port system (pkg). About one month has passed since then and a new version of bash has been released in the mean time (just as I hoped it would!). So let’s see how to update packages, right?

The most common case is that you want to update all your packages. There are two commands you should know in this regard:

# pkg update

This updates the repository catalogue so that the system knows which package versions are available in the remote repo. You don’t normally have to run this explicitly since FreeBSD will automatically fetch the latest catalogue if it thinks that the local one is too old.

# pkg upgrade

This will tell you which packages can be updated and perform the actual update if you choose to do it.

Updating binary packages

In this case, a new version of the package management tool was also released. Pkg must be updated before any other updates can happen but other than that it works just like any other update does.

The ports system

What are “ports”? The process of making a software (for which the source code is available) build on a system that it was not necessarily meant for is called porting. Depending on the piece of software this can be easy (the program builds out of the box) or extremely challenging (a lot of code needs to be patched to make it work). In order to make things easier for everybody, FreeBSD developed the ports system which is basically a directory for each application that was ported and a Makefile as well as some support files in it. These contain everything needed to build the respective application on FreeBSD simply by issuing make inside that dir. The directories make up what is known as the ports tree.

Fetching a port snapshot

The ports system originated in early FreeBSD and quickly spread to the other BSDs as well. And even on Linux there are people who like concept: Gentoo Linux for example is based on portage which builds on the very same concept (but works rather differently in the end). Well, since I told you to deselect the ports tree during the installation you do not have it on your system. So let’s first get it in place!

Getting the ports

All newer versions of FreeBSD offer the portsnap command which makes that very easy:

# portsnap fetch

If you do not have the ports tree on your system this downloads a snapshot, verifies it and also fetches any patches for ports changed after the snapshot was created. You can use the same command to fetch the newest patches if you already have a ports tree and receive any changes made in the meantime.

# portsnap extract

With this command you tell the system to actually unpack the snapshot and populate the ports tree. Only use this the first time you install the ports tree to your system. It doesn’t make sense to use it afterwards!

# portsnap update

You do not need this if you have just installed the ports tree for the first time. It is used to update the local ports tree after downloading any patches with fetch. If you wish you can also combine the two parameters to update the ports tree (portsnap fetch update).

Extracting the ports tree

You could also get the ports tree via Subversion. But portsnap is just so convenient to use that there’s barely any reason to do so.

Finding your way around the ports tree

So now let’s take a look at it! Where are all the files? They are in subdirectories of /usr/ports. We’ve installed bash in the last post using binary packages. Where would we find it in case we wanted to build it from ports? Being a shell, /usr/ports/shells/bash is quite a logical place, don’t you think? And where would you look for, say, the ruby interpreter? You’ll find multiple versions of it in /usr/ports/lang/ruby2x (ruby 2.0, 2.1, 2.2).

If you work with the ports tree for a while you’ll get at least an idea where things belong to. But what is the best way to locate a specific port? You can use the whereis command followed by the program name and it will tell you where the port lives! Just make sure you type in the right name. You won’t find php for example. But you will find the port if you look for php55 or php56 instead.

Finding applications in the ports tree

Still having trouble? Perhaps the page FreshPorts can help you. You can search there and chances are good that you find what you are looking for and can find out the category and port name that way.

Building from ports

The first question is of course: Why should you build programs from ports? The ports system was invented to automate the build process when there were no binary packages available and you had to build every program from source. Today you can easily work with FreeBSD without ever touching ports.

But when does it make sense to use ports? The simple answer: If you have special needs! The binary packages are pre-build and there’s no way to change any compile-time options. If you’ve ever manually build a program on e.g. Linux, there’s a good chance that you have met configure which takes options like –prefix=/usr –without-package-xz –enable-newest-feature and so on. If you need some program feature that the pre-packaged program does not come with on FreeBSD, you can use ports. Or if you do not want a certain feature built-in which is selected by default, you can also use ports.

Selecting build options for a port

For packages that can be built with different options which the author of the port thought were interesting, you will be given a nice dialog window in which you can select or deselect certain options. Just navigate into the directory of the ports tree where the files for the application you want to build live and issue make.

This will bring up the configuration window if there are any options to set. Please note that your selection will be saved so you are not asked the next time you build the port. If you changed your mind and want to reconfigure the options, you can use make config.

Building links from ports

If you order the port to “make”, the source code will be downloaded from a known location (you do not have to do this yourself), decompressed, probably applied some patches and then built. Once the build is complete, you can use make install to install the program and make clean to clear the build directory of files remaining from the built.

It is also possible to combine several commands which the program make takes (these are called targets and are defined in a file called Makefile). So you can build, install and clean up one port by issuing make install clean.

You also don’t have to worry about dependencies. If a port needs other programs (or libraries) which are not present on the system, they will automatically be built from ports, too. And one more important thing: Don’t hesitate to mix binary packages and ports on your system. You don’t have to choose one and stick with that all the time. In fact the ports produce custom binary packages which are then installed using the normal package system. That’s why pkg is aware of any program that you installed via ports and can for example remove it from the system if you tell it to. You could also go to the port’s directory and use make deinstall.

Recursive operations

If you want to build a complex program that has lots and lots of dependencies (like e.g. Libre Office), it is a good idea to let FreeBSD build it overnight. There is, however, a big problem that you’ll face if you try out large unattended builds: Every now and then, when a new port is built as a dependency, FreeBSD displays the configuration window and pauses until you make your choice…

This is why there are recursive targets: You can use make config-recursive and the ports system will go through all the dependencies and display the configuration. So you can select all the options that you need at once before you use just make to build all those programs.

Recursive source fetching

Mind one thing, though: If you enable more options, you may want to run config-recursive again. Why? Because the options that you selected may have pulled in new dependencies which are not yet configured. Running config-recursive will only display the configuration dialog for ports that were not configured previously. If you need to re-configure all ports, you can use the make rmconfig-recursive target to delete the stored configuration for the port and all dependencies and configure them again afterwards.

And in case you want to pre-load all source tarballs before starting an unattended build, there are the make fetch and make fetch-recursive targets. In very rare cases it can happen that all the sources that one port knows for its tarballs are no longer available (this is more likely to happen if you’re using a no longer supported version of FreeBSD and/or an out-of-date ports tree). You can fix this if you simply find another source of the needed file on the net and download it to the /usr/ports/distfiles/ directory where all those source tarballs for the ports live.

Updating the system from source

Just like with ports the first question ought to be: Why should you? And in this case the answer is even more: You probably shouldn’t. There are people who like to build from source and that’s ok. But if binary updates work for you, in general you should stick to them.

When do you need to compile the system from source? Well, obviously this is the method of choice if you are a developer who needs to build the absolutely newest code. But if that’s the case you’re probably not reading this tutorial anyways, right?

So – why should you do it? There are basically three main cases:

  • To have it done once
  • Because you want to aim for the stable branch
  • You want to customize e.g. your kernel configuration

Do not laugh at the first one. It is a perfectly valid reason. While building FreeBSD from source is extremely easy, it is good to have done it at least once. It will help you to get a little bit closer to your system.

FreeBSD comes in several branches. You can decide to follow another branch and compile the code for it. We’ll talk about that in a minute.

And last but not least if you have special requirements and want to customize your system for that. E.g. you man decide to compile your firewall of choice (FreeBSD offers three of them) into the kernel. In that case you have to build from source.

Getting the source

We cannot discuss scenario three (customizing FreeBSD) here. That would require its own post (or even more). Besides – I’m not too knowledgeable in that field.

Installing the certificate bundle

Let’s assume we want to follow the stable branch. First we need the appropriate source code. FreeBSD uses Subversion for version control and a slimmed version of it comes with the system (“svnlite”).

You may want to install the certificate bundle first so using a secure connection does not result in an error because the certificate is unknown. To do that you can simply use the following command: # pkg install ca_root_nss.

Next we need to checkout the current version of stable code with svn. FreeBSD source code always goes into /usr/src.

Checking out system source with svn

Start the checkout process with

svnlite checkout https://svn.freebsd.org/base/stable/10 /usr/src

and wait for Subversion to finish. This can take quite a while because the source code is quite large.

Once it’s done, you’re set. Go to /usr/src and issue make buildworld. This will build the userland part of FreeBSD (and – depending on your CPU – take a long time to finish).

System source checkout completed

What gets build goes into /usr/obj, btw. So the source code is kept separate from it and anything in /usr/obj can be easily removed anytime before doing a clean new build.

Building the FreeBSD userland from source

When the world build has completed, it’s time to build the kernel as well: make buildkernel – this does not take such a long time to complete.

Now both parts of the system need to be installed with make installkernel and make installworld. Always remember the correct order:

  1. Build world
  2. Build kernel
  3. Install kernel
  4. Install world

The reason is that “buildworld” needs to run first, is that it uses the system compiler to bootstrap the new compiler which is then used to build the whole userland and, after that, the kernel. And the reason that the kernel should be installed first is that after updating the userland you really should reboot. You’ll probably get away without rebooting if you just updated within the same release version but updating to a new release from source will mean that you cannot count on the system to just keep running like before due to incompatible changes made. In theory you are even encouraged to boot into single user mode to do the update! But I have not found that this is really required. Just mind the right order and stick to it.

Building the FreeBSD kernel from source

After rebooting you should find that the system is running on the new kernel. Now we’re on FreeBSD stable. However… That does not at all mean what you’re probably thinking it does!

New kernel is running

FreeBSD branches

I’ve stated before that there are multiple branches of FreeBSD, one of which is stable. Let’s take a look at what they are.

First there’s release. If you followed this tutorial along, version 10.1.0 was the system that we started with. Uname denotes the kernel as 10.1-RELEASE. Release is just that: A certain release. It will stay as-is forever, no changes applied to it.

Then there’s the patch branch or “releng“. This is “release + patches” and in fact the most stable branch available due to error corrections and security fixes. Uname will report something as kernel 10.1-RELEASE-p12. The patch branch is meant for conservative production systems.

We’ve already touched stable and even updated to it. If the patch branch is the most stable version, why is this one called “stable”? Yes, it is a bit confusing, I know. The reason is that this branch receives new features (which the patch branch does not) but the APIs are kept stable. Hence the name. This branch is not officially recommended for production use but the company that I work for has used servers with stable for years and they behaved absolutely fine.

Finally there’s current (called “head” in the repository). This is where the development takes place. If you’re not a developer or somebody who wants to test the newest features as early as possible, this is not for you.

What’s left

I would very much have liked to cover file flags & secure levels as well as jails. I’d liked to have written about tools like portmaster and system components like the three firewalls. But that might or might not happen in a future post…

What’s next?

In exactly one month I’m going to write my final exams to become a qualified IT specialist. So I’ll have to see what topic (if any) I manage to write about next month. Since I’ve always wanted to write the followup to my post about licenses, this may be a good candidate.

Exploring FreeBSD (2/3) – a tutorial from the Linux user’s perspective

This is the second post of the “Exploring FreeBSD” tutorial. If you didn’t do so already, may want to read the first part before this one. And if you are completely unfamiliar with FreeBSD, the posts installing FreeBSD and the general introduction of the OS may also be of interest for you.

In the first part we have configured SSH insecurely to allow root login, set up port forwarding in VirtualBox and briefly explored commands and the default shell on FreeBSD. Now it’s time to continue our journey!

Since some screenshots show a lot of lines I sometimes cut out the relevant part to save a bit of space.

Updating the system

FreeBSD is an operating system well-known for its reliability. But of course just like any other complex systems, it is not perfect – there are bugs and security holes. These are addressed rather quickly since FreeBSD takes those issues seriously. To always have the currently most secure and stable system you need to perform updates. As updating is important, let’s get right to it!

FreeBSD binary update

In FreeBSD there are several branches, but we will ignore this for now and save it for the last part of this series. There are also two supported methods of updating the operating system: Using binary updates and building from source. We’ll just cover the former here and also take a look at the later in the next post.

Updating your system within the current release basically boils down to issuing two commands (well, actually one command and two parameters): freebsd-update fetch and freebsd-update install.

Summary of files to update

Once the fetching is complete, freebsd-update will display a list of changes that will be applied to the system. There you can take a look at which files will be replaced with newer ones. If you agree with the changes, you can install the actual update and – if the kernel was affected as well – reboot.

Installing the update and rebooting

Updating the operating system is actually as easy as that. Keep in mind however, that with the BSDs operating system (“world”) and installed software (“packages”) are things separate from each other! We’ll cover updating packages in the next post.

Release upgrade from 10.1 to 10.2

But what if you want to update to a new release? We’re lucky here: Since this series of articles started with installing FreeBSD 10.1, a new release has happened: 10.2! So let’s update to the new release version, shall we?

Again the freebsd-update command is our friend. It can also do a binary upgrade from one release to another. It’s freebsd-update upgrade this time and with the -r 10.2 option we choose to upgrade to that particular release version.

The release upgrade needs a lot of patches and files!

This process takes a whole lot longer because there are of course far more files affected. But it is essentially the same process: Fetching patches, fetching new files and displaying three lists: files that will be removed, added and modified if the upgrade is installed.

After the upgrade: Rebooting again

After installing the upgrade, just reboot the system. A moment later you should be greeted by your new 10.2 FreeBSD system!

The updated 10.2 system

Adding users

User management is surprisingly easy on FreeBSD. There’s the powerful pw command that can do just about anything user related. And for adding users there’s also the convenient adduser script that makes adding users to the system a breeze.

Adding a new user

Most things are completely self-explanatory. What may however be new to you is the login class concept. Chances are that you won’t need them, but it’s nevertheless good to know that they exist and what they are. On a system with multiple users it could be that some of these users want to use e.g. different locale settings. FreeBSD allows you to create login classes that control localization – only affecting users who have that login class set for their account.

Take a look at the available shells. Missing something? In that case the shell is not installed on the system. If it was, adduser would offer it to you. All shells present on the system are recorded in /etc/shells by the way. Feel free to cat it out and compare it to what adduser offers!

Now let’s switch to the new user and try to become root again. Nope, sudo is not part of the base system. We could install it, but for now we’ll go without it. Fortunately we know the root password. So let’s su to root!

Only “wheel” members are allowed to “su”!

“Sorry”? Now what’s that? Certainly not a very helpful error message! Well, you just met another peculiarity of FreeBSD: You need to be part of the wheel group for su to allow you to become root.

So let’s try that out. And indeed: After logging out, adding the user to the group using pw usermod [username] -G wheel and logging in – su let’s us become root.

Package management

Traditionally programs have been built from source in an automated manner using something called the ports tree. We’ll cover that in the last part of this article series.

The other choice obviously is to use binary packages. FreeBSD has used what is used the pkg_*tools for a long time. Up to the still supported FreeBSD 9.x, these are the default package management utilities. You use pkg_add -r to add packages to the system, pkg_info to display information about them, pkg_delete to remove them, and so forth.

On FreeBSD 8.4 and 9.x the new pkg-ng tool could be optionally used. Since release 10.0 it is the new standard tool for dealing with packages. It is however not part of the base system and thus does not come installed by default.

It will however be binary-bootstrapped (a package manager is needed to install the first package, too, right?) if you first try to use it. For that process it doesn’t make a difference if you provide any parameters to pkg or not.

Bootstrapping the pkg binary package manager

Pkg-ng uses just the unified pkg binary which allows for subcommands. Once you have it on your system, you can use pkg install to add packages to your system, pkg info to view package-related information and so on.

Let’s just install the BASH shell for now. It is as easy as typing pkg install bash. Pkg will list the package and its dependencies and ask for confirmation. If you give it, it will download and install the packages. That’s really all there is to it.

Package management is of course an important topic. The new pkg-ng tool is so easy to use however, that I won’t make any additional examples here. Be sure to have a look at the man pages or read the handbook article.

Installing a familiar shell: Bash

User management again

Ok, we have BASH installed now. Just a moment ago I told you that it should now be available if you add a new user. Now what to do if our already created user should get it as the default shell?

What we could do is pw userdel on it and then re-create it. We could even manually mess with the /etc/passwd file. But this is not the best way to do it. In fact it is much simpler than that.

About to change the shell for the current user

There’s a program that let’s you conveniently change user information of existing users. It goes by multiple names and allows editing different user information. In our case we want to use chsh – “change shell”. It will fire up an editor and let’s you edit the user’s login shell among other things.

DO MIND that FreeBSD keeps packages separate from the base system! There is no /usr/bin/bash! If you enter that as the path to your login shell, the user won’t be able to login anymore. In FreeBSD you’ll find it in /usr/local/bin/bash. The same is of course true for other shells that are not part of the base system and for any other software that is installed from packages in general!

Altering user information

It is also noteworthy that you should NOT change the login shell for root. Leave it as it is and start bash manually if you really have to. On a toy system it may not be much of an issue, but there is no need to grow bad habits in the first place. If you ever need to repair a damaged system and you have changed the default shell for root, you have a good chance that it will bite you.

BASH is now the new shell for my user

System services

We have one more topic to deal with in this article. Knowing how to manage users and adding packages is nice, but being unable to mess with services kind of makes FreeBSD useless for you. So let’s take a brief look at how FreeBSD works with services.

You can simply ps -aux like on linux to take a look at what is running on your system. But how to manipulate daemons properly?

For a while now (it was introduced sometime in the 8.x release versions, if I remember correctly) FreeBSD comes with the service command. It is a valuable tool that you should start using right away. Sure, you can use the init scripts by hand, too. But service has the advantage that you don’t have to think if something belongs to the base system (and is thus located in /etc) or not (in which case you’d have to look in /usr/local/etc)!

Taking a look at services

It also provides a few more nice features. First let’s have a look at which system services are currently enabled (run automatically on startup). To do this, simply type service -e.

If you want to know all services on the system (which you could start), use service -l. This produces a long list, so it might be a good idea to use a pager like less or grep for something if you already know what you are looking for. In our example let’s look for the ntp daemon: service -l | grep ntp. No surprise: It’s called ntpd.

It won’t keep running without the right parameter because my clock difference is too big. But we’re not covering ntpd here, right? It’s just an example and you can of course use other services as well.

How to enable a service?

First let’s ask the system about ntpd’s status: service ntpd status. Now that error message tells us that ntpd is not enabled in the system configuration. It could still be started manually but as long as it is not enabled in the rc config file, FreeBSD keeps reminding you of that fact (which is actually a good thing).

Just like it suggested, we can use service ntpd onestatus. Truthfully it tells us that the daemon is not running. We can start it despite not being enabled in rc.conf using service ntpd onestart. Now onestatus lets us know that it’s running. Keep in mind however that such a manually started process will not be started when the system boots!

We could stop the daemon again using the service command. But to show off the init script way we’ll do it without it one time: /etc/rc.d/ntpd onestop.

And now finally let’s take a look at the configuration file and how we can enable any service. Fire up any editor on /etc/rc.conf and add the line ntpd_enable=”YES” to it. That’s all in fact. In case you want to give any parameters to the daemon, you can do so by adding an optional line like the following: ntpd_flags=”-x”.

Where services are configured: The “rc.conf” file

There’s a lot more to know about services and I encourage you to take a look at the FreeBSD handbook on that topic. But for our short introduction of the very basics, that’s it.

What’s next?

The next post will be the last one of the FreeBSD introduction. It will deal with the ports system, updating from source and a few other things.

Exploring FreeBSD (1/3) – a tutorial from the Linux user’s perspective

The previous blog post detailed the FreeBSD installation and the one before introduced the OS in general.

This one is going to show you around a little bit. There’s a lot to see here – so let’s jump right in!

Convenient access

Let’s start the VM with the fresh FreeBSD installation and login as root. The Virtualbox graphical console does allow you to work with your VMs but it is not a convenient way: You cannot copy just some lines of configuration into the buffer or paste a longer command to execute. Also if you’re like me and use an optimized keyboard layout, chances are that in VBox you’ll have to use US or any standard keymap.

Or maybe you want to scroll up your terminal when you need to see what that strange output five minutes again was. Sure, you can do that in a Virtualbox console, but it’s much more comfortable using the mouse. When you’d try SHIFT-PG UP or SHIFT-PG DOWN like on a Linux system it won’t work BTW. FreeBSD has its own scrolling mode which you can enable/disable pressing the scroll lock key.

And of course it’s always nice if you can see multiple logins at once – say with split-screen tmux. You can switch to another TTY with CTRL-ALT-Fx on FreeBSD just like in Linux. In Virtualbox you use right CTRL and one of the Fx keys. That works but you can’t see two or more logins at the same time.

Anyways: There are more than enough perfectly valid reasons to get rid of Virtualbox’s graphical console. So let’s just do that. But first we have to edit the configuration file for the ssh daemon:

# vi /etc/ssh/sshd_config

Find the line #PermitRootLogin no, remove the comment sign and change it to yes. Save and exit vi.

Now we’d have to restart the ssh daemon for the changed config to be applied. But we won’t do that. Instead we’ll just power down the machine. You can do that as you’re most likely used to from Linux:

# halt

A moment later you’ll see something like this:

All buffers synced.
Uptime: 50s

The operating system has halted.
Please press any key to reboot.

This is FreeBSD’s way of saying: “Alright, I’ve finished. You can safely power off the machine now”. Close the VM.

Note: To properly power off your FreeBSD machine you should use the shutdown command instead of halt or reboot as that will try to stop running services instead of just killing them. Shutdown can take the parameters -p to power down or -r to reboot the machine respectively. Thanks, TK, for reminding me of that as it really is something that a tutorial reader should know!

Now click on settings for your VM, choose network and click on advanced. Then click the button labeled Port Forwarding. In the new window add a new rule and enter 22 (the default port for SSH) as the guest port. Redirect it to any free port (higher than 1023) on your host machine – I’m using 20022 here.

Port forwarding rules for Virtualbox

Now let’s start the machine again. But since we don’t want the graphical console, let’s start the VM without it. Virtualbox offers something called headless mode which does just that: Start a VM in the background without a console for it. You could do so on the command line – but did you know that more recent versions also allow you to do this by double-clicking a VM while holding down SHIFT? Try it out!

Look at what Virtualbox calls the “preview” to watch the boot process and see that it is really starting. When it has booted up, it’s time to connect. User your favorite terminal to issue the following command:

$ ssh root@localhost -p 20022

You’ll then be prompted for the root password and then – there we are!

Alright! I don’t have to tell you that in general it is a very bad idea to enable root login for SSH – even more so if you allow logging in with password. But here for our purpose of just playing around on a non-production system it’s fair enough.

And what if you did something funny to your host’s ssh configuration and it won’t connect? Well, SSH problems are a special topic of their own. But here’s one not to miss: Whenever you run in trouble, be sure to include the “-v” switch for verbose output. This is bound to give you some hint at what is happening (and an idea what to search for with your preferred search machine if don’t know at all).

Welcome to FreeBSD!

That’s what your terminal is probably saying right now (if you’re just reading this post without following along, have a look at the screenshot).

Connected to the headless VM using SSH

Release notes and security advisories are probably not of too high importance for you if you’re new to FreeBSD. But I cannot recommend the handbook enough. It covers a large part of the system in detail and starts at a rather low-level (even for people without much command line experience). It is extremely helpful if you want to find your way around with FreeBSD. Have a look – and take your time reading. Really.

Don’t just jump at the forums. You are not ready for them, yet. Sure you’ll face problems. But this is Unix not Linux. kindergarten’s over. Learn to stand on your own feet and use your brain when you run into trouble. People in the forums will probably be happy to help you if you’ve read the handbook, FAQ, did some research on the net and still have no clue how to solve your problem. Do NOT consider it the first place to look for help! If the information you need is on the net and can be found easily then you’d steal everybody’s time asking stupid questions (and make yourself look like a dumbass at the same time).

Oh, and you may want to get used to consulting the man pages if you haven’t already. You may know that those exist from Linux but you may just have lived happily without them. They are a great resource of information. Embrace them and immediately benefit from the knowledge that they hold (without having to ask somebody and wait for an answer)!

Console commands

Use CTRL-L to clear the screen. Let’s see what FreeBSD can do! whoami, pwd, etc. You know the score. They all work and do what you’d expect.

Now let’s create two directories:

$ mkdir -p test/directories

Works like a charm. But perhaps we don’t really want them so let’s delete them again:

$ rm test
rm: test/: is a directory

Ah, stupid me. That command does not remove directories if it’s not given the -r switch! So let’s try again; the up-arrow key brings back the previous command so I can just append the missing switch. Now that should work, shouldn’t it?

$ rm test -r
rm: test/: is a directory
rm: -r: No such file or directory

What the…?! Why is stupid rm interpreting the switch as a file? This should work, right? Nope, rm is reacting just fine, it’s really my bad. But it does work on Linux, doesn’t it? That’s correct. So what’s happening here? In fact the above line is syntactically wrong. You should always state: 1. command, 2. option(s), 3. parameters. When rm met the first parameter (“test”) here, it stopped expecting options and thus treats “-r” as another parameter. And that’s fine – there could be a file “-r” after all as that’s a perfectly legitimate file name!

$ rm -r test

No surprise here: This works.

Lesson 1: FreeBSD is much more strict while Linux (with its GNU tools) lets you get away with a lot of strange things. Some people say that this makes GNU/Linux more “friendly” but that totally depends on how you look at it. What if you want to remove the files “test” and “-r” in GNU-land? It’s possible there, too. But you have to type this:

rm -r -- test -r

The double dash tells rm to stop parsing options and treat every following input as a parameter. Works just as well, for sure. But you need to know how to use the double dash or you can run into pretty bad edge cases without a clue what’s going on. The BSD way of doing things is not aiming at this kind of “friendliness” but just doing things right in the first place.

Mind the syntax!

Let’s take a look at what df can tell us. Except for the device name there’s nothing too strange here. If you wonder how that device naming works, you may take a look at my previous post, where I explained it in section “Excursion: Partitions and *BSD”.

But there is one special thing about df which you might run into if you decide to do some *BSD: Capacity can go over 100%! Yes, you can come across e.g. the root partition being at 105% or something like that. This is not an error in df – it is something that is conceptionally different. FreeBSD reserves 8% of disk space for the OS and root and df considers the complete space minus the reserved space to be 100%.

The shell

Now let’s try to play a little with environment variables. Setting and displaying one should be easy, no?

$ export ALL_FINE=”sure!”
export: Command not found.

Uh, what’s happening here? The system doesn’t know about export! Do you know what it belongs to? Try running which export on any Linux system. You won’t find it there either. What a mysterious case! Ok, not really. Export is an internal command provided by bash, the standard shell of most Linux distributions. FreeBSD uses another default shell, so let me introduce: The C-Shell (csh)!

Different shell: csh.

The most important thing that you need to know about the C-Shell is that it is fundamentally different in many ways. I’ll just show setting an environment variable here to show off something; the whole topic is huge and if you care enough, you should easily find something helpful to read. But we’ll simply be installing bash later in this tutorial. Why? Because there’s enough to learn for FreeBSD anyway and it doesn’t hurt if people can at least explore with a shell they know and feel comfortable with.

$ setenv ALL_FINE “sure!”

This is what the csh expects if you want to set an environment variable. Mind the missing equals sign! And if you want to display all the variables currently present, just run setenv without any parameters.

What’s next?

This post tried to give some very basic information for you to get along on FreeBSD systems. In the next two parts we’ll have a look at user, package and service management, system configuration, ports and updating the system.

Installing FreeBSD – a tutorial from the Linux user’s perspective

This article deals with installing FreeBSD. It’s meant for the first time FreeBSD user who has a bit of Linux background.

The installation process is actually quite simple and straight-forward. I’ll guide you through in some
detail, anyways, because I think that there are a few things that are good to know even if they are not strictly necessary.

On a side note: My blog turned three a few days ago. I decided not to write a birthday post this year. Let’s just get on with some real content!

Preparations

The easiest way to try out a new operating system these days is to run it in a virtual machine. I’m using VirtualBox to create the screenshots and you are invited to create one, too, and follow along. Just reading the post is also fine, though. And of course nobody will stop you if you decide to dedicate a real machine to this project because you have some spare hardware around.

If you opted for VirtualBox, you just need to create a new VM first. Select FreeBSD as the OS and give it a bit more RAM and far more disk space (since we’re actually going to put our new system to some use) than VirtualBox suggests. Other than that, you’re set up. You should not need to make any other changes (though you can of course adjust things if you feel like it).

Now download an ISO image if the installer CD from here.

We’ll be using FreeBSD 10.1-RELEASE. Pick the right one for your CPU architecture – if you’ve got a PC then i386 means “32-bit” and amd64 means “64-bit”. It doesn’t matter if you’ve got an Intel or an AMD. Make sure you choose an installer image and not a virtual machine image since we want to install FreeBSD after all!

The smallest image is fine; that’s FreeBSD-10.1-RELEASE-amd64-bootonly.iso for a 64-bit system (or the .xz one which you need to decompress after the download). If you were going to install FreeBSD on several machines or on one that is not connected to the net, it would make sense to choose a bigger one which actually includes the OS you want to install. But for our purpose the bootonly (which will download the OS files) suffices.

Got everything? Excellent. Start up your VM and put the installer image into the virtual drive. Then you’re set to go.

Installing FreeBSD: Keyboard and file sets

You should be greeted by the bootloader. Either wait ten seconds or just press the Enter key (we don’t want to set any special options at this early stage).

The FreeBSD bootloader screen

Now the kernel will probe your hardware and the system will come up. Once everything is done, the installer will be started automatically. It’s not the most beautiful installation program ever, but it does the job. And it actually makes the installation really easy. Until FreeBSD 8 there was the old installer, BTW. It was more powerful but the installation was also more complex (and it was even more ugly! ;)).

Meet the FreeBSD installer!

After choosing “install” you can select a keymap or go with the default one (US). It’s all up to you here.

Keymap selection in the installer

Then we have to give the virtual computer a name. I decided to call the machine beastie which is the name of FreeBSD’s mascot (the little red daemon with the fork).

Setting the hostname of the machine

Next select which optional sets to install. You won’t need doc. This may have been very useful in the past but today you’ll probably want to simply browse the documentation online instead of locally, anyways. If you really want it, it won’t hurt you, though, but you could just as well save yourself (and the FreeBSD project) some bandwidth.

Games is not what you probably think, BTW. It’s pretty obvious that this won’t install DooM or Quake or the like. But it’s not even something like Solitaire or Freecell that comes with Windows. It’s in fact just a bunch of command-line based games, the most popular being “fortune” mentioned by the installer. It is a “fortune cookie simulator”. Most people probably won’t call any of the programs from this set an actual game. They are more or less just little fun or joke programs. Actually they have a long tradition with Unix and they are very small, too. For that reason they are included in a FreeBSD install by default. You may deselect this set if you really want but they definitely won’t hurt you, either.

Choosing the sets to install

The lib32 set contains the 32-bit libraries which are needed if you want to run 32-bit executables on a 64-bit FreeBSD system. There are actually more cases where you need them than you might think. So if you aren’t really, really short on disk space or know exactly that you won’t ever need them (hint: When you are just starting with FreeBSD, you don’t) just leave this one checked.

For our tutorial please make sure you uncheck the ports and src sets which contain the ports tree and the system source code respectively. We will need both later – but we’ll fetch the current version of both them using different a different means!

Installing FreeBSD: Network

Now you need to configure your network so that the sets can be downloaded. This is a pretty straight-forward thing to do: Select your NIC (chances are you only have one, anyways), select IPv4 and choose DHCP if you wish to automatically receive an IP address (otherwise configure the NIC manually). You probably don’t need IPv6 so skip that unless you’re really actually using it in your LAN. If you selected DHCP, FreeBSD should have received the address of at least one DNS server as well. If it didn’t, make sure to provide one. Otherwise name resolution won’t work and the installer cannot download the file sets.

Selecting the mirror

The next step is to select a mirror server from which to download. It obviously makes sense to choose one that is located in a place near to you as it is more likely to provide a good connection for you.

Installing FreeBSD: Disk layout

All that’s left is setting up the hard disk(s). The basic choice you have to make is which file system you want to use. FreeBSD basically supports two native file systems: The traditional UFS (“Unix File System” aka. FFS or “Fast File System”) and the next-gen filesystem ZFS. The later is a sophisticated FS with a lot of interesting features. For quite some people, fact that ZFS is considered stable on FreeBSD is a killer feature of this operating system and even the main reason why they choose this OS.

ZFS is however well beyond the scope of this post. A dozen of posts like this could be written on that topic (probably not by me, though, since my ZFS knowledge is rather basic, at least at the time being).

So we’ll opt for UFS now. Partitioning a FreeBSD system is a little bit more complex than partitioning Linux. Fortunately there’s the Auto (UFS) option in the installer. We select that and want to use the entire disk.

Disk partitioning

Today there are two partitioning schemes in use on the PC. FreeBSD defaults to the newer one, GPT. You could also use the older MBR instead if you have any reason to do so. And in fact you could even go without any of them! But that’s going deeper than we need right now.

The installer suggests a default partitioning: A boot partition, one for the root file system and one more for the swap space. Depending on what you want to do with your FreeBSD machine, this is most likely not what you want. But for our test system it’s fair enough.

The default partitioning layout

After hitting “Finish” and “Commit” the changes are written to disk and the actual installation begins.

Excursion: Partitions and *BSD

Partitions are a topic which can be highly confusing for the beginner (at least it has been for me before I did some research in this area). The problem here is that in the FreeBSD world a partition is something different what you might think. And to make matters worse, the terminology differs even between the BSD distributions!

Most Linux distros use the old MBR (“Master Boot Record”) partitioning aka “DOS partitioning”. Same thing if you come from a Windows background. You know the score: Up to four primary partitions and extended partitions if you need more than that. Chances are that your Linux distribution uses three primary partitions: One for /boot, one for SWAP and one for /. If they are on the first hard drive (sda) of your pc then they will be called /dev/sda1, /dev/sda2 and /dev/sda3.

These partitions are known as disk slices in the FreeBSD terminology. So it’s just a different name for the same thing, right? That cannot be so bad! Wrong. The MBR partitions are the same thing as the slices, yes. But the fun starts when you learn that the BSD systems use a mechanism called BSD disklabels. These divide the MBR partitions further and if you think in the DOS terminology of partitions, disklabels actually allow for what might be called sub-partitions as that is what they are. Unfortunately these are just called partitions in the FreeBSD context!

So remember this: In FreeBSD a “partition” is what you may think of as a sub-partition and a “disk slice” is what you commonly know as a partition. How come that we have all this confusion? Who’s guilty of causing it? Well, things are not so easy here…

Unix began its life not on the PC but on bigger research computers which did not support partitions at all. So the Unix people came up with disklabels to partition disks into up to 8 partitions. Since that is what they are, it was an obvious choice to call them partitions. Quite some time later the PC platform supported MBR partitions which were also called thus. The real trouble started however when Unix was ported to the PC: Now there were two different things with the same name! For compatibility’s sake, FreeBSD embeds its partitions (sub-partitions, remember) into what they call disk slices (MBR partitions) to be able to distinguish between the two.

As a consequence, FreeBSD needs only one disk slice (MBR partition) because it can create partitions (sub-partitions) on it, e.g. for SWAP.

If you are using the MBR scheme, it leads to a naming like e.g. /dev/da0s1b. This means the first SCSI disk, slice 1 (primary MBR partition 1), partition 2. /dev/ada1s5f means the second SATA disk, slice 5 (extended MBR partition 1), partition 6.

FreeBSD can also do without slices. This is called “dangerously dedicated” mode. The name sounds quite worrying but in fact the only “danger” is that it is highly unlikely non-BSD systems will be able to read any data from it (since they don’t know about disklabels). If you ever come across something like /dev/da1d, you know that it’s the 4th partition (sub-partition) on the “dangerously dedicated” (MBR partition-less) disk da1. If you just have *BSD on your drive, you can use this mode and there’s no “danger” for your data.

Fortunately things became easier with the introduction of GPT (“GUID Partition Table”), a newer partitioning scheme that supports more than the 4 primary partitions of MBR. And since more than enough partitions can be created this way, FreeBSD does not use disklabels to subpartition them further. The good thing is that many other operating systems know GPT partitions, too. The bad thing is that we have another kind of partition that’s just called… Partition.

Newer FreeBSD installations default to GPT partitioning. If you look for your drives then, you’ll probably find them as something like /dev/ada0p1 which means the first GPT partition on the first disk.

If you have to use the older MBR partitioning for some reason, there are a few things you should know about disklabels. Disklabel a is meant to hold the root filesystem (/), b is for SWAP. C is completely special; you cannot use it as a normal partition. It’s always there and covers the whole disk. This is used by some tools to access the disk in raw mode, neglecting any partitions or whatever. Historically d stood for the whole disk and c for the complete slice – but that time has passed.

That’s a lot of information, I know. But you’ll get the hang of it if you want to. It’s not that difficult once you got rid of the confusion.

Installing FreeBSD: Putting the system on the disk

Now lean back for a while; the installer will fetch the distribution packages from the net first.

Fetching the distribution packages

As said before, this step is skipped if you use a bigger image. In that case you’ll also configure the network settings later in the installation process.

Installing FreeBSD: Final steps

When all distribution packages are downloaded and extracted, you are prompted for a password for the root user.

Setting a password for the root user

Then you have to tell FreeBSD about your time settings. If it is the only OS on your machine or it shares it with other Unix-like systems like Linux, go for UTC. If you also have Windows on the same computer, however, be sure to select No here. Windows and UTC is a mess.

Choosing the time setting

In the next screen you can choose which daemons should be started during the system boot process. Unselect dumpdev, it’s of no use to us (if you were able to read crash dumps and debug the applications with this info, you wouldn’t really read this post now, would you?). Keep sshd selected if you want it.

Selecting the daemons for autostart

Choose not to add any users right now. We’ll be doing this later and learn a bit about user management on FreeBSD! Next hit “Exit”, tell the installer that you don’t want to make any final changes.

Exit the installer

Choose to reboot. And that’s it.

Reboot to finish the installation

If you remove the installer image, your machine should boot into your new FreeBSD system. Welcome on board, new BSD user!

What’s next?

The next blog post will deal with some of the very basics of a fresh FreeBSD system.