Bacula on FreeBSD (pt. 4): Jobs, volumes, pools & a restore

This is part four of my Bacula tutorial. The first part covered some basics as well as installing Bacula and starting the three daemons that it consists of. Part two dealt with allowing all components of Bacula to interact with each other locally, debugging config problems and doing a first test backup. In part three the configuration was cleaned up and split into smaller parts, the first self-created resources (fileset, device and storage) were added and a backup job customized using the bconsole.

Part four will discuss jobs, and show how to do a restore. We will change the default settings for the backup job so that it’s no longer necessary to modify it using the bconsole. Also volumes, labels and pools will discussed.

The fourth part continues where the third one left off. During this tutorial series we use VirtualBox VMs managed by Vagrant to make things as simple as possible. If you don’t know how to use them, have a look here and here for a detailed introduction into using Vagrant and on how to prepare a VM template with FreeBSD that is used in the Bacula tutorial.


We’ve already seen and used backup jobs. There are also jobs for different actions like restore (and some others). But what is a job? It basically is your way of telling Bacula what to do and how to do it. The job type or action defines what Bacula should do in the first place. Back up data? Restore it? Verify (compare) data? The client determines who to operate on; it answers the question: Which host to back up data from or restore data to?

Then there’s the fileset which in case of a backup determines which files should be included and the pool that determines where to store the data. And finally there’s the schedule that determines when a job is run. So far we have only started jobs manually using the bconsole – but that’s certainly not what you have in mind for your backup solution (or maybe for simple backups it is. But in that case using Bacula for your backups is most likely overkill and you might want to look for a simpler backup utility)!

When we did our second backup, we changed a lot of settings using the bconsole. Now let’s modify the configuration instead so that those will be the new defaults for the backup job. Of course we’ll first have Vagrant spin up the VM, SSH into it and so on (you now the score by now):

% cd ~/vagrant/backuphost
% vagrant snapshot restore tut_3_end
% vagrant ssh
% sudo su -
# cd /usr/local/etc/bacula/

Then we can edit the job defaults (if you haven’t read the previous part(s) and wonder why you don’t have that file or even the directory – that’s because we’ve split the configuration for better readability!):

# vi includes/dir_job.conf

The topmost resource should be the JobDefs one that has the name DefaultJob. This is a kind of template for all jobs which only overwrite directives that differ from the default one and just use the rest as set here. Change FileSet to etc, Storage to File3 and Pool to Default. Save the file and exit the editor.

Now restart the director and prepare to run the backup job again using the bconsole:

# service bacula-dir restart
# bconsole
* run

Automatically selected Catalog: MyCatalog
Using Catalog “MyCatalog”
A job name must be specified.
The defined Job resources are:
1: Backuphost.local
2: BackupCatalog
3: RestoreFiles
Select Job resource (1-3):

Choose 1:

Run Backup job
JobName: Backuphost.local
Level: Incremental
Client: backuphost.local-fd
FileSet: etc
Pool: Default (From Job resource)
Storage: File3 (From Job resource)
When: 2016-09-23 23:48:01
Priority: 10
OK to run? (yes/mod/no):

That’s looking like it should. No more need to use mod multiple times! Type no now as we don’t actually need to do another backup at this time.

Restoring files

Instead we’ll be doing a restore next. Issuing the command to initiate a restore job leads to a long list of choices:

* restore
To select the JobIds, you have the following choices:
1: List last 20 Jobs run
2: List Jobs where a given File is saved
3: Enter list of comma separated JobIds to select
4: Enter SQL list command
5: Select the most recent backup for a client
6: Select backup for a client before a specified time
7: Enter a list of files to restore
8: Enter a list of files to restore before a specified time
9: Find the JobIds of the most recent backup for a client
10: Find the JobIds for a backup for a client before a specified time
11: Enter a list of directories to restore for found JobIds
12: Select full restore to a specified Job date
13: Cancel
Select item: (1-13):

Pick option 5 – the most frequent one that I use, BTW:

Defined Clients:
1: backuphost.local-fd
2: fbsd-template.local-fd
Select the Client (1-2):

Huh? Where’s that fbsd-template.local client coming from? Haven’t we removed it from the configuration completely? Yes, we have. However we did our very first backup when this was still the hostname of the virtual machine and the catalog remembers that it holds a backup for that client! Ignore that for now and select 1:

424 files inserted into the tree.

You are now entering file selection mode where you add (mark) and
remove (unmark) files to be restored. No files are initially added, unless
you used the “all” keyword on the command line.
Enter “done” to leave this mode.

cwd is: /

Notice that the prompt symbol changed ($)? You’re in a virtual shell now from which you can navigate through a filesystem rebuilt from the files contained in the selected backup. It is fairly limited, however. The most obvious limitation is that it does not provide auto-completion. You’ll have to live with that. And of course it does only provide a basic set of commands that allow you to change paths, list files, etc.

Bacula said that we’re in /. Let’s see what we have there:

$ ls

Ok, so obviously our filesystem consists of a subset of both /etc and /usr (subset because we’ve excluded /etc/caspar from /etc and only included /usr/local/etc and not the whole /usr, remember?).

Let’s see what is in /usr/local/etc, shall we:

ls /usr/local/etc

Nothing? Ouch. Does that mean that the backup is broken for whatever reason? No, in fact everything is fine. The problem here is that even this rather simple command is too advanced for Bacula! You want to see the contents of some directory? Go there and have a look again:

$ cd /usr/local/etc
cwd is: /usr/local/etc/

$ ls

There you go. It’s all there. Let’s mark the sudoers file so that it’ll be added to the restore job (you can also use the mark command, but add is shorter!):

$ add sudoers
1 file marked.

Ok, that worked. Just bear in mind that you always have to enter the directory first before you can mark (or view) any files. Even if you know where in the filesystem something is, Bacula can’t cope with anything more complicated than the very basic way of doing things.

Now let’s change to /etc:

$ cd /etc
cwd is: /etc/

I won’t show an ls here since that’d be too much output. But do it yourself and see if /etc/casper was really left out from the backup. Alright. Now let’s assume we want to restore csh.cshrc, csh.login and csh.logout as well. Thankfully Bacula’s virtual shell does support globbing (wildcard expansion):

$ add csh*
3 files marked.

After selecting a bunch of files, let’s tell Bacula that we’ve finished adding files:

$ done
4 files selected to be restored.

Run Restore job
JobName: RestoreFiles
Bootstrap: /var/db/bacula/backuphost.local-dir.restore.1.bsr
Where: /tmp/bacula-restores
Replace: Always
FileSet: Full Set
Backup Client: backuphost.local-fd
Restore Client: backuphost.local-fd
Storage: File3
When: 2016-09-25 08:02:20
Catalog: MyCatalog
Priority: 10
Plugin Options:
OK to run? (yes/mod/no):

Bacula has prepared a restore job and shows us a summary so we can either run, modify or cancel it. One thing to take note of is the Where: line. All files that are restored will have their path prefixed with /tmp/bacula-restores. You could choose another directory or set it to just / if you want Bacula to overwrite the current files in-place. For now accept the current settings by entering yes:

Job queued. JobId=3

Wait a moment and hit Enter to see if Bacula has any news for you. It should:

You have messages.

Let’s look at those:

* mes

You know the job report by now. Look for the following line that shows that everything went right:

Termination: Restore OK

The restore job completed successfully. There are some more useful commands that you can use when you select the files for the restore. I just want to mention two of them: unmark and lsmark. What the former does should be pretty obvious: It deselcts files that were marked for restore before. This allows you to e.g. add * and then unmark a few files which can be a much less painful way if you have more files that are to be restored than files that shouldn’t! The other one shows marked files in and below the current directory. That means if you want to see the full list of marked files, change to / before you use lsmark!

File examination

Let’s quit the bconsole now and take a look at the files that we just recovered from the backup:

* exit
# ls -1 /tmp/bacula-restores/etc/


Looks like something was restored. Since the original files have not actually been modified since we’ve backed them up, comparing the original and the restored ones should assure us of the files being intact:

# diff -q /usr/local/etc/sudoers /tmp/bacula-restores/usr/local/etc/sudoers

No output means that the files match exactly. Good! But where did those files get restored from? Remember what we did when we configured our backup device. Let’s take a look at the directory that we specified there:

# ls -lh /var/backup/
total 1952
-rw-r—– 1 bacula bacula 1.9M Sep 24 22:08 file3a

This is the volume that we specified in the configuration and that was actually created when we had Bacula label it.

For learning purposes our very simple setup (just one volume) worked great. But before we move on, it’s time to take care of creating a storage system that’s a little bit more advanced: We need a pool! But how do those work?

Volumes, pools and labels

Speaking of labels… In the previous part we had to create one before the job that we queued could actually start. To be able to come up with a sensible backup solution for your use case you will have to understand how Bacula stores backup data. It uses so-called volumes. Think of a volume as some kind of storage medium. This could either be tape or disk-backed storage (i.e. a file). Backup data can be written to a volume until the maximum capacity is reached. Additional data will have to be written to another volume.

We’re not really talking about using tapes here (which comes with its own set of problems from what I ‘ve read in Bacula’s manual). Still it makes sense to remember that tapes are the reason for some design choices of Bacula. Volumes are such a case. While supporting multiple files may not seem like a huge benefit (for one host that is), it’s easy to see that supporting more than one tape does. Once it’s full, write to the next. But to be able to distinguish them, Bacula needs some means of telling them apart. This is where the label comes in. A label basically means that some medium is marked as a volume that Bacula may use combined with a unique name so multiple volumes won’t get confused. So each volume needs a label before Bacula will use it to put data on.

If backup jobs were tied to a volume this could work for some cases but would probably lead to problems sooner or later. Imagine the case when a volume is probably only half full but nevertheless the next backup won’t fit on it. That backup would have to be written to the next volume, wasting the free space on the former. Issues and inflexibilities like that are solved by introducing pools. A pool is basically a list of volumes (plus some options). If your job targets a pool, it no longer matters which volume to put it on – Bacula can take care of that for you in a dynamic way. Pools also allow enforcing some restrictions (like maximum size, maximum time to use) on volumes depending on what your needs are and what you are trying to do.

Since this post is already long enough, it’s time to end this part. As always, let’s save our progress by shutting down the VM and taking the next snapshot (when the status has reached poweroff:

# shutdown -p now
% vagrant status
% vagrant snapshot save tut_4_end


After this part of the series we finally know how to restore files from a backup. We also have a better understanding of what jobs, volumes, labels and pools are.

In the next post we’ll create and test a new pool, do some configuration cleanup and reset the catalog. This should conclude the single node part of the tutorial.

Have any comments for me? Or did you perhaps find a mistake? Just leave me a comment.

Bacula on FreeBSD (pt. 3): Customizing configuration

This is the third part of my Bacula tutorial. The first part covered some basics as well as installing Bacula and starting the three daemons that it consists of. Part two dealt with modifying the default configuration files in a way that allowed all components of Bacula to interact with each other locally. And finally a test backup was done (without knowing details like what exactly would even be backed up!) just to ensure that communication between the daemons really works.

In this post we’ll clean up and split the configuration files to make them more comprehensible. We’ll create our own fileset, add a new device and assign that to a new storage resource. We’ll also customize a backup job on the bconsole to do a new backup using our self-created resources.

The third part continues where the second one left off. During this tutorial series we use VirtualBox VMs managed by Vagrant to make things as simple as possible. If you don’t know how to use them, have a look here and here for a detailed introduction into using Vagrant and on how to prepare a VM template with FreeBSD that is used in the Bacula tutorial.

Configuration cleanup

Let’s jump right into it! So restore the snapshot from the previous post, SSH into the machine, become root and change into Bacula’s configuration directory:

% cd ~/vagrant/backuphost
% vagrant snapshot restore tut_2_end
% vagrant ssh
% sudo su -
# cd /usr/local/etc/bacula/

Ok, here we are. Before we can do anything actually somewhat useful, we need to sort out things and get an idea of how the configuration works (what all those resources do, etc). Bacula’s default configuration is rather huge which doesn’t exactly make this an easy task. But there’s an easy solution to it: Let’s start by deleting each and every line that’s commented out in all of the configuration files!

Yes, some of these lines provide useful information but that’s not really lost:

# ls -1

As you can see, there are files like bacula-dir.conf.sample, bacula-fd.conf.sample, and so on. These are the unaltered default configuration files and you can always look at those as a reference. So before we move on: Edit the four config files that we’ve worked with so far (bacula-dir.conf, bacula-fd.conf, bacula-sd.conf and bconsole.conf) and get rid of all the commented out lines.

Removing these lines (and a few empty ones that were between them and didn’t have any purpose on their own) shrinks the configuration down from over 600 lines to less than 300. While that’s much better, it’s still a lot. Let’s throw more stuff away, shall we? There’s a “restricted director” that’s used for monitoring. We’re not going to use that here so all those resources with the name of fbsd-template.local-mon can be deleted from all three daemon’s config files. Do that now.

At this point restart all three daemons (and check if they are running afterwards) to make sure the configuration is still valid:

# service bacula-dir restart
# service bacula-fd restart
# service bacula-sd restart

The files bconsole.conf and bacula-fd.conf are trivial now with less than 20 lines each. Still bacula-sd.conf with about 80 lines is barely acceptable and especially bacula-dir.conf with over 170 remains too big for my taste!

Configuration splitting

Of course we cannot simply go rampage and delete more and more resources from the director configuration. So how to make it smaller (while maintaining the functionality)? Bacula supports breaking up larger configuration files. To do so we put some resources into their own file and include that in the main config file! Bacula doesn’t care in which files you stuff its resources and it also doesn’t really care about the order. But we do that to make things easier to maintain and so it makes sense to think about what resources should be grouped together.

To keep things organized, it’s best to create a separate directory for includes (or even multiple directories if you have complex configuration and need to separate things further). Choose a name that fits your taste:

# mkdir includes

Here’s my suggestion (you might want to do this completely differently – just do what makes sense for you) for splitting bacula-dir.conf:

  • Move all JobDefs and Job resources into includes/dir_job.conf
  • Move all FileSet resources into includes/dir_fileset.conf
  • Move all Schedule resources into includes/dir_schedule.conf
  • Move all Storage resources into includes/dir_storage.conf
  • Move all Pool resources into includes/dir_pool.conf

Now let’s include those configuration files that we split off! We could do so by including them one by one by adding the following lines at the end of bacula-dir.conf like this:


That would work (just make sure you use absolute paths to save you some potential headaches). But don’t put it in there (or remove it again if you already have)! There’s a better way of doing it; just add the one following line instead:

@|"sh -c 'for f in /usr/local/etc/bacula/includes/dir_*.conf ; do echo @${f} ; done'"

This will use the shell to automatically include all files in /usr/local/etc/bacula/includes/ that start with dir_ and end in .conf. Much better, eh? And it has the advantage of automatically pulling in includes/dir_messages.conf or something like that should you ever decide to split e.g. the Messages resources from the main dir config file as well.

Now we have a bunch of files with nicely separated resource types and the biggest ones are around 40 lines long. Much more comprehensible if you ask me. Let’s quickly split up the sd config as well by just putting all the Device resources into include/sd_device.conf and including it adding the following line into bacula-sd.conf:

@|"sh -c 'for f in /usr/local/etc/bacula/includes/sd_*.conf ; do echo @${f} ; done'"

Restart dir and sd again to see if the configuration is still ok:

# service bacula-dir restart
# service bacula-sd restart


We’ve given our base box a hostname of “fbsd-template.local”. That’s fine for a template – but this machine is not a template anymore. We’ll spin up more VMs from the same base box (template) later so it makes sense to do some proper naming. We could just change the hostname and Bacula would not care at all here since a resource’s name is just a name. But for human operators things can get pretty confusing pretty fast – and that’s what we wanted to avoid, right? So let’s edit the four config files now and change the Name of the various resources from e.g. fbsd-template.local-dir to backuphost.local-dir. Once you’ve replaced it nine times in the actual config files, you’re done here.

But we must not forget to also change it twice in includes/dir_job.conf or the director won’s start since the jobs refer to the old name! And while we’re at it, let’s also change the first Job resource’s name in the same file from BackupClient1 to Backuphost.local. Save the file and exit the editor.

Now let’s change the hostname that FreeBSD will assign itself during startup. To do so we need to edit rc.conf:

# vi /etc/rc.conf

Change the corresponding line so that it holds the new hostname:


Now is an excellent time to take another snapshot (so you can restore that if something goes wrong and don’t have to redo all the config splitting, etc.) and since we’ve changed the hostname we need to reboot anyways. So let’s shutdown the VM, snapshot and start the VM again so that it comes back up with the new hostname:

# shutdown -p now
% vagrant status
% vagrant snapshot save tut_3_mid
% vagrant up

Let’s SSH back into it, switch to root and see if the hostname changed:

% vagrant ssh
% sudo su -
# hostname


That looks good. Also all Bacula services should be running (feel free to double-check) so we can move on.

What to back up: Filesets

Let’s talk about FileSets next. We’ve put definitions of this kind of resources into their own configuration file that’s then included into the dir configuration. Now we want to add a new fileset. We do so by editing:

# cd /usr/local/etc/bacula
# vi includes/dir_fileset.conf

Just add the following lines to it:

FileSet {
Name = "etc"
Include {
Options {
signature = MD5
File = /etc
File = /usr/local/etc
Exclude {
File = /etc/casper

What does this do? It defines another fileset named “etc” which includes the directory /etc. By default Bacula will recursively include files and subdirectories when it is told to back up a directory. So this fileset will effectively backup almost all of the system’s configuration files (which reside in /etc and /usr/local/etc).

Let’s assume we don’t use Capsicum (FreeBSD’s sandboxing framework in case you’ve never heard of it) and don’t want to back up configuration for the Casper daemon (which is a support daemon for Capsicum). The above example excludes /etc/casper from being backed up.

And then there’s the Signature directive. It tells Bacula to create a signature for each file it backs up using the MD5 algorithm. While it is not necessary it is strongly recommended to let Bacula calculate signatures (checksums). Bacula uses these to find out if a file has been altered since it was last backed up even if e.g. the file size didn’t change. Even though it takes a little more space if you use signatures, doing so totally makes sense.

Save the file and exit. You don’t have to restart the director, yet, because we’re going to make one more change.

Where to back up to: Storage

Let’s say we want to store our backups in /var/backup. To do this, we first create the directory and make sure Bacula can access it:

# mkdir /var/backup
# chown bacula:bacula /var/backup

Now we have to inform the sd of the new “device”. Edit the configuration:

# vi includes/sd_device.conf

and add the following lines:

Device {
Name = Var-File
Media Type = File3
Archive Device = /var/backup
LabelMedia = yes;
Random Access = Yes;
AutomaticMount = yes;
RemovableMedia = no;
AlwaysOpen = no;
Maximum Concurrent Jobs = 5

Here we create a new “device” for Bacula to use – this “device” is in our case just a directory (/var/backup) on the disk. It is important that you use unique media types; since there’s already File1 and File2 in the default configuration, we go with File3 here. In fact you could choose any unique name that makes some sense to you. Ignore the other settings (or look them up in the Bacula documentation if you want to know more). Save the file and exit. However you won’t be able to use it, yet, as the dir does not know anything about it. So we need to edit its configuration as well:

vi includes/dir_storage.conf

and add the following lines:

Storage {
Name = File3
Address = localhost
SDPort = 9103
Password = "sdPASSWORD"
Device = Var-File
Media Type = File3
Maximum Concurrent Jobs = 10

This creates a new storage that can hold backups. It is defined for the director who will send it to the fd if it initiates a backup. Save and exit. Now we need to restart dir and sd to load the new configuration:

# service bacula-dir restart
# service bacula-sd restart

Backup time!

We’ve made a lot of changes to the configuration so let’s attempt to do another backup:

# bconsole
* run

Automatically selected Catalog: MyCatalog
Using Catalog “MyCatalog”
A job name must be specified.
The defined Job resources are:
1: Backuphost.local
2: BackupCatalog
3: RestoreFiles
Select Job resource (1-3):

Select the first one:

Run Backup job
JobName: Backuphost.local
Level: Incremental
Client: backuphost.local-fd
FileSet: Full Set
Pool: File (From Job resource)
Storage: File1 (From Job resource)
When: 2016-09-23 18:36:24
Priority: 10
OK to run? (yes/mod/no):

Choose mod this time:

Parameters to modify:
1: Level
2: Storage
3: Job
4: FileSet
5: Client
6: When
7: Priority
8: Pool
9: Plugin Options
Select parameter to modify (1-9):

Select 4:

The defined FileSet resources are:
1: Full Set
2: Catalog
3: etc
Select FileSet resource (1-3):

Of course we want to use 3, then mod and then 2:

The defined Storage resources are:
1: File1
2: File2
3: File3
Select Storage resource (1-3):

Choose 3, then mod and then 8:

The defined Pool resources are:
1: Default
2: File
3: Scratch
Select Pool resource (1-3):

Go with the first one:

Run Backup job
JobName: Backuphost.local
Level: Incremental
Client: backuphost.local-fd
FileSet: etc
Pool: Default (From User input)
Storage: File3 (From user selection)
When: 2016-09-23 18:38:24
Priority: 10
OK to run? (yes/mod/no):

Now type in yes:

Job queued. JobId=2
You have messages.

Let’s see how that went:

* mes
23-Sep 18:41 backuphost.local-dir JobId 2: Using Device “Var-File” to write.
23-Sep 18:41 backuphost.local-sd JobId 2: Job Backuphost.local.2016-09-23_18.41.29_03 is waiting. Cannot find any appendable volumes.
Please use the “label” command to create a new Volume for:
Storage: “Var-File” (/var/backup)
Pool: Default
Media type: File3

Oh my! What’s happening now again? We need to create a label first before Bacula can use the new storage. Let’s just do that now and talk about the detail later (there’s a bit more to storage actually):

* label
The defined Storage resources are:
1: File1
2: File2
3: File3
Select Storage resource (1-3):

Select target 3 of course. Bacula will then ask you to give that volume a name:

Enter new Volume name:

Let’s just call it file3a:

Defined Pools:
1: Default
2: File
3: Scratch
Select the Pool (1-3):

Select 1:

Connecting to Storage daemon File3 at localhost:9103 …
Sending label command for Volume “file3a” Slot 0 …
3000 OK label. VolBytes=212 VolABytes=0 VolType=1 Volume=”file3a” Device=”Var-File” (/var/backup)
Catalog record for Volume “file3a”, Slot 0 successfully created.
Requesting to mount Var-File …
3001 OK mount requested. Device=”Var-File” (/var/backup)

Seems like everything is ok. Wait a few seconds and see if the job that has been waiting in line has been run automatically:

* mes
[ … quite some output … ]

Again look especially for the most important line. If you can find this one:

Termination: Backup OK

everything has worked out.

Let’s save our progress by shutting down the VM and taking the next snapshot:

* exit
# shutdown -p now
% vagrant status
% vagrant snapshot save tut_3_end


Alright! We just did our second backup with Bacula and this time we actually even know what we did there. While it’s not perfect to modify the backup job like this we’re certainly making progress (and it makes sense to have done it like that at least once). We’ll take care of the job defaults, discuss labels, volumes, etc. next time and also do a restore. But for now take a break. You’ve earned it.

Got any comments or figured that there’s something wrong with what I wrote? Just drop me a line in the comments.

Bacula on FreeBSD (pt. 2): Bconsole – ruling the Director

This is the second part of my hands-on Bacula tutorial. The first part introduced Bacula for people without any previous knowledge, discussed some basics and covered getting all three daemons up and running. This post will explain how to use bconsole to connect to the director, ensure that the director can connect to the other two daemons and show how to do a simple backup. Plus we’ll create an invalid configuration on purpose and debug the issue.

The second part continues where the first one left off. During this tutorial series we use VirtualBox VMs managed by Vagrant to make things as simple as possible. If you don’t know how to use them, have a look here and here for a detailed introduction into using Vagrant and on how to prepare a VM template with FreeBSD that is used throughout all parts of this Bacula tutorial.

Excursion: Vagrant snapshots

We have to start the VM and SSH into it so we can continue setting up Bacula. While it’s not actually required if you didn’t do anything else with the VM in the meantime, it doesn’t hurt to just restore the snapshot that we took at the end of the previous part. But what are snapshots anyway?

A snapshot is a recording of the state of a VM at a specific point in time. You can create a snapshot of any VM; that VM may be running or powered off when the snapshot is created. Snapshots basically hold filesystem contents and (in case of a running VM) system memory, etc.

Snapshots allow you to return to the state that a VM had at the moment that the snapshot was created. If you deleted some files after making the snapshot, returning to the previous state means that the files are back again. At the same time any files that were created after the snapshot are gone and e.g. configuration files will lose any changes made to them. If you created a snapshot of a running VM, returning to the previous state also means that any processes that you stopped after sharpshooting will be running again and so on.

Snapshots work by recording the differences between the state that they captured and the current one. That way they do not double the space that the VM needs (this is what a full clone does, BTW.). The more the states differ however, the more space the snapshot will require. You can keep multiple snapshots for a VM and change back and forth between them. Just keep in mind that they can eat up free space quite fast and are not tied to the disk size quota of the VM itself!

To list all available snapshots (for the particular VM) use the following command:

% vagrant snapshot list

If you want to delete an old snapshot use:

% vagrant snapshot delete SNAPSHOTNAME

Now restore the snapshot from the previous post, SSH into the machine and switch to root:

% cd ~/vagrant/backuphost
% vagrant snapshot restore tut_1_end
% vagrant ssh
% sudo su -

Connecting to the director

We left off discussing configuration – and once it’s configured properly, Bacula is meant to run on its own with little to no manual intervention. However we’re nowhere near a proper configured Bacula, yet! So the first thing to do is to ensure that manual intervention is possible at all.

We control the dir (which in turn controls both the sd and the fd) using the Bacula console or bconsole. There are other programs available but bconsole is a text mode utility that comes with Bacula and will be our default way of interacting with the daemons. Let’s try out what happens if we invoke the console:

# bconsole
Director authorization problem.
Most likely the passwords do not agree.


We can’t connect. Bacula uses passwords to prove that one component of it is allowed to connect to another. These passwords are randomly generated when Bacula is installed and right now the director has a password set that’s different from the one in bconsole’s configuration. So let’s edit the director’s config file now:

# cd /usr/local/etc/bacula/
# vi bacula-dir.conf

Look for the Director resource with the name fbsd-template.local-dir. Change the Password directive to Password = "dirPASSWORD now. Save the file and exit the editor. And yes, the missing double quote is intentional; you’ll probably ruin your config multiple times as you follow along this series (or try to figure out things yourself) so it makes sense to show you right now how to debug configuration issues!

Edit the bconsole config file next:

# vi bconsole.conf

It should only have the Director resource with the name fbsd-template.local-dir. Change the Password directive there to the same password (however with the ending double quote!). Save and exit.

The director daemon needs to be restarted to pick up the configuration changes. Issue to following command to do so:

# service bacula-dir restart
# service bacula-dir status

bacula_dir is not running.

Our config is broken and so the director won’t start. BTW: Form a habit of checking that the daemon actually runs after making config changes and restarting it! I won’t explicitly mention it from here on. Just get used to always doing it yourself.

Debugging config errors

Let’s assume for a moment that we don’t know of the invalid config file. How do we learn about it? The director won’t start so let’s take a look at the log, shall we?

# cat /var/log/bacula.log
[…Just some old, unrelated messages…]

Hm! There’s nothing new in there?! Only message indicating the failed attempt of bconsole to connect to the dir and even older stuff? Yes, that’s true. Unfortunately the director seems to open the log file and to post messages into it only after it processed the configuration file. And if it runs into a parser error while doing so, it will terminate. So what do we do now?

In such cases it’s a good idea to start the director daemon manually in verbose and debug mode to test the configuration:

# bacula-dir -v -d 1 -c /usr/local/etc/bacula/bacula-dir.conf
bacula-dir: ERROR TERMINATION at parse_conf.c:465
Config error: Could not find config Resource DefaultJob referenced on line 50 : JobDefs = “DefaultJob”
: line 50, col 24 of file /usr/local/etc/bacula/bacula-dir.conf
JobDefs = “DefaultJob”

There we have it: It’s a config error in line 50. Hit CTRL+C to quit the process and take a look at the config file. But what’s that? Line 50 is a perfectly fine directive! Why does the director give that error? Read again and read carefully. The problem is NOT line 50. The problem is that this line references a resource named DefaultJob that is not present.

Search for it in the config file and you’ll find it (beginning on line 29). And worse: it’s perfectly fine as well! Strange, isn’t it? Well, yes and no. From a human perspective: Certainly. But look at the lines above and you’ll see line 25 with the missing double quote. That means that Bacula treats all following characters and lines until it finally finds the next double quote as part of the Password directive… That includes the resource type JobDefs and its name DefaultJob!

That’s why Bacula rightfully claims that a resource with that name wasn’t defined anywhere in the configuration file. And that’s just one “nice” example of how complicated it sometimes is to repair things once they start going sideways!

Now let’s fix the missing double quote in bacula-dir.conf real quick, restart the daemon and continue on:

# vi bacula-dir.conf
# service bacula-dir start


So can we finally connect now?

# bconsole
Connecting to Director localhost:9101
1000 OK: 102 fbsd-template.local-dir Version: 7.4.2 (06 June 2016)

Excellent! We’re eventually connected to the director. The asterisk (*) is the prompt symbol. We can now use the console to issue commands to the director. Let’s try one:

* status dir
fbsd-template.local-dir Version: 7.4.2 (06 June 2016) amd64-portbld-freebsd11.0 freebsd 11.0-RC3
Daemon started 20-Sep-16 20:26, conf reloaded 20-Sep-2016 20:26:01


OK, there’s some output about the director itself, about jobs and so on. The director seems to be alright. Now for the file daemon:

* status client
Automatically selected Client: fbsd-template.local-fd
Connecting to Client fbsd-template.local-fd at localhost:9102
Failed to connect to Client fbsd-template.local-fd.
You have messages.

So connecting to the file daemon failed! And we have messages. Let’s look at the latter first:

* mes
20-Sep 20:33 fbsd-template.local-dir JobId 0: Fatal error: Unable to authenticate with File daemon at “localhost:9102”. Possible causes:
Passwords or names not the same or
Maximum Concurrent Jobs exceeded on the FD or
FD networking messed up (restart daemon).

The command mes is just short for messages which would also work and do the same. The error here is that the connection was refused – we need to check passwords again. But before doing so, we need to quit bconsole first:

* exit

File daemon configuration

Time to edit the fd configuration file:

# vi bacula-fd.conf

Find the Director resource with the name fbsd-template.local-dir and change the password to "fdPASSWORD". Save and exit. Then edit the dir configuration again:

# vi bacula-dir.conf

Look for the Client resource with the name of fbsd-template.local-fd and change the password to "fdPASSWORD" as well. Save the file and exit the editor. Now restart both the fd and the dir:

# service bacula-fd restart
# service bacula-dir restart

Let’s see if that fixed the problem:

# bconsole
* status client

Automatically selected Client: fbsd-template.local-fd
Connecting to Client fbsd-template.local-fd at localhost:9102

fbsd-template.local-fd Version: 7.4.2 (06 June 2016) amd64-portbld-freebsd11.0 freebsd 11.0-RC3
Daemon started 20-Sep-16 20:51. Jobs: run=0 running=0.

All looking good! Now only the sd remains. Let’s give that a password that fits the scheme, too.

Storage daemon configuration

Now for the sd config:

* exit
# vi bacula-sd.conf

Go to the Director resource with the name fbsd-template.local-dir and set the password to "sdPASSWORD" this time. Save changes and exit. Next is the dir config once again:

# vi bacula-dir.conf

Find the Storage resources (named File1 and File2) and set their password to "sdPASSWORD" (and ignore that the configuration file says that you should not use localhost for sd while it configures it to use localhost by default…). Save and exit.

Now restart the sd and the dir and try to connect to the sd using bconsole:

# service bacula-sd restart
# service bacula-dir restart
# bconsole
* status sd

Automatically selected Storage: File1
Connecting to Storage daemon File1 at localhost:9103

fbsd-template.local-sd Version: 7.4.2 (06 June 2016) amd64-portbld-freebsd11.0 freebsd 11.0-RC3
Daemon started 20-Sep-16 21:09. Jobs: run=0, running=0.

Great, now bconsole can connect to the dir and the dir to both the fd and the sd!

Doing a backup

Before finally doing a first test backup, we’re going to edit the director config once more to tweak messaging a bit:

* exit
# vi bacula-dir.conf

Skip down to the Messages resource named Standard and find the following four lines:

mailcommand = "/usr/local/sbin/bsmtp -h localhost -f \"\(Bacula\) \\" -s
\"Bacula: %t %e of %c %l\" %r"
operatorcommand = "/usr/local/sbin/bsmtp -h localhost -f \"\(Bacula\) \\"
-s \"Bacula: Intervention needed for %j\" %r"
mail = root@localhost = all, !skipped
operator = root@localhost = mount

By default, Bacula sends mails if errors occur. We haven’t configured mailing, so this would only result in more errors… Since we’re not building a production environment here, let’s just get rid of this and make life a bit easier while exploring the essential functionality of Bacula. Comment out all four. Then save the file, exit and restart the director once again:

# service bacula-dir restart

Now issue the following commands:

# bconsole
* run

Automatically selected Catalog: MyCatalog
Using Catalog “MyCatalog”
A job name must be specified.
The defined Job resources are:
1: BackupClient1
2: BackupCatalog
3: RestoreFiles
Select Job resource (1-3):

Pick number 1.

Run Backup job
JobName: BackupClient1
Level: Incremental
Client: fbsd-template.local-fd
FileSet: Full Set
Pool: File (From Job resource)
Storage: File1 (From Job resource)
When: 2016-09-20 21:24:13
Priority: 10
OK to run? (yes/mod/no):

Answer “yes”.

Job queued. JobId=1
You have messages.

Ask the director to display the messages:

* mes

You should get a lot of output. If you only got a few lines, you probably were too fast and caught Bacula while it was still working. Give it a second and type mes again. The important line to look for is this one near the end:

Termination: Backup OK

Take a look at all the lines and get a rough idea what has happened here. Do so especially if at this point something bad happened (i.e. the backup failed). In this case check that you did everything as I wrote it here and if that doesn’t help, you’re unfortunately in for some research… But I’ll assume that it worked well and you just did your first Bacula backup. Good work! This means that you’ve also successfully ensured that all daemons can interact with each other.

That’s it for this part of the tutorial. Power down the VM and save another snapshot (when the status has reached poweroff):

* exit
# shutdown -p now
% vagrant status
% vagrant snapshot save tut_2_end


We’re still in the early stages of setting up Bacula but at least this time we were able to actually do something by backing up some data. You probably have no idea what you’ve just backed up or where to – but that’s ok for now. We’ll discuss these things in the next part. And in addition to making all daemons able to interact and doing a backup, you’ve done some debugging work. So I think having a break here and talking about backups in more detail later (I also want to cover configuration some more before I do so) makes sense.

I hope that this has been a useful read. Feel free to leave me a comment if you like it (or if you don’t) and of course tell me if I made some mistakes that should be corrected!

Bacula on FreeBSD (pt. 1): Introduction – Bacula backup basics

Looking for a practical tutorial to get into Bacula hands-on? You’ve come to the right place. So this post is going to cover the three “B”: Bacula Backup Basics, and it will do so on a fourth “B” – (Free)BSD! Read “basics” auds “does not require previous knowledge”. However it’s Bacula basics. Even though the project’s manual officially discourages (!) using Bacula for people unfamiliar with “sophisticated backup tools”, don’t be afraid of it. The reason for the statement in the manual is that Bacula requires quite a bit of configuration work before it can do anything useful. Due to its modularity it’s also a bit harder to understand compared to simple backup tools.

All of that is true. That’s why we won’t be doing backups in this first part. Instead I’ll discuss some basics here that you definitely have to know. We’ll also install Bacula and get all of its parts running. That’s in fact enough for a single post.

While this post does not expect you to know anything about Bacula, you should have a general understanding of backup fundamentals. Simple test: If you know the terms „incremental backup“ and „differential backup“ and you know what both mean, chances are good that you’ll be fine. If you don’t know them, please do a bit of research first and come back later.

What is Bacula?

Bacula is an Open Source backup tool with a lot of advanced features (which I don’t claim to know much about). Or more precisely: Bacula is a set of tools to perform backups. As I touched upon above, Bacula is modular and it is crucial to know its parts:

  • bacula-dir
  • bacula-sd
  • bacula-fd
  • bconsole
  • catalog

First we have bacula-dir, the Bacula director daemon. Just like the name suggests, this is the central instance that supervises and coordinates backup work.

Then there’s bacula-sd, the Bacula storage daemon. The name also kind of implies what it does: It is responsible for storing the backup data that it receives or for reading and sending it in case of a restore.

The third daemon, bacula-fd, is the Bacula file daemon. I’d prefer to call it “client daemon” as that would in my opinion fit it better, but that’s not what it is called. What does it do? It reads files from a backup client and sends them to the storage daemon (sd). Or, in case of a restore, the file daemon (fd) receives data from the sd and writes it back to the disk.

Not a daemon but rather a small administration utility is bconsole. It provides a way of connecting to the director daemon (dir) and interact with it. You need it for maintenance, to debug problems or to test new configuration.

The catalog is where Bacula stores various information about backups (NOT the backup data itself!). We’re just starting with Bacula, so let’s keep things simple. All you have to know for now is that Bacula needs a catalog and that it is kept in a database (MySQL, PostgreSQL and SQLite are supported).

Ok, so much for Bacula’s structure. Next station: The practical part, Installing and getting it to work! But one more thing first: Whenever something about Bacula doesn’t seem to make sense to you, here’s a clue for you. Bacula was designed with tape backups in mind. Recalling this fact often helped me understanding why something in Bacula is the way that it is and not built differently!


My previous post showed how to prepare a Vagrant base box for playing with Bacula. I’ll assume that you are either using that or have a FreeBSD system available that’s configured likewise. You can ignore the Vagrant stuff in that case. I do recommend however that you use Vagrant. For now we’ll stick with a simple setup of one node, but once we add more nodes to our setup in later parts of this series, Vagrant comes in really, really handy. If you don’t know the tool, I’ve written another post that introduces Vagrant + VirtualBox for beginners.

Let’s create a directory for the VM “backuphost” first and populate it with a default Vagrantfile:

% cd ~/vagrant
% mkdir backuphost
% cd backuphost
% vagrant init

Edit the Vagrantfile, delete its content and put the following into it:

Vagrant.configure("2") do |config| = "fbsd-template" "private_network", type: "dhcp"
config.vm.synced_folder ".", "/vagrant", disabled: true

The network setting (creating a private network) is not needed right now but we’re setting it up already so that it can simply be used when we need it later. Now fire up the VM and SSH into it:

% vagrant up
% vagrant ssh

We’ve already prepared things during the base box creation. Now become root and install bacula-server:

% sudo su -
# cd /usr/ports/sysutils/bacula-server/
# make install clean

FileDaemon and StorageDaemon

Now that Bacula is installed, let’s see if we can get the various daemons to run. If you know what they are called, that’s usually not too hard. We’ll start with launching the fd:

# sysrc bacula_fd_enable="YES"
# service bacula-fd start

Starting bacula_fd.

Fair enough. But has it started successfully? Let’s check:

# service bacula-fd status
bacula_fd is running as pid 14003.

It has, everything’s looking fine here. Ignore the PID, BTW, it will most likely be different for you. Time to enable, start and check the sd next:

# sysrc bacula_sd_enable="YES"
# service bacula-sd start

Starting bacula_sd.
# service bacula-sd status
bacula_sd is running as pid 14056.

Again: Nothing unexpected happening here. The default configuration that comes with Bacula on FreeBSD is sufficient for both daemons to be happy with and start up. Now for the remaining daemon, the director… So far things were downright boring. But now they get a bit more interesting (you decide if that’s a good thing. Most administrators hate it when software behaves in interesting ways).

Bringing up the director

Let’s start the director daemon after enabling it for system start and see if it’s running:

# sysrc bacula_dir_enable="YES"
# service bacula-dir start

Starting bacula_dir.
# service bacula-dir status
bacula_dir is not running.

It isn’t! What could be wrong here? What do you do if you have no clue? Take a look at the system log of course:

# tail /var/log/messages | grep bacula-dir
Sep 20 19:00:00 fbsd-template bacula-dir: 20-Sep 19:00 Message delivery ERROR: fopen /var/log/bacula.log failed: ERR=Permission denied

Aha! The director was not allowed to open its log file. Probably because there is no log file! So let’s create it and give it to the bacula user so that it has the permissions needed. Then we’ll try to start the director again:

# touch /var/log/bacula.log
# chown bacula:bacula /var/log/bacula.log
# service bacula-dir start

Starting bacula_dir.
# service bacula-dir status
bacula_dir is not running.

Too bad! But we should have a log now. Let’s take a look at that:

# tail /var/log/bacula.log
20-Sep 19:11 bacula-dir JobId 0: Fatal error: Could not open Catalog “MyCatalog”, database “bacula”.
20-Sep 19:11 bacula-dir JobId 0: Fatal error: sqlite.c:199 Database /var/db/bacula/bacula.db does not exist, please create it.

Right, we forgot the catalog! I told you to configure it for SQLite when we built the base box so that’s what I’m going to assume here. If you chose MySQL or PostgreSQL, refer to the manual or search the net (PostrgeSQL is the default backend on FreeBSD; so if you installed bacula-server from packages, you’re expected to use that).

Bacula provides scripts for each database backend to created the tables needed. Let’s create the catalog and see if the director can start afterwards:

# su -m bacula
% /usr/local/share/bacula/make_sqlite3_tables
% exit
# service bacula-dir start

Starting bacula_dir.
# service bacula-dir status
bacula_dir is running as pid 14069.

All the daemons are up and running and you know a few essential things about Bacula. However we cannot do anything with it just yet. We need to dig into Bacula’s configuration before we can use the bconsole to interact with the director – which can then interact with the other two daemons.

Bacula configuration files

You have to know a few things before making changes to the default configuration makes sense. As these can be considered “basics”, too, I’ll cover them here.

On FreeBSD, Bacula’s configuration files live in /usr/local/etc/bacula/. Bacula’s configuration is divided into several configuration files – one for each daemon and one for the bconsole (as well as another one that we’ll ignore): bacula-dir.conf, bacula-sd.conf, bacula-fd.conf and bconsole.conf.

There are various passwords in the config files and those can be a little confusing at first. This is why I’ll be using simple passwords that hint which daemon they are actually belonging to. As long as you are just testing Bacula, I recommend you do the same. It’s hard enough to work through Bacula’s configuration and the more clear, easy to understand bits you’ve got in there, the better.

All those files have a common structure. They are composed of definitions which assign a value (or values) to a variable. Definitions which belong together are in the same group. Bacula calls these definitions directives and the groups are known as resources. Since there can be multiple resources of the same type, each one must have a unique name so Bacula can tell them apart. When it comes to resource types, think of them as the context which the resource is meant for (e.g. Director, FileDaemon, Messages, etc.)

The structure of a resource always looks like this:

Type {
directive A
directive B
directive C

In configuration files lines starting with a “#” are commented out. If you see a comment sign later in the line that means that anything from there is ignored as a comment while the part before it isn’t.

Take a look at bconsole.conf and bacula-fd.conf (these are the shortest ones) and get a first impression of what Bacula configuration looks like. Think about some of the resources and guess what they maybe do. Don’t look at bacula-sd.conf or bacula-dir.conf if you’re not looking for some more confusion. If things seem crystal clear for you however, of course feel free to go ahead. It’s also OK if you feel a little overwhelmed after reading the files. We’ll be tearing them apart together in a later part.

Saving the progress

IMPORTANT: DON’T destroy your VM if you intend to follow the other parts, too! We’re not starting fresh each time but instead build upon what we’ve done so far.

Maybe you want to play with Bacula on your own in the meantime. That would however complicate things a lot since the configuration would probably no longer match what I take for granted in the next post. But there’s a simple solution to this: Create a snapshot of your VM now! As long as you don’t remove that snapshot, feel free to do inside the VM as you please.

Shutdown the VM properly first and then use Vagrant to create a snapshot; but before you do, make sure that the VM has reached the poweroff state (if it hasn’t, wait a moment!):

# shutdown -p now
% vagrant status
% vagrant snapshot save tut_1_end


That’s it for now. I originally wanted to stuff more into this first post and at least have you do your first backup. But that would have meant to either make this post much longer (which would probably scare some people away) or to skip over some details and have you figure them out yourself (which kind of defeats the idea of writing such a “practical introduction” in the first place).

And since Bacula is a complex topic, I also think that this may be enough information for somebody who’s completely new to it (and perhaps to Vagrant as well). If you feel like this could have been longer you can of course just continue with part 2 whenever you want.

Oh, and if you notice any mistakes or anything, please tell me. I’m by no means a Bacula expert and chances are of course that I’m doing things horribly wrong (it has worked for me so far, though!).