Backing up PostgreSQL with Holland

 Linux Guides  Comments Off
Feb 022011
 

Introduction

Backups are a subject I return to semi-frequently with a passion to never be in an “oh shit” scenario.  Last time I built my backup system, bacula with a postgresql DB backend, I determined that I would move to a common database backup script for all of my databases.  Holland fit this bill perfectly with support for postgres, sqlite and mysql.  This allows one command to backup all of my databases on all of my servers and subsequently creates a much simpler bacula configuration (the database job is defined the same as the catalog job).

The Solution

The problem I had when configuring holland to backup postgresql is that there was no example configuration file.  It wasn’t hard to craft a working default postgres configuration and the following is what I came up with (/etc/holland/backupsets/default.conf:

[holland:backup]
plugin = pgdump
backups-to-keep = 1
auto-purge-failures = yes
purge-policy = after-backup
estimated-size-factor = 1.0

[pgdump]
role = postgres

[pgauth]
username = postgres

Conclusion

Setting up holland to backup databases is incredibly easy and flexible.  By having a common backup solution for all databases other configurations become easier and processes can be streamlined.

 Posted by Alex Brandt at 7:10 PM  Tagged with: , , , , , ,

Holland on Gentoo

 Linux Guides  Comments Off
Aug 012010
 

Introduction

There is a new king of backups in town, holland. This little framework written in Python allows one to easily backup anything that might need to be converted to a more flat file style before being backed up. Right now there is support for mysql, sqlite, and postgresql but with a little finesse it could potentially support directories as well as databases. This would make not only mysql backups a breeze but LDAP as well.

Progress Update

I have added a preliminary set of ebuilds to my overlay (which could use some code review if anyone is interested) that allows holland to easily be installed on Gentoo systems. So easy in fact that all it takes is `emerge holland`.

It accepts a set of use flags to bring in the “providers” you want to be able to backup for and makes sure that those packages are installed on the system.

Examples

The holland ebuilds have three providers right now:

  • mysql
  • postgresql
  • sqlite

You can install any of these three you want in any combination; it doesn’t care. It will default to installing the mysql but can easily be told not to by placing -mysql in the use flags for holland. Diego Pettenò — Flameeyes mentioned to me that in EAPI 4 we’ll get the cool option of being able to specify one of a set of use flags must be set without forcing the choice but until then we have this slick solution.

There is also lvm support for snapshotting off the database directory before grabbing the database and a myriad of other features I haven’t had a chance to explore yet.

To perform a rudimentary backup after installing holland simply run `holland bk`. This will read the configurations in `/etc/holland` and backup the databases it finds.

Conclusion

The new kid on the block, holland, will make backups of complex databases and directories a breeze. Simply change that cronjob from using mysqldump to calling holland and you’re finished.

 Posted by Alex Brandt at 1:32 PM  Tagged with: , , , , , , , , , , ,

Bacula Update Database Woes

 Linux Guides  Comments Off
Jun 272010
 

Introduction

Backups, a subject that is always talked about but rarely put into practice. If you do happen to have a backup solution, excellent, you probably sleep better than our comrades without a recovery plan. I’ve been using bacula for my backup solution for over a year now and one thing I’ve never had enough experience with has been upgrading bacula. The upgrade process can be pretty hairy depending your distribution, level of investment, etc.

Bacula Updates

Recently bacula-5.0.2-r1 was marked as stable for the Gentoo distribution. Upon finishing the emerge (laugh if you like at my compilation of all packages), I attempted the restart of all bacula services to bring the new version live:

/etc/init.d/bacula-sd restart
/etc/init.d/bacula-fd restart
/etc/init.d/bacula-dir restart

All went well until I did the restart on the director (bacula-dir). At that point things took a nasty turn for the worse. The director didn’t want to start and there were no messages on the screen indicating why this might be the case.

Long story short: the database needed some schema changes to be applied before the director could start up and get running again. In order to determine this I needed to run: `bacula -u root -g bacula -c /etc/bacula/bacula-dir.conf -fvm`. Once I ran this I got some meaningful output that told me I needed to update the database a couple of versions. It’s important to know which versions you are dealing with as you have to run each update to the database individually.

First, a dump of all databases that are relevant is a good idea: `mysqldump -A -p > backup.sql`. Then, we can move onto updating the databases knowing full well that if something goes awry we simply import our backup and revert bacula to the old version.

The database update scripts were located in `/usr/libexec/bacula/updatedb/` and when run in the correct order will get you back up and running in short order. When all is said and done utter the `/etc/init.d/bacula-dir` and your installation should whir to life once again.

Conclusion

When updating bacula don’t forget the catalog database may have changes that require it to be updated using the various bacula updatedb scripts. Backups should commence at their regularly scheduled time.

 Posted by Alex Brandt at 3:47 PM  Tagged with: ,
© 2011 Alunduil's Hosting Suffusion theme by Sayontan Sinha