RSS
 

Bacula Update Database Woes

27 Jun

Introduction

Backups, a subject always talked about but rarely put into practice. If you do happen to have a backup solution, excellent, you probably sleep better than our comrades without a recovery plan. I’ve been using bacula for my backup solution for over a year now and one thing I’ve never had enough experience with has been upgrading bacula. The upgrade process can be pretty hairy depending your distribution, level of investment, etc.

Bacula Updates

Recently bacula-5.0.2-r1 was marked as stable for the Gentoo distribution. Upon finishing the emerge (laugh if you like at my compilation of all packages), I attempted the restart of all bacula services to bring the new version live:

/etc/init.d/bacula-sd restart
/etc/init.d/bacula-fd restart
/etc/init.d/bacula-dir restart

All went well until I did the restart on the director (bacula-dir). At that point things took a nasty turn for the worse. The director didn’t want to start and there were no messages on the screen indicating why this might be the case.

Long story short: the database needed some schema changes to be applied before the director could start up and get running again. In order to determine this I needed to run: bacula -u root -g bacula -c /etc/bacula/bacula-dir.conf -fvm. Once I ran this I got some meaningful output that told me I needed to update the database a couple of versions. It’s important to know which versions you are dealing with as you have to run each update to the database individually.

First, a dump of all databases that are relevant is a good idea: mysqldump -A -p > backup.sql. Then, we can move onto updating the databases knowing full well that if something goes awry we simply import our backup and revert bacula to the old version.

The database update scripts were located in /usr/libexec/bacula/updatedb/ and when run in the correct order will get you back up and running in short order. When all is said and done utter the /etc/init.d/bacula-dir and your installation should whir to life once again.

Conclusion

When updating bacula don’t forget the catalog database may have changes that require it to be updated using the various bacula updatedb scripts. Backups should commence at their regularly scheduled time.

 
Comments Off

Posted in Linux Guides

 

Blueman with Dolphin: Bluetooth File Browsing

27 Jun

Introduction

Bluetooth is a convenience that shouldn’t be underestimated. Bluetooth has invaded pretty much every peripheral device we’ve come to know and love: mice, keyboards, ad-hoc networks, etc. One of the most convenient aspects of bluetooth is the ability to browse filesystems through a simple ad-hoc network connection from a phone or other peripheral device.

Enter Blueman

Blueman is my current preferred bluetooth management utility but others do exist including (gnome-bluetooth and kbluetooth). I have found the configurability and freedom of blueman to give me a bit more flexibility than the other solutions.

The problem with blueman under KDE is that first and foremost blueman is a gnome application. Now this isn’t inherently bad but it makes interoperability a little trickier than if I used kbluetooth (which of course hooks directly into KDE). The first issue (and pretty much only issue so far) is that blueman expects you to use nautilus to browse files on a remote device. Again, not bad but I don’t have nautilus so it’s a little tricky to see those files.

Blueman Transfer Settings

Blueman Transfer Settings

What we see in the image above is that we can modify the obex ftp browser that is used by blueman. We’re going to change this to a custom script (shown below) that uses a FUSE obexfs and dolphin to achieve our desired behavior, browsing our bluetooth file with dolphin.

#!/bin/bash
# bluemount.sh

obexfs -b $1  ~/.bluemnt 
dolphin ~/.bluemnt --nofork
fusermount -u ~/.bluemnt

This script requires obexfs, fuse and of course dolphin to work properly. What the script does is mounts the bluetooth file system to a location that we can actually browse (I arbitrarily chose ~/.bluemnt and you can choose another directory if you like). The argument we get from blueman, $1, is the device id that obexfs expects in order to mount our filesystem (very convenient). Once the filesystem is mounted we simply tell dolphin to open that directory but stay in the foreground so we can properly unmount the device when we finish.

Conclusion

Integrating blueman with KDE can seem daunting but a few hooks and scripts allows the two to inter-operate with ease. Now we can simply open our device list and click the browse button to get dolphin to show our files on the bluetooth device in question.

 
Comments Off

Posted in Linux Guides

 

Memcached Remote Memory Sharing

24 Jun

Introduction

Memcached is a simple key/value memory store that allows values from pretty much any application to be stored in memory for quick retrieval. Most languages have simple APIs that make this very easy from a programming standpoint. This also means that it’s not quite as simple to configure as APC (but nearly for modern applications).

Installation and Configuration

To install memcached I recommend using your choice of installation (mine being emerge) and then checking the configuration (again in the case of Gentoo) located at /etc/conf.d/memcached.

Update the file to listen on the interface you want, use the memory you want, etc and then start the daemon up. Now if you’re using memcached on the same server as your apache server most webapps will be able to just turn on memcached and connect and work but if you’re using memcached and apache on separate servers it’s a little more work.

WordPress and memcached

I’ve already touted the wordpress plugin: W3 Total Cache but unfortunately this plugin doesn’t allow you to easily configure memcached usage through the web interface. It defaults to using the memcached server located at 127.0.0.1:11211 which if we’re using separate servers is less than ideal. The file we need to modify with a simply sed (or by hand if you prefer knowing what’s going on) is %{DOCROOT}/wp-content/w3-total-cache-config.php.

We just need to change the memcached servers to point at our server’s location before enabling it in the interface. To easily accomplish this we can use sed:

sed -i -e 's/127.0.0.1:11211/<host>:<port>/g' %{DOCROOT}/wp-content/w3-total-cache-config.php

Mediawiki and memcached

The other application we can easily setup for memcached is mediawiki. Mediawiki wants us to modify its LocalSettings.php with the following additions:

## Shared memory settings 
$wgMainCacheType = CACHE_MEMCACHED; 
$wgParserCacheType = CACHE_MEMCACHED; 
$wgMessageCacheType = CACHE_MEMCACHED; 
$wgMemCachedServers = array("giskard.alunduil.com:11211"); 

$wgSessionsInMemcached = true;

These settings simply turn on memcached (CACHE_MEMCACHED) for the various caching areas that mediawiki uses and specifies an array of memcached servers to store that caching information in.

If you do have more than one memcached server configured and available you can use the following syntax to include all of them with various weights:

$wgMemCachedServers = array(array("host:port", weight), array("host2:port2", weight));

Conclusion

Using memcached can greatly improve the performance of webapps or other applications that don’t need to present the most up to date information. This can reduce traffic to an overloaded database or configuration file or whatever other store you want to place a cache in front of.

 
1 Comment

Posted in Linux Guides

 

Backup Roaming Clients Easily

27 May

Introduction

We all know that backups are awesome … until we try to find the time to set them up or the space to store them on. We all know that without backups we may see that day when we boot up our computer and our data isn’t there for whatever reason. How can we simplify backups and more importantly initiate them client side?

rdiff-backup

The backup utility that I use to accomplish backing up roaming clients is rdiff-backup. This simple python wrapper for rsync simplifies keeping incremental backups and backup sets for a historical set of backups. By default rdiff-backup is pretty versatile and its documentation is fairly extensive.

What they don’t mention is how to automate these backups with an ssh key to a remote system and a cron job as a scheduler.

The cron job

The cron job is very very simple and highly adaptable:

/usr/bin/rdiff-backup --remote-schema 'ssh -i /home/alunduil/.ssh/backup_dsa %s rdiff-backup --server' --exclude-other-filesystems --print-statistics /home/alunduil daneel.alunduil.com::elijah-backup && /usr/bin/rdiff-backup --remote-schema 'ssh -i /home/alunduil/.ssh/backup_dsa %s rdiff-backup --server' --remove-older-than 7D --force daneel.alunduil.com::elijah-backup

Now let’s break this down so you can make it yours:

  • /usr/bin/rdiff-backup – Our script of course.
  • –remote-schema – The trick that makes it work. This actually allows you to specify the way ssh is called by rdiff-backup allowing a lot of control over how it authenticates and what happens on the server.
  • –exclude-other-filesystems – Specifies to stay on the mountpoint it started on.
  • –print-statistics – Prints some nice information about the backup when it finishes.
  • /home/alunduil daneel.alunduil.com::elijah-backup – The source and destination of the backup.
  • && – Just making sure we don’t delete old backups without having a good one (trust me you want it this way).
  • –remove-older-than 7D –force – Force the removal of any backups that are older than one week.

Conclusion

Creating a backup of a roaming client should be easy and with rdiff-backup with cron it certainly is. You have complete control over when and where your backups reside and it can be set to run on boot or any other condition. This is ideal for hosts that don’t always have an internet connection or who have uptimes of less than 50%.

 
Comments Off

Posted in Linux Guides

 

APC Caching PHP

26 May

Introduction

Making PHP run faster is usually pretty easy when you see some of the code that people can write (myself included) but what if we either don’t want to look at their code or don’t want to fix the code? What else can we do to improve the runtime and the load time of PHP web sites (this website included)?

We’ll cover how to use APC (Alternative PHP Cache) to help alleviate some of the issues of a slow web site. Just like last time we optimized MySQL and talked about optimizing Apache we’ll continue on the path to getting PHP applications as fast as we can (without touching a line of code).

Enter APC

APC is PEAR project and a PHP module that after installed and a restart of Apache is already doing some work to make life better. By default APC turns on op-code caching which saves on the compile and execution times of PHP. The configuration for this module is pretty straight forward but let’s look at a copy of the one I’m using:

; /etc/php/apache2-php5/ext-active/apc.ini (On Gentoo anyways ...)
extension=apc.so
apc.enabled="1"
apc.shm_segments="4"
apc.shm_size="128"
apc.num_files_hint="1024"
apc.ttl="7200"
apc.user_ttl="7200"
apc.gc_ttl="3600"
apc.cache_by_default="1"
;apc.filters=""
;apc.mmap_file_mask="/tmp/apcphp5.XXXXXX"
apc.slam_defense="0"
apc.file_update_protection="2"
apc.enable_cli="0"
apc.max_file_size="1M"
apc.stat="1"
apc.write_lock="1"
apc.report_autofilter="0"
apc.include_once_override="0"
apc.rfc1867="0"
apc.rfc1867_prefix="upload_"
apc.rfc1867_name="APC_UPLOAD_PROGRESS"
apc.rfc1867_freq="0"
apc.localcache="0"
apc.localcache.size="512"
apc.coredump_unmap="0"

Zooming In

These options are documented fairly thoroughly in the APC PHP Manual.

Most of these options can stay at their defaults and provide a pleasant experience but if we have the memory we should probably tweak these to get more out of them: * apc.shm_segments – The number of chunks to use from /dev/shm. * apc.shm_size – The size of aforementioned chunks.

These two parameters are the heart of APC and dictate the memory usage you’ll get out of it. If you take segments*size you’ll get the maximum amount of shm space used by APC for the cache. A limitation on shm_size is usually in place through the kernel and to determine what this is you can simply cat /proc/sys/kernel/shmmax (remember this command prints out B and APC expects MB).

Further Caching

APC also has an object cache available which can be used for just about anything but the application has to be modified to support this. If you’re using WordPress to host your site an excellent plugin for doing this is W3 Total Cache.

Conclusion

APC can help you get an edge out of your server but without careful tuning won’t get you more than the last inch of performance. Play with the memory settings until it isn’t over-utilizing memory (have a larger cache than your code base needs) or under-utilizing memory (having a smaller cache than your code base needs) but don’t forget you don’t want it to run into your swap space.

 
Comments Off

Posted in Linux Guides

 
« Older Entries