Jan 182011
 

Introduction

Portage is an amazingly simple and complex piece of technology.  The simplicity in each piece’s ability to do a specific function comes together in a complex package management system that rivals all other forms of package management (at least in my opinion).  Automating updates is something that admins everywhere do out of necessity.  Heck, automating everything is an admin’s life.  Automating portage’s updates is a bit more harrowing than other package management systems but it isn’t impossible.

Problem

As admins we attempt to simplify the work we actually do by writing scripts and programs to do most of our job for us.  It’s often been said that systems admins are the only people whose job description is to remove their job responsibilities.

Portage doesn’t have any default automation for doing nightly or even weekly portage updates but that doesn’t stop the creative from coming up with their own solution.  A simple but elegant solution is to create a small cron script that runs every day.  The problem comes when you want to read the wonderful output of portage (sometimes these messages can guide you when problems are about to occur) to avert disasters.  If the updates are performed from cron, the output will be preserved in an e-mail to the appropriate user but then we have to sift through all of the output at once.  This also doesn’t solve the issue if the updates are performed by another utility such as puppet.  These annoying little changes to the problem require a slightly more elegant solution.

Solution

The solution is to take advantage of portage’s logging specifications.  From the make.conf man file:

  • PORTAGE_ELOG_CLASSES
  • PORTAGE_ELOG_SYSTEM
  • PORTAGE_ELOG_COMMAND
  • PORTAGE_ELOG_MAILURI
  • PORTAGE_ELOG_MAILFROM
  • PORTAGE_ELOG_MAILSUBJECT

Using a combination of these directives in the make.conf file allows us to log the reports from portage to a large number of locations.  If we wanted to simply add mailing output (not the full build output just the messages) we would add the following directives to make.conf:

PORTAGE_ELOG_SYSTEM="save mail"
PORTAGE_ELOG_MAILFROM="portage@alunduil.com"

This simply adds the mailing log utility to portage and specifies that the e-mails come from the address portage@alunduil.com.  Of course, much more complex configurations can be crafted to suit any admins’ needs.

Conclusion

Letting your servers notify you of possible actions is one way of automating maintenance tasks; making maintenance eventually disappear from your task list. By starting with the tasks that are repeated the most frequently, you can quickly free up time for higher level automation and organization which leads to a cleaner and sturdier infrastructure.

May 272010
 

Introduction

We all know that backups are awesome … until we try to find the time to set them up or the space to store them. We all know that without backups we “may” see that day when we boot up our computer and our data isn’t there for whatever reason. How can we simplify backups and more importantly initiate them client side?

rdiff-backup

The backup utility that I use to accomplish backing up roaming clients is rdiff-backup. This simple python wrapper for rsync simplifies keeping incremental backups and backup sets for a historical set of backups. By default rdiff-backup is pretty versatile and its documentation is fairly extensive.

What they don’t mention is how to automate these backups with an ssh key to a remote system and a cron job as a scheduler.

The cron job

The cron job is very very simple and highly adaptable:

/usr/bin/rdiff-backup –remote-schema ‘ssh -i /home/alunduil/.ssh/backup_dsa %s rdiff-backup –server’ –exclude-other-filesystems –print-statistics /home/alunduil daneel.alunduil.com::elijah-backup && /usr/bin/rdiff-backup –remote-schema ‘ssh -i /home/alunduil/.ssh/backup_dsa %s rdiff-backup –server’ –remove-older-than 7D –force daneel.alunduil.com::elijah-backup

Now let’s break this down so you can make it yours:

  • /usr/bin/rdiff-backup – Our script of course
  • –remote-schema – The trick that makes it work. This actually allows you to specify the way ssh is called by rdiff-backup allowing a lot of control over how it authenticates and what happens on the server
  • –exclude-other-filesystems – Specifies to stay on the mountpoint it started on
  • –print-statistics – Prints some nice information about the backup when it finishes
  • /home/alunduil daneel.alunduil.com::elijah-backup – The source and destination of the backup
  • && – Just making sure we don’t delete old backups without having a good one (trust me you want it this way)
  • –remove-older-than 7D –force – Force the removal of any backups that are older than one week

Conclusion

Creating a backup of a roaming client should be easy and with rdiff-backup with cron it certainly is. You have complete control over when and where your backups reside and it can be set to run on boot or any other condition. This is ideal for hosts that don’t always have an internet connection or who have uptimes of less than 50%.

© 2011 Alunduil's Hosting Suffusion theme by Sayontan Sinha