As I mentioned last time, you want to be comfortable with the existing documentation on CFLAGS before going crazy trying to play with. It also helps to have a good understanding of what you’re doing to the code when you modify these “sacred” parameters.

Alright, now that the CYA is out of the way let’s take this one step further. Last time we talked about figuring out which instruction sets your processor understood and how to figure out what `-m` flags would get those instruction sets into the binaries on your system. This time we’ll be talking about making sure those same flags are in your use flags (just to be sure they’re picked up by the system).

Finding Flag Names

So how do we find the flags that do what we want? Well, as always BASH is our friend and can be used to find this answer in a mostly automated fashion:

. /etc/make.conf && gcc -Q -c -v ${CFLAGS} --help=target | grep enabled

This displays the currently enabled flags based on your CFLAGS parameter and allows us to find which flags have use flags with the following one liner:

gawk '/-m.*/ { print $1 }' | cut -d 'm' --complement -f 1 | xargs -I{} equery h "{}"

Conclusion

Using a little scripting we can extract the necessary information to quickly determine if there are any use flags we should be adding for particular compiler flags that our system might support.  With this last level of optimization beyond the previous time’s we should be ready to move on to -O3 (for the daring) and watch our machine’s nose bleed.

Portage is an amazingly simple and complex piece of technology.  The simplicity in each piece’s ability to do a specific function comes together in a complex package management system that rivals all other forms of package management (at least in my opinion).  Automating updates is something that admins everywhere do out of necessity.  Heck, automating everything is an admin’s life.  Automating portage’s updates is a bit more harrowing than other package management systems but it isn’t impossible.

Problem

As admins we attempt to simplify the work we actually do by writing scripts and programs to do most of our job for us.  It’s often been said that systems admins are the only people whose job description is to remove their job responsibilities.

Portage doesn’t have any default automation for doing nightly or even weekly portage updates but that doesn’t stop the creative from coming up with their own solution.  A simple but elegant solution is to create a small cron script that runs every day.  The problem comes when you want to read the wonderful output of portage (sometimes these messages can guide you when problems are about to occur) to avert disasters.  If the updates are performed from cron, the output will be preserved in an e-mail to the appropriate user but then we have to sift through all of the output at once.  This also doesn’t solve the issue if the updates are performed by another utility such as puppet.  These annoying little changes to the problem require a slightly more elegant solution.

Solution

The solution is to take advantage of portage’s logging specifications.  From the make.conf man file:

• PORTAGE_ELOG_CLASSES
• PORTAGE_ELOG_SYSTEM
• PORTAGE_ELOG_COMMAND
• PORTAGE_ELOG_MAILURI
• PORTAGE_ELOG_MAILFROM
• PORTAGE_ELOG_MAILSUBJECT

Using a combination of these directives in the make.conf file allows us to log the reports from portage to a large number of locations.  If we wanted to simply add mailing output (not the full build output just the messages) we would add the following directives to make.conf:

PORTAGE_ELOG_SYSTEM="save mail"
PORTAGE_ELOG_MAILFROM="portage@alunduil.com"

This simply adds the mailing log utility to portage and specifies that the e-mails come from the address portage@alunduil.com.  Of course, much more complex configurations can be crafted to suit any admins’ needs.

Conclusion

Letting your servers notify you of possible actions is one way of automating maintenance tasks; making maintenance eventually disappear from your task list. By starting with the tasks that are repeated the most frequently, you can quickly free up time for higher level automation and organization which leads to a cleaner and sturdier infrastructure.

The utility, pclean, does all of this and only has one major problem (so far) before I shall call it good enough for a 1.0 release. If you would like to try this little utility; it’s available in my overlay and if you notice any odd behavior please report it to my bugzilla.

Hamachi is a lightweight personal VPN connector that is a breeze to setup but there can be some pain if you don’t know what to expect. As always Gentoo provides us with an ebuild that simplifies the installation process but getting up and running is still a little confusing.

Installation and Setup

User Specific Configurations

The obvious first step is `emerge -av hamachi` (this only available to ~arch right now so add to `package.keywords` as necessary). The following are the typical instructions to install hamachi on Gentoo from portage:

1. Add to `packages.keywords` if necessary
2. `emerge -av hamachi`
3. `rc-update add tuncfg default`

After these steps have been taken you can run hamachi as any user on the system for ad-hoc VPN creation.

Server Wide Configuration

If you prefer to do a system wide on boot VPN with hamachi this is also possible but requires a slightly different setup:

1. Add to `packages.keywords` if necessary
2. `emerge -av hamachi`
3. rc-update add hamachi default

Now all configuration should be placed inside `/etc/hamachi` for this setup so the system will automatically start routing traffic correctly.

Kernel Configuration

For hamachi to work correctly you do need the tun parameter in your kernel or loaded as a module. This parameter is located in Device Drivers->Network device support->Universal TUN/TAP device driver support.

Using Hamachi

Now that hamachi is on the system we need to start using it. The server-wide installation doesn’t require this (but I’m sure you can use this method to create a configuration usable by the server-wide instance) but the user specific usage does.

Starting hamachi is as simple as the following:

1. `hamachi-init`
2. `hamachi start`
3. `hamachi login`
4. `hamachi create [ ]`
5. `hamachi join [ ]`
6. `hamachi go-online `

That’s it. You’re now connected to a private network named . You can view who else is connected to your network with `hamachi list` and `hamachi get-nicks`.

Conclusion

Setting up a VPN can be daunting (see the OpenVPN configuration documentation) or it can be a breeze with hamachi. Need a quick VPN for LAN gaming or a VPN for performing maintenance over the internet on a device behind a firewall? Hamachi may be the quick solution you’re looking for.

I’ve been using the minimal “server” profile on my Gentoo servers for a few years now (Ever since I first found out that hardened didn’t support the glibc version that the stage tarballs used).  This profile has been excellent for the minimal needs of a headless server but it constantly reminded me that I should be using hardened; so, I finally decided to take the plunge and moved all my Gentoo servers to the hardened profile. The migration was mostly painless (with the errors being my own fault as usual).

Migration Process

The process for migrating to hardened is actually well documented and very simple (I’m simply summarizing the Official Documentation).

1. Switch to a hardened profile
2. Install `hardened-sources` following the directives for PAX in the Documentation
3. Reboot the machine
4. Rebuild tool chain (gcc, glibc, binutils)
5. Rebuild world
6. Reboot the machine

That’s all there is to it. Once the reboot finalizes all processes should be working without an issue. Although, because you have now increased security you can expect some more strange bugs to work their way into your software: https://wwws.clamav.net/bugzilla/show_bug.cgi?id=2092.

Conclusion

Although strange things may be afoot I would still recommend making the plunge from a server profile to a true hardened profile. The stability far outweighs the issues that are to be encountered but be wary if something begins misbehaving in an unidentifiable manner.

There is a new king of backups in town, holland. This little framework written in Python allows one to easily backup anything that might need to be converted to a more flat file style before being backed up. Right now there is support for mysql, sqlite, and postgresql but with a little finesse it could potentially support directories as well as databases. This would make not only mysql backups a breeze but LDAP as well.

Progress Update

I have added a preliminary set of ebuilds to my overlay (which could use some code review if anyone is interested) that allows holland to easily be installed on Gentoo systems. So easy in fact that all it takes is `emerge holland`.

It accepts a set of use flags to bring in the “providers” you want to be able to backup for and makes sure that those packages are installed on the system.

Examples

The holland ebuilds have three providers right now:

• mysql
• postgresql
• sqlite

You can install any of these three you want in any combination; it doesn’t care. It will default to installing the mysql but can easily be told not to by placing -mysql in the use flags for holland. Diego Pettenò — Flameeyes mentioned to me that in EAPI 4 we’ll get the cool option of being able to specify one of a set of use flags must be set without forcing the choice but until then we have this slick solution.

There is also lvm support for snapshotting off the database directory before grabbing the database and a myriad of other features I haven’t had a chance to explore yet.

To perform a rudimentary backup after installing holland simply run `holland bk`. This will read the configurations in `/etc/holland` and backup the databases it finds.

Conclusion

The new kid on the block, holland, will make backups of complex databases and directories a breeze. Simply change that cronjob from using mysqldump to calling holland and you’re finished.

I’m assuming for this quick guide that you already have a website installed and consequently apache.

emerge -av awstats

A quick breakdown on the use flags from gentookit’s equery:

• apache2 : Add Apache2 support
• geoip : Add geoip support for country and city lookup based on IPs
• vhosts : Adds support for installing web-based applications into a virtual-hosting environment

Configuration

Lots of configuration needs to occur for awstats to work correctly:

First, we need to setup a configuration file for the web site so we can update the statistics.
cp /etc/awstats/awstats.model.conf /etc/awstats/awstats..conf

Where fqdn is the fully qualified domain name of your website you’ll be monitoriing. After you’ve copied the default configuration customize it for your particular website.

Second, you need to enable awstats in your apache vhost configuration:

CustomLog /var/www/localhost/log/apache/production.log combined

Alias /awstats/classes “/usr/share/webapps/awstats/6.9-r1/htdocs/classes/”
Alias /awstats/css “/usr/share/webapps/awstats/6.9-r1/htdocs/css/”
Alias /awstats/icons “/usr/share/webapps/awstats/6.9-r1/htdocs/icon/”
ScriptAlias /awstats/ “/usr/share/webapps/awstats/6.9-r1/hostroot/cgi-bin/”
ScriptAlias /awstats “/usr/share/webapps/awstats/6.9-r1/hostroot/cgi-bin/awstats.pl”
ScriptAlias /awstats.pl “/usr/share/webapps/awstats/6.9-r1/hostroot/cgi-bin/awstats.pl”
Options None
AllowOverride None

Order allow,deny
Allow from all

Options ExecCGI
AllowOverride None

Order allow,deny
Allow from all

Options None
AllowOverride None
Order allow,deny
Allow from all
Options None
AllowOverride None
Order allow,deny
Allow from all

And verify the logging output in /etc/apache2/modules.d/00_mod_log_config.conf:
# The following directives define some format nicknames for use with
# a CustomLog directive (see below).
LogFormat “%h %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-Agent}i\”" combined
LogFormat “%h %l %u %t \”%r\” %>s %b” common

LogFormat “%{Referer}i -> %U” referer
LogFormat “%{User-Agent}i” agent
LogFormat “%v %h %l %u %t \”%r\” %>s %b %T” script
LogFormat “%v %h %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-Agent}i\” VLOG=%{VLOG}e” vhost
# You need to enable mod_logio.c to use %I and %O
LogFormat “%h %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-Agent}i\” %I %O” combinedio
LogFormat “%v %h %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-Agent}i\” %I %O” vhostio

# The location and format of the access logfile (Common Logfile Format).
# If you do not define any access logfiles within a
# container, they will be logged here. Contrariwise, if you *do*
# define per- access logfiles, transactions will be
# logged therein and *not* in this file.
CustomLog /var/log/apache2/access_log common

# If you would like to have agent and referer logfiles,
# uncomment the following directives.
#CustomLog /var/log/apache2/referer_log referer
#CustomLog /var/log/apache2/agent_logs agent

# If you prefer a logfile with access, agent, and referer information
# (Combined Logfile Format) you can use the following directive.
#CustomLog /var/log/apache2/access_log combined

And Lastly, you need to add a cron entry to update the statistics on a regular basis:
# AWStats
*/15 * * * * perl /usr/share/webapps/awstats/6.9-r1/hostroot/cgi-bin/awstats.pl -config=www.alunduil.com -update > /dev/null

Everything should be running smoothly but give your installation some time to begin collecting statistics.

My overlay has been available via subversion for quite some time now, but not in any easy to use format (integration with layman).  Getting this overlay to work with yours must have been a pain if you wanted to try something I had been working on.

Making My Overlay Available

The first thing I had to do to make my overlay talk with layman, let alone get along with layman, was add an xml definition of the overlay somewhere.  I chose the easy to manipulate path of http://www.alunduil.com/svn/portage/trunk/portage/alunduil-overlay.xml.

Adding My Overlay List to Your Layman

To add my overlay(s) to your layman list, simply add the following path to your overlays variable in /etc/layman/layman.cfg: `http://www.alunduil.com/svn/portage/trunk/portage/alunduil-overlay.xml`.

Make sure you’ve at least looked at the following document: Gentoo Optimization Guide.

Checking Flags

The only real change we want to make to our system are simple things we know will not break it. Thus, we only look at making sure that we have the correct flags for enabling all of the instruction sets that our processor has available.

Checking Processor

grep flags /proc/cpuinfo | grep uniq

This pulls out all of the features of the processor as detected by the kernel. Example output is shown (line breaks added for readability):

alunduil@elijah ~ $ grep flags /proc/cpuinfo | uniq
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr
pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall
nx mmxext fxsr_opt rdtscp lm 3dnowext 3dnow rep_good
extd_apicid pni cx16 lahf_lm cmp_legacy svm extapic
cr8_legacy 3dnowprefetch

Checking Default Flags

gcc does a great job of determining what flags should be set by using the new -march=native flag (which you should have set by this point). Using the following command we can double check that all of the instructions we want enabled are enabled:

gcc -Q -c -v -march=native –help=target | grep disabled

If anything appears in the resulting list, it’s not enabled and should be enabled by adding the appropriate -m flag. For my cpu this results in: -msse3 -m3dnow.

Putting it Together

By adding the original guide with a check for instructions, we allow gcc to utilize the instructions that were specifically created for uses we may have (multimedia, extended math, etc).

The resulting CFLAG variable that I placed in my make.conf from the above discussion:

CFLAGS=”-march=native -O2 -pipe -msse3 -m3dnow”

After a while the files in /etc/portage become cluttered with the common, “Let’s try this . . . whoops that didn’t work . . . let’s try this.” No matter how hard you try to keep this clean, you have probably forgotten something along the way. Installing a testing package then removing it because you found a better one later, etc. Well, what’s an easy way to clean these files up and make sure that we minimize their sizes and keep our Gentoo system crisp (or as crisp as we can by just managing these files)? I’ve written some bash one-liners that assist with this and could easily be adapted into a script that automates a lot of the cleaning for you.

In all of these scripts change the /etc/portage/package.use to the file you are interested in cleaning.

Checking for Multiple Occurrences of an Atom Within a File

for atom in $(gawk ‘{print $1}’ /etc/portage/package.use); do [ "$(grep ${atom} /etc/portage/package.use | wc -l)" -gt "1" ] && echo “${atom}”; done

Checking for N Uses of a Use Flag in /etc/portage/package.use

I use this to move frequently used use flags to /etc/make.conf if it seems appropriate.

for flag in $(gawk ‘{print $2}’ /etc/portage/package.use); do [ "$(grep "${flag}" /etc/portage/package.use | wc -l)" -gt "2" ] && echo “${flag}”; done

Checking for Removed Atoms Within a File

for atom in $(gawk ‘{print $1}’ /etc/portage/package.use); do [ "$(portageq match / ${atom} | wc -l)" -lt "1" ] && echo “${atom}”; done