Alunduil's Hosting Gentoo Hackery and Other Fun …

2Mar/10

Running awstats on Gentoo

Requirements

I'm assuming for this quick guide that you already have a website installed and consequently apache.

emerge -av awstats

A quick breakdown on the use flags from gentookit's equery:

  • apache2 : Add Apache2 support
  • geoip : Add geoip support for country and city lookup based on IPs
  • vhosts : Adds support for installing web-based applications into a virtual-hosting environment

Configuration

Lots of configuration needs to occur for awstats to work correctly:

First, we need to setup a configuration file for the web site so we can update the statistics. cp /etc/awstats/awstats.model.conf /etc/awstats/awstats..conf

Where fqdn is the fully qualified domain name of your website you'll be monitoriing. After you've copied the default configuration customize it for your particular website.

Second, you need to enable awstats in your apache vhost configuration:

CustomLog /var/www/localhost/log/apache/production.log combined

Alias /awstats/classes "/usr/share/webapps/awstats/6.9-r1/htdocs/classes/"
Alias /awstats/css "/usr/share/webapps/awstats/6.9-r1/htdocs/css/"
Alias /awstats/icons "/usr/share/webapps/awstats/6.9-r1/htdocs/icon/"
ScriptAlias /awstats/ "/usr/share/webapps/awstats/6.9-r1/hostroot/cgi-bin/"
ScriptAlias /awstats "/usr/share/webapps/awstats/6.9-r1/hostroot/cgi-bin/awstats.pl"
ScriptAlias /awstats.pl "/usr/share/webapps/awstats/6.9-r1/hostroot/cgi-bin/awstats.pl"

<Directory "/usr/share/webapps/awstats/6.9-r1/htdocs">
   Options None
   AllowOverride None
   <IfModule mod_access.c>
       Order allow,deny
       Allow from all
   </IfModule>
</Directory>

<Directory "/usr/share/webapps/awstats/6.9-r1/hostroot/cgi-bin">
    Options ExecCGI
    AllowOverride None
    <IfModule mod_access.c>
        Order allow,deny
        Allow from all
    </IfModule>
</Directory>

<Directory "/usr/share/webapps/awstats/6.9-r1/hostroot">
    Options None
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

<Directory "/usr/share/webapps/awstats/6.9-r1/htdocs/icon">
    Options None
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

And verify the logging output in /etc/apache2/modules.d/00_mod_log_config.conf:

<IfModule log_config_module>
    # The following directives define some format nicknames for use with
    # a CustomLog directive (see below).
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %b" common

    LogFormat "%{Referer}i -> %U" referer
    LogFormat "%{User-Agent}i" agent
    LogFormat "%v %h %l %u %t \"%r\" %>s %b %T" script
    LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" VLOG=%{VLOG}e" vhost

    <IfModule logio_module>
        # You need to enable mod_logio.c to use %I and %O
        LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
        LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" vhostio
    </IfModule>

    # The location and format of the access logfile (Common Logfile Format).
    # If you do not define any access logfiles within a <VirtualHost>
    # container, they will be logged here.  Contrariwise, if you *do*
    # define per-<VirtualHost> access logfiles, transactions will be
    # logged therein and *not* in this file.
    CustomLog /var/log/apache2/access_log common

    # If you would like to have agent and referer logfiles,
    # uncomment the following directives.
    #CustomLog /var/log/apache2/referer_log referer
    #CustomLog /var/log/apache2/agent_logs agent

    # If you prefer a logfile with access, agent, and referer information
    # (Combined Logfile Format) you can use the following directive.
    #CustomLog /var/log/apache2/access_log combined
</IfModule>

And Lastly, you need to add a cron entry to update the statistics on a regular basis: # AWStats */15 * * * * perl /usr/share/webapps/awstats/6.9-r1/hostroot/cgi-bin/awstats.pl -config=www.alunduil.com -update > /dev/null

Everything should be running smoothly but give your installation some time to begin collecting statistics.

22Jan/10

CMake … Java … Pain …

I've been working quite fervently to get CMake working for a Java project I've taken on and have found that it has a couple of issues still. The first issue is an undefined CMAKE_Java_LINK_EXECUTABLE variable, which because Java doesn't really do linking (JIT takes care of doing something that I haven't looked into yet) we don't really need, but CMake expects it to be there. By simply making it a mv string and setting up the classpath option (via INCLUDE_DIRECTORIES directives) correctly, we can compile our java project using CMake almost the same way we would expect to make a CXX project.

If anyone knows of the platform agnostic way to move files in CMake I'd love to know.

Unfortunately the following changes are required to some CMake 2.8 system files for this to work:

In /usr/share/cmake/Modules/CMakeJavaInformation.cmake:

IF(NOT ${CMAKE_Java_LINK_EXECUTABLE})
SET(CMAKE_Java_LINK_EXECUTABLE 
    "mv <OBJECTS> <TARGET>")
ENDIF(NOT ${CMAKE_Java_LINK_EXECUTABLE})

In /usr/share/cmake/Modules/CMakeJavaCompiler.cmake.in:

SET(CMAKE_Java_LINK_EXECUTABLE "@CMAKE_Java_LINK_EXECUTABLE@")
Tagged as: , , No Comments
20Jan/10

The Mother System Load

Today I found out that some machines can't handle the load I throw at them. The following was the status of my web server after apache filled up its memory. A quick (quite slow actually) reboot of apache and everything was stabilized.

Ridiculous System Load

11Jan/10

Layman Overlay

My Overlay

My overlay has been available via subversion for quite some time now, but not in any easy to use format (integration with layman).  Getting this overlay to work with yours must have been a pain if you wanted to try something I had been working on.

Making My Overlay Available

The first thing I had to do to make my overlay talk with layman let alone get along with layman was add an xml definition of the overlay somewhere.  I chose the easy to manipulate path of http://www.alunduil.com/svn/portage/trunk/portage/alunduil-overlay.xml.

Adding My Overlay List to Your Layman

To add my overlay(s) to your layman list simply add the following path to your overlays variable in /etc/layman/layman.cfg: http://www.alunduil.com/svn/portage/trunk/portage/alunduil-overlay.xml.

18Dec/09

Optimizing Gentoo CFLAGS

Starting Off

Make sure you've at least looked at the following document: [http://www.gentoo.org/doc/en/gcc-optimization.xml](Gentoo Optimization Guide).

Checking Flags

The only real change we want to make to our system are simple things we know will not break it. Thus, we only look at making sure that we have the correct flags for enabling all of the instruction sets that our processor has available.

Checking Processor

grep flags /proc/cpuinfo | grep uniq

This pulls out all of the features of the processor as detected by the kernel. Example output is shown (line breaks added for readability):

alunduil@elijah ~ $ grep flags /proc/cpuinfo | uniq
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr 
pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall 
nx mmxext fxsr_opt rdtscp lm 3dnowext 3dnow rep_good 
extd_apicid pni cx16 lahf_lm cmp_legacy svm extapic 
cr8_legacy 3dnowprefetch

Checking Default Flags

gcc does a great job of determining what flags should be set by using the new -march=native flag (which you should have set by this point). Using the following command we can double check that all of the instructions we want enabled are enabled:

gcc -Q -c -v -march=native --help=target | grep disabled

If anything appears in the resulting list it's not enabled and should be enabled by adding the appropriate -m flag. For my cpu this results in: -msse3 -m3dnow.

Putting it Together

By adding the original guide with a check for instructions we allow gcc to utilize the instructions that were specifically created for uses we may have (multimedia, extended math, etc).

The resulting CFLAG variable that I placed in my make.conf from the above discussion:

CFLAGS="-march=native -O2 -pipe -msse3 -m3dnow"
23Jul/09

Cleaning /etc/portage/package.*

Inroduction

After a while the files in /etc/portage become cluttered with the common, "Let's try this . . . whoops that didn't work . . . let's try this." No matter how hard you try to keep this clean you have probably forgotten something along the way. Installing a testing package then removing it because you found a better one later, etc. Well what's an easy way to clean these files up and make sure that we minimize their sizes and keep our Gentoo system crisp (or as crisp as we can by just managing these files)? I've written some bash one-liners that assist with this and could easily be adapted into a script that automates a lot of the cleaning for you.

In all of these scripts change the /etc/portage/package.use to the file you are interested in cleaning.

Checking for Multiple Occurrences of an Atom Within a File

for atom in $(gawk '{print $1}' /etc/portage/package.use); do [ "$(grep ${atom} /etc/portage/package.use | wc -l)" -gt "1" ] && echo "${atom}"; done

Checking for N Uses of a Use Flag in /etc/portage/package.use

I use this to move frequently used use flags to /etc/make.conf if it seems appropriate.

for flag in $(gawk '{print $2}' /etc/portage/package.use); do [ "$(grep "${flag}" /etc/portage/package.use | wc -l)" -gt "2" ] && echo "${flag}"; done

Checking for Removed Atoms Within a File

for atom in $(gawk '{print $1}' /etc/portage/package.use); do [ "$(portageq match / ${atom} | wc -l)" -lt "1" ] && echo "${atom}"; done
22Jul/09

“Disabling” KScreenSaver via DBUS

KDE 4 has some major improvements over older versions, but it also seems to have gone backwards in places. The new libraries probably contribute to this, and are absolutely the way to go. A nice ability that I've been looking for in powerdevil (the new power manager in KDE 4) is how to have the screensaver "disable" when entering presentation mode. This is behavior that I know I expected, but found to my dismay partway through a presentation that the screensaver still kicked in.

After looking around for ways to "fix" this problem I finally found some interesting information in the form of the DBUS interface provided by the screensaver in KDE. Using qdbusviewer I was able to find an API for the screensaver that can be invoked at any point and from anywhere (assuming that you're part of the session). Using this new ammunition for more Google searching I found that I could write a daemon in python that would keep the screensaver from displaying while it was turned on.

The result of this work can be found in my subversion repository as stop_kscreensaver.py. This script only has 3 parameters and is very easy to use. When starting the daemon you simply pass a time between activity simulations (by setting this just shorter than the timeout for your screensaver activation it is much more efficient) and if desired a different log level. To stop the daemon you simply pass the kill parameter which reads the PID from a standard file and makes sure the daemon dies.

The timing parameter for this script is fairly functional in that you can pass the time in with various units and the conversion will be taken into account. For example, one could pass a time of 2h32.1m94.34s. Why anyone would is beyond me, but hey I figured with regular expressions it might be easy to do. If no units are passed the script assumes that the number passed was in seconds. As always if any bugs are found please e-mail me, Alex Brandt with a description of the problem (or if you're ambitious a patch file would be appreciated).

Now the important part. How do we get this to work with powerdevil? That's the easiest part of all with powerdevil's execute this script when switching to this profile feature. We simply save the script somewhere, make it executable (chmod 755), and then set the path (or browse to it) in the powerdevil configuration interface.

Once that is in place you can switch to the profile you set the daemon up to start in and the screensaver although active will not start up until you switch profiles again. This let's you watch that movie you wanted to just like our favorite comic XKCD tells us about.

26Apr/09

Create a Symfony Development Environment

What are We Talking About?

Symfony, the hot new framework for PHP web application development, can drastically reduce development times on intricate applications. By utilizing a framework and the Rapid Application Development, Agile Development, XP, or whatever development cycle you might like best, elegant code is easier to produce and maintain. With proper design patterns already in use, one simply needs to fill in the appropriate logic at each level, and let the application grow into being. For more information on Symfony the following resources are available:

This is all great, but not nearly as useful if we can't develop and test locally. To get a nice development area installed what do we need to do? The following steps outline how to setup a machine so that you can have a projects directory that will automatically map to a nice set of URLs with minimal configuration.

What Do We Need?

We need the following installed (Gentoo use flags are in parentheses):

  • apache
  • mysql
  • php (cli ctype reflection spl simplexml xml pcre session apache2 mysql pdo xsl)
  • symfony

What's the Setup?

So how do we turn a directory full of projects into something we can get at by browsing to http://localhost/project? Well, we start by symlinking (or just changing) the docroot to the directory we want to use.

# rm -rf /var/www/localhost/htdocs
# ln -snf <path to projects> /var/www/localhost/htdocs

What we want to do now is change the vhost declaration for the default virtual host in apache. It's only an addition of a few simple lines (placed after the docroot declaration):

AliasMatch ^/((?!cgi-bin|icons).+)/sf/(.*) /usr/share/php5/data/symfony/web/sf/$2
AliasMatch ^/((?!sf|cgi-bin|icons).+?)/(.*) /var/www/localhost/htdocs/$1/web/$2

<Directory "/usr/share/php5/data/symfony/web/sf">
  Options Indexes FollowSymLinks               
  Order allow,deny                             
  Allow from all                           
</Directory>

The last thing that needs to be done is make sure the default redirect in your symfony project's .htaccess file redirects to //index.php.

Conclusions

That should be all there is to it, but as always your mileage may vary. If you notice significant problems or shortcomings with this how to, please, leave a comment denoting that. I will update this guide appropriately as quickly as possible.

Tagged as: , , No Comments
16Apr/08

C++ Programming Students

It's strange that I'm the only tutor for our Computer Science and Information Systems department, but I can live with that. What irks me though, is how these students come to gain their knowledge about programming.

To be fair I will not mention names, and not blame anyone. I truly believe this to be a simple matter of miscommunication, and easily solved. I also feel that the scope of these classes needs to be monitored a little more closely, and what follows is simply what I understand the classes should teach in an introductory programming class, and my understanding of what is being taught in our introductory programming classes.

The classes in question are our first introductory classes to C++, and then our follow up course. These courses together should leave the student comfortable with the basic constructs and built-ins of C++. It should also make them comfortable with classes, and working in an OOP(Object Oriented Programming) paradigm. The problem crops up in the transition between these two classes. It seems (and this I know because of my tutees) that their understanding of the basics gets muddled in the first class, and thus they cannot build upon this knowledge in the second class. I have heard many things about what may be happening wrong in this class to cause such a terrible calamity, but let us refocus and start by going over what should be covered in the first class in order to succeed in the following class and throughout the whole program.

In an introductory programming class it should be important for students to become familiar with functions, control structures, and various keywords (not all of them mind you just most of them). Thus they should be comfortable looking through a piece of code like the following, and have an understanding of the majority of it:

#include <cstdlib>
#include <iostream>

using namespace std;

int factorial(int n)
{
    if (n == 0)
        return 1;
    return n * factorial(n - 1);
}

int main(int argc, char argv[])
{
    if (argc != 2)
    {
        cerr << "You must pass a number to get the factorials!" << endl;
        return EXIT_FAILURE;
    }
    int n = argv[1] + 1;
    while (--n, n > 0)
        cout << "The factorial of " << n << " is " << factorial(n) << "." << endl;
    return EXIT_SUCCESS;
}

They should also be able to understand Makefiles (rudimentary Makefiles not automake or anything of its ilk), header files vs. source files, the stages of compiling and how to deal with different errors in each (i.e. A linking error versus a compilation error), and they should understand array mechanics (if not exactly what they are then how to use them). They should be able to put all of this together, and make a nice modular program that allows them to expand and reuse code in an intelligent way. Then if there is time (although this gets covered in great detail in the second class) a quick coverage of structs should ensue. At this point the education they should have gained from the first class should be enough of the basics that they understand the constructs of the langauge, and can start reading simple programs like the one above.

In reality, this goal (mind you this is my personal goal and does not reflect the goals of the professors or the department) was not met, and based on experience with people I'm tutoring who are in the second class at this time it is quite obvious where their shortcomings are. Those shortcomings are outlined below:

  • Lack of understanding about the looping control structures
  • Lack of knowledge as to how arrays function, and should be used
  • Lack of knowledge as to how the compiler actually takes the source code (possibly spread over many files), and makes an executable
  • Lack of understanding as to how to solve simple problems like manipulating all the entries of an array
  • Lack of understanding as to how to do a simple search, or utilize functions provided in an API(Advanced Programming Interface).

This list will be updated appropriately as I gain more knowledge on what the students actually know. Thus this list may have inaccuracies and shortcomings, and may not reflect reality.

The list goes in in much the same fashion here after, and this is obviously a product of the class not fulfilling its primary purpose of laying down a foundation that the students can build on. To better achieve this goal it may be necessary to at the very least recommend some supplemental texts (see list at the end of this article for recommended books for the various programming courses).

Now, what I've heard reports of in this class is that (1) these concepts outlined above are not getting covered in the detail that is required of them in the subsequent classes (which only confirms what I've observed in students), and (2) there has been a report that a new teaching tool "Alice" was used in the introduction class.

The first point appears to be just a shortcoming of the class, and could be ameliorated by simply spending more time and going over more examples of the concepts in question.

Also, the students themselves need to realize that they control their education, and must participate for a class to truly be successful for them. When entering a class, students should think to themselves what do I want from this class, and how can I have this class help me achieve that goal? By asking questions and looking for more material on the specific area of study the student is interested in they can get a much more fulfilling experience and better understanding of the topics involved, but may hit a point where they just can't keep up with the class and need further assistance that isn't self-guided. Tutors are available from the tutoring department, and should not be shied away from. If you really want to learn the material you have to be willing to get help and help yourself towards that goal. That's only part of the problem, students who do try are still having a hard time within the status quo.

The second point requires that I do not put forth my personal opinion, but does require that I state its purpose (from the Alice website):

Alice is an innovative 3D programming environment that makes it easy to create an animation for telling a story, playing an interactive game, or a video to share on the web. Alice is a freely available teaching tool designed to be a student's first exposure to object-oriented programming. It allows students to learn fundamental programming concepts in the context of creating animated movies and simple video games. In Alice, 3-D objects (e.g., people, animals, and vehicles) populate a virtual world and students create a program to animate the objects.

In Alice's interactive interface, students drag and drop graphic tiles to create a program, where the instructions correspond to standard statements in a production oriented programming language, such as Java, C++, and C#. Alice allows students to immediately see how their animation programs run, enabling them to easily understand the relationship between the programming statements and the behavior of objects in their animation. By manipulating the objects in their virtual world, students gain experience with all the programming constructs typically taught in an introductory programming course.

In conclusion, it is my perception that there is a missing communication link in the way these classes are handled, but it's not just between the professors of the two classes (they seem to not have a smooth break between the classes), it's between the students and the professors that the communication has really broken down. The students must speak up for their education or they may see it going down a path that does not maximally further their education. This is wider spread than just the simple class example I've given here. Almost everywhere one looks, it seems that students are becoming more lethargic; pushing to just get through the courses. There is a lack of genuine interest in the education being provided, and more of a view that college is now a necessity to continue in society. Unfortunately, that has begun to be so, but we can still fight for the freedom of our minds.

Supplemental Books for CSIS Courses: 1. 152 * C++ in Plain English by Brian Overland 2. 252 * C++ in Plain English by Brian Overland 3. 352 * C++ in Plain English by Brian Overland * Beyond the C++ Standard Library: An Introduction to Boost by Björn Karlsson * C++ Coding Standards: 101 Rules, Guidelines, and Best Practices by Herb Sutter & Andrei Alexandrescu * Design Patterns: Elements of Reusable Object-Oriented Software by Erich Gamma, Richard Helm, Ralph Johnson, John M. Vlissides

1Oct/07

XBOX 360

I finally broke down and purchased an XBOX 360 with Halo 3. It is an incredible improvement over the first system, and is backwards compatible. The first intriguing thing about the system from a networking perspective is their idea of "Open", "Moderate", and "Closed" NAT. I didn't know that this was possible, but after a few sniffing experiments later I learned a few things: * XBOX Live uses port 3047 * XBOX Live uses UDP * XBOX Live thinks open NAT means that port and protocol is forwarded to your XBOX through the NAT device.

This leads me to the conclusion that the XBOX Live and Vista ready devices are just engineered to provide holes in your network. I suppose this is fine for most, but what if you want that security back?