Kerry D. Wong » Apache

Apache2 Logrotate

kwong — Thu, 12 Aug 2010 00:14:04 +0000

Whenever I setup a new web server, I always like the idea of keeping the Apache logs roated on a daily basis. While the procedures for changing the log rotate frequency is pretty straight forward, I have seen a lot of people having problems with it so I decided to document it.

To change the Apache log rotate frequency, you would want to change the apache2 configuration file located under /etc/logrotate.d. Configurations located in this directory overrides the default settings in /etc/logrotate.conf (the configurations in /etc/logrotate.d are included in logrotate.conf)

Here’s what my settings look like (note that I had commented out compress option so that all the logs can be easily browsed):

/var/log/apache2/*.log {
daily
missingok
rotate 365
#compress
#delaycompress
notifempty
create 640 root adm
sharedscripts
postrotate
if [ -f "`. /etc/apache2/envvars ; echo ${APACHE_PID_FILE:-/var/run/apache2.pid}`" ]; then
/etc/init.d/apache2 reload > /dev/null
fi
endscript
}

Common problem

A common problem after changing the log rotate frequency is that the Apache logs do not appear to be rotating daily even though the configuration file had been changed correctly.

This seems to be a known issue with logrotate. If you read the manual for logrotate you’d see the following explanation:

-f, –force
Tells logrotate to force the rotation, even if it doesn’t think
this is necessary. Sometimes this is useful after adding new
entries to a logrotate config file, or if old log files have
been removed by hand, as the new files will be created, and
logging will continue correctly.

Since the default cron daily job does not force the log rotation, you may want to add -f parameter to /etc/cron.daily/logrotate:

#!/bin/sh

test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate -f /etc/logrotate.conf

With this change, the logs will be forced to rotate daily according to the setup.

Parsing Apache Server Log for Unique IPs

kwong — Sun, 01 Feb 2009 16:41:01 +0000

I created this short Perl script to tally the number of unique IPs from the Apache server log. Since by default, the logrotate daemon compresses Apache logs, we need to disable the compression option for the Apache log configuration. In Ubuntu, this setting can be changed in /etc/logrotate.d/apache2. To disable compression, just comment out the following lines:

#compress
#delaycompress

The Perl script is quite straight forward: it basically builds a hash table and use the IP address as the key:

#!/usr/bin/perl

$apacheLogDir = "/var/log/apache2";

opendir(DIR, $apacheLogDir);

@files = grep(/^access/, readdir(DIR));

closedir(DIR);

%hashTable=();

$uniqueIP = 0;

foreach $file (@files) {

open FILE, "$apacheLogDir" . "/" . "$file" or die $!;

@lines = ;

foreach $line (@lines) {

@IP = split(/\s+/, $line);

if (exists($hashTable{$IP[0]})) {

$hashTable{$IP[0]}++;

} else {

$hashTable{$IP[0]} = 1;

$uniqueIP++;

#$resolved = qx{resolveip $IP[0]};

#print "$resolved";

print "$IP[0]\n";

}

close(FILE);

}

print "Total unique IPs: $uniqueIP.\n";

The two lines commented out:

#$resolved = qx{resolveip $IP[0]};
#print "$resolved";

can be used to identify the domain name the IP is associated with. Since reverse DNS lookup is a rather slow process, it will take a long time to resolve all the IPs. I may change it to use multiple threads to speed up the DNS lookup in the further.

Apache Log Rotation on Ubuntu Server

kwong — Thu, 04 Oct 2007 01:57:50 +0000

Interestingly, the log rotation utility rotatelogs which came with Apache does not work with Ubuntu 7.04 installation. At least not when the Apache is installed via apt-get.

By default, Ubuntu server utilizes logrotate not rotatelogs to rotate all its server logs. Appache access and error logs are rotated on a weekly basis when first installed. To change the log rotation frequency, you will need to change the following file:

/etc/logrotate.d/apache

The main difference between logrotate and rotatelogs is that logrotate is triggered by a cron job whereas rotatelogs runs in the background as a daemon. In Ubuntu, cron jobs are neatly organized into cron.hourly, cron.daily, cron.weekly and cron.monthly folders. This makes creating ad-hoc cron jobs a very easy task.