logrotate

Linux daemons write log files. Left to itself, these log files would grow with time and take up the disk space. It is necessary to purge the log files periodically so that the disk space is conserved and is available for smooth running of the system. logrotate is a utility for rotation, compression, removal and mailing of log files. The compression and mailing activities are optional. logrotate does this work daily, weekly, monthly, or when the log file becomes bigger than a predefined limit. The syntax of using the logrotate command is,

logrotate [OPTION ...] <config-file>

logrotate is run as a cron job. There is a script logrotate in the /etc/cron.daily directory, which is run by anacron, or by cron, if anacron is not installed. The line which calls logrotate in the script /etc/cron.daily/logrotate is,

/usr/sbin/logrotate /etc/logrotate.conf

where /etc/logrotate.conf is the logrotate configuration file. There is a state file, which is, by default, /var/lib/logrotate/status. logrotate keeps the date of file rotation for each log file in the state file. When logrotate is invoked, it checks up the last rotation date for a log file from the state file, looks at the rotation criteria and works out whether the log file is due for rotation.

For using logrotate, one has to add directives to the configuration file, /etc/logrotate.conf.

/etc/logrotate.conf

A sample /etc/logrotate.conf is given below:

# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
#compress

# packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp, or btmp -- we'll rotate them here  
/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0660 root utmp
    rotate 1
}

# system-specific logs may be configured here

Lines with the first non-whitespace character as # are treated as comments. The configuration file contains global directives and definition of log files to be rotated. Directives specific to a log file are given inside braces after the log file name. Local directives inside the braces override the global directives. Also, directives coming later on in the configuration file override the earlier ones. It is possible to give multiple configuration file names to logrotate on the command line. However, this is not recommended and single configuration file name should be given on the command line and other configuration files should be included in the main configuration file using the include directive. For example, in the above /etc/logrotate.conf configuration file, we have the include directive,

include /etc/logrotate.d

The logrotate configuration files for the individual packages are located in the directory, /etc/logrotate.d.

$ ls -ls /etc/logrotate.d
total 72
4 -rw-r--r-- 1 root root  194 2012-02-14 23:25 apache2
4 -rw-r--r-- 1 root root  126 2011-10-09 13:01 apport
4 -rw-r--r-- 1 root root  173 2011-10-06 21:00 apt
4 -rw-r--r-- 1 root root  330 2011-01-28 01:20 atop
4 -rw-r--r-- 1 root root  135 2011-05-21 14:42 consolekit
4 -rw-r--r-- 1 root root  248 2011-09-27 18:46 cups
4 -rw-r--r-- 1 root root  232 2011-10-06 13:34 dpkg
4 -rw-r--r-- 1 root root  146 2011-10-06 10:07 jockey-common
4 -rw-r--r-- 1 root root  880 2012-03-29 00:58 mysql-server
4 -rw-r--r-- 1 root root  302 2012-04-23 20:17 nginx
4 -rw-r--r-- 1 root root  157 2011-07-09 05:21 pm-utils
4 -rw-r--r-- 1 root root   94 2011-02-04 14:13 ppp
4 -rw-r--r-- 1 root root 1061 2011-01-28 01:20 psaccs_atop
4 -rw-r--r-- 1 root root  512 2011-01-28 01:20 psaccu_atop
4 -rw-r--r-- 1 root root  515 2011-10-04 02:51 rsyslog
4 -rw-r--r-- 1 root root  513 2011-06-06 07:59 speech-dispatcher
4 -rw-r--r-- 1 root root  178 2011-07-19 04:06 ufw
4 -rw-r--r-- 1 root root  114 2010-08-02 13:05 unattended-upgrades  

And, as an example, /etc/logrotate.d/rsyslog configuration file is,

$ cat /etc/logrotate.d/rsyslog 
/var/log/syslog
{
	rotate 7
	daily
	missingok
	notifempty
	delaycompress
	compress
	postrotate
		reload rsyslog >/dev/null 2>&1 || true
	endscript
}

/var/log/mail.info
/var/log/mail.warn
/var/log/mail.err
/var/log/mail.log
/var/log/daemon.log
/var/log/kern.log
/var/log/auth.log
/var/log/user.log
/var/log/lpr.log
/var/log/cron.log
/var/log/debug
/var/log/messages
{
	rotate 4
	weekly
	missingok
	notifempty
	compress
	delaycompress
	sharedscripts
	postrotate
		reload rsyslog >/dev/null 2>&1 || true
	endscript
}

If there are multiple files before a block of configuration directives inside braces, the block applies to all the files named before the block.

The following paragraphs give details about individual directives.

File rotation

Create new files

Compression

File extensions

Scripts

As an example, consider the following script for rotation of nginx webserver log files.

/var/log/nginx/mywebserver/error.log
/var/log/nginx/mywebserver/access.log
{
        rotate 7
        daily
        missingok
        notifempty
        compress
        delaycompress
        sharedscripts
        postrotate
            kill -USR1 `cat /var/run/nginx.pid`
        endscript
}

The command kill -USR1 `cat /var/run/nginx.pid` causes nginx to re-open its log files, after a rotation has been done.

Mail

Miscellaneous

logrotate options

The above discussion looked at the various directives in the logrotate configuration file. Now we look at the command line options to the logrotate command. The command syntax is,

logrotate  [-d|--debug] [-f|--force] [-m|--mail=command] [-s|--state=statefile]  
              [-v|--verbose] [-?|--help] [--usage] <configfile>

The -d, -f and -s options are quite useful in debugging the configuration files (global and those for individual packages), by running logrotate from the command line. The -d option only prints the diagnostic messages but does not rotate the log files. If actual rotation is required along with the diagnostic messages, the -v option should be used.