How to set up logrotate to manage and monitor a linux server

Log files contain valuable information on what is happening inside the system. They are generally inspected during troubleshooting processes included in server performance analysis. In a production environment, log files may grow quickly into large sizes. This can complicate matters and becomes server resources exhausts up space. Besides, opening and inspecting an individual particular or large log file is often tricky.

logrotate has proved to be a useful tool that can automate and simplify the operation of troubleshooting and/or rotating, compressing, and deleting old log files. For example it is possible to set up logrotate so that the log file /var/log/foo is rotated every 1 month, and logs more than six months are deleted. Once configured, the process is fully automated using logrotate without need for server monitoring. Logrotate can be set in a way that it sends email.

The logrotate package is normally pre-installed in major distros of linux. If however, it is missing, you can download it as a binary or rpm from a third party website using yum or apt-get tool.

# apt-get install logrotate cron 

On Fedora, CentOS or RHEL:

# yum install logrotate crontabs 

The configuration file for logrotate is /etc/logrotate.conf. However no modification is needed in this file. The log files to be rotated are defined in separate configuration file(s) placed under /etc/logrotate.d/ directory

Example

we will create a 10 MB log file /var/log/log-file to highlight how you can use logrotate to manage this log file. We start by creating a log file, and populating it with a 10 MB worth of random bit stream.

# touch /var/log/log-file
# head -c 10M < /dev/urandom > /var/log/log-file

Now that the log file is ready, we will configure logrotate to rotate this log file. Let’s create a configuration file for this.

# vim /etc/logrotate.d/log-file

/var/log/log-file {
    monthly
    rotate 5
    compress
    delaycompress
    missingok
    notifempty
    create 644 root root
    postrotate
        /usr/bin/killall -HUP rsyslogd
    endscript
}

definations

  • monthly: The log file will now be rotated monthly. Other possible values are ‘daily’, ‘weekly’ or ‘yearly’.
  • rotate 5: A total of 5 archived logs will be stored at a time. For the 6th archive, the oldest stored archive will be deleted.
  • compress: The rotated archive will be compressed using gzip, after the rotation task is complete.
  • delaycompress: Always used together with compress option, the delaycompress parameter instructs logrotate to not run compression on the most recent archive. Compression will be performed during the next rotation cycle. This is useful if you or any software still needs to access the fresh archive.
  • missingok: During log rotation, any errors will be ignored, e.g., “file not found”.
  • notifempty: Rotation will not be performed if the log file is empty.
  • create 644 root root: A fresh log file will be created with specified permissions as logrotate may rename the original log file.
  • postrotate/endscript: The command(s) specified between postrotate and endscript will be carried out after all other instructions are completed. In this case, the process rsyslogd will re-read its configuration on the fly and continue running.

Example 2

to rotate a log file when its size reaches 50 MB

# vim /etc/logrotate.d/log-file

/var/log/log-file {
    size=50M
    rotate 5
    create 644 root root
    postrotate
        /usr/bin/killall -HUP rsyslogd
    endscript
}

Example Three

you can set old log files to be named with the date of creation. This can be achieved by adding dateext parameter.

# vim /etc/logrotate.d/log-file
/var/log/log-file {
monthly
rotate 5
dateext
create 644 root root
postrotate
/usr/bin/killall -HUP rsyslogd
endscript
}

This will cause the archived files to contain the date in their name.

Troubleshooting

Here are a few troubleshooting tips for logrotate setup.

Running logrotate manually

logrotate can be invoked manually from the command line .

To invoke logrotate on all logs as configured in /etc/logrotate.d/*:

# logrotate /etc/logrotate.conf

To invoke logrotate for a particular configuration:

# logrotate /etc/logrotate.d/log-file
  1. Dry run

The first and the best command to run for any troubleshooting , a dry run. It operates a dummy log rotation and displays its output without rotating any log files.

# logrotate -d /etc/logrotate.d/log-file

Force run

a forced logrotate can be set to rotate log files even when rotation options are not set, by using ‘-f’ option. The ‘-v’  (verbose) output.

# logrotate -vf /etc/logrotate.d/log-file
reading config file /etc/logrotate.d/log-file
reading config info for /var/log/log-file
  1. Logrotate logging

Logs for logrotate itself are usually stored in the directory /var/lib/logrotate/status. If you want to troubleshoot any specific file  specify that from the next command .

# logrotate -vf –s /var/log/logrotate-status /etc/logrotate.d/log-file
  1. Logrotate cron job

cron jobs are automatically installed during OS installation.

# cat /etc/cron.daily/logrotate
#!/bin/sh

# Clean non existent log file entries from status file

cd /var/lib/logrotate
test -e status || touch status
head -1 status > status.clean
sed 's/"//g' status | while read logfile date
do
    [ -e "$logfile" ] && echo "\"$logfile\" $date"
done >> status.clean
mv status.clean status

 
test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate /etc/logrotate.conf

logrotate has proved to be a very useful tool for preventing log files from taking up storage space. The process of log rotation is fully automated, and can run without the need for physical intervention on the server. This tutorial has focused on a few basic examples of how to use logrotate. Further optimizations have been included which you can implement to match your requirements.

Advertisements

Comments are closed.

%d bloggers like this: