Installing Full-Featured Rsyslog 5.7.x On CentOS 5.X

This tutorial shows how you can install new generation of syslog servers by using Rsyslog. According to Rsyslog web site (www.rsyslog.com), Rsyslog is an enhanced syslogd supporting, among others, MySQL, PostgreSQL, failover log destinations, syslog/tcp, fine grain output format control, high precision timestamps, queued operations and the ability to filter on any message part. It is quite compatible to stock sysklogd and can be used as a drop-in replacement. Its advanced features make it suitable for enterprise-class, encryption protected syslog relay chains while at the same time being very easy to setup for the novice user.

Goals

This tutorial shows how you can compile and install full-featured Rsyslog 5.7.9 on CentOS 5.5 server. I do not issue any guarantee that this will work for you!

Pre-Installation

First we need to install the following packages:

yum install -y pcre pcre-devel mysql-server mysql-devel gnutls gnutls-devel gnutls-utils net-snmp net-snmp-devel net-snmp-libs net-snmp-perl net-snmp-utils libnet libnet-devel

Download additional package:

librelp (Reliable Event Logging Protocol Library) is an easy to use library for the RELP protocol. RELP in turn provides reliable event logging over the network. RELP (and hence) librelp assures that no message is lost, not even when connections break and a peer becomes unavailable. Please note that RELP is a general-purpose, extensible logging protocol. Even though it was designed to solve the urgent need of rsyslog-to-rsyslog communication, RELP supports many more applications.

cd /tmp
wget http://sourceforge.net/projects/libestr/files/libestr-0.1.0.tar.gz/download
tar -xvf libestr-0.1.0.tar.gz 
cd libestr-0.1.0
./configure --prefix=/usr
make
make install 

cd /tmp
wget http://www.libee.org/files/download/libee-0.1.0.tar.gz
tar -xvf libee-0.1.0.tar.gz 
cd libee-0.1.0
./configure --prefix=/usr
make
make install 

cd /tmp
wget http://honeynet.ir/software/rsyslog/librelp-1.0.0.tar.gz
tar -xvf librelp-1.0.0.tar.gz 
cd librelp-1.0.0
./configure --prefix=/usr 
make
make install

Download Rsyslog package:

At the time of writing this tutorial, I find rsyslog 5.7.9 is the best version of Rsyslog which support most of the good features you might need.

cd /tmp
wget http://www.rsyslog.com/files/download/rsyslog/rsyslog-5.7.9.tar.gz 
tar -xvf rsyslog-5.7.9.tar.gz
cd rsyslog-5.7.9

Compile & Install Rsyslog :

For more information about options which are available in Rsyslog , you can run ./configure --help

The following command enable most of the rsyslog feature such as Compression, Multithreading, MySql ,SNMP ,Mail ,RELP support and etc.

./configure --enable-regexp --enable-zlib --enable-pthreads --enable-klog --enable-inet --enable-unlimited-select --enable-debug --enable-rtinst --enable-memcheck --enable-diagtools --enable-mysql --enable-snmp --enable-gnutls --enable-rsyslogrt --enable-rsyslogd --enable-extended-tests --enable-mail --enable-imptcp --enable-omruleset --enable-valgrind --enable-imdiag --enable-relp --enable-testbench --enable-imfile --enable-omstdout --enable-omdbalerting --enable-omuxsock --enable-imtemplate --enable-omtemplate --enable-pmlastmsg --enable-omudpspoof --enable-omprog --enable-impstats 
make
make install

Prepare MySQL database:

Installing mySQL is Mandatory if you want to save syslog records to db otherwise skip this part

mysql -u root -p < plugins/ommysql/createDB.sql
mysql -u root -p mysql
GRANT ALL ON Syslog.* TO rsyslog@localhost IDENTIFIED BY 'your-mysql-password';
flush privileges;

Configure init script

vi /etc/init.d/rsyslog

#!/bin/bash
#
# rsyslog        Starts rsyslogd/rklogd.
#
#
# chkconfig: - 12 88
# description: Syslog is the facility by which many daemons use to log \
# messages to various system log files.  It is a good idea to always \
# run rsyslog.
### BEGIN INIT INFO
# Provides: $syslog
# Required-Start: $local_fs $network $remote_fs
# Required-Stop: $local_fs $network $remote_fs
# Default-Stop: 0 1 2 3 4 5 6
# Short-Description: Enhanced system logging and kernel message trapping daemons
# Description: Rsyslog is an enhanced multi-threaded syslogd supporting, 
#              among others, MySQL, syslog/tcp, RFC 3195, permitted 
#              sender lists, filtering on any message part, and fine 
#              grain output format control.
### END INIT INFO

# Source function library.
. /etc/init.d/functions

RETVAL=0

start() {
        [ -x /usr/local/sbin/rsyslogd ] || exit 5
        #[ -x /usr/local/sbin/rklogd ] || exit 5

        # Do not start rsyslog when sysklogd is running
        if [ -e /var/run/syslogd.pid ] ; then
                echo $"Shut down sysklogd before you run rsyslog";
                exit 1;
        fi

        # Source config
        if [ -f /etc/sysconfig/rsyslog ] ; then
                . /etc/sysconfig/rsyslog
        else
                #SYSLOGD_OPTIONS="-c3"
                SYSLOGD_OPTIONS="-c5"
                #KLOGD_OPTIONS="-2"
        fi

        if [ -z "$SYSLOG_UMASK" ] ; then
              SYSLOG_UMASK=077;
        fi
        umask $SYSLOG_UMASK

        echo -n $"Starting system logger: "
        daemon /usr/local/sbin/rsyslogd $SYSLOGD_OPTIONS
        RETVAL=$?
        echo
        #echo -n $"Starting kernel logger: "
        #daemon rklogd $KLOGD_OPTIONS
        #echo
        [ $RETVAL -eq 0 ] && touch /var/lock/subsys/rsyslog
        return $RETVAL
}
stop() {
        #echo -n $"Shutting down kernel logger: "
        #killproc rklogd
        #echo
        echo -n $"Shutting down system logger: "
        killproc rsyslogd
        RETVAL=$?
        echo
        [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/rsyslog
        return $RETVAL
}
reload()  {
    RETVAL=1
    syslog=`cat /var/run/rsyslogd.pid 2>/dev/null`
    echo -n "Reloading system logger..."
    if [ -n "${syslog}" ] && [ -e /proc/"${syslog}" ]; then
        kill -HUP "$syslog";
        RETVAL=$?
    fi
    if [ $RETVAL -ne 0 ]; then
        failure
    else
        success
    fi
    echo
    RETVAL=1
    #echo -n "Reloading kernel logger..."
    #klog=`cat /var/run/rklogd.pid 2>/dev/null`
    #if [ -n "${klog}" ] && [ -e /proc/"${klog}" ]; then
        #kill -USR2 "$klog";
    #    RETVAL=$?
    #fi
    #if [ $RETVAL -ne 0 ]; then
        #failure
    #else
        #success
    #fi
    #echo    
    return $RETVAL
}
rhstatus() {
        status rsyslogd
        #status rklogd
}
restart() {
        stop
        start
}

case "$1" in
  start)
        start
        ;;
  stop)
        stop
        ;;
  restart)
        restart
        ;;
  reload|force-reload)
        reload
        ;;
  status)
        rhstatus
        ;;
  condrestart)
        [ -f /var/lock/subsys/rsyslog ] && restart || :
        ;;
  *)
        echo $"Usage: $0 {start|stop|restart|reload|force-reload|condrestart}"
        exit 2
esac

exit $?

Note: make sure SYSLOGD_OPTIONS="-c5" is set!

Configure Syslog and Rsyslog:

service syslog stop
chkconfig syslog off
chmod 755 /etc/init.d/rsyslog
chkconfig --add rsyslog
chkconfig rsyslog on

Init script is available for download on Iran Honeynet Project - Rsyslog

Rsyslog configuration

Some configurations that outline features are available in Rsyslog.com Web Site.

vi /etc/rsyslog.conf

# Input Modules -----------------------------------This line is comment
#--------------------------------------------------This line is comment
$ModLoad impstats.so
$PStatsInterval 300
syslog.info  /var/log/rsyslog-stats
#--------------------------------------------------This line is comment
$ModLoad immark.so      # provides --MARK-- message capability
$ModLoad imuxsock.so    # provides support for local system logging (via logger command)
$ModLoad imklog.so      # provides kernel logging support (previously done by rklogd) 
#--------------------------------------------------This line is comment
$ModLoad imudp.so       # provides UDP syslog reception
$UDPServerAddress *     # all local interfaces
$UDPServerRun 514       # start UDP server (log server receiver)
#--------------------------------------------------This line is comment
$ModLoad imtcp.so       # provides TCP syslog reception and GSS-API (if compiled)
$InputTCPServerRun 514  # start TCP server (log server receiver)
#--------------------------------------------------This line is comment
$ModLoad imrelp.so      # RELP input
$InputRELPServerRun 20514 # start RELP Protocol
#--------------------------------------------------This line is comment
$ModLoad imfile.so      # Text file input
$InputFileName /var/log/i-am-a-text-file.log
$InputFileTag my-text-file:
$InputFileStateFile stat-file1
$InputFileSeverity error
$InputFileFacility local7
$InputFilePollInterval 10 # check for new lines every 10 seconds
$InputRunFileMonitor
#--------------------------------------------------This line is comment
#$ModLoad imgssapi.so   # Plain TCP and GSSAPI
#$ModLoad im1395.so     # Messages via RFC1395

# Output Modules ----------------------------------This line is comment
#--------------------------------------------------This line is comment
$ModLoad omsnmp.so      # Send SNMP traps
#$actionsnmptransport udp
#$actionsnmptarget 192.168.x.x
#$actionsnmptargetport 162
#$actionsnmpversion 1
#$actionsnmpcommunity public
#*.* :omsnmp:
#--------------------------------------------------This line is comment
$ModLoad ommysql.so     # Log to MySQL
#$ModLoad ompgsql.so    # Log to PostgreSQL
#--------------------------------------------------This line is comment
$ModLoad ommail.so      # Send mail
#$ActionMailSMTPServer mail.example.net
#$ActionMailFrom rsyslog@example.net
#$ActionMailTo operator@example.net
#$ActionMailTo admin@example.net
#$template mailSubject,"disk problem on %hostname%"
#$template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"
#$ActionMailSubject mailSubject
#$ActionExecOnlyOnceEveryInterval 21600
#if $msg contains 'hard disk fatal failure' then :ommail:;mailBody
#--------------------------------------------------This line is comment
$ModLoad omrelp.so      # Send to another host via RELP
#$ModLoad omlibdbi.so   # Log via generic DB output
#$ModLoad omgss.so      # GSS enabled output

# Globals -----------------------------------------This line is comment
$umask 0000
$DirCreateMode 0640
$FileCreateMode 0640
$RepeatedMsgReduction on

$WorkDirectory /var/log/rsyslog  # default location for work (spool) files
$ActionQueueType LinkedList      # use asynchronous processing
$ActionQueueFileName queue       # set file name, also enables disk mode
$ActionResumeRetryCount -1       # infinite retries on insert failure
$ActionQueueSaveOnShutdown on    # save in-memory data if rsyslog shuts down
$MainMsgQueueMaxFileSize 100M  
$ActionQueueMaxFileSize 5M     

#--------------------------------------------------This line is comment
# Below find some samples of what a template can do. Have a good
# time finding out what they do [or just tun them] ;)

# A template that resambles traditional syslogd file output:
$template TraditionalFormat,"%timegenerated% %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n"

# a template useful for debugging format issues
$template DEBUG,"Debug line with all properties:\nFROMHOST: '%FROMHOST%', HOSTNAME: '%HOSTNAME%', PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%', APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%',\nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%',\nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\nrawmsg: '%rawmsg%'\n\n"

# A template that resembles RFC 3164 on-the-wire format:
# (yes, there is NO space betwen syslogtag and msg! that's important!)
$template RFC3164fmt,"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag%%msg%"

# a template resembling traditional wallmessage format:
$template wallmsg,"\r\n\7Message from syslogd@%HOSTNAME% at %timegenerated% ...\r\n %syslogtag%%msg%\n\r"

# The template below emulates winsyslog format, but we need to check the time
# stamps used. for now, it is good enough ;) This format works best with
# other members of the MonitorWare product family. It is also a good sample
# where you can see the property replacer in action.
$template WinSyslogFmt,"%HOSTNAME%,%timegenerated:1:10:date-rfc3339%,%timegenerated:12:19:date-rfc3339%,%timegenerated:1:10:date-rfc3339%,%timegenerated:12:19:date-rfc3339%,%syslogfacility%,%syslogpriority%,%syslogtag%%msg%\n"

# A template used for database writing (notice it *is* an actual
# sql-statement):
$template dbFormat,"insert into SystemEvents (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%',%syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",sql

$template FileFormat,"%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"

$template ForwardFormat,"<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"

# Selector lines are somewhat different from stock syslogd. With
# rsyslog, you can add a semicolon ";" after the target and then
# the template name. That will assign this template to the respective
# action. If no template name is given, a hardcoded template is used.
# If a template name is given, but the template was not defined, the
# selector line is DEACTIVATED.
#--------------------------------------------------------------------

#--------------------------------------------------This line is comment
# Forward via TCP with maximum compression:
#$AllowedSender TCP, 127.0.0.1, 192.0.2.0/24, [::1]/128, *.example.net, somehost.example.com
#*.*       @@(z9)192.168.x.x:514
# Forward via UDP with maximum compression:
#$AllowedSender UDP, 127.0.0.1, 192.0.2.0/24, [::1]/128, *.example.net, somehost.example.com
#*.*       @(z9)192.168.x.x:514
# Forward via RELP Protocol :
#*.*      :omrelp:192.168.2.4:20514;TraditionalFormat      
# Store all log files in MySQL DB  :
#*.*       :ommysql:127.0.0.1,Syslog,rsyslog,your-mysql-password
#--------------------------------------------------This line is comment


#--------------------------------------------------This line is comment
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console;TraditionalFileFormat

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog

# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                 *

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log

#--------------------------------------------------This line is comment
$IncludeConfig /etc/rsyslog.d/*.conf

#--------------------------------------------------This line is comment
#if message contains 'network error' then run the restart-network.sh shell script!!!
#:msg, contains, "network error" ^/root/restart-network.sh

Important Note: For more information please check Rsyslog.com

Rsyslog config file is available for download on Iran Honeynet Project - Rsyslog

Start Rsyslog

chmod 640 /etc/rsyslog.conf
service rsyslog start 
tail -f /var/log/messages

Test Rsyslog

logger "this is a test message" 
logger -p local0.info -t testtag "this is a test message"

Links

Iran Honeynet Project: http://www.honeynet.ir/
Rsyslog Project:http://www.rsyslog.com/
CentOS: http://www.centos.org/