There are two primary reasons for using log files. The first, monitoring system events such as hardware issues, is secure on a local machine. If your reason for monitoring log files is the second reason, to monitor security, you must ask yourself: if the system has a break-in, can I rely on the log files?

For this reason, rsyslog is often used to log information remotely. Designating a remote server with heightened security as the recipient of copies of log information is an easy way to duplicate information, provide a central repository, and improve your chances of tracing any surreptitious activity.

Consider this simple scenario

A junior system administrator makes a mistake when logged in as root. In order to cover his tracks, he simply deletes the entries in /var/log/secure that show he logged in as root. Since the wtmp log only shows direct root logins (not su logins), there is no record of a root user being logged on when the damage occurred.

This scheme might sound pretty good if you're the junior administrator, but not if you're responsible for the system. If the system had been configured to log authorization records to a secure server as well as the local system, the suspicious login could be traced.

Remote logging with rsyslog

Configuring rsyslog to log remotely consists of two parts. First, the client must have a remote log destination indicated in rsyslog's configuration files. This is simple. For example, suppose in foo.org they have designated a secure logging machine as x.foo.org. (Of course, x.foo.org should not be accessible to any but the most trusted users.) You could create an entry in the rsyslog configuration of other machines in foo.org by placing the following single-line file in /etc/rsyslog.d:

authpriv.*;auth.info        @x.foo.org

This logs all authpriv and auth messages (except auth.debug) remotely using UDP to x.foo.org. (To specify the higher overhead but more reliable protocol TCP, double the @ symbol.) For brevity, we will only cover UDP and TCP logging.

Their destination on x.foo.org is determined by the rsyslog configuration on x.foo.org (they are intermingled with the local messages of the same type on x.foo.org )

Second, on x.foo.org, you must configure rsyslogd to listen to the syslog port (udp or tcp port 514). Traditionally syslogd was always configured to listen to its port for remotely logged messages. This is not a good idea for two reasons:

• since there is no filtering of messages, this is an invitation to remote hackers to compromise a machine by logging messages repeatedly. This hack can compromise the target system in two ways:
• it can bog the system down with a lot of network traffic and syslog messages to handle
• worse, it can create huge log files that could threaten the integrity of the root filesystem if log files are not stored on a separate partition! 

• second, rsyslogd has a history of security issues that were fodder for exploits. These program flaws allow hackers to gain root access by such tricks as overflowing syslogd's message buffer. (If a C program is not written carefully, such a trick can permit the execution of code transmitted across the internet as root!) Again, only exposing rsyslogd's open port to your LAN will protect against this. You must also ensure that rsyslogd is up-to-date with security patches. (Hopefully we are nearing the end of patching these buffer overrun bugs in poorly written C code! Or at least we are nearing the time when we can rely on SELinux to intercept such breaches!)

After consideration of these two potential problems, if you decide to go ahead and enable remote logging (the receipt of remotely-logged messages), open the firewall for TCP port 514, copy the following lines from /etc/rsyslog.conf, uncomment them, place them in a file whose name ends in .conf, and drop it into /etc/rsyslog.d:

Then you must restart rsyslogd. 

systemctl restart rsyslog

(Note the confusion between rsyslog and rsyslogd. Be careful which name is used when. It is inconsistent. Why is the name for rsyslogd's startup configuration file /etc/sysconfig/rsyslog for example? Better yet, why does the rsyslogd daemon still create a file named syslogd.pid in /var/run?(we will learn about these pid files later when we cover subsystems)) Worse, why is the service named rsyslog, whereas most are named the same as the daemon? (sshd, atd, crond,...)

Excessive log traffic

As mentioned in the previous section, a common hack is to overwhelm rsyslog with log traffic. This causes problems arising from two issues:

• the time taken to execute code on rsyslog's behalf. There is not much you can do about this other than to disallow remote logging.
• the amount of data written to the system log. There are two safeguards for dealing with this
• rsyslogd will refuse to output the same message repeatedly. Instead, it will output the message once, then count the repetitions, and output a message like 'last message repeated X times' after some time interval.
• the log file growth should be constrained by placing log files on their own filesystem. Thus /var and / should never be on the same physical partition. This prevents log files from taking up all the space on the root partition and compromising the system.

Historical note:

if you are logging to a machine that is running sysklogd rather than rsyslogd (RH5 and older), you must configure your default format for forwarded messages on the client like this:

$ActionForwardDefaultTemplate RSYSLOG_TraditionalForwardFormat

Even using this default template for forwarded messages you will see that the hostname is duplicated in each message, but the message will be logged at least! (Without the directive, sysklogd will ignore the remote message as invalid.)