Managing Linux Logs with rsyslog

Linux operating systems constantly generate logs from the kernel, running services, and applications. Proper log management is vital for troubleshooting, tracking activities, and ensuring smooth operations.

rsyslog is a highly adopted logging daemon that gathers, processes, stores, and can forward logs to other destinations. It offers flexible configuration, filtering, and multiple output options, making it ideal for both simple setups and complex infrastructures.

This guide explains how to use rsyslog for viewing logs, creating custom rules, and forwarding logs to remote servers.

What is rsyslog and Why It Matters

rsyslog is a modern logging service included in most Linux distributions. It started as an improved version of syslog, offering better performance and more features. Unlike traditional syslog tools, it supports multiple input sources, advanced filtering, rule-based routing, and forwarding to remote servers. These features make it useful not only for local log management but also for building centralized logging systems where logs from many machines are collected in one place.

Key Features of rsyslog

• Modular design: Uses plugins for inputs, filters, and outputs.
• Flexible configuration: Create rules by priority, facility, program name, or message content.
• Remote forwarding: Send logs via TCP or UDP to remote servers.
• Structured data support: Works with JSON and RFC 5424 for easier analysis.
• High performance: Multi-threaded for large log volumes.
• Custom rules: Define exactly which logs are stored and where.
• Multiple inputs: Collects logs from kernel, services, apps, and custom scripts.
• Structured output: Allows formatted log storage for better readability and parsing.

How rsyslog Works: Input, Filter, and Output

rsyslog works with a pipeline consisting of three stages: Input, Filter, and Output.

• Input: Collects logs from sockets (imuxsock), kernel messages (imklog), files (imfile), or network protocols (imtcp, imudp).
• Filter: Applies rules by facility, severity, or expressions to determine how logs are handled.
• Output: Routes logs to destinations such as files, databases, remote servers, or queues, with customizable formats.

This model makes it possible to adapt logging setups for both single servers and large infrastructures.

Common Log File Locations in Linux

Linux stores logs in /var/log/. Here are common log files:

• /var/log/syslog: General system and application logs (Debian/Ubuntu).
• /var/log/messages: General-purpose system log (Red Hat, CentOS, Fedora).
• /var/log/auth.log: Authentication, logins, sudo, SSH (security monitoring).
• /var/log/kern.log: Kernel-related logs for debugging hardware and modules.
• /var/log/dmesg: Kernel boot messages, hardware initialization.
• /var/log/cron.log: Scheduled cron job execution logs.

How to View Logs with Basic Linux Commands

Before configuring rsyslog, you can inspect log files using common Linux commands:

• cat: Show file contents.

$ cat <LOGFILE_PATH>

• more: Display file content screen by screen.

$ more <LOGFILE_PATH>

• less: Scroll through logs forward and backward.

$ less <LOGFILE_PATH>

• tail: View last 10 lines of a log file.

$ tail <LOGFILE_PATH>

• tail -f: Stream new log entries in real time.

$ tail -f <LOGFILE_PATH>

• zless / zcat / zgrep: Read compressed .gz logs.

$ zless <GZ_LOGFILE_PATH>

Filtering Logs with grep, awk, and sed

Linux provides text processing utilities to filter and analyze logs:

• grep: Search for patterns or keywords.

$ grep "error" /var/log/syslog
$ grep -i "failed" /var/log/auth.log

• awk: Extract fields from structured log entries.

$ awk '{print $1, $2, $5}' /var/log/syslog

• sed: Edit or extract text blocks.

$ sed -n '/Start/,/End/p' /var/log/syslog
$ sed 's/error/ERROR/g' /var/log/syslog

By combining these tools, you can create powerful pipelines to analyze logs and detect problems such as failed logins or service errors.

Configure rsyslog Using /etc/rsyslog.conf

The primary behavior of rsyslog is defined in its main configuration file located at /etc/rsyslog.conf. This file controls how log messages are processed, filtered, and routed depending on their source (facility), importance (severity), and final destination (action).

Basic Rule Format

A rule in rsyslog.conf generally follows this structure:

FACILITY.LEVEL ACTION

• FACILITY: The origin of the log message (e.g., auth, cron, daemon, kern, mail, user).
• LEVEL: The severity of the log (e.g., debug, info, notice, warning, err, crit, alert, emerg).
• ACTION: The log’s destination (e.g., a file, remote server, or program).

Common Configuration Examples

Send Auth Logs to a Separate File:

auth.*    /var/log/auth.log

Store Kernel Logs in a Separate File:

kern.*    /var/log/kern.log

Log Only Critical Mail Errors:

mail.crit    /var/log/mail_critical.log

Log Everything Except Mail and Auth to a Common File:

*.info;mail.none;authpriv.none    /var/log/general.log

Discard Unwanted Logs:

*.debug    ~

Log User Messages to Console:

user.*    /dev/console

Add Custom Logging Rules

rsyslog allows creating custom rules to capture events for specific services or programs. This is useful to separate logs for analysis or to closely monitor critical activity.

Log SSH Activity

Create a new configuration file in /etc/rsyslog.d/:

$ sudo nano /etc/rsyslog.d/10-ssh.conf

Add the following rule:

authpriv.*    /var/log/ssh.log

Restart the rsyslog service:

$ systemctl restart rsyslog.service

Log Activity for a Custom Application

Create a new config file:

$ sudo nano /etc/rsyslog.d/20-myapp.conf

Add this rule to capture logs:

if ($programname == 'myapp') then /var/log/myapp.log
& stop

Restart rsyslog:

$ systemctl restart rsyslog.service

Forward Logs to a Remote Server

Centralizing logs simplifies monitoring across multiple systems. rsyslog can forward logs from a client machine to a remote server.

Configure the Client

Create a new configuration file:

$ sudo nano /etc/rsyslog.d/60-remote.conf

Add this rule:

*.* @@192.2.0.3:514

Use @ for UDP or @@ for TCP. Replace the IP with your server address.

Configure the Server

Edit /etc/rsyslog.conf on the server:

module(load="imtcp")
input(type="imtcp" port="514")

Optionally, store logs by host and program name:

$template RemoteLogs,"/var/log/remote/%HOSTNAME%/%PROGRAMNAME%.log"
*.* ?RemoteLogs

Restart rsyslog on both systems:

$ sudo systemctl restart rsyslog

Test the Configuration

Send a test log message:

$ logger -p local0.info "Test message from rsyslog"

Check the appropriate log file (e.g., /var/log/syslog or your defined path) to confirm receipt.

Best Practices for Using rsyslog

• Use separate files for different services or applications.
• Protect log files with secure permissions.
• Enable log rotation to prevent oversized files.
• Leverage centralized logging for multiple systems.
• Monitor logs regularly to identify and resolve issues quickly.

Conclusion

This article explained how to configure rsyslog on Linux. You learned about the /etc/rsyslog.conf structure, created custom rules for SSH and applications, forwarded logs to a remote server, and tested your setup. Following best practices ensures reliable, organized, and secure logging for better system management.