Installing LogAnalyzer and rsyslog on CentOS

Instructions on how to set up Linux modules needed to get a LogAnalyzer log aggregation/analysis server up and running and collecting logs.

Prerequisites

These instructions are specific to CentOS 6.2. If you are using a different distro, many of the installation commands and paths to files will be different from what I've documented below. I strongly suggest that you document the steps to perform a similar install for your distro.

You will need to install the prerequisites by using the following commands:

yum install httpd
yum install mysql
yum install mysql-server
yum install php
yum install php-mysql
yum install php-gd
yum install rsyslog
yum install rsyslog-mysql
/usr/bin/updatedb 

The '/usr/bin/updatedb' command updates the file index so that the 'find' and 'locate' commands work properly. If you've already properly set up your system to index the files daily, this will be unnecessary.

If your distro of Linux is using a different syslog server such as syslog-ng or sysklogd, you'll need to remove it.

MySQL

Set up MySQL

/sbin/chkconfig --levels 235 mysqld on
/etc/init.d/mysqld start
/usr/bin/mysql_secure_installation

Hit enter key after last command has run since no password has yet been set for root MySQL account. Hit 'y' and enter when asked to set up a root password and type in a strong password. Hit 'y' and enter for the following questions: "Remove anonymous users?", "Disallow root login remotely?", "Remove test database and access to it?", and "Reload privilege tables now?"

Set up database and tables

Create the user/database/table and table schema:

Log in to mysql:

mysql -u root -p

Create a user:

CREATE USER rsyslog;
SET PASSWORD FOR rsyslog= PASSWORD('yourpasswordgoeshere');

Set up database and table schema:

CREATE DATABASE rsyslogdb;
USE rsyslogdb;

Paste contents below to mysql to set up the schema:

CREATE TABLE SystemEvents
(
 ID int unsigned not null auto_increment primary key,
 CustomerID bigint,
 ReceivedAt datetime NULL,
 DeviceReportedTime datetime NULL,
 Facility smallint NULL,
 Priority smallint NULL,
 FromHost varchar(60) NULL,
 Message text,
 NTSeverity int NULL,
 Importance int NULL,
 EventSource varchar(60),
 EventUser varchar(60) NULL,
 EventCategory int NULL,
 EventID int NULL,
 EventBinaryData text NULL,
 MaxAvailable int NULL,
 CurrUsage int NULL,
 MinUsage int NULL,
 MaxUsage int NULL,
 InfoUnitID int NULL ,
 SysLogTag varchar(60),
 EventLogType varchar(60),
 GenericFileName VarChar(60),
 SystemID int NULL
);
CREATE TABLE SystemEventsProperties
(
 ID int unsigned not null auto_increment primary key,
 SystemEventID int NULL ,
 ParamName varchar(255) NULL ,
 ParamValue text NULL
);

Next, we need to grant permissions to the rsyslog account we created earlier:

GRANT ALL PRIVILEGES ON `rsyslogdb`.* TO 'rsyslog'@'%' IDENTIFIED BY 'yourpasswordgoeshere';
flush privileges;

Leave MySQL:

exit

Configure rsyslog

Setting up

How to configure rsyslog:

nano /etc/rsyslog.conf

Make your #### Modules #### section the same as the following:

#### MODULES ####

$ModLoad ommysql        # provides support for MySQL
$ModLoad imuxsock.so    # provides support for local system logging (e.g. via logger command)
$ModLoad imklog.so      # provides kernel logging support (previously done by rklogd)
#$ModLoad immark.so     # provides --MARK-- message capability

# Provides UDP syslog reception
$ModLoad imudp.so
$UDPServerRun 514

# Provides TCP syslog reception
$ModLoad imtcp.so
$InputTCPServerRun 514

Just above ### begin forwarding rule ### section add info similar to the following line to limit IP addresses that can send syslog info to the server, for each class C subnet the server will be collecting from, you'll need to enter the subnet info followed by /24 (such as 172.18.22.0/24) to allow that subnet to send syslog data. Alternatively, you can limit by single IP addresses. The 127.0.0.1 is necessary so the server can send logs to itself:

$AllowedSender TCP, 127.0.0.1, 172.18.22.0/24
$AllowedSender UDP, 127.0.0.1, 172.18.22.0/24

Add the following line to the ### begin forwarding rule ### section. Replace the "<yourrsyslogpasswordhere>" bit with the password you set for rsyslog MySQL user above:

*.*       :ommysql:127.0.0.1,rsyslogdb,rsyslog,<yourrsyslogpasswordhere>

When done modifying the file, hit Ctrl+x, then y and then enter to save the file.

Restart the rsyslog service:

service rsyslog restart

Test rsyslog

Check if messages are arriving at the syslog server:

tail -f /var/log/messages

Check if messages are being stored in mysql database:

mysql -u root -p
use rsyslogdb;
select * from SystemEvents;

If you see anything other than “empty set” it’s working. Exit out of MySQL:

exit

Configure Apache

Configure CentOS to start the web server at bootup and manually start the service:

chkconfig --levels 235 httpd on
service httpd start

modify 2 lines to match your server's respective ip and fqdn in /etc/httpd/conf/httpd.conf

nano /etc/httpd/conf/httpd.conf
from:
Listen 80
to:
Listen ip.address.of.server:80
and from:
#ServerName www.example.com:80
to:
ServerName fully.qualified.domian.name:80

Hit CTRL+x, then Y and then enter to save and exit the file.

Restart the server:

/etc/init.d/httpd restart

Set up IPTables

Edit the iptables file:

nano /etc/sysconfig/iptables

Add these lines to the /etc/sysconfig/iptables file (before the COMMIT line):

-I INPUT -p tcp --dport 80 -m state --state NEW,ESTABLISHED -j ACCEPT
-I OUTPUT -p tcp --sport 80 -m state --state ESTABLISHED -j ACCEPT

You'll need to enter lines similar to the following based on your network environment. For more info on how to use IPTables in CentOS see http://wiki.centos.org/HowTos/Network/IPTables:

-I INPUT -p tcp --dport 514 -s 172.18.22.0/24 -j ACCEPT
-I INPUT -p udp --dport 514 -s 172.18.22.0/24 -j ACCEPT

Restart the network service and IPTables:

/etc/init.d/network restart
/etc/init.d/iptables restart

Configure LogAnalyzer

Install LogAnalyzer

Check for the latest stable release by going to http://loganalyzer.adiscon.com/downloads in a browser. Current latest release is http://loganalyzer.adiscon.com/downloads/loganalyzer-3-4-2-v3-stable

Download it on your CentOS server by doing the following:

cd /tmp
wget http://download.adiscon.com/loganalyzer/loganalyzer-3.4.2.tar.gz

Uncompress the file:

tar -xvzf loganalyzer-3.4.2.tar.gz

Copy the source directory to the Apache html directory and create config.php file:

cd loganalyzer-3.4.2/src
rm -R -f /var/www/html
mkdir /var/www/html
cp -R * /var/www/html
cd /tmp/loganalyzer-3.4.2/contrib/
cp * /var/www/html
cd /var/www/html
chmod +x configure.sh secure.sh
./configure.sh

The last line will create a blank “config.php” file, and will give everyone write access to it. It won´t generate any output.

Check if the config.php file has been created (initial setup via browser will make changes to this file):

ls

Create LogAnalyzer MySQL user and database:

mysql -u root -p
create database loganalyzerdb;
CREATE USER loganalyzer;
SET PASSWORD FOR loganalyzer= PASSWORD('yourpasswordgoeshere');
GRANT ALL PRIVILEGES ON `loganalyzerdb`.* TO 'loganalyzer'@'%' IDENTIFIED BY 'yourpasswordgoeshere';
flush privileges;
exit

Initial setup of Log Analyzer, Step One:

On a client system go to the Log server's URL using a web browser (http://yoursystemnamehere.blah.org).

A message stating "Critical Error Occurred: Error main configuration file is missing! Click here to install Adiscon LogAnalyzer!" will appear in browser. Click on the word "here" to start the install.

Click "Next" twice and you should get to the "Basic Configuration" screen. The recommend settings are:

• Number of syslog messages per page: 200 (set this lower if the log server is on a slow system)
• Message character limit for main view: 80 (default)
• Character display limit for all string fields: 80
• Show message details popup: Yes (default)
• Automatically resolved IP Addresses (inline): Yes (default)
• Enable User Database: Yes
• Database Host: localhost (default)
• Database Port: 3306 (default)
• Database Name: loganalyzerdb
• Table prefix: logcon_
• Database User: loganalyzer
• Database Password: <enter in the loganalyzer database user password that you set earlier here>
• Require user to be logged in: Yes

Click "Next".

Initial setup of Log Analyzer, Step Two:

Click "Next" on the "Create Tables" page, then click "Next" on the "Check SQL Results" page and then set up the admin user:

• Username: <enter in the username here that you want>
• Password: <enter in the user password that you want to use>
• Repeat Password: <re-enter in the user password that you want to use>

Click "Next".

Initial setup of Log Analyzer, Step Three:

The recommended settings for the "Create the first source for syslog messages" page are:

• Name of the source: All Syslog Sources
• Source type: MySQL Native
• Select view: Syslog Fields (default)
• Table type: MonitorWare
• Database host: localhost (default)
• Database name: rsyslogdb
• Database table name: SystemEvents
• Database user: rsyslog
• Database password: <enter in the rsyslog database user password that you set earlier here>
• Enable row counting: "Yes"

Click "Next" and then click "Finish".

The install of LogAnalyzer has now been completed. Now other users can be created and there are many settings that can be tweaked as needed.

Point all of the syslog capable devices to the new log server and begin analyzing the aggregated logs.

References

The following sites were used to help figure this all out

http://en.tiagomarques.info/2011/07/centos-syslog-server-rsyslog-mysql-and-loganalyzer/

http://www.pantz.org/software/mysql/mysqlcommands.html

http://pkgs.fedoraproject.org/repo/pkgs/phplogcon/README.fedora/5aa1ea186764ba0a7ea239131141734a/README.fedora

http://www.linuxhelp.in/2010/10/how-to-configure-syslog-server-or.html

http://www.beguelin.com/2009/05/locate-and-updatedb-on-centos.html

Chris Borte's brain

Comments

[Marcin]

This is very sweet tutorial, can you also use splunk with this setup?

[Marcin]

Why are you deleting my comment? I have asked for Splunk integration with it. Will it work?

[Alex]

Hello Marcin,

I didn't delete your comments, they just don't get published automatically. Most of the comments I get are spam so I have to resort to manually controlling comments.

Yes, you should be able to pretty easily get this to work with Splunk as well by also forwarding logs from this box to your Splunk box by adding an additional line on your rsyslog.conf.

You'd have to look up the exact syntax, but it's something like:

*.* @@servername:514

Just add that after the other *.* line that sends everything to the MySQL database.

[mgalvez]

After i install yum install rsyslog-mysql and reboot centos, it is not able to boot up again, and the virtual machine got stuck trying to start up NFS, what can i do????

[Chris Noffsinger]

Alex, Thank you so much for this. Adiscon should put this freakin guide on their page.

[joe]

cannot get the loganalyzer installed. Not sure whats going on. Anyone else run into issues with that part?

[joe]

Seems like its as simple as opening a browser, but maybe i am doing something wrong.

[Marcel]

Really nice Howto. Do you have experience with the scalability? How many clients are logging to this server?

Maybe you can use an own xfs formatted partition/hdd for the collected data to boost up I/O.

eg:
mkfs.xfs -l internal,size=64m -d agcount=45 /dev/sdX

/etc/fstab:
/dev/sdX /var/lib/mysql xfs noatime,nodiratime,nobarrier,logbufs=8,allocsize=128m 0 2

[Sean Pattee]

I used this writeup to install on CentOS 6.3

After getting stuck with errors authenticating while trying to "TEST RSYSLOG tail -f /var/log/messages", I was able to finish the install by changing the following line in the writeup:

GRANT ALL PRIVILEGES ON `rsyslogdb`.* TO 'rsyslog'@'%' IDENTIFIED BY 'yourpasswordgoeshere';

to read:

GRANT ALL PRIVILEGES ON rsyslogdb.* TO rsyslog@localhost IDENTIFIED BY 'yourpasswordgoeshere';

Other than that this writeup was 100% for me. Maybe this is common sense to a Linux/MySQL guy but it had me stumped for a day.

[Alex]

Hi Sean,

Looks like I screwed up on the single quotes. There's a mix of ` and ' in my instructions. I'll have to double check and see which it needs. I had problems getting that part to work without the single quotes/backtics, whichever it was that was needed.

Glad it worked for you without the ' or `.

Regards,

Alex

[ian]

Great tutorial. Very clear! Yesterday I struggled for about 3 hours trying to install on loganalyzer centos 5.6 and failed. With your tutorial I managed it in an hour! Thanks very much! btw is it possible to install loganalyzer in centos 5.6?

[Marco]

Hello Alexander,
I've just installed everything but after that occurred an error: No syslog records found.
Please, can you help me?

Thanks for your attention

[dmi]

Thank you very much for the tutorial. I've never seen so clean, error free and details info. Again thank you.

[Diego Villa]

The Logs of swicht and router are configured to reach the server but they do not reach me, I just arrive server logs.

What should I do to reach me swicht logs, Router and Cisco ASA?