Here I want to show how a centralized logserver could be setup. The logdata is stored in flat files and in a mysql
database.
One benefit of such a solution is that one can use a central server which checks logs of the whole network for violations. This can be done with fail2ban
, a very handy tool for such stuff.
Another benefit of a centralized logserver is that it's much harder for an attacker to clean his/her traces in the logs of the attacked system.
syslog-ng
(it will work with other loggers like rsyslog
, but I prefer syslog-ng
fail2ban
(on the logserver, most respositories have it)mysql-server
and mysql-client-libraries
(on the logserver)
On the logserver (in the follwing called ServerLocal) one has to install the required packages. Preferably via the systems package management. In my case - as it's Debian - apt-get
apt-get update apt-get install fail2ban mysql-server syslog-ng libmysqlclient18 mysql-client
On the other server, which should send its logs to ServerLocal (called ServerRemote in the following) only one package is needed
apt-get update apt-get install syslog-ng
First we have a look at syslog-ng
on ServerRemote. The configuration has to be changed so that syslog-ng
sends the logs to a remote destination
With Debian it's /etc/syslog-ng/syslog-ng.conf
Add a new destination for the remote logs
destination d_net { udp("ServerLocal" port(514)); };
this defines destination named d_net
, defines that UDP is used and defines that the connection has to be made to ServerLocal and port 514.
Instead of the hostname for ServerLocal I would suggest to use the IP address. Otherwise if DNS is in trouble the destination might cannot be reached anymore.
Next add a „target“ for the local logs that point to the new destination
log { source(s_src); destination(d_net); };
s_src
is default and should already be defined in syslog-ng.conf. At least with Debian it's already there.
.
With ServerLocal (the logserver) there must be changed some more stuff.
First we need to define a source which listens on UDP port 514 and receives remote logmessages
source s_external { udp ( ip(0.0.0.0) port(514)); };
s_external
binds to any ipaddress available to ServerLocal on UDP port 514. One can also specify a specific IP address to listen on
Then we add a destination for mysql
destination d_mysql { program("/usr/bin/mysql --user=USERNAME --password=PASSWORD DATABASE" template("INSERT INTO TABLE (host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC','$PROGRAM', '$MSG' );\n") template-escape(yes)); };
change USERNAME, PASSWORD, DATABASE and TABLE to your environment!
Next we need another destination because the logs should not only be written to mysql but also to flat files on ServerLocal
destination d_hosts { file("/var/log/ext/$HOST/$FACILITY.log" owner(root) group(root) perm(0600) dir_perm(0751) create_dirs(yes) ); };
create /var/log/ext manually.
Last we create a instruction how the remote logs should be processed
log { source(s_external); destination(d_hosts); destination(d_mysql); };
so s_external
will be sent to d_hosts (flat files) and to d_mysql (mysql).
So we need to create the database/table that is needed for the loging
mysql -uroot -pyourMysqlRootPassword CREATE DATABASE syslog; CREATE TABLE `logs` ( `host` varchar(32) DEFAULT NULL, `facility` varchar(10) DEFAULT NULL, `priority` varchar(10) DEFAULT NULL, `level` varchar(10) DEFAULT NULL, `tag` varchar(10) DEFAULT NULL, `datetime` datetime DEFAULT NULL, `program` varchar(32) DEFAULT NULL, `msg` text, `seq` bigint(20) unsigned NOT NULL AUTO_INCREMENT, PRIMARY KEY (`seq`), KEY `host` (`host`), KEY `program` (`program`), KEY `datetime` (`datetime`), KEY `priority` (`priority`), KEY `facility` (`facility`), ) ENGINE=MyISAM DEFAULT CHARSET=utf8; CREATE USER 'mysqlUser'@'localhost' IDENTIFIED BY 'yourPassword'; GRANT ALL on syslog.* TO 'mysqlUser'@'localhost'; FLUSH PRIVILEGES; \q
change mysqlUser
and yourPassword
to fit your environment. But these values must be identical to the valies you provided in syslog-ng.conf on ServerLocal
I used MyISAM
as table engine because my script that displays the messages from logs does fulltext searches on logmessage. This requires fulltext indecies which so far are only supported in MyISAM and not in InnoDB (at least not in the version that debian actually provides)
Ensure that the user you just created is allowed to access the database
mysql -umysqlUser -pmysqlPassword
If the test went well restart syslog-ng
first on ServerLocal then on ServerRemote
service syslog-ng restart #or /etc/init.d/syslog-ng restart
If everything went well then you should see new logiles on ServerLocal below /var/log/ext/
Check as well that the data has been written to mysql
Finally we setup fail2ban
so it monitors the new logfiles. I do not want to show the basis configuration of fail2ban
as there are good tutorials and HowTos in web. Just ask Google
fail2ban
knows a wildcard feature which makes it posssible to easily monitor logfiles from many remote servers
This example monitors the auth.log
files of the remote systems
[ssh] enabled = true filter = sshd logpath = /var/log/ext/*/auth.log maxretry = 3 port = all
So all auth.log
are monitored. fail2ban
will start to fill its block chains. So these can be dumped regulary and distributed to remote systems.
This can quite easy be done via a shell script with some iptables
and ssh
. But more about this another time