Nov 8, 2013

Ubuntu Fail2ban fails to parse Apache access.log

I was trying to setup Fail2ban to block Wordpress login bruteforce attacks, but Fail2ban somehow failed to parse access.log

When parsing log file with command:
fail2ban-regex /var/log/apache2/access.log /etc/fail2ban/filter.d/apache-wp-login.conf
CPU rises to 100% usage until I kill the process. I made a quick fix - workaround with redirecting needed content from access.log to another log file.

My setup is as follows:

OS: Ubuntu Server 12.04 LTS
Fail2ban: 0.8.6-3wheezy2build0.12.04.1
Python: 2.7.3-0ubuntu2.2

Fail2ban configuration:

/etc/fail2ban/fail2ban.conf
# Fail2Ban configuration file
#
# Author: Cyril Jaquier
#
# $Revision$
#
[Definition]
# Option:  loglevel
# Notes.:  Set the log level output.
#          1 = ERROR
#          2 = WARN
#          3 = INFO
#          4 = DEBUG
# Values:  NUM  Default:  3
#
loglevel = 4
# Option:  logtarget
# Notes.:  Set the log target. This could be a file, SYSLOG, STDERR or STDOUT.
#          Only one log target can be specified.
# Values:  STDOUT STDERR SYSLOG file  Default:  /var/log/fail2ban.log
#
logtarget = /var/log/fail2ban.log
# Option: socket
# Notes.: Set the socket file. This is used to communicate with the daemon. Do
#         not remove this file when Fail2ban runs. It will not be possible to
#         communicate with the server afterwards.
# Values: FILE  Default:  /var/run/fail2ban/fail2ban.sock
#
socket = /var/run/fail2ban/fail2ban.sock
We add new entry for Wordpress

Add configuration to /etc/fail2ban/jail.conf
[apache-wp-login]
enabled  = true
port     = http,https
filter   = apache-wp-login
logpath  = /var/log/apache2/apache-wp-login.log
maxretry = 3
findtime = 60
Now me make new filter for Wordpress. Some servers logs are in different format so we have to make different regex entries. These are two most common configurations:

Log format example 1:
www.domain.si:80 188.65.115.90 - - [08/Nov/2013:13:20:46 +0100] "POST /en/wp-login.php HTTP/1.1" 200 1784 "http://www.domain.si/en/wp-login.php" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1623.0 Safari/537.36"
Log format example 2:
89.222.15.152 - - [08/Nov/2013:13:40:12 +0000] "POST /wp-login.php HTTP/1.1" 200 1756 "http://domain.wordpress.com/wp-login.php" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0"
Create /etc/fail2ban/filter.d/apache-wp-login.conf

For log format example 1 use:
[Definition]
failregex = ^[^\:]+\:80 <HOST> .* "POST
ignoreregex =
ATTENTION! Don't apply this filter to default access.log, it would block all POST request, not only Wordpress. Only use it with modified log file we will create bellow.

For log format example 2 use:
failregex = <HOST>.*] "POST /wp-login.php
ATTENTION! Don't apply this filter to log format example 1, it would block your own server, because <HOST> regex would match your domain (www.domain.si).

Now we create new log file for parsing (/var/log/apache2/apache-wp-login.log). We only want to filter out POST requests for wp-login.php and write them to new log file called apache-wp-login.log.
tail --follow=name /var/log/apache2/access.log | grep --line-buffered wp-login.php > /var/log/apache2/apache-wp-login.log &
Add this command to /etc/rc.local so it would run at reboot:
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

# Fail2ban Wordpress Login
tail -f /var/log/apache2/access.log | grep --line-buffered wp-login.php > /var/log/apache2/apache-wp-login.log &

exit 0
Restart Fail2ban and you're done.
/etc/init.d/fail2ban restart