Install fail2ban

Published on: 22nd Apr 2014

Updated on: 30th Jan 2022

What is fail2ban

fail2ban is a service program (i.e., daemon) that is looking for the unauthorized user who is trying to access the server or some misbehave activities by reviewing the system log, user access log (/var/log/auth.log) or even Nginx web access log (/var/log/nginx/access.log).

Why fail2ban

Since our server is facing the Internet, this means anyone with the correct SSH ID and password will be able to access the server from anywhere. In this case, it will attract someone to try hacking the server by guessing the ID and password. With this consideration, we need to add another layer of protection after the firewall.

Another use case is that if a web crawler is trying to scan the website backdoor and we should block those web crawlers.

Here's how you install and configure it

To install fail2ban,
```
sudo apt-get install fail2ban
```
The following package is for saving the firewall policy
```
sudo apt-get install iptables-persistent
```
Then, review the default configuration by open the config file,
```
sudo nano /etc/fail2ban/jail.conf
```
Change the following setting in the configuration file and save it.
```
ignoreip = 127.0.0.1/8  
bantime = 10m
```

For the valid time unit,

s    seconds = 1
m    minutes = 60
h    hours = 3600
d    days = 86400
w    weeks = 604800

Add a new configuration file for your server.
```
sudo nano /etc/fail2ban/jail.local
```

The local jail config file should have the following settings:

[sshd]
enabled = true

# incremental banning time
bantime.increment = true
bantime.factor = 1
bantime.formula = ban.Time * (1<<(ban.Count if ban.Count<20 else 20)) * banFactor

Customizing fail2ban filter/policies

fail2ban allows you to add a new filter. I found the an article (https://david.ramsden.cloud/2016/03/21/banning-repeat-offenders-with-fail2ban/ that shows the details on how to create new filter. You may want to do this to automate the battle with the hacking bots.

Edit /etc/fail2ban/jail.local file by adding the following

# the filter will be created below.
[repeat-offender]
enabled = true
filter = repeat-offender
port = all
banaction = iptables-allports
logpath = /var/log/fail2ban.log
# Repeat offender if previously banned 3 times within 5 hours.
maxretry = 3
findtime = 5h
# Ban for 48 hours.
bantime = 48h

Create this new file: /etc/fail2ban/filter.d/repeat-offender.conf,

With the following contents,

[Definition]
failregex = fail2ban\.actions\[\d+\]: WARNING \[.*\] Unban <HOST>$
ignoreregex = fail2ban\.actions\[\d+\]: WARNING \[repeat-offender\].*$

Customized filter in fail2ban #2

After I have enabled the SSH key, some guys are still trying their luck to access the server through SSH. I found many of the following lines in the /etc/log/auth.log file,

...
Nov 29 20:29:48 myserver sshd[62008]: Received disconnect from 104.248.143.188 port 38968:11: Normal Shutdown, Thank you for playing [preauth]
...
Dec  1 03:22:20 myserver sshd[69945]: Unable to negotiate with 23.239.13.194 port 48784: no matching key exchange method found. Their offer: diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1 [preauth]
...
Dec  6 16:29:18 myserver sshd[117410]: Invalid user admin from 141.98.9.167 port 33901
...
Dec  6 16:29:10 myserver sshd[117402]: Connection closed by authenticating user root 141.98.9.163 port 44229 [preauth]
...
Dec  7 09:39:14 myserver sshd[121992]: Connection closed by 212.109.195.222 port 52660 [preauth]
...

So, I decided to automate the banning on these IP addresses.

Create this new file: /etc/fail2ban/filter.d/my-filter.conf

[Definition]
failregex = Received disconnect from <HOST>.*Normal Shutdown, Thank you for playing \[preauth\]$
        Unable to negotiate with <HOST>.*no matching key exchange method found. Their offer.*
        Invalid user .*from <HOST> port .*
        Connection closed by .*<HOST> port .* \[preauth\]$

ignoreregex =

Edit /etc/fail2ban/jail.local by adding the following to the bottom of the file

[my-filter]
enabled = true
filter = my-filter
port = all
banaction = iptables-allports

logpath = /var/log/auth.log

maxretry = 5
findtime = 5h
bantime = 48h

Save the changes.
To test the new filter:
```
fail2ban-regex /var/log/auth.log /etc/fail2ban/filter.d/my-filter.conf
```
and you will find the result in the console.

In case there are zero matches, it means that the failregex is not functioning properly. In this case, you may refer to the existing filter files in /etc/fail2ban/filter.d directory.
Finally, restart fail2ban.

Customized filter in fail2ban #3

We are hosting our website (https://ciysys.com) on a cloud server and we don't want the "crazy" web crawler that consumes too much of the bandwidth. After some research on the Internet, we found out that we can block the web crawler by using fail2ban.

For example, we don't want to let 'petalbot' crawl our website at all. In this case, we reviewed the web access log file (/var/log/nginx/access.log) and found the following

...
114.119.136.64 - - [13/Dec/2020:03:33:07 +0000] "GET /robots.txt HTTP/1.1" 301 178 "-" "(compatible;PetalBot;+https://aspiegel.com/petalbot)"
...

So, I decided to create a new fail2ban filter to handle this automatically,

Create this new file: /etc/fail2ban/filter.d/my-badbots.conf. Basically, the failregex is able to catch multiple bots that is separated with pipe symbol ("|").
```
[Definition]
failregex = <HOST>.*(GET|POST|HEAD).*(PetalBot|otherBots).*

ignoreregex =
```

Edit /etc/fail2ban/jail.local by adding the following to the bottom of the file,

[my-badbots]
enabled = true
filter = my-badbots
port = all
banaction = iptables-allports  
logpath = /var/log/nginx/access.log
maxretry = 1
findtime = 10m
bantime = 30d

Save the changes.

To test the new filter:

fail2ban-regex /var/log/nginx/accesslog /etc/fail2ban/filter.d/my-badbots.conf

and you will find the result in the console.

Finally, restart fail2ban.

`fail2ban` operating commands

Finally, restart the service
```
sudo systemctl restart fail2ban
```
To check status
```
sudo fail2ban-client status
```
Manually ban an IP in 'repeat-offender' jail (or any jail that you have setup).
```
sudo fail2ban-client set repeat-offender banip 87.251.77.206
```

To view the given jail,

sudo fail2ban-client status repeat-offender

View the fail2ban log file,
```
sudo tail /var/log/fail2ban.log
```
Basically, fail2ban analyze SSH access in the following file,
```
sudo tail /var/log/auth.log
```
To confirm the "unknown user id" has been banned,
```
sudo iptables -S | grep f2b
```

References

For more details about fail2ban: https://www.fail2ban.org/wiki/index.php/MANUAL_0_8#Definitions
How To Protect SSH with Fail2Ban on Ubuntu: https://www.digitalocean.com/community/tutorials/how-to-protect-ssh-with-fail2ban-on-ubuntu-14-04

Jump to #UBUNTU blog

Author

Lau Hon Wan, software developer.