I have servers hosted at Linode.com, have had them for several years now. While I am working or sleeping I rarely ever have an issue. But that doesn’t mean something isn’t happening with them. In fact, beyond the normal traffic I would expect them to get, I often get attacks via port scans, sshd login attempts, etc.
I take what some would consider above average security precautions. I’ve been called anal in the past by friends because I wouldn’t expose FTP or unnecessary protocols blindly. Instead I tell them to get an SSH tunnel client and use any service they want to through the secure pipe. Its easy enough to do even for customers from their desktops.
Besides the public SMTP/POP/IMAP and WEB, the only other major service I may expose to the WWW is SSH. Which ends up resulting in someone who doesn’t belong on my server, trying to get on it. Oh yes, I see every attempt as I use logcheck to drop me occasional emails throughout the day informing me of what is going on. To be honest, I’ve had this email log checker being sent to me for years. And I often think of the day that I will turn it off…but then the what if scenarios pop into my head, and I don’t do it.
Well I had enough with the failed sshd login attempts.
Aug 27 16:32:00 smallbox sshd: Failed password for root from 220.127.116.11 port 44533 ssh2
Aug 27 16:32:03 smallbox sshd: Failed password for root from 18.104.22.168 port 44789 ssh2
Aug 27 16:32:06 smallbox sshd: Failed password for root from 22.214.171.124 port 44974 ssh2
Aug 27 16:32:10 smallbox sshd: Failed password for root from 126.96.36.199 port 45191 ssh2
Aug 27 16:32:14 smallbox sshd: Failed password for root from 188.8.131.52 port 45456 ssh2
Aug 27 16:32:17 smallbox sshd: Failed password for root from 184.108.40.206 port 45698 ssh2
Aug 27 16:32:20 smallbox sshd: Failed password for root from 220.127.116.11 port 45919 ssh2
Aug 27 16:32:23 smallbox sshd: Failed password for root from 18.104.22.168 port 46153 ssh2
Aug 27 16:32:26 smallbox sshd: Failed password for root from 22.214.171.124 port 46379 ssh2
Aug 27 16:32:30 smallbox sshd: Failed password for root from 126.96.36.199 port 46619 ssh2
Before you say….but wait you listed their IP here! Do you really think I care? 🙂 BTW…it came from Amsterdam.
Adding fail2ban into the mix…
fail2ban will monitor events (in my case log files from auth), mark IP’s as bad for a configurable time (minutes, days) and release the jailed IP after that configurable time.
Your asking yourself, why release an IP from jail? There are several reasons. First that IP could be shared by different households where the IP owner rotates the IP’s dynamically. Second, the performance of your firewall will degrade as the list gets rather large.
Steps I took…to implement. I have my servers using debian stable, shorewall as the firewall.
1) Install fail2ban on debian:
aptitude install fail2ban
Note: There is something worth mentioning here. I run stable debian, and found that a much older version of fail2ban existed in the stable distro (0.7.5). Initially I opted to get it running, but found I had to write my own SSHD regex, and during testing their test client did not find matching values in auth.log. So I decided to upgrade to the most recent version (0.8.3). Installing a Debian package from testing in stable is not covered here. These settings below reflect the NEWER package settings.
2) Configure fail2ban with shorewall:
a) copy the /etc/fail2ban/jail.conf to /etc/fail2ban/jail.local
b) edit jail.local, leaving in for my purposes:
destemail = firstname.lastname@example.org banaction = shorewall [ssh] enabled = true port = ssh filter = sshd logpath = /var/log/auth.log maxretry = 6
The [ssh] section is not needed, as the default in jail.conf has ssh enabled, but I wanted to include it here in case a future update disabled the default for SSH checking.
The purpose of the .local file is protect your changes from the default values. It protects you from Debian’s future updates to the fail2ban package. It is important to note that the .local files are read AFTER the .conf files, thus the .local settings override the .conf settings.
If your not catching the “invalid user” lines from your auth file, adjust the regular expression as needed.
4) Ensure the service is started:
A few notes:
- Initially with the older release of fail2ban, I could not get the filter.d/sshd.conf regex to match those entries in my auth.log file. This began a lengthy process in writing a regex to find those matches in my auth.log file.
- I am doing all of this on my new MacBook. I don’t have those tools I am use to using, so first I wanted to find a decent regex editor that I could use to test. I found RegExhibit, which seemed to be the best regex editor without all the text editor functionality. And it is open source… 🙂
- It takes a while sometimes for things to happen! Be patient…and double check things while your waiting for the pot of water to boil!
I wanted to make sure everything was working before posting wrong instructions. Last night it seems someone was attempting to access my SSHD yet again. This one is quite interesting. Here is a sample message I have emailed to me when someone has failed to log in more than six times with failure:
The IP 188.8.131.52 has just been banned by Fail2Ban after
6 attempts against ssh.
Here are more information about 184.108.40.206:
OrgName: Peer 1 Dedicated Hosting
Address: 101 Marietta Street
Address: Suite 500
What I found amusing is I live near Atlanta, GA. I doubt that the Peer1.net company itself was scanning other servers out there (my server resides in Dallas, TX). It was most likely a client of theirs who has purchased a dedicated machine. Then again, I could always stop by and show them my logs and see what they know.
And here is the logcheck email that told me the were up to no good while fail2ban had an eye on them:
Aug 31 03:47:22 smallbox sshd: Failed password for root from 220.127.116.11 port 44705 ssh2
Aug 31 03:47:25 smallbox sshd: Failed password for root from 18.104.22.168 port 44961 ssh2
Aug 31 03:47:27 smallbox sshd: Failed password for root from 22.214.171.124 port 45195 ssh2
Aug 31 03:47:29 smallbox sshd: Failed password for root from 126.96.36.199 port 45409 ssh2
Aug 31 03:47:32 smallbox sshd: Failed password for root from 188.8.131.52 port 45615 ssh2
Aug 31 03:47:35 smallbox sshd: Failed password for root from 184.108.40.206 port 45857 ssh2
Aug 31 03:47:37 smallbox sshd: Failed password for root from 220.127.116.11 port 46091 ssh2
Ahhh…enough with the logs. I am quite content with shorewall, logcheck, portsentry and fail2ban. I’ll have to talk about portsentry on another day.