Tag: Security

Here I am, peacefully working at my computer when I’m interrupted by a text message on my phone:

** PROBLEM alert – someserver.somewhere.com/SSH is CRITICAL **

That’s not a good thing. Hoping it’s a false alert, I try to ssh in. No luck. I try again. Still no luck. And a third time, Yes! I’m in. Let the troubleshooting begin.

I check to see what processes are running and sure enough, I find a culprit. There are a ton of sshd processes going. I take a look at auth.log and it’s full of “Failed password for root from 218.145.160.100 port 55739 ssh2” messages (about 9,000 of them.) Here’s what’s going on: someone is trying to login to the server most likely by trying a bunch of passwords in a brute force attack. A brute force attack consists of trying every possible password until you find the right one. The attack doesn’t really concern me since I don’t allow password logins on most of the servers I manage. The excessive login attempts are a little annoying.

One command later and all traffic from that IP address drops into oblivion.

iptables -A INPUT -s 218.145.160.100 -j DROP

With that band-aid applied, it’s time to get something better in place for the long term. A while back there was some discussion about preventing or slowing down such attacks on the SLLUG email list and some people posted scripts they use to deal with it. Here is my current version of one of those scripts:

#!/bin/bash
case "$1" in
start)
# Put IP addresses for allowed hosts into this, separated by spaces.
SSH_ALLOWED="123.45.67.89 98.76.54.32"

iptables -A INPUT -p icmp -m state --state NEW,ESTABLISHED --icmp-type echo-request -m limit --limit 2/s -j ACCEPT
iptables -A INPUT -s 127.0.0.1 -j ACCEPT

# Allow TCP/UDP connections out. Keep state so conns out are allowed back in.
iptables -A INPUT -p tcp -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -p udp -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p udp -m state --state NEW,ESTABLISHED -j ACCEPT

# Allow ICMP out and anything that went out back in.
iptables -A INPUT -p icmp -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p icmp -m state --state NEW,ESTABLISHED -j ACCEPT

#put any custom rules for you rserver in this section
iptables -A INPUT -s 218.145.160.100 -j DROP
iptables -A INPUT -p tcp -m tcp --dport 111 -j REJECT
iptables -A INPUT -p tcp -m tcp --dport 11211 -j DROP
iptables -A INPUT -p icmp -j DROP
iptables -A INPUT -p udp -j DROP

#now for the ssh stuff
iptables -N SSH_Brute_Force
iptables -F SSH_Brute_Force
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -j SSH_Brute_Force
for IP in $SSH_ALLOWED; do
iptables -A SSH_Brute_Force -s $IP -j RETURN
done
iptables -A SSH_Brute_Force -m recent --name SSH --set --rsource
iptables -A SSH_Brute_Force -m recent ! --rcheck --seconds 60 --hitcount 5 --name SSH --rsource -j RETURN
iptables -A SSH_Brute_Force -j LOG --log-prefix "SSH Brute Force Attempt: "
iptables -A SSH_Brute_Force -j DROP

;;
stop)
iptables -F
iptables -X SSH_Brute_Force
;;

*)
echo "Usage: $0 {start|stop}" >&2
exit 1
;;
esac

This is an init script, so I put it in my /etc/init.d directory and set it up to run when the server boots up. What it does is only allows 5 SSH connection attempts per minute based on the source’s IP address. From there, it blocks and logs any connection attempts. Two words of warning when working with IP tables; be careful. It’s very easy to block yourself from accessing your own server. I’ve done this more times than I care to mention and had to take a drive to the datacenter or call their helpdesk to make things available again.