Databases, Systems & Networks

Source: defendagainstddos.wordpress.com

On the 25th of January one of my sites was struggling to stay up, my “Dos Deflate” emails were popping into my inbox at a great frequency.

A quick run of the following command:

netstat -tn 2>/dev/null | grep :80 | awk ‘{print $5}’ | cut -d: -f1 | sort | uniq -c | sort -nr | awk ‘{print $2}’

Gave me a list of addresses connected to the site. The number of addresses was way way above the number normally connected to the site.  A site which although modestly doing 100,000s of page views per day will normally have only 300-500 port 80 connections at any one instance.  This time however we had a 2 or 3 thousand.

A quick copy and paste into the following tool http://software77.net/geo-ip/multi-lookup/ (max 2000 ips per lookup) which is very usefully for bulk ip address to location lookups, yielded the following

1.26.199.253 # CN China
 1.81.157.62 # CN China
 1.83.168.34 # CN China
 1.180.57.96 # CN China
 1.26.201.167 # CN China
 1.62.140.66 # CN China
 1.189.82.78 # CN China
 1.193.76.146 # CN China
 1.26.241.209 # CN China
 1.190.209.36 # CN China
 1.189.162.213 # CN China
 1.31.57.105 # CN China
 1.62.17.196 # CN China
 1.58.137.111 # CN China
 1.83.229.4 # CN China
 1.180.10.45 # CN China
 1.180.5.200 # CN China
 1.26.169.201 # CN China
 1.180.187.10 # CN China
 1.190.2.102 # CN China
 1.25.134.125 # CN China
 1.62.161.233 # CN China
 1.182.28.181 # CN China
 (And on and on)

We clearly had an issue with China, but a quick log in to Google Analytics indicated almost no real-time traffic visiting the site.

Due to the sheer volume of connections for the next 45 mins, I was regularly running this command pasting into excel, then bulk iptabling the the ip address as well as the 65,000 neighbouring addresses.

sudo iptables -A INPUT -s 59.56.0.0/16 -m comment –comment “China IP” -j DROP
sudo iptables -A INPUT -s 1.26.0.0/16 -m comment –comment “China IP” -j DROP

etc etc.

Lire la suite…

Stopping a DDOS (distributed denial of service attack) or DOS (denial of service attack) is no simple task.  Frequently, these attacks become more than just a nuisance, they completely immobilize your server’s services and keep your users from using your website.

We’ve found a few common sense ways to help ease the pain of DDOS and/or DOS attacks.  While no method is fool proof, we certainly can minimize the profound effect these attacks have on your users and subsystems.

Identify the Source

Good luck with that one.  Many DDOS and DOS attacks are from roaming IP addresses.  A distributed denial of service attack can come from many different IP addresses and it quickly becomes impossible for the Linux system administrator to isolate and confine each IP with a firewall rule.

Wikipedia does a great job of describing the various types of attacks here: http://en.wikipedia.org/wiki/Denial-of-service_attack.  For the purpose of this tutorial, I’ll leave the research on the types of attacks up to you, and address the most common form that we’ve encountered over the years, the Apache directed DDOS or DOS attack.

Apache Based Attacks

Symptoms of the Apache DDOS or DOS attack:

• Website(s) serve slow
• You notice hanging processes
• Apache Top tells you that the same IP address is requesting a system resource
• The system resource continues to multiplex, causing more processes to spawn
• The Command:

  netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

• Says that you have a few too many connections to feel comfortable with.

The end result:

• Apache goes down
• System load goes sky high
• Server stops responding
• You cant ssh to the server node
• You’ve lost connectivity completely and a reboot is mandatory in order to restore access to the system

Preventative Measures and Counter Measures:

• Enable SYN COOKIES at the kernel level

  echo 1 > /proc/sys/net/ipv4/tcp_syncookies

• Enable and Configure iptables to prevent the attack or at least work to identify the attack

  /sbin/iptables -N syn-flood
  /sbin/iptables -A syn-flood -m limit --limit 100/second --limit-burst 150 -j RETURN
  /sbin/iptables -A syn-flood -j LOG --log-prefix "SYN flood: "
  /sbin/iptables -A syn-flood -j DROP

• Install the APF firewall to work to identify risky behavior
  • APF stands for Advanced Policy Firewall.  Its a rock solid firewall that normally plays nice with iptables.  You can grab a the most recent copy here: http://www.rfxn.com/projects/
• Install (D)DosDeflate
  • Great software, rock solid, and plays nice with either APF or iptables.  Install and configure the service in seconds using the commands below.  Edit the .conf file to utilize whichever flavor of firewall you’d like to integrate it with.  Set a few configuration settings and you’re done.
  • To Install (D)DosDeflate:

    wget http://www.inetbase.com/scripts/ddos/install.sh
    chmod 0700 install.sh
    ./install.sh

• If it doesnt workout, its simple to uninstall too.  To uninstall:

  wget http://www.inetbase.com/scripts/ddos/uninstall.ddos
  chmod 0700 uninstall.ddos
  ./uninstall.ddos

So a few tools are outlined above.  We’ve found that this will stop 90% of the attacks that are out there.  Some nice firewall rules above your server (at the router or switch level) also help.  Most of the time we can identify suspicious traffic before it even hits your servers, so a shameless plug here is probably in order.

Source: Liquid Communications

Source: Phil Chen

This how to article will go over stopping a DDoS attack when all you have access to is the targeted Linux host using netfilter and iptables. The two methods are either to simply drop packets from the offending IP/range or to only allow the offending IP/range X number of requests per second, if the range exceeds the requests per second rate traffic is dropped from the range.

*NOTE This method is for small attacks on services you are running on your Linux host. For large attacks using your gateway’s (firewall, load balancer, switch, or router) anti DDoS features maybe necessary or even having your ISP mitigating maybe the only option. I do often see attacks on HTTP from a hundred hosts or so and this article works on that scale.

Here is a example of a script for dropping packets from a offending IP/range lets say for our purposes the range is 206.250.230.0/24

#!/bin/bash
/sbin/iptables -I INPUT 1 -s 206.250.230.0/24 -j DROP
/sbin/iptables -I OUTPUT 1 -d 206.250.230.0/24 -j DROP
/sbin/iptables -I FORWARD 1 -s 206.250.230.0/24 -j DROP
/sbin/iptables -I FORWARD 1 -d 206.250.230.0/24 -j DROP

Here is a example of a script for dropping packets from a offending IP/range if it exceeds 30 requests per second lets say for our purposes the range is 206.250.230.0/24

#!/bin/bash
/sbin/iptables -I INPUT 1 -m limit --limit 30/sec -s 206.250.230.0/24 -j ACCEPT
/sbin/iptables -I INPUT 2 -s 206.250.230.0/24 -j DROP
 
/sbin/iptables -I OUTPUT 1 -m limit --limit 30/sec -d 206.250.230.0/24 -j ACCEPT
/sbin/iptables -I OUTPUT 2 -d 206.250.230.0/24 -j DROP
 
/sbin/iptables -I FORWARD 1 -m limit --limit 30/sec -s 206.250.230.0/24 -j ACCEPT
/sbin/iptables -I FORWARD 2 -s 206.250.230.0/24 -j DROP
 
/sbin/iptables -I FORWARD 1 -m limit --limit 30/sec -d 206.250.230.0/24 -j ACCEPT
/sbin/iptables -I FORWARD 2 -d 206.250.230.0/24 -j DROP

You can see your changes applied by running iptables -L command as seen below:

-bash-4.1# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  206.250.230.0/24     anywhere            limit: avg 30/sec burst 5 
DROP       all  --  206.250.230.0/24     anywhere            
 
Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             206.250.230.0/24    limit: avg 30/sec burst 5 
DROP       all  --  anywhere             206.250.230.0/24    
ACCEPT     all  --  206.250.230.0/24     anywhere            limit: avg 30/sec burst 5 
DROP       all  --  206.250.230.0/24     anywhere            
 
Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             206.250.230.0/24    limit: avg 30/sec burst 5 
DROP       all  --  anywhere             206.250.230.0/24

Source: bortzmeyer.org

% iptables -A INPUT -p tcp --dport 80 -m connlimit \
     --connlimit-above 20 -j DROP

Source: Stickpoll