Ok first just take a breath, you are getting nothing done by worrying. This guide is meant to try and help mitigate the effects of an attack that is in progresss. I have compiled a lot of the things that I do but it is very possible I have forgotten something. Please feel free to add suggestions at the bottom if you would do it a different way or if this works good for you.
This guide is not meant to take the place of a professional looking at your server but it should give you a very good idea of what is going wrong and point you in the right direction to solving the problem. A lot of this requires you to have an idea of what is happening with your server on a normal basis so you can see what is abnormal with it. It would not be a bad idea to run a few of these tests to see what is normal under your normal busy periods.
If you have a dual processor server your server is going to start slowing down when the loads are above 4. If you only have a single processor server with no hyperthreading much above 1 and you will start having trouble. Chances are if you are looking at this you are already having trouble. If your load is not high but your server is slow it is some sort of a ping attack meant to use all of the available bandwidth.
First install bwm-ng from http://www.gropp.org/ which is a very simple way to monitor the servers bandwidth.
tar -zxf bwm-ng-0.5.tar.gz
./configure; make; make install
In the bottom right is the total transfer in KB/s. Keep in mind some providers only provide a 10mbit uplink which is only 1024KB/s. Most servers are not going to be using much more then 800-1500KB/s. This is where knowing your server comes in handy. If you know that your server normally runs at 800k/sec and you see it using 3000k/sec something is obviously wrong. If the bandwidth is only 200-300KB/s the chances are very low you are under some sort of a bandwidth DOS attack.
If this does not show anything you need to enable apache status in your httpd.conf
pico -w /etc/httpd/conf/httpd.conf
Look for the following:
Deny from all
Allow from localhost
If will probably be commented out or in some way not look like the above, make it look like that. This will only be accessible via localhost so it must be done via the httpd status command listed above.
Now we are going to look at what this all means. Here is the important part of the status:
CPU Usage: u6047.55 s364.33 cu121.44 cs19.23 – 29.3% CPU load
25.5 requests/sec – 0.7 MB/second – 28.6 kB/request
130 requests currently being processed, 63 idle servers
Another very important thing to look at is how many active connetions your server is currently processing.
netstat -n | grep :80 |wc -l
The first command will show the number of active connections that are open to your server. Many of the attacks typically seen work by starting a connection to the server and then not sending any reply making the server wait for it to time out. The number of active connections from the first command is going to vary widely but if you are much above 500 you are probably having problems. If the second command is over 100 you are having trouble with a syn attack.
Ok so now we have an idea of what is happening what to do about this. If you have a bandwidth related attack you are pretty much SOL unless your ISP filters it. Even if you block it with a firewall the traffic is still making it to your server which is going to bog it down. Imagine the ethernet cord going into your computer as a highway, once it is full there is very little you can do to go fast on it.
First make a backup copy of the apache config and start to edit it
cp /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf-GOOD
pico -w /etc/httpd/conf/httpd.conf
I would suggest commenting out the current lines in your config with a # and just adding mine right below. Though they are good for a DOS attack they are not really optimal for normal activity.
Those are the 3 main configuration options, notice how much lower they are being put. You may have to play with them around a little but those should work fine. Now if you want to adjust the number of servers it would be a good time to adjust them. This is one of those tweaking things that will really depend on how busy your server is. Assuming the server is very busy I would set it at:
If the server is not as busy you can lower the numbers to say 10/15. I would not set them much more then 10 apart unless you are sure of what you are doing. Basically this will help your server respond to a quick burst of traffic as it will not have to open up more processes.
echo 1 > /proc/sys/net/ipv4/tcp_syncookies
Like I said above it is not supposed to substitute for hiring a professional but I know plenty of people like to do it themselves and want to learn or they just don’t have the money. The things I have posted above may not help you in the end depending on the type and size of attack. There are some attacks which very little can be done other then waiting until the storm has passed. As always please post any success stories or questions