Archive

Articles taggués ‘commands’

Bash Shell Loop Over Set of Files

13/08/2020 Comments off

bash shell loopBash Shell Loop

How do I run shell loop over set of files stored in a current directory or specified directory?

You can use for loop easily over a set of shell file under bash or any other UNIX shell using wild card character.

Syntax

The general syntax is as follows:

for f in file1 file2 file3 file5
do
 echo "Processing $f"
 # do something on $f
done

You can also use shell variables:

FILES="file1
/path/to/file2
/etc/resolv.conf"
for f in $FILES
do
	echo "Processing $f"
done

You can loop through all files such as *.c, enter:

$ for f in *.c; do echo "Processing $f file.."; done

Lire la suite…

Categories: Système Tags: , , ,

Find Duplicate Files (based on size first, then MD5 hash)

12/08/2020 Aucun commentaire
Terminal – Find Duplicate Files (based on size first, then MD5 hash)
find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate
Find Duplicate Files (based on size first, then MD5 hash)

This dup finder saves time by comparing size first, then md5sum, it doesn’t delete anything, just lists them.

Terminal – Alternatives
find -type f -exec md5sum '{}' ';' | sort | uniq --all-repeated=separate -w 33 | cut -c 35-
Find Duplicate Files (based on MD5 hash)

Calculates md5 sum of files. sort (required for uniq to work). uniq based on only the hash. use cut ro remove the hash from the result.

find -type d -name ".svn" -prune -o -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type d -name ".svn" -prune -o -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate
Find Duplicate Files, excluding .svn-directories (based on size first, then MD5 hash)

Improvement of the command “Find Duplicate Files (based on size first, then MD5 hash)” when searching for duplicate files in a directory containing a subversion working copy. This way the (multiple dupicates) in the meta-information directories are ignored.

Can easily be adopted for other VCS as well. For CVS i.e. change “.svn” into “.csv”:

find -type d -name ".csv" -prune -o -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type d -name ".csv" -prune -o -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate

find -not -empty -type f -printf "%-30s'\t\"%h/%f\"\n" | sort -rn -t$'\t' | uniq -w30 -D | cut -f 2 -d $'\t' | xargs md5sum | sort | uniq -w32 --all-repeated=separate
Find Duplicate Files (based on size first, then MD5 hash)

Finds duplicates based on MD5 sum. Compares only files with the same size. Performance improvements on:

find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separateThe new version takes around 3 seconds where the old version took around 17 minutes. The bottle neck in the old command was the second find. It searches for the files with the specified file size. The new version keeps the file path and size from the beginning.

find . -type f -exec md5 '{}' ';' | sort | uniq -f 3 -d | sed -e "s/.*(\(.*\)).*/\1/"
Find Duplicate Files (based on MD5 hash) — For Mac OS X

This works on Mac OS X using the `md5` command instead of `md5sum`, which works similarly, but has a different output format. Note that this only prints the name of the duplicates, not the original file. This is handy because you can add `| xargs rm` to the end of the command to delete all the duplicates while leaving the original.

Categories: Système Tags: ,

Typical iptables

11/08/2020 Aucun commentaire
# Modify this file accordingly for your specific requirement.
# http://www.thegeekstuff.com
# 1. Delete all existing rules
iptables -F

# 2. Set default chain policies
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT DROP

# 3. Block a specific ip-address
#BLOCK_THIS_IP="x.x.x.x"
#iptables -A INPUT -s "$BLOCK_THIS_IP" -j DROP
 Lire la suite...
Categories: Réseau, Sécurité, Système Tags: , , , ,

(D)DoS Deflate

10/08/2020 Aucun commentaire

About

(D)DoS Deflate is a lightweight bash shell script designed to assist in the process of blocking a denial of service attack. It utilizes the command below to create a list of IP addresses connected to the server, along with their total number of connections. It is one of the simplest and easiest to install solutions at the software level.

netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

IP addresses with over a pre-configured number of connections are automatically blocked in the server’s firewall, which can be direct iptables or Advanced Policy Firewall (APF). (We highly recommend that you use APF on your server in general, but deflate will work without it.)

Notable Features

  • It is possible to whitelist IP addresses, via /usr/local/ddos/ignore.ip.list.
  • Simple configuration file: /usr/local/ddos/ddos.conf
  • IP addresses are automatically unblocked after a preconfigured time limit (default: 600 seconds)
  • The script can run at a chosen frequency via the configuration file (default: 1 minute)
  • You can receive email alerts when IP addresses are blocked.

Installation

wget http://www.inetbase.com/scripts/ddos/install.sh
chmod 0700 install.sh
./install.sh

Uninstallation

wget http://www.inetbase.com/scripts/ddos/uninstall.ddos
chmod 0700 uninstall.ddos
./uninstall.ddos

Questions?

Although most things are explained on this page, if you have any further questions, you may contact the original developer of the script, Zaf.

Categories: Réseau, Système Tags: ,

cssh: un outil pour les paresseux…

06/08/2020 Aucun commentaire

Cluster ssh

Cluster ssh

Cluster ssh est un outil merveilleux ! Combien de fois ai-je du retaper les mêmes commandes dans deux fenêtres shell différentes sous prétexte que les serveurs faisaient la même chose… Je pense à mes deux serveurs DNS, par exemple.

Évidemment, il faut faire attention à ce qu’on tape car la moindre erreur est reproduite en double… Mais quel bonheur de faire deux fois le travail et surtout d’avoir des configurations identiques quand on le désire.

Autre avantage: si on ouvre plusieurs sessions ssh dans la même commande shell, cssh redimensionne les fenêtres automatiquement.

Attention: si vous ajouter des fenêtres après coup sur Ubuntu, il les superposera sur les précédentes ce qui n’est pas folichon… Il faut prévoir votre nombre de session au départ. Par contre, sur Mac OS X, l’excellent csshX fait le boulot correctement.

Categories: Système Tags: ,