Articles taggués ‘shell’

Find Duplicate Files (based on size first, then MD5 hash)

14/05/2021 Aucun commentaire
Terminal – Find Duplicate Files (based on size first, then MD5 hash)
find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate
Find Duplicate Files (based on size first, then MD5 hash)

This dup finder saves time by comparing size first, then md5sum, it doesn’t delete anything, just lists them.

Terminal – Alternatives
find -type f -exec md5sum '{}' ';' | sort | uniq --all-repeated=separate -w 33 | cut -c 35-

Calculates md5 sum of files. sort (required for uniq to work). uniq based on only the hash. use cut ro remove the hash from the result.

find -type d -name ".svn" -prune -o -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type d -name ".svn" -prune -o -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate

Improvement of the command “Find Duplicate Files (based on size first, then MD5 hash)” when searching for duplicate files in a directory containing a subversion working copy. This way the (multiple dupicates) in the meta-information directories are ignored.

Can easily be adopted for other VCS as well. For CVS i.e. change “.svn” into “.csv”:

find -type d -name ".csv" -prune -o -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type d -name ".csv" -prune -o -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate

find -not -empty -type f -printf "%-30s'\t\"%h/%f\"\n" | sort -rn -t$'\t' | uniq -w30 -D | cut -f 2 -d $'\t' | xargs md5sum | sort | uniq -w32 --all-repeated=separate

Finds duplicates based on MD5 sum. Compares only files with the same size. Performance improvements on:

find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separateThe new version takes around 3 seconds where the old version took around 17 minutes. The bottle neck in the old command was the second find. It searches for the files with the specified file size. The new version keeps the file path and size from the beginning.

find . -type f -exec md5 '{}' ';' | sort | uniq -f 3 -d | sed -e "s/.*(\(.*\)).*/\1/"

This works on Mac OS X using the `md5` command instead of `md5sum`, which works similarly, but has a different output format. Note that this only prints the name of the duplicates, not the original file. This is handy because you can add `| xargs rm` to the end of the command to delete all the duplicates while leaving the original.

Categories: Système Tags: ,

Typical iptables

13/05/2021 Aucun commentaire
# Modify this file accordingly for your specific requirement.
# 1. Delete all existing rules
iptables -F

# 2. Set default chain policies
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT DROP

# 3. Block a specific ip-address
#iptables -A INPUT -s "$BLOCK_THIS_IP" -j DROP
 Lire la suite...

How to change the MAC address of an Ethernet interface

09/05/2021 Aucun commentaire

Change the MAC address of an Ethernet interface temporarily

Check MAC addresses:

$ ifconfig -a | awk '/HWaddr/ {print "Interface: " $1 "\t MAC: " $NF}'
Interface: eth0	 MAC: 08:00:27:2c:a4:69
Interface: eth1	 MAC: 08:00:27:9a:21:24

Shut down desired Ethernet interface (eth0 in this example):

$ sudo ifconfig eth0 down

Specify new MAC address:

$ sudo ifconfig eth0 hw ether 08:00:00:00:00:01

Activate modified Ethernet interface:

# ifconfig eth0 up

Verify changed MAC address:

$ ifconfig -a | awk '/HWaddr/ {print "Interface: " $1 "\t MAC: " $NF}'
Interface: eth0	 MAC: 08:00:00:00:00:01
Interface: eth1	 MAC: 08:00:27:9a:21:24

This change is not permanent as the MAC address on the interface eth0 will revert to the default on next system reboot. Lire la suite…

Categories: Réseau, Système Tags:

Clone Your Ubuntu installation

09/05/2021 Aucun commentaire

If you want to create a system that is similar to a different system you have already set up, it can be difficult to remember each and every package you had installed.This method works best when you are exporting to and importing from the same distribution and, specifically, the same releasefor example, exporting from Ubuntu Dapper to Ubuntu Dapper or ubuntu edgy to ubuntu edgy.

Ubuntu uses the APT package management system which handles installed packages and their dependencies. If we can get a list of currently installed packages you can very easily duplicate exactly what you have installed now on your new machine. Below is a command you can use to export a list of your installed packages.

sudo dpkg --get-selections | grep '[[:space:]]install$='| awk '{print $1}' > installedpackages
Now you should end up with a file called “installedpackages” which consists of a long list of every package your currently have installed.

The next step would be to tell the clone machine to install each of those packages. You’ll have to copy that file to the clone machine (via network, usb drive, email, etc) and also make sure to duplicate the /etc/apt/sources.list file. Without the same access to repositories it may not be able to find the packages.

To tell your system to use the previously exported package list use the following command (after making sure to also clone your /etc/apt/sources.list file).

Update the source list using the following command

sudo aptitude update

Import the package list using the following command

cat installedpackages | xargs sudo aptitude install


cssh: un outil pour les paresseux…

08/05/2021 Aucun commentaire

Cluster ssh

Cluster ssh

Cluster ssh est un outil merveilleux ! Combien de fois ai-je du retaper les mêmes commandes dans deux fenêtres shell différentes sous prétexte que les serveurs faisaient la même chose… Je pense à mes deux serveurs DNS, par exemple.

Évidemment, il faut faire attention à ce qu’on tape car la moindre erreur est reproduite en double… Mais quel bonheur de faire deux fois le travail et surtout d’avoir des configurations identiques quand on le désire.

Autre avantage: si on ouvre plusieurs sessions ssh dans la même commande shell, cssh redimensionne les fenêtres automatiquement.

Attention: si vous ajouter des fenêtres après coup sur Ubuntu, il les superposera sur les précédentes ce qui n’est pas folichon… Il faut prévoir votre nombre de session au départ. Par contre, sur Mac OS X, l’excellent csshX fait le boulot correctement.

Categories: Système Tags: ,