Archive

Articles taggués ‘shell’

Bash Shell Loop Over Set of Files

13/08/2020 Comments off

bash shell loopBash Shell Loop

How do I run shell loop over set of files stored in a current directory or specified directory?

You can use for loop easily over a set of shell file under bash or any other UNIX shell using wild card character.

Syntax

The general syntax is as follows:

for f in file1 file2 file3 file5
do
 echo "Processing $f"
 # do something on $f
done

You can also use shell variables:

FILES="file1
/path/to/file2
/etc/resolv.conf"
for f in $FILES
do
	echo "Processing $f"
done

You can loop through all files such as *.c, enter:

$ for f in *.c; do echo "Processing $f file.."; done

Lire la suite…

Categories: Système Tags: , , ,

Find Duplicate Files (based on size first, then MD5 hash)

12/08/2020 Aucun commentaire
Terminal – Find Duplicate Files (based on size first, then MD5 hash)
find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate
Find Duplicate Files (based on size first, then MD5 hash)

This dup finder saves time by comparing size first, then md5sum, it doesn’t delete anything, just lists them.

Terminal – Alternatives
find -type f -exec md5sum '{}' ';' | sort | uniq --all-repeated=separate -w 33 | cut -c 35-

Calculates md5 sum of files. sort (required for uniq to work). uniq based on only the hash. use cut ro remove the hash from the result.

find -type d -name ".svn" -prune -o -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type d -name ".svn" -prune -o -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate

Improvement of the command “Find Duplicate Files (based on size first, then MD5 hash)” when searching for duplicate files in a directory containing a subversion working copy. This way the (multiple dupicates) in the meta-information directories are ignored.

Can easily be adopted for other VCS as well. For CVS i.e. change “.svn” into “.csv”:

find -type d -name ".csv" -prune -o -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type d -name ".csv" -prune -o -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate

find -not -empty -type f -printf "%-30s'\t\"%h/%f\"\n" | sort -rn -t$'\t' | uniq -w30 -D | cut -f 2 -d $'\t' | xargs md5sum | sort | uniq -w32 --all-repeated=separate

Finds duplicates based on MD5 sum. Compares only files with the same size. Performance improvements on:

find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separateThe new version takes around 3 seconds where the old version took around 17 minutes. The bottle neck in the old command was the second find. It searches for the files with the specified file size. The new version keeps the file path and size from the beginning.

find . -type f -exec md5 '{}' ';' | sort | uniq -f 3 -d | sed -e "s/.*(\(.*\)).*/\1/"

This works on Mac OS X using the `md5` command instead of `md5sum`, which works similarly, but has a different output format. Note that this only prints the name of the duplicates, not the original file. This is handy because you can add `| xargs rm` to the end of the command to delete all the duplicates while leaving the original.

Categories: Système Tags: ,

Typical iptables

11/08/2020 Aucun commentaire
# Modify this file accordingly for your specific requirement.
# http://www.thegeekstuff.com
# 1. Delete all existing rules
iptables -F

# 2. Set default chain policies
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT DROP

# 3. Block a specific ip-address
#BLOCK_THIS_IP="x.x.x.x"
#iptables -A INPUT -s "$BLOCK_THIS_IP" -j DROP
 Lire la suite...

How to change the MAC address of an Ethernet interface

07/08/2020 Aucun commentaire

Change the MAC address of an Ethernet interface temporarily

Check MAC addresses:

$ ifconfig -a | awk '/HWaddr/ {print "Interface: " $1 "\t MAC: " $NF}'
Interface: eth0	 MAC: 08:00:27:2c:a4:69
Interface: eth1	 MAC: 08:00:27:9a:21:24

Shut down desired Ethernet interface (eth0 in this example):

$ sudo ifconfig eth0 down

Specify new MAC address:

$ sudo ifconfig eth0 hw ether 08:00:00:00:00:01

Activate modified Ethernet interface:

# ifconfig eth0 up

Verify changed MAC address:

$ ifconfig -a | awk '/HWaddr/ {print "Interface: " $1 "\t MAC: " $NF}'
Interface: eth0	 MAC: 08:00:00:00:00:01
Interface: eth1	 MAC: 08:00:27:9a:21:24

This change is not permanent as the MAC address on the interface eth0 will revert to the default on next system reboot. Lire la suite…

Categories: Réseau, Système Tags:

Clone Your Ubuntu installation

07/08/2020 Aucun commentaire

If you want to create a system that is similar to a different system you have already set up, it can be difficult to remember each and every package you had installed.This method works best when you are exporting to and importing from the same distribution and, specifically, the same releasefor example, exporting from Ubuntu Dapper to Ubuntu Dapper or ubuntu edgy to ubuntu edgy.

Ubuntu uses the APT package management system which handles installed packages and their dependencies. If we can get a list of currently installed packages you can very easily duplicate exactly what you have installed now on your new machine. Below is a command you can use to export a list of your installed packages.

sudo dpkg --get-selections | grep '[[:space:]]install$='| awk '{print $1}' > installedpackages
Now you should end up with a file called “installedpackages” which consists of a long list of every package your currently have installed.

The next step would be to tell the clone machine to install each of those packages. You’ll have to copy that file to the clone machine (via network, usb drive, email, etc) and also make sure to duplicate the /etc/apt/sources.list file. Without the same access to repositories it may not be able to find the packages.

To tell your system to use the previously exported package list use the following command (after making sure to also clone your /etc/apt/sources.list file).

Update the source list using the following command

sudo aptitude update

Import the package list using the following command

cat installedpackages | xargs sudo aptitude install