Archive

Articles taggués ‘commandes’

How to check the file size in Linux/Unix bash shell scripting

28/09/2017 Comments off

 

How to check file size in unix using wc command

The wc command shows the number of lines, words, and bytes contained in file. The syntax is as follows to get the file size:
wc -c /path/to/file
wc -c /etc/passwd

Sample outputs:

5253 /etc/passwd

You can easily extract the first field either using the cut or awk command:
wc -c /etc/passwd | awk '{print $1}'
Sample outputs:

5253

OR assign this size to a bash variable:

myfilesize=$(wc -c "/etc/passwd" | awk '{print $1}')
printf "%d\n" $myfilesize
echo "$myfilesize"

How to get the size of a file in a bash script using stat command

The stat command shows information about the file. The syntax is as follows to get the file size on GNU/Linux stat:
stat -c %s "/etc/passwd"
OR
stat --format=%s "/etc/passwd"
To assign this size to a bash variable:

myfilesize=$(stat --format=%s "/etc/passwd")
echo "$myfilesize"
## or ##
myFileSizeCheck=$(stat -c %s "/etc/resolv.conf")
printf "My file size = %d\n" $myFileSizeCheck

The syntax is as follows to get the file size on BSD/MacOS stat:
stat -f %z "/etc/passwd"
Please note that if the file is symlink you will get size of that link only with the stat command.

du command example

The syntax is

du --apparent-size --block-size=1  "/etc/passwd"
fileName="/etc/hosts"
mfs=$(du --apparent-size --block-size=1  "$fileName" | awk '{ print $1}')
echo "$fileName size = ${mfs}"

Sample outputs from above commands:

Fig.01: How to check size of a file using a bash/ksh/zsh/sh/tcsh shell?Fig.01: How to check size of a file using a bash/ksh/zsh/sh/tcsh shell?

 

Find command example

The syntax is:

find "/etc/passwd" -printf "%s"
find "/etc/passwd" -printf "%s\n"
fileName="/etc/hosts"
mysize=$(find "$fileName" -printf "%s")
printf "File %s size = %d\n" $fileName $mysize
echo "${fileName} size is ${mysize} bytes."
Categories: Système Tags: , , , , ,

How to get domain name from URL in bash shell script

28/09/2017 Comments off
How can I extract or fetch a domain name from a URL string (e.g. https://www.cyberciti.biz/index.php) using bash shell scripting under Linux or Unix-like operating system?

You can use standard Unix commands such as sed, awk, grep, Perl, Python and more to get domain name from URL. No need to write regex. It is pretty simple.

Let use see various commands and option to grab the domain part from given variable under Linux or Unix-like system.

Get domain name from full URL

Say your url name is stored in a bash shell variable such as $x:
x='https://www.dbsysnet.com/'
You can use the awk as follows:
echo "$x" | awk -F/ '{print $3}'
### OR ###
awk -F/ '{print $3}' <<<$x

Sample outputs:

www.dbsysnet.com

Extract domain name from URL using sed

Here is a sample sed command:
url="https://www.dbsysnet.com"
echo "$url" | sed -e 's|^[^/]*//||' -e 's|/.*$||'

Extract domain name from URL using bash shell parameter substitution

Another option is to use bash shell parameter substitution:

# My shell variable 
f="https://www.cyberciti.biz/faq/copy-command/"
 
## Remove protocol part of url ##
f="${f#http://}"
f="${f#https://}"
f="${f#ftp://}"
f="${f#scp://}"
f="${f#scp://}"
f="${f#sftp://}"
 
## Remove username and/or username:password part of URL ##
f="${f#*:*@}"
f="${f#*@}"
 
## Remove rest of urls ##
f=${f%%/*}
 
## Show domain name only ##
echo "$f"

Shell script example

A shell script to purge urls from Cloudflare by matching domain name part:

#!/bin/bash
zone_id=""
api_key=""
 
urls="$@"
bon=$(tput bold)
boff=$(tput sgr0)
c=1
[ "$urls" == "" ] && { echo "Usage: $0 url"; exit 1; }
 
clear
echo "Purging..."
echo
for u in $urls
do
     echo -n "${bon}${c}${boff}.${u}: "
     ## Get domain name ##
     d="$(echo $u | awk -F/ '{ print $3}')"
     ## Set API_KEY, Email_ID, and ZONE_ID as per domain ##
     case $d in
	     www.cyberciti.biz) zone_id="ID_1"; api_key="MY_KEY_1"; email_id="webmaster@cyberciti.biz";;
	     theos.in) zone_id="ID_2"; api_key="MY_KEY_2"; email_id="webmaster@theos.in";;
	     *) echo "Domain not configured."; continue;;
     esac
     ## Do it ##
     curl -X DELETE "https://api.cloudflare.com/client/v4/zones/${zone_id}/purge_cache" \
     -H "X-Auth-Email: ${email_id}" \
     -H "X-Auth-Key: ${api_key}" \
     -H "Content-Type: application/json" \
     --data "{\"files\":[\"${u}\"]}"
     echo
     (( c++ ))
done
echo
Categories: Système Tags: , , ,

How to count total number of word occurrences using grep on Linux or Unix

28/09/2017 Comments off

I want to find out how many times a word (say foo or an IP address) occurs in a text file using the grep command on Linux or Unix-like system?

You can use the grep command to search strings, words, text, and numbers for a given patterns. You can pass the -coption to grep command. It only shows the number of times that the pattern has been matched for each file.

 

 

 

Show the total number of times that the word foo appears in a file named bar.txt

The syntax is:
grep -c string filename
grep -c foo bar.txt

Sample outputs:

3

To count total number of occurrences of word in a file named /etc/passwd root using grep, run:
grep -c root /etc/passwd
To verify that run:
grep --color root /etc/passwd
Pass the -w option to grep to select only an entire word or phrase that matches the specified pattern:
grep -w root /etc/passwd
OR
grep -c -w root /etc/passwd
In this example only match a word being with root:
grep --color -w '^root' /etc/passwd
grep -c -w '^root' /etc/passwd

To show only the matching part of the lines.
grep -o 'root' /etc/passwd
grep -c -o 'root' /etc/passwd

Sample session:

Fig.01: Counting occurrence of words/strings using grep commandFig.01: Counting occurrence of words/strings using grep command

How to find and delete directory recursively on Linux or Unix-like system

28/09/2017 Comments off

I type ‘find . -type d -iname foo -delete‘ command to find the foo directories and delete them. However, I am getting an error message that read as find: cannot delete './hourly.4/data/foo': Directory not empty on Linux server. How do delete directories based on find command output on Linux or Unix-like system?



The -delete option remove the DIRECTORY(ies), if they are empty. You need to use the -execoption to delete all directories and its contents. The syntax is as follows.

Find command syntax to delete dirs

Try:
find /dir/to/search/ -type d -name "dirName" -exec rm -rf {} +
OR
find /dir/to/search/ -type d -name "dirName" -exec rm -rf \;

Warning: Be careful with the rm command when using with find. You may end up deleting unwanted data.

Find will execute given command when it finds files or dirs. For example:
find . -type d -name "foo" -exec rm -rf {} +
OR
find . -type d -name "bar" -exec rm -rf "{}" \;
Sample outputs:

removed './daily.0/bar/.cache/motd.legal-displayed'
removed directory './daily.0/bar/root/.cache'
removed './daily.0/bar/.lesshst'
removed './daily.0/bar/.viminfo'
removed './daily.0/bar/.vim/.netrwhist'
removed directory './daily.0/bar/root/.vim'
removed './daily.0/bar/root/.bashrc'
removed './daily.0/bar/.ssh/authorized_keys'
removed directory './daily.0/bar/root'
removed directory './daily.0/bar/var/spool/cron/crontabs'

You can find directories that are at least four levels deep in the working directory /backups/:
find /backups/ -type d -name "bar" -depth +4 -print0 -exec rm -rf {} +

Find and xargs

The syntax is as follows to find and delete directories on Linux/Unix system:
## delete all empty dirs ##
find /path/to/dir/ -type d -empty -print0 | xargs -0 -I {} /bin/rm -rf "{}"
## delete all foo dirs including subdirs in /backups/
find /backups/ -type d -name "foo*" -print0 | xargs -0 -I {} /bin/rm -rf "{}"

The second command is secure and fast version as it deals with weird dir names such as:

  • “foo bar”
  • “Foo _ *bar”
 
Categories: Système Tags: , , ,

Bash Shell: Replace a String With Another String In All Files Using sed and Perl -pie Options

28/09/2017 Comments off

String search and replace

How do I replace a string with another string in all files? For example, ~/foo directory has 100s of text file and I’d like to find out xyz string and replace with abc. I’d like to use sed or any other tool to replace all occurrence of the word.

The sed command is designed for this kind of work i.e. find and replace strings or words from a text file under Apple OX, *BSD, Linux, and UNIX like operating systems. The perl can be also used as described below.

sed replace word / string syntax

The syntax is as follows:
sed -i 's/old-word/new-word/g' *.txt

GNU sed command can edit files in place (makes backup if extension supplied) using the -i option. If you are using an old UNIX sed command version try the following syntax:

sed 's/old/new/g' input.txt > output.txt

You can use old sed syntax along with bash for loop:

#!/bin/bash
OLD="xyz"
NEW="abc"
DPATH="/home/you/foo/*.txt"
BPATH="/home/you/bakup/foo"
TFILE="/tmp/out.tmp.$$"
[ ! -d $BPATH ] && mkdir -p $BPATH || :
for f in $DPATH
do
  if [ -f $f -a -r $f ]; then
    /bin/cp -f $f $BPATH
   sed "s/$OLD/$NEW/g" "$f" > $TFILE && mv $TFILE "$f"
  else
   echo "Error: Cannot read $f"
  fi
done
/bin/rm $TFILE

A Note About Bash Escape Character

A non-quoted backslash \ is the Bash escape character. It preserves the literal value of the next character that follows, with the exception of newline. If a \newline pair appears, and the backslash itself is not quoted, the \newline is treated as a line continuation (that is, it is removed from the input stream and effectively ignored). This is useful when you would like to deal with UNIX paths. In this example, the sed command is used to replace UNIX path “/nfs/apache/logs/rawlogs/access.log” with “__DOMAIN_LOG_FILE__”:

#!/bin/bash
## Our path
_r1="/nfs/apache/logs/rawlogs/access.log"
 
## Escape path for sed using bash find and replace 
_r1="${_r1//\//\\/}"
 
# replace __DOMAIN_LOG_FILE__ in our sample.awstats.conf
sed -e "s/__DOMAIN_LOG_FILE__/${_r1}/" /nfs/conf/awstats/sample.awstats.conf  > /nfs/apache/logs/awstats/awstats.conf 
 
# call awstats
/usr/bin/awstats -c /nfs/apache/logs/awstats/awstats.conf

The $_r1 is escaped using bash find and replace parameter substitution syntax to replace each occurrence of / with \/.

perl -pie Syntax For Find and Replace

The syntax is as follows:
perl -pie 's/old-word/new-word/g' input.file > new.output.file

Categories: Système Tags: , , , ,

BASH: Vérifier qu’une commande existe

28/09/2017 Comments off

Comment vérifier qu’une commande existe en Bash ?
Comment vérifier la disponibilité des prérequis d’un script ?

Une méthode permettant de vérifier qu’une commande existe consiste à tester le retour de « command -v <commande_a_tester> » :

fhh@mafalda ~ $ command -v ls
alias ls='ls --color=auto'
fhh@mafalda ~ $ echo $?
0
fhh@mafalda ~ $ command -v ls > /dev/null && echo "ls existe" ls existe
 

Si la commande existe et est accessible via le PATH ou au chemin spécifié, « command -v » retourne « 0 », sinon, le retour est supérieur à « 0 » :

fhh@mafalda ~ $ command -v youhou
# youhou n'existe pas dans le PATH ou comme commande interne :
fhh@mafalda ~ $ echo $?
1
fhh@mafalda ~ $ command -v ../../usr/bin/date
# date est accessible au chemin spécifié :
fhh@mafalda ~ $ echo $?
0

Lire la suite…

Categories: Système Tags: , , ,

BASH : Suppression des accents, cédilles, etc

28/09/2017 Comments off

Comment supprimer les accents, cédilles, etc, dans une chaine de caractères ?

Méthode classique : la substitution

La suppression des caractères accentués et autres cédilles peut être effectuée, en Bash, en utilisant « sed » ou « tr » :

fhh@aaricia ~ $ _str="Une chaine avec des é, des Ù, des À, des ç et des œ"
fhh@aaricia ~ $ echo $_str | sed 'y/áàâäçéèêëîïìôöóùúüñÂÀÄÇÉÈÊËÎÏÔÖÙÜÑ/aaaaceeeeiiiooouuunAAACEEEEIIOOUUN/' Une chaine avec des e, des U, des A, des c et des œ

La méthode est fonctionnelle mais sous entend que tous les caractères à substituer aient été définis. Dans l’exemple, le « œ » n’a pas été remplacé car aucun caractère de remplacement ne lui est alloué.

Autre problème de cette méthode, remplacer une lettre par deux autres tel que « œ » par « oe » ou le « ß » allemand par « ss » nécessite la définition de règles particulières à chaque cas.

Ce sont ces raisons qui nous poussent à éviter cette méthode au profit de la conversion de chaines de caractères.

Méthode recommandée : la conversion

Plus complète, la méthode de conversion présente en sus l’avantage d’être plus concise.

« iconv » est utilisé pour « convertir » la chaine de caractères du format de base, UTF-8 dans l’exemple (option « -f » pour « from »), vers le format ASCII (option « -t » pour « to »).

Avec l’option « TRANSLIT », si un caractère ne peut être transcrit dans le format de destination, il est converti en une chaine de caractère équivalente.

fhh@aaricia ~ $ _str="Une chaine avec des é, des Ù, des À, des çÇ et des œ"
fhh@aaricia ~ $ echo $_str | iconv -f utf8 -t ascii//TRANSLIT Une chaine avec des e, des U, des A, des cC et des oe

La méthode fonctionne sur un large panel de caractères :

fhh@aaricia ~ $ echo "\"ß\"" | iconv -f utf8 -t ascii//TRANSLIT "ss"
fhh@aaricia ~ $ echo "āáǎàēéěèīíǐìōóǒòūúǔùǖǘǚǜĀÁǍÀĒÉĚÈĪÍǏÌŌÓǑÒŪÚǓÙǕǗǙǛ" | iconv -f utf8 -t ascii//TRANSLIT aaaaeeeeiiiioooouuuuuuuuAAAAEEEEIIIIOOOOUUUUUUUU

et peut être utilisée sur des fichiers :

fhh@aaricia ~ $ cat myfile.txt Une chaine avec des é, des Ù, des À, des ç et des œ   et même des "ß"
fhh@aaricia ~ $ iconv -f utf8 -t ascii//TRANSLIT < myfile.txt > noaccents.txt
fhh@aaricia ~ $ cat noaccents.txt Une chaine avec des e, des U, des A, des c et des oe   et meme des "ss"
 
Categories: Système Tags: , , , ,

Bash: How do I get the command history in a screen session?

05/07/2017 Comments off

If I start a screen session with screen -dmS name, how would I access the command history of that screen session with a script?

Using the , the last executed command appears, even in screen.

ANSWER:

I use the default bash shell on my system and so might not work with other shells.

this is what I have in my ~/.screenrc file so that each new screen window gets its own command history:

Default Screen Windows With Own Command History

To open a set of default screen windows, each with their own command history file, you could add the following to the ~/.screenrc file:

screen -t "window 0" 0 bash -ic 'HISTFILE=~/.bash_history.${WINDOW} bash'
screen -t "window 1" 1 bash -ic 'HISTFILE=~/.bash_history.${WINDOW} bash'
screen -t "window 2" bash -ic 'HISTFILE=~/.bash_history.${WINDOW} bash'

Ensure New Windows Get Their Own Command History

The default screen settings mean that you create a new window using Ctrl+a c or Ctrl+a Ctrl+c. However, with just the above in your ~/.screenrc file, these will use the default ~/.bash_history file. To fix this, we will overwrite the key bindings for creating new windows. Add this to your ~/.screenrc file:

bind c screen bash -ic 'HISTFILE=~/.bash_history.${WINDOW} bash'
bind ^C screen bash -ic 'HISTFILE=~/.bash_history.${WINDOW} bash'

Now whenever you create a new screen window, it’s actually launching a bash shell, setting the HISTFILE environmental variable to something that includes the current screen window’s number ($WINDOW).

Command history files will be shared between screen sessions with the same window numbers.

Write Commands to $HISTFILE on Execution

As is normal bash behavior, the history is only written to the $HISTFILE file by upon exiting the shell/screen window. However, if you want commands to be written to the history files after the command is executed, and thus available immediately to other screen sessions with the same window number, you could add something like this to your ~/.bashrc file:

export PROMPT_COMMAND="history -a; history -c; history -r; ${PROMPT_COMMAND}"
Categories: Système Tags: , , ,

Scripts shell de sauvegarde

21/06/2017 Comments off

Une des façons les plus simples de sauvegarder un système utilise un script shell. Par exemple, un script peut être utilisé pour configurer les répertoires à sauvegarder et transmettre ces répertoires comme arguments à l’utilitaire tar, ce qui crée un fichier d’archive. Le fichier d’archive peut ensuite être déplacé ou copié dans un autre emplacement. L’archive peut également être créée sur un système de fichiers distant tel qu’un montage NFS.

L’utilitaire tar crée un fichier d’archive de plusieurs fichiers ou répertoires. tar peut également filtrer les fichiers par le biais des utilitaires de compression, réduisant ainsi la taille du fichier d’archive.

Categories: Système Tags: , , ,

8 Practical Examples of Linux Xargs Command for Beginners

27/03/2017 Comments off

The Linux xargs command may not be a hugely popular command line tool, but this doesn’t take away the fact that it’s extremely useful, especially when combined with other commands like findand grep. If you are new to xargs, and want to understand its usage, you’ll be glad to know that’s exactly what we’ll be doing here.

Before we proceed, please keep in mind that all the examples presented in this tutorial have been tested on Ubuntu 14.04 LTS. Shell used is Bash, and version is 4.3.11.

1. How Xargs command works?

Well, before jumping onto its usage, it’s important to understand what exactly Xargs does. In layman’s terms, the tool – in its most basic form – reads data from standard input (stdin) and executes the command (supplied to it as argument) one or more times based on the input read. Any blanks and spaces in input are treated as delimiters, while blank lines are ignored. 

 

If no command is supplied as argument to xargs, the default command that the tool executes is echo. For example, in the following example, I just executed ‘xargs’ and entered ‘Hello World’ on stdin. As I pressed Ctrl+D (to tell xargs that we’re done with the input), the echocommand was automatically executed, and ‘Hello World’ was printed again.

How xargs command works

2. How to use xargs with another command?

While echo is the default command xargs executes, you can explicitly specify any other command. For example, you can pass the find command along with its ‘-name’ option as argument to xargs, and then pass the name of the file (or type of files) you want find to search as input through stdin.

Here’s the complete command in question:

xargs find -name

For example, we provided « *.txt » in input through stdin, which means we want the find command to search all .txt files in the current directory (as well as its subdirectories).

Here’s the command in action:

Combine xargs with other commands

Lire la suite…

Categories: Système, Tutoriel Tags: , ,