Shell Scripting Tutorial

1.0 Shell Script

bash is the commonly used shell in GNU-Linux. bash can be run interactively, as a login shell or as an interactive process from a terminal emulator like the gnome-terminal. bash can be run non-interactively by running a bash script. A shell script, or bash script, is a list of commands written to automate some system operations work in GNU-Linux. The first script prints Hello World! on the display. It is,

#!/bin/bash

echo "Hello, World!"

The first line in the script is a shebang [pronounced shuh–bang]. It is the first line in the script and the first two characters are #!, followed by the path to the interpreter program (/bin/bash). The interpreter is run with the path to the script as the first argument. There are no restrictions on the file name. The only requirement is that the execute permission must be on. So if the script file name is hello, we can do something like this,

$ vi hello     # Enter the text as in the script above
$ chmod +x hello
$ ./hello
Hello, World!

Actually, we can run the above script as bash hello and in that execute permission on the script file is not required.

$ chmod -x hello
$ ls -ls hello
4 -rw-rw-r-- 1 user1 user1 402 Jan  4 07:25 hello
$ bash hello
Hello World!

2.0 Variables

The variable names comprise of alphanumeric characters and underscores and start with an alphabetic character or an underscore. Variables are often called parameters in shell jargon. The values contained in the variables are essentially strings but can be considered as integers depending upon the value and the context of usage. The variables can be assigned values. To retrieve the value from a variable, a $ has to be prefixed to the variable. For example,

i=5       # assignment
echo $i   # prints 5

$i prints the value of i, but it is better to put braces around i, for it delimits the string value of the variable. This is particularly helpful in more complicated expressions. For example,

i=5         # assignment
echo ${i}   # prints 5
t="to"
echo "I will see you ${t}morrow."  # prints "I will see you tomorrow."

We can regard that the construct ${t} expands to the value of t.

3.0 Quoting

Double quotes indicate a string to bash, but it looks inside and expands the variables, as in the case of ${t} above. What if you wish to print ${t} as a part of a string? There are multiple answers. Backslash turns off the meaning of special characters. So if you put \ before ${t}, bash does not expand it. Also, if you enclose the string in single quotes, bash does not look inside it and whatever is there is printed as it is.

t="to"
echo "I will see you ${t}morrow."  # prints "I will see you tomorrow."
echo "I will see you \${t}morrow."  # prints "I will see you ${t}morrow."
echo 'I will see you ${t}morrow.'  # prints "I will see you ${t}morrow."

4.0 Backquote/backtick/grave accent quoted command execution

If a command is enclosed between two backquote characters, like `command`, the shell expands it by replacing it with the standard output of that command after discarding the trailing newlines. It is also possible to write $(command) with the same effect. For example,

echo "The time is `date`"   # prints date and time
echo "The time is $(date)"  # prints date and time

5.0 Command Line Parameters

When a bash script runs, it gets certain parameters as a part of its environment. $0 is the name of the script as it was run. $1, $2, $3, .. are the positional parameters, the arguments to the script. $# gives the number of arguments passed to the script. For example, the script hello, as given below,

#!/bin/bash
echo "Number of arguments = $#"
echo "Hello, World!"
echo '$0' = "$0"
echo '$1' = "$1"
echo '$2' = "$2"
echo '$3' = "$3"
echo '$4' = "$4"
echo '$5' = "$5"
echo '$6' = "$6"
echo '$7' = "$7"
echo '$8' = "$8"
echo '$9' = "$9"
echo '$10' = "${10}"
echo '$11' = "${11}"
echo '$12' = "${12}"

gives the output,

$ ./hello one two three four five six seven eight nine ten eleven twelve
Number of arguments = 12
Hello, World!
$0 = ./hello
$1 = one
$2 = two
$3 = three
$4 = four
$5 = five
$6 = six
$7 = seven
$8 = eight
$9 = nine
$10 = ten
$11 = eleven
$12 = twelve

For printing the values of $10, $11 and $12, it is necessary to put braces around 10, 11 and 12 respectively. Of course, we could have written $1, $2, $3, …, $9 as ${1}, ${2}, ${3}, …, ${9} respectively.

6.0 Special Parameters

There are Special Parameters available in the bash script, which help in programming. These are,

7.0 String manipulation

7.1 String length

${#string} expands to the length of the string.

$ str="I am a string"
$ echo ${#str}
13

Alternatively, we can use the expr command.

$ echo `expr length "${str}"`
13
$ expr "${str}" : '.*.'
13

7.2 Substring extraction

${string:position} extracts the string, starting at the position. The string index starts at zero. For example,

$ str="Hello, World!"
$ echo "${str:7}"
World!

We can specify the length of the required substring after position.

$ echo "${str:7:5}"
World

7.3 Substring replacement

${string/substring/replacement} replaces the first occurrence of substring in the string with the replacement.

$ str1="apple mango guava apple pineapple"
$ echo "${str1/apple/orange}"
orange mango guava apple pineapple

${string//substring/replacement} replaces all occurrences of substring in the string with the replacement.

$ echo "${str1//apple/orange}"
orange mango guava orange pineorange

If the replacement is not given, the substring is simple deleted.

$ echo "${str1//apple}"
 mango guava  pine

However, the original string is not modified.

$ echo "${str1}"
apple mango guava apple pineapple

${string/#substring/replacement} searches the string for the first occurrence of substring, from the front end, and replaces it with the replacement.

$ echo ${str1/#apple/orange}
orange mango guava apple pineapple

${string/%substring/replacement} searches the string for the first occurrence of substring, from the rear end, and replaces it with the replacement.

$ echo ${str1/%apple/orange}
apple mango guava apple pineorange
$ echo ${str1/%apple/orange}
apple mango guava apple pineorange

7.4 Removing substrings

7.4.1 Removing matching prefix pattern

${parameter#word}
${parameter##word}

The parameter is expanded. The word is a regular expression. If the word matches the beginning of the value of parameter, the matching portion is deleted and the remaining part is the result of this expansion. A single “#” matches the minimum, whereas “##” matches the maximum. For example,

$ FILENAME="somefile.ext"
$ # get file name extension
$ echo "${FILENAME##*.}"
ext
$ # full pathname
$ PATHNAME=`pwd`/${FILENAME}
$ echo "${PATHNAME}"
/home/user1/src/shell/somefile.ext
$ # get the filename only
$ echo ${PATHNAME##*/}
somefile.ext

7.4.2 Removing matching suffix pattern

${parameter%word}
${parameter%%word}

The parameter is expanded. The word is a regular expression. If the word matches the end of the value of parameter, the matching portion is deleted and the remaining part is the result of this expansion. A single “%” matches the minimum, whereas “%%” matches the maximum. For example, assuming the parameters FILENAME and PATHNAME have the values assigned in the section above,

$ # get filename without extension
$ echo "${FILENAME%.*}"
somefile
$ # get the directory name
$ echo ${PATHNAME%/*}
/home/user1/src/shell

8.0 Terms

8.1 Command

A command can be a simple command, a pipeline or a list.

A simple command is a command followed by arguments as in cat file1, file2, …. The return value of a simple command is its exit status. If a command is terminated with a signal n, the return value is 128+n.

A pipeline is a sequence of simple commands connected by the | character. The commands in the pipeline are executed concurrently as separate processes. The standard output of a simple command on the left of a | symbol is connected to the standard input of the next command on the right of the | symbol.

command1 | command2 | command3 ...

The return value of a pipeline is the exit status of the last command, if pipefail option is not enabled. However, one can enable the pipefail option,

set -o pipefail

And, after the pipefail option is enabled, the return value is the exit status of the rightmost command in pipeline returning a non-zero status. If all commands return a zero exit status, the return value is zero.

A list is a sequence of pipelines, separated by one of the operators, ;, &, &&, ||. A list may be optionally terminated by a ;, & or newline. Of the list operators, && and || have equal precedence. Also, the operators ; and & have equal precedence. The precedence of && and || is higher than that of ; and &.

Commands separated by a semicolon (;) are executed sequentially. The shell starts a command and waits for it to finish before starting the next command in the list.

For commands that end with an &, the shell starts the command and moves on to the next command. The command executes in the background.

Command exit statuses are different from the values of true and false in programming languages. A return status of 0 indicates success and is deemed true. Similarly, a non-zero status indicates an error and is considered false. So for command lists, with commands separated by && and ||,

command1 && command2:
command2 is executed if and only if command1 returns a zero exit status.

command1 || command2:
command2 is executed if and only if command1 returns a non-zero exit status.

The return value of a command list is the exit status of the last command executed in the list.

8.2 Compound Command

There are four types of Compound Commands, viz., (list), { list; }, ((expression)) and [[ expression ]].

8.2.1 (list)

A new shell is created and the list is executed in it. For example, the shell script named, hello1

#!/bin/bash
(echo "Hello, World"; ps -f)
echo "Hello, the brave new World!"
ps -f

gives the output,

$ ./hello1
Hello, World
UID        PID  PPID  C STIME TTY          TIME CMD
user1    4187  3083  0 16:26 pts/9    00:00:00 bash
user1    4265  4187  0 16:29 pts/9    00:00:00 /bin/bash ./hello1
user1    4266  4265  0 16:29 pts/9    00:00:00 /bin/bash ./hello1
user1    4267  4266  0 16:29 pts/9    00:00:00 ps -f
Hello, the brave new World!
UID        PID  PPID  C STIME TTY          TIME CMD
user1    4187  3083  0 16:26 pts/9    00:00:00 bash
user1    4265  4187  0 16:29 pts/9    00:00:00 /bin/bash ./hello1
user1    4268  4265  0 16:29 pts/9    00:00:00 ps -f

As you can see, a new process (subshell) with id 4266 above, is created to execute the commands enclosed in parenthesis.

8.2.2 { list; }

This is grouping of commands and the command group is executed by the current shell. How does it help? Consider the script,

command1 && { command2 || command3; }

If, and only if, command1 returns a zero status, the command group, { command2 || command3; }, is executed.

8.2.3 ((expression))

The expression is evaluated as per the rules of arithmetic expressions. $((expression)) gives the value of the expression. Only integer arithmetic is supported. Also, spaces may not be put in a way that invalidates the syntax. For example,

a = 3 # wrong. bash tries to execute a file named a
a=3   # correct. a is assigned the value of 3.
b=7
c=9
d=6
echo $((a+b))                       # prints 10
echo $(( ((c * d)/2) ))             # prints 27
echo $(((a+b) * ((c * d)/2)))       # prints 270

8.2.4 [[ expression ]]

The expression is a conditional expression. The value returned is either 0 (true) or 1 (false). A conditional expression is made up of unary and binary primaries. The primaries check file attributes, compare strings and do arithmetic comparisons. The primaries are,

A primary is an expression. Expressions may be combined to give another expression, as given below,

( expression )
Returns the value of expression.

! expression
Returns true if expression is false

expression1 && expression2
Returns true if both expression1 and expression2 are both true. expression2 is evaluated only if expression1 is true.

expression1 || expression2
Returns true if either expression1 or expression2 is true. expression2 is evaluated only if expression1 is false.

For example, a script named hello1, with the set group id bit on,

#!/bin/bash

x="hello"
if [[ (${x} = "hello") && (-g "hello1") ]] ; then echo $?; echo "true";
else
    echo "false"
fi

gives the output,

$ ls -ls hello1
4 -rwxrwsr-x 1 user1 user1 119 Jan 11 15:09 hello1
$ ./hello1
0
true

9.0 Reserved Words

You might have noticed the spaces in { list; } and [[ expression ]]. That is because {, }, [[ and ]] are reserved words and spaces are required around these so that bash can recognize these words. The full list of bash reserved words is rather small and is,

! case coproc do done elif else esac fi for function if in select then until while { } time [[ ]]

10.0 Flow Control Statements

if list; then list; [ elif list; then list; ] … [ else list; ] fi
First the if list is executed. If the exit status is zero, then the then list is executed and the command completes. Otherwise, the succeeding elif lists, if present, are executed one by one. If the exit status of an elif list is zero, the then list of that elif is executed and the command completes. Otherwise, the else list, if present, is executed. The exit status is the exit status of the last command executed and is zero if no condition tested true and the else part was not present.

case word in [ [ ( ] pattern [ | pattern ] … ) list ;; ] … esac
The left parenthesis is optional and is almost never written. So, effectively, the word is matched with each pattern, and for the first match, the corresponding command list is executed and the command completes. The vertical bar (|) in pattern1 | pattern2 means pattern1 or pattern2. Pattern matching rules like, * for matching everything and specifying a range in brackets can be used for writing patterns. The exit status is the exit status of the last command executed and is zero if no pattern matched. The double semicolon operator (;;) separates a pattern and the corresponding command list from the next one. For example, a script that tells to work or play based on the three character code for the day of the week, passed as the first argument is,

case $1 in
    [Ss]un* | [Ss]at* )  echo "Play";
                 ;;
    [Mm]on* | [Tt]ue* | [Ww]ed* | [Tt]hu* | [Ff]ri* ) echo "Work";
        ;;
    *) echo "Error in input";
        ;;
esac

for name [ [ in [ word … ] ] ; ] do list ; done
The word is expanded. The name is assigned the first word in the expansion and the list is executed. This is repeated for all the remaining words in the expansion. If in is missing, the list is executed for each of the positional parameters. If word expands to an empty list, the list is not executed at all. The return status is the exit status of the last command executed or is zero when no command was executed. For example,

#!/bin/bash
# script for-check (* expands to all file-names)
for name   in *  ;  do echo $name ; done

$ ls -ls    # list files in directory
total 20
4 -rw-rw-r-- 1 user1 user1 14 Jan 12 12:45 dry
4 -rwxrwxr-x 1 user1 user1 72 Jan 12 13:08 for-check
4 -rw-rw-r-- 1 user1 user1 14 Jan 12 12:45 fry
4 -rw-rw-r-- 1 user1 user1 14 Jan 12 12:45 pry
4 -rw-rw-r-- 1 user1 user1 14 Jan 12 12:45 try
$ ./for-check    # run the script
dry
for-check
fry
pry
try

$ # now trying for without in
#!/bin/bash
# script for-check (without in)
for name do echo $name ; done

$ # Running above script, echo prints positional parameters
$ ./for-check hello world
hello
world

$ # introducing word which expands to an empty list
#!/bin/bash
# script for-check (x is not defined, so ${x} expands to empty list)

for name in ${x}  ;  do echo $name ; done

$ # Running above script
$ ./for-check hello world
$ # no output, as echo is not executed.

for (( expr1 ; expr2 ; expr3 )) ; do list ; done
This resembles the for statement in C language. expr1, expr2 and expr3 are arithmetic expressions. First expr1 is evaluated. Then, expr2 is evaluated. If expr2 evaluates to zero, the execution of the statement is complete. If expr2 evaluates to non-zero, the list is executed. Then expr3 is evaluated. The sequence, evaluating expr2 and if zero, command being complete, executing the list and evaluating expr3 is done repeatedly till expr2 evaluates as zero. If any of expr1, expr2 or expr3 is missing, it is assumed to be 1. For example,

#!/bin/bash
# print 5 random strings
for (( i=0 ; i<5 ; i++ )) ; do
date | md5sum | cut -f1 -d' ';
sleep 1;
done

$ ./for-check
2357b9a70a2920457d9681c7bfc303a5
603ba8c778432d75640b94abda065d07
9f67756f46fc4b09576c8ccc3cf39ca4
eaeda2cbea43b1c9f4878695739d9b69
5fecdc9e399ed1178a07e8512b9727fd

while list-1 ; do list-2 ; done
The while command executes list-1. If the last command in list-1 returns an exit status of zero, it executes list-2. This is repeated till the last command in list-1 returns a non-zero exit status. For example,

#!/bin/bash
# For the next hour show logged in users, every minute
counter=61
while [[ ${counter} -gt 0 ]] ; 
do
w -h ;
echo ;
echo "--------------------------------------------------------------" ;
echo ;
if [[ $((--counter)) -gt 0 ]]; then
sleep 60 ;
fi ;
done

until list-1 ; do list-2 ; done
The until command is just like the while except that list-2 is executed till the time the last command in list-1 returns a non-zero exit status. The above script can be written using the until command,

#!/bin/bash
# For next hour show logged in users, every minute
counter=61
until [[ ${counter} -eq 0 ]] ; 
do
w -h ;
echo ;
echo "--------------------------------------------------------------" ;
echo ;
if [[ $((--counter)) -gt 0 ]]; then
sleep 60 ;
fi ;
done

select name [ in word ] ; do list ; done
The select command can be used for presenting a menu of options to the user and executing commands corresponding to the option chosen by the user. The word is expanded and the resulting items are printed on the standard error, with each item being preceded with a number. The user is prompted with PS3 to choose an option. Based on the option number entered by the user, the corresponding item value is assigned to name, which can be used by the commands in the list. If the user enters just ENTER (that is, an empty line), the options are printed again. If the user enters a number outside the range, name is set to null. For example, the following script allows an administrator to monitor processes run by a user.

#!/bin/bash

# find procceses by a logged in user
USERS=`who | cut -d' ' -f1 | uniq`
USERS="${USERS} Quit"
PS3="Select user: "
echo "Find processes being run by a user"
select user in $USERS ;
do
if [[ ${user} = "Quit" ]] ; then
break;
fi
if [[ -n ${user} ]] ; then
ps -f -u $user ;
else
echo "Error in input, please try again"
continue;
fi;
echo ;
done

If, in the select command, the in word is omitted, the positional parameters are used. For example, to print a file in current directory, the following script is run with the * parameter.

#!/bin/bash

PS3="Print file: "
select file ;
do
if [[ -n ${file} ]] ; then
cat ${file} ;
echo "${file} printed" ;
else
break;
fi;
done

$ # running the above script
$ ./select-check *
1) dry		  3) fry	    5) select1-check  7) try
2) for-check	  4) pry	    6) select-check   8) until-check
Print file: 7
"Try not to become a man of success, but rather try to become a man of value."
-- Albert Einstein

try printed
Print file: 0
$

11.0 See also

• Bash Tutorial
• Bash idioms