Linux Shell Scripting -A Beginner's handbook [Part -1]

Chapter 1: Introduction: Quick Introduction to Linux

Prologue

This Linux Shell Scripting Tutorial is designed for people who wish to learn the fundamentals of shell scripting/programming. It also contains an introduction to power tools such as awk and sed. It is not the manual for the shell. While reading this tutorial you may find the manual quite useful (type man bash at $ prompt to see the manual pages). The manual contains all of the necessary information you need, but will not contain any examples which would make the ideas clear. Therefore, this tutorial contains examples rather than all of the features of the shell.

Audience for this tutorial

I assume you have a working knowledge of Linux: That you know basic commands such as how to create, copy, and remove files and directories; That you know how to use a text editor like vi or emacs; And that you can login to your system. However, I do not expect you to have any programming language experience. If you have access to Linux or (any UNIX like operating system), this tutorial will provide you with an easy-to-follow introduction to shell scripting.

So what is different about this tutorial?

Most of the other tutorials on Linux shell scripting are either too basic or skip too many important intermediate steps. The concept of this tutorial is to maintain a balance between these two. It covers many of the real life examples of shell scripting that are missing from many other tutorials. I have used a hands-on approach in this tutorial. The idea is very clear "do it yourself or learn by doing" i.e. trying things yourself is the best way to learn, so examples are presented as complete working shell scripts, which can be typed in and executed. I hope you get as much pleasure from reading this tutorial as I had writing it. If, after reading this tutorial you are able to write your own powerful shell scripts, then I think the purpose of writing this tutorial has been served. Finally, if after reading this tutorial you do get time to drop me an e-mail message with your comments or suggestions I would greatly appreciate it.

What is Linux?

Linux is a free open-source operating system based on Unix. Linus Torvalds originally created Linux with the assistance of developers from around the world.

• Free
• Unix Like
• Open Source
• Network operating system

Strictly speaking, Linux is a kernel. A kernel provides access to the computer hardware and control access to resources. The kernel decides who will use a resource, for how long and when. You can download the Linux kernel from the official web site.

However, the Linux kernel itself is useless unless you get all the applications such as text editors, email clients, browsers, office applications, etc. Therefore, someone came up with idea of a Linux distribution. A typical Linux distribution includes:

• Linux kernel
• GNU application utilities such as text editors, browsers
• GUI (X windows)
• Office application software
• Software development tools and compilers
• Thousands of ready to use application software packages
• Linux Installation programs/scripts
• Linux post installation management tools daily work such as adding users, installing applications, etc

Corporate and small businesses users need support while running Linux, so companies such as Red Hat or Novell provide Linux tech-support and sell it as product. Nevertheless, community driven Linux distributions do exist such as Debian, Gentoo and they are entirely free. To be frank, there are over 200+ Linux distributions.

Who created Linux?

In 1991 Linus Torvalds was studying UNIX at a university, where he was using a special educational experimental purpose operating system called Minix (a small version of UNIX to be used in the academic environment). However, Minix had its limitations and Linus felt he could create something better. Therefore, he developed his own version of Minix, known as Linux. Linux was Open Source right from the start.

Linux is kernel developed by Linus. The kernel was bundled with system utilities and libraries from the GNU project to create a usable operating system. Sometime people refer Linux as GNU/Linux because it has system utilities and libraries from the GNU project. Linus Torvalds is credited for creating the Linux Kernel, not the entire Linux operating system.

Linux distribution = Linux kernel + GNU system utilities and libraries + Installation scripts + Management utilities etc.

Please note that Linux is now packaged for different uses in Linux distributions, which contain the sometimes modified kernel along with a variety of other software packages tailored to different requirements such as:

1. Server
2. Desktop
3. Workstation
4. Routers
5. Various embedded devices
6. Mobile phones etc

More information on Linus Torvalds, can be found on Linux.org's short biography page or visit his home page.

Where can I download Linux?

Linux is available for download over the internet. However, this is only useful if your internet connection is fast. Another way is to order the CD-ROMs, which saves time, and the installation is fast and automatic. I recommend visiting LinuxISO.org to download most popular distributions.

If you are in India then you can get a Linux distribution from the leading computer magazines such as PC Quest or Digit. Most Linux books from you local bookstore also include a Linux distribution. See the list of recommended Linux books.

What is the Linux Kernel?

The kernel is heart of the Linux operating system. It manages the resources of Linux. Resources include:

• File management
• Multitasking
• Memory management
• I/O management
• Process management
• Device management
• Networking support including IPv4 and IPv6
• Advanced features such as virtual memory, shared libraries, demand loading, shared copy-on-write executables etc

The kernel decides who will use these resources and for how long and when. It runs your programs or sets up to execute binary files.

The kernel acts as an intermediary between the computer hardware and various applications.

What is the Linux Shell?

Computers understand the language of zeros and ones known as binary language. In the early days of computing, instructions were provided using binary language, which is difficult for all of us humans to read and write. Therefore, in an operating system there is a special program called the shell. The shell accepts human readable commands and translates them into something the kernel can read and process.

The shell is a user program or it is an environment provided for user interaction. It is a command language interpreter that executes commands read from the standard input device (keyboard) or from a file.

Shell is not part of system kernel, but uses the system kernel to execute programs, create files etc.

Several shells are available for Linux including:

Shell Name Developed by Where Remark BASH ( Bourne-Again SHell )Brian Fox and Chet Ramey

Free Software Foundation

Most common shell in Linux. It's Freeware shell.CSH (C SHell)Bill JoyUniversity of California (For BSD)The C shell's syntax and usage are very similar to

the C programming language.KSH (Korn SHell) David Korn AT & T Bell Labs --TCSHSee the man page.

Type $ man tcsh --TCSH is an enhanced but completely compatible version of the Berkeley UNIX C shell (CSH).

To find all of the available shells in your system, type the following command:

$ cat /etc/shells

Note that each shell does the same job, but each understands different command syntax and provides different built-in functions.

Under MS-DOS, the shell name is COMMAND.COM which is also used for the same purpose, but it is not as powerful as our Linux Shells are!

Command line interface (CLI)

The shell provides an interface to Linux where you can type or enter commands using the keyboard. It is known as the command line interface (CLI).

To find out your current shell type following command:

$ echo $SHELL

$ ps $$

But how do you use the shell?

To use the shell you simply type commands. A command is a computer program to built perform a specific task. Examples of commands include:

• ls
• clear
• cal
• date
• vi

If you started your Linux system in text mode, you start to use the shell as soon as you log in. If you started in a graphical mode, such as the Gnome desktop, you can open a shell by going to Applications >> System Tools >> Terminal. Alternatively, you can switch to a virtual console by pressing Ctrl-Alt-F1 and logging in.

What is a Shell Script or shell scripting?

Normally shells are interactive. It means the shell will accept command from you (via keyboard) and execute them. However, if you store a sequence of commands to a text file and tell the shell to execute the text file instead of entering the commands, that is known as a shell program or shell script.

Shell script defined as

"Shell script is a series of command(s) stored in a plain text file. A shell script is similar to a batch file in MS-DOS, but is much more powerfull."

Each shell script consists of

• Shell commands such as if..else, do..while
• Linux text processing utilities such as grep, awk, cut
• Linux binary commands such as w, who, free etc

Did you know?

• It is the shell that lets you run different commands without having to type the full pathname to them even when they do not exist in the current directory
• It is the shell that expands wildcard characters, such as * or ?, thus saving you laborious typing
• It is the shell that gives you the ability to run previously run commands without having to type the full command again by pressing the up arrow, or pulling up a complete list with the history command
• It is the shell that does input, output and error redirection

Why shell scripting?

• Shell scripts can take input from a user or file and output them to the screen
• Whenever you find yourself doing the same task over and over again you should use scripting i.e. repetitive task automation
• Time saving
• Creating your own power tools/utilities
• Customizing administrative task
• Since scripts are well tested, the chances of errors are reduced while configuring services or system administration task such as adding new users
• Practical examples where shell scripting actively used:
  • Monitoring your Linux system
  • Data backup and creating snapshots
  • Dumping Oracle or MySQL database for backup
  • Creating email based alert system
  • Find out what processes are eating up your system resources
  • Find out available and free memory
  • Find out all logged in users and what they are doing
  • Find out if all necessary network services are running or not. For example if web server failed then send an alert to system administrator via a pager or an email
  • Find out all failed login attempt, if login attempt are continue repeatedly from same network IP automatically block all those IPs accessing our network/service via firewall
  • User administration as per your own security policies
  • Find out information about local or remote servers
  • Configure server such as BIND (DNS server) to add zone entries

Shell scripting is fun. It is useful to create nice (perhaps ugly) things in shell scripting. Here are few script example I use everyday:

• Find out today's weather (useful when you are busy in a chat room)
• Find out what that site is running (just like netcraft)
• Download RSS feeds and display them as you login or in your email
• Find out the name of the MP3 file you are listening to
• Monitor your domain expiry date every day

Which Shell we are going use in this tutorial?

We are using bash shell.

Learning Objectives

After completing this tutorial, you will be able to:

• Understand the basis of Linux shell scripting
• Write shell scripts and use it to save time with automated scripts
• Customizing shell start-up files
• Creating nifty utilities
• Control your administration tasks such as Linux user management, Linux system monitoring etc

Getting started with Shell Programming

Objective

Upon completion of this module, you will be able to do following:

• Understand basic steps to write a shell scripts
• Define variables and use them in a shell
• Shell arithmetic
• Basic input/output statements
• Use of wild card characters
• Understanding necessary concepts such as pipes, filter and processes

Understand basic steps to write a shell scripts

Following steps are required to write shell script:

1. Create a script
2. Setup a executable permission on a script
3. Run a script
4. Debug a script if required

Let us try to understand each step in depth, if you are already familiar with above four steps please skip to next section.

Create a script

As discussed earlier shell scripts stored in plain text file, generally one command per line. You can use text editor such as vi or emacs. I recommend using vim (Vi Improved) as it is equipped with features such as syntax highlighting and indenting. Most modern Linux (or *BSD) distribution comes with vim. You can start vi or vim from shell prompt by typing any one of the following command:

$ vi  myscript.bash

$ vi myscript.sh

$ vim myscript.bash

Make sure you use .bash or .sh file extension for each script. This ensures easy identification of shell script.

Setup executable permission

Once script is created, you need to setup executable permission on a script. Why to setup executable permission, might be next question in your mind, right?

Simple,

• Without executable permission, running a script is almost impossible
• Besides executable permission, script must have a read permission

Syntax to setup executable permission:

chmod permission your-script-name

Examples:

$ chmod +x your-script-name

$ chmod 755 your-script-name

Run a script (execute a script)

Now your script is ready with proper executable permission on it. Next, test script by running it.

Syntax:

bash your-script-name

sh your-script-name

./your-script-name

Examples:

$ bash bar

$ sh bar

$ ./bar

In last example, you are using . (dot) command which read and execute commands from filename in the current shell. If filename does not contain a slash, file names in PATH are used to find the directory containing filename. For example:

./bar

bar script executed from current directory. The specialty of dot (.) command is you do not have to setup an executable permission on script.

Debug a script if required

While programming shell sometimes you need to find out errors (bugs) in shell script and correct all errors (remove errors i.e. debug script). For this purpose you can pass -v and -x option to sh/bash command to debug the shell script. General syntax is as follows: Syntax:

sh   option   { shell-script-name }

OR

bash   option   { shell-script-name }

Where,

• -v: print shell input lines as they are read
• -x: after expanding each simple-command, bash displays the expanded value of PS4 system variable, followed by the command and its expanded arguments.

Debugging discussed later in depth. Now you are ready to write first shell script that will print "Knowledge is Power" on screen. See the common vi command list , if you are new to vi.

$ vi first

#

# My first shell script

#

clear

echo "Knowledge is Power"

After saving the above script, you can run the script as follows:

$ ./first

This will not run script since we have not set execute permission for our script first; to do this type command

$ chmod 755 first

$ ./first

First screen will be clear, then Knowledge is Power is printed on screen.

Script Command(s) Meaning$ vi first Start vi editor#

# My first shell script

## followed by any text is considered as comment. Comment gives more information about script, logical explanation about shell script.

Syntax:

# comment-textclearclear the screenecho "Knowledge is Power"To print message or value of variables on screen, we use echo command, general form of echo command is as follows

syntax:

echo "Message"

How Shell Locates the file (My own bin directory to execute script)

Tip: For shell script file try to give file extension such as .sh, which can be easily identified by you as shell script.

Exercise:

1)Write following shell script, save it, execute it and note down the it's output.

$ vi ginfo

#

#

# Script to print user information who currently login , current date & time

#

clear

echo "Hello $USER"

echo "Today is \c ";date

echo "Number of user login : \c" ; who | wc -l

echo "Calendar"

cal

exit 0

Future Point: At the end why statement exit 0 is used? See exit status in below sections for more information.

Variables in Shell

To process our data/information, data must be kept in computers RAM memory. RAM memory is divided into small locations, and each location had unique number called memory location/address, which is used to hold our data. Programmer can give a unique name to this memory location/address called memory variable or variable (Its a named storage location that may take different values, but only one at a time).

In Linux (Shell), there are two types of variable:

(1) System variables - Created and maintained by Linux itself. This type of variable defined in CAPITAL LETTERS.

(2) User defined variables (UDV) - Created and maintained by user. This type of variable defined in lower letters.

You can see system variables by giving command like $ set, some of the important System variables are:

System Variable Meaning BASH=/bin/bashOur shell nameBASH_VERSION=1.14.7(1)Our shell version nameCOLUMNS=80No. of columns for our screenHOME=/home/vivekOur home directoryLINES=25No. of columns for our screenLOGNAME=studentsstudents Our logging nameOSTYPE=LinuxOur Os typePATH=/usr/bin:/sbin:/bin:/usr/sbinOur path settingsPS1=[\u@\h \W]\$Our prompt settingsPWD=/home/students/CommonOur current working directorySHELL=/bin/bashOur shell nameUSERNAME=vivekUser name who is currently login to this PC

NOTE that Some of the above settings can be different in your PC/Linux environment. You can print any of the above variables contains as follows:

$ echo $USERNAME

$ echo $HOME

Exercise:

1) If you want to print your home directory location then you give command:

a)

$ echo $HOME

OR

(b)

$ echo HOME

Which of the above command is correct & why?

Caution:Do not modify System variable this can some time create problems.

How to define User defined variables (UDV)

To define UDV use following syntax

Syntax:

variable name=value

'value' is assigned to given 'variable name' and Value must be on right side = sign.

Example:

$ no=10

# this is ok

$ 10=no

# Error, NOT Ok, Value must be on right side of = sign.

To define variable called 'vech' having value Bus

$ vech=Bus

To define variable called n having value 10

 $ n=10

Rules for Naming variable name (Both UDV and System Variable)

(1) Variable name must begin with Alphanumeric character or underscore character (_), followed by one or more Alphanumeric character. For e.g. Valid shell variable are as follows

HOME

SYSTEM_VERSION

vech

no

(2) Don't put spaces on either side of the equal sign when assigning value to variable. For e.g. In following variable declaration there will be no error

$ no=10

But there will be problem for any of the following variable declaration:

$ no =10

$ no= 10

$ no = 10

(3) Variables are case-sensitive, just like filename in Linux. For e.g.

$ no=10

$ No=11

$ NO=20

$ nO=2

Above all are different variable name, so to print value 20 we have to use $ echo $NO and not any of the following

 $ echo $no

# will print 10 but not 20

 $ echo $No

# will print 11 but not 20

 $ echo $nO

# will print 2 but not 20

(4) You can define NULL variable as follows (NULL variable is variable which has no value at the time of definition) For e.g.

$ vech=

$ vech=""

Try to print it's value by issuing following command

 $ echo $vech

Nothing will be shown because variable has no value i.e. NULL variable.

(5) Do not use ?,* etc, to name your variable names.

How to print or access value of UDV (User defined variables)

To print or access UDV use following syntax

Syntax:

$variablename

Define variable vech and n as follows:

$ vech=Bus

$ n=10

To print contains of variable 'vech' type

$ echo $vech

It will print 'Bus',To print contains of variable 'n' type command as follows

$ echo $n

Caution: Do not try $ echo vech, as it will print vech instead its value 'Bus' and $ echo n, as it will print n instead its value '10', You must use $ followed by variable name.

Exercise

Q.1.How to Define variable x with value 10 and print it on screen.

Q.2.How to Define variable xn with value Rani and print it on screen

Q.3.How to print sum of two numbers, let's say 6 and 3?

Q.4.How to define two variable x=20, y=5 and then to print division of x and y (i.e. x/y)

Q.5.Modify above and store division of x and y to variable called z

Q.6.Point out error if any in following script

$ vi variscript

#

#

# Script to test MY knowledge about variables!

#

myname=Vivek

myos = TroubleOS

myno=5

echo "My name is $myname"

echo "My os is $myos"

echo "My number is myno, can you see this number"

echo Command

Use echo command to display text or value of variable.

echo [options] [string, variables...]

Displays text or variables value on screen.

Options

-n Do not output the trailing new line.

-e Enable interpretation of the following backslash escaped characters in the strings:

\a alert (bell)

\b backspace

\c suppress trailing new line

\n new line

\r carriage return

\t horizontal tab

\\ backslash

For e.g. $ echo -e "An apple a day keeps away \a\t\tdoctor\n"

How to display colorful text on screen with bold or blink effects, how to print text on any row, column on screen

Shell Arithmetic

Use to perform arithmetic operations.

Syntax:

expr op1 math-operator op2

Examples:

$ expr 1 + 3

$ expr 2 - 1

$ expr 10 / 2

$ expr 20 % 3

$ expr 10 \* 3

$ echo `expr 6 + 3`

Note:

expr 20 %3 - Remainder read as 20 mod 3 and remainder is 2.

expr 10 \* 3 - Multiplication use \* and not * since its wild card.

For the last statement not the following points

(1) First, before expr keyword we used ` (back quote) sign not the (single quote i.e. ') sign. Back quote is generally found on the key under tilde (~) on PC keyboard OR to the above of TAB key.

(2) Second, expr is also end with ` i.e. back quote.

(3) Here expr 6 + 3 is evaluated to 9, then echo command prints 9 as sum

(4) Here if you use double quote or single quote, it will NOT work

For e.g.

$ echo "expr 6 + 3" # It will print expr 6 + 3

$ echo 'expr 6 + 3' # It will print expr 6 + 3

More about Quotes

There are three types of quotes

Quotes Name Meaning"Double Quotes"Double Quotes" - Anything enclose in double quotes removed meaning of that characters (except \ and $).'Single quotes'Single quotes' - Enclosed in single quotes remains unchanged.`Back quote

`Back quote` - To execute command

Example:

$ echo "Today is date"

Can't print message with today's date.

$ echo "Today is `date`".

It will print today's date as, Today is Tue Jan ....,Can you see that the `date` statement uses back quote?

Exit Status

By default in Linux if particular command/shell script is executed, it return two type of values which is used to see whether command or shell script executed is successful or not.

(1) If return value is zero (0), command is successful.

(2) If return value is nonzero, command is not successful or some sort of error executing command/shell script.

This value is know as Exit Status.

But how to find out exit status of command or shell script?

Simple, to determine this exit Status you can use $? special variable of shell.

For e.g. (This example assumes that unknow1file doest not exist on your hard drive)

$ rm unknow1file

It will show error as follows

rm: cannot remove `unkowm1file': No such file or directory

and after that if you give command

$ echo $?

it will print nonzero value to indicate error. Now give command

$ ls

$ echo $?

It will print 0 to indicate command is successful.

Exercise

Try the following commands and not down the exit status:

$ expr 1 + 3

$ echo $?

$ echo Welcome

$ echo $?

$ wildwest canwork?

$ echo $?

$ date

$ echo $?

$ echon $?

$ echo $?

The read Statement

Use to get input (data from user) from keyboard and store (data) to variable.

Syntax:

read variable1, variable2,...variableN

Following script first ask user, name and then waits to enter name from the user via keyboard. Then user enters name from keyboard (after giving name you have to press ENTER key) and entered name through keyboard is stored (assigned) to variable fname.

$ vi sayH

#

#Script to read your name from key-board

#

echo "Your first name please:"

read fname

echo "Hello $fname, Lets be friend!"

Run it as follows:

$ chmod 755 sayH

$ ./sayH

Your first name please: vivek

Hello vivek, Lets be friend!