Winter 2017 - January to April 2017 - Updated 2018-11-01 17:32 EDT
A shell (e.g. bash
) can be used in one of two ways:
This page discusses shell scripts.
You may find some parts of these advanced Lynda.com videos useful as you start writing your own shell scripts. These links require you to have created a free account on lynda.com via the Algonquin Lynda.com Link:
A shell script is a list of commands, stored in a text file, that can be executed by a shell. The shell reads the commands from the file instead of reading commands from your keyboard.
If we have a set of commands that we want to run on a regular basis, we could create a text file containing those commands. The text file allows us to use a single file to hold many commands. This text file is called a Shell Script. For example:
$ cat myscript.sh # show the content of the file
echo "This is a shell script."
date
echo "Goodbye $USER"
$ sh myscript.sh # pass the file to a shell "sh"
This is a shell script
Wed Nov 2 04:47:44 EDT 2016
Goodbye idallen
Like most commands that process files, if you give the shell a file to read, it will read from that file and not from your keyboard. Unlike most other commands, shells will treat each line in the file as a command to be run.
There are two ways to run the list of commands in the script file:
The most obvious way to run a list of commands stored in a shell script is to type the name of the shell program that we want to use to read the script file (e.g. sh
or bash
) and give it the script file name as a file name argument:
$ bash -u myscript.sh
$ sh -u myscript.sh
The
-u
argument to the shell tells the shell to report undefined variables used in the script and abort if any are used. Using-u
is always good practice, in case there might be a typing mistake in the script.
With a file name argument, this script-reading copy of the shell program does not read your keyboard but instead reads the script file and executes each line in the script file as a command line. When it reaches the end of the script file, the script shell exits and you get your original interactive shell prompt back.
A separate shell process is reading the script file, not your interactive shell that is reading your keyboard. Changes made to the environment of the shell running the script (e.g. changing directories, setting variables) will not affect your interactive shell. Your interactive shell is protected from environment changes that the shell script might make as it runs.
The other, more elegant way to run the list of commands in a script file is to make the script text file executable (only needs to be done once) and then simply type its name as a command name to the shell:
$ chmod ugo+x myscript.sh # only needs to be done once
$ ./myscript.sh
The name of the script can be typed into your interactive shell and a second copy of the shell will execute the script by reading the script file and finding and running the commands in the script file. When that script shell reaches the end of the script file, that script shell exits and you get your original shell prompt back.
Again, a separate shell process is reading the script file, not your interactive shell that is reading your keyboard.
bin
directory for your scriptsIndexIf you put your scripts in your own directory, e.g. ~/bin
and append that directory name at the end of your shell $PATH
, then your script names behave just like other command names:
$ mkdir ~/bin # only need to do this once
$ PATH=$PATH:~/bin # could do this in your .bashrc
$ mv myscript.sh ~/bin/myscript # choose a unique name
$ myscript # execute your script!
As with every command name typed into the shell, the shell will search for the command name myscript
in all the directories in your search $PATH
, finally finding it in the file ~/bin/myscript
and executing it. Most people set their $PATH
variable to include their personal bin/
directory at log-in time via their .bashrc
.
In short:
$PATH
As a system administrator, you can make your job easier by writing your own custom shell scripts containing lists of commands to help automate tasks. You can often bring these scripts with you when you change jobs. Sysadmin may have dozens of personal shell scripts in their own personal
bin
directory.
Many command names in the system are not binary executable programs; they are actually shell scripts. You can find out how many scripts are in the standard system command directories /bin
and /usr/bin
:
$ file /bin/* /usr/bin/* | fgrep 'script' | wc
446 3232 40941
$ file /bin/gunzip
/bin/gunzip: POSIX shell script, ASCII text executable
Above we see that the gunzip
command is actually a shell script, not a binary executable program. Because it is a text file, you can read it and discover that all it does is call gzip -d
with whatever arguments you give it.
Shell scripts need a consistent environment when they run. For this, course we will specify three things at the start of every shell script:
/bin/sh -u
/bin:/usr/bin
umask
that will be used in the script: 022
To do this, use this exact set of Standard Script Header lines as the first three lines of shell scripts you write in this course:
#!/bin/sh -u
PATH=/bin:/usr/bin ; export PATH # (2) PATH line
umask 022 # (3) umask line
Each of the three lines above is described in detail below.
#
indicates the start of a shell comment. The hashtag and everything to the right, to the end of the line, are ignored by the shell.#!
IndexThe very first line of every executable standard shell script in this course must be an interpreter magic number line, or shebang line (short for “shell bang” or “shell exclamation”) with this exact text (12 characters plus newline):
#!/bin/sh -u
The first two characters #!
need to be the first two characters in the file, because together they form a Magic Number that tells the kernel this is an executable script file with a special format.
No additional shell comments (#
) are allowed on the first line of the script, because this first shebang #!
line must be executed by the Linux kernel, not by the shell, and the kernel does not understand shell comments. Use the exact line given above for all the scripts in this course.
#!
line tells the kernel which program to runIndexThe #!
magic number at the start of the file must be followed by the absolute path of a binary executable program file that kernel will run when you execute this file.
This file pathname on this first line is how the kernel chooses which program will process your script file. The program specified for a shell script is almost always the standard system shell /bin/sh
or (less commonly) /bin/bash
:
#!/bin/sh -u
If an executable file starts with #!
, the kernel will execute the program name that follows the #!
and hand the file name being executed to the program as an argument. A shell program (e.g. /bin/sh
) will read and execute the lines in the file as commands. For example, if the first line of a script named myscript.sh
is the standard shebang line #!/bin/sh -u
, then the kernel will create and then execute this command line when it executes the script file:
/bin/sh -u myscript.sh
The /bin/sh
program will read and execute commands from the myscript.sh
file that is its argument, using the -u
shell option.
A limited number of option arguments can be supplied to the program on the shebang line, after the program name. The -u
option argument to a Bourne shell program tells the shell to generate an error if the script tries to make use of a variable that’s not set and has no value. We always use -u
to force the shell to detect undefined variables:
#!
line is a comment when the shell reads itIndexThe shebang line deliberately begins with a hashtag shell comment character #
and will be ignored as a shell comment when the shell program specified after the #!
reads the script file.
If, instead of executing the shell script, you want to pass the shell script as an argument to a shell program, remember to also pass the options given in the shebang line. For example, if the shebang line in the file myscript.sh
is #!/bin/sh -u
, you must type this command line to run it properly, remembering to include the -u
option:
$ /bin/sh -u myscript.sh
When you call the shell directly and pass the script as an argument, the shebang line is treated as a comment line and is ignored by the shell. You have to remember to include the options on the shebang line yourself, as shown in the above command line.
#!
IndexAny executable program name can be specified after the #!
on the shebang line, and the kernel will run that program, passing to the program the name of the file being executed as an argument. If the shebang line in file myscript.sh
is:
#!/bin/sh -u ...then the kernel executes: /bin/sh -u myscript.sh
#!/bin/bash -u ...then the kernel executes: /bin/bash -u myscript.sh
#!/usr/bin/csh ...then the kernel executes: /usr/bin/csh myscript.sh
Q: What would happen if you changed the first line of a script file to be:
#!/bin/ls -l
#!/usr/bin/wc
#!/bin/cat
#!/bin/rm
The second line of the Standard Script Header sets the search PATH for the script:
PATH=/bin:/usr/bin ; export PATH # (2) PATH line
PATH
so that the script will run the standard commands from the standard locations.PATH
that may not be correct for the commands used in the shell script.$PATH
if your script needs to run those commands:
PATH=/bin:/usr/bin:/sbin:/usr/sbin ; export PATH
Remember that, since the script is being executed by its own shell program, nothing you set or change inside the script will affect any shell outside the script. Settings in your login shell remain unchanged.
The third line of the Standard Script Header sets the permissions umask
for the script:
umask 022 # (3) umask line
umask
value.umask
value that may not be correct for the commands used in the shell script.umask
of 077
: umask 077
Remember that, since the script is being executed by its own shell program, nothing you set or change inside the script will affect any shell outside the script. Settings in your login shell remain unchanged.
The “body” of the script is the code that follows the script header.
#
and extend to the end of the line.#!
line of the script is a comment line and is ignored when the shell reads the script file.stdin stdout stderr
are unchangedIndexThe stdin, stdout, stderr of the commands inside the script are the stdin, stdout, stderr of the script as it is run.
When a command in your script prints output to stdout, the output goes to stdout just as if the command were not in the script.
When a command in your script reads from stdin, the input comes from stdin just as if the command were not in the script.
Redirection applied to the script applies to every command that is run inside the script file:
$ sh -u ./myscript.sh >out # all script stdout will go into file out
$ ./myscript.sh >out # all script stdout will go into file out
$ myscript >out # all script stdout will go into file out
We may supply arguments to our script on the command line, the same as we do when using any command we type:
$ sh -u myscript.sh arg1 arg2 "arg 3" "arg 4"
$ ./myscript.sh arg1 arg2 "arg 3" "arg 4"
$ myscript arg1 arg2 "arg 3" "arg 4"
These command-line arguments are accessible inside the script using positional parameter variables such as $1
, $2
, etc.
$0
, $1
, $2
, $#
, etc.IndexWhen our script is running, the command line arguments are available as Positional Parameter variables inside the script starting at 1: $1
, $2
, etc.
The script may access the argument values through these variable names.
The nine shell positional parameter variables $1
through $9
contain the first nine arguments passed to the script on the command line.
To access arguments past $9
, surround the number with braces and use ${10}
, ${11}
, etc.
The variable $#
holds the number of arguments used on the command line, not counting the command name (which is never an argument).
Shell variable $0
is the pathname of the script itself, often used in script messages to tell you which script is running:
echo "$0: Input file '$file' will be used"
$*
and $@
IndexShell variables $*
and $@
both expand to be all of the arguments supplied on the command line. They behave differently when double quoted:
"$*"
is one word (one argument; one token) with spaces between the contained command line argument values."$@"
behaves unlike any other double-quoted string. It is a list of multiple words (multiple arguments; multiple tokens) where each command line argument is separately quoted. Using double-quoted "$@"
inside a shell script creates multiple arguments, not a single argument as you would expect. Double-quoted "$@"
is the only variable that creates multiple arguments when quoted.We will explore how these variables work in a separate document.
Below is a small sample shell script that demonstrates the use of various shell variables, including positional parameters. Put these lines below in a file named positional.sh
and make it executable and then run it.
The first three lines of the script are a copy of the standard script header:
#!/bin/sh -u
PATH=/bin:/usr/bin ; export PATH
umask 022
# The lines below are the body of this shell script:
#
myvar="howdy doody"
echo "The value of \$myvar is: $myvar" # use backslash to hide first $
echo "The command name (the script name) is $0"
echo "The number of command line arguments is: $#"
echo "All the command line arguments are: $*"
echo "The first argument is: $1" # fails if no arguments
echo "The second argument is: $2" # fails if not two arguments
echo "The third argument is: $3" # fails if not three arguments
Because the script is running the shell with the -u
option, the shell will issue an error message if any variables, including positional parameter variables, are undefined. To avoid these errors, make sure you execute the above script with at least three command line arguments:
$ chmod ugo+x positional.sh # only needs to be done once
$ ./positional.sh one two three # give at least three arguments
The value of $myvar is: howdy doody
The command name (the script name) is ./positional.sh
The number of command line arguments is: 3
All the command line arguments are: one two three
The first argument is: one
The second argument is: two
The third argument is: three
You can also run any script file, even if it isn’t executable, by passing its file name to the shell as a file name argument:
$ sh -u positional.sh
The shebang line is not used when you run a shell script the above way, which is why we must explicitly supply the -u
option to check for undefined variables.
shift
IndexThe built-in shell command shift
will delete the first command line argument, causing all the positional parameter numbers to shift down one:
$ cat myscript.sh
#!/bin/sh -u
echo "All arguments: $*"
echo "First argument \$1 is '$1' and second argument \$2 is '$2'; count is $#"
echo "Doing a shift"
shift
echo "All arguments: $*"
echo "First argument \$1 is '$1' and second argument \$2 is '$2'; count is $#"
echo "Doing a shift"
shift
echo "All arguments: $*"
echo "First argument \$1 is '$1' and second argument \$2 is '$2'; count is $#"
$ ./myscript.sh one two three four
All arguments: one two three four
First argument $1 is 'one' and second argument $2 is 'two'; count is 4
Doing a shift
All arguments: two three four
First argument $1 is 'two' and second argument $2 is 'three'; count is 3
Doing a shift
All arguments: three four
First argument $1 is 'three' and second argument $2 is 'four'; count is 2
As you can see, after a shift
, all the arguments shift down one place. Argument $2
becomes $1
(because the first argument is gone); argument $3
becomes argument $2
, etc. The shift
command is most useful inside a looping control structure such as a while
or for
loop.
9 Comments in your own shell scripts – no “Instructor-Type” commentsIndex
Many of the comments in script file examples in this course are “Instructor-Type” comments and are not appropriate for real scripts that you write. (e.g. Comments such as
This is the body of the shell script
oruse backslash to hide first $
)“Instructor-Type” comments explain features about the syntax and structure of a shell script and are used to teach scripting to beginners. I put Instructor-Type comments in my examples because I am teaching you how to write scripts.
You would not put Instructor-Type comments in your own scripts, because the scripts you write always assume that you and your script readers already know how to write shell scripts.
Do not put “Instructor-Type” comments into the scripts that you submit for marking. Comments should address what the script is doing, not how ordinary script features work. Do not submit my Instructor-Type comments back to me again in scripts that you write.
Plain Text - plain text version of this page in Pandoc Markdown format
Author Ian! D. Allen