Updated: 2017-04-11 12:35 EDT

1 Shells can read from keyboards or from text files

A shell (e.g. bash) can be used in one of two ways:

This page discusses shell scripts.

2 Create Shell Scripts to automate sysadmin tasks

A shell script is a list of commands, stored in a text file, that can be executed by a shell. The shell reads the commands from the file instead of reading commands from your keyboard.

If we have a set of commands that we want to run on a regular basis, we could create a text file containing those commands. The text file allows us to use a single file to hold many commands. This text file is called a Shell Script. For example:

$ cat myscript.sh                      # show the content of the file
echo "This is a shell script."
echo "Done"

$ sh -u myscript.sh                    # pass the file to a shell "sh"
This is a shell script
Wed Nov  2 04:47:44 EDT 2016

Like most commands that process files, if you give the shell a file to read, it will read from that file and not from your keyboard. Unlike most other commands, shells will treat each line in the file as a command to be run.

There are two ways to run the list of commands in the script file:

2.1 Run Method 1: Using the shell to read the script file

The most obvious way to run a list of commands stored in a shell script is to type the name of the shell program that we want to use to read the script file (e.g. sh or bash) and give it the script file name as a file name argument:

$ bash -u myscript.sh
$ sh -u myscript.sh

The -u argument to the shell tells bash to report undefined variables used in the script and abort if any are used. Using -u is always good practice, in case there might be a typing mistake in the script.

With a file name argument, this script-reading copy of the shell program does not read your keyboard but instead reads the script file and executes each line in the script file as a command line. When it reaches the end of the script file, the script shell exits and you get your original interactive shell prompt back.

A separate shell process is reading the script file, not your interactive shell that is reading your keyboard. Changes made to the environment of the shell running the script (e.g. changing directories, setting variables) will not affect your interactive shell. Your interactive shell is protected from environment changes that the shell script might make as it runs.

2.2 Run Method 2: Making the script file executable

The other, more elegant way to run the list of commands in a script file is to make the script text file executable (only needs to be done once) and then simply type its name as a command name to the shell:

$ chmod ugo+x myscript.sh              # only needs to be done once
$ ./myscript.sh

The name of the script can be typed into your interactive shell and a second copy of the shell will execute the script by reading the script file and finding and running the commands in the script file. When that script shell reaches the end of the script file, that script shell exits and you get your original shell prompt back.

Again, a separate shell process is reading the script file, not your interactive shell that is reading your keyboard.

3 Use your own bin directory for your scripts

If you put your scripts in your own directory, e.g. ~/bin and append that directory name at the end of your shell $PATH, then your script names behave just like other command names:

$ mkdir ~/bin                          # only need to do this once
$ PATH=$PATH:~/bin                     # could do this in your .bashrc
$ mv myscript.sh ~/bin/myscript        # choose a unique name
$ myscript                             # execute your script!

As with every command name typed into the shell, the shell will search for the command name myscript in all the directories in your search $PATH, finally finding it in the file ~/bin/myscript and executing it. Most people set their $PATH variable to include their personal bin/ directory at log-in time via their .bashrc.

In short:

  1. Create an executable shell script with a unique name.
  2. Move the script file to a directory that is in your search $PATH
  3. Type the name and the script runs!

As a system administrator, you can make your job easier by writing your own custom shell scripts containing lists of commands to help automate tasks. You can often bring these scripts with you when you change jobs. Sysadmin may have dozens of personal shell scripts in their own personal bin directory.

3.1 Many command names are shell scripts

Many command names in the system are actually shell scripts.

You can find out how many scripts are in the standard system command directories /bin and /usr/bin:

$ file /bin/* /usr/bin/* | fgrep 'script' | wc
443    3203   37857

$ file /bin/gunzip
/bin/gunzip: Bourne-Again shell script, ASCII text executable

Above we see that the gunzip command is actually a shell script.

4 Standard Script Header – three lines

Use this exact set of Standard Script Header lines as the first three lines of most shell scripts you write in this course:

#!/bin/sh -u
PATH=/bin:/usr/bin ; export PATH          # (2) PATH line
umask 022                                 # (3) umask line

Each of the three lines above is described in detail below.

4.1 Line 1: Interpreter Magic Number line, or Shebang

The very first line of every executable standard shell script in this course must be an interpreter magic number line, or “shebang” line (short for “shell bang” or “shell exclamation”) with this exact text (12 characters plus newline):

#!/bin/sh -u

The two characters #! need to be the first two characters in the file, because they form a Magic Number that tells the kernel this is an executable script.

No additional shell comments are allowed on the first line of the script, because this line is executed by the Linux kernel, not by the shell, and the kernel does not understand shell comments. Use the exact line given above for all the scripts in this course.

This first line begins with a hashtag character # and is a shell comment that will be ignored by the shell when the shell reads the script file.

4.1.1 The Shebang line specifies the program to run

The #! magic number is followed by the absolute path of a binary executable program file that kernel will run when you execute this script.

This file pathname on this first line is how the kernel chooses which program will process your script file. The program specified is almost always the standard system shell /bin/sh or (less commonly) /bin/bash.

The kernel will hand the script file name to the program as an argument. A shell program will read and execute the lines in the file as commands. For example, if the first line of a script named myscript.sh is #!/bin/sh -u, then the kernel will execute this command line when it executes the script file:

/bin/sh -u myscript.sh

The /bin/sh program will read and execute commands from the myscript.sh file that is its argument.

A limited number of option arguments can be supplied to the program on the same first line of the script. The -u option argument to the shell tells the shell to generate an error if the script tries to make use of a variable that’s not set. We always use -u to force the shell to detect undefined variables:

  • Undefined variables should never happen if the script is well written and tested.
  • If the error does happen, it’s better to have the shell stop processing than continue on using incorrect data.

If you want to pass the shell script as an argument to a shell, remember to also pass the options given in the shebang line. For example, if the shebang line in the file myscript.sh is #!/bin/sh -u, you must type this command line to run it, including the option:

$ /bin/sh -u myscript.sh

When you call the shell directly and pass the script as an argument, the shebang line is treated as a comment line and is ignored by the shell.

4.1.2 Any program name is possible after #!

Any executable program name can be specified after the #! and the kernel will run that program, passing to the program the name of the script as an argument. If the shebang line in file myscript.sh is:

#!/bin/sh -u      ...then the kernel executes: /bin/sh -u myscript.sh
#!/bin/bash -u    ...then the kernel executes: /bin/bash -u myscript.sh
#!/usr/bin/csh    ...then the kernel executes: /usr/bin/csh myscript.sh

Q: What would happen if you changed the first line of a script file to be:

  • #!/bin/ls -l
  • #!/usr/bin/wc
  • #!/bin/cat
  • #!/bin/rm

4.2 Line 2: Set the shell search PATH

The second line of the Standard Script Header sets the search PATH for the script:

PATH=/bin:/usr/bin ; export PATH          # (2) PATH line

4.3 Line 3: Set the shell umask

The third line of the Standard Script Header sets the permissions umask for the script:

umask 022                                 # (3) umask line

5 The body of the script (after the script header)

5.1 stdin stdout stderr are unchanged

6 Arguments to the script on the command line

6.1 Positional Parameter variables: $0, $1, $2, $#, etc.

6.2 Aggregate Positional Parameter variables: $* and $@

Shell variables $* and $@ both expand to be all of the arguments supplied on the command line. They behave differently when double quoted:

We will explore how these variables work in a separate document.

7 Sample script One – variables and positional parameters

Below is a small sample shell script that demonstrates the use of various shell variables, including positional parameters. Put these lines below in a file named positional.sh and make it executable and then run it.

The first three lines of the script are a copy of the standard script header:

#!/bin/sh -u
PATH=/bin:/usr/bin ; export PATH
umask 022

# The lines below are the body of this shell script:
myvar="howdy doody"
echo "The value of \$myvar is: $myvar"       # use backslash to hide first $

echo "The command name (the script name) is $0"
echo "The number of command line arguments is: $#"
echo "All the command line arguments are: $*"
echo "The first argument is: $1"             # fails if no arguments
echo "The second argument is: $2"            # fails if not two arguments
echo "The third argument is: $3"             # fails if not three arguments

Because the script is running the shell with the -u option, the shell will issue an error message if any variables, including positional parameter variables, are undefined. To avoid these errors, make sure you execute the above script with at least three command line arguments:

$ chmod ugo+x positional.sh               # only needs to be done once
$ ./positional.sh one two three           # give at least three arguments
The value of $myvar is: howdy doody
The command name (the script name) is ./positional.sh
The number of command line arguments is: 3
All the command line arguments are: one two three
The first argument is: one
The second argument is: two
The third argument is: three

You can also run any script file, even if it isn’t executable, by passing its file name to the shell as a file name argument:

$ sh -u positional.sh

The shebang line is not used when you run a shell script the above way, which is why we must explicitly supply the -u option to check for undefined variables.

8 Comments in your own shell scripts – no “Instructor-Type” comments

Many of the comments in script file examples in this course are “Instructor-Type” comments and are not appropriate for real scripts that you write. (e.g. Comments such as This is the body of the shell script or use backslash to hide first $)

“Instructor-Type” comments explain features about the syntax and structure of a shell script and are used to teach scripting to beginners. I put Instructor-Type comments in my examples because I am teaching you how to write scripts.

You would not put Instructor-Type comments in your own scripts, because the scripts you write always assume that you and your script readers already know how to write shell scripts.

Do not put “Instructor-Type” comments into the scripts that you submit for marking. Comments should address what the script is doing, not how ordinary script features work. Do not submit my Instructor-Type comments back to me again in scripts that you write.

| Ian! D. Allen, BA, MMath  -  idallen@idallen.ca  -  Ottawa, Ontario, Canada
| Home Page: http://idallen.com/   Contact Improv: http://contactimprov.ca/
| College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/
| Defend digital freedom:  http://eff.org/  and have fun:  http://fools.ca/

Plain Text - plain text version of this page in Pandoc Markdown format

Campaign for non-browser-specific HTML   Valid XHTML 1.0 Transitional   Valid CSS!   Creative Commons by nc sa 3.0   Hacker Ideals Emblem   Author Ian! D. Allen