Winter 2019 - January to April 2019 - Updated 2021-09-03 12:03 EDT
Single quotes, double quotes, and backslashes tell the shell which parts of command lines to treat as ordinary (not special) characters.
The quoting delimits (identifies) a string of characters; the quoting mechanism is removed and is not part of the string passed to the command. Only the characters being quoted are passed to the command.
Another Quote Tutorial: http://www.grymoire.com/Unix/Quote.html
Below and in the next link are explanations of what happened in the above comic.
Databases are accessed using a query language, typically Structured Query Language. A database query in SQL might look like this:
SELECT * FROM Students WHERE ( NAME = '$NAME' ) ;
The variable $NAME
is the name of the student, and the query would
be modified by inserting that name into the query in place of the
$NAME
variable. For a normal student name used in place of $NAME
,
e.g. Leslie Smith
, the query simply looks up the student by name in
the database:
SELECT * FROM Students WHERE ( NAME = 'Leslie Smith' ) ;
For little Bobby Tables, his name Robert'); DROP TABLE Students;
also gets inserted into the database query in place of $NAME
in the
same way, but the presence of quotes and other special characters in
the name completely changes the meaning of the query:
SELECT * FROM Students WHERE ( NAME = 'Robert'); DROP TABLE Students;' ) ;
The SQL query now tries to find a student named 'Robert'
and then (after
the semicolon ;
) deletes the entire Students
table from the database.
“Sanitizing inputs” refers to the process of removing special characters from the input so that they don’t defeat the quoting used in the query and change the meaning of the query.
Shells read input and find and run commands, passing command line arguments to the commands. Shells treat a number of characters specially on the command lines; these are called shell “meta-characters”.
The most common shell meta-character is the blank or space character
that shells use to separate arguments. The shell separates arguments
with one blank or a dozen blanks. Other shell meta-characters include:
$
, *
, ;
, >
, ?
, &
, |
, etc.
“Quoting” is the generic name given to the action of protecting shell meta-characters from being treated specially by the shell, usually by surrounding the string containing the special character(s) with quotes, or using backslashes in front of the individual characters:
$ echo Semicolon is special; it divides command lines.
Semicolon is special
it: command not found
$ echo Semicolon is special\; quoting hides it from the shell.
Semicolon is special; quoting hides it from the shell.
$ echo 'Semicolon is special; quoting hides it from the shell.'
Semicolon is special; quoting hides it from the shell.
Quoting hides the special character(s) from processing by the shell. The characters are passed, unchanged, to the command.
Sometimes you don’t want the shell to act on its meta-characters; you want these characters passed to a command as part of the command arguments:
$ touch a b c d
$ echo *** BOOM *** # shell expands GLOB characters
a b c d BOOM a b c d
$ echo "*** BOOM ***" # quoting protects characters
*** BOOM ***
$ ls My Documents # shell splits command line on blanks
ls: cannot access My: No such file or directory
ls: cannot access Documents: No such file or directory
$ ls 'My Documents' # hide the space from the shell
resume.odt assignment.txt
The quoting mechanism protects the special shell meta-characters from expansion by the shell. The protected characters are passed as part of command arguments.
The shell treats quotes and backslashes specially when they are used on a command line or in a shell script. Shells do this because shells are designed to find and run commands, and supply arguments to the commands.
Sometimes you need to supply arguments to commands that contain special meta-characters that the shell would otherwise expand or process. The quoting mechanism stops the shell from processing the special characters.
Double quotes, single quotes, and backslashes can quote other shell meta-characters only when typed as command-line input to the Unix shells.
Quotes and backslashes are not special characters when fed as input to
most other non-shell programs such as cat
, head
, sort
, wc
, etc.:
$ echo "It's a nice day" # shell will process a quoted string
It's a nice day. # note how quotes have been removed
$ cat
"It's a nice day" # typed input to cat command
"It's a nice day" # unchanged program output on stdout
^D
$ head -1
"It's a nice day" # typed input to head command
"It's a nice day" # unchanged program output on stdout
Most non-shell programs do not treat quotes or backslashes specially when they read them as input. They are just normal characters.
grep
and find
IndexBackslash-style quoting may also be used by other programs to hide special characters in their command arguments, which results in us having to quote the backslashes to hide them from the shell so that the backslashes can then quote characters used in the command arguments.
We have to hide the backslash and other shell meta-characters from the shell using shell single quoting so that the programs get and use these characters and not the shell.
For example, the grep
program uses backslash quoting to turn off the
meaning of Regular Expression characters such as periods:
$ grep '\.' filename # look for a real period character
The single quotes hide the backslash from the shell, and the backslash
works inside grep
to hide the period Regular Expression character from
grep
.
The find
program also uses backslashes to prevent GLOB expansion
inside its --name
argument, and again we have to hide the backslashes
(and the GLOB characters) from the shell inside single quotes so that
find
gets them:
$ find . -name '\*boom.txt' -o -name '\?query.txt'
The backslashes tell find
to look for a real asterisk and a real
question mark, not the GLOB pattern characters asterisk and question mark.
This document focuses on shell quoting mechanisms: Hiding shell meta-characters from the shell so that commands can receive them as command line arguments.
Quoting is done using either double quotes, single quotes, or backslash characters. Not all quoting mechanisms protect all shell meta-characters.
While technically we only need to apply the quoting to the individual shell meta-characters we want to protect from the shell, not to the whole command line argument, it usually looks better to surround the whole argument with matching quote characters. For example, we can use double or single quotes to “quote” the asterisk and blanks in argument to the echo command:
$ echo "* "star # quote only the four meta-characters
* star
$ echo "* star" # quoting the whole argument looks better
* star
$ echo '* star' # single quotes work, too
* star
Backslashes may also be used to “quote” or turn off the special meaning of each immediately following character, one at a time. Each backslash only protects the one single character immediately to its right. For example:
$ echo \*\ \ \ star
* star
On a shell command line, a “quoted” semicolon (;
) is not a
meta-character and it does not separate commands. A “quoted” asterisk
(*
) is not a GLOB meta-character and it does not match file names:
$ echo one ; echo two # shell splits on semicolon
one
two
$ echo one ";" echo two # quoting protects semicolon
one ; echo two
$ echo "one ; echo two" # style: quote the whole argument
one ; echo two
$ touch a b c d # create four file names
$ echo * # shell expands GLOB character
a b c d
$ echo "*" # quoting protects GLOB character
*
$ echo \* # quoting protects GLOB character
*
Quoting is used to prevent the shell from acting on and expanding its meta-characters. The quoting causes the shell to ignore the special meaning of the character, so that the character gets passed unchanged to a command as part of a command argument.
The most common shell meta-character is the blank or space character that shells use to separate arguments. The shell does not normally pass any blanks or spaces to the command; the shell uses the blanks and spaces to separate and identify individual command line arguments, and the shell passes only those arguments to the command. One blank separates arguments the same way as ten or a hundred.
In both examples below, the echo
command receives from the shell
exactly three single-character arguments and no spaces:
$ echo a b c # one blank between arguments
a b c
$ echo a b c # many blanks between; same thing
a b c
The same arguments would be given to any command, not just echo
.
A touch
command would create three single-character files; a mkdir
would create three single-character directories. No blanks are passed
to the command; blanks serve only to separate the arguments.
A “quoted” blank is no longer a shell meta-character; it does not separate
arguments. To pass the folder name My Documents
to a command, the
space in the name must be hidden from the shell by some form of quoting:
$ ls My" "Documents # quoted blank is part of name
$ ls My' 'Documents # quoted blank is part of name
$ ls My\ Documents # quoted blank is part of name
The quotes and backslashes are used by the shells to locate a string of characters to protect. The quotes and backslashes themselves are not part of the string; they are removed as the string of characters is collected. For example:
$ echo "hi ho"
hi ho
$ echo hi\ \ ho
hi ho
The double quotes and backslashes above both delimit a single
six-character string; the quoting characters are not part of the six
characters. The shell collects one six-character command line argument,
and passes that argument to the echo command. If the touch
command had
been used, it would create a six-character file name with two embedded
spaces in the name. The quoting is removed from the argument.
The quotes and backslashes used in the quoting mechanism are not part of the string passed to the command – they only delimit the string and protect the meta-characters. The quoting mechanism is removed from the string as it is collected and before it is passed to the command.
Without the quoting mechanism, blanks act as meta-characters to separate command line arguments and in the example below the shell creates two two-character arguments for the echo command, instead of one six-character argument:
$ echo hi ho
hi ho
Blanks are meta-characters to the shell unless they are hidden from the shell by quoting. The shell splits the command line into tokens on any unquoted blanks. You may type one blank to separate the arguments or a hundred blanks; it makes no difference to the shell or to the arguments:
$ echo a b c
a b c
$ echo a b c
a b c
The three one-character string arguments passed to the echo command in the above two cases are identical; the echo command sees none of the blanks because the shell uses unquoted blanks to separate arguments.
Blanks that are quoted are not seen as special meta-characters by the shell; they do not separate arguments; they become ordinary characters made part of the argument string passed to the command being invoked.
You can create a file name that is a single blank using any of these quoting mechanisms to turn off its meta-character meaning:
$ touch \ # there is a space after the backslash
$ touch " " # there is a space inside the quotes
$ touch ' ' # there is a space inside the quotes
Quotes and backslashes tell the shell which parts of the input to treat as ordinary (not special) characters. The quoting delimits (identifies) a string of characters; the quoting mechanism is removed and is not part of the string passed to the command. Only the characters being quoted are passed to the command, not any of the quotes or backslashes used to do the quoting:
$ echo "***" # argument is three characters, not five
***
$ echo '***' # argument is three characters, not five
***
$ echo \*\*\* # argument is three characters, not six
***
The shell removes the quoting mechanism before passing the delimited text
as an argument to a command. In the all the examples above, the echo
command receives one three-character command line argument containing
three asterisks ***
. The quotes and backslash characters used to
protect the meta-characters are removed before the argument is passed to
the command. The command sees only the one three-character argument, no
matter what kind of quoting was used. All the quoting syntax is removed.
This next echo command line has six string arguments. None of the quoting
mechanism is part of the arguments; it is all stripped out. The only
blanks passed to the echo
command are the ones hidden inside the quotes:
$ echo "a b" c 'd e' f "g h" "i j"
a b c d e f g h i j
^^^^ ^ ^^^^ ^ ^^^^ ^^^^^^^
1 2 3 4 5 6
Only the characters being quoted are passed to the command, not any of the quotes or backslashes used to do the quoting. All the unquoted blanks are stripped away; they only serve to separate the arguments.
Quotes are processed and matched on shell command lines in strict left-to-right order. Let’s examine how this works using this example:
$ echo 'one "two" three'four"five"six"seven 'eight' nine"ten
one "two" threefourfivesixseven 'eight' nineten
The above echo
command line argument is actually one single argument
string containing six pieces as seen by the shell:
The first quote character found by the shell, starting from the left,
is a single quote. The shell collects all the characters up to the
next single quote as part of the first argument to echo
. All these
single-quoted characters are protected from further analysis by the shell.
No characters are expanded or special inside single quotes.
The words one
, two
, and three
and the contained blanks and
double-quotes are all inside this first single-quoted string.
The double quotes, inside single quotes, have no special meaning
to the shell. They are simply part of the string. Same with the
blanks – they are quoted and thus do not perform their meta-character
function of separating shell arguments. The single quotes themselves,
used to delimit the single-quoted string, are not part of the string
and thus are not part of the first argument being collected.
Right after the closing single-quote, the word four
is outside of
any quotes. The word immediately follows the closing single quote;
there is no space there to indicate the start of a second argument to
echo
and so the four
is still part of the same first argument.
Right after four
, with no blank to start a new argument, we start a
double-quoted string that extends until the next double-quote character.
The characters inside this double-quoted string are still part of
the first argument; the shell has not yet seen any unquoted blank that
indicates the end of the first argument. The word five
is inside this
double-quoted string. The double quotes themselves, used to delimit
the string, are not part of the string.
Right after the closing double-quote, the word six
is outside of
any quotes. The word immediately follows the closing double quote;
there is no space there to indicate the start of a second argument to
echo
and so the six
is still part of the same first argument.
Right after six
we start another double-quoted string that extends
until the next double-quote character. Most, but not all,
double-quoted characters are protected from further analysis by
the shell. (Double quotes are weaker than single quotes; some
meta-characters inside double quotes do still expand.)
The words seven
, eight
, and nine
, the single-quotes, and the
contained blanks are all inside this double-quoted string. The single
quotes, inside double quotes, have no special meaning to the shell.
They are simply part of the string. Same with the quoted blanks –
they are quoted and thus do not perform their meta-character function
of separating shell arguments. The double quotes themselves, used
to delimit the string, are not part of the string.
Right after the closing double-quote, the word ten
is outside of any
quotes, but since it is not separated from the other characters by any
shell-visible blanks, it is still part of the same first argument.
The one command line argument to echo
is built up of six pieces of
text that includes three quoted strings. Each quoted string connected
immediately to another non-blank character, and all the blanks that are
inside quoted strings are protected from the shell. Thus, the echo
command is handed only one command line argument. (Command line
arguments are separated by meta-character blanks. There were no unquoted
blanks used.)
The fact that this one argument was made up of six pieces on the command
line, including both quoted and unquoted parts, is invisible to the
echo
command. The shell did the work; the echo
command gets its
one argument.
The quotes seen by the shell are not part of the argument. The quotes delimit the string; they are not part of the string.
Even though the shell used multiple quoting mechanisms to assemble the argument above, no unquoted blanks were found to separate arguments, so the string above is passed as one argument to the echo command.
The shells can use single- and double-quote characters to delimit and protect strings of characters used as arguments to commands. Backslashes also protect special characters from the shell; but, sometimes they are less convenient to use. For example, one could protect all the special shell meta-characters in a line of text, each with its own backslash:
$ echo \*\*\*\ Here\ is\ a\ backslash\ \"\\\".\ \*\*\*
*** Here is a backslash "\". ***
Note that quotes and backslashes are themselves meta-characters to the shell, so if you want quotes and backslashes to appear in command line arguments you have to quote them with backslashes to protect them!
Alternately, one could surround the shell meta-characters, and the whole argument string, with matching single quotes to protect the whole string from the shell:
$ echo '*** Here is a backslash "\". ***'
*** Here is a backslash "\". ***
This works because the string of text being quoted doesn’t itself contain any single quotes, so surrounding it with single quotes is possible.
In most cases the use of quotes is easier than using a lot of backslashes.
Single quotes are “stronger” than double quotes. Nothing is special inside single quotes; the shell treats all the characters, no matter what they are, as part of the string being collected and it does not expand any of them inside single quotes.
Inside double quotes, the shell still sees and expands some special meta-characters. The most important character expanded by the shell inside double quotes is the dollar sign that signals the start of a shell variable:
$ echo '$SHELL'
$SHELL # single quotes prevent expansion
$ echo "$SHELL"
/bin/bash # double quotes permit expansion
Backslashes are also still special meta-characters inside double-quoted strings, so you can use backslashes inside double quotes to protect other double quotes (and other backslashes, and dollars, etc.):
$ echo "This is a \"double\" quoted \$SHELL string and backslash \\."
This is a "double" quoted $SHELL string and backslash \.
For maximum protection and maximum quoting, use single quoted strings. The only character you can’t safely hide inside a single-quoted string is another single quote, since it ends the quoting.
Which quotes can we use to surround a string that contains either single and double quotes? We need to put the single quotes inside of double quotes to protect them, and put the double quotes inside of single quotes to protect them:
$ echo "single ' quote"
single ' quote
$ echo 'double " quote'
double " quote
Inside a double-quoted string, a single quote is not a meta-character had has no meaning. Inside a single-quoted string, a double quote is not a meta-character and has no meaning.
Single quotes are not meta-characters inside of double-quoted strings; they are just ordinary characters. Double quotes are not meta-characters inside of single-quoted strings; they are just ordinary characters.
Adding single quotes inside double quotes doesn’t turn off variable expansion, because the shell doesn’t see the single quotes as special:
$ echo "This is my $SHELL variable."
This is my /bin/bash variable.
$ echo "This is my '$SHELL' variable."
This is my '/bin/bash' variable.
Adding double quotes inside single quotes doesn’t turn on variable expansion, because the shell doesn’t see the double quotes as special:
$ echo 'This is my $SHELL variable.'
This is my $SHELL variable.
$ echo 'This is my "$SHELL" variable.'
This is my "$SHELL" variable.
How can we quote some text that contains both kinds of quotes?
To quote a command line argument that contains both types of quotes, we can alternate quoting mechanisms in the same line. We still put the single quotes inside of double quotes to protect them, and put the double quotes inside of single quotes to protect them:
$ echo "single ' and "'double " quote'
single ' and double " quote
Note how the double quoted section ends at the second double quote, after which we immediately start a single-quoted section that extends to the end of the line. The single quote is treated as an ordinary character inside double quotes and the double quote is treated as an ordinary character inside single quotes.
The shell sees no spaces in the argument above: all the spaces are hidden
inside quotes. Without spaces to separate arguments, there is only one
single argument string to echo
The only space the shell sees is the
one after the command name. If touch
had been used, only one file
would have been created.
Sometimes you want to fully quote a piece of text that already contains double quotes, but still allow a variable inside the text to expand. You can’t simply single-quote the entire text, since that would prevent the variable from expanding:
$ echo 'The value of SHELL is "$SHELL".'
The value of SHELL is "$SHELL".
To allow the variable to be seen and expanded by the shell, you have to end the single-quoted string, double-quote the variable, then resume the single-quoted string:
$ echo 'The value of SHELL is "'"$SHELL"'".'
The value of SHELL is "/bin/bash".
You might also consider double-quoting the entire string and using backslashes (which do work inside double-quotes) to further quote any double-quotes inside the string:
$ echo "The value of SHELL is \"$SHELL\"."
The value of SHELL is "/bin/bash".
Variables must always be expanded inside double quotes, so that metacharacters contained in the variables don’t themselves expand:
$ x='* BOOM *'
$ touch a b c d
$ echo $x # WRONG: missing double quotes!
a b c d BOOM a b c d
$ echo "$x" # correct use inside double quotes
* BOOM *
$ dir='My Documents'
$ ls $dir # WRONG: missing double quotes!
ls: cannot access 'My': No such file or directory
ls: cannot access 'Documents': No such file or directory
$ ls "$dir" # correct use inside double quotes
resume.odt assignment.txt
You must always double-quote variable expansions, to prevent the shell from further blank-splitting and GLOB expansion of the substituted value.
grep
and find
expressionsIndexThe shell normally doesn’t tell you that a GLOB expansion failed (unless
you set the failglob
option), and so you might not notice that an
argument to grep
contains an unquoted and unintended GLOB expression
until the contents of the file system change one day:
$ mkdir empty ; cd empty
$ echo go* # shell fails to expand GLOB characters
go*
$ grep go* /etc/fstab # shell fails to expand GLOB characters
[... several lines print here matching "go*" search pattern ...]
$ touch golf good goofs # create three file names
$ grep go* /etc/fstab # now the shell expands the go* GLOB pattern
$ # nothing found !!
$ echo grep go* /etc/fstab # use echo to see the GLOB problem
grep golf good goofs /etc/fstab # looks for pattern "golf" in three files !!
$ grep "go*" /etc/fstab # quoting protects GLOB characters
[... several lines print here matching "go*" search pattern ...]
The same problem can happen with a find
GLOB expression that the shell
expands using GLOB instead of it being passed to and expanded by find
:
$ mkdir empty ; cd empty
$ echo *go* # shell fails to expand GLOB characters
*go*
$ find /usr/bin -name *go* # shell fails to expand GLOB characters
/usr/bin/pbmtogo
[... find finds many pathnames matching -name *go* pattern ...]
/usr/bin/gouldtoppm
$ touch xxxxgoxxxx # create a name containing "go"
$ echo *go* # now shell expands GLOB characters
xxxxgoxxxx
$ find /usr/bin -name *go* # now shell expands GLOB characters
$ # NO OUTPUT FROM FIND !!
$ echo find /usr/bin -name *go* # use echo to see the GLOB problem
find /usr/bin -name xxxxgoxxxxx # xxxxgoxxxxx does not exist in /usr/bin
$ touch algorithm # create another name with "go" in it
$ echo *go* # now shell expands GLOB characters
algorithm xxxxgoxxxxx
$ find /usr/bin -name *go* # now shell expands GLOB characters
find: paths must precede expression: algorithm # ERROR FROM FIND !!
$ echo find /usr/bin -name *go* # use echo to see the GLOB problem
find /usr/bin -name algorithm xxxxgoxxxxx # bogus argument to find
$ find /usr/bin -name '*go*' # quoting protects GLOB characters
/usr/bin/pbmtogo
[... find finds many pathnames matching protected -name '*go*' pattern ...]
/usr/bin/gouldtoppm
$ echo find /usr/bin -name '*go*' # use echo to see what happens
find /usr/bin -name *go* # confirms GLOB characters not expanded
Don’t let the shell expand arguments to
grep
orfind
– quote the arguments to protect them from the shell and pass them unchanged.
Double quotes are essential when expanding variables, because without them the shell will blank-split and GLOB-expand the result of a variable substitution. The unquoted and unwanted GLOB expansion is harder to notice when it comes from inside the expansion of a shell variable:
$ mkdir empty ; cd empty
$ x='*** BOOM ***' # variable with GLOB characters
$ echo $x # shell fails to expand GLOB characters
*** BOOM ***
$ touch a b c d # create four file names
$ echo $x # now shell expands GLOB characters
a b c d BOOM a b c d
$ echo "$x" # quoting protects GLOB characters
*** BOOM ***
$ mkdir empty ; cd empty
$ x='We *go* far!' # variable with GLOB characters
$ echo $x # shell fails to expand GLOB characters
We *go* far!
$ touch Algonquin algorithm good # create three file names
$ echo $x # now shell expands GLOB characters
We Algonquin algorithm good far!
$ echo "$x" # quoting protects GLOB characters
We *go* far!
$ mkdir empty ; cd empty
$ x='My Documents' # variable with blanks in it
$ mkdir $x ; ls -l # shell splits on blanks: two arguments
drwxr-xr-x 2 idallen idallen 40 Dec 12 11:55 Documents/
drwxr-xr-x 2 idallen idallen 40 Dec 12 11:55 My/
$ rmdir *
$ mkdir "$x" ; ls -l # blanks are hidden from shell
drwxr-xr-x 2 idallen idallen 40 Dec 12 11:57 My Documents/
Unwanted shell splitting on blanks after variable expansion also causes
problems, especially when the blanks create extra (or missing) arguments
that cause syntax errors in test
conditional logic in IF and WHILE
statements:
$ arg="a" # simple argument works fine
$ [ $arg = a ] && echo hi # unquoted $arg used - WRONG!
hi
$ arg="a b" # now $arg contains blanks
$ [ $arg = a ] && echo hi # unquoted $arg used - WRONG!
sh: [: a: unexpected operator # shell syntax error !
$ arg=" " # now $arg is a blank string
$ [ $arg = a ] && echo hi # unquoted $arg used - WRONG!
sh: [: =: unexpected operator # shell syntax error !
$ [ "$arg" = a ] && echo hi # quoted $arg works correctly; no errors
Always double-quote your variable expansions, to prevent GLOB expansion and blank-splitting!
You must always double-quote variable expansions, to prevent the shell from further blank-splitting and GLOB expansion of the substituted value.
Due to the order of processing of the command line, all quoted strings are located and identified by the shell before any variables are expanded. This means quotes embedded inside variables are not seen as special characters by the shell:
$ x='aaaaa " bbbbb'
$ echo $x
aaaaa " bbbbb
In the above echo
command line, the shell first looks for and identifies
quoted strings before it expands the variable $x
. The double quote is
hidden inside the variable and is not seen. Next, the shell expands the
$x
variable and splits it on blanks into three arguments. The double
quote hidden inside $x
appears “too late” – it is not treated as a
special character by the shell. The echo
commands sees three arguments.
Only “exposed” quotes are treated as special on shell command lines, not quotes embedded inside variables. Quotes inside variables are not special.
Exposed quotes are special; embedded quotes inside variables are not:
$ mkdir empty ; cd empty; touch a b c d ; ls
a b c d
$ echo " * "
* # quotes protect the glob character
$ q='"'
$ echo $q * $q # double quotes are inside the variables
" a b c d " # quotes inside variables are not special
Above, the quotes embedded inside the variable $q
are not treated as
special characters by the shell and they do not stop the GLOB expansions.
Note: Blanks and glob (wildcard) characters embedded inside unquoted variables are seen as special to the shell and may cause multiple arguments to be created and other problems:
$ x="a b c"
$ touch $x ; ls # creates three files; embedded blanks are special
a b c
$ touch "$x" ; ls # creates another single file named: a b c
a a b c b c d
$ y='*'
$ echo "$y" # embedded glob char is protected by quoting
*
$ ls $y # unprotected glob char matches four file names!
a a b c b c
You must double-quote all uses of variables to prevent the embedded blanks and glob (wildcard) characters from being seen as special by the shell after the variable expands.
Understand how the shell handles quotes and blanks:
$ echo hi there
hi there
$ echo "hi there"
hi there
$ echo 'hi there'
hi there
Explain the above three outputs. How does the shell find arguments?
How many arguments are passed to the echo
command in each case?
The touch
command creates empty files by name.
Try this:
$ touch "a b"
$ ls
a b
$ rm a b
Explain the error message that is output by the above rm
command.
How many arguments are passed to the touch
and rm
commands?
How can you remove a file name containing a special character?
Here are some more things to try, and to understand.
$ echo "'hello'"
'hello'
$ echo '"hello"'
"hello"
$ mkdir empty ; cd empty ; touch a b c d ; ls
a b c d
$ echo ' * '
*
$ echo '" * "'
" * "
$ echo '"' * '"'
" a b c d "
$ echo '"'" * "'"'
" * "
$ echo ' * ' * " * "
* a b c d *
$ echo \' * \'
' a b c d '
You must be able to predict the output of each of the above command lines without having to type them in to try them.
argv.sh
to count command line argumentsIndexHow many arguments are there to the following echo command?
$ echo abc \ def \\ ghi \ \ jkl
How many characters are in each of the arguments? Fetch and run the
argv.sh
program (available in the Class Notes) to
help you:
$ ./argv.sh abc \ def \\ ghi \ \ jkl
Argument 0 is [./argv.sh]
Argument 1 is [abc]
Argument 2 is [ ]
Argument 3 is [def]
Argument 4 is [\]
Argument 5 is [ghi]
Argument 6 is [ ]
Argument 7 is [jkl]
Since the shell is the program that parses arguments, the number of
arguments passed to the argv.sh
program will be exactly the same as the
number of arguments passed to the echo
program (or to any program).
When in doubt about how the shell will parse a command line, use echo
or argv.sh
to confirm the arguments that the shell is providing.
ssh
)IndexThis is optional advanced material for people who execute shell commands on other machines: Unix/Linux Shell Quoting for remote shells
eval
: Using quotes inside shell variablesIndexThis is optional advanced material for people who write scripts.
How to make quotes inside variables work using the shell eval
mechanism.
This section is taken from a reply email I sent to the OCLUG mailing list:
From: "Ian! D. Allen" <idallen@idallen.ca>
To: General Membership Discussion List <oclug@lists.oclug.on.ca>
Subject: Re: [oclug] Must be Friday.....
> This doesn't work:
> $ SUDO="/usr/bin/sudo -u root -p \"Enter password for user '%U': \""
> $ $SUDO ls
Most shell meta-characters such as quotes, backslashes, line separators, pipes, redirection, etc. don’t have special meaning coming from inside shell variables. They also mean nothing if they appear in the command line due to GLOB pattern matches. The shell only treats them specially the first time it sees them on the command line:
$ mkdir empty ; cd empty ; touch a b c # three visible files
$ echo " * "
* # the shell sees the quotes and protects the GLOB pattern
$ q='"'
$ echo $q * $q
" a b c " # the hidden quotes have no special meaning
$ echo hi ; date
hi
Sun Feb 25 18:55:34 EST 2007
$ s=';'
$ echo hi $s date
hi ; date # the hidden semicolon has no special meaning
$ echo hi >out # put "hi\n" into file "out"
$ r=">out"
$ echo hi $r
hi >out # the hidden >out has no special meaning
The user continues:
> # eval $SUDO ls
> Um.. okay, that works. <scratching few remaining hairs on head>
> What is special about this case that requires the use of 'eval'?
The eval
tells the shell to process the command line twice. This gives
hidden meta-characters a second chance. The first time through, the
shell expands the variable. The second time through, the shell now sees
the quoted strings and treats them correctly (since they aren’t hidden
inside the variable any more).
The user notes that this works without problems:
> # LS="ls -l"
> # $LS /usr
The above only works because no shell meta-character processing is needed. A simple variation fails:
# LSW="ls -l | wc"
# $LSW
ls: |: No such file or directory
ls: wc: No such file or directory
Don’t put commands inside variables – use shell aliases instead.
The user says:
> eval is the shell command that executes the contents of a shell variable.
The eval
causes the shell to process what was in the variable as if you
had typed it directly into the shell. It may “execute” and it may not,
depending what else is on the command line in front of it:
$ v="date"
$ date=foo
$ eval $v
Sun Feb 25 19:06:12 EST 2007
$ eval echo $v
date
$ eval echo \$$v
foo
The last example above is how we do crude associative arrays in older versions of the Bourne shell. Assignment also has to use eval, since the shell won’t assign to a name hidden inside a variable:
$ echo $v $date
date foo
$ $v=hoho # wrong way
bash: date=hoho: command not found
$ eval $v=hoho # right way
$ echo $v $date
date hoho