Winter 2015 - January to Apil 2015 - Updated 2017-01-20 00:48 EST
System administrators often need to run command lines on remote machines without doing a full remote login. We just want the output of the one command line; that is all. We don’t want to log in, run the command, and log out again. Or, we might want to run a command on a remote machine and redirect the output into a local file, which can’t be done using a full login.
For example, one might want to verify an account disk quota on a remote machine using the quota
command remotely, without logging in. To run a command remotely, simply add the command line after the ssh hostname
on the command line. You may need to accept the machine’s host key, and you will need to enter your remote system password:
$ ssh localhost hostname
abcd0001@localhost's password:
idallen-ubuntu
$
$ ssh cst8207.idallen.ca quota
abcd0001@cst8207.idallen.ca's password:
Disk quotas for user abcd0001 (uid 1000):
Filesystem blocks quota limit grace files quota limit grace
/dev/sda1 52 204800 512000 15 5000 8000
/dev/sdc1 568 204800 512000 237 5000 8000
$
$ ssh cst8207.idallen.ca quota >quota.txt # save output in a file
abcd0001@cst8207.idallen.ca's password:
$ wc quota.txt
4 30 255 quota.txt
The single command line is executed by a shell on the remote machine and the output is displayed. The remote output can be redirected into a file on the local machine. No full login to the remote machine is done.
Another example: Perhaps you might need to create a directory on several (dozen? hundred?) machines. A quick way to do this is with a shell for
loop that runs an ssh
remote command line to each machine without needing to log in, type each command manually, and then log out:
$ for h in host1 host2 host3 host4 host5 host6 host7 ; do
> ssh $h.example.com mkdir newdir
> done
The above loop works best when you have enabled password-free public key logins to the remote machines, to avoid typing many passwords, but even without that it’s much faster than having to log in to each machine, execute the command, then log out!
Two shells are involved in reading and parsing every command line sent to a remote system using ssh
:
ssh
.ssh
.The local shell processes the command line you type in, doing such things as local GLOB and variable expansion on the local machine, and then stripping off one layer of quotes and passing the result to to the ssh
command, which will send the arguments to the remote machine.
The remote machine passes the stripped command line arguments to a remote shell, and that remote shell does more GLOB and variable expansion and quote stripping on those arguments on the remote machine.
The command line is processed by two shells.
For example, we can run multiple commands on the remote machine by passing semicolons in the command line sent to the remote shell. We have to hide these semicolons from the local shell by quoting them. Without the quoting, the semicolons would be seen and acted on by the local shell, creating two commands in the local shell, like this:
$ cd /tmp
$ ssh cst8207.idallen.ca hostname ; pwd # unquoted semicolon
idallen-ubuntu # hostname executes on remote system
/tmp # pwd executes on local system
Above, the ssh
sends only the hostname
command to the remote system; the local shell splits the command line we typed on the semicolon and the pwd
runs on the local system.
Using quotes to hide the semicolon, all the special characters can be sent to the remote shell, creating two commands on the remote machine:
$ ssh cst8207.idallen.ca "hostname ; pwd" # quoted semicolon
idallen-ubuntu # hostname executes on remote system
/home/abcd0001 # pwd also executes on remote system
This is what is happening in the ssh
command above:
ssh
command as part of an ordinary argument. (The quotes are removed from the argument by the local shell before handing the argument to the ssh
command. Quotes delimit the argument, but are never made part of it when the argument is passed to the command, as described in Quoting.)ssh
command then passes its argument string to the shell on the remote machine. (Remember: The string does not have quotes around it when ssh
receives it and passes it to the remote machine.)The act of using SSH to send a command to a remote machine uses two shells. Each shell recognizes and strips a layer of quoting from the line being sent. You can see this by putting an echo
in front of the command line, to see how the local shell strips one layer of quotes. The stripped line is what is actually being passed to the ssh
command and then sent to the remote system’s shell:
$ echo ssh cst8207.idallen.ca "hostname ; pwd" # insert echo in front
ssh cst8207.idallen.ca hostname ; pwd
Above, the echo on the screen shows us that the ssh
command is sending the argument string hostname ; pwd
to the remote machine. (The string received by ssh
and passed to the remote machine has no quotes around it, because the local shell stripped the quotes.)
When that (unquoted) string argument hostname ; pwd
is received by the shell on the remote machine, the remote shell will split the argument into three tokens (because shells split on unquoted blanks). The remote shell will see the unquoted semicolon metacharacter, and execute two separate commands on the remote machine. The output of the two commands will return and display on your local screen, where you could redirect it into a local file.
Two shells are involved in every ssh
remote command. One layer of quotes used on an ssh
command line hides the metacharacters from expansion by the local shell, but not from expansion by the remote shell.
Two shells are involved in every ssh
remote command. One layer of quotes used on an ssh
command line hides the metacharacters from expansion by the local shell, but not from expansion by the remote shell.
Where things get tricky is when the command line you want to run on the remote machine contains shell metacharacters that need quoting on both the local machine and on the remote machine. One level of quoting doesn’t work:
$ echo "Here today ; gone tomorrow."
Here today ; gone tomorrow.
$ ssh localhost echo "Here today ; gone tomorrow."
Here today
bash: gone: command not found
To see the problem more clearly, again insert an echo
into the start of the ssh
command line to show what arguments are being passed to the ssh
command for execution on the remote system:
$ echo ssh localhost echo "Here today ; gone tomorrow."
ssh localhost echo Here today ; gone tomorrow.
Above, the echo on the screen shows us that the ssh
command is sending an unquoted semicolon to the shell on the remote system. The remote shell splits the command line in two on the unquoted semicolon and tries to execute a nonexistent command named gone
.
To quote something on the remote machine in the remote shell requires careful thought. One level of quotes isn’t enough. You need one level of quotes to hide things from the shell you’re typing into on the local machine, and inside those quotes you need another level of quotes that get sent to the remote shell to do the quoting and hiding remotely as well.
Let’s modify our example and put in single quotes, inside the double quotes, to hide the semicolon from the shell on the remote system:
$ echo ssh localhost echo "Here today ';' gone tomorrow."
ssh localhost echo Here today ';' gone tomorrow.
Above, the echo on the screen shows us that the ssh
command will send a single-quoted semicolon to the remote system, which will keep it from being interpreted as a shell metacharacter. We can remove the leading echo
and confirm that it works as expected:
$ ssh localhost echo "Here today ';' gone tomorrow."
Here today ; gone tomorrow.
Before you think we’ve solved the problem, let’s explore a bit further. Spaces are also shell meta-characters, and the above remote echo
is not receiving exactly the same arguments as its local version. The local echo
is receiving only one argument; the remote echo
is receiving five:
$ echo "Here today ; gone tomorrow." # one argument to echo
$ ssh localhost echo "Here today ';' gone tomorrow." # five arguments to echo
The remote shell executes an echo
command with five arguments, not one, because the ssh
command line is sent to the remote machine unquoted and the remote shell splits the line into tokens separated by spaces. This is a problem if the remote command is not expecting multiple arguments. To make the problem more obvious, let’s try touching a file containing a space both locally and remotely:
$ touch "my file" # creates one file
$ ssh localhost touch "my file" # creates two files!
Let’s remind ourselves what ssh
is passing to the remote system by inserting an echo
in front of the ssh
command line:
$ echo ssh localhost touch "my file" # insert echo in front
ssh localhost touch my file # two arguments to touch
Above, the echo on the screen shows us that the ssh
command is sending a command line that will cause two separate arguments to the touch
command, not one. There are no quotes being sent to the remote system to hide the space in the file name.
To fix the problem of the space in the name on the remote system, we need to hide the blanks from both the local shell and the remote shell by putting quotes inside of quotes. On easy way to do this is by putting single quotes inside double quotes:
$ echo ssh localhost touch "'my file'" # single inside double
ssh localhost touch 'my file' # one argument to touch
Above, the echo on the screen shows us that the ssh
command is now sending a command line containing one single-quoted argument to the touch
command. We can remove the leading echo
and the command will work as expected:
$ ssh localhost touch "'my file'" # now creates one file
We need “quotes inside of quotes” to hide all the metacharacters on both the local system and the remote system.
We can also put double quotes inside single quotes to hide them. For example, suppose we want to run the command mkdir "Space Dir Name"
on the remote machine. The command line below doesn’t work; it creates three directory names not one directory name with two spaces in it.
$ ssh localhost mkdir "Space Dir Name" # creates three directories
Inserting a leading echo
command at the start helps us see this:
$ echo ssh localhost mkdir "Space Dir Name" # insert echo in front
ssh localhost mkdir Space Dir Name # three arguments to mkdir
Above, the echo on the screen shows us that the ssh
command is sending a command line that will cause three arguments to the mkdir
command, not one argument inside quotes.
We have to hide the double quotes from the local shell so that they get sent as double quotes to the remote shell. One way to do this is to put the double quotes inside single quotes:
$ ssh localhost mkdir '"Space Dir Name"' # creates one directory
Inserting a leading echo
command at the start helps us see this:
$ echo ssh localhost mkdir '"Space Dir Name"' # insert echo in front
ssh localhost mkdir "Space Dir Name" # one argument to mkdir
Here is another example showing how one level of quotes isn’t enough to preserve metacharacters on the remote system:
$ ssh locahost echo "Hello World" # only one level of quotes
Hello World # no quotes; blanks disappear
$ echo ssh locahost echo "Hello World" # insert echo in front
ssh locahost echo Hello World # no quotes are sent
$ echo ssh locahost echo "'Hello World'" # put quotes inside quotes
ssh locahost echo 'Hello World' # one level of quotes remain
$ ssh localhost echo "'Hello World'" # remove the leading echo
Hello World # quoted blanks are preserved
Quotes that we want sent to the remote shell themselves need quoting, to protect those quotes from the local shell.
Here is another example. We need to use a real asterisk (*
) on the remote system, so it needs to be quoted on the remote system, as well as on the local system, to hide it from the shell on both systems.
The first step is to know how to hide the asterisk from the local shell. Either single or double quotes will work:
$ echo '*' # works fine locally
*
Now we need to add the extra quoting needed to pass both the single quotes and the asterisk to the remote shell. To make this work we use echo
to develop a correctly quoted ssh
command line. When the echo
command shows us a correctly quoted line, we know it will work remotely. We see this doesn’t work:
$ echo ssh localhost echo '*' # WRONG
ssh localhost echo * # WRONG - note lack of quotes
The above line echoed on the screen shows that no quotes are being passed to the remote shell; that won’t work on the remote machine because the remote shell will expand the unquoted asterisk. Let’s hide the single quotes and the asterisk inside double quotes, and then it works:
$ echo ssh localhost echo "'*'" # hide the single quotes
ssh localhost echo '*' # now it has quotes
$ ssh localhost echo "'*'" # remove the leading echo
* # it works
Suppose we want to echo
a double quote on the remote system. The first step is to know how to hide the double quote from the local shell. Single quotes will work:
$ echo '"' # double inside single
"
The command line below does not work; the first level of single quotes is stripped by the local shell and the double quote is sent alone to the remote system where it causes a syntax error:
$ ssh localhost echo '"' # doesn't work with ssh
bash: -c: line 0: unexpected EOF while looking for matching '"'
bash: -c: line 1: syntax error: unexpected end of file
Inserting a leading echo
command at the start helps us see this:
$ echo ssh localhost echo '"' # insert echo in front
ssh localhost echo " # only one quote is being sent
We need to quote the single quotes so that they get passed to the remote system to protect the double quote from the remote shell. The way we did this above was to put the whole string inside double quotes, but we can’t do that if there is a double quote inside the string we want to surround with double quotes. The quotes won’t match up correctly and we will get an input continuation prompt from the local shell:
$ ssh localhost echo "'"'" # unmatched quote
> # unmatched double quote!
One solution is to hide the internal double quote from the local shell:
$ ssh localhost echo "'\"'" # hide embeded double quote
" # works!
Another solution is to leave single quotes around the double quote to hide it from the local shell and send two more single quotes to the remote shell using backslashes to hide them locally:
$ ssh localhost echo \''"'\' # add two single quotes
" # works!
Inserting a leading echo
command at the start helps us see this:
$ echo ssh localhost echo \''"'\' # insert echo in front
ssh localhost echo '"' # correctly quoted
Above, the echo on the screen shows us that the ssh
command is sending a single-quoted double quote to the shell on the remote system. The single quotes will hide the double quote from the remote shell, as desired.
Things get more complicated if we are trying to protect something that looks like a variable expansion in a remote command line.
The following example shows how the shell variable $$
is protected by single quotes when used locally, but it expands to be the process ID when the command line is sent to the remote system:
$ echo 'It costs $$.' # single quotes stop expansion
It costs $$.
$ ssh localhost echo 'It costs $$.' # doesn't work remotely
It costs 18029. # $$ variable expands
$ echo ssh localhost echo 'It costs $$.' # insert echo in front
ssh localhost echo It costs $$. # unprotected $$ variable
Above, the echo on the screen shows us that the ssh
command is sending an unquoted $$
variable to the remote shell, which will expand it. To prevent this expansion by the remote shell, we also need to send single quotes to the remote shell, to protect the $$
variable.
You might think that you could simply double-quote the single-quoted string to protect the single quotes, as we did above, so that the single quotes would prevent the variable expansion on the remote system, but it’s not that easy:
$ ssh localhost echo "'It costs $$.'" # surround with double quotes
It costs 25761. # doesn't work!
$ echo ssh localhost echo "'It costs $$.'" # insert echo in front
ssh localhost echo 'It costs 25761.' # $$ variable expanded locally
Above, the echo on the screen shows us that the ssh
command is now getting the single quotes correctly, but it is also getting an already-expanded $$
variable, because the local shell expanded $$
in the now-double-quoted string! We need to tell the local shell not to expand the $$
inside the double-quoted string, and one way to do that is to use backslashes to hide the dollar signs inside the double-quoted string:
$ ssh localhost echo "'It costs \$\$.'" # protect $$ from expansion
It costs $$. # works!
$ echo ssh localhost echo "'It costs \$\$.'" # insert echo in front
ssh localhost 'It costs $$.' # correctly quoted
Another way to send single quotes to the remote shell is to individually quote (using backslashes) the single quotes on either side of the single-quoted string being sent, just as we did in the case of the single-quoted double quote earlier:
$ echo ssh localhost echo \''It costs $$.'\' # add two more single quotes
ssh localhost echo 'It costs $$.' # this looks good
$ ssh localhost echo \''It costs $$.'\' # remove the leading echo
It costs $$. # it works!
$ ssh localhost echo "'"'It costs $$.'"'" # another way to use single quotes
It costs $$. # it works!
The last example above shows that you can also double-quote the single quotes to hide them from the local shell, instead of using backslashes to hide them.
Things get much more complicated if we are trying to protect both variable expansions and different types of quotes all in a single remote command line. Getting the mixed quoting right is tricky even when you only have to quote for one local shell.
Suppose we want to echo the string It's "not" easy $$.
locally. This string contains both single and double quotes, and also dollar signs. All of these have to be hidden from the local shell:
$ echo "It's \"not\" easy \$\$." # use backslashes
It's "not" easy $$.
$ echo 'It'"'"'s "not" easy $$.' # use alternating quotes
It's "not" easy $$.
Making this same echo
work for a remote shell is not easy, since we need to quote the quotes so that the remote shell gets all the correct quoting shown above.
A strategy to get different types of quotes sent to the remote shell is to single-quote the double quotes and double-quote the single quotes. We always start using a leading echo
to see what would be sent to the remote shell first:
$ echo ssh localhost echo "'It'"'"'"'"'"'"'s "'"not" easy $$.'"'"
ssh localhost echo 'It'"'"'s "not" easy $$.' # looks good
$ ssh localhost echo "'It'"'"'"'"'"'"'s "'"not" easy $$.'"'" # remove the leading echo
It's "not" easy $$. # it works!
You can also use backslashes to escape individual characters so that they are each sent to the remote shell:
$ echo ssh localhost echo \'It\'\"\'\"\'s\ \"not\"\ easy\ \$\$.\'
ssh localhost echo 'It'"'"'s "not" easy $$.' # looks good
$ ssh localhost echo \'It\'\"\'\"\'s\ \"not\"\ easy\ \$\$.\' # remove the leading echo
It's "not" easy $$. # it works!
When the leading echo
command shows us a correctly quoted line on the screen, we know it will work as a remote command without the leading echo
. Always check your complex quoting with echo
, first!
Sending a command line to a remote login shell works similarly to sending a command line from the current shell into another local bash
shell through a pipeline. A pipeline is useful and faster for working the bugs out of a remote command line before you use it remotely through ssh
since no passwords or remote connections are needed to pipe into bash
:
$ ssh localhost "date ; whoami ; echo 'Hello World'" # slow using ssh
abcd0001@localhost's password:
Tue Apr 15 23:35:29 EDT 2014
abcd0001
Hello World
$ echo "date ; whoami ; echo 'Hello World'" | bash -u # fast using bash
Tue Apr 15 23:36:29 EDT 2014
abcd0001
Hello World
Above, we can see that in both the ssh
case and the pipeline case two shells are involved and two levels of quotes are stripped. What works in the pipeline case will also work in a remote ssh
command line. Use bash
to help you debug your quoting quickly.
Here is a previous example showing how one level of quotes isn’t enough to preserve metacharacters on the remote system:
$ echo echo "Hello World" | bash -u # only one level of quotes
Hello World # no quotes; blanks disappear
$ echo echo "'Hello World'" | bash -u # two layers of quotes
Hello World # quoted blanks are preserved
Another example:
$ ssh localhost echo Eat\\\ \\\ \\\;\\\ \\\ Pray\\\ \\\ \\\;\\\ \\\ Linux
Eat ; Pray ; Linux
$ echo echo Eat\\\ \\\ \\\;\\\ \\\ Pray\\\ \\\ \\\;\\\ \\\ Linux | bash -u
Eat ; Pray ; Linux
While you are testing and developing a properly-quoted remote command line, you can get the same quoting output results more quickly using echo
and a pipeline into a second copy of bash
instead of using the much slower ssh
. Once you get the bash
quoting working, switch to using ssh
.
eval
instead of remote loginIndexThe bash
shell (and most other shells) contains a keyword that lets you strip two layers of quotes from a single command line.
Use the shell keyword eval
in place of ssh localhost
to get the same effect. The example below shows using both remote shell, a second bash
shell, and eval
to test a quoted string:
$ ssh localhost echo Eat\\\ \\\ \\\;\\\ \\\ Pray\\\ \\\ \\\;\\\ \\\ Linux
Eat ; Pray ; Linux
$ echo echo Eat\\\ \\\ \\\;\\\ \\\ Pray\\\ \\\ \\\;\\\ \\\ Linux | bash -u
Eat ; Pray ; Linux
$ eval echo Eat\\\ \\\ \\\;\\\ \\\ Pray\\\ \\\ \\\;\\\ \\\ Linux
Eat ; Pray ; Linux
Using eval
removes two levels of quotes just the way that ssh localhost
or a pipeline into bash
does. You can use eval
for testing because it’s faster and doesn’t need us to type in any ssh
remote password.
argv.sh
to count argumentsIndexYou can use the argv.sh
script from the Class Notes to verify that a string is one single argument to the remote echo
command:
$ eval ./argv.sh Eat\\\ \\\ \\\;\\\ \\\ Pray\\\ \\\ \\\;\\\ \\\ Linux
Argument 0 is [./argv.sh]
Argument 1 is [Eat ; Pray ; Linux]