% Shell search PATH -- finding and running commands % Ian! D. Allen -- -- [www.idallen.com] % Fall 2015 - September to December 2015 - Updated 2017-02-13 01:44 EST - [Course Home Page] - [Course Outline] - [All Weeks] - [Plain Text] Shells find and run commands ============================ The job of a shell is to find and run commands. This file explains how the shell identifies commands names and then finds the commands to run, such as the `date` command: $ date Sat Feb 21 05:36:58 EST 2015 The shell finds the `date` command as an executable program file, often located in the file system as `/bin/date`: $ ls -l /bin/date -rwxr-xr-x 1 root root 59984 Nov 19 17:25 /bin/date $ /bin/date Sat Feb 21 05:37:12 EST 2015 The shell has an exact system for finding this executable file and executing it when you type just the name `date` at the start of a shell command line. This document explains how that works. Identifying the command name ============================ After the shell has processed and removed I/O redirection on a command line, the first token/word remaining at the beginning (left) of the line is assumed to be a command name. The command name in both lines below is the token `date`: $ date >out $ >out date Redirection is done first, is removed, and is never part of a command name. Shell built-in command names ---------------------------- A few shell *built-in* commands are executed directly by the shell and are not searched for and run as external programs, e.g. `echo`, `cd`, `umask`, `pwd`, `history`, `kill`, `help`, etc. Some built-in commands also have external versions, so that they are both external and built-in. (Often this is because something that was very useful as an external command was added later to the shell as a built-in command so that it would run faster or have built-in features.) Built-in commands that have no external version have no separate man pages; you can't say `man 1 cd` to find a manual page since there is no external `cd` command. For the BASH shell you can type `help` to get information about these built-in commands, e.g. `help cd`. Searching for command names in `PATH` ===================================== Any command name that is not built-in (e.g. `date`) is assumed to be the name of an executable program file, and the shell attempts to find an executable file with that name and run it. (Shells find and run commands.) If the command name contains no slashes (like most command names, e.g. `date`), the shell looks for the executable file with that exact name in the list of directories kept in the `PATH` environment variable. Because `PATH` is a shell environment variable, you can change the list, and the list is usually exported and inherited by child processes of your shell. Directories in `PATH` are separated by colons, e.g. the following `PATH` variable contains three directories separated by two colons: $ echo "$PATH" /usr/local/bin:/bin:/usr/bin When you type the command name `date` (with no slashes), the shell goes looking for the `date` executable file in the list of directories kept in the `PATH` environment variable, looking for an executable file named `date` to execute. The shell tries each directory in the `PATH`, left-to-right, and runs the first executable program with the matching command name that it finds. Using the above `PATH` list of directories, you can see that when you type `date`, the first directory tried in the `PATH` is `/usr/local/bin`, so the shell looks for an executable file named `/usr/local/bin/date`. This pathname does not usually exist: $ ls -l /usr/local/bin/date ls: cannot access /usr/local/bin/date: No such file or directory The shell next tries the second directory name in the `PATH` variable (`/bin`) and looks for an executable file named `/bin/date`, and this is the usual location of the `date` executable file: $ ls -l /bin/date -rwxr-xr-x 1 root root 59984 Nov 19 17:25 /bin/date Since this file exists, the shell runs this executable program and the date appears on your screen: $ date Sat Mar 16 20:39:13 EDT 2013 You could get exactly the same results on your screen if you typed the full pathname of the executable `date` file yourself: $ /bin/date Sat Mar 16 20:39:43 EDT 2013 Slashes in the pathname prevent the shell from using `PATH` to look up the command name, so the shell executes `/bin/date` directly. Only one executable file is run ------------------------------- If the same command name appears in multiple `PATH` directories, only the **first** one found is run. Even if both `/bin/date` and `/usr/bin/date` existed, the shell would stop at the first directory in `PATH` containing the `date` command and only execute that one file, once. Command not found -- not found in `PATH` ---------------------------------------- If the shell can't find the command name as an executable file in any of the `PATH` directories, it tells you it can't find the command: $ echo "$PATH" /usr/local/bin:/bin:/usr/bin $ fdisk bash: fdisk: command not found The message doesn't mean that there is no such command anywhere on the system, only that the shell can't find that command name in any of the directories listed in your current `PATH` variable. In the above example, none of these files existed: `/usr/local/bin/fdisk` `/bin/fdisk` `/usr/bin/fdisk` If you know the command really does exist, e.g. you know the path to the command is `/sbin/fdisk`, you could either type the full path to the command directly, or add the `/sbin` directory that contains `fdisk` to your `PATH` if you want the shell to find it: $ ls -l /sbin/fdisk # I know where the command is -rwxr-xr-x 1 root root 99448 Jun 17 21:21 /sbin/fdisk $ /sbin/fdisk # use the full path to the command Usage ... $ PATH=$PATH:/sbin ; echo "$PATH" # add /sbin to end of $PATH /usr/local/bin:/bin:/usr/bin:/sbin $ fdisk # now the shell can find it using $PATH Usage ... You can set your own list of directories in the `PATH` environment variable. People usually set and export their own `PATH` environment variable in one of the shell start-up files, such as `.bashrc`. Current directory in `PATH` (don't do it) ----------------------------------------- A leading or trailing colon in `PATH`, or two adjacent colons (`::`), indicate that the *current directory* is one of the directories that the shell should try when looking for executable command names. This is a security risk and you must not do it: $ echo "$PATH" :/bin/:/usr/bin # EXTREME SECURITY RISK! $ echo "$PATH" /bin/::/usr/bin # MODERATE SECURITY RISK! $ echo "$PATH" /bin/:/usr/bin: # SECURITY RISK! **Do not put the current directory (or any relative path directory) in `PATH` -- it is a security risk.** You might accidentally execute a malicious program in the current directory. > For example: If you put the current directory at the start of your `PATH`, > e.g. `PATH=:/bin:/usr/bin`, then when you type `ls` the first program that > is tried is `./ls` (the program named `ls` in the current directory) which > might be a malicious program. Command names with slashes avoid `PATH` ======================================= If the command name found by the shell at the beginning of the command line contains any *slashes*, the shell does not use `PATH` to find the executable file. If there are slashes, the shell executes that file pathname directly as a program and does not need to search for it: $ /bin/foo # executes the file /bin/foo; does not search PATH $ bar/foo # executes the file foo in the sub-directory bar; does not search PATH $ ./foo # executes the foo in the current directory; does not search PATH $ ../foo # executes the foo in the parent directory; does not search PATH Either the file exists (and is executable) exactly where the pathname leads, or else you get an error: $ ./foo bash: ./foo: No such file or directory $ doc/file.txt bash: doc/file.txt: Permission denied The rule is that no `PATH` searching is done if a command name contains any slashes. Executing a program in the current directory -------------------------------------------- The shell will not execute a program in the current directory unless you put `./` in front of the name to avoid the `PATH` search: $ cp /bin/date foo # create a program named foo in the current directory $ foo bash: foo: command not found $ ./foo Sat Mar 16 18:21:55 EDT 2013 If you don't use `./` in front of the name, the shell will look for `foo` only in the directories in `PATH`, and it won't be found. You must put slashes into the name to avoid the `PATH` search. **Do not put the current directory in `PATH` -- it is a security risk.** Examples of `PATH` and command names ==================================== Here are examples of how the shell finds and runs commands: $ PATH=/bin:/usr/bin ; ls - shell tries to find "ls" in first directory in PATH: /bin - shell looks for /bin/ls - this exists, so it is executed $ PATH=/bin:/usr/bin ; gcc - shell tries to find "gcc" in first directory in PATH: /bin - shell looks for /bin/gcc - this is not found - shell tries to find "gcc" in next directory in PATH: /usr/bin - shell looks for /usr/bin/gcc - this exists, so it is executed $ PATH=/bin:/usr/bin ; nosuch - shell tries to find "nosuch" in first directory in PATH: /bin - shell looks for /bin/nosuch - this is not found - shell tries to find "nosuch" in next directory in PATH: /usr/bin - shell looks for /usr/bin/nosuch - this is not found - shell issues message "bash: nosuch: Command not found" $ PATH=/bin:/usr/bin ; /usr/games/fortune - command name /usr/games/fortune contains slashes, PATH is NOT used - shell executes /usr/games/fortune directly (if it exists) - PATH is NOT used $ PATH=/bin:/usr/bin ; ./foo - command name contains slashes, PATH is NOT used - shell executes ./foo directly (if it exists) - PATH is NOT used $ PATH=/xxjunkxx ; ls - shell tries to find "ls" in first directory in PATH: /xxjunkxx - fails because /xxjunkxx/ls does not exist - shell issues message "bash: ls: Command not found" Because the shell appends your typed command name to each colon-separated pathname in the `PATH` variable when searching for an executable file, each pathname in `PATH` should be a directory, not a file name: $ PATH=/bin/ls ; ls - shell tries to find "ls" in first directory in PATH: /bin/ls - fails because /bin/ls/ls does not exist (/bin/ls is not a directory!) - shell issues message "bash: ls: Command not found" Your shell has some *built-in* commands that are executed directly by the shell itself and not looked up in `PATH`, e.g. `echo`, `cd`, `umask`, `pwd`, `history` $ PATH=/xxjunkxx ; date # fails (because /xxjunkxx/date fails) $ PATH=/xxjunkxx ; echo hi # works because echo is built-in to shell $ PATH=/xxjunkxx ; cd .. # works because cd is built-in to shell $ PATH=/xxjunkxx ; umask # works because umask is built-in to shell $ PATH=/xxjunkxx ; pwd # works because pwd is built-in to shell $ PATH=/xxjunkxx ; history # works because history is built-in to shell Commands related to `PATH` and finding commands =============================================== which - tell which PATH directory contains a command whereis - locate commands in "standard" directories (ignores PATH) (also locates man pages for you, if any) `which` -- which PATH directory contains the command ---------------------------------------------------- The command `which` tells you which directory in your `$PATH` contains a command name: $ echo "$PATH" /usr/local/bin:/bin:/usr/bin $ which date /bin/date $ which head /usr/bin/head Only the first directory containing the command name is printed. `whereis` -- where is this command (and man page) in the system --------------------------------------------------------------- The command `whereis` may tell you that a command exists in some standard directory, but when you try to execute the command it may not be found, if the standard directory is not one of your shell's `PATH` directories, or if you don't have permission to execute the command: $ fdisk bash: fdisk: command not found $ which fdisk # no output - not found in PATH $ whereis fdisk fdisk: /sbin/fdisk /usr/share/man/man8/fdisk.8.gz $ echo "$PATH" /bin:/usr/bin # PATH does not contain /sbin $ PATH=$PATH:/sbin ; echo "$PATH" # append :/sbin to PATH /bin:/usr/bin:/sbin $ which fdisk # now fdisk is found in PATH /sbin/fdisk $ fdisk # now fdisk is found in PATH Usage: [...] Most shells only look in `PATH`, not in any "standard" places. If you have a bad `PATH`, no non-built-in commands will be found. Appending to `PATH` ------------------- Since the `PATH` variable is a standard shell environment variable, you can modify it in the usual ways that you modify any variable. See **Appending to a variable** in [Shell Variables]. When you modify `PATH`, make sure that each directory in `PATH` is separated by a colon character `:` from each other directory. Never use two adjacent colons or a leading or trailing colon in `PATH` -- having the current directory in your `PATH` is a security risk. Summary ======= 1. Non-slash command names (the most common) are searched for in `PATH` 2. A command name containing any slashes is *not* looked for in `PATH`. The pathname is executed directly (if it exists). 3. You must put `./` in front of any program that you want to execute in your current directory, e.g. `./foo` 4. Do not put the current directory in `PATH` -- it is a security risk. PATH in Shell Scripts ===================== Put a correct `PATH` setting at the start of all your shell scripts and export it into the environment. If you don't set the `PATH` at the start of your script, the script will inherit the `PATH` from the person or program that executes your script. The inherited `PATH` may or may not contain the correct directories needed to find the commands used by your script -- your script may fail. Choose `PATH` in your script to include the system directories that contain the commands your script needs. Directories `/bin` and `/usr/bin` are almost always necessary. System scripts may need `/sbin` and `/usr/sbin`. GUI programs will need the X11 directories. Choose appropriately for the script. -- | Ian! D. Allen, BA, MMath - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [www.idallen.com]: http://www.idallen.com/ [Course Home Page]: .. [Course Outline]: course_outline.pdf [All Weeks]: indexcgi.cgi [Plain Text]: 400_search_path.txt [Shell Variables]: 320_shell_variables.html [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/