You might store tunes by artist, genre, and mood. You don’t want three copies of all your music files, so you create hard links to have only one copy, stored with names in three directories.
You’re working on a database compression routine. The program compresses the database into a new file, then removes the uncompressed database. Since the database is 10GB, you don’t want to have to create a 10GB copy of the database every time for testing. You create a hard link to the database, so that even when the database is removed by your program the data is still available using the other name and can be instantly restored for the next test.
You want to give someone access to some program you wrote. You don’t want to give them a copy of the program, since if you update the program they won’t get the updated version. Instead, you let them hard-link to your program so that there is only one copy of the program with two names. You can now update the program without having to remember to give new copies to anyone.
You might want to have a letter from your mother containing a chocolate chip cookie recipe filed under both your mom directory and your recipies directory, but you don’t want to have two copies of the file. If mom sends you an updated recipe, you don’t want to have to remember to edit two different files to make the change. You only want one file with two ways of accessing it.
Unix/Linux makes this easy by allowing you to make the pathnames mom/letter and recipies/cookie be two names for the same file data. You can edit the single copy of the file data using either pathname; the changes are done to the single copy of the file data that is accessed by either name.
To make multiple names possible, in Unix/Linux the file data is stored in a different place than the file name. The file data is given a unique number, called an index node or inode number, and the file data is stored on disk using only that number (not the name).
The names of files are stored on disk in a separate directory. Each file name is paired with just the inode number that matches the file data that goes with that name. The data is not stored with the name; only the inode number of the data is stored with the name. This makes it easy to have multiple names refer to the same inode number.
The ls command with the -i option will show you at the start of each output line the inode number paired with each file system name. The -d option to ls shows the actual directory permissions, not the directory contents:
$ ls -ild /bin /bin/bz*
711 drwxr-xr-x 2 root adm 4096 Jan 20 02:49 /bin/
601 -rwxr-xr-x 3 root root 31112 Dec 15 2011 /bin/bzcat
601 -rwxr-xr-x 3 root root 31112 Dec 15 2011 /bin/bzip2
444 -rwxr-xr-x 1 root root 2140 Dec 15 2011 /bin/bzdiff
[...]
We see above that directory /bin has its own inode number 711. The file names /bin/bzcat and /bin/bzip2 are both paired with inode 601 and the file name /bin/bzdiff is paired with inode 444.
The matching leading 601 inode numbers above show that the pathnames /bin/bzcat and /bin/bzip2 are not two different files. These are two names for the same 601 inode – two names for the same 601 file data. The file data is stored on disk just once under inode 601 and you can access this 601 inode data using either the pathname /bin/bzcat or the pathname /bin/bzip2. If you edit /bin/bzcat, you are editing the data stored in inode 601, and the changes will also be there when you access the same inode 601 using the alternate pathname /bin/bzip2. It’s the same as one person being called both “Bob” and “Robert” – both names lead to the same man; they are two names for the same thing.
The numeric field to the immediate right of the permissions in the long listing output above is the “link count” of a pathname. The link count counts how many names an inode has. We see that the 601 inode has a link count of 3, which means it has a total of three names in the file system. Only two of the three names are given in the ls output above; somewhere, there is a third name for this 601 inode. (The third name might be almost anywhere; finding the third name involves an exhaustive search of the entire file system to find what third name is paired with the 601 inode number – this can be very slow.)
You can have a large number of different names for the same inode in most Unix/Linux file systems – at least several tens of thousands of names.
To understand how Unix hard links and link counts work, it helps to understand how files and directories are stored in Unix file systems.
On Unix, the collection of data that makes up the contents of a directory or a file isn’t stored directly under a name; the data is stored as part of a numbered data structure called an “index node” or “inode”.
In the inode, Unix stores information about which disk blocks are allocated to store the contents of the directory or file, as well as attribute information about who owns the directory or file and what access permissions it has. Every directory or file on Unix has its storage and attributes managed by its own numbered inode.
Unix doesn’t store the name of the directory or file in the inode. Every inode in the file system has a unique number, and the file system locates the contents of a directory or file strictly by its inode number.
Directories give names to inode numbers. When you access a file by name, the system looks in a directory to see what inode number corresponds to that name, then the system goes to that inode to read the file data for that file name. There is no file data stored with the name in the directory, only an inode number.
Unix directories are very simple – they are just tables that pair names with inode numbers. Given a name, the directory will tell the system the inode number that corresponds to that name, and that is all. To read the actual data that corresponds to that name, the system still has to look in the file system to find the data for that inode (by number), and then read the data from that inode.
The inodes that contain directories and the inodes that contain files are similar. Directories are simply special kinds of inodes whose data blocks are tables of name-to-inode pairings. On some older versions of Unix (e.g. IBM AIX) you can even open a directory as if it were a regular file and read the binary name-inode tables directly!
Because names are stored separately from data, the name of a file or directory is stored in a directory inode that is different from the inode that actually holds the contents of that file or directory. The name-inode pairing information is stored in a directory inode; the actual data content of the directory or file is stored in its own separate inode. Recall this listing from above (note the use of the -d option to ls to show the actual directory permissions, not the directory contents):
$ ls -ild /bin /bin/bz*
711 drwxr-xr-x 2 root adm 4096 Jan 20 02:49 /bin/
601 -rwxr-xr-x 3 root root 31112 Dec 15 2011 /bin/bzcat
601 -rwxr-xr-x 3 root root 31112 Dec 15 2011 /bin/bzip2
444 -rwxr-xr-x 1 root root 2140 Dec 15 2011 /bin/bzdiff
[...]
The /bin directory inode 711 is where the names bzcat, bzip2, and bzdiff are actually stored – names are stored in directories. The file inodes 601 and 444 are where the content of the files is stored. The names are kept in a different place from the content.
The name-inode pairing for a file isn’t necessarily unique. A directory may contain several file names that are all paired with the same inode number; using any of the several names will lead to the same inode number and thus to the same file in the file system; these are called “hard links”.
The names bzcat and bzip2 in the /bin directory above are paired with the same 601 inode number, so these two names are really two names (hard links) for the same data inode 601.
Different directories may also contain name-inode pairings for the same inodes. It might be true that /etc/foo – in a different directory than /bin – might be a third name for inode 601:
$ ls -ild /etc/foo
601 -rwxr-xr-x 3 root root 31112 Dec 15 2011 /etc/foo
Thus, on Unix, a file (the data in a file) can have many names, even across different directories. You may use any of the several names of a file to find the unique data inode for the file. Unix calls these multiple names for the same inode “hard links” to the file.
5 File attributes are kept with the data inodeIndex
File attributes such as owner, modify time, and permissions are stored as part of the inode that contains the data for the file. This means it doesn’t matter what name you use to access the file data, you always arrive at the same inode and the same owner, modify time, and permissions. All that information is kept with the file data inode, not with any of the file names.
This means that even if you have a name for one of my files in one of your directories, it doesn’t give you any permissions to modify my file since the permissions to access and change my file are kept with the file inode that belongs to me. All you have is a name; nothing more.
Having a name for a file does not necessarily give you any permissions to read or change the file. To change data in a file, you must have permissions to modify the actual inode containing the data for the file. The location of the name for the file doesn’t matter.
5.1 Directories holding file names have different permissions than file dataIndex
The names of files are kept in directories that are separate from the file data. Directories have their own storage inodes, and so the directory containing a name of a file may have a different set of attributes than the inode that contains the file data itself. Revisiting our example above (note the use of the -d option to ls to show the actual directory permissions, not the directory contents):
$ ls -ild /bin /bin/bz*
711 drwxr-xr-x 2 root adm 4096 Jan 20 02:49 /bin/
601 -rwxr-xr-x 3 root root 31112 Dec 15 2011 /bin/bzcat
601 -rwxr-xr-x 3 root root 31112 Dec 15 2011 /bin/bzip2
444 -rwxr-xr-x 1 root root 2140 Dec 15 2011 /bin/bzdiff
[...]
The directory /bin (inode 711) that holds the names bzcat, bzip2, and bzdiff has different attributes than the inodes of the files whose names it holds. The attributes that apply to the directory holding the file names are not the same as the attributes that apply to the file content for those names.
Changing the attributes of the directory that contains a name-inode pairing for the file doesn’t affect the owner and permissions of the file content itself. The directory containing the name and the file content are in different inodes.
The permissions you need to change the name of a file are contained in the inode corresponding to the directory that contains the name. The permissions you need to change the data in the file itself are contained in the inode corresponding to the file data. These are two different inodes, and each inode may be owned and controlled by a different userid. An example:
555 -rw-r--r-- 3 chris chris 12 Dec 5 2013 /home/pat/foo
555 -rw-r--r-- 3 chris chris 12 Dec 5 2013 /home/leslie/foo
555 -rw-r--r-- 3 chris chris 12 Dec 5 2013 /home/chris/foo
Users Pat and Leslie can each own directories (their HOME directories) containing a name-inode pairing (a name) for a file whose actual data inode is owned by Chris. Chris retains full control of access to the file data, because the inode containing the data is owned by him. Pat and Leslie can’t write onto the file’s data inode; they don’t have permission.
Pat and Leslie can only manipulate their local name-inode pairings in their own directories that they own. They can change their own names for Chris’ file in their own directories. They can create new names (hard links) for the file, change their existing names for the file, or remove their names for the file, but they can’t affect the file data itself, unless file owner Chris changes the permissions to permit it.
Even if both Pat and Leslie remove their names for this file, Chris still has a name for it and the file continues to exist.
A name is stored separately from content, and you need one set of permissions to manipulate the directory inode that contains the name and a different set of permissions to manipulate the inode that is the content.
6 Hard links: manipulating names for inodes using ln and rmIndex
The ln (“link”) command will create a new name-inode pairing in a directory, if you have permission to write in the directory. Given an existing name for a file, ln will create another name that pairs the new name with the same inode number, creating another name for the same data inode. This is called “making a hard link” to a file:
$ date >foo
$ ln foo bar
$ ls -i foo bar
5001 foo 5001 bar
$ cat foo
Mon Feb 18 14:06:31 EST 2013
$ cat bar
Mon Feb 18 14:06:31 EST 2013
There is no record as to which name was the “first” name for the inode. All names are equally valid; all names lead to the same unique data inode.
You do not normally need any permissions on the data inode to make a new name for the inode in a directory; you can make a local name in your own directory for any file you can see, even if you can’t read or modify the file data because you have no permissions on the data inode for that file.
Recent versions of the Linux kernel (since 3.6) have created a new kernel security option that, when enabled, prevents you from making a hard link to (new name for) a file that you do not own and cannot read or write. See the kernel sysctl option fs.protected_hardlinks or entry /proc/sys/fs/protected_hardlinks in the man page proc(5). This Linux course assumes that this option is not enabled, so that you can create a hard link to any file you can access in the current file system without restriction.
6.1 Link count: counting the names for the inodeIndex
For every inode on the system, a link count in the inode keeps track of how many names this inode has. If an inode has only one name-number pairing (only one name), its link count is one. If the inode has two name-number pairings (two names), its link count is two.
The numeric field to the right of the permissions in the long listing output below is the “link count” of a pathname:
$ ls -il /bin/bzcat
601 -rwxr-xr-x 3 root root 31112 Dec 15 2011 /bin/bzcat
We see that the 601 inode above has a link count of 3, which means it has a total of three names in the file system. One of those names is /bin/bzcat.
The ln command creates a new name (a “hard link”) for a file and increases its link count by one:
$ date >foo
$ ls -li foo
34 -rw-r--r-- 1 idallen idallen 29 Feb 18 17:52 foo
$ ln foo bar
$ ls -li foo bar
34 -rw-r--r-- 2 idallen idallen 29 Feb 18 17:52 bar
34 -rw-r--r-- 2 idallen idallen 29 Feb 18 17:52 foo
$ ln bar dog
$ ls -li foo bar dog
34 -rw-r--r-- 3 idallen idallen 29 Feb 18 17:52 bar
34 -rw-r--r-- 3 idallen idallen 29 Feb 18 17:52 foo
34 -rw-r--r-- 3 idallen idallen 29 Feb 18 17:52 dog
Notice how the link count of inode 34 increases with each new name.
You do not normally need any permissions on a file to make a hard link to it (but see the kernel security note above) .
6.2 Removing names: decreasing the link countIndex
The rm command will delete from a directory a name-inode pairing for a file. It only removes the name from the directory, nothing else. The link count for the data inode of that file will go down by one when that name for the inode no longer exists.
When a link count goes to zero in an inode, that means that no directory has any name for that inode and the system is free to release and reclaim the disk space used by the inode. Only when the link count goes to zero is the actual file content deleted.
The way the system knows it can release and free up data blocks for a file is to watch the number of names a data inode has. As long as the data inode has even one name, the data must be kept. When the number of names for the data inode goes to zero, the system knows that this data doesn’t have any names anywhere and the system can release and free up the used data blocks.
Contrary to what the man page says, the rm command does not delete a file – it only deletes a single name for a file. Only when all the name-inode pairings for that data inode are gone does the actual file data space get reclaimed. A file with a link count of ten needs ten names to be removed before the file content will be released.
Disk space will not be reclaimed until all the names for a file are removed and the link count goes to zero.
7 Linking, Moving, and Removing are directory operations: ln, mv, rmIndex
Anyone can normally create a hard link (ln) to any file to which they have access (except see the earlier note on added kernel security options that may prevent this). They don’t need to be able to read or write the file data to make the link; they only need to be able to access the inode of the file and they need write permission on the directory in which the name-inode pairing (the name, or “link”) is being created. (Also, unrelated to permissions, the inode of the thing to which the link is being made must be on the same file system as the directory that will contain the link. See “File Systems”, below.)
Anyone can rename (mv) a file name if they can write on the directory holding the file name. Moving a file name doesn’t change the data; it only moves the name for the file data. The inode number stays the same after the move (unless you are moving the file between file systems). The data itself does not move and is not touched; only the name moves.
Anyone can remove (rm) a file name if they can write on the directory holding the file name. Removing a name doesn’t change the data; it only decreases the link count of the data inode by one. (Only when the link count goes to zero does the file data get released.)
Linking, moving, and removing are all purely directory operations. They only affect file names; they don’t touch or affect the data. You don’t need any permissions on the data to link, move, or remove a file name.
Unix permits you to give files many additional names (“hard links”). You are not allowed to create additional hard links to directories because it would break many useful file system properties, such as:
Every directory has a unique parent directory. Hard-linked directories would mean that one directory could have multiple parent directories, one for each hard link to the directory. Where would cd .. lead, if there are multiple choices?
There is a unique symlink-free absolute pathname for every file system basename. Hard-linked directories would allow multiple different absolute pathnames with different directory hard links for the same basename, e.g. /a/b/c and /a/x/c. Not good.
The Unix file system is a Directed Acyclic Graph, which means that if a directory contains the name of a sub-directory, other than . or .., that sub-directory is guaranteed not to be an ancestor of the current directory or to lead to any ancestor of the current directory. Hard links would let a sub-directory name be a link to a parent directory, creating a file system “cycle” or loop. (Only the special name .. is allowed to refer to an ancestor directory.)
This prohibition against hard links to directories means that every inode number that refers to a directory inode is allowed to appear only once in exactly one parent directory and no more. Every directory has exactly one name given to it by its parent directory. This restriction also means that every sub-directory only has one unique parent directory, and that means the special name .. (dot dot) in a sub-directory always refers unambiguously to its unique parent directory.
This directory linking restriction prevents loops and cycles in the file system tree, preventing cases where a sub-sub-directory might contain a link back up to a parent directory. Many things are simpler if the file system tree has no loops or cycles.
Every directory has a name. That name is stored as a name-inode pairing in the unique parent of the directory, and that name counts as one link in the link court field of the directory inode.
Every directory also contains the special name . (dot), a short name that is always paired with the inode number of the directory itself. Since . is also a real name for the current directory, it is also counted as a link in the link count field for the directory.
Thus, the smallest link count of any Unix directory is two: count one link for the unique name-inode pairing in the parent directory that gives the directory its Unix “name”, and count another link for the self-referential . (dot) pairing in the directory itself.
Every directory, even an empty one, must have two names. Every directory, even an empty one, has a link count of at least two.
Files aren’t directories and don’t have a self-referential ., so new files have a link count of one, not two.
9.1 Each subdirectory adds one to the link countIndex
A directory may have sub-directories. Since the special name .. (dot dot) in every one of those sub-directories is a link to the inode number of the parent directory, the link count of the parent directory is increased by one for every sub-directory the parent contains.
The link count counts how many name-inode pairings point to this inode, and that includes the special . (dot) and .. (dot dot) name-inode pairs! A directory with five sub-directories will show a link count of 2+5=7.
Note that creating files in a directory does not affect the link count of the directory, since files don’t create new name-inode pairings for the directory containing the file names. Nothing in a file points back up to the parent directory in the same way that .. does in a sub-directory. The files themselves will have their link counts set to one by virtue of having new name-inode pairs created in this directory, but the directory link count will not be affected.
$ ls -ild /bin /bin/bz*
711 drwxr-xr-x 2 root adm 4096 Jan 20 02:49 /bin/
601 -rwxr-xr-x 3 root root 31112 Dec 15 2011 /bin/bzcat
601 -rwxr-xr-x 3 root root 31112 Dec 15 2011 /bin/bzip2
444 -rwxr-xr-x 1 root root 2140 Dec 15 2011 /bin/bzdiff
[...]
The inode numbers show which names are links to the same content. In the example above, two file names in the /bin directory are paired with the same inode number 601, and the link count on this inode is three. This is one file (one inode) with three names, two of which are visible in the listing above. There is a third name somewhere.
The only way you can tell which directory names point to the same files is by looking at the inode numbers using the -i option to ls.
If you know one name for an inode and want to find the other names, you have to use the super-user account and walk the entire file system looking for the inode number (don’t do this on a shared machine; it’s very disk intensive):
The above find command walks the entire file system, looking for names that are paired with inode 601. Doing this is both slow and disk-intensive. Some new versions of find have a -samefile option to do the same thing without having to look up the inode number first. It’s still a very slow and disk-intensive operation. Don’t do it unless you have to.
12 File System Damage – Orphans and Lost+FoundIndex
When a Unix file system suffers damage, one or more inodes may become unreadable. This usually has one of two consequences:
If the damaged inodes are file inodes, the file data blocks pointed to by those inodes will be missing or incomplete. The file will be all or partially unreadable.
If any of the inodes are directory inodes, containing the names of files and sub-directories, then the files and sub-directories that were once pointed to by those inodes will lose their names and become “orphans” – inodes with data and a positive link count, but no names.
At boot time, the Unix file-system checking program, fsck, notices the existence of files and sub-directories that have a positive link count but no longer have any names in any undamaged directories in the file system. (On an undamaged file system, an inode with a positive link count will always have that many names in directories somewhere in the file system.) These are “orphan” inodes – they have data but no name(s).
The fsck program gives the orphan inodes false names (most Unix systems create new names that are just the inode numbers) and links them into a special directory named /lost+found when the system reboots itself. The system administrator must go into the directory and figure out what the files are, what their names are, and where they belong.
Note that only the names are lost when a directory is damaged. The owner, permissions, and access times are all stored with the data, and are not lost. Only the names are lost.
A Unix file system is equivalent to a single disk partition (e.g. a “drive letter” in Windows). Each Unix file system (disk partition) has its own set of inode numbers, and you can’t make hard links between different file systems. Since the overall hierarchical tree on a Unix system may transparently include pieces from several file systems (from several partitions) all connected together, some items in the tree will have the same inode numbers, but will actually be different files residing in different file systems:
$ df / /home
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 29528020 5458708 22569364 20% /
/dev/sdc1 516060600 3008500 486837752 1% /home
$ ls -lid /lost+found /home/lost+found
11 drwx------ 2 root root 16384 Sep 7 2012 /lost+found
11 drwx------ 2 root root 16384 May 19 2013 /home/lost+found
Above, we see that the ROOT file system is on disk partition /dev/sda1 and the /home directory is actually another file system (another disk partition) /dev/sdc1. The directories /lost+found and /home/lost+found have the same inode numbers, but they are in different file systems.
A directory’s name-to-inode-number pairing applies only within a single Unix file system. It isn’t possible for a directory to pair a name with (hard link) an inode number in a different file system (i.e. in a different disk partition). Inodes are only unique within one disk partition.
The df command can show on which disk partition (file system) a file or directory resides:
$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 1511856 1121408 313648 79% /
/dev/sdb1 198337 5663 182434 4% /mnt/ext3
/dev/sdb2 99150 5646 88384 7% /mnt/ext4
$ df /etc/passwd
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 1511856 1121408 313648 79% /
$ df /mnt/ext3/foo
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdb1 198337 5663 182434 4% /mnt/ext3
Using the above listing, we can see that the system has three file systems mounted from three different disk partitions. Pathnames under /, /mnt/ext3, and /mnt/ext4 will have some of the same inode numbers; but, being on different file systems, they are really different files. You can’t make hard links between any of these three file systems; you can only make hard links inside the file systems.
The -i option to the ls command shows which inode number is associated with a file or directory:
$ ls -lid / /mnt/ext?
2 dr-xr-xr-x 22 root root 4096 Nov 22 05:03 /
2 drwxr-xr-x 3 root root 1024 Nov 24 12:21 /mnt/ext3
2 drwxr-xr-x 3 root root 1024 Nov 24 12:34 /mnt/ext4
We see that the ROOT of every Linux file system is inode 2, and each inode 2 above is a different inode 2 since each is on a different file system residing in a different partition.
If you want to create a link from one file system to another, you can’t use hard links but you can use symbolic links:
$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 1511856 1121408 313648 79% /
/dev/sdb1 198337 5663 182434 4% /mnt/ext3
/dev/sdb2 99150 5646 88384 7% /mnt/ext4
$ ln /mnt/ext3/foo /mnt/ext4/bar
ln: failed to create hard link `/mnt/ext4/bar' => `/mnt/ext3/foo': Invalid cross-device link
$ ln -s /mnt/ext3/foo /mnt/ext4/bar
$ ls -l /mnt/ext4/bar
lrwxrwxrwx 1 alleni alleni 13 Mar 5 07:35 /mnt/ext4/bar -> /mnt/ext3/foo
The fsutil command in Microsoft Windows will create Windows hard links and tell you information about Windows hard links (information from Charles Dougan):
From: Charles Dougan
> Is there a way to determine (i.e via command line) that
> ben.bat is "hardlinked" to charles.bat?
C:\> fsutil hardlink list c:\charles\hardlink.txt
\Ben\ben.txt
\Charles\hardlink.txt
C:\> fsutil hardlink create
Usage : fsutil hardlink create <new filename> <existing filename>
Eg : fsutil hardlink create c:\foo.txt c:\bar.txt
C:\> fsutil hardlink create c:\Ben\bogus.txt c:\charles\hardlink.txt
Hardlink created for c:\Ben\bogus.txt <<===>> c:\charles\hardlink.txt
C:\> fsutil hardlink list c:\charles\hardlink.txt
\Ben\ben.txt
\Ben\bogus.txt
\Charles\hardlink.txt
Author:
| Ian! D. Allen, BA, MMath - idallen@idallen.ca - Ottawa, Ontario, Canada
| Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/
| College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/
| Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/
