Winter 2015 - January to Apil 2015 - Updated 2015-04-03 02:59 EDT
Do not print this assignment on paper!
- On paper, you will miss updates, corrections, and hints added to the online version.
- On paper, you cannot follow any of the hyperlink URLs that lead you to hints and course notes relevant to answering a question.
- On paper, scrolling text boxes will be cut off and not print properly.
23h59 (11:59pm) Tuesday March 31, 2015 (start of Week 12)
Do not print this assignment on paper! On paper, you cannot follow any of the hyperlink URLs that lead you to hints and course notes relevant to answering a question.
This is an overview of how you are expected to complete this assignment. Read all the words before you start working.
For full marks, follow these directions exactly.
You will create some minimal file system structure in your HOME directory on the CLS.
Most work will involve changes in your own Linux Virtual Machine running Centos 6.6. You can use the Checking Program to check your work as you go. You can check your work with the Checking Program as often as you like before you submit your final mark.
When you are finished, leave the files and directories in place on both the CLS and your own CentOS Virtual Machine as part of your deliverables. Do not delete any assignment work until after the term is over! Assignments may be re-marked at any time on the CLS; you must have your term work available on the CLS right until term end.
All references to the “Source Directory” below are to the CLS directory
~idallen/cst8177/15w/assignment08/
and that name
starts with a tilde character ~
followed by a user name
with no intervening slash. The leading tilde indicates to the shell that the
pathname starts with the HOME directory of the account idallen
(seven letters).
You do not have permission to list the names of all the files in the Source Directory, but you can access any files whose names you already know.
The previous term’s course notes are available on the Internet here: CST8207 GNU/Linux Operating Systems I. All the notes files are also on the CLS. You can learn about how to read and search these files using the command line on the CLS under the heading Copies of the CST8207 course notes near the bottom of the page Course Linux Server.
Review your work from CST8207 GNU/Linux Operating Systems I:
passwd, useradd, userdel, groupadd, gpasswd, usermod, chown, chgrp, chmod, chsh
Review your work from CST8207 GNU/Linux Operating Systems I:
fdisk, mkfs, mount, /etc/fstab
fdisk, mkfs, mount, swap
I recommend that once you have booted your CentOS VM, you connect to it
and work using a remote login session (e.g. ssh
or
PuTTY
) where copy-and-paste works and where you can have
multiple simultaneous connections into the VM. The VMware console is not
friendly.
If you can’t get an SSH (PuTTY or ssh
) connection working
into your Linux VM, see the Network
Diagnostics page.
Note that SSH sessions (and whatever you are doing inside them) do not survive across a VMware suspend. Make sure you save your editor files and exit your SSH session before you pause or suspend your virtual machine. (Editor sessions that run inside the VMware console do survive across suspend and resume, since they don’t depend on a network connection.)
Advanced users may look into the various virtual terminal programs such as
tmux
andscreen
that do allow you to suspend and resume your sessions even from a remote login.
Complete your CentOS Install and Configure.
Before you begin this assignment, create a snapshot of your CentOS Virtual Machine.
Do a Remote Login to the Course Linux Server (CLS) from any existing computer, using the host name appropriate for whether you are on-campus or off-campus.
Make the CLS directory ~/CST8177-15W/Assignments/assignment08
~/CST8177-15W/Assignments/assignment08
(the same hierarchy
as you have already made on the CLS).This CentOS assignment08
directory in your sysadmin
account is the Base
Directory for all pathnames in this assignment. Store your CentOS files
and answers below in this sysadmin Base
Directory.
Run the Fetch and Checking Program to verify your work so far.
You will use the standard account management tools to create a few ordinary (non-admin) accounts, just as you did last term. You will force password expiry so that the users must change their passwords when they first log in.
Log in to your CentOS system administration account, if necessary, and
obtain root
shell privileges using the sudo
command, if necessary.
$
to include the
#
character that indicates root
privileges.root
PATH
that
includes /sbin
Type whoami
or id
to confirm that you are
the root
user.
Take a VMware snapshot that you can return to if things go wrong.
Create three new users by running the appropriate command three times:
user001
, user002
,
user003
User One
,
User Two
, User Three
/etc/skel/
directory.Set different, good initial passwords for the three users.
Force these users to change their password upon first login.
Set their HOME directory permissions to be as follows:
Run the Fetch and Checking Program to verify your work so far.
Few organizations create users manually. The batch
newusers
command (RTFM) can read a text file and create user accounts in bulk. In this section, you will be creating a text file suitable for input to thenewusers
command, then usingnewusers
to quickly create almost a hundred new accounts.
Log in to your CentOS system administration account, if necessary.
root
shell, if you are running as
root
.Type whoami
or id
to confirm that you are
not running as root
.
Take a VMware snapshot that you can return to if things go wong.
On CentOS, create your Base Directory in which you will create the files and scripts resulting from the following tasks. (You already did this on the CLS; now do it here on CentOS.)
userlist.csv
userlist.csv
from the Source Directory on the CLS to your CentOS
Base Directory.
scp
command in Unix/Linux
SCP Command.scp
command.scp
command name.
cp
command.Imagine that the userlist.csv
file was given to you from the
Human Resources department by someone who created it with a spreadsheet.
Examine this file, and notice that it is in Colon-Separated-Value format. It is 98 lines: a header line and a username and a real name for each of 97 new users that need an account on your system.
The file contains five fields, separated by colons (:
). Read
the header line to know what the five fields are. (A real spreadsheet export
would be separated by commas, but we’re making it easier for you.)
We need to create a text file suitable for batch input to the
newusers
command. Every line in the file we give to
newusers
must have the correct format: it must have the userid
at the start and the seven colon-separated fields described at the top of the
newusers
man page.
The file given to us only has five fields, and it has a poor password set
for all the accounts. We need to fix this file before we can feed it to the
newsers
command.
Note: If you read all the words in this section before you start working, you will save yourself some file copying by using one command pipeline (no temporary files needed) instead.
userlist.newusers
userlist.newusers
based on
userlist.csv
, but make the following changes.
Copy the userlist.csv
file into the new file
userlist.newusers
.
The first line in userlist.csv
is a header line, not a
user to be created; the first line must be deleted. Use a command to read the
file, remove the first (header) line, and write a temporary output file. The
temporary output file should be only 97 lines long. (Hints:
What command shows the last 97 lines of a file? That same
command has a syntax to “print beginning with the Kth item from the start of
each file” which allows you to skip the first line without knowing how many
lines are in the file. Don’t use the number 97, since it might change in
future.)
Move the temporary output file to be the
userlist.newusers
file.
Use sed
to read the new file and on every line insert the
two colon characters that correspond to the location of the missing
pw_uid
and pw_gid
fields needed by
newusers
, and write a temporary output file. For example, use
sed
to change this line:
user066:password:User 066:/home/user066:/bin/bash
to this line with two more colon characters in the right place:
user066:password:::User 066:/home/user066:/bin/bash
on all 97 lines. (This is a one-expression sed
substitution.)
(Hint: You can’t simply change a colon to three colons. Use
some fixed context around your expression, to select the correct colon to
change.)
Move the temporary output file to be the
userlist.newusers
file.
All the seven fields in the file are acceptable except the
pw_passwd
field that currently contains password
,
which is not a good default password for all these accounts. RTFM to
see how the pw_passwd
field is used by
newusers
.
sed
to read the new file and on every line change the
pw_passwd
field from the dummy value password
to a
single, common password that all of these new users will get. You choose the
new password. (This is not very secure, but it’s the best you can do without
writing a more complex script.)password
.sed
command to
read this file and change the word password
to the password that
you made up, redirecting the output of the sed
command to a
temporary output file.Move the temporary output file to be the
userlist.newusers
file.
Verify that your output file is 97 lines (no header line) and the only
field that has changed in each line is the new password
field.
pw_passwd
field position.convert_userlist.sh
Realize that the above three edits could be done as one pipeline that
reads the original userlist.csv
file, and makes each of the
above three changes using filters. With a pipeline, no temporary files are
needed.
Write this command pipeline and when it is working, remove all the
file names and put the commands in a script named
convert_userlist.sh
. With file names removed, the script should
act as a “filter” and read standard input and write standard output, so that
you can type:
$ ./convert_userlist.sh <userlist.csv >userlist.newusers
Hints: The script file will contain two or three commands
(which might inclue tail
and sed
and maybe another
sed
) separated by pipe characters. If you RTFM, you can combine
the two sed
commands into one sed
command with two
expressions, or perhaps even into one single expression that does both edits
at the same time. Remove all file names from the script, so that the script
reads standard input and writes to standard output, as shown above. Do not
put file names in the script.
Verify that the userlist.newusers
file created by your
script contains 97 lines and 194 words, with seven fields per line, with good
passwords.
Use sudo
to run the newusers
command with
this file to create all of these 97 new users.
Make sure all the new users and HOME directories exist:
user001
through user100
/etc/skel/
in them. Only useradd
copies these
files.Run the Fetch and Checking Program to verify your work so far.
This reviews the account management commands you learned above and in your previous term. Links to previous term notes and worksheets are given above under Review of CST8207 account management. None of the items below require you to text-edit any system files using a text editor. Actions can be performed using the correct account management commands. Most account management commands will require
root
permissions to run.
Make sure you have correctly followed all the above steps, including
using the newusers
command to create 97 accounts. Verify that
you have created all the users and HOME directories for accounts
user001
through user100
before continuing.
Use a system admin command to create a new group called
common
.
Use a system admin command to add users User 004
and
User 005
to the common
group.
/home/common
Create a directory called /home/common
owned by your
sysadmin user, and group-owned by the new common
group.
Change the permissions on /home/common
so that your
sysadmin user can read/write/search, members of the common
group
can read/write/search, and it is not accessible in any way to other
users.
/home/common/README
User 004
without using a password (using your
sysadmin powers)
User 004
create a file /home/common/README
containing the text:This common directory is for members of the common group.
common
group you just created.user004
shell to revert back to your
superuser-self.User 005
, and ensure that as user005
you
can edit the file /home/common/README
and change the text
“for members
” to “for all members
”.
user005
shell to revert back to your
superuser-self.User 006
and ensure that as user006
you
have no access to the /home/common/
directory.
user006
shell to revert back to your
superuser-self.User 005
to: CommonUser 005
User 006
and User 007
.
User 008
and User 009
to
/bin/sh
.
User 010
and User 011
without deleting their HOME directories.
ls
tells you about the file owner now!)user010
and
user011
groups, if they are still present on your machine.
newusers
command did not create shadow group entries. Ignore the
error – the groups don’t exist in the group shadow file.User 012
and User 013
using the option that also deletes the HOME directory at the same time.
None of the items above require you to text-edit any system files using a text editor. Actions can be performed using the correct account management commands. Most account management commands will require
root
permissions to run. Do not text-edit the system files!
Run the Fetch and Checking Program to verify your work so far.
sdb
You will add a second hard disk to your CentOS Virtual Machine, and partition it. The procedure for adding a hard disk to an actual physical computer is different only in the steps that take place while the machine is powered off. Any step carried out while the machine is running would be the same for physical machines as it is for virtual machines. The console of a physical machine is its actual keyboard and monitor, but in the case of a VM, the console is the VMware window of the machine.
Most of the system admin commands in this assignment access the raw disk
and will require you to prefix the actual command name with sudo
to gain root
permissions (unless you are in single-user mode and
therefore running everything as root
). If you get
“permission denied” errors, you forgot to use sudo
.
If your CentOS Virtual Machine is not already powered off, login and use the correct command to power off the virtual machine.
With your CentOS machine still powered off, use the
VMware Settings menu for your CentOS VM to
add to your VM a virtual 10GB
hard disk, accepting defaults for
everything except the size. (You did similar work in CST8207 adding a
VMware disk; review the notes.)
After adding the new disk, power on your VM, then login as your system administrator user.
Ensure the /proc/partitions
file contains the second disk
you added.
10485760
.10485760/1024/1024
to confirm the number of
gigabytes.sdb1
or sdb2
or other
sdb
partitions, this is not a new disk with no
partition table. Get help.partitions_before.txt
/proc/partitions
to
file partitions_before.txt
in your CentOS sysadmin Base Directory (6
lines, 20 words).
sd_all.txt
/dev
directory. Put a long (ls -l
) listing of all names under /dev
that
start with the first two letters of the new disk name into file
sd_all.txt
in your Base Directory.
Advanced Usage: If you need to add a disk to a machine that is powered on (and you can’t afford to reboot the machine), a modern Linux kernel can find the disk if you tell it to re-scan the SCSI host bus containing the new disk, something like this (the exact host number may difer):
echo "- - -" >/sys/class/scsi_host/host2/scan
If the scan is successful, runningdmesg
will show the new disk being found and you will see the device appear under/dev/
. You can use this as an alternative to shutting down and rebooting.
Run the Fetch and Checking Program to verify your work so far.
fdisk
First, you must have added a new 10GB
hard drive in
VMware and rebooted, as described above. Log in to the
machine.
Run (always with root
privileges) fdisk -cul /dev/sdb
and make sure you see Disk /dev/sdb: 10.7 GB
with no errors and no partitions
listed under it.
$ sudo fdisk -cul /dev/sdb
Disk /dev/sdb: 10.7 GB, 10737418240 bytes
If you don’t see 10.7 GB
, then shut down, delete the disk,
recreate the disk, and reboot until your 10GB disk install
works.
Make sure you only change things on this new
sdb
disk in this section! Thesda
disk is your Linux ROOT disk; if you damage it you will need to recover back to your snapshot. Make sure you have a snapshot to go back to!
man
page for the fdisk
command, locate
and make a note of two option letters:
fdisk
devicename
, where devicename
is the
absolute path of the device corresponding to the new disk under
/dev
. This will start the fdisk
program, just as
you did in CST8207
Fdisk Command.
fdisk
starts, read the WARNING
about
DOS-compatible mode.fdisk
program.fdisk
command, this time inserting the two option
letters you found in the man
page. (Keep the same device
name.)WARNING
about DOS-compatible mode
should be
gone when you start fdisk
with those two options. Always use
these two options on CentOS. (Other versions of fdisk
use these
options as defaults.)Warning
about an invalid flag; ignore
it.fdisk
, display the partition table and verify that
the disk you are working on is the 10GiBi disk with no partition table.If you don’t know the difference between a power-of-two MebiByte and a power-of-ten Megabyte, read up on Binary Prefixes. The
fdisk
program can create partitions using either MebiBytes (e.g. using syntax+500M
) or Megabytes (e.g. using syntax+500MB
). We always want power-of-two MebiBytes (e.g.+500M
) in this assignment.
fdisk
use the command to display the partition table
and verify that the disk you are working on is the 10GiBi
(10.7 GB
) disk with no partition table. Use fdisk
commands to partition the new disk as follows:
fdisk
on the wrong
disk. Make sure you use fdisk
on the new disk device
name!500
megabyte (MebiByte) primary partition as
Partition 1:
+sizeM
where size
is the number of
MebiBytes.M
and not MB
so that you create
power-of-two MeBi
bytes instead of power-of-ten Mega bytes.MB
suffix in this assignment, since that
creates power-of-ten Megabytes. We always want power-of-two MebiBytes in this
assignment.400
megabyte (Mebibyte) logical partition.fdisk
tells you as it quits whether you need
to reboot for the new partition table to take effect. Do what it says.partitions_after.txt
/proc/partitions
(showing the new
partitions you just created) to partitions_after.txt
in your
sysadmin Base
Directory.
/dev
directory.partitions_diff.txt
Use the diff
command to find the differences between the
old and new partitions_{before,after}.txt
and redirect the
results to partitions_diff.txt
in your sysadmin Base
directory.
Examine the differences file, and verify that your new partitions are the only differences. You should see four additional lines in the new partition file, corresponding to the four partitions you created:
6a7,10
> 8 17 512000 sdb1
> 8 18 1 sdb2
> 8 21 409600 sdb5
> 8 22 9561088 sdb6
If your numbers differ, perhaps you forgot to use the fdisk
options that turn off DOS-compatibility mode and switch to using sectors
instead of cylinders, or perhaps you used MB
instead of
M
when creating the partitions. Delete and start over.
Run the Fetch and Checking Program to verify your work so far.
/home
directory to its own filesystemYou will create an
ext4
filesystem on the primary partition of the new hard disk. Then, in single user mode, you will migrate the contents of the/home
directory to that new filesystem. You will configure the/etc/fstab
so that the new filesystem will be automatically mounted on/home
, with the option for giving the users disk space quotas.
lsof
package. RTFM to see what it does.fdisk.txt
Run the command file -s /dev/
somedevice
to check the type of the device special
file somedevice
that corresponds to your new primary
partition on your new disk. Because it has nothing on it, you should see
nothing but unknown data
:
/dev/sdb1: data
Put the above output into file fdisk.txt
in your sysadmin Base Directory. The
file will contain one line.
Create an ext4
filesystem on the only primary partition
on the new disk.
After creating the filesystem, again check the type of the device that
corresponds to your new primary partition on your new second disk. It should
show an ext4
filesystem that is not active:
Linux rev 1.0 ext4 filesystem data (extents) ...
Append the above output to file fdisk.txt
in your sysadmin Base Directory. The
file will contain two lines.
Also use file -s
to check the type of file
system in the first partition of the first disk. This first partition of the
first disk has an active (in use) ext4
filesystem. Note the
warning “needs journal recovery” indicating this filesystem is open
and being modified:
Linux rev 1.0 ext4 filesystem data (needs journal recovery) ...
Append the above output to file fdisk.txt
in your sysadmin Base Directory. The
file will contain three lines.
Also check the type of the second partition of the first disk. It will
say that it is not an ext4
filesystem.
Append the above output to file fdisk.txt
in your sysadmin Base Directory. The
file will contain four lines.
The fdisk.txt
file should contain four lines.
Run the Fetch and Checking Program to verify your work so far.
/home
firstYou are about to make a copy of all the files in the /home
directory.
now
./home
directory. On a real
system, you would run a full back-up before you attempted this. We don’t have
a back-up system running on CentOS, but since the /home
is
small, we can create a tar
archive back-up file.Close down any remote login sessions you are running into your CentOS
machine. Exit all PuTTY and SSH connections. These connections will not work
when you shut down to single-user, but they could leave processes running
that might interfere with moving the /home
directory.
From the console (the actual VMware window, not a remote PuTTY or an
SSH login that will be disconnected) take the system down to single user mode
using shutdown now
to do so. (Do not halt the
machine!)
Telling INIT to go to single-user mode.
1 S
whoami
and confirm that you are always the
root
user when running in single-user mode. Be
careful!home.tar.gz
Create a compressed tar
archive of /home
and
save it under the HOME directory of the root
account (which is
not the ROOT directory) using the name home.tar.gz
and use
file
to confirm that it is a compressed file:
home.tar.gz: gzip compressed data, from Unix, last modified: ...
The archive should contain everything under the
/home
directory.
tar
index should list over 130 pathnames, including almost
100 directories created by the newusers
command in a previous
assignment./home
is empty directories or duplicate files.(You will check the validity of the home.tar.gz
file at the
end of this task when you move it into your Base
Directory.)
You will copy the files in /home
to the new partition. First
record the file names so you can make sure the copy works:
home_before.txt
Record a recursive, sorted listing of all of the pathnames of your
/home
directory using find /home | sort
and redirecting the output to a file named home_before.txt
in
root
’s home directory. Your file should
contain more than 130 lines, one for each pathname in the /home
directory.
Ensure no processes are using the /home
directory or any
files under it, with lsof +D /home
/home
is not yet a mount point, you need to use the
+D
option to include every directory under
/home
./home
directory./home
./home
, you probably forgot
to exit all remote sessions before going into single-user mode. Kill all the
processes that are using /home
.lsof
command isn’t found, you missed an earlier
step. Exit single-user to multi-user, install the package, and return to
single-user.)You must finish correctly the remaining steps in this section before you reboot, or your sysadmin account will be missing its HOME directory and you will get an error message about that when you log in. You must completely finish the remaining steps in this section correctly to regain log-in access to your sysadmin files in your HOME directory. Do not shut down or reboot your machine in this section, since the reboot will cause the
/home
directory to unmount and all your HOME directories, including the one for your sysadmin account, won’t work. You might want to take another snapshot here before you continue.You can safely use VMware to PAUSE or SUSPEND your CentOS VM in the middle of this work, just don’t shut it down and reboot until you finish this section.
/old_home
lsof
that nothing is using the
/home
directory. Rename the existing /home
directory to /old_home
/home
has been renamed and is therefore missing./home
Re-create a new empty /home
directory that will be used
as a mount point for the new filesystem you just created, above.
Mount onto the empty /home
directory the new 500 MB
ext4
filesystem that you created earlier. (Review the
mount
command in CST8207
Partitions and File Systems.)
EXT4-fs (sdb1): mounted filesystem...
.Run the mount
command and confirm that you can see
/dev/sdb1
mounted on the /home
directory. Do not
proceed until this is true:
/dev/sdb1 on /home type ext4 (rw)
The df -h
command will also show
/dev/sdb1
mounted on /home
, with approximately
these sizes:
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 477M 2.3M 449M 1% /home
Verify that there is a lost+found
directory under
/home
now, because /home
is now a file system mount
point instead of just a plain directory. Do not accidentally delete this
directory, or else the system won’t have a place to put orphan files! (If you
delete it, read the man page for the mklost+found(8)
command and
recreate it.)
Use the copy command with the archive option to copy the contents of
the old /old_home
directory to the new 500 MB /home
filesystem.
/old_home
into /home
and do not copy the name /old_home
in to
/home
!/home
and confirm that you do
not see the old_home
directory name there.lost+found
directory inside
/home
.home_after.txt
/home
again, in the same
way, sorted, except redirect the output into a new file
home_after.txt
also in root
’s
home directory.home_diff.txt
Record the differences between the two
home_{before,after}.txt
files, in a file named
home_diff.txt
also in root
’s
home directory. (The files should differ by exactly one line; the new HOME
directory has one additional directory in it that wasn’t in the original.
We’ve already told you what its name is.)
Add a record to the system fstab
file so the new
/home
filesystem is mounted automatically, with default options
and added quota options for both users (usrquota
) and groups
(grpquota
).
fstab
.1
) for the fifth field (dump or archive flag).2
) for the sixth field (fsck
pass
number). (RTFM for fstab
and note that using pass number
1
is reserved for the ROOT file system.)Use the mount
command with the remount
option to remount the /home
filesystem. The remount will use the
new options given in fstab
man mount
and look for the remount
section
under the -o
flag (options).mount
command doesn’t read the new quota options from
fstab
, then you are specifying both the device and the mount
point to the mount
command, which means it won’t read the file
to get the new options. Don’t do it that way.mount
command has other errors, do not continue. Fix
it!Use the mount
command to verify that /home
is now remounted with the two quota options that you set in
fstab
:
/dev/sdb1 on /home type ext4 (rw,usrquota,grpquota)
Verify that your sysadmin HOME directory is now valid:
su
with the --login
option to log in to
your sysadmin account. If this fails (home directory not found), restore from
a snapshot and try again./home
.root
, do not continue. Fix it!root
shell.In your single-user root
shell, unmount
/home
and then mount it again, relying on the fstab
to provide the device name and file system options:
# umount /home ; mount /home ; mount
You should see no errors, and mount
should show
/home
mounted with the same two quota options again.
At this point you have verified that the new /home
directory
is working. Your system could be safely shut down and rebooted, but let’s
clean up first.
Return from single-user to runlevel 3 by typing exit
at
the single-user shell. The system will boot multi-user into the default run
level.
Log in using your account (you may use SSH again) and verify that you are in runlevel 3 by running the command that displays the system’s runlevel.
Normally, you would remove the /old_home
directory, and
everything beneath it to free up space on the /
filesystem,
reaping the rewards of moving the /home
directory to its own
filesystem; however, leave the /old_home
directory in place for
marking purposes. Do not remove
/old_home
.
Move into your sysadmin Base directory the
tarball and all of the *.txt
files you created in the
home directory for root
, and then change the owner and group of
those files from root
to yourself (your sysadmin user whose name
is of the form abcd0001
).
sudo
. Why? (Hint: Who is running the shell that is doing
the GLOB expansion before executing sudo
?)Run the Fetch and Checking Program to verify your work so far.
rsync
The rsync
command is an intelligent form of
copy command that only transfers data if the data isn’t
already there. You will practice using rsync
between your CentOS
VM and its loop-back network adapter, which we will call the Backup machine
(even though it’s really the same machine). A trivial change to the remote
host name lets you transfer files to any machine on the Internet that lets
you run rsync
.
The modern
rsync
command uses an underlying SSH protocol to actually transfer the data, so any configuration you have done for SSH (such as private keys, host aliases, or SSH agents) applies torsync
as well.
Take a snapshot of your CentOS VM. You can never have too many snapshots.
Log in to CentOS as your sysadmin account.
/mnt/disk02
ext4
file system on the first logical partition of
your second disk and create a system fstab
entry that mounts it
on the new directory /mnt/disk02
(that you will have to create).
noatime
in the
fstab
entry.disk02
file system and then check the mount to
make sure it has the noatime
option listed:
type ext4 (rw,noatime)
/mnt/disk02/backup
backup
with these options:
Backup Account
/mnt/disk02/backup
backup
account a really short password (because
you will be typing it a lot in this assignment).
Tip: The root
user can give an account a
“too short” password if you persist by typing it multiple times:
$ sudo passwd backup
Changing password for user backup.
New password:
BAD PASSWORD: it is WAY too short
BAD PASSWORD: is a palindrome
Retype new password:
passwd: all authentication tokens updated successfully.
backup
account:
ssh backup@localhost id
UID_MIN
and GID_MIN
in
/etc/login.defs
ssh backup@localhost pwd
/mnt/disk02/backup
ssh backup@localhost df .
backup
account HOME directory
is on the first logical partition of your second disk, which is mounted on
/mnt/disk02
Mounted on
you must see /mnt/disk02
and
if not, go back a few steps and mount it again.Have you taken a snapshot recently?
Install the rsync
package.
Make sure this works without error: ssh backup@localhost df .
backup
account HOME directory
is on the first logical partition of your second disk, which is mounted on
/mnt/disk02
Mounted on
you must see /mnt/disk02
and
if not, go back a few steps and mount it again.As a simple test, use rsync
to transfer a single file to
the Backup machine using the standard three archive,
verbose, and hard-links rsync
options as follows:
$ date >foo
$ rsync -avH foo backup@localhost:
Note the trailing colon (:
) character after the host name
in the destination pathname! Since nothing follows the colon, the same file
name will be used in the HOME directory on the remote machine. (If you forget
the colon, rsync
will create a local file with that name; this
is not what we want.)
The three options -avH
are standard sysadmin use for this
command and are almost always used, just as sysadmin must always use the
-p
option to both cp
and scp
to
preserve modes and times.
You should see:
sent 121 bytes received 31 bytes
Confirm that the HOME directory of the backup
account now
contains an exact copy of file foo
backup
account is not
under the usual /home
directory. Look in the right place.Confirm that the foo
file in the backup
account has exactly the same time and date as the one in your own
account:
$ sudo diff foo /mnt/disk02/backup/foo
$ sudo ls -l foo /mnt/disk02/backup/foo
The output of ls
must show identical sizes, times, and dates.
See your instructor if you can’t get this one-file transfer working.
Repeat the exact same rsync
command with the same
foo
file.
sent 49 bytes received 12 bytes
rsync
protocol.Touch foo
and repeat the same rsync
again.
sent 92 bytes received 37 bytes
Redirect a new date
into foo
and repeat the
same rsync
again.
sent 121 bytes received 31 bytes
Remove the foo
file and reverse the rsync
to
restore a local copy from the remote Backup machine:
$ rm foo
$ rsync -avH backup@localhost:foo .
:
) character after the host name in
the source pathname, followed by a relative pathname!.
) at the end to copy into the current
directory as a destination pathname! The same file name will be used in the
current directory.-avH
are standard sysadmin use for this
command and are almost always used, just as you must always use the
-p
option to both cp
and scp
to
preserve modes and times.sent 30 bytes received 122 bytes
foo
is restored into the current
directory.The
rsync
command only does the least amount of work needed to make the remote file or directory the same as the local one (or vice-versa).
This section is optional. Typing backup@localhost
is too much
work. We can shorten that.
.ssh
in your sysadmin account
HOME directory and remove all permissions for group or other..ssh/config
Put the following four lines into file config
in the
above .ssh
directory:
Host backup back bk b
Hostname localhost
HostKeyAlias localhost
User backup
Remove all permissions for group or other from the config
file.
From your sysadmin account, try out the new SSH Host
alias using any ssh
command:
$ ssh backup id
CentOS release 6.6 (Final)
CST8177-15W idallen@idallen.ca
backup@localhost's password:
uid=497(backup) gid=497(backup) groups=497(backup)
You can use the shorter aliases, too. Try any of these:
$ ssh backup id
$ ssh back id
$ ssh bk id
$ ssh b id
Now try these; all should work using the above SSH Host
aliases:
$ rsync -avH foo backup:
$ rsync -avH foo back:
$ rsync -avH foo bk:
$ rsync -avH foo b:
$ rsync -avH b:foo .
You can now use the short SSH host and user alias b:
instead
of typing backup@localhost:
as either a source or destination
host name.
rsync
with the standard three sysadmin options to send
your entire HOME directory to the Backup machine under remote directory
test1
.
Always use the relative path on the remote machine.
Use rsync
with the added dry-run option
so that you can see what pathnames are being copied. When the pathnames look
correct (see below), remove the dry-run option.
You may use the optional SSH host alias b:
as part of the
destination pathname, if you created it above, otherwise you need to use the
full backup@localhost:
name.
As noted in your rsync
course notes
(12-sshkeys_yum_rsync.pdf
), be careful how you specify the
source pathname for your HOME directory. You must ensure that every local
file /home/abcd0001/
foo
transfers to the Backup machine HOME directory
as test1/
foo
and not as abcd0001/test1/
foo
. If rsync
displays pathnames that
begin with your userid, such as this:
sending incremental file list
created directory test1
abcd0001/
abcd0001/.bash_history
[...etc...]
then your source pathname is NOT correct. If you are not using
the dry-run option, you have to remove the abcd0001
directory from the backup
account and
re-read your rsync
course notes
(12-sshkeys_yum_rsync.pdf
). The pathnames transferred should
look similar to this:
sending incremental file list
created directory test1
./
.bash_history
[...etc...]
Only when the pathnames look correct should you remove the
rsync
dry-run option and actually transfer the
files.
Make sure there is no extra abcd0001
directory
under the test1
directory in the backup
account
HOME directory.
Compare a local file and a backed-up file to make sure they are the same, including the time and date:
$ pwd ; echo ~backup
/home/abcd0001/CST8177-15W/Assignments/assignment08
/mnt/disk02/backup
$ sudo diff do.sh ~backup/test1/CST8177-15W/Assignments/assignment08/do.sh
$ sudo ls -l do.sh ~backup/test1/CST8177-15W/Assignments/assignment08/do.sh
Make sure the files are exactly the same. The output of ls
must show identical sizes, times, and dates.
rsync_home_test1.txt
Put the exact rsync
command line you used above into file
rsync_home_test1.txt
in your sysadmin Base
Directory.
Test that you can restore an existing file from the Backup machine to
the /tmp
directory on the local machine. Compare the
tmp
copy to the original file. The two files should be exactly
the same, including the time and date:
$ rsync -avH backup@localhost:test1/CST8177-15W/Assignments/assignment08/do.sh /tmp/foo
$ diff /tmp/foo ~/CST8177-15W/Assignments/assignment08/do.sh
$ ls -l /tmp/foo ~/CST8177-15W/Assignments/assignment08/do.sh
b:
in the source
pathname, if you created it above.assignment08/do.sh
rsync
says failed: No such file or directory
then verify that the file exists where you think it should be under the
test1
directory in the backup
HOME directory.ls
must show identical sizes, times, and dates.Repeat the exact same rsync
command to the
test1
directory that you did in Step 1 above and that you saved
in the rsync_home_test1.txt
file:
sh rsync_home_test1.txt
.bash_history
file and the new
rsync_home_test1.txt
file should be the only files that have
changed.)speedup is
line at the bottom of the
rsync
verbose output that tells you how much faster it was to
compare files and not have to transfer any of the files that were already
there.Change to your sysadmin Base Directory. (Perhaps you are already there?)
In your sysadmin Base Directory, touch
your existing rsync_home_test1.txt
file and repeat the full HOME
directory backup again.
rsync_base_test1.txt
In your sysadmin Base Directory copy
rsync_home_test1.txt
to the new file
rsync_base_test1.txt
Use a new rsync
command line with the
dry-run option to attempt to update just the current
sysadmin Base
Directory (not your whole HOME directory) to the corresponding
remote sysadmin directory on the Backup machine.
Always use the relative path on the remote machine.
The source pathname to rsync
must be simply
.
(the current, sysadmin Base Directory) not
your HOME directory.
If you get the command correct, rsync
will propose to
update only one single file to the remote machine – the new
rsync_base_test1.txt
file.
sending incremental file list
./
rsync_base_test1.txt
sent 463 bytes received 18 bytes 962.00 bytes/sec
total size is 28770 speedup is 59.81 (DRY RUN)
If rsync
proposes to transfer all the pathnames in the
current directory, then you have the destination directory wrong.
When rsync
with the dry-run option says
only one file will be updated from this sysadmin Base Directory to the
remote sysadmin directory, remove the dry-run option and
update the Backup machine with the current directory.
rsync_base_test1.txt
Put the exact rsync
command line you used into file
rsync_base_test1.txt
in your sysadmin Base
Directory.
Remove just the one file rsync_base_test1.txt
from the
remote sysadmin Base
Directory as backed up under the test1
directory in the
Backup account, like this:
$ sudo rm ~backup/test1/CST8177-15W/Assignments/assignment08/rsync_base_test1.txt
If you get an error message, you have the pathname wrong or else you
didn’t do the previous rsync
correctly to back up the
file.
From your sysadmin Base Directory repeat
the exact same base-directory-only rsync
command that you saved
in the rsync_base_test1.txt
file: sh rsync_base_test1.txt
rsync_base_test1.txt
Since rsync
can transfer a lot of files in a very short time,
always do a dry-run rsync
before doing the real
thing, just to make sure that you have the pathnames correct! As it says in
the rsync
course notes (12-sshkeys_yum_rsync.pdf
),
the source pathname syntaxes foo
and foo/.
are NOT the same, and it’s usually foo/.
that you want to use as a source pathname! Always use
the dry-run option first!
Run the Fetch and Checking Program to verify your work so far.
rsync
to other machinesYou can probably see that using rsync
to send files to
another machine is simply a matter of choosing the remote userid and machine
name for the SSH login:
$ date >foo
$ rsync -avH foo backup@localhost:
$ rsync -avH foo abcd0001@cst8177.idallen.ca:
Of course, you need an SSH account on the remote machine, and
rsync
(and the underlying SSH protocol) must be installed
there.
Tip: You might choose to back up your CentOS sysadmin account HOME directory to a backup directory in your account on the CLS every now and then.
Warning: An incorrect use of rsync
to the
CLS can overwrite important files on the CLS. Some rsync
options
may even delete files. Always use the dry-run option to see
what rsync
proposes before actually letting rsync
do the transfer.
If you find yourself locked out of a Linux machine, and you have access to the console, booting into single user mode will will often not require a password, and in single-user mode you can change passwords or perform various other repair tasks. (Some systems do password-protect single-user mode, in which case you would need to boot a “live” or “rescue” CD to reset your
root
password.)
Use the correct command to reboot your CentOS VM, and when you see the
GNU GRUB
menu and the 30-second countdown timer, halt the GRUB
countdown by pressing the space bar or an arrow key.
Use GRUB to edit your boot options so you boot into single user mode. (Refer to last term’s CST8207 Booting and GRUB.)
Verify that you are in single user mode: when you issue the command
runlevel
, the output should be either N S
or unknown
Note that you are running as root
and can change the
password of any user in single-user mode, including the root
password.
ps_auxww.txt
Put the output of the command ps auxww
into a file named
ps_auxww.txt
in your sysadmin Base Directory, and
change the ownership and group of this file to your ordinary sysadmin user.
(Don’t leave root
-owned files in ordinary
user accounts!)
Exit this single-user shell, which will allow the system to boot into the default runlevel.
Log in (using SSH if possible) and verify you’re in the default
runlevel by issuing the runlevel
command.
S 3
Run the Fetch and Checking Program to verify your work so far.
If you find a Linux machine is unbootable, and you have console access, you may be able to rescue it by booting the machine from a “Live CD”. You will use the CentOS installation DVD to boot into “rescue” mode, which is a “Live CD” mode.
Shut down or power down your CentOS VM gracefully using the proper command.
Attach the CentOS Installation ISO image file to your VMware virtual DVD drive, connect it, and make sure it will be connected at Power On. (You did exactly this when you first installed CentOS.)
Access the VMware Settings for your Virtual Machine and increase the RAM to at least 1024MB. (The installer and Rescue mode needs more RAM for the graphics than the server-style CentOS machine.)
Boot into the VMware BIOS of your virtual machine, as follows:
*.vmx
file.*.vmx
file to give a longer
pause on the VMware BIOS screen: bios.bootDelay = "60000"
In the VMware BIOS menu, use the keyboard to change the “Boot” settings so that the CD/DVD drive is before the hard disk in the boot order, if it isn’t already.
Save and Exit the VMware BIOS. The machine should boot from the virtual CD/DVD drive that contains the CentOS installation ISO image file. You should see the familiar Welcome to CentOS installation menu and a 60-second countdown timer:
If you boot into your regular CentOS on your hard disk, then you didn’t set up either the Boot menu or the CD/DVD device correctly. Wait until the machine has finished booting, log in, shut it down and try again.
On the Welcome to CentOS screen, use the arrow keys
to select the item Rescue installed system
. Boot it by pressing
Enter.
Follow the instructions on the screen, choosing the defaults for everything except for networking. Do not enable networking.
You do not need to enable networking – choose No
Use Continue
(not Read-Only
) for
your Linux Rescue installation, since we need to write on
the file system.
If it says it can’t mount your disk under /mnt/sysimage
,
see the Rescue CD Appendix I.
When it says Your system has been mounted under
write
down the directory under /mnt
that will be used to mount and
access your Linux installation.
At the three-item menu that starts with
shell Start shell
, choose the first item (start a shell). It
will give you a root
shell prompt in a black console screen of
the Rescue system.
When you finally have a bash
root
prompt,
type hostname
and then cat
the password file to see
that this is not your own CentOS system running. It is the Rescue system,
with its own Rescue machine name and Rescue password file.
Running df
in this Rescue CD will confirm that your
CentOS ROOT partition /dev/sda1
is now mounted on directory
/mnt/sysimage
and your CentOS HOME partition
/dev/sdb1
is mounted on directory
/mnt/sysimage/home
/mnt/sysimage
file
systems.livecd_df.txt
df
(all 10 lines) to the file
livecd_df.txt
in your sysadmin Base Directory in your
mounted CentOS system.
/home
when mounted on the Rescue CD! Read all
the words above.)ls -l
on all the HOME directories in the
HOME partition (which is NOT currently mounted under
/home
) and note that all the accounts have numeric owners and
groups.
ls -l
.livecd_passwd.txt
livecd_passwd.txt
in your
sysadmin Base
Directory in your mounted CentOS system.
/home
when mounted on the Rescue CD! Read all
the words above.)sum
of your livecd_passwd.txt
file should
be 63933 2
root
.
chown: invalid user: '
abcd0001
'
chown
fail when
run from the Rescue CD?invalid
(doesn’t exist) in the
Rescue CD environment?chroot /mnt/sysimage
root
shell running with
/mnt/sysimage
(your hard disk CentOS ROOT) as its ROOT
directory, instead of the Rescue CD ROOT.chroot
shell, the
/mnt/sysimage
directory will used as be actual ROOT directory
named /
.cat
the password file, you will see the
password file relative to the new chroot
ROOT directory, which
is your CentOS ROOT directory, so you see your CentOS password file, not the
LiveCD password file.df
command now and note the familiar list of file
systems, with sda1
mounted on the ROOT and sdb1
mounted on /home
, etc.
chroot
command hides the /mnt/sysimage
mount point and makes it look like the real ROOT directory as long as we stay
in this chroot
shell.All programs you run from this
chroot
shell will behave as if they used your CentOS file system as the ROOT. The file name/etc/passwd
now refers to your CentOS password file, not the Rescue CD password file.You could fix a broken MBR with the command
grub-install
at this point, or do any other repairs to your CentOS Linux file system.
In the chroot
shell you are running, pathnames work as if
your CentOS machine were running. Everything works as expected.
Confirm that you can now see your two livecd_*.txt
files
in your usual CentOS sysadmin Base Directory using
its usual path with respect to the usual CentOS ROOT directory:
# cd ~abcd0001/CST8177-15W/Assignments/assignment08
# ls -l livecd_passwd.txt livecd_df.txt
(Always use your own sysadmin userid in these assignments, never the
place-holder userid abcd0001
.) Note that the above
files in your account are still owned by root
.
Change the owner and group of the livecd_*.txt
files to
your system admin account user name. The command will succeed this time.
chown
succeed in
the chroot
shell but fail in the Rescue CD shell before using
chroot
?Exit the chroot
shell. You will return back to the Rescue
CD shell and prompt.
Run the df
command again in this Rescue CD shell and note
how everything again appears mounted under
/mnt/sysimage
At the Rescue CD root
shell prompt, try to use the usual
command to shut down and halt the machine safely. Do not use VMware
Power Off!
shutdown: Unable to shutdown system
root
shell and return to the three-itme
Rescue CD text menu.reboot Reboot
from
the menu.Welcome to CentOS 6.6
screen, use the VM -> Power
-> Power Off menu to power off the system without starting
CentOS.
With the machine powered off:
256MB
to make live snapshots quick and small.Power on your CentOS VM. You should see a familiar
GNU GRUB
menu from your hard disk CentOS installation.
Boot from local drive
.When your CentOS has rebooted, log back in as your system admin account (using SSH if possible, since it’s nicer than the console).
Change the ownership and group to you of any remaining
root
owner or group files anywhere in your CentOS system admin
account. (If you’ve done your work carefully, there should be nothing owned
by root
.)
root
-owned files as an indication that someone has
broken into the system. Don’t leave root
-owned files in your own CentOS sysadmin
account.root
files in your CLS assignment directories. This is
intentional: don’t delete these!Run the Fetch and Checking Program to verify your work so far.
That is all the tasks you need to do.
Log in to the CLS and submit your mark from the CLS following the Checking Program instructions below.
Read your CLS Linux EMail and remove any messages that may be waiting. See Reading eMail for help.
Optional: Keeping your main configuration snapshots, remove any intermediate snapshots you no longer require, to free up disk space. - Be careful not to remove your current work!
See CentOS Remote Checking, Marking, and Submitting your Work.
Use this Appendix if the Rescue CD tells you it can’t
mount your system under /mnt/sysimage
and tells you to do it
manually. We will also mount some useful /dev
directories) so
that chroot
works.
First, use the Rescue CD menus to get to a
root
shell prompt.
At the Rescue CD root
shell prompt, mount
these five file systems:
# mount /dev/sda1 /mnt/sysimage
# mount -o bind /dev /mnt/sysimage/dev
# mount -t tmpfs /dev/tmpfs /mnt/sysimage/dev/shm
# mount /dev/sdb1 /mnt/sysimage/home
# mount /dev/sdb5 /mnt/sysimage/mnt/disk02
The output of df
should now be 9 lines, with five lines
including the above five /mnt/sysimage
file systems.