3.1 The Source DirectoryIndex

All references to the “Source Directory” below are to the CLS directory ~idallen/cst8207/13f/assignment03/ and that name starts with a tilde character ~ followed by a userid with no intervening slash. The leading tilde indicates to the shell that the pathname starts with the HOME directory of the account idallen (seven letters).

4.1 Worksheet and PromptIndex

These worksheets prepare you to do the numbered tasks listed below by executing commands via Remote Login to the Course Linux Server.

You can download Libre Office (or Open Office) for Windows to edit the Worksheet *.odt files and save your answers. (Or you can print the PDF and write your answers on paper.) Do not use MSWord.

Record and save all your worksheet answers for study and quizzes!

See Assignment #02 for how best to fill in the worksheets.

1. Do the worksheets Lab Worksheet #04 ODT and Lab Worksheet #05 ODT before this assignment. Record and save all your worksheet answers for study and quizzes!

2. Set your prompt every time you log in to the CLS, so that the prompt changes to tell you you the basename of your current working directory, as you did the worksheets and the previous assignment. Hint: You can use your saved shell history to set the prompt very quickly at login time, as demonstrated in lectures.

4.2 Working with directories and filesIndex

3. Create the following directory structure in your CLS HOME directory and record (for study purposes) the series of Unix commands you used to create it. Spelling and capitalization must be exactly as shown:

   CST8207-13F
   `-- Assignments
       `-- assignment03
           `-- top1
               |-- dir1
               |   `-- dir1-1
               `-- dir2
                   `-- dir2-1

   Note: You can create the entire directory tree above using one single command with one option and two pathname arguments. System administrators like to work efficiently – they learn how to do things quickly.

4. Make a directory named warez under your assignment03 directory. You will put files here later.

5. Make a directory named osfiles under your assignment03 directory. You will put files here later.

6. Make the top1 directory your current directory and then use a command to recursively generate a list of all pathnames under your current directory. The recursive output of all pathnames under your current directory will be exactly five lines long and will contain this exact line in the output somewhere (along with another four lines):

   ./dir2/dir2-1

   Note that there is no punctuation at the end of any pathname.

7. When the recursive output above on your screen looks correct, redirect the five lines of output of the command into the file named top1find.txt in the assignment03 directory (not in the current directory). The word count of this file must be: 5 5 44. If the word count is wrong, read all the words in the previous step.

8. RTFM and find the single-letter option to the copy program that preserves mode, ownership, and timestamps when you copy.

9. Under assignment03 create another directory top2 that contains a copy of everything you created under top1, with the timestamps preserved. Hint: The fastest way to do this is with a single recursive copy command, as you did in Lab Worksheet #03 HTML, but add the option to preserve timestamps.

10. Create an empty file named MT in each of the five directories from top2 down. (You can do this with one command name and five relative pathnames.)

11. Make the top2 directory your current directory and then use a command to recursively generate a list of all pathnames under your current directory. The recursive output of all pathnames under your current directory will be exactly ten lines long and will contain this exact line in the output somewhere (along with another nine lines):

    ./dir2/dir2-1/MT

    Note that there is no punctuation at the end of any pathname.

12. When the recursive output above on your screen looks correct, redirect the ten lines of output of the command into the file named top2find.txt in the assignment03 directory. The word count of this file must be: 10 10 103. If the word count is wrong, read all the words in the previous step.

13. Under assignment03 create another directory top3 that contains a copy of everything that you created under top2, with timestamps preserved. Hint: See the previous Hint!

14. Under top3 rename each of the MT files to have the number of the parent directory that contains it, e.g. rename ./MT to be ./MT3 and dir2-1/MT to be dir2-1/MT2-1, etc. (You don’t know enough scripting yet to do this with a script; do each file manually.)

15. Make the top3 directory your current directory and then use a command to recursively generate a list of all pathnames under your current directory. The recursive output of all pathnames under your current directory will be exactly ten lines long and will contain this exact line in the output somewhere (along with another nine lines):

    ./dir2/dir2-1/MT2-1

16. When the recursive output above on your screen looks correct, redirect the ten lines of output of the command into the file named top3find.txt in the assignment03 directory. The word count of this file must be: 10 10 112. If the word count is wrong, read all the words in the previous steps.

Run the Checking Program to verify your work so far.

4.3 Finding multiple files in a maze using a GLOB patternIndex

17. Under the Source Directory there is a name maze that is a symbolic link that leads to the same directory maze used in a previous assignment: Assignment #2.

    Use the absolute pathname of the maze name in the Source Directory as an argument to ls along with an option that shows the long information about the pathname. You probably have the right option if the first word of the output is lrwxrwxrwx, indicating that maze is a symbolic link. Save the output (one line) into file mazeinfo.txt under your assignment03 directory.

18. Again, you must find files in this maze, but the symbolic link requires some special handling because the command that finds files does not follow symbolic link arguments on the command line without using some options:

    1. You must choose one of these approaches to reach the maze:

       1. Use the maze pathname from the previous assignment, not the pathname in the Source Directory that is the symbolic link. OR
       2. Use an option to the finding command that makes it follow symbolic links while processing the command line arguments. Hint: RTFM and do not use the -L option. OR
       3. Make the maze your current directory and then recursively search the current directory. (A current directory can never be a symbolic link – it must be a real directory.)

    2. Important: Look for the string quote in the course notes web page on Searching for and finding files by name, size, use, modify time, etc. Read the paragraphs containing this word and remember the importance of quoting. You will need this information for this next question:

    3. In this maze, use a single command (not a pipeline) to recursively find all pathnames with a basename that contains your eight-character userid anywhere in the name. For example, if your userid were abcd0001 then you might output pathnames containing basenames such as abcd0001, abcd0001YYY, XXXabcd0001, and XXXabcd0001YYY where XXX and YYY can be anything (zero or more characters). You must use a single command that finds files by using a pattern basename to find these pathnames. Do not try to use cd and ls to find the names in the maze; the maze is very big. (You have previously used this command many times without a pattern for a basename. You should find approximately 26 pathnames.)

    4. Modify the same single command you used to find the names above and add the option that makes the command show the full information about the names (including permissions, owner, size, date, etc.) instead of just the pathname. (You know this option from your answers in Worksheets 2 and 3 and from reading the More features section of the Searching for and finding files by name notes.) You will know you have the right option if the output of the command shows approximately 26 lines and 291 words.

    5. Pipe the pathname output of the above command (26 lines of output) into a sorting program and put the sorted output into file mazefound.txt under your assignment03 directory. The sorted file will still contain approximately 26 lines and 291 words.

Run the Checking Program to verify your work so far.

4.4 Using shell GLOB patterns to select namesIndex

19. In your HOME directory, create two symbolic links to the old and new course notes for CST8207 using the method described in Copies of the CST8207 Course Notes. (The old notes must be term 13w and the new notes must be term 13f in the pathnames you use.)

    In the notes, see the example use of grep and fgrep with GLOB patterns to match *.txt files in these old and new notes directories.

    These text-searching commands grep and fgrep are case-sensitive by default – searching for abc won’t find ABC unless you use an option to ignore case distinctions. (What option? RTFM)

20. In your HOME directory, use the echo command with a shell GLOB pattern to match all pathnames under oldnotes/ that end in .txt and display all the names on your screen. The shell will find 88 pathnames ending in .txt, and the echo command will display those 88 names on your screen.

    When the echo output on your screen is correct (88 names), redirect the output into file textfound.txt under your assignment03 directory. The file must contain 88 names on one line.

    Hints: Use a command to count the words to be sure you have 88. The example in the notes already uses the given GLOB pattern to generate pathnames for the fgrep command. Use echo not fgrep to display the pathnames on your screen and then redirect the names into the output file.

21. In your HOME directory, use the echo command with a shell GLOB pattern to match all pathnames under oldnotes/ that contain the word symbolic anywhere in the file name and display the names on your screen. The shell will find two pathnames, one ending in .html and the other in .txt, and the echo command will display those two names on your screen.

    When the echo output on your screen is correct (two names), redirect the output into file symfound.txt under your assignment03 directory. The file must contain two names on one line.

22. In your HOME directory, use the echo command with a shell GLOB pattern to match pathnames under oldnotes/ that contain the word vi anywhere in the file name and end in the extension .pdf at the end. The shell will find two pathnames, each ending in .pdf at the end, and the echo command will display those two names on your screen.

    When the echo output on your screen is correct (two names), change the command name from echo to ls and add an option to show the full, long information about the pathnames. You should see two long lines on your screen, showing the full file information for each of the two files.

    Now redirect the long output into file vifound.txt under your assignment03 directory. The file must contain two lines.

Run the Checking Program to verify your work so far.

4.5 Searching for text inside files (e.g. course notes)Index

As mentioned in Lab Worksheet #03 HTML, choose which text search command you use depending on whether special characters are being used in the search string.

Always verify that the correct output appears on your screen before you redirect the output into a file. You can only redirect what you can see.

23. Search for lines containing your login userid in the password file and redirect the output into file mypasswd.txt under assignment03. You should find exactly one line.

24. Search for lines containing the special character asterisk (*) in the file bash.bashrc in the Source Directory and redirect the output into file star.txt under assignment03. The word count of the output (and the file) should be: 2 6 36.

25. In your HOME directory, create two symbolic links to the old and new course notes for CST8207 using the method described in Copies of the CST8207 Course Notes, unless you have already created these links earlier in this assignment.

26. In your HOME directory, use the fgrep command with a shell GLOB pattern to match all pathnames under oldnotes/ that end in .txt and search for the word Filezilla in those files. One line of text from inside one of the files will display.

27. Repeat the above, but add the fgrep option that ignores case distinctions. Now, nine lines are found in three different files.

    Redirect the the nine lines of output into a file named filezilla.txt under your assignment03 directory. The word count of the output (and the file contents) should be: 9 72 816.

Run the Checking Program to verify your work so far.

4.6 The cracker WAREZ 100 filesIndex

The “story” here is that a cracker has hidden a bunch of WAREZ files in a directory on the server. (See https://en.wikipedia.org/wiki/Warez.) Your job is to take a copy of these files, and only these files, for use in a court case. You must not touch or copy any other files, only the WAREZ files.

28. Under the Source Directory there is a directory named start. Hidden deeper under this directory is one single directory containing about 158,730 files. Be careful about typing ls in this directory without using any output pagination pipe – the amount of output may flood your terminal window for some time and even ^C may take a minute or two to interrupt the command! Make this (huge) directory your current directory.

    Exactly 100 files in this one (huge) directory have names that contain your userid (lower-case) followed somewhere later by the string warez, where warez may appear in any combination of upper- and lower-case letters, e.g. warez,Warez,wArez,waREz, etc. Any amount of text may appear before your userid, between your userid and the warez, and after the warez.

    Some sample file names for userid abcd0001 might look like these (note that the warez word must always follow the userid in all the required file names):

    • HhUtfgYtyGhjJADGekCAkgtZEKsTGKdYZZabcd0001ADGekCwaREZZaFSrXJnxGex
    • zynabcd0001uKVUFOsCXaGFWZPECbYWVFKzynuKWaREZv

    Using one single copy command and a GLOB expression, copy all 100 (exactly 100) of these cracker files (and no others) into your own directory named warez that you created above. Make sure you preserve the modify times of the copied files, as you did in a previous lab. (In this simulation, all the files are empty.)

    Hints: Use a shell GLOB pattern to match the right file names. Use echo and word count to see if your file names are correct before you try to use the copy command. Do not use a pipe or find to select the file names. The shell can do it all with one copy command using the right GLOB pattern.

    When you succeed, you can check you work using:

    $ \ls warez | wc

    which should show a count of 100 lines. (The backslash before ls turns off any alias you might have defined for ls that would affect the output.) All the files should have their original modify dates preserved – verify this.

4.7 Three different O/S Text FilesIndex

29. Somewhere under that same start directory are exactly three non-empty files whose names contain your userid (lower-case) somewhere (anywhere) in the name.

    Find these three non-empty (size larger than zero) files and copy them (preserving modify times) to the osfiles directory that you created above. You will find your userid mentioned inside each file, but because the files are not all Unix/Linux text files, some of the text content may not display correctly on your terminal screen.

    Hints: What command finds files based on size and a basename that can be a GLOB-style pattern? You have used this command many times this term. See the end of Worksheet #2 HTML. The less command is better than cat when displaying files containing strange (e.g. unprintable) characters.

    Since there are only three file names, you can use your mouse to copy-and-paste the three file names you need to copy. You can also read this optional material on a better way to Use find -exec and xargs (optional).

30. In your osfiles directory, determine which operating system created each of the three non-empty files. Rename the Unix/Linux file to be unix, the Windows file to be windows and the Macintosh file to be macintosh.

    Hints: In Assignment #1 you used a command that can determine file type to identify the text inside a file.txt created with Windows Notepad. You will also find this command listed under Week 01 in the List of Commands in your notebook. Use this command and the notes on Text File Line End Differences to identify the special line endings of the Windows and Macintosh files.

Your instructor will mark the assignment03 directory in your account on the due date. Leave them there on the CLS. Do not delete anything.

Run the Checking Program to verify your work so far.

4.8 Appending to filesIndex

31. Count the lines, words, and characters in the file services under the /etc directory and put the count in file wc.txt under your assignment03 directory. (Use the absolute pathname of the services file when you count and do not use any pipes.) The file wc.txt should contain one line.

32. Extract just the first line of the same services file and append this one line to the end of the wc.txt file, so that the file wc.txt now has two lines in it (the word count and the first line of services). Hint: You know a command that shows lines at the start of a file.

33. Append the count of the lines, words, and characters in the file protocols under the /etc directory to the end of file wc.txt, so that the wc.txt file now has three lines in it. (Use the absolute pathname of the protocols file when you count and do not use any pipes.)

34. Extract just the last line of the same protocols file and append this one line to the end of the wc.txt file, so that the file wc.txt now has four lines in it. Hint: You know a command that shows lines at the end of a file.

    Confirm that the word count of the file gives 4 20 140. If you see the right number of lines but the other values differ, go back and re-read the parenthesized sentences that start with “Use the”.

Run the Checking Program to verify your work so far.

That is all the tasks you need to do. Verify your word counts and then mark your work: