% CST8207 Week 04 Notes -- I/O redirection, pipes, stdout, stderr, `2>&1`, awk fields $1 $NF % Ian! D. Allen -- -- [www.idallen.com] % Winter 2019 - January to April 2019 - Updated 2019-02-04 14:08 EST - [Course Home Page] - [Course Outline] - [All Weeks] - [Plain Text] Readings, Assignments, Labs, Tests, and ToDo ============================================ Read (at least) these things (All The Words) -------------------------------------------- 1. [Week 04 Notes HTML] -- this file -- **Read All The Words** 2. [Practice Tests and Answers] -- practice questions for your midterm test. 3. [Shell I/O Redirection] -- stdin, stdout, stderr, and Pipes 4. [Selecting Fields with `awk`] -- extracting fields from input lines and columns from input files using `awk` and `$1` and `$NF` 5. [The VI (VIM) Text Editor] -- how to edit files. Don't use `nano`! 6. Next week: [Shell Variables] -- quoting, local and environment variables, `USER HOME SHELL PATH TERM $$` 7. [List of Commands] -- Command names you should know, listed by week 8. [Video Tutorials on Lynda.com] -- tagged by week number 9. [Linux and Sysadmin News in the World] -- Linux skills earn $100,000 salaries. - Follow last week's instructions to create your lynda.com account, then watch these sections of [Unix for Mac OS X Users] - these videos are for *all* Unix/Linux students, not just Mac OSX users - **3. Working with Files and Directories** - Searching for files and directories 6m 32s - `find` using "wildcard" (GLOB) patterns - same as shell GLOB - Unix text editors 6m 39s - `vi/vim, nano` - `head, tail` - Naming files 5m 41s - use quotes to surround names with blanks or special characters - **6. Directing Input and Output 20m 39s** - Standard input and standard output 1m 24s - Directing output to a file 4m 13s - Appending to a file 2m 44s - Directing input from a file 5m 28s - Piping output to input 4m 40s - Suppressing output 2m 10s Assignments this week --------------------- Check the due date for each assignment and put a reminder in your agenda, calendar, and digital assistant. Just like in the Real World, not all due dates are on the same days or at the same times. - Review last week. Did you do everything assigned last week? - Read All The Words, Do, and then Submit via Brightspace: - [Assignment #02 HTML] -- Course Linux Server simple commands - [Assignment #03 HTML] -- working with files and pathnames: `PS1 find mkdir mv cp rm touch file` - Remember to set your `PS1` prompt every time you log in to the CLS. - This assignment does *not* use shell GLOB patterns or pipes. - See the list of **Prerequisites** at the start. - OPTIONAL: Read [The VI (VIM) Text Editor] and optionally do the bonus [Assignment #04 HTML] for extra credit. - Coming soon: [Assignment #05 HTML] -- using GLOB wildcard patterns, selecting files using GLOB, redirection and pipes. This assignment requires reading [Shell GLOB patterns] and [Shell I/O Redirection] and then doing [Worksheet #04 HTML] and [Worksheet #05 HTML]. - Really do **Read All The Words**. You don't get a second chance to get it right. Lab work this week ------------------ - Finish the readings, worksheets, and [Assignment #03 HTML] from last week. - GLOB patterns and pipes are *NOT* needed in Assignment 3. See Assignment 5. - Study the first [Practice Tests and Answers] for your upcoming Midterm Test #1 in your one-hour lecture class in Week 6. - Complete and save the ODT or PDF versions of [Worksheet #04 HTML] and [Worksheet #05 HTML] so you can do [Assignment #05 HTML] -- GLOB wildcard patterns, finding files using GLOB, redirection and pipes - These worksheets require you to understand [Finding Files], [Shell GLOB patterns] (wildcard pathname matching) and [Shell I/O Redirection] (including Pipes). - Coming soon: [Assignment #05 HTML] -- GLOB wildcard patterns, selecting files using GLOB, redirection and pipes - This assignment requires you to understand the material in the web pages and both Worksheets 4 and 5. - OPTIONAL BUT HIGHLY RECOMMENDED: Read [The VI (VIM) Text Editor] and optionally do the bonus [Assignment #04 HTML]. Real Linux sysadmin all know how to use this text editor. ### Worksheets Worksheets are preparation for your assignments. You can't do the assignments without having done the worksheets first, and you can't do the worksheets without having first read the Course Notes: 1. Read the web notes. (Please: **Read All The Words**) 2. Do the relevant Worksheet(s). 3. Do the relevant Assignment(s). Make notes from the worksheets on *how each command works*. What do the options used in the worksheets mean, for each command? (See the weekly [List of Commands].) Form a small study group to do the worksheets. Each person tries the example given, and you make sure you all get the same answers. Worksheets are not for hand-in; they are not worth marks; the assignments test your knowledge of the lectures and worksheets. > The worksheets are available in four formats: Open Office (ODT), PDF, HTML, > and Text. Only the Open Office format allows you "fill in the blanks" in > the worksheet. The PDF format looks good but doesn't allow you to type into > the blanks in the worksheet. The HTML format is crude but useful for quick > for viewing online. Do **NOT** open the Worksheet ODT files using any Microsoft products; they will mangle the format and mis-number the questions. Use the free Libre Office or Open Office programs to open these ODT documents. On campus, you can get a copy here: [Course Introduction: Install Libre Office]. Worksheets #02 and #03 require you to have read [File System and Pathnames]: - [Worksheet #02 ODT] -- Using standard Linux commands I - [Worksheet #02 PDF] - [Worksheet #02 HTML] - `PS1, cd, find, less, ls, man, mkdir, passwd, pwd, rmdir` - Remember to set your `PS1` prompt every time you log in to the CLS. - [Worksheet #03 ODT] -- Using standard Linux commands II - [Worksheet #03 PDF] - [Worksheet #03 HTML] - `cat, clear, cp, find, fgrep, history, less, man, mv, rm, sleep, touch` Worksheets #04 and #05 require you to have read [Shell GLOB patterns] and [Shell I/O Redirection]: - [Worksheet #04 ODT] -- shell GLOB patterns and Aliases - [Worksheet #04 PDF] - [Worksheet #04 HTML] - shell GLOB patterns (wildcards), Aliases: `alias, sum, unalias` - [Worksheet #05 ODT] -- shell I/O redirection (including pipes) - [Worksheet #05 PDF] - [Worksheet #05 HTML] - shell I/O redirection, `date, head, nl, tail, tr, wc` Worksheet #06 is for the optional VIM Bonus Assignment (extra credit): - [Worksheet #06 HTML] -- *Optional* Bonus VIM Text Editor Practice - This is an *optional* worksheet for a BONUS assignment using `vim` - Optional command-line VIM tutorial: the `vimtutor` program on the CLS. - Bonus (optional) [Assignment #04 HTML] -- the VIM text editor - Read [The VI (VIM) Text Editor] Worksheets prepare you for the upcoming assignments. Upcoming tests -------------- Follow this link to see all your upcoming [Quizzes and Tests]. The dates are also posted on the [Course Home Page] and on [Brightspace CST8207]. For full marks, read the [Test Instructions] (all the words) **before** your midterm tests. 1. First Midterm test: 45 minutes; in your one-hour lecture class in Week 6. 2. Second Midterm test: 45 minutes; in your one-hour lecture class in Week 10. Tests take place in your one-hour lecture class, not in your lab period. You must write the test in the lecture class in which you are registered. ### Practice Test #1 and Brightspace Quiz #2 [Practice Tests and Answers] -- practice questions for your midterm test. [Quizzes and Tests] -- Quiz #2 is taken from Practice Test #1 and is good practice for your upcoming Midterm Test #1 in Week 6. Notes from the Classroom ======================== - **Take notes in class!** Keep a pad open on your desk. - This week, we learn about I/O redirection and pipes. - I recommend that you learn the VIM text editor. - How do you know a pathname exists? What command can you use to verify that a pathname you type actually exists? - Remember to set your prompt (`PS1=`) every time you log in to the CLS. It only takes three keystrokes to recall and execute the command line. Simplifying pathnames --------------------- You will see lots of these: Rewrite each pathname as a simplified absolute pathname with no `.` or `..` references: 1. `/home/me/../you/../../etc/../home/me/../you/../me/../foo` 2. `/home/me/../you/../../etc/../home/me/../you/../me/../..` 3. `/usr/./bin/../lib/../../etc/../usr/./lib/../bin/./bar` 4. `/usr/./bin/../lib/../../etc/../usr/./lib/../bin/..` 5. `/../../var/./a/../../var/b/../../etc/./bar/../foo` 6. `/../../var/./a/../../var/b/../../etc/./bar/..` These questions and the answers are part of Practice Test #1 in [Practice Tests and Answers]. Look for questions starting with "Simplify this pathname". Draw file system diagrams ------------------------- See last week's note about drawing file system diagrams: [Week 03 Notes HTML] Did not change password -- CLS access revoked --------------------------------------------- Students will have their CLS accounts disabled if they have not changed their default password. (I do not want people breaking into my machine because of you.) See my online timetable for how to make an office appointment to see me if you want your account back. Commands Used ------------- > Keep a notebook with a [List of Commands] in it. > > - You need to write down yourself what each command *does*. > - Check the updated list of commands each week. > - I will check for this list in your lab periods. > - Bring your notes to class! Stop wasting time looking up commands. - Are you making notes from the worksheets on how each command works? - What do the options used in the worksheets mean, for each command? - Don't copy and use options that you don't understand! - GLOB patterns and pipes are *NOT* needed in Assignment 3. See Assignment 5. - About `ls -d`: See last week's notes: [Week 03 Notes HTML] Tutoring ======== Many students find that hiring a personal tutor helps them get through the first term. Financial assistance is available. See the **Tutoring** heading in the [Course Introduction]. Network Tutor Available on Saturday mornings -------------------------------------------- See your Networking course announcements. SSH and HTTP Attacks on the CLS =============================== You are not paranoid if they really are out to get you! See the [Week 03 Notes HTML] for command pipelines to count the ongoing attacks on the CLS. Student Class Representative ============================ **Andre D'Aoust** was selected as your **Class Representative** for for 19W. Reach out to him with your concerns when they *affect the entire class*. Individual concerns are handled between the student and the professor. Locked out of the CLS ===================== When you are locked out, [follow the directions] in the notes to get your IP address re-enabled. ![Take Notes in Class] -- | Ian! D. Allen, BA, MMath - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [www.idallen.com]: http://www.idallen.com/ [Course Home Page]: .. [Course Outline]: course_outline.pdf [All Weeks]: indexcgi.cgi [Plain Text]: week04notes.txt [Week 04 Notes HTML]: week04notes.html [Practice Tests and Answers]: 000_practice_tests.html [Shell I/O Redirection]: 200_redirection.html [Selecting Fields with `awk`]: 187_selecting_fields_awk.html [The VI (VIM) Text Editor]: 300_vi_text_editor.html [Shell Variables]: 320_shell_variables.html [List of Commands]: 900_unix_command_list.html [Video Tutorials on Lynda.com]: 910_lynda_index.html [Linux and Sysadmin News in the World]: 950_linux_world.html [Unix for Mac OS X Users]: http://www.lynda.com/Mac-OS-X-10-6-tutorials/Unix-for-Mac-OS-X-Users/78546-2.html [Assignment #02 HTML]: assignment02.html [Assignment #03 HTML]: assignment03.html [Assignment #04 HTML]: assignment04.html [Assignment #05 HTML]: assignment05.html [Shell GLOB patterns]: 190_glob_patterns.html [Worksheet #04 HTML]: worksheet04.html [Worksheet #05 HTML]: worksheet05.html [Finding Files]: 180_finding_files.html [Course Introduction: Install Libre Office]: 050_course_introduction.html#install-libreoffice-or-openoffice-into-windows [File System and Pathnames]: 160_pathnames.html [Worksheet #02 ODT]: worksheet02.odt [Worksheet #02 PDF]: worksheet02.pdf [Worksheet #02 HTML]: worksheet02.html [Worksheet #03 ODT]: worksheet03.odt [Worksheet #03 PDF]: worksheet03.pdf [Worksheet #03 HTML]: worksheet03.html [Worksheet #04 ODT]: worksheet04.odt [Worksheet #04 PDF]: worksheet04.pdf [Worksheet #05 ODT]: worksheet05.odt [Worksheet #05 PDF]: worksheet05.pdf [Worksheet #06 HTML]: worksheet06.html [Quizzes and Tests]: 000_quizzes_and_tests.html [Brightspace CST8207]: https://brightspace.algonquincollege.com/d2l/home/117776 [Test Instructions]: 000_test_instructions.html [Week 03 Notes HTML]: week03notes.html [Course Introduction]: 050_course_introduction.html [follow the directions]: 070_course_linux_server.html#getting-locked-out-of-the-server [Take Notes in Class]: data/remember.jpg "Take Notes in Class" [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/