% CST8207 Week 04 Notes -- GLOB patterns, I/O redirection, pipes, stdout, stderr, `2>&1` % Ian! D. Allen -- -- [www.idallen.com] % Fall 2017 - September to December 2017 - Updated 2017-10-10 14:00 EDT - [Course Home Page] - [Course Outline] - [All Weeks] - [Plain Text] How To Succeed in Linux ======================= _ _ _____ ____ _ _ _ _ | | | | _____ __ |_ _|__ / ___| _ _ ___ ___ ___ ___ __| | (_)_ __ | | (_)_ __ _ ___ __ | |_| |/ _ \ \ /\ / / | |/ _ \ \___ \| | | |/ __/ __/ _ \/ _ \/ _` | | | '_ \ | | | | '_ \| | | \ \/ / | _ | (_) \ V V / | | (_) | ___) | |_| | (_| (_| __/ __/ (_| | | | | | | | |___| | | | | |_| |> < |_| |_|\___/ \_/\_/ |_|\___/ |____/ \__,_|\___\___\___|\___|\__,_| |_|_| |_| |_____|_|_| |_|\__,_/_/\_\ [Some advice from CST student Joshua McNeely] *"learn how to use the website early, because it is an incredibly useful resource (it's the only resource needed)."* Readings, Assignments, Labs, Tests, and ToDo ============================================ Read (at least) these things (All The Words) -------------------------------------------- 1. [Week 04 Notes HTML] -- this file -- **Read All The Words** 2. [Shell GLOB patterns (wildcard pathname matching)] 3. [Shell I/O Redirection] -- stdin, stdout, stderr, and Pipes 4. [Practice Tests and Answers] -- practice questions for your midterm test. 5. [List of Commands You Should Know] 6. [Video Tutorials on Lynda.com] -- tagged by week number 7. [Linux and Sysadmin News in the World] - Create your [lynda.com] account and watch these sections of [Unix for Mac OS X Users] - these videos are for *all* Unix/Linux students, not just Mac OSX users - **3. Working with Files and Directories** - Searching for files and directories 6m 32s - `find` using "wildcard" (GLOB) patterns - same as shell GLOB - Unix text editors 6m 39s - `vi/vim, nano` - `head, tail` - Naming files 5m 41s - use quotes to surround names with blanks or special characters - **6. Directing Input and Output 20m 39s** - Standard input and standard output 1m 24s - Directing output to a file 4m 13s - Appending to a file 2m 44s - Directing input from a file 5m 28s - Piping output to input 4m 40s - Suppressing output 2m 10s Assignments this week --------------------- Check the due date for each assignment and put a reminder in your agenda, calendar, and digital assistant. Just like in the Real World, not all due dates are on the same days or at the same times. - Review last week. Did you do everything assigned last week? - Read All The Words, Do, and then Submit via Blackboard: - [Assignment #03 HTML] -- working with files and pathnames: `PS1 find mkdir mv cp rm touch file` - OPTIONAL: Read [The VI (VIM) Text Editor] and optionally do the bonus [Assignment #04 HTML]. - [Assignment #05 HTML] will be out soon -- using GLOB wildcard patterns, selecting files using GLOB, redirection and pipes - Really do **Read All The Words**. You don't get a second chance to get it right. Lab work this week ------------------ - Finish the readings, worksheets, and [Assignment #03 HTML] from last week. - Study the first [Practice Tests and Answers] for your upcoming Midterm Test #1 on Thursday or Friday. - Complete and save the ODT or PDF versions of [Worksheet #04 HTML] and [Worksheet #05 HTML] in preparation for Midterm Test #1 and for doing [Assignment #05 HTML] - These worksheets require you to understand [Finding Files], [Shell GLOB patterns][Shell GLOB patterns (wildcard pathname matching)] (wildcard pathname matching) and [Shell I/O Redirection] (including Pipes). - Get ready for upcoming [Assignment #05 HTML] -- GLOB wildcard patterns, selecting files using GLOB, redirection and pipes - This assignment requires you to understand the material in the web pages and Worksheets 4 and 5. - OPTIONAL: Read [The VI (VIM) Text Editor] and optionally do the bonus [Assignment #04 HTML]. ### Worksheets Worksheets are preparation for your assignments. You can't do the assignments without having done the worksheets first, and you can't do the worksheets without having first read the Course Notes: 1. Read. 2. Worksheets. 3. Assignment. Make notes from the worksheets on how each command works. What do the options used in the worksheets mean, for each command? (See the weekly [Command List][List of Commands You Should Know].) Form a small study group to do the worksheets. Each person tries the example given, and you make sure you all get the same answers. Worksheets are not for hand-in; they are not worth marks; the assignments test your knowledge of the lectures and worksheets. > The worksheets are available in four formats: Open Office (ODT), PDF, HTML, > and Text. Only the Open Office format allows you "fill in the blanks" in > the worksheet. The PDF format looks good but doesn't allow you to type into > the blanks in the worksheet. The HTML format is crude but useful for quick > for viewing online. Do **NOT** open the Worksheet ODT files using any Microsoft products; they will mangle the format and mis-number the questions. Use the free Libre Office or Open Office programs to open these ODT documents. On campus, you can [download Libre Office here]. These first two worksheets require you to have read [File System and Pathnames]: - [Worksheet #02 ODT] -- Using standard Linux commands I - [Worksheet #02 PDF] - [Worksheet #02 HTML] - `PS1, cd, find, less, ls, man, mkdir, passwd, pwd, rmdir` - [Worksheet #03 ODT] -- Using standard Linux commands II - [Worksheet #03 PDF] - [Worksheet #03 HTML] - `cat, clear, cp, find, fgrep, history, less, man, mv, rm, sleep, touch` - [Worksheet #04 ODT] -- shell GLOB patterns and Aliases - [Worksheet #04 PDF] - [Worksheet #04 HTML] - shell GLOB patterns (wildcards), Aliases: `alias, sum, unalias` - [Worksheet #05 ODT] -- shell I/O redirection (including pipes) - [Worksheet #05 PDF] - [Worksheet #05 HTML] - shell I/O redirection, `date, head, nl, tail, tr, wc` - [Worksheet #06 HTML] -- *Optional* Bonus VIM Text Editor Practice - This is an *optional* worksheet for a BONUS assignment using `vim` - Optional command-line VIM tutorial: the `vimtutor` program on the CLS. - Bonus (optional) [Assignment #04 HTML] -- the VIM text editor - Read [The VI (VIM) Text Editor] Worksheets prepare you for the upcoming assignments. Upcoming tests -------------- > Note that your upcoming Midterm Test #1 is worth more than three > assignments put together. Don't spend all your time doing an assignment at > the expense of studying the [Practice Tests and Answers] for the Midterm > Test. For full marks, read the [Test Instructions] (all the words) before your midterm tests. Midterm tests and the final exam are closed-book, no aids (just like your job interview). You will be given scrap paper to write on, and you may write on the question sheet. Put the test dates into your phone! Use the [Algonquin Registered Name Game] link to test your name before the test. I don't answer questions about the instructions during the test. 1. **First Midterm test:** 45 minutes; in your one-hour lecture class on Thursday or Friday in Week 5 (October 5 or 6). 2. **Second Midterm test:** 45 minutes; in your one-hour lecture class on Thursday or Friday in Week 9 (November 2 or 3). Tests take place in your 50-minute lecture class, not in your lab period. Each midterm test is 45 minutes long and contains approximately 45 multiple-choice questions similar to those found in [Practice Tests and Answers]. You must write the test in the lecture class in which you are registered. ### Midterm Test #1 -- October 5 or 6 - Midterm #1 takes place on Thursday or Friday, October 5 or 6 (Week 5), in your scheduled lecture class (not in your lab period). - The tests and exams are all closed-book, no aids, just like your job interview. You will be given some scrap paper to write on. - For full marks, you must read the [Test Instructions] before the test for important directions on how to enter your name, student number, answers, your lab (not lecture) section number, and the test version code on the question sheet and the mark-sense forms. - There may be more questions on the test than you can answer in the time allowed; answer the ones you know, first. - A set of practice questions and answers for the first midterm test is posted in [Practice Tests and Answers]. - The tests use the semicolon `;` to separate multiple commands on the same line, to save space and paper, e.g. three separate commands can be written on one line like this: `date ; echo "hi" ; ls` - **Do not use semicolons as an interactive human user!** Type each command on its own line with the \[Enter\] key. ### Midterm #1 Topics - The Weekly Notes 1, 2, 3, and 4 (this week), including the readings. - Weekly web pages up to and including [Shell I/O Redirection]. - Worksheets 2, 3, 4, and 5. - Assignments 2 and 3. - The questions in the first practice test in [Practice Tests and Answers]. - Does *not* include anything about editors `vim` or `nano`. - Does *not* include anything about the [Course Introduction]. For full marks, you must read and understand the [Test Instructions]. ### Blackboard Quiz #1 - Blackboard has some quizzes taken randomly from the first practice test in [Practice Tests and Answers]. - The practice test itself is *not* part of your course grade, but these quizzes *are* part of your course grade. - See the **Blackboard Quizzes** section located in [Practice Tests and Answers][1]. Notes from the Classroom ======================== - **Take notes in class!** Keep a pad open on your desk. - This week, we learn how to match file names using GLOB patterns and then about I/O redirection and pipes. - Note that your upcoming Midterm Test #1 is worth more than three assignments put together. Don't spend all your time doing an assignment at the expense of studying the [Practice Tests and Answers] for the Midterm Test. Draw file system diagrams ------------------------- See last week's note about drawing file system diagrams: [Week 03 Notes HTML] Commands Used ------------- > Keep a notebook with a [List of Commands][List of Commands You Should Know] > in it. > > - You need to write down yourself what each command *does*. > - Check the updated list of commands each week. > - I will check for this list in your lab periods. > - Bring your notes to class! Stop wasting time looking up commands. - Are you making notes from the worksheets on how each command works? - What do the options used in the worksheets mean, for each command? - Don't copy and use options that you don't understand! - Using GLOB patterns as `-name` arguments to `find` - Remember to quote the `find` GLOB patterns to hide them from the shell! - GLOB patterns are *NOT* needed in Assignment 3. See Assignment 5. - About `ls -d`: The man page says "list directories themselves, not their contents". It doesn't say "list only directories;, do not list files". If you give a file and a directory to `ls -d`, you will get the file name and the directory name in the output. If you give a file and a directory to `ls` without `-d`, you will get the file name and the *contents of the directory* (not the directory name itself) in the output. See the difference? The `-d` shows the directory itself, not what is in the directory. Tutoring ======== Many students find that hiring a personal tutor helps them get through the first term. Financial assistance is available. See the **Tutoring** heading in the [Course Introduction]. SSH Attacks on the CLS ====================== You are not paranoid if they really are out to get you! See the [Week 03 Notes HTML] for command pipelines to count the attacks on the CLS. HTTP attacks on Course Linux Server ----------------------------------- In addition to the SSH attacks, machines on the Internet receive attacks using bogus HTTP requests. These unsuccessful attacks below attempt to use a chain of parent directories to fetch a file out of the file system: [Dec 13 13:50:57 2015] [client 54.201.251.33] Invalid URI in request GET /../../ratings.html HTTP/1.1 [Dec 13 13:51:01 2015] [client 54.201.251.33] Invalid URI in request GET /../../index.html HTTP/1.1 [Apr 26 17:09:59 2016] [client 67.79.37.250] Invalid URI in request GET /../../../../../../../mnt/mtd/yqcP HTTP/1.1 [May 02 12:21:45 2016] [client 222.235.67.147] Invalid URI in request GET /../../../../../../../mnt/mtd/IJSj HTTP/1.1 [Jun 27 17:54:40 2016] [client 222.198.128.207] Invalid URI in request GET /../../../../../../../mnt/mtd/3qS9 HTTP/1.1 [Oct 07 09:34:11 2016] [client 64.137.234.42] Invalid URI in request GET /../../../../../../../mnt/mtd/ahMk HTTP/1.1 [Oct 15 06:49:57 2016] [client 177.142.99.103] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 [Oct 19 20:49:33 2016] [client 189.100.188.31] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 [Oct 23 00:43:39 2016] [client 191.189.55.9] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 [Nov 02 21:35:09 2016] [client 201.37.223.72] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 [Jan 07 20:51:33 2017] [client 201.17.154.181] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 [May 10 21:22:29 2017] [client 187.67.167.31] Invalid URI in request GET /../../../../../../../mnt/mtd/qt HTTP/1.0 The Apache server blocked all the above attacks. ![Take Notes in Class] -- | Ian! D. Allen, BA, MMath - idallen@idallen.ca - Ottawa, Ontario, Canada | Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/ | College professor (Free/Libre GNU+Linux) at: http://teaching.idallen.com/ | Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/ [Plain Text] - plain text version of this page in [Pandoc Markdown] format [www.idallen.com]: http://www.idallen.com/ [Course Home Page]: .. [Course Outline]: course_outline.pdf [All Weeks]: indexcgi.cgi [Plain Text]: week04notes.txt [Some advice from CST student Joshua McNeely]: notes/data/joshua_testimony.txt [Week 04 Notes HTML]: week04notes.html [Shell GLOB patterns (wildcard pathname matching)]: 190_glob_patterns.html [Shell I/O Redirection]: 200_redirection.html [Practice Tests and Answers]: PRACTICE_TEST_README.html [List of Commands You Should Know]: 900_unix_command_list.html [Video Tutorials on Lynda.com]: 910_lynda_index.html [Linux and Sysadmin News in the World]: 950_linux_world.html [lynda.com]: http://algonquincollege.com/onlineresources/mobileStudent/lynda.htm [Unix for Mac OS X Users]: http://www.lynda.com/Mac-OS-X-10-6-tutorials/Unix-for-Mac-OS-X-Users/78546-2.html [Assignment #03 HTML]: assignment03.html [The VI (VIM) Text Editor]: 300_vi_text_editor.html [Assignment #04 HTML]: assignment04.html [Assignment #05 HTML]: assignment05.html [Worksheet #04 HTML]: worksheet04.html [Worksheet #05 HTML]: worksheet05.html [Finding Files]: 180_finding_files.html [download Libre Office here]: 050_course_introduction.html#install-libreoffice-or-openoffice-into-windows [File System and Pathnames]: 160_pathnames.html [Worksheet #02 ODT]: worksheet02.odt [Worksheet #02 PDF]: worksheet02.pdf [Worksheet #02 HTML]: worksheet02.html [Worksheet #03 ODT]: worksheet03.odt [Worksheet #03 PDF]: worksheet03.pdf [Worksheet #03 HTML]: worksheet03.html [Worksheet #04 ODT]: worksheet04.odt [Worksheet #04 PDF]: worksheet04.pdf [Worksheet #05 ODT]: worksheet05.odt [Worksheet #05 PDF]: worksheet05.pdf [Worksheet #06 HTML]: worksheet06.html [Test Instructions]: 000_test_instructions.html [Algonquin Registered Name Game]: http://cst8207.idallen.ca/~idallen/namegame.cgi [Course Introduction]: 050_course_introduction.html [1]: PRACTICE_TEST_README.html#blackboard-quizzes [Week 03 Notes HTML]: week03notes.html [Take Notes in Class]: data/remember.jpg "Take Notes in Class" [Pandoc Markdown]: http://johnmacfarlane.net/pandoc/