e) Input and Output Redirection, Pipes, and Filters

Input and Output Redirection

We can change the behaviour of programs to redirect input from a file instead of the keyboard and write to a file instead of the screen. The '>' character is used to redirect output to a file and '<' to redirect input.

Redirection of program I/O
-bash-4.1$ echo Hello World > afile
-bash-4.1$ cat afile
Hello World
-bash-4.1$ wc -l < wordsworth 
25
-bash-4.1$ 

The wc -l command counts the number of lines typed. In this example we have redirected the input from a file.

Pipes

Pipes allow the output of one program to be fed into the input of another. The '|' is the pipe symbol.

This example counts the number of lines in a set of files. We write the output to a file. The file is then sorted, using the sort command, into ascending order to give us the file order by number of lines.

Count lines in a file
-bash-4.1$ ls
carroll  keats  milton  tennyson  thomas  wordsworth
-bash-4.1$ wc -l * > linecount
-bash-4.1$ cat linecount 
  733 carroll
  423 keats
  156 milton
   11 tennyson
   28 thomas
   25 wordsworth
 1376 total
-bash-4.1$ sort -n -k 1 linecount 
   11 tennyson
   25 wordsworth
   28 thomas
  156 milton
  423 keats
  733 carroll
 1376 total
-bash-4.1$ 

First we count the lines in each of the six file and redirect the output to a file. We then sort the file by the first column in numerical order.

A quicker and more efficient way without using the intermediary file is to use a pipe.

Line counting and sorting using a pipe
-bash-4.1$ wc -l * | sort -n -k 1
   11 tennyson
   25 wordsworth
   28 thomas
  156 milton
  423 keats
  733 carroll
 1376 total
-bash-4.1$ 

Filters

A filter is a program that transforms an input stream into an output stream. Almost all Linux programs do this. The pipe is used to connect the filters. Here is an example of finding all the user-names of people logged into the computer.

Find a list of users who has logged in
-bash-4.1$ last |more
abs4     pts/14       gallifrey.york.a Thu Sep 11 08:47   still logged in   
jg757    pts/3        elecpc111.ohm.yo Thu Sep 11 01:51   still logged in   
dl792    pts/3        host-172-18-1-89 Wed Sep 10 23:10 - 23:34  (00:24)    
yx664    pts/9        :1001.0          Wed Sep 10 17:55   still logged in   
yx664    pts/7        :1001.0          Wed Sep 10 17:54   still logged in   
yx664    pts/5        :1001.0          Wed Sep 10 17:40   still logged in   
yx664    pts/13       :1001.0          Wed Sep 10 16:07   still logged in   
yx664    pts/10       :1001.0          Wed Sep 10 16:05   still logged in   
rm591    pts/14       mandle.york.ac.u Wed Sep 10 16:01 - 16:01  (00:00)    
yx664    pts/13       :1002.0          Wed Sep 10 15:56 - 16:03  (00:07)    
yx664    :1002        :1002            Wed Sep 10 15:54 - 16:04  (00:09)    
jdr500   pts/9        10.240.171.184   Wed Sep 10 15:53 - 17:14  (01:20)    
yx664    pts/7        :1001.0          Wed Sep 10 15:51 - 17:38  (01:47)    
yx664    pts/6        :1001.0          Wed Sep 10 15:47 - 16:05  (00:17)    
yx664    :1001        :1001            Wed Sep 10 15:45   still logged in   
rm591    pts/14       mandle.york.ac.u Wed Sep 10 15:36 - 15:38  (00:01)    
rm591    pts/14       mandle.york.ac.u Wed Sep 10 14:48 - 15:34  (00:45)    
rm591    pts/14       mandle.york.ac.u Wed Sep 10 14:32 - 14:35  (00:02)    
yx664    pts/13       :1001.0          Wed Sep 10 13:41 - 15:42  (02:00)    
yx664    pts/10       :1001.0          Wed Sep 10 13:17 - 15:44  (02:26)    
yx664    pts/9        :1001.0          Wed Sep 10 12:51 - 15:42  (02:50)    
yx664    pts/6        :1001.0          Wed Sep 10 12:21 - 15:42  (03:20)    
yx664    pts/9        :1001.0          Wed Sep 10 11:57 - 12:23  (00:26)    
--More--
-bash-4.1$ last | sort | uniq -w 9 | cut -c1-9

abs4     
at568    
dl792    
ff555    
fjg504   
jdr500   
jg757    
kb1024   
klcm500  
ma725    
msr514   
pbc500   
pbk1     
rfle500  
rm591    
root     
sjb508   
sl561    
sy757    
tao500   
tm588    
wtmp begi
yw679    
yx664    
-bash-4.1$ 

the last command displays all users and the dates and times they have logged in. We then sort this,and  pass it through uniq, which removes duplicate lines by comparing only the first 9 characters. We then remove the remainder of the line after the username with the cut command.

Other ways of doing this
 -bash-4.1$ last | cut -c1-9 | sort | uniq

abs4     
at568    
dl792    
ff555    
fjg504   
jdr500   
jg757    
kb1024   
klcm500  
ma725    
msr514   
pbc500   
pbk1     
rfle500  
rm591    
root     
sjb508   
sl561    
sy757    
tao500   
tm588    
wtmp begi
yw679    
yx664    
-bash-4.1$