b) Files and Directories

Filesystem organisation

The file system is the component of the operating system that organises data into files. These files are organised into directories.

When you have logged in you will be placed in a directory which is called your home-directory. To find the name of the directory use the pwd (print working directory). 

Locating your home directory and files
-bash-4.1$ pwd
/usr/researchcomp/elecclust/abs4
-bash-4.1$ cd /usr
-bash-4.1$ pwd
/usr
-bash-4.1$ cd
-bash-4.1$ pwd
/usr/researchcomp/elecclust/abs4
-bash-4.1$ cd ..
-bash-4.1$ pwd
/usr/researchcomp/elecclust
-bash-4.1$ cd .
-bash-4.1$ pwd
/usr/researchcomp/elecclust
-bash-4.1$ 

The output of the pwd command, /usr/researchcomp/elecclust/abs4, is called a pathname, and this specifies the location of the users home directory. The first '/' in the pathname is the root directory. names following the '/' are directory names. Directories within directories are called sub-directories. Pathanmes can also specify the location within the filesystem of files. Only the last name of a pathaname can be a file or directory.

The cd command lets you change your working directory to another location in the file system. cd with no arguments places you back in your home directory. The special directory '..' references the directory above your current directory (known as the parent directory). The is another speicial direcory '.' which references the current directory. These two directories can be viewd as links.

Listing files and directories

To list the files in a  directory use the ls (list) command.

Lisiting directory contents
-bash-4.1$ ls
afile  bin  Chemistry  Desktop  examples  Experiments  intel  jobs  logs  tmp
-bash-4.1$ ls -l
total 296
-rw-r--r-- 1 abs4 csrv           0 Sep  8 16:26 afile
drwxr-xr-x 2 abs4 csrv        4096 Jun 24 09:39 bin
drwxr-xr-x 3 abs4 csrv        4096 Jun  6 09:23 Chemistry
drwxr-sr-x 2 abs4 elecclust   4096 Mar 11 10:53 Desktop
drwxr-xr-x 3 abs4 csrv        4096 Jun 30 12:21 examples
drwxr-xr-x 5 abs4 csrv        4096 May 23 11:34 Experiments
drwxr-xr-x 3 abs4 csrv        4096 Aug 14 12:26 intel
drwxr-sr-x 3 abs4 elecclust   4096 Aug 15 12:49 jobs
drwxr-xr-x 2 abs4 csrv      266240 Aug 15 13:48 logs
drwxr-xr-x 3 abs4 csrv        4096 Aug 14 14:50 tmp
-bash-4.1$ ls -a
.              Experiments      intel             .profile
..             .felix           jobs              .pulse
afile          .gconf           .lesshst          .pulse-cookie
.bash_history  .gconfd          .local            .python_history
bin            .gnome2          logs              .qmon_preferences
Chemistry      .gnote           .matlab           .Rhistory
.config        .gnupg           .mcrCache8.3      .ssh
.dbus          .gstreamer-0.10  .modulerc         .subversion
Desktop        .gvfs            .nautilus         tmp
.emacs.d       .history         .ngspice_history  .Xauthority
examples       .ICEauthority    .nx
-bash-4.1$ ls -al
total 440
drwx------ 30 abs4   elecclust   4096 Sep  8 16:26 .
drwxrws--- 14 jaw500 elecclust   4096 Sep  8 16:25 ..
-rw-r--r--  1 abs4   csrv           0 Sep  8 16:26 afile
-rw-------  1 abs4   elecclust  16495 Sep  8 15:40 .bash_history
drwxr-xr-x  2 abs4   csrv        4096 Jun 24 09:39 bin
drwxr-xr-x  3 abs4   csrv        4096 Jun  6 09:23 Chemistry
drwxr-sr-x  4 abs4   elecclust   4096 Mar 11 10:53 .config
drwx--S---  3 abs4   elecclust   4096 Mar 11 10:51 .dbus
drwxr-sr-x  2 abs4   elecclust   4096 Mar 11 10:53 Desktop
drwxr-xr-x  3 abs4   csrv        4096 May 23 14:52 .emacs.d
drwxr-xr-x  3 abs4   csrv        4096 Jun 30 12:21 examples
drwxr-xr-x  5 abs4   csrv        4096 May 23 11:34 Experiments
drwxr-xr-x  2 abs4   csrv        4096 Jul  1 12:00 .felix
drwx--S---  4 abs4   elecclust   4096 May  2 16:09 .gconf
drwx--S---  2 abs4   elecclust   4096 May  2 16:34 .gconfd
drwx--S---  4 abs4   elecclust   4096 Mar 11 10:53 .gnome2
drwxr-sr-x  3 abs4   elecclust   4096 Mar 11 10:53 .gnote
drwx--S---  2 abs4   elecclust   4096 Mar 11 10:52 .gnupg
drwxr-sr-x  2 abs4   elecclust   4096 Mar 11 10:53 .gstreamer-0.10
drwx--S---  2 abs4   elecclust   4096 Mar 11 10:52 .gvfs
-rw-------  1 abs4   csrv         978 Jun  6 09:32 .history
-rw-------  1 abs4   elecclust    314 Mar 11 10:52 .ICEauthority
drwxr-xr-x  3 abs4   csrv        4096 Aug 14 12:26 intel
drwxr-sr-x  3 abs4   elecclust   4096 Aug 15 12:49 jobs
-rw-------  1 abs4   csrv          46 Jun  6 09:31 .lesshst
drwxr-sr-x  3 abs4   elecclust   4096 Mar 11 10:52 .local
drwxr-xr-x  2 abs4   csrv      266240 Aug 15 13:48 logs
drwxr-xr-x  3 abs4   csrv        4096 May  2 16:06 .matlab
drwxr-xr-x  9 abs4   csrv        4096 Jul  3 11:54 .mcrCache8.3
-rw-r--r--  1 abs4   csrv          32 Sep  5 08:05 .modulerc
drwxr-sr-x  2 abs4   elecclust   4096 Mar 11 10:53 .nautilus
-rw-------  1 abs4   elecclust      0 Jan 13  2014 .ngspice_history
drwx--S---  6 abs4   elecclust   4096 Apr 25 13:36 .nx
-rw-r--r--  1 abs4   elecclust    145 May 19 11:59 .profile
drwx------  2 abs4   csrv        4096 Mar 11 10:54 .pulse
-rw-------  1 abs4   elecclust    256 Mar 11 10:54 .pulse-cookie
-rw-------  1 abs4   csrv          49 Jun  3 13:42 .python_history
-rw-r--r--  1 abs4   csrv         342 Jun 16 12:57 .qmon_preferences
-rw-------  1 abs4   csrv          40 May 23 11:09 .Rhistory
drwxr-sr-x  2 abs4   elecclust   4096 Jun  5 12:53 .ssh
drwxr-xr-x  2 abs4   csrv        4096 May  2 16:06 .subversion
drwxr-xr-x  3 abs4   csrv        4096 Aug 14 14:50 tmp
-rw-------  1 abs4   csrv         488 Sep  8 15:48 .Xauthority
-bash-4.1$ 

ls without any options or arguments lists the name of the files and directories in the current working directory. In this example above it is hard to see which names refer to files or directories. We will show you how to do this later on. The next example displays the directory in the long format using the '-l' option, much more information is displayed about the directories and files. The '-a' option shows all files, filenames starting with '.' are usually hidden from display. We can combine options to give more detail.

ls can take arguments as well. When specifying an argument ls displays the information for that file or directory.

Displaying file or directory information
-bash-4.1$ ls Experiments
architest.dtr                   OLD               simple_verbs.dtr~
architest.dtr~                  OLD CART          simple verbs_to_Dunstan.txt
exploded.csv                    ordered           simple verbs_to_Dunstan.xlsx
Latest CART                     phon.csv          simple_verbs_to.txt
mian.rp                         phonsorted        simple_verbs.txt
NotesAboutInfixPredictions.pdf  simple_verbs.dtr  simpverbsort
-bash-4.1$ ls -l Experiments
total 368
-rw-r--r-- 1 abs4 csrv    919 May 23 11:08 architest.dtr
-rw-r--r-- 1 abs4 csrv    909 May 23 11:08 architest.dtr~
-rw-r--r-- 1 abs4 csrv   3613 May 23 11:08 exploded.csv
drwxr-xr-x 2 abs4 csrv   4096 May 23 11:34 Latest CART
-rw-r--r-- 1 abs4 csrv   4019 May 23 11:28 mian.rp
-rw-r--r-- 1 abs4 csrv 193602 May 23 11:08 NotesAboutInfixPredictions.pdf
drwxr-xr-x 2 abs4 csrv   4096 May 23 11:08 OLD
drwxr-xr-x 2 abs4 csrv   4096 May 23 11:08 OLD CART
-rw-r--r-- 1 abs4 csrv   3613 May 23 11:08 ordered
-rw-r--r-- 1 abs4 csrv   6217 May 23 11:08 phon.csv
-rw-r--r-- 1 abs4 csrv   6217 May 23 11:08 phonsorted
-rw-r--r-- 1 abs4 csrv  17663 May 23 11:08 simple_verbs.dtr
-rw-r--r-- 1 abs4 csrv  17647 May 23 11:08 simple_verbs.dtr~
-rw-r--r-- 1 abs4 csrv   8058 May 23 11:08 simple verbs_to_Dunstan.txt
-rw-r--r-- 1 abs4 csrv  30416 May 23 11:08 simple verbs_to_Dunstan.xlsx
-rw-r--r-- 1 abs4 csrv   4696 May 23 11:08 simple_verbs_to.txt
-rw-r--r-- 1 abs4 csrv  17525 May 23 11:08 simple_verbs.txt
-rw-r--r-- 1 abs4 csrv  17647 May 23 11:08 simpverbsort
-bash-4.1$ ls -ld Experiments
drwxr-xr-x 5 abs4 csrv 4096 May 23 11:34 Experiments
-bash-4.1$ ls -l afile 
-rw-r--r-- 1 abs4 csrv 0 Sep  8 16:26 afile
-bash-4.1$ ls /usr
appl          cmsmigratetest  lfa       phpweb        src       vleexam
archive       cvs             lib       puppet        systems   vle-sysadmin
backups       datasets        lib64     puppetdev     tmp       webmisc
bin           etc             libexec   researchcomp  transfer  work
central       games           local     rlink         userfs    yorkroot
cert          idm             logfiles  sbin          UserFS    yorkweb
cmsmedia      include         mirror    scratch       vle       yorkwebtest
cmsmediatest  java            misc      secbuffer     vle-arch
cmsmigrate    kerberos        opapp     share         vle-eldt
-bash-4.1$ 

Using a directory name as an option causes ls to list the contents of the directory. To list the attributes of the directory use the '-d' option. You can use a pathname as the argument.

Creating, moving and copying files and directories

You can create directories, move or copy files or directories to other locations in the filesystem using the mkdir (make directory) mv (move) and cp (copy) commands.

Create a new directory
-bash-4.1$ ls
afile  bin        Desktop   Experiments  jobs  new-dir
bfile  Chemistry  examples  intel        logs  tmp
-bash-4.1$ mv afile new-dir
-bash-4.1$ cp bfile new-dir
-bash-4.1$ ls
bfile  Chemistry  examples     intel  logs     tmp
bin    Desktop    Experiments  jobs   new-dir
-bash-4.1$ ls new-dir
afile  bfile
-bash-4.1$ mv new-dir/afile .
-bash-4.1$ ls
afile  bin        Desktop   Experiments  jobs  new-dir
bfile  Chemistry  examples  intel        logs  tmp
-bash-4.1$ 

This example creates a new directory, 'new-dir', We then move the file 'afile' to it and create a copy of 'bfile'. We then move the file 'afile' back to our current working directory. Note the use of the '.' file to reference the current working directory. We can use full or partial pathnames to reference other parts of the file system.

Copying a directory is a little more complicated and the directory may contain files and directories. We use the '-r' command to cp to do this.

Copying a directory
-bash-4.1$ ls
afile  bin        Desktop   Experiments  jobs  tmp
bfile  Chemistry  examples  intel        logs
-bash-4.1$ ls tmp
icc-start  ifort-start  logs       mpi-stop  start  test
icc-stop   ifort-stop   mpi-start  new-dir   stop
-bash-4.1$ cp tmp/test .
cp: omitting directory `tmp/test'
-bash-4.1$ cp -r tmp/test .
-bash-4.1$ ls 
afile  bin        Desktop   Experiments  jobs  test
bfile  Chemistry  examples  intel        logs  tmp
-bash-4.1$ ls test
test.c  test.cpp  test.f  test.f90  test.x
-bash-4.1$ 

In this example we wish to copy the contents of the directory 'tmp/test' into the current directory. cp will not copy a directory. we have to use the '-r' (recursive) option to tell cp to copy all files and directory within the directory.

Deleting files and directories

The rm (remove) command is used to delete files.

Deleting files and directories
-bash-4.1$ ls
afile  bin        Desktop   Experiments  jobs  test
bfile  Chemistry  examples  intel        logs  tmp
-bash-4.1$ rm bfile
-bash-4.1$ ls
afile  Chemistry  examples     intel  logs  tmp
bin    Desktop    Experiments  jobs   test
-bash-4.1$

To deleting directories use the rmdir (remove directory) command.

Deleting directories and their contents
-bash-4.1$ rmdir dira
rmdir: failed to remove `dira': Directory not empty
-bash-4.1$ rm -r dira
-bash-4.1$ ls
afile  Chemistry  dirb      Experiments  jobs  test
bin    Desktop    examples  intel        logs  tmp
-bash-4.1$ rm -ri dirb
rm: descend into directory `dirb'? y
rm: descend into directory `dirb/dirb'? y
rm: remove regular empty file `dirb/dirb/afile'? y
rm: remove directory `dirb/dirb'? y
rm: remove regular empty file `dirb/afile'? y
rm: remove directory `dirb'? y
-bash-4.1$ 

rmdir will only remove empty directories. To remove a directory and all it's contents use the rm -r (recursive) option to the rm command. To be safe and check the files before you remove them use -ri (recursive and interactive) options.

Editing and displaying the contents of files

Text Editors

  • vi and vim
  • emacs
  • nano

Displaying the contents of files

The commands cat (concatenate files) and more displays the contents of file.

cat and more
 -bash-4.1$ cat snark2

The Hunting of the Snark
By Lewis Carroll
Fit the First
            The Landing

"Just the place for a Snark!" the Bellman cried,
   As he landed his crew with care;
Supporting each man on the top of the tide
   By a finger entwined in his hair.

"Just the place for a Snark! I have said it twice:
   That alone should encourage the crew.
Just the place for a Snark! I have said it thrice:
   What I tell you three times is true."

-bash-4.1$ more snark
The Hunting of the Snark
By Lewis Carroll
            Fit the First
            The Landing
"Just the place for a Snark!" the Bellman cried,
   As he landed his crew with care;
Supporting each man on the top of the tide
   By a finger entwined in his hair.
"Just the place for a Snark! I have said it twice:
   That alone should encourage the crew.
Just the place for a Snark! I have said it thrice:
   What I tell you three times is true."
The crew was complete: it included a Boots—
   A maker of Bonnets and Hoods—
A Barrister, brought to arrange their disputes—
   And a Broker, to value their goods.
A Billiard-marker, whose skill was immense,
--More--(2%)

The cat command displays all the test in the users file on the screen. This can prove difficult to read if there are large amounts of text. The more command paginates the text and displays portions of it on the screen. The user can use  character command to move through the file:

  • SPACE - display the next screen of text
  • q - quit displaying the file
  • b - skip backwards through he file
  • /pattern - search for text in the file

Files and directory permissions

Groups are provided to manage sets of users and control access to fie and directories. All users belong to a default group and may be a member of other groups.

Group membership
-bash-4.1$ groups
csrv pfs17 pfs34 csys cvssys itsilg rhpcs sshfix git5 git6 git7 elecclust yc-gauss yc-install yc-colum
-bash-4.1$ ls -l
total 332
-rw-r--r-- 1 abs4 csrv           0 Sep  8 16:26 afile
drwxr-xr-x 2 abs4 csrv        4096 Jun 24 09:39 bin
drwxr-xr-x 3 abs4 csrv        4096 Jun  6 09:23 Chemistry
drwxr-sr-x 2 abs4 elecclust   4096 Mar 11 10:53 Desktop
drwxr-xr-x 3 abs4 csrv        4096 Jun 30 12:21 examples
drwxr-xr-x 5 abs4 csrv        4096 May 23 11:34 Experiments
drwxr-xr-x 3 abs4 csrv        4096 Aug 14 12:26 intel
drwxr-sr-x 3 abs4 elecclust   4096 Aug 15 12:49 jobs
drwxr-xr-x 2 abs4 csrv      266240 Aug 15 13:48 logs
-rw-r--r-- 1 abs4 csrv       25678 Sep  9 10:24 snark
-rw-r--r-- 1 abs4 csrv         433 Sep  9 10:27 snark2
drwxr-xr-x 2 abs4 csrv        4096 Sep  8 17:08 test
drwxr-x--- 5 abs4 csrv        4096 Sep  8 17:01 tmp
-bash-4.1$ 

The groups command displays which groups you are a member of. Each file and directory you create will be owned by you and be potentially accessible to a group. In the above example the file 'afile' is owned by 'abs4' and is accessible to the 'csrv' group.

There is a 'special' group sometimes called world, or other,  which contains all users of the system.

In the above example the first column of the directory listing shows the permissions of the files. These permissions control who is allowed acces the files and directories. There are three categories of user who can has potential rights to access the files - owner, group, world. The access rights to the files are displayed in the form of a sequence of letters like 'drwxr-xr-x'. The meaning is:

  • d - if present this is a directory, otherwise it is a file
  • the following 3 letters are in three groups and state the access permissions for the owner, group, world users
    • w - the file can be written to
    • r - the file can be read
    • x - if a file it can be executed, if a directory it can be accessed
  • example - drwxr-x--- 5 abs4 csrv        4096 Sep  8 17:01 tmp
    • this is a directory
    • the owner, abs4, can read, write and access the directory
    • members of the group, elecclust, can read and access the directory, they can not creat files in the directory
    • all other users do not have any access to the directory

To change file permissions use the chmod command.

Changing file permissions
-bash-4.1$ ls -l snark
-rw-r--r-- 1 abs4 csrv 25678 Sep  9 10:24 snark-bash-4.1$ chmod go-rwx snark
-bash-4.1$ ls -l snark
-rw------- 1 abs4 csrv 25678 Sep  9 10:24 snark

The chmod command has the form:

chmod <mode> <file>

mode takes the form of:

[ugoa...][[+-=][perms...]...]

  • u = user (owner)
  • g = group
  • o = other (world)
  • a = all (user, group, other)

  • + add permission
  • - remove permission
  • = explicitly set permission

  • w = can be written to
  • r = can be read
  • x = can be executed if a file, if a directory it can be accessed
Some examples
-bash-4.1$ ls -l snark
-rw-r--r-- 1 abs4 csrv 25678 Sep  9 10:24 snark
-bash-4.1$ chmod a+rx snark
-bash-4.1$ ls -l snark
-rwxr-xr-x 1 abs4 csrv 25678 Sep  9 10:24 snark
-bash-4.1$ chmod go-rwx snark
-bash-4.1$ ls -l snark
-rwx------ 1 abs4 csrv 25678 Sep  9 10:24 snark
-bash-4.1$ chmod u-w snark
-bash-4.1$ ls -l snark
-r-x------ 1 abs4 csrv 25678 Sep  9 10:24 snark
-bash-4.1$ chmod a=r snark
-bash-4.1$ ls -l snark
-r--r--r-- 1 abs4 csrv 25678 Sep  9 10:24 snark
-bash-4.1$ chmod u=w snark
-bash-4.1$ ls -l snark
--w-r--r-- 1 abs4 csrv 25678 Sep  9 10:24 snark
-bash-4.1$ chmod u+r snark
-bash-4.1$ ls -l snark
-rw-r--r-- 1 abs4 csrv 25678 Sep  9 10:24 snark
-bash-4.1$ 

To change the group of a file use the  command chgrp <groupname> <filename>.

Changing group membership of a file
-bash-4.1$ ls -l snark
-rw-r--r-- 1 abs4 csrv 25678 Sep  9 10:24 snark
-bash-4.1$ groups
csrv pfs17 pfs34 csys cvssys itsilg rhpcs sshfix git5 git6 git7 elecclust yc-gauss yc-install yc-colum
-bash-4.1$ chgrp rhpcs snark
-bash-4.1$ ls -l snark
-rw-r--r-- 1 abs4 rhpcs 25678 Sep  9 10:24 snark
-bash-4.1$