Navigating the LMS and basic Using the research/teaching servers with the Linux command line
to get you startedinterface
Once you have successfully logged into the LMS it research or teaching servers, they may look very different to what you are use tooused to, particularly if you are use used to using windowsWindows. Please do not let this put you off. The research computing team have successfully managed to help many people use these computers who have never used the Linux command line before. It takes a bit of getting use used to but the more you use it the easier and quicker it will become over time.
The command line, or shell, has been the major interface for the Unix/Linux operating system since it was first conceived in the late 1960s. The shell allows interaction with the operating system through a text based interface. The shell provides the user with the following features:
- An easy to use command line interface (CLI)
- Combine existing tools to create new tools
- Input/output redirection
- Wildcard characters for filename abbreviation
- Variables and options for environment customisation
- Ability to write shell functions and programs
- Command-line editing
- Command history (quick access to previous commands)
- Arithmetic
- Command abbreviations
The user starts the , rather than the graphical interface you are used to. While the graphical interface is easy to learn, and usually makes simple things easy to do, it can be hard to do complex things like operate on large numbers of files, or make different tools work together. The shell can be hard to learn, but is much more powerful and flexible than most graphical interfaces, so can be very useful for research, where we often want to try new things on large data sets.
In this tutorial, we will only scratch the surface of the shell's features, just to get you started, but we will note some further features at the end of the tutorial that you may want to look into.
The user starts the shell by logging into the computer with a userid and password:
| Code Block |
|---|
| language | bash |
|---|
| title | Logging in |
|---|
|
******************************************************************************
*** THE UNIVERSITY OF YORK IT SERVICES ***
*** ***
*** THIS IS A PRIVATE COMPUTER ***
*** UNAUTHORISED ACCESS STRICTLY PROHIBITED ***
******************************************************************************
login: user001
password:
Last login: Mon Sep 8 14:12:44 2014 from gallifrey.york.ac.uk
-bash-4.1$
|
The last line is a command prompt and it is the means by which the computer is telling you that it is ready to accept a command from you. If you do not see the prompt, the computer is probability still executing the last command you have typed. The user types commands which take the form:
command program [ options ] [ arguments ]
Options Roughly speaking, program is the name of the program we want to run, arguments are objects we want to process (typically data files or folders), and options modify how the program will run. Options to a command are usually proceeded by a '-' or '- -' , this differentiates to differentiate them from the arguments. The following example shows the echo command which prints the arguments and the ls command which take options arguments. The ls command display the users file. There will be more explanation of files and the ls command laterexercise demonstrates using the echo program with a series of arguments and the ls program with or without options.
Exercise 2 - Running commands in the Linux shell
| Expand |
|---|
When you see the prompt type the following command . You (you can also copy and paste the command into your terminal).
| Code Block |
|---|
| echo I love York |
What happens? The terminal should write out "i I love York". The echo command is a small program that takes an input called a string, a series of characters arguments and repeats them to the user. Now try typing the following commandcommands. What do you see? Something like this? | Code Block |
|---|
| language | bash |
|---|
| title | Exmaple Example of command execution |
|---|
| -bash-4.1$ echo Hello World
Hello World
-bash-4.1$ ls
bin Chemistry Desktop examples Experiments intel jobs logs tmp
-bash-4.1$ ls -l
total 296
drwxr-xr-x 2 abs4 csrv 4096 Jun 24 09:39 bin
drwxr-xr-x 3 abs4 csrv 4096 Jun 6 09:23 Chemistry
drwxr-sr-x 2 abs4 elecclust 4096 Mar 11 10:53 Desktop
drwxr-xr-x 3 abs4 csrv 4096 Jun 30 12:21 examples
drwxr-xr-x 5 abs4 csrv 4096 May 23 11:34 Experiments
drwxr-xr-x 3 abs4 csrv 4096 Aug 14 12:26 intel
drwxr-sr-x 3 abs4 elecclust 4096 Aug 15 12:49 jobs
drwxr-xr-x 2 abs4 csrv 266240 Aug 15 13:48 logs
drwxr-xr-x 3 abs4 csrv 4096 Aug 14 14:50 tmp
-bash-4.1$ |
The ls command lists all of the files in your current directory. A directory is equivalent to a folder. The ls -l command gives you a long list, showing more information about each file or folder such as who is the owner of the file, who can access it, when it was last accessed. You can use most linux commands Linux programs in a number of ways by adding extra flagsoptions. Here for ls -l we added the -l flag option. If you need to know more about a command program you can use either of the following:
| Code Block |
|---|
| ls --help
or
man ls |
|
Filesystem organisation
The
ls can also take arguments as input. For example, to see the contents of a directory in your current directory, pass the name of the directory to ls as an argument. You can combine options and arguments, like the second command below: | Code Block |
|---|
| ls Desktop
ls -l Desktop |
Filesystem organisationThe file system is the component of the operating system that organises data into files. These files are organised into directories (just like directories in Windows Explorer or the Mac Finder). When you have logged in you will be placed in a directory which is called your home |
-|
| language | bash |
|---|
title | directory. To find the name of the directory use the program pwd (print working directory). | | Code Block |
|---|
Now try running the following commands. Here we have shown the prompt (-bash-4.1$) followed by the command to type on one line, then the output of the command on the following line (e.g after the -bash-4.1$, the first command is pwd, and the output is /home/userfs/e/ejb573; do not type the prompt as well, only the command). Your output will be slightly different as it will display the path to your home directory. | Code Block |
|---|
| language | bash |
|---|
| title | Locating your home directory and files |
|---|
| -bash-4.1$ pwd
/ |
usrresearchcompelecclustabs4usrusrhome
-bash-4.1$ cd
-bash-4.1$ pwd
/ |
usrresearchcompelecclustabs4ejb573
-bash-4.1$ cd ..
-bash-4.1$ pwd
/ |
usrresearchcompelecclust.usrresearchcompelecclust
The output of the pwd command, / |
usrresearchcompelecclustabs4ejb573, is called a pathname, and this specifies the location of |
the users user ejb573's home directory. The first '/' in the pathname is called the root directory |
. names , the top-level directory in the hierarchy. Names following the '/' are directory names. Directories within directories are called sub-directories. |
Pathanmes Path names can also specify the location |
within the filesystem Only The last part of a pathname (after the last /) is typically the name of a |
pathaname can be a command program lets you change your working directory to another |
location directory in the file system. cd with no arguments places you back in your home directory. The special |
directory references means the directory above your current directory (known as the parent directory). |
The is another speicial direcory '.' which references the current directory. These two directories can be viewd as links.Listing files and directories
To list the files in a directory use the ls (list) command
Creating, moving and copying files and directoriesYou can create new files with the touch program and new directories with mkdir. You can move or copy files or directories to other locations with the mv (move) and cp (copy) programs. First, let's create a new file and directory. | Lisiting | contentsls
afile bin Chemistry Desktop examples Experiments intel jobs logs tmp
afile
-bash-4.1$ touch bfile
-bash-4.1$ mkdir new-dir
-bash-4.1$ ls |
-ltotal 296
-rw-r--r-- 1 abs4 csrv afile bfile Chemistry examples intel logs |
0Sep 8 16:26 afile
drwxr-xr-x 2 abs4 csrv tmp
bin Desktop Experiments jobs |
4096Jun 24 09:39 bin
drwxr-xr-x 3 abs4 csrv You should now see the files afile and bfile and the directory new-dir. Let's experiment with afile and new-dir: | Code Block |
|---|
| language | bash |
|---|
| title | Create a new directory |
|---|
| -bash-4.1$ ls
afile bin |
4096Jun609:23 Chemistry
drwxr-sr-x 2 abs4 elecclust 4096 Mar 11 10:53 Desktop
drwxr-xr-x 3 abs4 csrvExperiments jobs new-dir
bfile Chemistry examples intel logs tmp
-bash-4.1$ mv |
4096 Jun 30 12:21 examples
drwxr-xr-x 5 abs4 csrv afile new-dir
-bash-4.1$ cp bfile new-dir
-bash-4.1$ ls
bfile Chemistry examples intel |
4096May2311:34Experiments
drwxr-xr-x3abs4csrv4096Aug1412:26 intel
drwxr-sr-x 3 abs4 elecclust 4096 Aug 15 12:49 jobs
drwxr-xr-x 2 abs4 csrv 266240 Aug 15 13:48 logs
drwxr-xr-x 3 abs4 csrv 4096 Aug 14 14:50 tmp new-dir
-bash-4.1$ ls new-dir
afile bfile
-bash-4.1$ mv new-dir/afile .
-bash-4.1$ ls |
-a.afile bin Desktop Experiments |
Experiments jobs new-dir
bfile Chemistry |
intel.profile
.. .felix jobs .pulse
afile .gconf .lesshst This example creates a new directory, 'new-dir', We then move the file 'afile' to it and create a copy of 'bfile'. We then move the file 'afile' back to our current working directory. The '.' argument in "mv new-dir/afile ." means the current working directory, so this command moves 'afile' to your working directory.
Copying a directory is a little more complicated as the directory may contain files and directories. We use cp's '-r' option (recursive) to do this. | Code Block |
|---|
| language | bash |
|---|
| title | Copying a directory |
|---|
| -bash-4.1$ ls
afile bin |
.pulse-cookie
.bash_history .gconfd Desktop Experiments jobs tmp
bfile Chemistry |
.local .python_history
bin logs
-bash-4.1$ ls tmp
icc-start ifort-start logs mpi-stop |
.gnome2 start test
icc-stop ifort-stop mpi-start |
logs new-dir stop
-bash-4.1$ cp tmp/test .
cp: omitting directory `tmp/test'
-bash-4.1$ cp -r tmp/test . |
qmon_preferences
Chemistry .gnote.matlab test
bfile Chemistry examples intel |
.Rhistory
.config .gnupg .mcrCache8.3 .ssh
.dbus .gstreamer-0.10 .modulerc .subversion
Desktop .gvfs .nautilus tmp
.emacs.d .history .ngspice_history .Xauthority
examples .ICEauthority .nx
-bash-4.1$ ls -al
total 440
drwx------ 30 abs4 elecclust 4096 Sep 8 16:26 .
drwxrws--- 14 jaw500 elecclust 4096 Sep 8 16:25 ..
-rw-r--r-- 1 abs4 csrvtmp
-bash-4.1$ ls test
test.c test.cpp test.f test.f90 test.x
-bash-4.1$
|
In this example we wish to copy the contents of the directory 'tmp/test' into the current directory. cp will not copy a directory; we have to use the '-r' (recursive) option to tell cp to copy all files and directories within the directory. Deleting files and directoriesYou can use the rm (remove) program to delete files. Please be careful with these commands, because Linux has no undo! To remove a directory and all of its contents use the rm -r (recursive) option to the rm command. To be safe and check the files before you remove them use -ri (recursive and interactive) options.
|
Displaying and editing the contents of files
There are a variety of different tools to help you display and edit the contents of your files. We will provide some examples below but you may find other ones which you prefer to use in the future.
Exercise 3 - Displaying the contents of files
| Expand |
|---|
File contents can be displayed with several different Linux programs. cat (concatenate files) will print your file to the screen, but this is not a good idea when the files are large. Instead, we can use less to view the file one screen at a time. Try them both on an existing file on the research/teaching servers (here we are using a file we already have called snark2). | Code Block |
|---|
| language | bash |
|---|
| title | cat and more |
|---|
| -bash-4.1$ cat snark2
The Hunting of the Snark
By Lewis Carroll
Fit the First
The Landing
"Just the place |
0Sep816:26 afile
-rw------- 1 abs4 elecclust 16495 Sep 8 15:40 .bash_history
drwxr-xr-x 2 abs4 csrv 4096 Jun 24 09:39 bin
drwxr-xr-x 3 abs4 csrv 4096 Jun 6 09:23 Chemistry
drwxr-sr-x 4 abs4 elecclust 4096 Mar 11 10:53 .config
drwx--S--- 3 abs4 elecclust 4096 Mar 11 10:51 .dbus
drwxr-sr-x 2 abs4 elecclust 4096 Mar 11 10:53 Desktop
drwxr-xr-x 3 abs4 csrv 4096 May 23 14:52 .emacs.d
drwxr-xr-x 3 abs4 csrv 4096 Jun 30 12:21 examples
drwxr-xr-x 5 abs4 csrv 4096 May 23 11:34 Experiments
drwxr-xr-x 2 abs4 csrv 4096 Jul 1 12:00 .felix
drwx--S--- 4 abs4 elecclust 4096 May 2 16:09 .gconf
drwx--S--- 2 abs4 elecclust 4096 May 2 16:34 .gconfd
drwx--S--- 4 abs4 elecclust 4096 Mar 11 10:53 .gnome2
drwxr-sr-x 3 abs4 elecclust 4096 Mar 11 10:53 .gnote
drwx--S--- 2 abs4 elecclust 4096 Mar 11 10:52 .gnupg
drwxr-sr-x 2 abs4 elecclust 4096 Mar 11 10:53 .gstreamer-0.10
drwx--S--- 2 abs4 elecclust 4096 Mar 11 10:52 .gvfs
-rw------- 1 abs4 csrv 978 Jun 6 09:32 .history
-rw------- 1 abs4 elecclust 314 Mar 11 10:52 .ICEauthority
drwxr-xr-x 3 abs4 csrv 4096 Aug 14 12:26 intel
drwxr-sr-x 3 abs4 elecclust 4096 Aug 15 12:49 jobs
-rw------- 1 abs4 csrv 46 Jun 6 09:31 .lesshst
drwxr-sr-x 3 abs4 elecclust 4096 Mar 11 10:52 .local
drwxr-xr-x 2 abs4 csrv 266240 Aug 15 13:48 logs
drwxr-xr-x 3 abs4 csrv 4096 May 2 16:06 .matlab
drwxr-xr-x 9 abs4 csrv 4096 Jul 3 11:54 .mcrCache8.3
-rw-r--r-- 1 abs4 csrv 32 Sep 5 08:05 .modulerc
drwxr-sr-x 2 abs4 elecclust 4096 Mar 11 10:53 .nautilus
-rw------- 1 abs4 elecclust 0 Jan 13 2014 .ngspice_history
drwx--S--- 6 abs4 elecclust 4096 Apr 25 13:36 .nx
-rw-r--r-- 1 abs4 elecclust 145 May 19 11:59 .profile
drwx------ 2 abs4 csrv 4096 Mar 11 10:54 .pulse
-rw------- 1 abs4 elecclust 256 Mar 11 10:54 .pulse-cookie
-rw------- 1 abs4 csrv 49 Jun 3 13:42 .python_history
-rw-r--r-- 1 abs4 csrv 342 Jun 16 12:57 .qmon_preferences
-rw------- 1 abs4 csrv 40 May 23 11:09 .Rhistory
drwxr-sr-x 2 abs4 elecclust 4096 Jun 5 12:53 .ssh
drwxr-xr-x 2 abs4 csrv 4096 May 2 16:06 .subversion
drwxr-xr-x 3 abs4 csrv 4096 Aug 14 14:50 tmp
-rw------- 1 abs4 csrv 488 Sep 8 15:48 .Xauthority
-bash-4.1$
|
ls without any options or arguments lists the name of the files and directories in the current working directory. In this example above it is hard to see which names refer to files or directories. We will show you how to do this later on. The next example displays the directory in the long format using the '-l' option, much more information is displayed about the directories and files. The '-a' option shows all files, filenames starting with '.' are usually hidden from display. We can combine options to give more detail.
ls can take arguments as well. When specifying an argument ls displays the information for that file or directory.
| Code Block |
|---|
| language | bash |
|---|
| title | Displaying file or directory information |
|---|
|
-bash-4.1$ ls Experiments
architest.dtr OLD simple_verbs.dtr~
architest.dtr~ OLD CART simple verbs_to_Dunstan.txt
exploded.csv ordered simple verbs_to_Dunstan.xlsx
Latest CART phon.csv simple_verbs_to.txt
mian.rp phonsorted simple_verbs.txt
NotesAboutInfixPredictions.pdf simple_verbs.dtr simpverbsort
-bash-4.1$ ls -l Experiments
total 368
-rw-r--r-- 1 abs4 csrv 919 May 23 11:08 architest.dtr
-rw-r--r-- 1 abs4 csrv 909 May 23 11:08 architest.dtr~
-rw-r--r-- 1 abs4 csrv 3613 May 23 11:08 exploded.csv
drwxr-xr-x 2 abs4 csrv 4096 May 23 11:34 Latest CART
-rw-r--r-- 1 abs4 csrv 4019 May 23 11:28 mian.rp
-rw-r--r-- 1 abs4 csrv 193602 May 23 11:08 NotesAboutInfixPredictions.pdf
drwxr-xr-x 2 abs4 csrv 4096 May 23 11:08 OLD
drwxr-xr-x 2 abs4 csrv 4096 May 23 11:08 OLD CART
-rw-r--r-- 1 abs4 csrv 3613 May 23 11:08 ordered
-rw-r--r-- 1 abs4 csrv 6217 May 23 11:08 phon.csv
-rw-r--r-- 1 abs4 csrv 6217 May 23 11:08 phonsorted
-rw-r--r-- 1 abs4 csrv 17663 May 23 11:08 simple_verbs.dtr
-rw-r--r-- 1 abs4 csrv 17647 May 23 11:08 simple_verbs.dtr~
-rw-r--r-- 1 abs4 csrv 8058 May 23 11:08 simple verbs_to_Dunstan.txt
-rw-r--r-- 1 abs4 csrv 30416 May 23 11:08 simple verbs_to_Dunstan.xlsx
-rw-r--r-- 1 abs4 csrv 4696 May 23 11:08 simple_verbs_to.txt
-rw-r--r-- 1 abs4 csrv 17525 May 23 11:08 simple_verbs.txt
-rw-r--r-- 1 abs4 csrv 17647 May 23 11:08 simpverbsort
-bash-4.1$ ls -ld Experiments
drwxr-xr-x 5 abs4 csrv 4096 May 23 11:34 Experiments
-bash-4.1$ ls -l afile
-rw-r--r-- 1 abs4 csrv 0 Sep 8 16:26 afile
-bash-4.1$ ls /usr
appl cmsmigratetest lfa phpweb src vleexam
archive cvs lib puppet systems vle-sysadmin
backups datasets lib64 puppetdev tmp webmisc
bin etc libexec researchcomp transfer work
central games local rlink userfs yorkroot
cert idm logfiles sbin UserFS yorkweb
cmsmedia include mirror scratch vle yorkwebtest
cmsmediatest java misc secbuffer vle-arch
cmsmigrate kerberos opapp share vle-eldt
-bash-4.1$
|
Using a directory name as an option causes ls to list the contents of the directory. To list the attributes of the directory use the '-d' option. You can use a pathname as the argument.
Creating, moving and copying files and directories
You can create directories, move or copy files or directories to other locations in the filesystem using the mkdir (make directory) mv (move) and cp (copy) commands.
| Code Block |
|---|
| language | bash |
|---|
| title | Create a new directory |
|---|
|
-bash-4.1$ ls
afile bin Desktop Experiments jobs new-dir
bfile Chemistry examples intel logs tmp
-bash-4.1$ mv afile new-dir
-bash-4.1$ cp bfile new-dir
-bash-4.1$ ls
bfile Chemistry examples intel logs tmp
bin Desktop Experiments jobs new-dir
-bash-4.1$ ls new-dir
afile bfile
-bash-4.1$ mv new-dir/afile .
-bash-4.1$ ls
afile bin Desktop Experiments jobs new-dir
bfile Chemistry examples intel logs tmp
-bash-4.1$
|
This example creates a new directory, 'new-dir', We then move the file 'afile' to it and create a copy of 'bfile'. We then move the file 'afile' back to our current working directory. Note the use of the '.' file to reference the current working directory. We can use full or partial pathnames to reference other parts of the file system.
Copying a directory is a little more complicated and the directory may contain files and directories. We use the '-r' command to cp to do this.
| Code Block |
|---|
| language | bash |
|---|
| title | Copying a directory |
|---|
|
-bash-4.1$ ls
afile bin Desktop Experiments jobs tmp
bfile Chemistry examples intel logs
-bash-4.1$ ls tmp
icc-start ifort-start logs mpi-stop start test
icc-stop ifort-stop mpi-start new-dir stop
-bash-4.1$ cp tmp/test .
cp: omitting directory `tmp/test'
-bash-4.1$ cp -r tmp/test .
-bash-4.1$ ls
afile bin Desktop Experiments jobs test
bfile Chemistry examples intel logs tmp
-bash-4.1$ ls test
test.c test.cpp test.f test.f90 test.x
-bash-4.1$
|
In this example we wish to copy the contents of the directory 'tmp/test' into the current directory. cp will not copy a directory. we have to use the '-r' (recursive) option to tell cp to copy all files and directory within the directory.
Deleting files and directories
The rm (remove) command is used to delete files.
| Code Block |
|---|
| language | bash |
|---|
| title | Deleting files and directories |
|---|
|
-bash-4.1$ ls
afile bin Desktop Experiments jobs test
bfile Chemistry examples intel logs tmp
-bash-4.1$ rm bfile
-bash-4.1$ ls
afile Chemistry examples intel logs tmp
bin Desktop Experiments jobs test
-bash-4.1$ |
To deleting directories use the rmdir (remove directory) command.
| Code Block |
|---|
| language | bash |
|---|
| title | Deleting directories and their contents |
|---|
|
-bash-4.1$ rmdir dira
rmdir: failed to remove `dira': Directory not empty
-bash-4.1$ rm -r dira
-bash-4.1$ ls
afile Chemistry dirb Experiments jobs test
bin Desktop examples intel logs tmp
-bash-4.1$ rm -ri dirb
rm: descend into directory `dirb'? y
rm: descend into directory `dirb/dirb'? y
rm: remove regular empty file `dirb/dirb/afile'? y
rm: remove directory `dirb/dirb'? y
rm: remove regular empty file `dirb/afile'? y
rm: remove directory `dirb'? y
-bash-4.1$
|
rmdir will only remove empty directories. To remove a directory and all it's contents use the rm -r (recursive) option to the rm command. To be safe and check the files before you remove them use -ri (recursive and interactive) options.
Editing and displaying the contents of files
Text Editors
Displaying the contents of files
The commands cat (concatenate files) and more displays the contents of file.
| Code Block |
|---|
| language | bash |
|---|
| title | cat and more |
|---|
|
-bash-4.1$ cat snark2
The Hunting of the Snark
By Lewis Carroll
Fit the First
Bellman cried,
As he landed his crew with care;
Supporting each man on the top of the tide
By a finger entwined in his hair.
"Just the place for a Snark! I have said it twice:
That alone should encourage the crew.
Just the place for a Snark! I have said it thrice:
What I tell you three times is true."
-bash-4.1$ less snark
The Hunting of the Snark
By Lewis Carroll
Fit the First
The Landing
"Just the place for a Snark!" the Bellman cried,
As he landed his crew with care;
Supporting each man on the top of the tide
By a finger entwined in his hair.
"Just the place for a Snark! I have said it twice:
That alone should encourage the crew.
Just the place for a Snark! I have said it thrice:
What I tell you three times is true."
The crew was complete: it included a Boots—
A maker of Bonnets and Hoods—
A Barrister, brought to arrange their disputes—
And a Broker, to value their goods.
A Billiard-marker, whose skill was immense,
|
To move through the file with less, or to quit back to the command line, use the following keys: - SPACE - display the next screen of text
- q - quit displaying the file
- b - skip backwards through he file
- /pattern - search for text in the file
There are many text editors available on Linux. The easiest to use is probably nano: To edit text within nano, just type; to move around the file, use the arrow keys; to exit nano, type Control+X (^X is shown in the shortcuts at the bottom of the screen); to save the file, type Control+O (^O) and hit Enter when prompted for a filename. You might explore the other shortcuts shown at the bottom to try out some other features. You might also try vi or emacs, which are harder to learn and use but very powerful once you get used to them. If you are using X forwarding (see above), you might also try gedit, which is a graphical text editor.
|
