Page tree
Skip to end of metadata
Go to start of metadata

Below we will summarise the typical way users can move/copy their data to Viking.  As more methods or data sources become apparent we will add this information here. 

Viking is a self contained machine, therefore you will notice your normal UOY home directories are not available. This is intentional for the following reasons. 

  • If the dedicated network link between campus and Viking goes down, it may cause slowness or jobs to fail. Instead, jobs should continue to run until the link is re-established. 
  • If a user tries to read/write from the university filestores using Viking, it is possible that they could bring the entire storage system down for the university. 
  • UOY home directories are not designed for high performance computing.   We have a special high performance filestore available on viking called scratch.  You will see a link to your scratch area in your home area on Viking. 


Windows 

If you are copying data from a Windows device it is recommended you use WinSCP.  Details on how to set it up can be found here 

ARCHIVED - Copying files from another host or desktop to interactive Linux servers

Please replace "research0.york.ac.uk" with "viking.york.ac.uk"

Also if you are trying to move lots of data, you will be better off copying it directly to your scratch area. 


Linux

You can copy your data from any Linux device to Viking using the following commands 

  • scp
  • rsync

Here are a couple of examples.  

scp 

Recommended  for a small number of files 

Example

You wish to copy your data from YARCC to Viking and place it in your scratch area on Viking.

#For an individual file
scp filename viking.york.ac.uk:~/scratch/ 

#For a folder with lots of files
scp -r dirname viking.york.ac.uk:~/scratch/

There are many options you can use with scp.  To view these options either run

man scp 

on the device you are using scp on or have a look at this scp wiki page

rysnc

Recommend for a large number of files.  Rsync will compare what is already in place, therefore if the network is interrupted you can run the command again and it will pick up where you left off.  It will only copy files that do not exists on the other server or files that have been changed. 

Example

You wish to copy your data from YARCC to Viking and place it in your scratch area on Viking.

Here is an example

rsync -avz dirname viking.york.ac.uk:~/scratch

There are many options you can use with rsync.  To view these options either run

man rsync 

on the device you are using rsync on or consult the rsync webpage




  • No labels