Transfer Data to and from CSAR Machines

Introduction

There are a number of ways of transferring the data to/from the machines.
In the examples given below, the user must supply the username and host for the machine to which the data is being transferred.

• sftp (or use ftp if sftp unavailable on remote machine)

• rcp
• scp (or use rcp if scp unavailable on remote machine)

• tape

Data is migrated to tape storage after being placed in the user's /hold directory on the SAN. A cache of the /hold directories is kept on the SAN but once a particular file is no longer in the cache, it can take a considerable amount to retrieve it from tape. Files can be prefetched on Wren using the dmget command and their status checked using dmls -l. Files can be transferred to remote machines using any of the standard techniques listed here. For more information on /hold, see http://www.csar.cfs.ac.uk/user_information/san/san_files_hold.shtml

All the above rely on the network bandwidth to the individual machines, which is currently 100MB/s. However they are connected to the Janet Backbone which operates at a much higher rate than this.

Transfer methods

Most transfer methods should be done to the interactive machines, Newton and Wren, and are restricted.

Transfers using sftp/scp often see an improvement when using Newton rather than Wren as the bottleneck can be the encryption algorithm performing better on the faster processors of Newton.

Green permits no form form of interactive login and thus transfers initiated remotely cannot be performed, however it is possible from within a batch job which could prove costly if running a large job.