To effectively use a Unix or linux, you need to be able to transfer files between systems using command-line tools. There are two programs to do this: secure transfer file protocol (sftp) and secure copy protocol (scp). 

Both sftp and scp are secure, which means that your password is encrypted when you enter it. In fact, the entire transmission is encrypted, since both use ssh encryption, meaning no one can read your data in transit. You may not care about this, but you certainly care about your password. For this reason you should never use insecure protocols (ftp or telnet) to login to a site on which your personal data resides. (Some systems allow public ftp connections, with a public password. For these sites, guarding the password is unnecessary, as anyone is allowed to log on.)

sftp and scp each have different strengths. We can see these strengths by some examples of how they work. We will start with sftp, which is used to transfer individual files between two systems. It proceeds like this:

• the system you are currently on is the local system. Your current directory is called your local current directory
• you initiate a connection to another system, called the remote system, using the command

sftp [login@]host

and respond with your password when prompted. When you log in you are connected to your default directory (in the case of a personal login, this is your home directory) on the remote system.

You are now ready to issue sftp commands

sftp commands

The most confusing thing to remember about commandline sftp is that you are simultaneously connected to two different systems. Simple sftp commands allow you to change directory, create directories, list files, remove files, and, finally, transfer files. These commands can be applied to either one of the systems, or, in the case of file transfer, to transfer a file in either direction.

For simplicity, sftp commands look very similar to standard Unix commands. They are not standard Unix commands, however. They are sftp commands that have the same names as standard Unix commands. The sftp versions [with one exception] do not have options, and, since no shell is involved, wildcards are not expanded. 

The remote system (the one you sftp-ed to) is the default system used for commands. The familiar commands (such as ls for 'list') apply to the remote system. The corresponding command for the local system has the prefix l (for local), in this case lls for 'local list'.

sftp is very useful when you want to transfer multiple files from different directories OR when you are unsure where a piece of data is. This makes it ideal for public repositories, where you may not know the exact name of the file or where it is located, requiring some searching.

Let's try a couple of simple problems using sftp:

Example #1 - You are logged into hills.ccsf.edu and you want to back up your archive file files.tar.gz (a gzipped tar archive is similar to a zip file) to your home system. Your login at home is gsmith and your home system's IP address is 204.23.44.121. The archive is in the current directory on your local system, and you want to put it in the directory /backups on your home system. Below are the commands to do it. (The text appearing after # on some lines are annotations, or comments.)

Example #2 - there is a sample output file for the shell script for assignment one in the public directory for cs160b on hills. You know the file is beneath /pub/cs/gboyd/cs160b, but you are unsure what it's name is and what directory it is in. You want to transfer the file to your home system and place it in the directory asmt01 beneath your home directory. (In the example, you think the asmt01 directory exists, but it does not.) Your login on hills is you1. Here are some commands to do it. (Note: the example is for a lecture section of cs160b. An online section would use the directory asmt01-online)

This last example, while complicated, shows you abilities only available in sftp: searching for a file, and creating a directory as needed. Some problems require this type of flexibility, but the most common problems do not. For these, the overhead of sftp outweighs the flexibility, as does the limitation of only transferring one file at a time.

The non-secure counterpart of sftp, ftp, can still be used for public repositories. When accessing a public repository, it is easiest to use the ftp protocol via a browser. To see if your favorite website has a public repository, simply prefix the web domain with ftp in a browser window. Using this technique I found public ftp sites for Apple (ftp.apple.com), CCSF (ftp.ccsf.edu), and Electronic Arts (ftp.ea.com), but not one for Google. If logging in without a password is allowed (or if your browser can login using your email address as password (below)), the ftp site will show up in the browser window.

Some public ftp sites insist on a password to connect. In this case you can connect using command-line ftp. Use the account anonymous and your email address as the password. (You can probably use any email address.) One site that I found that refused a browser connection but allowed public command-line ftp access was ftp.irs.gov

Differences between ftp and sftp

At the level of our discussion, the biggest difference between sftp and ftp is that ftp does not have the local commands. Instead, prefix a standard shell command with ! (exclamation point) to run the command on the local machine. Note that the command !cd x will not change ftp's current local directory, as the cd command runs in a separate process.

Secure Copy 

If you know the exact path of the data you want to transfer and exactly where you want to put it, scp is more convenient than sftp. scp functions the same as any Unix command: you issue the command, the command is executed, either succeeding or failing, and control is returned to your shell. The syntax is strange at first, but it is consistent with another standard Unix command, cp.

scp [ -r ] source destination

where each of source and destination have the following syntax:

[[user@]host:]path

Here, [[user@host:] must be present for the remote system, and absent for the local system. Most versions of scp only allow one of source and destination to be remote. The path can be absolute or relative. Remote paths are relative to the default directory on the remote host (in our case, the home directory). 

Note that the host specification ends with a colon (:). If there is no colon, the host specification is not recognized, and it is treated as a path! If you run scp with two local paths, it functions the same as a cp command does! 

Let's analyze a few scp commands:

scp you1@hills.ccsf.edu:cs160b/typescript .

This scp command proceeds as follows

• it attempts to login to hills.ccsf.edu as you1, asking for a password. 
• If the login is successful, it then attempts to transfer the file cs160b/typescript (relative to the home directory of you1 on hills) to the current directory. 
• The resulting file will, of course, be named typescript. Just like cp, scp overwrites any typescript file that is in its way, if permissions permit.

scp typescript hills.ccsf.edu:

• This command attempts to login on hills.ccsf.edu using the current login from the local machine (your login is the same on both systems). It asks you for your hills password.
• If the login is successful, it attempts to transfer the file typescript from the current directory to the home directory on hills.
• Note that the path is empty here. This is legal, and . is the default. If this makes you nervous, you can append . or ~ as in  scp typescript hills.ccsf.edu:~

Now let's try a couple of problems:

Example #1 - You are logged into hills.ccsf.edu and you want to back up your archive file files.tar.gz to your home system. Your login at home is gsmith and your home system's IP address is 204.23.44.121. The archive is in the current directory on your local system, and you want to put it in the directory /backups on your home system. (This is the problem we solved as Example #1 in sftp.) Here is the command to do it:

scp files.tar.gz gsmith@204.23.44.121:/backups

Example #2 - You are logged into your home system and you want to backup up your entire home directory from hills.ccsf.edu to your home system. Again, your login at hills.ccsf.edu is you1. You want to place the copy in the existing directory hillshome beneath your home directory:

scp -r hills.ccsf.edu:~ ~/hillshome

Remember to use the -r option when you are transferring a directory!

scp -r hills.ccsf.edu:* ~/hillshome

would work as well. (Note that hidden files and directories would be skipped.)

Copyright 2011 Greg Boyd - All Rights Reserved.