Comp:ssh

With the Linux ssh command you can logon to another server. You can configure ssh such that you do not have to enter your password every time again.

This is the C&CZ wiki page on ssh

First make sure ssh is working

From you local linux computer login to the remote server, e.g., lilo7.science.ru.nl

 ssh username@lilo7.science.ru.nl

where you will have to replace username with your own username. If your username on the remote server is the same as on you local computer you can drop it:

 ssh lilo7.science.ru.nl

You will be asked to enter your password. The first time you give this command you will be asked to trust that you are connecting to right server. Later you will get a warning if lilo7.science.ru.nl is not the same server anymore - e.g., when your connection is being hacked or, more likely, when a mistake was made during a software upgrade on lilo7.

Login without password

First generate keys on your local computer:

 ssh-keygen -t ed25519 -a 100

Accept all the defaults by giving enter (up to three times).

Now copy the local key to the remote server, e.g., to lilo8.science.ru.nl

 ssh-copy-id username@lilo8.science.ru.nl

You will have to enter your password, but the next time you logon with ssh you will not be asked for your password again.

Warning: if someone can break into your local computer, they will now also have access to lilo8.science.ru.nl

A more secure solution is to protect you private key with a passphrase.

The first time you login with ssh to a new server you will be asked whether this server should be added to the "~/.ssh/known_hosts" file. If the operating systems on the server was re-installed you may get a warning suggesting that the server may have been hacked. To prevent these messages follow the "SSH tips and settings" of the C&CZ wiki: wiki.cncz.science.ru.nl/SSH

Using a passphrase and an ssh-agent for security and ease

Your keys are kept in the .ssh directory in the files, e.g,

 % ls -l ~/.ssh
 -rw------- 1 gerritg gerritg  1766 May 19  2020 id_rsa
 -rw-r----- 1 gerritg gerritg   395 May 19  2020 id_rsa.pub

The public key id_rsa.pub is added to the authorized_keys file in the .ssh directory on the remote server. Above we used the ssh-copy-id command to do that. Before we protect the keys, first remove the keys we just made. On the local computer:

 cd ~/.ssh
 rm id_rsa id_rsa.pub

and on the remote server (e.g. lilo7):

 cd ~/.ssh
 rm authorized_keys

If you have more than one key in authorized_keys use an editor to remove keys.

Now, on your local computer re-generate the keys

 ssh-keygen

Accept the default file .ssh/id_rsa with Enter, and then enter a passphrase.

Now copy your public key again to lilo7 with the ssh-copy-id command, and try ssh to lilo7. Instead of your password, you will now have to enter your passphrase. Even if you passphrase is the same as your password, this is more secure, because your passphrase will never leave you computer. However, you will now have to re-enter your passphrase every time you logon.

The next step is to use an ssh-agent to remember your passphrase until you logout, so you don't have to enter it too often.

Using and ssh-agent

Start the agent, and give it your passphrase

 eval `ssh-agent`
 ssh-add

Now see if it works:

 ssh lilo7.science.ru.nl

The agent remembers your passphrase until you logout. To save some typing, you can define a function in your .bashrc

 function agent () {
   eval `ssh-agent`
   ssh-add
 }

so you will only have to enter agent to start the ssh-agent.

You can save a few more key-strokes by using the ssh configuration file:

Configuring ssh

When ssh starts it reads the configuration file ~/.ssh/config

Add these lines to this file:

 Host lilo7
    Hostname lilo7.science.ru.nl

If your username is different on lilo7 you can also add a line, right after Hostname, with

    User username_on_lilo7

From your local computer, you cannot directly login to a clusternode, e.g., cn10.science.ru.nl, you first have to go to a login server (lilo7 or lilo6 etc), and from there you can ssh to other servers on the network.

You can configure <ssh> to do this for you. On your local computer, add this to your ~/.ssh/config file:

 Host cn10
    ProxyCommand ssh -q lilo7 nc -q0 cn10.science.ru.nl 22

This assumes you have define lilo7 in the config file as well. You should now be able to directly ssh to cn10 from your local computer:

 ssh cn10

Here nc is the netcat command that is doing the forwarding and 22 is the ssh port number.

You can now give command from your local computer directly to cn10:

Keep your session alive

You may find that if you haven't used a ssh login session for 10 minutes or so, the connection is lost. This can be prevented by adding two lines to your ~/.ssh/config file:

 Host *
   ServerAliveInterval 15
   ServerAliveCountMax 3

Use ssh to give remote commands

With ssh you can give commands on the server, e.g.,

 ssh lilo7.science.ru.nl hostname

Slurm commands (to run batch jobs) must be given from a cluster node, and do not run on, e.g., lilo7. If you have configure ssh as described above, you can check the queue status from your local computer with

 ssh cn10 squeue

The rsync command for copying files and directories over internet

To copy an entire directory from your university account to your PC or laptop with linux the rsync command is very convenient:

   rsync -avz lilo7.science.ru.nl:projects/stage .

This will copy the directory projects/stage with all its subdirectories and files to your current directory, because of the -a flag. The v flag means verbose and the z flag means that the files will be compressed during the copy, which is useful on slow networks.

This command also works if a directory with the name stage already exists in your current directory. In that case, files that are the same on the remote and local machine will not be copied. When the remote file is different, a local file with the same name will be overwritten.