Using a computational cluster
This tutorial is an introduction on how to start using a linux-based computational cluster from the command line of your device. It covers some basic aspects: generating a log in key pair, logging in and navigating the system, and transferring files between the remote system and your device. The approaches discussed here should work on any client system - Windows, macOS, or linux.
Linux, macOS, and recent releases of Windows usually have an ssh client enabled by default. To confirm this, run ssh
in the terminal. If the result of this command is not a “command not found” message, you are good to proceed with this
tutorial. Otherwise, it is necessary to configure an ssh client on your system before continuing the tutorial.
If you are doing this tutorial on Windows, it is necessary to use PowerShell and not the cmd.exe command line.
Navigating the command line
To log in to the remote machine, run the following command:
ssh <username>@<hostname><username> should be replaced by the name assigned to you by the system administrator and <hostname> should be replaced
by the web address of the remote machine. When done with the session, make sure to log out and terminate your session by running:
exitOnce logged in to the remote machine, the command line will be initialized in the home directory. The command line is like a file browser - it lets you create, move, open, and delete files on the computer. To switch between folders, manipulate files, and run programs, text commands need to be typed into the terminal. Here I provide a reference list of some of the most fundamental commands that you will need while working with a linux machine.
ls command
| Mnemonic | list |
| Action | list files in the current directory |
| Arguments | none |
pwd command
| Mnemonic | print working directory |
| Action | print the path of the current directory |
| Arguments | none |
cd command
| Mnemonic | change directory |
| Action | move into another directory |
| Arguments | path or none |
| Notes | If a directory name or a complex path is provided, opens the specified directory. If used without arguments, returns to the home directory |
| Example: return to home | cd |
| Example: move to parent directory | cd .. |
| Example: open directory my_directory | cd my_directory |
| Example: open a path | cd /usr/bin |
mv command
| Mnemonic | move |
| Action | move or rename file |
| Arguments | (1) old path, (2) new path |
| Notes | If you want to move a file from some source directory to the current directory, the second argument should be a dot . |
| Example: rename file my_file.txt | mv my_file.txt readme.txt |
| Example: move file my_file.txt and rename it | mv ~/Downloads/my_file.txt readme.txt |
| Example: move file my_file.txt without renaming it | mv ~/Downloads/my_file.txt . |
cp command
| Mnemonic | copy |
| Action | copy file |
| Arguments | (1) old path, (2) new path |
| Example: copy file my_file.txt to readme.txt | mv my_file.txt readme.txt |
rm command
| Mnemonic | remove |
| Action | delete file |
| Arguments | path |
| Example: delete file my_file.txt | rm my_file.txt |
| Example: delete directory my_directory | rm -r my_directory |
mkdir command
| Mnemonic | make directory |
| Action | create directory |
| Arguments | path |
| Example: create directory my_directory | mkdir my_directory |
cat command
| Mnemonic | concatenate |
| Action | print the contents of the file |
| Arguments | path |
| Example: print file my_file.txt | cat my_file.txt |
wget command
| Mnemonic | web get |
| Action | download file from internet |
| Arguments | URL |
| Example: download the source code of soot-dem | wget https://github.com/egor-demidov/soot-dem/archive/refs/tags/v2.tar.gz |
There are many more essential and convenience commands in linux, but these will help you get started. Refer to web resources for other commands.
Some useful commands to learn next would be the archive manipulation commands (zip, unzip, tar) or text manipulation commands
(vi, nano).
Copying files to/from the cluster
To copy files to/from the remote computer, the scp command needs to be used from the client device. The scp command is similar
in syntax to cp, except it can copy files over the internet using the ssh protocol. If the source or destination path
is on the remote host, it needs to be prefixed with <username>@<hostname>:.
For example to copy the file my_file.txt from the cluster, I would use:
scp <username>@<hostname>:/home/egor/my_file.txt .To copy a local file my_file.txt from my device to the cluster I would use:
scp my_file.txt <username>@<hostname>:/home/egor/my_file.txtAlternatively, Windows users can use MobaXterm, which is a program that combines an ssh-capable terminal for logging into a remote machine and a graphical interface for file transfer between the remote and the client. Thus, you can just use the drag and drop semantics to exchange files with the remote machine. MobaXterm would need to be configured to log in using your private key instead of password-based authentication, which is enabled by default.