Hands-On: Cluster Setup & File Transfer

Check that your private key is readable only by you. Open the section for your platform:

Linux / macOS / WSL

Check the permissions of ~/.ssh/ and your private key files:

ls -ld ~/.ssh
ls -l ~/.ssh/id_*

The directory must be drwx------ (700), private keys must be -rw------- (600). Fix any that are not with chmod 700 ~/.ssh or chmod 600 ~/.ssh/<key>.

Public keys (*.pub) being world-readable is fine and expected.

Windows (OpenSSH)

Windows uses access control lists (ACLs) instead of POSIX modes. Right-click your private key file → Properties → Security → Advanced. Make sure only your user account has access; remove inherited permissions and any other entries. Alternatively, run in PowerShell:

icacls "$env:USERPROFILE\.ssh\<key>" /inheritance:r
icacls "$env:USERPROFILE\.ssh\<key>" /grant:r "${env:USERNAME}:F"

If you're inside a Command Prompt, use:

icacls "%USERPROFILE%\.ssh\<key>" /inheritance:r
icacls "%USERPROFILE%\.ssh\<key>" /grant:r "%USERNAME%:F"

If OpenSSH refuses your key with "permissions are too open", these commands fix it.

Windows (MobaXterm)

MobaXterm does not impose a single location for your key. Each saved session records the path to the private key it uses. Usually that is an ordinary Windows file you pointed it at, for example C:\Users\<you>\.ssh\<key>, though it can also live inside MobaXterm's own home directory (shown in Settings → General as the Terminal home). Because the key is a Windows file, running chmod in the MobaXterm terminal is not the reliable protection it is on Linux: what governs access is the Windows file and its permissions (ACLs).

• Find the key path: right-click your session → Edit session → Advanced SSH settings → Use private key.
• Protect that file with the same icacls commands, run in PowerShell (not the MobaXterm terminal), using the path from your session:

  icacls "C:\path\to\your\key" /inheritance:r
  icacls "C:\path\to\your\key" /grant:r "${env:USERNAME}:F"

  If the key is under your user profile (C:\Users\<you>\...), default Windows permissions already restrict it to your account.
• Wherever the key lives, never keep it on a USB stick, a network share, or a cloud-synced folder (OneDrive, Dropbox), and never share your MobaXterm.ini or data folder, which also holds your saved sessions and any stored passwords.

Connect to Hunter using the SSH alias from your ~/.ssh/config:

ssh hunter

If you have not set up the alias yet, see the SSH config slide from day 1. If you are using MobaXterm, double click your connection.

To avoid retyping your passphrase for every connection in this session, add the key to your local SSH agent.

Linux / macOS:

ssh-add ~/.ssh/<your_key_filename>

The agent forgets the key when you log out or close the terminal session, so you will need to re-add it next time.

Windows (OpenSSH): the SSH agent runs as a Windows service that is disabled by default. Enable it once (PowerShell as Administrator):

Set-Service ssh-agent -StartupType Automatic
Start-Service ssh-agent

Then add your key (regular PowerShell):

ssh-add $env:USERPROFILE\.ssh\<your_key_filename>

Unlike on Linux, the Windows agent persists keys across reboots once added.

Windows (MobaXterm): MobaXterm includes MobAgent, its built-in SSH agent. Enable it in Settings → SSH → "Use internal SSH agent (MobAgent)" and add your private key in the same dialog. The agent is active whenever MobaXterm is running.

On Hunter, check the permissions of your home and SSH directories:

ls -ld ~ ~/.ssh

Both should be drwx------ (700). If ~/.ssh is too permissive, SSH key-based login will silently fail next time.

Confirm where you are and check your home directory quota:

pwd
quota -s

What is your current disk usage? How close are you to the limit?

List existing workspaces with ws_list. If this is your first day on Hunter, the output is likely empty.

Allocate a new workspace for this course:

ws_allocate course-handson 7

The 7 is the lifetime in days (maximum is 60). Note the path that ws_allocate prints. This is where your workspace lives on the parallel filesystem.

If you skip the lifetime, you get the default. Use ws_allocate --help to see all options.

Verify the workspace exists and write a small test file into it:

ws_list
cd $(ws_find course-handson)
echo "Hello workspace" > test.txt
ls -l

ws_find <name> prints the workspace path, which is convenient for shell substitution.

Workspaces have limited lifetimes. Extend yours to 14 days:

ws_extend course-handson 14

The number is the new total lifetime, not an increment. There is also a limit on how often you may extend; see ws_extend --help.

For convenience, create a symbolic link from your home directory to the workspace so you can reach it without typing the long path:

ln -s $(ws_find course-handson) ~/work
ls -l ~/work
ls ~/work/

The first ls -l shows the symlink itself; the second lists the contents of the workspace through the symlink.

Compare the size of your home and your workspace:

du -sh ~ ~/work/

Workspaces are sized for working data; home is for configuration and small files.

Where would you put it? Decide whether each of the following belongs in $HOME or in a workspace:

• Source code for a simulation you are developing
• 50 GB of intermediate output from a parameter sweep
• Your ~/.bashrc and shell aliases
• A 2 GB Docker image you transferred from your laptop
• Long-term archival of finished project results

$HOME: small, slow, no backup. Workspace: large, fast, #files is limited, no backup, expires. Long-term archival: neither, that is what HPSS is for.

On your local machine, copy a small file to Hunter with scp (for copying entire folders, use scp -r):

echo "test" > /tmp/test.txt
scp /tmp/test.txt hunter:work/

Confirm on Hunter that the file arrived:

ls ~/work/test.txt

Now do the same with rsync, which is more efficient for repeated transfers and works recursively on directories by default:

mkdir /tmp/my_dir
echo "a" > /tmp/my_dir/a.txt
echo "b" > /tmp/my_dir/b.txt
rsync -av /tmp/my_dir/ hunter:work/my_dir/

Add a third file locally, then run the same rsync command again. Notice that only the new file is transferred.

The trailing slash on the source path matters: my_dir/ copies the contents, my_dir copies the directory itself.

Finally, retrieve a file from Hunter back to your local machine. Use this command to bring results home from a job:

rsync -av hunter:work/test.txt /tmp/retrieved.txt
cat /tmp/retrieved.txt

Open a second terminal on your local machine and start the reverse tunnel. Pick any unused port number (e.g. 12345, use your own random 5-digit number) and use it consistently below:

ssh -R localhost:12345 hunter

Keep this session open because closing it will close the tunnel.

In your original Hunter session, set the proxy environment variables to match the port you chose:

export MY_PROXY_PORT=12345  # ← pick your own 5-digit number
export https_proxy=socks5://localhost:$MY_PROXY_PORT
export http_proxy=$https_proxy

Test that the proxy works:

curl -I https://github.com

You should see HTTP/2 200 or similar. If curl hangs, the tunnel is not connected.

Clone the course example repository into your home directory:

git clone https://github.com/jordan-hlrs/hello-jobstep-hlrs
ls hello-jobstep-hlrs/

You should see several files. We will compile and run this on day 3.

Set up a Python environment for your own packages. The order matters: load the module first, then create the venv with --system-site-packages so you inherit the optimized Hunter packages:

module load cray-python
cd ~   # env in $HOME, not the workspace: it survives workspace expiry
python3 -m venv --system-site-packages my_env
source my_env/bin/activate
python3 -m pip install /sw/general/x86_64/development/python/share/PySocks-1.7.1-py3-none-any.whl

The last line installs PySocks from a local copy on Hunter, which pip needs to talk to the SOCKS5 proxy.

Now install a small package through the proxy and verify it works:

python3 -m pip install requests
python3 -c "import requests; print(requests.__version__)"

If the import succeeds and prints a version number, your environment is ready for actual project work.

Close the tunnel, either by breaking the connection with Ctrl+c or by closing the terminal. Confirm that the tunnel is closed by running:

curl -I --max-time 5 https://github.com

or repeat one of the previous git clone, pip, or curl commands. If you unset the proxy variables:

unset https_proxy http_proxy

then commands relying on internet access will fail fast with a clear error instead of waiting for a timeout. You can also check whether anything is still listening on the tunnel port on hunter:

ss -tlnp | grep $MY_PROXY_PORT        # or: lsof -i :$MY_PROXY_PORT

Either run this in the Hunter shell where you set MY_PROXY_PORT, or export the variable again.