Creative Inquiry

Logo

The hub for CUFCTL creative inquiries.

View the Project on GitHub CUFCTL/mlbd

Palmetto Cluster

The Palmetto Cluster is Clemson’s high-performance computing cluster. In our lab we use Palmetto 2 extensively to run experiments that we can’t run on our own machines. Full documentation on Palmetto 2 can be found here. There is also a YouTube tutorial for Palmetto created by Clemson CITI.

Accessing Palmetto via Browser

The easiest way to access Palmetto is through Palmetto 2 Open OnDemand. Once you sign in with your Clemson username and complete DUO authentication, you can do anything you would do in a terminal, and much more.

Accessing Palmetto via SSH

Once you have an account, you can access Palmetto through SSH with your Clemson username and password:

ssh <username>@slogin.palmetto.clemson.edu

You will also be prompted to complete DUO authentication.

NOTE: The login node should not be used for compute-heavy tasks, and your home directory has limited storage space. Read the Jobs section to learn how to use compute nodes, and read the Data Storage section to learn how to use the scratch directory.

Jobs

When you log into Palmetto, you will be on the “login node”, which should be used only for simple tasks such as moving files around, editing files, and submitting or monitoring jobs. For more compute-intensive tasks such as building software (e.g. running make) or training a machine learning model, you need to first log in to a compute node, which can be done by submitting jobs. If you try to run something compute-intensive on the login node, it may be terminated without notice.

Interactive jobs

The easiest way to log in to a compute node is to run salloc. When the node is ready, your shell prompt will change to something like nodeXXXX and you will be able to run whatever you want. For example, the following requests one CPU core, 1 GB of memory, and 30 minutes of wall time:

salloc --cpus-per-task 1 --mem 1gb --time 00:30:00

You can adjust this syntax to request the resources that you need. Below is a useful alias to append to your .bashrc for logging in to a node with a GPU:

alias gpu='salloc --cpus-per-task 8 --gpus 1 --mem 16gb --time 12:00:00'

This requests any available GPU. You should only request a particular GPU model if your software requires it, because doing so may increase the amount of time your job spends waiting in the queue.

Interactive jobs should not be left idle. When you are finished, run exit to end the job and return the resources to the cluster.

Batch jobs

You can also run sbatch example.slurm, where example.slurm is a shell script with a few Slurm directives at the top to specify the resources, like so:

#!/bin/bash

#SBATCH --job-name example
#SBATCH --cpus-per-task 1
#SBATCH --mem 2gb
#SBATCH --time 00:10:00

pwd
env
module list

You can use squeue to view the status of your jobs, like so:

squeue --me

More information can be found in the Palmetto job-management documentation.

Data Storage

When you are logged in, your default directory is your home directory /home/$USER. You can use this directory for long-term storage. Palmetto also provides temporary scratch storage at /scratch/$USER.

The downside of the scratch directory is that it will not keep your data permanently. Files that have not been accessed, modified, or had their metadata changed within the last 30 days are deleted automatically. Scratch data is not backed up and cannot be recovered after it is deleted. Therefore, you should try to keep things like code, input data, and results in your home directory or project storage, while using the scratch directory to store temporary and intermediate data.

On Palmetto 2, /home, /scratch, and /project are on the same Indigo file system, so there is no general performance difference between them. Compute nodes also provide /local_scratch, which is generally the fastest option because it is stored directly on the node. However, /local_scratch is temporary, is only available during an active job, and cannot be shared between multiple nodes.

Below is an alias that is useful to have in your .bashrc. This script is in your home directory, and it is run when you log in, so this kind of setup gives you a shortcut when working with the scratch directory.

alias scratch="cd /scratch/$USER/"

You can check your home and shared-scratch usage from the login node with:

checkquota

Modules

Since you can’t install system packages on Palmetto through apt or yum, many software packages are provided as modules. Below are some simple commands to get you started:

These commands are typically very fast because they only update a few environment variables; the software packages themselves are already installed.

Module versions change over time, so use module spider to find the versions that are currently available before adding module commands to your .bashrc. For example:

module spider anaconda3
module spider cuda

You could then load the versions required for your work:

module purge
module load anaconda3
module load cuda

Be careful about loading modules automatically in .bashrc. A module needed for one workflow may conflict with another, and optimized modules may depend on the hardware and compiler environment of the compute node. Loading the required modules inside each batch script is often more reproducible.

Transferring Data

You can use scp to transfer data from the command line through one of Palmetto’s dedicated data-transfer nodes:

scp [-r] <username>@hpcdtn01.rcd.clemson.edu:<remote-path> <local-path>

scp [-r] <local-path> <username>@hpcdtn01.rcd.clemson.edu:<remote-path>

A second data-transfer node is also available:

hpcdtn02.rcd.clemson.edu

These commands should be run from a terminal on your local computer, not from a Palmetto login node.

To manage file transfers from a GUI, you can use an SFTP client such as FileZilla and connect to one of the data-transfer nodes. You can also upload and download files through the Open OnDemand Files dashboard. Open OnDemand browser uploads are limited to 50 MB total, so scp or Globus should be used for larger transfers.