Getting Started with HiPerGator

Software tutorials of interest.
Post Reply
Message
Author
jstenner
Site Admin
Posts: 87
Joined: Wed Jan 27, 2010 7:33 pm

Getting Started with HiPerGator

#1 Post by jstenner »

hipergator.01.jpg

BASICS

Alright, we've worked through basic principles of AI using P5.js and ML5.js. Now let's work with AI using some tools that allow us to interact with the outstanding resources available via: UF's HiPerGator AI.

Python is commonly used in AI work and many resources are available online. A common technique is to prototype and develop an AI application using a development tool like Google Collab or Jupyter. Once the app is designed, tested, and optimized, it is typically run as a Python program in production. Many of the programs with which we'll experiment are intended to run on Collab or via a local installation. To get them to work on HiPerGator we'll need to make a few adjustments.

Since HiPerGator is a cluster of computing resources, it manages the distribution of work to the various nodes using the SLURM workload management system. If you are reading this and you're in the Art + Technology program, you can think of SLURM as something similar to our own Thinkbox Deadline renderfarm submission system, but SLURM is more commonly available in the world of high-performance computing systems. HiPerGator provides resources for using Jupyter, which we'll focus on below. You should recognize that you can install and run Jupyter, locally, on your personal computer; you just won't have ready access to all the CPUs and GPUs available via HiPerGator. Fortunately, you can remotely access HiPerGator from your personal computer and we'll show you how to do that as well.

REFERENCE

Pre-recorded Training
Jupyter Notebooks
AI HELP
TensorFlow
Modules (see also Basic Usage)
Locally available Datasets (i.e. on the HiPerGator system)

CONFIG

Let's log into UF HiPerGator using the Secure Shell protocol (SSH). Substitute your gatorlink username for <gatorusername> below:

Code: Select all

ssh <gatorusername>@hpg.rc.ufl.edu
If you've set up your computer using my instructions via Zsh Setup, you can log into UF HiPerGator via SSH using our previously created alias, "hgator" (assuming that's what you called yours...check your .zshrc or run "alias" to see all of your aliases if unsure):

Code: Select all

hgator
Alright, so we're logged into HiPerGator and we are placed in our home directory (~) on one of the login nodes.

The default shell on the Linux login nodes on HiPerGator is BASH. Let's set up a basic shell configuration by adding some things to your ".bash_profile" using the "nano" text editor.

Code: Select all

nano .bash_profile
These are a good starting point if you haven't already configured your HiPerGator shell config. Add the following after any existing lines in this file. At the least, ensure the "umask 007" line is included!

Code: Select all

EDITOR=nano

#make data accessible to all group members, but not to the users outside of the group
umask 007

#configure the command prompt
export PS1='\[\033[32m\][\u@\h]\[\033[34m\](\w)\$ \[\033[00m\]'

#modify command-line-completion
bind 'set completion-ignore-case on'
bind 'set print-completions-horizontaly on'
bind 'set show-all-if-ambiguous on'

# ls with color, human readable file sizes, reversed, etc.
alias lr='ls -hartl'
alias ll='ls -halG'
alias l='ls -G'

# shorter cd ..
alias .='cd ..'

# nano without wrapping, which is essential for editing config files
alias edit='nano -w'
Now we need to save these edits by writing out the changes. Type Control + O, press Enter, then Control + X to exit.

To make the changes register with the shell we need to tell the shell to re-read the configuration. Do that by typing:

Code: Select all

source .bash_profile

Next, recall in your HPC New User training, they want us to store our working files on the "blue" storage system rather than our home area or "orange," right? Also recall that these storage locations are not available to us (i.e. mounted) until we first access them by navigating to them. To make this easier for ourselves in the future, we're going to create a shortcut in our home area that points to our most commonly used storage directory; we're going to create a symbolic link (like an alias) to our class directory on the HiPerGator "blue" drive. Any time you log in, that's where we want to do most of our daily work.

First, we'll need to "mount" our "blue" storage system. Substitute your groupname (art4630, cota-dept, <username>, etc.) for "art4659" in the command below:

Code: Select all

cd /blue/art4659
Now that we've mounted the blue file system, return to your home directory so we can create the symbolic link:

Code: Select all

cd ~
A symbolic link is created using the "ln -s" command followed by the path to the directory of interest, followed by the name of the link you want to create:

Code: Select all

ln -s /blue/art4659/ blue_ai
After you've done that, list the files in your home directory and see how it represents a symbolic link...see the arrow?

Code: Select all

ls -al

Code: Select all

blue_ai -> /blue/art4659/
Let's use our symbolic link to navigate to our class directory. If you've enabled autocompletion in your shell configuration you can hit <tab> after typing a few letters of "blue" and it will autocomplete the full name for you:

Code: Select all

cd blue_ai

Code: Select all

ls -al
Notice there is a list of directories named after each class member. Permissions are set so each of these directories are private. Notice also, there is a directory called "shared." Files placed in this directory will allow us to work on them together because they are assigned our shared groupname, "art4659." You should create a shared directory using your username within the "shared" directory. Once you've created this working directory, navigate to it. So, here's an example:

Code: Select all

cd shared
mkdir <gatorusername>
cd <gatorusername>
mkdir  code (or whatever you want to call it)
cd code
From now on, any code or notebooks you want to use should be placed in this directory. We'll work with GitHub to grab sources as we get further along.

Configure Anaconda:

You'll probably want to use Anaconda to manage working environments for projects. You'll use HPC's module system to load Anaconda, but you should edit the .condarc file to store your packages and env files on "blue" instead of your home area, which will fill up quickly. If you've never loaded the Anaconda module, load it for the first time so it will create a default .condarc file for us to edit.

Code: Select all

module load conda
Notice that there is now a .condarc file in your home directory. Edit the file to point to the proper locations for your environment and package directories. The "${USER}" portion of the path is a shell parameter expansion call that will insert your username in the path.

Here's an example for my ART 4659 class:

Code: Select all

envs_dirs:
  - /blue/art4659/${USER}/conda/envs
pkgs_dirs:
  - /blue/art4659/${USER}/conda/pkgs
Here's an example for my COTA research allocation:

Code: Select all

envs_dirs:
  - /blue/cota-dept/saah/${USER}/conda/envs
pkgs_dirs:
  - /blue/cota-dept/saah/${USER}/conda/pkgs
The master of disaster!

jstenner
Site Admin
Posts: 87
Joined: Wed Jan 27, 2010 7:33 pm

Getting Started with HiPerGator: Open on Demand (OOD)

#2 Post by jstenner »

INTRO

HiPerGator uses what is called a "module" system. You can load pre-installed modules rather than install them into your working directory environment. They have also created task-specific "kernels" designed for various types of work using Jupyter Lab.

There are several different ways to use Jupyter that are available to us. Perhaps the easiest is to use Open on Demand (OOD) via a web browser. Another way is to launch a Jupyter notebook via the command line interface (CLI or terminal) and create a secure shell (SSH) tunnel to the running notebook. Yet another way is to use OOD and launch a virtual "desktop" environment to run your notebook.

OOD

From on-campus or after starting a VPN connection from home, access Open on Demand (OOD) in your web browser:

https://ood.rc.ufl.edu
OOD1-Screen Shot 2023-02-27 at 11.54.43 AM.png
Under the "Interactive Apps" menu choose "Jupyter Notebook."
OOD2-Screen Shot 2023-02-27 at 11.55.15 AM.png

For SLURM account and QoS enter our class group name: "art4659" (or whatever it happens to be). If you're wanting to work using your COTA account, you'd substitute "cota-dept" for example. Note also how you'd specify particular modules that should be available as well as how you'd select a specific GPU type.
OOD3-Screen Shot 2023-02-27 at 12.00.14 PM.png


Creating Your Own Kernel

A lot of people prefer to use Jupyter Notebooks as opposed to the command-line. A key issue is how to configure your "kernel" in Jupyter and make it available to your notebook. Here's a nice tutorial describing how:

Using Virtual Environments in Jupyter Notebook and Python

Assuming you're logged into HiPerGator:

Code: Select all

module load conda
Create a conda environment, giving it whatever name makes sense for you, and whatever minimum Python version makes sense for your project:

Code: Select all

conda create -n myenv python=3.8
Enter the environment:

Code: Select all

conda activate myenv
Add ipykernel to your working environment:

Code: Select all

pip install --user ipykernel

Code: Select all

python -m ipykernel install --user --name=myenv
Now, if you've logged into jupyterlab or ood.rc.ufl.edu you can load the kernel name into your notebook and you're good to go!
The master of disaster!

jstenner
Site Admin
Posts: 87
Joined: Wed Jan 27, 2010 7:33 pm

Getting Started with HiPerGator: Jupyter Standalone with SSH Tunnel

#3 Post by jstenner »

Reserved for instructions on how to access Jupyter via a SSH Tunnel to HiPerGator.
In the meantime, info can be found here:
https://help.rc.ufl.edu/doc/Jupyter_Notebooks
The master of disaster!

Post Reply