Open OnDemand - uwsph/hpcusers GitHub Wiki

Open OnDemand is a web interface for accessing high performance computing environments. From the comfort of your web browser, you can manage your files, check resource usage, access a shell, and even launch browser-based applications. In our deployment, we currently offer a Linux virtual desktop, Jupyter Notebook, and R Studio Server.

Click Here to access OnDemand (Husky OnNet required).

Features

File Manager (Files)

The integrated file manager allows you to transfer files, change permissions, edit text documents, and much more. In addition, the system supports working with remotely hosted files, through an integration with rclone.

Remote File Services

In order to access remote file systems / services from within OnDemand, you must first add the service to "rclone".

  1. Login to the cluster via SSH or using the Shell Access feature
  2. Add the remote system to rclone
rclone config
  1. This will bring you into a text menu. rclone supports a wide variety of services. You'll find details on how to configure each of them, in their setup guide.
  2. Once configured, you'll see the remote system listed within the Files menu. Select it, to access that system.

Interactive Apps

Directly from your browser, you can launch a virtual desktop, R Studio Server, and Jupyter Notebook. After launching an interactive app, you'll find the running session under "My Interactive Sessions". If needed, you can close your browser window and resume right where you left off (unless the session has reached the maximum runtime requested).

R Studio Server

Our R Studio Server application uses containers to package R, R Studio, and a base set of packages. The application is designed to use containers from the Rocker Project, or containers based on theirs. In other words, you can take one of their containers, customize it for your needs, and use it with OnDemand. And, if you place that custom container in your group's shared folder, then everyone within your group can use the same environment.

For example, to use the latest Rocker Project produced Docker container:

  1. Login to the cluster via SSH or Shell Access
  2. Request an interactive session (you'll need to adapt the account and partition to those you have access to)
salloc -A sph -p 12c128g -N 1 -c 1 --mem 4G -t 1:00:00
  1. Pull the latest container
apptainer pull docker://rocker/verse:latest
  1. Login to OnDemand
  2. Select R Studio Server, and enter the path to your new container

Jupyter Notebook

The Jupyter Notebook integration uses containers to package Python, Jupyter, and a base set of packages. Our application is designed to use containers from Anaconda, or containers based on theirs. Thus, you can take one of their containers, customize it, and use it with OnDemand. Placing the container within your group's shared directory, will enable everyone in your group to share the same environment.

For example, to use the latest Anaconda produced Docker container:

  1. Login to the cluster via SSH or Shell Access
  2. Request an interactive session (you'll need to adapt the account and partition to those you have access to)
salloc -A sph -p 12c128g -N 1 -c 1 --mem 4G -t 1:00:00
  1. Pull the latest container
apptainer pull docker://continuumio/anaconda3:latest
  1. Login to OnDemand
  2. Select Jupyter Notebook, and enter the path to your new container

Shell Access (Clusters > Shell Access)

Shell Access is a browser based SSH client. You can either access it from the Clusters menu, or from within Files. When accessed from Files, the session will place you in the directory you have navigated to.

Jobs

The Jobs menu has tools for monitoring your jobs, and creating new batch jobs from templates.

Active Jobs

As the name implies, the Active Jobs feature shows you the active jobs on the cluster. This includes jobs that are waiting to run, and those that are currently running. In either case, you can view the requested resources, and if the job is running, the resource utilization.

Job Composer

Using Job Composer, you can create a batch job using your browser. The composer supports creating templates, simplifying the process of running a similar job in the future. The first time you run the tool, it'll provide an overview of the user interface.