https://jhpce.jhu.edu/

JHPCE is an HPC some of us in the Langmead lab have access to (and others could get access to it if they attend the training and their project warrants it).

JHPCE is collocated along with MARCC at the Johns Hopkins Bayview campus.

It is administrated completely separately from MARCC (other than shared components such as power/network).

To request a membership (if you're in the Langmead lab):

https://jhpce.jhu.edu/register/user/

Support email:

[email protected]

SGE and Hardware Resources

Unlike the way Slurm is typically configured (at least on MARCC and possibly Stampede2), JHPCE uses [Sun] Grid Engine (SGE) to schedule jobs.

The way SGE is configured on JHPCE follows a strict resource allocation system. Meaning that if you request more threads and/or memory than what as been scheduled for your job, your process(es) will fail.

For example multiple of us JHPCE users have noticed that when running STAR on a JHPCE node we can't use the full set of threads we requested (N), so STAR has to be run with N-2 threads.

This likely because STAR starts up more threads than it actually fully uses or is specified by --runThreadN. This is referenced in:

https://github.com/alexdobin/STAR/issues/195

Another aspect of this is memory needed for Singularity containers and building images, documented at the Singularity page:

https://github.com/BenLangmead/jhu-compute/wiki/Singularity

An example SGE qrsh for an interactive session requesting specific resources:

qrsh -pe local 32 -l h_vmem=1.2G,mem_free=1G,h_fsize=100G

This requests 32 CPUs (cores/threads), a total ceiling of 1.2GB * 32 = 38.4 GBs of virtual memory, a machine with at least 1GB * 32= 32 GBs of RES memory free, and says your job can write files up to 100GBs in size.

The -pe local 32 and -l h_vmem=1.2G are the key constraints to which your job will be held.