๐Ÿš€Additional AI Resources - DrAlzahrani/HPC-AI-Resources GitHub Wiki

๐Ÿš€ Expanse HPC Jupyter Notebook Workshop Guide ๐Ÿš€

๐Ÿ‘‹ Welcome to Your Supercomputing Journey! ๐Ÿ‘‹

This fun guide will help you access powerful Jupyter Notebooks on SDSC's Expanse supercomputer. Let's get computing! โœจ

๐Ÿ”‘ Step 1: Portal Login ๐Ÿ”‘

  1. ๐ŸŒ Open your browser and go to: https://portal.expanse.sdsc.edu/
  2. ๐Ÿ” Log in with your credentials
    • You'll see the Expanse Portal with the red SDSC logo and navigation menu
    • You should be logged in as your username (shown in top right corner)

๐Ÿ’ป Step 2: Launch Shell Access ๐Ÿ’ป

  1. ๐Ÿ‘€ Find and click on the "expanse Shell Access" app
    • Look in the Pinned Apps section with the terminal ">" icon
    • It's in the first row of apps, on the right side

๐Ÿ› ๏ธ Step 3: Set Up Your Environment ๐Ÿ› ๏ธ

  1. ๐Ÿงฉ Add Galyleo to your PATH:

    export PATH="/cm/shared/apps/sdsc/galyleo:${PATH}"

  2. ๐Ÿ” Check your account information:

    expanse-client user -r expanse

  3. ๐Ÿ“‹ You'll see a table like this:

    โ•ญโ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚   โ”‚ NAME  โ”‚ STATE โ”‚ PROJECT โ”‚  TG PROJECT  โ”‚ USED โ”‚ AVAILABLE โ”‚ USED BY PROJECT โ”‚
โ”œโ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ 1 โ”‚ usr42 โ”‚ allow โ”‚ cs270   โ”‚ TG-ABC123456 โ”‚  10  โ”‚    3000   โ”‚        25       โ”‚
โ•ฐโ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ

  4. โœ๏ธ Note your PROJECT value (e.g., "cs270" in this example)

๐Ÿš€ Step 4: Launch Your Notebook ๐Ÿš€

  1. ๐Ÿ”ฅ Launch Jupyter with your project information:

    galyleo launch --account cs270 --partition shared --cpus 2 --memory 4 --time-limit 00:30:00 --env-modules cpu/0.17.3b,anaconda3/2021.05
    • โš ๏ธ Replace cs270 with YOUR project ID!
    • This config gives you:
      • ๐Ÿง  2 CPU cores
      • ๐Ÿ’พ 4GB of memory
      • โฑ๏ธ 30 minutes of compute time
      • ๐Ÿ“ฆ CPU and Anaconda modules

  2. โณ Wait for the system to process (be patient!)

๐ŸŒ Step 5: Connect to Jupyter ๐ŸŒ

  1. ๐ŸŽ‰ When successful, you'll see a message: "Success! Token linked to jobid."

  2. ๐Ÿ“ Look for the HTTPS URL that looks like:

    https://mountain-sunrise-butterfly.expanse-user-content.sdsc.edu/?token=b42f8d71c9e3516ab723490c8d51f3e9

  3. ๐Ÿ–ฑ๏ธ Copy the ENTIRE URL (this is your secure access key!)

  4. ๐Ÿ”— Paste it into a new browser tab

  5. ๐ŸŽŠ Voilร ! Your Jupyter Notebook interface will load when resources are ready

โš ๏ธ Important Tips โš ๏ธ

  • ๐Ÿ”’ Never share your URL/token - it's your private key!
  • โฐ Your session will end after the time limit (30 min in our example)
  • ๐Ÿ’พ Files save to your home directory by default
  • ๐Ÿ›‘ To end early: use "File" โ†’ "Shut Down" or cancel the job in terminal

๐Ÿ†˜ Troubleshooting ๐Ÿ†˜

If something goes wrong:

  • ๐Ÿงพ Check your account allocation (is it active?)
  • ๐Ÿ”ข Verify you used the correct project ID
  • ๐Ÿ”„ Try a different partition if resources are unavailable
  • ๐Ÿ“ž Still stuck? Contact SDSC support!

๐ŸŽ“ Happy Computing! ๐ŸŽ“

Remember, with great computing power comes great responsibility. Use your HPC time wisely! ๐Ÿง™โ€โ™‚๏ธ