Tutorial: Parallel Computing - nthu-ioa/cluster GitHub Wiki

:warning: This page is still being written. Please get in touch with the admins if you have any questions.

This page gives a basic introduction to the keywords and concepts involved in using an HPC cluster.

Nodes, CPUs, cores and hyperthreads

A node is a computer (metal box with flashing lights on the front) that is part of a network. Our cluster has 22 nodes. The 17 "cpu" nodes are specifically for for parallel computing.

A CPU is a very small, very expensive block of silicon packed in plastic. Our cluster nodes have 2 CPUs each.

Each CPU consists of many discrete sub-units etched onto the same tiny bit of silicon. Most of the space is taken up with cores. Cores are collections of nano-scale paths and gates that move electrons around in different sequences, which correspond to a set of logical instructions (add, multiply, reverse etc.). Passing numbers (electrons) from the computer's memory through these instructions turns them into other numbers. Programs tell the cores what numbers to pass to which instruction in which order.

A core can do only one job (related set of instructions) at one time. Since our CPUs have 18 (c01-04) or 24 (c05-c17) cores each, they can do 18 or 24 things at the same time.

A technical trick called hyperthreading can turn one real-life, physical core into two `logical' cores. This is not a free lunch, because each logical core is slightly slower than the real, physical core would be on its own. We use hyperthreading on our cluster, so each of our CPUs has 48 (or 36) logical cores. You, the user, almost never need to care that these are logical rather than physical cores.

Each of our nodes has 2 CPUs, so the total number of logical cores per node is 2x48 = 96 (or 2x36 = 72).

The point of telling you about hyperthreading is that should not be confused with the 'threads' and 'multithreading' described below. They are not the same at all. You need to understand threads, but you don't need to understand hyperthreading except possibly to know that it is used on our cluster.

Processes and threads

A process is a single instance of a program being executed on a CPU core. It consists of a set of instructions to the core that have to be executed one after another in a particular order. The program is a recipe, the core is the kitchen, and the process is the chef.

A program can split its work into many sub-processes that do different jobs. These are threads.

A simple program usually has only one thread. A complicated program might have thousands.

The operating system (Linux in our case) takes care of deciding which thread can use each core at any give time. This is called scheduling. It can stop threads executing temporarily, to give some time to other more urgent threads (called context switching).