Software: GCC - nthu-ioa/cluster GitHub Wiki

Golden rule

Always module load gcc, especially for compiling intensive code, otherwise you will get the non-optimized system compiler.

Cross-compiling on login nodes

The cluster SOP is to compile on fomalhaut and then run on the compute nodes. It is generally fine to compile code that will be run on the gXX or cXX nodes with --march=native on fomalhaut, as shown above. The memory node has the greatest difference with fomalhaut, but even this will only be noticeable for intensive calculations (benchmarks would be very welcome).

Power users may care about small differences, which correspond to the following choices made by --march=native:

  • Fomalhaut: l2-cache-size=11264, -mno-avx512vnni `
  • c nodes, 6140: l2-cache-size=25344, -mno-avx512vnni
  • c nodes, 6240R: l2-cache-size=36608 -mavx512vnni
  • g nodes: l2-cache-size=25344, -mno-avx512vnni
  • m nodes: l2-cache-size=28160, -mavx512vnni

The vnni refers to Intel's vector neural network extensions for AVX 512.

What do command line flags to gcc expand into?

Use a combination of -Q and --help={topic}. See the manpage for values of {topic}

From the manpage:

If the -Q option appears on the command line before the --help= option,
then the descriptive text displayed by --help= is changed.  Instead of 
describing the displayed options, an indication is given as to whether 
the option is enabled, disabled or set to a specific value (assuming 
that the compiler knows this at the point where the --help= option is 
used).

To check what -march=native actually implies: gcc -march=native -Q --help=target | grep -- '-march=' |cut -f3

On login node: skylake-avx512 On gXX nodes: skylake-avx512 On cXX nodes: skylake-avx512 On mXX nodes: skylake-avx512

Likewise for -mtune: gcc -march=native -Q --help=target | grep -- '-mtune=' |cut -f3

On login node: skylake-avx512 On gXX nodes: skylake-avx512 On cXX nodes: skylake-avx512 On mXX nodes: skylake-avx512

So --march=native gets the right thing in --mtune.