Using OptClim with CESM - optclim/ModelOptimisation GitHub Wiki

Linking CESM into OptClim2

The following are used to connect CESM with the OptClim framework:

  1. OptClim class CESM.py
  • manages the cloning and setting of parameter values
  • sets up the execution of the model

  1. Internal workflow in CESM, https://esmci.github.io/cime/versions/ufs_release_v1.0/html/users_guide/workflows.html

    • This is expressed as SLURM dependencies, with serial or parallel jobs
    • We have a non-standard workflow, that includes a postrun task specific to OptClim.

  2. The base model - that used to make clones - is created from a script including that distinctive workflow's name. ${CIMEROOT}/scripts/create_newcase --case CASENAME --compset F2000climo --res f19_f19_mg17
    --workflow optclim1 --walltime 00:20:00 --project ACCOUNT --queue=short

  3. Cloning the base case to make a new case is simple - a CIME command included in an OptClim script for CESM that manages the setting of parameter values. It avoids the need to rebuild the model.

  4. CESM uses namelists, but rather than editing these directly CESM provides a file into which modifications can be written

The user sets up an OptClim study by defining parameter values, simobs etc as is always needed but other than specifying that CESM is the model to run, there is no added action, so long as the base case is set up as noted above.

An example is on ARCHER2 Directories/files in porting and in initial test:

directory/file purpose
​ /work/n02/shared/mjmn02/cesm/cases/Wa2
/work/n02/shared/mjmn02/OptClim/optclim3/base/CESM/base1 BASE_CASE: Recreation of above, built not run,.This is the initial base case for testing.  This should not be a clone. When clone to make eg. zd001 then subdirectory IN THIS base1 directory hold the run data.

The file "lite_setup_for_cesm" adds the minimum environment definitions, in addition to the generic OptClim setup of environment:

          # define CIME directory and the model name we are using
    export CIMEROOT="/work/n02/shared/mjmn02/cesm/CESM2.1.3/my_cesm_sandbox/cime"
    export CIME_MODEL="cesm"

Adding a new workflow with an associated task:

  • config_workflow: establishes a new CIME workflow, inserting a postrun task between the model completing and archiving
  • template_a2.postjob: causes a script optclim_finished to run in a serial job, and this script was created by optclim in the model's OptClim interface directory, to release the corresponding array job.
cd $CIMEROOT/config/cesm/machines

ln03@ machines> diff config_workflow.xml config_workflow.original_xml 
53,88d52
<  <workflow_jobs id="optclim1">
<     <!-- order matters, with no-batch jobs will be run in the order listed here -->
<    <job name="case.run">
<       <template>template.case.run</template>
<       <prereq>$BUILD_COMPLETE and not $TEST</prereq>
<     </job>
<     <job name="case.test">
<       <template>template.case.test</template>
<       <prereq>$BUILD_COMPLETE and $TEST</prereq>
<     </job>
< 
<     <job name="case.a2_postjob">
<       <template>template.a2_postjob</template>
<       <dependency>case.run or case.test</dependency>
<       <prereq>1</prereq>
<       <runtime_parameters>
<         <task_count>1</task_count>
<         <tasks_per_node>1</tasks_per_node>
<         <walltime>0:20:00</walltime>
<       </runtime_parameters>
<     </job>
< 
<     <job name="case.a2_archive">
<      <!-- If DOUT_S is true and case.run (or case.test) exits successfully then run st_archive-->
<       <template>template.st_archive</template>
<       <dependency>case.a2_postjob</dependency>
<       <prereq>$DOUT_S</prereq>
<       <runtime_parameters>
<         <task_count>1</task_count>
<         <tasks_per_node>1</tasks_per_node>
<         <walltime>0:20:00</walltime>
<       </runtime_parameters>
<     </job>
< 
<   </workflow_jobs>
< 


ln03@ machines> cat template.a2_postjob
#!/bin/bash
# Batch system directives
{{ batchdirectives }}
# added in template file:
#SBATCH --time=00:10:00
#SBATCH --nodes=1
#SBATCH --mem=4G
#SBATCH --tasks-per-node=1
#SBATCH --output={{ job_id }}.%j
#echo Running  $@
pwd
ls

./xmlquery RESUBMIT CONTINUE_RUN STOP_OPTION   STOP_N    RESUBMIT  

# ifESBMIT >0  and CONTINUE_RUN =FALSE and job seems ok then
# CONTINUE_RUN=TRUE and case.submit
# else

OPTFINISHED='optclim_finished'
if [[ -f $OPTFINISHED ]]
then
  ls -l $OPTFINISHED
  bash $OPTFINISHED
else
  echo OPTFINISHED not found: $OPTFINISHED 
fi
⚠️ **GitHub.com Fallback** ⚠️