CROW workflow definition explained - TerrenceMcGuinness-NOAA/global-workflow GitHub Wiki

TerrenceMcGuinness-NOAA: @workspace Trying really hard and going over all aspects of this system how would specific workflows be defined?

Collecting workspace information

To define triggers in your YAML configuration file for ecFlow that are combined using an OR operator instead of the default AND, you need to include the operand: OR key at the same indentation level as the triggers list. This tells the workflow generator to combine the trigger conditions using logical OR.

Here's how you can do it:

suites:
  prod00:
    nodes:
      gfs:
        tasks:
          jgfs_forecast:
            triggers:
            - task: jgfs_atmos_analysis
              event: release_fcst
            - task: jgfs_wave_prep
              state: complete
            operand: OR
        atmos:
          analysis:
            tasks:
              jgfs_atmos_analysis:
                events:
                  - release_fcst
        wave:
          prep:
            tasks:
              jgfs_wave_prep:

Explanation:

• The triggers: key is followed by a list of trigger conditions.
• Each trigger condition is specified as a dictionary with keys like

task:

• By default, multiple triggers are combined using logical AND. To change this behavior to logical OR, you need to add operand: OR at the same indentation level as triggers:.
• The operand: OR tells the workflow generator that any one of the trigger conditions being true will satisfy the trigger for the task.

Resulting ecFlow Definition:

#5.8.1
suite prod00
  edit ECF_HOME '/path/to/your/ecf/home'
  edit ECF_INCLUDE '/path/to/your/ecf/include'
  edit ACCOUNT 'your_account'
  edit QUEUE 'your_queue'
  edit MACHINE 'your_machine'
  edit RUN_ENVIR 'your_run_environment'
  family gfs
    task jgfs_forecast
      trigger ( /prod00/gfs/atmos/analysis/jgfs_atmos_analysis:release_fcst or /prod00/gfs/wave/prep/jgfs_wave_prep == complete )
    endtask
    family atmos
      family analysis
        task jgfs_atmos_analysis
          event release_fcst
        endtask
      endfamily
    endfamily
    family wave
      family prep
        task jgfs_wave_prep
        endtask
      endfamily
    endfamily
  endfamily
endsuite
# enddef

In the ecFlow definition:

• The trigger for jgfs_forecast is combined using or:

  trigger ( /prod00/gfs/atmos/analysis/jgfs_atmos_analysis:release_fcst or /prod00/gfs/wave/prep/jgfs_wave_prep == complete )
  

• This means that jgfs_forecast will start when either jgfs_atmos_analysis fires the release_fcst event or jgfs_wave_prep reaches the complete state.

Important Notes:

• Indentation Matters: In YAML, the indentation level is significant. Make sure that operand: OR is at the same indentation level as triggers: so that it's correctly associated with the triggers list.
• Triggers List: Each trigger condition under triggers: is an item in a list (-). This list can include conditions based on tasks, families, events, and states.
• Events in Triggers: When using an event in a trigger, ensure that the event is defined in the corresponding task using the events: key.

Example with Additional Details:

suites:
  prod00:
    nodes:
      gfs:
        tasks:
          jgfs_forecast:
            triggers:
            - task: jgfs_atmos_analysis
              event: release_fcst
            - task: jgfs_wave_prep
              state: complete
            operand: OR
            # You can add other task configurations here
        atmos:
          analysis:
            tasks:
              jgfs_atmos_analysis:
                events:
                  - release_fcst
                # Additional task configurations can go here
        wave:
          prep:
            tasks:
              jgfs_wave_prep:
                # Task configurations for jgfs_wave_prep

Summary:

• Use operand: OR under the triggers: section to combine multiple trigger conditions with a logical OR.
• Place operand: OR at the same indentation level as triggers:.
• Ensure that each trigger condition is properly defined with required keys (
• Define events in tasks using the events: key to reference them in triggers.

By following this structure, you can effectively control the execution flow of your tasks in ecFlow, making tasks dependent on multiple conditions combined with logical OR.

Let me know if you have any more questions or need further clarification!