How is the climatology formed for thresholds and metrics? - NOAA-OWP/wres GitHub Wiki

There are two separate uses of climatology within WRES:

  1. To compute climatological quantiles that serve as evaluation thresholds; and
  2. As a baseline or reference for skill score calculations.

In both cases, the climatology is formed at the same temporal scale, and in the same measurement units, as the evaluation pairs.

How is the climatology formed when computing quantile-based thresholds?

The climatological dataset employed to compute quantile-based thresholds (i.e., the real values associated with probability_thresholds), is the full observed dataset at the evaluation time_scale and in the evaluation measurement units. No other filters are applied. For example, when an evaluation is conducted for daily accumulations, the climatological data will be formed with daily accumulations (and will be upscaled to that time-scale, as needed).

How is the climatology formed when computing skill-score metrics?

When computing skill scores, the baseline may be declared explicitly and it may represent a climatological data source, which is fully controlled by the user. For some skill scores, such as the Nash Sutcliffe Efficiency and the Brier Skill Score, a default climatological baseline is assumed when no explicit baseline is declared. In this case, the climatological data originates from the observed source after pairing. For example, the baseline used in the Nash Sutcliffe Efficiency is the mean of the paired observed data.