How Climatology is Formed for Thresholds and Metrics - NOAA-OWP/wres GitHub Wiki
There are two separate uses of climatology within WRES:
- to compute climatological quantiles that serve as evaluation thresholds; and
- as baseline or reference for skill score calculations.
The latter is split into two further categories:
a. an explicit climatological baseline supplied by a user; and
b. a default climatological baseline that is integral to a particular skill score, such as the Brier skill-score, which is applied when no explicit baseline
is supplied by the user.
In all cases, the climatological data is at the same temporal scale, and in the same measurement units as the evaluation pairs.
How is the climatology for computing quantile-based thresholds formed?
The climatological dataset employed to compute quantile-based thresholds (i.e., probability_thresholds
), is the full observed
dataset at the evaluation time_scale
and in the evaluation measurement units
. No other filters are applied. For example, if you're performing an evaluation at a daily time-scale, then the climatology will have that time-scale, too (and will be upscaled to that time-scale, as needed).
How is the climatology for computing skill-score metrics formed?
There are some evaluation metrics that use a climatological baseline by default, which is mostly consistent with the above definition for thresholds, but does vary on some details. Notably, the metrics that have a default climatological baseline all use the paired observations, i.e., the evaluation pairs formed using observed
and predicted
(and optionally baseline
if declared using cross pairing). That may be a subset of the overall observed
dataset. There are also some small technical differences when compared with thresholds. For example, the climatological baseline for the Brier skill-score is the average over all observed outcomes/indicators, which is a (climatological) probability of the event being measured.
Climatology for computing skill-score metrics serves as a default for those metrics and is employed only when an explicit baseline
is not declared.