List of Metrics Available - NOAA-OWP/wres GitHub Wiki
The metrics available within WRES are listed below. They are organized by input data type and output data type. Transformations between data types are possible. For example, an ensemble forecast may be transformed into a single-valued forecast by calculating an ensemble average (e.g., mean). Similarly, a single-valued prediction may be transformed into a dichotomous prediction using an event threshold.
When declaring use of the metric in a YAML file, use the name provided in the table below in lower case and replace any dashes, '-', with spaces, ' ', so that, for example, "Time-to-Peak Error" is declared as "time to peak error".
* To calculate the difference between a predicted and baseline dataset for this metric, simply add the word difference to the metric name.
+ When the Mean Square Error Skill Score is declared without an explicit baseline, a default baseline of climatology is used (i.e., the variance of the observations). In this case, the score is equivalent to the Nash-Sutcliffe Efficiency.
ǂ These metrics reveal the paired values directly, rather than summarizing the paired values with statistics. As such, these metrics generate voluminous outputs and should be used with caution, particularly when the evaluation contains a large number of paired values. In such cases, it is not recommended to generate svg graphics, as these graphics are encoded with uncompressed text, which will be extremely large. The spaghetti plot contains the paired values from all ensemble forecast traces, so the outputs associated with this metric are particularly large.
| A | B | C | |
|---|---|---|---|
|
1
|
INPUT TYPE (PAIRED VALUES) | METRIC NAME | OUTPUT TYPE |
|
2
|
SINGLE VALUED | Bias Fraction* | SCORE |
|
3
|
Coefficient of Determination* | ||
|
4
|
Pearson Correlation Coefficient* | ||
|
5
|
Index of Agreement* | ||
|
6
|
Mean Absolute Error* | ||
|
7
|
Mean Error* | ||
|
8
|
Median Error* | ||
|
9
|
Root Mean Square Error* | ||
|
10
|
Root Mean Square Error Normalized* | ||
|
11
|
Sum Square Error* | ||
|
12
|
Volumetric Efficiency* | ||
|
13
|
Sample Size* | ||
|
14
|
Kling Gupta Efficiency* | ||
|
15
|
Mean Square Error* | ||
|
16
|
Mean Square Error Skill Score+ | ||
|
17
|
Quantile-Quantile Diagram | DIAGRAM | |
|
18
|
Box Plot of Errors | ||
|
19
|
Box Plot of Percentage Errors | ||
|
20
|
Scatter plotǂ | ||
|
21
|
SINGLE VALUED TIME-SERIES | Time-To-Peak Error Summary Statistics | SCORE |
|
22
|
Time-To-Peak Relative Error Summary Statistics
|
||
|
23
|
Time-To-Peak Error | PAIRED VALUES | |
|
24
|
Time-To-Peak Relative Error | ||
|
25
|
Time Series Plotǂ | ||
|
26
|
DISCRETE PROBABILITY | Brier Score* | SCORE |
|
27
|
Brier Skill Score | ||
|
28
|
Relative Operating Characteristic Score* | ||
|
29
|
Reliability Diagram | DIAGRAM | |
|
30
|
Relative Operating Characteristic Diagram | ||
|
31
|
DICHOTOMOUS | Critical Success Index* | SCORE |
|
32
|
Threat Score* | ||
|
33
|
Equitable Threat Score* | ||
|
34
|
Frequency Bias* | ||
|
35
|
Peirce Skill Score* | ||
|
36
|
Probability of Detection* | ||
|
37
|
Probability of False Detection* | ||
|
38
|
MULTICATEGORY | Contingency Table (2x2) | TABLE |
|
39
|
Peirce Skill Score | SCORE | |
|
40
|
ENSEMBLE FORECAST | Sample Size | SCORE |
|
41
|
Continuous Ranked Probability Score* | ||
|
42
|
Continuous Ranked Probability Skill Score | ||
|
43
|
Rank Histogram | DIAGRAM | |
|
44
|
Box Plot of Errors by Observed Value | BOXPLOT | |
|
45
|
Box Plot of Errors by Forecast Value | ||
|
46
|
ENSEMBLE FORECAST TIME SERIES | Spaghetti Plotǂ | PAIRED VALUES |