finnpos–eval - mpsilfve/FinnPos GitHub Wiki

finnpos-eval sys_data gold_data (model)

Evaluate a tagged file against a gold standard file.

The data files sys_data and gold_data need to conform the the FinnPos data format. Additionally, they need to represent the same input data. that is their word forms need to agree.

If a model is provided, finnpos-eval can provide statistics separately for in-vocabuary and out-of-vocabulary words.

$ finnpos-eval ftb.test.sys ftb.test.gold ftb.model
Comparing ftb.test.sys and ftb.test.gold (gold standard).
Label accuracy for all words: 0.932939
Label accuracy for IV words:  0.970814
Label accuracy for OOV words: 0.787924
Lemma accuracy for all words: 0.902333
Lemma accuracy for IV words:  0.989879
Lemma accuracy for OOV words: 0.567137

$ finnpos-eval ftb.test.sys ftb.test.gold
Comparing ftb.test.sys and ftb.test.gold (gold standard).
Label accuracy for all words: 0.932939
Label accuracy for IV words:  -1
Label accuracy for OOV words: -1
Lemma accuracy for all words: 0.902333
Lemma accuracy for IV words:  -1
Lemma accuracy for OOV words: -1
-1 denotes unknown value.