Test files - adamb924/mortal-engine GitHub Wiki

A test file contains a bunch of tests of your model. When you think about it, there's no point in having a morphological model if you don't know that it works correctly. So the test file is just as important as the morphology file. As I note elsewhere, a smart way to work would be to write the test file first, and then write the morphology file.

The example below is test-gallery.xml from the repository (examples/test-gallery.xml). Different elements of the file are explained using comments. The file examples/03-Optional-Affixes.xml is just chosen as a convenience. Suffice it to say that the language has a bare stem “ata” and a plural suffix “lar” (in Arabic script, “آتا” and “لار”).

<?xml version="1.0" encoding="UTF-8"?>
<!-- <tests> is the root element -->
<tests
    xmlns="https://www.adambaker.org/mortal-engine/tests"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="https://www.adambaker.org/mortal-engine/tests tests.xsd">

    <!-- The <tests> element contains one or more <schema> elements. 
         There is no requirement to use more than one schema. It just
         provides a more organized output if you want that. -->
    <schema label="Just a single schema in this example">

        <!-- The <morphology-file> should be a path to a morphology file.
            This is the morphology that Mortal Engine will use for these tests. -->
        <morphology-file>03-Optional-Affixes.xml</morphology-file>

        <!-- A recognition test is simply whether a form should be 
            accepted by the model or not. -->
        <recognition-test label="Acceptance of a correctly affixed form">
            <input lang="wk-LA">atalar</input>
            <should-be-accepted>yes</should-be-accepted>
        </recognition-test>

        <recognition-test label="Acceptance of a correctly affixed form">
            <input lang="wk-LA">this-is-not-a-good-input</input>
            <should-be-accepted>no</should-be-accepted>
        </recognition-test>

        <!-- The following two shorthands are equivalent to the two previous tests -->
        <accept lang="wk-LA">atalar</accept>
        <reject lang="wk-LA">this-is-not-a-good-input</reject>

        <!-- A parsing test checks whether the model produces correct
            morphological analyses of your input. The <label> elements
            contain the labels of morphemes, which are specified in your
            model. You can include multiple <output> tags if that is correct
            for your model (i.e., amgbiguous inputs). The test will fail if
            the list of parsings from the model is not identical to the list
            you provide here. (The order of the output is irrelevant, though.)  -->
        <parsing-test label="Check that the correct morphemes are selected">
            <input lang="wk-LA">atalar</input>
            <output>
                <label>Stem</label>
                <label>Plural</label>
            </output>
        </parsing-test>

        <!-- A transduction test is similar to a parsing test. You can have
            multiple <output> tags. The test fails if the transductions
            generated by the model are not identical to those you list. -->
        <transduction-test label="Changing writing systems">
            <input lang="wk-LA">atalar</input>
            <output lang="wk-AR">آتالار</output>
        </transduction-test>

        <!-- The suggestion test suggests different possible parsings for
            a given input. (In this case, “unknown” is not a word in the
            language.) -->
        <suggestion-test label="Suggest possible stems">
            <input lang="wk-LA">unknownlar</input>
            <output>
                <stem>Stem</stem>
            </output>
            <output>
                <stem>Stem</stem>
                <label>Plural</label>
            </output>
        </suggestion-test>
    </schema>
</tests>

You can run this file with this command:

met test-gallery.xml output.txt --verbose

It should produce the following in output.txt:

Schema: Just a single schema in this example [03-Optional-Affixes.xml]
Success: Acceptance of a correctly affixed form: The input atalar (wk-LA) was accepted by the model, which is correct. [Stem][Plural]
Success: Acceptance of a correctly affixed form: The input this-is-not-a-good-input (wk-LA) was rejected by the model, which is correct. 
Success: The input atalar (wk-LA) was accepted by the model, which is correct. [Stem][Plural]
Success: The input this-is-not-a-good-input (wk-LA) was rejected by the model, which is correct. 
Success: Check that the correct morphemes are selected: The input atalar (wk-LA) produced [Stem][Plural], which is correct.
Success: Changing writing systems: The input atalar (wk-LA) produced (آتالار (wk-AR)), which is correct.
Success: Suggest possible stems: The input unknownlar (wk-LA) produced ([Stem], [Stem][Plural]), which is correct.

Notes

  • Each input or output has the lang attribute. This is the lang identifier from the <writing-systems> tag.
  • All of the label attributes are optional. They will be displayed in the output file. I have found it helpful for the label to be a short description of why the test is there, e.g., “Vowel-final stems should lenite”.
  • In recognition tests, don't neglect tests where the result should be “no”. This is particularly important when you're testing allomorphy.
  • If you'd like to output a line of text in a test file, use the <message> tag (e.g., <message>My message</message>).
  • If you'd like to create a blank line between tests in the output, use the <blank/> tag.
  • You can add the show-model attribute to a particular schema if you just want to look at one model. E.g., <schema show-model="true">
⚠️ **GitHub.com Fallback** ⚠️