ModelLang (Language for Defining Box Models) - mn-mikke/Model-driven-Pretty-Printer-for-Xtext-Framework GitHub Wiki

This chapter discusses what a language allowing for defining formatting rules that are linked to rules of grammar look like. This formatting rules consists from usages of operators defined by a code written in the [MetaModLang](https://github.com/mn-mikke/Model-driven-Pretty-Printer-for-Xtext-Framework/wiki/MetaModLang-(Language-for-Defining-Box--Meta-models\)) language. Further, it is possible to set a value of some parameter if a default value does not meet. The final solution is inspired by the syntax selected for the usage of operators of the ideal box meta-model. This language is called as the ModelLang in the remaining text.

The following text describes what a language satisfying the requirements from the previous paragraph could look like. As the [MetaModLang](https://github.com/mn-mikke/Model-driven-Pretty-Printer-for-Xtext-Framework/wiki/MetaModLang-(Language-for-Defining-Box--Meta-models\)) language was associated with a concrete extension, so the extension .ppf expressing a term Pretty Printer's Formatting will belongs to files storing a code of this language.

The user should be allowed to choose a box meta-model for a subsequent usage. Moreover, he should have a chance to define ad-hoc alias rules and constants for a definition of formatting. These both requirements can be solved at once so that the grammar of this language would be an extension of the grammar of the [MetaModLang](https://github.com/mn-mikke/Model-driven-Pretty-Printer-for-Xtext-Framework/wiki/MetaModLang-(Language-for-Defining-Box--Meta-models\)) language. This is possible because the language dedicated for a definition of a grammar allows for inheriting a grammar from one another. The slot for an ancestor is by default occupied by the default grammar containing specifications of terminals. Since the inheritance of grammar is transitive and the grammar of previous language extends the default grammar of terminals, the slot can be occupied by the grammar of the previous language.

As it is shown in the Listing what an interconnection of formatting rules with rules of a SDF grammar can look like, thus it have to be designed formatting rules linked to Xtext's grammar. The listing considers only the formatting rules for non-terminals but because it will be used operators transforming text described by a terminal and the Xtext framework contains a concept of syntax highlighting on the lexical level, it has to be also defined formatting rules for terminals. This fact implies that should be designed suitable keywords indicating a formatting rule. Any keyword with the suffix "BOX" looks as a reasonable variant because the formatting rule is essentially a box adding an appearance and a shape to abstract thing.

Furthermore, a file contained some grammar should be somehow referenced by this language in order to grammar rules be accessible for formatting rules. Since this language references meta-models and meta-models are referenced among themselves by utilizing imports with an URI, this imports will be also used for this case in order to preserve the convention.

A definition of nonterminal rule consists of rule's name and definition elements as it can be seen in the Listing. Firstly, the formatting rule should reference a qualified name of the grammar rule in order to be clear with which one is it associated as well as the alias operator references an another operator. Further, it can follow a body containing usages of operators that encapsulate references to defining elements of the grammar rule as well as redefinitions of default values of an alias operator reference corresponding parameters. Unlike a definition of parameters of some operator which structurally flat, definition elements form a tree structure as it can be seen in the Listing.

The Defining Elements of the Parser Rule section discusses in detail that the language for defining a grammar contains not only essential defining elements such as keywords, rule calls and cross references but also composite defining elements such as ordered and unordered groups, alternatives and assignments for which the body of grammar rule is a tree structure. It is good to realize that composite defining elements can not be referenced in any way because they do not contain any name or any other identifier. The exception is an assignment but there is still a problem how to reference these composite defining elements and how to knit them them with usages of some operators. The solution could be to reference only essential defining elements and references to composite defining rules supply by organization of references to essential defining elements into a similar tree structure such the elements are organized. The references will be grouped by composite elements of formatting rule as well as composite defining elements group essential defining elements of a grammar rule. It means that composite elements of a formatting rule will use the same delimiters ("~", "|", "&") for a separation of sub-elements as composite defining elements of a grammar rule. The exception is also an assignment because it does not separate sub-elements but it encapsulates one sub-element. Since priorities are defined among composite defining elements and it is possible to adjust the priorities by utilizing parentheses, composite elements of a formatting rule should be enriched with this feature. Now it can be created a formatting rule whose structure reflects a structure of grammar rule and further it can be used operators used at any level of structure. Usages of operators can used as its body references to essential defining but also composite elements of formatting rules. Hence the structure of formatting rule is the same as a structure of grammar rule with the exception of interleaving formatting structure by usage of operators. This solution is typified in the following figure.

Linking of a structure of formatting rule (right) to a rule of Xtext's grammar (left) is depicted . The grammar rule called Greeting enables to the user write two greetings. The first "good morning" consists form two keywords. The second "hello" is represented by only one keyword. The formatting rule defines that all greetings will be aligned horizontally but this information will be reflected only with the first greeting. Moreover, the formatting rule defines a various font color for each greeting.

Usages of the operator that surrounds references or compositions have been typified but it should be possible to be usages of operators surrounded by themselves. This will require some changes in the syntax of a usage of the ideal meta-model because operator's name and a rule call is represented by an identifier it would not be clear which object should be created since Xtext's parser works with a view to only one token. The solution could be to try to extend parser's outlook but it can be a bit slower than the default variant. A more elegant solution could be enclose the usage of a operator by some keywords. Since angle brackets have not been used so they look like a good choice.

This problem with usages of operators has been solved. Now it has to be decided which kinds of operators should be allowed to use for these formatting rules. Definitely, it should be enabled positional operators because they give to grammar rule an shape. Further, it should be enabled highlight operators because the Xtext framework contains a concept of semantic highlighting at the semantic level (see syntax highlighting) and these formatting rules is a good place for an integration of the concept into the configuration of the pretty printer. Since the transforming operators were designed in order to format comments and comments do not exist at this level, they will be disabled.

This example of a grammar rule and its corresponding formatting rule are related to the Figure. The same operators are used. The formatting rule is identified by the PBOX keyword because grammar rule for nonterminals are also called parser rules (see the Parser Rules section).

// Grammar rule
Greetings:
    'good' 'morning' | 'hello'
;

// Formatting rule
PBOX[Greetings]:
    <H>[<F c="#00ff00">['good' 'morning'] | <F c="#ff0000">['hello']]
;

The situation with defining formatting rules is more easier in comparison with formatting rules for nonterminals. The grammar rule of a terminal does not contain any defining elements like in a grammar rule of a nonterminal. Thus the formatting rule should contain only usages of operators. It also follows that positional operators can not be used in any way. Conversely, transforming operators could be used for formatting comments and highlight operators would serve to define a configuration of syntax highlighting at the lexical level. Furthermore, it makes sense to use maximally one transforming operator and one highlight operator.

Although, terminals defined by terminal rules (see the Terminal Rules section) exist in the Xtext framework, which can be referenced similarly like nonterminals, the keywords used as a defining element has a terminal character and it would be beneficial to resolve them by various fonts. Since they have no own formatting rule, they should be referenced alternatively. The possible solution is to categorize keywords by regular expression that work on the wanted lexical level. The similar situation is how to define a default font for terminals which do not have any corresponding formatting rule defining a font. The whole solution is typified in the following listing.

// Formatting rule referencing a terminal defined by grammar rule.
TBOX[ML_COMMENT]: <MC>, <F c="#00ff00">;

// Formatting rule with a regular expression 
TBOX[Keyword, "^\\w.*\$"]: <F c="#0000aa", w=bold>;

// Formatting rule for terminals that have no
// formatting rule of two previous categories.
TBOX[default]: <F c="#222222">;

This listing depicts what a file containing code written in ModelLang may look like. A box model is related to grammar from the listing on this page, whose pretty-printing effect is reflected in the Listing 2 on this page.

xtext "platform:/resource/cz.gpp/src/cz/gpp/Example.xtext"

import "platform:/resource/gpp/settings/operators.ppo"

TBOX[Default]: <F>;
TBOX[INT]: <F c="#7F7F7F">;
TBOX[ML_COMMENT]: <F i=italic,c="#00FF00">, <MC>;
TBOX[SL_COMMENT]: <F i=italic,c="#00FF00">, <SC>;
TBOX[Keyword, ["^\\w.*\$"]: <F w=bold,c="#7F0055">;

PBOX[Model]:
    <V vs=2>[
        package:Package
        (<V>[imports:Import] & class:Class)
    ]
;
PBOX[QualifiedName]: <H hs=0>[ID ('.' ID)];
PBOX[Package]: <H>['package' name:QualifiedName];
PBOX[Import]: <H>['import' className:QualifiedName];
PBOX[Class]:
    <V>[
        <H>[
            abstract:'abstract' 'class' name:ID
            ('extends' superClass:[Class|ID])
        ]
        '{'
        <I>[<V vs=2>[(methods:Method | internalClasses:Class)]]
        '}'
    ]
;
PBOX[Method]:
    <V>[
        <H>[
            visibility:Modifier returnValue:[Class|ID]
            <H hs=0>[
                <F i=italic,c="#55007F">[name:ID] '('
                (parameters:Parameter <H>[(',' parameters:Parameter)])
                ')' '{'
            ]
        ]
        <I>[<H>[body:INT]]
        '}'
    ]
;
PBOX[Parameter]: <H>[SpecificParameter name:ID];
PBOX[SpecificParameter]:
    <V>[IntParameter | StringParameter | ObjectParameter]
;
PBOX[ObjectParameter]: <V>[type:[Class|ID]];
PBOX[IntParameter]: <V>['int'];
PBOX[StringParameter]: <V>['string'];

All realization steps seem to be straightforward with one exception. The [Scoping](https://github.com/mn-mikke/Model-driven-Pretty-Printer-for-Xtext-Framework/wiki/MetaModLang-(Language-for-Defining-Box--Meta-models\)) section discusses that it has to be defined some mechanism allowing for creating suitable scopes that are important for referencing parameters of some operator, there are exist a problem how to reference essential defining elements so that it would be possible to reference elements that are on a equivalent position in the tree structure like an identifier of a corresponding cross reference. This problem can not be solved like referencing parameters of operators because connection between a cross reference and its referenced object is created on the base of existence of cross reference's identifier in a scope which is a set of descriptions of possible objects. The object description contain information about its identifier. It means that the scoping in this form does not guarantee any restriction regard to order of elements. Moreover, because composite defining elements are defined by a language developer and hence are defined dynamically, the scopes have to contain identifiers of all defining elements of grammar rule. Thus it can not be captured any restriction regard to nesting in the tree structure.

One possible solution is to change a method identifying cross references and corresponding defining elements internally for scoping so that an identifier be able to capture the exact position of a cross reference or defining element in the tree structure. The newly created identifier for the essential defining element and its reference should contain elements name, a number expressing its order in a parent element and parent's identifier where the parent could be an composite element, a grammar rule or a formatting rule. The identifier of a composite element consists of its delimiter, position and parent's identifier. Moreover, grammar rules and formatting rules will be identified by its names, which will allowing for identifying references and defining elements uniquely. Thus the scopes of defining elements do not have to be define within a grammar rule.

Newly proposed identifiers for the essential defining elements from the Listing 1.

Greetings.|.0. .0.good
Greetings.|.0. .1.morning
Greetings.|.1.hello

The system of creating identifiers is designed and it remains to solve how to implement this change into the code of the Xtext framework.

The linking cross references to referenced objects is realized in the Xtext framework by a class implementing the ILinkingService interface specially its method getLinkedObjects. The method has tree parameters. The first one called context represents a model element containing a given cross reference. The second one called reference represents a description of a given cross reference. And the last one called node is an element of a node model which is an AST-based structure. The node represents a position of a given cross reference in this structure. The default implementation of the interface gets a required type of from the reference description and it passes the type together with the context to some default scope provider thereby giving a scope. Further, it get an identifier of a given reference from the node which it then uses for getting an object description from the scope. The object description then allows for getting a given object that the method returns.

This behavior of the method is inappropriate with use of newly designed version of identifiers. In this case, the user has to enter these long identifiers manually. It follows that it has to be written a new implementation of the interface where the method would contain translating simple identifiers into a new version. Furthermore, it has to be solved how to get the new version of identifiers into object descriptions located in a scope.

Identifiers are represented in the Xtext framework by the QualifiedName class, which is essentially a name formed from several hierarchical segments. These identifiers can be obtained from the getFullyQualifiedName method, whose only parameter represents object for who is an identifier wanted. This method is specified in the IQualifiedNameProvider interface. Therefore the task how to transform simple identifiers into a new version can be solved by inserting the Reference grammar rule containing only cross reference to a type of essential defining elements into a grammar of this language. Because of this step the instances of the Reference will be located at the same place in the tree structure like essential defining elements. The next step is to write a new implementation of the IQualifiedNameProvider interface that would involve the new version of identifiers for the model generated from a code written in this language where each instance of the Reference would have a blank name. Moreover, the implementation has to be registered by utilizing Google Guice in order to be the implementation active. The name from this new implementation would be concatenated with a simple identifier obtained from a node in order to get a whole identifier of the new version.

As the previous problem was solved by creating the new implementation of the IQualifiedNameProvider interface, a change of identifiers contained in descriptions of defining elements forming scopes into the new version can be realized similarly. Moreover, there is no need to concatenate an identifier from more parts because essential defining elements has name or other attributes that can serve as a simple identifier. Now it has to be decided how to integrate identifiers of the new version into descriptions The mere registration of the new implementation of the interface by utilizing Google Guice in order to change a default implementation, which is used by a code creating concrete descriptions, is not sufficient because each language has own Google Guice configuration and the default implementation is registered into configuration of the language defining Xtext's grammar.

Thus it has to be rewritten a code that is registered under the configuration of this language and serves to obtaining descriptions from another languages. The descriptions are obtained together as a pack represented by the IResourceDescription interface. The place for obtaining descriptions is located in the getResourceDescription method of the LoadOnDemandResourceDescriptions class, where the method finds out a manager of resource descriptions for a particular language on the base of a URI of resource descriptions forwarded as an only parameter. Further, a selected manager returns a needed resource description. Thus it has to be the method rewritten so as the methods selects newly created manager for descriptions of the language of Xtext's grammar. The creation of a new manager involves not only to implement IResourceDescription.Manager interface representing a manager but also implement the IDefaultResourceDescriptionStrategy interface that is responsible for creating descriptions and it is exploited by manager. Thus the strategy is the right place for getting a new version of identifiers into object descriptions.