Code Completion - vilinski/nemerle GitHub Wiki

This page needs to be updated, and it will be when the Code Completion Engine gets a more mature stage.

Table of Contents What is all about? Areas of the project Building the Type Tree Using the Type Tree ConstructedTypeInfo DeclaredTypeInfo and NemerleMemberInfo Compiler Messages

The Code Completion project for Nemerle aims to create an easy but powerful platform so development environment and tools could use the same lexer, parser, typer, etcetera... that the compiler uses. This has two great advantages:

1. Environments do not need to create their own parser, but use the compiler, so this is a work they don't have to do.

2. Creating an entire parser and typer is not an easy task, much more in Nemerle which has type inference as one of its main features.

3. If the compiler changes, the environments do not have to update anything, just link to the new Nemerle.Compiler DLL.

This project has been subdivided in two main areas

1. Type Tree: you can obtain a tree representing all your types and members of the current code, fast enough for an IDE, and with a bunch of information about all of them. This tree also has information regarding the errors that may be found in the code.

2. Code Completion: in any point, you can ask the compiler to tell you which members/types/... are allowed in this scope, so you can put them in a drop-down list and show it to the user.

If you want to build a type tree using the compiler features, you need to follow this easy steps:

1. You must add a reference to the Nemerle.Compiler.dll. You should also open the namespace Nemerle.Completion. All types that are refenced here live in that namespace.

2. Create a new Engine instance. You must create a new instance each time you are going to use the engine, or errors may arise.

 <span style="color: #0600FF;">mutable</span> engine = Engine(); <small>(Nemerle)</small>

 Engine engine = <span style="color: #0600FF;">new</span> Engine(); <small>(C#)</small>

3. Add references for the libraries you need. Your program must take care about knowing which to add. There are 2 ways to add a reference: via its name (for absolute paths or assemblies in the GAC), or loading yourself the assembly using System.Reflection.Assembly.Load. Anyway, then you have to call the AddReferencedAssembly method.

• Loads the assembly from the GAC. This is specially recommended for assemblies that come with the Framework (System.Xml, System.Data...):

 engine.References.Add("System.Windows.Forms"); <small>(both Nemerle and C#)</small>

• Loads an assembly and then passes it as argument. You can catch the exceptions while loading the assemblies, which is much better than relying in the Code Completion engine:

 <span style="color: #0600FF;">def</span> assembly = System.Reflection.Assembly.Load (<span style="color: #808080;">"System.Windows.Forms"</span>);
 engine.References.Add(assembly); <small>(Nemerle)</small>

 System.Reflection.Assembly assembly = System.Reflection.Assembly.Load (<span style="color: #808080;">"System.Windows.Forms"</span>);
 engine.References.Add(assembly); <small>(C#)</small>

4. Add any define for the preprocessor. As stated before, you should take care of the defines in your programs and add them at this point:

 engine.AddDefine (<span style="color: #808080;">"DEBUG"</span>); <small>(both Nemerle and C#)</small>

5. By default, compiler messages goes to the console. However, you can change it by modifying the Output property. If you don't want to output the messages automatically (you can always check them later), you must tell the compiler to redirect it to a MemoryStream this way:

 engine.Output = System.IO.StreamWriter (System.IO.MemoryStream ()); <small>(Nemerle)</small>

 engine.Output = <span style="color: #0600FF;">new</span> System.IO.StreamWriter (<span style="color: #0600FF;">new</span> System.IO.MemoryStream ()); <small>(C#)</small>

6. You need to tell the engine from which code it must generate the tree. The method AddCode takes two parameters: the first one is the code itself, the second one is the filename (for example 'Main.n', or things like that). This filename will be used in information and messages, so it should be the same as the keys you use to open files in your application, for example.

 engine.AddCode (TheCodeITookFromTheEditor, <span style="color: #808080;">"filename.n"</span>); <small>(both Nemerle and C#)</small>

7. And finally you call GetTypeTree, which returns a TypeTree with all the information. Errors in the code may create a bad tree, or with some members missing. However, this is not a problem of the engine, but from the code. Errors that the engine find more difficult to recover from are missing '}' (closing brackets).

 <span style="color: #0600FF;">def</span> tree = engine.GetTypeTree();  <small>(Nemerle)</small>
 TypeTree tree = engine.GetTypeTree(); <small>(C#)</small>

Type Tree is made from a bunch of interconnected classes, that are shown in the graph below (this may not be totally correct UML, but the idea is clear).

Image:TypeTreeNew.png

As you can see, there are four different classes for type information:

• DeclaredTypeInfo: represents a type that has been written in your code.
• ReferencedTypeInfo: represents a type declared outside (in a DLL, in the class library). Contains a reference to the type in terms of System.Type in its field Type.
• NemerleTypeInfo: the base class for this two.
• ConstructedTypeInfo: this is the base class for the 6 types that live nested on it. Represents information about a type, but not as a definition. Maybe an example will make it clear:

Example is a DeclaredTypeInfo

 <span style="color: #0600FF;">public class</span> Example['a] {...}

Example[int] is a ConstructedTypeInfo, because it is being used in a bigger expression, in that case the return type of a method.

 <span style="color: #0600FF;">public</span> ExampleMethod() : Example[int] {...}

So the 6 subclasses of ConstructedTypeInfo are:

• Class: contains a reference to a type (NemerleTypeInfo), and its type parameters, if it has any. Be careful because the Type Parameter maybe another Class or a Generic Specifier.
• GenericSpecifier: this subclass is used when the type is open, that is, when the type parameter has not been substituted for anything yet. It may contain additional information about the constraints, both type ones and special ones, that are taken from the Constraint enum.

 ExampleClass[GenericClass[int], 'a] <span style="color: #0600FF;">where</span> 'a : IComparable, <span style="color: #0600FF;">struct</span>

maps to:

Class
|--> Type = ExampleClass as a NemerleTypeInfo
|--> SubstitutedArguments
     |--> Class
     |    |--> Type = GenericClass as a NemerleTypeInfo
     |    |--> SubstitutedArguments
     |         |--> Class
     |              |--> Type = int as a NemerleTypeInfo (int is always a ReferencedTypeInfo)
     |              |--> SubstitutedArguments = empty array
     |--> GenericSpecifier
          |--> Name = 'a
          |--> TypeConstraints
          |    |--> Class
          |         |--> Type = IComparable
          |         |--> SubstitutedArguments = empty array
          |--> SpecialConstraints = Constraint.Struct (value 0x02 from the enum)

• Tuple: a tuple. Its Types property has the information from it. This is again a ConstructedTypeInfo, so thing like (IDictionary[int, 'a], 'b) could be represented.
• Function: a function as an argument (int->void...). More than an argument in either the arguments or the return types are represented as tuples.

 int*string->void

maps to

Function
|--> From
|    |--> Tuple
|         |--> Types
|              |--> Class
|              |    |--> Type = int
|              |--> Class
|                   |--> Type = string
|--> To
     |--> Void

• Array: its name is self-describing. Just a Type property and a Rank one. Ranks are also called dimensions, so a bidimensional array[int] is:

Array
|--> Type
|    |--> Class
|         |--> Type = int
|--> Rank = 2

• Void: just nothing. Only used as return types from methods and functions.

And for all of you, people who just want a way to get a string with the type from a ConstructedTypeInfo, and have read all this long, annoying writing, here's the code for it (in C#). It suposses you have opened the Nemerle.Compiler.CodeCompletion namespace:

    <span style="color: #0600FF;">if</span> (ar != <span style="color: #0600FF;">null</span>)
    {
        <span style="color: #0600FF;">return</span> <span style="color: #808080;">"array["</span> + get_type_name(ar.Type) + <span style="color: #808080;">"]"</span>;
    }
    <span style="color: #0600FF;">else if</span> (cl != <span style="color: #0600FF;">null</span>)
    {
         DeclaredTypeInfo dti = cl.Type <span style="color: #0600FF;">as</span> DeclaredTypeInfo;
         ReferencedTypeInfo rti = cl.Type <span style="color: #0600FF;">as</span> ReferencedTypeInfo;
         <span style="color: #0600FF;">string</span> nameByNow = "";
         <span style="color: #0600FF;">if</span> (dti != <span style="color: #0600FF;">null</span>)
         {
             <span style="color: #0600FF;">if</span> (!dti.IsNested)
                 nameByNow = dti.Namespace + <span style="color: #808080;">"."</span> + dti.Name;
             <span style="color: #0600FF;">else</span>
                 <span style="color: #007F00;">// Nested classes follow the convention Namespace.DeclaringType+NestedType</span>
                 nameByNow = dti.DeclaringType.Namespace + <span style="color: #808080;">"."</span> + dti.DeclaringType.Name + 
                             <span style="color: #808080;">"+"</span> + dti.Name;
         }
         <span style="color: #0600FF;">else if</span> (rti != <span style="color: #0600FF;">null</span>)
         {
             nameByNow = rti.Type.FullName;
         }
         <span style="color: #0600FF;">if</span> (cl.SubstitutedArguments.Length > 0)
         {
             nameByNow += <span style="color: #808080;">"["</span>;
             <span style="color: #0600FF;">foreach</span> (ConstructedTypeInfo cdt <span style="color: #0600FF;">in</span> cl.SubstitutedArguments)
                 nameByNow += get_type_name(cdt) + <span style="color: #0600FF;">", "</span>;
             nameByNow = nameByNow.TrimEnd(<span style="color: #0600FF;">','</span>, <span style="color: #0600FF;">' '</span>);
             nameByNow += <span style="color: #0600FF;">"]"</span>;
         }
         <span style="color: #0600FF;">return</span> nameByNow;
     }
     <span style="color: #0600FF;">else if</span> (fu != <span style="color: #0600FF;">null</span>)
     {
         <span style="color: #0600FF;">return</span> get_type_name (fu.From) + <span style="color: #808080;">"->"</span> + get_type_name(fu.To);
     }
     <span style="color: #0600FF;">else if</span> (gs != <span style="color: #0600FF;">null</span>)
     {
         <span style="color: #0600FF;">return</span> gs.Name; <span style="color: #007F00;">// It only shows the name, no constraints</span>
     }
     <span style="color: #0600FF;">else if</span> (tu != <span style="color: #0600FF;">null</span>)
     {
         <span style="color: #0600FF;">string</span> nameByNow = <span style="color: #808080;">""</span>;
         <span style="color: #0600FF;">foreach</span> (ConstructedTypeInfo cdt <span style="color: #0600FF;">in</span> tu.Types)
             nameByNow += get_type_name(cdt) + <span style="color: #808080;">"*"</span>;
         <span style="color: #0600FF;">return</span> nameByNow.Trim(<span style="color: #808080;">'*'</span>);
     }
     <span style="color: #0600FF;">else</span>
         <span style="color: #0600FF;">return</span> <span style="color: #808080;">"void"</span>;
 }

Most of the properties of this classses are self-describing, but some of them need a little explanation:

• Nested Types: nested types are the ones declared inside another type. They can be accessed via NestedTypes (for pure nested types) or via VariantOptions which are nested types with an special meaning. In that case, Namespace contains the namespace of its declaring type, which is saved at DeclaringType. For any other type, DeclaringType = <span style="color: #0600FF;">null</span>.
• Constructors: constructors are included inside the Methods collection, but its IsConstructor or IsStaticConstructor is set to true. Its return type is always Void for the engine, but it doesn't make much sense to show it in an IDE.
• BaseType: contains the direct superclass for the type.
• Interfaces: contains the interfaces that the developer needs to implement in the class. That is, if you subclass List[T], which implements IList, you also implement IList, but it is not shown here because it has already been implemented in List[T].
• And finally, be careful with things that may or not be present. In Nemerle option['a] is used, but this does not exist in the Framework, so we assign <span style="color: #0600FF;">null</span> in cases like a property without setter, IndexerParameters if the property is not a indexer..., and can throw a NullReferenceException, of course.

Messages thrown by the lexer, parser, scanner and typer are picked up and put inside the CompilerMessages property in the engine object. A compiler messages consist of just 3 properties:

• Message: the message itself ("there no type called foo", "you are not using the variable bla", and so on). Sometimes you may receive a message telling about an "unbound type 'int'", but this is not a problem. int, char, an all the primitive types are inside Nemerle.Core: this namespace is opened automagically by the compiler, and everything works OK, but it complains. The solution is very simple: add <span style="color: #0600FF;">using</span> Nemerle.Core; at the start of yor code.
• Location: for the start of a method, a variable...
• '''match (MessageKind) {'''
  

    | Error => an error that will make the compiler to refuse to compile the code (missing brackets, non-existing keywords...)
    | Warning => something that has some severity, but which will work fine
    | Hint => an advice given for free to you
}