Eisen Extension: unless - blancas/eisen GitHub Wiki

Eisen's extension mechanism works by adding to the compiler a parser and translator functions for a new kind of expression. In order to avoid conflicts with existing language constructs, new expressions must start with a unique word or combination of words.

To illustrate a simple extension we'll add unless to the language. This should work similarly to if but it doesn't have an else and it may have multiple expressions inside a do-end block. So we want to write:

unless test-expr do
   expr1;
   expr2;
   expr3
end

The reserved word do both separates the test expression from the consequent expressions and also starts a block of sequenced expressions. The word end ends the block and the whole expression.

Reusing Compiler Code

The compiler defines parsing and translation functions that we can reuse for our new expression. In particular, Eisen uses the Kern library to define parsing functions, which are located in the blancas.eisen.parser namespace. The Kern Wiki has a guide, tutorials and samples to get started and use the library.

Common patterns of parser code have their corresponding translation function in the blancas.eisen.trans namespace. Usually we can just reuse the appropriate one for new expressions.

For the unless command we can reuse the following:

  • The parser expr for parsing the test expression.
  • The parser doex for parsing a do-end block.
  • Function trans-expr for translating the parsed data into Clojure data structures.

We'll start the code by importing the namespaces mentioned above.

(use 'blancas.eisen.core)
(use '[blancas.eisen.parser :only (expr doex word)])
(use '[blancas.eisen.trans :only (trans-expr)])
(use '[blancas.kern.core :only (bind return run)])
(use '[blancas.morph.core :only (monad)])
(use '[blancas.morph.transf :only (->right)])

Our parser function must expect the code in the right order and generate a map with three fields:

  • A token to tell this expression apart from any other.
  • The parsed test expression.
  • The parsed do-end expression block.

The Parsing Function

Thus we write the following function according to the Kern usage and the map requirement:

(def unless-expr
  "Parses an unless expression with this syntax:

   'unless' test-expr 'do' expr* 'end'"
  (bind [_    (word "unless")
         test expr
         body doex]
    (return {:token :unless-expr :test test :body body})))

We can try our parsing function by running it with Kern's run function.

(run unless-expr "unless false do 500 end")               
;; {:token :unless-expr,
;;  :test
;;  {:token :bool-lit, :value false, :pos {:src "", :line 1, :col 8}},
;;  :body
;;  {:token :seq-expr,
;;   :value
;;   [{:token :dec-lit, :value 500, :pos {:src "", :line 1, :col 17}}]}}

The resulting map contains the three fields we need: :token, :test and :body. The text expression is in this case a boolean literal and the expression block is a sequenced expression, as their :token fields indicate. Getting familiar with the parsed abstract syntax tree (AST) requires some practice. There are many parser functions in blancas.eisen.parser that can be used like the one above to gain experience.

The Translation Function

Translating an AST into Clojure data structures is a sequential process that can fail at any point. We want the process to stop at the first failure and return an error message. In order to comply with this requirements, Eisen uses Morph a library of constructs called monads. Using the Imperative monad gives us the ability to return successful result as Right values, and unsuccessful ones as Left values, while nicely sequencing our processing and stopping as soon as some function returns a Left value. You can find a guide and examples in the Morph Wiki.

Since our parsed data are both expressions, we'll reuse the function trans-expr for their translation. Having those Clojure forms, we then proceed to put the whole resulting expression together. This is how we write for our translation function:

(defn trans-unless
  "Translates an unless expression."
  [ast]
  (monad [test (trans-expr (:test ast))
	  body (trans-expr (:body ast))]
    (->right `(if ~test nil ~body))))

You may notice a similarity between bind and monad. They both follow the monadic technique of sequenced processing, since both parsing and translation are fail-fast operations. In the above code, if trans-expr returns an error (e.g., a Left value), that will be the result of trans-unless. Otherwise, the function puts together a Clojure if expression that reverses the consequent expression. We don't have to splice the forms in body because a do-end block is translated as a Clojure do form.

Installing Unless

Now that we have our parsing and translation functions we now have to add it to the Eisen library with the function add-expression. We supply the unique token, the parsing function, the translation function, and any words we want to reserve for this expression, in this case unless.

(add-expression :unless-expr unless-expr trans-unless "unless")

For a quick test we can evaluate:

(eisen= "unless false do 999 end")
;; 999
(eisen= "unless true do 999 end")
;; nil