ANTLR and traits

Traits in Scala are a wonderful way of composing functionality; in this post, I want to show you how to take Java classes and add some functionality into them using traits. In short, I used ANTLR to generate lexer and parser for a grammar in my project, but I needed to provide an interface to report the lexer and parser errors to the users.

ANTLR generates the Java code for the lexer and parser of your grammar; to then build the AST from your grammar, you first create the lexer and then give the lexer to the parser. The lexer analyses the input stream of characters and produces tokens, the parser consumes the tokens from the lexer and produces the AST. In code, it looks something like this:

grammar MyGrammar;

options { output=AST; }

tokens {
  SCRIPT_DEF;
  DECLARATION_DEF;
}

@header {
package org.cakesolutions.akkapatterns.dsl;
}

@lexer::header {
package org.cakesolutions.akkapatterns.dsl;
}

script: declarations element -> ^(SCRIPT_DEF declarations element);

declarations: declaration* -> ^(DECLARATION_DEF declaration*);
	
declaration: String ':' url;
	
url: String '://' Ref;
	
element:
  LParen name attribute* element* value? RParen 
    -> ^(ELEMENT_DEF name attribute* element* value?);
	
...
		
String:	('a'..'z'|'A'..'Z'|'0'..'9'|'.'|'-'|'_'|'#')+;
Ref:	('a'..'z'|'A'..'Z'|'0'..'9'|'.'|'-'|'_'|'/'|'#')+;
LParen:	'(' ;
RParen:	')' ;

WS:     (' '|'\r'|'\t'|'\u000C'|'\n') {$channel=HIDDEN;};

Given this grammar, ANTLR will generate MyGrammarLexer and MyGrammarParser, which you can use in your project like so:

class AntlrMapper {

  def getAst(source: String): Either[Seq[String], CommonTree] = {
    val lexer = new MyGrammarLexer(new ANTLRStringStream(source))
    val tokens = new CommonTokenStream()
    tokens.setTokenSource(lexer)
    val parser = new MyGrammarParser(tokens)
    parser.setTreeAdaptor(new CommonTreeAdaptor())

    try {
      Right(parser.script().getTree.asInstanceOf[CommonTree])
    } catch {
      case e: Exception =>
        Left(e.getMessage)
    }
  }

}

The problem is that if the script contains lexing or parsing errors, you will not see them–or, rather, you will see only one, reported by the final exception. This is not good. It would be useful to be able to somehow collect the errors from the lexer and parser, but ideally without modifying the lexer or the parser. You see, the generated MyGrammarLexer and MyGrammarParser both allow you to override method called emitErrorMessage(msg: String): Unit, which by default simply System.out.printlns it. (Oh, the humanity!). We would like to add this behaviour to both the generated classes ad hoc, when we instantiate them… a perfect place for a trait.

class AntlrMapper {

  trait ErrorReporting {
    this: BaseRecognizer =>
    final val errors = new collection.mutable.ArrayBuffer[String]

    override def emitErrorMessage(msg: String) {
      errors + msg
    }
  }

  def compile(source: String): Either[Seq[String], CommonTree] = {
    val lexer = new MyGrammarLexer(new ANTLRStringStream(source)) 
                  with ErrorReporting
    val tokens = new CommonTokenStream()
    tokens.setTokenSource(lexer)
    val parser = new MyGrammarParser(tokens) with ErrorReporting
    parser.setTreeAdaptor(new CommonTreeAdaptor())

    try {
      Right(parser.script().getTree.asInstanceOf[CommonTree])
    } catch {
      case e: Exception =>
        Left(lexer.errors ++ parser.errors + e.getMessage)
    }
  }

}

And this is it. Notice the self-type annotation on the ErrorReporting trait, which lets me override the emitErrorMessage method. I then mix in this trait to the instances of the MyGrammarLexer and MyGrammarParser and then jump into lexing and parsing the input DSL. In case of failures, I have all the error messages I need.