<img height="1" width="1" src="https://www.facebook.com/tr?id=1076094119157733&amp;ev=PageView &amp;noscript=1">

Type-safe DSL

Posted by Jan Machacek on Sat, Dec 10, 2011

I very much like the new query DSL in Grails; in this post, I will show you how to construct a similar domain-specific language in Scala. We are going to implement a DSL to create queries. You are going read about the mental steps needed to construct your DSL and see how to use implicit conversions (the implicit def functions) to do some of the heavy lifting.

The follow-up post deals with simplifying the DSL we construct here using pattern matching.

The query should feel like a regular SQL or HQL query, such as

val q1 = "foo" &#x003D; "a"
val q2 = (("foo" &#x003D; "a") && ("bar" like "c")) ||
          ("x" &#x003D; "y") orderBy "foo"

To make things slightly more complex, let's allow a guard expression that could cause a term to be skipped. In the following example the entire ("username" = username) term will be skipped, because the value of username is empty.

val username: String = ""
val q1 = ("id" &gt; 1L) && ("username" &#x003D; username when username != "")

The underlying structure

So, we are creating some queries; these queries contain restrictions, groupings and order by clauses. Right. That gives us the first cut of the classes we will need:

class Restriction
class OrderBy
class GroupBy
class Query(val restriction: Restriction,
            val orderByClauses: List[OrderBy],
            val groupByClauses: List[GroupBy])

It seems that ("username" like username when username != "") should end up as a subclass of Restriction. If the Restriction class contains functions &&, when and ||, it should all work. Before we jump in to the code, let's consider the kinds of restrictions we'll need: equality, non-equality, greater-than, less-than, ..., like, is-null, not-null. Classifying the restrictions, we could arrive at:

abstract class Restriction
case class Binary(column: String, op: Symbol, value: Any) extends Restriction
case class Like(column: String, value: String) extends Restriction
case class NotNull(column: String) extends Restriction
case class IsNull(column: String) extends Restriction

Let's complete the restrictions by adding the logical operations as well as three other special restrictions: Tautology(), Contradiction() and Skip(). Contradiction() and Tautology() are obvious: they represent always-false and always-true restriction. The Skip() restriction is the restriction that gets added if the when guard fails.

case class Disjunction(lhs: Restriction, rhs: Restriction) extends Restriction
case class Conjunction(lhs: Restriction, rhs: Restriction) extends Restriction
case class Tautology() extends Restriction
case class Contradiction() extends Restriction
case class Skip() extends Restriction

So, the q should become:

val username: String = ""
val r = ("id" &gt; 1L) && ("username" &#x003D; username when username != "") 

//r == Conjunction(Binary("id", '&gt;, 1L), Skip())

Convert me!

So, how do we get there? To construct a query, I must have a valid restriction; to construct a restriction, I start with a column name and then refer to the appropriate operator and, if required, supply a value. Let me rephrase that into something that we can use to construct Scala code. We turn the column name--a String into some instance that contains methods that construct the appropriate Restriction subclasses. For example, the >(value: Any) method returns Binary(the-column, '>, value), the =(value: Any) returns Binary(the-column, '=, value); the like(value: String) method returns Like(the-column, value), and so on.

Excellent! We have:

class PartialRestriction(val column: String) {
  def &gt;(value: Any) = Binary(column, '>, value)
  def &#x003D;(value: Any) = Binary(column, '=, value)
  def like(value: String) = Like(column, value)
  ...
}

In other words, we can take a String, construct the PartialRestriction instance and use its methods to get the Restriction subclasses that represent the concrete restrictions. So, we can write:

val r = new PartialRestriction("username").like("Jan%")

//r == Like(username, Jan%)

Nice, but inconvenient! I'd like to automatically turn the "username" string into a PartialRestriction: enter implicit conversions. All I need to do is to add an implicit function into scope:

implicit def toPartialRestriction(column: String) =
  new PartialRestriction(column)

We can now write code such as:

val r = "username" like "Jan%"

//r == Like(username, Jan%)

What just happened? Well, the Scala compiler saw that we're calling a function like on a String; and there is no such function. Scala will now see if it can find an implicit conversion from String into something that contains the like(value: String) function. And it is in luck: it find the implicit def toPartialRestriction(property: String) function, so it "replaces" our "username" String with new PartialRestriction("username"). The PartialRestriction contains the function like, which expects one argument of type String, which we supply as Jan%.

The guard

We can now turn Strings into the appropriate Restriction subclasses via the PartialRestriction instance that we got using the implicit conversion function. How do we now deal with the guard? Remember, we'd like to be able to write:

val username: String = ""
val r = "username" &#x003D; username when username != ""

// r == Skip()

The solution is not that difficult: "username" = username is an instance of subclass of Restriction--all we need to do is to add the when function to the Restriction class:

abstract class Restriction {
  def when(guard: => Boolean) = if (guard) this else Skip()
}

Yes, that's all there is to it! If the guard is false then we replace the Restriction by Skip(), else we keep the very same Restriction. To complete, we need the && and || functions to combine the Restrictions:

abstract class Restriction {
  ...
  def &amp;&amp;(rhs: Restriction) = Conjunction(this, rhs)
  def ||(rhs: Restriction) = Disjunction(this, rhs)
}

That was easy! We can now apply our DSL (using our implicit conversion function to turn a String to PartialRestriction and then to Restriction) to construct rather complex restrictions:

val username: String = ""
val r = (("id" &gt; 1L) && ("username" &#x003D; username when username != "")) ||
          ("length(username)" > 5)

// r == Disjunction(
//        Conjunction(Binary(id, '>, 1L), Skip()),
//        Binary(length(username), '>, 5))

The next challenge is to allow us to use the orderBy and groupBy clauses in our query DSL. However, an expression with either of the two clauses is no longer just a simple Restriction, it is a Query. Looking at the constructor of our Query class, in order to construct the instance, we'd need to write:

val r = "username" like "J%" ...
val q = new Query(r, Nil, Nil)

Clumsy. But wait, remember the implicit conversions? There's nothing stopping us from creating an implicit function that turns a Restriction into a Query with that restriction and empty order by and group by clauses.

implicit def toQuery(restriction: Restriction) =
  new Query(restriction, Nil, Nil)

Now, how do we add the order by and group by clauses--simply by adding the appropriate methods to the Query class like so:

class Query(val restriction: Restriction,
            val orderByClauses: List[OrderBy],
            val groupByClauses: List[GroupBy]) {

  def orderBy(orderBy: OrderBy) =
    new Query(restriction,
              orderBy +: orderByClauses,
              groupByClauses)

  def groupBy(groupBy: GroupBy) =
    new Query(restriction,
              orderByClauses,
              groupBy +: groupByClauses)
}

So, what's left? We can take a String and turn it into arbitrarily complex Restriction; we can then turn that Restriction into a Query; we can add the group by and order by clauses, as required.

val q = (("foo" &#x003D; "a") && ("bar" like "c")) ||
         ("x" &#x003D; "y") orderBy "foo" desc groupBy "bar"

Notice that the groupBy and orderBy functions do not accept Strings, they accept the appropriate instances. What we need to do is to turn a String into OrderBy or GroupBy. That should be easy!

class PartialOrderBy(val column: String) {
  def asc = new OrderBy(column, true)
  def desc = new OrderBy(column, false)
}

case class OrderBy(column: String, ascending: Boolean)

implicit def toPartialOrderBy(column: String) = new PartialOrderBy(column)
implicit def toGroupBy(column: String) = new GroupBy(column)
implicit def toOrderBy(pob: PartialOrderBy) = pob.asc

Summary

That completes the picture. I have the Query whose restrictions I can combine to make more complex queries using its functions (&&, || and others); I have the PartialRestriction that works as a builder for the final Restriction subclass. To make all this easier to construct in code, I supply the implicit conversions; from String to PartialRestriction and then from Restriction to Query. This means that I can "naturally" construct data access restrictions in my code

val simple =   "foo" &#x003D; "a"
val and    = (("foo" &#x003D; "a") && ("bar" like "c")) || ("x" &#x003D; "y")
val or     =  ("foo" &#x003D; "a") || ("bar" like "c")
val neg    =  ("foo" &#x003D; "b")
val gt     =   "foo" > 5
…

When building the Restrictions, I can use the when guard and, naturally, the orderBy and groupBy functions. All this gives me type-safe DSL!

In the next post, I will show you how to use pattern matching to simplify the restrictions.

Recent Posts

Posts by Topic

see all

Subscribe to Email Updates