Blog has moved to github pages. mostlyblather.com

Grails and DBUnit integration testing

I need to do a bit of integration testing with Grails and both of the DBUnit plugins I found didn’t seem to be maintained.  To create a simple base class for DBUnit based integration tests turned out to be relatively simple to create.

For the purposes of this sample I created a grails project called bookstore with a single domain class called Book in the bookstore package.  This is what book domain class looks like:

package bookstore

class Book {
String title
String author
}


First I had to add the DBUnit dependency to /grails-app/conf/BuildConfig.groovy and also enable the maven central repository.  This is what the grails.project.dependency.resolution section now looks like:



grails.project.dependency.resolution = {
inherits( "global" ) {
}
log "warn"
repositories {
grailsPlugins()
grailsHome()
mavenCentral()
}
dependencies {
test 'org.dbunit:dbunit:2.4.7'
}
}


Next I created a base class for my DBUnit based tests.  I created a class named DbunitGroovyTestCase in /test/integration/bookstore.  The DataSource property named dataSource is injected from the Spring container when the integration tests are run, and I used the DatabaseDataSourceConnection class to create the DBUnit connection.



package bookstore

import javax.sql.DataSource;

import org.dbunit.database.DatabaseDataSourceConnection;
import org.dbunit.database.IDatabaseConnection;
import org.dbunit.dataset.xml.FlatXmlDataSetBuilder
import org.dbunit.dataset.xml.FlatXmlDataSet
import org.dbunit.dataset.IDataSet
import org.dbunit.operation.DatabaseOperation;

import groovy.util.GroovyTestCase;

class DbunitGroovyTestCase extends GroovyTestCase {

DataSource dataSource
IDatabaseConnection connection

protected void setUp() {
connection = new DatabaseDataSourceConnection(dataSource)
DatabaseOperation.CLEAN_INSERT.execute(connection, dataSet)
}

protected void tearDown(){
connection.close()
}

protected IDataSet getDataSet() {
return new FlatXmlDataSetBuilder().build(new FileInputStream("test/dbunit/dataset.xml"));
}

}


Here is a sample data set I created for the book table, located in test/dbunit/dataset.xml:



<dataset>
<book id="1" version="1" author="Mark Twain" title="The Adventures of Tom Sawyer"/>
<book id="2" version="1" author="Mark Twain" title="The Prince and the Pauper"/>
<book id="3" version="1" author="Mark Twain" title="Adventures of Huckleberry Finn"/>
<book id="4" version="1" author="Mark Twain" title="A Connecticut Yankee in King Arthur's Court"/>
</dataset>


Finally here is a sample integration test, located in test/integration/bookstore/BookPersistenceTest.groovy



package bookstore;

import grails.test.*

class BookPersistenceTest extends DbunitGroovyTestCase {

public void testFindAllByAuthor() throws Exception {
def books = Book.findAllByAuthor("Mark Twain")
assertEquals(4,books.size)
}

public void testFindByTitle() throws Exception {
def book = Book.findByTitle("Adventures of Huckleberry Finn")
assertEquals("Mark Twain",book.author)
}

}

Scala with Querulous vs. Java with Spring JdbcTemplate

While (still) looking around for a Scala SQL equivalent of iBatis, I found this post on stackoverflow.  Daniel’s answer let me to Querulous.  Several years ago I used the Spring Framework JdbcTemplate and it worked fine, but it was kind of clunky.  That experience with JdbcTemplate led directly to using iBatis on that very same project.  I have been mostly working with Hibernate/JPA since then, but for a certain class of problems, a SQL centric solution seems a better fit that a full blown ORM.

After looking at Querulous, and seeing how it manages to de-clutter the anonymous inner class clunkyness that is JdbcTemplate I’m left wondering if this isn’t the way I should go.

Here is a quick example from the JdbcTemplate documentation in Java:

Collection actors = this.jdbcTemplate.query(
"select first_name, surname from t_actor",
new RowMapper() {

public Object mapRow(ResultSet rs, int rowNum) throws SQLException {
Actor actor = new Actor();
actor.setFirstName(rs.getString("first_name"));
actor.setSurname(rs.getString("surname"));
return actor;
}
});


Here is the equivalent with Querulous in Scala:



  val actors = queryEvaluator.select("select first_name, surname from t_actor") { row =>
new Actor(row.getString("first_name"), row.getString("surname"))
}


Simple and easy to understand.  Can’t beat that.

Scala External DSL and Combinator Parsing experimentation

Yesterday I was looking around on the inter-webs for a Scala project that was “iBatis” like, but more Scala oriented.  I found some cool stuff (ScalaQL, and scala-sql-dsl for example) but nothing quite what I was looking for.

I kind of chewed on what exactly it was that I was looking for and this is my 10k/foot feature list:

  • Ability to define the underlying data store in a DSL that can be understood by Scala
  • Ability to map Scala classes to JDBC results by binding to artifacts of the data store

This is more or less what iBatis accomplishes with xml (now annotations) and a bunch of reflection.  I’d like to see it done without XML and minimal reflection.

That sent me down the rabbit hole on how exactly one goes about defining a DSL in scala.  I re-read chapter 31 in Programming in Scala.  I also read chapter 11 in Programming Scala.  Then I took a stab at writing a parser combinator for a very, very small subset of the “CREATE TABLE” DDL syntax for SQL.

import scala.util.parsing.combinator._

class TableDdlParser extends JavaTokenParsers {

def tables: Parser[Map[String, Any]] = rep(table) ^^ { Map() ++ _ }

def table: Parser[(String,Any)] =
("TABLE" ~ tableName ~ columns
^^ { case "TABLE" ~ tableName ~ tableContents => (tableName,tableContents) })

def tableName: Parser[String] = ident ^^ { case ident => ident }

def columns: Parser[Map[String, Any]] = "("~> repsep(column, ",") <~")" ^^ { Map() ++ _ }

def column: Parser[(String,Any)] =
columnName ~ dataType ^^ { case columnName ~ dataType => (columnName,dataType) }

def columnName: Parser[String] = ident ^^ { case ident => ident }

def dataType: Parser[Any] = "VARCHAR" | "INTEGER"

}

object TableDdlParserRunner extends TableDdlParser {

def main(args: Array[String]) {
val input =
"""TABLE person (first_name VARCHAR, last_name VARCHAR, age INTEGER)
TABLE place (city VARCHAR, state VARCHAR)"""
println(parseAll(tables,input))
}

}


Here is the output



[2.51] parsed: 
Map(person -> Map(first_name -> VARCHAR,
last_name -> VARCHAR,
age -> INTEGER),
place -> Map(city -> VARCHAR,
state -> VARCHAR))


This experiment only supports two data types (VARCHAR and INTEGER), without any of the other metadata that is required for those types.  I just wanted a feel for how hard it would be to write and didn’t want to get bogged down in the details.  It turned out to be quite a bit easier than I thought it would be.



Next up is to figure out the best way to map this DDL information to a scala class (If I don’t get distracted by something else first).

Save TortoiseHg Remote Repository Credentials

A quick tip so you don’t have to re-enter your user name and password every time you pull or push from a remote Mercurial repository when using TortoiseHg.

Right click in the folder explorer and select TortoiseHg –> Repository Settings.

repository_settings

Select the “Sync” tab in the “TortoiseHg Configure Repository” window.  Either select the desired Remote repository path and click “Edit” or create a new Remote repository path with the “Add” button.

sync

That will bring up the “Edit remote repository path” dialog.  Enter your User and Password, then click OK.

 

edit

Finally click the “Apply” button on the “TortoiseHg Configure Repository” window.

Scala With RESTEasy (JAX-RS)

In the "code club" / "dojo" or whatever you want to call it that Ben (aka Johnny Coder) organized we have been learning about Scala the last few weeks. I have been puttering with Scala on my own for about a year now, and it has been fun to engage with other folk in the learning process. We have mostly been doing a mix of Project Euler and the S-99 problems as exercises. Those problems are fun, but they don't really reflect the kind of day to day programming tasks that I am usually engaged in.

To scratch that slightly more practical itch, I decided to see how well Scala fits in with other tools in the Java ecosystem. The Java ecosystem being broad, and my Scala kung fu being not that strong, all I have are some possibly interesting bits I discovered while trying to get Scala and RESTEasy working together.  I picked RESTEasy because in the middle of reading RESTful Java with JAX-RS.

I started off with an existing example project from chapter 11 in the RESTEasy book. I added the Scala plugin to the maven build, and I was off.

The first class I converted over was a JAXB annotated class com.restfully.shop.domain.Customer

I ran into a few things that caused me problems.  The first was this compiler message:

"';' expected but ',' found"


At the @XMLType annotation line.  The resolution was to switch from using the Array literal "{}" Java syntax to the Scala Array object.



I also got a JAXB runtime error during the JUnit tests:



....
Caused by: com.sun.xml.bind.v2.runtime.IllegalAnnotationsException: 15 counts of IllegalAnnotationExceptions
There are two properties named ....


I resolved that by using the @XmlAccessorType(XmlAccessType.FIELD) annotation.



For compatibility with the existing code that expected Java Bean style property access, I added the @BeanProperty annotation.   The annotation causes Scala to automatically generate get/set methods.



Here is the java version of Customer:



package com.restfully.shop.domain;

import javax.xml.bind.annotation.XmlAttribute;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlType;

@XmlRootElement(name = "customer")
@XmlType(propOrder = {"firstName", "lastName", "street", "city", "state", "zip", "country"})
public class Customer
{
private int id;
private String firstName;
private String lastName;
private String street;
private String city;
private String state;
private String zip;
private String country;

@XmlAttribute
public int getId()
{
return id;
}

public void setId(int id)
{
this.id = id;
}

@XmlElement(name = "first-name")
public String getFirstName()
{
return firstName;
}

public void setFirstName(String firstName)
{
this.firstName = firstName;
}

@XmlElement(name = "last-name")
public String getLastName()
{
return lastName;
}

public void setLastName(String lastName)
{
this.lastName = lastName;
}

@XmlElement
public String getStreet()
{
return street;
}

public void setStreet(String street)
{
this.street = street;
}

@XmlElement
public String getCity()
{
return city;
}

public void setCity(String city)
{
this.city = city;
}

@XmlElement
public String getState()
{
return state;
}

public void setState(String state)
{
this.state = state;
}

@XmlElement
public String getZip()
{
return zip;
}

public void setZip(String zip)
{
this.zip = zip;
}

@XmlElement
public String getCountry()
{
return country;
}

public void setCountry(String country)
{
this.country = country;
}

@Override
public String toString()
{
return "Customer{" +
"id=" + id +
", firstName='" + firstName + '\'' +
", lastName='" + lastName + '\'' +
", street='" + street + '\'' +
", city='" + city + '\'' +
", state='" + state + '\'' +
", zip='" + zip + '\'' +
", country='" + country + '\'' +
'}';
}
}


Here is my Scala conversion of Customer:



package com.restfully.shop.domain

import javax.xml.bind.annotation.XmlAttribute
import javax.xml.bind.annotation.XmlElement
import javax.xml.bind.annotation.XmlRootElement
import javax.xml.bind.annotation.XmlType
import javax.xml.bind.annotation.XmlAccessorType
import javax.xml.bind.annotation.XmlAccessType

import scala.reflect.BeanProperty

@XmlRootElement(name = "customer")
@XmlType(propOrder = Array("firstName", "lastName", "street", "city", "state", "zip", "country"))
@XmlAccessorType(XmlAccessType.FIELD)
class Customer {

@XmlAttribute
@BeanProperty
var id: Int = _

@XmlElement(name = "first-name")
@BeanProperty
var firstName: String = _

@XmlElement(name = "last-name")
@BeanProperty
var lastName: String = _

@XmlElement
@BeanProperty
var street: String = _

@XmlElement
@BeanProperty
var city: String = _

@XmlElement
@BeanProperty
var state: String = _

@XmlElement
@BeanProperty
var zip: String = _

@XmlElement
@BeanProperty
var country: String = _

override def toString =
"Customer{" +
"id=" + id +
", firstName='" + firstName + '\'' +
", lastName='" + lastName + '\'' +
", street='" + street + '\'' +
", city='" + city + '\'' +
", state='" + state + '\'' +
", zip='" + zip + '\'' +
", country='" + country + '\'' +
'}'

}


The next class I converted was com.restfully.shop.domain.Customers



This conversion was very similar, but it was dealing with collections.  I found that Scala 2.8 has a new object scala.collection.JavaConversions that contains many implicit conversion for Java collections. 



Here is the Java version of Customers:



package com.restfully.shop.domain;

import javax.xml.bind.annotation.XmlElementRef;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlTransient;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;

/**
* @author Bill Burke
* @version $Revision: 1 $
*/
@XmlRootElement(name = "customers")
public class Customers
{
protected Collection customers = new ArrayList();
protected List links;

@XmlElementRef
public Collection getCustomers()
{
return customers;
}

public void setCustomers(Collection customers)
{
this.customers = customers;
}

@XmlElementRef
public List getLinks()
{
return links;
}

public void setLinks(List links)
{
this.links = links;
}

@XmlTransient
public String getNext()
{
if (links == null) return null;
for (Link link : links)
{
if ("next".equals(link.getRelationship())) return link.getHref();
}
return null;
}

@XmlTransient
public String getPrevious()
{
if (links == null) return null;
for (Link link : links)
{
if ("previous".equals(link.getRelationship())) return link.getHref();
}
return null;
}

}


Here is my Scala conversion of Customers:



package com.restfully.shop.domain

import java.util.{ArrayList => JArrayList}
import java.util.{Collection => JCollection}
import java.util.{List => JList}

import javax.xml.bind.annotation.XmlElementRef
import javax.xml.bind.annotation.XmlRootElement
import javax.xml.bind.annotation.XmlTransient
import javax.xml.bind.annotation.XmlAccessorType
import javax.xml.bind.annotation.XmlAccessType

import scala.collection.Iterable
import scala.collection.JavaConversions._
import scala.collection.mutable.Buffer

@XmlRootElement(name = "customers")
@XmlAccessorType(XmlAccessType.PROPERTY)
class Customers {

var customers = Iterable[Customer]()
var links: Buffer[Link] = Buffer()

@XmlElementRef
def getCustomers: JCollection[Customer] = customers
def setCustomers(customers: JCollection[Customer]) = {
this.customers = customers
}

@XmlElementRef
def getLinks: JList[Link] = links
def setLinks(links: JList[Link]) = {
this.links = links
}

@XmlTransient
def getNext(): String = findHref("next")

@XmlTransient
def getPrevious(): String = findHref("previous")

private def findHref(rname: String): String = {
val rel = links.filter(_.getRelationship == rname)
rel match {
case Buffer() => null
case _ => rel.head.getHref
}
}

}


That’s about as far as I got this weekend.  I guess I was actually mostly working with JAXB this time.  The same example code has JPA as was as JAX-RS, and I’m going to try and convert the entire project to Scala, and if I do, I’ll upload it to BitBucket.

GWT 2.0 ms1 with Ivy Dependency Resolution

I decided to take a stab at finding out how hard it would be to integrate Ivy with a GWT Ant build.

Generate Project

I started out by generating the GWT application with webAppCreator.

C:/Programs/gwt-2.0.0-ms1/webAppCreator com.mostlyblather.IvyGwt

Create Ivy Configuration

Next I created the ivy.xml file. After playing around with a couple of different configurations, I settled on this one. Any future libraries added to the project would be either assigned to "compile" or "runtime" configurations. The "compile" runtime would include things like 3rd party GWT libraries. The "runtime" configuration would be used for dependencies executed in the Servlet container.


ivy.xml file:


<ivy-module version="2.0">
<info organisation="com.mostlyblather" module="ivygwt"/>
<configurations>
<conf name="runtime" description="needed in servlet container"/>
<conf name="compile" description="needed for compilation (other GWT libraries)"/>
<conf name="sdk" description="GWT SDK (compile time, seperated for Eclipse GWT IDE Integration)"/>
</configurations>
<dependencies>
<dependency org="com.google.gwt" name="gwt-servlet" rev="2.0.0-ms1" conf="runtime->default"/>
<dependency org="com.google.gwt" name="gwt-user" rev="2.0.0-ms1" conf="sdk->default"/>
<dependency org="com.google.gwt" name="gwt-dev" rev="2.0.0-ms1" conf="sdk->default"/>
</dependencies>
</ivy-module>

Modify Ant Build

In order to use the Ivy resolved dependencies, several modifications to build.xml need to be made.

First an automatic ivy installation code section.


<property name="ivy.install.version" value="2.1.0" />
<condition property="ivy.home" value="${env.IVY_HOME}">
<isset property="env.IVY_HOME" />
</condition>
<property name="ivy.home" value="${user.home}/.ant" />
<property name="ivy.jar.dir" value="${ivy.home}/lib" />
<property name="ivy.jar.file" value="${ivy.jar.dir}/ivy.jar" />

<target name="download-ivy" unless="offline">
<mkdir dir="${ivy.jar.dir}"/>
<!-- download Ivy from web site so that it can be used even without any special installation -->
<get src="http://repo2.maven.org/maven2/org/apache/ivy/ivy/${ivy.install.version}/ivy-${ivy.install.version}.jar"
dest="${ivy.jar.file}" usetimestamp="true"/>
</target>

<target name="init-ivy" depends="download-ivy">
<!-- try to load ivy here from ivy home, in case the user has not already dropped
it into ant's lib dir (note that the latter copy will always take precedence).
We will not fail as long as local lib dir exists (it may be empty) and
ivy is in at least one of ant's lib dir or the local lib dir. -->
<path id="ivy.lib.path">
<fileset dir="${ivy.jar.dir}" includes="*.jar"/>
</path>
<taskdef resource="org/apache/ivy/ant/antlib.xml"
uri="antlib:org.apache.ivy.ant" classpathref="ivy.lib.path"/>
</target>
Next, add ivy to the build.xml xml namespace.


<project name="IvyGwt" xmlns:ivy="antlib:org.apache.ivy.ant" default="build" basedir=".">



Create a new target named "resolve" to actually fetch the needed jars.

  
<target name="resolve" depends="init-ivy" description="retrieve dependencies with ivy">
<ivy:retrieve pattern="${ivy.lib.dir}/[conf]/[artifact].[ext]"/>
</target>


Additionally create a "resolve" target dependency to the "libs" target.

    
<target name="libs" depends="resolve" description="Copy libs to WEB-INF/lib">


The property definition "gwt.sdk" is no longer required so it can be removed.

The "project.class.path" property has to be changed to pick up the new Ivy provided jars.

 
<path id="project.class.path">
<pathelement location="war/WEB-INF/classes"/>
<fileset dir="lib/compile" includes="**/*.jar"/>
<fileset dir="lib/sdk" includes="**/*.jar"/>
</path>


Target "libs" similarly needs to be updated. Notice that it is only picking up the "runtime" configuration.

 
<target name="libs" depends="resolve" description="Copy libs to WEB-INF/lib">
<mkdir dir="war/WEB-INF/lib" />
<copy todir="war/WEB-INF/lib" >
<fileset dir="lib/runtime" includes="**/*.jar"/>
</copy>
</target>


The "clean" target has some additional work to do now.

 
<target name="clean" description="Cleans this project">
<delete dir="war/WEB-INF/classes" failonerror="false" />
<delete dir="war/WEB-INF/lib" failonerror="false" />
<delete dir="war/ivygwt" failonerror="false" />
<delete dir="lib" failonerror="false" />
</target>


Here is the final build.xml

 
<?xml version="1.0" encoding="utf-8" ?>
<project name="IvyGwt" xmlns:ivy="antlib:org.apache.ivy.ant" default="build" basedir=".">

<path id="project.class.path">
<pathelement location="war/WEB-INF/classes"/>
<fileset dir="lib/compile" includes="**/*.jar"/>
<fileset dir="lib/sdk" includes="**/*.jar"/>
</path>

<property name="ivy.install.version" value="2.1.0" />
<condition property="ivy.home" value="${env.IVY_HOME}">
<isset property="env.IVY_HOME" />
</condition>
<property name="ivy.home" value="${user.home}/.ant" />
<property name="ivy.jar.dir" value="${ivy.home}/lib" />
<property name="ivy.jar.file" value="${ivy.jar.dir}/ivy.jar" />

<target name="download-ivy" unless="offline">
<mkdir dir="${ivy.jar.dir}"/>
<!-- download Ivy from web site so that it can be used even without any special installation -->
<get src="http://repo2.maven.org/maven2/org/apache/ivy/ivy/${ivy.install.version}/ivy-${ivy.install.version}.jar"
dest="${ivy.jar.file}" usetimestamp="true"/>
</target>

<target name="init-ivy" depends="download-ivy">
<!-- try to load ivy here from ivy home, in case the user has not already dropped
it into ant's lib dir (note that the latter copy will always take precedence).
We will not fail as long as local lib dir exists (it may be empty) and
ivy is in at least one of ant's lib dir or the local lib dir. -->
<path id="ivy.lib.path">
<fileset dir="${ivy.jar.dir}" includes="*.jar"/>
</path>
<taskdef resource="org/apache/ivy/ant/antlib.xml"
uri="antlib:org.apache.ivy.ant" classpathref="ivy.lib.path"/>
</target>

<target name="resolve" depends="init-ivy" description="retrieve dependencies with ivy">
<mkdir dir="lib/runtime" />
<mkdir dir="lib/compile" />
<mkdir dir="lib/sdk" />
<ivy:retrieve pattern="${ivy.lib.dir}/[conf]/[artifact].[ext]"/>
</target>

<target name="libs" depends="resolve" description="Copy libs to WEB-INF/lib">
<mkdir dir="war/WEB-INF/lib" />
<copy todir="war/WEB-INF/lib" >
<fileset dir="lib/runtime" includes="**/*.jar"/>
</copy>
</target>

<target name="javac" depends="libs" description="Compile java source">
<mkdir dir="war/WEB-INF/classes"/>
<javac srcdir="src" includes="**" encoding="utf-8"
destdir="war/WEB-INF/classes"
source="1.5" target="1.5" nowarn="true"
debug="true" debuglevel="lines,vars,source">
<classpath refid="project.class.path"/>
</javac>
<copy todir="war/WEB-INF/classes">
<fileset dir="src" excludes="**/*.java"/>
</copy>
</target>

<target name="gwtc" depends="javac" description="GWT compile to JavaScript">
<java failonerror="true" fork="true" classname="com.google.gwt.dev.Compiler">
<classpath>
<pathelement location="src"/>
<path refid="project.class.path"/>
</classpath>
<!-- add jvmarg -Xss16M or similar if you see a StackOverflowError -->
<jvmarg value="-Xmx256M"/>
<!-- Additional arguments like -style PRETTY or -logLevel DEBUG -->
<arg value="com.mostlyblather.IvyGwt"/>
</java>
</target>

<target name="hosted" depends="javac" description="Run hosted mode">
<java failonerror="true" fork="true" classname="com.google.gwt.dev.HostedMode">
<classpath>
<pathelement location="src"/>
<path refid="project.class.path"/>
</classpath>
<jvmarg value="-Xmx256M"/>
<arg value="-startupUrl"/>
<arg value="IvyGwt.html"/>
<!-- Additional arguments like -style PRETTY or -logLevel DEBUG -->
<arg value="com.mostlyblather.IvyGwt"/>
</java>
</target>

<target name="build" depends="gwtc" description="Build this project" />

<target name="war" depends="build" description="Create a war file">
<zip destfile="IvyGwt.war" basedir="war"/>
</target>

<target name="clean" description="Cleans this project">
<delete dir="war/WEB-INF/classes" failonerror="false" />
<delete dir="war/WEB-INF/lib" failonerror="false" />
<delete dir="war/ivygwt" failonerror="false" />
<delete dir="lib" failonerror="false" />
</target>

</project>