May 2009 Archives

This is a simple one.

Mysteriously the auto-compile for controllers (but not views) in my grails v1.1.1 application stopped working.  It worked fine for the other engineers on this project so I did the normal poking around.  Surprisingly I could not find anything on Google.

To fix this problem, try deleting your ~/.grails directory.  You'll need to download all of your plugins again, but autocompile will work once again.

Like many people in the web industry I’ve been using MySql for quite some time as part of LAMP projects). When I started my current project using PostgreSQL I looked around for easy replication technique. I found a couple including PGCluster and Slony, but they seemed much more complicated to set up than replication in MySql.

The solution was actually right there in the PostgreSQL manual, I was just googling with the wrong terms. I should have been looking for WAL PITR (seems so obvious now!).

Anyway, these acronyms, which combined and expanded stand for Write Ahead Log Point in Time Recovery, provide a built in solution to disaster recovery. This technique is also very well documented in the online manual.

The gist of the solution is that you take a snapshot of the database and then archive incremental changes by copying the write ahead logs. Your backup DB hangs out and consumes these logs as they’re produced which keeps it quite close to the production DB. When your primary DB server fails, you let the backup DB process any remaining logs and then point your applications at it.

This has the advantage of being easy to set up and official supported, but it only provides a warm backup and required manual intervention in the case of a hardware failure. Also, since the default WAL file size is 16mb, you will lose up to 16mb of data in the standard configuration which may be a big deal for low activity databases.

In spite of these flaws, I opted for this solution at least to get started. Later when I have more load I’ll revisit proper replication.

Grails does not like loading the same class twice.  I've observed this several times in Grails v1.1.1 and each time there was a different fix.  The stack traces look pretty similar no matter what the root cause.

Example stack trace:

[groovyc] Compiling 33 source files to /Users/me/.grails/1.1.1/projects/trunk/classes
Error executing script TestApp: : java.lang.LinkageError: loader constraint violation: loader (instance of ) previously initiated loading for a different type with name "org/w3c/dom/Attr"
gant.TargetExecutionException: : java.lang.LinkageError: loader constraint violation: loader (instance of ) previously initiated loading for a different type with name "org/w3c/dom/Attr"
	at gant.Gant$_dispatch_closure4.doCall(Gant.groovy:331)
	at gant.Gant$_dispatch_closure6.doCall(Gant.groovy:334)
	at gant.Gant$_dispatch_closure6.doCall(Gant.groovy)
	at gant.Gant.withBuildListeners(Gant.groovy:344)
	at gant.Gant.this$2$withBuildListeners(Gant.groovy)
	at gant.Gant$this$2$withBuildListeners.callCurrent(Unknown Source)
	at gant.Gant.dispatch(Gant.groovy:334)
	at gant.Gant.this$2$dispatch(Gant.groovy)
	at gant.Gant.invokeMethod(Gant.groovy)
	at gant.Gant.processTargets(Gant.groovy:495)
	at gant.Gant.processTargets(Gant.groovy:480)
Caused by: : java.lang.LinkageError: loader constraint violation: loader (instance of ) previously initiated loading for a different type with name "org/w3c/dom/Attr"

Case 1: /lib/ jar file conflicts with a Grails plugin

I was using a functional test plugin which included xerces.  I already had xerces in the /lib folder of my grails project.

Solution to Case 1:

Remove the jar file from /lib as long as I was using this plugin...  I found the conflicting jar file the crude way: grep -r 'org.w3c.dom.Attr' ./ This solution is kind of messy but solved the problem until I stopped using this functional testing plugin at which time I had to add the jar back into the /lib/ folder.

Case 2: A maven assembly jar-with-dependencies contained a class that conflicted with a grails dependency

In this case I was using a library of my own creation built in maven. To keep the dependency nightmare to a minimum, I used the super-jar provided by the assembly plugin to package all of the dependent classes into my library. This works great in most cases, but as it turns out one of the included libraries, Xerces, is also used by grails internally.

I discovered the conflicting jar files the same way as last time: grep

Solution to Case 2:

I didn't find a satisfactory solution to this issue. Lacking time to investigate the inclusion / exclusion features of the maven assembly plugin, I simply used the standard maven jar and selectively copied the dependencies into the /lib folder of my grails project.

This 'solution' is fairly unsatisfying, but for now it works. Once I end up with dependency hell I'll revisit this issue and update this blog entry. In the mean time if anyone else solves this problem, please comment.

Other Solutions

Although I have not run into any of these solutions, other people have had luck with them

• Deleting ~/.ant/lib
• Removing JVM provided libraries that conflict

Grails comes with a database schema manager that is perfect for development and testing.  It's fast, transparent, and sometimes magical.  Being magic, though, it gets confused on occasion such as when you do a roll back to an older application version.  In development and testing this is easily solved by dropping the DB.  This is not sufficient for production.

It's time for me to explore production database management.  It seems that there are a few options:

• Liquibase

The most mature player.  It's been proven on many other projects that I've done.  It has two big downsides, though.  First off, it's another hand-edited XML configuration nightmare, which is precisely why I had stopped developing Java web applications for years (yes, I'm talking about you Spring).  Second, although it does have a grails plug in, the plug in does not actually do much except script some commands.  Integration is hence very loose.

• dbmigate

I read that this behaves a lot like the rails which has an OK database migration technique, although I always had challenges merging migrations from many developers and hated the 1000 migrations that built up after a few months of development.

I did not test this tool very much since at the time of this blog post, it seems to be an abandoned project.

• Autobase

This tool is still a baby, but it looks incredibly promising.  It claims to offer the features of Liquibase with the convience of the built in grails migration tool.  It's also being actively developed.

While I'd love to start contributing to Autobase, I'm afraid that I don't have time to dive into that project at the moment.  In the end I decided to start with Liquibase and keep a close eye on Autobase.  Perhaps in a couple of months I'll be able to ditch Liquibase and be free of XML files once again.

SQL logging is important not only for debugging problems but for keeping an eye on the database usage of your application.  No matter how efficient we try to be frameworks that isolate us from the database often result in problems like N+1 queries which can be catastrophic on large production data sets.

To enable SQL Logging in Grails v1.1.1, just add a single line to your data source configuration in /grails-app/conf/DataSource.groovy.

loggingSql=true

For example, if our development data source looks like this:

environments {
    development {
        dataSource {
            dbCreate = "update"
            url = "jdbc:postgresql:foobar"
        }
    }
}

This will be our version with SQL Logging enabled:

environments {
    development {
        dataSource {
            dbCreate = "update"
            url = "jdbc:postgresql:foobar"
            loggingSql=true
        }
    }
}

The time came for me to export data to Excel in my partially enterprise targeted application built on Grails v1.1.1.  My first impulse was to go for a grails plugin.  The export plugin seemed to be pretty close to what I needed.  Sadly I could not get it to work and anyway I need to export denormalized data that spans several domain objects.

Having given up on the plugin I hit google and found a really good solution that seems to offer about the best possible performance short of pregenerating the reports. 

This solution which uses Hibernate Projections seems like a good candidate for a plugin.

If you've found this entry, you probably ran into your first SVN Tree conflict. This is also called an 'evil twin conflict' in some other revision control systems.

The message generated by svn merge or later svn stat operations will look something like this:

A     C web-app/images/widget.png
      >   local obstruction, incoming add upon merge
...
Summary of conflicts:
  Tree conflicts: 1

The Cause

This special conflict message is created when the same file has been added to both the place your merging from as well as the place your merging to since the last merge. Since these evil twins both have completely different histories and no common state (as would exist if the image had been added before you branched), svn is totally unable to provide you advice. This is why you will see no merge-left or merge-right files.

The Solution

In the example above the same binary file has been added to both a branch and a trunk. Since the image is identical, the solution was simply to pick one. I picked the working copy.

svn resolve --accept working ./web-app/images/widget.png

Another possible case is that two totally different files have been checked in with the same name and path. In this case, you're going to need to rename one version and refactor the rest of your code to accommodate that name change. SVN has no easy way to do this so you'll need to do it manually.

The last case is that the same file has been created but you need a super set of the functionality provided by both versions. Again, SVN does not offer any speedy tools for this so you'll simply need to do a manual merger.

There are a lot of great validations built into Grails v1.1, but they are all focused on the object that is being validated. There are none that are capable of inspecting relationships. This is when we need to take advantage of the powerful custom validators that can be created with Grails. Here's an example of a simple case. I have a car class and I want to make sure that my car has at least one tire. < class="brush: groovy"> class Car { static hasMany[tires:Tire] static constraints = { tires(validator: { val, obj -> def retval = true if (!obj?.tires?.size()) { retval = 'car.validator.hasnotires.error' } return retval }) } } One tire a useful car does not make, though. We need four tires on our car. Let's tweak this to require exactly four tires. It's an easy change of just a few characters.

class Car {
  static hasMany[tires:Tire]
  static constraints = {
    tires(validator: { val, obj ->
      def retval = true
      if (!obj?.tires?.size() != 4) {
        retval = 'car.validator.haswrongtirecount.error'
      }
      return retval
    })
  }
}

It's easy to write similar validators that check for children of children, more detailed aspects of children, and so on. One more note. Although it may be obvious to some of you, it has tripped me up at least once. To save these data structures you must save them depth first. This means all children must be saved before you can save any parent object otherwise this validator will save since your children objects (your tires) will still be considered transient.

Was your grails v1.1 app just working and suddenly reports something like this whenever you fire it up? I bet your data source is set to 'create-drop' too.

2009-05-12 17:32:09,671 [main] ERROR hbm2ddl.SchemaExport  - Unsuccessful: create table foo (id int8 not null, ...
2009-05-12 17:32:09,672 [main] ERROR hbm2ddl.SchemaExport  - ERROR: relation "foo" already exists
2009-05-12 17:32:09,694 [main] ERROR hbm2ddl.SchemaExport  - Unsuccessful: create table bar (id int8 not null, ...
2009-05-12 17:32:09,694 [main] ERROR hbm2ddl.SchemaExport  - ERROR: relation "bar" already exists
2009-05-12 17:32:09,695 [main] ERROR hbm2ddl.SchemaExport  - Unsuccessful: create table myfoo (id int8 not null,...
2009-05-12 17:32:09,695 [main] ERROR hbm2ddl.SchemaExport  - ERROR: relation "myfoo" already exists
2009-05-12 17:32:09,703 [main] ERROR hbm2ddl.SchemaExport  - Unsuccessful: create table mybar (id int8 not null,...
2009-05-12 17:32:09,703 [main] ERROR hbm2ddl.SchemaExport  - ERROR: relation "mybar" already exists

Well you're in luck. I've discovered three possible causes and they're all easy to fix:

• Your database is locked by an existing grails process.  This happens to me sometimes when I close my IDE (IntelliJ) without killing the process.  Just kill the orphaned process to resolve this.
• You have some DB tool connected to your database which is preventing the drop command from being executed.  Simply close all of your database tools.
• You are attempting to do a backwards migration.  If you have moved from a new schema to an older one (revisiting an old tag, for instance) Grails may not be able to drop the DB.  Drop and recreate it by hand.

If none of these work, and you find another root cause, please drop me a comment and I'll update this list.

I've recently started using Joyent for my production hosting needs at work.  It's been a great experience so far.  The virtual hardware is great (very fast and never crashes), their support has been very helpful, and if you have a bandwidth intense app their prices are impossible to beat.

Now that things are starting to get up and running at Joyent, I decided it was time to set up monitoring.  Many people on Joyent use munin, but I'm more familiar with Nagios so I decided to give it a go.  It went pretty well, but I did run into a couple hiccups so I'm documenting my installation here.

1. Create the nagios user and group if they're not there already

   groupadd nagios
   useradd -g nagios -s /usr/bin/false -d /wherever -c 'Nagios User' nagios

2. Run the package installer if nagios isn't already installed (as it turns out I already had nagios installed)

   pkg_add http://pkgsrc.joyent.com/2008Q3/net/nagios-base-3.0.3.tgz

3. Create a virtual host for Nagios

   sudo vi /opt/local/etc/httpd/virtualhosts/nagios.conf

   with these contents:

   ScriptAlias /nagios/cgi-bin "/home/jill/cgi-bin/nagios"
   
   <Directory "/home/jill/cgi-bin/nagios">
       AllowOverride None
       Options ExecCGI
       Order allow,deny
       Allow from all
       AuthName "Nagios Access"
       AuthType Basic
       AuthUserFile /opt/local/etc/nagios/htpasswd.users
       Require valid-user
   </Directory>
   
   Alias /nagios "/opt/nagios/share"
   
   <Directory "/opt/nagios/share">
      Options None
      AllowOverride None
      Order allow,deny
      Allow from all
      AuthName "Nagios Access"
      AuthType Basic
      AuthUserFile /opt/local/etc/nagios/htpasswd.users
      Require valid-user
   </Directory>
   

4. This should have been it, but I was seeing a cryptic error from suExec. As it turns out suExec was preventing them from executing. Since I didn't have the time to learn suExec, I took the easy way out and moved the CGI files to the docroot (which was the seed user's folder)

Jetty 6 is my favorite servlet container.  It's super fast, easy to embed, and very easy to use.  I have run into one issue pretty often, though.

The built in jetty.sh script often hangs when given the stop command.  This isn't a big deal when I'm doing development, but it's a problem when I'm trying to bounce Jetty in a test environment from continuous integration software like Hudson.

To solve this problem I came up with a very crude shell script to kill all jetty processes (including hung jetty.sh stop processes).  It also manually exits 0 to feign success even if no Jetty is found.  This keeps Hudson happy.

killjetty.sh:

#!/usr/xpg4/bin/sh
ps -ef | grep jetty | grep -v killjetty | grep -v grep | awk '{ print $2 }' | xargs kill -9 
exit 0

This issue has gotten me more than once. I've just checked a Grails project out of SVN and I fire it up. It complains that it 'Could not open ServletContext resource [/WEB-INF/applicationContext.xml]'. What the heck does that mean? I didn't do any xml hacking. Grails is supposed to take care of this, right? Well, kind of. If you followed the '[how to check grails into svn](http://www.grails.org/Checking+Projects+into+SVN)' document, you did not check in the WEB-INF folder. This is your problem. To fix this issue, just run this command and respond 'yes' to any prompts:

grails upgrade

Just to validate that you're experiencing the same issue, here's exactly what I see when I run into this problem:

$  grails run-app
Welcome to Grails 1.1 - http://grails.org/
Licensed under Apache Standard License 2.0
Grails home is set to: /local/dev/grails-1.1

Base Directory: /local/grails_app/trunk
Running Grails application..
2009-04-29 13:21:03,085 [main] ERROR context.ContextLoader  - Context initialization failed
org.springframework.beans.factory.BeanDefinitionStoreException: IOException parsing XML document from ServletContext resource [/WEB-INF/applicationContext.xml]; nested exception is java.io.FileNotFoundException: Could not open ServletContext resource [/WEB-INF/applicationContext.xml]
	at grails.web.container.EmbeddableServer$start.call(Unknown Source)
	at _GrailsRun_groovy$_run_closure5_closure11.doCall(_GrailsRun_groovy:145)
	at _GrailsRun_groovy$_run_closure5_closure11.doCall(_GrailsRun_groovy)
	at _GrailsSettings_groovy$_run_closure10.doCall(_GrailsSettings_groovy:269)
	at _GrailsRun_groovy$_run_closure5.doCall(_GrailsRun_groovy:137)
	at _GrailsRun_groovy.runInline(_GrailsRun_groovy:104)
	at _GrailsRun_groovy.this$4$runInline(_GrailsRun_groovy)
	at _GrailsRun_groovy$_run_closure1.doCall(_GrailsRun_groovy:58)
	at RunApp$_run_closure1.doCall(RunApp.groovy:33)
	at gant.Gant$_dispatch_closure4.doCall(Gant.groovy:324)
	at gant.Gant$_dispatch_closure6.doCall(Gant.groovy:334)
	at gant.Gant$_dispatch_closure6.doCall(Gant.groovy)
	at gant.Gant.withBuildListeners(Gant.groovy:344)
	at gant.Gant.this$2$withBuildListeners(Gant.groovy)
	at gant.Gant$this$2$withBuildListeners.callCurrent(Unknown Source)
	at gant.Gant.dispatch(Gant.groovy:334)
	at gant.Gant.this$2$dispatch(Gant.groovy)
	at gant.Gant.invokeMethod(Gant.groovy)
	at gant.Gant.processTargets(Gant.groovy:495)
	at gant.Gant.processTargets(Gant.groovy:480)
Caused by: java.io.FileNotFoundException: Could not open ServletContext resource [/WEB-INF/applicationContext.xml]
	... 20 more
2009-04-29 13:21:03,086 [main] ERROR mortbay.log  - Failed startup of context org.mortbay.jetty.webapp.WebAppContext@61c8cd04{/grails_app,/local/svncopy/grails_app/trunk/web-app}
org.springframework.beans.factory.BeanDefinitionStoreException: IOException parsing XML document from ServletContext resource [/WEB-INF/applicationContext.xml]; nested exception is java.io.FileNotFoundException: Could not open ServletContext resource [/WEB-INF/applicationContext.xml]
	at grails.web.container.EmbeddableServer$start.call(Unknown Source)
	at _GrailsRun_groovy$_run_closure5_closure11.doCall(_GrailsRun_groovy:145)
	at _GrailsRun_groovy$_run_closure5_closure11.doCall(_GrailsRun_groovy)
	at _GrailsSettings_groovy$_run_closure10.doCall(_GrailsSettings_groovy:269)
	at _GrailsRun_groovy$_run_closure5.doCall(_GrailsRun_groovy:137)
	at _GrailsRun_groovy.runInline(_GrailsRun_groovy:104)
	at _GrailsRun_groovy.this$4$runInline(_GrailsRun_groovy)
	at _GrailsRun_groovy$_run_closure1.doCall(_GrailsRun_groovy:58)
	at RunApp$_run_closure1.doCall(RunApp.groovy:33)
	at gant.Gant$_dispatch_closure4.doCall(Gant.groovy:324)
	at gant.Gant$_dispatch_closure6.doCall(Gant.groovy:334)
	at gant.Gant$_dispatch_closure6.doCall(Gant.groovy)
	at gant.Gant.withBuildListeners(Gant.groovy:344)
	at gant.Gant.this$2$withBuildListeners(Gant.groovy)
	at gant.Gant$this$2$withBuildListeners.callCurrent(Unknown Source)
	at gant.Gant.dispatch(Gant.groovy:334)
	at gant.Gant.this$2$dispatch(Gant.groovy)
	at gant.Gant.invokeMethod(Gant.groovy)
	at gant.Gant.processTargets(Gant.groovy:495)
	at gant.Gant.processTargets(Gant.groovy:480)
Caused by: java.io.FileNotFoundException: Could not open ServletContext resource [/WEB-INF/applicationContext.xml]
	... 20 more
Server running. Browse to http://localhost:8080/grails_app

A few weeks back I posted a <a href="http://www.mikenicholson.net/2008/12/what_rails_could_do_better.php">vent blog entry</a> about some shortcomings of Ruby.  Well, it looks like one of my concerns has been at least partially resolved.

<a href="http://dev.ctor.org/">Hiroshi Nakamura</a> created, and continues to maintain, a wrapper around Net::Http called <a href="http://dev.ctor.org/http-access2">httpclient</a> which adds enough functionality to save lots of time.  Sure the performance still sucks compared to curl, but with this great library you can do <a href="http://dev.ctor.org/doc/httpclient/classes/HTTPClient.html#M000015">multipart form posts</a> and <a href="http://dev.ctor.org/doc/httpclient/classes/WebAgent/Cookie.html">basic cookie functionality</a>.  Now my rails app can finally make use of some more complex HTTP based APIs out there (not everything is simple REST).

<a href="http://dev.ctor.org/">Hiroshi</a>, you rock, but back to my other complaint about Ruby stuff being scattered about the wind, please submit your gems to one of the big repositories so we can find them :)

Hey Everyone.

Welcome to my new technical blog.  I've begun to post lots of technical stuff, especially related to Groovy on Grails, on my personal blog.  Unsurprisingly this is not the kind of content the few readers I have over there wanted to see.

So here we go.  Get ready for lots of blog entries about cryptic Groovy on Grails 1.1 error messages and QQing that 'Java sucks at X, Y and Z'.

And, no, this blog neither implements HTCPCP nor the even more esoteric HTTPCP (HyperText Tea Pot Control Protocol)