418 I'm a teapot

Grails Dependency Manager Woes - This version of SLF4J requires log4j version 1.2.12 or later

2010-06-10T18:38:03Z

With the release of Grails 1.3.1 we decided that it was finally time to stop our ‘copy all the jars to lib’ process and use the awesome dependency manager.

As with any move from static lib directory to a smarter dependency manager we ran into some puzzling issues. One engineer set it all up and it worked great on our build system, but then about 50% of our engineers ran into some strange issues including this one which prevented any grails scripts from running:

...
SLF4J: This version of SLF4J requires log4j version 1.2.12 or later. See also http://www.slf4j.org/codes.html#log4j_version
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/john_doe/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.5.8.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/john_doe/.ivy2/cache/org.slf4j/nlog4j/jars/nlog4j-1.2.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
...

Hacking around randomly seemed to fix it in every case. Here’s the crude process we ended up following to resolve these strange issues:

Purge the local caches by deleting ~/.ivy and ~/.grails
Reboot (Yes this fixed the problem for someone… go figure)
Run grails upgrade over and over until it started working

What is the root cause? I have no idea. Perhaps our dependency on the public repositories is to blame: one should never trust a free third party service.

Groovy Deadlocks

2010-05-28T02:01:45Z

Today I managed to set up my grails environment in such a way that it triggered deadlocks quite easily when stressed with a simple jmeter script. While this was unintentional, it did give me an opportunity to diffuse some ticking time bombs.

A few of the deadlocks were caused by some of the creative ways that my project uses the Groovy on Grails APIs for rendering pages from GSP code stored in a database, and with some tweaks most of those were resolved.

Some other deadlocks, though, seem to be the fault of Groovy itself. A quick google search turned up over 300 hits relating to deadlocks in the Groovy bug tracker:

http://www.google.com/search?hl=en&q=site:jira.codehaus.org+%2Bgroovy+%2Bdeadlock

This is when I had a sad.

I know that as an open source project if a bug really bothers me, I should submit a patch, which I am likely to do, but even if I find fixes for some of the deadlocks I’m facing, I’ll either have to wait for the changes to make it into the Groovy codebase, Groovy to release, grails to pick up the changes, etc. or I’ll have to hack the patch into my application and hope for the best.

Hopefully I’ll find some magic work around that does not tickle these bugs :)

Update: Someone must have seen this blog entry, or maybe they’re just cool. Anyway, they fixed this problem wicked fast: http://jira.codehaus.org/browse/GRAILS-6398 Grails continues to rock as a result.

iPad and OneNote

2010-05-26T19:22:39Z

Tablet computing is one technology that I’ve been attempting to early-adopt for years, but it’s never quite been there. What I’ve always dreamed of is the PADD device that everyone walked around with in Star Trek TNG: A small, light, sleek, multifunctional device that accepts pen input, accepts touch input, and is reasonable to carry around.

Around 2003 I purchased a Toshiba m200. It was a hybrid Windows XP Tablet computer. It had a keyboard but could also be converted into tablet mode. The operating system was a clunky afterthought on Windows XP, but it did have one shining star in the product offering (and it was not stupid marble game). It was OneNote. At the time everything Microsoft made seemed clunky and over-complicated, but OneNote did everything that made sense, and nothing else. It was, and still is, a great product.

I stopped using it, though. The application was great, but the operating system and hardware were just not there yet. Also people kept asking me why I was drawing on my laptop (since tablet computers were still fringe).

Now its 2010. Everyone has heard of the iPad. Tablet computing has gone mainstream but there does not seem to be a single killer OneNote like application for the iPad. This may have something to do with the lack of an included stylus. This void, however, can be filled using a combination of applications.

First off, you’ll need a system for organizing those notes. This is where Evernote comes in handy. The interface is clean, it supports cool features like automatic geotagging and you can even attach voice sound clips to your notes. Even better than OneNote, it supports automatic synchronization between all sorts of devices. It’s pretty mature, and has been around for awhile, but the iPad is the killer device for this service.

Unfortunately, since it started as a web browser / desktop application, Evernote does not yet allow you to scribble those quick notes. On the bright side it does allow you to email content to your primary notebook, and this is the trick.

For your stylus sketches and note taking, go grab a copy of Penultimate. Of the dozen or so drawing / sketching / scribbling apps I’ve tried, this one is by far the best. Input is clean and precise. It is also the only one I’ve found that works while your wrist is rested on the screen, which makes scribbling a lot more comfortable.

Once you’re done with that scribbled note in penultimate, use the ‘email this note’ feature to fire it off to Evernote. A quick synchronisation and there it will appear in your notebook.

It’s not perfect yet, and it’s a bit awkward, but it works well enough for me!

Checking opensolaris physical memory inside a zone

2010-05-04T20:55:36Z

Joyent uses OpenSolaris zones for its accelerators. At some point I needed to verify the physical memory size of one of these zones but was unable to use the webmin tool that Joyent provides. This seemingly simple operation was actually pretty tricky to figure out. Here are the steps I followed:

 
$ sudo rcapadm -E
                                      state: enabled
           memory cap enforcement threshold: 0%
                    process scan rate (sec): 15
                 reconfiguration rate (sec): 60
                          report rate (sec): 5
                    RSS sampling rate (sec): 5
$ rcapstat -z 1 1
    id zone            nproc    vm   rss   cap    at avgat    pg avgpg
    46 foo            -    0K    0K 2048M    0K    0K    0K    0K
$ sudo rcapadm -D
                                      state: disabled
           memory cap enforcement threshold: 0%
                    process scan rate (sec): 15
                 reconfiguration rate (sec): 60
                          report rate (sec): 5
                    RSS sampling rate (sec): 5
$

rcapadm seems to be a resource management tool. I’m not sure if there’s an impact to leaving it running, but I disabled it just in case.

Grails Override Configuration with Properties File

2010-04-06T23:30:25Z

Sometimes you will need to switch up configuration in a hurry. For example, you may need to switch the database connection URL in order to use your warm standby database. This can be a pain in Grails because the convient-during-development Config.groovy is compiled both by grails run-app and grails war.

The simplest way to change configuration without rebuilding and redeploying your Grails application is to make use of an externalized configuration file. Although would appear to be very straight forward based on the docs, you will need to jump through a couple undocumented hoops to get it working in all environments.

Create your override configuration file in the grails-app/conf folder. I called mine override.conf. For now you can leave it empty.
Enable this external configuration file by pasting this into the top of your grails-app/conf/Config.groovy file:

grails.config.locations = [ "classpath:override.properties"  ]

This is where the docs end. When you start up grails you’ll see the following error message:

Unable to load specified config location classpath:override.properties : class path resource [override.properties] cannot be opened because it does not exist

1. Grails uses different paths in run-app mode and when being run as a war. We’re going to need to add a step post compilation to ensure our configuration file is available at all times. We’ll do this by modifying _Events.groovy (or Events.groovy if you’ve been migrating your grails app since v1.1). Open scripts/_Events.groovy, or create it if it does not exist. 1. Add the following to _Events.groovy (or merge it in if you already have an eventCompileEnd defined):

eventCompileEnd = {
    ant.copy(todir:classesDirPath) {
      fileset(file:"${basedir}/grails-app/conf/override.properties")
    }
}

Try setting a new property and fire up grails as usual. For example, if you want to set the database URL, you’d add this to your grails-app/conf/override.properties file:

dataSource.url=jdbc:postgresql://example.host/somedb:shutdown=true

Source: Some Grails - user mailing list post by Ian Roberts

Zeus ZXTM Upgrade Cannot fork: Not enough space

2010-03-22T20:15:32Z

While upgrading one of my Zeus ZXTM traffic managers from v5.1r2 to v6.0r4 it crashed on startup. This pretty surprising because the upgrade process appeared to have proceeded without a hitch. Here’s the error I saw in the logs and when I attempted to fire up the zxtm program using SSH:

$ ./start-zeus
Initializing Zeus Application Framework. (C) 1995 - 2010 Zeus Technology Limited
Zeus Administration Server already running: 235
Zeus Traffic Manager - (C) 1995 - 2010 Zeus Technology Limited
Version 6.0r4, Build date: Feb 10 2010 08:32:37
Process permissions set to zeus:zeus
 INFO   Zeus Traffic Manager starting
 INFO   Version 6.0r4, Build date: Feb 10 2010 08:32:37
 FATAL Parent 1234 hit FATAL at Cannot fork:Not enough space
[0x6ac417] function __1cOcommkeyChanged6FpknNConfigSection_rknKStringBase_pknKConfigFile_p6_nIRetValue__ + 0x417
[0x8d4376] function __1cFFATAL6Fpkc1i_v_ + 0x66
[0x6ad2b3] function __1cUreally_nice_shutdown6F_v_ + 0x883
[0x6b3a3b] function __1cKParentBoot6Fpkc_v_ + 0xa8b
[0x5e83c1] function main + 0x571
[0x5c532c] function _start + 0x6c
[0x0] function ?? + 0xffffffffffa3ad40
$

Not enough space? Something was seriously amiss. The solution turned out to be pretty simple, but was not something I could find in the manual.

In v6.0, a new configuration parameter was added: sharedpoolsize. This was not set since I was upgrading from v5.1. The ZXTM made its best guess, it guessed wrong on my virtualized environment and picked a value that exceeded the memory available to my zone. The fix was simple: set sharedpoolsize in $ZEUSHOME/zxtm-6.0r4/conf/settings.cfg to a size small enough to fit into my available memory. Since this configuration did not exist, I mad to add it at the bottom of the file.

GoDaddy SSL Certificate in Zeus ZXTM Traffic Managers

2010-03-04T22:58:45Z

Almost a year ago I purchased a wildcard SSL certificate from GoDaddy. When it was first issued I simply loaded it into into my Zeus ZXTM load balancer with the import button.

Everything seemed fine for quite awhile. I visited my web site in Firefox and in all of the flavors of IE. It seemed to work great. That is, until, someone called me to let me know that Safari was not accepting the certificate! I thought I was in trouble until I googled around and found many blog entries about the root cause. It turns out that my server, or in this case my Zeus ZXTM load balancer, was not configured to display the whole certificate chain back to the root authority.

This makes me wonder how this worked at all in every other web browser. Perhaps this is such a common problem that the other browsers hack around it?

Anyway, the fix was easy enough but the terminology was different. Rather than an SSLCertificateChainFile, my ZXTM called it an ‘Intermediate Certificate’. One click of a button, browsing to gd_bundle.crt (provided with my original certificate), and it was loaded up and the issue was fixed.

channel 2: open failed: administratively prohibited: open failed

2010-03-02T23:16:57Z

I attempted to set up an SSH tunnel today to access a web application that was only bound to a private IP. I tunneled to my opensolaris box, which worked fine, and then attempted to establish a connection with telnet. Telnet reported that the connection was refused and I saw this on my command line for the tunnel:

$ ssh johndoe@foo.bar.com -L 8080:foo.bar.com:8080 -N
channel 2: open failed: administratively prohibited: open failed

I made a pretty silly mistake, but I figure it may save someone’s time to post it here. Of course the connection was refused. My ssh tunnel attempts to establish the connection to the public domain name, which uses the public IP (the same one I’m SSHing into). The web server I’m trying to reach is not bound to that adapter!

Here’s a correct command:

$ ssh johndoe@foo.bar.com -L 8080:localhost:8080 -N

Java Mime Type Content Type

2010-02-25T22:55:37Z

I’ve been working on a Java servlet these days which for many reasons needs to determine the mime type of files so that files being rendered are correctly used by my clients. For example, I’m generating a crossdomian.xml, which if not matched with the text/xml content type, really confuses flash players.

Anyway, I fumbled around for a bit in Google and found one great solution: mime-util. It’s Apache License and works great. The documentation is even very good.

Since I can trust my file names I opted for the simple file name extension detection method. It’s a perfect fit except for one thing: I need to detect the application/json content type which is not part of the default property bundle.

It took me a few minutes of digging trough the JavaDocs to find the answer, so to save you some time, here’s my solution:

Create a file named mime-types.properties and put it somewhere in your class path.
Add the types you need as extension=type pairs like json=application/json

Java Mime Type Content Type

2010-02-25T22:55:37Z

Anyway, I fumbled around for a bit in Google and found one great solution: mime-util. It’s Apache License and works great. The documentation is even very good.

It took me a few minutes of digging trough the JavaDocs to find the answer, so to save you some time, here’s my solution:

Create a file named mime-types.properties and put it somewhere in your class path.
Add the types you need as extension=type pairs like json=application/json

E575: viminfo: Illegal starting char in line:

2010-02-18T20:19:56Z

Once upon a time I tried to open a file for editing with MacVim and saw this message:

$  vi foo.xml 
E575: viminfo: Illegal starting char in line: ion.properties
$

I assumed there was something wrong with the file that I was editing, so I tried a different file. I observed the same result.

**The Problem Your .viminfo file is corrupt. This file does not contain settings or anything, mostly temporary and working data.

**The Fix Delete your ~/.viminfo file

Grails restart loop

2010-01-27T23:35:55Z

The Problem

The other day I was doing some bulk file operations on a grails project. I was moving some classes in bulk from a plug-in into an application. After I made the move I ran into a puzzling issue: grails kept restarting in a loop. Each time the plugin scanner ran (every 5 seconds after startup), grails would complain that it needed to compile 1 file. This went on for a few minutes until my app ran out of PermGen and crashed. To make matters worse, grails would not tell me which file it was recompiling even with logging set to trace!

This seems to be a classic grails issue that is rare, but frustrating.

The Cause

The likely cause is a groovy file outside the correct package or with a file name that does not match the case name. The latter can happen when tinkering around between case sensitive and case insensitive file systems as is common when developing with OS X.

The Fix

To fix it you need to find the messed up class and fix it. The below method is crude, but works.

If your app crashes quickly due to PermGen issues, provide more PermGen space. You’ll need at least 2 or 3 minutes of run time for this to work. Use the JVM argument -XX:MaxPermSize=1024m if you have to.
Let your app run for a few minutes. Let it recompile that file a few times.
If your app has not already crashed, stop it
Now to find the compiled class file. Go to the directory where grails claims to be compiling the file. In my case it was in ~/.grails/1.2.0/projects/fooApp/classes
Run this shell command to see when most of the files were updated: find ./ | xargs ls -al The vast majority of the listed files should all have an updated time within the same minute. In my case it was 15:17 Lets fileter those out and see what’s left behind.
Run another find command and see the newer files: find ./ | xargs ls -al | grep -v 15:17. I saw a few files and headers that had to be ignored, but in the middle I found the class name which was compiled from a .groovy file of a different name.
Go back to your app code and resolve the conflicted state of your groovy file(s)
Quit cursing under your breath and go have a beer if you need one

Simple Basic Authentication from Java Servlet Example

2010-01-04T19:13:17Z

Basic Auth is a very simple way to secure your web application. When combined with a secure transport technology like SSL, it’s also good enough in most cases.

Here is a dirt simple example of how to access the basic authentication information from the HTTP header in your servlet.

protected void doGet(HttpServletRequest request, HttpServletResponse response) {
    String pathInfo = request.getPathInfo();
    //will contain "Basic Ym9iOnNlY3JldA=="
    String header = request.getHeader("Authorization"); 
    //always wise to assert your assumptions
    assert header.substring(0, 6).equals("Basic "); 
    //will contain "Ym9iOnNlY3JldA=="
    String basicAuthEncoded = header.substring(6);
    //will contain "bob:secret"
    String basicAuthAsString = new String(
        new Base64().decode(basicAuthEncoded.getBytes()));
    ...
}

You can test it with this curl command:

$  curl -vv http://bob:secret@localhost:8090/blah
* About to connect() to localhost port 8090 (#0)
*   Trying ::1... connected
* Connected to localhost (::1) port 8090 (#0)
* Server auth using Basic with user 'bob'
 GET /blah HTTP/1.1
 Authorization: Basic Ym9iOnNlY3JldA==
 User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4 OpenSSL/0.9.8k zlib/1.2.3
 Host: localhost:8090
 Accept: */*
...

grails Exception in thread "main" java.lang.NoClassDefFoundError: org/codehaus/groovy/grails/cli/support/GrailsStarter

2009-12-31T21:35:57Z

Maybe you just upgraded Groovy on Grails, or maybe you’re doing a fresh install. Either way you may be confronted with this curious error message:

$ grails test
Exception in thread "main" java.lang.NoClassDefFoundError: org/codehaus/groovy/grails/cli/support/GrailsStarter

What’s wrong? You have a grails mismatch. Your GRAILS_HOME environment variable is not pointing to the same place as the grails in your PATH. Make them agree and try your grails command again.

System call sent warnings to stderr: pst3: This program can only be run by the root user!

2009-12-08T02:49:59Z

I ran into this very cryptic one while setting up Nagios at Joyent. I copied my plugins from one nrpe client to a new server. Three of my checks used check_procs which all failed with a message like this:

check_procs
System call sent warnings to stderr: pst3: This program can only be run by the root user!

To make this even more annoying, sudo did not fix it. The same error message was displayed. What was the problem? File permissions! The error message should say “This program must be owned by the root user!” The fix:

sudo chmod root:root pst3