Niggle: History, Current Status and Future

History

The Niggle framework really dates back to the summer of 1999. I was working on the initial prototype of the Medimom site, for Reminders On-Line (ROL). At that stage, we were using a home-grown page template system that had a lot of limitations. I had become aware of the open source Freemarker library which I preferred. Vastly. It was far less awkward to use than the template system we were using. The problem was that we already had a fair bit of existing work, both on the java coding and graphical design front, based on the other template library. Switching over all of the existing page templates to Freemarker seemed like a daunting task, the kind of thing that typically never happens, because it requires a huge all-at-once expenditure of energy and it never quite seems worth it at any given moment.

Still, I was determined to make the switch. I started off by creating a higher-level API whereby the two template systems could coexist, albeit temporarily. I created a Page interface and the concrete implementations LegacyPage and FreemarkerPage and refactored the code so that it only used the higher-level Page interface. Once I created a PageFactory that vended the appropriate concrete instance, I could then gradually change the implementations from LegacyPage to FreemarkerPage one by one over a period of time, while still having a working system throughout. This worked out quite well. Once I had finally converted all of the page templates from the LegacyPage to FreemarkerPage, I got rid of the older implementation class, yet kept our application code using the more abstract Page/PageFactory API, in the full knowledge that, if we found a better presentation solution than Freemarker, we could switch over to that in a similar way. (As it turned out, we never did find a better presentation solution than Freemarker. Or at least, Freemarker was good enough that we never actively looked for anything any better.) As of this writing there are implementations of this API that wrap the popular WebMacro and Velocity template engines.

Actually, I would not say that this is the point at which Niggle was born. But it was gestating. Meanwhile, another front was opening up -- the data side. It was becoming increasingly evident that there was an ugly error-prone coding pattern that was constantly repeated throughout the code. A user would submit data in an HTML form and we would create a data record from that. However, since a user can put anything in a field, we would, at various places, be checking that the data was valid. The problem was that this validation code was dispersed (and repeated) all over the place throughout our dispatching logic.

The solution was to find some way of specifying a record's field metadata separately and then, the code could always just iterate over the fields and check whether the data was valid. If the data was invalid, it would simply redisplay the page with the error message. If we wanted a different constraint to apply to a given record, we would simply change the field constraint in the metadata and all of the right things would simply continue to happen.

The other thing that this scheme introduced was that, on the basis of the metadata, we could have code that automatically (we used the term automagically a lot when discussing this) generated the appropriate wrapper objects that Freemarker needed to expose the data on the page.

I think that this core functionality was running pretty robustly by November of 1999. Now, the Niggle framework was born!

Niggle still went through a lot of growth from there. Initially, I developed a custom way of specifying the record metadata in java code. This was rather awkward in retrospect. It certainly was not particularly easy for newbies to learn and feel comfortable with. It was Lawrence Austen, CTO of Oilspace who suggested that it would be better to specify record metadata in an external XML file. The switch to an external XML schema was made by June of 2000. At that point, there was in existence something that very much resembled the current version.

Current Status

Niggle is definitely in a usable state. In fact, that is probably quite the understatement. The Niggle code has been highly robust for a good while now. Reminders On-line have been building all of their servlet-based software on top of Niggle for well over a year. ROL's java servlet-based reminder service, Medimom, runs on top of Niggle. Still, there are various issues that caused me to hold off on classifying previous versions as a 1.0 final release.

The most important reason was a lack of documentation and examples. This has been largely remedied. Over the past few months, the examples have become more polished, the javadoc has been made more complete, and there are a couple of HOWTO documents on different aspects of Niggle.

Nonetheless, documentation is still the greatest priority. In my opinion, the best documentation consists of examples that are well-constructed and annotated. And that will be my emphasis over the next while on that front.

Besides the examples in the ./examples hierarchy, there is also a separate web-based forum application that I recently wrote to show off Niggle. This is a separate download. Unlike the examples in the ./examples directory of the main Niggle distro, this is not really a toy app. Also, it is not well documented now, but if you have gone through the examples that came with Niggle, and you want something more complex to sink your teeth into, feel free to download that code and hack it.

License and Copyright Issues

I wrote most of the Niggle code while working for Reminders On-Line in 1999 and 2000. From fairly early on, it was agreed that Niggle -- which was really the generally useful code not specific to ROL's application domain -- would be made available under a liberal open source license.

Reminders On-Line owns the original copyright. I have not been working for ROL since March of 2001 and hence, I own the copyright to subsequent modifications. In any case, both the original code and the subsequent modifications are licensed under the same BSD-style open source license.

Addendum: Reminders On-Line closed its doors in September of 2001 along with countless other "dot-com" enterprises. Nonetheless, I am confident that their use of the Niggle framework was not a cause of their business failure.

Future

Niggle is an ongoing project and you can fully expect further enhancements and refinements.

There are really 3 different fronts on which I see progress happening from here on:

• More documentation. (I'm a bit of a broken record on this.)
• More adaptor code so that Niggle wraps popular (and maybe not even so popular) external datasources out-of-the-box.