Niggle Design Goals

Been there, done that...

To my way of thinking, there are two basic ways in which an application framework can make you more productive. There is the obvious way: it provides pieces of prebuilt functionality that you can just use. And of course, this saves time, since you have to write less code yourself. However, that is true of class libraries in general. What makes a class library a framework is that it partitions the overall problem space for you; it provides a skeletal structure that has proven itself time and again in practice.

An application framework is most likely to be useful when it is written by people who have "been there, done that". When you use such a framework, you are leveraging the hard-won experience of others who went before you to avoid the pitfalls typical of a given application space. So, on the one hand, Niggle is designed to get you going writing web applications as quickly as possible. But probably more importantly, the applications you write will be more robust and maintainable, and this represents a huge time savings over the course of a project's lifetime. Maintainability is one of those things best defined by what it is not: code is not maintainable if relatively small changes in requirements lead to having to rewrite large pieces of code. This is a great peril particularly in web application development, where there is often a need to make minor changes or implement new functionality very quickly. Niggle is designed so that relatively small changes will not require large modifications to your code base. In fact, Niggle-based code tends to be so robust in this sense that, often, you can change your entire site's user interface or make significant changes to your database schema without hardly changing a single line of java code.

In the beginning was data....

Tools very typically embody an approach to a certain class of problem. For example, when you use Visual Basic or some similar GUI builder sort of tool, you tend to design the user interface of your program and then gradually flesh out the other details afterwards. So you could characterize those tools as being UI or presentation-centric.

Niggle's approach, on the other hand, could be characterized as data-centric. Once you become experienced with Niggle, when you start off a new web-based app, you will find yourself naturally sketching out your data structures in XML and then, on that basis, gradually filling in the other details such as presentation templates and the basic dispatching logic. In my view, this tends to lead to more robust, well engineered applications, as opposed to an approach that starts off at the user interface level.

Now, it cannot be denied that tools like GUI builders have great strengths. They certainly allow you to get a lot of stuff working without writing a lot of code. As far as I can see, this is because they give you a lot of objects that simply work out of the box. Where they fall down, I think, is that, though they allow you to make a lot of progress very quickly in the initial stages of a project, they do not really encourage a maintainable approach. (In fact, quite possibly the opposite!) I suspect that, in many cases, a lot of fast progress is made at first and then things gradually get bogged down.

One might say, then, that Niggle's goal is to give you that kind of benefit -- extremely rapid prototyping of the skeletal functionality of an app -- while also pushing you towards a much more flexible, maintainable approach.

It should "just work"

The mind-set behind Niggle is relentlessly pragmatic. It was meant to be used! To the greatest extent possible, Niggle is designed to help you to get started writing web-based applications as quickly as possible. Having downloaded and used a lot of open source software, I do have one very big axe to grind. Things should work out of the box! I have tried my best to to avoid situations where a beginner is presented with an intimidating array of settings and options that must be mastered just to get started. Of course, this is easier said than done. Earlier versions of Niggle did have a much steeper learning curve. However, in the last few months, as I have approached the current 1.0 release, I have been trying to make sure that there are generally useful, out-of-the-box default settings that allow neophyte users to be productive with the framework very quickly. Ideally, this means users who are not even necessarily that experienced with server-side Java.

It's all the same to me.... Data Source Independence

Niggle is based very much on the idea of data source independence, which means that your application code is shielded from any knowledge of how the system is retrieving/storing its data. This allows you to build up a prototype that uses flat-files, say, for data storage, and then later switch to an industrial-strength RDBMS like Oracle, without changing any of your application code. You simply change the external XML-configuration of your data sources. You could then switch from Oracle to another database vendor in a similarly easy fashion.

To achieve this, Niggle uses the Oreo data API that abstract away the notion of an external data source. There are currently 3 implementations (really sub-implementations, since they share some of the same code.)

The first is in org.oreodata.standalone. The DataSource implementations in this package simply store all of the records in memory and log/persist to a flat text file in a kind of key=value format. This is the implementation of greatest vintage. It is quite efficient for applications with moderate data storage requirements. This implementation has the great virtue that it allows Niggle to work out of the box without any external RDBMS being installed and configured. This release (1.0.2) introduces a better in-memory data source implementation in org.oreodata.xml.XMLFileDataSource. This keeps all of the data record in memory, but persists/logs to an XML file.

The DataSource implementation is in org.oreodata.jdbcimpl.JDBCBackedRecordSet represents a façade in front of an external RDBMS. It provides for a configurable level of in-memory caching of records.

In general, the Oreo Data API could be concretely implemented in various different ways. I would fully expect expert-level programmers to write custom data source implementation classes with different performance characteristics. (I would hope that they would contribute these classes back to the open-source code base, but there is no obligation.) For one thing, a very performance-conscious programmer could write a custom implementation that caches data in very smart ways for a given application domain. On the other hand, just about any arbitrary data source could be rolled up as an Oreo DataSource. For example, though I have not done it (yet) I have every reason to believe that the folders of an IMAP email server or the groups on an internet news server could be abstracted away as an Oreo MutableDataSource. At an earlier point, as proof of concept, at ROL, Paul Murphy wrote a wrapper around a JNDI data source that conformed to the Oreo API. Thus, the potential exists to tie together disparate sources of data and treat them the same way programmatically.

Niggle also provides a presentation API. It assumes that the end result of fulfilling an HTTP request is a org.revusky.niggle.templates.Page object. By default, the Page interface is implemented by the concrete FreemarkerPage class, which wraps the open source Freemarker template engine. In this version, there is also an implementation that uses WebMacro. In any case, there is no reason that one could not write adaptors so that Niggle could use other presentation solutions. I would be interested in this, generally. That said, there is probably less real motivation to do so than in the case of the data side, since the default Freemarker-based adaptor code actually works extremely well and I anticipate that most people will simply use that. The data side presents a lot more incentive for custom wrappers since people already have their various data in an enterprise living in all kinds of different "containers ".