The Java Blogs Meetup is scheduled at 6:30pm in the back of the Thirsty Bear Monday night. I plan on attending!

Read more...

I have been working on a handful of web applications for over 3 years now. Over the course of this timeframe, business rules have changed, technologies have changed, and bugs have manifested their way into the production environment from time to time. All of these reasons have, at one time or another, led to unclean data entering the persistence layer. Our data model for these web applications is highly normalized. Unfortunately, the proper constraints are not present in some cases to enforce referential integrity. Over time, this has led to data issues where something is halfway input or halfway deleted. Not to mention, bugs that have sometimes allowed for invalid inputs. All in all, our data is not as clean as it should be. Shame on us I know! After looking over a section of code, I see us checking for null values in many places, checking for the presence of certain data and the absence of other data. This defensive code is spackled from the web layer to the EJB layer and back. Much of this code was written as a work around for unclean data. If we would have mandated good data from day 1, I wonder how much smaller our code base would be now.

The difficulty solving this type of problem is that what was once a clean data model can be tainted when business rules change. An easy example of this is when a field that allows null values suddenly doesn't allow them anymore. You can't change the database to require this field since many rows of data do not currently have it populated. Sure you could update the table and insert a surrogate value, but isn't this the very thing that we are trying to avoid. The easy fix for this problem is to throw a null check into the application code but leave the database as is allowing null. This is step one to bad data and unmaintainable code. Unfortunately this problem never seems to get better, only worse.