Having Fun with Testing

Well, I just finished writing my first junit test case. It took quite a bit of effort, a bit more than I'd expected to be honest. However, I really do think it's worth the effort in the end. There is no way that I would perform all the test cases by hand otherwise, particularly not on a regular basis. I tried to follow the philosophy of writing the unit test before writing any implementation of the code. I wrote the skeleton of ConnectionManager beforehand so I could use auto-completion while writing the test.

On another completely different note, I'm writing this blog entry using a toolbar applet for Gnome. It allowed me to enter the blog info. for the shoechicken blog and then bring up a mini blog entry window with a single click. So, if this entry comes out all garbled up, you know why.

I also realized today that java 1.5 added a thread pool implementation in java.util.concurrent. This could be very useful when implementing the Bufunkalo. I'm also wondering, way ahead of when I need to be, whether java.nio might be very useful for the rating part of the Shoechicken; I'll have to look further into that, just wanted to have it out there to remind us.

I spent some time looking into writing an ant build script for Shoechicken this week. I help maintain ant build files at work, but haven't ever written one from scratch. Luckily one for Shoechicken would be fairly rudimentary, but may be worth the time. Currently Michael and I are both using Eclipse to build the project, but a standardized build file from which to create the project may be a good idea. I'm thinking that the build file would only take a few hours to make too. Michael and I can talk about it when we meet tomorrow.

I've been checking the search engines and we're still only up on Yahoo, Altavista, and AllTheWeb. I make sure to click on our links on each search engine; not sure that it does anything but it would seem logical for the search engines to keep track of what people actually click on?

That's all for now, but should be plenty more to post after Michael and I meet tomorrow.

Wiki Wiki Wiki

James and I have been working on designing the classes related to handling database connections and performing database queries and updates. Since we originally designed the ConnectionManager class, the design has had to be retooled because not everything was thought through thoroughly enough. I think that part of these issues fall from the fact that there is no formal specification for the modules or classes. Being the Software Engineering major, and apparently insane, I feel that there is a need to start building at least semi-formal specifications of the modules and classes. This will lend some traceability to our design decisions and, if we do it properly, should privide us a better understanding of the system and reduce the number of design mistakes we make. So, to that end, I have created a Shoechicken Wiki that we may post specifications on.

It is my suggestion that Wiki pages be created for each module/class that we design, before it is actually "designed." These pages should outline the purpose of the class, how it will interact with other classes, and most importantly its behaviors and error conditions. The Wiki will allow us to easily update the specification as the need arises, which in the beginning will likely be very common but should hopefully become infrequent over time. We should probably develop a "template" for the specification of a class or module, so that all entries will contain the same information and present it in a common format.

The Shoechicken Website received a few small updates. The Documentation section saw the addition of a design page for the database classes. It was the creation of this page that spawned the idea of the Wiki, which could hold formal specifications of these designs. The site now uses a feature of Apache called Server Side Includes (SSI) to include the header on each page. This will make page maintenance much easier. Other, smaller, changes have been made across the site.

The class which spawned this mad tirade of mine is the ConnectionManager class. Originally, James and I had designed the class with a public constructor, so that any class could create one. Then, probably because it was getting late at night, decided (at my erroneous encouragement) that the class should be a Singleton which will load the settings for a ConnectionManager from a properties file. Later, James pointed out the likelyhood that eventually the item and user model databases may split, and that this approach would limit the ability to connect to multiple databases. James and I have started dicussing a Factory approach that would create one ConnectionManager per database, but would allow for multiple databases. This approach is still under discussion.

The Sky is Falling

Well, Shoechicken is encountering a few setbacks at the moment. It turns out that by leaving the rating agent as a black box, we did not appropriately design the rest of the system to support the Rating Agent. The key issue is that items (news articles) should not be delivered individually, but instead in groups, to the Rating Agent. The benefits of this (according to James) are less computation by the Rating Agent by not having to rebuild a large matrix for each article and more accurate ratings by comparing related articles to each other at the time of rating.

The architecture of the system is in a state of flux. Over the weekend James and I went back to the drawing board (quite literally a whiteboard) to resolve some of the above issues. Much of the functionality has been moved to the UI Agent, which is responsible for managing data from the Feed Subscription Service and the Shoechickens. It has been decided that there will be a 1-to-1 ratio between users and Shoechickens, meaning that for each user in the system there will be a Shoechicken (read Rating Agent + Feedback Interpreter) assigned to that user. The new design still supports the ideas of services mentioned by James, and that of a multi-user environment. The approach will add some additional modules and work, but should benefit the project in the long run.

The UI Agent is now responsibly for managing requests for updates, and ensuring that Shochickens are notified of these updates. The UI Agent will keep track of all active requests from all users, and when requests for article downloads are fulfilled the UI Agent will post events to the appropriate Shoechickens for the users issuing the requests.

A module accidentally left out in the previous design is now called the Bufunkalo, in honor of Hooper P. Bufunkalo (supposed discoverer of the Shoechicken). This module is responsible from capturing feeds and breaking them up into individual items before sending them to the Shredder. The module also weeds out articles that the Shredder has already seen, so that they are not processed again. Before sending articles to the Shredder, it downloads the content of the article if it is available so that the Shredder may break up the content into keywords. Another key aspect of the Bufunkalo is that it will contain a thread pooling system, which will allow multiple articles to be processed simultaneously. This will also prevent any other services which have called the Bufunkalo from having the wait for the Bufunkalo to finish processing (such as the Feed Subscription Service).

Additionally, we determined the need for a UserToFeed table in the database. This table maps a username to a list of feeds that user is subscribed to. This will allow the UI Agent to recreate subscriptions when a user reconnects. This will also provide the Rating Agent with the ability to determine which feeds it should be checking for new articles in the database. This table has been added to the SQL script used to create the database and the database itself in CVS.

The package names have been changed for most of the project. They were getting ridiculously long. For instance, the longest package name was com.shoechicken.shoechicken.bufunkalo.shredder.keywordcounter.event. Now, the same package falls under com.shoechicken.sc.bfnklo.shred.kwcntr.event. The new, more compact, package names should save some typing.