12/28/2005

Java Closure Chaining

I am playing with closures in java. Not as simple as the Ruby counter parts but a benefit to our programming arsenal I think.

I have come to realize two new things from this experience.

Testability

Compare


Iterator iter = list.iterator();
while (iter.hasNext())
{
A a = (A)iter.next();
do something
}

With

ListTools.each(list, new ClosureObject()
{
public void call(Object a_object)
{
A a = (A)a_object;
do something
}
});

How many pre-bugs do you see in each version. Both have 4 lines of code but the first requires 3 dot references which adds opportunities for those methods to do something unexpected.

Because of the the "final" requirement in java I found this construct to be a rarity. Instead we need something like this.

ListItemMunger munger = new ListItemMunger();
ListTools.each(list, munger);
System.out.println(munger.something());

class ListItemMunger implements ClosureObject()
{
public void call(Object a_object)
{
A a = (A)a_object;
do something
}
});

The most significant benefit with using Closures in java is that the loop-core is removed and is more testable than ever before.

public void testLoopCore()
{
ListItemMunger munger = new ListItemMunger();
munger.call(new SomeObject());
munger.call(new SomeObject());
munger.call(new SomeObject());
assertTrue(munger.something());
}

Now that I have classes that scan directories recursively, scan the files within them and then the line of text within each file this testability feature has been invaluable.

Closure Chaining

So I can scan directories, files and lines within files. How do I do that with these closure things.

DirectoryRecurseLookup.each("my path", new ClosureFile()
{
public void call(File a_directory)
{
FileLookup.each(a_directory, new ClosureFile()
{
public void call(File a_file)
{
StreamLineLookup.each(a_file, new ClosureIntString()
{
public void call(int a_lineNumber, String a_line)
{
System.out.println(a_lineNumber + " : " + a_line);
}
}
}
}
}
});


Well that's a lot of nesting but relatively simple syntacticly, no pre-bugs waiting to pop out at me. My first version took each of the closures into a separate class to simplify the code and make it more testable and it all worked fine.

Then I realized what I was doing wrong. I am just chaining closures so why can't the closures encapsulate that.

StreamLineLookup lineScanner = new StreamLineLookup();
FileLookup fileScanner = new FileLookup(lineScanner);
DirectoryRecurseLookup dirScanner = new DirectoryRecurseLookup(fileScanner);

DirectoryRecurseLookup.each("my path", dirScanner);

So as an example:

class DirecotryRecurseLookup implements ClosureFile
{
public static each(String a_path, ClosureFile a_closure)
{
// find files and pass them to the closure
}

private ClosureFile m_closure;
public DirectoryRecurseLookup(ClosureFile a_closure)
{
m_closure = a_closure;
}

public void call(File a_directory)
{
each(a_directory, m_closure);
}
}

10/27/2005

XP Cincinnati - Frappr

I am probably late to the party but this Frappr idea rocks. I created a frappr map for XP Cincinnati to see if we can get an idea of how spreadout we are.

Frappr XP Cincinnati

9/07/2005

Pre-bug - Variable Notation

A prebug, is a piece of code that is about to become a bug.

This is much like Martin Fowler's code smells (http://www.martinfowler.com/books.html#refactoring) but his writing comes from the perspective of how to find and recognize bad code and refactor it to improve it. The pre-bug concept is more like patterns for defensive coding strategies.

I coined the phrase prebug to describe code that you are typing in that will very soon be re-classified as a bug. Hopefully the test turns your prebug into a bug by failing but what if you can work out a way to type in your code that will prevent you from entering pre-bugs in the first place.

For example, one of the old C bugs that was so common was the single-equal-if problem.


if (a = 1) { printf("I think its one but who really knows"); }

For those unfamiliar with C, this statement would assign 1 to a and return true, always.

As a developer is typing in this code they may not be considering the possible bugs they are generating. So what if we retrained ourselves to always type this.

if (1 = a) { printf(("I know its one"); }

In this case the compiler will thrown an error saying that you can not assign a to the constant 1. What we have done here is slightly change the way we type in code and avoided a nasty bug from passing on down the development stream, potentially to production.

Here is another prebug idea that I want you to suggest solutions for. How would you enter this code to reduce the likelihood of the bug getting passed your ever vigilant eyes?

class Counter extends Thread
{
private static int counter = 0;
public void run()
{
while (counter < 10)
{
counter++;
doSomething();
}
}
}

The bug is that the counter is static and is able to run inside a thread, clealey a problem in this case. The same problem can occur with any static variable even its not so obviously in a class intended for multi-threaded operation.

The best solution I have found for spotting this kind of bug as I am entering it is to classify the variable names by scope and lifetime. We have all seem these notations in coding standards docs for years but I think the application of these ideas for the specific purpose of preventing bugs is not so commonly realized.

The java prefixes I use for these problems are:

statics: s_
instance: m_
arguments: a_

With the statics, instance and argument variables clearly identify to the reading eye what the scope of the variables are. The previous code should be entered like this.

class Counter extends Thread
{
private static int s_counter = 0;
public void run()
{
while (s_counter < 10)
{
s_counter++;
doSomething();
}
}
}

Eclipse offers auto-prefixing so all of its code generation features work once you have configured these notations.

8/23/2005

Emailing to a Blog

This is an experiment to see if this is a worth while feature. I am a little concerned that I will want to post too much crap. The editing process is important and slowing my posting down to 1 or so a month keeps things sane.
 
Anyway, one can but try the new processes and see what happens.
 

3/29/2005

Decoupled UI

My original question:

I want to be able to do this but it will not compile because the EventHandler does not have a no arg constructor.


IDictionary collection = new Hashtable();
collection.Add("myevent", new EventHandler());

A single event has the syntactic oddities

event EventHandler fred;
fred += new EventHandler(this.SomeMethod);
fred("x", eventArgs);

So how do I add fred to a collection so that I can do this
collection["myevent"] += new EventHandler(this.SomeMethod);


The answer:

I found the answer so here it is. Thanks to Mike Wood for his expertise.

The first and simple answer is that you can't do what I was asking. Events can't be constructed in the way that we would expect so can not be stored in a hashtable.

Background

The new information that I learned was about multicast delegates. These are the normal delegates that are created for us in our WinForms apps but they actually have the ability to store more than one callback.

It turns out that an event is completely unnecessary when declaring a callback. MSDN says that its just a safer syntactic representation of a multicast delegate (An Event to Remember).
delegate void MyCallback();
MyCallback fred;
fred += new MyCallback(this.SomeMethod);
This will fail to compile because fred is not assigned and you should not += to it.
delegate void MyCallback();
MyCallback fred;

fred = new MyCallback(this.SomeMethod);
fred += new MyCallback(this.SomeMethod);
This will work because fred is first assigned with a delegate and then one more is added. Note that we can not re-assign a callback to the delegate to clear all existing delegates. This is the danger of the delegate syntax. This is now a multi-cast delegate since fred contains multiple callbacks that can all be invoked in turn by:
fred();
So the alternate syntax allows for the += in any situation. By placing the keyword "event" in front of your delegate declaration you allow the += to always work. This doesn't prevent you from doing an assignment so its not much protection but it does mean that whenever you see an assignment to an event you should be suspicious.
delegate void MyCallback();
event MyCallback fred;
fred += new MyCallback(this.SomeMethod);

Problem

What I wanted was a single class that could manage a set of delegates, categorized by data type, that could be subscribed to and published from different areas of your code to ensure that there was no compile time coupling between them.

Example of the problem:

class A
{
private B b;

public A()
{
b = new B();
}

public OnButtonPressed()
{
b.MakeBread();
}
}

class B
{
public MakeBread()
{
Console.Write("rise rise rise");
}
}

Notice how A has a reference to B. This means that when I want C to do something when the ButtonPressed event occurs I have to edit A to add a new reference. Also, If I want to add another layer between A and B then I have to change A to make the new reference and move the B reference into the new location. Lots of code changing is bad from a maintenance and testing perspective.

Design principle. Open Close Principle

"Software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification."

C2 Wiki - OpenClosedPrinciple

In this example I had to change lots of existing code to rearrange my structure. This is very common in UI applications as you move the UI around the elements of the UI have to be moved between source files and the call backs have to be moved as well.

Solution

The solution to the original problem is to put a delegate in the hashtable and then add additional delegates to the same element. No events were used during the making of this code.

I wrapped this in a class called PubSub with two static methods Subscribe and Publish (Download Solution)

Example:

class A
{
public A()
{
}

public OnButtonPressed()
{
PubSub.Publish(new UiEventButtonPressed());
}
}

class B
{
public B()
{
PubSub.Subscribe(typeof(UiEventButtonPressed), new EventHandler(MakeBread));
}

public MakeBread(object sender, EventArgs args)
{
Console.Write("rise rise rise");
}
}

Notice now that class A doesn't know who is reacting to the ButtonPressed event, all its doing it notifying the world that something happened. Class B on the other hand is a simple standalone class that knows it has to do something when the button is pressed so subscribes for that event.

I could now write a class C that subscribed for the same event without changing any existing code at all. This is a massive maintenance and test advantage. Imagine being able to extend an existing system by dropping a new dll into the deployment. Sweetness.

This type of abstraction also offers extension advantages. Imagine the day when your UI gets so slow that you have to start backgrounding some of the actions. You can add extend your PubSub class to submit all the callbacks on background threads.

What about the idea of distributing your application. Say you want a background service to manage the synchronization of data from your UI to some external storage. You could extend PubSub to post events across process or machine boundaries and plug into your existing application without changing it.

Obviously, I am simplifying this a little but the abstractions are the key to making your system flexible enough to be tolerant of these kinds of changes.

2/18/2005

CinDnug 2-20-05

Doug Rohrer, of Avanade Inc employ, presented an overview of the Microsoft Enterprise Library (entlib).

Apparently Avanade are the original writers of these libraries as the copyright at the top of the source indicates. Oh did I mention that we get the source? The license allows you to modify for your own use but you can not redistribute your alternations and you must leave any existing copyright notice at the top if you do not change the code.


// Copyright © Microsoft Corporation. All rights reserved.
// Adapted from ACA.NET with permission from Avanade Inc.
// ACA.NET copyright © Avanade Inc. All rights reserved.

This is a library of "application blocks" that are intended to help developers with the basic plumbing of an application. A block is just a subset of a library which is akin to an assembly. In java terms its just a jar. I don't know why "block" replaced "library".

Doug writes about the entlib in his blog.

Doug Rohrer's Blog

Download it here after some registration steps:

Microsoft Enterprise Library

I am afraid that I arrived 30 minutes late so missed a couple of the initial block descriptions but the website doc is good. These are the blocks in the entlib

o Configuration Application Block
o Data Access Application Block
o Caching Application Block
o Exception Handling Application Block
o Logging & Instrumentation Application Block
o Cryptography Application Block
o Security Application Block

One the most interesting features of this library is that it comes with source. When you download it you get a selfextracting exe that unloads the source to your disk and, with compile checkbox checked, proceeds to compile the library in place. Having the source available makes this a far more valuable resource since we can not only use it but also learn from the code.

All the libraries use the Configuration block and most use the Logging block so there is a good synergy between them all. They all have good factory abstractions so its simple to replace the various providers with your own implementations. Each folder in the source contains a Test directory where NUnit tests sit.

Here is a simple sample of each.

Logging:

    Logger.Write("My message", "someMessageCategory");
Each write can optionally, specify a category that can be configured in the configuration file and drives where the message is written to.

There is no method level distinction between error, warning, info and debug messages. There is also no class level message filtering. You could use the category names for either but not both.

This being a class level API, instead of instance level, its flexibility is limited. A simple improvement would be to allow us to create our own reference to the logger and then we can choose to make it an instance or static property.

So, compared to Log4Net (http://logging.apache.org/log4net/), this solution is not adequate.

Configuration:

    settings = (MySettingsType)ConfigurationContext.GetConfiguration("section name");
settings.MyProperty
The advantage of this package over the built-in App.config is that you can select the storage type. This seems like a good solution for .Net app config loading.

Data Access:

     myDataSet = DatabaseFactory.CreateDatabase("Sales").
ExecuteDataSet("GetOrdersByCustomer", myCustomerId );
I missed the comments on this one but the code snippet looks simple enough. I asked Doug after the presentation about support for Transactions but he didn't have any concrete information.

Longer term I am more interested in libraries like NHibernate (http://nhibernate.sourceforge.net/) that support a full Object/Relational mapping solution. Then you can look into Spring for .Net (http://sourceforge.net/projects/springnet/) and abstract your transaction management out of your code completely.

Caching:

    factory = new CacheManagerFactory(Context);
cacheMgr = factory.GetCacheManager("SmallInMemoryPersistence");

cacheMgr.Add("key", "cache");
cacheMgr.Remove("key");

cacheMgr.Dispose();
The location of a cache can be configured as in-memory or in-database so it can be used within a clustered environment. It supports various cache ageing algorithms that allow you to control how stale you data gets.

I am unsure about this library. With a IDictionary type interface you are limited in the types of data that you can cache. Not that you can't store anything but that you can't get it back as easily. Say I wanted to store a row from a dataset, what would I use for a key? It would have to be the unique key to the row but if that is a compound key I would have to come up with a string representation of that information.

I am sure that there are some good uses this library can be put to but with caching being such a complex layer I am not sure that it is flexible enough for the more complex applications.

Exception Handling:

    catch (Exception ex)
{
bool shouldRethrow = ExceptionPolicy.HandleException(ex, "policyname");
if (shouldRethrow)
{
throw ex;
}
}
The goal here is to centralize the processing from all our catch blocks. The HandleException method reads configuration where a set of exception handlers are configured. You can relate a list of handlers to a policyname that can log, write to the event log or add you own custom handler.

The biggest downside to this approach is that you loose all your context logging. Most exception handlers are going to format a message that includes information from the local context that will help debug the problem. The only way I can see to add this feature and still use this exception handler is to wrap the thrown exception in your own exception and pass that in.
        MyException mx = new MyException(ex, "Context specific message");   
bool shouldRethrow = ExceptionPolicy.HandleException(mx, "policyname");
I question whether the function that I want to perform in my exception handlers should be abstracted to the configuration file level. Certainly, common code should be extracted into some application specific class but to standardize that and push it all the way to a config file seems unnecessary and not very useful.

Crypto:

    Cryptographer.EncryptSymmetric("dpapiSymmetric1", "plain text");
Cryptographer.DecryptSymmetric("dpapiSymmetric1", encrypted,);

Cryptographer.CreateHash(hashInstance, plainTextString);
Doug also demoed encrypting with sha1 and it was as simple. This is going to be really easy and will, hopefully, ensure that every programmer gets comfortable with this subject matter and uses it. There is way too much data out there that is not secure because the average programmer doesn't know how to use the cryptography api.

Security:

    factory = new AuthenticationProviderFactory();
IAuthenticationProvider provider = factory.GetAuthenticationProvider();

if (provider.Authenticate(new NamePasswordCredential("Fred", "password"), out identity))
{
// Woohoo
}

factory = new AuthorizationProviderFactory();
provider = factory.GetAuthorizationProvider();

if (provider.Authorize(principle, context))
{
// Woohoo
}
A simple authorization and authentication abstraction that will simplify your login pages and access checks in your code. You can plug in your own providers and perform application specific checks as needed.

Conclusion:

Some of these seem to be very useful libraries that I expect to start using in my current project.

A possible improvement that could be made to the distribution would be to formalize the mock classes that are used. The "Test" folders are full of mock classes that help prove that various pieces of functionality work. When I am testing my code I will have to concoct some mock configuration to drive the various paths though my code. If the library supplied some mock implementations it would save me from writing my own.

Thanks Doug for a great presentation.

2/11/2005

Cold basement

My office is in my basement. Yes, a little corner of a cold dark hole in the ground. Winter makes it intensley cold with one electric heater making my body hot on one side and leaving the other blue.

Still, it beats trecking through the snow to a small dark cube.

Cincinnati doesn't get too cold in the winter, not compared to the more northern areas. However, this is my home and that gives me the right to whine about it a little.

The view from our back window is nice after a fresh snow. The last snow was windless so produced some excellent depth on the individual branches.



2/06/2005

Referencing EXE assemblies in VS.net

The problem, as discussed in the CinDnug use group, was that a VS.net project that contains tests can not reference a project that generates an exe. The only design solution is to make the exe code a one liner that calls a main dll but this has quite an impact on other design decisions.

When you try to add a reference from a dll project to an exe project in VS.net you get a warnings stating that the reference target has to be a dll. You can solve this problem with some judicious editing of your .csproj file.

Editing your .csproj file will show references declared like this


<Reference
Name = "ProtoTabBack"
Project = "{6292AE62-0514-40F7-99C0-C2C181BF9A15}"
Package = "{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}"
/>
<Reference
Name = "TabulatorFront"
AssemblyName = "TabulatorFront"
HintPath = "..\TabulatorFront\bin\debug\TabulatorFront.exe"
/>

Note there are two types of declarations here. The first type uses the package project guid that you can find in your .sln file in the following format. The first guid is the Package and the second the Project.

Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "TabulatorFront",
"TabulatorFront\TabulatorFront.csproj",
"{DD6F402B-1823-489F-AC6C-B9B8FE18391D}"
ProjectSection(ProjectDependencies) = postProject
EndProjectSection
EndProject

Using this "project reference" in your .csproj file results in VS.net flagging the reference with a little yellow triangle and saying that it has to be a .dll file.

However, using the second Reference format and referring to the exe file directly works. I am now able to code to and link my tests with the exe project code, utilize existing test infrastructure to write tests against UI code and keep my test code out of my production deployments.

Thanks James for the suggestion.

2/03/2005

Robust code

Trying to get people to stop hard coding expected results in tests is not easy.


IList rowList = db.GetSomeRows()
Assert.Equals(4, rowList.Count);

EEEEEEkkkkkkk

Code is like poetry
blowing in the wind,
beaten from left and right
crashed and regularly skinned,
choose the best of parchment
to ward against the eternal din.

1/19/2005

Recap CinNug meeting 1/18/2005

CinNug - http://www.cinnug.com/.

This was a Microsoft product pitch for Team System by Alex Lowe who runs with the title Developer Evangelist. He made a point of apologizing for his title at the beginning.

To be released in the second half of the year sometime, this solution includes a Team Server and three possible client options, Team Architect, Team Developer and Team Test. Each offers extensions to Visual Studio 2005 which is due for release in the middle of the year.

The target market is the enterprise development team so cost will be high. Alex kept comparing it with the Rational target market. He mentioned that MS would beat their price by 20% but that we should not do a feature for feature comparison since they are very different products. MSDN Universal owners will have a license for one of the three client options but would have to by Team Server anyway. It is not clear which of the client features require Team Server to operate.

The idea behind this solution is to establish a standard framework from which teams can manage their projects. Apparently MS is starting to use the product internally but so little of it is working at the moment there is little internal feedback. It is their intention to completely adopt this development framework within the next 5 years.

Disclaimer 1: I have never used Whidbey or Team System so this write-up is probably filled with assumptions. I am copying Alex so he can correct us if he feels like it.

Disclaimer 2: I am also a long time java developer with perhaps 4 months of full time .Net work under my belt so please forgive any ignorance based bias. With years of VC++ and ASP development I have some background but java swept me off my feet many years ago.

So into the new features of Team System.

Team Architect contains a design diagramming tool replacing the use of Visio for class diagrams. Alex talked about the MS's preference for Domain Specific Languages (DSL) as opposed to the standards based UML syntax. He explained that customers are more likely to understand a language that uses the terms and shapes that they are familiar with.

The demo included a class diagram which showed up in non-UML format. Though we only saw two classes, one that inherited from the other, the inheritance arrow was UML like but the class boxes had rounded corners and contained non-UML sections and verbiage. I am all in favor of DSL's when they best represent the subject you are communicating but developers understand classes and I would wager most understand or at least have seen UML so why go non-standard for class diagrams?

The class diagramming tool included two way dynamic code synchronization so we you add to your diagram the code is being written and as you edit your code the diagram is being updated.

To my mind the biggest problem with class diagrams of larger systems is complexity so the selling points of a class diagramming tool should be the filters that can be applied to simplify the representation. Unfortunately we didn't see any of that.

Enhancements to VSS improve performance, scalability and offer new features like "shelving" which allows you to check your code into a new branch so that you don't pollute the main branch with your failing tests until you are ready. Since VSS already supported branching I assume all they have done is to make it easier to do and given it a fancy name.

Most significant improvement to VSS is that you can integrate with it remotely over HTTP so it can finally be used outside a LAN. While I love CVS, it would be better if we could use a source control system that integrated into VS.

Alex presented the unit test features that allow you to select the classes and methods to create tests for. Team Developer produced NUnit like code to exercise them. Not just test stubs but completed methods with some best guess code in them. This months .Net Developers Journal discusses the same feature with the similar screen shots in Whidbey which will be Visual Studio 2005 so I presume that there is some overlap in these products.

Notice I said "NUnit like". Microsoft has changed the "attributes" that you use to tag the classes and methods that are exercised as tests. NUnit uses "TestFixture" and "Test" but the new MS unit test integration uses "TestClass" and "TestMethod". I view this as a change in standard names that are out in the industry and is completely untenable.

Imagine a 100 developer team with 1000's of NUnit tests thinking about migrating to VS.Net 2005. I would lay money of the fact that no admin is going upgrade 100 developer machines in a weekend, the risk would be enormous. So as desktops are upgraded they become unable to run the product test suite. Alex mentioned that a tool would be provided to convert all your existing test cases but you are still leaving a portion of your developer community out of the testing process until they migration is complete.

Code attributes were invented as a way to abstract the code location from class and method names. This can be a fine idea if one wants that abstraction. MS has ignored the industry standard names and is forcing developer shops to build repositories of tests that are dependent on Microsoft testing tools. Choosing to use these new unit test tools is as bad an idea as choosing .Net as a cross platform development strategy. I see myself continuing to use TestDriven as my VS integrated NUnit framework.

The demonstration of test automation went pretty smoothly but I missed asking one question. When Alex entered into the test creation steps a new VS like window opened up with a new view layout targeted on test management, I believe Alex even called it a "Test Manager". From this new "Test Manager" Alex was able to run the tests and we saw the sample test being scheduled, we waited a little and then it was in run state and finally the red/green bar appeared. Alex then exited this new window to get back to the code.

This UI will impact the way test first developers work because it introduces a time and complexity barrier between your fingers in the code and the red/green bar of feedback. This will reduce the number of times tests get run which is contrary to the goals of test first development.

One of the major reasons that JUnit, NUnit and all the other XUnit frameworks work so well is that they are ultra simple to use. With IDE integration the single click to run a test is a really important feature. I think MS missed the boat on this one.

---------------------------------
By the way Alex, you mentioned that test first required writing tests before the code. You should rephrase that as "writing the tests while writing the code". The process requires that a single, simple test is written before code is written and when the test passes, another is written. Think really small cycles like 60 seconds or so.
---------------------------------

Team System has an integrated build and deploy process. Not much was said about it but a short demo showed the creation of a diagram that you can add IIS server definitions to, then you can drag your Web Projects into your diagram to deploy them.

Since best practice suggests isolating your build and deploy tools from your IDE this may not be a good solution. However, I didn't ask the question, and Alex didn't say whether the build/deploy tools could or could not be used from outside the IDE or whether an IDE deployment/license would be necessary to run them. It strikes me as insane to require a VS IDE on a build server. We need nant integration please. Then we can plug-in it into our CruiseControl.Net environment without a care in the world.

When you create a project you have the option to select a "profile" which is allows you to chose MSF Agile methodology (the only one available at the moment) which sets up your project with the "standard set of documentation required". I think they missed the point of agile if they think it requires any documentation. Anyway, I guess we can say that they are thinking about the concept anyway. Alex also mentioned that we will be able to set out own standard profiles to help with company standardization.

Alex admitted that the product was pretty shaky but promised performance improvements before its release date. Advertised release dates are in question and Team System will probably be released after the VS 2005 which it is targeted to integrate with.

Conclusion
----------------
Team System allows us to integrate the management of design, source code control, testing and deployment as well as bug tracking and reporting. This is a really good thing for development teams.

The architecture of Team System appears to be completely MS centric which could work for pure MS shops. Alex even mentioned that Team Server only works with the packaged SQL server and they have already received feedback from Oracle shops about having to support another database.

I look forward to trying it out when it is available.

Thanks to Alex for a good presentation.

Additional References
--------------------------------

Microsoft product page

http://lab.msdn.microsoft.com/vs2005/teamsystem/

The .Net Developer Journal story on Team Developer:

http://sys-con.com/story/?storyid=47759&DE=1

If you have a subscription you can see the story on Team Test:

http://sys-con.com/magazine/?issueid=507&src=false