Waterfall Symptoms

As I stand in a sit down status meeting to listen to another hour long discussion I ponder what is going on to see if I can offer some help. The obvious suggestions have already been ignored: stand up during the meetings; stop designing the system in meetings; stop with all the detail please, just a summary is all I need.

This meeting is a little special though. We have a list of components to be completed before we go live and another list of the outstanding bugs. 20 of one component and 197 of the other. We go live in 6 weeks.

An analyst, that doubles as QA tester, spent their weekend testing but entered bugs like "couldn't test" because of environment problems. The release process is still pretty new and untested.

Our leader asks for QA of a new module and the same dedicated analyst asks for help. There is none, of coarse, just a few extra of their hours in their day. They were allocated to their kids, but they are dedicated folk. Dedicated to their work I mean, can't be dedicated to work and kids on a waterfall project. Laughable.

They mention code freeze, except for bugs of coarse; you understand, can't code freeze code that doesn't work. Laughable.

6 weeks left and now we start raining in scope. Oh wonderful, just in time. Laughable.

QA has no automation so there will be no regression, just wild retesting. They do a great job, work really hard and find lots of bugs. I just wonder how many are missed for the next go around.

The project plan is stored in MS Access, a horrible solution, that few people use. We just code anyway, why should we know what the plan is, we weren't part of its creation, or its revisions. It was presented to us once, I asked some questions about it which had the usual optimistic answers.

Question: "There isn't enough time to complete coding and QA testing before the release date"

Optimism: "Oh, that's ok, they will run in parallel. Each component we finish will go into QA test."

It all sounds so logical. Of coarse, I am an Agile developer, parallel coding and testing are the norm but my Agile projects are setup to support that feat. Waterfall projects aren't, well this project isn't anyway, seems a shame to speak for all falling water.

Of coarse the reality has proved itself over the optimistic plan. It has a way of doing that. The QA regression testing is required and is being attempted. Best of luck to them. They work so hard. They deserve success.

I try to get the ant scripts to release to production. Not to big a job, shame I couldn't get them to work on it 6 months ago. Anyway, I am talking to the tech lead and we have to have this finished by tomorrow. There is a problem with my ant properties so he exclaims that he will manually complete the production deployment so we don't need ant.

"We don't release to production often anyway, so it doesn't matter"

I suggest that the less you deploy the more important the scripts, after all we got dev deployments reliable months ago, and we really want production deployments to work first time every time right?

"Yes, that's right, but we need it by tomorrow. We can try again in two weeks."

He really does have the best intentions. He has been put in a predicament. No production release practice for 10 months, now we have about a week to get it right.

Why does our industry do this to itself? They can't predict where the next problem will be before it arrives. Experience? No, they have lots. Understanding? No they are smart people. They have always done it this way. This is how software projects always work. This is just the life of a software developer. It is all we can expect. Why change.

If it weren't for change,
I couldn't tell that I am standing still.
If it weren't for standing still,
I couldn't tell that the world around me was leaving me behind.

I see what I see with clarity,
I understand what I understand with certainty,
With this clarity of certainty,
I stand and watch and re-affirm my motionless state.

If I stand on the certainty,
That my knowledge is true,
Then my only conclusion must be,
That all that I see is foolish change.

The gulf between knowledge and change, stands in a persons need,
A persons need depends on their situations urgency,
A situations urgency is created by those who don't change,
so the lack of change is our choice, because that has always been our situation.

New Knowledge vs Productivity

Update: 08/22/2007: add references to J curve

I can get task X done using knowledge I already have. I would take longer if I used a technique that I don't currently know.

However, if I learned a new technique I might complete task Y so fast that the slowdown on task X is insignificant.

This is the balance of the J curve. The J idea proposes that your productivity remains flat until you learn something new. At the beginning of your learning your productivity decreases until your skills are sufficient to surpass your original productivity.

The amount we produce is related to our current position on the J's learning curve and the number of J's in our past.

So, worst case, you could imagine living on the downward slant of the J forever. There is always something new to learn after all, why produce when you can learn. I guess pure research organizations work like this.

The other worst case, is never entering the J. Living your life with a static set of knowledge.

OK, so how often should we dive into the J?

Life needs to exist,
In a rising J,
With every day,
Finding another rise.

If the bad poetry doesn't make sense then consider learning 5 things at once and every day push one of your tracks past the base of the curve.

Growth ends with the belief that you will be less productive by learning.

Do something new today, even if it hurts. Oh, and it will hurt.

References:

Meilir Page-Jones: The Productivity Curve

Gerry Weinberg: quoted as part of his pinball analog

Amazing Image Integration

This movie has been around since May but I just found it and just in
case others havn't seen it, here it is.

http://www.ted.com/index.php/talks/view/id/129

The future of connected imagery is here.

Twitter

Yes, I am now sending tweets. Not sure why but it appears to be addicting.

Twitter is another social network but is far more focused in its goals. It just asks "What are you doing" and with frequent updates you end up with a general idea of what your friends are doing.

Perhaps this is just too much information and no one really cares but I think that is what it is all about. Why does and now much will they be able to pay?

With a simple beginning there are now addons to the twitter service like a search of all the things that people are doing and, my personal favorite mashup, TwitterVision which merges the posts with the geo locations of the poster and swoops you around the world from language to language and irrelevant activity to irrelevant activity.

But don't count this thing out yet. Check out my personal twitter vision. There are many times that we need to keep track of things. Oh, did we think this was limited to people? When are you going to get that new watch that will tweet so you never loose it. Should FedEx make their packages tweet? It's just a 140 character piece of text but twitter vision encodes ":L[position]" into it as a really simple way to track things.

FexEx, of course, built their own package tracking system back in the days when web services were new and SOA was a Canadian SO. If the task was started again today would they have chosen to leverage existing web services.

Each web service we build we examine granularity as an architectural problem. Perhaps the success of twitter indicates we should stop examining and just think smaller.

Who knows perhaps your next watch will make a request for the time somewhere.

Visual Studio 2005 and ReSharper

I have used ReSharper with VS 03 in the past but decided to try the new VS 05 features before installing the new ReSharper version. After fighting with the stupid stupid VS features my course is clear.

ReSharper is to C# what Eclipse is to Java. A rebirth.

Java, C#, Groovy, Ruby, Python

Is the question "which" or "which ones"?

I am a member of a local user group that exclusively pitches Ruby as the savior of all things linguistic. I had made a strategic decision to learn Groovy instead because it integrates better into my primary language, Java.

This has worked pretty well for me up to now with a couple of jobs getting groovyized nicely.

Well, Chris Nelson, mentioned something to me a few user groups ago that just sunk in today. An extension to the logic of why I chose Groovy.

If we assume the future is a combination of the best of today. Perhaps a tenuous bet but I am an optimist. Then we have to assume that a VM is the correct runtime environment for future systems. This knocks out c-Ruby from the running.

If we think that dynamically typed languages win over statically typed languages then we have to choose Python, Ruby or Groovy as the most blogged about at the moment. I am completely sold on dynamic languages and have been for years.

I asked Chris why he choose JRuby/Rails over Groovy/Grails since he is a Java developer from long back (Trails). He answered, "It's a different language".

My first impression was that Ruby was a different language as well so what's your point. Then today, while reading Charles Nutter's response to Ola's IronRuby post it occurred to me that the language Ruby is key for the simple reason that it is likely to be the first multi-vm language and already represents the best of what a dynamic language can offer.

Based on this there is one choice, quit groovy, learn Ruby in what ever form and the runtime environment will follow.

Did anyone notice how I dropped python so fast? Dynamically typed and runs on JVM (Jython) and CLR (IronPython) today but I hate tabs :-)

Unfortunately, the question is "which ones". Things move to fast to play just one pipe.

Standup Meetings, for Accountability not Communication

We Agile folks like out stand-up meetings but when asked why we do it we usually start talking about communication being a core principle.

A recent project involved a customer that was tentatively interested in Agile so started doing stand-ups. When it came to a month or so before my departure I stopped leading to help encourage the team to take ownership of the new processes that they had been learning.

The first thing that went was the stand-up meetings and the result of this lead me to the decision that we don't stand-up for communication. The best reason for stand-up meetings is to force every programmer to prove that they are working to the plan instead of just working.

This forces plan driven work which is prioritized and focused. My customer quickly descended back into panic driven work with programmers being approached by QA reps telling them of the latest problem and urging them into action. The plan was forgotten.

The chatter and sharing is nice and there are certainly other benefits to stand-ups but from a management perspective we do this to ensure that we are driving accountability for everything that we do each day.

The shorter iterations of Agile drive accountability to the customer. The daily discussion on what we are focusing on drives programmers accountability to themselves and their team.

Star Wars ! What really happened after the Deathstar blew up

This is really funny.

[snipped old dead version]

The moral of this story that before long everything will be a copy. Thanks for the comment Mr Anonymous.

The yahoo version is still active.

TIP: Dos Sleep

I needed a batch file to sleep between starting up a couple of processes but dos doesn't support sleep. Google found me this result which gave me the idea but their solution is wrong.

The actual answer is this:

ping 1.1.1.1 -n 1 -w 10000 > nul

In this case we are pinging a non-existent IP address, once, and timing out in 10000 milliseconds.

Of coarse you could pop this into a sleep.bat to centralize it

cd c:\windows\system32
copy con sleep.bat
@echo off
ping 1.1.1.1 -n 1 -w %1% > nul
ctrl-Z

Now you can just type "sleep 5000" to sleep for 5 seconds.

The actual resolution of this timer is one second or more. Any number less than one second is going to end up with a 1 second'ish delay.

The cost of this is one network ping with a standard 32 byte payload. The total message is 74 bytes on a Windows XP machine. This is low cost for a couple of sleeps in your program. If you want to do this all the time then you may want a less hackey solution.

The better solution might be to use the Windows Resource Kit which includes a sleep.exe command.

Speed of Time

If you could set the speed of time,
how long would it take you to change it.

MonoRails - .Net is cool after all

After two days of fighting with msbuild I thought that I was going to loose hope but along comes MonoRails. We all know that Rails rocks, we have loved tinkering with Grails and now we find that wherever we go someone is building another version of this idea.

For the uninitiated, go and learn Rails first, it's the granddaddy of them all. These are new ways to build web applications, they obey the idea that conventions can be more productive and just as flexible as configuration files so you type in the name of something and it magically gets extra powers just because you spelled it correctly. Don't be afraid of the magic, it is your friend.

So, MonoRails is more of the same but here are a couple of differences that I noticed on my first night at playing with it:

1. It is normal statically typed C#. All these frameworks use dynamically typed languages to achieve much of the magic. Injecting methods, imbuing properties, you name it they do it. The static language approach is certainly going to offer some additional challenges and, some might say, reduce the benefit of this implementation of the new approach.
   
2. Parameters posted from a form are dynamically mapped to a controller method and passed in as parameters. Perhaps not ideal for large forms but certainly the simpler ones can use this instead of having to deal with a property hash. They can also mangle the form names into an object like good old structs forms.
   

Ends up at the same place. After a couple of hours going through tutorials I have another tabled scaffolded.

Tripping in .Net

It has been a few years now but I have another opportunity to play on a .Net project. Being so used to the Java way this is going to take a little getting used to. My colleges keep snickering at me. It has something to do with the audible sound I make when frustration sets in. "Oh but this is so easy with ant, what evil is this msbuild trying to foist on us", "This would be so much easier in Eclipse".

It's the little things. I want to copy a file and replace a string within it. A simple template replacement process or in java terms, ant copy filter. NAnt doesn't support it either so I feel the need to learn msbuild.

So I have a fred.msbuild file but you open it in VS and it gives you schema errors on all the property tags because msbuild uses the tag name as the property name. So where is the MSBuild editor? MS makes everything a dialog and thats too hard for build scripts. I just need an editor.

The script can now "Copy" a template file, then "FileUpdate" is to do string replaces in it. Oh I have a lot to learn.

I fully expect to be assimilated in another few weeks and will start extolling the beauty of this new world. Until then, it is painful while the new gadgets are being surgically implanted into my brain.

I am looking forward to the new C# language features.

Sieve of Eratosthenes - After thoughts

The most interesting part of this exercise was that each attempt performed better and better until I the DSL version. Even the objectisizing and then groovyizing resulted in better response times. Now perhaps I doing something silly in the DSL version because the extra method dispatch for the DSL keywords shouldn't slow it down 5 times.

It is clear that the need to use a DSL for this problem is irrational.

Here are the tests for the Sieve of Eratosthenes problems and the results

.Time: 488.8ms: Algorithm: Sieve1 Primes: [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97]
.Time: 248.4ms: Algorithm: Sieve2 Primes: [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97]
.Time: 270.4ms: Algorithm: Sieve3 Primes: [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97]
.Time: 1145.6ms: Algorithm: Sieve5 Primes: [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97]

Sieve of Eratosthenes - Try 4 - DSL

Now to re-describe the basic parts of the algorithm into domain terms. Not saying its better just cooler.

Sieve of Eratosthenes - Try 3 - Groovyize

How about those groovy categories, the really allow you to put the code in the place you want it. Note I didn't say the right place?

Sieve of Eratosthenes - Try 2 - Objectize

Add in some OO, not much, just a sprinkling.

Sieve of Eratosthenes - Try 1

The groovy quiz this week was a prime number problem and I evolved through 4 different solutions.

The first was the basic script style approach, the second used OO, the third was groovyized with categories and the last converts the whole thing into a DSL. DSLs are cool right?

I am going to let each unfold in different posts so that you can compare and contrast if you are interested.

Here is the first, the basic "Sieve of Eratosthenes"

A groovy finite state machine

Well this is my first DSL. The people on the groovy users list are talking about trying to model something like this from the ruby example posted.

Sing: "Anything they can do, we can do better, or at least as well."

"Language Meshing" - April 10, 2007

I have created a new term, "Language Meshing" that refers to the ability to mix difference languages within the same source file.

Not a new concept I know, but I like the term so am blogging it now. Yes I might have created something completely new and completely useless.

I was review what Charles Nutter was posting about for JRuby and found their use of the Java 6 scripting extensions.

http://headius.blogspot.com/2007/03/can-your-ruby-do-this.html

I now dub it "Language Meshing". It might catch on.

WheresTheWater

WheresTheWater
Originally uploaded by esumerfd.