Security and Quality through Testing or Process?

As an Agile software developer I talk alot about testing. I promote and teach Test Driven Development and push to change the habits of developers to encourage quality traits that make the software we produce just work.

I want to draw a distinction between the understanding that I am "testing" and the fact that I am following a "process". This may be moot in the grand scheme of things but it strikes me as important because of a sentence I was about to make to a security professional.

Consider this:

As a software developer, my primary goal is to make software work.

A QA developer is focused on making software NOT work.

This is why I am not capable of testing.

Similar logic demands that
I am not capable of producing secure code.

The problem with this logic is that is contradicts everything I teach. Test, test, test but, oh by the way, you are not capable of succeeding.

So, perhaps I should describe TDD as a development process that improves quality instead of a process that tests code. While the eventual result is a set of tests that verify features of code, the developer gets there by following the TDD process, not by trying to make things break.

This is not a statement of whether TDD is a design process or not. Difference discussion. I am addressing developer intent, not defining the outcome of the TDD process.

In TDD we first follow a simple mantra:

Run the Test,
Change the Test,
Change the Code,
Repeat every 60 seconds.

Step two of this process could be argued to be an attempt to break code. However, this is not true since all we are doing is writing tests that will break because the functionality is not implemented yet.

TDD offers us a process to ensure that we end up with a suite of testing code that matches our running code. So TDD is not a testing process, just a development process that encourages a higher quality of working code than other processes.

From a higher level, the software development process we use already "values" a Quality Assurance team that double checks the work that we do. These people focus on breaking our code and do a really good job at it. They are a critical step to producing reliable software.

Another important role that software developers are unable to be good at is producing secure code. While we can learn the best practices and gain an understanding of the OWASP Top 10 attacks we are still trying to make software work for the business, we are not trying to prevent customers from attacking it.

Even with this problem, the software development process doesn't "value" a security organization reviewing the product that is produced.

Our industry is just starting to learn how to integrate security intelligence into the software build process and, to date, we software developers do not have a Security Driven Development process to improve our odd for success.

Scott Ambler's 2003 article predicted its creation in 2004 but his predicted internet shutdown wasn't as impressive a game changer as he thought. Unfortunately, that remains a learning curve in our future.

Dynamic vs Static Typing: An Observation

This is a dynamically typed language


pron. This v. is prep. a adj. statically n. typed n. language

Careful Java Programmers going to Ruby

Good evening everyone, I am a Java programmer.
It happened quickly, I didn't even notice.
One day I was merrily strncpy'ing and the next it was immutable.

We joke about languages of old while new shinny languages are paraded in front of us, but in the end they all teach us something and make us forget other things.

Today I came across the first serious change that the Java world thrust upon my weak mind. I am working a Ruby project, a great new shinny language which is lots of fun to learn and work with.

When working in Java I use the usual Agile development process, write your test, write some code and repeat. As I progress, my code and indeed the API design evolve into a nice usable, stable set of libraries.

Working in Java, with Eclipse, the simplest technique is to start by writing your test, reference a public method and flush out the implementation of that method. As you go, you will naturally refactor pieces of code into new methods to keep it simple.

Well, this is where things get a little too easy for Java developers in Eclipse. Select the code, press a couple of keys and Eclipse create the new method for you. A new private method in the same class.

Sounds good right? Well it is, except, that you have now learned to not thing about the scoping your your methods when you create them. Your first is always public or protected because your test is driving it and subsequent methods are private by default as created by the IDE. It's a beautiful system.

Now you switch to Ruby.

You write your test and implement your public method, you refactor with the cut/paste/rename/suffle and low and behold, because you have learned not to think about it, you have loads of public methods in your class.

The project I am working on takes it one step further and has tests for every method, examining all its nuances, mocking all its dependencies and going to town making the implementation as couple to the test as possible. It's easy to do, they are all public, what else is a programmer to do?

Unit testing philosophy does vary depending on who you talk to but my basic goal is to:

• Test all public methods,


• Test all protected methods,


• If a private method is particularly complex make it protected and test it,


• If I have loads of "untested" private methods I need to move some to new classes and make them public. I leave it to you to quantify "loads".


• Trust your coverage tool to tell you the private code you never use and delete it.

So, starting tomorrow, I change my Ruby development process and consider all new methods private unless part of a test driven process. This will do lots of things to improve the final product:

1. Simplify the test/code dependencies making the tests less brittle.


2. Prevent call sequence bugs where you have to call one method before another or it all falls over. Too many publics will do that you know.


3. Make the calling conventions exposed by the class easier to grok by new programmers reading the code.


4. Improve Class API design which is one of the primary benefits of TDD, lost to the lazy public manic people like me.

Oh, yes, an stop putting java semicolons in Ruby code. That really upsets the Ruby people.

Tests and Blank Lines

Add blank lines please.

This might sound stupid and, indeed, alternate DSL's for unit testing are making this less of an issue but there are still a lot of the old XUnit frameworks that prevent isolation of the 3 areas of your tests.

Every test you have needs to be setup, executed and validated. Let's start with a Java example.

public static void testSomething()
{
Address address = new Address();
address.setName("The House");
address.setLine1("1234 Street Ave");
address.setCity("Cityville");
address.setCountry("US");
address = dao.geocode(address);
assert_equals(-86.234, address.getLatitude());
assert_equals(46.234, address.getLongitude());
}


What does this test do? Which lines of code are being tested? Ok, lets try again with blank lines.

public static void testSomething()
{
Address address = new Address();
address.setName("The House");
address.setLine1("1234 Street Ave");
address.setCity("Cityville");
address.setCountry("US");

address = dao.geocode(address);

assert_equals(-86.234, address.getLatitude());
assert_equals(46.234, address.getLongitude());
}


Doesn't that help? A simple change and you can clearly see the three areas of your code.

I hear you shout "setup method", I know but junit sucks in that the only way to write enough tests with enough setup's is to have lots of separate test classes. Certainly go that way if you have a complex class to test but simpler stuff can be done in one class with a little setup and tests with blank lines.

OK, what is the future of these blank lines? Well, Ruby has unit test libraries like Shoulda that make these concepts easy to express.

context "an address" do

setup do
@address = Address.new(
:name => "The House",
:line1 => "1234 Street Ave",
:city => "Cityville",
:country => "US"
)
end

should "be geocoded to a specific lat and long" do

@address = @address.geocode

assert -86.234, @address.latitude
assert 46.234, @address.longitude
end
end


This allows us to separate and describe the areas of our tests, however, common usage has taken this API to the next level which, I think speaks to the need for the original idea behind the blank lines.

context "an address" do

setup do
@address = Address.new(
:name => "The House",
:line1 => "1234 Street Ave",
:city => "Cityville",
:country => "US"
)
end

context "when geocoded" do

setup do
@address = @address.geocode
end

should "have a specific latitude" do
assert -86.234, @address.latitude
end

should "have a specific longitude" do
assert 46.234, @address.longitude
end
end
end


As you can see, the use of the sub-context for each "test" and the use of the setup method to actually run the test, allows us to isolate the specifics of what the test is all about and describe each area.

Extreme Arguments and the Slippery Slope

So often we hear doubters ripping into ideas for their "potential" for destroying the world we know. Why the crazy extreme?

Most recently it was argued that code refactoring was bad because a programmer will spend so much time "perfecting" the code that they will never finish the project.

Another, states that we spend so much time unit testing that we don't write production code.

Another that you have to "corrupt" your beautiful domain design so much to make it testable that it makes OO irrelevant.

The Republicans are doing with the whole stimulus thing. A few billion spent means that we are on the slippery slope that results in the government owning every house in the country.

The "Slipper Slope" argument is the favorite argument of the person that doesn't have an argument for the proposal on the table.

Perhaps we should attempt assess the benefits of a proposal based on the proposal.

If you don't understand why something is bad you should not announce your opinion about your fears of what it might lead to. If it ends up leading there, announce away, until then, lets improve our world one step at a time.

Sticking to the facts is the single hardest thing we do. Especially, since facts can be a little slippery themselves.

Garmin nuvi 760

I had a Gadget Frenzy this morning that worked out quite well. This years birthday produced a Garmin nuvi 760 which will prevent me from getting lost unless I am too busy playing with its features to notice the turns it is telling me about.

Oh, by the way, don't play with your Garmin while driving. This is the first and largest warning the device gives you, even ahead of the one about how the device may, or may not, get you where you want to go.

This device also includes an SD Card slot and mp3 player, plus Bluetooth phone headset connectivity, an FM transmitter and an audio jack. Oh, and a picture viewer, but what on early is the point if that.

Unfortunately the FM transmitter is no use in Cincinnati because all the radio stations are too close and powerful, however, the audio jack and an audio cassette adapter work wonderfully.

So, imagine the scene. Car stereo blaring but with no sound, just yet, a fresh set of podcasts loaded onto the SD Card and a willingness to connect as many pieces simultaneously as possible.

Select the media player on the Garmin, create a play list and hit play for the latest episode of Furled Sails sailing podcast.

Hit Back and select "Where To" enter my work address and press Go. The mp3 pauses and a nice lady with British accent says, "Please drive to the beginning of the route" and back to the sailing adventures. She even says route like the Brits say it, root, I had forgotten, how sweet.

My Treo 650 Bluetooth has already synced with the Garmin so if someone would call me, please, we can have all three audio streams passing through this device, hopefully, appropriately paused.

So my problem now is, do I continue listening to the Sailing podcast or switch over to my His Excellency: George Washington audio CD. Either way, I should arrive at my destination, precisely at 8:29am.