The Rontologist

Random Thoughts of an IT Professional

Unit testing – When is enough enough?

Feb 1

Posted by The Rontologist in General Software, Programming | No Comments

It is pretty much a given these days that unit testing increases to quality of software. It increases the quality of code, and makes paying off any technical debt easier and cheaper. Unfortunately, they are also among the first casualties once the release date approaches and crunch time begins. There are many excuses, er, reasons why this occurs and I will not cover them here.

If we assume that unit testing time will be cut at the end then it should be possible to reduce the amount of unit testing we do at the start of the project in order to have more near the end. Assuming that this is true (which may or may not be depending on the project manager) then we need to answer a very interesting question. How can we tell how much unit testing is enough?

Having a code coverage number would be ideal. It is easy to get metrics for that and it easy to grasp. If I said 80% was a good enough number then we could calibrate our automated build to ensure that we hit it. But this doesn’t work in reality. If you set the bar at 80% then the easiest 80% of the code base gets unit tested. This leaves 20% of code (which has a very good chance of being complex) untested, while your trivial methods (e.g. accessors) are sitting at 100% tested. Setting a higher number helps offset this, but it is still not going to hit the intended target.

More complex metrics provide a better guide. A useful metric is C.R.A.P. (Change Risk Analysis and Predictions). It gives methods a score based off of complexity and unit test coverage. It provides a better rule of thumb as code coverage, but it can still be gamed. In fact, any metric that you can think of can be gamed.

Another tactic is to have a unit test for every bug logged. While this is a great idea, it doesn’t help determining if you have enough unit tests. It is too reactive for that. You are relying on your QA staff to determine coding behaviour and that is never a good idea.

Unfortunately there is no good answer to this problem. Every system, and team, is different and has different unit testing requirements. It takes a fair bit of experience and intuition to find out what is right for your project.

I would suggest the followings rules that must, at a minimum, be met:
- All complex code (if it cannot be simplified) should be tested as much as possible.
- All bugs should have unit tests that reveal them, and prove they are fixed.
- All modules that have a higher than average bug count should be tested as much as possible.
- Drop UI unit tests in favour of business tests, since they do not provide as much value.

Tags: ,

Even Sun was bitten by generics failings

Jan 4

Posted by The Rontologist in Java, Programming | 4 Comments

One of the biggest failings of generics in Java (and I know this is old news) is
compile time erasure. I came across this code and could not help but laugh that
even the owners of the language could not work around its failings.

EnumMap<MyEnum, Integer> map = new EnumMap<MyEnum, Integer>(MyEnum.class);

Wow. How many times do you need to type MyEnum there? Thanks Sun…

Tags: , ,

Ruby mixin fun

Dec 7

Posted by The Rontologist in Programming, Ruby | No Comments

We have a monthly programming challenge at my current job. I was reviewing
some of the submissions I created when I came across this piece of ruby code.

class String
def adj?(word)
0.upto(word.length – 1).inject(0) do |d, i|
word[i] != self[i] ? d + 1 : d
end == 1
end
end

It adds a function (adj?) to the built-in string class that lets me check if two strings differ by a single character easily. There is no bounds checking on this, but the dataset it was working with was guaranteed to have strings of equal length.

“abcd”.adj?(”abce”) == true
“abcd”.adj?(”efgh”) == false
“alpha”.adj?(”gamma”) == false

This could have been done by extending String, or having a util function, but that is not nearly as clean and neat as this. This small mixin is one of the many reasons why I love ruby.

Tags: , ,

cynicalman says…

Nov 12

Posted by The Rontologist in Funny, Misc | No Comments

cynicalman says…

I’ve got an idea. Make sure it f**king works!

Keeping developers happy – Up-to-date tech

Nov 2

Posted by The Rontologist in Programming | No Comments

Keeping software developers happy is hard. However, if done properly, it can decrease turnover and make hiring new talent easier. After all, who wouldn’t want to work for a place where people are genuinely happy to work?

The technology that developers use plays a part in this. I don’t mean hardware (although that is also important) as much as the software platform they work with. This includes programming languages, libraries, web servers, etc. Keeping these up-to-date helps to keep the software feeling fresh to those who work in it.

Technology, like anything else, ages. There is nothing that can be done about it. All the money that is invested in software development will eventually (with a bit of luck and a lot of work) yield a product that will, in turn, age. The good news is that there is a point where the developers know the platform really well. When you first enter this time then you are receiving the most bang for the buck with your team. They will be able to minimize the number of dead ends they explore, and be able to design your software better using the tools at hand.

The bad news is that if there is not a conscious effort to keep it up-to-date then you will try to remain in this stage. While it makes sense in the short term, it has horrible long term effects. Eventually, your products technology will be so old that no one wants to work in anymore (COBOL anyone?). If your software ever reaches this point it will be very hard not only to keep your existing talent, but to get new talent as well.

This is of up most concern if your business revolves around updating the same product, and continually making it better. You do not want to risk losing your most seasoned and talented developers on something like this. While upgrading technology can be expensive, so too is losing your best people.

Good developers are kind of like kids when it comes to technology. They both look forward to getting something new to play with. Tech is the toys that good developers like. When they first get it, they want to play with it and discover its secrets. It provides them with new problems to solve, and can possibly make their lives easier.

This is the behaviour you want to drive. An engaged developer is going to become more energized about their work, and will put more of an effort into their day to day job.

While you may not be able to implement a new tech (or an upgrade of an old one) in every release cycle, it should be something that is considered in every release cycle. Keeping your tech new not only makes keeping your people easier, it also makes recruiting easier. After all, new people are more likely to know new tech than old. Just ask the next person you interview if they know COBOL…

Tags: ,

Feeds moved to FeedBurner

Oct 30

Posted by The Rontologist in Misc | No Comments

I have finally got around to updating my feeds to use FeedBurner. I’m mainly posting this because the plugin I added requires it before it updates the RSS feed.

Update: Looks like that didn’t work. Time for me to debug…
Update 2: It appears to be working. Now I will be able to revel in FeedBurner goodness.

Avoiding part of a language

Oct 2

Posted by The Rontologist in Programming | 1 Comment

A few years ago I was working at a small company that was doing development. After a while we decided that it was time to implement coding standards for the usual reasons. While there were quite a few religious arguments (such as where do the braces belong and other silly topics), there was one point that really stuck in my head.

There was one part of the language that everyone except for a few people dreaded, and it was proposed that we disallow its use. We went back and forth a few dozen times about whether we should put it in or now. I voiced my opinions about why we should keep using it, and did not really hear a compelling reason on why we shouldn’t. While I don’t remember who won the argument, I still hold that it was not a very good idea.

Programming, at the very basic level, is problem solving. There is a task to do, and it is the programmers responsibility to distil that down into the most basic form until it is so simple that even a dumb machine can do it. Doing this well takes a lot of patience and is a lot of work. If you can do this then count yourself lucky, because not a lot of people can. So why do paid problem solvers want to avoid the problem of figuring out a part of their language? I can think of a few reasons.

Maybe it is a tricky part of the language? I really hope this is not the case. The problems that occur in most software projects are huge compared to a language. If they wasnt the case, then software development would look a lot different than it does today.

Maybe it is syntax sugar and therefore useless to learn? I don’t think this is valid either. There is plenty of syntax sugar out there, but it is there for a reason. It might make the code look nicer, in which case it is a good idea. Better yet, another developer will use it, in which case for maintenance sake it is an even better idea.

Maybe it is just too much work? I am not even going to go there.

There is not any good reason (then or now) that I can think of that would make a convincing argument for avoiding a piece of a language. Programmers only do themselves a disservice (and maybe cause a lot of extra work for themselves) by going down this path. I am not advocating that programmers learn everything about their language. However, they should not be afraid of any part of it either.

(for the curious: perl’s map was what some of my colleagues wanted to avoid).

It’s not just a theory!

Sep 18

Posted by The Rontologist in Misc | No Comments

I came across this site today. Very awesome indeed!

Not just a theory

Using anonymous inheritence to isolate untestable code.

Aug 17

Posted by The Rontologist in Java, Programming | No Comments

Unit testing bad code is hard. If it is not designed to be tested than odds are it will be hard to test. One of the biggest thorns in my side has been trying to unit test classes that use static methods to access resources.

Static methods tend to be used quite a bit in older code. The singleton pattern was pretty popular in the java world, especially when dealing with resources. Unfortunately, when we are unit testing, we want to avoid dealing with these resources.

Take this fragment for example:

public class AwesomeClass {
    public void performAwesome() {
        // some code
        AwesomeUtil.talkToServer();
        // some more code
    }
}

public class AwesomeUtil {
    public void static talkToServer() {
        // code to connect and communicate with a server
    }
}

Testing performAwesome is made more complicated because of the static method. It talks to a server, and that would make this an integration test (which we want to avoid). We could use a mocking framework that supports static rewriting, but I am not a huge fan of that (I love mocking, but I prefer using OO to solve this particular problem because it tends to lead to better design decisions when making good code).

Ideally we would have a free hand and refactor it fully using an IoC framework such as spring, but that rarely happens, and would add a huge load on QA if we did. One of the nicest tricks I have learned is to do a micro-refactoring.

public class AwesomeClass {
    public void performAwesome() {
        // some code
        talkToServer();
        // some more code
    }

    public void talkToServer() {
        AwesomeUtil.talkToServer();
    }
}

This modification to the code is logically equivalent. The only change is that the call to the static method is now handled in a non-static function in our class. It is a very simple and low risk change.

And it allows the following:

public class AwesomeClassTest {
    int talkedToServerCount;
    AwesomeClass awesomeClass
    @setUp
    public void setUp() {
        talkedToServerCount = 0;
        awesomeClass = new AwesomeClass() {
            talkToServer() {
                ++talkedToServerCount;
            }
        }
    }
}

I find that pretty cool. The version of the class we are testing no longer has a static method, and we can not test is as we would any other class.

Ruby fun – testing multiple objects the cool way

Jan 20

Posted by The Rontologist in Programming, Ruby | No Comments

A common task to do in programming is to iterate through a list and test each element in a collection. In most languages this is done exactly how you would think.

strings = ["param1", "param2", "param3"]

passed_all = true;
strings.each {|s|
    passed_all = false unless s.length == 6
}

puts "Matched all" if passed_all;

However Ruby is kind enough to give us the any? method on its collections. This allows for doing the same test in more more straightforward manner.

strings = ["param1", "param2", "param3"]

if (strings.all? { |s| s.length == 6 })
  puts "Matched all"
end

It also gives us an any? method too if you need to do an action if, you guess it, any item in the collection passes a test.