login
Blurts on the Art of Software Development

Today | RSS | RDF | Atom | Other Tags
Categories : All | All | CI | .NET | General | Humour | Java | Personal | Reviews | Ruby | SW Eng

Once again, there's a discussion going on about whose unit testing framework is bi... I mean, better. Cedric is asking are dependent test methods really evil to which Robert Watkins replied that they're at least mildly naughty.


I don't care about which is better... I've no vested interest in JUnit, and there are a number of features of TestNG that I like. What gets to me is the misconceptions, as I see them. I can't help it - I feel compelled to put forward what I see as the truth. :) One of these days I'll learn not to bite.

Likewise. There are a bunch of stuff about JUnit that are mildly annoying and there's a bunch of stuff in TestNG that make me go "wow", but so far I'm still using JUnit and JUnit only.

For me, perhaps the most important thing is that a framework enables instead of restricts the developer from doing stuff. I haven't yet familiarized myself enough with TestNG to be able to determine whether JUnit or TestNG satisfies this test better.

So I look at a tool like Agitator, that is truly, in my mind, an extreme unit testing tool and then some. True unit testing and for that mater functional testing is trying to *break* an application or bit of code. Most of the unit tests I see only test that something works the way its expected to. They don't throw tons of different tests at the same one bit of code, be it a method or functional test that groups many method calls. Agitator does this. For example, in a method with a String argument, it will pass null, 0-length string, a very long string, strings with different characters (unicode, double-byte, etc), various odd character combos (such as the < character for web pages that tends ot break pages) and more. I haven't actually used Agitator, a bit out of my price range. But what it does is simply amazing. I truly believe it does what all unit tests should do, it throws the unimaginable at a given method trying to break it, and often it does. How many times do we write a method knowing that its only called here and here (2 places) and we KNOW that we dont pass null? Well, that's great sometimes, but you never know when someone is going to extend it and somehow a null creeps in. I think what I like about the approach the Agitator methodology takes is that it makes programmers better. The first time they are hit with agitator they are seeing stockpiles of failures. WOW! Then they start learning... to check for nulls, to make sure Files exist, to double-check URLs, and more, just for starters. I can't even fathom what it's totally capable of, but from everything I've read, it's an amazing tool and well worth a funded company to purchase and make use of from the start. Now, getting back to junit/testng/etc, my thought is that a unit test framework, or for that matter a test framework should try to make use of reflection in the case of java, figure out the arguments a method takes, and try throwing tons of different tests at it. So, we have our normal unit tests where we fill in some mock objects, pass them along and test. We have our UI automation tests where we try to find components, click/type, and ensure dialogs pop up correctly, localization works right, etc. Then we have the "depper" testing that tries to figure out each and every method and argument, and with it tries to throw tones of different data at each argument.
For those wondering what the hell "Agitator" is from the previous post, he means "Agitar". http://www.agitar.com/
Well, he means Agitator, from the compnay Agitar...


Add a comment

Title
Body
HTML : b, i, blockquote, br, p, pre, a href="", ul, ol, li
Math Quiz 9 + 1 = (Helps stop blog spam)
Name
E-mail address
Website
Remember me Yes  No 

E-mail addresses are not publicly displayed, so please only leave your e-mail address if you would like to be notified when new comments are added to this blog entry (you can opt-out later).

TrackBack to http://radio.javaranch.com/lasse/addTrackBack.action?entry=1111128030438