login
Blurts on the Art of Software Development

Today | RSS | RDF | Atom | Other Tags
Categories : All | All | CI | .NET | General | Humour | Java | Personal | Reviews | Ruby | SW Eng

Recently a lot of people have been asking how to test for expected exceptions. I kind of cover this in my book but since not everyone will buy it and considering that my old blog entry about creating suites with JUnit 4 is one of the most popular ones according to my logs, I thought I should write a quick blog post about it.

So here it is:

import static org.junit.Assert.fail;
import org.junit.Test;

public class TestExpectedExceptions {

    @Test
    public void testForExpectedExceptionWithTryCatch()
            throws Exception {
        try {
            Integer.parseInt("This should blow up...");

            // Uh-oh! No exception was thrown so we 
            // better make this test fail!
            fail("parseInt() should've thrown an exception!");
        } catch (NumberFormatException expected) {
            // this is exactly what we were expecting so 
            // let's just ignore it and let the test pass
        }
    }

    @Test(expected = NumberFormatException.class)
    public void testForExpectedExceptionWithAnnotation()
            throws Exception {
        Integer.parseInt("This should blow up...");
    }
}

In simple terms, here's what the above code snippet illustrates:

Classic Try-Catch Structure

The first test uses a JUnit 3 and 4 compatible try-catch structure for making sure that, if the code under test does not throw an exception, we'll fail the test case by invoking Assert#fail(). We leave the catch-block empty here but we could also make further asserts about what information we expect the thrown exception to contain. For example, we might assert that the exception's getMessage() method returns a description that contains some specific keyword.

Compact Annotation

The second test uses a JUnit 4-only feature that builds on the concept of annotations introduced into the language by the release of Java 5. In this less verbose example, we simply interact with the code under test in a way that we expect to cause an exception to be thrown. Normally, a test method throwing an exception would make JUnit flag that test as a failed one but since we have defined the expected parameter to our @Test annotation, JUnit instead fails if the method doesn't throw an exception (and, specifically, that type of an exception).

Which one should I use?

So these are the two options. If you're using JUnit 3, then your only option is the first one. If you're using JUnit 4, then you can choose between the two and decide, case by case, whether you like the compact form more. The compact form does not allow assertions against the thrown exception, though, beyond its type so you'll probably end up using both anyway.

I hope this was helpful.


I think that assertions against thrown exception are very important. Actually I usually found very few cases where asserting the type is enough. When exception handling is done right, the exceptions should contain some information about context (e.g. IllegalArgumentException should always include information about the argument which was erroneous). This info is important part of the object behaviour and there should be an assert for it.
There's one more indication for going the classic way. Sometimes you want to check that an object (or a config of objects) is in a determined state after an exception was thrown. Classic example: a DB session might still be usable after some exceptions but not after others. Other example: An account should not have changed its balance when withdrawing money resulted in a WithdrawalNotCoveredException. You cannot do such checking in annotational style.
In such a case as checking a correct balance in an account after an exception is thrown, it sounds as though you are making your unit test too complicated by mixing two different tests. Test one should be: Is the exception thrown under correct circumstances? Test two should be: Is the balance of the account correct after a failed withdrawal transaction? In the case of test two, you don't need to re-check that the exception is thrown under the correct circumstances, (if test one was correctly designed it would have already proved that) test two can then safely "eat" the exception, and then move on to test the "after exception" balance condition. This is how unit testing works. The tests are meant to be as small and specific as possible. Any individual unit test should only be capable of failing in 'one way' If your test can fail because of a dropped exception OR a failed balance, then the test is probably too complex. --But that's just my .02.
Wouldn't "eating the exception" almost look like checking that the exception is thrown in the first place? Since both aspects are part of the correct exceptional behaviour I'd probably like to see it in a single spot.
Having descriptive names for the @Test methods should help with that.
Let's consider the following test:
@Test
public void uncoveredWithdrawalFails() {
  account.setBalance(100);
  try {
    account.withdraw(101);
    fail()
  } catch (UncoveredWithdrawalException expected) {
    assertEquals(100, account.getBalance();
  }
}
How would you transform it into two tests?
Technically, like this:
@Test(expected=UncoveredWithdrawalException.class)
public void withdrawingOverBalanceRaisesException() {
    account.withdraw(account.getBalance() + 1);
}

@Test
public void attemptToWithdrawOverBalanceDoesNotAffectBalance() {
    int balanceBeforeWithdrawal = account.getBalance();
    try {
        account.withdraw(balanceBeforeWithdrawal + 1);
        fail()
    } catch (UncoveredWithdrawalException expected) {
        assertEquals(balanceBeforeWithdrawal, account.getBalance());
    }
}
It is more verbose and I probably wouldn't choose this over the combined test even though it does make the intended behavior very clear by giving names to the two aspects we're testing. Then again, using a tool other than JUnit (e.g. JDave) might level things out and make it more favorable to go with the two tests approach. In practice, I'd probably just give your example test a more descriptive name:
@Test
public void uncoveredWithdrawalFailsWithoutAffectingBalance() {
    account.setBalance(100);
    try {
        account.withdraw(101);
        fail()
    } catch (UncoveredWithdrawalException expected) {
        assertEquals(100, account.getBalance();
    }
}

Someone (Joonas I think) just started using this nice little testing class from Spring (spring-mock) in our project:

        new AssertThrows(NumberFormatException.class) {
            public void test() throws Exception {
                Integer.parseInt("This should blow up...");
            }
        }.runTest();

To make Markus happy, you can also get the actual exception and assert something about the message (I like doing that, always used to have StringAssert(from junit-extensions) .assertContains() on the message in the traditional Java 1.4 style catch block)

        AssertThrows aThrows = new AssertThrows(NumberFormatException.class) {
            public void test() throws Exception {
                Integer.parseInt("This should blow up...");
            }
        };
        aThrows.runTest();
        StringAssert.assertContains("For input string: \"This should", 
            aThrows.getActualException().getMessage());
Works on Java 1.4 too.
If you're willing to do some more hacking with imposterization, you can also do, for example:
assertThrownException(is(e)).when(emptyList).get(0);
Unfortunately, it currently only works for instance methods http://shareandenjoy.saff.net/2006/12/assertthrownexception_20.html
Thanks a lot :)


Add a comment

Title
Body
HTML : b, i, blockquote, br, p, pre, a href="", ul, ol, li
Math Quiz 7 + 5 = (Helps stop blog spam)
Name
E-mail address
Website
Remember me Yes  No 

E-mail addresses are not publicly displayed, so please only leave your e-mail address if you would like to be notified when new comments are added to this blog entry (you can opt-out later).

TrackBack to http://radio.javaranch.com/lasse/addTrackBack.action?entry=1179405760728