login
Blurts on the Art of Software Development

Today | RSS | RDF | Atom | Other Tags
Categories : All | All | CI | .NET | General | Humour | Java | Personal | Reviews | Ruby | SW Eng
Permalink
JUnit 4 trivia
Test suites using annotations

It seems that not that many developers migrating to JUnit 4 are aware of the replacement for the old way of building suites--using the static suite() method and the junit.framework.TestSuite class--I decided to blog a simple example for Google's indexing hamsters to grab hold of.

Here we go:

import org.junit.runner.RunWith;
import org.junit.runners.Suite;
 
@RunWith(Suite.class)
@Suite.SuiteClasses({
  TestCalculatorAddition.class,
  TestCalculatorSubtraction.class,
  TestCalculatorMultiplication.class,
  TestCalculatorDivision.class
})
public class CalculatorSuite {
    // the class remains completely empty, 
    // being used only as a holder for the above annotations
}

The above class is a simple placeholder for the suite annotations, not containing any other functionality as such. The key is in the @RunWith annotation, which tells the JUnit 4 test runner to use the org.junit.runners.Suite class for running this particular class. The @Suite annotation, on the other hand, tells the Suite runner which test classes to include in this suite and in which order.

I hope this helps folks out there since it doesn't seem to be documented too well in junit.org.


As with so many of these JUnit4 additions I'm struggling to see the advantage of this. It seems to be just replacing one clunky syntax with another.</p.

Do you have any suggestion of why (or when) it might be better to use this than the old suite method approach?

I love most of what's changed in JUnit 4 but I do have to say that I kind of agree with you on the syntax being a bit clunky.

Personally, I would've preferred having an alternative enabling the building of a test suite dynamically. Something like this:

@Suite
public class MySuite {
    public static Test suite() {
        // do it the old way
    }
}

...where there's the old suite() method and the class is annotated with @TestSuite (mainly to help tool developers).

You'll have to pose the question to the JUnit developer list, though, to get the answer you're looking for...

Thank you!!! (It really isn't documented "too well" in junit.org or anywhere else I found so far).
This works excellently in Eclipse, but in addition to using the IDE GUI I'm using an Ant Build file to generate Emma Code Coverage statistics. Emma requires a Main method that forces all execution paths to be executed, is there a way to do this in JUnit4?

Mark,

You can simply create a main method that invokes the JUnit API as follows:

public class MainForEmma {
    public static void main(String[] args) {
        JUnitCore.runClasses(new Class[] { MyTestSuite.class });
    }
}
i would prefer public static void main(String[] args) { junit.textui.TestRunner.run(suite()); }
I hope you don't mind my saying "Thank You" about a hundred times... I was using the basically excellent tutorials at http://www.cs.umanitoba.ca/~eclipse/, but their JUnit TestSuite example just would not come together. Spent hours looking at docs but nada. Googled your site ( Thanks Google)and came away HAPPY. I'm a newbie with Java, Unit Testing and Eclipse. Coming from VB6 and this is alot at one time... Keep up the good work.
nice. thank you
or you could use the following in ant
${testSrc} - is the top level test directory
${testDir} - is the directory where the classes you want to run are

<target name="runTests" description="run the junit tests">
<junit outputtoformatters="false" showoutput="no" fork="on" haltonfailure="no" printsummary="yes">
<formatter type="plain" usefile="false"/>
<classpath refid="classPath"/>
<batchtest>
<fileset dir="${testSrc}">
<include name="${testDir}/**/*Test.java/"/>
</fileset>
</batchtest>
</junit>
</target>
hi, i need to execute LoginTest.class first and then the other *Test.class and finally LogoutTest.class i tried with ANT in this format, <junit> <test name="LoginTest" /> <batch> <fileset dir="${classes.dir}" includes="*Test.class" excludes="LoginTest.class,LogoutTest.class" /> </batch> <test name="LogoutTest" /> </junit> Problem it is executing in its own order and not in specific order. is there any way to execute these in specific order with ANT. and, i'm using JUnitCore.runClasses(new Class[] {}); or how to execute this type of Suite with ANT. now i'm using the formatters in xml format ant generating HTML reports with <junitreport /> tag. Thanks in advance.

Did I understand correctly that you don't actually have a suite but rather multiple test classes that you collect from the file system for execution?

If this is the case, how do you run those tests in your IDE?

hi,
thanks for the reply.
i have several test classes to test, my Struts-Actions. i'm using <junit> and <junitreport> elements in Ant to test them and to generate the reports in HTML format.

up to this is fine. the problem is i'm using <batchtest /> in <junit/> which is executing the test classes in alphabet sequence. and not in the desired sequence.

to avoid that i'm using <test /> element in the <junit/>, which follows the desired sequence, but the problem is i need to mention more than 30 tags in <junit/> which is more adequate.

when i tried to run the test classes from java with JUnitCore api, it is executing in the desired sequence.

so my need is, i need to run the test cases from ant in some sequence.
is there any format for that or is there any tag that executes the TestSuite.

Point your Ant script to run a single Suite class where you define the tests you want to run and their desired order. Have you tried that?
Thank you, this was very helpful, I was up and running with a test suite within 5 minutes of reading your blog entry, I have not found this info elsewhere.
Same here - searched how to make suit and tests to work, your little sample helped right away! Thanks!
Nicely Done, Finally some documentation on using TestSuites. thanks Surj
What is the replacement of suite.addTest method where it was possible to form test suite with diffrent test methods from multiple classes. With "@Suite.SuiteClasses" we can only pass the class which will result in running all the tests from that class. Using suite.addTest, it was possible to do something like this : suite.addTest(new MyClassA("testA")); suite.addTest(new MyClassB("testB")); Could you please suggest any way of achieving this using JUnit4. Regards, Afroz

Afroz,

I don't know of a way to do that in JUnit 4. In fact, I don't even know that I would want such an ability. Why? Because the fact that you want to execute only some tests from a given test class is an indicator of that test class being too big and incoherent. A design smell. Instead, you should consider splitting your test class into two or more.

Thaks for your reply. The problem here is not because of the big test class. Basically if i want to do some scenario testing where it requires to call methods from multiple test classes. To do that I would create one test suite for one scenario and add different methods to it. This way I can reuse my test classes for multiple scenarios. I belive that JUnit is not for this type of testing !!!
You're quite correct, Afroz. JUnit isn't designed for that kind of a thing.
When debugging a single test case, I have found it useful to temporary isolate just the single test case by using the old (JUnit 3.8) method: suite.addTest(new MyClassA("testA")); This way you do not have to run the entire set of test cases just to observe the results of the one test case. My interest here is expediency, and not necessarily because the class is too big. So, to Afroz's question, is this possible in JUnit 4.x? Thanks!

What I usually do is I run the single test case in Eclipse by just right-clicking it from the JUnit runner and selecting "Run" (or "Debug" if I'm having a bad day ;).

I'm not sure how to do this from the command line, though. I mean, Ant (look for "run-single-test") and Maven have means of running a single test class but I'm not aware of a way to run a single test method with them.

Hello Thanks for the post. Is really helpful in learning how to build suites in jUnit4.But i had an idea that would be difficult to implement this way. I want to have a method in my suite in which i want to set a system property(something like environment setting) for the tests to use while execution. So when the tests i include in the suite, run they run using that property. But if the class is empty how can i achieve this? Thanks, Sriki

You can add @BeforeClass and @AfterClass methods to your suite class and have them coordinate the system properties.

For example:

import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;

@RunWith(Suite.class)
@SuiteClasses( { ExampleTest.class, AnotherExampleTest.class })
public class MySuite {

    public static final String PROPERTY_NAME = "whatever";

    private static String previousValue;

    @BeforeClass
    public static void setUp() throws Exception {
        previousValue = System.getProperty(PROPERTY_NAME);
        System.setProperty(PROPERTY_NAME, "value for this suite");
    }

    @AfterClass
    public static void after() throws Exception {
        if (previousValue != null) {
            System.setProperty(PROPERTY_NAME, previousValue);
            previousValue = null;
        }
    }
}
Hello, Does anyone know how I can run a suite from a suite in JUnit 4, so that I have a hierarchy of tests? Thanks, Martyn
Like this:
@RunWith(Suite.class)
@SuiteClasses( { SomeTestSuite.class, AnotherTestSuite.class })
public class CompositeSuite {
}
Hi,

I also have a problem with nested test suites. My hierarchy is:
test suite A
|-test suite B
| |-test case 1
|-test suite C
  |-test case 2
test suite A contains:
@SuiteClasses( { B.class, C.class })

test suite B contains:
@SuiteClasses( { 1.class })

test suite C contains:
@SuiteClasses( { 2.class })

but when I run test suite A, the test methods of 1 and 2 will shown directly under A, and my hierarchy is lost.

What am I doing wrong? I am using NetBeans 6.0.1

Thanks, Oli
In JUnit 3x, it was possible to attach a descriptive name to a suite via TestSuite(String name), and that name would display in the runner results (instead of the class name). Is there a similar feature in JUnit4? Thanks, Alex
Well, can't the class name be descriptive?
Well, I too hold that the class name absolutely should be descriptive. But in the JUnit output the class name is preceded by the long package name and scrolls out of the view. Also, you can't have blanks in a class name and those are very useful in descriptions :-)

I understand how to make these JUnit 4 suites, now, but the problem is I don't want to have to explicitly put in the class names under @Suite.SuiteClasses().

I'm running my tests from Eclipse, and I want a suite class that creates a list dynamically. Is that even possible?

We're mixing JUnit 3 and JUnit 4 tests, but the old method of getting all the tests only picks up the JUnit3 tests. I'd love something that uses the new system.

I'd want something like this:

@RunWith(Suite.class)
@Suite.SuiteClasses(
    getClassesInPackage()
)
public class PackageSuite {}

If you're running tests from Eclipse, how about just right-clicking the package and selecting "Run As > JUnit test"?

In any case, you could take this as a base and modify it to suit your purposes.

It's a shame this isn't better documented at junit.org. For taking the time to post this:
public class ShowAppreciation
{
    public static void main(String[] args)
    {
        for (int i = 10; i<1000000; i++)
        {
            System.out.println("Thank you Lasse!");
        }
    }
}
Michael, you could do it the following way: put your junit 4 tests in a suite and wrap it in a suite() method returnig a Junit4TestAdapter (see its doc). this is a "downgrade" to 3.8 and can be run either by 3.8 runners or the 4.4 runners, e.g. the JUnitCore I am using. Unfortunately you would still have to annotate the jUnit 4 testclasses manually, but they could be picked up along with your 3.8 testclasses and/or suites dynamically.
Has anyone else run into the problem where you have a test suite with dozens of test cases that have the same test method names, and as a result it is difficult to spot which classes are failing from the junitreport?
The example is great. But how about this kind of suite():
public static Test suite() {
    TestSuite suite= new TestSuite();
    suite.addTest(new MoneyTest("testMoneyEquals"));
    suite.addTest(new MoneyTest("testSimpleAdd"));
    return suite;
}
I just want add some specific methods into a suite(), not the whole class, how could I do that?

You extract those test methods into a separate class...

Out of curiosity, what differentiates the tests that you want in your suite from the rest of the tests in that class?

Thanks Lasse.

I am reading Pragmatic Unit Testing in Java with Junit, it suggests to put part of test cases in suite(), but leave those time-consuming ones in a high level class which contains all tests.

Then by default only test contained in suite() run, and those "big" tests only run when we invoke the high level test.

Unfortunately this book only talks about junit 3.

Maybe it is not a good way to arrange test cases?

I tend to separate my unit tests from integration and system level tests by placing them into separate source trees.
public class TestSingleMethod{
	public static void main (String [] args )
	{
		JUnitCore junitcore = new JUnitCore();
		junitcore.addListener (new TextListener());
		junitcore.run(Request.method(TestClass.class, "method1"));
	}
}
... will execute method 'method1' of class 'TestClass' (Junit 4.4)
It appears to me that TestSingleMethod does not do anything... Maybe because JUnit assumes that you already ran the whole suite/class? (Eclipse plugin seems to suffer from the same problem: only re-run of single test method works). Can anyone confirm that it works? Or is there an easier way to do this? I receive this when I run it (with some modifications though): Time: 0.008 OK (0 tests)
package junittest;

import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.PrintStream;
import java.util.ArrayList;

import org.junit.Test;
import org.junit.internal.TextListener;
import org.junit.runner.JUnitCore;
import org.junit.runner.Request;


public class TestSingleMethod
{
	public static void main (String [] args )
	{
		JUnitCore junitcore = new JUnitCore();
		try {
			junitcore.addListener (new TextListener(new PrintStream (new FileOutputStream("testresult.txt"))));
		} catch (FileNotFoundException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		junitcore.run(Request.method(TestSingleMethod.class, "test1"));
	}
	
	@Test
	public void test1 ()
	{
		System.out.println("test1");
		ArrayList <Object> emptyList = new ArrayList <Object> ();
		// Results in an error!
		Object o = emptyList.get(0);
	}
	@Test
	public void test2 ()
	{
		System.out.println("test1");
	}
}
I made my example above a bit more detailed. Copy it, execute it and you will see, that test1 ist executed and its junit-testouput is written to the file "testresult.txt". However you will also see, that test2 is NOT executed by junit. I hope the short example will be of some help. (Env: jre1.5.0_12, eclipse 3.3, junit 4.4)
I stand corrected. Thank you for that follow-up. However, what I was trying to do is more advanced: I'd like to run a @Test method that resides in another (arbitrary) class. The idea would be to give command line args for both test class and test method (and possible classpath). My scenario still results in a no-op, unfortunately.


Add a comment

Title
Body
HTML : b, i, blockquote, br, p, pre, a href="", ul, ol, li
Math Quiz 8 + 10 = (Helps stop blog spam)
Name
E-mail address
Website
Remember me Yes  No 

E-mail addresses are not publicly displayed, so please only leave your e-mail address if you would like to be notified when new comments are added to this blog entry (you can opt-out later).

TrackBack to http://radio.javaranch.com/lasse/addTrackBack.action?entry=1154024535662