JUnit was probably the first Java unit test framework (I don't know any which has its beginning in 2002. Then in 2004 Cédric Beust frustrated of JUnit deficiencies, has created TestNG. In this article refers to JUnit (v 4.12) and TestNG (v 6.10) but new version of JUnit is in progress.
Both frameworks are dedicated to create unit, functional and integration tests but with TestNG it is all possible without any third party libraries. TestNG is more configurable in case of grouping and ordering of test execution.
The main reason why TestNG was created is that creator of TestNG was frustrated that JUnit creates a new instance of class for each test and he changed this approach in the TestNG to create one instance of a test class for each test methods in this class. This has some consequences - JUnit class is state full - contains class attributes which are refreshed before each test method. In case of TestNG class's attributes share some state for all test methods in a class. This assumption allows to create in an easy way test sequence.
Both frameworks allows to categorise and group tests. However the TestNG is much more flexible and allows to create some sequence of tests.
Bellow I gathered most important advantages of each framework and attached some simple examples of use it.
TestNG (v 6.10)
1. Dedicated for unit, functional, integration tests.
2. Make dependences between methods and groups in one class.
2. Make dependences between methods and groups in one class.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | @Test() public void testZAdd() throws Exception { calc.addMemory(BigDecimal.TEN); Assert.assertEquals(calc.getMemory(), BigDecimal.TEN); } @Test(dependsOnMethods = {"testZAdd"}) public void testAdd2() throws Exception { calc.addMemory(BigDecimal.TEN); Assert.assertEquals(calc.getMemory(), new BigDecimal(20)); } @Test(dependsOnMethods = {"testZAdd", "testAdd2"}) public void testClean() throws Exception { Assert.assertEquals(calc.getMemory(), new BigDecimal(20)); calc.cleanMemory(); Assert.assertEquals(calc.getMemory(), new BigDecimal(0)); } |
3. One class can be supported by one thread
4. To organise all test in a testng.xml file - something like suite case in JUnit
5. Support input parameters of test method.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | <test name="MyTestGroup"> <groups> <run> <exclude name="brokenTests" /> <include name="checkinTests" /> </run> </groups> <classes> <class name="test.IndividualMethodsTest"> <methods> <include name="testMethod" /> </methods> </class> </classes> </test> |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | @DataProvider(name = "test1") public Object[][] createData1() { return new Object[][] { { null, null, BigDecimal.ONE, BigDecimal.TEN}, { BigDecimal.TEN, new BigDecimal(2), new BigDecimal(4), new BigDecimal(4)}, }; } @DataProvider(name = "test2") public Object[][] createData2() { return new Object[][] { { null, null, BigDecimal.ONE, BigDecimal.TEN}, { new BigDecimal(12), new BigDecimal(2), new BigDecimal(2), new BigDecimal(3)}, }; } @org.testng.annotations.Test(dataProvider = "test1") public void testAdd(BigDecimal sum, BigDecimal a, BigDecimal b, BigDecimal c) throws Exception { Assert.assertEquals(ComplexOperations.add(a, b, c), sum); } @org.testng.annotations.Test(dataProvider = "test2") public void testMultiply(BigDecimal sum, BigDecimal a, BigDecimal b, BigDecimal c) throws Exception { Assert.assertEquals(ComplexOperations.multiply(a, b, c), sum); } |
6. Reusable of methods and classes.
7. Support parallel testing: per class, method, instance, etc. - It is possible to execute one method in parallel many times, ex. 60 times in 10 threads.
or by test.xml
8. Inject dependences alone or by an external library ex. by Guice1 2 3 4 5 | @Test(dependsOnMethods = {"testClean"}, threadPoolSize = 10, invocationCount = 60, timeOut = 1000) public void testAfterClean() throws Exception { System.out.println("To " + Thread.currentThread().getId() + " -> " + this); Assert.assertTrue(true); } |
or by test.xml
1 | <suite name="My suite" parallel="classes" thread-count="5"> |
JUnit (v. 4.12)
1. Categorisation of test ??class or method??
2. Parametrisation of test but test class instance have to be statefull.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | @RunWith(Parameterized.class) public class ComplexOperationsJUnitDataProvider { private BigDecimal sum; private BigDecimal a; private BigDecimal b; private BigDecimal c; @Parameterized.Parameters public static Object[][] createData1() { return new Object[][]{ {null, null, BigDecimal.ONE, BigDecimal.TEN}, {BigDecimal.TEN, new BigDecimal(2), new BigDecimal(4), new BigDecimal(4)}, }; } public ComplexOperationsJUnitDataProvider(BigDecimal sum, BigDecimal a, BigDecimal b, BigDecimal c) { this.a = a; this.b = b; this.c = c; this.sum = sum; } @Test public void testAdd() throws Exception { Assert.assertEquals(ComplexOperations.add(a, b, c), sum); } } |
3. Basic test ordering by @FixMethodOrder(MethodSorters.NAME_ASCENDING)
4. Assumption to tested code. If assumptions is false then skip test.
1 2 3 4 5 6 | @Test public void testAdd1() throws Exception { Assume.assumeThat(calc.getMemory(), is(BigDecimal.ZERO)); calc.addMemory(BigDecimal.TEN); Assert.assertEquals(calc.getMemory(), new BigDecimal(10)); } |
5. Continues testing - external library. - This solution execute a test in background after it discover change in code.
6. Concurrent test - ConcurrentUnit
7. Ignore test
Test class with 2 test methods, one with Ignore annotation, gives following result
Tests run: 2, Failures: 0, Errors: 0, Skipped: 1
1 2 3 | @Ignore @Test public void testMyAlgorythm(){ ... } |
Test class with 2 test methods, one with Ignore annotation, gives following result
Tests run: 2, Failures: 0, Errors: 0, Skipped: 1
8. Timeout for test
1 2 | @Test(timeout = 10) public void testMyAlgorythm() { ...} |
9. Rules in test class body
10. @Theory mechanism - helps with generation of input data
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | @RunWith(Theories.class) public class CalculatorWithMemoryJUnit { private CalculatorWithMemory calc = new CalculatorWithMemory(); @Theory public void testSum(@TestedOn(ints = {0, 10, 30, 60}) int val, @TestedOn(ints = {0, 10, 30, 60}) int sum) { Assume.assumeThat(val, is(sum)); System.out.println("Test with val " + val + ", sum=" + sum); Assume.assumeThat(calc.getMemory(), is(BigDecimal.ZERO)); calc.addMemory(new BigDecimal(val)); Assert.assertEquals(0, new BigDecimal(sum).compareTo(calc.getMemory())); } } |
Next time I am going to present mock frameworks.