Executing Tests Selectively in TestNG

TestNG offers various approaches to re-run only failed test methods, allowing for efficient use of resources in large test suites. The TestNG framework automatically generates a testng-failed.xml file after each test execution, which can be used to re-run failed tests.

Why This Matters

In large test suites, re-running all tests after a failure can be time-consuming and wasteful. By executing only failed tests, developers can quickly identify and fix issues, reducing the overall testing time and improving productivity. For instance, a study found that selective test execution can reduce testing time by up to 70% in some cases.

Key Insights

TestNG’s testng-failed.xml file allows for easy re-running of failed tests: this file is generated automatically after each test execution and includes only the failed test methods.
Using in testng.xml enables fine-grained control over test execution: developers can manually specify which tests to run by adding entries in the testng.xml file.
Maven Surefire’s single method execution simplifies test re-running: developers can use the -Dtest parameter to execute a single test method, making it easier to debug and fix issues.

Working Example

public class ExecuteSelectivelyUnitTest {
    @Test
    public void givenTest_whenFails_thenExecuteSelectively() {
        Assert.assertEquals(5, 6);
    }
    
    @Test
    public void givenTest_whenPass_thenExecuteSelectively() {
        Assert.assertEquals(5, 5);
    }
}

To re-run only the failed test, use the following command:

mvn test -Dsurefire.suiteXmlFiles=target/surefire-reports/testng-failed.xml

Practical Applications

Use Case: A developer can use TestNG’s selective execution to quickly re-run failed tests after making changes to the code, ensuring that the fixes are correct and efficient.
Pitfall: Manually editing the testng.xml file to include only specific tests can become cumbersome and error-prone in large test suites, leading to maintenance issues and potential test execution errors.

References:

On This Page

Executing Tests Selectively in TestNG