Screenshots of test failures

On some occasions a functional GUI test will run perfectly from within the IDE but will break when executed in a batch with other tests (such as when you are using Ant). This is because functional GUI tests are vulnerable to certain environment-related events, and AssertJ Swing is no exception.

It occasionally may happen that anti-virus software runs a scheduled scan while a GUI is under test. If the anti-virus software pops up a dialog in front of the GUI, the AssertJ Swing robot will not be able to access the GUI and will time out eventually, making the test fail. In this case, the failure is not related to a programming error; it is just a case of bad timing.

AssertJ Swing is capable of taking a screenshot of the desktop when a GUI test fails, regardless of the test framework you use (TestNG or JUnit). You then can use this screenshot to analyse the cause of a failed test and discover whether it is programmatic or environmental.

There are many ways to take a screenshot of the desktop when a GUI test fails. Regardless of how AssertJ Swing takes a screenshot, it needs to know which tests should be considered GUI tests. In order to do so, we only need to add the annotation @GUITest. This annotation can be placed at class or method level, and it is inherited by subclasses of annotated classes.

Once our GUI tests have the @GUITest annotation, we can take a screenshot of the desktop when a GUI test fails. See the following sections for the how-to.

How to take screenshots manually

Taking a screenshot of the desktop is quite simple. The class ScreenshotTaker provides two methods:

• takeDesktopScreenshot() takes a screenshot of the desktop and returns it as a BufferedImage.
• saveDesktopAsPng(String) takes a screenshot of the desktop and saves it as a PNG image using the file path passed as argument.

Example

This example takes a screenshot of the desktop when a test fails and saves the image as myTest.png:

// import static org.assertj.core.util.Files.currentFolder;
// import static java.io.File.separator

ScreenshotTaker screenshotTaker = new ScreenshotTaker();

String currentFolderPath = currentFolder().getCanonicalPath();

File imageFolder = new File(currentFolderPath + separator + "failed-tests");
String imageFolderPath = imageFolder.getCanonicalPath() + separator;

try {
  // perform your test
} catch (Exception e) {
  // test failed
  screenshotTaker.saveDesktopAsPng(imageFolderPath + "myTest.png");
  throw e;
}

How to take screenshots of test failures with JUnit

AssertJ Swing can take a screenshot of the desktop when a JUnit GUI test fails, either when running tests using Ant or inside an IDE (e.g. Eclipse). To take screenshots of failed GUI tests you have to add the @GUITest annotation and make sure that you've included the assertj-swing-junit artifact.

Running with the JUnit Runner

For running GUI tests in an IDE, AssertJ Swing provides a custom JUnit runner, GUITestRunner, which takes screenshots of failed GUI tests. To use it, just annotate your test class with @RunWith(GUITestRunner.class). Screenshots of failed tests will be saved in the directory failed-gui-tests (relative to the directory where tests are executed).

Running with Ant

In order to take screenshots of failed GUI tests with Ant please follow these steps

• Add a definition of the Ant task assertjreport
• Use the formatter ScreenshotOnFailureResultFormatter inside the junit Ant task
• Use the Ant task assertjreport instead of junitreport, and specify in its classpath where the assertj-swing-junit jar file is.

Example

<target name="test" depends="compile">
  <taskdef resource="assertjjunittasks" classpathref="lib.classpath" />

  <junit forkmode="perBatch" printsummary="yes" haltonfailure="no" haltonerror="no">
    <classpath refid="lib.classpath" />
    <classpath location="${target.test.classes.dir}" />
    <classpath location="${target.classes.dir}" />
    <formatter classname="org.fest.swing.junit.ant.ScreenshotOnFailureResultFormatter" extension=".xml" />
    <batchtest fork="yes" todir="${target.junit.results.dir}">
      <fileset dir="${target.test.classes.dir}" includes="**/*Test*.class" />
    </batchtest>
  </junit>

  <festreport todir="${target.junit.report.dir}">
    <classpath refid="lib.classpath" />
    <fileset dir="${target.junit.results.dir}">
      <include name="TEST-*.xml" />
    </fileset>
    <report format="frames" todir="${target.junit.report.dir}/html" />
  </festreport>
</target>

assertjreport works exactly as junitreport but additionally produces a link as seen in this screenshot:

How to take screenshots of test failures with TestNG

AssertJ Swing can take a screenshot of the desktop when a TestNG GUI test fails. Because of TestNG's excellent support for extensibility, it is quite easy to setup AssertJ Swing with TestNG. Just make sure that you've included the assertj-swing-testng artifact.

The provided listener, ScreenshotOnFailureListener, is capable of taking a screenshot of the desktop when a test fails and saves it in TestNG's default output directory. Due to TestNG's flexible architecture, this listener can be called either from Ant, TestNG plug-in for Eclipse, command line, etc.

To take screenshots of failed GUI tests, regardless of how they are executed, just add the @GUITest annotation.

Running with Ant

This example shows how to register the ScreenshotOnFailureListener with TestNG's ant task.

<testng listeners="org.assertj.swing.testng.ScreenshotOnFailureListener"
        outputDir="${target.test.results.dir}" haltOnFailure="true"
        verbose="2">
  <classfileset dir="${target.test.classes.dir}" includes="**/*Test.class" />
  <classpath location="${target.test.classes.dir}" />
  <classpath location="${target.classes.dir}" />
  <classpath refid="test.classpath" />
</testng>

A screenshot of a failed TestNG GUI test