Reports & failure evidence

as a manual tester

Your bug reports attach screenshots and steps — evidence is what makes them actionable.

→

in automation

Automated failures need the same evidence: a report humans can read, and a screenshot from the moment of death.

A failing test at 3 a.m. in CI is only useful if the morning human can diagnose it without re-running. Two upgrades deliver that: readable reports and automatic screenshots on failure.

What you already have: Surefire reports

Every mvn test writes reports to target/surefire-reports/ — a .txt per class and XML that CI systems parse into dashboards (that's how GitHub Actions will show you red/green per test):

report files

target/surefire-reports/
├─ LoginTest.txt
├─ TEST-LoginTest.xml
└─ InvalidLoginTest.txt

# inside LoginTest.txt after a failure:
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0
validLogin  Time elapsed: 3.41 s  <<< FAILURE!
org.opentest4j.AssertionFailedError:
    expected: <Products> but was: <Swag Labs>

Screenshots on failure — the pro move

Selenium can photograph the browser at any moment. JUnit can detect that a test failed. Glue them in BaseTest and every failure leaves evidence:

BaseTest.java — teardown with evidence

import org.junit.jupiter.api.extension.*;
import org.openqa.selenium.*;
import java.nio.file.*;

public class BaseTest {
    protected WebDriver driver;

    @RegisterExtension
    TestWatcher watcher = new TestWatcher() {
        @Override
        public void testFailed(ExtensionContext ctx, Throwable cause) {
            try {
                byte[] png = ((TakesScreenshot) driver)
                                 .getScreenshotAs(OutputType.BYTES);
                Path path = Paths.get("screenshots",
                                       ctx.getDisplayName() + ".png");
                Files.createDirectories(path.getParent());
                Files.write(path, png);
                System.out.println("Saved failure evidence: " + path);
            } catch (Exception ignored) {}
        }
    };

    /* @BeforeEach / @AfterEach as before */
}

predict: what does this print? think it through, then reveal

output

LoginTest > Valid user lands on Products page FAILED
Saved failure evidence: screenshots\Valid user lands on Products page.png
[INFO] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0

A saved failure screenshot — the browser exactly as it looked when the assertion failed

Now every red test comes with a picture of exactly what the browser showed — usually enough to diagnose in seconds ("the error banner! the app was down") instead of a re-run.

Levelling up later: teams often add Allure for rich HTML dashboards with history, steps, and embedded screenshots. It's a bolt-on to what you built here — same XML, prettier face. Worth adopting at capstone-polish time, not before.

⚠ when it breaks — the classics

Tests failed but screenshots/ is empty

(folder missing or empty after a red run)

Fix: Three checks: the watcher must be a field annotated @RegisterExtension (not static, not a local variable); evidence only appears on failures — a passing run produces nothing; and take the screenshot before anything quits the driver.

My screenshot file name is garbled

java.nio.file.InvalidPathException: Illegal char <:> at index…

Fix: Display names can contain characters Windows paths forbid. Sanitize before saving: ctx.getDisplayName().replaceAll("[^a-zA-Z0-9 -]", "_").

⚡ exercise · produce a diagnosable failure

Add the TestWatcher to your BaseTest.
Break the valid-login test (wrong expected title), run mvn test.
Open the PNG in screenshots/ and the class .txt in target/surefire-reports/. Diagnose the failure using only those two artifacts, as the morning human would. Fix the test.

references

key takeaways

Surefire already writes per-class reports and CI-readable XML on every run.
A TestWatcher in BaseTest saves a screenshot at the moment of every failure.
Evidence turns 'it failed, re-run it' into 'diagnosed in seconds'.