How to compare Java objects in tests

If you followed my blog or looked at any of my projects, you probably noticed that I am a big fan of Shazamcrest. I have introduced this library in every project I worked on, whether it’s personal or at work. Recently I came across AssertJ‘s isEqualToComparingFieldByField(...) which tries to achieve something similar. All of these frameworks have certain problems that I would like to show you. I would also like to show you how easy it is to implement a good assertion method yourself.

Let’s consider these two classes:

public class Example {
    public final Point xy;
    public final int z;

    public Example(Point xy, int z) {
        this.xy = xy;
        this.z = z;
    }
}

public class Point {
    public final int x;
    public final int y;

    public Point(int x, int y) {
        this.x = x;
        this.y = y;
    }
}

And we want to assert that the object returned by the method we are testing is equal to our expectation.

JUnit

The simplest implementation with JUnit:

Example actual = methodWeAreTesting();
Example expected = new Example(new Point(1, 2), 3);
assertEquals(expected, actual);

However it will always fail, as neither of the two classes implement equals methods. They also don’t implement toString() method, so the failure message will be pretty useless:

expected:<uk.co.jpawlak.Example@10dba097> but was:<uk.co.jpawlak.Example@1786f9d5>

You could implement these methods but then you end up with test code in production. You could write multiple assertions (one for each field) but it is also bad – not only difficult to maintain but also doesn’t give you a nice overview, seeing a comparison of all fields at once usually allows you to identify the problem much faster.

Finally, because of the method signature, it’s quite easy to confuse expected and actual values.

AssertJ

With AssertJ, we could do:

Example actual = methodWeAreTesting();
Example expected = new Example(new Point(1, 2), 3);
assertThat(actual).isEqualTo(expected);
import static org.assertj.core.api.Assertions.assertThat;

But it has all the same issues as JUnit. Due to the fluent nature of the framework, it’s flexible and makes it easy to create code structures that are syntactically correct but semantically wrong. If you forget .isEqualTo(expected) part, the test will just go green (in case of other frameworks like JUnit or Hamcrest it would be a compilation error instead). You might think that it is unlikely to happen, but you can chain calls this:

assertThat(actual).describeAs("This is a really useful message that will be displayed when the test fails. It is so long that it exceeds the view margin and without scrolling you cannot see what is called afterwards");

And in this case someone forgot .isEqualTo(expected) at the end.

Going back to the original topic, AssertJ has an interesting method:

assertThat(actual).isEqualToComparingFieldByFieldRecursively(expected);

However it has some problems. Firstly it uses toString() method to show you the objects:

java.lang.AssertionError:
Expecting:
<uk.co.jpawlak.Example@6ddf90b0>
to be equal to:
<uk.co.jpawlak.Example@57536d79>
when recursively comparing field by field, but found the following difference(s):

And although the remaining part of the message is more useful:

Path to difference: 
- actual : <7>
- expected: <3>

Path to difference: 
- actual : <6>
- expected: <2>

You still don’t have a nice side by side comparison. And this is because it throws AssertionError, rather than its subclass ComparisonFailure, which IDEs know how to interpret and show you a nice <Click to see difference> button.

Shazamcrest

Example actual = methodWeAreTesting();
Example expected = new Example(new Point(5, 2), 3);
assertThat(actual, sameBeanAs(expected));
import static com.shazam.shazamcrest.MatcherAssert.assertThat;
import static com.shazam.shazamcrest.matcher.Matchers.sameBeanAs;

At first, Shazamcrest looks almost identical to AssertJ. However, in contrary to AssertJ, it throws ComparisonFailure and IntelliJ is able to generate a really useful diff:

However, Shazamcrest also has its own problems. Because it is based on Hamcrest, it needs to conform to Hamcrest’s Matcher interface:

public interface Matcher<T> extends SelfDescribing {
    boolean matches(Object actual);
    void describeMismatch(Object actual, Description mismatchDescription);
}

And as you can see, there is no place here for throwing the exception. What throws the exception is assertThat(...) method and both org.junit.Assert.assertThat and org.hamcrest.MatcherAssert.assertThat throw AssertionError rather than ComparisonFailure. Therefore Shazamcrest comes with its own com.shazam.shazamcrest.MatcherAssert.assertThat which throws ComparisonFailure. But this is only the matter of an import and it is really easy to import the wrong method. And when that happens, although IntelliJ will try to be smart and still generate the diff based solely on the message, the diff will be pretty useless:

More complex assertions

And there is another problem. What if you want to test whether the returned list contains an expected element? You can mix Shazamcrest’s sameBeanAs(...) with other Hamcrest matchers:

List<Example> actual = asList(
        new Example(new Point(1, 2), 3),
        new Example(new Point(4, 5), 6),
        new Example(new Point(7, 8), 9)
);
assertThat(actual, hasItem(sameBeanAs(new Example(new Point(5, 2), 3))));
import static com.shazam.shazamcrest.MatcherAssert.assertThat;
import static com.shazam.shazamcrest.matcher.Matchers.sameBeanAs;
import static java.util.Arrays.asList;
import static org.hamcrest.Matchers.hasItem;

However, in this case it is quite important to see what was in the actual list. This will allow you to notice that there was a difference in one value only in the first element. Unfortunately, the error message will show a diff between the last element of the list and your expectation, which is not good:

Do it yourself

So how can we make it better? We have the following requirements:

  1. throw ComparisonFailure
  2. show actual and expected object without using its toString() method
  3. make a comparison of objects without using its equals(...) method

The first point is really easy. The ComparisonFailure‘s constructor takes 3 Strings – message, expected and actual. The last 2 will be used by the IDE to generate a diff.

Second point requires converting an object to a human-readable, formatted text. Rather than doing it ourselves, we can use existing JSON serializers (e.g. Jackson or Gson). You can even serialise to XML or any other format you want – now you have a full control!

Third point requires comparing the objects somehow. And given that we have already converted them to String, we can simply compare these Strings.

All of this is just a few lines of code:

private static final Gson GSON = new GsonBuilder().setPrettyPrinting().create();

public static <T> void assertCollectionContains(Collection<T> actual, T expected) {
    String expectedJson = GSON.toJson(expected);

    if (actual.stream().noneMatch(element -> GSON.toJson(element).equals(expectedJson))) {
        String actualJson = GSON.toJson(actual);
        String message = format("Expected to find%n%s%nin collection%n%s", expectedJson, actualJson);
        throw new ComparisonFailure(message, expectedJson, actualJson);
    }
}
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import org.junit.ComparisonFailure;
import java.util.Collection;
import static java.lang.String.format;

And it gives us the following output:

We need to still show the actual collection and the expected element in the message so that it shows when we run the test outside of IntelliJ, in some build automation system.

You could say that the diff is not exactly accurate. Even if this assertion passed, there would be still a difference, because we are comparing object against the list, so there are always going to be [ and ] symbols in the actual. Although true, this is a minor issue compared to the problem we had before. Also, you can customise both sides of the diff and for example add an extra sentence at the top explaining what these are – yes, the sentence is always going to be different on both sides, but the rest of the diff will be still useful. The point here is not to make a perfect diff, but to show all the information that are needed to quickly and easily understand why the test failed.

As you can see, doing it yourself solves the problem that none of the above testing frameworks solve, is really easy to implement and finally gives you a full control so that you can tailor the assertion to your needs and/or preferences.

About Jaroslaw Pawlak

I have done MSci in Computer Science at King’s College London and currently work as Software Engineer specialising in Java. I spend most of my time in front of computer improving my programming (and other) skills or just relaxing with a good game. I also train some sports but at the moment I am not a member of any club. I love cycling and volleyball, but I have also played a lot of football, tennis and trained martial arts.

Posted on June 15, 2016, in Test Driven Development. Bookmark the permalink. Leave a comment.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: