Whatever happened to the test pyramid?

The humble unit test has taken a beating in recent times. What did it ever do wrong?

Mike Cohn’s wonderful book Succeeding with Agile talks about the Test Automation Pyramid, shown in the image above.

This pyramid is beautifully explained by Martin Fowler and I certainly can’t do even half as good a job as he can, so if you’re unfamiliar with the concept I suggest you read that post.

A key point is that unit tests should make up the bulk of your tests because they are cheaper to write and maintain, and because they are faster to run.

Recently though, I keep seeing and hearing from JavaScript folks who believe that unit testing is more trouble than it’s worth. There are smart, thoughtful front-end programmers who truly believe that end-to-end tests are the way to go.

So what happened to the humble unit test? Is the test pyramid still relevant?

Front-end frameworks make unit testing techniques less obvious

I think the main reason for lack of faith in the test pyramid is that front-end unit testing is fraught with danger. It can take a great deal of experience and confidence to be able to write a front-end unit test suite that is cheaper and easier to maintain than a set of UI-level tests.

Unit testing is proving harder and harder to do as we begin using frameworks that mesh together static data with behavior, like how React source files are a mixture of HTML and JavaScript.

Isn’t the front-end just the UI tip of the pyramid?

No.

Modern front-end applications involve user workflow, receiving and sending data through network requests, handling session state and data validation. For applications that involve content creation, there’s plenty of business logic to contend with, too.

All of these lend themselves well to unit testing.

If you’re working in a modern JavaScript codebase, you would do well to structure it with a standard hexagonal architecture. Then unit testing becomes straightforward, and the test automation pyramid begins to make sense again.

Knowing what not to test: the declarative nature of the front-end

Broswer-based development (i.e. “the front-end”) is a lot of HTML and CSS mixed in with some JavaScript. Thankfully the HTML and CSS is static data, and unit tests aren’t generally meant for static data.

Unit tests really shine when you’re testing behavior, not simply repeating static information from the production codebase.

This applies to any framework you might be using, be it React or Vue or Svelte, or simply just plain JavaScript.

If you can create a very clean separation between your static data and your behaviour, then unit testing becomes painless.

Unfortunately, that separation isn’t very easy to do. You can read about my way of doing it in React in this blog post.

HTML and CSS can easily be verified by visual regression and golden master tools

If you’ve got a QA team, they are, without a doubt, interested in verifying that your static HTML and CSS does what it’s meant to.

They can use visual regression tools for this exact purpose.

Jest has a mechanism called snapshot testing which does just this, and it’s quite nice in that it can run on individual components in a component tree. Smaller units reduces the brittleness of these tests.

Unfortunately, these tests are often written in the same test suite as your unit tests, and snapshot testing is promoted as a kind of replacement for unit tests, even though the tests serve a different purpose (they don’t test behavior) and aren’t necessarily a good fit for a unit test framework like Jest.

My take on why the original pyramid isn’t working

The assumption that unit tests are cheaper is no longer true simply because it’s harder to write unit tests.

Unit tests are only cheap if you have enough experience on your team to know how to write unit tests well.

What I observe from years of working with component-based frameworks is that people really struggle with unit tests.

If this is you, then don’t be pressured (by people like me!) into thinking you’ve got to start writing unit tests. Instead, the onus is on us (people like me!) to help simplify unit testing practice. Until then, stick with whatever form of testing brings you most value.