Your UI Needs More Than Unit Tests

Testing code is a polarizing topic in the industry, and it always has been. Some people say that if a line of code is not tested, it should be deleted. This may sound dogmatic, but you will find the topic of testing popping up in almost every area of software development. For example, sometimes what is legacy code is defined by the absence of tests for the code in question. The list of examples goes on, but the claims are not universally accepted. Sceptics will argue that they are too radical and aiming for full test coverage is simply not worth the hustle.

It is clear that Test-Driven Development (TDD) is an important part of software engineering. That said, I tend to view that kind of simplistic reasoning with suspicion. The idea that every line of code needs to be tested, or it won’t be maintainable, does not sound practical. It is an easy solution to a nuanced problem.

Simple solutions have a special kind of charm. They are easy to explain and enforce, but they are rarely optimal. In this article, I will explain my testing strategy for the applications I build. I will focus on the front-end perspective primarily because testing user interfaces is challenging. I should warn you that it is fairly involved as it requires learning about various techniques of testing UI, but before we get there, we need to take a step back and answer a more fundamental question.

Should you test your code?

First, let’s acknowledge that the case for testing is overwhelming. It is widely accepted that automated testing improves project maintainability, decreases the number of bugs in production, and prevents regressions. Tests can serve as reliable documentation of business logic, and indirectly improve the overall design of code being tested. My experience supports these claims, but for the sake of brevity, I will skip the details and avoid listing every reason to cover your code with automated tests. It is a broad topic that could easily become a separate series of articles. If you are not convinced yet, there is no shortage of excellent material online explaining the benefits of testing code, and I am sure you will find strong resources in minutes.

Full disclosure: I have always had an extremely positive attitude toward automated tests, especially unit tests. For some reason, the idea of automated testing resonated strongly with me. I genuinely enjoy writing my code using a strict Test-Driven Development method, and I aim for a comprehensive test suite in all my personal projects as well. Because of this bias, in most engineering teams I joined, I advocated for higher test coverage. Something that was obvious to me was that it should include covering UI code with unit tests. In hindsight, it was a mistake.

So despite the undoubtedly beneficial effects of automated software testing and my personal sentiment, today my answer to the question “Should you test all your code?” is: it depends. While I still consider myself a huge fan of TDD and automated testing in general, these days I’m also more aware of the trade-offs we need to consider when making that choice.

Why unit testing should not be your primary way of testing UI

When I ask candidates during interviews about their experience with automated tests, the most common answer I get is: unit testing. Other types of testing are also mentioned from time to time, integration tests, E2E tests, visual regression tests, synthetic monitoring/smoke tests, but unit tests seem to be the most widely adopted.

The main reason why unit tests are so popular is quite clear: they are fast. By definition, a code unit (function, component, class, etc.) needs to be tested in isolation, which implies that dependencies of the unit under test often have to be mocked. Mocking a dependency replaces a real dependency with a fake one, usually a simplified version prepared specifically for testing. This technique ensures that a correctly written unit test is independent of potentially heavy runtimes. In the case of JavaScript web applications, that costly runtime is the browser environment, so to test it efficiently we need to mock it. Luckily, mature browser-like environments such as jsdom and happy-dom already exist and we can simply use them. That allows a correctly written unit test to run in anything from a couple to a couple hundred milliseconds, and it is normal to run thousands of unit tests in every test cycle.

The problem with this approach is the mocking itself. Every mock is a simplification of something larger. It can be inaccurate or incorrect, and it may miss an obscure edge case, so the more you mock, the less confidence you have in the test you write. Tests with mocks verify how your code behaves in a particular mocked environment, not in the real application. Not to mention that someone needs to write the mock in the first place and keep it up to date as the mocked dependency changes. Browser mocks aside, frameworks like React can increase the burden because they often require additional mocking to simulate the way components render.

In general, testing UI tends to be a frustrating experience. Part of it can be attributed to the aforementioned browser and framework mocks that developers have to use in order to test even the simplest component. Things get only worse when you consider “smarter” components. These are components that import parts of the application that require additional mocking. For example, when a UI component imports a function that fetches data from a REST API, that function will need mocking as well. Correctly splitting the UI from the business logic can help alleviate the issue, but it can never be fully avoided.

As a result, in many projects UI is left untested or tested sporadically. In others, an alternative is proposed, a method to fully replace unit testing of UI components. Most often, that alternative is end-to-end tests.

E2E tests are not a silver bullet

End-to-end testing is a type of automated testing that takes a more user-centric perspective. To run it, we need to launch a browser and have a way to simulate interactions such as clicking, scrolling, and navigating. While it may sound like a lot, E2E is a well-established technique for testing user interfaces. The tooling around it, with frameworks like cypress and playwright, is stable and mature.

Compared to unit tests, E2E tests can look like a step up that could fully replace them. While mocking is still required, at least we can benefit from a real browser environment, which gives us much better confidence in the written tests. This is especially true for functionality that is practically impossible to test well with unit tests. One classic example is drag and drop, where the amount of mocking required for unit tests can get so high that it defies the purpose. E2E is a much better tool for testing this.

However, the main issue with E2E tests is speed. Browsers are complicated pieces of software, and running them does not come cheap. While unit tests run in milliseconds, a single E2E test executes in seconds. Depending on its complexity, it can take up to half a minute, and that can still be considered OK for an E2E test. Of course, those times can vary depending on the machine they are executed on, and in CI they can be even longer. No matter how well provisioned your environment is, we are often looking at around 100x longer execution times compared to unit tests.

The fact that E2E tests take seconds to execute means we cannot fully replace unit tests with them. Let us break down why. We can assume that a single E2E test gives better confidence than a single unit test, which means one E2E test replaces multiple unit tests. But how many? If we want to replace unit tests with E2E tests without increasing CI run times, one E2E test would need to replace roughly 100 unit tests. Practice shows that this is not the case. It is highly dependent on the project and the code we are testing, but whenever logic branches out, we need multiple tests to cover those branches. For example, testing a web form with complex business logic can easily require a dozen E2E tests. Our simple math shows that this would be equivalent to running a few thousand unit tests. Needless to say, you need far fewer unit tests to fully tests to fully cover a form even if it has very complicated conditional rendering logic or validation rules.

Another issue is that E2E tests validate whole functionality at once, which requires putting many independent code units together and running them in a browser. As a result, they are not the right choice for documenting the intended behavior of smaller units such as functions and classes. They do an excellent job at documenting whole user flows, for example, whether a user can sign in to an application with invalid credentials. But documenting that signIn(userId: string) should throw a 401 Unauthorized error when supplied credentials are incorrect is a job for a unit test. So the idea that E2E tests can be a 1:1 replacement for unit tests is fundamentally flawed. They each come with their own trade-offs and should be treated as complementary tools rather than alternatives.

All that said, I understand where the idea of replacing unit tests with E2E tests comes from. Both techniques share one trait: they are good at assessing code behavior, whether a function returned the expected result or whether a callback was called when specific conditions were met. They answer the question: “When the user does X, does Y happen?” They are meant to validate intent. However, there is another kind of testing that answers a different question and, if used correctly, can be very efficient for testing user interfaces.

Validating behavior vs detecting changes in structure

The other kind of tests we can employ are regression tests. These tests, unlike unit tests and E2E tests, do not describe the expected behavior. Instead, they preserve some state and detect if that state has changed. It starts with creating a snapshot of some testable structure - it can be a serialization of some large object or a screenshot, but in principle it can be anything. The first run of a regression test is always a “pass” and the goal is to create a saved state. Only the subsequent runs of the test may fail. The test will fail if the tested state is different than the saved state. If the test fails, however, it doesn’t necessarily mean that the new state is not correct. After all, regression tests do not encode any expectation. A failed regression test is a signal for the developer that the state changed and it requires their attention, and it is for them to decide if the new state is correct or not. If it’s incorrect, then it should be fixed. If it is correct, then it should be accepted as a new saved state.

Regression testing is not as good at expressing intent as unit tests and E2E tests, but they allow for catching unintentional changes in complex structures. It’s worth noting that each time a regression test fails, we rely on the developer to decide if the change is intended. If we want it to be reliable, the snapshot has to have some properties that will help the developer to decide quickly about the nature of the failure. Let’s keep that in mind as it’s going to be important in the later part of this section. For now, let’s focus on two widely adopted types of regression testing: snapshot testing and visual regression testing. It’s interesting to see how they take the same idea of regression testing and achieve the completely different level of usability.

Let’s take snapshot testing first. It’s usually implemented by frameworks for unit testing like vitest and jest. The snapshot they take is some string representation of the UI that is saved on the filesystem and committed alongside the test files. That representation is defined by the framework we use to render our test UI, but with React the most common choice these days is testing-library, which produces a human-readable string representation that resembles HTML code.

Even though the snapshot is human-readable, it doesn’t automatically mean it’s easy to comprehend because it’s still very far from what a user sees on the screen. For simple components, it may be sufficient, but it’s not enough for more involved ones. When such a test fails, the developer may have a hard time assessing if the new state should be accepted or not. The less readable the representation of the UI is, the bigger the chance of accepting an unexpected change by mistake. Unfortunately, that is what happens often with snapshot tests and for this reason I don’t recommend using them. Even though they are relatively fast and easy to introduce since they come with the popular unit-testing frameworks, they are not worth it. They can create a false sense of security which in my opinion is worse than not having any tests at all.

While I don’t recommend snapshot testing, it doesn’t mean regression testing as a whole does not have its use for the front-end. Quite the contrary. The other type, visual regression testing, has proven to be far more useful in testing user interfaces. The key seems to be the right snapshot. Instead of a string representation of an interface, it produces a screenshot of the actual render of the component under test. The benefits of such an approach are obvious - it’s much easier for the developer to decide if the change is a regression by comparing two screenshots. With snapshot testing we would be limited to a file diff, while visual regression testing tools usually offer more sophisticated ways to compare changes like side-by-side view, diff strobing, etc.

Under the hood, visual regression testing needs a browser to render the UI. So similar to how E2E tests are slower than unit tests, visual regression tests are slower than snapshot tests. Also, similar to E2E tests, they can’t be a replacement for unit tests, especially because regression testing isn’t well suited for testing the behavior. That said, it’s not a flaw. User interfaces are inherently a mix of presentation and behavior. Therefore, for enforcing that our UI behaves as we expect, unit and E2E tests are the natural candidates. For ensuring our interface stays the way we want it to look, we should use visual regression testing. Using all three testing techniques forms the foundation of my front-end testing strategy.

Complete strategy for testing front-end applications

The core principle of my recommendation and strategy for testing user interfaces is already highlighted in the previous section:

• Unit tests and E2E tests should be used to test UI behavior
• Visual regression tests should be used for testing UI look

That means that in every project I work with, I want to have all three capabilities ready to be used, even if there are no actual tests written yet. I want to set up the CI pipeline to have vitest running unit tests, which is already a common practice in the industry. I will work to set up playwright running E2E tests, even if there are none yet. Most of the time I will want to have storybook in good shape so it can be used with one of the visual regression testing tools like Chromatic. That said, there are other visual regression testing setups that work well for different use cases. As I mentioned earlier, the use of snapshot testing, even though it usually comes for free with vitest, should be discouraged.

All three together make up a solid testing setup for any modern front-end project. Of course, it should not stop there, as there are more testing methods you should invest in as the project grows, but the setup above will get you a long way. That said, there is one very important aspect that we still need to cover: the rule that complements the front-end testing strategy.

• Business logic should be separated from the UI.

To make myself clear, by saying that, I am not pushing back against modern libraries like React. The true power of components lies in the fact that they are a mix of presentation and behavior, which, some may argue, is a direct violation of separating them by classic models like MVC. I take the position that this is not a flaw. It is one of the reasons why component-based frameworks are so popular these days. The key distinction to keep in mind is that not all behavior is automatically business logic. The definitions may vary, but business logic is behavior that exists because of the business domain of our application. It’s invariant to the implementation. And it is, generally speaking, a good idea to separate it from the presentation not only to make it easier to test.

That effort will eventually result in a growing number of visual regression tests covering larger portions of the UI. A similar effect will also be seen for the test coverage of the business logic. After all, when logic is properly separated from presentation, it is much easier to unit test than when it is embedded within a UI component. And so, when the discipline of extracting behavior from UI is combined with the right tooling for testing it, you gain much-needed confidence in your front-end changes, even if coverage is not 100%.