What I Learned About Unit Testing Working at Volvo Group
Recently, I resigned from Volvo Group Connected Solutions AB (VGCS). It was a great workplace, they are developing surveillance systems for truck and bus fleets. The company is as large-scaled as it gets. They run many different kinds of tests on multiple levels and also have many different roles for employed testers. Despite all the tests, a handful of bugs creep all the way to production occasionally. A product cannot get tested enough. You can spend too much time on testing it though, and this article will explain why it happens and how you can avoid it.
All teams on VGCS have their own rules. In the team I worked in we aimed for a 100 % coverage of unit tests. Our team alone had quite a few thousands of unit tests for the code we managed. Other teams were more into integration tests and spent less time on unit tests. Today, I will give you my two cents regarding unit testing.
Unit Tests Takes Time, Is It Worth It?
Writing unit tests is a slow process. I would claim experienced developers spend as much time writing unit tests as writing code. Novel developers would spend maybe up to three or four times as much as they write their code, along with that they need support learning it. It's a great opportunity to enlighten them how to write better code that is more easily tested, although I personally prefer doing that during code reviews or weekly tech meetings.
The time it takes to write the tests is not in vain. You will get the time back if you do it wise. Shooting wildly and hoping to catch bugs would not be considered wise. That will do nothing more than emptying your magazine and you will end up with a lot of time spent on nothing.
Most Unit Tests Are Useless
If you aim at 100 % coverage, chances are that most of them are useless. Among all the code I have updated, I have very rarely failed unit tests due to bugs I have introduced in the code. That doesn't mean I haven't introduced bugs, I certainly have.
What I mean is that bugs I introduce almost never is the reason to why the unit tests fail. Rather they fail purely because the updated code isn't compatible with how the tests were written. Let me clarify that with an example.
import React from 'react'
import { shallow } from 'enzyme'
import TodoList, { Todo } from '../TodoList'
it('should pass title to Todo component', () => {
const todos = [
{ id: 1, title: 't1' },
{ id: 2, title: 't2' },
]
const wrapper = shallow(<TodoList todos={todos} />)
const firstTodo = wrapper.find(Todo).at(0)
expect(firstTodo.prop('title')).toEqual('t1')
})
Enzyme test for a todo list. Code available at CodeSandbox or GitHub.
Above is a typical Jest test for React written with Enzyme. What it does is to render a TodoList component and ensures that the correct title is passed to the first Todo component.
Let's say that we are updating the code, maybe letting each Todo component getting its own title from a context or backend. That means we would stop passing in the title to the Todo component. The test would then fail since we don't pass in a title anymore. Since we don't test anything else, we can simply remove the test, which means it was an unnecessary test to write in the first place.
Maybe the same TodoList component is showing a message when we have no todos and we have written a test that checks that the message is visible. A possible improvement could be to let the user create a new component instead of showing the message. In that case we would once again end up with a test that fails due to a change we actually intended to do.
These kinds of edits are very common. If you follow best practices and keep your components (units) small, most of your test failures will be of this kind. With that design, components will have a narrow use case, and as soon as you change the use case, its unit tests will be invalidated. The tests will fail just because you deliberately chose to design the code or UI in another way, not because you have introduced a bug in the old code. In many cases it isn't sufficient to update the tests and you will have to write completely new tests from scratch.
This means that most tests are only valid for as long as you don't touch the code, and as soon as you update it, you either discard or rewrite the tests. You have basically simply tested that the code continues to work as long as you don't touch it. What do you think Einstein would have said about that?
Should You Skip Unit Tests?
Occasionally we borrow developers from other teams at Volvo Group. One time one of those developers came from a team that preferred integration tests over unit tests. I understand his reasoning and I prefer to keep stuff minimal and keeping development at a fast pace, I could to some extent agree with him. But in large scaled projects a lot of people think he is wrong in that, you should really have both unit and integration tests.
When Are Unit Tests Useful?
When I previously accused unit tests to be useless I never meant all of them are. What I was talking about was that it is inefficient to test simple code that doesn't include very much logic or code that will change drastically whenever you make an update to it. That kind of code is common when it comes to UI components or boilerplate code. Not all code looks like that.
Math functions, utils functions, hooks and different kind of pure functions like reducers are all perfect examples of when you should write unit tests. Sometimes they contain complex logic which you absolutely should test. Other functions may have many edge cases to tests. Maybe one of the most common causes to bugs in Javascript is when dealing with mutations. With unit tests that's very fast and easy to test.
These kinds of functions should be tested even if you only plan to write them once and then never updating them. It is ridiculously easy to introduce bugs in logic-heavy code and you can't always test it graphically to see that it works. I would strongly recommend Test Driven Development, TDD, when writing that kind of code. TDD forces you to think of edge cases on beforehand which often can save you time already when writing the code. Without it, you may end up rewriting the code multiple times just because you find new edge-cases with every new solution you come up with.
How to Write Good Unit Tests
I have already touched upon what good unit tests are. When testing logical code, it's important to test edge-cases and test that functions doesn't mutate the code. That can be achieved by invoking functions multiple times or by using the strict equal operator in Javascript.
I won't go into any more details there. Instead, I want to turn back to testing UI components again, that's the kind of unit testing I claimed to be useless in many cases. In details, we will discuss the concepts of shallow and mounting tests with Enzyme, and also interactional unit testing with Testing Library. Testing Library can be used with many libraries, including React.
Unit Testing with Enzyme
If you don't know the difference between shallow and mount component testing, the main difference is that when you test a component shallowly you only test that component's logic without rendering its child components. Mounting will instead render the full DOM tree including all child components which aren't explicitly mocked. A more detailed comparison between Enzyme's shallow and mount can be found here.
Enzyme vs React Testing Library
Regarding the differences between Enzyme and React Testing Library, one can see at npm trends that Testing Library is more used nowadays. Meanwhile Enzyme is slowly dying since it isn't being maintained and lacks unofficial support for React 17.
Shallow Tests
Many people prefer shallow testing. Not all are of the same opinion but personally I would recommend it over mounting components, or maybe I would recommend to use a mix of them. What shallow testing means is that you basically test each component's logic without caring about how it would integrate with other components when you run your code.
Maybe the non-integrating part doesn't sound very inviting, we will come to that. At least shallow testing tests the component itself and if you test every component fully you will get a 100 % test coverage in the end. If you update your components, you are likely to rewrite your complete tests as I talked about earlier in this article.
Mounting Tests
Mounting tests are very much like shallow tests. The positive thing is that mounting also tests the integrations to child components. You can ensure that the components work together.
Since you will test child components when testing component, you will end up with a lot more than 100 % coverage for some deeply nested child components. A button that is used in tenth or hundredth of components will be tested over and over and over again. That's where the drawback hides. At a first glance it doesn't seem to hurt very much. But wait until you update that button component in a way that affects all components that uses it. You will end up with failed unit tests in all those tens or hundreds of components you had written tests for.
Interactional Tests
The third type of tests I wanted to bring up is unit tests that focuses on interactions. The idea behind it is to test the components in their real environment based on what really happens when you interact with the DOM nodes. In that way, we can test React components in their natural environment as they would behave in a real browser. It's one step closer to integration tests even though we still are testing units.
Interactional tests in React Testing Library will behave more like Enzyme's mount tests than the shallow tests, since it will render child components as well. You are of course free to mock whatever component you want to mock, so it's completely possible to tests all components shallowly if you would prefer that, just mock all child components.
Not convinced yet? Let's continue, I'm getting to it. The huge advantage I like about interactional unit testing is that you often will be able to keep your unit tests untouched even if you refactor components, or even multiple components. Just like if you would have tested your code with an integration testing tool like Cypress or Selenium.
Let's look at the Todo example again. This time using React Testing Library.
import React from "react"
import { render } from "@testing-library/react"
import TodoList from "../TodoList"
test("it should pass title to Todo component", () => {
const todos = [
{ id: 1, title: "t1" },
{ id: 2, title: "t2" }
]
const { getAllByRole } = render(<TodoList todos={todos} />)
const todoItems = getAllByRole("listitem")
expect(todoItems[0]).toHaveTextContent("t1")
})
React Testing Library test for a todo list. Code available at CodeSandbox or GitHub.
With the code above, we can update the TodoList component and Todo component in any way we want without having to update the test, as long as we keep using list items for the todo items. If you think it is annoying to depend on list items, we can remove that dependency as well. Testing Library allows looking at data-test-id:s or pure texts as well. Read about supported queries here. Here's some examples of what you can do.
// Checking presence of text using a regex.
getByText(/t1/i)
// Checking for data-test-id with the text.
expect(getByTestId('todo-item-1')).toHaveTextContent('t1')
// Checking for a button with the text "Press me".
expect(getByRole('button')).toHaveTextContent('Press me')
Code available at CodeSandbox or GitHub.
Conclusion
Unit tests and integration tests are both necessary. Keeping unit tests at a 100 % coverage is not a bad thing. But if you don't test your code in an efficient manner, it will cost you tremendous of time. Be smart when designing your unit tests and choose the right tools for it.
Code with a lot of logic and calculations are easy to mess up, and it's hard to think of all edge cases and to always have mutability in mind. Test that kind of code thoroughly and preferably with a TDD approach to force you to consider all edge cases before you start writing the code.
When it comes to testing UI and React components you should really think twice about how to write your tests. Using React Testing Library instead of Enzyme is a great start. Not only because Enzyme is poorly maintained, but rather because Testing Library approaches unit testing in a more efficient way. Testing library focuses on testing DOM elements and elements visible to the user. That kind of interactive unit testing is also possible to write using Enzyme, but Enzyme isn't written for that purpose.
By focusing on DOM elements or the UI visible to the user, rather than the implemented components, you can avoid rewriting your tests over and over again. The tests can then fulfil their purpose of catching bugs whenever the code is being updated. When focusing too much on testing implementation details you will end up rewriting your tests every time you update the code, which makes the unit tests more or less useless.
