November 04, 2021 By adi
A few days ago I was talking to a colleague from my team about the fact that in one of the integration tests I had:
- a test name that contained implementation details
- assertion on an internal property of a model
The use case that I had to implement in the story was: As a product content operator I want to approve a product request so that I can create a new product. Besides other constraints, I had to test the following one: if the product request was not approved before, we can't create a new product from it, and my test looked like this:
Above all, a few technical remarks:
- in the command handler, we check if the $productRequest was approved before, and if not we make an earlier return
- the product id is generated after the product request becomes a new product and it is different by the product request id
My colleague said that the tests should not contain implementation details and the test itself should be a business use case, and we do not have in our business logic the fact that someone will want to create a product from a rejected request. If you try to look at things from an application user's eyes, you will say that he has right. But what are the risks if you will test your application just from the user's point of view? I think is obvious to everyone what are the risks... To be the devil's advocate, I could say that, when we write tests, by default, we should try to name our tests from the business perspective. But there can be cases, like mine, when we want to test corner situations and we have to name our tests by saying exactly what they are testing.
Another way to write that test without checking if the $productRequest->productId() is null, is using a mock for ProductRepository which expects that the save method will be never called, but again we do not get rid of the implementation details.
This misunderstanding about what point of view you should have when testing your application reminded me about the classification of tests made by Chris Hartjes in his book The Grumpy Programmer's Guide To Testing PHP Applications. Chris classified tests by customer and developer, as follows:
Customer tests are any test that proves your application is behaving in a way that the customer expects. This would cover what we traditionally use for functional tests. Inspired by my case, a simple example is a functional test that verifies if a product is visible on site after a product content operator created a new product.
Developer tests are any test that proves a specific set of code is behaving the way the developer expects it to. This would cover what we traditionally use for unit tests or integration tests. The goal of the developer tests is to guide anyone working with the application at a coding level to create testing scenarios that ensure the code, at the lowest level, is functioning.
Let's go back to the origins and try to remember the definitions of each type of test:
- a unit test verifies the behavior of a small piece of code (method or object) and often uses doubles in the place of dependencies for the system under test
- an integration test verifies the behavior of a small number of objects that are part of a use case, using them together without the use of doubles of dependencies for the system under test
- a functional test verifies a small subset of the behavior of the application as a whole by using the application itself
The main goal of the customer tests is testing the main functions of an application. It involves a basic usability testing of the system and checks whether a user can freely make some actions without any difficulties. This means that we mostly test happy paths and a few corner cases that a user can meet. The great benefit of these kinds of tests is they ensure that the result meets the business and the user’s requirements. Of course, that comes with a cost: they are more expensive to make and maintain, slowest and you don't get quick feedback from them.
A developer test can verify different behavioral aspects of the system under testing, but mainly it verifies that the system under testing produces the correct results. From the developer’s perspective, the purpose of these tests is to create a strong codebase at a cheap cost. Another important purpose is to provide proper documentation for the customer tests or new developers.
If you are familiar with these testing techniques, we can say that customer tests use the black box testing technique and developer tests use the white box testing technique.
In conclusion, personally prefer more this classification because is simple, clear, and easy to understand. Experience has taught me that the more abstract and slightly different options we have, the greater the chance of misunderstanding and poorly implementing them. But if the whole community uses a different classification for tests, in order to have a common language, we have to master that one and then we can come up with our own philosophy.
References
