When considering the term “meaningful testing”, it looks like a rather strange abstraction that extends to the entire chain of work with software – from setting a development task to deployment. What does the word “meaningful” mean in this context?
In short, the software creator should focus on testing that benefits the end product and its users. There are several approaches to meaningful testing. Next, I will talk about them, the techniques that are included in these approaches, and how to implement new knowledge in my work.
I think you are all familiar with the testing pyramid. In the blog Martin Fowler there is a good explanation for this concept from Ham Wok. If you agree with what is said there, then I will allow myself to pay attention to one of the proposals of the publication.
“Despite the fact that the concept of a test pyramid has been around for quite some time, teams are still trying to put it into practice.”
So why do teams have problems with the practical implementation of the QA strategy? The answer seems simple: you can’t test everything that makes sense. Isn’t it?
When processing the “testing pyramid” query in Google, you may see something like this.
Obviously, this concept can be considered at different levels of detail. However, over time, you will turn any abstract QA strategies into your own implementation. From my point of view, the QA strategy can be divided into three main blocks, described in the following image.
Abstract Testing Pyramid
I want to explain how, in my opinion, you should implement “meaningful testing”, guided by the categories presented above. At the same time, I will mention practices that together can lead to a transparent and meaningful implementation of the QA strategy (from the bottom up).
Stylistic and software code testing
This category is fundamental to the testing pyramid and, as a result, uncompromising. This is a well-known component of testing, included in programming standards and recognized in terms of terminology. It is on it that our pyramid is built.
You should always try to cover code name and style checks, as well as deal with code defects at the earliest stage of testing (as far as possible). After that, it is the turn of unit testing and other isolated approaches to product verification that can be implemented by the developer. Ideally, this process should be carried out before adding the source code to the GIT repository.
Moreover, this approach should be reasonable and meaningful. There is no reason to try to achieve one hundred percent coverage or strict compliance with the rules if it does not make obvious sense to the author of the code. I often came across projects in which the test blocked the work, and did not provide any significant benefit. I consider such tests as a fundamental condition for moving to the next stages of testing.
Another process I am considering, as part of the very foundation of the pyramid– is code review. Checking the code should become part of the daily routine associated with software testing. You can take a break from your own tasks and help other team members solve their problems.
If you do this work as a routine, daily, then you can get a cleaner and clearer code. Without a serious basic check, you are likely to spend time on corrections that will relate to established rules. Therefore, we can again return to the topic of why we need meaningfulness during the tests.
Moving up the pyramid, we move from isolated tests to more complex ones. Integration testing for me has always been based on the right architecture and well-planned development. If such prerequisites as clean interfaces, contracts, basic data models and APIs are performed correctly, then nothing will prevent you from determining a high level of integration testing with a large number of mockups and simulations.
Now there are a large number of tools used to create their own integration testing stack. I note the evolution of Swagger, Postman, or such a promising cloud product as Azure API Management. Last but not least, I would like to mention well-known libraries and tools like Faker and others.
Security dependency checks should also be performed at this stage, even if it is a service from a VSC provider, a managed external SaaS service, or an open source DependencyTracker tool.
The most difficult of the disciplines always depends on various “buts”. The correct implementation can allow the entire chain to work much faster, reacting in advance to potential incidents.
Load and performance testing (performance testing) are ideal for this category. With their help, you can answer questions that should be asked before the release of the product.
The ease of implementing e2e tests within your project/company strictly depends on how the previous testing levels are implemented. In fact, the top of the pyramid depends on the base and cannot exist without it.
However, at this stage, you can forget about isolated tests. You need to implement the most necessary for the end user. This avoids endless testing at the product support level.
Expect true to be “truthy”
In conclusion, I would like to start by explaining the meaning of the title of this section: “Expect true to be “truthy”. I think everyone involved in the development is doing everything necessary to have a high level of QA (as much as possible). At the same time, we are still focused on best practices, and not on understanding how they are implemented. That is why time is spent on removing obstacles that eventually force us to do something that does not bring any benefit.
We still believe that following common patterns can make the product better. However, this is not the case. We need to stop thinking that a system that works within one implementation will be suitable for another. It is necessary to prove the viability of this implementation and the potential benefits of expanding the current solution to other cases, benefits for the product and the end user.
Originally published in TG channel QA team