Along with awareness of the need to write automated tests, programmers
are increasingly bogged down in the swamp of testers terminology (they
think it's unified >: - E).
I decided to put some more historical context to the Pyramid understanding.
All testers who started just as testers, trying to automate something, intuitively want in their "auto-tests" to repeat the same steps that they did with their hands. But at some point most of them face the challenge of supporting the UI auto-tests. And after the painful shots of destiny they finally have a "sacred pyramid of autotesting", which orders them:
On the other hand, when the "pyramid" is shown to programmers, trying to justify why they MUST write basillions of tests, this raises the perturbation of their logically consistent universe. Here is the well-known Brian Okken @ brianokken , who works as a programmer without the testers and is responsible for the quality of his code alone, in his last podcast (https://testandcode.com/31 ) Brian faced difficulties understanding this approach. I will try to give a few of his and my own critical thoughts, why the "pyramid" is a bad model:
I decided to put some more historical context to the Pyramid understanding.
Just nice pyramids from TUTORIAL 1 - OpenGL Fundamentals http://www.naturewizard.at/tutorial0104.html
All testers who started just as testers, trying to automate something, intuitively want in their "auto-tests" to repeat the same steps that they did with their hands. But at some point most of them face the challenge of supporting the UI auto-tests. And after the painful shots of destiny they finally have a "sacred pyramid of autotesting", which orders them:
- Take care of user-oriented exploratory testing;
- write less UI-tests, because you're gonna rewrite them all the time;
- learn to do a back-end testing without using the UI;
- programmers have to test their code, and since they are programmers - they should test it with unit tests, which are also code. And there should be a lot of these unit tests! Much more than your UI-autotests.
- And from here, the most important conclusion is: Do not worry to test very deeply, let the programmers go deep down and cover all the depth with their basillions of unit tests!
A real pyramid of auto-testing
On the other hand, when the "pyramid" is shown to programmers, trying to justify why they MUST write basillions of tests, this raises the perturbation of their logically consistent universe. Here is the well-known Brian Okken @ brianokken , who works as a programmer without the testers and is responsible for the quality of his code alone, in his last podcast (https://testandcode.com/31 ) Brian faced difficulties understanding this approach. I will try to give a few of his and my own critical thoughts, why the "pyramid" is a bad model:
- What is the "unit" that should be tested by "unit tests"? What kind of granulation on the unit is sufficient? Why not deeper? Or why not more general?
- The same with "integration tests". Should an every layer of abstraction be covered? Which level Is the highest abstraction to cover? the lowest?
- Looking at this model, it's easy to think that everything is sticking to the unit tests, and if they do not exist - it's kaput to everything. But practice shows that this is very much often not the case. There are so many products and the whole eco-systems (I have seen many successful PHP projects), where they do not write unit tests - and it's considered normal. One may say that this is a lower quality code. But is the "highest grade" required everywhere? If the overwhelming majority of users are satisfied with the quality of the product (do not forget that the concept of "quality" may include the cost of development), then maybe more expensive development does not make sense?
- Is "the number of tests" such an false metric? The quality of the test design is very context-dependent. 10 bad unit tests can reach 100% code coverage (expressions, conditions, solutions). But they will not be valid in terms of the semantics of the use of this module. The "Holy Fathers" of programming has long refused to measure the code by "number of rows". The "Holy Fathers" of testing almost convinced everyone that the number of executed manual tests also means nothing about functionality coverage. So why is the most commonly used automated quality control model is based on the banal "number of tests"? This is precisely primitive, and it seems not very reliable. Emperically confirmed.
- Imagine that there is one widely-parameterized automated test that runs with a large number of different data. Do you care that it is one? No. Are we interested in knowing the number of datasets? I do not think so. What we are really interested in is the variability of these data, which functional areas they cover, and how they relate to the real production data. And this can not be drawn on the pyramid in its present form.
Немає коментарів:
Дописати коментар