10 Signs Testing is Not Enough (Part 3)

What Are the Next Three Signs That Strike Us Most That There Is Something Not “Quite Enough"?

Sign 3: Bug Counts Grow

Well, it may seem strange and yet, there are projects where after each build of the product the number of bugs encountered continues to grow. If that happens to you, do not just continue the testing.

There is a high probability that the number of bugs you are going to find in the next and the next build will continue to grow. As soon as you notice this tendency, you must sit and analyze why this is happening, outline the reasons and find solutions. Otherwise, not taking the preventive measure, your software product is highly possible to go in a Death Spiral.

“We cannot control what we cannot measure”

For the QA Lead/ Manager the controlling phase means that you have a view on your project and you could quickly respond to any deviations from the initially set goals. So, to have a clear view, it is good to also bear in mind and use the metrics. Good software testing metrics are really important as they encapsulate metrics that measure the effectiveness of the project and help gauge its progress, quality, and health of the testing effort.

Moreover, the carefully chosen and defined metrics can aid in improving the testing and testing process within the organization and also help track its status from time to time. It is quite common practice today, for test vendors, to share with the client metrics that exemplify productivity, efficiency and effectiveness in conjunction with the business benefits of the software projects. Never forget that without metrics it will be almost impossible to quantify, explain, or even demonstrate software quality. They also easily provide a quick insight of the status of the testing effort, thus influencing smart decisions (if taken seriously and into account).

Never forget that without metrics it will be almost impossible to quantify, explain, or even demonstrate software quality. They also easily provide a quick insight of the status of the testing effort, thus influencing smart decisions (if taken seriously and into account).

The basic metrics are quite popular and easy to be applied and they are really great for analyzing the trends in your project. So, there are several layer of metrics and we can offer you to use these ones:

Project Level Metrics – Requirements and Requirement Coverage and Defect Open and Close Rates. These two metrics are often used to measure the state of development of the project – in the beginning, in the middle or ending and what is the state of the fulfillment of the requirements of the business.

Here, you can see the simple formula to measure the requirements coverage and the defect open and close rates:

Department Level Metrics– the Meantime to Detect (MTTD) and Meantime to Repair (MTTR). MTTD measures how long it takes the QA professionals to find a problem, while the MTTR demonstrates the amount of time needed to effectively address it. As you can see, even the way and the velocity with which the QAs work can be measured and then, if there are any problems met, they should be carefully addressed to the people who are working slowly and there is no reason for this. That metric can also show that your product is quite complicated and a lot of time takes to discover a bug. Or even, maybe, that the testing done (if the metric is applied at a late stage in the software testing project) is quite enough and sufficient.

So, here you can see how to apply those metrics:

Company Level Metrics – Number of Issues reported by Customers – this is the metric by which you can measure the “done” you have succeeded in testing.

The metrics above are useful for the assessment of the stage of the software testing you are doing. You should apply them at every stage of the project as you have to make sure that everything is going as planned and there are no deviations from your expected results. They will show you clearly and in a statistical manner if there is something wrong with the project. And thus, you can take the necessary actions.

Sign 4: Not Investing Enough

Just look at your competitors and the investment in testing they are doing. If your investment is less, then you probably are out of date with the latest tendencies and quite soon enough it will turn out that your testing and the way you are doing it is not enough and will not satisfy the needs of the client.

Sign 5: No Clear Criteria

This is quite easy to imagine. If you have no criteria for the coverage (test or code), the execution of the test cases – their priority and set deadlines, the acceptable level of defects and the plan for fixing the defects found, it will be quite difficult for you to assure proper testing. The lack of everything above brings an almost disastrous situation to you and your team and even if one element is missing, the process will become more difficult. So, before starting the test process, why don’t you and your team go through the criteria you have on the table, see what else you need and after you are provided with all possible criteria on behalf of the business, start testing. Otherwise, if you assume you can manage with only partial criteria, then you will find yourself in a middle of an unclear testing phase with unclear requirements regarding the testing and for sure, delay or a serious pitfall.

Now, you are half way or more through our topic. Stay tuned, so you don’t miss the next remaining 5 signs helping you understand how much testing is enough.

Quality House Team