A Tester’s Litmus Test

Testers nowadays are generally acknowledged as valuable members of an Agile or DevOps team. I often wonder if this newly acquired status stems from positive experiences with testers or from the general recognition that testing is important. To be honest, I fear it stems from the latter.

My personal experiences with testers is very diverse, both in Agile development and in more traditional approaches. I’ve worked with people I hold in high esteem when it comes to testing knowledge, but I’ve seen at least as many testers, some of them even with years of experience, that I just can’t acknowledge as a professional tester, as someone practicing, or even knowing, the things it takes to earn that title.

In my opinion, there are a couple of simple questions that set aside the ‘good’ from the ‘bad’, but it starts with attitude, a good tester knows you can’t test everything, so he/she asks what the risks are if we would refrain from testing … after all, testing takes time and effort, so let’s do as little as possible of it! The age old testing adagio ‘no risk, no test’ still holds true.

And then the real work starts because acknowledging and identifying risks constitutes merely the basis for testing. Where the tester really shows its worth is in translating those risks into a proper test design. Because creating security test cases for security testing requires a different approach than creating e.g. performance, functional or usability test cases. And areas with high risk requires more thorough testing than areas with low risk. The difference is not just ‘more test cases’ (although that usually is the case also), but other, additional test cases resulting in a test design with higher coverage.

Secondly, a sound test approach balances manual and automated testing. Not everything can be tested automatically, but we cannot go completely without it either.

Which brings me to the tester’s litmus test:

Question 1: Describe in a few words how you would start putting together a test approach? To pass this first question, a good tester mentions keywords or key phrases like risk, coverage, manual/automated and several test types like regression, functional, security, performance, etc.

Question 2: Which coverage types do you know? A good tester would mention some of the following:  experience based coverage, data coverage, path coverage, decision coverage.

Question 3: How is high risk reflected in the test design for any of the coverage types you just mentioned? Keywords or phrases a good tester would reply are: for data coverage: pairwise testing (or, if he/she would really like to impress you ‘orthogonal arrays’!), for path coverage: variations in test depth for the process cycle test (in true test jargon, PCT test depth 1, 2, 3, … etc.), for decision coverage: DC (decision coverage), MCDC (modified condition decision coverage), MCC (multiple condition coverage).

I think, when you consider adding someone to your team as tester and he or she does not get nervous of these questions (or even gets excited about the prospect of working together with someone who understands testing  and comfortably answers them using a considerable amount of the keywords/phrases mentioned above, you can safely assume your team is about to be strengthened by a true tester!