Repetition in testing

Many people in the IT industry seem to hate repetition. Doing something more than once? Silly you, automate it! What strikes me as funny is that it’s probably not all that easy to automate some of these things. It takes time! But apparently, that’s not what matters. It’s more about the avoidance of repetition. 

In testing, I see this same notion trickling through. Repetition? Yuk! Write an automated check. Humans are too good to be doing the same things.

I want to break a lance for repetition in testing though.

With repetition I don’t mean ‘follow a test script to the letter and ignore everything else time after time’. That’s a human trying to be like a computer. That’s not where we shine.

No, when you repeat some steps during testing, force yourself to pay attention to all the details around what you’re doing. Maybe we can call this ‘mindful testing’. Be in the moment when you test. Is that too vague for you? It just means: turn off all your notifications, sit in a quiet spot and really only focus on the testing; try to quiet your thoughts. You’ll see that there’s much more to see than if you test and talk with your colleagues at the same time, or get distracted by a Slack notification. Of course, you don’t have to do this all day. But a session or two like this? Golden.

Another never disappointing heuristic you can use while repeating some tests is ‘vary variables’ (you can read about this in the book Explore It!). Maybe your test path is the same each time, but if you feed it new interesting input each time, you might uncover some defects. Last week, I was testing an app that gave me as output: an amount of money with localised currency. That is interesting to test, because there are quite some rules when it comes to localisation and currency. Do you display “€2,44”, “€2.44”, “2.44€” (not even looking at different currency symbols here)? That all depends on the country you’re in. The programmer hadn’t used all the standard options the library provided, so we had some issues there.

Could I have automated this as checks? Sure, but before I did these repetition tests with the variable ‘currency’, I hadn’t even realised how many different ways of displaying currency there are. Engaging with the application made me think about this. Repeating some tests with this variable made me notice some things, made me do some research on the matter. Now that I have learned more about it, I might immediately automate them as checks next time in another context.

So, repeating a simple test might seem boring or silly to some, but to me it is not. I learned something valuable. That, to me, is worth a lot.

Next time you test something you’ve tested before, slow down. Don’t think of repetition as your enemy, but try to get the most out of it. Need inspiration to vary your ‘repetitive test’? Just look at the never ending awesomeness the Test Heuristic Cheat Sheet provides.

3 thoughts on “Repetition in testing

  1. I can only agree to that. That’s why I prefer defining rather only defining the goal (why?) of the test and leave the steps and test data open. With that you prevent exact repetition and enable yourself to actually find new bugs with a test script you performed already 20-50 times.

    I had a colleague who notes every exact step of the tests he made to repeat them exactly like that. And I found spread over multiple client configurations and versions always the same test data. When I used his scripts I tried to guess the why (because it wasn’t documented) and used my own test data, based on the cheat sheet ideas or others, and guess what… scripts that passed for years suddenly failed.

    So, repetition is good, but only when using variation. And explaining that to an automation script is something I try to challenge hopefully this year.

    Patrick (TestPappy)

  2. Using repetition like this can be a nice way to explore and learn especially when as you describe things are varied to some degree. Repetition doesn’t necessarily unlock this, in some cases it might even deter it. The currency and culture settings you use as an example cuts both ways. You may catch insight from the first round of repetition, or as Patrick commented a fresh perspective might uncover missed issues but what if you had to go through all of your supported currencies? The interest may wear thin and things might get missed. It’s a tough part of testing.

    Great post though, nice to see some love being shown to a beleaguered aspect of testing!

  3. Great blog post.

    I think people assume that repetitive tasks should always be automated as there is little to gain from people doing repetitive tasks and also it could be seen as a waste of money. However in my experience just varying the data and steps slightly can sometimes reveal some issues that you may of missed before.

    Its interesting that you mentioned about being mindful whilst testing. I wrote a post on this a while back which you may find interesting.

    http://testingsoftwaretimes.blogspot.co.uk/2015/06/testing-and-mindfulness.html

    Dave

Leave a Reply

Your email address will not be published.