The abysmal state of ‘testing’ in 2016

I wish I didn’t have to write this blogpost, but here goes. Time to bring out the famous hammer metaphor. As a child I learned how to use a hammer. Not that I’m an expert user, but I’m skilled enough to slap some nails into a wall with it. As a child, I even participated in a yearly week-long ‘carpenter camp’ (timmerdorp, for the Dutch readers) and we built huts together; great fun!

Now, imagine that I’m going to start using the hammer for everything. My sink is clogged?? Time to bring out the hammer and bust it open. My oven has stopped working?? HAMMER TIME. You’re probably already tired of this metaphor and I hope you get the gist.

It’s ridiculous to use a hammer for everything.

Then WHY IN GODS NAME do we think it is okay to equal testing to automation!?

rage

I’m freaking tired of seeing lousy job ads asking for an agile tester, when in the description I read ‘We’re looking for an agile tester who is constantly busy with automating tests. We don’t like manual work, we are too busy for that’. I’m looking at you Albert Heijn (text of the ad is in Dutch). If you don’t like manual work, maybe you should fire all the people and get robots to do the coding for you.

Moreover, during my work I sometimes feel like I have to defend myself when I do some testing myself. Sometimes a developer is truly puzzled why I don’t automate something. They can’t look inside my head and probably think I’m doing the same thing over and over, while in fact I look for subtle variations and new combinations. I’m exploring! I’m using the software while trying to ‘think like a user’, ‘think like a hacker’, or whatever persona I can think of. I always use tools to support me during testing.

Do I hate automation? No, of course not. For the first time, I’m working with developers who are heavily into unit testing and doing it quite good! So I’d rather team up with them to see what we are unit testing and what we can automate on another level. Put that in a CD pipeline and you have a great baseline of checks running without having to lift a finger. Awesomesauce. But that’s not gonna prove your product works. It’s only proof that you’ve built it right.

Did you build the right thing?? That’s only for humans to evaluate. And a tester (or anyone else from the team) can help you with that in various ways.

Please, dear people, stop equating testing to automation. I get that you want the world to be a simple place, but it is not. And if you think I’m wrong, please hire the developer (because you’re not going to hire a tester if you’re asking for automation only) and enjoy testing software with only boolean outcomes.

And now I’m going to relax and sip some tea, cheerio folks

Thanks to Petri Kainulainen and Patrick Prill for enraging me (I see that as a good thing when it’s inspiring me to write blogs), cheers guys!

11 thoughts on “The abysmal state of ‘testing’ in 2016

  1. Maaike, I’m not sure how I did it, but I’m happy I inspired you to this post. It describes one big part of the problem that testing as a profession still seems to have on a regular basis. Skilled testing seems to be something that many people don’t realize that it’s possible and necessary. I’m afraid there are to many phonies out there. Too many people with the idea that testing is so simple even they could do it.
    Testing as a profession needs to improve and not by a few certification courses. Testers need to be better advocates for their own work.
    Managers won’t change their opinion about testing as long they haven’t seen the difference between a sapient tester and someone who’s just doing something based on instructions someone else wrote a good while ago. Of course do managers get the impression all that can be automated. I’d think that, too, if I see shallow “testing” in action.
    Keep on fighting!
    Patrick

  2. Testers (and as a community), I think, we are bad at explaining what we do. I don’t think things will change until we can all start marketing/selling ourselves better.

    Of interest, might be, ‘What Do Testers Do?’ http://www.ministryoftesting.com/2016/01/what-do-software-testers-do-version-0-1/ – I’m working on something around this, however, the basic of it is that we should all be constantly practicing the pitch ourselves until we feel we are confident in our message.

  3. Sometimes it becomes important to log all issues you find somehow. This is especially true in agile teams. Just create some list (a task issue or the like in e.g Jira) and note found/fixed for each issue/line.

    Otherwise, because of attitudes etc, people night start arguing that you don’t deliver any value. If you have the list they’ll have to argue that you’re lying.

    This is a reality in some situations. Better safe than played out!

  4. Very well written. Right on the mark in many ways. It sounds funny and frustrating at the same time when I read people looking for a excellent manual Tester who is hands on coder too or Automation guy. I wish them luck.

  5. Very true, market right now is like, if you do not know any automation tool then you are not a good tester and we do not hire you, this is not good because testing is not only knowing any language testing is thinking or approach.

    Most of the time people ask you know selenium or automation? instead the correct way to put this question is, do you know any language because if a person knows any language then he/she will able to script using any tool which support that language. And to learn a language is not every body’s cup of tea its just like not everyone loves to play or see football.

    Knowing a language is good but filtering out people in such criteria is not good. There are lot of automation testers who can’t think or take a proper approach to make software better instead they just want to prepare scripts to run. Also people think testing is very easy & everyone can do it but that’s wrong thinking because everyone is not able to think in that way. So If anyone want a tester they should see the person ability to find bugs in application or approach to test it instead how he/she can prepare automation script or he/she has any certification.

    If you see the most of questions which are present in ISTQB or any certification are irrelevant to practical scenarios, as practical scenarios are very much different as compared to theory. So thinking in a way that he/she has certification it means he/she is good in testing is also not correct.

    These are the few points there are lot which we can discuss on this…

  6. Please check my software vs please test my software. Checking vs testing…these are not mutually exclusive nor are they be interchangeable.

    Automation is wonderful to check software — observe its interaction, evaluate it algorithmically and report any failed checks.

    Whereas, testing is exploring, learning something new about the software and providing information to generate informed risks to stakeholders.

  7. Very well stated Maaike. If I look at the hiring practices of companies in South Africa, it almost seems like if you do not knowing any testing tools, you are deemed as not worth hiring. Everybody wants testers with Automation experience or knowledge but in reality, tools cannot be used for each and every project. Manual testing can and should never seen as optional.

  8. Yep this resonates. I think the “every problem is a nail” view is largely due to stakeholders not grasping what automation is for.

    Automation is NOT for setting up a computer to run every single manual test you have in your battery. That creates a maintenance suck, doesn’t matter how great the tool is that you automate with.

    Automation IS for assisting testers. The best results let testers know whereabouts a problem might be, and every automated test should give you information about the system that you didn’t have before.

  9. Great article Maaike! I recruit and manage testers every day and the number of times a candidate admitted when asked that they were limited in automation despite it be plastered all over their resume was that they wouldn’t get an interview otherwise! your analogy stands, automation is great, as a tool, wielded by humans. The intuition, invention and insight of an experienced QA professional cannot be automated…Never mind empathy with user/customer!

    I might direct you to my colleague Rich Rogers’ blogs on the same subject.

    https://richrtesting.wordpress.com/

    And also my company AccessHQ, with our emphasis on Human Quality!

    Keep fighting the good fight. 🙂

    Harvie,

  10. Hey there I am so grateful I found your site, I really found you by error, while I was researching on Bing for something else, Anyways I am here
    now and would just like to say thank you for
    a remarkable post and a all round enjoyable blog (I also love the theme/design), I don’t have time to browse it all at the minute but I have book-marked it
    and also added in your RSS feeds, so when I have time I will be
    back to read a lot more, Please do keep up the awesome work.

Leave a Reply to Pankaj Sharma Cancel reply

Your email address will not be published. Required fields are marked *