An Agile Test Strategy: Some Ideas

There are a couple of misconceptions when it comes to an agile way of working. Some people still believe you don’t need documentation (seriously, that myth has proven really hard to debunk). I also notice that a lot of focus is on automating everything when it comes to testing. I think that is a noble goal, but you will never be able to automate all testing tasks, since the ‘real testing’ happens inside a human brain.

So, even in an agile context it is a good idea to have some kind of test strategy. By this I explicitly do not mean a test plan. You know, those 50 page documents that nobody ever reads. An agile test strategy should be short and concise. It should explain to people who matter how you are going to approach testing in your context. It is also a good way to think about testing with your team.

Continue reading

Mapping Biases to Testing: Confirmation Bias

I use terminology from earlier blog posts about biases. If you have missed those posts, read part 1 here. I explain the terminology there. In the second post I wrote about the Anchoring Effect.

Let me state the ‘bad news’ up front: you cannot fully avoid the confirmation bias. That’s actually a good thing, because if you could you wouldn’t be human. We jump to conclusions (with System 1) when our assumptions are likely to be correct[1] and a mistake is acceptable. For example, when you meet a new person you immediately judge him or her based on stereotypes, what type of clothes the other person wears, his/her posture, etcetera. This happens so quickly that you cannot stop it.


However, jumping to conclusions can be a bad thing when the situation is unfamiliar, the stakes are high and when there is no time to collect more information[2] . This screams ‘testing!!’ to me. We are often dealing with unfamiliar situations, the stakes are high and we usually face a deadline. How do we deal with this? Let’s explore what the confirmation bias has to do with testing. Continue reading

How I ended up in IT

This is a long read, be warned :-)

Honestly, I never thought I would end up working in IT. Now I’m here and enjoying myself, but it took me years to figure it out. I would like to tell you my story of how I got here.

When I was fifteen years old, I thought I would end up as a musician. I started playing the clarinet when I was nine years old. When I was fifteen I joined an orchestra with adult people in it for the first time and I enjoyed it so much that I formed a dream of doing that as a pro. Music was something I was at least excelling in, and besides, the rest of the school curriculum didn’t really interest me much. I only loved writing, philosophy (they offered that as a class in my high school) and reading for my literature class. So I poured my energy into getting better at making music. Getting extra classes on music theory, learning to play the piano and practising a lot of clarinet. After I had done my final exams for high school I also had to do an audition to get into music school. I was rejected in Utrecht (horrible experience), but accepted in Rotterdam. Continue reading

The abysmal state of ‘testing’ in 2016

I wish I didn’t have to write this blogpost, but here goes. Time to bring out the famous hammer metaphor. As a child I learned how to use a hammer. Not that I’m an expert user, but I’m skilled enough to slap some nails into a wall with it. As a child, I even participated in a yearly week-long ‘carpenter camp’ (timmerdorp, for the Dutch readers) and we built huts together; great fun!

Now, imagine that I’m going to start using the hammer for everything. My sink is clogged?? Time to bring out the hammer and bust it open. My oven has stopped working?? HAMMER TIME. You’re probably already tired of this metaphor and I hope you get the gist.

It’s ridiculous to use a hammer for everything.

Then WHY IN GODS NAME do we think it is okay to equal testing to automation!?


I’m freaking tired of seeing lousy job ads asking for an agile tester, when in the description I read ‘We’re looking for an agile tester who is constantly busy with automating tests. We don’t like manual work, we are too busy for that’. I’m looking at you Albert Heijn (text of the ad is in Dutch). If you don’t like manual work, maybe you should fire all the people and get robots to do the coding for you.

Moreover, during my work I sometimes feel like I have to defend myself when I do some testing myself. Sometimes a developer is truly puzzled why I don’t automate something. They can’t look inside my head and probably think I’m doing the same thing over and over, while in fact I look for subtle variations and new combinations. I’m exploring! I’m using the software while trying to ‘think like a user’, ‘think like a hacker’, or whatever persona I can think of. I always use tools to support me during testing.

Do I hate automation? No, of course not. For the first time, I’m working with developers who are heavily into unit testing and doing it quite good! So I’d rather team up with them to see what we are unit testing and what we can automate on another level. Put that in a CD pipeline and you have a great baseline of checks running without having to lift a finger. Awesomesauce. But that’s not gonna prove your product works. It’s only proof that you’ve built it right.

Did you build the right thing?? That’s only for humans to evaluate. And a tester (or anyone else from the team) can help you with that in various ways.

Please, dear people, stop equating testing to automation. I get that you want the world to be a simple place, but it is not. And if you think I’m wrong, please hire the developer (because you’re not going to hire a tester if you’re asking for automation only) and enjoy testing software with only boolean outcomes.

And now I’m going to relax and sip some tea, cheerio folks

Thanks to Petri Kainulainen and Patrick Prill for enraging me (I see that as a good thing when it’s inspiring me to write blogs), cheers guys!

Mapping Biases to Testing: the Anchoring Effect

Dear reader, welcome back to the Mapping Biases to Testing series. Today it is my pleasure to discuss the first bias in this series: the Anchoring Effect. Before we start mapping that to testing, I want to make sure that we have a clear understanding of what the anchoring effect is. 

Anchoring is a cognitive bias that describes the common human tendency to rely too heavily on the first piece of information offered (the “anchor”) when making decisions. During decision making, anchoring occurs when individuals use an initial piece of information to make subsequent judgments. Once an anchor is set, other judgments are made by adjusting away from that anchor, and there is a bias toward interpreting other information around the anchor. For example, the initial price offered for a used car sets the standard for the rest of the negotiations, so that prices lower than the initial price seem more reasonable even if they are still higher than what the car is really worth.”

I highlighted the important parts. Decision making is something we constantly have to do during testing, and it is important to realise which anchors might affect you. Also, to make this clear, I think ‘testing’ is not just the act of doing a test session, but thinking about everything that involves quality. You can apply a testing mindset to all that is needed to make software: the process, the specifications, the way the team works, etc. Continue reading