A TestBash conference in my own hometown, that is like a dream come true! Going there on my bike, super nice. I’ll try to live blog the whole conference day, hope it’ll be useful to you.
Technically, this is already the second day of TestBash NL 2017. Yesterday we had a day filled with workshops. In the morning, Cirilo and I gave a workshop on Specification by Example. Our group was very involved, so we had a great time.
After lunch, I chose to go to Gitte’s workshop on finding your courage. That was an amazing afternoon! A great serendipity moment as well, because this was just what I needed. I cannot tell you what was discussed in the group, but it was really special to hear what personal stories people had to share. What we did discuss that can be shared publicly is the question: ’what is courage’. We came to the conclusion that it’s a very personal thing. What is a courageous act for someone else, can be a piece of cake for you. And most courageous acts require you to go out of your comfort zone. You have to be motivated to do that! If you do courageous acts only for someone else, you risk going too far of you comfort zone into the ‘danger zone’. So, take baby steps and try to find a safe space to do new things that are (a bit) scary for you.
After the workshop day, about 50 people went to a bar nearby. There was a lot of talk about testing (of course), about the workshops, about what people are going to do in the future. I got to know a couple of new folks and had a great evening overall. I felt very immersed for the whole day, so I had a weird sensation when I got out of the bar and found myself in my own city. For some reason, being in my own city felt very odd. Like being at a conference is always associated with being somewhere else, abroad. But now I was home in 15 minutes, doing yoga in my living room.
With that summary of the workshop day, let’s get on to today! First speaker of the day is Alan Richardson. So far, I have loved every talk I’ve seen from him, so I’m curious!
Alan Richardson – How to misuse ‘automation’ for testing
He wants us to do different things with automation that are maybe perceived as ‘wrong’ by others. We have a certain way of thinking how automation ‘should be done’, so he wants to fuck with our minds and entice us to do something different.
What do testers do anyway? Don’t we misuse the software anyhow? (How often were you told that the defect you found would never happen in production because ‘users don’t do that’?)
But what is wrong use of software anyway? We are biased creatures anyway, anything we use is affected by how we perceive it.
So Alan turned to an objective source of truth: Google *cough*.
Turned out, Google wasn’t really helpful, although it turned up some funny answers about what a tool is, and automation.
Alan says that a tool is anything that he uses to support his testing. I agree with him there, your brain is just as much a tool as a technical test framework.
So, if you follow this train of thought, then any tool can have more than one use other than the obvious. We should be careful to label tools. Is Cucumber a testing tool? (No, it’s a vegetable).
This all seems very nitty gritty, but it’s about thinking differently about something you think is obvious. Getting out of the paradigms about what you should be doing with tools according to….people, unwritten rules, etc. Those are just belief systems.
We all have a model of the world, of its creations and how those creations ‘should be used’. When you think you wrote the best test strategy in the world, you pass it to someone else and they think something completely different. That’s life. You’ll have to respect that others can (and probably should) think different things than you.
If you focus on the behaviour (the what) in stead of the justification of it (the why), you’ll gain a different perspective on a lot of things. You can then focus on getting more out of a tool, whatever tool that might be. You can let go of the accompanying dogma’s that surround tools, frameworks, test processes…In that way you can ‘misuse’ automation; you’ll do different things with tools that others might view as crazy.
If we look into the tool Cucumber, many people would say it’s a BDD tool. But that doesn’t help you when you need to work with REGEX and data separation (which is a part of Cucumber). If you have not mastered those things, you’ll fail at using Cucumber effectively.
Alan then gives an example about how he ‘misused’ RestAssured by just not using the given/when/then structure that this tool offers (and sort of implies you should use).
That poses a question in my head. It’s all fine and dandy to misuse testing tools on your own, but I have worked with RestAssured in projects and I don’t think my teammates would have liked if I had suddenly let go of how RestAssured ‘should be used’. So, is this mainly something you should do in your homegrown projects? Like, how feasible is it to misuse in a professional setting?
I cannot help but smile throughout this talk. I’m quite sure this talk will confuse some people, it has surely got me thinking! I now realise how often I skip parts in my thinking about testing and take testing decisions for granted (because we should always do cross browser testing, right? But who stops and thinks about the technical ‘why’ behind that every time!?).
In conclusion, what Alan would like to see and I will pass this message along through this blog is how people ‘misuse’ testing tools. So, write a blog about this, share it with the world! And even more importantly, have fun misusing tools!!!
Break time
Met a couple of new people during the break! Such a great atmosphere already. In the end, I did ask my question to Alan: how would you handle misuse of automation in a team setting? He said: either mess with people’s belief systems until they fit to yours, conform to their belief system, collaborate, compromise…Of course, the way he framed the answer was way funnier than I picture here, but I have to write fast here, forgive me.
Gitte Klitgaard – Having the courage to be yourself
Gitte, oh how I love her. Her workshop yesterday was also about this topic and it’s so important. It’s not about testing in the ‘normal sense’, but in some way it’s about testing yourself.
Do you do what you do in order to fit in?
Do you dare to speak up or would you rather blend with the wallpaper behind you?
Gitte used to be a very plain person, although that’s hard to believe now. When she stopped giving a fuck about what the world expects of her, her personality started developing. She wanted to dye her hair blue, so she did. She wanted to play with kids as a kid, so she did.
At first, she didn’t dare to speak in public. Her manager back then dared her to speak in front of her team. And when she saw the poster-women in a poster (perfect, slim creatures) she got pissed, because is that the image we want to display to the world? Only if you look perfect you are welcome as a woman in IT? This anger turned into courage and she grew into being a speaker.
Is courage without fear? No, it’s not. Doing something new is often scary. If you jump out of an airplane without fear, you’re probably just stupid.
So, what is courage to Gitte. Firstly, it’s about being who you are even though your surroundings find you strange. Be different, if you want to be. By being courageous, you’ll also make yourself vulnerable. By nature, we like to stay in our comfort zone, but can we grow there? It’s also about sharing your fears and joys. In work, we often forget to do that, because we are supposed to be ‘professional’ there. Who dares to show up at work and say “I’m going to have a shitty day, I feel bad after a fight with my partner”? There’s no such thing as a little courage. If you worked up the courage to do something, it doesn’t feel little to you.
Sidenote: this is only the second talk and the wooden chairs are already fucking up my back…grrr
Oh shit, we have to dance now, brb.
How does all this relate to agile? The most important thing is to inspect and adapt.
It takes courage to look at the stuff you did with your team and see how it can be improved. You’ll need some form of self-critique, introspection and being honest with yourself. Learning is scary. The more we learn the more we realise that there are so many things we don’t know.
Makes me think about the time I just started as a tester. The world seemed simpler back then, but that was just because I knew very little about testing and software development in general.
Changes are scary too. We are wired to avoid change, but agile forces us (almost) to face this fear and change often.
Asking for help is scary as well and so on. There are so many things: telling the truth, doing something new, working together with new people, etc.
Gitte got a great round of applause from the audience.
After this talk, I’m going to stand for awhile, because these chairs are absolutely killing my back.
Tom and Sharath – Do Testers Dream of Electric Sheep
Of course, this title comes from Bladerunner, for those who don’t know. I once started in the book, but didn’t finish it for some reason. Saw the movie, so there’s that at least.
Will testers be replaced by machines? Is testing still a viable long term career path? A very actual question, right now! It’s something I struggle with myself.
Tom started as a tester and he was quite content to stay there, but others were like ‘don’t you want to be a test manager?’. He was shown a career matrix and that sort of nonsense. So, because utopia was promised to him he tried it. He learned some typical management skills and while he disliked some of them, he had to admit that some of these skills were useful.
Now Sharath is speaking. He had a very different introduction to testing. He had never been trained in testing, but was asked to test. (I think it’s barely different from the hordes of young professionals with an ISTQB certificate who are let loose in the world, but I digress). He even went into test management quite quickly, but he wanted to go back to testing. Because of his culture (Indian) he felt like he had to conform. He wonders why it is not more widely accepted to do cool stuff with the skills you have instead of fitting into a described role.
What we do as testers is very hard to put into a box. There’s so much stuff we do, so much skills to obtain. It’s hardly feasible that one ‘tester’ possesses all of these skills. Yet, when you see a tester role described in a job offer, it’s often very focussed on one thing (following process X, automate the tests, ugh). That hardly does us justice!
I think it’s good for us to realise that we probably do more than we think or realise. I do hope Tom and Sharath share this giant list of what testers do, it would be a good reminder for the community.
Given this enormous list of things we can do as testers, you soon realise that it’s sort of important to have the skill of deciding how to spend your time wisely. You always have limited time to test, so what are you going to do? That’s a huge skill on its own! I think many testers (including myself) could benefit from improving it. How to maximise what you get out of your time?
Use a Quality Map, is their first suggestion. It’s partly about good ole’ quality attributes, but combines it with persona’s. When is software successful, useful, usable, secure, deployable? Again, I hope they share the slides/pictures because it’s hard to only convey in words.
Use Heuristics, is their second suggestion. Use learning lists, ask a lot of questions, study competing products, interview users, analyse risk. On the surface these are very obvious things, but do you do them a lot in your daily work? I only do some of them, and usually the same. I should definitely do more with interviewing users, for example.
And their suggestions go on and on, it’s so much! They should share this in some way, because I just cannot keep up with my writing. A little note of critique is also that I’m not sure the audience is going to parse all these suggestions. It all seems important, but it’s a LOT.
[I had to go for a restroom break, so I missed a bit of the talk]
Holy moly, the list of suggestions how to improve your testing goes on and on. The conclusion is: You need to build the skills and practices to combat any context.
Diana and Simon – Have you ever been embedded?
Hey, I recognise them both! Diana from Agile Testing Days and Simon from the courage workshop yesterday.
Have you ever been embedded as a tester in a software development team? What do you do if that’s the case. Lead the way in testing for the rest of your team and work together with the developers.
They are both telling their story of how they ended up feeling truly embedded in their team instead of standing apart as a tester in a different team.
I notice that my concentration is slipping because I’m a hungry ogre at this point. It takes a lot energy to pay attention all the time and distill the core points of each talk and write it down at the same time.
Lunch time
Time to get some conversations with people: Tom, Cirilo, Patrick, Rob, Klaas. Great time and an opportunity to get some fresh air outside, because boy, is it warm inside here or what?!
And food. Food.
Rutger – My Life as an Agile Test Coach
His first time speaking at a conference, although he did give a workshop at the Agile Testing Days last year. For a first timer, he seems very calm and collected!
We will get to hear his story about what he did as test coach at ING. He is sketching the context about his team and the IT landscape first.
They had some testing problems like almost no documentation about how stuff is tested and insanely complicated test cases that almost no one understood anymore.
So how do you know if you are doing the correct things? Basically, his team needed a better test strategy and approach.
This talk for me is a bit too specific about his team situation. It might sound selfish, but I wonder what I should do with this information. I miss a ‘mission statement’ in his talk at the beginning, like how the audience will benefit from his story (or I missed it?). I always appreciate when talks offer me either something practical, ruffle my feathers or even better, blow me off my socks. A very contextual talk is often very clear for the one giving it, but harder to follow for the audience. Your mileage may vary of course, just honestly sharing how it is for me.
That being said, I hope others do get something of value out of this talk!
Ash Winter – The forgotten ‘ility’
Finally, a talk about testability!
Being able to observe, control and understand is key to testability… testers often don’t get it, he claims.
If you take this statement seriously, than most of us aren’t good testers? To control the product is often depending on if you can code and mess around with test environments. Ahs learned how to work with perl, php, bash and more in order to have more control over his testing. I do think that it’s okay not to have every skill though, pairing up with another teammate is fine too.
How do you deal with integration? You have to get to know the third parties and how to effectively mock stuff.
Also don’t forget how the world looks for people in operations. They are the ones having to deal with the problem when stuff goes wrong in production. So if you make an effort to know what they would like and how you can help them, that’s a great thing. If your team can give operations small tools to help them with their work you’ll make their life easier. Also, is it useful if you log all the things? How does that help operations? If you log what matters it’ll save operations a lot of time wading through errors that don’t matter.
In the end, Ash got to know PHP a lot better. A blessing, or a curse? 😀
He drops a bomb: More environments doesn’t improve testability. I agree with him if he means old school test environments. But if it’s really easy for you to spin up a Docker container and have control over your test environment, is that bad? I wouldn’t think so.
He also says: “the past is a foreign country when it comes to adding unit tests way after the fact. you miss the intent.” HEAR HEAR!
That was about the end of the talk. I feel like my concentration keeps being a problem, because this talk was also not so easy to follow for me. My brain is not really working at full force, or something.
Mary Gilmartin – Just Enough Security
Can perfect security exist?
It has been done recently, but not where you expect it. The US military built a new helicopter and flew in (see what I did here?) a security pentesting team to break into the helicopter. At first, they succeeded almost at once. The software was improved and now, for 6 weeks the security team couldn’t break in anymore.
Software ages like milk. Most security issues stem from software that isn’t updated anymore (Windows XP anyone?).
Cyber crime can work because the exploits that are found can be sold to people who want to do bad things with them. So, there’s a market for it basically!
Why is patching so hard then?
Even if you’re doing your very best to keep everything patched, people can have blockages to receive the patches. Some of these things aren’t in your control (firewall at your internet provider, that sort of stuff). This is known as the dependency problem.
[I’m learning a lot of new stuff during this talk, wow!]
Package managers can help you to minimise the dependency problem.
So, what is just enough security? It’s a hard balance to reach, for sure. Developers need to think about it, testers need to keep an eye out, the ops team needs to think about it. Everybody needs to work together again.
I personally see a problem here though, not many people have a lot of knowledge about security. I sure as hell don’t. To me, it feels like a complete domain on its own to become an expert in. So, I would like to know when you know just enough about security to be of value to any software development team as a tester.
Mary uses charters to help teams think about security. An account charter is about how many test accounts there are, if they should all exist, if that list of accounts is up to date, who can create accounts etc. It’s just a reminder-type thing for teams to think about accounts and security. Of course, there will be more types of charters that can link to security (for example: password charter, config charter, recovery charters).
List of 500 worst passwords is on github: github.com/danielmiessler/seclists/tree/master/passwords
Remember that website ‘have I been pwnd’? Yeah? They have been breached themselves as well. No one is safe!
This talk hits quite close to home for me, since my own website got hacked this week. I was forced to dive into solving security issues and felt like I opened a can of worms. There’s so much information about security (in WordPress, in my case) and I just didn’t know where to start. I installed an automatic scanner on my website now, changed all the passwords, made fresh backups, but who knows if it’s enough? I still feel like quite a security n00b, actually.
Mary also says: automation is your friend. Try to have automated security tests running and try to figure out what you’ve got covered with those.
[Again: please give us n00bs some practical starting points, because there’s too much to learn and where on earth do you start?]
I took a short break from blogging, because I’m really tired and my concentration is gone. It’s still not really up to par but the next talk is about my favourite topic.
Mirjana Kolarov – Reaching Symbiosis of exploratory and test automation
The big question: what do you automate in testing and what do you explore. How can you combine the best of both worlds.
Often, it is phrased like a boolean: are you for automation or manual (exploratory) testing. This is never a black or white thing!
You cannot automate everything, but neither should you use manual testing 100%.
You want to have both, I hope we all can agree about that.
We see the video from Daniel Simons, where people in white and black t-shirts pass a ball while in the background the curtain changes color, a gorilla enters the scene and a player leaves the field. This video proves that we are very bad at focussing on multiple things at the same time. If you would have automated this test case, it would have perfectly counted how many times the ball got passed around, but not the other stuff.
Now we play the 20 questions game. We could only ask yes or no questions to figure out a word. The old school way would have required us to prepare all the 20 questions before hand (test script does that) and then ask all them. In Exploratory Testing, we can ask a question and immediately change our strategy. There’s a freedom in that.
Now to the automation part.
On unit test level, testers are usually not involved as much.
Integration testing is when testers jump in.
E2E testing is the hardest. Don’t put all tests through the GUI, API e2e testing is a thing too!
[Guys, wtf, my brain is messing with me. I’m so tired, hah! I feel barely capable of blogging anymore. And I volunteered to the a 99 second talk, which I highly regret at this moment, lol.]
The Day After
And those were the last words I wrote yesterday…
Like I already said on Twitter, live blogging is a bit like exploratory testing: you parse information, think about how to summarise it and then write about it at the same time. It’s simply impossible to keep that going at a good level for a whole day. Add a lot of tiredness into the mix and I couldn’t deliver anymore. It’s okay though, I’m still happy with my attempt.
I am a bit sad that I couldn’t give you the live part of Zeger’s talk “Becoming a software skeptic”. His talk was really good. His slides are always of superb quality to begin with, but he also had lots of small clips worked into his presentation, humor, great examples. He managed to engage a tired audience for 45 minutes. I hope you’ll have the option to see his talk one day.
The conference ended with 30 minutes of 99 seconds talks. I had signed up right away in the morning, and tried to estimate what I could say. And yes, of course, my estimation was wrong. I was in the middle of trying to inspire people to also get on Twitter, blog or on stage when the horn sounded. Shit, 99 seconds is really short. Many of the other 99 second speakers did a great job! Tony Bruce was the baller, just getting on stage to collect his Ministry of Testing beanie. I don’t blame him, the hat kept me warm when I cycled back home.
Some people went to the bar next door to get some drinks. I really wanted to join them, but I was too sleep deprived. I knew I had to take care of myself and rest. I’m writing this to you the day after, with the support of 12 hours of sleep. Feeling really content with the last 2 days, I can say from the bottom of my heart: I hope TestBash returns to Utrecht next year. Thanks so much, Huib and Rosy and all the volunteers, for organising this.