Mapping Biases to testing in the LLM-era
This page contains "the receipts" for my talk Mapping Biases to Testing in the LLM-era. I have given this talk at Øredev and Agile Testing Days in 2026.
Slides
PDF version of slides here: https://maaikees.stackstorage.com/s/biases-talk-maaike-brinkhof
What is a cognitive bias
A cognitive bias is a systematic pattern of deviation from norm or rationality in judgment. Individuals create their own "subjective reality" from their perception of the input.
An individual's construction of reality, not the objective input, may dictate their behavior in the world. Thus, cognitive biases may sometimes lead to perceptual distortion, inaccurate judgment, illogical interpretation, and irrationality
System 1, System 2
System 1: fast, automatic, emotional, stereotypic and unconscious thinking that YOU cannot stop.
System 2: slow, effortful, infrequent, calculating, conscious
Risks of using System 1 in testing
- Not thinking things through, jumping to conclusions
- overconfidence, based on experience
- not seeing risks/problems
- not being aware of biases
- if only one person does testing, you can be sure that some problems are not spotted.
Benefits of using System 1 in testing
- fast thinking has its place (blink testing)
- experience can be a valid way of spotting problems
- serendipity in testing -> an experienced tester follows a hunch and finds an important problem this way.
The Risks of using System 2 in testing
- it's too slow, you don't get the time. This is more common than you think, as testing is usually done under a lot of pressure
- even if you use it, it doesn't guarantee that you're doing the right thing. You can think long and hard and still make mistakes
Benefits of System 2 in testing
- useful for test design and exploratory testing
- can use this way of thinking to ask questions/spot problems/make people consider different points of view.
- very powerful way to find problems at all points in the development cycle (but you cannot find all problems!!)
Software development has a bias problem
Software development as a whole has a huge bias problem. For some reason, too many tech workers think that they are "rational" and "smart". It's exactly this overconfidence that is an extra risk for biases to occur. We need to stay humble, and accept that we are fallible.
The fact that software is largely made by white men that have studied computer science is a huge problem on its own. There are so many blind spots! So many lived experiences missing, and it shows in how software functions these days.
https://sciencepolicy.hsites.harvard.edu/blog/racial-discrimination-face-recognition-technology
Vignettes to look at biases and problems in LLMs
I have not answered all these questions during my talk (30 mins isn't enough time), but I think these are important questions to ask ourselves.
| what | central question |
|---|---|
| creation of LLMs | to what degree are biases an even bigger problem than they already were? Did the problem grow? |
| process of using | How is widespread LLM usage changing the way we work together to make software in development teams? What is biasing us here? |
| human that uses | How does LLM usage change the human using it? Which biases come into play? Why does that matter? |
| Result | Is software quality changing because of LLM usage? How do we test code that has been mostly LLM generated? |
| influence on work | To what degree has the culture of software development changed? Are developers and testers better off now? |
From a testers POV + "just my opinion, man"
The creation of LLMs
They are unlawful by design (https://www.amnesty.org/en/documents/pol40/0996/2026/en/)
trained on stolen data,
labeled by exploited workers (https://www.theguardian.com/global-development/2026/feb/05/in-the-end-you-feel-blank-indias-female-workers-watching-hours-of-abusive-content-to-train-ai),
and the data centers required to run all this crap ruin the planet by consuming vast amounts of water and electricity.
https://memeburn.com/ai-water-consumption-statistics-2026-and-key-findings/
Joep Schuurkes blog: https://smallsheds.garden/blog/2026/on-the-acceptance-of-genai/
Book tip: Empire of AI by Karen Hao
Sources:
https://dl.acm.org/doi/10.1145/3442188.3445922
https://academic.oup.com/ijpp/article/32/6/524/7748890
https://www.nature.com/articles/d41586-024-00674-9
https://www.unesco.org/en/articles/generative-ai-unesco-study-reveals-alarming-evidence-regressive-gender-stereotypes
Process of using the LLMs
Lowering the barrier to get "a result" = feeding into confirmation bias, eroding your own skills.
Some developers notice this and dislike it.
Sources:
More developer stories:
Human that uses the LLMs
https://www.science.org/doi/10.1126/science.adi0248
There are two excellent blog posts you can read about this topic.
https://deadsimpletech.com/blog/no-such-thing-as-just-a-tool
https://aworkinglibrary.com/writing/toolmen
To quote Mandy Brown:
Engaging with AI as a technology is to play the fool—it’s to observe the reflective surface of the thing without taking note of the way it sends roots deep down into the ground, breaking up bedrock, poisoning the soil, reaching far and wide to capture, uproot, strangle, and steal everything within its reach.
Iris Meredith explains it by arguing that any tool becomes an extension of you. She calls it the human-tool gestalt. And since every human is impacted by ideologies, has their own way of looking at the world, this makes the tool not neutral.
when I hear a tech person say that something's "just a tool", I can't help but read it as an unwillingness to take responsibility, as an industry, for what we put into the world and for what we do to ourselves. Tools, unfortunately, simply cannot be the neutral, separate thing that so much of the tech world would like them to be, and pretending otherwise, let alone telling people affected by the tools that they're simply using them wrong, is an abdication of our moral duty to avoid, inasmuch as is possible, doing harm. We really ought to do better.
The result of LLM usage
For this point, I recommend the excellent article If No One Pays for Proof, Everyone Will Pay for the Loss
https://freakonometrics.hypotheses.org/89367
Influence on work culture as a whole
So what is the influence of LLM usage on work culture as a whole?
- Dunning-Kruger Effect
- confirmation bias & cognitive dissonance
- cognitive surrender
- automation bias
- groupthink (or not working in a group at all anymore)
- discrimination
- illusion of control
But more important than any list of biases can every be: to me, it feels like the LLM era is the culmination of everything that was already rotten in tech.
I am not alone in thinking this way, feeling this way.
https://ky.fyi/posts/ai-burnout
https://blog.glyph.im/2025/06/i-think-im-done-thinking-about-genai-for-now.html
https://karolinaszczur.com/ai
End
Imagine being confident enough in your own skills and abilities that you don't see the need to offload this to a machine. 4 years ago this was normal, and now this is somehow strange. "You will be left behind!" Fine, then leave me behind.