Talking Testing with Michael Bolton

Please can you introduce yourself and give us a brief overview of current role?

I’m Michael Bolton. No, not the singer; no, not the guy from Office Space. I had my name long before either of those guys (the singer’s real last name is Bolotin).

I help organizations and individual people to get better at testing, so that they can find important problems in their products before their customers do. I provide participant observation, coaching, and consulting on how to do that. I teach classes on Rapid Software Testing (RST), which I co-author with my colleague James Bach. The goal of the RST approach is to help people learn how to design, perform, and explain testing when products need deep testing. In the classes, we help testers to think, speak of, and do testing in powerful ways.

Looking over your LinkedIn profile, you have had a varied and rich career. How did you first break into the world of software testing?

I started working with computers as a hobbyist. I did work as a data entry clerk; as a kind of database masseur; as a programmer; as a trainer; then as a tech support person; then at Quarterdeck in 1994, I finally got a job formally called “tester”. That’s the first time I got paid to do testing work full-time.

After that, I became a program manager; a developer (again); a documenter (I’d been documenting all along). These days I’m a teacher of testing; an author of a testing class; an independent consultant; a tester for hire.

The thing is, though, if you’re working with computers and software at all, you’re testing: evaluating and learning through exploration and experimentation. When you’re not testing, you’re stymied by errors, or oblivious to them, or tolerating them, or dumping them on your clients and customers. And you’re not learning.

You must have seen many changes in how we look at testing and how we value testers over the years. Do you feel that software testing is now looked upon as being “equal” to development?

I don’t think about it that way. Diagnosis isn’t equal to health care; diagnosis is part of health care. Testing is not equal to development; testing is part of development. Testing is the part of development that is focused on learning about the actual status of the product; dispelling illusions about it; seeking and finding trouble so that customers don’t. That’s a socially difficult role, because many people have a natural aversion to thinking about problems and errors and bugs.

We mostly live in societies where the builder mindset is given higher value, by and large, than the tester mindset. The creator is valued more highly than the critic. The optimist is easier for most people to deal with, psychologically, than the skeptic.

It’s understandable that people value builders more highly than testers. It’s easy to see the value of the visible things builders produce. It’s harder to see the value of looking for problems – especially when we’ve been successful at it! People never notice pain they would have felt from the bug that might have shipped, but didn’t.

I don’t think there’s anything intrinsically wrong with having more builders than testers, either. Still, if we’re building something important and we don’t want to be fooled, I’d say it’s a really good idea to have at least a few people around who remain professionally uncertain that everything is okay.

It’s important for those people to have the skills to be able to explore and experiment, to anticipate problems that are deeply buried or rare or subtle or emergent. Those people need to be able to model the whole system, and to investigate it with the specific goal of looking for trouble. We’ll probably always have some challenges in helping people to see the value of that.

It’s heavily debated whether or not all testers should learn to code, or their role may become redundant. What is your opinion on this?

The world of software development is dominated by programming. We write code to build things and solve problems. Because of that, it’s tempting to believe that testers should be programmers too. Ask a programmer – everybody should be a programmer!

I’ve written a blog post on why it might be a good idea for testers to learn to code. These days, it seems to me, a fair number of testers know how to code, and that’s a pretty a good thing, by and large. What worries me is that we might not have enough testers who are really serious about learning to test.

To test is not simply to show that the product can work. That’s demoing the product. To test is to approach the product and the system around it from many different directions; to analyze them; to model them; to challenge them; to gain rich experience with them; to find the usual kinds of problems, and to develop awareness of the new or unusual ones.

When a tester knows how to write code, but hasn’t learned solid testing skills, a couple of bad things can happen: testing work may transition into a building role, or shallow testing may get turned into shallower checking. On the other hand, skilled coders collaborating with skilled testers can get lots of useful testing work done, and each person can learn a ton from the other.

Software development is in the middle of a long head cold, in which people have come to believe that checking the build is the same thing as testing the product. Unit testing and automated checks are really helpful; it’s good to find low-level problems in code at the when they’re easy to find. Checking the build for relatively shallow problems, close to the surface, is a really good idea too. But let’s not stop there.

Far too much of what passes for testing these days, in my view, is about demonstrating that we can build the product frequently and that it can work. Now, sometimes shallow testing might be enough. A lot of software isn’t that important; low-risk; simple.

When money, health, safety, privacy, or fairness are at issue, though, complex systems need deep testing. We need to analyze them deeply, explore them thoroughly, push lots of data through them, given them a good hard shake, uncover hidden risk. We need to develop some experience with using those systems as they will be used, as realistically as possible, before we inflict them on the wider world.

The tech world is ever moving. How do you as a teacher and consultant who isn’t fully hands on anymore, ensure your skills and knowledge around software testing are continually kept up to date?

Yes, the tech world is always moving… Yet people talk about computers today in a way that’s startlingly similar to the way they talked about them in the 1950s. Here’s an example.

I don’t worry about technologies changing that much. I’m used to it. I’ve seen many things changing over a long career. I know I can learn what I need quickly in a given situation. We all do that, don’t we? We all plunge in, get help from people around us, and learn to swim.

The skills that I think are most important are not vulnerable to obsolescence, the way the language of the month or the testing framework of the week may be. Systems thinking, critical thinking, scientific thinking, risk focus, analysis, sensemaking, fitting in to a new situation, rapid learning… those things vary, but the essentials of them remain relatively consistent over time. Learning about those things, and practicing them, prepares you to learn quickly about the more turbulent stuff.

So I get my hands on stuff as often as I can. In consulting work with my clients, I’m constantly being exposed to new technologies, and I love that kind of work. It’s often helpful to bring fresh eyes into a project; everybody learns. In the Rapid Software Testing Applied classes, James Bach and I present people with real products and systems to test. To be effective, we test those products ourselves. We review and critique student work and our own, and debrief it extensively. And if someone surprises us with a test idea or test technique or tool that we weren’t aware of, we learn from that!

It is clear from following you across your social media you enjoy getting involved in conversations and debates surrounding software testing. Has this even landed you in hot water?

People will sometimes disagree with what I have to say, and sometimes they will get upset. In a sense, it’s okay for people to be upset. I’m not happy that someone is upset, of course, but feelings can point our attention to things that matter. When someone is upset, it suggests that something important might be at stake.

In a community that wants to make progress, debate and controversy are essential. When debate isn’t happening within a discipline, that discipline is standing still. The fact that people rarely questioned Aristotle basically froze Western science for almost 2000 years. Questioning perceptions and the status quo is the basis of good science, and to good testing, and to the ways we think about ideas. Debate anti-fragilizes ideas and makes them stronger.

I admire the way Van Jones puts things here.

I have noticed a number of conversations recently suggesting that the Test Community is too closed off and doesn’t welcome in people from the wider tech community. Is this something you would agree with?

I have some difficultly answering that question. First, I don’t think we can talk about “the test community”. There isn’t just one; are lots of them. They intersect and overlap to some degree, and they’re also estranged from each other to some degree.

The kind of testing community that I would like to see and foster would reach out more communities outside of our own, outside of testing, and outside of technology, too. My desktop and library and bedside table and reader apps are filled with all kinds of stuff: mathematics and medicine and sociology and baseball and qualitative research. All along I’ve tried to learn from all of those domains and to see the links between them and testing. Qualitative research books are better books on testing than most testing books are. Bill James’ 1988 Baseball Abstract is a more serious analysis of what numbers can and can’t tell us than you’ll find in most conference presentations on metrics.

Women in Tech / Women in Testing is also a hotly debated topic. Do you feel as a powerful male in the testing sector you have a personal responsibility to ensure diversity is taken seriously?

First, I don’t think of myself as “a powerful male in the testing sector”. I don’t feel as though I have much power, honestly. If I did, a lot of testing that doesn’t fit its context would go away. I don’t hire people, and (EuroSTAR 2013 aside) I don’t generally choose the programme for conferences. I like having ideas, and I like to share them, but as Fiona Charles said at EuroSTAR 2013, we should all be a little cautious – even squeamish – about “thought leadership”. We can all be thought leaders. We can all be powerful.

Every person must be treated with respect, but not every idea should. Some ideas should be treated seriously, and we do that by discussing them and debating them thoughtfully. Weak or silly ideas won’t last long, given that treatment.

Diversity in testing is an idea that has to be taken seriously, because it’s essential to have different perspectives to foster critical distance between mindsets. The more we’re all alike, the greater the chance that we’ll be fooled by missing something important. Diversity is a powerful heuristic for addressing that problem, not only diversity of genders and identities, but also of cultures, ethnicities, experiences, temperaments, skills, approaches… Growing recognition of that is important, and a wonderful thing.

On this note many men have actually come out and said they won’t participate in conferences that don’t offer a diverse line up. Do you agree with this movement or is there a better way to tackle things?

It seems to me that people should follow their own lights on how to effect change. I wanted to make sure the EuroSTAR 2013 programme was eclectic and covered a lot of perspectives, and I chose a variety of colleagues to help me with that. It’s pretty obvious to me that a conference that doesn’t offer a diverse lineup won’t be a very interesting or worthwhile conference.

Finally if you hadn’t ever got into testing, what job do you think you’d be doing now?

Testing, and helping people learn to do it, is already a pretty perfect fit. I’m engaged with testing because it’s focused on finding risks and problems that cause trouble for people. That’s a key step in helping to make the world a better place.

 

To keep up to date with Michael Bolton on social media visit his Twitter and LinkedIn, and you can also check out the rest of my Talking Testing blogs here.

Share this article...

Other Articles...