Testing As Consultancy

Introduction

Recently, I read Jerry Weinberg’s book “The secrets of consulting: a guide to giving and getting advice successfully”. This book has many pearls of wisdom, but I’d like to highlight one in particular.

Before reading this book, my working definition of consultancy was “coming up with ways to solve problems for organisations and people”. For example, suppose that a car factory wanted to increase the number of cars they build per day, and they hired a consultant to help them out. I’d imagine that the consultant would do things like: gather loads of data about their performance; analyse the data; determine the bottlenecks in the manufacturing process; come up with a process that reduces the effects of these bottlenecks; present the new process; teach the factory how to use it. The next time the factory had a problem that they couldn’t solve themselves, they’d hire the consultant to fix it, and the process would begin again.

However, in the book, Jerry explains that the role of a consultant is NOT to fix problems for other people. He describes a “powerful consultant” as a consultant whose style has at least the following characteristics:

“””

  • You strive to make people less dependent on you, rather than more dependent.
  • You try to obey The Law of the Jiggle: The less you actually intervene, the better you feel about your work.
  • If you succeed (in helping a client solve a problem), the least satisfying approach is when you solve the problem for them.
  • More satisfying is to help them solve their problems in such a way that they will be more likely to solve the next problem without help.
  • Most satisfying is to help them learn how to prevent problems in the fist place.
  • Your ideal form of influence is first to help people see their world more clearly, and then to let them decide what to do next.

“””

Based on Jerry’s description of consulting, I’d say that the role of a consultant is “to help organisations/people be able to solve all their problems without a consultant’s help”. I agree it seems odd that, when a consultant is hired at an organisation, their aim should be to essentially make themselves redundant!

So why on Earth am I talking about consulting when we’re a software engineering company? I think there’s an interesting parallel between consulting and software testing…

How is testing like consultancy?

Within some testing organisations, I think that the following definition of testing is one that many would use: “Testing is an empirical, technical investigation conducted to provide stakeholders with information about the quality of the product or service under test”. In practice, what this means is that I spend every day at work doing various bits of investigation to try and find problems with the product as a whole.

However, recently I’ve been worrying that there’s a fundamental problem with the above: I’m just one person – that places an enormous restriction on the number of problems I can find with the product as a whole. Taking inspiration from Jerry’s description of consulting, perhaps my role should NOT be solely to find the problems in the product. Perhaps “powerful testing” looks a bit more like this:

  • I strive to make the organisation less dependent on testers, rather than more dependent.
  • I try to obey The Law of the Jiggle: the less I actually intervene (by finding ship-stopping issues personally), the better I feel about my work.
  • If we succeed in finding a problem in the product, the least satisfying approach is when I find a bug/problem myself.
  • More satisfying is to help the organisation fix the bugs in such a way that they will be more likely to fix the next bug of a similar nature without my help.
  • Most satisfying is to help the organisation learn how to prevent the bugs in the first place.
  • My ideal form of influence is first to help people see their world more clearly, and then to let them decide what to do next.

I think that last bullet point is particularly important, as it yields a different definition of what a tester should do: “A tester helps other people to learn how to view things in a different light”.

Humble Inquiry

This blog was the inspiration for my talk at the recent Test Leadership Congress. I would highly recommend the conference!

Recently, I had my 18-month review. During the review meeting, one particular conversation I had particularly stood out to me: (Note – this isn’t a direct quote, but I’m pretty sure it captures the gist of the conversation)

“””

Manager: “You’re now reasonably good at critically thinking when it comes to questioning a product, but I’m not confident that you’re as good when it comes to critically thinking about people’s assertions.”

Kwesi: “What do you mean?”

Manager: “For example, suppose you were in a design meeting for a product, and an experienced developer said ‘the product should behave like X in situation Y’, or ‘it’s really important that you spend lots of time testing Z’. I think there’s a risk that you’d think about it for a few seconds, and then respond ‘yes, you’re completely right’, rather than giving it more thought.

Kwesi: ***Thinks for a few seconds***

Kwesi: “Yes, you’re completely right!”

“””

Admittedly, I have thought about it more carefully since my review, and I think the above conversation is accurate. I’m entirely capable of critically thinking about things. I spend most of my days at work interacting with our products, critically thinking about what I observe, and thereby exposing problems with the products. If I read a functional specification, I’m capable of generating many test ideas based on critically thinking about what I read. However, if I’m in a conversation with someone and they express opinion X, I find myself nodding along in enthusiastic agreement, even if I think they’re wrong.

So why do I behave so differently depending on the circumstances?

Conflict Avoidance

Throughout my life, I’ve always been very conflict avoidant. In fact, the only people I’ve ever had heated arguments with are members of my immediate family. I suspect there’s one key reason for this: I simply don’t “get” anger/irritation. It’s not that I don’t understand that these are pretty common emotions (see http://www.mindyouranger.com/anger/anger-statistics/ to get a sense of just how prevalent they are in day to day life). It’s more that they’re emotions that I basically never feel myself. I would say that I feel “irritated” an average of less than once per week. Anger is even rarer for me – if I try and think about the last time I felt “angry”, I’d need to go back about 7 months!

As a consequence, I equate irritation with “this is the worst thing that will happen to me this week”, and anger with “this is the worst thing that will happen to me in the next few months”. Unsurprisingly then, when I see other people being “angry” or “irritated”, my immediate reaction is one of fear and worry. And it’s this fear of people being “angry” or “irritated” with me that leads me to be so conflict avoidant.

How does this relate to work? Imagine you’re a tester in a design meeting, and the lead developer says “the product should do X”. With your critical thinking hat on, you think “actually, the product should do Y, as well as X”. How do you communicate this to the developer? Clearly the developer has invested time and effort into this design, and challenging their design might anger or irritate them. Based on the previous two paragraphs, I hope you can see why the prospect of challenging their design terrifies me.

Fear of not knowing things

A digression about the education system. In my experience, a key value that schools and university teach is the following: “revealing that you don’t know something is bad”. For example, how many school/university courses did you attend that were like this: 1) Teacher tells you “correct” theory/facts about thing; 2) Student is rewarded with good grade if they learn theory/facts, and regurgitate them in the appropriate fashion; 3) Student is given bad grade if they don’t do step 2. (I have thoughts about how this system could be improved, but that’s for another time).

With the benefit of experience and hindsight, I can see that I derived various terrible values from my education, but “fear of not knowing things” has to be amongst the most pernicious, deleterious lessons that I learnt. Here are four ways that it’s affected me personally since starting at work (I’m hoping that at least one of these ways will resonate with you):

  1. When I was asked a question by a colleague that I didn’t know the answer to, rather than say “I don’t know, here’s somebody else you can ask”, I would come up with the best plausible answer that I could, in the hope that they’d be satisfied with the answer. This was true even if my response was skilfully disguised bullshit. Also, if it’s a question that I felt that I was supposed to know the answer to, I’d be significantly more likely to bullshit.
  2. When I was trying to do a task, and things weren’t working as expected, rather than just ask somebody, I would spend as much effort as possible trying to diagnose the issue, and only ask somebody as a last resort.
  3. Generally feeling shit about my competency as a professional when I don’t know how to do tasks that I’ve been set, or answer questions that I’m asked. This was particularly acute during my first year or so at the company, having spent the previous 21 years generally knowing how to answer questions set by teachers, or at the very least, knowing how to make progress towards getting the correct answer.
  4. A fear of taking on tasks that are outside my experience or comfort zone, because I’ll regress to not knowing how to do things, and that’s bad.

To be clear, the above 4 things are not to be emulated! If I could go back in time and talk to myself on my first day of work, I’d scream at myself not to do these things!

So, let’s go back to the example of being a tester in a design meeting, and the lead developer saying “the product should do X”. You happen to know that it’s crucial that the product has quality Y, and you have a worry that if the product does X, it might threaten quality Y. However, you have no idea of the technical details required to rigorously demonstrate your hunch.  How do you communicate this to the developer? In particular, what do I say, given that I don’t understand exactly what’s going on, and I don’t have a concrete suggestion for what to change? Moreover, it’s likely that the lead developer knows more than I do, so is the problem my understanding or their design? I hope I’ve explained why the “fear of not knowing things” might make me reluctant to speak up in a similar situation.

So, here are two problems preventing me from doing my job as well as I could. So what should I do?

So, what is Humble Inquiry?

Coincidentally, at about the same time I had my review, I’d just ordered a book called “Humble Inquiry: the gentle art of asking instead of telling” (https://www.amazon.co.uk/Humble-Inquiry-Gentle-Instead-Telling/dp/1609949811).

A couple of concepts struck me as important: firstly, the definition of “here-and-now humility” – when person X is dependent on person Y to impart some knowledge or do a thing so that person X can achieve one of their goals. This actually applies to all roles in an organisation, especially managers, but this also applies to testers. As an example, in order to have a product that behaves in a way that is desirable to customers, a tester is dependent on the developer writing some code in order to do this. Moreover, the tester almost certainly doesn’t know exactly what code would achieve the desirable outcome.

Secondly, the book talked about a specific type of inquiry derived from an “attitude of interest and curiosity”. I interpreted this as the sort of questioning that isn’t leading, rhetorical, embarrassing, or a statement described as a question, but rather questioning with no other motive than interest in the answer.

To give you a (short) example from the book:

“””

My house was on a street that feeds directly onto the main highway into Boston. I was in my front garden when a woman drove up and asked me whether I could direct her to Massachusetts Avenue. This would have meant turning around and going back across several streets. I asked her, “Where are you trying to get?” (Humble Inquiry). She replied that she was trying to get to downtown Boston. She was already on the direct road to Boston, so I told her to just keep going on the road she was on. To this day I wonder what would have happened to her if I had told her how to get to Massachusetts Ave.

What I Learned

  • Don’t jump in telling answers until you know what the other person really needs to know.
  • Don’t assume that the person with the question has asked the right question.

“””

(If you want an example that is more specific to testing, read this short anecdote by Richard Feynman: http://tools.jiscinfonet.ac.uk/downloads/lsd/feynman.pdf)

As I was reading this book, a light bulb went off – I don’t need to confront people or know things in order to critically evaluate their assertions! All I need to do is humbly enquire! For example, I’ve been to design meetings where they’re discussing things, and I have almost no idea what’s going on. But I do know what the aim of the product is. Beforehand, my fear of conflicting people and not understanding what’s going on meant that I’d say basically nothing. Now, simply asking questions like “what does this product do”, or “what happens in this case” or “what happens if this particular feature fails” is a way for me to provide value whilst avoiding the two problems mentioned before.

In the past, my manager had spoken about a tester they’d managed: the tester would talk to a developer, and ask questions such as “if this component failed, what would happen?” and the developer would think about it, realise there’s a bug, and fix it before even coding it. I’ve always thought that was excellent testing, but I’d mistakenly assumed it was because that tester happened to be a product expert and was really good at spotting issues in products. But until very recently, I’d misunderstood – this tester was simply good at humbly inquiring!

So testers – no longer do you need to fear that you can’t contribute at story, whiteboard, design meeting or functional specification review! Attend those meetings, “humbly inquire”, and let your curiosity and interest guide them into thinking about interesting questions! And if you want to get better at humble inquiry, I’d start by reading the book, and practicing the techniques it describes.