RiskStorming - Maping Risks with TestSphere

Why RiskStorming?

Beren van Daele and Andreas Faes were chosen to give a workshop at the Agile & Automation days in Krakau. In that workshop, they were going to utilize the famous TestSphere Cards as a tool to create a test strategy. In the end, Andreas sadly couldn’t make it and Beren asked me if I want to jump in. Feeling a bit guilty and thrilled at the same time I agreed.

In our first hangout, we discussed, that we want the created test strategy to be risk-based and I volunteered to look for a nice, visualizing format to generate risks, that would be fun to do during a workshop. Andreas said something along the lines of “this is not easy to find” and he was absolutely right. In the end, I found nothing, that really suited our needs so on a weekend I just put my phone on the table and sorted the TestSphere cards around it only to think to myself that this seems to look like a circle. Thus “RiskStorming” was born.

The idea behind it is to use the TestSphere cards to steer a discussion about product risk and help come up with a test strategy, that actually tackles the identified risks.

How does RiskStorming work?

RiskStorming can help you to guide your thinking towards product risks and how to mitigate them either when on your own or with a group of people. Therefore I will first describe how RiskStorming generally works and then address some topics, which can come up when you facilitate a group session.

(Re)Inventing your test strategy

RiskStorming itself is structured in circles with the application under test in the centre. I found it helpful to have some form of representation of that application on the table, while you go forward, like a smartphone showing the application or any other token, e.g. the company’s mascot. This helps to visualize the rather abstract concept of software. If you are lucky and actually are testing a smartphone app or website you can even access it during the RiskStorming session.

The first circle surrounding the application focuses on “Quality Aspects”, which are covered by the blue TestSphere cards. Think about the most important Quality Aspects for your application and lay them around your representation. The hard thing is not to come up with potentially important Quality Aspects, but acknowledging that not all of them can or will be equally important. Focus. If you use our board we try to foster this by actively limiting the number of Quality Aspects to six.

The second circle focuses on the actual risks. Take a pen and sticky notes then start writing down risks, which threaten the application, specifically those connected to the Quality Aspects you choose to be the most important in the last phase, e.g. if you chose “Security and Permissions” maybe “loss of personal credit card information and social security number” is a valid risk to your application. Put the sticky notes on the Quality Aspects and try to sort them into the ones they belong to.

The last circle deals with risk mitigation. Look at the identified risks and use the full TestSphere card deck to find heuristics, patterns and techniques, which can help you to mitigate them. Since the TestSphere cards are by no means complete you might come up with more ideas than you find cards. Write down your ideas on additional sticky notes, preferable different coloured ones than those showing the risks, and add them. The idea behind using TestSphere cards is to give you new inspirations about how to approach a testing problem, e.g. a lot of testers don’t think about utilizing “Log-digging” for their testing yet this is a very powerful tool.

In the end, you will come up with a mixture of your own sticky notes and TestSphere cards, which is absolutely fine.

Using Riskstorming in a timebox

I believe it is helpful to timebox RiskStorming phases since it makes sure you stay on track and arrive at a result after a given amount of time because the meeting you scheduled for you and your team will end eventually. You can always double check the resulting test strategy after a good night's sleep.
In my experience so far I would spend most of the time on risks and mitigating them and not too much on the Quality Aspects. Here is a setup, that worked now for several rounds:

• Phase 1: Discussing Quality Aspects: 10 minutes
• Phase 2: Finding Risks: 25 minutes
• Phase 3: Finding Risk Mitigations: 25 minutes

Facilitating a Riskstorming session

In addition to knowing the rules and having an idea about good timeboxes, it is also helpful to be aware of questions and struggles participants might have when facilitating a session.

Thinking in bugs, not in risks

More than one group noted down bugs instead of risks, e.g. “order of elements is wrong” instead of “users don’t finish the ordering process” for the Quality Aspect of “User Friendliness”. This can subsequently lead to a shallow testing strategy, which only takes care of very specific incidents. Make sure to remind the participants to come up with risks, not bugs.

Making decisions by reducing resources

In the beginning, we did not give participants a limit on the Quality Aspects, but we changed this once Beren came up with the board. The point is not that there are never more than six important Quality Aspects, but to foster discussions and eventually a decision by the whole group. These discussions give the participants a better understanding of each other's view on the product and what's most important about it.

Giving them time to read the cards

There are a lot of TestSpehre cards. Your participants should have the time to read them properly when trying to assign certain techniques or heuristics to their test strategy. If possible try to make the cards available for them before the session, if not you might want to prolong the “Finding Risk Mitigation” phase.

Make them read the cards

The participants should not only have the time to read the cards, they should actually do so. Look out for people only reading the headline and then applying the card in a wrong way, because they have not grasped the whole meaning, but just assumed they knew what that card is about. Encourage them to take their time and read the cards entirely, especially if a term is new to them.

Don’t restrict them to the cards

It is not possible to cover every technique, every pattern, every heuristic or Oracle, that ever came up in the world of testing with a TestSphere card. This means the participants may come up with ideas how to test for or mitigate a risk they identified, which is not covered by a card. Encourage them to still write it down and put it on the board. Don’t let a finite set of cards get in the way of finding the best possible test strategy.

Make them use the cards

Not restricting to the participants to the cards does not mean they should not use them at all. The idea of using the TestSphere cards is to give people new ideas how to approach a testing strategy. The cards can inspire them to use methods, they have never tried before, e.g. a group of people, which previously approached their testing problems strictly from a business point of view is often surprised that “Log-Digging” can provide them with a lot more options than before.

Be aware if they lack experience in an area

In some sessions, we encountered, that participants did not have much experience with testing in general or certain aspects testing, e.g. security testing. Try to place at least one person in a group proficient in topics, that might come up. If that is not possible be prepared that you will have more coaching to do.

Use the board

Beren came up with a board, that helps you to keep in mind what the different phases are about and also makes it more fun since it amplifies the gamified character of RiskStorming. Then Thomas Harvey made it beautiful. If you print it in the right size the cards perfectly fit the slots.

Here is the board in different sizes for you to download:

TestSphere Riskstorming 4xA3 - A1

TestSphere Riskstorming 8xA4 - A1

TestSphere Riskstorming A1 (can be printed A3 or A4)

TestSphere Riskstorming A3

Impressions

Here are a few pictures of what this actually looks like:

The power of @TestSphere, lots of testers talking testing. #TestBash pic.twitter.com/eexaFLCr6B

— Richard Bradshaw (@FriendlyTester) 25. Oktober 2017

We'll be doing a riskstorming session at the wednesday pre-meetup of TestBash Manchester! @_Testheader, @Marcel_Gehlen & @EnquireTST pic.twitter.com/7rDSntSJFi

— TestSphere (@TestSphere) 9. Oktober 2017

Riskstorming like they've been doing this since forever!! Awesome work!#testbash Munich pic.twitter.com/ITeKt6PvDM

— Beren Van Daele (@isleoftesting) 7. Oktober 2017

Riskstorming with @TestSphere. An experiment gone right.

Did I mention @Marcel_Gehlen is brilliant to come up with the format? pic.twitter.com/tWBsAjvkSz

— TestSphere (@TestSphere) 26. September 2017