Exploring with QA

Our QA process, how we run exploratory testing, and why it works

By Elbat Mesfin

At TribalScale we practice XP (Extreme Programming) — a way of work that is in harmony with our personal and corporate values — and Quality Assurance (QA) engineering is no exception. QA applies XP values and practices; we constantly communicate, show mutual respect, and work to our common goal of delivering amazing products to our clients. These values are demonstrated in the way we test at TribalScale. We go further than simply testing features, we engage in exploratory testing. Exploratory testing opens up conversations that go beyond the feature sets and allows QA to provide feedback to the project teams.

What is exploratory testing?

Exploratory testing is a form of manual testing. The difference between traditional-manual and exploratory-manual testing is that with exploratory, the test engineers use their creativity and test beyond the delivered stories. They go outside of the box and push boundaries with the app, playing with new user flows and environments to expose unforeseen flaws. It’s an essential part of our QA process, as it allows developers to discover vulnerabilities in their code so that we can deliver an exceptional product to our clients.

How do we do it?

Exploratory testing does not mean RANDOM testing! Often, it’s confused with “let me click some buttons and see what happens.” Instead, a test engineer will think of multiple user flows that may not be part of the current user acceptance criteria of stories. For example, if the feature created was, “if a user is on a login page, and selects sign-up, they should be taken to the sign-up page.” Not only do we verify that, but we also think of a different flow that uses this new feature. For example, “what happens when I select sign-up, then go back to the login page and select forgot password? Am I taken to the correct page?” or “if I lose network connection while being redirected to the sign-up page, am I given the correct error?” Since developers implement functionality based on stories written by their product manager (PM) (which were all previously discussed and prioritized with the client) any flows that deviate from those stories could be vulnerable. Our QA process focuses on these vulnerabilities to find issues quickly.

How TribalScale does it differently

At TribalScale our QA team goes a step further. We always put ourselves in the end-user’s shoes. This way, we can best critique the end-user experience. We can come across parts of an application that function properly, but with the perspective of an end-user, we might also see room for improvement. TribalScale QA provides suggestions and comments that go beyond testing functionality. We believe in working as one team, which means everyone does their part for the success of a project. Unlike other organizations, QA at TribalScale has the chance to provide input, which often results in new features and improves the app overall.

QA’s fresh perspective

When you’ve been staring at the same code base for over a week you start to develop a very narrow focus. As the expression goes, “you see what you choose to see.” When developers create a new feature, their focus is on correctly implementing that functionality and it can be hard to take a step back and get the whole context of the app. That is where QA comes in! QA provides fresh eyes on the application, and through exploratory testing, we have the chance to see how the new feature affects the app in its entirety.

Our “Charters”

When our test engineers are engaging in exploratory testing, we use “Charters,” which are helpful tools to organize our thoughts. A charter is a means of encompassing 3 essential pieces of information: 1) what we are exploring, 2) what tools or resources we are using, and 3) what we are trying to discover.

So a general template is:

Explore (target) with (resources) to discover (information).

For example:

Explore player with a poor connection to discover pausing issue.

Beyond organizing our thoughts, Charters help us clearly define reproduction steps and write concise bugs for our developers. Implementing Charters allows our team to focus testing on key areas of the application that have potential vulnerabilities.

Conclusion

There are many benefits of exploratory testing. It gives our test engineers the chance to take a holistic view of the products being tested. Through tools such as “Charters” we are able to better organize the way we explore, which maximizes our chances of finding issues quickly and effectively. Exploratory testing is great to measure a tester’s skill and creativity, just as other Agile XP values emphasizes those qualities for developers.

author

About the author

Elbat is a fourth year student studying Computer Science at the University of Waterloo. She is working as the Fall 2018 Software Test Engineer Intern at TribalScale.

Join our fast growing team and connect with us on Twitter, LinkedIn, Instagram & Facebook! Learn more about us on our website.

Visit Us on Medium

You might also be interested in…