Differences between Moderated & Unmoderated Usability Testing

Today’s post is one of the most interesting topics that we will be discussing in our blog (and one of my personal favorites). 

Usability testing and its great advantages have been known for a long time now. It was Mr. Jacob Nielsen that first stressed the importance of performing usability tests in our designs or prototypes. He specifically said that 5 users are, most of the time, more than enough to identify potential issues and pitfalls in the interaction between your users and your designs. He also stated that focus groups -widely used by marketers in the past- were not as efficient as user testing because people can easily mislead you. 

Of course, if you are into UX you might already know this stuff but sometimes getting back to basics could be a lifesaver. 

It was only last week that a UX “pro” tried to convince me that A/B testing belongs in the usability testing sphere.

Anyway, let’s get back to our question:

Moderated – Unmoderated testing dilemma

First things first.

What is Moderated Usability testing?

The term Moderated Usability testing implies that a moderator is around to better control the whole test. The moderator is preferably someone knowledgeable and experienced in such kinds of tests and is the one that has full responsibility for the successful guidance of the user across the whole test.

As we are not going to expand on this topic right now, you could read this guideline to learn more about conducting moderated usability testing.

What is Unmoderated Usability testing?

Unmoderated Usability testing, on the other hand, does not require any moderator whatsoever. The participants get specific guidance before taking the test and they complete it on their own at their convenience wherever they want to. 

Which one to prefer?

To better address our dilemma, I would suggest that these 2 types of testing should be considered complementary, not contradictory. 

Moderated testing on “test”

1. The main advantage of employing the moderated testing method is that you don’t wait until your participants finish the test to see how they interacted with your prototype. This allows you to further ask your users about the actions they took or didn’t take. 
2. Another advantage is that this type of testing is good for companies that care about privacy.
3. Involving users for more than 30 min will surely increase engagement and will create anticipation. 
4. Moderated usability testing is great for early-stage designs. 
5. It is extremely expensive and requires a lot of resources.
6. It requires careful planning ahead of the test.
7. Hawthorne effect
8. The long sessions might irritate your audience.

Unmoderated testing on “test”:

1. Extremely fast and efficient.
2. Very cheap.
3. Avoiding the Hawthorne effect.
4. Ideal for design comparison.
5. If done remotely It will not take your privacy concerns seriously.
6. You won’t have the opportunity for follow-up questions 
7. Finally, until today the participants of such tests were professionals that needed specific hardware/software to complete the tests making the process far from efficient.

Conclusion

To conclude, it is a fact that one size does not fit all. My recommendation would be to constantly try different things until you find what works best for you. For example, you could do moderated usability testing combining Useberry and Skype.

Get creative.

Just make sure you are getting closer to your final goal: making a “product” that rocks!