Why User Testing Services are Useless
Building software that people enjoy using isn’t an easy task, and it takes a ton of effort.
It’s critical that you get consistent, honest feedback to help you create a better experience for the end user. In an effort to create this feedback loop, user testing sites have real people record themselves visiting and interacting with the site.
While I want to love what they are doing (I certainly agree with their mission), I find myself despising these services and finding them to be completely useless.
Why? Let me explain…
Context is King
First, your customer won’t visit your website, press a “record” button, and then give 15 minutes of feedback. They have a purpose to their visit, and they are on your site to perform a specific job.
As someone who desperately tries to put myself in the mindset of a first-time user, I still fail miserably. That’s why I don’t trust someone who records 10-15 videos a day to give me valuable insight.
There’s no context to the visit. *It’s empty feedback. *Now, some user testing companies have figured this out and allowed companies to filter users by demographic information, but once again, the context for why they are visiting isn’t there.
Fluffy Feedback
This next section depends who is giving feedback, but after watching more of these videos, I was amazed at how fluffy the feedback was. Take a look at this video below. At Safari, we are fully aware that our website needs a major overhaul, yet listen to how nice Barbara is!
While I appreciate nice people, I want brutal & honest feedback. It’s like asking your best friend to review your startup idea…they will be nice and make you feel good about yourself, but that’s exactly what you don’t want.
Why is it so popular?
To be honest, I’m not sure why these sites are so popular, but I have a theory that boils down to this:
**It’s not fun to get honest feedback, and it’s time consuming to recruit the right people to participate. **
User testing sites make it easy to get feedback, and the “best part” is that you hear what you want to hear, not what you need to hear.
If you spent a few months perfecting a design, and someone tears it apart, that’s not fun at all. It doesn’t feel good, and hurts your pride. You might even get a bit defensive. People in general pursue a path of least resistance, and user testing is one of them.
Alternatives?
I could rant about this longer, but here’s what I suggest if you are trying to get valuable feedback from customers. To me, the key is to collect feedback in the user’s natural habitat, with minimal intrusion. My recommendations below start with the most intrusive ways of collecting feedback, to the least intrusive.
Phone
Yes, people still use phones. I suggest soliciting feedback from people that use your service – use analytics software to find users of varying usage, and interview them. Dangle a carrot for their feedback (i.e. – a gift card) and show that their opinion matters.
Services like Intercom make it really easy to collect feedback over email. If a user has performed (or not performed) a specific action, send them an email and ask them why. Don’t abuse this though.
Qualaroo
I’m not a fan of “pop-ups” but I find Qualaroo to be one of my favorite tools for collecting feedback. Sure, some users won’t like it, but the trade-off of getting valuable insight (in context) will enable you to make the experience better for everyone.
Heatmap/Video Tracking
If you are looking for aggregate data about how visitors are interacting with your site, I suggest using a heatmap tool like CrazyEgg or Inspectlet.
To wrap things up, context is king. A user who gives feedback with context is gold. Do everything you can to get this feedback. It won’t be easy, but it will be worth it.