USABILITY TESTING REI ONLINE REVIEWS

What’s most important in an online review?

Background

What works well about REI’s current set up? What opportunities exist?

The goal of our project was to perform exploratory usability testing on REI.com’s online review section. REI wanted to know what components of an online review were important, as well as a better understanding of what worked well and what opportunities existed with their current configuration. At the time, REI was using a third party to assist with their online review section and they were planning to change providers shortly after our study.

Working in a team of four, I was able to be an active participant in the Usability Testing process from end to end. My responsibilities included: being the liasion between my team and our client, conducting user research, screening usability participants, moderating usability test session, analyzing usability data, reporting results, and acting as an overall project manager checking in frequently with the team to ensure we were on track with each of our deliverables.

My role

Working in a team of 4, I was the project lead and primary contact with REI. We all took turns facilitating the usability sessions and writing up our findings.

Skills used

• Usability Testing

• Project management

PRELIMINARY RESEARCH

We began by performing a competitive analysis on various online sites and their online review sections. We emphasized finding sites similar in nature to REI as well as sites that used the third party vendor REI would be switching to.

We also created and sent out a general online review survey to begin understanding what users felt was important in an online review as well as gather feedback on REI’s online review section. This helped us narrow our focus for usability testing.

USABILITY PLANNING

With initial research complete, we began creating a usability test plan and a usability study kit. Our usability test plan served as charter document for us, representing our collective decisions about the best approach and consolidating details so that the overall approach can be understood by interested parties, including REI. Additionally, our test plan demonstrates our strategic thinking and inventive approach. The test plan included the following:

• Purpose (including goals/objectives)

• Research Questions

• Participant profiles

• Method

• Task lists (with details about starting states, successful completion conditions, etc.)

• Test environment, equipment, and related logistics

• Facilitation approach

• Data that will be collected

• Reporting (document, presentation, etc.)

Usability research is guided by a plan, which typically specifies a set of activities that are mediated by texts, forms, and other instruments. These elements are the tools that facilitate the testing activities envisioned by the team. Since we were going to be executing the test we planned, we created a test kit to support our hands-on work during our testing sessions.

Our test kit included the following:

• Screening questionnaire

• Consent form

• Scripts used during facilitation

• Pre-test questionnaire

• Post-task questionnaires

• Data-logging / note-taking forms

With our deck complete, we had nothing left to do but recruit for participants. We sent out our screener to gather usability participants matching our criteria. Each user was contacted via phone before being brought in to our downtown Seattle office. Users were compensated with a gift card for their participation.

USABILITY TESTING

Our usability testing took place over 3 days. We conducted 1 pilot study several days before conducting 4 more tests. The purpose of our pilot was to ensure we had all the technical equipment working as expected and that our test plan components were what we were expecting.

I was the moderator for 2 of the 5 sessions, taking notes during the other 3 sessions.

Findings and recommendations

When our usability testing was complete, we help a debriefing session to begin drawing conclusions. We started with affinity mapping general themes each team member noticed throughout all sessions. After we sorted these, we took our high level findings and went back to our research to confirm our data.

After this we were able to return to our original question – what’s most important in an online review? – and say that we learned the following:

• The Overall Product Rating is important in weighing a decision

• Negative Feedback – 58% of those surveyed strongly agreed negative feedback is important

• Real Customer Responses – Based on our usability testing, users want reviews to come from actual product owners and speak to their needs

• General reviews – 65% of those surveyed strongly agreed reviews are important

In addition, we came up with the following findings and recommendations:

Finding 1: Participants were interested in seeing specific star rating reviews for products. Some of our participants would click on the overall star rating hoping it would show the specific rating reviews they were interested in.

Recommendation: Make the overall star rating clickable allowing users to find specific star rating reviews.

Finding 2: Participants mentioned disliking the Review section being in a separate tab next to the Description section and Specs section.

Recommendation: Have the Review section appear on the same page as the Description and Specification sections.

Finding 3: Participants really liked the Pros and Cons table on the top of the Reviews section. Some of our participants would click on the keywords listed in this section hoping that it would direct them to the reviews that mentioned the keyword they were interested in.

Recommendation: Link the keywords listed in the Pros and Cons table to the associated reviews.

Client handoff

We presented our findings to the REI UX team and they incorporated the work into their 2016 complete site redesign. Check out the reviews page for the MSR Pocket Rocket here.