Launching a UX Lab

Situation

My teammates and I were able to successfully advocate for, and launch an in-house usability testing lab at our World Headquarters in Kansas City. This was pre-COVID and preceded the widespread use of remote usability testing. As the primary researcher for the product, I was responsible for planning and preparing the test - anything that went on within the testing room or the product itself. My teammate was responsible for the operations of the lab including the means and method the data would be collected. The product owner was responsible for recruiting participants and ensuring their arrival. We worked together to pull this off with 3 weeks notice.

Purpose​ of study

Validate changes made to designs through Rounds 1–3 of remote usability testing. Verify mental models around key workflows, presentation of information, and interaction with the software.​

Research Questions​

• How intuitive are the features of the Offline Rehab app as currently built?​

• How well are users able to complete the tasks presented to them?​

• Is the information being presented useful to the user?​

Core Workflows​

• Logging in and downloading PHI​

• Editing and canceling an appointment​

• Return to app via PIN after putting the device to sleep​

• Navigating to the Time Tracking area​

• Editing time tracking and mileage information

Method

A live, in-person usability test was conducted to confirm the design, development, and interactions of the Offline capabilities of the Rehab product for WellSky’s client, Rehab Without Walls. ​

During the test, participants were encouraged to provide feedback while completing tasks. A set of standard questions were administered after each task scenario. Each session was 1 hour long. Sessions were moderated by a UX team member and streamed for the development team. 

High Level Findings

Priority 1: The Time Tracking page left every user with questions​

• 4/5 users frustrated that inline edit not a function ​

• 5/5 users track time in 15 min. increments​

• Ave. Time on Task – 225 sec. (Longest)​

• Most users have a difficulty consistently tracking time – don’t always remember​

Priority 2: Edit/Cancel Appointment​

• 5/5 users' first impulse was to tap-and-drag appointment to desired reschedule time ​

• Every user, regardless of tech literacy, understood this gesture intuitively​

Priority 3: Login, Create Pin, Download PHI workflow​

• 5/5 users frustrated by not being able to backspace when typing in PIN​

• 3/5 users didn’t read instructions or notices​

• 4/5 users misinterpreted the language/messaging in the login workflow​

• Ave. Time on Task – 214 sec. (2nd longest)​

• Industry-standard UX heuristics support reducing steps in login workflow​

​

RESEARCH RECOMMENDATIONS

The objectives of the usability testing is to be able to give the product team actionable insights. The following actions were recommended and prioritized based on the above findings from usability testing.

Learnings

The lab was completed about a month before the world shut down due to the COVID-19 pandemic. Our team did a fantastic job pivoting quickly to take all of the usability testing we had scheduled and converting them to remote testing. We have since learned after a year of conducting remote tests that having an in-person lab was fairly superfluous. While an in-person lab was a great way to demonstrate to prospective clients that we put a lot of effort and weight into gaining insight into the user experience, it didn’t significantly impact the quality of the data we could collect from users. In retrospect, we likely could have used our budget more constructively elsewhere.