Foundational testing to decide site features
I lead qualitative testing that resulted in the company aligning on a product roadmap for the next year based on end user value balanced with development effort.
Rocket Auto Dashboard Qualitative Testing
The purpose of the research:
To investigate if the company should design an online dashboard for their online used automotive buying process and if so, what features would bring the most value to the end user.
Result of testing:
Proved the value of a dashboard to our founder resulting in more financial support.
Helped the team decide the order for the future product roadmap for dashboard features
Created company excitement for future UX testing.
Methods
Survey,
Interviewing,
Card sorting,
Qualitative data analysis,
Usability testing on Usertesting.com
Timeframe
2 week for planning
1 week for testing
2 weeks for analysis
1 week for summary design for presentation
Work I Completed
Collaborated with project managers, development, Design, and product on the research plan.
Created the test with assistance from the Rocket Mortgage ReOps team.
Synthesized all of the data
Created and presented the findings to the entire company.
Research Planning
Since I was the lead and only researcher in the small startup I took the initiative for startup and lead all the research planning meetings. These meetings were divided by the teams for more laser focused workshopping.
Company wide collaboration
After putting together a quick research plan I met with the the company executives to review and gather insights on the following:
What assumptions do we have about our end users?
What is our target end user?
What dashboard features are being considered?
What KPIs would we improve with this research?
Development Effort Scoring
With the list of features the team was considering I met with the head of development and their scrum master to gather insights on the following:
What dashboard features do they think should be included?
Using a difficulty scoring system, research how much effort it would take to create each feature.
Rank each feature on the difficulty level ranking system we created.
Review with the ReOps
As a startup, I had to make a proposal for the testing to the founder, Rocket Mortgage’s, ReOps team. That process included:
Reviewing the test and screener questions with ReOps for clarity, and bias.
Letting them know how many users we wanted to test with. For this test it was 15 users.
Creating the test online, and synthesizing the results
Once the research plan was reviewed and agreed upon with each the team I to began to create the test on Usertesting.com to gather end user feedback.
The Test Itself
The test was a collection of the following activities.
Screener questions
To make sure the user was looking to purchase a vehicle within the next month.
The activities included
Survey questions,
A usability test of Rocket Autos landing page and sign up process,
A card sorting activity to weigh the value the end users saw.
The results and Synthesis
Since the test had a card sorting activity the test called for 15 user, and 1 pilot to go through the test. The average time for the test was around 20 minutes.
My test review process:
Create the chart above with the questions, end users and their answers to the questions.
Color code the users answers while filling out my synthesis chart. Red for negative, green for positive, orange for quotes of interest, and yellow for general answers.
Video clipped quotes of interest.
Create suggestions
After reviewing all 16 users I found and relayed the following to the team:
Users are more willing than expected to give more information up front, so we can ask more about their trade in.
User’s expected a dashboard with a tracker status to provide information to the company on what vehicle they are looking to purchase.
Knowing the effort score of each feature I recommended the the team prioritize creating a tracker and vehicle details as they were low effort but were viewed as providing high value.
Summarizing the Results
Designing the research summary
My process is to create a powerpoint that relays the key results in the beginning and then goes deeper into the results to prioritize time considering the busy startup environment. Other things I include are:
Graphs and infographics at every opportunity for quicker understanding by users.
A page to talk about opportunities to improve found during testing from the usability test.
Explanation of why we are running the test and the reason behind the sample size as the company was new to UX research.
Videos of our potential end users to create a greater impact among the viewers from empathy with the user.
Presentation and results
The presentation was presented with my design team, the stakeholders, and the CEO of the startup with the entire company at our quarterly meeting. It was met with company wide excitement since it resulted in:
Process updates, as users were found to be willing to provide more info online, upfront, which allowed the company to process them faster.
Product roadmapping for the next year as the company took my suggestions and then decided based off the research findings,
We received more funding from our founder Rocket Mortgage as we proved our end users expected a online experience more than the founder expected.
My research gained awards and additional research support from our founder for the research summary design which can be reviewed here.