Tableau: Dashboard Usability Study

HCDE 517: Usability Studies

Purpose & Goals

The purpose of this study was to learn more about how beginner Tableau Desktop users experience the dashboard creation and interaction process. Specifically, our goal was to discover pain points and barriers beginners face when using Tableau to create dashboards as well as provide recommendations for improvements and next steps. In doing so, we hoped to provide useful, data driven insights into how the dashboard experience can be improved for beginners thereby making it more accessible to them.

Scope

Our specific focus in this study was to investigate the dashboard creation and interaction process. We scoped user tasks to include only interactions in these processes in an effort to get deeper and more detailed insights. To this end, all worksheets necessary to construct were created prior to testing and given to the participants.

Methods & Procedures

Our team conducted 7 moderated, in-person usability test sessions at the University of Washington Odegaard Library and Sieg Hall. Prior to each session, we asked every participant a series of pre-test questions gauging things like their current attitudes towards Tableau and their use patterns. We then presented them with a scenario which provided context for the 20 tasks they were given. Upon completing all tasks, each participant was asked a series of post-test questions to reflect on their experiences as well as a Net Promoter question. All but one participant were then administered the System Usability Scale.

METHODOLOGY

For our moderated usability testings we used “Thinking out loud”, we wanted to understand the process and steps the users went through to finish the tasks. We also wanted to understand the thoughts they had as they approached and went through the task, this method was successful to understand the process. We also recorded each session and had a note taker observing, this was helpful in the analysis portion of the project to refer back to and also see different perspectives real time as we ran the study.

We focused on having the user completing tasks to create a new dashboard. Having them resize, add, format, filter, etc. different elements of the dashboard. We had about 20 different mini tasks, but focused on creating and setting up the dashboard, set up interactions in the dashboard, and testing the created dashboard.

At the end of each usability study we had them fill out a System Usability Scale (SUS) to understand the measure of the study quickly, we saw the benefits of the easy scale to administer to our user. To understand if they would use the system, the complexity/ease, the function, and also the overall thoughts of the system. The data gave us a at-a-glance snapshot of how usable the system was. We also administered a Net Promoter Score (NPS), to see if our users were likely or not to recommend the product to a friend of relative, this allowed us to understand and get more in depth if the product was easy enough or helpful enough to recommend.

Analysis

After we got all the data we needed for our study, we sat down as a group to look at the data broadly. We noticed patterns throughout our study, but wanted to look at the main points of each user. We each focused on a participant and wrote down each participants main finding on a different color sticky note, to begin affinity diagraming. The affinity diagram was coded by color, and then grouped with similar findings from other users.

Once grouped together we were able to find the main patterns, and findings from the study. The benefits of affinity diagraming allowed us to brainstorm and look at each individual participants main pain points and findings. Then to look at it at a higher level by looking at the data in a big group we were able to prioritize the main findings. It also allowed us as a team to discuss and come to a consensus, working in a group we all had different perspectives of the problem and the findings. So the affinity diagram allowed us to look at it, and hear each others thoughts.

Reflection

WHAT I WOULD DO DIFFERENTLY

We primarily recruited on the University of Washington campus, and used UW Slack channels and mailing lists. As well as reached out to several academic programs to see if other students have used Tableau, because of this it was difficult to find participants. I think it would have been beneficial to reach out to a broader audience, to pick participants and have a bigger pool of data. Since we did pull from a student base, some of them were not as interested because they were worried about their performance of Tableau not being good enough and felt like they would be judged. I think it also would’ve been beneficial to get more participants to perform a pilot run, being new to Tableau myself there was a learning curve I faced myself when running the study if the participant clicked on the wrong button and got completely re-directed. I would also clarify more in the screener to understand if participants used Mac OS versus Windows, since the layout is different some participants were unfamiliar with the Mac Os version since we tested using our personal computers we all had macs. I think clarity on that would’ve allowed some participants to be more comfortable as well as remember what keys and things were placed as they use it normally.

TAKEAWAYS

This project was very interesting to be a part of, because I was an undergraduate at the time sitting in on a graduate level class I was able to learn a lot from the class itself but also the other students. It was a great opportunity to work on an industry based project and to work with a client. The experience of designing a usability study end-to-end, was something I haven’t done before. I learned how to collaboratively and efficiently run a usability test as a team, and to draw meaningful conclusions from the study data. I also got to learn how to design and run a usability test on a complex product, that was unfamiliar to me but ended up learning a lot form users and their insights. urpis.