Voice User Interface for Illinois Voter Registration
OVERVIEW
Project Summary
The right to vote is a fundamental part of the American democracy but, according to a 2016 report by Reform for Illinois, 24% of the 10.2 million eligible Illinois voters were not registered for the 2016 presidential election. That’s an additional 2.4 million votes. Furthermore, the voter registration process can be challenging. Our team looked at current voter registration processes in the State of Illinois and designed an alternative service in the form of a voice user interface (VUI).
CLIENT
HCI Capstone
TOOLS
Voiceflow
Amazon Alexa
Trello
OVERVIEW
Goals
Users will find the flow of the registration process natural, allowing them to easily register to vote.
Benchmark: A 50% success rate during Wizard-of-Oz testing, increased to a 75% success rate in the final round of usability testing.
Design a conversation that is tolerant of dialogue variations.
Benchmark: Throughout the development and testing process, increase the amount of accepted utterances by 35%.
Users are as satisfied using a VUI to register to vote as they are with the Illinois State Online Application.
Benchmark: Meet or exceed the current online application, comparing both systems using a SUS score.
RESEARCH
Competitive Review
During the exploratory research phase the team reviewed competitors and alternative services that assist in the voter registration process to influence the approach and help benchmark success. After analysis, it was determined that the VUI voter registration skill should include the following features:
Auditory affordances that let users know where they are in the process.
Inform users of the required materials they will need to complete the process.
Inform users of the requirements to vote in Illinois and check that they meet these requirements
Guide users through the process, using easy to understand directions and questions
Validate the information that the user provides to promote error prevention
Help users identify, diagnose, and recover from errors
Allow visibility of system status, clearly communicate to users what the system’s state is
Provide users with clear deadlines for voter registration
Promote user control and freedom; allow users to move throughout the process
in a non-linear manner if necessary (go backwards to change information)
Allow users to review information before submission
Provide affordances to reduce cognitive load, including breaking the process down into manageable portions
RESEARCH
User Flow Chart V1
From research into the national and Illinois voter registration requirements, a user flow chart was designed to visualize task requirements for the voter registration process.
TESTING
Table Read
The table reading was the first chance to review the voter registration process with real users. Following the first iteration of the user flow, a sample dialogue was developed. Six users were recruited to read sample dialogues aloud with a moderator to catch low-hanging fruit in communication issues and get a feel for the conversational style. Below is a sample of our findings:
SECURITY CONCERNS
Four of the six participants that we spoke with brought up security as a major concern when using this type of system to register to vote. The concerns ranged from not trusting a company like Amazon, to wondering where the data would be stored and who would have access to it. Users needed to be informed about how their data will be transmitted, stored, used, and who will have access to it. The system should also check if the user is in a secure location to transmit their data verbally.
NEED FOR PAUSE
Four of the six participants mentioned that they would have likely timed out the process when asked for a Driver's License ID number as this is something they would need to locate. These four participants also noted that they would like to know what type of information they would need before they started so they could get it. This system needed the added ability for the user to pause the process and, in areas where users might need extra time, have the system ask them if they need a to pause. Users must also be informed about what they will need to complete the registration process.
AWKWARD LANGUAGE
For the most part, the flow of the “happy path” (user flow) made sense to our participants, but there were still parts of the process that confused them or felt awkward. Four areas of concern were identified. These sections needed to be updated and the new language analyzed in the Wizard-of-Oz testing sessions.
QUICK AND EASY
Participants commented on the ease of use and were happy with the time it took to complete. Five of the six said this process took less time than expected.
“It was easy and it was short, which is ideal”
Participant 5
TESTING
User Flow Chart V2
Taking the insight from the table read, as well as further research, the user flow was updated.
TESTING
Wizard-of-Oz Testing
Using the updated user flow chart and insight from the table read, a voice script was developed for this testing session. Each team member was given a specific role; one was the designated moderator, another member was the designated Alexa operator (who would launch Alexa like voice replies based on the participants responses), and the third member acted as an observer and note-taker. Below is a sample of our findings:
ISSUES WITH ID QUESTIONS
In order to minimize questions asked, we asked the user to say either the last four of their social or the driver’s license ID number and the system would decipher which one they input. Users expected to choose between the two options and this approach confused them. Users wanted to choose between the two options before inputting this information as this felt like a more natural flow.
LENGTH OF SPEECH
There were two substantial updates to the start and ending of the process after our table reading sessions. Knowing that people were curious about their data, what information they would need to provide and more about the system, we added a disclaimer at the start that had all this information. We also updated the affirmation section at the end to sync up better with what Illinois Online Voter Registration Application website. Users noted issues about the length of these text. This text needed to be broken down into digestible sections and meet a more conversational flow.
RESEARCH
User Flow Chart V3
Taking the insight from Wizard-of-Oz testing, as well as further research, the user flow was updated.
RESEARCH
User Survey
A user survey was used to identify natural utterances and confirm user insights on a larger scale than the table readings and Wizard-of-Oz test. Below is a sample of our findings:
SPEED IS IMPORTANT
70% of the survey respondents expected that the VUI voter registration process would take less than 10 minutes. This made the 6-minute average completion time from our Wizard-of-Oz test seem like an important first milestone.
MINIMIZING CONFUSION
The survey asked participants how they would respond to Alexa if they got confused with one of her questions. We received 20 unique responses. This method showed us how important it is to have a high level of variation on how a user can express confusion to Alexa.
SECURITY AND DATA CONFIRMATION
One theme that has been present in every usability method so far is the concerns around data collection and storage. The responses to the survey were no different and 12 respondents said this would be a concern of theirs with this type of technology.
QUESTION VS PROMPT
Survey respondents were asked how they would start this VUI Skill. 14 of the 20 respondents asked a question such as “Hi Alexa, how can I register to vote?”, “Alexa, tell me how to become a registered voter? and the other 7 responses were in the form of a command like “Alexa I'd like to register to vote”. When programming the ways a user can start the skill, it needs to allow users ask a question of Alexas as well as accept a command. There also needs to be a high level of tolerance for accepted utterances.
Users are more likely to start this skill with a question
There are many ways to express confusion
Security and data confirmation could hold this technology back
It needs to be fast
TESTING
Comparative Usability Testing
The comparative usability test was used to gain insight into two things: additional insights about a hi-fidelity prototype and review performance against the current Illinois online voter registration system.
The comparative usability test was a within-subjects study, with 8 participants. The participants asked to complete the voter registration process using both the VUI prototype and Illinois online voter registration website. Below are a sample of our findings:
COMPLETION TIME
The average time to complete the VUI prototype was five minutes and fifty-eight seconds. The average time for the clickable prototype was seven minutes and thirty-four seconds.
NEEDED INCREASE IN TOLERANCES AROUND ID TYPE
The only breakdown in the VUI system was the need to increase the accepted utterances when the user is choosing which type of ID to input. This bank of accepted utterances will include those discovered during this test and there will be an implementation of a system that directs users to accepted utterances if they have issues with this question.
SUS SCORE
The average SUS score for the VUI was 84.4. The average SUS score for the clickable prototype was 57.2.
ONLINE FORM FRUSTRATION
Seven of the participants thought the IL Website was outdated, frustrating, and asked for too much information. This means that there is definitely user interest in an alternative way to register to vote.
REFLECTION
Goals Revisited
Users will find the flow of the registration process natural, allowing them to easily register to vote.
Benchmark: A 50% success rate during Wizard-of-Oz testing, increased to a 75% success rate in the final round of usability testing.
This goal was partially met met, there was only a 100% success rate during the Wizard-of-Oz Testing but, during the final round of usability testing, this dropped to 62%. The section where the user is choosing which type of ID to input caused 100% of these failures and will be altered (as discussed in the comparative usability testing section.
Design a conversation that is tolerant of dialogue variations.
Benchmark: Throughout the development and testing process, increase the amount of accepted utterances by 35%.
This goal was met, there was at least a 50% increase in accepted utterances for all sections. However, this still needs to increase as noted in the comparative usability testing section.
Users are as satisfied using a VUI to register to vote as they are with the Illinois State Online Application.
Benchmark: Meet or exceed the current online application, comparing both systems using SUS evaluation.
This goal was met, The average SUS score for the VUI was 84.4. The average SUS score for the clickable prototype was 57.2. Furthermore, the VUI registration process took users less time: The average time to complete the VUI prototype was five minutes and fifty-eight seconds. The average time for the clickable prototype was seven minutes and thirty-four seconds.
REFLECTION
Future Work
Fix the issue with the ID selection section, including add more utterance tolerance and reduce risk of user ending up in a loop. If the user’s answer is not accepted by our system, they will first be asked to repeat their answer. If it is not accepted for a second time, the system will guide them with specific answers they need to say in order to progress.
Increase utterance tolerance
Improving information security will involve finding the most secure way to link the VUI system to the government voter registration system.
Learning the prototyping process within Voiceflow had its challenges, further exploration of this platform will be required.