Download Keynote presentation to hear audio
As a part of our class, employees from the company Focus Brands came to us with a prompt: figure out a way to best implement voice interaction in the context of Moe's Southwest Grill. Since this was a very open-ended problem because of the facets of voice interaction that exist, our team had to find ways in which voice interaction would be useful to Moe's in a variety of contexts.
Our team came up with Burrito Quest: a voice interaction storytelling game that is played on a device such as an Amazon Echo. This entertainment medium not only provides an interesting story, but also provides rewards points and the opportunitiy to win a free burrito upon completion of the game. This game would inspire interesting PR for Moe's because of this fun use of voice interaction technologies, as well as provide customers with great benefits.
• Created task analysis and summaries of popular Alexa Skills for competitive analysis
• Created and distrubuted survey
• Helped with concept creation and fruition
• Assembled design guidelines using Alexa's Voice Design Guide for the heuristic evaluations
• Ran the interactive user demo and feedback sessions
• Bang Tran
• Rachel Feinberg
• Suyash Thakare
• Survey for Moe's Customers
• Competitive Analysis
• Empathy Maps
• Follow-Up Survey from Pivot
• Concept Creation
• Feedback Session 1: Choosing a Concept
• Feedback Session 2: Improving Chosen Concept
• Heuristic Evaluations
• Interactive User Demo and Feedback
We took our background research on what we understood about Moe's Southwest Grill from our partnered clients, and conducted a survey to understand more about what customers enjoy or dislike about the Moe's dining exprience, as well as opinions and usage on voice interaction technologies and online food ordering experiences. We split these two ideas (online food ordering experiences and voice technology opinions) into 2 seperate surveys that we sent to 2 different sets of equal customers, since there was a lot of data we wanted to collect, and we did not want our participants to exprience survey fatigue.
For Moe's Experiences:
• Customers really like the ease, efficiency, customization, and visibility in the Moe's in-store ordering exprerience.
• Customers really dislike the long wait time and ordering difficulties in the Moe's in-store ordering experience.
• Customers really like how easy the Moe's app is to use, and enjoy the rewards points they can obtain through the app.
• Customers dislike the unreliability and interface difficulties they can have on the Moe's app.
For Voice Technologies:
• Most people found voice technologies more complicated and inconvenient to use than normal touch interfaces.
• Most people used voice technologies in their home or car, and were mostly alone or with family and friends.
• Most people used voice technologies for phone calls, text messaging, music, searching, timers, and weather.
Taking our survey results, we decided to delve into the realm of Alexa Skills and popular voice ordering technologies in order to understand what works for these technologies and how our team could gain a better understanding on how Moe's can fit into voice technologies. We also conducted task analyses on some of these systems to help better understand how they worked.
• Voice interaction technologies can be extremely complex or extremely simple
• Voice interaction does not have to be disembodied—it can be grounded with visuals
• Voice interaction can be fun and interactive in novel ways
• Chunking information in voice interaction technology can save time
• Linking voice technology to an existing account solves some logistical issues like payment and addresses
• Offering very important features upon start-up can help users be aware of what’s possible
• Use familiar attributes can help users better familiarize themselves even when using a new technology
Taking our survey results, we also created empathy maps to understand two different types of groups: Moe's customers and voice interaction users.
After discussing the results from our competitive analysis, we realized that there was a fundamental issue with the way we were approaching this problem. Currently, we had been approaching the problem from a "top-down" approach, that is, figuring out how Moe's can fit into voice. After lots of discussion, we knew that Moe's could really only fit into voice through only a few means - including food and ordering. Based off of our research, we knew that voice ordering was not very well received, so we were stumped on other ways that Moe's could fit into voice.
After this discussion, we decided on a better approach. It made it easier for our team to think of design ideas based off of our research if we looked at it from a "bottom-up" approach, that is, figuring out ways that voice can fit into Moe's. This mentality freed ourselves to think outside of the box on the ways that popular voice technology ideas can be incorporated into the Moe's brand.
From this point forward, we fixated our research on this new approach, which would help us to more freely think about the problem and come up with solutions.
After our pivot, we decided to conduct another survey to ask more about the "Why" behind voice interaction, instead of focusing our energy on Moe's experiences and food ordering. We had 103 participants for this survey.
• Our voice technology needs to be simple. This is because participants most often use voice technologies for simple tasks, such as playing music, checking the weather, asking questions, calling and texting, asking for directions, and setting an alarm
• Our interactions do not necessarily need to be private, since participants use voice technologies both alone and with people
• Convenience and entertainment are two key reasons why our participants used voice technologies, so catering our design concepts around these ideas would be best
We designed 3 concepts in order to conduct feedback sessions for which concept potential users would like best.
In this feedback session, we ran through the three concepts with our participants using a script interaction involving one of our team members as "Alexa" and the participant as the user, using the scripts and pictures found above. Here are our results.
• Could relate to online ordering
• Not sure when using voice for ordering could be useful over traditional ordering
• Simulation of the Moe’s in-store experience
• Visual component allows food ordering without voice interaction
• Voice interaction makes the process of ordering online tougher
• Listening to a lot of unnecessary steps sometimes
• Personalizing the experience
• Flexibility in navigation
• Enjoyed the experience, particularly the musical components
• Percentile system was confusing
• Earning reward points
• “Corniness” and strong connection to brand makes it more fun
• Confusing percentile system
•Need to own an Alexa-enabled device to play the game
• Using a ranking system instead of the percentile system
• Include an integration with the Moe’s mobile app
• Fun to engage in (lots of smiling/laughing during the session)
• Defining your own path in the story
• The minigame component of the story
• Length of script in between interactions - lost engagement
• Increased cognitive load due to the absence of a visual interface
• Remove Moe’s daily deals at the beginning of the interaction
• Break up the script into smaller, more digestible parts
In this stage, we fleshed out the Burrito Quest concept some more in order to test the overall concept with participants.
We conducted a Wizard of Oz scenario using a voice-recorded script and an audio puck simulating an Alexa Echo. The participant interacted with the puck as if it was an Alexa, and the "wizard" controlled which audio recordings (based off of the prototype at the top of this page) were played based on the participant's words.
Portion of the script
We used this general outline for the script. With this outline, it gives an "illusion of choice" for the interaction, allowing us to simplify our prototype while also demonstrating freedom of choice for the participants interacting with our system.
• Overall participants were able to successfully interact with the concept
• Participants laughed/smiled while listening to dialogue
• Generally well received and enjoyed
• Some feedback indicated there was too much narration
• Likes: Liked the presentation of the concept, particularly in terms of story and sound
• Dislikes: Primarily disliked the length of the narration and low frequency of user interaction
• Story: Story flow and progress made sense to participants
• Interaction: Participants were able to understand what input was required for each interaction
• Generally easy to stay engaged with concept, though this could have been hindered by the infrequent interactions
• Only received feedback from one participant: increase frequency of interaction, integrate visual component, revise points system
• Shootout: Generally well-received, added an exciting element to the concept; some felt that the music lasted a little too long and could be shortened
• Minigames: The concept of minigames was well-liked; liked earning points and how they could fit in with the story
• Liked the ability to earn points, but there was confusion about how they worked and how they were presented
Taking this concept feedback, we constructed a more detailed prototype to use in the usability testing phase. We kept the Wizard of Oz scenario since it was cheap and quick, and would allow us to make easier changes to the prototype while still allowing it to be customized to the voice and atmosphere we wanted to create.
The prototype can be found by downloading the Keynote presentation at the top of the page. It's functionality can be used by following the flow of the presentation based off of the user's responses to the story. An example of this flow is below.
During our concept feedback, many participants stated that a visual component would help increase engagement in the story. Therefore, we decided to create a visual component in the form of a physical map in order to test how well a visual component would enhance the user's experience.
For our evaluations, we wanted to ensure that the system was expertly designed using appropriate prompts and responses, and followed common voice interaction standards. Instead of using Neilsen's 10 heuristics, since they did not quite meet the functionality we were trying to improve with a voice interaction system, we decided to use "heuristics" from Amazon Alexa's Design Guide, found here. The design guidelines we chose participants to evaluate are listed below.
• Earcons were generally well received, and enhanced the experience. However, they could also be added in the background during the user’s interaction as not to have an abrupt stop when waiting for a response.
• The visual handout was also generally well received, and participants liked having it as an option. While some participants said that they did not think it was necessary to have, they also all agreed that it enhanced their experience and liked to reference it whenever they felt like it.
• The points system could be improved, including making the rewards more relevant in the game, making it more obvious how the points system works, or making it more obvious how many points the user has as they move through the game. Evaluators also expressed that the point system might not be for all users (Moe’s Rewards members and others), so design accordingly.
• Narration is a key aspect of this interaction, and should be emphasized in the system. However, the length of narration and the character interaction could be improved to make the user more engaged.
• Interactions are overall very simple, which makes it easy to progress through the story.
In addition to an expert evaluation, we wanted general user feedback into some key components of Burrito Quest, such as the storyline and its interaction components, the points system, the visual map, and the minigames.
• Overall, engagement in the story was high, and no participants had trouble identifying their next moves in the story. Laughing and smiling could be identified throughout the story, and all participants answered to having high rates of enjoyment while interacting with the story.
• The minigame duration was too long, and should be shortened by about 25%.
• Participants were split by the story having too little or too much narration. Some participants felt that it could use more narration, while others felt that they wanted less narration, and more interaction with the story. This is an idea to keep in mind in the future, as we develop the narration between personal preferences.
• The rewards system was still confusing, and should be made more clear at the beginning of the interaction and during the story. For instance, during the story, instead of saying "You just won 30 rewards points," say "You just won 30 rewards points! You now have 65 out of 500 rewards points, and you just need 35 more to redeem a free queso!"
• Options to go backwards or jump to a certain point in the story would be ideal for users who either made a mistake in their interaction choice, or are replaying the game and would like to skip to certain aspects of the story.
• Designing voice technologies is tough, and requires a plethora of different design guidelines in order to design correctly for voice.
• When it comes to brainstorming a system for a unique technology that might not quite fit a certain brand, it's important to think more outside-of-the-box on how their brand could fit successfully into a less-explored technology.
• The most obvious solution (in this instance, food ordering), may not always be the best one. It takes lots of research to determine what exactly would be best to create. This means not relying on your own assumptions as a designer before any research, but allowing the research to guide you in your process to create the best solutions for your users.
• How do we expand this concept to a fully-fledged system? How long should the duration of the game be to win a free burrito, and what research would we need to do in order to solidify the rest of the story?
• What are our limitations on voice technology? Since this was only a prototype, actually programming the system may cause us to face new challenges, especially in terms of the minigames, or other complicated interactions. Several design decisions may need to be made at that point depending on the limitations of the system.
• How might we need to change the points system to comply with what Moe's standards are in terms of coupons?
• What can we do to encourage replayability for this system? For instance, might we change the storyline every few months for a fresh new game, or have new rewards, or both?