Intro
This platform helps older adults manage their day-to-day through remote assistance from family members, and makes it easier for multiple family members to manage shared tasks.
Problem
How might we help seniors stay in their homes longer, allowing them better control of their every day lives, and make it easier for caregivers to coordinate efforts to ensure their loved ones are properly cared for?
Work Covered & Teams
Nov 2019 - August 2020 | (7x) 2-Week Sprints, (12x) 3-Week Sprints
This project example covers 2 features I designed for the Amazon Echo Show 5 skill.
Our team: 2 Project Managers, 2 Product Managers, 7 Experience Designers, 24 Developers
Client team: 6 Stakeholders, 4 Product Owners, 2 Product Managers, 3 Developers
Client Background
Providing insurance for over 100 years. The firm specializes in auto insurance for farmers, homeowners and life insurance, banking, and financial services.
Project History
Product Features & Terms
Refining high-level concepts
Skills are voice and touch interface applications that can be added to Amazon devices. The skill is linked to the mobile app and it has simplified life management tools like a group calendar and shared lists. Older adults can receive updates from their family members (caregivers), and share status updates with their caregivers through the skill.
Mobile App
The mobile application helps family members coordinate management of key tasks and allows everyone to stay up-to-date on the group calendar, shared photos, to-dos, grocery, and wellness.
Older Adults
The primary users of the Echo Show skill. Over the age of 65, and have a range of capabilities, but may have vision, hearing, medical, or mobility issues.
Caregivers
Primary users of the mobile application. Adults between ages 40-60, actively helping with some of the organizational aspects of their parents lives, running errands, or providing reminders.
Alexa Presentation Language (APL)
The virtual assistant developed by Amazon, and the voice that users interact with on the Echo Show. Markup language specifically designed to build skills for Amazon Alexa devices.
Constraints
The Echo Show device has many default behaviors that cannot be overridden in the interface: One screen = One VUI interaction, common mobile & web design patterns were not possible, and not able to set typefaces or font weights.
Challenges
Needed to meet accessibility requirements for a device that is not handheld. Design for both touch and voice, ensuring both were usable by those with diminished vision and/or hearing. The Alexa Presentation Language is a work in progress, causing many road blocks for our development and design teams.
Accessibility
Inclusive design was a primary focus for both platforms. The older adults and caregivers ages ranged from mid 30’s to early 80’s, which meant the text needed to meet at least AA requirements, UX best practices, and common UI patterns need to be employed to accommodate for a range of technical knowledge/experience.
Designing for Accessibility on a Small Screen
Meet accessibility requirements while considering this device is not handheld. Needed to design for both touch and voice, ensuring both were usable by those with diminished vision and/or hearing.
One screen = One VUI interaction
Common UX considerations, like scroll bars or showing peaks of additional content proved to be difficult.
Consistency of Design Styles across Platforms
Common mobile & web design patterns were not possible with the Echo Show. We could not specify typefaces or font weights for the Echo Show skill, so the mobile app and skill experiences exist with different text styles.
Onboarding
April 15th - April 30th 2021
11
New designers participate in design team sprint review with client product owners
12
Design team staffing and reorganization
New designers familiarize with project and allocate sprint work
13
Familiarize with design files and design tool stack
16
Client onboarding
17
Read through Alexa APL documentation
Read NN/g’s UX design for seniors citizens report
18
2x 1-hour user testing sessions with client focus group
19
Final 1-hour user testing session with focus group
Synthesize notes from testing sessions
20
Synthesize notes from testing sessions and incorporate feedback into designs
21
Wireframe review of design work for next development sprint
Get access to developer APL dashboard
23
Create mock VUI interactions using APL
Read documentation on VUI design by Apple and Google’s voice assistants
New Design Team Members
3 new designers including myself were added to the team to take on the new features being added as the product grew. I got involved in designing experiences for the Echo Show and was given a sprint to familiarize with the team’s design process and ramp up on voice interface design (VUI).
User Feedback
We worked directly with the client’s user researcher who asked questions the designers wanted to address with the focus group. The focus group was a diverse representation of family’s caring for their older independent parents(caregivers).
We attended sessions remotely and documented the responses. Hearing feedback from the participants of the focus group helped us empathize with them, and heavily informed the direction of the product’s features.
Daily Summary & Tutorial Features
May 15th - June 24th, 2021
Daily Summary
We designed a daily summary experience for the Echo Show we termed “Start My Day”. Building in routine interactions helps older adults remember what they can do and surfaces key features to engage with the product. Check-ins and notifications are examples of routine interaction features that prompt the older adult to communicate with their family.
I outlined the experience, making updates after each meeting with the team to incorporate feedback. Our senior product manager and I worked with the client to curate the type of information for the feature. We landed on 4 different types of information to cover during the interaction. During the ~30 second readout, Alexa greets the older adult by name, states the date, the weather, the first event of their day, the items they are responsible for in their lists, and a fun fact. Alexa asks the older adult if they want to check-in with their family at the end of the readout.
“I don’t think my parent would complain if they heard the weather more than once, their memory isn’t always the best, so it doesn’t hurt to include it.” —Caregiver
Working Through Problems
We wanted to incorporate the weather in the daily summary however, we were unable to integrate with Darksky’s weather API. We knew that older adults already recieved weather updates from multiple sources, but the focus group felt strongly that weather updates needed to be included in the interaction, explaining that resurfacing information was important. We made a case to our product team to push for a solution. We knew the zip code of where the device was located, so our dev team opted to pull weather data from an open source API instead.
Refining The Interaction
A general rule was to keep UI and VUI interactions under 30 seconds. The focus group helped the team determine the extent of older adult’s ability to retain information. The maximum length advised was no more than 30s, and between 20 and 25 was ideal.
I mapped out many scenarios of the experience to help us decide what information to pair down. Although the fun fact was received well by the client and focus group, it was the least essential information in the readout. We decided to compromise and rotate a fun fact into the readout every few days and adjust their frequency later based on feedback after the feature’s release.
The last issue we needed to account for was the time of day the older adult opened the skill. What if the older adult did not open the skill at the same time everyday. We created conditional logic to adjust the content in the readout depending on when the skill was opened. The time of day in the greeting needed to be morning, afternoon, evening, and any events that had already occurred earlier in the day needed to be excluded.
Flex Development Chops
I wanted to add some delight for the team and client, so I created a high fidelity mockup of Alexa reciting the daily summary interaction. I got set up in the Alexa developer console and started crafting the dialogue using Alexa's speech synthesis markup language. We had to jump through some hoops to bring the dialogue into the next design review, but the team practiced with me and we pulled it off.
Hearing Alexa run through an example was exciting for the client and for us; their voices sounded truly delighted.
“It gave us real confidence about the idea.”
Tutorial
June 25th - August 15th, 2021
Tutorial
We know that older adults in the 75 to 85 age range can have diminished short term memory and may have limited to no exposure to voice assistance technology.
Our client wanted to include a tutorial feature to onboard users when opening the skill for the first time. Our focus group of caregivers supported the idea and were interested to see what we came up with.
I started by selecting our 6 most essential features and gave each its own screen with text and graphics. I felt like there was an opportunity to make the experience more interactive, so I proposed the idea of practicing with Alexa during the tutorial. In addition to showing the graphic and having Alexa read the on-screen text, Alexa will also prompt the user to try saying a command. The tutorial covers each main feature of the skill, so each step of the tutorial includes practicing that features specific commands.
Technical Challenges
We pushed the limits of the Echo Show with the tutorial feature. We needed to be cognizant of our ~30 interaction limit for a single screen. We designed each of the 6 steps in the tutorial so that they could be skipped. Alexa will ask the user if they want to try practicing, allowing them to spend more time on a single step or continue if they do not feel like practicing.
The practice utterances were also tricky. We had to disable the entire library of utterances to make it work. Without turning them off, the skill will accept the practice commands, causing those features to open during the tutorial. We also wanted users to be able to return to the tutorial.
Adding Delight
Similar to creating a demo of the interaction with the presentation language for the start my day feature, I wanted to add some personality to Alexa’s voice specifically for the practice portion of the tutorial. Alexa congratulates the user after they practice speaking the commands, but the readout of Alexa’s voice is fairly uninspiring. The APL allows developers to specify tone and inflection. I created the readout in the markup language and specified how Alexa should sound when congratulating the user for practicing. Now Alexa sounded genuinely excited when rewarding the user for practicing, sparking some life into the dialogue, ensuring that extra confidence boost reads naturally to new users of the skill.