Sarah Yukes

Amazon Engage

How can we help frontline leaders plan, conduct, and report engagements, improving relationships with theirs associates?

My role

I was the sole lead UX designer working with a product manager, technical product manager, and three engineers.

Skills Used

• user interviews

• usability testing

• product strategy

• interaction design

• information architecture

Background

My team, PeopleInsight (PI), set out to be the single source of truth for people data across Amazon Worldwide Operations. After spending two years standardizing over 500 people metrics, the team was ready to turn data into products in ways that would allow Amazon operational leaders to make more data-driven decisions.

<image of all the places>

User problem

Frontline leadership in the Amazon Worldwide Operations space is responsible for managing anywhere from 2-400 hourly associates at any given time. Keeping track of who their direct reports are, what their schedule was, what to talk to them about - you can imagine all of this is hard to keep track of. With no centralized tool, leadership accessed 4-10 systems each day to create their own “talk to” lists, often recorded on post-notes or Excel.

Business opportunity

There are over 20 associate engagement types operational leaders are asked to conduct on a monthly basis and tracking engagements and their impact on associates/business was difficult. Data was scattered across multiple tools, some engagements were not logged anywhere, it was essential a big disorganized mess that no one had ever tried to clean up. Additionally, our sister machine-learning team was working on automating and prioritizing some of these engagements, but needed a product to put them in.

For these reasons, the idea of Amazon Engage was born.

when i joined

I was less than a week into the Amazon life when I took over this project. The product manager at the time had already handed off a two page wireframe of what they wanted built, but I quickly learned no user research had been completed nor did the team know who among the operational leader staff would be using this tool the most. In general, I was really worried about the product strategy but had no idea what was possible being SO new. After a few discussions, I was able to negotiate 6 weeks to hand off a new design.

my process

1. Figure out who the primary user is

2. Prioritize product requirements against primary user needs, creating a product vision

3. Get the product built using a design and development framework

4. Hand off a scalable design on time (6 weeks)

Step 1. figure out who the primary user is

If our team didn’t know who among Amazon operational leadership would most likely be using our tool, I was not sure how to optimize the experience for them. So, I tracked down the developer of a tool we were getting engagement data from and quickly added a survey banner to get users to interview. I also asked them questions about why they were using the aforementioned tool.

Thankfully, people responded. After 8 interviews and a deep dive on the survey and interview results, it was clear the Area Manager would be our primary user in Amazon operations. The Area Manager role (in North America) is often a college graduate or an hourly associate that worked their way up into the role. In both cases, they often had no previous management experience. The top findings:

1. Often, it would be difficult to know who was on their team, and if they were even in the building. Further, managers were not necessarily working at the same time as their reports, so they regularly relied on peers to help out.

2. Because managers had to access so many different tools to determine who to talk to, they regularly planned this out ahead of their shift. How they did this varied - post it notes, excel sheets, and OneNotes were commonly mentioned.

3. There was nowhere to record what conversations took place, so it was difficult to remember what they talked about last with an associate.

4. Every manger was passionate about helping their associates, and they were overwhelmed but motivated to be the best manager they could be.

Step 2. Prioritize product requirements against primary user needs, creating a product vision

With a better understanding of who the primary user was for our product, I was able to set a product vision: help frontline leaders plan, conduct, and report engagements, improving relationships with their associates.

With the vision in place, I quickly got to work taking apart the existing wireframes the product manager had put together before I joined, working through the requirements and ideas with the vision and our user research in mind. There was still a lot I didn’t know about Amazon operations or even about area managers. A few stakeholders including my manager used to work in operations, so I met with them each day to work through VERY messy design ideas as I continued to increase fidelity.

One product feature I had to fight for was the concept of a digital list that a user had 100% control over. It was going to add an extra week of dev time and push our pretty firm launch date, but without the component the product didn’t address the “planning” aspect of our vision. Further, because users were already making lists ahead of shift, I hypothesized this product feature would be a delight factor and help support adoption at launch. Ultimately, we were able to include it, albeit scoped down to a simpler feature - My engagement list.

Step 3.Get the product built using a design and development framework

While working on those aforementioned messy design ideas, I was also evaluating two internal to Amazon design and development frameworks. Our team qualified to use the system meant for all Amazon HR tools as well as the framework for all Amazon Operations tools. As I was evaluating the design components, I worked with the software engineers to facilitate meetings with both system teams so we could understand the technical tradeoffs. Ultimately, we went with the HR framework because it offered better engineering support and the differences from a UI component perspective were minimal.

Step 4: Hand off a scalable design on time (6 weeks)

Ultimately I handed off a completely redesigned “product” that, while being built, I was able to usability test. A top project highlight for me was working along side the front-end enginner to get all priority 1 (meaning the worst issues) addressed before launch.

Results

In the end, Engage surpassed the launch adoption goal (33% of v1 sites used the tool > 5 times a week against a goal of 20%), and as of Q2 2020, 17.3K WW Ops managers used Engage to inform and conduct 5.4M conversation across 954k unique associates. From a business perspective, there is an estimated 28k hours of monthly, manual work saved each month with the tool and Europe has seen a 2.5% decrease in attrition for associates with logged engagements (representing ~4.2M cost savings). But most importantly, Engage solved for Amazon Worldwide Operations managers’ needs, as one area manager said:  ‘I think Engage is great for area managers. It's a one stop shop to set up engagements with them.’  We also The SUS score ranks at 84%

Lessons learned

This product will always hold a special place in my career memories because I was able to see the user-centered design system work successfully, not only at lightning speed but also with minimal (but essential) user research. By focusing the team on the primary user and solving for their main issues related to our product, we saw success at launch, allowing us to build and iterate off something working as opposed to going backwards and aligning the product to user needs. If I did this project again, I would have continued research after I completed my usability study, so I could have been more prepared for post launch updates.