- Ethan Evans, a former Amazon VP, led a failed project in 2011 that interrupted Jeff Bezos’ launch plans.
- The failure involved a critical design flaw in the Amazon Appstore’s “Test Drive” feature.
- It taught Evans to communicate in a crisis, take ownership of a problem, and rebuild trust slowly.
I worked at Amazon for 15 years, starting in 2005 as a senior manager. When I left in 2020, I was a vice president.
My biggest launch failure was in 2011 on a project Jeff Bezos personally cared about. The failure interrupted Bezos’ plan to present the feature publicly, causing me to miss my promotion and almost leave the company. However, I went on to be promoted from director to VP and have a long and happy career.
I learned a lot about dealing with a crisis and rebuilding trust as well as a lot about Bezos as a leader. He taught me the importance of maintaining high standards while being willing to forgive and move on.
Here’s the story of my biggest failure
When I started at Amazon, I was assigned to Prime Video and had periodic direct exposure to Bezos.
When I was promoted to director, I continued working with Bezos on creating Amazon Studios. Throughout my first six years at the company, I met with him at least once a quarter about one of my projects.
In 2010, I started working on the Amazon Appstore. We planned a new feature called “Test Drive,” which allowed you to simulate an app on your phone before buying it.
Bezos was excited about this feature and planned to make it the focus of his launch announcement. At the time, when Amazon launched something new, the company would replace the normal homepage with a personal letter from him explaining the new offering.
Our launch’s “Jeff Letter” focused on the “Test Drive” feature. The night before the launch, our team launched the new store.
Everything except the “Test Drive” feature was working well
We worked through the night to debug the intermittent failures, but as morning came and the announcement was supposed to go out, the feature wasn’t working. At 6 a.m., I got an email from Bezos asking why his letter was not on the homepage.
I replied that we were working on some problems, hoping he would get in the shower, go on the treadmill, or do anything else to buy us more time. Within a few minutes, he responded and asked, what problems?
All hell broke loose.
The VP and SVP above me both woke up and started asking questions, and more and more leaders were CC’ed into the email thread. We quickly realized that our feature had some critical design flaws and wouldn’t be a quick or easy fix.
The first three lessons I learned were during the crisis, and the next three were learned after.
Mid-crisis
Lesson one: Communicate clearly and predictably
I began sending hourly updates to Bezos and the other leaders, working to slowly re-establish trust. Each message briefly explained where we stood and what we would be doing in the next hour, and they each promised a further update in the next hour.
Lesson two: Accept help
Other leaders who had experienced similar problems reached out and offered help from their teams, so within a couple of hours, several very senior engineers were working with my team.
They quickly figured out the problems and announced that we had a design flaw that needed to be re-written. The temporary solution was to work around it with extra hardware. Without this workaround, it could’ve been days before the feature was up and running.
Lesson three: No all-night launches
Planning an early morning launch that required us to work all night became an obvious flaw. I needed to be sharp to manage the crisis, and my team needed to be able to help with the fixes. We started rotating people home to sleep in shifts, and we learned never to accept a launch schedule that would put us in this position again.
As my team and I became increasingly exhausted, Bezos became increasingly frustrated. He wanted a fix that day. This led to the other leaders ramping up the pressure, and the weight on us kept getting heavier.
We were finally saved when the CTO, Werner Vogels, intervened and said the team could not fix this problem in one day. Bezos fell silent on the email threads.
Over the next few days, we patched the design problem and rewrote the code to eliminate the issue, but as the technical obstacles were removed, the management problems only increased.
The “Jeff Letter” never went live on the website. By the time we had everything fixed and tested, the news cycle had moved on, and Bezos’ moment to tell his customers about the exciting new feature was gone.
After the crisis
Lesson four: Own the problem
My direct report volunteered to take the fall. The engineer who wrote some of the code did the same. My manager also sought to take overall responsibility. Ultimately, Bezos knew it was my team and code, meaning I had to own the problem.
Amazon has a process called COE (Correction of Errors), which involves a written investigation of a problem’s root causes and a plan to prevent similar problems in the future. I wrote this report and was asked to share it with all my peers in the organization. Publicly sharing an analysis of our mistakes was embarrassing, but doing a good job of it helped me re-establish trust in my leadership ability.
The week after the launch, I was scheduled to attend a meeting with Bezos about another project. I considered skipping it, but I decided that if I couldn’t face Bezos, I should probably pack my desk and find a new place to work.
I went to the meeting.
Bezos always sat in the same chair in his conference room. I went early and chose a chair right next to where he would sit. He came in, sat next to me, and ran the meeting. As the meeting ended, he asked me how I was doing because it must’ve been a tough week.
Bezos showed empathy for my experience and concern for my well-being. He could’ve just as easily asked for a status report or taken me to task for the problems; instead, he chose to focus on me as a person rather than on any frustration or curiosity about the project.
Lesson five: Face your leaders
Don’t hide. I understand the temptation to avoid those who might criticize you, but facing Bezos reassured me that he was over his initial frustration and was willing to give me the time to rebuild trust.
In short, going to the meeting allowed me to stay at the company. I knew my job was on the line, and a single word from Bezos would’ve sent me packing.
Lesson six: Patiently rebuild trust
I’d been close to a promotion to VP, but now I had to re-establish that I could operate a key business carefully and consistently. I was eventually promoted, but it took two more years.
I learned that trust can be rebuilt but that it takes time.
Bezos taught me how important it is to hold your teams to high standards but also be willing to forgive and move on. He chose to be kind, empathize, and offer encouragement to me, which inspired me to spend the rest of my corporate career with Amazon.
I left in 2020, less than a year before Bezos stepped down, to focus on teaching leadership lessons to the next generation.
An Amazon representative didn’t comment on this story when contacted by Business Insider.
Ethan Evans is a retired Amazon vice president with over 23 years of experience as a business executive.
Read the full article here
