If you want to know what a man’s like, take a good look at how he treats his inferiors, not his equals.

– Sirius Black, Harry Potter and the Goblet of Fire.

I vividly remember my first (of several) ? Jeff B. escalation emails. Sebastian Gunningham was still a “Section 16 officer”, and Amazon had only one CEO. I literally had no idea what I was supposed to do – surprisingly there was (and still is) no training manual for this kind of thing. After asking around, I figured out that I had 24 hours (the time to respond has since then been extended to 48 hours) to come up with a response that indicated a description of the issue, the root cause, any immediate fixes, and long-term actions that address the issue. The first Director in your chain of command emails Jeff your response and marks you and Amazon’s senior leadership team in the email. It can be unnerving but is quickly forgotten if the issue is addressed, or if there is a plan to fix.

Yes, this is a absolutely a REAL thing. I’m not kidding. Jeff literally forwards customer emails with a “?” to his senior leaders. Here are two examples; these are samples, based on (but not) actual emails:

In a recent interview with Ken Hersh during the Forum on Leadership at the Bush Center, Jeff described this mechanism. While the entire interview (below) is a worthwhile watch, he describes the ? email mechanism from 22m:35s.

Surprisingly, the response to his disclosure about sending ? emails has been mixed. Several are in awe like this one:

Yet, a few like the one below have rightly pointed out that although glorifying executives who invoke fear in underlings is part and parcel of Corporate America folklore, it should be questioned, like this one:

So what is it then? Is this a mechanism that works, or is this a questionable mechanism that deserves its own ? email?

Well, like most things in life, the truth is somewhere in the middle. I am in favor of using anecdotes to augment data in order to ascertain if systems are running as designed or can be improved further. Allow me to demonstrate using an example.

This example is an automatic email sent out by Amazon’s vendor systems to encourage vendors to spend on enhanced marketing content to improve their product sales (and Amazon’s bottomline). And, it’s darn obvious that this particular email is worthless to the vendor – they would’ve spent $0.36 in just reading the email.

Now, the metrics for this email automation might look just fine. It is unlikely for a handful of low value emails (like $0.36) to affect the open rate, click-through rate, action rate, and opt-out rate, and as a result the product manager may not even notice the need for filters to filter out emails that do not add a significant improvement to vendor’s sales. These vendors are unlikely to opt out because the other emails they receive might be valuable.

In such a case, it is clear that a ? email anecdote can be really powerful in making a small change to the email algorithm – if the product sales improvement is less than a threshold dollar value, or less than a percentage change, do not send that vendor an email. Of course, the team should have caught this and built these filters before launching the automated email product, but sometimes when you are moving fast and launching products at scale, you may miss out edge cases, and anecdotes are more powerful than data in catching those.

Sure, this is only an anecdote and I have not provided any empirical data that backs up why Bezos’s ? email works. But that’s the point isn’t it? Sometimes the anecdotes are powerful enough for you to make a decision. Make no mistake, a not-so-great leader can use the ? email mechanism to belittle employees for their mistakes, or simply to ask wtf? In fact, one of my peers once sent a ? email to his manager as a response to a request from the manager. Let’s just say, the manager sent more than just a request in the next email – he clearly wasn’t thrilled. Taunting employees, or making them feel ashamed of a mistake is not the intent of Jeff’s ? email mechanism at Amazon. The intent is to seek product improvement opportunities that data sometimes fail to catch.

In the end, though, whether you forward such customer emails with just a ? or take a few more seconds to write, “Please look into this.”, or ask “What do you think about this?”, that’s a choice you have to make. I know what I would do. Do you?

Note: This blog post does not contain confidential Amazon information; these are my personal views and does not represent the views of Amazon or its management.