measurement missteps
|
Surveying Satisfaction in SoftwareEarly in my management career, the HR department sent out a single-question survey which asked how satisfied we were with our jobs on a scale of 0 (very unsatisfied) to 4 (highly satisfied). To my surprise, my group had the lowest average score (2.6) in the company. The HR VP predicted a mass exodus and told me that my annual bonus would be negatively impacted. I was mystified because I had heard very few complaints and nobody had left in the previous year. Not one to leave such mysteries unsolved, I convinced HR to ask my group three more questions:
The results made me feel better. My group reported they were unlikely to leave the company in the next six months, their current job was slightly better than ones they’d had in the past and they felt the same about their job today as they did 3 months ago. The evidence convinced HR I didn’t have a major problem brewing. And saved my bonus. Despite this, I was curious why my group’s results were lower than the company average. During my regular one-on-ones in the subsequent weeks, I described the situation to my employees and asked for their opinion. Apparently software engineers are tough graders. Many said they would never give a higher score than 3.5, no matter how happy they were. It irritated them that the company soda machines didn’t carry Pepsi products; enough to lower their grades. A 2.6 score for my group might have been equivalent to 3.0 or higher for another group. This confusion could have been avoided if HR hadn’t chosen to benchmark groups against each other. Comparing software engineers to salespeople is probably not a good idea. Instead, HR should have ignored the raw results and focused on the trend over time. Are the results from this quarter’s survey result going up or down as compared to last quarter? Since we only had one data point, I was trying to approximate this by asking, “How does your satisfaction with your job compare with how it was 3 months ago?” As it turns out, no one left my group over the next year despite the VP’s warning. While interesting work or a soft job market might have been the primary reasons, I like to think that it had something to do with the fact that I stocked my office mini-fridge with free Pepsi. 100% of the people surveyed were happy they followed me on Twitter (@jbecher). An earlier version of this blog appeared in 2007. Please follow me on Twitter, LinkedIn, and Google+. This blog was originally posted on Manage By Walking Around on May 26, 2013.
|
|
The Invisible Gorilla EffectRemember the invisible gorilla video? In an experiment popularized by the book of the same name, volunteers were told to keep track of how many times a basketball was passed between players. While the ball was being tossed, someone in a gorilla suit walked between them in plain view. Very few people noticed the gorilla because they were so focused on counting the passes. The invisible gorilla study is the most famous example of a phenomenom called “inattentional blindness.” When we pay close attention to one thing, we often fail to notice other things – even if they are obvious. The authors of the book modified the original video to reinforce the point. Watch the updated video to see if you notice anything unusual: One thing has always bothered me about the gorilla experiment. Counting basketball passes is not a real-life task and the participants have no training in it. It’s possible they are easily fooled or just lazy. Would the same thing happen to experts? A recent study from Brigham and Women’s Hospital confirms experts suffer from inattentional blindness and raises some uncomfortable questions in the process. The scientists asked experienced radiologists to identify white nodules within five CT scans made up of hundreds of images of lung tissue – a process used to identify potential lung cancer. In one of the scans, the scientists inserted an image of a gorilla almost 50 times the size of a nodule. Despite its relatively large size and its incongruous presence in the CT scans, only four of the 24 radiologists noticed the gorilla. It’s not that it was difficult to see; eye-tracking data showed clearly that the radiologists looked right at it. And when later told to look for a gorilla, nearly everyone found it. The scientists repeated the experiment on untrained volunteers and none noticed the gorilla. Not surprisingly, the radiologists were also much better at spotting the warning signs of lung cancer. The confirmation that expert observers suffer from inattentional blindness raises some troubling questions. By training radiologists to identify white nodules, are they more likely to miss other life-threatening anomalies? Could the same issues pertain to other expert observers like MRI technicians, air traffic controllers, and police officers? That’s one scary gorilla. Follow me on Twitter: @jbecher This blog was originally posted on Manage By Walking Around on February 10, 2013.
|
|
The Myth Of Fair ValueTo prepare for the upcoming holiday shopping season, I’m reading ‘Priceless: The Myth of Fair Value (and How to Take Advantage of It)‘ by William Poundstone. Poundstone references a wide variety of psychologies studies that show consumers are unable to accurately estimate fair prices and are “strongly influenced by the unconscious, irrational, and politically incorrect.” In fact he makes the enticing claim that prices are a collective hallucination. One of my favorite stories describes what happened when Williams-Sonoma added a $429 premium breadmaker on the shelves next to their normal $279 model. While they sold very few of the premium model, sales of the $279 breadmaker more than doubled. In post-purchase surveys, shoppers reported the lower-priced model seemed like a bargain. Poundstone also explains why nearly 2/3 of all retail prices end in the number 9 and why the profit margin of the 99 Cents Only store is twice as high as Wal-Mart. Here’s a video of him discussing the phenomenon: The field of decision theory tries to explain these examples of people making seemingly illogical buying decisions. As Poundstone himself says: “although humans spend in numbered dollars, we make decisions based on clues and half-thinking that amount to innumeracy.” If you’re interested in learning more, Poundstone occasionally blogs here; most recently about a $69 hotdog. As for me, I feel better prepared to resist those holiday shopping bargains. Follow me on twitter @jbecher. This blog was originally posted on Manage By Walking Around.
|
|
Home Field AdvantageMost sports fans know that home teams have an advantage over visiting teams even though they have never seen any hard facts that prove it. While fans believe in the home field advantage, they hotly debate the reason for its existence. Over the years, I’ve heard a variety of explanations from not having to travel to playing in familiar settings to being in front of supportive fans. In addition to these psychological explanations, some people claim the formats of the games themselves are the cause: for example, in baseball the home team bats after the visitors giving them one last chance to win while in hockey the home team is allowed to make player substitutions after the visiting team which can create better match-ups.
In the book Scorecasting, the authors compile won/loss percentages for each of the major U.S. sports league and show that the home field advantage really does exist. (I’ve recreated the table from the book.) While the edge is minimal in baseball, it’s pronounced in major league soccer – nearly seven in ten games are won by the home team. However, the authors debunk the commonly held theories around travel and familiarity with the surroundings. Instead they claim that home teams get slightly preferential treatment from the officials, which gives them an edge in winning the game. Despite allegations of corruption in officials, the bias is likely unconscious and perhaps even involuntary. Every once in a while officials get caught up in the emotion of the crowd and make a call that pleases the home team. As this article points out, the increased home field advantage in soccer might be explained by the fact an official in soccer has more opportunity to influence a game’s outcome than in other sports. If the home field advantage exists in sports, it’s natural to wonder whether it exists in business as well. Are M&A deal prices influenced by whether they are negotiated at the acquirer’s office or in a neutral location? Sales calls are typically held at the customer site but would vendors achieve a higher close rate on their own turf? Should job candidates avoid salary negotiations in their future boss’ office? In this vein, researchers conducted a series of experiments to determine if people were more successful in negotiations that happened in their own offices than in unfamiliar locations. As might be expected, home team negotiators outperformed those in ‘away’ or ‘neutral’ locations. This suggests you shouldn’t ask your boss for a raise in her office but rather in your own office or even in a neutral location like the cafeteria. Unlike sports in which the home field advantage is attributed to the officials, the researchers believe psychological factors are at work here. If changing offices isn’t possible, the researchers suggest improving your confidence:
You can be confident that in business, like sports, the advantage goes to the home team. This blog was originally posted on Manage By Walking Around.
|

I'm fascinated by the concept of aligning execution and strategy; in other words – making sure what we actually do matches what we say we will do. It might sound obvious but finding ways to close the gap between the two makes a great subject for a blog and I’ve been writing about it for years. Even as CMO of SAP, I do my best to literally “manage by walking around” but since SAP is a global company it might be more aptly called manage by flying around. While I occasionally write about marketing topics, most of my entries deal with general people and business issues. Please share your ideas and experiences in the comments.
RSS Feed