I'm posting excerpts from a few of the most popular posts from my other blog called Manage By Walking Around. In an extract from this post, I share a cautionary story of using surveys to measure employee satisfaction.
_____________________________________________________________________________
Many years ago, the HR department in the company I worked for sent out a single-question survey that asked how satisfied we were with our jobs on a scale of 0 to 4, with 4 being the highest. When the results came back, my department had the lowest average score (2.6) of any group in the company. The HR VP solemnly explained that my group was in danger of mass exodus and that my annual bonus was going to be negatively affected.
I was mystified because I had heard very few complaints from my group and no one had left in the last year. Not one to leave such mysteries unsolved, I convinced HR to ask my group three other questions: “how likely are you to leave the company in the next 6 months”, “how does your job compare to other jobs that you’ve had”, and “how does your satisfaction with your job compare with how it was 3 months ago”. The results made me feel much better. On average, my group reported that they were very unlikely to leave the company in the next six months, that their current job was a little better than ones they’d had in the past, and that they felt the same about their job today as compared to 3 months ago. The evidence convinced HR that I didn’t have a major problem brewing.
Despite this, I was still curious why my group responded so much lower than the company average. In the following weeks, during my regular one-on-ones, I described the situation to each of my employees and asked them for their opinion. The answers were striking. Software engineers, it seems, are tough graders. Even though the theoretical maximum was 4.0, many said that they would never give a higher score than 3.5 no matter how happy they were. And it irritated them that the company soda machines only carried Coke products and not Pepsi ones; enough that they lowered their grades. So, a 2.6 average score for my group might indicate equivalent satisfaction to a 3.4 score for another group.
All of this confusion might have been avoided if the HR group hadn’t chosen to benchmark my group against others in the company. Instead, they would have been better served to ignore the raw number and to focus on the trend over time. Are the results from this quarter’s survey result going up or down as compared to last quarter? Since we only had one data point, I was trying to approximate this trend by asking, “how does your satisfaction with your job compare with how it was 3 months ago”.
As with everything in performance management, understanding time is critical to interpreting results. If there is seasonality in your business (like there is in retail), you might consider also comparing the results to the same period last year, as satisfaction is notoriously lower during the Christmas blitz than during other slower parts of the year. For the software company I worked for, we needed to make we weren’t accidentally comparing the end of a release push to the relative calm of a planning period.
As it turns out, no one ever left my group during my tenure as a manager despite the VP’s warning. While interesting work or a soft job market might have been the primary reasons, I like to think that it had something to do with the fact that I stocked my office mini-fridge with free Pepsis.
