Skip to Content

Strongly Disagree? Should you use Agree/Disagree in survey questions?

It’s surprising but true. One of the most common kinds of survey questions, used widely in course evaluation surveys, employee attitude surveys and other surveys of opinion or attitude in education and training, is actually a source of survey error and should be avoided.

It’s very common in surveys to ask whether a respondent agrees or disagrees with a statement (sometimes called a Likert Scale question). Here’s an example:

My overall health is excellent. Do you:

  • Strongly Agree
  • Agree
  • Neither Agree or Disagree
  • Disagree
  • Strongly Disagree

Is this a good question or a poor one? The answer: it is usually a poor question. It’s much better to replace it with something direct. Here’s an example:

How would you rate your health overall?

  • Excellent
  • Very good
  • Good
  • Fair
  • Bad
  • Very bad

One reason why agree/disagree questions are weak is due to something called “acquiescence bias”.

People are much more likely to agree with a statement than disagree with it. Whether due to people wanting to be polite or just because they don’t want to put in the effort, people tend to agree with survey statements much more than they disagree. This is a particularly strong tendency for some of those with less formal education, who are more likely to agree, as they may think the survey author knows more than they do. This factor has been shown repeatedly in research. On average, studies have shown that with two questions asking the same thing in different ways, around 14% more people agree with an assertion compared to those who express the same point of view via a direct question.

Another reason that agree/disagree questions should be replaced is that they require the respondent to evaluate the statement and then answer. The respondent has to think “what do I feel about the statement?” and then “how should I answer?”, and this complexity reduces how accurate the results are. Furthermore, the distinction between “Strongly Agree” and “Agree” is imprecise and different people will select one or the other with different criteria.

To quote learning researcher Will Thalheimer in his excellent book on smile sheets (course evaluation surveys) “Performance-focused smile sheets”:

    “Likert-like scales are ubiquitous on smile sheets – but they are very problematic and should be avoided in most circumstances”

For more information on why these questions can be improved on, read Will’s’ book, see this survey research article by eminent researchers Krosnick and Presser or read my article on good survey practice on the Questionmark blog.

It takes a bit more time to write direct questions than agree/disagree ones, but you will get more accurate results.

If you have an existing survey with past results and you want to ask it again to compare trends from today with those of the past, it can make sense to stick with the previous question format for comparison purposes. But if you are writing a new survey (in SAP Assessment Management or other systems), it’s usually better to ask the question directly, not make a statement and ask the respondent whether they agree with it.

Agree or disagree or have your own thoughts – feel free to comment below!

You must be Logged on to comment or reply to a post.
  • This is informative post, thanks for sharing.

    VIP Plumbing is Melbourne’s leading service provider of quality plumbing services. We put the customer and client experience at the top of our list. We have solid reliable plumbing and that is why our referrals are second to none.

    Plumbing Maintenance

  • Looking for the right car loan in Brisbane? Our expert car loan brokers in Brisbane will get you the right car loan finance that fits in your budget. Compare rates from 40+ lenders and get the best deal on your car loan. For more information about car finance Brisbane, just visit our website or Call us today on 07 3889 9250.

  • Carrum Downs Dental Care is one of the leading dental and orthodontics practices in Melbourne. With the latest technology our caring and experienced dentists will be able to provide top dental care in a modern clinic. At our practice, patients will be able to get the dental care they need in a very comfortable and caring environment.

    For more information about Emergency Dentist, just visit our website or call us today.