3 Examples of Bad Survey Questions & How To Fix Them

Surveys are a data collection tool that can help your company identify opportunities, assess challenges, and set direction. Surveying your customers, internal team members, and other key stakeholders can reveal powerful insights that enable you to make data-driven decisions and propel your business forward.

The quality of the data captured, however, can only be as good as the questions themselves. If you're looking to design a survey for your organization, you should be aware of the common pitfalls that novice survey creators encounter.

Here are three survey question development mistakes made by startups and well-established companies alike.

Examples of Bad Survey Questions to Avoid

1. The Biased or Leading Question

“Our product reduces your tension by 10 percent. Would you like to buy it?”

A biased survey question, like the one above, prompts or leads a respondent to answer in favor or against a specific outcome, resulting in inaccurate data. The example mentions a reduction in tension that might influence the respondent to indicate they would make a purchase. A better question would be, "How likely are you to buy this product?" Examples don’t need to be as extreme as the one highlighted to be considered biased.

Tips to avoid bias:

• Use neutral language. Don’t favor one option—explicitly or implicitly—and don’t lead a respondent to an answer.
• Vary the order of options in a list. This should be done across questions with similar answers to ensure a respondent won’t make a decision based on chronological sequencing.

2. The Ambiguous Question

“How do you feel about your purchase?”

This question uses ambiguous and imprecise language. Quantifying or assessing subjective attitudes is difficult, and the burden shouldn’t fall on the respondent. A better approach is to provide options: "How satisfied are you with your purchase? Extremely satisfied, somewhat satisfied, neutral, somewhat unsatisfied, or extremely unsatisfied?"

Tips to avoid ambiguity:

• Think critically and develop precise questions. Add options for respondents to choose from. These can be “yes” or “no” options, multiple choice answers, or a Likert scale that prompts the respondent to select a number that indicates their level of agreement with a statement, satisfaction with your product, or another measure of their feelings.
• Additionally, consider conjoint analysis, which is conducted via a specialized survey to reveal how much a customer values a certain set of attributes in a product or service. Its tailored design ensures each question is direct and specific.

3. The Complex Question

“If you had to get to work using a bicycle, bus, train, car, or on foot, which would you choose? Consider annual precipitation, your transportation budget, and carpooling opportunities.”

The question above assesses propensity for transportation methods, perhaps to help a city decide whether to allocate funding to bike-share programs, commuter rail services, or bus routes. However, this question is overly complex, making it difficult for the respondent to answer.

When a researcher overcomplicates a question, it can become what’s known as a double-barreled question, or one that asks about more than one subject but requires a single answer. Such questions can confuse the reader and greatly diminish the quality of the data collected.

Tips to avoid complexity:

• Trim the fat. Cutting unnecessary qualifiers could ease the intellectual burden. Understand, however, that you risk perpetuating the ambiguity discussed previously by making assessments more open-ended.
• Split a complex question into multiple questions that each have one focus. By doing so, you can ensure you get your questions answered without overwhelming respondents or sacrificing data quality.
• Employ a pairwise ranking system, which prompts respondents to rank options in order of preference or compare each option to the others to determine a hierarchy of preference.

In addition, qualitative interviews and focus groups are alternative methods of receiving detailed answers if time and effort allow.

Collecting Reliable Insights

When thoughtfully executed, surveys can be an effective means of collecting data and unlock powerful business insights.

To ensure your data is reliable, design clear, concise survey questions. When your company conducts surveys, look for red flags that could lead to inaccurate conclusions.

These three tips are only the beginning of becoming a proficient survey designer. Knowing the best way to design assessments—and being aware of when other tools may be more appropriate—can lead to more informative company research.

Are you interested in unlocking the full potential of your organization’s data? Explore our online analytics courses to get started.