Asking double-barreled questions not only wastes your audiences’ valuable time; it costs you money. Running additional surveys to make up for the cost of having not done it right the first time will only inflate your research costs - and likely give you a splitting headache to go with it.
The way you phrase test questions in a survey is crucial in determining the value of your results. One of the biggest mistakes new researchers and veteran surveyors alike are prone to making: they ask a participant to answer two questions with one response.
For example:
While it may feel like you’re getting the most out of your surveying budget by cramming multiple queries into a single question - you’re actually doing harm to your research.
Questions that ask panelists to respond to two separate issues or topics with only one answer are known as double-barreled questions. The reason for such a name is obvious; you’re firing two questions at the participant at once. Most often, when researchers include double-barrelled questions in a study or survey, they are not aware of the mistake they’re making.
These types of questions dilute the quality of your research and invalidate your results. As participants can only select one answer for two questions, you can never be sure which question they hoped to respond to with their choice; therefore, you’ve effectively wasted their opportunity to answer two of your questions.
Asking double-barreled questions not only wastes your audiences’ valuable time; it costs you money. Running additional surveys to make up for the cost of having not done it right the first time will only inflate your research costs - and likely give you a splitting headache to go with it.
The way to avoid creating confusing questions is simple: split your questions into two.
Instead of asking customers to answer:
Ask them:
The part some researchers have trouble with is proofreading their surveys. Double-barrelled questions are easy to identify, once you know what they are; you simply have to review your surveys before sending them out.
As a survey designer, double-barreled questions are not the only mistake you need to be aware of. When designing your test, you should take care to avoid these common survey question errors:
Hypothetical questions are often too abstract in nature for most customers to want to handle. Your survey questions should be direct and relevant, never asking “what ifs''. The more concise and pragmatic your question is, the better chances you have of your panelist providing a complete answer.
Asking embarrassing or taboo questions can turn panelists away from wanting to answer your survey. The theory of social desirability bias asserts that people are more likely to answer survey questions in a way that makes them appear more socially accepted. If you plan on asking panelists to divulge potentially sensitive information, it is best to keep the results private - or allow them to submit their answers anonymously.
A leading question is any that attempts to guide a participant’s decision, or push them towards a particular answer. Questions like, “How much do you love our product?” could instead be phrased more neutrally by saying, “How would you rate your enjoyment of our product?”
Extremely positive or extremely negatively-worded questions - much like leading questions - attempt to pull a panelist towards one side of an argument. Researchers sometimes include these questions in their surveys in a misguided attempt to boost their customer satisfaction metrics; what they don’t understand is that these types of questions burn just as many panelists as they fool.
Questions like, “On a scale of 1-10, how undyingly loyal are you to our organization?” or perhaps “How would you rate our product compared to our competitor’s cheap imitation product?” should be avoided outright. They will sour your reputation with your customers, and the data you receive from these faulty questions will not be accurate or correct.
When asking panelists to respond to questions with a predetermined set of multiple choices, it is vital that you include an opt-out or neutral answer. When a panelist is unable to find a choice they feel represents their answer, then they either drop out of your survey or pick an incorrect/incomplete answer.
All of the answer choices we offer must be clear and distinct; including overlapping answer choices will dilute the quality of our results and confuse our participants.
To get the most out of your surveys, you’re going to need to avoid including these aforementioned faulty questions. Double-barreled questions, leading questions, extreme or humiliating questions, and all the rest can spell doom for your survey results - if you’re not careful.
Here are some tips you can use when drafting survey questions:
Writing survey questions is not always as straight-forward as it seems; the looming presence of faulty questions (like double-barreled questions) threaten to invalidate our survey results. So long as you are aware of what common mistakes to watch out for, the way to write effective questions becomes far clearer.
Proofreading your test questions, running single-question surveys, and removing improper questions are just a few simple ways you can dramatically improve the quality of your survey.
If you find yourself doubting whether or not your question is valid, return to this guide for reference - or visit our guide to survey question writing.
Helpfull is a living, growing platform built around the core principles of accessibility and usability. A community of thousands are ready and waiting to test your products; giving you the feedback you need to turn your project into a resounding success.
An intuitive user-interface, coupled with the ability to gather hundreds of consumer responses in just minutes, are just a few of the features that make Helpfull the ultimate tool for any artist, designer, marketer, or inquisitive spirit.
Set up and begin receiving results within minutes.
Sign up for free, no contract required.
Helpfull is the easiest way to get feedback from thousands of people in minutes. Whether you're comparing images, text or surveys; our pool of qualified testers give you their real detailed opinion to help you make better decisions and be more informed.
