What is a good questionnaire, and how should you frame questions and answer options?
Most questionnaire guidance is about how to phrase a question. In practice, more studies are damaged by the options underneath it, because a badly built option list forces respondents to give an answer that was never true.
In brief
A good questionnaire asks one thing per question, in the respondent's own vocabulary, with answer options that are mutually exclusive, collectively exhaustive and balanced between positive and negative. Question wording decides whether the respondent understands what you asked. Answer options decide whether they can tell you the truth, and whether the resulting data is comparable across respondents.
What makes a questionnaire good
Three tests, applied to every question.
Can the respondent answer it accurately from memory? A question asking how many times someone bought a category product in the last twelve months will be answered, but with a guess. Shorten the recall window until the answer is retrievable.
Is there exactly one true answer available? If a respondent has to pick the closest available option rather than the correct one, you have collected a measurement error and it will not be visible in the output.
Will the answer be comparable across respondents? Two people reading the same question the same way is the whole point. Vague quantifiers like “regularly” or “sometimes” fail this, because one respondent's regularly is another's rarely, and the percentages you report are averaging two different definitions.
A questionnaire that passes all three across every question is good. Style and tone are secondary.
How to write a survey question
Ask about one thing. Ask about it in the words your respondent uses. Keep the question shorter than the options underneath it.
Write in the respondent's vocabulary, not the category's. Brand teams say “usage occasion” and “purchase driver”; respondents say “when I use it” and “why I picked it.” Internal language does not just confuse, it flattens variance, because a respondent who is unsure what you mean picks the safest available option.
Anchor the timeframe explicitly. “How often do you buy” is not answerable. “In the last four weeks, how many times did you buy” is. Every behavioural question needs a window, and the window should be short enough to remember and long enough to capture the behaviour.
Separate behaviour from attitude. “Do you prefer natural ingredients” measures a stated value, and almost everyone says yes. “Which of these did you buy most recently” measures behaviour. Do not use one as a proxy for the other, and do not put them in the same question.
Keep the question neutral about its own subject. The moment a question signals that one answer is expected, correct, or socially preferable, you have stopped measuring and started prompting.
Question wording that biases results
| Fault | Example | What to do instead |
|---|---|---|
| Double-barrelled | Was there a time you needed this, and why? | Split into a yes or no question, then an open follow-up shown only to those who said yes |
| Leading | How much did you enjoy the new packaging? | Ask how they would rate it, with an equal negative range available |
| Loaded | Do you agree that sustainability should matter? | Ask what factors influence their choice, sustainability among them |
| Vague quantifier | Do you use this regularly? | Ask how many times in a defined recent period |
| Unanswerable recall | How many units did you buy last year? | Shorten the window to four weeks or ask about the most recent occasion |
| Jargon or internal language | What is your primary purchase driver? | Ask what made them choose it |
| Assumed behaviour | Where do you usually buy this? | Establish that they buy it at all, then route to the follow-up |
| Hypothetical intent | Would you buy this if it launched? | Ask about the most recent comparable purchase, and treat intent as directional only |
The double-barrelled fault is worth singling out because it fails invisibly. Respondents answer the first half honestly, the open-text field fills with “no” and “none,” and the resulting data looks like respondent apathy rather than a design fault. Nothing in automated cleaning distinguishes the two.
Why agree and disagree scales cause bias
Agreement framing is the most common construction in questionnaires and one of the weakest. Survey researchers have repeatedly found that agree and disagree formats produce acquiescence bias, where respondents agree more readily than they otherwise would, and the recommended alternative is an item-specific scale.
The fix is to move the substance out of the statement and into the scale. Instead of asking whether someone agrees that a product is good value, ask how they would rate its value, from very poor to very good. The respondent now has to make a judgement rather than accept or reject yours.
This matters most in grids, since a grid of agreement statements is where straight-lining concentrates. Item-specific scales force the respondent to re-evaluate each row rather than defaulting down a column.
How to write survey answer options
Two properties are non-negotiable.
Mutually exclusive. No respondent should qualify for two options. Numeric ranges are the usual failure, since 18 to 25 and 25 to 35 both contain 25 and the respondent picks arbitrarily. Write 18 to 24 and 25 to 34.
Collectively exhaustive, meaning the list covers the full range of possibilities. A vehicle question offering van, SUV and sedan has no answer for a truck owner, and that respondent will either abandon or pick something untrue.
Beyond those two, four working rules.
- Balance the range. Count the positive options and the negative options, and make them equal. A scale running excellent, very good, good, fair, poor has four positive positions and one negative, and it will return a flattering result regardless of what respondents think.
- Match the option granularity to the decision. Offering eleven price bands when the decision needs three creates thin cells that cannot be read at cohort level.
- Use the respondent's categories, not the company's. Internal segment names, product family names and channel taxonomies mean nothing outside the building.
- Keep option text parallel in length and structure. A list where one option is a full sentence and the rest are two words biases selection towards the detailed one.
How many scale points to use
| Points | Best for | Trade-off |
|---|---|---|
| 3 | Simple direction, low-literacy or mobile-heavy audiences | Too coarse to detect movement between waves |
| 5 | Most attitude and satisfaction measurement | The default for good reason, balances precision and ease |
| 7 | Attitude work where you need finer discrimination | Respondents rarely use the extremes, so effective range narrows |
| 10 or 11 | Likelihood and recommendation measures, benchmark comparability | Interpretation of the middle varies widely between respondents |
Hold the same number of points across every scale in one questionnaire. Mixing 5-point and 7-point scales makes the results incomparable and makes the survey harder to answer, since the respondent has to relearn the scale each time.
Label every point rather than only the endpoints. Numbered scales with only the ends labelled leave the middle open to interpretation, and different respondents assign different meanings to the same number.
Neutral, don't know and other options
A neutral midpoint belongs in a scale where indifference is a real position, which is most attitude measurement. Removing it forces a lean that does not exist and inflates whichever side is more socially acceptable.
Don't know is different from neutral and the two should not be combined. Neutral means the respondent has a view and it sits in the middle. Don't know means they have no view, or lack the knowledge to form one. Collapsing them makes an ignorance measure look like an indifference measure.
Offer don't know only where genuine ignorance is plausible. On awareness and knowledge questions it is essential. On preference questions about a product they have just been shown, it is an escape hatch that costs you data.
Prefer not to answer options are not supported by current evidence as improving data quality or response rates, though many respondents appreciate having the option. Reserve it for genuinely sensitive questions, typically income, health and household composition, and leave it off everything else.
An “other, please specify” field belongs on any question where your list might be incomplete, particularly brand and channel lists. Read what comes through it after fielding, because it is the cheapest available signal that your option list was wrong.
How option order and list length change answers
Position affects selection independently of content. Options near the top of a long list get chosen more often, and options at the end of a spoken or scrolling list benefit from recency. Randomise option order to distribute the effect rather than letting it accumulate on whichever brand you happened to list first.
Some options must stay fixed. None of the above, other, and don't know belong at the bottom regardless of randomisation, because their meaning depends on following the substantive options. Pin them as anchors rather than letting them float into the middle of the list.
Keep lists short enough to read. Past roughly ten options, respondents stop evaluating and start scanning for something acceptable, which produces selections concentrated in whatever they read first. If the list genuinely needs to be long, split it or use a two-stage question that narrows before it asks for detail.
How to write open-ended questions
Ask an open question only where a closed one cannot capture the answer, because open text is the most expensive question type for both respondent and analyst.
Be specific about what you want. Asking for suggestions to improve a product returns answers about taste, packaging, price and advertising all at once, none of them comparable. Asking what would make them buy it more often returns something analysable.
Never fuse an open question to a closed one. If you need to know whether something happened and then why, that is two questions and a piece of routing logic.
Place open questions early, while attention is high. The last open question in a long survey returns single words while the first returns paragraphs, and that difference is fatigue rather than opinion.
How to test question wording before fielding
Read the questionnaire aloud. Anything you would not say to a person in conversation is too long or too formal to be answered accurately.
Then pilot it and check three things in the returns. Whether any option list produced a heavy “other” response, which means the list was incomplete. Whether open answers address what you asked, since off-topic answers usually indicate a question that could not be answered as written. And whether any grid came back flat, which indicates either a fatigue problem or an agreement construction that needs converting to an item-specific scale.
Fixing wording after full fieldwork is not possible. Fixing it after a pilot costs a day.
Questionnaire checklist
One subject per question. Respondent vocabulary throughout. Every behavioural question carries a defined timeframe. Options mutually exclusive with no overlapping ranges. Options collectively exhaustive, with other where the list may be incomplete. Positive and negative options counted and balanced. Scale points consistent across the questionnaire and fully labelled. Neutral and don't know kept separate. Option order randomised with none of the above anchored. Open questions specific, early and never fused to a closed question. Piloted, with other responses and open answers read before launch.
Why Flickly
Flickly builds questionnaires with balanced option sets, randomisation, anchored options and item-specific scales configured from the start, then scores response quality on every completed interview.
Frequently asked questions
Run your next study with verified Indian consumers
Design the study, reach the right respondents, and get decision-ready insights in 72 hours.