Survey Question Design: Asking the Right Questions to Get the Right Answers
Think about the last time someone asked you a question that confused you. Maybe a friend asked, "Do you like movies?" and you weren't sure if they meant any movies, or just the one playing this weekend. Or a teacher asked, "How often do you study?" — and you wondered: every day? Last week? Compared to what?
That confusion is exactly what survey question design tries to avoid. In economics, surveys are how we gather data on what people earn, spend, save, and believe. If the questions are poorly designed, the data is useless — and so are the policies or business decisions built on it.
The Core Idea: A Good Question Gets One Clear Answer
A well-designed survey question has three properties:
-
It means the same thing to every respondent. If "income" could mean salary before tax, salary after tax, or total household earnings, different people will answer the same question differently — not because their situations differ, but because they interpreted the words differently.
-
It asks one thing at a time. A question like "Do you support the government's new tax policy and the accompanying subsidy cuts?" is actually two questions. Someone might support the tax policy but oppose the subsidy cuts — and have no way to say so.
-
It gives answer choices that are exhaustive and mutually exclusive. "Exhaustive" means every possible answer fits somewhere. "Mutually exclusive" means no answer fits into more than one category. If you ask "What is your monthly income?" and give options ₹0–₹10,000, ₹10,000–₹20,000, ₹20,000–₹30,000, a person earning exactly ₹10,000 doesn't know which box to tick.
Why This Matters in Economics
Economics is not a lab science. You cannot put a consumer in a test tube and measure their preferences. You have to ask them. And the answers you get depend entirely on how you ask.
Consider the Consumer Expenditure Survey, which the government uses to calculate the Consumer Price Index (CPI). If the survey asks, "How much did you spend on food last month?" — most people will guess, and guess badly. A better question might ask about the last week, or ask about specific items one by one. The design of the question directly affects the accuracy of the inflation rate the country reports.
A Common Mistake
Students often think survey design is just about "being polite" or "using simple words." It is not. It is about validity — whether the question actually measures what you think it measures. A question that everyone understands but that measures the wrong thing is worse than a confusing question.
The Key Types of Questions
Open-ended questions let the respondent answer in their own words. "What factors influenced your decision to buy this car?" These give rich data but are hard to tabulate. They are useful in pilot studies, not in large-scale surveys.
Closed-ended questions give fixed options. "Which of the following factors most influenced your decision to buy this car? (a) Price, (b) Fuel efficiency, (c) Brand reputation, (d) Resale value." These are easy to analyse but may force a respondent into a choice that does not truly reflect their thinking.
Rating scales ask respondents to rank something on a scale, typically 1 to 5 or 1 to 7. "On a scale of 1 to 5, how satisfied are you with your current bank?" The problem here is that different people use scales differently — one person's "3" might be another person's "4".
The Trap of Leading and Loaded Questions
A leading question nudges the respondent toward a particular answer. "Don't you agree that the new GST rates are unfair?" This question signals that the expected answer is "yes, they are unfair." A neutral version would be: "How would you describe the new GST rates — fair, unfair, or neither?"
A loaded question contains an assumption that may not be true. "How often do you waste money on unnecessary luxuries?" This assumes the respondent does waste money. A respondent who believes they spend wisely has no good way to answer.
The Double-Barreled Problem
This is so common it deserves its own warning. A double-barreled question asks about two things in one sentence. Example: "Should the government reduce taxes and increase spending on healthcare?" A respondent might want lower taxes but oppose higher healthcare spending — or vice versa. The question cannot separate these views.
What Makes a Good Answer Set
For closed-ended questions, the answer choices must follow two rules:
-
Exhaustive: Every possible answer fits somewhere. If you ask "How many times did you visit a bank last month?" and give options 0, 1–2, 3–5, 6+, someone who visited 10 times is covered by 6+. But if you ask "What is your employment status?" and give only Employed and Unemployed, you miss students, homemakers, and retirees.
-
Mutually exclusive: No answer fits into two categories. If you ask "What is your age group?" and give 18–25, 25–35, 35–45, a 25-year-old could tick either of the first two boxes. The correct ranges are 18–24, 25–34, 35–44.
A Useful Shortcut
When designing answer categories, always check the boundaries. If a person at the exact boundary could fit into two categories, the categories are not mutually exclusive. Fix it by making the upper bound of one category one less than the lower bound of the next.
The Order Effect
The sequence of questions matters. If you ask "How satisfied are you with your life in general?" and then ask "How satisfied are you with your job?" — the first answer may influence the second. Worse, if you ask about specific problems before asking about overall satisfaction, the respondent's attention is drawn to those problems, skewing the overall answer.
Economists who design surveys randomise the order of questions across different respondents to cancel out this effect. This is called randomisation — not a formula, but a technique.
A Concrete Example from Economics
Suppose you want to measure the marginal propensity to consume (MPC) — the fraction of an additional rupee of income that a household spends. You cannot observe MPC directly. You have to ask.
A bad survey question: "If your income increased by ₹10,000, how much of it would you spend?" This is hypothetical, vague, and invites guesswork.
A better approach: Ask about actual past behaviour. "Last year, did your household income increase, decrease, or stay the same? By approximately how much? And did your household spending increase, decrease, or stay the same? By approximately how much?" From these two answers, you can calculate the MPC for that household — not ask for it directly.
This is the heart of survey design: ask about what people know, not about what you want to know. You want to know the MPC. You ask about income and spending changes. The MPC emerges from the data.
What This Means for You as a Student
When you encounter survey data in your economics textbook — the unemployment rate, the CPI, the poverty line — remember that behind every number is a survey. And behind every survey is a set of design decisions. The poverty line is not a fact of nature; it is the answer to a question that someone designed. If the question was badly designed, the poverty line is wrong.
Survey question design is not a formula you memorise. It is a skill you practise. Every time you read a statistic, ask yourself: What question was asked? How was it worded? What answer choices were given? Could the answers be misinterpreted? That critical habit is what the concept is really teaching you.