The Psychology of Response
Teaching Note
I: The Problem You Didn’t Know You Had
Let’s start with something that might ruin surveys for you forever.
When a respondent answers your question, they’re not telling you the truth. They’re not lying either (at least, not usually). They’re doing something a little more complicated. They’re constructing an answer, in real time, using a process so fast and so automatic that they don’t even realize it’s happening.
Say you ask someone, “how satisfied are you with your bank?” It’s a simple question that probably feels like it should have a simple answer. But in the two or three seconds before they respond a lot of subconscious mental gymnastics unfold.
First, they have to figure out what you’re asking. Satisfied how? With the app? The fees? The person at the branch who helped them last Tuesday? The ATM that ate their card in 2019? “Satisfaction” could mean a dozen things. They pick one and it is probably whatever’s most mentally available at that moment.
To answer, they have to search their memory. Memory is a curious function. It’s more like a Wikipedia article than a filing cabinet. It gets edited every time you read it. What they “remember” about their bank is a reconstruction, shaped by their mood today, what happened recently, and perhaps whether they recently saw a commercial for a competitor.
Then they have to make a judgment. They take this fragmentary, reconstructed, possibly contaminated information and decide: am I satisfied? But that judgment is warped by what they think they should feel, or what a “reasonable person” would feel, or whether the survey itself has been nudging them in any direction.
Finally, they have to map that judgment onto your five-point scale. Is their feeling a 4 or a 5? What’s the difference, really? They pick one. They really are quite satisfied with their bank, but 5 seems extreme. They don’t like extremes. They think you pick the 1s and the 5s only when something extraordinary has occurred. So, they click on a 4. You record that as one of a stream of data from many customers, assuming it is an accurate depiction of that customer’s state of satisfaction.
Total elapsed time: maybe three seconds. The respondent has no idea any of this happened. You have no idea any of this happened. But it did. And every step introduced a bit of distortion.
This is the machinery underneath every survey response. And once you see it, you can’t unsee it.
II: The Four-Stage Model (Or, Where Things Go Wrong)
The process just described has been studied by cognitive psychologists for decades. The most influential framework comes from Roger Tourangeau and his colleagues, laid out most comprehensively in The Psychology of Survey Response (Tourangeau, Rips & Rasinski, 2000). Tourangeau identified four stages that every respondent passes through: comprehension, retrieval, judgment, and response.
Think of it like a relay race. The baton has to pass cleanly through all four stages to produce a valid answer. Drop it at any stage and you get the potential for error.
Comprehension: “Wait, What Are You Asking?”
Before respondents can answer, they have to understand the question. This seems obvious. Yet the harsh reality is that most respondents will answer your question whether they understand it or not.
Surveys don’t have a “huh?” button. There’s rarely an option for “I’m not sure what you mean.” Respondents encounter a question, experience a brief moment of interpretation, and move on. If their interpretation doesn’t match your intention, you’ll end up with data that measures something other than what you thought you were measuring.
Comprehension errors occur for many reasons but jargon is a frequent culprit. Technical terms that seem clear to you and anyone in your industry may be meaningless or misleading to respondents. Ambiguity is another frequent suspect. “How often do you work out?” seems like a straightforward question until you realize that one respondent counts a walk to the refrigerator while another only counts 5 AM sessions in their garage gym with a rucksack and cold plunge chaser. Same question, different universes. Your definition of workout was left open to the respondent’s interpretation and the results will probably lead you to over-report frequency.
The pioneering survey methodologists Stanley Payne and Howard Schuman documented these problems extensively. Schuman’s Questions and Answers in Attitude Surveys (Schuman & Presser, 1981) remains a masterwork on how small wording changes produce large response differences. It’s a humbling read for anyone who thinks question design is easy.
The most common way to catch comprehension failures is to watch people encounter your questions in real time. Researchers call this cognitive interviewing. You sit with someone, ask them to think aloud as they answer, and listen for the moments of hesitation, the wrong turns, the interpretations you never anticipated. It’s tedious. It’s also the cheapest insurance you’ll ever buy.
Retrieval: “Let Me Try to Remember…”
Let’s say your respondent comprehended your question correctly. Now they have to retrieve relevant information from their memory.
This is the juncture where things can get philosophically weird. Most of us assume memory is playback. You’re accessing a recording of the past. Yet decades of research in cognitive psychology have shown that’s not how memory works. Memory is reconstruction. Every time you remember something, you’re rebuilding it from fragments, filling in gaps with plausible guesses, and editing the result based on what you know now. The psychologist Elizabeth Loftus built a career demonstrating how easily memories can be distorted, even implanted, by subtle cues and suggestions.
In her most famous study, she convinced adults they had been lost in a shopping mall as children. It was an event that never happened. Subjects did more than accept the suggestion. They elaborated, adding vivid details their own minds invented. If memory can fabricate an entire childhood trauma from a researcher’s suggestion, imagine what it does with something as mundane as how often you visited Starbucks last month.
This means that what respondents “remember” is partly true, partly confabulated, and heavily biased toward certain kinds of information.
Recency Bias—Recent events are easier to recall, so they dominate. Ask someone how often they eat fast food “in a typical month,” and they’ll anchor1 on the last few weeks—even if that period was unusual.
1 Anchor: A reference point that disproportionately influences subsequent judgments, even when it’s arbitrary or irrelevant.
Salience Bias—Vivid, emotional, unusual events stick. Routine events blur together. A single terrible customer service call will loom larger in memory than fifty adequate ones.
Retrieval Cues—What people remember depends heavily on what triggers the memory. Ask “how often do you snack?” and you’ll get different answers than if you ask “how often do you snack while watching television?” The second version provides a retrieval cue that surfaces a different set of memories.
The implication is a little unsettling. When respondents report their past behavior, they’re giving you a narrative reconstruction, produced in the moment, shaped by the questions you happened to ask. It isn’t necessarily historical fact. Change the questions, change the narrative. I know. You are now questioning the results of every poll you’ve ever read. Hang on. It gets better.
Judgment: “What Do I Actually Think About This?”
Respondents have interpreted your question and they retrieved some relevant information. But raw information isn’t an answer. Respondents still have to evaluate it and form a judgment about what it means and how they feel.
Judgment is the psychological stage most vulnerable to heuristics (mental shortcuts), biases (systematic distortions), and context effects (influences from the surrounding environment, including the survey itself).
We’ll dive deeper on heuristics in a moment. For now, understand that respondents aren’t dispassionately weighing evidence at this stage. They’re pattern-matching. They’re relying on feelings. They’re figuring out what a “normal” person would say. And they may be editing their own thoughts in the process.
The judgment they produce shifts depending on what question came before, what words you used, what images are on the page, or perhaps whether or not they’re in a good mood or a bad one. Norbert Schwarz (a professor of Psychology and Marketing here at USC) and his colleagues demonstrated this in a series of elegant studies showing that even incidental factors like finding a dime on a copy machine can influence how people report their life satisfaction (Schwarz & Clore, 1983).
This is why two surveys, asking what seems like the same question in different ways, can produce wildly different results. It’s not that one is right and one is wrong. It’s that each survey created a slightly different context, eliciting a slightly different judgment.
Response: “I Guess I’ll Say… 4?”
The mental marathon of a millisecond is nearly over. The respondent understood the question, retrieved information from their memory, and has decided how to respond. Now, they need to translate their internal judgment into your external response format.
Seems easy, right? Just pick a number or check a box. But the format itself shapes the answer. Your response scale is a lens that can distort whatever passes through it.
The most common problem is granularity mismatch. You offer a 10-point scale because it feels precise. But respondents don’t have 10 distinct levels of satisfaction in their heads. They have something more like three, maybe five categories. So they do their best to convert their fuzzy feeling into your precise format and that conversion can add noise. Jon Krosnick, one of the leading scholars of survey methodology, has written extensively about how response scale properties affect data quality (Krosnick, 1999).
Consider the endpoints. A scale from “extremely dissatisfied” to “extremely satisfied” implies that the extremes exist and that respondents might reasonably land there. But most people avoid extremes. They cluster toward the middle, using only a portion of the scale you gave them.
The response format is part of the question. It influences how the respondent answers and what you (as the researcher) receive in the data.
III: The Shortcuts in Their Heads
The four-stage model tells us a lot, but it’s only the foundation of the psychology we need to grasp in order to make our surveys more effective.
Human cognition has a limited budget. We can’t carefully deliberate over every decision. If we did we’d never make it through the day. So the brain takes shortcuts. Cognitive psychologists call these heuristics—quick, efficient rules that usually produce good-enough answers with minimal effort.
For example, when a respondent’s cognitive budget is running low they are likely to satisfice–a term coined by the economist and cognitive scientist Herbert Simon, combining “satisfy” and “suffice.” Instead of optimizing (carefully evaluating every option to find the best answer), they settle for the first answer that seems good enough. Think of satisficing as efficiency under constraint, rather than mere laziness. It’s a rational response to the fact that your survey is the fourteenth thing demanding their attention today. But it means your carefully constructed response options may never get fully considered. The respondent sees an acceptable choice, clicks it, and moves on. The four options below it might as well not exist.
The foundational work here belongs to Daniel Kahneman and Amos Tversky, whose 1974 paper “Judgment under Uncertainty: Heuristics and Biases” launched a revolution in how we understand human decision-making. Kahneman later won the Nobel Prize in Economics for this research.
Think of heuristics as features, not bugs. They’re why you can navigate a grocery store without having an existential crisis at every shelf. But in survey research, they may become a source of error. Respondents are using shortcuts to generate answers, and those shortcuts can have predictable effects.
Here are the big ones.
Availability: What Comes to Mind Easily
The availability heuristic says that we judge the frequency or importance of things by how easily examples come to mind. If instances pop up quickly, we assume the phenomenon is common. If we have to work to think of examples, we assume it’s rare.
This is usually adaptive. Things that happen often are easier to remember. But it misleads when some events are memorable for reasons other than frequency.
Consider risk perception. Plane crashes and shark attacks are rare but vivid. They make the news. They generate memorable images. As a result, people systematically overestimate their likelihood. Meanwhile, mundane risks—car accidents, diabetes, the flu—kill far more people but generate less mental availability. We underestimate them. Tversky and Kahneman documented this pattern, and it’s been replicated countless times since.
In surveys, availability means that vivid and memorable experiences carry disproportionate weight. Ask someone to evaluate a brand, and the encounters that made an emotional impression will dominate their assessment even if those moments were exceptions. The danger compounds in group settings. In a focus group, one participant might mention Squirt as their favorite soda. Suddenly, everyone’s discussing niche beverages. Meanwhile, every person at the table is drinking Diet Coke. If you don’t recognize and control for the availability heuristic, it can anchor an entire conversation.
What to do about it: Vary time frames. Ask about specific periods rather than “in general.” Probe for less obvious examples. Don’t prime respondents with salient cases before asking for recall.
Anchoring: The Gravity of First Numbers
Here’s a weird one. Show people a number—any number—and it will pull their subsequent judgments toward it, even if the number is transparently irrelevant. This is known as the anchoring bias.
In one of Tversky and Kahneman’s most famous demonstrations, they asked people to estimate the percentage of African countries in the United Nations. But first, they spun a wheel of fortune that landed on either 10 or 65. The wheel was obviously random. Everybody knew the wheel was random. And yet people who saw 65 gave higher estimates than people who saw 10.
The anchor shouldn’t have mattered. But it did. Once a number is in your head, it exerts gravitational pull on nearby judgments. Subsequent research has shown that anchoring persists even when people are warned about it, even when they’re offered incentives for accuracy, and even among experts in their field of expertise.
In surveys, anchors are everywhere. They’re in the question stems (“Given that the average household spends $300 per month on groceries…”). They’re in previous questions. They’re in the response scale itself. And they’re shaping judgments in ways neither you nor the respondent can fully see.
What to do about it: Be intentional about every number respondents encounter before making a judgment. Randomize order when you have multiple numeric questions. Consider using open-ended formats before introducing specific quantities.
Representativeness: Does It Look Like the Thing?
The representativeness heuristic says we judge the likelihood that something belongs to a category by how much it resembles our mental prototype of that category.
Is a person who is “meticulous, introverted, and enjoys organizing systems” more likely to be a librarian or a sales representative? Most people say librarian. The description matches the stereotype. But this ignores base rates. There are far more salespeople than librarians in the working population. Statistically, the meticulous introvert is more likely to be selling software than shelving books. But the story feels right, so the statistics get ignored.
Kahneman and Tversky called this “base rate neglect,” and it’s one of the most robust findings in the judgment literature. We’re seduced by narrative fit and largely blind to underlying probabilities.
In survey research, representativeness means that vivid descriptions will trigger prototype matching. Describe a product user with specific personality traits, and respondents will evaluate the product through that lens even if the description came from your imagination rather than your data. The story overwrites the statistics.
What to do about it: Scrutinize descriptions for stereotype-activating details. Consider whether vivid language is adding information or activating bias. Be especially careful with personas and user profiles that feel compelling but lack empirical grounding.
Affect: Feelings Come First
The affect heuristic says that emotional responses precede and shape rational judgments. When we encounter something for the first time our opinions are formed by many factors–some rational, some not so rational. We have gut reactions— feelings—and these can overpower our responses. In fact, we might construct rational reasons to justify what we’re feeling. That short-circuiting can distort your findings if you don’t control for it.
Paul Slovic and his colleagues developed this concept in the early 2000s, showing that if something feels good, we perceive it as less risky and more beneficial. If something feels bad, we perceive it as more risky and less beneficial. The feeling comes first. The analysis is post-hoc rationalization.
In surveys, feelings are contagious. Put a product concept against a beautiful image, and the beauty inflates the concept rating. Respondents think they’re evaluating your idea. They’re actually evaluating how everything on the page made them feel.
What to do about it: Control the emotional context of evaluation tasks. Recognize that aesthetic pleasure (or displeasure) will contaminate judgments about focal stimuli. When you want to measure reactions to a specific element, be deliberate about what else is on the page—because respondents can’t cleanly separate their reaction to your concept from their reaction to everything surrounding it.
IV: The Biases We Don’t Admit
Heuristics are cognitive shortcuts. They are efficient, usually helpful, and occasionally misleading. Biases are a different matter. They’re systematic distortions that tilt responses in particular directions, often without respondents’ awareness.
Acquiescence: The Urge to Agree
Present respondents with a statement and ask whether they agree or disagree. Many will drift toward agreement, regardless of content.
This is called acquiescence bias, and it’s been documented since at least Lee Cronbach’s work in the 1940s. It probably reflects a mix of cognitive ease (agreeing requires less mental effort than disagreeing) and social dynamics (disagreement feels confrontational, even with a piece of paper). The result is a systematic tilt toward “yes” that inflates agreement with whatever statements you happen to offer.
If all your survey items are positively worded, acquiescence might inflate your scores across the board. You’ll think respondents endorse everything, when actually they were just being agreeable. This is why well-designed scales include reverse-coded items—statements where agreement indicates the opposite of what you’re measuring. It’s a check against mindless acquiescence.
What to do about it: Include negatively worded items. Use response formats other than agree/disagree when you can. Watch for suspiciously uniform agreement patterns that suggest the respondent is on autopilot.
Confirmation Bias: The Researcher’s Shadow
This one’s on you.
Researchers have hypotheses. We have theories we favor, conclusions we’re hoping to support. And confirmation bias means we’re more likely to notice evidence that confirms what we already believe and more likely to dismiss or overlook evidence that doesn’t.
The psychologist Peter Wason demonstrated this tendency in the 1960s with his famous “2-4-6 task,” showing that people systematically seek confirming evidence and fail to test their hypotheses by looking for disconfirmation. It’s human nature. But in research, it’s dangerous.
In survey work, confirmation bias shows up in how we interpret ambiguous data, which findings we emphasize in reports, and which anomalies we wave away as “noise.” The bias is usually unconscious. That’s what makes it so hard to fight.
What to do about it: Actively seek disconfirming evidence. Take surprising results seriously. Have someone outside your project review your analysis with fresh eyes. Be especially skeptical of findings that conveniently support what you wanted to be true.
V: Design Failures That Masquerade as Respondent Failures
Sometimes the problem isn’t in the respondent’s head. It’s in how the survey was designed. You can’t blame the player when the game is rigged.
Order Effects: First and Last Win
Items at the beginning and end of a list get disproportionate attention. Items in the middle get lost. This is true for response options, for questions in a sequence, and for stimuli in an evaluation task.
The psychological mechanisms are well-documented. Primacy effects occur because early items get more attention and processing time. Recency effects occur because late items are still active in working memory when the judgment is made. Middle items suffer from both—less initial processing and memory decay.
In a list of response options, early items will be selected more often simply because respondents start at the top and often stop when they find something acceptable. Krosnick and his colleagues have documented this pattern extensively, showing that response order can meaningfully shift survey results.
What to do about it: Randomize option order across respondents. Keep lists short. Don’t bury important options in the middle, where they’ll be overlooked.
Framing: The Words Change the Thing
There’s no such thing as a neutral question. Every word choice is a frame, and the frame shapes the response.
Tversky and Kahneman demonstrated this dramatically in 1981 with their “Asian disease” problem. Presented with the same statistical outcomes framed as lives saved versus lives lost, people reversed their preferences. The underlying reality was identical. The words changed the decision.
This extends to every aspect of survey design. “Government spending” polls differently than “public investment.” “Climate change” lands differently than “global warming.” “Undocumented immigrants” evokes different responses than “illegal aliens.” You cannot escape framing. You can only choose your frame deliberately and consider how it might be shaping your results.
What to do about it: Test alternative wordings when the stakes are high. Consider what associations your language might trigger. Be transparent in your reporting about the specific language you used—because someone using different language might get different results.
Self-Selection: The Sample Biases Itself
Who responds to your survey matters as much as what they say.
If your recruitment appeals to enthusiasts, you’ll get enthusiasts. If your survey is long and tedious, you’ll lose everyone except the highly motivated—who are not representative of the general population. If you’re studying customer satisfaction and only people with complaints bother to respond, your data will look worse than reality.
Non-response bias is the shadow lurking behind every survey. A 20% response rate means 80% of your target population declined to participate. What makes them different from the 20% who responded? You don’t know. But assuming they’re identical to responders is almost certainly wrong.
What to do about it: Scrutinize your recruitment for selection effects. Consider who might be missing from your sample. Weight your data if you have external benchmarks. Be humble about generalizability—your findings describe the people who answered, not necessarily the population you wanted to understand.
V ½: The New Frontier
Everything we’ve discussed so far makes a lot of important and often falable assumptions. It assumes a human respondent, sitting reasonably still, giving your survey their full attention.
The Mobile Reality
Most survey responses now happen on phones. Respondents are answering while waiting in line, half-watching television, lying in bed at midnight, or (sorry to be graphic) sitting on the toilet. The cognitive budget we discussed earlier? It just got slashed.
Mobile context amplifies every problem we’ve covered. Satisficing increases because attention is fragmented. Primacy effects intensify because fewer options are visible without scrolling—whatever fits on the first screen gets disproportionate selection. The affect heuristic becomes wildly unpredictable because you have no idea what emotional state surrounds the survey experience. Your carefully designed instrument is competing with text notifications, TikTok, and a toddler demanding snacks.
Distraction is the ever-present new reality of survey research, leading some managers to believe surveys are no longer effective at all for gathering consumer insight. That conclusion is premature. The real answer is to design for distraction. Keep your surveys short. Write simpler questions. Provide fewer response options. Avoid those nasty, attention-hogging matrix questions with numerous columns and rows for the respondent to navigate. And, most importantly, design for mobile first. Assume your respondent is taking the survey from their phone, sitting on a bus, wedged between someone’s unheadphoned TikTok scroll and another passenger’s conviction that everyone needs to hear their gym playlist.
The AI Transformation
Researchers are beginning to explore whether AI can replace human survey responses altogether, skipping the expense and hassle of real data collection by having LLMs simulate how people would answer. Why recruit a thousand distracted respondents that give you flawed data when you can generate a thousand synthetic ones with highly rational responses in seconds? Early results are mixed. Argyle and colleagues (2023) showed that GPT-3 could produce response patterns that matched demographic subgroups with surprising accuracy.
At first glance, this sounds like a big upgrade. An LLM doesn’t have social desirability concerns. It doesn’t experience retrieval failures. It doesn’t satisfice out of boredom.
But think carefully about what we just covered.
Social desirability bias creates survey noise, but it is also ever-present in human life. It’s a real psychological force that shapes real behavior. People really do buy the hybrid car partly because they want their neighbors to notice. They really do order the salad when eating with colleagues. They really do donate more generously when the contribution is public on a Venmo stream.
Consider a study conducted during a flu outbreak at a university. Researchers placed a sanitizer station at the entrance of a campus dining hall with two options: an eco-friendly, natural brand and a clinical, industrial-strength brand. When the station was positioned at the front of the room where everyone could see which bottle you reached for, students disproportionately chose the eco-friendly option. When researchers moved the station to the back of the room, out of public view, students switched to the industrial-strength sanitizer. Same students. Same flu. Different context, different choice.
In theory, the LLM responses would predict what was observed when the station was placed at the back of the room. But you just heard me tell you that when it was in the front, the other option was chosen. The LLM would not have predicted this.
In public, students wanted to signal environmental virtue. In private, they wanted to not get sick. An LLM simulating these students without social desirability concerns would miss half of this picture. It would tell you what people choose when no one is watching. But that’s not how a lot of life is lived.
The same logic applies to other “biases” that LLMs lack. Retrieval failures are more than memory glitches. They reflect what’s actually salient and accessible in a person’s mind when they make decisions. Satisficing represents how people actually navigate a world with too many choices and too little time. Think of that menu board at the new build-your-own poke place. Did you really read all the options?
These new synthetic samples and LLM response models offer a lot of promise. In one study that used the General Social Survey (GSS) for its questionnaire, the LLM matched human response 86% of the time. That’s impressive and it warrants further exploration. But for now, treat AI-generated responses as a tool for stress-testing your instrument, and as a supplement to other fieldwork, not as a substitute for real data. You’re trying to gauge human behavior, and much of our behavior is shaped by biases and heuristics.
VI: Living With Uncertainty
If you’ve made it this far, you might be feeling a little demoralized. Everywhere you look, there’s another source of bias, another way for the data to mislead you.
Good. That’s the appropriate reaction.
Survey research is imperfect. It’s more fragile than the crisp charts and precise percentages suggest. The numbers feel authoritative. You now know that underneath the numbers rest on a foundation of cognitive processes that are messy, context-dependent, and partially unknowable.
Rather than despair, what we need is humility plus rigor.
Pretest your instruments. Cognitive interviews take a few hours and can reveal comprehension failures, retrieval struggles, and response mapping problems before they contaminate your actual data. This is not optional. This is how professionals work.
Triangulate. Don’t rely on a single question or a single method. If multiple approaches converge on the same answer, you can have more confidence. If they diverge, you’ve learned something important—even if what you’ve learned is that the phenomenon is harder to measure than you thought.
Report your methods transparently. Let readers see how questions were worded, in what order, with what response options. The details that seem boring are the details that allow someone to evaluate whether your data mean what you claim they mean.
Stay curious about anomalies. When results surprise you, resist the urge to explain them away. The weird finding might be noise. Or it might be signal you weren’t expecting. Or it might be revealing a flaw in your instrument that affects everything else too. You want to know which.
Surveys remain one of the most powerful tools in marketing research. They let us reach populations we couldn’t otherwise access, quantify attitudes that would otherwise stay hidden, and make comparisons across groups and time. But they’re not magic, and they’re not neutral. They’re a measurement technology with known limitations.
The researchers who developed these tools did it because understanding the machinery makes us better at using it. Knowing where the weaknesses are lets you design around them.
Your job is to use these tools wisely—which starts with understanding the weird, wonderful, and occasionally humbling psychology of how people respond to the questions we ask.
References
Cronbach, L. J. (1946). Response sets and test validity. Educational and Psychological Measurement, 6(4), 475-494.
Crowne, D. P., & Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24(4), 349-354.
Kahneman, D., & Tversky, A. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124-1131.
Krosnick, J. A. (1999). Survey research. Annual Review of Psychology, 50, 537-567.
Schuman, H., & Presser, S. (1981). Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context. Academic Press.
Schwarz, N., & Clore, G. L. (1983). Mood, misattribution, and judgments of well-being: Informative and directive functions of affective states. Journal of Personality and Social Psychology, 45(3), 513-523.
Slovic, P., Finucane, M. L., Peters, E., & MacGregor, D. G. (2007). The affect heuristic. European Journal of Operational Research, 177(3), 1333-1352.
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The Psychology of Survey Response. Cambridge University Press.
Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211(4481), 453-458.
Wason, P. C. (1968). Reasoning about a rule. Quarterly Journal of Experimental Psychology, 20(3), 273-281.
Social Desirability: Looking Good to the Survey
People want to seem like good people. This is true in life, and it’s true in surveys—even anonymous ones. Respondents systematically overreport behaviors they perceive as virtuous and underreport behaviors they perceive as shameful.
The researchers Douglas Crowne and David Marlowe developed the first systematic measure of this tendency back in 1960, and the “Marlowe-Crowne Social Desirability Scale” remains a standard tool for detecting it. What they found is that some people are more prone to socially desirable responding than others, and certain topics trigger it more reliably.
Voter turnout is consistently overreported. So is charitable giving, exercise frequency, and consumption of vegetables. Meanwhile, alcohol intake, time spent on social media, and various other vices are underreported. Are respondents willfully lying? Not exactly. It’s a slight (often subconscious) thumb on the scale that causes us to present ourselves more favorably than our actual behavior would warrant.
Think of every survey response as a LinkedIn post. Nobody’s writing “Thrilled to announce I got fired for missing deadlines” or “Excited to share that I mass-applied to 200 jobs and mass-deleted the rejections.” They’re posting the humble brags, the grateful reflections on “the journey,” the performative vulnerability that somehow always ends with a promotion. Survey responses are the same theater, just without the “I’m happy to share…” preamble. Your respondents are projecting what they want the world to see, and perhaps even a version of themselves they’d like to be. Your job is to see past the highlight reel so that you’re capturing real life rather than a fictional life reel.
What to do about it: Assume respondents will shade answers toward whatever makes them look better. One useful technique is indirect or projective questioning—asking what “most people” think or why “someone might” behave a certain way. This gives respondents psychological cover to reveal attitudes they wouldn’t claim as their own. They’re ostensibly talking about others, but they’re often projecting their own views onto those hypothetical people. It’s not perfect—projective techniques introduce their own interpretive noise—but research suggests they reduce distortion on sensitive topics (Fisher, 1993). You can also frame socially undesirable behaviors as common (“Many people occasionally…”), which normalizes the behavior and makes honest responses feel less risky.