Quantitative Research

Overview

Introduction to survey research and sampling methods for quantitative customer research.

Presented by:
Larry Vincent,
Professor of the Practice
Marketing

Presented to:
MKT 512
October 7, 2025

Why survey research

Provides systematic way to understand attitudes, behaviors, and experiences at scale
Enables population statistical inferences from small samples
Generates standardized, quantifiable data for decision-making
More cost-effective than other research methods
Critical and flexible validator for decision-making

Modalities

Online
Telephone
Mail
In-person intercept

A note on sample size

You’ve learned sampling principles in our previous session. The assigned HBS reading provides formulas for:

Calculating confidence intervals
Determining required sample size for desired precision
Understanding standard error

We won’t re-cover these statistical mechanics today. Questions? Office hours or async discussion.

Today’s focus: Once you have your sample, how do you design scales that minimize measurement error?

Design principles

What type of study?

Basic structure

Screener

Most surveys have upfront questions used to determine if respondent is qualified to take survey.
Screener questions may also be used to group respondents within the survey flow, and for managing quotas.
Some questions in screener may be used to gather information from respondent that could bias or “prime” questions in main body of study.
When using paid sample, it is important to ask qualification questions early, and provide quick termination if respondent is disqualified.
In some instances (i.e. surveying your existing customers), you may not need a screener because you can qualify and group respondents with internal business data.
Panel providers can help qualify respondents using profiling data; however, you may still want to validate with a screener because circumstances change.

The screener is your first line of defense for quality control.

Quotas

Quotas ensure sample representativeness. They help your study match population characteristics (age, gender, income, etc.) and they prevent over/under-representation of specific groups.

Benefits:

Ensure statistical validity and reliability
Reduce sampling bias
Enable meaningful subgroup analysis
Control research costs while maintaining quality

Types:

Hard quotas: Strict limit that cannot be exceeded–survey closes when filled
Soft quotas: Flexible targets–survey does not automatically close and may exceed target

Quality control

Common quality issues

Lack of attention
Speeding
Straight-lining
Gibberish in open-ends
Bots and use of AI
“Gaming” the system

Prevention

Keep you surveys as brief as possible; and mobile friendly.
Use your survey platform’s features to identify speeders–you can choose to give them a warning or immediately terminate their interview.
Use a survey platform with fraud detection features (good for identifying bots).
Some platforms can identify straight-liners real-time; for others you will have to identify and remove those responses in data analysis (SD = 0).
AI is increasingly helpful at finding gibberish responses and assessing quality of open-ends.
Make your survey enjoyable to take!

Methodological Tips

Rotate!

Earlier in the course we discussed recency and primacy biases. To overcome these heuristics in your survey research it is usually advisable to rotate the order of statements in a categorical question for each new respondent. While it is advisable to rotate categorical scales, you should not rotate ordinal scales.

Ordinal scales

How many scale points?
Unipolar or biploar?
Odd or even numbered?
Where to anchor?

How much precision is needed?

How much neutrality is allowable?

Number of scale points

Even-Numbered

Forces respondents to take a position
(no middle ground)
Good when you want to minimize social desirability bias
Useful when neutral responses aren’t meaningful for your research
Helpful when evaluating controversial or sensitive topics
Best when respondents should reasonably have an opinion

Odd-Numbered

Allows for true neutral responses
Better reflects natural human judgment patterns
Appropriate when respondents might genuinely be neutral
Good for topics where people may lack experience/knowledge
Reduces respondent frustration by not forcing choice

Deciding even/odd

Consider if neutral is a valid response for your research question
Think about your target audience’s familiarity with the topic
Evaluate if forced choice might bias your results
Assess whether you need to measure direction or just intensity
Match your scale choice to your research objectives

To label or not to label

To label or not to label

Labeling scale points

Labeling scale points is usually a best practice; however, as the number of scale points increases, the labels may introduce subjectiveness that results in response errors.

Use Labels for Clarity
Labeling all points can reduce ambiguity, especially if the scale is longer. This helps ensure that all respondents interpret the scale in a similar way, which is particularly important if you’re seeking detailed distinctions in satisfaction levels.
When to Leave Points Unlabeled
If the scale is very granular (e.g., a 7- or 11-point scale), leaving the middle points unlabeled can be appropriate to reduce potential confusion and subjective interpretation. In such cases, you might only label the endpoints (e.g., “Not Satisfied at All” and “Completely Satisfied”), as it gives respondents some flexibility to rate their feelings without specific wording influencing their choices.
Balanced Labels for Consistency
If labeling, ensure the labels are equidistant in their meaning. For example, avoid combining an objective term with a subjective one, like “Slightly Satisfied” followed by “Fairly Satisfied,” as this could lead to inconsistent interpretations.

Two types of scales

Degree of representation vs. polarity

Scale choice

Unipolar

Best when measuring the intensity/magnitude of a single concept or feeling
Use when the opposite of the concept is just its absence (e.g., not confident)
Preferred for measuring frequency
(Never → Very often)
More intuitive for respondents when the concept naturally starts at zero
Good for concepts like intent, agreement, or importance
When used for satisfaction, measures intensity of construct

Bipolar

Best when measuring concepts with true opposites
Use when both positive and negative ends are meaningful and distinct
Effective for measuring attitudes or emotions (Love ↔︎ Hate)
Good for comparative judgments
(Too slow ↔︎ Too fast)
Works well for semantic differential scales
(Good ↔︎ Bad)
When used for satisfaction, measures volatility of construct

Selecting polarity

Consider how people naturally think about the concept
Test if respondents view “not satisfied” as the same as “dissatisfied”
Check if zero point interpretation makes sense
Ensure both ends of bipolar scales are true opposites
Keep consistent scale type within similar question sets

Anchoring

War of the biases

Acquiescence bias–pushes people toward agreeing with statements
Negativity bias–pushes people toward disagreeing with negatively worded statements

Anchoring

Default to negative → positive / low → high
(e.g., Strongly disagree → Strongly agree)
Match natural cognitive progression from absence to presence
Follow reading direction (left-to-right in Western cultures)
Keep consistent direction across all scales in your survey
Consider cultural context for international research
Be consistent

Other anchoring challenges

Your turn

Your client wants to ask about household income. They propose showing options HIGH to LOW because “our target customers are affluent and we want the survey to feel premium.”

Considerations

What’s your recommendation on order?
How do you explain it to the client?
What would you say about their “premium feel” concern?

Results

Anchoring best practices

Response order affects answers—not just anchor labels
Primacy effects favor options presented first/left
Ascending order (low → high) minimizes bias
For sensitive questions, ascending order feels less presumptive

Digital considerations

Vertical scales: Negative/low at top is standard
Radio buttons: Follow left-to-right progression
Drop-down menus: Start with negative/low options
Mobile surveys: Consider thumb-reaching patterns
Visual analog scales: Left = low, Right = high

Exceptions

Academic grading scales (A → F) may follow cultural norms
Price sensitivity questions might work better high → low
Match industry standards for your specific field
Consider prior surveys if comparing to historical data
Specialized scales (e.g., Net Promoter Score) have set formats

Cognitive considerations

People typically process “bad to good” more naturally
Mental number lines usually run low to high
Starting negative/low reduces satisficing behavior
Consistent direction reduces cognitive load
Clear endpoints help anchor respondent understanding

Your turn

You’re measuring satisfaction with a NEW campus food delivery service. It launched 2 weeks ago. You need to understand if students are happy or identify problems quickly.

Considerations

Unipolar or bipolar? (Why?)
How many points? (Why?)
Odd or even? (Why?)
What anchors? (Which end? What labels?)

Next steps

Takeaways

Good survey design balances methodological rigor with respondent experience
Scale choices (polarity, points, anchoring) should align with how people naturally think
Quality control starts with screener design and continues through data collection
Consistency in scale direction and presentation reduces cognitive load, and prevents over/under-reporting
Rotate options to minimize bias, but maintain fixed scales when measuring ordinal data