Fieldwork Management

Overview

Tips and tricks to make your data collection easier.

Presented by:
Larry Vincent,
Professor of the Practice
Marketing

Presented to:
MKT 512
February 17, 2026

Today’s Agenda

1. Pre-testing & Soft Launch — Catching problems before they become data
2. Respondent Experience — The UX of surveys
3. Live Demo: Guardant Health — Display logic, DQ termination, quotas, and tracking in Qualtrics
4. Sample Suppliers — Where respondents come from (briefly)

Pre-Testing &
Soft Launch

Two Kinds of Pre-Testing

There’s a critical distinction between testing whether your survey makes sense and testing whether it works.

	Cognitive Pre-Test	Technical Pre-Test
Question	Do respondents understand what we’re asking?	Does the survey work as programmed?
Method	Think-aloud protocols with 5–8 people	Take the survey yourself, every path
Catches	Confusing wording, unfamiliar terms, double-barreled questions	Broken skip logic, missing options, display errors
When	Before programming (ideally) or right after	After programming, before any respondents

Ask: “How many of you have tested your KCRW survey by having someone outside your team take it?” This is the cognitive pre-test. Then ask: “How many of you have tested every possible skip-logic path?” That’s the technical pre-test.

The Technical QA Checklist

Before a single real respondent sees your survey:

Logic & Flow

• Every skip pattern tested end-to-end
• Conditional display logic verified
• Randomization working correctly
• No dead-end paths
• Back button behavior checked
  (do you want them to go backward?)

Content & Display

• Every question displays correctly
• Response options are complete
• No typos or formatting errors
• Progress bar is accurate
• Thank-you / redirect page works

Transition: “You’ll see exactly how display logic and termination paths work when we look at the Guardant Health survey in a few minutes. For now, the point is: you need to test every path before you launch.”

The Soft Launch

A soft launch means fielding to 5–10% of your target sample before opening the full study. You’re looking at:

Completion rate — If it’s below ~80%, something is driving people out.

Median duration — Compare this to your estimate. If you estimated 8 minutes and actual median is 14, you have a problem.

Drop-off analysis — Where are people abandoning? The question before the biggest drop-off point is usually the culprit.

Open-end quality — Are people writing thoughtful responses or pasting gibberish?

Distribution checks — Are your key variables showing reasonable variance, or is everyone selecting the same option?

Key insight: A soft launch is not a pilot study. You’re not testing hypotheses — you’re testing the instrument. If everything looks good, these respondents become part of your final dataset. If something is broken, you discard them and fix it.

Pair Activity: Soft Launch Diagnostic

Your team soft-launched a 25-question survey to 40 respondents. Here’s what you found:

Metric	Result
Completion rate	62%
Median duration	18 min (estimated: 10 min)
Largest drop-off	Between Q11 and Q12 (lost 22%)
Q7 (income)	35% selected “Prefer not to say”
Q19 (open-end: “Describe your ideal experience”)	60% wrote ≤ 3 words

With your partner: Diagnose what’s happening and propose fixes. Be specific — don’t just say “make it shorter.”

Debrief: Q11-Q12 drop-off likely means Q12 is problematic (too sensitive? too complex? page break issue?). 18-min median means the survey is roughly twice as long as it should be. The income question needs a reframe or relocation. The open-end is too vague — respondents don’t know what “ideal experience” means without context. Completion rate of 62% is a red flag for data quality even among completers.

Respondent
Experience

You Are Designing an Experience

Every survey is a product. Your respondent is the user. And just like any product, if the experience is bad, people leave — or worse, they stay and give you bad data.

The average American is invited to take dozens of surveys per year. Your survey is competing for attention with everything else on their phone.

This means respondent experience is a data quality strategy.

Ask students: “Think about the last survey you abandoned. Why did you quit?” Collect a few answers. They’ll say: too long, boring, confusing, felt like it would never end. These are all design failures.

The Length Problem

The single biggest driver of survey abandonment is perceived length. Note the word perceived — it’s not just actual minutes, it’s how long the survey feels.

What the research tells us:

• Surveys over 15 minutes see sharp drops in completion and data quality
• After ~10 minutes, satisficing behavior increases significantly
• Respondents estimate time based on the first few questions — if those are slow, they assume the rest will be too

The uncomfortable truth: Every question you add has a cost. Not in dollars (though it does cost more) — in data quality across the entire survey. A 30-question survey with focused, engaged respondents produces better data than a 60-question survey where people are racing to finish.

Connect this to the KCRW project: “How long are your KCRW surveys right now? Have you timed yourself taking it? That’s your homework tonight.”

Mobile-First Is Not Optional

More than 60% of online survey responses now come from mobile devices. If you haven’t tested your survey on a phone, you haven’t tested your survey.

What breaks on mobile

• Grid/matrix questions require horizontal scrolling
• Long response option text gets truncated
• Images don’t resize properly
• Progress feels slower (more scrolling per question)

Mobile-friendly design

• Single-column layouts only
• Replace grids with individual rating questions
• Keep response option text short
• Use vertical radio buttons, not horizontal
• Test on an actual phone, not just a browser preview

Qualtrics has a mobile preview, but it’s not always accurate. The gold standard is to pull the survey up on your phone and take it. This is especially relevant for the KCRW audience — younger digital-first listeners will overwhelmingly respond on mobile.

The Psychology of Progress

How you signal progress profoundly affects respondent behavior.

Progress bars — Useful when accurate. Harmful when they jump unpredictably (especially backwards, which happens with skip logic). If your survey has complex branching, consider hiding the progress bar entirely.

Page structure — One question per page feels slower but produces better data. Many questions per page feels faster but increases satisficing.

Sectioning — “You’re now starting Section 3 of 4” gives respondents a sense of structure and endpoint. This is especially valuable in longer surveys. But it can also be daunting.

The first impression — The opening questions set expectations. If Q1 is a boring demographic question, you’ve told the respondent this survey isn’t worth their attention. Lead with something engaging.

Practical tip: In Qualtrics, you can add descriptive text blocks between sections. A simple “Thanks for your responses so far — just two more sections to go” can measurably reduce drop-off.

The Respondent’s Internal Monologue

Every survey interaction triggers an internal cost-benefit calculation:

“Is this worth my time?” → Your intro screen and first question

“How much longer is this going to take?” → Your progress indicators

“Why are they asking me this?” → Your question relevance and framing

“I don’t understand this question but I’ll just pick something” → Satisficing from confusion

“I’m done caring” → Fatigue from length or monotony

Your job as a researcher is to keep respondents in the first three states and out of the last two — for the entire survey.

This slide is the conceptual takeaway for the block. Everything in survey UX reduces to managing this internal monologue. Transition: “Now let’s look at how all of this comes together in a real survey. I’m going to show you a study I fielded for Guardant Health.”

Live Demo

Where Do Respondents
Come From?

A Brief Word on Sample Suppliers

When you’re not recruiting your own sample (the way you are for KCRW), respondents typically come from panel companies — firms that maintain databases of people who’ve agreed to take surveys for compensation.

The key economics: What you pay per completed survey (CPI) is driven by three things:

• Survey length — Longer = more expensive
• Incidence rate — The rarer your target, the more it costs to find them
• Quota complexity — More constraints = harder to fill = higher cost

A general population, 10-minute survey might cost $5–8 per complete. A 20-minute survey targeting C-suite healthcare executives could run $100+.

You will work with these companies in your careers. Understanding their economics helps you design studies that are both rigorous and feasible.

Keep this brief — 5 minutes max. The point isn’t to make them experts on sample procurement. It’s to give them vocabulary so they’re not lost when it comes up in a job interview or their first week at work. Transition: “For KCRW, you’re being the sample supplier — you’re finding your own respondents. Which means the fielding management skills we covered today are even more important.”

Putting It
All Together

The Fielding Lifecycle

Phase	What You Do	Common Pitfalls
Design QA	Cognitive pre-test, technical test all paths	Skipping this because “we’re running out of time”
Soft Launch	Field to 5–10%, review diagnostics	Launching full field without checking soft launch data
Full Field	Monitor quotas, check quality daily	Set-it-and-forget-it mentality
Mid-Field	Communicate with team, adjust if needed	Not flagging fill-rate problems early
Close	Final quality checks, close survey, export data	Forgetting to close the survey

Every shortcut in this process shows up in your data — and by then, it’s too late to fix.

For Your KCRW Surveys

Before you field, make sure you’ve done the following:

This week:

☐ Have at least 2 people outside your team take the survey (cognitive pre-test)

☐ Test every display logic and skip path in Qualtrics

☐ Take the survey on your phone

☐ Time yourself — is the LOI what you expected?

☐ Set up embedded data to track your distribution channels

When you launch:

☐ Soft launch to your first 10–15 respondents

☐ Check completion rate, median duration, and drop-off points

☐ Review open-end response quality

☐ Check your quota fill rates (if using quotas)

☐ Only then open to full distribution

End the session here. This gives them a concrete, actionable checklist for their KCRW work this week.