How to Choose a Data Gathering Method: A Decision Framework for Research Teams

There is no single best data gathering method, only the best fit for a particular question, audience, budget and deadline. Teams that choose well tend to follow the same reasoning each time, even if they never write it down. This guide turns that reasoning into a repeatable decision process, with a scoring approach you can adapt and three worked examples.

If you need a refresher on the options themselves, our overview of data collection methods in research describes each family. Here the focus is on choosing between them.

Start with the decision, not the method

The most common error in research design is picking the method first. Someone suggests "a quick survey", and the rest of the brief is bent to fit. Reverse the order.

Write down, in one or two sentences:

• The business or policy decision the research will inform.
• What you need to know to make it, phrased as questions.
• How precise the answer must be. A go or no-go on a concept needs less precision than a market size used in a forecast.
• When the answer is needed, working back from the decision date rather than forward from the kick-off.

The six questions that narrow the choice

Once the decision is clear, six questions eliminate most options quickly.

1. What kind of answer?Measurement (how many, how much) or explanation (why, how). This decides the quantitative or qualitative lead.
2. Who holds the answer?The target population, how rare it is and how easy it is to reach.
3. Is there a sampling frame?A list, panel or other source that covers that population, and how well.
4. How sensitive is the topic?Whether people will answer honestly with an interviewer, in a group, or only alone.
5. What will it cost to reach them?Incentives, recruitment effort, interviewer time and fieldwork length.
6. How long do you have?Calendar time for design, fieldwork and analysis.

Kind of answer

If the decision needs a number, such as share, incidence, preference between options or change over time, you need a structured technique with enough sample to read the subgroups that matter. If it needs understanding, you need conversation or observation. Many briefs need both, which is the territory of mixed designs covered in quantitative and qualitative data collection techniques.

Population and sampling frame

This is where method choice becomes sampling choice. A method for data collection is only as good as the source of respondents behind it.

• Probability samples start from a frame that gives everyone in the population a known chance of selection. They support stronger claims about the population but are slower and more costly to build.
• Non-probability samples, such as opt-in online panels, are faster and more economical, and they are widely used in commercial research. Their results depend on careful quotas, sound sourcing and quality checks.

Pew Research Center's description of its U.S. survey methodology shows one way of combining the two worlds: its American Trends Panel is probability-based, panel members are recruited offline, and surveys are then completed online, with tablets provided for people without internet access. Most commercial projects will not build a panel like that, but the example shows that the survey mode and the way the sample is recruited are separate decisions.

Sensitivity and survey mode

Topics such as health, money or workplace behaviour change how people answer. Self-completion, usually online, tends to suit sensitive questions because there is no interviewer to impress. Interviewer-led modes suit complex screening, low-literacy audiences and long interviews where encouragement keeps people engaged. Group settings are the wrong choice for anything people would not say in front of strangers.

Cost and time

Cost is driven by how hard the audience is to reach, how much effort you ask of them and how much interviewer or moderator time is needed. Time is driven by recruitment and by how many stages the design has. A two-stage design with a pause for analysis will always take longer than a single survey, and it is often worth it.

A simple scoring approach

When two or three methods remain, a weighted score makes the trade-offs explicit and easy to discuss with a client. Agree the criteria and their weights first, then score each option against them.

Fit to the research question90

Coverage of the target audience80

Data quality controls available75

Speed to results60

Cost per usable response55

Depth of explanation35

The weights above are an example for a fast concept screen, where reaching the right people and getting a clear answer quickly matter more than depth. A study exploring why customers leave would weight depth much higher and speed lower. The value is in the conversation the weights force, not in the arithmetic.

This comparison is deliberately qualitative. Treat it as a starting point and adjust it for your markets and audiences.

Three worked examples

Example 1: sizing a new product category

A consumer goods brand needs to know how many households buy a category and how often, across three countries. That is a measurement question across a broad population. An online survey with quotas on age, gender and region in each country is the natural lead, with secondary data used to set the quota targets. If one market has weak online coverage, a mixed-mode approach there is worth considering.

Example 2: understanding why enterprise clients churn

A software company wants to know why some large clients do not renew. The population is small, senior and known by name. In-depth interviews with former and current clients are the right lead. A short survey of the wider client base can follow to check how common each reason is.

Example 3: a sensitive health behaviour

A public health team wants to understand a stigmatised behaviour. Self-completion is likely to produce more candid answers than an interviewer. The team also needs to think carefully about consent, data minimisation and support information for participants, and the sample source has to reach the relevant group without exposing them.

Example 4: tracking satisfaction over time

A service business wants to monitor customer satisfaction every quarter. The key requirement is consistency: the same questions, the same mode and a comparable sample each wave, so that changes reflect customers rather than method. An online survey sent to a sample drawn from the customer list in the same way every quarter is usually the practical choice. The bigger risk is not the initial method choice but later drift, such as someone rewording a question or switching sample source mid-series. Lock the instrument and document any change.

When to revisit the choice

A method chosen at the proposal stage is not fixed. Revisit it if any of these happen before fieldwork starts:

• Feasibility comes back low. If the audience is much smaller or harder to reach than expected, a survey may need to become a set of interviews, or the quotas may need loosening with the client's agreement.
• The brief changes. A new question added late may need a different technique, not just extra questions.
• The timeline shrinks. A two-stage design squeezed into one stage often loses its main benefit, so it may be better to drop a stage cleanly.

Pre-launch checks for your chosen method

Before committing, run through a short checklist with the team and, ideally, the client.

• The method answers the decision written in the brief
• The sample source covers the population, with known gaps written down
• Feasibility has been checked per market and segment
• The mode suits the sensitivity of the topic
• Quotas and subgroup sizes support the analysis plan
• Data quality rules are agreed before launch
• Timeline includes time between stages for multi-stage designs

Data quality rules deserve particular attention in online work; our post on online survey data quality checks sets out the main ones. If you are weighing different interview, group and observation techniques, our guide to primary data collection techniques covers what each needs in the field.

Where Sampledge fits

Sampledge helps once the method is chosen and the work becomes operational. SampAudience lets you define an online audience and check feasibility and cost before you commit, which is useful when deciding whether an online design is realistic for a hard-to-reach group. SampTrek then manages the project itself: quotas, suppliers, live fieldwork and reconciliation. For agencies pricing several design options for a client, SampBidx keeps bids and proposals organised.

To see how the pieces connect, read how Sampledge works, or get in touch to talk through a current brief.