Data Collection Methods

Why This Matters

In biostatistics, the method you choose to collect data fundamentally shapes what conclusions you can draw and how confident you can be in those conclusions. You need to distinguish between methods that can establish causation versus those that only reveal association, and to recognize when each approach is appropriate given practical and ethical constraints.

The methods covered here demonstrate core principles like randomization, control, bias reduction, and generalizability. On exams, you'll need to identify which method fits a given research scenario, explain why certain designs are stronger for specific questions, and recognize the trade-offs inherent in each approach. Don't just memorize definitions; know what makes each method powerful and where its limitations lie.

---

Experimental Methods: Establishing Causation

Experiments are the gold standard for determining cause-and-effect relationships because the researcher controls who receives an intervention. By manipulating the independent variable and holding other factors constant, experiments isolate the effect of interest.

Experiments

Three features make experiments uniquely powerful:

• Manipulation of variables allows researchers to test cause-and-effect hypotheses directly. This is the only design that truly establishes causation.
• Random assignment to treatment and control groups distributes confounding variables evenly across groups, reducing selection bias and strengthening internal validity. Note that random assignment (who goes in which group) is different from random sampling (who gets into the study in the first place). Both matter, but random assignment is what makes experiments special for causal inference.
• Control groups provide a baseline for comparison, making it possible to attribute observed differences to the intervention rather than to external factors.

---

Observational Approaches: Studying What Exists

When randomization is unethical or impractical, researchers observe subjects without intervening. These methods sacrifice causal inference for real-world applicability and ethical feasibility.

Observational Studies

• No manipulation of variables. Researchers record exposures and outcomes as they naturally occur. For example, you can't randomly assign people to smoke for 20 years, so studying smoking's health effects requires observation.
• Confounding variables remain a major threat since groups may differ systematically in ways beyond the exposure of interest.
• Cohort and case-control designs are two common subtypes. Cohort studies start with exposed and unexposed groups and follow them forward to see who develops the outcome. Case-control studies start with people who already have the outcome (cases) and look backward to compare their exposures to those of healthy controls.

Longitudinal Studies

• Repeated measurements on the same subjects over time allow researchers to track individual change trajectories and developmental patterns.
• Temporal sequence can be established. Knowing that an exposure preceded an outcome strengthens causal arguments even without randomization.
• Attrition bias poses a significant threat as participants drop out over months or years. If dropout is related to the outcome being studied, the remaining sample becomes unrepresentative and results can be skewed.

Cross-Sectional Studies

• Single time point data collection provides a snapshot of a population, making these studies faster and cheaper than longitudinal designs.
• Prevalence (the proportion of people with a condition at a given time) can be estimated, which is useful for public health planning and hypothesis generation.
• Cannot establish causation because exposure and outcome are measured simultaneously. You can't determine which came first.

---

Survey-Based Methods: Gathering Self-Reported Data

When researchers need information about attitudes, behaviors, or experiences, they must ask participants directly. The structure and format of questions significantly influence data quality and the types of analysis possible.

Surveys

• Structured questionnaires yield quantitative data that can be statistically analyzed across large samples. They're ideal for measuring prevalence and associations.
• Administration modes (online, phone, in-person) each introduce different response biases and affect who participates. For instance, phone surveys may underrepresent younger adults, while online surveys may miss populations without internet access.
• Sampling strategy determines generalizability. A well-designed survey of 1,000 people can represent millions if the sample is truly random.

Interviews

• Qualitative depth allows exploration of complex topics, capturing nuance and context that closed-ended survey questions miss.
• Flexibility in format: Structured interviews standardize questions across all participants (higher reliability), while unstructured interviews follow participant responses wherever they lead (richer data, but harder to compare across participants).
• Interviewer effects can introduce bias through leading questions, tone, or the participant's desire to give socially acceptable answers (known as social desirability bias).

Focus Groups

• Group dynamics generate data through participant interaction. Ideas build on each other in ways individual interviews cannot capture.
• Moderator skill is critical for managing dominant personalities and encouraging quieter participants to contribute.
• Not generalizable to broader populations due to small, non-random samples, but excellent for exploratory research and generating hypotheses that can later be tested with larger studies.

---

Secondary and Specialized Approaches

Not all research requires collecting new data. Leveraging existing information or focusing intensively on specific cases can answer questions efficiently or reveal insights that large studies miss.

Secondary Data Analysis

• Existing datasets (hospital records, census data, prior studies) can be reanalyzed for new research questions without the cost of primary data collection.
• Limited control over how variables were originally measured. You're constrained by decisions the original researchers made, which means the data may not perfectly fit your question.
• Replication and validation of findings becomes possible when multiple researchers analyze the same data independently.

Case Studies

• Intensive analysis of one or a few cases provides rich contextual detail that's impossible in large-sample studies.
• Hypothesis generation is the primary strength. Unusual or rare cases can reveal mechanisms or patterns worth testing in larger populations.
• Low external validity means findings may not generalize beyond the specific case. Still, case studies excel at documenting rare conditions or exploring complex phenomena in depth.

---

Sampling: The Foundation of Valid Inference

How you select participants determines whether your findings apply beyond your sample. Probability sampling allows statistical inference to populations; non-probability sampling does not.

Sampling Techniques

Probability sampling gives every member of the population a known, non-zero chance of being selected. The main types are:

• Simple random sampling: Every individual has an equal chance of selection (like drawing names from a hat).
• Stratified sampling: The population is divided into subgroups (strata) based on a characteristic like age or sex, then random samples are drawn from each stratum. This ensures important subgroups are represented.
• Cluster sampling: Naturally occurring groups (like hospitals or schools) are randomly selected, and then everyone within those clusters is studied. This is more practical when a complete list of individuals isn't available.

Non-probability sampling is easier and cheaper but introduces selection bias:

• Convenience sampling: Recruiting whoever is readily available (e.g., students in your class).
• Purposive sampling: Deliberately selecting participants who meet specific criteria.
• Snowball sampling: Existing participants recruit others, useful for hard-to-reach populations.

A key principle: a large biased sample is worse than a smaller representative one. Sample size and representativeness jointly determine precision, but representativeness matters more for valid inference.

---

Quick Reference Table

---

Self-Check Questions

1. A researcher wants to determine whether a new drug lowers blood pressure. Which method would establish causation, and what two design features are essential?

2. Compare longitudinal and cross-sectional studies: what can longitudinal studies reveal that cross-sectional studies cannot, and why?

3. A public health team needs to estimate the current prevalence of diabetes in a city quickly and affordably. Which study design should they use, and what is its main limitation?

4. You're reviewing a study that used convenience sampling from a university campus to draw conclusions about all adults in the country. What type of validity is threatened, and why?

5. When would a researcher choose interviews over surveys, and what trade-off does this choice involve?