Epidemiology - Validity Bias and Error

Characterization, Validity, and Bias Understanding Measurement Error When we conduct epidemiologic studies, we measure exposures, outcomes, and other variables. However, our measurements are never perfect. There are two fundamental types of measurement error that can affect study results: Random Error Random error arises from unpredictable fluctuations during data collection, coding, transfer, or analysis. Think of it as noise in your data—sometimes measurements fall slightly above the true value, sometimes below, with no consistent pattern. This type of error comes from sampling variability, meaning that different samples from the same population will produce slightly different results just by chance. Precision is the key concept here. Higher precision means less random error. You can reduce random error by: Increasing your sample size (more observations provide more stable estimates) Using more precise measurement instruments or methods Careful standardization of data collection procedures When you see a confidence interval around a relative risk estimate, that interval reflects precision. A narrow confidence interval means high precision (low random error), while a wide interval means low precision (high random error). Systematic Error (Bias) Systematic error is fundamentally different from random error. It occurs when measurements consistently deviate from the true value due to a consistent source of bias. Unlike random error that cancels out across many measurements, systematic error pushes all measurements in the same direction—either consistently too high or too low. Two important validity concepts relate to systematic error: Internal validity is the accuracy of measurements and associations within your study sample. A study has good internal validity when it correctly measures what it claims to measure and the associations it finds are real (not due to bias). External validity is your ability to generalize findings beyond the studied sample to other populations. A study can have perfect internal validity but poor external validity if the study population is too different from the populations you want to apply the findings to. Biases in Epidemiologic Studies Understanding specific types of bias is essential because different biases require different solutions. Here are the major categories: Selection Bias (Differential Participation) Selection bias occurs when who participates in your study is related to the outcome you're measuring. More specifically, it happens when characteristics that affect participation are also associated with both exposure and outcome status. Here's the critical distinction: Selection bias only becomes a problem if the difference in participation is associated with a systematic difference in the outcome between exposed and unexposed groups. For example, imagine you're studying whether an occupational exposure causes cancer. If both exposed and unexposed workers are equally likely to participate in your study (perhaps both groups refuse participation at the same rate), then selection bias will not distort your results—even though you didn't enroll everyone. The key is that participation is unrelated to the outcome, so the people who participate adequately represent the people who didn't. However, if exposed workers are more likely to participate because they're more health-conscious and also more likely to be cancer-free, then selection bias is present and distorts your results. Information Bias (Systematic Measurement Error) Information bias arises from systematic errors in measuring or classifying study variables. This is different from selection bias—you're not missing certain people, but rather measuring the exposure, outcome, or other variables incorrectly for everyone (or for some groups systematically). Recall Bias Recall bias is particularly important for studies that rely on participants' memory. It occurs when participants' ability to remember past exposures differs by their outcome status. Here's a concrete example: You're conducting a case-control study on birth defects. You interview mothers of children with birth defects (cases) and mothers of healthy children (controls), asking them about infections during pregnancy. Mothers of affected children are likely to think more carefully about their pregnancies—they may have revisited the events in their minds many times, talked to doctors about what happened, and be motivated to remember details. In contrast, mothers of healthy children may not have thought as carefully about their pregnancies. If cases remember exposures better than controls, the association between the exposure and outcome will be overestimated. Immortal Time Bias (Design-Related Bias) Immortal time bias occurs in cohort studies with follow-up periods and emerges from how you classify time during which the outcome cannot possibly occur. Here's the conceptual issue: In some studies, there's a period of follow-up during which a participant is logically "immortal"—they cannot experience the outcome. If you incorrectly count this immortal time as exposure time, you artificially inflate the apparent protective effect of the exposure. Example: You study whether statin use prevents heart attacks. You recruit patients, classify them as exposed (on statins) or unexposed (not on statins), and follow them forward. But if a patient started taking statins during the follow-up period, the time before they started statins should not count as "exposed time." If you incorrectly count pre-exposure time as exposure time, you attribute person-years to the wrong exposure group, making the exposure appear more protective than it really is. The solution is straightforward: correctly assign each person-year of follow-up to the appropriate exposure category based on when the exposure actually occurred. Confounding (Mixing of Effects) Confounding is bias that results from the mixing of the effect of an extraneous factor (a confounder) with the effect of your exposure of interest. Unlike the biases above, confounding stems from real causal relationships, not from measurement error or study design problems. The Counterfactual Framework To understand confounding precisely, we use counterfactual thinking. The true causal effect is what would happen if we could observe the same person in two states: exposed and unexposed. The true causal effect for a population is the difference (or ratio) between: $R{A1}$: the risk of the outcome if the population were exposed $R{A0}$: the risk of the outcome if the population were unexposed The problem is that $R{A0}$ cannot be observed—we can't have the same person exist in both exposed and unexposed states simultaneously. So in real studies, we compare two populations: Population A (the exposed group): we observe their risk $R{A1}$ Population B (the unexposed group): we observe their risk $R{B0}$ We calculate measured contrasts like $R{A1} - R{B0}$ or $R{A1} / R{B0}$ and use these as estimates of the true causal effect. When Confounding Occurs Confounding is present when the unobserved risk $R{A0}$ (the risk in the exposed population if they were unexposed) differs from the comparator risk $R{B0}$ (the actual risk in the unexposed population). In other words, if the two groups differ in ways that would affect their outcome risk aside from the exposure, those differences confound the association. Key distinction: Unlike selection bias or information bias, which result from flawed study methodology, confounding results from real differences between the groups being compared. The confounder has a genuine causal effect on the outcome, and it differs between exposure groups.