Estimands

In any scientific endeavor, moving from raw data to a meaningful conclusion is a foundational challenge. We often start with a simple question, such as "Does a new drug work?" but find that real-world complexities quickly make such a question too vague to answer reliably. The key to rigorous, reproducible science lies in first establishing an unambiguous target for our inquiry. This precise target—the "what" we want to know at a population level—is called an estimand. This article addresses the critical knowledge gap between forming a general research question and specifying a precise, answerable one, a gap that often leads to ambiguous or conflicting results.

This article will guide you through the estimand framework, a powerful tool for achieving clarity in research. In the "Principles and Mechanisms" chapter, you will learn the fundamental distinctions between an estimand, an estimator, and an estimate. We will deconstruct the five pillars required to build a precise estimand and explore how this framework helps us navigate common research challenges like patient dropouts and unexpected events. Following this, the "Applications and Interdisciplinary Connections" chapter will demonstrate the estimand's transformative impact in practice. You will see how it brings clarity to clinical trials in medicine, synthesizes knowledge in meta-analyses, and provides a universal language for discovery in fields from public health to genomics.

Imagine you want to know a simple fact: the average height of every adult in France. There is a single, true number out there, a characteristic of the entire population of France. It’s an ideal, a perfect piece of knowledge we want to capture. In the language of science, this target quantity—this perfect, population-level fact—is called the estimand​. It's the "what" we want to know.

Now, you can't possibly measure all 50 million adults. It’s impractical. So, you devise a strategy. You'll randomly sample, say, 1,000 people, measure their heights, and calculate the average of that sample. This rule, this procedure for getting from a sample of data to a guess about the truth, is the estimator​. Before you've collected the data, the estimator is a curious thing; it's a random variable, because its value will depend entirely on the luck of the draw—which 1,000 people you happen to pick.

Finally, you execute your plan. You measure the 1,000 people, you punch the numbers into a calculator, and out comes a value: 175.6 cm. This concrete number, this result of your procedure on your specific data, is the estimate​. It is your single best guess for the estimand.

These three concepts—estimand, estimator, and estimate—are the three musketeers of statistical inference. The estimand is the truth we seek. The estimator is the method we use to hunt for it. The estimate is the trophy we bring back from the hunt. In a well-designed study, like measuring biomarkers in a patient population, we choose our estimator carefully. We might use the sample mean, $\bar{X}_n$, as our estimator for the true population mean, $\mu$, because it has beautiful properties. On average, it hits the target (a property called unbiasedness​), and as our sample size $n$ grows, it gets closer and closer to the true value (a property called consistency​). This doesn't require the data to follow a perfect bell curve; it works under much broader conditions, making it a robust and reliable tool.

This framework seems straightforward enough for measuring a static quantity like average height. But science is rarely so simple. We usually want to ask more dynamic questions, like "What is the effect of a new drug on lowering blood pressure?" At first glance, this question also seems simple. But let's look closer, because the universe is a messy place, and the people in it are complex.

Imagine we are running a clinical trial for a new drug designed to treat a serious inflammatory disease. We randomize patients into two groups: one gets the new drug, the other a placebo. We plan to measure their disease activity at the end of 24 weeks. But what happens when real life intervenes?

• A patient in the new drug group has a severe side effect and has to stop taking the medication at week 4. What is the drug's "effect at 24 weeks" for them?
• A patient in the placebo group finds their symptoms so unbearable that at week 12 their doctor gives them a powerful, standard-of-care steroid to manage the disease. This is called "rescue medication." How can we measure the effect of the placebo when this patient is now on a different, powerful drug?
• Tragically, another patient in the trial dies at week 20. Their disease activity cannot be measured at week 24. What do we do with them?

These events—treatment discontinuation, use of rescue medication, death—are called intercurrent events​. They are not just annoyances; they are fundamental challenges to the very meaning of our question. If we just ignore these patients, our study becomes biased. If one researcher decides to use the last measurement taken before a patient dropped out, and another decides to label them as "failures," they will get completely different answers from the exact same trial data. Both could claim to be measuring the "drug's effect," but they would be talking about different things. This is not a recipe for reliable scientific knowledge. Science demands precision, and the vague question "What is the effect of the drug?" is simply not precise enough.

To escape this ambiguity, the modern scientific community, particularly in medicine, has adopted a powerful discipline: you must specify exactly what you mean by "effect." You must build your scientific question with the rigor of an engineer. This precisely constructed question is your estimand, and it stands on five pillars:

1. The Population: Who are we talking about? We must define the group to whom our conclusion applies. (e.g., "Adults over 18 with stage-2 hypertension who meet the trial's entry criteria.")

2. The Variable: What, precisely, are we measuring? (e.g., "The change in systolic blood pressure from baseline to week 24.")

3. The Intervention: What is the exact comparison we are making? (e.g., "150 mg of the new drug taken once daily" versus "a matched placebo taken once daily.")

4. The Handling of Intercurrent Events: How do we account for life's messiness? This is the revolutionary step. We must declare, in advance, our strategy for every anticipated intercurrent event. Will we measure the outcome regardless of adherence? Will we define the outcome in a hypothetical world where the event didn't happen? This choice defines the question.

5. The Summary Measure: How will we summarize the effect for the whole population? (e.g., "The difference in the mean (average) outcome between the two intervention groups.")

When you have specified these five things, you don't just have a vague goal. You have an estimand​. You have a question so precise that it has only one right answer.

Here is the most beautiful and profound insight of the estimand framework: there is often no single, "true" effect of a drug. There are different, valid questions one can ask, and the "right" question depends on who is asking it and for what purpose.

Let's return to our trial where patients might need rescue medication. We can define at least two different, perfectly valid estimands:

• The Treatment-Policy Estimand: Imagine you are a doctor or a health regulator. Your main concern is what will happen in the real world. You ask, "If I prescribe this new drug to my patients, what will their overall outcome be, knowing that some of them might end up needing rescue medication anyway?" This is a pragmatic question about the effectiveness of a treatment strategy as a whole. To answer it, we use a treatment-policy strategy for our estimand: we measure the patients' outcomes exactly as they occur, rescue medication and all. We embrace the messiness of reality because the messiness is part of the question.

• The Hypothetical Estimand: Now, imagine you are a scientist in a lab, trying to understand the pure biological mechanism of the drug. You might ask, "What is the effect of this drug on its own, in a perfect world where no one ever takes rescue medication?" This is a very different question. It's not about the real-world policy; it's about the drug's direct causal power. To answer it, we use a hypothetical strategy: we define our estimand in a counterfactual world where the intercurrent event is forbidden. This requires different, and often more complex, statistical methods, but it answers a different and equally important scientific question.

The estimand framework doesn't tell you which question to ask. It forces you to be honest and clear about which question you are asking. The choice of estimand—specifically, the strategy for handling intercurrent events—is the choice of the scientific adventure you are embarking on.

To truly appreciate the depth of an estimand, we must go one level deeper and ask what a "causal effect" really is. The most elegant way to think about this is through the lens of potential outcomes​.

For any single person in our trial, we can imagine two parallel universes. In one, they receive the new drug, and their blood pressure at 24 weeks is a certain value, which we'll call $Y(1)$. In the other, they receive the placebo, and their blood pressure is $Y(0)$. For that individual, the true, personal causal effect of the drug is the difference: $Y(1) - Y(0)$.

Of course, we face the fundamental problem of causal inference​: we can never observe both potential outcomes for the same person at the same time. We can't send someone down both paths of the multiverse.

So, we aim for the next best thing: the average causal effect across our entire target population. This is the Average Treatment Effect (ATE), defined as $E[Y(1) - Y(0)]$. This is the ultimate, causally-defined estimand. It’s the average difference we would see if we could somehow give everyone the drug and then, turning back time, give everyone the placebo and compare the results.

This is where the magic of a Randomized Controlled Trial (RCT) comes in. By randomly assigning people to either the drug or the placebo, we create two groups that are, on average, identical in every respect at the start of the trial. They have the same average age, the same distribution of disease severity, the same mix of everything. Because they are balanced, the difference in the average outcomes we observe between the two groups becomes a valid estimate of the unobservable ATE we truly care about. Randomization builds a bridge from the data we can see to the causal truth we want to know. This whole elegant structure rests on some foundational assumptions, chief among them the Stable Unit Treatment Value Assumption (SUTVA), which essentially states that your outcome depends only on the treatment you received (not your neighbor's) and that there aren't hidden, different versions of the treatment.

Once we have gone to all the trouble of carefully defining our estimand, we must be vigilant not to accidentally switch the question during our analysis. The five-pillared structure gives us a precise checklist to ensure we stay on track.

Suppose our pre-specified estimand targets the difference in mean blood pressure change. If, after seeing the data, we decide it looks better to report the odds ratio of achieving a "responder" status (e.g., a drop of at least 10 mmHg), we have not just changed our analysis. We have changed the estimand. We changed the variable (from a continuous number to a binary yes/no) and the summary measure (from a difference to a ratio). This new result, however impressive, does not answer our original question.

Likewise, analyzing the data on a logarithmic scale might be statistically convenient, but it changes the estimand from a difference of arithmetic means to a ratio of geometric means—a fundamentally different summary of the effect. In contrast, simple shifts, like subtracting a constant from every measurement, often leave the core estimand unchanged, as the difference between the groups remains the same.

This discipline is not mere pedantry. It is the bedrock of reliable and reproducible science. By defining our estimand with precision, we draw a clear map of our scientific question. This map guides not only our primary analysis but also how we handle inevitable complications like missing data, ensuring that our methods remain aligned with our goal. The estimand framework transforms the potentially chaotic process of interpreting trial data into a logical, transparent, and unified journey from a precise question to a trustworthy answer.

Having grappled with the principles of what an estimand is, you might be thinking, "This is all rather abstract. A nice piece of logical hygiene, perhaps, but what does it do​?" This is a fair question, and the answer is exhilarating. The estimand framework is not just a philosophical footnote; it is the very engine of modern scientific discovery, the crucial bridge between a fuzzy, real-world question and a sharp, reliable answer. It is the tool that allows us to navigate the unavoidable messiness of reality—from patients who don't take their medicine to the inherent randomness of gene expression—and still arrive at a piece of solid knowledge.

Our journey to see this in action will begin in the demanding world of medicine, where lives literally depend on asking the right question. We will then see how this same disciplined thinking extends outward, providing a common language for discovery in fields as disparate as genetics and public health.

Imagine you are a doctor. A new drug has been developed to lower high blood pressure. The big, important question is simple: "Does this drug work?" But what does "work" mean? This is where the trouble—and the fun—begins.

A modern clinical trial won't settle for such a vague question. It will force us to be precise. For instance, in a trial comparing a new drug (let's call it Treatment 1) to an old one (Treatment 2), the question might be refined to: “What is the average difference in systolic blood pressure at 12 weeks between patients assigned to Treatment 1 versus those assigned to Treatment 2?”.

Suddenly, we have a precise target. It's not just a vague notion of "working." It is a specific, measurable quantity: the difference between the average blood pressure in two well-defined populations. In mathematical shorthand, we're targeting $\mu_1 - \mu_2$, where $\mu_j$ is the true, population-level average blood pressure for everyone who follows strategy $j$. This target, $\mu_1 - \mu_2$, is our estimand. It is the clear, unambiguous destination for our scientific journey. Everything else—the statistical test, the p-value, the confidence interval—is just the vehicle we use to get there.

But reality, as it often does, throws a wrench in the works. In the real world, people are not perfect automatons. In a trial of a new vaccine, some people assigned to get the vaccine might miss their appointment. In a study of a lifestyle-coaching app, some people assigned to use the app might forget about it, while some in the control group might download a similar app on their own.

This is where the true power of the estimand framework shines. It forces us to confront this messiness head-on and decide what question we are really asking.

Effectiveness vs. Efficacy: Two Sides of the "Truth"

Consider a vaccine trial. We could be asking two very different questions:

1. The "Treatment Policy" Question​: What is the overall public health benefit of a policy that offers this vaccine to a population? This question accepts the real-world messiness. It counts infections in everyone, including those who didn't get their shot, because that's part of what happens when you roll out a vaccination program. This estimand measures the vaccine's effectiveness​. Formally, it's the effect of assignment to the vaccine, often called the intention-to-treat (ITT) effect.

2. The "Hypothetical" Question​: What is the biological effect of the vaccine in a person who actually receives it as intended? This question wants to isolate the pure, biological protective mechanism of the vaccine. It asks what would have happened if everyone had perfectly followed the protocol. This estimand measures the vaccine's efficacy​.

Notice that neither question is "more correct" than the other! They are simply different questions, serving different purposes. A public health official deciding on a nationwide vaccination campaign cares deeply about the "treatment policy" estimand, because it predicts the real-world impact. A virologist trying to understand the vaccine's mechanism of action might be more interested in the "hypothetical" estimand.

The estimand framework gives us the vocabulary to distinguish these. The "treatment policy" estimand is what we target when we say we want the effect of assignment, regardless of what happens afterward. The "per-protocol" or "hypothetical" estimand, on the other hand, targets a causal effect in an idealized world—a world where, contrary to fact, everyone followed the rules perfectly.

You might be tempted to think that to estimate the "per-protocol" effect, you can just analyze the people who followed the rules. This is a trap! The group of people who choose to adhere perfectly to a protocol are often systematically different from those who don't. They might be healthier, more motivated, or have a better prognosis for reasons that have nothing to do with the treatment. Comparing the adherers in the treatment group to the adherers in the control group is like comparing apples to oranges; you've broken the beautiful balance that randomization gave you. Estimating a well-defined per-protocol estimand requires sophisticated causal inference methods that go far beyond a simple subgroup analysis. The estimand guides us away from this naive and often misleading comparison.

Perhaps most profoundly, the choice of estimand dictates the entire design of an experiment. If you want to know the "real-world" effect of a treatment policy, you might design a pragmatic trial that allows for flexibility, like letting doctors use rescue medication as they see fit. If you want to know the pure biological effect, you might design an "explanatory" trial with a strict run-in period to exclude non-adherent participants and intensive monitoring to ensure the protocol is followed. You must design the experiment to answer the question you care about, and the estimand is that question, stated with unflinching precision. This principle even extends to more complex designs, like trials asking if a new drug is "no worse than" an old one, where the clinically acceptable margin of "no worse" must be defined for the same estimand—the same causal question—as the primary analysis.

Science doesn't stop at a single study. We build knowledge by synthesizing evidence from many trials. Here too, the estimand concept brings startling clarity. When we perform a meta-analysis, combining the results of, say, ten different studies, what are we trying to estimate?

One approach, called a fixed-effect meta-analysis, assumes that all ten studies are just noisy measurements of one single, common true effect​, a parameter we might call $\mu$. This is a very strong assumption—that the effect of the drug is identical in a trial run in Sweden and one run in Japan, despite differences in population and medical practice.

A second approach, a random-effects meta-analysis, makes a more realistic assumption. It posits that each study has its own true effect, $\theta_i$, and these effects are themselves drawn from a grand distribution of possible true effects. The main target of this analysis is not any single study's effect, but the average of that entire distribution​, a parameter we call $\bar{\theta}$.

These are two different estimands, born from two different views of the world. The fixed-effect estimand $\mu$ asks, "What is the best estimate of the effect, assuming it's the same everywhere?" The random-effects estimand $\bar{\theta}$ asks, "What is the average effect across all the different contexts where this treatment could be used?" For a doctor wanting to know what to expect for their next patient, who may not perfectly match any of the previous trial populations, the random-effects estimand is often the more relevant and honest answer.

The beauty of the estimand is that it is not confined to medicine. It is a universal principle of science. Anytime we move from a sample to make a claim about a larger reality, we must define what feature of that reality we are targeting.

Consider the field of genomics. A scientist wants to know which genes are affected when a cancer cell is treated with a new drug. They can measure the activity of thousands of genes in treated and untreated cells. But the raw data are just a sea of numbers. The scientific question gives direction, but the estimand provides the coordinates. A proper estimand for this experiment might be: "the population-level average log-fold change in the true abundance of mRNA molecules for each gene". This precise target guides the entire complex statistical analysis, ensuring the scientist is estimating a real biological quantity, not an artifact of their measurement technique.

This way of thinking even clarifies our choice of statistical tools. In medical research, we often deal with data that is clustered—for example, patients within different hospitals. Suppose we want to know if a program reduces hospital readmissions. We could use two different statistical methods: a Generalized Linear Mixed-Effects Model (GLMM) or Generalized Estimating Equations (GEE). It turns out these models aren't just two ways of doing the same thing; they are aimed at two different estimands.

• The GLMM typically targets a conditional estimand. It asks: "For a given hospital, what is the change in a patient's odds of readmission?" It's a hospital-specific question.

• The GEE targets a marginal estimand. It asks: "If we roll out this program across the entire healthcare system, what is the change in the odds of readmission for an average patient in the population?" It's a population-averaged question.

Imagine looking at a forest. The conditional question is like asking, "For this specific part of the forest, how much taller is this pine tree than the oak next to it?" The marginal question is like asking, "On average, how much taller are pine trees than oak trees across the entire forest?" Both are valid questions, but they are different. A hospital administrator might care about the conditional effect in their hospital; a secretary of health might care more about the marginal, population-wide effect. The estimand framework forces us to choose which question we're answering.

From the doctor's office to the genetics lab, from a single experiment to the synthesis of all available knowledge, the principle remains the same. Before we can find an answer, we must first dare to state, with absolute clarity, what it is we are looking for. The estimand is our declaration of intent, a guiding star that protects us from the siren song of spurious correlations and the fog of ambiguous questions. It is the first, and most critical, step on the path to reliable knowledge.