Natural Direct Effect

In any scientific inquiry, establishing that a cause has an effect is only the beginning. The more profound question that follows is how or why. To truly understand a phenomenon, we must move beyond simple correlation and dissect the intricate causal pathways that produce it. A new drug might lower the risk of heart attack, but does it do so by reducing cholesterol, or does it have a separate, protective effect on the arteries themselves? Answering such questions requires a clear method for separating the direct influence of a cause from the indirect effects it creates by altering other variables along the way. This is the central challenge that mediation analysis and the concept of the Natural Direct Effect seek to resolve.

This article provides the conceptual tools for such a dissection, offering a clear framework for understanding causal mechanisms. It is divided into two main parts. The first chapter, ​​Principles and Mechanisms​​, will introduce the powerful counterfactual framework, define the Natural Direct and Indirect Effects, and explain the remarkable way they decompose the total effect. It will also confront the significant challenge of identifying these effects from real-world data and the strong assumptions this requires. The second chapter, ​​Applications and Interdisciplinary Connections​​, will then demonstrate how these seemingly abstract concepts are applied to solve critical problems in fields ranging from medicine and public health to social justice and the ethics of artificial intelligence. By the end, you will have a robust understanding of not just if a cause works, but the specific pathways through which it exerts its influence.

After we have established that a cause has an effect—that a new drug lowers cholesterol, or a new teaching method improves test scores—a deeper, more satisfying question naturally arises: Why? What is the mechanism? How does it work? Is it the drug itself that directly protects the arteries, or does it work entirely by lowering cholesterol? Does the teaching method directly impart some new way of thinking, or does it simply encourage students to study more? Science is not just about observing correlations; it is about understanding the intricate causal pathways that produce the phenomena we see.

To dissect these pathways, to separate the direct effect of a cause from the indirect effects it produces by changing other variables along the way, we need a language of exquisite precision. We need a way to talk about what would have happened in worlds that do not exist. This is the world of causal inference, and our primary tool is the concept of ​​counterfactuals​​.

Let’s imagine we are studying a treatment, which we’ll label $A$. For simplicity, let’s say a person either receives the treatment ($A=1$) or they don’t ($A=0$). We are interested in its effect on an outcome, $Y$. The counterfactual idea is simple: for any given person, there exists a potential outcome $Y(1)$, the outcome they would have if they got the treatment, and a potential outcome $Y(0)$, the outcome they would have if they did not. The ​​total causal effect​​ for that person is simply the difference, $Y(1) - Y(0)$. Of course, we can only ever observe one of these for any individual, which is the fundamental problem of causal inference.

Now, let’s introduce a ​​mediator​​, $M$. This is the intermediate variable we suspect is part of the story—the biomarker level, the hours studied. The mediator itself is affected by the treatment. So, we can define a potential mediator, $M(1)$, the value the mediator would take if the person received the treatment, and $M(0)$, the value it would take if they didn't.

Furthermore, the final outcome $Y$ doesn't just depend on the treatment $A$ anymore; it depends on both the treatment $A$ and the value of the mediator $M$. So we need to expand our notation. Let $Y(a, m)$ be the outcome that would be observed if the treatment were set to level $a$ and the mediator were somehow set to level $m$. With this elegant notation, we can now state a fundamental rule, often called ​​consistency​​ or ​​composition​​: the outcome a person would have under treatment $A=1$, which we called $Y(1)$, must be the same as the outcome they would have if we set their treatment to $A=1$ and let their mediator naturally become $M(1)$. In our symbols, this is simply $Y(1) = Y(1, M(1))$. This might seem like a trivial rule, but it is the link that allows us to connect the total effect to the underlying mediational pathways.

Our goal is to separate the direct effect of $A$ on $Y$ from the indirect effect that goes through $M$. Visually, we want to distinguish the path $A \to Y$ from the path $A \to M \to Y$.

How might we isolate the direct path? A straightforward idea is to hold the mediator $M$ constant. Imagine a grand experiment where we could not only give people the treatment ($A=1$) or the control ($A=0$), but we could also force everyone’s mediator level to be some specific, fixed value, say $m^*$. If we did this, the only reason for a difference in the outcome $Y$ between the two groups would have to be the direct effect of $A$. We call this the ​​Controlled Direct Effect​​ (CDE) at level $m^*$:

The $E[\cdot]$ here means we are taking the average effect over the whole population. This is a well-defined and potentially useful quantity. We could ask, "What is the effect of the drug if we could force everyone's blood pressure to be a healthy 120/80 mmHg?"

But there's a certain artificiality to this. What if the level $m^*$ we pick is biologically impossible for some people, or very atypical? The CDE gives us the answer to a question, but is it the most relevant one? We are asking about the mechanism in the real world, not in a world where we have god-like control over a person's internal biology.

This leads to a more subtle and beautiful idea. Instead of fixing the mediator to an external, artificial value $m^*$, what if we could set it, for each person, to the value it would have naturally taken if they had been in the control group? This "natural" baseline level is $M(0)$. Now we can define a new kind of direct effect. We compare the outcome when a person gets the treatment ($A=1$) but their mediator is held at their natural baseline level $M(0)$, to their outcome when they don't get the treatment ($A=0$) and their mediator is also at its natural baseline level $M(0)$. This is the ​​Natural Direct Effect​​ (NDE):

This quantity is profound. It represents the effect of the treatment on the outcome if the mediational pathway were blocked, not by forcing it to some universal constant, but by holding it at the natural level it would have had for each individual in the absence of the treatment.

Look closely at the first term in the NDE definition: $Y(1, M(0))$. This is a strange and wonderful creature. It is a ​​nested counterfactual​​, often called a "cross-world" counterfactual. Let's translate it into words: it is the outcome you would have if you received the treatment ($A=1$), while your mediator was at the value it would have had if you had been in the control group ($A=0$).

For any single individual, this is a physical and logical impossibility. You cannot both receive the treatment and not receive it at the same time. The world in which your outcome is measured ($A=1$) is different from the world from which your mediator's value is drawn ($A=0$). It seems like something out of science fiction.

Early critics of this framework argued that such quantities were "undefined" or "metaphysical" and had no place in science. But this misses the point. The power of mathematics and logic is that we can define concepts and explore their consequences, even if we cannot build them in a workshop. The term $Y(1, M(0))$ is not something we ever expect to observe for a person. It is a theoretical construct that allows us to ask our question—"What is the direct effect?"—with perfect clarity. Its role is to define the ideal we are trying to understand, even if we can only ever approximate it with real data.

The true beauty of these definitions is revealed when we see how they fit together. The total causal effect of the treatment is the difference between the world where everyone gets the treatment and the world where no one does:

Now, watch this. With a bit of simple algebra (adding and subtracting the same term), we can perform a remarkable decomposition:

The first part of this sum is, by definition, our Natural Direct Effect. The second part is called the ​​Natural Indirect Effect​​ (NIE).

The NIE has an equally intuitive, if mind-bending, interpretation. It asks: while holding the treatment fixed at the "on" position ($A=1$), what is the change in outcome caused purely by the mediator shifting from its natural control value ($M(0)$) to its natural treatment value ($M(1)$)? This perfectly isolates the effect transmitted through the mediator.

And the punchline is this:

This is not an approximation. It is an exact, algebraic identity. Our conceptual journey has led us to a perfect partitioning of causality. We have taken the total effect of a cause and cleanly split it into two pathways: the direct effect and the indirect effect operating through the mediator. This decomposition is a testament to the power and internal consistency of the counterfactual framework.

We have defined our ideal quantities. Now for the hard part: can we ever measure them? Can we connect these abstract counterfactuals to the messy, real-world data we collect from experiments or observational studies? This is the problem of ​​identification​​.

A common intuition is that a simple Randomized Controlled Trial (RCT) should solve everything. If we randomize the treatment $A$, we break all confounding between $A$ and the characteristics of the patients. This is true, and it allows us to get an unbiased estimate of the total effect, $E[Y(1)] - E[Y(0)]$. But it does not, by itself, allow us to estimate the NDE or NIE. Why? Because randomization ensures that $A$ is independent of the patient's potential outcomes, but it doesn't break the dependencies within the patient. For example, a person's potential outcome under treatment, $Y(1,m)$, and their potential mediator level under control, $M(0)$, could still be linked by their underlying genetics or physiology. Randomization of $A$ alone cannot peer inside this "cross-world" relationship.

To identify the NDE and NIE from data, we need to make a leap of faith, embodied by a set of strong, untestable assumptions. In the simplest case, these are often called the "four commandments" of mediation analysis:

1. ​​Consistency​​: The potential outcomes framework accurately describes the observed data.
2. ​​Positivity​​: For any type of person, there is some chance they could receive either treatment and could exhibit any of the relevant mediator values. There are no "doomed" subgroups.
3. ​​No Unmeasured Confounding​​: This is a huge leap. We must assume that we have measured all the baseline factors (let's call them $L$) that confound any of the relationships: the effect of $A$ on $Y$, the effect of $A$ on $M$, and, crucially, the effect of $M$ on $Y$.
4. ​​The Cross-World Assumption​​: This is the final, and perhaps most audacious, leap. We must assume that there is no unmeasured factor that links the potential mediator under one exposure, $M(0)$, with the potential outcome under the other, $Y(1,m)$. This is the assumption that lets us mathematically "stitch" the two worlds together.

If these four assumptions hold, then it is possible to write a formula—often called the ​​mediation formula​​—that expresses the NDE and NIE purely in terms of observable probabilities and conditional expectations from our data.

These assumptions are not just technical footnotes; they are fragile. Nature is complex, and it is easy for a real-world causal structure to violate them in subtle ways. Consider a classic, tricky case: an ​​exposure-induced mediator-outcome confounder​​.

Imagine a new drug ($A$) leads to patients having more frequent follow-up appointments with their doctors ($L$). These more frequent appointments, in turn, lead to more careful monitoring of their blood pressure ($M$), which improves their final health outcome ($Y$). But the appointments ($L$) also have a direct effect on the outcome ($Y$), perhaps because doctors give other useful health advice during these visits. The causal diagram looks like this: $A \to L$, $L \to M$, $L \to Y$, and $M \to Y$.

Here, the variable $L$ (doctor visits) is a common cause of the mediator $M$ and the outcome $Y$. That makes it a confounder of the $M \to Y$ relationship. But this confounder, $L$, is itself caused by the exposure $A$. This breaks the standard identification assumptions. Simply "adjusting for $L$" in a statistical model doesn't work; in fact, it can create even more bias by blocking off a part of the causal effect we want to understand. In such a scenario, the beautiful NDE and NIE that we so carefully defined are no longer identifiable with standard methods.

This serves as a crucial lesson. The Natural Direct Effect provides us with the perfect question. Causal inference provides a framework for how we might answer it. But it also forces us to be honest and humble about the powerful assumptions we are making about the world, and to recognize when the complexity of reality outstrips the reach of our tools. The journey to understand "why" is as much about appreciating the beauty of the question as it is about respecting the difficulty of the answer.

Having journeyed through the principles and mechanisms of causal decomposition, one might be tempted to view these ideas as elegant but abstract creations of the statistician's mind. Nothing could be further from the truth. The concepts of natural direct and indirect effects are not mere academic curiosities; they are powerful scalpels for dissecting the intricate causal chains that govern our world. To truly appreciate their power, we must see them in action. We must move from the chalkboard to the real world, where these tools are used to answer some of the most pressing questions in science, society, and engineering. This is a journey from if a cause has an effect, to the far more profound question of how it exerts its influence.

The human body is a labyrinth of interconnected pathways. A change in one corner can send ripples throughout the system, often in ways that are far from obvious. Causal mediation analysis provides a lantern to navigate this labyrinth.

Consider a classic public health puzzle: the link between high salt intake, blood pressure, and stroke. We are often told that high salt is dangerous because it raises blood pressure, which in turn increases the risk of stroke. This describes an indirect pathway: Salt $\to$ Blood Pressure $\to$ Stroke. But is this the whole story? Could high salt intake harm the cardiovascular system through other mechanisms, independent of its effect on blood pressure? This is precisely the question the natural direct effect (NDE) is designed to answer. The NDE quantifies the effect of salt on stroke that bypasses the blood pressure pathway. The natural indirect effect (NIE), conversely, isolates the portion of the harm that is transmitted through blood pressure. By dissecting the total effect into these two pieces, researchers can understand whether public health interventions should focus exclusively on blood pressure management, or if there are other salt-induced dangers to guard against.

This same logic is indispensable in modern medicine and pharmacology. Imagine a new anti-inflammatory drug is being tested. The drug is known to reduce a specific inflammatory biomarker in the blood, and patients taking the drug show improved clinical outcomes. The crucial question for drug developers is: Is the drug's benefit because it lowers this biomarker (an indirect effect), or does the drug have other, beneficial "off-target" effects (a direct effect)? The NDE helps to quantify these direct, pleiotropic effects. Understanding this is vital for both efficacy and safety. A drug might have a beneficial indirect effect through its intended target but a harmful direct effect, or vice-versa.

The power of this framework extends beyond molecules and into the realm of psychology and behavior. It is known, for instance, that optimistic individuals tend to live longer. But why? A psychologist might hypothesize that this is because optimistic people are more likely to adopt healthy behaviors, such as quitting smoking. In our language, this is an indirect pathway: Optimism $\to$ Smoking Cessation $\to$ Longevity. The natural direct effect, then, would capture any remaining survival benefit of optimism that exists even if we could hold a person's smoking behavior fixed. This could point toward direct psycho-physiological effects of a positive mindset on the body's resilience, a fascinating frontier in medical psychology.

The causal pathways that determine our health are not confined to our bodies; they extend deep into the fabric of society. The very same tools we use to trace the path of a drug can be used to trace the path of inequity.

Social epidemiologists seek to understand why factors like income and neighborhood can have such profound impacts on health. Consider the link between low income and depression. A plausible hypothesis is that low income leads to higher levels of chronic stress, which in turn causes depression. This is the indirect pathway. The natural direct effect would capture any effect of income on depression that is not explained by its influence on stress—perhaps through access to resources, social participation, or a sense of security.

In a striking (though hypothetical) analysis of healthcare disparities, researchers might investigate why living in a deprived neighborhood increases the risk of severe maternal morbidity. Suppose they find that a large fraction of this effect—say, 60%—is mediated through inadequate prenatal care. This number, the "proportion mediated" ($\frac{NIE}{NIE+NDE}$), is not just a statistic; it is a powerful signpost for policy. It suggests that interventions aimed at equalizing access to high-quality prenatal care could potentially eliminate a majority of the observed health disparity. It turns a complex social problem into a tractable target for action.

This quest for fairness has found a new and urgent application in the world of artificial intelligence. We are increasingly deploying algorithms to make life-altering decisions in areas like hiring, lending, and even medical resource allocation. How can we ensure these algorithms are not perpetuating or amplifying historical biases? Causal mediation provides a revolutionary answer.

Imagine an AI system designed to allocate scarce medical resources, which uses a patient's self-identified race ($A$) and a clinical risk score ($X$) to produce a decision ($Y$). The causal graph might look like $A \to Y$ (a direct path) and $A \to X \to Y$ (an indirect path).

• The ​​Natural Direct Effect​​ of race on the decision is the effect that bypasses the clinical risk score. In a linear model, this is captured by a single coefficient, $\gamma$. This corresponds to two individuals with the exact same clinical risk score receiving different decisions solely because of their race. This is the very definition of direct discrimination, and it is something we can demand our algorithms eliminate by forcing the NDE to be zero.

• The ​​Natural Indirect Effect​​ flows through the clinical score. This is more subtle. The link $A \to X$ might reflect systemic inequities (e.g., one racial group has historically had less access to preventative care, leading to worse clinical scores) or it might reflect genuine group-level differences in disease prevalence. The concept of ​​path-specific fairness​​ gives us a formal language to address this: we can design an algorithm that deliberately blocks the influence of disallowed pathways (like the direct effect) while retaining pathways deemed clinically justifiable. We can build an AI that makes a prediction as if the direct discriminatory path did not exist. This is a profound shift from merely observing bias to surgically removing it from our automated systems.

One of the most beautiful aspects of a fundamental scientific principle is its universality. The logic of causal mediation is not limited to biology or sociology. It applies to any complex system where one component's action influences another through an intermediary.

Consider a cyber-physical system, like a robot in a factory. A new control algorithm ($A$) is installed. Soon after, the robot's main arm begins to fail more frequently ($Y$). An engineer observes that the arm's motor is also running hotter than usual ($M$). The engineer's diagnostic question is precisely a mediation question: Is the new software causing failures because it's making the motor overheat (the indirect effect, $A \to M \to Y$)? Or does the software contain a bug that directly causes failures, irrespective of temperature (the direct effect, $A \to Y$)? Disentangling these two paths is critical. If the NDE is the culprit, the software must be debugged. If the NIE is the main driver, the problem might be solved by installing a better cooling system. The ability to distinguish between these scenarios is essential for effective root cause analysis in any complex engineering system, from a digital twin of a jet engine to the global internet.

Thus far, we have focused on how NDE and NIE help us understand the world as it naturally unfolds. But often, our goal is to change the world through intervention. Here, a subtle but crucial distinction comes into play, highlighting the relationship between the questions we ask and the answers we can act upon.

The Natural Direct Effect involves a "cross-world" thought experiment: what would happen if we administered a treatment, but the mediator were held at the level it would have been without the treatment? This is perfect for understanding mechanism but does not correspond to a straightforward real-world policy. We cannot, for each patient, know their unobserved counterfactual mediator level and intervene to set it there.

This is where a related concept, the ​​Controlled Direct Effect (CDE)​​, becomes invaluable. The CDE asks a different question: What is the effect of the treatment if we intervene to set everyone's mediator to the same fixed level $m$? For example, "What is the effect of our new anti-inflammatory drug if we also give every patient a second drug that forces their biomarker level to 50?" This is a concrete, implementable strategy. The CDE quantifies its effect.

The NDE is our tool for passive understanding; the CDE is our tool for planning active intervention. A complete scientist and a wise policymaker must understand both. They must first use the NDE to understand the natural pathways of the world, and then use that knowledge to formulate CDE-like questions about which interventions are most likely to succeed.

From the quiet workings of our cells to the bustling dynamics of our societies and the precise logic of our machines, the principles of causal decomposition provide a unified framework for inquiry. They empower us not just to see the connections between events, but to understand the very texture of those connections, revealing the hidden pathways that shape our reality.