Practical Non-Identifiability

In modern science, mathematical models are indispensable tools for understanding complex systems, from the molecular dance within a cell to the vast dynamics of an ecosystem. We build these models to represent reality, but a crucial challenge arises when we try to calibrate them against experimental data. Even when a model perfectly fits our observations, how can we be sure that the specific parameter values we've found are the one true reflection of the underlying processes? This gap between a good fit and true knowledge is often caused by a problem known as non-identifiability, where the available data is insufficient to uniquely pin down the model's parameters.

This article delves into the pervasive and often subtle issue of ​​practical non-identifiability​​. You will learn to distinguish it from its more fundamental counterpart, structural non-identifiability, and understand why it represents a challenge not of the model itself, but of the dialogue between the model and the experiment. First, the "Principles and Mechanisms" chapter will unravel the causes of this problem—from insensitive parameters to insufficient data—and introduce the powerful geometric tools, like the Fisher Information Matrix, used to diagnose it. Following this, the "Applications and Interdisciplinary Connections" chapter will demonstrate the real-world consequences of non-identifiability across diverse fields, illustrating how recognizing this issue is essential for designing smarter experiments and building more reliable scientific knowledge.

Imagine you are a detective investigating a complex case. You have a theory—a model—of what happened, involving several suspects and their actions. You gather evidence—data—and see if it fits your theory. A good fit feels like a victory! But what if your evidence is consistent with several different versions of the story? What if moving a key suspect's timeline by an hour, or even a day, doesn't contradict any of the facts you've gathered? In that moment, you haven't proven your theory; you've discovered that your evidence isn't strong enough to tell the full story.

This is the central challenge in much of modern science. We build beautiful mathematical models of the world, from the inner workings of a cell to the dynamics of an ecosystem. These models have parameters—knobs we can turn, representing things like reaction rates, degradation constants, or interaction strengths. We collect data and try to find the one true setting for all these knobs that perfectly describes reality. But often, we run into a frustrating but fascinating problem: ​​practical non-identifiability​​. It's the scientific equivalent of our detective's dilemma. It means that our experimental data, no matter how carefully collected, is insufficient to pin down the value of one or more parameters with any reasonable certainty.

Before we delve deeper, we must distinguish practical non-identifiability from its more stubborn cousin, ​​structural non-identifiability​​. Think of it this way:

A ​​structural non-identifiability​​ is a fundamental flaw in the model's equations themselves. It's like having two suspects, Alice and Bob, who only ever appear together in the story. Your model might depend on a term like the product of their efforts, say Alice's speed $\times$ Bob's strength. No matter how much data you collect about their combined work, you can never separate their individual contributions. If Alice was twice as fast and Bob was half as strong, the outcome would be identical. Mathematically, this happens when different combinations of parameter values produce the exact same model output. For example, in a simple decay model where a substance $X$ disappears over time, we might measure a signal proportional to its concentration, $y(t) = c X_0 \exp(-kt)$. The initial amount $X_0$ and the measurement scaling factor $c$ are structurally non-identifiable because our data only ever sees their product, the initial signal height $cX_0$. No amount of data, no matter how perfect, can untangle them. The model is permanently blind to their individual values.

A ​​practical non-identifiability​​, on the other hand, is not a flaw in the model itself, but a problem with the dialogue between the model and the experiment. The model is theoretically sound, and in a perfect world with infinite, noise-free data, every parameter could be determined. But in our real, messy world, our specific dataset just doesn't contain the right kind of information. It's not that the answer is impossible to find; it's that we haven't asked the right question with our experiment. This is a challenge, not a dead end. It’s a clue that tells us we need to be cleverer in how we probe the system.

A powerful tool for visualizing this is the ​​profile likelihood​​. Imagine a landscape where the position represents a parameter's value and the altitude represents how well the model fits the data (the "likelihood"). A well-identified parameter has a profile like a deep, sharp canyon—a clear, unambiguous peak. A structurally non-identifiable parameter corresponds to a perfectly flat plateau; moving along it causes zero change in the fit. A practically non-identifiable parameter is a long, shallow valley. There's a lowest point, a "best" fit, but you can wander very far from it before the altitude changes much. The data has a slight preference, but its voice is a whisper, not a shout.

If practical non-identifiability comes from a lack of information, where does the information hide? It turns out there are several common ways our experiments can fail to "ask the right questions."

​​1. Asking the Wrong Question: Insensitive Parameters​​

Sometimes, our experiment is simply conducted under conditions where the model's output is insensitive to the parameter we care about. Imagine trying to determine an enzyme's affinity for its substrate—a parameter called the Michaelis constant, $K_M$. This parameter describes how much substrate is needed to get the reaction running at half its maximum speed. If you design an experiment where you flood the system with a huge, saturating amount of substrate, the enzyme will be working at its maximum capacity, $V_{\max}$, regardless of its affinity. The reaction rate will be the same whether the affinity is high or low. The data you collect will be utterly silent about the value of $K_M$, leading to a huge confidence interval for its estimate. To learn about $K_M$, you must perform measurements at substrate concentrations around the value of $K_M$, where the reaction rate is actually sensitive to it.

​​2. Partners in Crime: Confounded Parameters​​

In other cases, the effects of two or more parameters become tangled together, or ​​confounded​​, under certain conditions. Consider a simple chemical cascade: substance $A$ turns into $B$, which then turns into $C$ ($A \xrightarrow{k_1} B \xrightarrow{k_2} C$). Let's say we start with an initial amount $A_0$ of substance A and measure the concentration of B over time. Right at the beginning of the reaction, for very small times $t$, the concentration of B is approximately $[B](t) \approx A_0 k_1 t$. The data only "sees" the product of the initial amount $A_0$ and the first rate constant $k_1$. If we double $A_0$ and halve $k_1$, the initial rise of B looks identical. The two parameters are acting as partners in crime, and our early-time data can't tell their individual roles apart. Their sensitivity profiles become nearly collinear, meaning they point in the same direction in parameter space, and we can't distinguish one from the other.

​​3. Blink and You'll Miss It: Insufficient Sampling​​

Many biological and chemical processes involve a sequence of events happening at vastly different speeds. Imagine our $A \to B \to C$ cascade again, but this time the first step ($A \to B$) is incredibly fast, while the second step ($B \to C$) is slow. Let's say the first step has a characteristic time of 1 millisecond. If we are only able to take measurements every second, the entire conversion of A to B will have happened and finished long before we take our first snapshot. Our data will look as if we just started with a pile of B which then slowly turns into C. The data will contain plenty of information about the slow rate, $k_2$, but it will be completely blind to the fast rate, $k_1$, because we missed the action. This is a form of aliasing—the fast dynamics are invisible to our slow sampling rate. The only way to catch the speed of the first step is to use a "high-speed camera": sample much, much faster than 1 millisecond.

​​4. The Deceptive Doppelgänger: Indistinguishable Mechanisms​​

Perhaps the most profound form of practical non-identifiability arises when our data cannot distinguish between two entirely different underlying mechanisms. Suppose we are trying to figure out if a protein $X$ activates another protein $Y$ directly ($X \to Y$) or through an unobserved intermediate helper, $Z$ ($X \to Z \to Y$). If the intermediate step is extremely fast—that is, $Z$ is created and consumed in a flash—the overall behavior of $Y$'s concentration over time can look almost identical in both models. The two-step process with a rapid intermediate effectively mimics the simpler, direct one-step process. Our data, unless it has extraordinarily high temporal resolution, fits both stories equally well. This isn't just about a single parameter being uncertain; it's about the very structure of our proposed reality being ambiguous.

So how do scientists get a handle on all this? We don't just throw up our hands. We draw a map. We create a mathematical object that maps out the landscape of information in our experiment. This map is the ​​Fisher Information Matrix (FIM)​​.

For a given model and experimental design, the FIM tells us how much information our data contains about each parameter and every combination of parameters. The real magic happens when we analyze its geometry. Like any matrix, the FIM has ​​eigenvectors​​ and ​​eigenvalues​​. You can think of the eigenvectors as special, "natural" directions in the high-dimensional space of our parameters. The corresponding eigenvalue tells us how much information we have in that specific direction—how steep the "likelihood landscape" is.

• ​​Stiff Directions:​​ These are directions associated with very ​​large eigenvalues​​. A small step in a stiff direction causes a huge change in the model's output, making the fit to the data much worse. This means our experiment is extremely sensitive to this particular combination of parameters. The landscape is a steep canyon, and the parameter combination is pinned down with high precision.

• ​​Sloppy Directions:​​ These are directions associated with very ​​small eigenvalues​​. You can move a long way in a sloppy direction before the model's output changes much at all. The landscape here is a long, flat-bottomed valley. Our experiment is insensitive to this combination of parameters, and its value will be very uncertain. This is the mathematical soul of practical non-identifiability.

Many complex models in biology are found to be ​​"sloppy."​​ This is a technical term meaning that the eigenvalues of their FIM span many orders of magnitude. They have a few very stiff directions and many, many sloppy directions. This means that while some combinations of parameters can be known with exquisite precision, others are astronomically uncertain.

This geometric view is incredibly powerful. The length of the uncertainty on a parameter combination (the semi-axis of its confidence "ellipsoid") scales as $1/\sqrt{\lambda}$, where $\lambda$ is the eigenvalue. A tiny eigenvalue means a gigantic uncertainty. The intuitive ideas we started with—a flat likelihood profile, an insensitive parameter, or confounded effects—are all symptoms of the same underlying geometric reality: a sloppy direction in parameter space, revealed by a small eigenvalue of the FIM.

Understanding this principle doesn't solve the problem, but it transforms it. It tells us precisely why our experiment is failing and, most importantly, it gives us a map to design the next, smarter experiment. By identifying the sloppy directions, we can devise new experimental conditions—new inputs, faster sampling, different measurements—that specifically target these directions, forcing the system to reveal the information that was once hidden in the sloppy shallows. The journey from non-identifiability to knowledge is a journey of turning sloppy valleys into stiff canyons.

Now that we have explored the principles of identifiability, we might be tempted to think of it as a rather abstract, mathematical concern. A bit of theoretical housekeeping for modelers. But nothing could be further from the truth. The ghost of non-identifiability is not a harmless specter haunting the pages of mathematics journals; it is a real-world trickster that lurks within our most critical scientific and engineering challenges. It affects how we discover new drugs, predict the course of epidemics, trace our own evolution, design safer materials, and assess the risks of new technologies.

Understanding where this ghost appears and how to deal with it is one of the most practical skills a scientist can possess. It is the art of knowing what questions we can reasonably ask of our data, and what questions will only be met with an enigmatic silence. Let us go on a journey through a few different fields of science and engineering to see this principle in action.

One of the most common ways non-identifiability appears is through an ambiguity of scale. Imagine you are looking at a photograph of a mountain. Can you tell if it's a very large mountain far away, or a smaller mountain close up? Without a familiar object in the frame for scale—a tree, a person—you simply can't know. Many scientific models have this exact problem baked into their structure.

A classic example comes from chemistry. Suppose we are studying a simple reaction where a substance $X$ turns into $Y$, which then degrades: $X \to Y \to \varnothing$. We want to measure the concentration of the intermediate substance $Y$ over time. We build a sensor, but we don't know exactly how sensitive it is. The signal our machine outputs, $y_{\text{obs}}(t)$, is related to the true concentration, $y(t)$, by some unknown amplification factor (or gain) $\alpha$ and an unknown baseline offset $\beta$. So, $y_{\text{obs}}(t) = \alpha y(t) + \beta$.

We run the experiment, starting with some unknown amount of $X$, which we'll call $x_0$. We get a beautiful curve of our observed signal over time—it rises, peaks, and then falls. We can perfectly determine the reaction rates, $k_1$ and $k_2$, from the shape of this curve. We can also determine the offset $\beta$ from the signal's starting value (since $y(0)=0$, our sensor reads $y_{\text{obs}}(0)=\beta$). But here's the catch: the height of the curve depends on the product of the initial amount of material and the sensor's gain, $\alpha x_0$. Did we see a big peak because we started with a lot of $X$ ($x_0$ is large) and our sensor is not very sensitive ($\alpha$ is small)? Or was it a small amount of $X$ amplified by a very sensitive sensor? Based on this one experiment, it is fundamentally impossible to tell. The parameters $\alpha$ and $x_0$ are structurally non-identifiable. The only thing our data can reveal is their product. This scaling symmetry is a blind spot in our experiment. To resolve it, we must break the symmetry—for instance, by running a second experiment with a known quantity of $X$, thereby calibrating our "faulty gauge".

This "problem of scale" is not just for simple chemical reactions; it is a central challenge in modeling complex biological systems. Consider the life-or-death struggle between a virus and a host. Virologists build models to describe the populations of target cells, infected cells, and free virus particles. A crucial piece of data is the "viral load" measured from a patient's blood. But the measurement we get is not a direct count of virions; it's a signal from a laboratory assay, and the conversion factor, $q$, between the true number of virus particles and the measured signal is often unknown. This introduces a subtle but profound structural non-identifiability. One can imagine a scenario where the patient has a certain number of target cells, $N$, and each infected cell produces virus at a rate $p$. Now, imagine a different scenario with twice as many target cells, $2N$, but where each infected cell is only half as productive, producing virus at a rate $p/2$. It turns out the dynamics of the observable viral load can be exactly the same in both scenarios. The model cannot tell them apart.

This has enormous practical consequences. If a model cannot distinguish the absolute number of host cells, any prediction about that number is built on sand. This exact issue appears in modern ecological risk assessment for synthetic biology. When modeling the environmental impact of an engineered microbe, a sensor might measure a combined signal of the engineered organism and a native species, with an unknown scaling factor. If we cannot independently determine this factor, we cannot know the absolute population of the native species. A regulatory agency, therefore, cannot accept a simple claim like "the native host population will remain above 1,000,000 individuals" if the model generating that claim has an unresolvable scaling symmetry. Governance standards must demand that evidence be framed only in terms of what is truly identifiable, or that the experimental design be improved to break the symmetry and provide an absolute scale.

Another form of non-identifiability arises when different combinations of underlying parameters can conspire to produce the same observable outcome. It's like a crime scene where two completely different sequences of events could lead to the same set of clues.

The simplest case comes from enzyme kinetics. The famous Michaelis-Menten equation describes how the rate of an enzyme-catalyzed reaction depends on the concentration of its substrate. This relationship is governed by two parameters: the maximum reaction rate, $V_{\max}$, and the Michaelis constant, $K_M$, which describes the enzyme's affinity for the substrate. If an experimenter is lazy and only measures the initial reaction rate at a single substrate concentration, they are left with one equation and two unknowns. An infinite number of $(V_{\max}, K_M)$ pairs—a whole line in the parameter space—can satisfy this single data point. The parameters are structurally non-identifiable. The cure, of course, is a better experiment: measuring the rate at several different substrate concentrations, or even better, observing the entire reaction over time as the substrate is depleted. This provides enough information to pin down a unique solution.

Sometimes, the "doppelgänger effect" comes not from a lazy experiment, but from a fundamental symmetry in our own concepts. In evolutionary biology, we might build a model where a species' trait (say, body size) evolves at different rates depending on which of two "hidden" environmental regimes, 'A' or 'B', it inhabits. But the labels 'A' and 'B' are our invention. A world where regime 'A' causes fast evolution and 'B' causes slow evolution is indistinguishable from a world where 'B' is fast and 'A' is slow. The likelihood of our data will be identical for these two parameter sets. This "label-swapping" is a structural non-identifiability. The only way to proceed is to make an arbitrary choice to break the symmetry, for instance by imposing a rule like, "we will always label the faster-evolving regime as 'A'".

This ambiguity is especially pronounced in engineering. Imagine you're an engineer characterizing a new rubber-like material for a critical application, like a seal in a spacecraft. You need a mathematical model that predicts how the material deforms under any kind of stress. Sophisticated models like the Ogden model use several parameters to capture this complex behavior. A common and easy test is to pull on a strip of the material and measure how much it stretches—a uniaxial tension test. The problem is that this single test only probes the material's response in one specific direction. It's like trying to understand a complex sculpture by looking at it from only one angle. Different combinations of the model's underlying parameters can collude to produce the exact same stress-strain curve in that one test. To truly identify the material's properties, you must be more creative. You need to probe it from multiple angles: stretch it in two directions at once (equibiaxial tension) or stretch it in one direction while holding its width constant (planar tension). Each of these tests provides a different piece of the puzzle, and by forcing a single set of parameters to explain all of them simultaneously, you can finally unmask the material's unique properties and build a reliable model.

So far, we have focused on structural non-identifiability, where different parameter sets produce exactly the same predictions. This is a property of the model's mathematics. But there is a second, more insidious type: practical non-identifiability. Here, the parameters are, in principle, unique. But in the real world of finite, noisy data, the information needed to distinguish them is so weak that it is effectively lost.

Think of trying to determine the parameters that describe a material's fatigue life. The total strain a material experiences is the sum of a recoverable, elastic part (like a spring) and a permanent, plastic part. When we perform tests at very high strains that cause failure in just a few cycles, the plastic deformation is enormous, while the elastic part is minuscule. The data are completely dominated by the plastic behavior. Trying to estimate the parameters that govern the tiny elastic contribution is like trying to hear a whisper during a rock concert. While they are structurally identifiable, they are practically non-identifiable from this dataset. The confidence intervals on our estimates for the elastic parameters would be astronomically wide, meaning the data are telling us almost nothing. The solution is not to pretend we can hear the whisper. Instead, we must be honest about the limits of our data and use information from other sources—perhaps from separate low-strain tests, or from our fundamental understanding of material physics—to provide reasonable constraints or priors for those elusive parameters.

This problem often arises when a system has processes occurring on very different timescales. In the evolutionary model with hidden regimes, if the environment switches between regimes 'A' and 'B' extremely rapidly, any observation of trait evolution over a longer timescale will only see a blurred average of the two. The data will be informative about the average evolutionary rate, but will be unable to resolve the individual rates in each regime. A similar phenomenon occurs in a bacterial toxin-antitoxin system, where a stable antitoxin molecule quickly neutralizes a toxin. The toxin's effective rate of disappearance is a combination of its natural degradation and its neutralization by the antitoxin. If the neutralization is fast and the antitoxin level is constant, the data can only tell us about the combined effective rate, not the two separate contributions.

Finally, practical non-identifiability can simply be the result of a poorly designed experiment. Suppose we want to understand how social influence affects people's choices. We build a model where the probability of an individual choosing a certain cultural variant depends on the frequency of that variant among their peers. But if, in our experiment, we only ever show them situations where the frequency is very high (say, 99% of their peers chose variant A), we have no information about what they would do if the frequency were 50%, or 10%. The parameter that measures the strength of this frequency-dependent bias will be practically non-identifiable because we never supplied the experiment with the necessary variation. Similarly, in studying how a photoreceptor cell in the eye responds to light, if we use a flash of light that is too dim, the cell's response will be so small that it is completely buried in the inherent biological noise. We cannot identify the parameters of a response that we cannot even see.

The journey through these diverse applications reveals a deep and unifying principle. Identifiability analysis is the scientist's tool for checking their own blind spots. It is a formal way of asking, "Given my model and my experiment, what am I truly capable of learning?"

The discovery of non-identifiability, whether structural or practical, should not be seen as a failure. It is an opportunity. It is a signpost telling us that we are at the limits of what our current approach can tell us. It guides us toward progress by forcing us to ask critical questions. Does our model have a fundamental symmetry? Then we must reframe our questions in terms of identifiable combinations. Is our experiment providing ambiguous or weak information? Then we must design a better, more informative experiment—by adding new measurements, probing the system from different angles, or expanding the range of conditions we test.

In a world awash with data and complex computational models, it is easy to be seduced by a model that fits our data well. But identifiability analysis provides a crucial dose of humility. It reminds us that a good fit is not the same as a true understanding. It teaches us to respect the boundary between what can be known and what remains, for now, a ghost in the machine. By learning to see these ghosts, we learn to conduct better, more honest, and ultimately more powerful science.