Practical Identifiability

In quantitative science, mathematical models are indispensable tools for unraveling complex systems, from the inner workings of a cell to the spread of a disease. Yet, a model's predictive power is entirely dependent on the parameters that define it. A critical question often overlooked is: can we uniquely and confidently determine these parameters from our experimental data? This challenge lies at the heart of ​​identifiability​​, a fundamental concept that acts as a check against creating models that are plausible but not truly meaningful. This article addresses the crucial problem of distinguishing between reliable models built on solid evidence and those built on a foundation of ambiguity.

To navigate this complex topic, we will first explore the core concepts in the chapter ​​Principles and Mechanisms​​. Here, you will learn the vital distinction between structural identifiability, an ideal property of the model itself, and practical identifiability, a real-world challenge tied to the quality and quantity of data. We will also introduce powerful diagnostic tools to detect and understand identifiability issues. Following this, the chapter on ​​Applications and Interdisciplinary Connections​​ will showcase how these principles are not just theoretical concerns but have profound, practical implications across diverse fields like systems biology, engineering, and even regulatory policy, demonstrating the universal importance of knowing what we can, and cannot, learn from our data.

Imagine you are a detective trying to solve a mystery. You have a theory—a model—of what happened. But a theory is only as good as the evidence you can gather to support it. Can you uniquely identify the culprits and their motives from the clues you find? In the world of scientific modeling, this is the central question of ​​identifiability​​. It is, in many ways, the conscience of the quantitative scientist, a crucial check against the ever-present danger of fooling ourselves with models that look good on the surface but are built on a foundation of sand.

After our introduction to the topic, let's now delve into the core principles. The first thing to understand is that there are two fundamentally different ways our investigation can fail. The problem might be in our theory itself, or it might be in the clues we collected. This distinction separates the world of identifiability into two domains: the ideal world of structural identifiability and the messy, real world of practical identifiability.

Think of ​​structural identifiability​​ as having a perfect, detailed blueprint for a machine. The question is: if you follow this blueprint, is there only one possible machine you can build? Or are there ambiguities in the design itself, allowing different parts to be swapped out to produce machines that look and function identically? Structural identifiability is a property of the model equations and the experimental setup in an idealized world of perfect, noise-free measurements. It asks: if we had infinite and perfect data, could we uniquely pin down the values of our parameters?

​​Practical identifiability​​, on the other hand, is like trying to reconstruct the machine from a limited number of blurry, poorly-lit photographs. Even if the original blueprint was perfect (i.e., the model is structurally identifiable), our real-world data—finite, noisy, and perhaps taken from unhelpful angles—might not be good enough to distinguish one version of the machine from another. It addresses the question: given the data we actually have, can we estimate our parameters with any reasonable degree of certainty?

Let's explore these two ideas by examining how they can go wrong.

Sometimes, a model has an intrinsic, mathematical flaw that makes certain parameters impossible to distinguish, no matter how good our data is.

A classic example of this arises in ecology. Imagine you're modeling a simple predator-prey system, like wolves and rabbits, using the famous Lotka-Volterra equations. The model includes an "attack rate" ($a$) for the wolves and a "conversion efficiency" ($e$) describing how many rabbits it takes to create a new wolf. Now, suppose your experiment only allows you to observe the rabbit population, $N(t)$. You can't directly count the wolves, $P(t)$. You might find that a certain population of wolves with a given attack rate perfectly describes the rise and fall of the rabbits. But could a different scenario work just as well?

What if there were twice as many wolves ($P'(t) = 2P(t)$), but each wolf was only half as efficient at hunting ($a' = a/2$)? To keep the system balanced, their conversion efficiency would also need to change ($e' = 2e$). From the rabbits' perspective, nothing has changed! The rate at which they are eaten, which depends on the product $a \cdot P$, remains identical. Since you are only watching the rabbits, you can never, ever tell these two scenarios apart. This is a ​​scaling symmetry​​, a fundamental ambiguity in the model-experiment structure. The parameters $a$ and $e$ are ​​structurally non-identifiable​​ when only the prey is observed. The blueprint itself is flawed.

This isn't just a problem in complex models. It can arise from a poorly designed experiment. In biochemistry, the Michaelis-Menten equation $v = \frac{V_{\max} S}{K_M + S}$ describes enzyme reaction rates. If you conduct an experiment where you only measure the initial reaction rate $v_0$ at a single substrate concentration $S_0$, you are left with one equation and two unknown parameters, $V_{\max}$ and $K_M$. An infinite number of ($V_{\max}, K_M$) pairs can satisfy this single equation, forming a line in parameter space. The parameters are structurally non-identifiable from this experiment. To solve this, you need more information—either measuring the rate at multiple different concentrations $S$ or measuring the entire time course of the reaction.

More often than not in science, our model's blueprint is perfectly sound—it is structurally identifiable. Yet, we still find ourselves unable to pin down our parameters. This is the domain of ​​practical non-identifiability​​, and it almost always comes down to one thing: a poor choice of experiment.

Imagine a simple gene circuit where a gene is constantly being produced at rate $a$ and degraded at rate $b$. The concentration of the gene's product, $x(t)$, follows the equation $\dot{x}(t) = a - b x(t)$. The system will eventually settle to a ​​steady state​​ where production balances degradation, at a level of $x^* = a/b$. Now, suppose you run your experiment by waiting for the system to reach this steady state and then taking all your measurements. What have you learned? You've learned the ratio $a/b$ with great precision. But you have learned absolutely nothing about the individual values of $a$ and $b$! A system with $a=10, b=1$ has the same steady state as one with $a=20, b=2$. By only looking at the system when it's "resting," you've missed all the action—the transient dynamics that reveal the individual timescales of production and breakdown.

This crime of conducting "boring" experiments is a primary cause of practical non-identifiability. To learn about a parameter, you must perform an experiment that is ​​sensitive​​ to that parameter.

Let's take a more subtle example from synthetic biology. A gene's activity is controlled by an input molecule, and the response follows a sigmoidal Hill curve, characterized by a maximum response $\gamma$ and an activation constant $K$ (the concentration of input needed for a half-maximal response). To find $K$, it seems intuitive that you must probe the system with input concentrations around the value of $K$. What if, instead, you only conduct experiments at input levels far below $K$? In this low-input regime, the sigmoidal curve looks like a simple power law: the response is just proportional to $(U/K)^n$. The parameters $\gamma$ and $K$ meld together into a single effective parameter, $\gamma/K^n$. Your data can determine this lumped parameter, but it cannot untangle $\gamma$ and $K$ individually. Even though the model is structurally identifiable, your experimental design has rendered $K$ practically non-identifiable. You took your photograph from an angle that completely obscured the feature you wanted to measure. This same principle applies when trying to determine the dissociation constant $K_d$ in protein-ligand binding studies. If you don't collect data around the $K_d$, you won't be able to estimate it reliably.

How do we, as careful scientists, diagnose these issues? We need tools to inspect our models and data for signs of identifiability trouble.

The Fisher Information Matrix: Our Information Accountant

The most fundamental tool is the ​​Fisher Information Matrix (FIM)​​. You can think of it as a mathematical accountant that tallies up the "information" each data point provides about the parameters. What is this "information"? It's the ​​sensitivity​​ of your model's prediction to a small change in a parameter. If wiggling a parameter a little bit causes a big, noticeable change in the model's output, then data in that region are highly informative about that parameter. The FIM is essentially a sum of these sensitivities over all your data points.

A model with good practical identifiability will have a "strong," well-conditioned FIM. But what if it's "weak"? Often, biological models are ​​sloppy​​. This means the FIM has some very large eigenvalues and some very small ones. Each eigenvalue corresponds to a direction in parameter space. A large eigenvalue means we have a lot of information in that direction. A tiny eigenvalue indicates a "sloppy" direction—a specific combination of parameters that can be changed dramatically without making the model's fit to the data much worse. The eigenvector corresponding to that tiny eigenvalue is the detective's holy grail: it tells you exactly which combination of parameters is causing the problem. For example, in a complex population model, it might reveal that the total population size $N$ is hopelessly confounded with the population's heterogeneity $\sigma^2$.

To make a model identifiable, then, is to design an experiment that provides information in all directions, especially the "sloppy" ones. This might involve choosing a more informative input signal—a concept formalized as ​​persistent excitation​​, where the input must be rich enough to continually "kick" the system and reveal all of its dynamic modes.

Profile Likelihoods: A Visual Diagnostic

The FIM can be abstract. A more intuitive and visual tool is the ​​profile likelihood​​. This technique lets you "interview" each parameter one by one. To create a profile for a parameter, say $K_M$, you systematically fix its value across a range. At each fixed value, you find the best possible fit to your data by letting all other parameters (like $V_{\max}$) adjust freely. You then plot this best-possible-fit "likelihood" against the value of $K_M$ you fixed.

The result is powerfully illuminating:

• If a parameter is ​​practically identifiable​​, its profile will show a clear, sharp peak. The location of the peak is your best estimate, and the width of the peak defines your confidence in that estimate.
• If a parameter is ​​practically non-identifiable​​, its profile will be flat, or have a long, flat valley. This flatness is a giant red flag. It tells you that a huge range of parameter values are all equally plausible given your data. Your experiment is simply uninformative.

The beauty of the profile likelihood is its directness. It turns an abstract statistical problem into a picture that screams "I'm confident about this one!" or "I have no idea about this one!"

In the end, identifiability analysis is not just a mathematical chore. It is a critical part of the scientific process. It forces us to think deeply about the connection between our theories and our experiments. It may tell us our model is too complex for our data, that our experiment was poorly designed, or even that our data quality is suspect due to outliers. By heeding its warnings, we can design better experiments, build more reliable models, and ensure that the scientific stories we tell are ones we can truly stand behind.

Now that we have grappled with the principles of identifiability, you might be thinking, "This is all very interesting, but what is it good for?" This is the truest test of any scientific idea. A concept that lives only on a blackboard is a ghost; a concept that helps us understand the buzzing, blooming, and bewildering world around us is a living thing. Our journey now is to see where the idea of identifiability lives, to find its footprints in fields as different as biochemistry, disease ecology, and even regulatory policy. We will see that this is not some esoteric concern for mathematicians, but a fundamental challenge that every quantitative scientist, engineer, and thinker must face. It is the art of knowing what we can know.

Let’s begin our adventure by shrinking down to the world within a single cell, a bustling metropolis of molecules. Here, biochemists and systems biologists build intricate models—maps of metabolic pathways, gene networks, and signaling cascades—to make sense of the chaos. But a map is only as good as the landmarks on it, and for these models, the landmarks are parameters: reaction rates, binding affinities, and Michaelis constants. How do we measure them?

Imagine you are a biochemist trying to understand how an enzyme works. You have the famous Michaelis-Menten model, a pearl of biochemical theory, which relates the rate of reaction to the concentration of a substrate $S$ using two magic numbers: the maximum rate $V_{\max}$ and the Michaelis constant $K_M$. A common experimental pitfall is to perform all your measurements at very low substrate concentrations, where $S$ is much smaller than $K_M$. In this regime, the elegant Michaelis-Menten equation, $v = V_{\max}S / (K_M + S)$, simplifies. It becomes approximately $v \approx (V_{\max}/K_M)S$. Notice what happened! The two parameters $V_{\max}$ and $K_M$ have collapsed into a single, identifiable ratio, the specificity constant. Your data can tell you this ratio with great precision, but it has no power to disentangle $V_{\max}$ from $K_M$. Any combination of a huge $V_{\max}$ with a huge $K_M$ would give the same result as a tiny $V_{\max}$ with a tiny $K_M$. You have discovered a shadow, but not the object that cast it. To see the full picture, to measure both parameters, you must design a better experiment, one that includes substrate concentrations near and well above $K_M$ to see the reaction saturate. This is our first lesson: a poor experiment can render even a perfect model structurally unidentifiable.

The challenge deepens when we consider not one enzyme, but a whole chain of them. Consider the breakdown of fats in your body, a process beginning with the conversion of triacylglycerols (TAG) to diacylglycerols (DAG) by one enzyme, followed by the conversion of DAG to other products by a second enzyme. Or think of a simple chemical assembly line $A \xrightarrow{k_1} B \xrightarrow{k_2} C$. To determine the rates of both steps, $k_1$ and $k_2$, you must watch the whole movie. You need to see the initial material $A$ disappear, the intermediate $B$ rise and then fall, and the final product $C$ accumulate. If you only watch the very beginning, you won't see enough of the second step to say anything about $k_2$. If you only watch the very end, the first step is long over, and you've lost your information about $k_1$. To identify both parameters, your data must have "dynamic richness"; it must capture the characteristic timescale of every important process in the system.

But what if you can't see the whole movie? In modern biology, we often use glowing reporters—like a firefly's luciferase—attached to a molecule of interest. Imagine studying the circadian clock, the 24-hour pacemaker in our cells. A beautiful model describes the feedback loop of a gene's mRNA ($M$) being translated into a protein ($P$), which then enters the nucleus ($P_n$) to shut down its own gene. Suppose we can only measure the mRNA levels, and even then, only via a luciferase reporter that gives off light, $Y(t) = k_{\mathrm{luc}} M(t)$. We have two problems. First, we aren't observing the proteins $P$ or $P_n$ at all. Second, the scaling factor $k_{\mathrm{luc}}$ is an unknown quantity of the experiment itself. It turns out that there is a "scaling symmetry" in the model. We can get the exact same light output $Y(t)$ from a model with a high transcription rate and one with a low transcription rate, just by adjusting our assumptions about the states and the unknown scaling factor $k_{\mathrm{luc}}$. This is a profound structural non-identifiability. No amount of data from this single reporter can solve the puzzle. To break the symmetry, we need a new kind of measurement—perhaps a second, different-colored reporter on the protein.

The quest for identifiability is not just about passive observation; it is the very foundation of engineering. You cannot hope to control what you cannot reliably model. In the burgeoning field of synthetic biology, scientists aim to build novel biological circuits and engineered ecosystems. Imagine a bioreactor containing a consortium of two microbial species, whose relative abundance we wish to control. We might build a controller that, for example, delivers a toxin that kills species 1 faster than species 2. To design this controller, we need a model with parameters for each species' growth rate and sensitivity to the toxin.

Now suppose our only measurement is the total "cloudiness" of the culture—the total optical density, which is the sum of both species' populations, $y(t) = x_1(t)+x_2(t)$. A daunting problem immediately emerges. The model has a perfect "permutation symmetry." If we swap all the parameters of species 1 with those of species 2, and also swap their initial populations, the sum $y(t)$ remains identical. From the measurement of the total population, we can never know which species is which! We might identify that there is a fast-growing species and a slow-growing one, but we can't assign those properties to "species 1" or "species 2." How, then, can we hope to design a controller to, say, maintain a ratio of $2:1$ for species 1 to species 2? It is impossible without breaking the symmetry. We need a species-specific measurement, like a unique fluorescent protein in each, to make the parameters structurally identifiable and the control problem solvable. Practical identifiability is even more critical. If our parameters for toxin sensitivity have huge uncertainty, a controller designed for the "best guess" parameter values might spectacularly fail—or even crash the system—if the true values are at the other end of the confidence interval.

This principle extends beyond biology. In electrochemistry, engineers study interfaces using techniques like impedance spectroscopy. A common model for an electrode is a simple electrical circuit with resistors and capacitors representing physical processes like solution resistance ($R_s$) and charge transfer ($R_{ct}$). By applying an oscillating voltage across a wide range of frequencies, $\omega$, and measuring the current, one can deduce the parameters. At very high frequencies, the capacitor acts like a short circuit, and the impedance tells us about $R_s$. At zero frequency (DC), the capacitor is an open circuit, and we measure $R_s + R_{ct}$. By observing the system's behavior at these two extremes, we can disentangle the parameters. But if we only measure in a narrow frequency band, the effects of the different components become hopelessly correlated, leading to practical non-identifiability.

Zooming out further, we see the same challenges on the scale of whole ecosystems and societies. Epidemiologists model the spread of a disease through a network of communities. A key part of the model is human mobility—how many people travel between city A and city B? A "gravity" model might propose that this flow depends on the populations of the two cities and the distance between them, each raised to some power. These exponents are the parameters we want to find. If we only have data for two cities, we can measure the infectious coupling between them, but we don't have enough information to uniquely determine all the exponents in the gravity law. There are too many knobs to tune for the single observation we have. However, if we add a third city, and then a fourth, each with different populations and at different distances, we generate more and more independent observations of the coupling. With a rich enough network of data, the parameters of the mobility model, which were once structurally non-identifiable, can be pinned down.

Even in immunology, when modeling the response to a stimulus—like a cytokine's concentration rising and falling after an infection—identifiability is key. One particularly illuminating technique for exploring practical identifiability is the "profile likelihood." In essence, we put each parameter on trial. We fix its value and then allow all other parameters to adjust to find the best possible explanation for the data. We then ask, "How much worse did our explanation get?" If we can change the parameter by a large amount without making the model's fit much worse, then that parameter is "shifty" and poorly identified by our data. Its profile is flat. A well-identified parameter has a sharp, V-shaped profile, where any deviation from its best-fit value quickly makes the model incompatible with the data. This technique can also reveal structural problems. If two parameters, like a production rate $k_{\mathrm{prod}}$ and a stimulus strength $A_0$, only ever appear in the model as a product, $k_{\mathrm{prod}}A_0$, the profile likelihood for $k_{\mathrm{prod}}$ will be a perfectly flat plateau along a curve where this product is constant. The data can identify the product, but not the individual factors.

Finally, we arrive at the grandest scales: reconstructing evolutionary history and regulating our technological future. When evolutionary biologists build a family tree from DNA sequences, they use complex statistical models like GTR$+\Gamma$ to describe the process of nucleotide substitution over millions of years. A modern temptation is to "partition" the data—say, by gene—and assign a separate, parameter-rich model to each partition. But if a partition is very small (contains few DNA sites), there is simply not enough information to estimate all those parameters reliably. This is a classic case of practical non-identifiability leading to overfitting. You are asking the data to tell a story more complex than it is capable of telling. The responsible path is to tie parameters across partitions, which is like assuming a common grammar for the language of evolution, a more parsimonious and robust approach.

This brings us to the intersection of science and society. Imagine a company wants to release an engineered bacterium into the environment. They present a risk assessment model to a regulatory agency. The model predicts the population dynamics of the engineered microbe and its impact on a native species. However, their field sensor, which measures the sum of the two organisms, has an unknown calibration factor, $q$. As we saw with the circadian clock, this creates a structural non-identifiability. The model cannot distinguish between a scenario with a low density of native hosts and a high sensor sensitivity, versus one with a high density of hosts and a low sensor sensitivity. Any claims about the absolute host population are therefore baseless without an independent calibration of the sensor. A regulatory body armed with the concept of identifiability would, and should, reject such evidence. It would demand that the model be reframed in terms of identifiable quantities, or that the monitoring experiment be redesigned. It would also demand that for any model, the proponents demonstrate that the crucial parameters related to risk are practically identifiable with low uncertainty, and apply a precautionary principle when they are not.

And so, we see that the simple question, "Can we know this number?" is not so simple after all. It is a unifying thread that runs through all of quantitative science. From the dance of a single enzyme to the fate of an ecosystem, the ability to build meaningful knowledge from data rests on our honest and rigorous assessment of what is, and is not, identifiable. It is a check on our hubris, a guide for our experiments, and a foundation for responsible innovation.