Multifidelity Modeling

In the pursuit of scientific discovery and engineering innovation, we increasingly rely on computational models to simulate complex realities. However, we constantly face a fundamental trade-off: the choice between highly accurate, "high-fidelity" simulations that consume immense computational resources, and fast, "low-fidelity" approximations that are quick but inherently biased. Simply choosing one over the other is often insufficient, creating a knowledge gap where we need both accuracy and speed. Multifidelity modeling provides the solution, offering a powerful framework to strategically combine information from these disparate sources. It treats the fast, biased model not as a poor substitute for the truth, but as a valuable scaffold upon which to build a more accurate, computationally affordable understanding.

This article explores the core concepts and widespread applications of this transformative methodology. In the "Principles and Mechanisms" chapter, we will dissect the mathematical and statistical engine that drives multifidelity modeling, from learning the model's error to making economically optimal decisions about data acquisition. Subsequently, the "Applications and Interdisciplinary Connections" chapter will showcase how these principles are applied in diverse fields, from quantum chemistry and materials discovery to climate science and the development of digital twins, illustrating the universal power of fusing information for smarter, faster science.

Imagine you are an explorer charting a vast, unknown mountain range. You have two tools at your disposal. The first is a blurry, low-resolution satellite map of the entire region. It’s cheap and fast to use, giving you a general sense of the terrain, but it misses all the crucial details—the cliffs, the crevices, the precise mountain peaks. Your second tool is a high-powered telescope. It’s incredibly expensive and time-consuming to set up, but wherever you point it, you see the landscape in perfect, crisp detail. How do you combine these two tools to create the best possible map of the entire range in the least amount of time?

This is the central question of ​​multifidelity modeling​​. We live in a world of approximations. From forecasting the weather to designing a jet engine or discovering a new material, we rely on computational models. These models exist on a spectrum. ​​High-fidelity (HF)​​ models, like our telescope, are rooted in complex, fundamental physics—solving the full Navier-Stokes equations for fluid flow or running a quantum-mechanical simulation of a molecule. They are accurate but computationally ravenous, sometimes taking weeks or months for a single run. On the other end are ​​low-fidelity (LF)​​ models, like our blurry map. They use simplified physics, coarser computational grids, or empirical relationships. They are lightning-fast but systematically biased; they get the general trends right but are often quantitatively wrong.

The naive approach of simply averaging their outputs is doomed to fail. The low-fidelity model isn't just a noisy version of the high-fidelity one; it has its own inherent, systematic errors. The true art lies not in trusting the low-fidelity model's answers, but in exploiting its structure. The secret is to use the cheap model to quickly sketch the landscape's broad features and then use the expensive, high-fidelity model to learn and correct the cheap model's specific errors.

The most powerful idea in multifidelity modeling is this: instead of trying to build a machine learning model that predicts the complex high-fidelity output $y_H$ from scratch, we build a model that predicts the difference, or ​​residual​​, between the two models. We model the error itself.

Let's define this correction, often called $\Delta$ (delta), as:

where $\mathbf{x}$ represents the input parameters to our models. Our multifidelity prediction for the high-fidelity output, $\hat{y}_H$, then becomes:

where $\hat{\Delta}(\mathbf{x})$ is the output of our correction model. This approach, sometimes called ​​$\Delta$-learning​​, transforms the problem. Instead of learning the full, complex physics of $y_H$, our machine learning task is to learn the much simpler structure of the error.

In some cases, this error can be captured by a very simple model. For instance, we might find that the high-fidelity result is roughly a scaled and shifted version of the low-fidelity one. This leads to a simple linear ​​autoregressive model​​:

Here, $\rho$ is a scaling factor, and the new correction term $\delta(\mathbf{x})$ might be a simple polynomial function of the inputs, like $\delta(\mathbf{x}) = \beta_0 + \beta_1 x + \beta_2 x^2$. We can use a handful of paired $(y_L, y_H)$ simulations to find the best-fit values for $\rho$ and the $\beta$ coefficients using standard methods like linear regression.

However, the world is rarely so simple. In many real-world problems, especially in materials science or complex physics, the error is not a simple linear function. When we plot the true error $\Delta$ against the low-fidelity prediction $y_L$, we might see tell-tale signs of structure: curvature, clustering of data points by chemical family, or an error that grows larger as the prediction value increases. These patterns are fingerprints of the "missing physics" in the low-fidelity model. In these cases, a simple linear correction is not enough. We need a more flexible, nonlinear machine learning model—like a neural network or a Gaussian process—to learn this complex error landscape.

It's crucial to understand what this approach is not. It is fundamentally different from a popular technique called ​​transfer learning​​, where a model trained on low-fidelity data is simply used as a starting point for fine-tuning on high-fidelity data. In our fusion model, the low-fidelity prediction $y_L(\mathbf{x})$ remains an active and essential input to the final prediction at all times.

We can sharpen our "learning the error" intuition with the rigor of statistics. Suppose our goal is not to predict the entire function $y_H(\mathbf{x})$, but something simpler: its average value, or mean, $\mu_H = \mathbb{E}[H]$. The standard way is to run the expensive HF model $m$ times and take the average, $\bar{H}_m$. The uncertainty of this estimate decreases slowly, proportional to $1/\sqrt{m}$.

Here is where the low-fidelity model becomes a powerful statistical lever. We know that the LF model is biased, so $\mathbb{E}[L] \neq \mu_H$. However, because it captures some of the same underlying physics, its output $L$ is correlated with the HF output $H$. We can exploit this correlation.

Consider a quantity like $(\bar{L}_{m+n} - \bar{L}_m)$, where we've used $m$ LF runs that are paired with our HF runs, plus an extra $n$ cheap LF-only runs. Since both $\bar{L}_{m+n}$ and $\bar{L}_m$ are estimates of the same mean $\mathbb{E}[L]$, their difference has an average of zero. This means we can add it to our HF estimate without introducing any bias. This gives rise to the ​​control variate​​ estimator:

This looks like magic. How can adding a zero-mean term help? The term we added, $(\bar{L}_{m+n} - \bar{L}_m)$, is correlated with our original estimator $\bar{H}_m$. By choosing the coefficient $\alpha$ cleverly, we can make it so that when $\bar{H}_m$ happens to be a high-side error, the correction term is likely to be negative, and vice-versa. We use the randomness in the cheap model to cancel out the randomness in the expensive one. The optimal choice for $\alpha$ turns out to be $\alpha^\star = \operatorname{Cov}(H, L) / \operatorname{Var}(L)$. With this choice, the variance of our new estimate is reduced by a factor of approximately $(1 - \rho^2)$, where $\rho$ is the correlation coefficient between the high- and low-fidelity models. If the models are highly correlated ($\rho \to 1$), the uncertainty in our estimate collapses dramatically. We get a much more precise answer for the same number of expensive simulations.

This idea can be generalized from estimating a single mean value to learning the entire function. This is the domain of ​​co-kriging​​, which uses ​​Gaussian Processes (GPs)​​ to put the autoregressive model $f_H(\mathbf{x}) = \rho f_L(\mathbf{x}) + \delta(\mathbf{x})$ on a fully probabilistic footing. In this framework, we treat the low-fidelity function $f_L(\mathbf{x})$ and the discrepancy function $\delta(\mathbf{x})$ as random functions drawn from a GP prior. A GP is a flexible model that can define a distribution over functions, characterized by a mean and a covariance kernel that describes how smooth the function is and how its values at different points are related.

By combining the HF and LF data within this single probabilistic model, we can make predictions for $f_H(\mathbf{x})$ at new, untried locations. The information from the dense, cheap LF data helps "fill in the gaps" between the sparse, expensive HF data points, dramatically reducing our uncertainty. This is not just a theoretical benefit; in practical applications like predicting nuclear mass properties, adding low-fidelity data can slash the predictive variance of the model.

A key insight from this framework is the importance of having at least some ​​co-located samples​​—input points where we have run both the low- and high-fidelity simulations. These paired data points are essential for the model to learn the crucial scaling parameter $\rho$. Without them, the model can get confused, unable to distinguish whether a change in the high-fidelity output is due to the influence of the low-fidelity function or the discrepancy term. Co-located points anchor the relationship between the two fidelities and allow for a robust fusion of the information.

So far, we have seen how to combine existing data from different models. But in many scientific endeavors, we are actively deciding what to do next. Should we run another expensive DFT simulation, or should we run a thousand fast classical potential calculations? This is a question of economics.

Multifidelity models provide a rational basis for making this decision. The goal of a new simulation is to gain information—to reduce the uncertainty in our model. But each simulation has a cost. The optimal strategy, therefore, is to choose the simulation that provides the maximum ​​expected information gain per unit cost​​.

Using a probabilistic model like co-kriging, we can mathematically calculate the expected reduction in our predictive variance at a target location that would result from acquiring a new piece of data—either a new low-fidelity point or a new high-fidelity point. For example, the expected reduction in variance of $f_H(\mathbf{x}_\star)$ from observing $y_L(\mathbf{x}_c)$ is given by:

We can compute a similar value, $\mathcal{V}_{H}(\mathbf{x}_{c})$, for a high-fidelity observation. We then simply compare the ratios $\mathcal{V}_{L}/c_{L}$ and $\mathcal{V}_{H}/c_{H}$, where $c_L$ and $c_H$ are the respective costs. We choose the fidelity that offers the biggest "bang for the buck." This logic forms the core of multifidelity ​​Bayesian optimization​​ and ​​active learning​​ algorithms, which intelligently guide the search for new materials or optimal designs by always making the most economically sound choice about what data to acquire next.

The principles of multifidelity modeling extend far beyond just combining two computer simulations. They offer a profound, unified framework for thinking about the relationship between models and reality itself.

Think of the hierarchy of information we deal with in science:

1. ​​Physical Reality​​: The ultimate, infinitely complex "high-fidelity" truth.
2. ​​Physics Model​​: Our best mathematical description of reality (e.g., a set of PDEs). This is a low-fidelity approximation of reality. The difference between its predictions and reality is the ​​structural model discrepancy​​.
3. ​​Numerical Solution​​: The solution of our physics model on a computer with a finite mesh. A coarse mesh is a lower-fidelity approximation of the continuum PDE solution on a fine mesh. The difference is the ​​discretization error​​.
4. ​​Experimental Data​​: Our noisy measurements of physical reality. The difference between a measurement and the true physical quantity is ​​instrument noise​​.

A comprehensive ​​Uncertainty Quantification (UQ)​​ framework must account for all these sources of error and uncertainty. And it does so using the very same principles we have just explored. We can use replicate measurements to characterize the instrument noise. We can use mesh refinement studies (treating coarse and fine meshes as different fidelities) to characterize and extrapolate away the discretization error. And we can use a rich set of experimental data to calibrate our physics model's parameters while simultaneously modeling the remaining structural discrepancy.

Viewed through this lens, multifidelity modeling is not just a clever computational trick. It is a fundamental scientific methodology for navigating the inescapable ladder of approximations that connects our abstract theories to concrete, messy reality. It provides a disciplined, quantitative language for understanding what our models know, what they don't know, and how to combine all sources of information—from the fastest, cheapest approximation to the most expensive experiment—into our single best picture of the world.

We have journeyed through the principles of multifidelity modeling, exploring the mathematical gears that make this powerful machinery work. But a machine is only as good as the problems it can solve. Now, we shall see this machinery in action. The true beauty of a fundamental idea in science is not just its elegance, but its universality—the surprising way it reappears, disguised in different costumes, across a vast stage of disciplines. The art of combining cheap, imperfect information with sparse, precious truth is one such idea. It is a recurring theme in humanity's quest to understand and engineer the world, a testament to our ingenuity in the face of immense complexity and finite resources.

Let us now take a tour of the many worlds where this idea has taken root, from the quantum dance of molecules to the grand sweep of atmospheric jets, from the design of new materials to the creation of virtual "digital twins" that mirror reality.

At its heart, one of the simplest and most profound multifidelity strategies is that of correction. Imagine you have a cheap map that is mostly accurate but has a systematic error—perhaps it's shifted a little to the north everywhere. If you could afford just a few very precise GPS measurements, you could figure out the exact shift and correct your entire map. This is the essence of additive bias correction.

Nowhere is this idea more beautifully embodied than in the world of computational chemistry. Chemists often need to calculate the energy of a large, complex molecule. Using the most accurate quantum mechanical methods on the entire molecule would be computationally ruinous. However, they had a brilliant insight: the most important and computationally expensive quantum effects are often local. The subtle dance of electrons that a simple model gets wrong is largely confined to the core, chemically active part of the molecule. This led to the development of methods like ONIOM (Our own N-layered Integrated molecular Orbital and molecular Mechanics).

In the language of multifidelity modeling, the strategy is stunningly simple. The total energy is estimated as:

Here, $\mathcal{R}$ is the full, "real" molecule and $\mathcal{M}$ is the small, "model" subsystem at its core. $E^{\mathrm{L}}$ is the energy from a cheap, low-fidelity calculation, and $E^{\mathrm{H}}$ is from an expensive, high-fidelity one. The formula tells us to take the cheap calculation for the whole system, $E^{\mathrm{L}}(\mathcal{R})$, and add a correction term. This correction, the term in parentheses, is the bias—the difference between the expensive and cheap models—calculated only for the small, manageable model system $\mathcal{M}$. It's a masterful piece of physical intuition translated into a multi-fidelity recipe.

This "correct the cheap model" philosophy is a general principle. The correction itself doesn't have to be a single number; it can be a function that captures how the error changes from place to place. We can use a few high-fidelity data points to build an interpolant, such as a polynomial, that approximates this error function, which we then add back to our low-fidelity model to get a more accurate result everywhere.

Moving from molecules to the design of solid materials, the same principle applies, but with added statistical sophistication. In materials science, we might use a cheap empirical model like the Modified Embedded Atom Method (MEAM) to predict properties, and a very expensive quantum model like Density Functional Theory (DFT) for high accuracy. Here, the relationship between the cheap and expensive models might be more complex than a simple offset. We can model it with a statistical relationship, such as the autoregressive model:

This equation states that the high-fidelity property $y_H$ is a scaled version of the low-fidelity property $y_L$ (with scaling factor $\rho$) plus a discrepancy term $\delta(x)$. Methods like co-kriging, based on the theory of Gaussian processes, provide a powerful framework to learn the parameters $\rho$ and the function $\delta(x)$ by combining a wealth of cheap MEAM data with a handful of precious DFT calculations. This allows scientists to efficiently screen vast arrays of potential new alloys for desirable properties like strength or fault tolerance, accelerating the discovery of next-generation materials.

The world of fluids is a realm of mesmerizing complexity. From the chaotic eddies shed by a wing to the majestic march of weather systems, simulating fluid flow is one of the grand challenges of computational science. Here, too, multifidelity modeling provides an indispensable toolkit.

Consider the flow of air right next to a surface, a region known as the boundary layer. Resolving the tiny, violent eddies in this layer is incredibly expensive. However, physicists have long known that in a certain part of this layer, the "overlap region," the average velocity profile follows a universal, elegant logarithmic shape known as the "logarithmic law of the wall." This law, $U^{+}(y^{+}) = \frac{1}{\kappa} \ln(y^{+}) + B$, contains parameters like the von Kármán "constant" $\kappa$ that can vary slightly with flow conditions. A brilliant multifidelity strategy emerges: instead of trying to correct a coarse simulation point-by-point, we can use a few high-fidelity results to learn the correct value of the physical parameter $\kappa$. We then enforce this physically-correct logarithmic shape onto the biased results of a coarse simulation, bending it into a more truthful form. This is a beautiful example of a "physics-informed" multifidelity approach, where we fuse data not just from different simulations, but from our fundamental understanding of the physics itself.

Scaling up to the entire planet, we face a different challenge: uncertainty. Our climate models are full of parameters that represent processes too small or complex to simulate directly, such as the effective viscosity $\nu_t$ generated by sub-grid scale eddies. We don't know the exact values of these parameters. To understand how this uncertainty affects large-scale predictions—for instance, the position of the jet stream—we must run our model thousands of times in a Monte Carlo simulation. Performing this with a full-blown, high-resolution climate model is a non-starter. The solution is to build a simple, fast-to-evaluate surrogate model. We can run the full, complex climate model (high-fidelity) and a simplified, cheaper version (low-fidelity) for a few parameter values. We then fit a simple statistical model, like a linear regression, that maps the output of the cheap model to the output of the expensive one: $y_{\text{jet-stream}}^{\text{High}} \approx \rho \, y_{\text{jet-stream}}^{\text{Low}} + c$. This cheap surrogate can then be run millions of times in the Monte Carlo simulation, allowing us to quantify the uncertainty in our climate predictions at a fraction of the cost.

So far, we have seen multifidelity models used to make better predictions. But their power extends far beyond that, into the realm of decision-making, design, and control.

Imagine you are designing a new antenna. The design is controlled by a set of parameters, and for each set, you can run a simulation to see how well it performs. Running a high-fidelity simulation is slow and expensive. How do you find the best design without spending a lifetime on simulations? This is where multifidelity modeling meets optimization. We can build a surrogate model—again, perhaps a co-kriging model—that learns the relationship between the design parameters and performance, using both cheap Finite-Difference Time-Domain (FDTD) simulations and expensive Finite Element Method (FEM) simulations. But here’s the clever part: this surrogate does more than just predict. It also tells us where its predictions are most uncertain. An intelligent optimization algorithm, like an Evolutionary Algorithm, can use this information. At each step, it asks: "Given my limited budget, what is the most valuable simulation to run next? Should I run a cheap one to quickly explore a new design region, or an expensive one to reduce the uncertainty around a promising candidate?" This strategy, known as Bayesian Optimization or active learning, uses the model to intelligently guide the search, balancing exploration (reducing uncertainty) and exploitation (finding the minimum). It turns the simulation process from a brute-force grind into an intelligent, targeted inquiry.

This idea of a dynamic, learning model of a system is at the heart of the "digital twin" concept. A digital twin is a virtual replica of a physical asset—a jet engine, a wind turbine, a bridge—that is continuously updated with real-world data. To be useful, this twin must run in real-time or faster. This often precludes using a full, high-fidelity FE model all the time. A common strategy is to use a fast Reduced-Order Model (ROM) by default, while constantly monitoring an error estimator. If the estimator signals that the ROM is drifting too far from reality, the system dynamically switches to the full FE model for one or more steps to re-calibrate and anchor itself back to the ground truth before switching back to the fast ROM. This dynamic switching between fidelities is a powerful and practical multifidelity strategy. Of course, this hybrid approach comes with its own complexities, including serial overheads from the error estimation and switching costs, which can limit its parallel scalability on supercomputers—a crucial, practical consideration in its design.

The multifidelity philosophy is so fundamental that it connects and enriches many other advanced fields of computational science. It is not a monolithic technique, but a symphony of methods, each suited to a different kind of problem.

In the field of Uncertainty Quantification (UQ), where the goal is to understand how uncertainties in model inputs propagate to the outputs, multifidelity methods are transformative. Techniques like Polynomial Chaos Expansions (PCE) create a surrogate model by representing the output as a polynomial of the uncertain inputs. Building a high-fidelity PCE requires many runs of the expensive model. By constructing a multi-fidelity PCE that synergistically combines results from many cheap runs and a few expensive ones, we can create far more accurate uncertainty estimates for the same computational budget.

In geomechanics, when trying to characterize subsurface properties like hydraulic conductivity, we might have data from different types of sensors (e.g., flowmeters and slug tests) and different predictive models (e.g., a simple 1D screening model versus a full 3D simulation). Here, a different multifidelity philosophy can be used: Bayesian Model Averaging (BMA). Instead of trying to construct a single blended model, we treat the low- and high-fidelity models as two competing "hypotheses." Using the language of Bayesian inference, we can calculate the "evidence" or marginal likelihood for each model, given the observed data. This tells us how plausible each model is. The final, combined prediction is then a weighted average of the predictions from each model, where the weights are their posterior probabilities. We let the data decide how much to trust each model.

The sophistication continues. When our low- and high-fidelity models are based on fundamentally different numerical methods—for instance, a local Discontinuous Galerkin (DG) method versus a global spectral method—we cannot simply mix their coefficient vectors. They live in different "function spaces." Advanced multifidelity Reduced Order Models (ROMs) must first build explicit mathematical mappings to align these spaces. Furthermore, for problems like fluid advection, a naive projection to a reduced model can be unstable. The design of a stable multi-fidelity ROM must incorporate principles from the underlying numerical methods, such as upwind stabilization, into the very structure of the projection, for example by using a Least-Squares Petrov-Galerkin (LSPG) formulation. This shows that deep domain knowledge is essential for creating robust multifidelity models.

From the simplest corrective nudge to a Bayesian debate between models, the multifidelity paradigm is a powerful and unifying thread running through modern computational science. It is the science of making smart compromises, of building bridges between the tractable and the true. It reminds us that often, the path to a profound answer is not a single, heroic leap, but a series of clever, calculated steps, leaning on every piece of information we can gather along the way.