Physics-Informed Machine Learning

In modern science and engineering, a fundamental dilemma exists: the trade-off between the unerring accuracy of high-fidelity physical simulations and the intoxicating speed of black-box machine learning. While simulations provide ground truth based on first principles, they are often computationally prohibitive. Conversely, data-driven models are fast but lack an understanding of underlying physics, leading to physically nonsensical predictions and poor performance on novel problems. This gap highlights the need for a new paradigm—one that can combine the speed of machine learning with the rigor of physical law.

Physics-Informed Machine Learning (PIML) emerges as this transformative solution. It is a class of methods designed to "teach" a machine to think like a physicist by embedding fundamental principles directly into the learning process. By doing so, PIML creates models that are not only faster than traditional simulations but also more robust, accurate, and generalizable than their purely data-driven counterparts. This article explores this revolutionary approach, detailing how it works and where it is making a profound impact.

The following chapters will guide you through the world of PIML. In "Principles and Mechanisms," we will dissect the core techniques, from using physical equations as a new kind of "teacher" in the loss function to building models that respect fundamental symmetries by design. Then, in "Applications and Interdisciplinary Connections," we will journey across various scientific domains to witness how PIML is being used to accelerate simulations, refine established theories, and forge entirely new tools for discovery.

Imagine you are a materials scientist searching for a new wonder material, one with extraordinarily high thermal conductivity. You have a library of 10,000 hypothetical crystal structures to test. How do you proceed? You face a classic dilemma. On one hand, you have powerful, high-fidelity physics simulations — think of them as the gold standard, the "ground truth." These simulations, perhaps based on quantum mechanics, can calculate a material's properties with stunning accuracy. The catch? They are incredibly slow. A single calculation might take hundreds of hours on a supercomputer. Simulating all 10,000 structures is simply out of the question; it would take millions of CPU-hours.

On the other hand, a colleague from the computer science department has just built a "black-box" machine learning model. It’s been trained on a database of known materials and can predict whether a new structure is "high-conductivity" or "low-conductivity" in a fraction of a second. The speed is intoxicating! But this model is an empiricist, not a physicist. It learns statistical correlations from the data it has seen, but it has no understanding of the underlying laws of thermal conductivity. It might correctly identify most of the promising candidates, but it also produces a significant number of false positives, and worse, it might behave erratically on structures that are truly novel and unlike anything in its training data.

As highlighted in a common screening scenario, a hybrid approach is often employed: use the fast ML model to create a shortlist, then run the expensive simulation on just those candidates. This is a pragmatic solution, but it leaves us wondering: can we do better? Can we create a model that combines the speed of machine learning with the rigor of physical law? Can we teach the machine to think like a physicist?

This is the very soul of Physics-Informed Machine Learning. It’s not just about using data; it’s about informing the learning process with the fundamental principles that govern the system. A purely data-driven model, when applied to complex scientific data like Mössbauer spectra, might produce results that are physically nonsensical—like negative absorption intensities, line shapes that violate quantum mechanical symmetries, or components that don't add up to 100%, violating the conservation of matter. A physicist would immediately reject such a result. A physics-informed model learns to reject it too, because the laws of physics are woven into its very fabric.

So, how do we "weave" the laws of physics into a machine learning model, which is fundamentally just a giant, differentiable function? The secret lies in changing how we train the model. In conventional supervised learning, we train a model by showing it an input and a correct output, and the "loss function"—the measure of the model's error—is simply the difference between the model's prediction and the true answer.

Physics-Informed Neural Networks (PINNs) take a revolutionary turn. Instead of relying on a vast dataset of pre-solved problems, they learn from the physical laws themselves. The central idea is to construct a loss function that represents the entire mathematical statement of a physical problem.

Let's consider a concrete example, the famous Black-Scholes equation from financial engineering, which describes how the price of a financial option $V$ changes with respect to the asset price $S$ and time $t$. It’s a Partial Differential Equation (PDE):

A well-posed physical problem isn't just the PDE; it's the PDE plus a set of boundary and initial (or terminal) conditions that constrain the solution. For a European call option, we know its value at the expiration time $T$ (the terminal condition) and how its value behaves at extreme prices (e.g., $V=0$ when $S=0$).

A PINN turns this entire problem on its head. We define a neural network, let’s call it $\hat{V}(S, t; \theta)$, that takes position $S$ and time $t$ as inputs and outputs a predicted value for the option, $\hat{V}$. The network's parameters $\theta$ (its weights and biases) are initially random. We then task an optimization algorithm with finding the parameters $\theta$ that minimize a very special loss function. This loss function acts as a "physics teacher," and it has several parts:

1. ​​The PDE Residual Loss:​​ We use the magic of automatic differentiation—a technique that allows us to compute exact derivatives of the network's output with respect to its inputs—to plug our network $\hat{V}(S, t; \theta)$ directly into the Black-Scholes equation. The equation is supposed to equal zero. For our network, it probably won't, especially at the beginning. The amount by which it doesn't equal zero is the ​​PDE residual​​. We calculate this residual at thousands of random points inside our domain and add its squared value to our total loss. This part of the loss essentially tells the network: "Obey the governing physical law everywhere!"

2. ​​The Boundary/Terminal Condition Loss:​​ We also check if the network is respecting the conditions at the edges of the problem. For the option pricing problem, we know $V(S, T) = \max(S-K, 0)$. So, we add another term to our loss: the squared difference between our network's prediction $\hat{V}(S, T; \theta)$ and this known payoff function, sampled at many points along the terminal time $T$. We do the same for all other boundary conditions. This part of the loss tells the network: "Respect the specific context and constraints of this particular problem!"

The total loss is the sum of all these components. By minimizing this total loss, the optimizer forces the neural network to find a function $\hat{V}(S, t; \theta)$ that simultaneously satisfies the governing PDE and all the boundary conditions. In essence, the network discovers the solution to the PDE from first principles, without ever being explicitly told what the solution looks like.

The method described above, where we add penalty terms to the loss function for any violation of a physical law, is known as ​​soft enforcement​​. It's like telling a student, "You'll be penalized for every rule you break." It is incredibly flexible and powerful.

However, sometimes we can be even stricter. We can design the network's very architecture such that it satisfies certain conditions by construction. This is called ​​hard enforcement​​. It's like building a car with a governor that physically prevents it from exceeding the speed limit.

Consider a heat conduction problem where we know the temperature must be exactly $g_D(\mathbf{x})$ on a certain boundary $\Gamma_D$. We could enforce this softly with a loss term. Or, we could use a clever trick. Let's say we have a function $d(\mathbf{x})$ that is zero on the boundary and positive everywhere else (a signed distance function is a good choice). We can then construct our network's output as:

Here, $N_\theta(\mathbf{x})$ is the raw output of our neural network. Look at what happens on the boundary $\Gamma_D$: since $d(\mathbf{x}) = 0$, the entire second term vanishes, and we are left with $\hat{T}(\mathbf{x}) = g_D(\mathbf{x})$, exactly as required! The neural network $N_\theta$ is now free to learn whatever it needs to satisfy the rest of the physics (the PDE in the interior), secure in the knowledge that it can't possibly violate this boundary condition.

Such hard constraints can be more complex to formulate, especially for derivative-based conditions like Neumann or Robin boundary conditions, but they can significantly simplify the learning process by reducing the space of possible solutions the network has to search through. The choice between hard and soft constraints is a key design decision, balancing mathematical elegance and practical implementation.

At this point, you might ask: This is clever, but why is it fundamentally better than a standard black-box model that just learns from data? The answer lies in a deep concept from machine learning: ​​inductive bias​​. An inductive bias is an assumption a model makes to generalize from the finite data it has seen to new, unseen situations. A simple black-box model has weak inductive biases; it might assume the world is locally smooth, but not much else. This makes it vulnerable to learning spurious correlations and failing spectacularly when it has to extrapolate outside its training data.

Physical laws are the most powerful, most truthful, and most effective inductive biases we have. By building them into our models, we are giving them a "cheat sheet" to the universe.

Imagine trying to predict the force of a nanoscale probe indenting a polymer film. A black-box model trained on data from one specific probe radius and loading speed will likely fail if you switch to a different probe or a different speed. But a physicist knows that the system is governed by fundamental principles:

• ​​Scaling Laws:​​ The laws of contact mechanics dictate a precise mathematical relationship (an equivariance) between force, indentation depth, and probe radius ($F \propto R^{1/2} \delta^{3/2}$). A physics-informed model can build this scaling directly into its structure, allowing it to generalize effortlessly to any probe radius.
• ​​Causality & Superposition:​​ The material's response must be causal (the effect cannot precede the cause) and, in many regimes, it obeys the linear superposition principle. This constrains the response to have the mathematical form of a convolution integral.
• ​​Thermodynamics:​​ The second law of thermodynamics demands that a passive material cannot create energy out of nothing (passivity). This places a strong mathematical constraint (complete monotonicity) on the material's relaxation function.

A model endowed with these principles is no longer just curve-fitting. It's learning the underlying material properties that are invariant across different experiments. It has a much better chance of "getting it right" when faced with a new scenario, because its internal reasoning mirrors the physical reasoning of the real world. This is why PINNs can dramatically improve generalization and enable plausible extrapolation where black-box models fail. This principle applies across fields, from enforcing causality through the Kramers-Kronig relations in optics and materials science to ensuring data assimilation in solid mechanics remains physically grounded.

So far, we've discussed PINNs that learn the solution to one specific problem—one set of boundary conditions, one initial state, one forcing function. For example, a single function $\mathbf{u}(\mathbf{x})$ that solves the elasticity equations for a single, fixed load $\mathbf{f}$. To solve a problem with a new load, we would have to retrain the network. This is akin to learning the result of "2 + 3 = 5" but having no idea how to calculate "2 + 4".

The next great leap is to learn the ​​solution operator​​ itself. An operator is a mapping from a function to another function. The solution operator, let's call it $\mathcal{G}$, is the abstract a mapping that takes any valid forcing function $\mathbf{f}$ as input and returns the entire corresponding solution field $\mathbf{u} = \mathcal{G}(\mathbf{f})$ as output. This is like learning the concept of addition itself. Once you've learned the operator, you can solve for a whole family of new problems instantly, without retraining.

Architectures like DeepONets (Deep Operator Networks) are designed for this very purpose. A DeepONet has two main parts: a "branch" network that processes the input function (e.g., the load $\mathbf{f}$) and a "trunk" network that processes the coordinate where you want to know the solution (e.g., the point $\mathbf{x}$). The outputs of these two networks are combined to produce the final prediction.

By training on a dataset of many different input functions and their corresponding solutions (e.g., from a set of FEM simulations), the DeepONet learns the underlying kernel of the solution operator. This is profoundly powerful. It represents a shift from learning a single answer to learning the very engine of cause and effect for a physical system.

With all this power, it's tempting to think PINNs can solve anything. But physics itself teaches us about humility, and there is no greater teacher of humility than chaos.

Consider the famous Lorenz system, a simple-looking set of three ordinary differential equations that famously models atmospheric convection and exhibits chaotic behavior. A hallmark of chaos is ​​sensitive dependence on initial conditions​​, a.k.a. the "butterfly effect." Any two starting points, no matter how close, will eventually lead to wildly divergent trajectories. The rate of this divergence is governed by a quantity called the Lyapunov exponent.

What happens when we train a PINN to solve the Lorenz equations? We can train it on a time interval, say from $t=0$ to $t=T$, and achieve incredibly low loss. The PINN learns the governing equations almost perfectly. But when we ask it to predict what happens for times $t>T$, the trajectory-wise accuracy will inevitably and rapidly deteriorate. Why? Because any tiny error in the PINN's approximation at time $T$ acts as a small perturbation to the initial condition for the future. And in a chaotic system, that tiny error is amplified exponentially.

This is not a failure of the PINN. It is a fundamental feature of the reality it is trying to model. No numerical method—be it a classic Runge-Kutta integrator or a sophisticated PINN—can predict the exact trajectory of a chaotic system indefinitely.

However, this is not the end of the story. While the exact trajectory may be lost to us, the model can still be incredibly useful.

• By using clever training strategies like "multi-shooting" (breaking a long time interval into smaller, connected pieces), we can significantly extend the time horizon for which the prediction remains accurate.
• Even more profoundly, we can add other physical constraints to our loss function. For the Lorenz system, we know that the total volume of any region of its state space must contract at a specific, constant rate. By enforcing this as another soft constraint, we can ensure that even when our predicted trajectory diverges from the true one, it remains on the correct "attractor"—the beautiful, butterfly-shaped structure that contains all possible long-term behaviors of the system.

This means our model can still correctly predict the statistical properties of the system—the climate, if you will—even if it can no longer predict the specific state—the weather. This is a beautiful final lesson: physics-informed machine learning is not about defying the fundamental nature of physical systems, but about building models that understand, respect, and ultimately, think in harmony with them.

Having journeyed through the principles and mechanisms that animate a physics-informed learning machine, we might find ourselves asking, "This is all very clever, but what is it good for?" The answer, as we shall see, is wonderfully broad and deeply transformative. We are not just discussing a new tool for data analysis; we are witnessing the emergence of a new way to do science itself. By teaching our machines the language of physics, we don't merely create better predictors; we build faster simulators, refine established theories, and even forge entirely new tools for scientific discovery.

Let's embark on a tour across the disciplines to see how these ideas are put to work, transforming the landscape of research from the swirling eddies of turbulent flow to the intricate dance of a folding protein.

One of the most immediate and practical uses of physics-informed machine learning is to create "surrogate models." Imagine you have a very complicated, time-consuming computer simulation—perhaps one that calculates the heat transfer from a hot cylinder placed in a cool cross-flow. Running this simulation for every possible combination of fluid velocity and thermal properties would be prohibitively expensive. The underlying physics is often captured in empirical correlations, like the famous Churchill-Bernstein correlation, but even these can be computationally intensive to evaluate millions of times within a larger design loop.

Here, we can use a machine to learn an approximation, a surrogate, that is vastly faster to compute. But we will not use a naive, "black box" approach. A physicist knows that the problem is not really about velocity and viscosity in isolation; it is governed by dimensionless numbers, in this case, the Reynolds number ($\text{Re}$) and the Prandtl number ($\text{Pr}$). These numbers are the natural language of fluid dynamics. By framing the learning problem in terms of these dimensionless groups, and perhaps even guiding the model by transforming the variables (say, by using logarithms, which are common in physical scaling laws), we can build an incredibly efficient and accurate surrogate with very little data. The machine isn't just memorizing points; it's learning the smooth functional relationship that our physical intuition told it must exist. This is PIML in its most pragmatic form: a smart shortcut, guided by physics, to accelerate engineering design and analysis.

Often in science, we have a theory that is good but not perfect. It captures the bulk of a phenomenon but misses some finer details or fails under certain conditions. Think of it as a beautiful pencil sketch of reality. Instead of throwing the sketch away and starting from scratch with a "black box" that knows nothing, PIML allows us to learn just the right colors and shading to add—it learns the residual, the difference between our theory and the ground truth.

Consider the world of quantum chemistry, where we want to calculate the exact energy of a molecule. A foundational result tells us that as we use more and more complex basis sets (indexed by a cardinal number $X$), the calculated energy $E(X)$ approaches the true energy $E_{\text{CBS}}$ with an error that shrinks like $a X^{-3}$. This gives us a simple formula to extrapolate to the exact answer using calculations from just two basis sets, say $X=3$ and $X=4$. This formula is our beautiful pencil sketch. However, the coefficient '$a$' is not a universal constant; it's a rich, molecule-specific quantity. Instead of just treating it as a nuisance to be eliminated algebraically, we can train a machine learning model to predict this residual behavior based on the molecule's structure. The model learns the subtle, system-dependent ways in which the convergence deviates from the simple scaling law, giving us a much more accurate extrapolation.

This same "$\Delta$-ML" philosophy, of learning the correction to a known physical model, is a recurring theme. In materials science, the movement of defects called dislocations is governed by a thermally activated process, following an Arrhenius-like law. But the precise energy barrier depends on a dizzying array of factors: film thickness, core structure, and character angle. A PIML model can start with the basic Arrhenius form and learn how these complex, nanoscale features modify the energy barrier, leading to a predictive model of material strength. Similarly, in modeling turbulent flows, we might have a simple model for the turbulent Prandtl number, $\text{Pr}_t$. A machine can learn a correction factor, $\chi$, that modulates this value based on local flow conditions, dramatically improving heat transfer predictions in simulations. In all these cases, physics provides the canvas, and machine learning provides the masterful finishing touches.

The deepest laws of physics are often expressed not as equations of motion, but as principles of invariance and conservation. Energy is conserved. The laws of physics don't change if you rotate your laboratory. A truly physical model must respect these symmetries. One of the most elegant aspects of PIML is its ability to bake these principles directly into the architecture of the learning machine, ensuring its predictions are physically meaningful by construction.

Imagine trying to predict the binding energy between a drug molecule and a protein. This energy is a scalar quantity; it cannot depend on the arbitrary coordinate system you've used to describe the complex. A naive model fed with raw Cartesian coordinates would be hopelessly confused, trying to learn a different reality for every possible orientation. The physics-informed approach is to build features that are intrinsically invariant. By describing the electrostatic interaction in terms of the relative orientations of multipole moments on the protein and the ligand, and forming scalar-valued dot products, we create features that are guaranteed to be rotationally and translationally invariant. The machine is then free to focus on learning the chemistry, because the physics of symmetry has already been taken care of during feature design.

This idea goes even deeper. In continuum mechanics, the stress in an elastic material is derivable from a scalar energy potential. This fact guarantees that the material's response is conservative and that its stiffness tensor possesses certain fundamental symmetries. We can design a neural network that does not predict stress directly, but instead predicts this scalar potential. The stress and stiffness are then defined as the first and second derivatives of the network's output, calculated using automatic differentiation. By this single architectural choice, we guarantee—without any extra cost or data—that the learned model obeys all the required symmetries and conservation laws of elasticity. The physics is no longer just a term in a loss function; it has been woven into the very DNA of the network.

So far, we have mostly used data from experiments or heavy simulations to teach our models. But what if we could have the laws of physics—the partial differential equations (PDEs) themselves—act as the teacher? This is the revolutionary idea behind Physics-Informed Neural Networks, or PINNs. A PINN learns to satisfy not only a set of data points but also the governing PDE over the entire domain of a problem.

Let's consider the solidification of a liquid, like water freezing into ice. As the material cools, it releases a large amount of latent heat at the phase transition. A simple heat equation that only considers specific heat, $\rho c_p \partial_t T - \nabla \cdot (k \nabla T) = 0$, is physically wrong for this problem; it's missing the latent heat term. If we train a neural network using this incorrect PDE, it will fail spectacularly, no matter how much data we give it. It is being penalized for deviating from a lie!

A true PINN for this problem must be informed by the correct physics, which is the enthalpy formulation: $\rho \partial_t h - \nabla \cdot (k \nabla T) = 0$. Here, the enthalpy $h$ is a function of temperature that explicitly includes the latent heat. By penalizing the residual of this correct equation, the neural network learns a temperature field that properly accounts for the physics of phase change, accurately capturing the motion of the solidification front. The PDE itself provides an infinitely rich source of training information, constraining the solution at every point in space and time. This paradigm allows us to blend the worlds of traditional scientific computing (based on discretizing and solving PDEs) and machine learning in novel ways, sometimes replacing parts of a conventional solver, and at other times, replacing the solver entirely.

Much of scientific modeling focuses on predicting static endpoints: the final folded structure of a protein, the equilibrium state of a system. But nature is not static; it is a process. A grand challenge is to model the pathways of change—the dynamic movie, not just the final snapshot.

Here too, physics-informed learning offers a path forward. Consider the monumental problem of protein folding. While some models can predict the final native structure with remarkable accuracy, they tell us nothing about how the protein gets there. By training on data from molecular dynamics simulations, which are essentially atomic-scale movies of the folding process, a PIML model can learn something much more profound than an endpoint. It can learn the one-step transition probability, $p(x_{t+1}|x_t, S)$, that governs the system's dynamics. By learning this transition kernel, the model can then be used to generate the most probable folding pathway from an unfolded state, providing unprecedented insight into the mechanisms of folding. This represents a leap from predictive modeling to a generative science of processes.

Perhaps the most exciting frontier for PIML is not just in solving existing problems faster or more accurately, but in creating entirely new kinds of scientific instruments. We can move beyond learning a corrective function and learn a fundamental mathematical operator.

In quantum mechanics, to make calculations tractable, physicists often replace the complicated interaction between the core and valence electrons of an atom with a simpler, effective object called a pseudopotential. Designing a good pseudopotential is a high art, guided by deep physical principles like norm-conservation, which ensures that the pseudo-atom scatters electrons in the same way as the real atom. We can now frame this design process as a learning problem: train a model to produce a pseudopotential, but with the fundamental physical constraints, derived directly from scattering theory, imposed as priors or hard constraints on the learning process.

Here, the machine is not just spitting out a number. It is generating a new piece of theory, a new physical operator, that can be downloaded and used by physicists around the world in their own simulations. This is the ultimate vision of physics-informed machine learning: a true synergy where the intuition and abstract principles of the physicist guide the powerful optimization and pattern-recognition capabilities of the machine, not just to analyze the world, but to help us write the very laws that describe it.