SciencePediaThe quest to understand and predict complex physical systems, from global climate patterns to turbulent fluid flows, has long been the domain of numerical simulation. However, these traditional methods, while powerful, are often computationally prohibitive, creating a major bottleneck in scientific discovery and engineering design. While deep learning has transformed many fields, conventional neural networks are fundamentally limited in this domain; they are designed to map fixed-sized vectors and struggle to learn underlying physical laws independently of the grid on which they are measured.
This article introduces neural operators, a revolutionary class of models that directly address this challenge by learning operators—mappings between entire function spaces. We will first explore the core Principles and Mechanisms that grant these models their power, including the concept of discretization invariance and the specific architectures of Fourier Neural Operators (FNOs) and Deep Operator Networks (DeepONets). Subsequently, we will survey their transformative Applications and Interdisciplinary Connections, showcasing how they are accelerating research across physics, engineering, and beyond. To begin, we must understand the fundamental shift in thinking that neural operators represent, moving from approximating functions to learning the operators that govern them.
To truly appreciate the revolution that neural operators represent, we must first journey back to the familiar world of conventional neural networks. For decades, these networks have been celebrated for their ability to learn complex relationships, acting as "universal function approximators." You give them a fixed-size list of numbers as input—say, the pixel values of a photograph—and they produce another fixed-size list of numbers as output—perhaps the probability that the photo contains a cat. They map vectors in to vectors in . But what if the problem we want to solve isn't about fixed lists of numbers?
What if we want to predict the weather? The input isn't a fixed-size vector; it's a continuous function of temperature, pressure, and wind defined over the entire globe. The output we desire is another set of functions, predicting these values for tomorrow. What if we want to design a new airplane wing? The input is the shape of the wing (a function), and the output is the airflow around it (another function). The world of science and engineering is filled with such problems, where we need to learn mappings not between vectors, but between functions. These mappings are known in mathematics as operators.
A traditional neural network, trained on a simulation of airflow over a wing discretized on a grid, is completely lost if you then ask it to predict the flow on a finer grid. The network's very architecture is tied to the input dimension. More subtly, even if we could retrain it, the properties that ensure its predictions are stable and reliable might degrade as the resolution increases. We aren't learning the physics; we are just learning a pixel-to-pixel mapping for one specific camera.
The quest, then, is to build a new kind of learning machine: one that learns the operator itself, the underlying physical law, independent of how we choose to measure or discretize it. This property is the holy grail of scientific machine learning: discretization invariance. A truly discretization-invariant model, once trained, could take data from a coarse weather simulation and produce a high-resolution forecast, or use sensor data from an arbitrary set of locations on an airplane wing to map out the complete pressure field. It learns the continuous physical reality, not the discrete shadow it casts on our sensors or grids.
How could one possibly build such a machine? Let's turn to physics for inspiration. Many fundamental operators in nature, from gravity to electromagnetism, can be expressed in the form of an integral transform. The output of the operator at a point is given by an integral of the input function over the entire domain, weighted by a function called the kernel:
The kernel encodes the interaction: it specifies how the input at point influences the output at point . This provides us with a universal blueprint. A neural operator is, at its heart, a sophisticated, learnable version of this integral operator. The general architecture, which can be adapted to any geometry, involves three steps:
Lifting: We first take the input function and, at each point, lift it into a higher-dimensional channel space, creating a representation . This is like giving the network a richer "scratchpad" to perform its calculations.
Iterative Updates: We then apply a sequence of layers. Each layer updates the function to by composing a learnable integral operator with a simple, pointwise linear transformation and a nonlinearity :
Projection: After the final layer, we take the rich, high-dimensional representation and project it back down, point by point, to the final output function .
This elegant blueprint is remarkably general. For functions defined on an arbitrary mesh or point cloud, we can approximate the integral with a weighted sum, where the weights account for the local density of points. This gives rise to Graph Neural Operators, which learn the underlying continuous physics even on complex, unstructured geometries by learning a kernel that depends on the properties of points, not their arbitrary indices on a grid. The core principle remains the same: learn the interaction kernel of a continuous integral operator.
Now, let's consider a special but incredibly important case: problems on a regular grid, like an image or a simulation on a rectangular domain. Here, we can employ one of the most powerful "magic tricks" in all of mathematics and physics: the Fourier transform. The celebrated Convolution Theorem tells us that a complex global operation in real space—convolution—becomes a simple, local multiplication in frequency space. Many physical processes, like diffusion (heat flow), are described by convolutions.
This is the central idea behind the Fourier Neural Operator (FNO). Instead of trying to learn a complicated, space-dependent kernel , the FNO learns a much simpler, translation-invariant kernel by parameterizing its Fourier transform. The workflow of a single FNO layer is a beautiful symphony of transformations:
To the Frequency Domain: First, the input function , represented on a grid, is transformed into its frequency components using the Fast Fourier Transform (FFT).
Filtering and Mixing: In this spectral domain, the FNO performs its key operation. It truncates the high frequencies, keeping only modes for some cutoff . This acts as a low-pass filter, which not only makes the model more efficient but also provides an implicit form of regularization, making the learning process more stable, especially when data is noisy. On these retained frequencies, it applies a learned complex-valued matrix , which multiplies the modes and mixes information across the different channels of the function. This multiplication is the learned "physics" of the layer. To ensure that a real-valued input function produces a real-valued output, the learned weights must obey a beautiful conjugate symmetry, .
Back to the Real World: The filtered modes are then transformed back to the spatial domain using the inverse FFT. This entire process effectively applies a global convolution to the input function.
Local Refinement: The result of the global convolution is combined with the result of a simple, pointwise linear map (and a skip connection), and finally passed through a non-linear activation function .
The true genius of this approach lies in its solution to the discretization invariance problem. The learned parameters define the function in the continuous frequency domain, not as a collection of weights tied to specific grid indices. Therefore, if we want to evaluate the operator on a new grid with a different resolution, we simply apply the FFT on that new grid and sample the very same learned function at the new grid's corresponding frequency locations. This grants the FNO its remarkable zero-shot super-resolution capabilities.
Furthermore, this frequency-domain approach is incredibly efficient. A direct global convolution on a grid with points would have a computational cost that scales quadratically with . By using the FFT, the FNO achieves the same goal with a cost of only . This computational leap makes it feasible to learn complex, long-range interactions in large-scale systems, a task that is often intractable for standard architectures like CNNs, which are restricted to small, local kernels.
The FNO is powerful, but it relies on a specific basis—the Fourier basis—which is most natural for periodic domains. What if we want a different kind of flexibility? Let's return to our integral operator blueprint and look at it through a different lens. What if we could approximate the kernel using a separated representation?
If we substitute this into our integral, something wonderful happens:
Look closely at this expression. The term in the parentheses, which we can call a coefficient , depends only on the input function . The other term, , depends only on the output coordinate where we want to evaluate the solution. The output is a linear combination of basis functions where the coefficients are determined by the input function .
This is the elegant idea behind the Deep Operator Network (DeepONet). It is architecturally a duet between two distinct neural networks:
The final output is simply the dot product of the outputs of these two networks. The branch net listens to the entire input function and decides what to express, while the trunk net learns a suitable set of basis functions to express it where needed.
The DeepONet's approach to discretization invariance is different but equally powerful. Because the trunk network takes a continuous coordinate as input, we can ask for the solution at any point in the domain, giving it natural output resolution independence. Input invariance is achieved by fixing the sensor locations for the branch network in physical space, independent of any particular grid.
These architectures are not just clever engineering tricks; they are grounded in deep mathematical principles. Just as standard neural networks are universal approximators for functions, it has been proven that neural operators are universal approximators for continuous operators. The combination of global integral transforms (like the FNO's spectral convolution) and local, pointwise non-linearities is powerful enough to approximate any continuous mapping from one function space to another. The theory confirms that by stacking these layers, we can build up arbitrarily complex, non-translation-invariant kernels from simple, efficient components.
Of course, the real world is messy. It has complicated geometries and boundaries. The FNO, in its purest form, assumes a periodic world. But the principles are adaptable. By combining operator learning with classic techniques from numerical analysis, these challenges can be overcome. Instead of forcing a non-periodic problem into a periodic box, we can transform the problem itself by using lifting functions to handle boundary conditions, or we can build an operator using a more suitable spectral basis, like Chebyshev polynomials, which are natural for bounded domains. DeepONets, by their very design, can incorporate boundary conditions through clever architectural choices, such as multiplying their output by a mask that vanishes at the boundary.
Perhaps the most exciting promise lies in predicting the future—learning the dynamics of evolving systems. We can train a neural operator to approximate the evolution of a system (like fluid flow or weather) over a single, short time step, . Then, to predict the long-term future, we simply apply the learned operator repeatedly in a "rollout." One might fear that small errors made by the model at each step would accumulate and quickly lead to catastrophic failure. But here, physics comes to the rescue.
A remarkable analysis shows that if the underlying physical system is dissipative (like heat flow, where energy decays), the long-term error of the learned model remains provably bounded, regardless of how many steps we take. The system's natural stability continuously corrects the model's small inaccuracies. Even for energy-conserving systems (like ideal wave propagation), the error grows at most linearly with time—a far cry from the explosive exponential growth one might naively expect. When a learned operator truly captures the underlying physics of a system, its predictions inherit the system's own stability, opening the door to reliable, long-horizon forecasting at a fraction of the cost of traditional simulators. This is the beautiful unity of physics and machine learning, a partnership that is just beginning to unfold.
Having peered into the inner workings of neural operators, we now embark on a grand tour of their domain. We have seen that their essence is the ability to learn mappings between entire function spaces—a concept that might seem abstract at first glance. But it is precisely this abstraction that unlocks a spectacular range of applications, weaving together fields as diverse as engineering, climate science, materials research, and even cosmology. We will see that neural operators are not merely a new tool for an old toolbox; they represent a fundamental shift in how we can approach complex scientific problems, moving us from solving single instances to learning the very laws that govern entire families of phenomena.
The most direct and perhaps most intuitive application of a neural operator is to act as a surrogate for a computationally expensive physical simulation. Imagine the challenge of modeling groundwater flow through porous rock, a problem described by Darcy's law. Solving the underlying partial differential equation (PDE) for every new rock permeability configuration can take hours or even days on a supercomputer.
A neural operator offers a breathtaking alternative. We can train it on a dataset of high-fidelity simulations, teaching it the mapping from the input function (the permeability field, ) to the output function (the pressure field, ). Once trained, the operator can predict the solution for a new, unseen permeability field in a fraction of a second. It has learned the physics, encapsulating the solution operator of the PDE itself.
What is truly remarkable is a property known as discretization invariance. Because the operator, particularly a Fourier Neural Operator (FNO), learns the relationship in a continuous space (like the Fourier domain), it is not tied to the specific grid resolution of its training data. An operator trained on a coarse grid can make startlingly accurate predictions on a much finer grid, a feat known as zero-shot super-resolution. It has learned the continuous physical law, not just a discrete pixel-to-pixel mapping.
This power is not limited to fluid dynamics. Consider the elegant problem of solving the Laplace equation, , inside a domain. A classic mathematical object associated with this is the Dirichlet-to-Neumann (DtN) map, an operator that takes a function defined on the boundary of the domain (the Dirichlet data) and gives back another function on the boundary (the normal derivative, or Neumann data). For simple shapes like a circle, this operator has a beautiful analytical form in the Fourier domain, where it simply multiplies each frequency mode by a factor proportional to its frequency. A neural operator can learn this mapping from data, effectively discovering the analytical solution on its own and generalizing across domains of different sizes.
While the idea of replacing an entire simulation is tempting, some of the most powerful applications arise from a more nuanced approach: hybrid modeling. Here, neural operators work in concert with traditional methods, each playing to its strengths.
One clever strategy is domain decomposition. Imagine simulating a fluid that is mostly smooth but contains a sharp shockwave, like in the Burgers' equation. A neural operator is excellent at modeling the smooth, well-behaved regions quickly and efficiently. The shock, however, with its sharp discontinuity, is better handled by a precise, classic numerical solver designed for such phenomena. We can thus partition the domain, letting the operator handle the "easy" part and the classic solver handle the "hard" part. The key is to ensure they communicate properly at the interface, for instance, by minimizing any mismatch in the physical flux between the two domains.
Another, immensely powerful, form of hybrid modeling is to use operators to augment existing physical models. For decades, engineers have relied on the Reynolds-Averaged Navier-Stokes (RANS) equations to model turbulent flows. These models are fast but are known to be inaccurate in many situations because they rely on simplified assumptions about turbulence. Rather than throwing these models away, we can train a neural operator to learn the correction term—the discrepancy between the RANS model and the true physics. The operator takes as input local features of the flow (such as invariants of the velocity gradient tensor) and outputs a correction field that, when added to the RANS equations, yields a much more accurate prediction of the turbulent stresses. This approach preserves the well-established structure of the original solver while patching its deficiencies with a data-driven component, a perfect marriage of physical insight and machine learning.
Neural operators can go beyond learning the solutions to equations; they can learn the fundamental physical laws themselves, some of which are far more complex than a simple input-output map.
Consider the behavior of materials. The stress in a simple elastic material depends only on its current strain. But for more complex materials like polymers or biological tissues—a class of materials known as viscoelastic—the current stress depends on the entire history of deformation it has experienced. The relationship is not a simple function but a history-dependent functional. This is a natural fit for an operator learning framework. We can train a neural operator to learn the mapping from a strain history function for to the stress vector at the present time. Both Fourier Neural Operators and Deep Operator Networks have shown promise in capturing this "fading memory" characteristic of real materials, opening new avenues for data-driven constitutive modeling in solid mechanics.
At an even more fundamental level, we can use operators to learn core components of our most foundational physical theories. In cosmology, the evolution of the cosmic microwave background photons is described by the Boltzmann equation. This equation contains a complex collision term that accounts for Thomson scattering between photons and electrons. This collision operator itself can be learned by a neural operator. By training on high-fidelity calculations, we can construct a surrogate that is not only fast but can be explicitly built to obey the fundamental symmetries and conservation laws of the underlying physics, such as photon number conservation and parity invariance. Here, we are not just accelerating a simulation; we are creating a fast, physically-constrained replica of a piece of fundamental physics.
Perhaps the most profound application of neural operators is when we turn them inward, teaching them not just to emulate physics, but to emulate and accelerate the very computational methods we use to study physics. Many of the most challenging scientific tasks are inverse problems, such as data assimilation in weather forecasting or finding an optimal control strategy for a fusion reactor. These are often formulated as large-scale optimization problems that require running a forward model and its adjoint (its derivative) thousands or millions of times.
If the forward model is an expensive PDE solver, this process is prohibitively slow. By replacing the forward model with a trained neural operator, we can accelerate the entire optimization loop by orders of magnitude. This has profound implications for tasks like 4D-Var data assimilation, where we seek the optimal initial state of a system (e.g., the atmosphere) that best explains a sequence of observations over time. It is also transformative for PDE-constrained optimal control, where finding the best way to steer a system to a desired state becomes computationally tractable.
We can push this idea even further. Instead of learning the solution operator, what if we could learn a crucial part of the solver algorithm itself? Many implicit numerical methods for solving stiff PDEs, like those in combustion or phase-field modeling, rely on iteratively solving a nonlinear system at each time step using Newton's method. This involves computing a "Newton correction" by solving a large linear system. This step can be a major bottleneck. In a stunning twist, we can train a neural operator to directly learn the map from the current state residual to the required Newton correction, effectively creating a learned, inexact Newton solver that can be much faster than the exact one while maintaining stability.
In a similar vein, many large-scale linear systems are solved iteratively, and their convergence speed is dictated by the system's condition number. We improve this by using a preconditioner, an operator that "massages" the system to make it easier to solve. The ideal preconditioner is often related to the physics of the problem but can be difficult to construct or apply. A neural operator can be trained to learn this ideal preconditioner, for example, by approximating a fractional power of a covariance operator, which can dramatically accelerate the solution of variational data assimilation problems. In these examples, the operator is not just a scientist; it is learning to be a numerical analyst.
The final step in this journey is to embrace uncertainty. A truly scientific prediction is not just a single number but an answer accompanied by an estimate of its uncertainty. Probabilistic neural operators do just this: instead of predicting a single output function, they predict a full probability distribution over the space of possible output functions, typically a Gaussian distribution defined by a mean function and a covariance operator.
This capability is revolutionary for scientific discovery. When we use such an operator in an inverse problem to estimate an unknown physical parameter, , the uncertainty in the operator's prediction () contributes to the final uncertainty of our estimate. This is captured by a beautiful mathematical object called the Fisher information, . For a Gaussian likelihood, its formula involves two terms: one related to how the mean prediction changes with , and one related to how the covariance prediction changes with :
The inverse of the Fisher information provides the Cramér-Rao lower bound, a fundamental limit on the best possible precision with which we can measure . Intuitively, higher uncertainty in our surrogate model (a larger or more variable ) leads to lower Fisher information and thus a poorer ability to constrain the physical parameter. By learning to predict their own uncertainty, these operators allow us to perform robust, uncertainty-aware science, moving from simple prediction to genuine scientific inference.
From accelerating simulations to augmenting existing models, from learning historical dependencies to discovering the building blocks of numerical algorithms, and finally to embracing the probabilistic nature of knowledge, neural operators are redefining the boundaries of computational science. They are a testament to the remarkable synergy between mathematics, physics, and computer science—a new language for describing the operational laws of the universe.