Deep Operator Network (DeepONet)

Scientific discovery has long been about finding relationships, from simple rules connecting numbers to the complex laws governing our universe. While standard neural networks excel at learning functions that map numbers to numbers, many fundamental laws of nature are not functions but operators—rules that map entire functions to other functions. For example, the laws of fluid dynamics take the shape of an aircraft wing (a function) and produce the pressure distribution across its surface (another function). Learning these operators directly is a major goal in scientific computing, promising to create surrogate models that can bypass prohibitively expensive simulations.

However, a significant hurdle has been the "tyranny of the grid," where models trained on a specific data discretization fail when applied to a different one. This has limited the ability of traditional deep learning to learn the true, underlying continuous physical laws. The Deep Operator Network, or DeepONet, was developed to overcome this fundamental challenge, providing an elegant framework for learning operators in a way that is independent of how we represent the data.

This article explores the world of DeepONet. First, in the "Principles and Mechanisms" chapter, we will deconstruct its innovative architecture, exploring how its Branch and Trunk networks work together to process and generate functions. We will also touch upon the mathematical guarantees that ensure its power. Following that, the "Applications and Interdisciplinary Connections" chapter will showcase how this powerful idea is being applied to solve formidable challenges across a vast landscape of science and engineering, from structural mechanics to climate modeling.

In science, we often begin by learning simple rules, relationships between numbers. If you double the force on an object, you double its acceleration. If you increase the temperature of a gas, its pressure rises. A standard neural network is a master at learning such rules, no matter how complex. It might learn to predict a single number, like tomorrow's high temperature, from a list of other numbers, like today's temperature, humidity, and wind speed. It learns a function: a mapping from a handful of numbers to another number.

But nature’s laws are often grander. They don't just relate numbers; they relate entire functions. Consider the flow of air over a wing. The law of fluid dynamics doesn't just relate the pressure at one point to the velocity at another. It provides a rule, an ​​operator​​, that takes the entire shape of the wing (a function describing its boundary) and the entire incoming velocity field (a function of space) and produces the entire pressure field over the surface of the wing (another function of space).

This is the world of operators: machines that take whole functions as inputs and spit out other functions as outputs. Learning these operators directly is the holy grail for creating fast and accurate surrogates for complex physical simulations. If we could learn the operator for weather prediction, we could feed it today's complete weather map and get tomorrow's map in an instant, bypassing the hours of computation on a supercomputer.

So, how do we teach a computer to handle a function? A function, like a curve drawn on a piece of paper, is made of an infinite number of points. A computer, being a finite machine, gets nervous around infinity.

The most straightforward idea is to cheat. We lay a grid over the function and just record its value at a finite number of points. Our smooth, continuous velocity field becomes a long list of numbers—the velocities at each grid point. Suddenly, our problem of learning an operator becomes a familiar one: learning a map from a big vector in $\mathbb{R}^n$ to another big vector in $\mathbb{R}^m$.

But this is a devil's bargain. The network we train is now fundamentally tied to the specific grid we chose. If we train on a coarse $100 \times 100$ grid and then want a high-fidelity prediction on a $1000 \times 1000$ grid, our network is useless. It was never taught the continuous physical law, only a pixelated approximation. It has no idea what to do with the new grid points. Furthermore, as the grid gets finer, these naive models can become unstable, with their predictions oscillating wildly or blowing up. This dependence on the discretization is a kind of digital myopia, and breaking free from it is the central challenge of operator learning. We need a way to build a model that is ​​discretization-invariant​​: a single, learned model that operates on the underlying continuous function, independent of how we choose to represent it on a grid.

How can we possibly build a machine that ingests an entire, infinite-dimensional function? The Deep Operator Network, or ​​DeepONet​​, offers an answer of profound elegance and simplicity. Instead of trying to "see" the entire function at once, it cleverly splits the problem into two smaller, more manageable questions. This is achieved with a two-part architecture: a ​​Branch Network​​ and a ​​Trunk Network​​.

The Branch Net: What Function Is This?

The Branch network's job is to identify the input function. But it doesn't need to see every point. Think about how a doctor diagnoses an illness. They don't need to scan every cell in your body. A few key measurements—temperature, blood pressure, a blood sample—are often enough to form a diagnosis.

Similarly, the Branch net takes a few measurements of the input function $u$ at a fixed set of "sensor" locations: $[u(x_1), u(x_2), \dots, u(x_m)]$. This small vector of sensor values acts as a "fingerprint" for the function. The Branch net, which is just a standard neural network, processes this fingerprint and outputs a set of coefficients. These coefficients are the network's internal summary of the input function it has just seen.

The Trunk Net: Where Are We?

The Trunk network answers the second question: where do we want to evaluate the output? Its input is simply a coordinate, $y$, in the output domain. For this coordinate, the Trunk net generates a set of pre-defined "basis functions." You can think of these as a collection of fundamental shapes or patterns, like the basic LEGO bricks of functions. The Trunk net's job is to tell us the value of each of these basis bricks at the specific location $y$.

Putting It All Together

The final step is breathtakingly simple. The DeepONet predicts the value of the output function at location $y$ by taking a weighted sum of the basis functions from the Trunk net. And what are the weights? They are precisely the coefficients computed by the Branch net.

This structure is a thing of beauty. It separates the "what" from the "where." The Branch net understands the input function, and the Trunk net understands the space where the output function lives. The final dot product marries the two to produce a prediction. This elegant design is the key to the DeepONet's power.

This design isn't just clever; it's backed by profound mathematical guarantees. The ​​Universal Approximation Theorem for Operators​​ states that, under reasonable conditions, a DeepONet can approximate any continuous operator to any desired accuracy. But why should this be true? Let's explore the two key ingredients.

First, the Branch net's sensors must be able to "see" the important differences between input functions. Imagine we want to learn an operator that depends on the slope of an input line, $u(x) = a_1 + a_2 x$, but our only sensor is at $x=0$. The sensor only ever sees $u(0)=a_1$. It is completely blind to the slope $a_2$! The network would receive the exact same fingerprint for the functions $u(x) = 1+x$ and $u(x) = 1-x$, and would be forced to produce the same output for both, even if the true operator should treat them differently. This tells us the sensors must be placed wisely, in a way that allows the network to distinguish between any two distinct input functions it is expected to handle.

Second, the Trunk net must be able to "build" any required output function from its basis shapes. Imagine if the Trunk net could only produce constant basis functions. Then any weighted sum would also be a constant. The network would be utterly incapable of approximating an operator whose output is, say, a parabola. The trunk's basis must be rich enough to express the full variety of shapes present in the true operator's output.

When these two conditions are met—the Branch net can see and the Trunk net can express—the magic happens. The architecture is guaranteed to be a universal approximator. And notice, the design is inherently ​​discretization-invariant​​. The sensors are at fixed physical locations, and the trunk can be queried at any continuous coordinate. We can train the model using data from one grid and apply it flawlessly to any other, because the model has learned the underlying continuous operator, not a pixelated artifact.

Let's ground this with a concrete example: predicting the steady-state temperature $T(x,y)$ inside a 2D plate, given the heat flux $q$ we are applying at its boundary. The operator is $\mathcal{G}: q \mapsto T$.

To build a DeepONet surrogate, our ​​Branch net​​ would take as input a vector of flux values sampled at, say, 16 fixed sensor locations around the plate's perimeter. Our ​​Trunk net​​ would take as input a coordinate pair $(x,y)$ from inside the plate. During training, we would feed the network a set of sensor readings for a known flux $q_i$, and a query point $(x_j, y_j)$. The network would predict a temperature $\widehat{T}_i(x_j,y_j)$. We would then compare this to the true temperature $T_i(x_j,y_j)$ (perhaps obtained from a slow but accurate traditional solver) and calculate the error. Using the familiar chain rule of calculus in a process called backpropagation, we can compute how to adjust every weight in the Branch and Trunk networks to nudge the prediction closer to the truth. After seeing thousands of such (flux, temperature) pairs, the network learns the intricate mapping from boundary conditions to the internal temperature field.

The true power of this architecture lies in its flexibility. What if not just the boundary flux $q$ changes, but also the material's thermal conductivity $\kappa$, or even the very shape of the domain $\Omega$? The DeepONet framework handles this with grace. We simply expand the notion of the "problem instance." The Branch net's job is to digest everything that defines the specific problem we want to solve. Its inputs would now be a representation of the triplet $(q, \kappa, \Omega)$. The Trunk net's job is still to build a basis in space, so its inputs would be the coordinate $y$ plus any local geometric information, like the distance to the nearest boundary. This principled separation of concerns allows DeepONet to learn across vast families of parametric problems.

But what if we don't have much data? We can give our network a head start by teaching it the laws of physics directly. We know the solution must satisfy the heat equation, $\nabla \cdot (\kappa \nabla T) = 0$. We can add a term to our training loss that penalizes the network if its output violates this equation at random points inside the domain. This is the core idea of ​​Physics-Informed Neural Networks (PINNs)​​. This isn't just an engineering hack; it has a deep justification in Bayesian statistics. The data error term in the loss corresponds to the likelihood of our observations, while the physics-penalty term corresponds to a strong prior belief that the laws of nature hold true. The weighting between these two terms is not arbitrary; it can be rigorously derived from the noise level in our measurements and our confidence in the physical model.

By combining a simple, powerful architecture with the eternal laws of physics, DeepONet provides a remarkable tool for decoding the complex operators that govern our world. It represents a beautiful synthesis of functional analysis, deep learning, and physical principles—a testament to the unifying power of mathematical ideas.

We have spent some time admiring the theoretical engine of operator learning, tinkering with the branch and trunk networks, and appreciating the mathematical elegance of their design. But an engine, no matter how beautifully constructed, is only truly understood when we see what it can drive. What happens when this abstract machinery meets the wonderfully messy and intricate reality of the physical world? Where does this new way of thinking take us?

Let us embark on a journey across the landscape of modern science and engineering. We will see that this single idea—learning the relationship between entire functions—is not some isolated curiosity. Rather, it is a kind of universal key, unlocking doors in fields that, at first glance, seem to have little in common. It is in these applications that the true power and beauty of operator learning are revealed.

Perhaps the most direct and intuitive application of a DeepONet is as a universal solver for partial differential equations (PDEs). Nearly all of the fundamental laws of physics, from the flow of heat to the vibrations of a guitar string to the bending of a steel beam, are described by PDEs. A PDE defines a relationship, but solving it for a specific scenario—a particular initial temperature, a unique pluck of a string—requires immense computational effort. Each new scenario demands a new, costly simulation.

What if we could learn the solution operator itself? What if we could train a network to understand the very essence of "solving," mapping any valid input condition to its corresponding solution? This is precisely what a DeepONet can do.

Imagine learning the 1D heat equation, which describes how temperature spreads along a rod. The "input function" is the initial temperature distribution along the rod, and the "output function" is the temperature distribution at some later time. A DeepONet can be trained on examples of this process, learning the characteristic smoothing and decay of heat. Once trained, it can instantly predict the final temperature profile for any new initial heat distribution it has never seen before, effectively acting as an infinitely fast PDE solver.

This idea scales from simple textbook examples to formidable engineering challenges. Consider the complex problem of determining how a mechanical part, like an airplane wing or a bridge support, deforms under various loads. Here, the input function is the force field applied to the structure, and the output function is the displacement field describing how the structure bends and twists. A traditional Finite Element Method (FEM) simulation can take hours or days for a single load case. A trained DeepONet, however, can provide an answer in milliseconds.

What is particularly beautiful here is how our physical intuition can guide the network's design. For a structure with holes or complex boundaries, the way it deforms is strongly influenced by its geometry. We can encode this knowledge directly into the network. For instance, the trunk network, which processes the spatial coordinates, can be fed not just the raw coordinates $(x,y,z)$, but also extra information like the distance from any point to the nearest boundary. By making the network "aware" of the object's shape, we help it learn the physics of stress concentrations and boundary layers much more efficiently.

While learning the full solution operator is powerful, physicists often seek a deeper, more fundamental understanding. For many linear systems, the entire, complex behavior is governed by its response to the simplest possible disturbance: a single, sharp "poke" at one point. This response is called the Green's function.

Think of it like dropping a single pebble into a still pond. The Green's function, $\mathcal{G}(x,y)$, describes the ripple you see at location $x$ from a pebble dropped at location $y$. The profound insight of the superposition principle is that if you know this fundamental ripple pattern, you can calculate the effect of any disturbance—a handful of pebbles, or a continuous shower of rain—by simply adding up the corresponding ripples. Mathematically, the solution $u(x)$ for a general forcing $f(y)$ is just an integral: $u(x) = \int \mathcal{G}(x,y) f(y) dy$.

A DeepONet can be trained to learn this very essence of the system. By feeding its branch network a series of sharp, localized input functions (approximations of a "poke," or Dirac delta function) centered at different locations $y_k$, and training it on the resulting solutions, the network learns to approximate the Green's function itself. The branch network learns to encode the location of the poke, $y$, while the trunk network learns the spatial pattern of the response, $x$.

This concept has remarkable interdisciplinary reach. In medical imaging or astronomy, the "blur" introduced by a microscope or telescope is described by a point spread function (PSF), which is nothing more than the imaging system's Green's function. Often, this blur changes depending on where you are in the image. This is a complex, space-variant inverse problem. By learning this spatially-varying kernel $k(x,y)$ with a DeepONet, we can build sophisticated algorithms that "de-blur" the image, revealing the true underlying structure. The learned operator becomes a crucial component in modern data assimilation and image reconstruction, where its differentiability allows it to be integrated seamlessly into variational frameworks that require adjoints for gradient computations.

So far, our operators have mostly mapped functions over space. But what about time? Many systems have memory. The state of the system now depends not just on the present input, but on the entire history of inputs.

This is the very soul of materials science. The stress in a piece of dough depends not on its current shape, but on the entire history of its kneading, stretching, and resting. This behavior, known as viscoelasticity, is governed by an operator that maps a function of time (the strain history) to a single value (the current stress). A DeepONet is perfectly suited for this task. Its branch network can ingest a discretized representation of the strain history, $\boldsymbol{\varepsilon}(s)$ for $s \in [0,t]$, and its trunk network can be a simple query for the current time $t$, allowing it to predict the stress $\boldsymbol{\sigma}(t)$.

This principle finds concrete application in fields like geotechnical engineering. When constructing a building, engineers must predict how the clay soil underneath will settle over decades. This long-term creep is a history-dependent process. The final settlement depends on the entire loading history from construction. We can design a surrogate model, inspired by the DeepONet structure, to learn this operator. Here again, we can embed physical knowledge. By constructing features for the branch network from physically-motivated models like the Prony series (which represents material memory with decaying exponentials) and by enforcing physical constraints like monotonicity (a heavier load cannot cause less settlement), we create a fast, reliable, and physically plausible predictive tool. This is a beautiful marriage of data-driven learning and classical engineering theory.

A common criticism of machine learning is its hunger for data. What if we don't have enormous datasets from simulations or experiments? In physics, we often have something just as valuable: the governing equations. The Physics-Informed Neural Network (PINN) was a revolution, showing that a network could be trained not on data, but by demanding that its output satisfy a PDE. However, a PINN learns the solution to only one specific problem instance.

The next leap forward is to combine the data-free training of PINNs with the generalizing power of DeepONets. This creates the "Physics-Informed DeepONet". Imagine a complex, coupled problem like the heating of a metal object as it is plastically deformed. The behavior depends on a whole field of material parameters: stiffness, thermal conductivity, yield stress, and so on. A standard PINN would need to be retrained from scratch for every new material.

A Physics-Informed DeepONet, however, learns the entire operator that maps the parameter field to the solution field. The branch network takes in the material properties, while the trunk processes the space-time coordinates. The network is trained by minimizing a loss function composed of the residuals of the governing PDEs (momentum balance, heat equation, plasticity laws). The network never sees a single "correct" solution. Instead, it explores the space of functions until it finds an operator whose outputs universally obey the laws of physics for any given material. It is like learning the rules of chess not by studying millions of recorded games, but by simply being given the rulebook and discovering for itself all the valid strategies that emerge.

Some of the largest-scale computations in science are dedicated to forecasting—predicting the weather, the climate, or the path of ocean currents. These systems are chaotic, meaning tiny errors in the initial state grow exponentially, making long-term prediction impossible. Modern forecasting relies on a process called data assimilation, which continually corrects the model's state with incoming observations.

A powerhouse technique is 4D-Var, which can be thought of as a cosmic-scale optimization. It seeks the perfect initial state of the atmosphere at the beginning of the week that, when evolved forward by the physics model, best matches all the satellite and weather station data collected over the entire week. This requires running the massive weather model (the flow map $\Phi$) and its adjoint forwards and backwards many times. It is fantastically expensive.

Here, a DeepONet can serve as an ultra-fast surrogate, $\widehat{\Phi}$. Once trained to mimic the expensive physical model, it can be dropped into the 4D-Var optimization loop, potentially slashing the computational cost by orders of magnitude while preserving the integrity of the variational framework.

Deeper still, operator learning offers a new lens through which to view chaos itself. The theory of Koopman operators tells us that even the most wildly nonlinear chaotic dynamics can be viewed as simple linear evolution, provided we look at them in a different, usually infinite-dimensional, space of "observable" functions. The challenge is finding this magic "Koopman viewpoint". A DeepONet can be architected to do just that, with its trunk network learning the basis of these special observables. By learning an approximate Koopman operator, we can forecast chaotic systems like the Lorenz-96 model with potentially greater stability than by trying to model the nonlinear dynamics directly.

Perhaps the most profound application of operator learning is in modeling what we cannot see. In many complex systems, like the Earth's climate, we can only afford to simulate the large-scale phenomena (global wind patterns, ocean gyres). Yet, we know that small-scale, unresolved processes (individual clouds, tiny ocean eddies) have a crucial collective effect on the large scales. The "closure problem" is one of the grand challenges of computational science: how do we represent this influence of the unseen on the seen?

We can frame this as an operator learning problem. The closure is an operator that maps the state of the resolved, large-scale field to a term representing the net effect of the unresolved small scales. A DeepONet can be trained on data from high-resolution simulations to learn this closure operator, providing a way to build more accurate and physically consistent coarse-grained models of multiscale systems.

Finally, we can turn the entire problem on its head. Instead of learning the forward map from cause to effect, can we learn the inverse map from effect back to cause? For many ill-posed inverse problems, this is the true goal. A remarkable strategy is to train a DeepONet to act as a learned regularized inverse. The network, $\mathcal{R}_{\theta}$, takes in blurry or incomplete data $y$ and directly outputs an estimate of the true state $x$. It is trained by forcing its output, $\hat{x} = \mathcal{R}_{\theta}(y)$, to minimize the very Tikhonov variational objective that defines the classical solution. In this way, the network learns the "art of inverting" directly from the mathematical principle of regularization, amortizing the cost of solving the inverse problem over the entire data distribution.

From the simple diffusion of heat to the fabric of chaos, from the memory of materials to the unseen influence of clouds on our climate, the Deep Operator Network provides a unifying language. It shows us that the relationships governing our universe are not just between numbers, but between entire functions, entire fields, entire histories. By learning these relationships, we are not merely fitting data; we are capturing a piece of the underlying physical law itself.