Local and Global Error in Numerical Methods

From forecasting the weather to designing an aircraft, numerical simulations are indispensable tools that solve the complex differential equations governing our world. These methods work by taking a series of small, discrete steps to approximate a continuous solution. However, each step is an approximation, introducing a small error. The central challenge lies in understanding how these tiny, individual errors accumulate over the course of a long simulation and whether they compromise the final result. Without a firm grasp of these errors, we cannot fully trust the predictions of our most powerful computational models.

This article demystifies the concepts of error in numerical analysis. It addresses the critical distinction between the error made in a single step and the total error accumulated over the entire journey. By navigating through the core principles and their real-world consequences, you will gain a clear understanding of what makes a numerical simulation accurate and reliable. The first chapter, "Principles and Mechanisms," will break down the fundamental concepts of local and global error, explaining how they relate to one another and the crucial roles that stability and consistency play. Following this, the "Applications and Interdisciplinary Connections" chapter will explore how these theoretical ideas manifest in practice, from verifying scientific code and simulating physical phenomena to understanding the limits of prediction in chaotic systems.

Imagine you are on a long journey, trying to follow a winding path drawn on a map. You can't see the whole path at once; you can only see the direction you're supposed to go right now. So, you take a step, check your map again, and take another. This is precisely the challenge of solving a differential equation numerically. The equation is your map, telling you the direction of your path at every point. A numerical method is your strategy for taking steps. But, of course, no step is perfect. And the small errors from each step can lead you far astray over the course of a long journey. Understanding these errors is not just an academic exercise; it's the key to trusting the predictions of computer simulations, from the weather forecast to the design of a new aircraft.

When we talk about error in numerical methods, we must be careful to distinguish between two fundamentally different kinds.

First, there's the error you make in a single step. Let’s say you are standing on the true path. Your map (the differential equation) tells you to head in a specific direction. Your stepping strategy (the numerical method) tells you to take a step of a certain length, say $h$, in approximately that direction. Because your strategy is an approximation, the place you land after one step will not be exactly on the true path. This discrepancy—the difference between where one perfect step would land you and where the true path actually goes—is called the ​​local truncation error​​. It's "local" because it's confined to a single step, and it's a "truncation" error because it usually arises from truncating an infinite Taylor series to create a finite, computable recipe. Critically, to define the local error, we imagine a perfect scenario: we start the step at the exact right spot on the true path.

But we don't get to start fresh at every step. The error from the first step means you begin the second step slightly off course. The error from the second step adds to this, and so on. The ​​global error​​, then, is the total accumulated deviation. It’s the difference between where you are after many steps and where you should be on the true path. It's the error of the entire journey, not just a single step.

So, how does the journey's error relate to the error of each step? A simple, optimistic guess would be that the global error is just the sum of all the local errors you've made along the way. While this isn't the whole truth, it gives us a crucial first insight.

Suppose you are simulating a satellite's orbit for a total time $T$, using a fixed step size $h$. The total number of steps you'll take is $N = T/h$. Now, let's say you're using a very sophisticated numerical method, one where the local truncation error is proportional to the fifth power of the step size, or $O(h^5)$. This means each individual step is incredibly accurate. If you halve your step size, the error of a single step drops by a factor of $2^5 = 32$!

But what about the global error at the end of the orbit? If we naively add up the local errors, the total error would be roughly the number of steps multiplied by the average local error:

This simple calculation reveals a general and profoundly important rule of thumb: for a stable numerical method of order $p$, its local truncation error is of order $O(h^{p+1})$, but its global error is of order $O(h^p)$. The global error's dependence on the step size is one order worse than the local error's. This is the price of accumulation: the accuracy of the whole journey is degraded by the sheer number of steps taken. Even with a high-quality method, halving the step size reduces the final error by a factor of $2^4 = 16$, not 32. This fundamental relationship holds true for a wide variety of methods, from simple one-step schemes to more complex multistep algorithms.

The "simple sum" model of error accumulation rests on a dangerous assumption: that an error, once made, just sits there passively as new errors pile on top of it. In reality, the system we are modeling—the "terrain" of our map—can interact with the error. It can either guide us back toward the correct path or shove us even further away. This property is called ​​stability​​.

Consider the seemingly innocuous equation $y'(t) = -100(y - \cos(t))$. The term $-100y$ acts like a very strong spring, pulling the solution $y(t)$ powerfully towards the curve $\cos(t)$. If our numerical solution strays, the system dynamics should correct it. But what happens if we use a simple method like the Forward Euler method with a step size that's too large, say $h=0.03$? The local truncation error at each step is tiny, on the order of $h^2 \approx 0.0009$. We might think we're safe. We're not.

The problem is that our numerical method's approximation of the "strong spring" is poor. Instead of pulling the error back, it overshoots so violently that it amplifies the error at every step. In this case, each step multiplies the existing global error by a factor of $|1 - 100 \times 0.03| = |-2| = 2$. A small initial error is doubled at every step, leading to an exponential explosion and a numerical result that is utter nonsense. This happens because the chosen step size violates the method's ​​stability condition​​, which for this problem requires $h \le 0.02$. In such a ​​stiff​​ system, stability, not local accuracy, is the tyrannical ruler governing our choice of step size.

The opposite can also happen. Imagine modeling a system with inherent exponential growth, like $y'(t) = \lambda y(t)$ for some positive $\lambda$. Here, the true path itself is unstable; any small deviation from it is naturally amplified over time. A numerical method with a constant source of local error, let's call it $\epsilon$ per unit time, will produce a global error that also grows exponentially. The final global error at time $T$ isn't just proportional to $\epsilon$; it's amplified by a factor related to $\exp(\lambda T)$. This is a sobering lesson for anyone performing long-term simulations: even if your local error control is perfect, the inherent nature of the system you're modeling can cause the global error to become unacceptably large.

We've seen that small local errors can accumulate and even be amplified into large global errors. But what if the local error isn't even small to begin with? What if, no matter how tiny you make your step size $h$, your method stubbornly insists on making a finite error at every step?

This brings us to the most fundamental requirement of all: ​​consistency​​. A numerical method is consistent if its local truncation error vanishes as the step size approaches zero. In our walking analogy, this means that as your steps get smaller and smaller, the direction of your step should become a better and better match for the direction given on the map.

If a method is inconsistent—if its local error approaches some non-zero constant—it is fundamentally flawed. Even if the method is perfectly stable, it will not converge to the correct solution. Instead, it will converge to the solution of a different differential equation. It's like using a miscalibrated compass that always points one degree east of true north. No matter how carefully you walk, you will end up in the wrong city. The famous ​​Lax Equivalence Theorem​​ puts it bluntly: for a large class of problems, a method converges to the correct solution if and only if it is both stable and consistent. There are no trade-offs. Consistency is non-negotiable.

This journey into the world of numerical errors reveals a landscape far more rich and subtle than one might first imagine. The accuracy of a simulation is a delicate interplay between the local precision of the algorithm, the cumulative nature of a long journey, and the inherent stability of the system being modeled. And beneath it all lies the bedrock principle of consistency: to have any hope of reaching your destination, you must, at the very least, be pointed in the right direction.

Having explored the mathematical machinery of local and global errors, we might be tempted to view them as a mere technical nuisance, a tax on the path to computational truth. But this is a narrow view. To a physicist, an engineer, or a scientist, understanding the nature of these errors is not just about cleaning up calculations; it is about understanding the limits and possibilities of simulation itself. The dialogue between our idealized models and our finite computations is where some of the most profound insights are found. Like a navigator on a vast ocean, we don't just curse the unpredictable currents and winds; we learn to read them, to account for them, and sometimes, even to harness them to reach our destination more cleverly.

The first step in mastering error is to understand its behavior. A beautifully simple, almost universal rule governs most numerical methods: if the error made in a single step (the local truncation error, or LTE) is proportional to the step size $h$ raised to some power, $h^{p+1}$, then the total accumulated error after integrating over a fixed interval (the global truncation error, or GTE) will be proportional to $h^p$. This happens because to cross a fixed interval, we must take a number of steps proportional to $1/h$. The global error is, roughly, the sum of all local errors, so its order is one power of $h$ lower than the local error.

Consider the simplest of all integrators, the Forward Euler method. Its local error is of order $O(h^2)$, meaning each step is quite accurate for small $h$. However, the accumulation of these small inaccuracies results in a global error of order $O(h)$. This means if you want to make your final answer ten times more accurate, you must take ten times as many steps—a costly trade-off. For a higher-order method, the relationship is even more favorable, but the principle remains: the global error's dependence on step size is the ultimate measure of a method's efficiency.

This very principle becomes a powerful diagnostic tool. Suppose you've just coded a new, complex numerical solver. How do you know it's correct? You can perform a convergence study. By solving a problem with a known answer for a sequence of decreasing step sizes and plotting the logarithm of the GTE against the logarithm of the step size, you should see a straight line. The slope of this line is the order of your method, $p$. If the theory promised a fourth-order method ($p=4$) but your plot reveals a slope of 2, you have a bug. This empirical verification of theoretical convergence rates is a cornerstone of code validation in computational science, a way to build trust in the digital instruments we use to probe the world.

Here, our perspective shifts. If the error is so predictable, can we do more than just observe it? Can we exploit it? The answer is a resounding yes. One of the most elegant ideas in numerical analysis is Richardson Extrapolation. Suppose you perform a calculation with a step size $h$ and then repeat it with $h/2$. You now have two different, imperfect answers. But because you know how the error depends on $h$, you can combine these two "wrong" answers with a clever bit of algebra to cancel out the leading error term, producing a new answer that is far more accurate than either of the originals. It's a bit like having two flawed maps of a coastline, but by understanding the specific distortions in each, you can synthesize a much more accurate chart.

This way of thinking also provides a solution to a profoundly practical question: how do we estimate the error of a simulation when we don't have an exact analytical solution, as is almost always the case in real research? The same strategy applies. We can run a "coarse" simulation with a lenient error tolerance and then run a "fine" one with a much stricter tolerance. Since the fine solution is presumed to be much closer to the unknowable true answer, the difference between the coarse and fine solutions gives us a reliable estimate of the global error in our coarse result. This technique is used ubiquitously in engineering and science to provide error bars for computational results, turning the abstract notion of GTE into a concrete, quantifiable measure of confidence.

So far, we have treated error as a quantitative issue. But in some situations, it becomes a qualitative one, leading to results that are not just inaccurate but physically nonsensical. This often happens in systems with processes occurring on vastly different time scales—so-called "stiff" systems, which are common in fields like chemical kinetics and circuit simulation.

If one applies a simple method like Forward Euler to a stiff problem, a strange paradox can emerge. The local truncation error at each step might be deceptively small, suggesting the simulation is proceeding accurately. Yet, the global solution can explode into meaningless, wild oscillations. The problem is not one of accuracy but of stability. The numerical method, if the step size is not chosen to be small enough to resolve the fastest time scale, can amplify tiny errors at each step, leading to a catastrophic accumulation.

This failure can manifest as a direct violation of fundamental physical laws. Consider simulating the diffusion of heat in a rod. The second law of thermodynamics, in the form of the maximum principle for the heat equation, dictates that a point within the rod cannot become hotter than its initial maximum temperature (assuming no external heat sources). However, a numerically unstable simulation can do just that, creating artificial "hot spots" that appear out of nowhere. Here, the global truncation error is no longer just a number; it represents a breakdown of physical reality within the simulation. Understanding the interplay between GTE and stability is thus essential to ensure our simulations respect the very laws they are meant to describe.

In any complex model, quantities are interconnected. The error in one part of a calculation does not live in isolation; it ripples through the system, affecting other derived quantities. A striking example comes from the world of computational finance. The famous Black-Scholes equation, an ODE in a particular formulation, is solved to determine the theoretical price of a financial option. The GTE of the numerical solver gives the error in this price.

However, for a trader or a risk manager, the price itself is only half the story. They are equally, if not more, interested in the "Greeks"—sensitivities like Delta and Gamma, which are the first and second derivatives of the option price with respect to the underlying asset's price. These quantities are calculated from the numerical price solution. Consequently, any error in the price propagates directly into the calculated Greeks. An inaccurate Delta or Gamma can lead to misjudged risk and significant financial loss. Fortunately, the structure of error propagation is often well-behaved; for many methods, an error of order $O(h^p)$ in the price leads to an error of the same order in the Greeks, allowing for a systematic analysis of the model's reliability.

We arrive at the final, and perhaps most profound, intersection: the role of error in chaotic systems. A hallmark of chaos is extreme sensitivity to initial conditions—the "butterfly effect." Any two nearby starting points diverge exponentially fast. This presents a terrifying prospect for numerical simulation. Any tiny, unavoidable local error from floating-point arithmetic acts as a perturbation that places our numerical trajectory on a different path. This new path will diverge exponentially from the true one that started at the exact same point. The GTE, in this context, grows exponentially, suggesting that any long-term simulation of a chaotic system is doomed to be completely wrong.

It seems like a fatal blow. But here, a beautiful piece of mathematics, the shadowing lemma, comes to our rescue. It reveals an astonishing truth: while our computed trajectory (a "pseudo-orbit") is indeed diverging from the true orbit with the same initial condition, it is not meaningless garbage. Under general conditions, there exists a different true orbit, starting from a slightly different initial condition, that remains uniformly close to our entire computed trajectory for all time. Our simulation is "shadowing" a genuine path of the system.

This insight is liberating. It means that even though we cannot trust our simulation to predict the exact state of a chaotic system far into the future (a feat that is impossible in principle anyway), we can trust its statistical properties. The overall geometric structure of the system's attractor, the frequency of certain behaviors, the average properties—all of these are faithfully captured. The shadowing lemma provides the mathematical foundation for why numerical weather prediction, simulations of turbulence, and models of ecological systems can provide invaluable statistical insights, even when they cannot pinpoint an exact future. It shows that in the intricate dance between order and chaos, our imperfect computations can still reveal a deep and meaningful truth.