Wide-Sense Stationary Process

In fields from engineering to physics, we constantly encounter signals that are random yet exhibit a form of statistical regularity over time. A steady background hiss, the random vibrations of a motor, or the fluctuations in a financial market may appear unpredictable moment-to-moment, but their overall character—their average level and internal rhythm—often remains consistent. The concept of a Wide-Sense Stationary (WSS) process provides the essential mathematical framework for understanding and manipulating such signals. It addresses the challenge of analyzing randomness by defining a practical form of "statistical sameness" that is not overly restrictive yet powerful enough for vast applications. This article breaks down the WSS process into its core components. First, we will explore the "Principles and Mechanisms," defining the two simple rules that govern WSS processes and examining the rich information encoded within the autocorrelation function. Following this, the section on "Applications and Interdisciplinary Connections" will demonstrate how these theoretical ideas apply to the real world, exploring what happens when we filter, sample, and measure WSS signals, and introducing the crucial concept of ergodicity that connects abstract theory to tangible data.

Imagine you're listening to the steady hiss of a radio tuned between stations, or watching the waves on a vast, open ocean. If you record a ten-second clip of the hiss today, and another ten-second clip tomorrow, the two clips will be completely different in their fine details. Yet, in a statistical sense, they will feel the same. The average loudness, the range of frequencies present, the "texture" of the sound—these characteristics don't change. This intuitive idea of statistical "sameness" over time is the heart of what we call ​​stationarity​​. It's a profoundly useful concept because it allows us to analyze a small piece of a process and make powerful predictions about its behavior at any other time.

In physics and engineering, we often don't need the strictest form of stationarity. We can relax the conditions a bit and still have a wonderfully powerful tool. This leads us to the idea of ​​Wide-Sense Stationarity (WSS)​​, which is built on two simple, common-sense rules. A random process is WSS if its mean and its autocorrelation meet these conditions. Let's look at them one by one.

​​Rule 1: The mean value must be constant.​​

This is the most straightforward requirement. The average level of the signal cannot be drifting up or down over time. It has to be stable. Suppose you have a sensor whose reading is slowly drifting upwards, perhaps due to heating. We could model this as a signal $X(t) = at + N(t)$, where $N(t)$ represents random noise and $at$ is the deterministic drift. The average, or expected, value of this signal at time $t$ is $E[X(t)] = at$. You can see immediately that this average value changes with time. The process is not stationary. Only if the drift rate $a$ is exactly zero does the process have a chance of being stationary. A process whose statistical properties change with the absolute time is called ​​non-stationary​​.

​​Rule 2: The autocorrelation must depend only on the time lag.​​

This rule is more subtle and more powerful. Let's first think about what ​​autocorrelation​​ means. The autocorrelation function, $R_{XX}(t_1, t_2) = E[X(t_1) X(t_2)]$, measures the statistical relationship between the signal's value at two different points in time, $t_1$ and $t_2$. It asks: "If the signal is high at time $t_1$, is it also likely to be high at time $t_2$?"

For a process to be WSS, this relationship must not depend on when you look, but only on how far apart you look. That is, the correlation between the signal at 1:00 PM and 1:01 PM should be the same as the correlation between the signal at 5:00 PM and 5:01 PM. The time difference, $\tau = t_2 - t_1$, is all that matters. So, for a WSS process, we can write the autocorrelation simply as a function of one variable, the time lag $\tau$: $R_{XX}(\tau)$.

A beautiful illustration of this is what happens when you simply delay a signal. If $X(t)$ is a WSS process, and we create a new, delayed process $Y(t) = X(t - t_0)$, what is the autocorrelation of $Y(t)$? A quick calculation shows that $R_{YY}(\tau) = E[Y(t)Y(t+\tau)] = E[X(t-t_0)X(t+\tau-t_0)]$. If we just shift our time reference, calling $u = t - t_0$, this becomes $E[X(u)X(u+\tau)]$, which is just $R_{XX}(\tau)$. The autocorrelation function is completely unchanged by the time shift. The internal "rhythm" of the process is independent of when it starts.

With these two rules, we can start to build a gallery of characters, some of whom are WSS and some who are not. The variety might surprise you.

• ​​The Random Constant:​​ What's the simplest "random" process you can imagine? How about one that doesn't change at all? Let $X(t) = C$, where $C$ is a random variable chosen once at the very beginning. Perhaps it's the final temperature of a chemical reaction, which has some randomness but is then fixed forever. Is this process WSS? The mean is $E[X(t)] = E[C]$, a constant. The autocorrelation is $R_{XX}(t_1, t_2) = E[X(t_1)X(t_2)] = E[C^2]$, also a constant. Since these values don't depend on time, the process is WSS, provided that the second moment $E[C^2]$ is finite. This might seem trivial, but it's a great sanity check: a process doesn't need to be "wiggling" to be a WSS process.

• ​​The Deceptive Oscillator:​​ Now for a more exciting case. Consider a process that looks like a pure sinusoid: $X_n = A \cos(\omega n) + B \sin(\omega n)$. Here, the time evolution is discrete ($n=0, 1, 2, ...$), and the randomness comes from the amplitudes $A$ and $B$. Let's say $A$ and $B$ are uncorrelated random variables, with a mean of zero and the same variance $\sigma^2$. Any single realization of this process is a perfect sine wave with a specific amplitude and phase. This doesn't look stationary at all! But remember, stationarity is a property of the ensemble—the collection of all possible outcomes. The mean is $E[X_n] = E[A]\cos(\omega n) + E[B]\sin(\omega n) = 0$, which is constant. What about the autocorrelation? After a bit of algebra involving trigonometric identities, we find a remarkable result: $E[X_n X_m] = \sigma^2 \cos(\omega(n-m))$. It depends only on the time lag, $n-m$! So, this process is perfectly WSS. This is a profound lesson: a process can appear highly structured and time-varying in any single instance, yet its underlying statistical "rules" can be completely stationary.

• ​​The Physicist's Noise Model:​​ In many real-world experiments, fluctuations tend to be correlated over short time scales but uncorrelated over long ones. A wonderful model for this is the Gaussian Process, where the voltage fluctuations $V(t)$ have a constant mean $\mu_0$ (a DC offset) and a covariance function like $K(s, t) = \sigma^2 \exp\left(-\frac{(s-t)^2}{\ell^2}\right)$. Since the mean is constant and the covariance only depends on the time difference $s-t$, this process is WSS. The function shape tells us that the correlation between two points drops off smoothly and rapidly as the time separation between them increases, a very common physical behavior.

The autocorrelation function $R_X(\tau)$ is far more than a mathematical check-box for stationarity. It is a rich fingerprint of the process, revealing its deepest physical and statistical properties.

• ​​The Peak at Zero: Average Power:​​ What is the meaning of $R_X(\tau)$ at a lag of zero? By definition, $R_X(0) = E[X(t)X(t+0)] = E[X(t)^2]$. If $X(t)$ represents a voltage across a 1-ohm resistor, then $X(t)^2$ is the instantaneous power. The expected value, $E[X(t)^2]$, is therefore the ​​average power​​ of the signal. So, the value of the autocorrelation function at $\tau=0$ is not just a number; it is the total average power carried by the process.

• ​​The Far Horizon: DC vs. AC Power:​​ What happens as the time lag $\tau$ becomes very large? For most physical processes that don't have a perfectly periodic component, the signal at time $t$ will have forgotten all about its state at time $t+\tau$. They become statistically independent. In that case, the expectation of the product becomes the product of the expectations: $\lim_{\tau \to \infty} R_X(\tau) = E[X(t)X(t+\tau)] \to E[X(t)] E[X(t+\tau)] = \mu_X \cdot \mu_X = \mu_X^2$. The value that the autocorrelation function asymptotes to is the square of the mean! This gives us a wonderful way to decompose the signal's power. For instance, if a sensor's signal has an autocorrelation of $R_V(\tau) = 13 \exp(-\frac{\tau^2}{2\sigma_0^2}) + 36$, we can immediately read off its power components. The total power is $R_V(0) = 13 + 36 = 49$ W. The value at infinity is $36$ W, which must be the ​​DC power​​ ($\mu_V^2$). The remaining part, the part that decays to zero, represents the fluctuations around the mean. Its power contribution is the ​​AC power​​, which is $R_V(0) - \mu_V^2 = 49 - 36 = 13$ W,. The entire story of the power budget is written in the shape of the autocorrelation function!

• ​​Frequencies and White Noise:​​ The story gets even more interesting when we look at a process in the frequency domain. The ​​Wiener-Khinchine theorem​​ tells us that the autocorrelation function $R_X(\tau)$ and the ​​Power Spectral Density (PSD)​​ $S_X(f)$—which describes how the signal's power is distributed across different frequencies—are a Fourier transform pair. This connection is incredibly powerful. Consider the ultimate random process: ​​white noise​​. This is a signal so unpredictable that its value at any instant is completely uncorrelated with its value at any other instant, no matter how close. What would its autocorrelation function look like? It must be zero for any $\tau \neq 0$, and have an infinitely sharp spike at $\tau=0$ to account for the signal's power. The mathematical object with this property is the ​​Dirac delta function​​, $\delta(\tau)$. If the PSD of white noise is a constant, $S_{VV}(f) = N_0/2$, its autocorrelation is precisely $R_{VV}(\tau) = \frac{N_0}{2}\delta(\tau)$. A flat spectrum—where all frequencies are equally present—thus corresponds to a signal that is perfectly uncorrelated in time for any non-zero lag.

We must be careful. Our definition of Wide-Sense Stationarity only looks at the first two moments of the process: the mean and the autocorrelation. What if higher-order statistical properties, like the skewness (asymmetry) or kurtosis ("peakiness") of the signal's probability distribution, change over time?

This brings us to a stronger condition: ​​Strict-Sense Stationarity (SSS)​​. A process is strictly stationary if its entire joint probability distribution is invariant to shifts in time. This means all statistical properties—the mean, variance, skewness, every moment, every possible statistical measure—are constant in time.

Clearly, SSS is a much stronger condition. If a process is SSS and has finite second moments, it must also be WSS. But is the reverse true? Does WSS imply SSS?

In general, the answer is ​​no​​. We can construct a process that is WSS but not SSS. Imagine a discrete-time process where at each step, we draw an independent random number. But we change the rules depending on the time step: at even time steps, we draw from a Laplace distribution (sharp peak, heavy tails), and at odd time steps, we draw from a Gaussian distribution (the classic bell curve). We can cleverly set the parameters so that both distributions have a mean of zero and the exact same variance. This process is WSS because its mean (0) and its autocovariance (a delta function at zero lag) are time-invariant. However, the fundamental shape of the probability distribution flips back and forth at every time step. The process is not strictly stationary.

There is one very important case where this distinction vanishes. For a ​​Gaussian process​​—a process where any collection of samples has a joint Gaussian distribution—WSS does imply SSS. This is because a Gaussian distribution is completely and uniquely defined by its mean and covariance. If those two are time-invariant, the entire statistical structure of the process must be as well. This is one of the reasons Gaussian processes are so central to signal processing and machine learning: their stationarity properties are uniquely simple and elegant.

And so, from a simple, intuitive notion of "sameness," we have journeyed through a landscape of random processes, uncovering deep connections between time, frequency, power, and probability. The concept of stationarity, in its wide-sense form, provides just enough structure to make random worlds predictable, without sacrificing the richness of their behavior.

In the previous chapter, we became acquainted with the wide-sense stationary (WSS) process, a mathematical abstraction of beautiful simplicity and order. We saw that its "personality"—its statistical character—doesn't change over time. Its mean is steadfast, and its autocorrelation depends only on the time lag between two points, not on when we start looking. This is a wonderfully clean picture. But the real world is rarely so pristine. We don’t just observe processes; we interact with them. We filter them, we sample them, we chop them up, and we try to measure them with finite instruments.

So, the natural question to ask is: what happens to our elegant WSS model when it collides with the real world of engineering and measurement? This is where the true power and utility of the concept come to life. We are about to embark on a journey from the abstract plane of theory to the bustling workshop of its applications.

A raw signal, even a well-behaved WSS one, is often not what we ultimately need. It might be contaminated with a constant bias, or perhaps we are only interested in its rapid fluctuations. This is where filtering comes in. A filter is like a sculptor's chisel, chipping away parts of the signal we don't want and shaping what remains.

Imagine you are trying to stabilize a high-precision laser. The position of the laser spot on a sensor jitters randomly, a motion we can model as a WSS process $x(t)$. We aren't just interested in its position, but how fast it's moving, its velocity $v(t) = \frac{dx(t)}{dt}$. This act of differentiation is, in fact, a filter. What does it do to the signal's power spectrum, $S_{xx}(\omega)$? It turns out that the power spectrum of the velocity is $S_{vv}(\omega) = \omega^2 S_{xx}(\omega)$. The multiplication by $\omega^2$ means that high-frequency components are dramatically amplified, while low-frequency drift is suppressed. The filter has sculpted the signal to emphasize its "wiggles" and ignore its slow wanderings.

This same principle applies in the digital world. A sensor monitoring a stable process might have a constant DC offset, a non-zero mean $\mu_x$. To see the quick changes, we can apply a simple "first-difference" filter: $y[n] = x[n] - x[n-1]$. The effect on the mean is immediate and striking: the output mean becomes exactly zero. The filter has perfectly blocked the DC component. Looking at its effect on the power spectrum, we find it multiplies the input spectrum by a factor of $4\sin^2(\omega/2)$. This function is zero at $\omega=0$ (DC) and increases with frequency, once again acting as a high-pass filter.

These examples reveal a profound and beautifully simple rule: when a WSS process passes through any stable Linear Time-Invariant (LTI) filter, the output is also WSS. Its power spectral density is simply the input power spectral density multiplied by the squared magnitude of the filter's frequency response, $|H(\omega)|^2$.

$S_{YY}(\omega) = |H(\omega)|^2 S_{XX}(\omega)$

This relationship is a cornerstone of statistical signal processing. It allows us to design filters to shape the spectrum of random noise in any way we please. But it also works in reverse, turning us into scientific detectives. Suppose we observe a noisy signal $Y(t)$ at the output of a known low-pass filter, and find its spectrum is, say, $S_{YY}(\omega) = P_0 / (1 + (\omega/\omega_c)^2)$. What was the original signal $X(t)$ that went into the filter? Using our magic formula, we can "deconvolve" the filter's effect. In this case, we'd find that the input spectrum $S_{XX}(\omega)$ must have been a constant. The seemingly structured, "colored" noise at the output was born from a completely unstructured, "white" noise at the input. We have inferred the hidden cause from the observed effect.

Nearly all modern analysis of signals happens on computers. This requires us to take a continuous river of information, $X(t)$, and capture it as a discrete sequence of numbers, $x[n]$. This is the act of sampling. What does this do to the statistics of our WSS process?

At first glance, the answer is wonderfully simple. If we sample a WSS process $X(t)$ every $T_s$ seconds, the autocorrelation of the resulting discrete sequence $x[n]$ is simply the T-sampled version of the original autocorrelation function: $R_{xx}[k] = R_X(kT_s)$. Likewise, if we have a sensor moving through a random spatial field, changing its speed is equivalent to time-scaling the resulting temporal signal. If we double the speed, the correlation structure of the measured signal gets compressed by a factor of two in the time-lag domain. Everything seems to scale in a straightforward way.

But this elegant simplicity hides a great peril: ​​aliasing​​. A fundamental theorem of signal processing tells us that sampling in the time domain causes the signal's spectrum to become periodic in the frequency domain. The spectrum of the sampled signal is a sum of infinitely many copies of the original spectrum, shifted by multiples of the sampling frequency. If the original process has frequencies higher than half the sampling frequency (the Nyquist frequency), these shifted copies overlap. The result is a catastrophic and irreversible corruption of the signal. High frequencies from the original signal masquerade as low frequencies in the sampled version.

This leads us to the Nyquist-Shannon sampling theorem, extended to random processes. To be able to perfectly reconstruct a random process from its samples (in the sense of minimizing mean-squared error), its power spectral density must be zero above the Nyquist frequency. If an electronic noise source has a bandwidth of $\omega_0$, we absolutely must sample it at a frequency $f_s$ such that $\omega_0 \le \pi f_s$. This isn't just a guideline; it is a rigid law that forms the bedrock of our digital world, from audio recording to medical imaging.

We saw that LTI filtering preserves wide-sense stationarity. But what about other, seemingly simple operations? Consider taking our WSS process $X(t)$ and multiplying it by a deterministic, periodic signal—for example, a pulse train that turns the signal "on" and "off" cyclically. This is a common operation known as gating or chopping.

Is the output process, $Y(t)$, still WSS? Let’s check. Its mean remains zero. But what about its autocorrelation, $E[Y(t)Y(t+\tau)]$? This now involves the periodic pulse train $p(t)$ and becomes $p(t)p(t+\tau)R_{XX}(\tau)$. Because of the $p(t)p(t+\tau)$ term, this function now depends on the absolute time $t$, not just the lag $\tau$. Stationarity is broken!

But chaos has not been unleashed. The statistics are not just arbitrarily time-varying; they are periodic in time, with the same period as our chopping signal $p(t)$. We have created a ​​cyclostationary process​​. This is not a pathology; it's a feature. Many of the most important signals in communications and signal processing are cyclostationary by design. The carriers and symbol rates in radio and wireless communications imprint a periodic statistical structure onto the signals. By understanding this cyclostationarity, we can build receivers that can lock onto and decode these signals far more effectively than if we pretended they were stationary.

We have one final, and perhaps most profound, connection to make. Throughout our discussion, we have spoken of the autocorrelation function $R_X(\tau)$ and the power spectral density $S_X(\omega)$ as if they were known quantities given to us by an oracle. But how do we ever find them in the real world?

The definition of the autocorrelation, $E[X(t)X(t+\tau)]$, is an ensemble average—an average over an infinite collection of parallel universes, each with its own realization of the random process. In our universe, we only ever get to see one realization, and for a finite amount of time at that. How can we possibly compute an ensemble average?

We are saved by a powerful and beautiful idea: ​​ergodicity​​. An ergodic process is a special type of stationary process for which time averages, if taken over a long enough period, are equivalent to ensemble averages. It means that by observing a single path of the process for a long time, we can learn the statistical properties of the entire ensemble. The system explores all its possible statistical states just by evolving in time.

This allows us to take our finite block of N data points and compute an estimate of the autocorrelation. A common way to do this is to calculate $\hat{R}_X[k] = \frac{1}{N} \sum_{n=0}^{N-1-k} x[n]x[n+k]$. But we must be careful. This is an estimator, a random variable in its own right, not the true deterministic autocorrelation function. It comes with its own quirks. For instance, this specific estimator is biased; its expected value is actually $\frac{N-k}{N}R_X[k]$, systematically underestimating the true value for non-zero lags.

The distinction between the true, theoretical autocorrelation $R_X(\tau)$ and its finite-record time-average estimate $\hat{R}_X^{(T)}(\tau)$ is critical. The Wiener-Khinchin theorem, which states that the PSD is the Fourier transform of the autocorrelation, applies strictly to the true, ensemble-based function $R_X(\tau)$. The Fourier transform of our estimate $\hat{R}_X^{(T)}(\tau)$ gives us an estimate of the PSD (called a periodogram), a quantity that is itself random and comes with its own sources of error and uncertainty.

Making the "ergodic leap"—assuming that the single world we can measure is representative of the whole ensemble—is the essential bridge that connects the elegant mathematical theory of WSS processes to the practical, messy, but fascinating world of real data. It is this leap that allows us to use these powerful ideas to design communications systems, control noisy electronics, analyze financial markets, and interpret geophysical data, turning the abstract beauty of stationary processes into tangible engineering and discovery.