Admissible Controls in Stochastic Control Theory

In the quest to command systems that evolve under uncertainty—from navigating a spacecraft through cosmic radiation to steering an investment portfolio through market volatility—a fundamental question must be answered before any notion of "optimality" can be discussed: What constitutes a valid strategy? The seemingly simple act of choosing a control action is fraught with mathematical subtleties when randomness is involved. Without a rigorous framework, our strategies might paradoxically require knowledge of the future, or they might lead to physically impossible scenarios where the system's state explodes to infinity. This is the knowledge gap addressed by the theory of ​​admissible controls​​, which provides the essential rules of the game for stochastic optimization.

This article explores the critical concept of admissible controls, revealing why these rules are not arbitrary constraints but the very bedrock of modern control theory. First, in the ​​Principles and Mechanisms​​ chapter, we will journey into the heart of what makes a control strategy mathematically sound, exploring the principles of causality, measurability, and the conditions required to tame random dynamics. Then, in the ​​Applications and Interdisciplinary Connections​​ chapter, we will see how this carefully constructed foundation enables us to solve profound problems, from charting optimal paths with the Dynamic Programming Principle to guaranteeing safety in robotics and modeling the collective behavior of entire populations.

Imagine you are the captain of a sophisticated ship, navigating a tumultuous, unpredictable sea. Your goal is not just to reach a destination, but to do so with the least amount of fuel and in the calmest way possible. The ship is your ​​system state​​ ($X_t$), the rudder and engines are your ​​controls​​ ($u_t$), and the stormy sea is the relentless, random buffeting of ​​Brownian motion​​ ($W_t$). You have a map and a set of navigation principles. These principles are not arbitrary; they are the laws of physics and engineering that ensure your journey is possible and that your goal is meaningful. In the world of stochastic control, these are the principles of ​​admissible controls​​.

Answering the question "What can we use as a control?" is not just a mathematical formality. It is the very foundation upon which the entire theory of optimal control is built. Without a clear set of rules, our equations might describe a journey that is physically impossible, mathematically nonsensical, or a goal that is infinitely far away. Let's explore these rules, not as a list of dry conditions, but as a journey into the heart of controlling randomness.

The most fundamental rule of control is ​​causality​​. Your decisions at any moment can only be based on what has happened in the past and what is happening right now. You cannot turn the rudder based on a wave that will arrive in ten seconds. This iron law of the universe is given a beautifully precise mathematical form using the concept of a ​​filtration​​, denoted $(\mathcal{F}_t)_{t \ge 0}$.

Think of $\mathcal{F}_t$ as the sum total of all information available to you at time $t$—the history of your ship's position, the path of the waves so far, everything. The principle of causality then becomes a simple requirement: your control action $u_t$ must be determined solely by the information in $\mathcal{F}_t$. A process that respects this condition is called ​​adapted​​ to the filtration. This non-anticipating nature is the bedrock of Itô calculus and any realistic control problem. Allowing a control to be "anticipating" would be like giving our ship's captain a crystal ball. While seemingly powerful, it breaks the mathematical framework we use to model the world, rendering tools like the dynamic programming principle and Itô's formula inapplicable. The entire logical edifice of deducing optimal strategies relies on the ordered, sequential flow of information encoded by the filtration.

Being adapted is necessary, but for the strange world of Itô calculus, it's not quite sufficient. We are dealing with an integral against Brownian motion, $\int \sigma(X_s, u_s) dW_s$, which is no ordinary integral. Brownian motion is a process so jagged and erratic that its path is nowhere differentiable. To define a meaningful integral against such a "function," the integrand—the term $\sigma(X_s, u_s)$—must be more than just adapted. It needs a slightly stronger property called ​​progressive measurability​​.

You don't need to be a mathematician to grasp the intuition. It means that when viewed over any time interval $[0, t]$, the process we are integrating is "well-behaved" as a joint function of time and random outcomes. This technical requirement ensures that the sums that define the Itô integral converge properly. For our purposes, we can think of it as the price of admission for using the powerful machinery of Itô calculus. Fortunately, if our control process $u_t$ is itself progressively measurable, and the function $\sigma(x,u)$ is continuous, then the composite integrand $\sigma(X_s, u_s)$ will have the property we need. Therefore, we decree that an admissible control must be ​​progressively measurable​​. This technical rule ensures the very language of our model—the stochastic differential equation—is grammatically correct.

Suppose we have a control that respects causality and is progressively measurable. We can write down our SDE. Are we done? Not yet. Imagine if a small turn of the rudder caused the ship to spin infinitely fast, or if the ship's engine was so powerful that it could accelerate the ship to infinity in a matter of seconds. The system would "explode," and our model would cease to be useful.

To prevent this, we must put "leashes" on the dynamics. These are conditions on the functions $b(t,x,u)$ and $\sigma(t,x,u)$ that define the system's evolution. Critically, these leashes must hold no matter which admissible control value $u$ we choose.

1. ​​The Lipschitz Leash:​​ This condition essentially says that the change in the system's dynamics is bounded by the change in its state. Formally, for some constant $L$, we require $|b(t,x,u) - b(t,y,u)| + \|\sigma(t,x,u) - \sigma(t,y,u)\| \le L|x-y|$. This prevents the system from being infinitely sensitive. It ensures that two ships starting close together will not fly apart at an arbitrarily fast rate. This condition is the key to ensuring a ​​unique​​ solution to our SDE.

2. ​​The Linear Growth Leash:​​ This condition puts a cap on how fast the dynamics can grow as the state moves away from the origin. Formally, for some constant $K$, we require $|b(t,x,u)|^2 + \|\sigma(t,x,u)\|^2 \le K(1 + |x|^2)$. This is like a governor on an engine; it prevents the system from accelerating itself to infinity. It ensures the solution ​​exists​​ for all time on our interval, without exploding.

When these conditions hold ​​uniformly​​ for all controls $u$ in our action set $A$, we can be confident that for any admissible control strategy we dream up, there will be one, and only one, resulting trajectory for our system.

We now have a well-behaved system. But the goal of optimal control is to minimize a ​​cost functional​​, $J(u)$. This cost might be the fuel consumed, the time taken, or the deviation from a desired path. For this optimization problem to be meaningful, the cost for any admissible strategy must be a finite number. An infinite cost is like an infinite price tag—it's impossible to compare.

This leads to another crucial condition for admissibility: an ​​integrability condition​​ on the control process itself. For many problems, especially the famous ​​Linear Quadratic Regulator (LQR)​​ where costs are quadratic in the state and control, the natural condition is that the control must have finite expected energy. This is written as:

This single condition, combined with the linear growth leash on the dynamics, is often powerful enough to guarantee that all quadratic costs on the state $x_t$ are also finite, making the entire optimization problem well-posed. Similarly, the functions defining the cost, $\ell$ and $h$, cannot grow too quickly. If they grow at a polynomial rate in $x$, the linear growth leash on the dynamics is usually enough to keep the total cost finite. If, however, they were to grow exponentially, the cost could become infinite even for a perfectly well-behaved system.

So far, we have been playing a game with a fixed set of rules. The probability space, our "universe," and the source of randomness, the Brownian motion $W_t$, are given to us in advance. Our only task is to choose a control process $u_t$ that is adapted to the filtration of this given $W_t$. This is known as the ​​strong formulation​​ of a control problem. It is the most direct and intuitive setup.

But what if we could be more powerful? What if, instead of just choosing our actions within a given universe, our control could choose the universe itself? This is the core idea behind the ​​weak formulation​​. In this view, an admissible control is not just a process, but an entire probabilistic system—a probability space, a filtration, a Brownian motion, and a state process—that is consistent with our desired dynamics.

This is a more abstract and powerful perspective. It gives the controller a larger set of tools. Since we are optimizing over a larger set, the minimum cost we can achieve in a weak formulation can be lower than (or equal to) the minimum cost in a strong formulation. This added power is particularly crucial when the diffusion coefficient $\sigma$ depends on the control. While some might think a clever change of probability measure (via Girsanov's theorem) could make any controlled diffusion look like a simple drift change, this is not so. The quadratic variation of a process—its intrinsic roughness—is determined by $\sigma \sigma^\top$ and cannot be altered by such measure changes. The ability to control $\sigma$ is a genuine power that the weak formulation fully embraces. The distinction vanishes, however, under certain ideal conditions. If the SDE is known to have a unique pathwise solution for every control, then the weak and strong formulations become equivalent—the extra freedom of the weak formulation doesn't actually create any new dynamics.

We have defined the rules of our game. But does a "best" strategy always exist? The surprising answer is no.

Imagine your control is a simple switch that can only be set to $u=-1$ or $u=+1$. This control set $U = \{-1, +1\}$ is not convex; it has a hole in the middle. Now, suppose the optimal strategy would ideally require a control value of $u=0$. Since this is not allowed, a minimizing sequence of controls might try to achieve this effect by "chattering"—oscillating infinitely fast between $-1$ and $+1$. The sequence gets closer and closer to the optimal cost, but it never settles on a single, well-defined control strategy. In the limit, there is no optimal control within the allowed set.

This is a deep problem related to the lack of ​​compactness​​ in the set of control strategies. To solve it, mathematicians came up with a brilliant idea: ​​relaxed controls​​. Instead of forcing ourselves to choose a single action $u_t \in U$ at each instant, what if we could choose a probability distribution over the actions? In our switch example, instead of just choosing 'on' or 'off', we could choose a control that is "60% on and 40% off" on average at that instant.

This measure-valued control, $\nu_t$, lives in the space of all probability measures on $U$, denoted $\mathcal{P}(U)$. A remarkable mathematical fact is that if $U$ is compact, then $\mathcal{P}(U)$ is also compact and convex. By enlarging our set of strategies to include these relaxed controls, we "fill in the holes." The chattering sequence that had no limit in the original space now has a perfectly well-defined limit in the relaxed space. This restored compactness guarantees that an optimal relaxed control always exists. In many cases, it turns out the optimal relaxed control is actually a simple Dirac measure, which corresponds to a classical, non-relaxed control. But by taking a detour through this larger, more abstract space, we can prove that a solution exists and discover its nature.

From the intuitive principle of causality to the abstract beauty of relaxed controls, the definition of "admissible" is a rich tapestry of ideas. It is the framework that ensures our quest to control random systems is not a fool's errand, but a well-posed and profoundly fascinating journey.

In our previous discussion, we laid the groundwork for what makes a control "admissible." We saw that this isn't just mathematical pedantry; it's the very soul of defining a problem that has a sensible solution. We demanded that our controls be "measurable" and "non-anticipative," which is our formal way of saying that a decision-maker can't be infinitesimally clever, nor can they see into the future. These might seem like abstract constraints, but it is precisely these sensible limitations that unlock a universe of applications. Now, let's embark on a journey to see how this carefully crafted concept of admissible controls allows us to navigate through some of the most fascinating problems in science and engineering, from charting the single best path to keeping a robot safe, and even to predicting the behavior of entire crowds.

Imagine you are the captain of a ship trying to sail from one port to another in the shortest possible time. You have maps of the currents and forecasts of the wind. How do you plan your entire journey? Do you plot it all out at the start and stick to it rigidly? What if you get blown off course? The most powerful idea for solving such problems is Richard Bellman's ​​Dynamic Programming Principle (DPP)​​. It's a wonderfully simple yet profound observation: if the best path from New York to Lisbon passes through the Azores, then the Azorean leg of that journey must be the best possible path from the Azores to Lisbon. Any optimal path is composed of optimal sub-paths.

This principle allows us to stop thinking about the entire colossal problem at once and instead focus on making the best possible decision at each and every moment. We can write down an equation, the famous ​​Hamilton-Jacobi-Bellman (HJB) equation​​, which crystallizes the DPP into a local rule. The HJB equation acts like a magical compass: at any given location and time, it tells you the value of being there—the "optimal cost-to-go"—and which direction to steer to achieve the best outcome. The "admissible controls" are all the possible steering directions we are allowed to consider at each instant. The HJB equation essentially says that the change in value over a tiny amount of time is determined by choosing the best possible admissible control right now.

Of course, the real world is rarely so predictable. Our ship is tossed by random waves and unpredictable gusts of wind. Here, the non-anticipative nature of admissible controls becomes paramount. We are now dealing with a stochastic system, and we want to minimize the expected travel time. The DPP still holds, but now it's a statement about expectations. The optimal strategy is no longer a fixed path, but a policy—a rule that tells us how to react to the random events as they unfold. The HJB equation gains a new term related to the variance of the noise, and our admissible controls are now functions that adapt to the information as it arrives, but crucially, cannot anticipate it. You can adjust your rudder in response to a wave that has just hit you, but not to one that is still brewing miles away. The careful definition of admissible controls as "progressively measurable" processes is the mathematical guarantee of this physical principle of causality.

So, if we can find a function that solves this HJB equation, a remarkable thing happens. This function is not only the true value function (the optimal cost-to-go from any point), but it also gives us the optimal control policy for free! This is the content of a ​​verification theorem​​. The solution to the HJB equation provides a "certificate" of optimality. We simply choose the control at each instant that minimizes the Hamiltonian—the core expression inside the HJB equation—and we are guaranteed to be following the optimal strategy. This powerful connection between a partial differential equation and an optimal real-time policy is one of the crown jewels of control theory.

Sometimes, our primary goal is not to be the fastest or most efficient, but simply to survive. Imagine an autonomous drone flying through a dense forest or a surgical robot operating near a vital organ. The absolute priority is to avoid collision. The system must remain within a pre-defined "safe set" for all time. This is the domain of ​​viability theory​​.

Here, the concept of admissible controls takes on a beautiful and intuitive new role. As our system approaches the boundary of the safe set—the edge of a cliff, the wall of an artery—the set of admissible controls shrinks. To guarantee safety, we must choose a control that steers the system back toward the interior of the safe set, or at the very least, tangent to the boundary. Any control that points "outward" becomes inadmissible at that point. The system's dynamics must be viable; they must allow for the existence of at least one such safe control at every point within the safe set.

This idea has been turned into a powerful and practical engineering tool with the development of ​​Control Barrier Functions (CBFs)​​. A CBF is a function that defines the safe set, much like a topographic map defines altitude. The condition that the system remains safe can be translated into a simple inequality that the control input must satisfy. This inequality defines, at every instant, the set of all safe, admissible controls.

What makes this so powerful is that it can be implemented as a real-time safety filter. Suppose a high-level planner gives a "nominal" command to our autonomous car, like "accelerate to pass." A CBF-based safety module can check if this command is safe. If the command would lead the car too close to the vehicle in front, it is deemed inadmissible. The safety module then solves a tiny, instantaneous optimization problem: "What is the closest possible control action to the one I was commanded, which still satisfies the safety inequality?" The result is a minimally modified, guaranteed-safe command that is sent to the actuators. This is a real-time projection onto the set of admissible controls, ensuring safety without completely overriding the system's performance goals. This technology is at the heart of modern robotics and autonomous systems, providing a provable guarantee of safety.

We've assumed so far that our captain or robot knows its exact state—its position, velocity, and so on. But what if the GPS is noisy, the sensors are imperfect, and we are navigating in a fog? This is the problem of control under ​​partial observation​​.

The celebrated ​​Linear Quadratic Gaussian (LQG)​​ problem provides the canonical framework for this situation. Here, the system is linear, the costs are quadratic, and the noises are Gaussian, but the controller cannot see the state $x_t$ directly. Instead, it receives a noisy observation $y_t$. The crucial step in formulating this problem is to redefine the meaning of "admissible control." An admissible control can no longer depend on the true state $x_t$, which is hidden. It can only depend on the history of observations, $\{y_s : s \le t\}$. The information available to the controller is fundamentally limited.

One might expect this to lead to an impossibly complex problem. But a result of breathtaking elegance, the ​​Separation Principle​​, comes to our rescue. It states that the problem miraculously separates into two simpler parts:

1. ​​Estimation​​: Use the history of observations to compute the best possible estimate of the current state. For the LQG problem, this is done using the famous ​​Kalman filter​​.
2. ​​Control​​: Solve the optimal control problem as if you could see the state perfectly. This gives an optimal state-feedback law. Then, simply apply this law to the estimate you computed in the first step.

The separation principle is not a given; it is a deep theorem that holds because of the specific structure of the problem and the careful definition of admissible controls. It tells us that we can first solve the problem of "seeing" and then, separately, solve the problem of "acting." This principle has been the workhorse of aerospace engineering since the Apollo program, guiding spacecraft to the Moon and back, and it remains fundamental to countless technologies, from econometrics to target tracking.

Let's push to the frontiers of control theory. What happens when our environment is not just a passive, random entity, but is itself composed of countless other agents, all making their own decisions? This is the world of ​​mean-field games​​, a revolutionary framework for studying the collective behavior of large populations of strategic agents. Think of traders in a stock market, drivers in city traffic, or even birds in a flock.

In a mean-field game, each individual agent tries to optimize its own objective (e.g., maximize profit, minimize travel time). However, the dynamics and the costs for that agent depend on the aggregate behavior of the entire population—the "mean field." For instance, the price of a stock depends on the average buying and selling behavior of all traders. At the same time, the collective behavior is nothing more than the result of all the individuals simultaneously implementing their own optimal strategies.

This creates a fascinatingly complex feedback loop. To find a solution, or a ​​Nash equilibrium​​, we need to find a state where no single agent can improve its outcome by changing its strategy, given that everyone else's strategy remains the same. The concept of admissible controls here becomes incredibly subtle. An agent's control might depend on its own private state (idiosyncratic noise) as well as on information that affects everyone (common noise), like a public news announcement. To solve this, the state of the problem must be augmented to include not just the state of the individual agent, but also the probability distribution of the entire population. The dynamic programming principle is then lifted to a space of probability measures, leading to a coupled system of two PDEs: an HJB equation that describes the optimal control for a single agent given the population's behavior, and a Fokker-Planck equation that describes how the population's distribution evolves as a result of all the agents' actions. This powerful framework is now being used to gain insights into systemic risk in finance, the formation of traffic jams, and the dynamics of social networks.

From the simple principle of optimality to the intricate dance of large-scale interacting systems, the journey of "admissible controls" is a testament to the power of abstraction. By starting with a carefully, almost philosophically, considered definition of what constitutes a valid strategy, we have built a theoretical edifice that allows us to tackle, with stunning success, problems of immense practical and intellectual importance across the entire scientific landscape.