Convexity Preservation

In mathematics and engineering, a "well-behaved" problem is often a convex one. Like a perfectly bowl-shaped valley with a single lowest point, convex problems guarantee that a solution, if it exists, is unique and can be found reliably. This property is the holy grail for optimization. But as we build complex models by combining simpler parts, a critical question arises: how do we ensure the final system doesn't lose this desirable convex nature? The answer lies in the principles of convexity preservation—a set of rules that act as an "algebra" for maintaining stability and solvability.

This article explores the art and science of preserving convexity. It addresses the challenge of building complex, realistic models that remain computationally tractable. You will first delve into the core "Principles and Mechanisms," uncovering the fundamental operations that maintain convexity, from simple sums and intersections to more advanced transformations like infimal convolution. Then, in "Applications and Interdisciplinary Connections," you will see these principles in action, embarking on a journey through fields as diverse as machine learning, economics, and solid mechanics to understand how preserving convexity underpins stability, enables optimal decision-making, and even describes the shape of the physical world.

Imagine you are standing in a landscape of rolling hills and deep valleys. If you are in a bowl-shaped valley, any step you take downhill will eventually lead you to the single lowest point. There are no false bottoms, no tricky local minima to get stuck in. This simple, reliable property is the essence of convexity. In mathematics, physics, and engineering, we are constantly searching for problems that have this "bowl-shaped" character. Once we find it, we know we can solve the problem, and the solution will be unique and robust. But how do we know if a problem has this wonderful property? And how can we build complex systems while ensuring we don't lose it? This is the art and science of preserving convexity.

At its heart, convexity is a geometric idea. A set of points—say, a region in a plane—is ​​convex​​ if for any two points you pick inside the set, the straight line segment connecting them lies entirely within the set. A perfect disk is convex. A square is convex. A banana shape, however, is not; you can easily find two points in a banana whose connecting line pops outside.

This simple geometric notion has a powerful counterpart for functions. A function $f(x)$ is said to be ​​convex​​ if the line segment connecting any two points on its graph never falls below the graph itself. The classic parabola $f(x) = x^2$ is a perfect example. Pick any two points on its curve, draw a straight line between them, and the parabola will always sag beneath that line.

What's the connection between these two ideas? It's a beautiful concept called the ​​epigraph​​. The epigraph of a function is simply the set of all points lying on or above its graph. For our parabola $f(x) = x^2$, its epigraph is the entire shaded region above the U-shaped curve. And here is the profound link: ​​a function is convex if and only if its epigraph is a convex set​​. This turns a property of functions into a property of shapes, allowing us to use our geometric intuition to understand functions.

A direct consequence of this is that the ​​sublevel sets​​ of a convex function are always convex sets. A sublevel set is just all the points $x$ where the function's value is below a certain threshold, $f(x) \le \alpha$. Geometrically, this is like flooding our convex epigraph with water up to a certain height $t=\alpha$ and looking at the shape of the flooded region on the "floor" (the $x$-axis). Because the epigraph was a convex "bowl," the shape of the water's edge will always define a convex region.

If we have a toolbox of convex functions and sets, we can start combining them to build more interesting things. The magic is that certain operations are guaranteed to preserve convexity, giving us an "algebra" for building well-behaved systems.

1. ​​Intersection, Sums, and Scaling:​​ The simplest rules are the most fundamental.

   • The ​​intersection​​ of any number of convex sets is always convex. Imagine overlapping two convex shapes; the region they share is also convex. This is why the intersection of two sublevel sets, like $\{x \mid \max\{f(x), g(x)\} \le \alpha\} = L_\alpha(f) \cap L_\alpha(g)$, is convex if $f$ and $g$ are.
   • ​​Scaling and Adding:​​ If you take convex functions, multiply them by non-negative numbers, and add them up, the result is still convex. For instance, if $f(x)$ and $g(x)$ are convex, so is $af(x) + bg(x)$ for any $a, b \ge 0$.

2. ​​The Pointwise Supremum (The Upper Envelope):​​ What if we have a whole family of convex functions, $\{f(x,t)\}$, indexed by some parameter $t$? We can define a new function, $G(x) = \sup_t f(x,t)$, which at every point $x$ takes on the highest value from the entire family. This operation always preserves convexity. The reason is elegant: the epigraph of the supremum, $\text{epi}(G)$, is precisely the intersection of all the individual epigraphs, $\bigcap_t \text{epi}(f(\cdot,t))$. Since intersection preserves convexity, the resulting function $G(x)$ must be convex. It's like draping a sheet over a collection of convex tent poles; the resulting surface is convex.

3. ​​The Pointwise Infimum (The Cautionary Tale):​​ You might guess that the infimum, $F(x) = \inf_t f(x,t)$, also preserves convexity. But here, nature throws us a curveball. The infimum (or minimum) of convex functions is ​​not​​ generally convex. Consider two simple parabolas, $f_1(x) = (x-1)^2$ and $f_2(x) = (x+1)^2$. Both are perfectly convex. But their minimum, $F(x) = \min\{f_1(x), f_2(x)\}$, looks like a "W" shape. You can easily pick two points on the outer arms of the "W" whose connecting line segment passes underneath the central peak, violating convexity. This is a crucial lesson: building a "floor" from convex components does not guarantee a convex floor.

The algebra of convexity gets even more interesting when we start plugging functions into other functions.

• ​​Affine Composition:​​ A simple yet powerful rule is that if $f$ is a convex function, then composing it with an affine map (a linear transformation plus a shift) preserves convexity. That is, $h(x) = f(Ax+b)$ is convex. This means we can stretch, rotate, and shift our convex functions, and they remain convex. This is vital, as it allows us to change coordinate systems without breaking the well-behaved nature of our problem. For example, if we have a simple quadratic function $f(\mathbf{y}) = \frac{1}{2}\|\mathbf{y}\|^2$, and we feed it a linear transformation of our variable $\mathbf{x}$, the resulting function $h(\mathbf{x}) = \frac{1}{2}\|A\mathbf{x}+\mathbf{b}\|^2$ remains convex, and its "convexness" (specifically, its strong convexity) is determined by the properties of the matrix $A$.

• ​​General Composition:​​ For a more general composition $h(x) = f(g(x))$, the rules are stricter. It's not enough for both $f$ and $g$ to be convex. We saw a counterexample with the "W" shape, but another striking one is taking $g(x)=x^2$ (convex) and $f(y) = \exp(-y)$ (also convex). The result, $h(x) = \exp(-x^2)$, is the famous Gaussian bell curve, which has a bump in the middle and is distinctly non-convex. The sufficient condition is that in addition to $f$ and $g$ being convex, the outer function $f$ must also be ​​non-decreasing​​. This makes intuitive sense: the inner function $g$ bends the input "upwards," and the outer function $f$ must not reverse this curvature by penalizing larger values.

• ​​The Magic of Smoothing (Infimal Convolution):​​ Remember how taking the infimum of convex functions can fail? There is a special kind of infimum, called ​​infimal convolution​​, that miraculously does preserve convexity. We can visualize it by returning to epigraphs. Instead of just taking a union of epigraphs, we can combine them using a ​​Minkowski sum​​. The Minkowski sum of two sets $A$ and $B$ is the set of all possible sums of their elements, $\{a+b \mid a \in A, b \in B\}$. If we take the Minkowski sum of the epigraphs of two convex functions $f$ and $g$, the result is another convex set which is the epigraph of a new function, $h(x) = \inf_u \{f(u) + g(x-u)\}$. This new function $h$ is guaranteed to be convex!

  A spectacular application of this is the ​​Moreau envelope​​. Here, we convolve a function $f$ with a simple quadratic bowl, $\frac{1}{2\lambda}\|x\|^2$. The resulting function,

  $$e_\lambda f(x) = \inf_{y \in \mathbb{R}^n}\left\{ f(y) + \frac{1}{2\lambda}\|x-y\|^2 \right\}$$

  has incredible properties. It takes any proper convex function $f$—even one with sharp corners, like the absolute value function $|x|$—and produces a new function $e_\lambda f(x)$ that is not only convex but also smooth and continuously differentiable! It's a universal smoothing machine that preserves convexity. As the parameter $\lambda$ goes to zero, the smoothed function sharpens back into the original function $f$. As $\lambda$ goes to infinity, it flattens out into a constant function equal to the minimum value of $f$. It is a tunable, beautiful approximation tool at the heart of modern optimization.

These abstract rules are not just mathematical games. They are the tools we use to understand and guarantee the behavior of complex systems.

• ​​Optimization's Holy Grail:​​ The main reason we hunt for convexity is that if a problem can be formulated as minimizing a convex function over a convex set, it has (at most) one global minimum. This is the bedrock of countless fields. In optimal control, engineers ensure the cost functional for guiding a rocket is strictly convex so there is a single, unambiguous optimal flight path to calculate.

• ​​When Physics Fights Back:​​ In the theory of solid materials, one might hope that the energy of a deformed elastic body is a convex function of the deformation. This would guarantee a unique equilibrium shape for any applied force. However, a fundamental principle of physics—frame indifference (the energy shouldn't change if you just rigidly rotate the object)—is incompatible with strict convexity for large deformations. Nature, it seems, refuses to be so simple! This forces mathematicians to invent weaker notions like ​​polyconvexity​​ and ​​quasiconvexity​​. These are just enough to prove that a solution exists, but not that it's unique. This lack of uniqueness is not a failure but a profound insight: it corresponds to real physical phenomena like phase transitions and the formation of complex microstructures in materials.

• ​​The Dynamics of Shape:​​ The principle of convexity preservation also governs how shapes evolve in time. A famous example is ​​Mean Curvature Flow​​, which describes how a surface shrinks under its own surface tension, like a soap film. A celebrated theorem by Gerhard Huisken states that if you start with a convex surface, it will remain convex as it shrinks down to a single point. It will never spontaneously develop a saddle-like dimple. This principle, and others like it, are consequences of a deep result called ​​Hamilton's Maximum Principle for Tensors​​. It states, roughly, that if a tensor quantity (like the curvature of a surface) lives inside a convex cone of "allowed" tensors, and the forces of its evolution equation don't push it out of the cone, then it will stay in the cone forever.

• ​​Probing the Fabric of Space:​​ Even in the purest realms of geometry, these ideas are central. The simple fact that the intersection of convex sets is convex is a key lemma in proving profound theorems about the structure of curved spaces. For instance, in a space with non-negative curvature, one can construct a nested sequence of closed, totally convex sets. Their intersection, which is also totally convex, forms the "soul" of the manifold—a compact submanifold that captures its entire topological essence. This powerful result, the Soul Theorem, grows from the same simple seed: convexity is preserved under intersection.

From finding the one true answer in an optimization problem to understanding the many possible forms of matter and the very structure of space, the principles of convexity preservation form a golden thread, unifying disparate fields with their simple, elegant, and powerful logic.

We have spent some time getting to know convex functions on a formal basis, learning their definitions and basic properties. At this point, you might be thinking that this is all a fine game for mathematicians, but you might wonder: what is the real point? Does this concept of "no hidden valleys" actually show up in the messy, complicated real world?

The answer is a resounding yes, and the reason is not just that we happen to find convex things here and there. The true magic, the secret to its vast importance, is that convexity is a remarkably resilient property. Once you have it, many of the most common operations in science and engineering—summing things up, averaging, taking the maximum, looking for the best course of action—do not destroy it. This "preservation of convexity" is a deep and beautiful principle that unifies an astonishing range of fields, from genetics to economics to the very geometry of space. It is the invisible thread that guarantees stability, enables optimization, and allows us to make sense of complex systems.

Let's embark on a journey to see this principle in action.

One of the simplest things we do is add things together or take an average. What happens to convexity when we do this?

Imagine studying the effect of a chemical mutagen on a population of cells. Within a single cell, there are intricate biochemical processes. A mutagen enters, the cell's enzymes try to detoxify it, and its repair mechanisms try to fix any damaged DNA. These processes, like many in biology, can become saturated. As the external dose $d$ of the mutagen increases, the cell's defenses can't keep up. The result is that the internal damage doesn't just grow linearly; it accelerates. The dose-response curve for a single cell, plotting mutation probability versus dose, curves upwards—it is a convex function.

Now, here is the wonderful part. In any real population of cells, or people, there is variation. Some individuals have more efficient detoxification enzymes than others due to their genes. You might think that averaging over this messy, heterogeneous population would wash out any clean mathematical property. But it does not! Because the expectation operator is, in essence, a sophisticated form of a weighted sum, and because the sum of convex functions is convex, the average dose-response curve for the entire population is also convex. The presence of a few highly susceptible individuals, whose response curves bend sharply upwards even at low doses, is enough to make the population average curve upwards as well. This preservation of convexity under averaging is a key reason why many population-level phenomena in toxicology and epidemiology exhibit non-linear, accelerating behavior.

This same principle is the bedrock of many modern machine learning models. When we train a model to rank items—say, search results or product recommendations—we might define a loss function that penalizes every pair of items that are in the wrong order. Each of these pairwise loss components is designed to be a convex function of the model's parameters. The total loss, the one we actually try to minimize, is simply the sum of all these individual penalties. By summing them, we preserve the convexity, ensuring our overall optimization problem has a single basin and no treacherous local minima to get stuck in.

Nature and science are full of chains of processes, where the output of one step becomes the input to the next. What happens to convexity when we compose functions? The rules here are more subtle, but just as powerful.

Consider an investor choosing how to allocate their wealth among different assets. The final wealth is a linear function of the portfolio weights $\mathbf{w}$. The investor, however, doesn't care about wealth directly, but about the utility it brings. A fundamental concept in economics is diminishing marginal utility: the first million dollars changes your life more than the tenth. This means the utility function, $u$, is concave. So, the utility as a function of our portfolio weights, $u(f(\mathbf{w}))$, is a concave function.

Now, suppose we want to solve an optimization problem involving this utility, perhaps by applying one more function, $g$. Our objective becomes $\mathbb{E}[g(u(f(\mathbf{w})))]$. For this to be a convex optimization problem (minimizing a convex function), the composition $g \circ u \circ f$ must be convex. We have an affine function $f$ inside a concave function $u$, which results in a concave function. For the final composition with $g$ to be convex, a specific rule must be followed: $g$ must be both convex and non-increasing. This chain of reasoning reveals the precise conditions needed to ensure a financial optimization problem is well-behaved and solvable.

This idea of "building" a function to have the right properties finds its ultimate expression in the design of modern artificial intelligence. We can create neural networks that are, by their very architecture, guaranteed to be convex functions of their inputs. These are called Input Convex Neural Networks (ICNNs). The trick is to enforce the rules of convexity preservation at every single layer. The weights that combine outputs from the previous layer must be non-negative, preserving convexity from the summation. Then, the activation function—the non-linear "spark" of the neuron—must be both convex and non-decreasing to pass the convexity along to the next layer. By following these rules, we can construct an incredibly complex and expressive function that still retains the beautiful simplicity of having no local minima.

Convexity truly shines when we need to make optimal decisions, especially under uncertainty.

Let's go back to our nurse staffing problem. A hospital manager must decide how many nurses to schedule for a baseline staff, $x$, before knowing how many will call in sick. After the random absences are revealed, they can hire expensive temporary nurses to meet the demand. The manager's first-stage decision $x$ is difficult because it affects the costs in all possible future scenarios. The key is to analyze the expected recourse cost, $Q(x)$—the average cost of hiring temporary nurses, taken over all possible absenteeism scenarios.

Here, we see two powerful preservation principles at work. For any single future scenario, the problem of hiring the minimum number of temps is a linear program, a simple form of convex optimization. Its optimal value is a convex (in this case, piecewise-linear and convex) function of the available baseline staff $a_s x$. Then, the total expected cost $Q(x)$ is the probability-weighted average of these convex functions. As we saw with the genetics example, this averaging operation preserves convexity. The result is that the complex, two-stage problem simplifies into minimizing a convex function $Q(x)$ (plus the initial cost $cx$), a problem we know how to solve efficiently.

This same logic extends to the heart of reinforcement learning and control theory. In a Markov Decision Process, we seek a policy that minimizes the total future cost. The solution is characterized by an optimal cost-to-go function, $J^{\star}(s)$, which tells us the best possible cumulative cost starting from any state $s$. Finding this function is the central challenge. However, under certain natural conditions—for instance, if the per-stage cost is convex and the system dynamics are linear—the famous Bellman optimality operator has a magical property: it preserves convexity. If you apply the operator to a convex function $J$, the resulting function $TJ$ is also convex. This implies that the true optimal cost-to-go function, $J^{\star}$, must itself be convex! This insight is a tremendous advantage. By knowing the solution we're looking for lives in the well-behaved world of convex functions, we can design much more efficient and reliable learning algorithms. It provides a powerful "inductive bias," guiding the learning process toward the right kind of solution. The same principle, in a more abstract form, underpins the existence of solutions in the classical calculus of variations.

Finally, convexity is not just an abstract tool for optimization; it is woven into the fabric of the physical world. It describes stability in materials and the evolution of shapes in geometry.

In solid mechanics, when a ductile metal is put under stress, it doesn't fail immediately. It can withstand a certain range of stresses. This "admissible stress set" must be a convex set for the material model to be physically stable. When engineers build sophisticated models for material failure, like the Gurson–Tvergaard–Needleman (GTN) model, they often combine different criteria. For example, they might say the stress state is safe if it satisfies both the GTN criterion and a simple pressure "cap." The resulting safe set is the intersection of the two original sets. If both original sets are convex, their intersection is also convex. This is often implemented by defining the combined failure function as the maximum of the individual functions, an operation that preserves the convexity of the resulting admissible set. This ensures the new, more complex material model doesn't lose the essential property of stability. Modern approaches even build this stability directly into machine learning models of materials, using ICNNs to construct what is known as a polyconvex stored-energy function, guaranteeing the learned model is physically realistic.

Perhaps the most elegant illustration of convexity preservation is in geometry itself. Consider the process of mean curvature flow, which describes phenomena like the smoothing of a crystal or the collapse of a soap bubble. If you start with a shape that is strictly convex—a smooth, rounded pebble, for instance—and let it evolve under this flow, it will remain convex at all times. It shrinks gracefully and smoothly, becoming more and more spherical until it vanishes into a single point. Contrast this with a non-convex shape, like a dumbbell. Under the same flow, the thin neck shrinks much faster than the heavy bells. The flow does not preserve its shape; instead, a singularity forms, and the neck pinches off in a finite time. Here, convexity is synonymous with structural integrity and a stable, predictable evolution.

From the average response of a cell population to the grand evolution of geometric shapes, convexity is a property that, once present, tends to endure. It is this resilience, this preservation across operations and disciplines, that transforms it from a mere mathematical definition into a profound and unifying concept in science and engineering.