Rotational Indeterminacy

When we use mathematical models to make sense of complex data, we expect them to give us clear, definitive answers. We seek the fundamental components, the latent factors, or the underlying states that govern a system. However, the concrete output we receive is often just one of many equally valid possibilities, like one of several correct routes to the same destination. This gap between a unique underlying reality and the non-uniqueness of its description is the core of rotational indeterminacy. This article confronts this often-overlooked ambiguity, reframing it not as a flaw in our methods but as a deep principle about the nature of structure and symmetry. The following chapters will guide you through this concept. First, ​​Principles and Mechanisms​​ will delve into the geometric and algebraic foundations of rotational indeterminacy, using examples from PCA to control theory. Subsequently, ​​Applications and Interdisciplinary Connections​​ will demonstrate the profound impact of this principle across diverse scientific domains, from psychology to structural biology, and explore the clever strategies researchers use to find a meaningful perspective.

Have you ever given someone directions? You might say, "Walk two blocks east and one block north." But someone else, facing a different direction, might describe the same destination as "Walk one block west and two blocks south." Both descriptions are correct; they just use a different frame of reference, a different coordinate system. The destination itself, the physical reality, hasn't changed. This simple idea, the distinction between a thing and its description, lies at the heart of a deep and beautiful principle that echoes across many fields of science and engineering.

When we build models to understand the world, we are often trying to find a "description" of a complex reality. We seek the fundamental factors, the principal axes, the essential states that govern a system. We run our data through powerful algorithms and get back a set of numbers, a loading matrix, a list of components. It's tempting to look at this single, concrete output and believe we have found the answer, the one true description. But often, what we have found is just one of infinitely many possible descriptions, like one set of directions to a destination. The underlying reality is invariant, but our description of it is subject to a kind of freedom, a ​​rotational indeterminacy​​. This chapter is a journey to understand this principle, not as a flaw in our methods, but as a fundamental truth about the nature of structure and symmetry.

Let's begin our journey with a common tool in a data scientist's toolkit: ​​Principal Component Analysis (PCA)​​. Imagine you have a cloud of data points. PCA is like finding the skeleton of this cloud; it seeks out the directions in which the data varies the most. If your data cloud is shaped like a long, thin ellipse, the answer is obvious and satisfying. There is one direction of maximum variance (the major axis) and another, perpendicular to it, of minimum variance (the minor axis). These "principal components" feel unique and fundamental.

But what happens if your data cloud is perfectly circular? If you try to find the direction of "maximum" variance, you'll find that every direction passing through the center is equally good. There is no longer a single, unique major axis. You can pick any diameter, and then another one perpendicular to it, and you'll have a perfectly valid set of principal axes. The algorithm will give you one pair, but it's an arbitrary choice, a spin of the roulette wheel. Any rotation of that pair would be just as correct.

This geometric picture has a precise algebraic counterpart. In PCA, these principal directions are the eigenvectors of the data's covariance matrix, and the amount of variance they capture is given by the corresponding eigenvalues. For the ellipse, the eigenvalues are distinct. For the circle, the two largest eigenvalues are equal. This degeneracy, this equality of eigenvalues, is the algebraic signature of symmetry. It's the mathematics telling you that the underlying structure has a rotational freedom, and that no single basis, no single set of "principal" directions, is sacred. This is our first, and simplest, encounter with rotational indeterminacy.

The practical consequences are immediate. If you are a biologist studying gene expression, you cannot simply perform PCA on two different datasets and assume that "principal component 1" from the first analysis means the same thing as "principal component 1" from the second. The sign of the component is arbitrary—an eigenvector $\mathbf{v}$ is just as valid as $-\mathbf{v}$. Worse, if the top two eigenvalues are close, the component labeled "PC1" in one dataset might be a rotated mixture of what the algorithm labeled "PC1" and "PC2" in the other. Direct comparison is a fool's errand without understanding this ambiguity.

This ambiguity is not just a special case for round data. It becomes a central feature in more sophisticated models like ​​Factor Analysis (FA)​​. In FA, we postulate that a large number of observable variables (like scores on dozens of different tests) can be explained by a small number of unobserved, latent "factors" (like "intelligence" or "creativity"). The model has the form $\mathbf{\Sigma} = \mathbf{\Lambda}\mathbf{\Lambda}' + \mathbf{\Psi}$, where we observe the covariance matrix $\mathbf{\Sigma}$ and try to estimate the factor loading matrix $\mathbf{\Lambda}$.

Here, the indeterminacy is built into the very structure of the model. The observable part of the covariance is captured by the term $\mathbf{\Lambda}\mathbf{\Lambda}'$. Now, imagine you have a set of factor loadings $\mathbf{\Lambda}$ that perfectly explains your data. What happens if you take your hidden factors and "rotate" them? For any orthogonal matrix (a rotation) $Q$, you can define a new set of loadings $\mathbf{\Lambda}^* = \mathbf{\Lambda}Q$. Let's see what happens to the observable covariance:

It's completely unchanged! This means that if $(\mathbf{\Lambda}, \mathbf{\Psi})$ is a valid solution, then so is $(\mathbf{\Lambda}Q, \mathbf{\Psi})$ for any rotation $Q$. If we are modeling with two factors ($m=2$), there's an entire circle of solutions. For three factors ($m=3$), there's a whole sphere of them. The model is fundamentally underdetermined.

This is not just a theoretical curiosity. It's a critical issue known as the problem of ​​identifiability​​. If you have too many parameters to estimate for the amount of data you have, your model might not be identifiable at all, even up to rotation. When a solution does exist, the rotational freedom means that the raw output of a factor analysis is arbitrary. This is why researchers developed methods like "Varimax rotation"—not to find the one "true" set of factors (which doesn't exist), but to rotate the arbitrary solution to a position that is simpler to interpret, for example, where each variable is strongly associated with only one factor. The special case is when you have only one factor ($m=1$). Here, the "rotation" is just multiplication by $1$ or $-1$, so the ambiguity is reduced to a simple sign flip.

At this point, you might think this is a peculiar quirk of statistical modeling. But the astonishing thing—the part that reveals the deep unity of scientific principles—is that this exact same mathematical structure appears in completely different domains.

Let's visit the world of ​​control theory​​, where engineers design algorithms to control systems like airplanes or chemical reactors. They use a "state-space" model, a set of equations that describe the internal state of the system. This model is often identified from data using matrix factorization, for instance, a data matrix $\mathbf{G}$ is decomposed as $\mathbf{G} = \mathcal{O}\mathbf{X}$. Sound familiar? Just as in factor analysis, this factorization is not unique. For any invertible matrix $\mathbf{T}$ (a generalized rotation or "similarity transformation"), the decomposition $\mathbf{G} = (\mathcal{O}\mathbf{T})(\mathbf{T}^{-1}\mathbf{X})$ is equally valid. A different choice of $\mathbf{T}$ corresponds to a different choice of "state variables" to describe the system. The internal description changes, but the external input-output behavior of the system remains identical. It is the same principle in a different guise.

Now let's go to the world of ​​physics and continuum mechanics​​. Imagine a steel bridge. Engineers use the equations of elasticity to calculate how the bridge deforms and where stresses concentrate under load. A solution to these equations gives the displacement of every point in the bridge. But is this solution unique? No. If you have one valid solution for the deformed shape, you can take that entire deformed shape and translate it ten feet to the left, or rotate the whole thing by one degree. These rigid body motions induce no strain and therefore no stress. They are "invisible" to the governing equations of static equilibrium. Therefore, any solution to a problem with only forces specified on the boundary (a pure Neumann problem) is only unique up to a rigid body motion. The set of all possible translations and rotations forms the "kernel" of the elasticity operator—they are the motions that the operator annihilates.

Let's take one more leap, into the elegance of ​​differential geometry​​. How do you define a curve in 3D space? The Fundamental Theorem of Curves states that a curve's essential shape is completely determined by two functions: its curvature $k(s)$ and its torsion $\tau(s)$. If you know these, you know everything about the curve's bends and twists. But where is the curve located in space? And which way is it pointing? You don't know. The description $(k(s), \tau(s))$ defines the curve uniquely only up to a rigid motion—a rotation and a translation in space. A helix is a helix, whether it's in your room or on Mars, pointing up or pointing sideways. Its intrinsic description is separate from its extrinsic embedding in space.

The story doesn't end with matrices. In the age of big data, we often encounter datasets with more than two aspects, which are naturally represented not as tables (matrices) but as multi-dimensional arrays, or ​​tensors​​. Think of a collection of videos (height $\times$ width $\times$ time $\times$ videos) or brain activity data (sensors $\times$ time points $\times$ subjects).

To find patterns in such data, we can use methods like the ​​Tucker decomposition​​, which is a higher-order analogue of PCA. It decomposes a large tensor $\mathcal{X}$ into a smaller "core" tensor $\mathcal{G}$ and a set of factor matrices $A^{(n)}$ for each dimension. And, as you might now guess, this decomposition is not unique. For each mode, we can transform the factor matrix, $\mathbf{A}'^{(n)} = \mathbf{A}^{(n)}\mathbf{M}_n$, as long as we apply an inverse transformation to the core tensor, $\mathcal{G}' = \mathcal{G} \times_n \mathbf{M}_n^{-1}$. The full tensor $\mathcal{X}$ is perfectly reconstructed. The ambiguity we saw with a single rotation matrix in factor analysis now blossoms into a richer, multi-faceted non-uniqueness, with a separate invertible transformation possible for each dimension. The standard way to reduce this ambiguity is to require the factor matrices to have orthonormal columns, which restricts the arbitrary transformations $\mathbf{M}_n$ to be orthogonal (rotation) matrices. Yet, the indeterminacy persists.

So far, our journey might seem a bit disheartening. It feels as if we can never find the "true" answer. But this is the wrong way to look at it. Recognizing this freedom is the first step toward mastering it. The indeterminacy is not a bug; it's a feature we can exploit.

The first lesson is one of caution. As we've seen, you must be extremely careful when comparing results from different models or datasets. The "first component" is just a label for an arbitrary vector chosen from a potentially infinite family of equally good vectors.

The second, more exciting lesson is that we can use this freedom to our advantage. Since the algorithm's default choice of rotation is arbitrary, why not choose a rotation that is meaningful to us? This is the idea of ​​targeted rotation​​ or ​​supervised analysis​​.

Let's return to the world of kernels and machine learning with ​​Kernel PCA​​. Suppose we find two principal components whose eigenvalues are nearly identical, giving us a 2D subspace with rotational ambiguity. The standard algorithm gives us two arbitrary orthogonal directions, $\mathbf{u}_1$ and $\mathbf{u}_2$. Now, suppose we also have some external information we care about—for example, a "target" label $y$ that tells us whether each data point belongs to a control group or a treatment group. Instead of passively accepting the arbitrary directions $\mathbf{u}_1$ and $\mathbf{u}_2$, we can ask: what is the specific direction within the subspace spanned by $\mathbf{u}_1$ and $\mathbf{u}_2$ that best aligns with our target $y$? We can solve this optimization problem to find the ideal rotation that makes our component maximally relevant to the scientific question at hand. We have tamed the beast, turning a source of ambiguity into a tool for discovery.

Our exploration of rotational indeterminacy has taken us from simple geometry to the frontiers of data science. We've seen that the descriptions our models provide are often just one perspective among many. Understanding this principle liberates us from the "tyranny of the basis"—the mistaken belief that any single output is the final truth. It encourages a more sophisticated, nuanced view of our models and, most powerfully, gives us the freedom to rotate our perspective until the world snaps into a clearer, more meaningful focus.

There is a profound beauty in discovering that a single, simple idea can echo through the halls of nearly every branch of science. The notion of rotational indeterminacy is one such echo. It begins as a straightforward geometric observation: the description of an object can be invariant under rotations. A sphere looks the same no matter how you turn it. But this simple idea blossoms into a deep and recurring challenge whenever we try to interpret data or model the world. The universe, it seems, often presents us with phenomena whose intrinsic properties are clear, but whose absolute orientation in some abstract or physical space is hidden from us. Our measurements are often blind to a "which way is up," and teasing out that information is a grand scientific detective story. This story takes us from the abstract world of statistical factors to the tangible reality of molecular machines.

Let us begin in the abstract realm of statistics, where we often hunt for hidden causes behind observable effects. Imagine an educational psychologist studying student performance. They collect scores from dozens of tests—vocabulary, algebra, geometry, reading comprehension, spatial reasoning, and so on. They notice that students who do well in algebra also tend to do well in geometry, and those good at vocabulary are often good at reading. This suggests that the myriad test scores are not all independent; they might be driven by a smaller number of underlying, latent "factors," such as "mathematical ability" and "verbal ability."

This is the goal of a statistical technique called Factor Analysis. We model the observed data as a combination of these hidden factors. The trouble starts the moment we find a set of factors that successfully explains the correlations in our data. It turns out that this solution is not unique. We can take our abstract "factor space" and rotate our factor axes. The newly rotated factors are different combinations of the old ones, yet they explain the observed test scores just as perfectly. This is rotational indeterminacy in its purest, most mathematical form. Which set of factor axes is the "right" one? Mathematically, none of them are privileged.

This leaves the scientist in a predicament. If we can't pin down the factors, how can we give them meaningful names like "verbal ability"? The answer is that we must add new information or new assumptions to "fix the rotation." One popular approach, known as Varimax, is to rotate the axes until they provide the "simplest" possible description of the data—for instance, making each test score load heavily on only one factor and weakly on all others. This is an aesthetic choice, a sort of mathematical Occam's razor.

A more powerful approach, used in what's called confirmatory factor analysis, is to use prior theory. A researcher might define "verbal ability" by creating a target structure based on previous knowledge and then rotating the new solution to align with this fixed target. This method, called Procrustes rotation, is like using an external map to orient ourselves, ensuring that the meaning of "Factor 1" remains consistent from one study to the next. The choice of how to resolve the ambiguity is not a mathematical trick; it is a profound statement about the scientific goal, whether it is exploratory (let the data suggest a simple structure) or confirmatory (test a pre-existing hypothesis). Not all constraints are created equal, either. Some, like requiring the columns of the loading matrix to be orthogonal or setting a specific loading to zero, can successfully lock down the rotation. Others, like constraining a variable's total variance explained by the factors (its "communality"), are useless because the communality itself is invariant under rotation—it doesn't change as you spin the factors, and so provides no leverage to stop the spinning.

This problem of un-mixing becomes more concrete in the world of signal processing. Consider the classic "cocktail party problem": you are in a room with two microphones recording two people speaking simultaneously. Each microphone records a different mixture of the two voices. Can you computationally separate the two original, clean voice signals from the two mixed recordings? This is a physical analog of factor analysis, where the hidden "factors" are the original voices. A first attempt might be to use Principal Component Analysis (PCA), a cousin of factor analysis that finds orthogonal directions of maximum variance in the data. However, if the sources were mixed in a non-orthogonal way (as is almost always the case), PCA will fail. It finds a set of decorrelated signals, but these are just different mixtures, not the original voices. It is still trapped by rotational ambiguity.

The solution comes from a more powerful technique: Independent Component Analysis (ICA). ICA works because it makes a stronger, more physically motivated assumption: the source signals are not just uncorrelated, but statistically independent. This is a much stricter condition that utilizes higher-order statistics beyond simple variance. By searching for the rotation that makes the output signals as independent as possible, ICA can break the ambiguity and, like magic, recover the original voices. It's a beautiful demonstration of a core principle: the richer your physical assumptions, the more ambiguity you can resolve.

The challenge of separating a mixture into its pure components is central to the physical sciences. Let's step into a chemistry lab where a reaction, $A \rightarrow B \rightarrow C$, is taking place in a solution. An analytical chemist monitors the reaction by measuring its full absorption spectrum at many points in time. The collected data forms a matrix, where each row is a spectrum at a specific time. Each measured spectrum is a linear combination of the unknown spectra of the pure species $A$, $B$, and $C$, weighted by their unknown, time-varying concentrations. The goal of Multivariate Curve Resolution (MCR) is to decompose this data matrix into a matrix of pure concentration profiles and a matrix of pure species spectra.

And there it is again, our ghost in the machine. The factorization is not unique. An infinite family of mathematically equivalent solutions exists, all related by a "rotation" or, more generally, an invertible linear transformation. As scientists, we can bring physical knowledge to bear. We know that concentration cannot be negative, and for absorption spectroscopy, the spectral absorbance values also cannot be negative. Imposing these non-negativity constraints drastically shrinks the family of possible solutions. But remarkably, it often isn't enough to force a single, unique answer. The rotational ambiguity can persist even under these strong physical constraints.

To truly conquer the ambiguity, we must bring in our deepest knowledge of the system: a mechanistic model. Instead of just saying concentrations are non-negative, we can write down the differential equations—the laws of chemical kinetics—that describe exactly how the concentrations of $A$, $B$, and $C$ must change over time. In a sophisticated technique known as global target analysis, we can then rotate the abstract mathematical solution until its kinetic part perfectly matches the evolution predicted by our kinetic model. The laws of physics itself provide the ultimate anchor, transforming an ill-posed decomposition into a well-posed measurement of nature.

This same principle extends to the modern world of big data, which often comes in the form of tensors—multi-dimensional arrays that are generalizations of matrices. A popular model for analyzing such data is the Tucker decomposition, which breaks a tensor down into a set of factor matrices and a smaller "core" tensor that describes their interactions. Just like in factor analysis, the factor matrices are not unique; they are subject to rotational ambiguity. What is uniquely determined are not the individual factor vectors, but the subspaces they span. This stands in contrast to other tensor models, like the CP decomposition, which is typically unique. Understanding this rotational freedom is crucial, as it tells us that the factors of a Tucker model represent collective modes, not individual latent variables, unless we impose further constraints to fix the rotation.

Thus far, our "rotations" have been in abstract data spaces. But the problem is born of, and returns to, the geometry of the physical world. Imagine a swarm of autonomous drones tasked with assembling a specific formation in the sky, using only distance measurements to their immediate neighbors. They can successfully arrange themselves into the correct shape—say, a giant, rigid cube. But is it the correct cube? The entire formation could be shifted ten meters to the left, or rotated 30 degrees, or even be a mirror image of the target shape, and every single inter-drone distance would still be perfectly correct. The system is ambiguous up to a rigid-body motion.

How do we solve this? We provide anchors. If we command one drone to go to a fixed GPS coordinate, we eliminate the translational ambiguity. The swarm is now tethered, but it can still rotate freely around that anchor point. If we then command a second drone to another fixed coordinate, the rotational freedom is eliminated. The swarm can no longer spin. Yet, one ambiguity remains: the formation could be a reflection of the true one across the line connecting the two anchors. Only by anchoring a third, non-collinear drone do we eliminate all ambiguity and lock the formation into a single, unique state in space. This provides a wonderfully tangible picture of how specifying information at a few key points can remove the global degrees of freedom.

This same logic appears in reverse in computational chemistry. To run a simulation of a molecule, we must first tell the computer where the atoms are. To define the absolute position of a single atom requires a reference frame. We can build this frame using a Z-matrix, a standard tool that defines atoms via internal coordinates (distances and angles) relative to previously defined points. To establish an absolute frame from scratch, we need three "dummy atoms": the first sits at the origin, fixing translation; the second is placed along an axis, fixing two rotational degrees of freedom; and a third, non-collinear point fixes the final rotation about that axis. We must explicitly build an unambiguous frame before we can even begin. A similar issue arises in solid mechanics: if we know the strain field within a deformed object, we can only reconstruct its final shape up to an arbitrary rigid rotation and translation. To find the unique deformed shape, we must specify boundary conditions, such as clamping one end of the object, which fixes its position and orientation.

Perhaps the most dramatic and subtle manifestations of this problem occur at the frontiers of structural biology. Scientists using cryo-electron microscopy (cryo-EM) and tomography (cryo-ET) seek to determine the 3D atomic structure of life's essential molecules, like proteins and viruses. This is done by taking many 2D projection images of flash-frozen molecules from different angles and computationally reconstructing the 3D volume. Here, rotational ambiguity appears in two devastating forms.

First is the "handedness problem." Most biological molecules are chiral; they are different from their mirror image, just like your left and right hands. During data processing, the algorithm must determine the orientation of each 2D projection. If it systematically confuses a "front" view with a "back" view (a 180-degree flip), the final 3D reconstruction will be a perfect mirror image of the true structure. For a biochemist, this is a fatal error, akin to mistaking a right-handed screw for a left-handed one.

Second, and more subtly, is the "missing wedge" artifact in cryo-ET. Due to physical limitations of the microscope, it is impossible to tilt the sample through a full 180-degree range. This results in a wedge-shaped blind spot in the 3D Fourier transform of the object. This missing wedge is itself rotationally symmetric about the axis parallel to the electron beam. The devastating consequence is that the data itself becomes insensitive to the particle's rotation around this very axis. Rotating the particle just shuffles information around within the blind spot, producing almost no change in the data. The experiment itself has a built-in rotational ambiguity, a direction in which nature hides the particle's true orientation from us.

From abstract factors to tangible drones and the very molecules of life, we see the same story. The raw data of our measurements or the basic laws of a system are often not enough. They leave a rotational freedom, an ambiguity that prevents a unique interpretation. To find the "truth," we must always bring more to the table: a theoretical assumption, a physical law, a statistical property, or a clever experimental design. Recognizing this shared challenge is more than a technical exercise; it is a deep insight into the process of scientific discovery itself. It is a lesson in the beautiful and creative ways we have learned to find our bearings in a universe that doesn't always tell us which way is up.