Safety Of The Intended Functionality (SOTIF)

As autonomous systems become more integrated into our daily lives, from cars that drive themselves to robots that work alongside us, ensuring their safety presents a profound engineering challenge. For decades, safety was synonymous with preventing failure; a system was safe if its components did not break. However, this traditional view, known as Functional Safety, is no longer sufficient. We are now faced with a new class of risk where a system can behave dangerously even when every sensor, computer, and line of code is executing perfectly as designed. This gap in our understanding—the problem of a "perfect" system creating an imperfect outcome—is one of the most critical hurdles in the development of trustworthy AI and automation.

This article introduces the Safety Of The Intended Functionality (SOTIF), a modern safety science developed to address this very challenge. It provides a framework for understanding, analyzing, and mitigating risks that stem not from faults, but from the inherent limitations of a system's design when interacting with a complex and unpredictable world. In the following chapters, we will embark on a journey to understand this crucial paradigm shift. First, we will explore the core "Principles and Mechanisms" of SOTIF, defining its relationship to functional safety and introducing key concepts like the Operational Design Domain (ODD). Following that, we will examine its "Applications and Interdisciplinary Connections," discovering how SOTIF provides a practical toolkit for engineers using simulation, data science, and machine learning to build and certify the safe autonomous systems of the future.

To truly grasp the safety of modern autonomous systems, we must venture beyond our traditional understanding of what it means for something to "fail." For centuries, safety engineering was largely about preventing things from breaking. A bridge collapses because a steel beam rusts through. A car crashes because its brake line snaps. The logic is simple: a component fails to perform its function, leading to a hazard. This realm of safety, which focuses on preventing hazards caused by faults and malfunctions, is known as ​​Functional Safety​​. It’s a mature and vital discipline, concerned with building robust systems that can withstand random hardware failures or systematic design errors. Imagine a glitch on a sensor's communication bus causes a stream of data to momentarily drop out; a well-designed functionally safe system would detect this anomaly and execute a safe fallback, like a controlled stop.

But what happens when nothing is broken? What if every wire is intact, every chip is functioning perfectly, and every line of code is executing exactly as intended, yet the system still behaves in a dangerous way? This is a more subtle, and in many ways more profound, challenge. Think of a supremely skilled human driver. They are not immune to accidents. They can be momentarily blinded by sun glare, their view can be obscured by dense fog, or they can be startled by a child darting out from behind a parked car. Their eyes, brain, and limbs are all functioning perfectly, but their intended functionality—driving safely—has inherent limitations when faced with a challenging world.

This is the heart of a new frontier in safety science, known as ​​Safety Of The Intended Functionality (SOTIF)​​. It addresses the risks that arise not from malfunctions, but from the inherent limitations of a system's intended behavior. Consider an autonomous vehicle's perception system. If it misinterprets a novel roadwork configuration because its machine learning model has never seen that specific pattern before, it's a SOTIF issue. If its camera sensor saturates in low-sun glare, consistent with its design specifications but leading to poor detection performance, it's a SOTIF issue. Even if an occupant allows slush to build up on a sensor, and the system—lacking a sufficiently robust self-check—continues to operate until a near-miss occurs, this too falls under the umbrella of SOTIF as a "reasonably foreseeable misuse". In all these cases, the system is not broken; it is simply insufficient for the task at hand. This crucial distinction is the bedrock of modern safety analysis for autonomous systems.

To see how a perfectly functioning system can lead to a hazard, let's build a simple mental model of a perception system, much like the one explored in a thought experiment. Imagine an automated emergency braking system. A sensor produces a continuous signal, let's call it $X$. The system's rule is simple: if the signal $X$ is greater than a certain threshold $t$, it means an obstacle is detected, and the brakes are applied. If $X \le t$, it assumes the path is clear.

Now, suppose an obstacle is truly present. In a perfect world, it would generate a strong, clear signal. But our world is not perfect. The signal can be weakened by environmental factors. Perhaps the obstacle is partially hidden behind another object, or obscured by rain or fog. We can represent this environmental challenge with a variable for "occlusion," $o$. The more occlusion, the weaker the signal. So, the signal's strength might be modeled as $\mu - k o$, where $\mu$ is the signal strength in ideal conditions and $k o$ represents the signal loss due to occlusion.

Furthermore, no real-world measurement is perfectly clean; there's always a small amount of random electronic noise, $\sigma$. So the final signal the computer sees is $X = (\mu - k o) + \text{noise}$.

Herein lies the mechanism of a SOTIF failure. On a particularly foggy day, the occlusion $o$ is very high. The signal strength $\mu - k o$ becomes dangerously weak. Now, it only takes a small, random fluctuation of noise in the negative direction for the final signal $X$ to dip below the decision threshold $t$. The computer, following its rules with perfect fidelity, sees that $X \le t$ and concludes, "No obstacle." It does not brake. A collision becomes inevitable.

No part of the system has failed. The sensor measured what it measured. The computer executed its logic flawlessly. The hazard arose from the intersection of a specific, challenging environmental condition (high occlusion) and an inherent performance limitation in the system's design (the fixed threshold $t$). The probability of this "perfect failure" is not zero; it is a calculable risk, $P_H$, that depends on how often obstacles appear ($\pi$), the probability distribution of different weather conditions ($p(o)$), and the system's own design parameters ($\mu$, $k$, $\sigma$, $t$). This simple model beautifully illustrates that hazards can be an emergent property of a system's interaction with its environment, entirely in the absence of faults.

If a system cannot be perfectly safe in all conditions, how can we ever trust it? The answer is that we must draw a line. We must create a contract that defines the boundaries within which the system is expected to operate safely. This set of specified conditions is called the ​​Operational Design Domain (ODD)​​.

An ODD is defined by a set of measurable variables. For an autonomous valet parking system, for instance, the ODD might be specified as: forward speed $v \in [0, 12]\,\mathrm{m/s}$, ambient illumination $L \in [2000, 100000]\,\mathrm{lux}$ (daylight, not nighttime), precipitation $P \in \{\text{dry}, \text{light rain}\}$ (not a blizzard), and so on.

The ODD is the manufacturer's promise. They are making a claim of safety within this specified envelope. A SOTIF analysis, therefore, focuses intensely on ensuring the risk is acceptable inside the ODD's boundaries. A scenario happening outside these boundaries, such as the valet system trying to operate at night with illumination below $2000$ lux, is not a failure of the intended functionality, because the system was never intended for that function. The ODD provides the fundamental context for any and all safety claims.

With the ODD defined, we have a map of the world where our system is supposed to be safe. But this map is vast and multidimensional. We cannot possibly test every single point on it. We can test the common scenarios—driving on a sunny day, parking in a well-lit garage—but what about the strange and rare corner cases? The "here be dragons" written in the uncharted corners of our map. These are the ​​triggering conditions​​ that can spark a SOTIF hazard: a bizarre reflection from a wet road at sunset, a uniquely shaped shadow that mimics a pedestrian, or a bicycle partially obscured by a steaming manhole cover.

This is the "long-tail" problem, and physically exploring it is impossible. This is where modern simulation tools and ​​digital twins​​ become absolutely essential. A digital twin is a hyper-realistic virtual replica of the autonomous system and its environment. Within this digital realm, engineers can systematically hunt for dragons. They can conjure any scenario imaginable—dialing up the fog, creating blinding glare, spawning complex traffic—all to discover the specific triggering conditions that push their perfectly functioning system over the edge.

This capability enables a powerful, data-driven approach to safety engineering: scenario-based risk assessment. Engineers can identify classes of known risky scenarios within the ODD (e.g., "curved ramp with daytime glare"), use the digital twin to estimate the probability of a hazard within that scenario ($\lambda_i$), and combine it with real-world data on how frequently that scenario occurs ($n_i$) to calculate a portion of the total risk.

Yet, the most profound challenge remains: what about the parts of the map we haven't explored? The "unknown-unsafe" regions, the dragons we don't even know exist. This epistemic uncertainty, the risk from the scenarios we haven't identified, is what keeps safety engineers awake at night. We can never eliminate it, but we can manage it.

This is the final, crucial piece of the SOTIF puzzle. Engineers perform massive-scale random testing across the ODD, often in simulation. If they run, say, $100,000$ random test cases and observe zero hazardous events, they do not naively conclude the risk is zero. Absence of evidence is not evidence of absence. Instead, they turn to statistics. They can state, for example, "With $95\%$ confidence, the true probability of a hazard in these unknown regions is no more than a very small number, $p_u$." A common approximation, the "rule of three," suggests this upper bound is about $3/N$, or $3/100,000$ in this case. This conservatively bounded risk from the unknown is then added to the risk from the known scenarios. If this total risk is below a pre-defined acceptable threshold, we can finally make a credible, evidence-based argument that the system is acceptably safe.

This journey—from distinguishing faults from limitations, to modeling performance, to defining operational boundaries, and finally to exploring and rigorously bounding the unknown—is the science and art of SOTIF. It represents a philosophical shift in safety engineering: an acceptance that perfection is impossible, and that true safety lies not in eliminating all failures, but in deeply understanding and rigorously managing the inherent limitations of our increasingly intelligent creations.

Now that we have explored the principles of Safety Of The Intended Functionality (SOTIF), we might be tempted to file it away as a collection of abstract rules. But to do so would be to miss the point entirely. SOTIF is not a theoretical curiosity; it is a wrench, a microscope, and a compass for the engineers building our future. It is a practical philosophy that is actively reshaping how we design, test, and, most importantly, trust the complex autonomous systems that are beginning to inhabit our world.

The principles of SOTIF ripple outwards, forging powerful connections with disciplines that might at first seem distant: data science, simulation technology, cybersecurity, and even the formal language of legal and regulatory certification. Let us take a journey through these connections to see how the ideas of SOTIF come to life.

Imagine the challenge of proving an autonomous car is safe. How many miles must it drive? A million? A billion? The sobering truth is that the number of miles required to statistically demonstrate safety from rare, catastrophic events is so astronomically large that it is impossible to achieve through physical road testing alone. We would be waiting decades, perhaps centuries, for the verdict.

Engineers, therefore, turn to a digital world—a high-fidelity simulation, often called a "digital twin," that serves as a virtual proving ground. But this raises a profound question: if we can't fully predict the behavior of the real system in the messy real world, how can we possibly trust its digital copy? This is where SOTIF provides the compass. Instead of blind faith in the simulation, we build a formal "acceptance argument" for it. This is not a matter of simply showing the simulation "looks good." It is a rigorous, scientific case built on SOTIF principles. Engineers must create a sub-claim within their safety case that the digital twin itself is fit for its purpose. This involves validating the simulation against targeted physical experiments, proving that the discrepancy between the twin's predictions and reality is understood and bounded. For instance, the twin's prediction of stopping distance might be proven to be accurate within a few centimeters under all relevant conditions. It involves quantifying all sources of uncertainty—from the model's physics to the parameters used—and propagating that uncertainty to the final result.

The goal is to move from a statement like "The simulation shows the car is safe" to a much more honest and powerful one: "We have validated this simulation to a specified degree of accuracy, and after running a statistically significant number of tests that account for all known uncertainties, the $95\%$ upper confidence bound on the dangerous failure rate is below the required safety target." This rigorous approach, which demands tool qualification, independent verification, and traceability, is the only defensible way to use simulation as a primary source of safety evidence for a high-risk system.

With such a validated twin, we can now use it as a SOTIF laboratory. Imagine a warehouse robot, previously confined to a pristine, predictable indoor environment, is now being prepared for work in an outdoor loading bay. This change in the Operational Design Domain (ODD) is a classic SOTIF challenge. New phenomena appear: rain slicking the pavement, wind gusts pushing the robot, and the glare of the sun blinding its sensors. Simply adjusting the old simulation's parameters for friction or light levels would be a dangerous mistake. SOTIF thinking demands that we recognize the physics of the environment has fundamentally changed. The digital twin's dynamic models must be updated to include these new forces and sensor failure modes. The entire safety analysis—from hazard identification to fault tree analysis—must be revisited from first principles. The trusted digital twin allows engineers to explore these new "known unknowns" and discover the "unknown unknowns" in the safety of simulation, long before the physical robot turns a single wheel in the rain.

Perhaps the greatest SOTIF challenge lies within the "brains" of modern autonomous systems: the machine learning (ML) models that perceive and interpret the world. An ML perception system—the part that identifies pedestrians, traffic lights, and obstacles—doesn't fail because a transistor burns out. It fails when it encounters a situation it wasn't adequately trained for, a scenario where its performance is insufficient. This is the very definition of a SOTIF hazard.

How, then, do we build confidence in an ML model? A simple metric like "$99\%$ accuracy" on a test dataset is close to meaningless from a safety perspective. Why? Because that dataset might not contain the rare, confusing, and dangerous "corner cases" that matter most. Think of it like studying for an exam. A student who only practices the most common questions may get a good score but will be stumped by the one tricky question that truly tests their understanding. SOTIF demands that we become expert hunters for those tricky questions.

To do this, engineers are developing sophisticated ODD coverage metrics. Instead of treating all test scenarios equally, they create a risk-based map of the world. Using a digital twin, they can estimate both the probability of a certain scenario occurring (e.g., driving in fog at dusk) and the inherent hazard of that scenario. The goal of testing then becomes to achieve a high degree of hazard-weighted coverage. We intentionally focus our testing resources on the scenarios that contribute the most to the total potential risk, even if they are rare. This risk-based approach ensures that we are not just building a system that works most of the time, but a system that doesn't fail in the moments that matter most.

This risk-based view provides a powerful, unified framework for analysis. We can treat different sources of risk—whether from the inherent limitations of an ML model or from a malicious cybersecurity attack—within the same mathematical language. Consider a system facing two hazards: a random misclassification by its perception system, and the possibility of a malicious data poisoning attack that corrupts its training data. Using a quantitative SOTIF approach, we can model both of these as hazard rates, each with an associated severity. We can then precisely calculate how different safety measures affect the total residual risk. For example, a data management control (like cryptographically signing training data) might reduce the occurrence rate of the data poisoning hazard. In contrast, a runtime monitor that detects strange behavior might not change the rate at which it occurs but dramatically reduces the effective severity by triggering a safe stop. This quantitative lens allows engineers to make rational, data-driven decisions about where to invest their efforts to achieve the greatest reduction in risk, elegantly bridging the disciplines of safety, SOTIF, and cybersecurity.

After all the analysis is done, the simulations are run, and the risks are calculated, one final task remains: convincing the world that the system is safe. This is not a marketing exercise; it is the ultimate application of SOTIF, where engineering rigor meets the social and legal requirements of certification.

What does "proof" of safety look like? The SOTif mindset has taught us what it is not. It is not a marketing video showing a flawless demonstration. It is not an anecdote from an expert. It is not even a simple statement of "we drove a million miles without an accident". Raw mileage, without context about the driving conditions (the ODD) and without a statistical analysis to place confidence bounds on the result, is just a large number, not evidence.

Instead, acceptable safety integrity evidence is a structured, auditable collection of artifacts. It is a bundle containing the results of a formal hazard analysis, the validation report for the digital twin, the test results from a simulation campaign with documented ODD representativeness, and the quantitative analysis showing that safety goals have been met with a high degree of confidence.

The final expression of this entire body of work is the ​​assurance case​​. An assurance case is the grand, coherent argument that connects a top-level claim (e.g., "The system is acceptably safe to operate in its ODD") to the mountain of supporting evidence. Notations like Goal Structuring Notation (GSN) provide a graphical grammar for this argument, creating a clear, traceable map from the highest goals, through the strategies used to achieve them, down to the specific solutions—the reports, analyses, and test data that form the bedrock of evidence.

This assurance case is the ultimate embodiment of the SOTIF philosophy. It forces engineers to be transparent about their assumptions, explicit about their reasoning, and rigorous with their evidence. It is the story they tell the regulator—and themselves—to build justified confidence in the systems they create. In this way, SOTIF provides more than just a method for engineering safer systems; it provides a language for communicating trust.