Safety is a cross-layer property whose representation changes with the level of abstraction, while its underlying objective remains the same. A unified framework bridges systematic hazard analysis and runtime enforcement using hazard-informed urban safety envelopes that represent the range of operating conditions under which an autonomous robot or vehicle can safely interact with its environment.
Introduction
The integration of robotics into urban mobility is shifting from controlled industrial environments to public spaces. Robotics is moving onto city streets, changing how people, goods, and services move, giving rise to three dominant use cases:
- Sidewalk autonomous delivery robots: Small, electric, cooler-sized robots equipped with cameras, GPS, and LiDAR travel on sidewalks at pedestrian speeds to deliver groceries and takeout.
- Autonomous vehicles and robotaxis: Passenger-carrying vehicles are transitioning from pilot phases to scale and operating continuously.
- Autonomous road-maintenance robots: Robots can patrol city streets at night to detect asphalt cracking and seal it immediately.
Deploying these heterogeneous robotic systems alongside human agents poses severe safety, operational, and regulatory challenges. Each domain occupies a distinct urban zone and uses a unique safety paradigm to prevent physical harm and minimize conflict.
Sidewalk robots operate on pedestrian pathways. Their safety is ensured by limiting kinetic energy through ultra-low speeds and relying on “yield-to-all” behavior trees.
Robotaxis are passenger-carrying vehicles that require strict, multilayered hardware and sensor redundancies—including cameras, LiDAR, and radar—alongside deterministic safety-envelope algorithms. The systems must conform to rigorous behavioral standards.
Road-maintenance robots operate in high-risk, unstructured construction zones. Their safety relies on real-time geofencing, vehicle-to-everything communication, and active sensor-clearing mechanisms to prevent collisions with oncoming traffic or nearby human crews.
Yet safety is not a static attribute of a single software component. Instead, it is a system-level feature that evolves as information and decisions flow through different layers of the system design.
For example, a pedestrian is represented as a vulnerable road user at the symbolic layer, as a dynamic obstacle during trajectory planning, and as a moving object with position and velocity at the control layer. Although these representations differ, they all capture the same safety objective: preventing harmful interaction with the pedestrian.
Urban Safety Envelope
Safe urban mobility requires a broad, universal framework aimed at maintaining sufficient safety margins before dangerous situations develop.
The urban safety envelope is learned using a hazard-informed pipeline. Rather than collecting only normal driving data, synthetic training scenarios are generated specifically around known safety-critical situations. Each scenario is derived from explicitly defined assets, exposure modes, and hazard scenarios, ensuring that the generated data focuses on conditions where safety margins are reduced.
For example, an urban intersection may include pedestrians, cyclists, parked vehicles causing visual occlusions, adverse weather, and varying traffic densities. By systematically varying these factors, the simulation generates scenarios ranging from completely safe situations to conditions that approach safety limits. This allows machine learning models to observe the gradual transition from normal operation to hazardous situations instead of learning only from rare collision events.
The resulting models continuously estimate how close the current driving situation is to violating predefined safety constraints. If the estimated safety margin becomes too small, the autonomous system can respond proactively by:
- Reducing speed
- Increasing following distance
- Selecting an alternative trajectory
- Transferring control to a higher-level safety controller
Unlike conventional hazard detection, which reacts after a dangerous situation has already emerged, the proposed approach enables predictive safety assessment. The objective is not simply to recognize hazards but to identify when the vehicle is approaching the boundary of safe operation.
An additional advantage of the proposed approach is its interpretability. Because every scenario originates from an explicitly defined hazard within the safety engineering process, the behavior learned by the model can be traced back to specific assets, vulnerabilities, and hazard scenarios identified during risk analysis. This improves transparency and provides stronger engineering evidence for safety validation and certification.
Safety Across Hierarchical World Models
Autonomous urban mobility requires reasoning across multiple levels of abstraction. No single representation of the environment is sufficient to support strategic decision-making, trajectory planning, and real-time vehicle control simultaneously. Instead, autonomous systems employ a hierarchy of world models, where each layer maintains a specialized representation of the environment that captures only the information relevant to its function.
From a safety perspective, each layer defines its own notion of safe operation. Consequently, urban safety cannot be represented by a single model but emerges from the consistent interaction of multiple safety abstractions across the control hierarchy.
Symbolic Safety Layer
The highest layer represents the environment using symbolic entities, relationships, and operational rules. At this level, the system reasons about mission objectives, traffic regulations, legal constraints, and ethical requirements rather than physical geometry.
Safety is expressed in terms of symbolic constraints, such as respecting pedestrian priority, obeying traffic signals, avoiding restricted areas, and complying with operational policies. The environment is therefore viewed as a structured network of permissible and prohibited actions rather than a physical space.
Within this representation, a pedestrian is treated as a vulnerable road user whose presence imposes behavioral constraints on the autonomous system.
Safety rules can be visualized as directed relationships, such as Yield and Stop, between entities.
Spatial Safety Layer
The intermediate layer transforms symbolic objectives into geometric motion plans. The environment is represented as a spatial map containing free space, obstacles, traversability costs, and predicted motion of surrounding traffic participants.
Safety is expressed through spatial relationships, including collision probability, minimum clearance distances, visibility constraints, and safe trajectory generation. Dynamic objects are represented by their predicted occupancy rather than by their semantic meaning.
Consequently, the pedestrian is no longer viewed as a legal entity but as a dynamic obstacle whose future motion influences the selection of a safe path.
The continuous environment is discretized into a grid to compute a safe geometric path.
Dynamic Safety Layer
The lowest layer governs the physical interaction between the vehicle and its environment. Here, the world is represented by continuous dynamics, actuator limits, and physical constraints.
Safety is defined by maintaining stable behavior while respecting limits on steering, braking, acceleration, tire-road friction, and actuator capabilities. The controller is not concerned with traffic regulations or navigation goals but with ensuring that commanded trajectories remain physically achievable.
Within this representation, a pedestrian is abstracted as a moving object characterized by position, velocity, and relative motion, enabling the controller to compute safe control actions in real time.
Safety Transformation Across Layers
The three world models described in the previous section represent the same physical environment but at different levels of abstraction. As information propagates downward through the hierarchy, safety is progressively refined from high-level symbolic objectives into low-level physical commands. Conversely, feedback from the real world is sent upward, updating representations at each layer and enabling closed-loop adaptation.
This bidirectional flow ensures that the urban safety envelope is not confined to any single layer but emerges from the consistent intersection of symbolic, spatial, and dynamic safety constraints.
Forward Transformation: From Rules to Actions
At the symbolic layer, safety is encoded as a set of mission objectives, traffic regulations, ethical principles, and operational policies. Examples include yielding to pedestrians at crosswalks, respecting speed limits, and avoiding restricted zones. These rules define permissible high-level behaviors but do not specify how to execute them geometrically or physically.
The spatial layer translates these symbolic constraints into geometric safety margins and collision-free trajectories. It reasons about the environment in terms of occupancy, clearance distances, and predicted motion of other agents.
For instance, the rule “yield to pedestrian” is transformed into a requirement to maintain a minimum lateral distance of 0.5 meters and to stop before the crosswalk if the pedestrian is within 2 seconds of arrival. The planner computes a path that satisfies these spatial margins while optimizing for efficiency and comfort.
The dynamic layer then converts the planned trajectory into physically feasible control commands—steering, braking, and acceleration—that respect actuator limits, tire-road friction, and vehicle dynamics. At this stage, safety is expressed as maintaining stability and tracking the reference trajectory within prescribed bounds.
Feedback Transformation: From Execution to Awareness
Sensor measurements update the dynamic model with information about actual vehicle behavior, environmental disturbances, and unexpected obstacles. This updated physical model informs the spatial planner about deviations from the expected path and changes in the surrounding scene.
If the planner detects that the original trajectory is no longer collision-free, such as because of a suddenly braking vehicle, it revises the geometric plan accordingly. In turn, persistent or significant deviations may trigger a revision of symbolic decisions—for example, aborting a lane change and reverting to a safe pull-over maneuver.
This closed-loop feedback ensures that safety remains responsive to the evolving real-world situation.
Emergence of the Urban Safety Envelope
The urban safety envelope is therefore the intersection of constraints propagated through all three layers. A state is safe only if it simultaneously satisfies symbolic rules, spatial collision-free criteria, and dynamic feasibility bounds.
Learning this envelope from hazard-informed synthetic scenarios enables the autonomous system to anticipate when it approaches the boundary of safe operation. Because the training data is systematically generated around known hazard types, the learned model captures the gradual degradation of safety margins, allowing proactive intervention before a violation occurs.
Consider a sidewalk delivery robot approaching an intersection with a pedestrian. Symbolically, the robot must yield to any crossing pedestrian. Spatially, the planner computes a path that stops before the blind corner, maintaining a buffer zone. Dynamically, the controller applies smooth deceleration within the motor’s torque limits.
If the pedestrian suddenly appears closer than expected, sensor feedback updates the dynamic state, prompting the spatial planner to recalculate a tighter stop trajectory, and the controller adjusts braking force accordingly. This continuous interplay exemplifies the safety transformation across layers.
In summary, the hierarchy enables each layer to reason using the most suitable abstraction for its computational role, while the coherent propagation of safety information ensures that the overall system remains safe at all times.
Runtime Safety Enforcement Through the Physical AI Harness
The previous sections describe how safety is represented and progressively transformed across multiple abstraction layers. However, preserving safety semantics during execution requires more than correct perception, planning, and control. Once machine learning models begin generating decisions in real time, an additional mechanism is required to ensure that their outputs remain consistent with the safety constraints established during system design.
The Physical AI Harness is a runtime software layer responsible for governing the execution of AI models within robotic systems. Unlike conventional robot middleware, which primarily coordinates communication between software components, the Physical AI Harness continuously supervises AI-generated decisions to ensure that they remain compatible with the physical capabilities of the robot and the operational requirements of its environment.
The harness is based on three complementary mechanisms: Projection, Isolation, and Transfer, or PIT. Each addresses a different aspect of runtime safety.
Projection
Projection constrains the outputs generated by machine learning models before they are executed by the robotic system. Rather than allowing arbitrary actions, the harness projects the predicted commands onto a predefined region of acceptable behavior.
Within the proposed urban safety framework, this acceptable region corresponds naturally to the urban safety envelope. Steering commands, vehicle speed, lane changes, or trajectory modifications are accepted only if they remain within the safety margins derived from hazard analysis and safety-envelope learning.
Commands that violate these constraints are modified or rejected before reaching the controller.
Projection therefore represents the final validation step between learned decision-making and physical execution.
Isolation
Modern autonomous vehicles execute numerous software components simultaneously, including perception, localization, planning, communication, and control. Machine learning models frequently compete with these components for computational resources, potentially degrading the performance of safety-critical functions.
Isolation prevents AI workloads from interfering with essential vehicle functions by allocating dedicated computational resources and enforcing execution budgets. This separation preserves the deterministic behavior
Frequently Asked Questions
What is an urban safety envelope?
It represents the range of operating conditions under which an autonomous robot or vehicle can safely interact with its environment.
Why are safety envelopes learned from synthetic hazard scenarios?
Hazard-informed scenarios focus training on safety-critical situations and show models how safety margins gradually deteriorate before a violation or collision occurs.
How does safety differ across the three world-model layers?
The symbolic layer handles rules and legal constraints, the spatial layer handles geometry and collision-free paths, and the dynamic layer handles physical feasibility and control limits.
What is the role of the Physical AI Harness?
It supervises AI-generated decisions during execution so they remain compatible with the robot’s physical capabilities and environmental safety requirements.