Validating the AI Driver

At Wayve, we have a deep conviction in AV2.0: autonomy that generalizes across sensors, computers, operational domains, and vehicles. Our data, road experience, and partnerships with Nissan, Stellantis, Mercedes-Benz, and Uber are helping us develop the Wayve AI Driver, designed to bring autonomy’s safety and comfort benefits to more vehicles and places.

Nine years of development have taught us that AV2.0 reaches well beyond the AI Driver, into the tools, the process, and the mindset we use to build and validate safety-critical AI. Many of the verification and validation paradigms that enabled AV1.0 and created this industry are ineffective, insufficient, or impossible to apply to AV2.0. This reality has led us to develop a collection of tools and processes we encapsulate as Validation 2.0.

Validation 2.0 is the companion to AV2.0. It applies the scaling properties of AI to validation itself: a validation system that improves with more data, generalizes across vehicles and operating domains, and produces the evidence a credible safety case needs. Like AV2.0, we believe this toolset is not only required to meet the challenges of frontier physical AI development, but also fundamentally enables the experience the public will want from ubiquitous AV technology: more capable, reliable AV journeys across broader, more complex operational design domains (ODDs), at a lower cost.

Validation 2.0 rests on three technical pillars:

• Highly efficient, low domain gap closed-loop simulation, powered by GAIA, our line of world models.
• Learned and generalizable behavior evaluation through GRADER (Generalizable Reward for Autonomous Driving Evaluation and RL), a data-driven reward model designed to assess driving behavior without relying on a hand-coded rule for every variation.
• Statistical evidence transfer, which determines how much evidence from previous software releases, vehicle platforms, and operating conditions can credibly support the safety argument for a new release.

Validation 2.0 remains strongly grounded in applicable functional safety and SOTIF standards, as our industry demands. We focus on behavioral validation, a major deployment bottleneck, by building credible evidence of the absence of unreasonable risk (AUR).

The rest of this article describes how innovative tooling and processes come together to make validation more scalable, generalizable, and credible for any safety-critical robotic AI system.

Validation is in need of disruption

The autonomous vehicle industry is at an architectural crossroads. Two decades of development have produced the most tangible and consequential manifestation of AI in the real world, and expectations have risen with it. As L2+ personal vehicles and L4 robotaxis reach more people, customers rightfully demand more capable and trustworthy systems. Just as we now expect frontier language models to match or outperform humans on many tasks, we also have high expectations for autonomous drivers.

Validation, the proof of deployment readiness and the safety argument behind it, has not had the same revolution.

Standard practice still rests on large sets of formal requirements, translated into large sets of tests, graded against test-specific metrics, with results aggregated by heuristics that transfer poorly to a new vehicle or operating domain. The resulting spider-web of requirements, tests, metrics, and acceptance criteria often exceeds the complexity of the driving stack under test. Untangling and making sense of this traceability web requires significant engineering teams, who become increasingly bogged down in maintenance versus advancement.

Most problematic, these validation systems do not share the one property that makes the system under test improve: more data. While learned components may exist inside them, the net result does not scale or generalize as expectations and requirements change.

If we want validation to advance at the same rate as the system under test, it has to share the same favorable scaling and generalization properties.

The Modular Myth: Why Component-Level Evidence is Not Enough

As detailed in “How Wayve Builds a Safe AI Driver,” AV1.0 systems are often defined by large separable subsystems–perception, planning, mapping–that suit traditional verification and validation (V&V) well. System requirements flow down to subsystem requirements, then to subsystem-specific tests, metrics, and acceptance criteria. The ability to decompose a system into meaningful parts, evaluate those parts independently, and then extrapolate claims about the entire system appears advantageous on the surface. 

However, strong subsystem-level performance and traceability does not guarantee system-level safety. 

Modular AV1.0 validation grants a false sense of security. While isolated subsystems like mapping and perception can pass their individual tests, dynamic real-world changes—such as roadworks obscuring landmarks and causing perceptual aliasing—can feed confidently incorrect data to the planner. Separability ensures test coverage and understandability, but it fails to guarantee system-level safety.

AV2.0, or any end-to-end AI architecture, does not expose perception, mapping, and planning as independently testable stages in the same way as a modular system. It may learn internal representations of maps, traffic lights and other scene elements, but it does not expose them as separate interfaces that conventional human QA can validate independently. Therefore, a new validation approach is required for AV2.0: one that evaluates the system behavior of the complete system and can scale across safety-critical AI systems.

Our Validation 2.0 framework takes this behavior-first approach.

Evidence of this generalization can be seen in data collected across countries, vehicle platforms, and sources. Although these situations appear different, each requires the same behavioral judgment: to stop. In each case, our offline validation system identified that the vehicle was expected to stop, demonstrating that its grading can generalize without relying on explicit AV1.0 perception features or distance measurements.

Pillar 1: Closed-Loop Simulation

GAIA, our multimodal world model, powers our closed-loop simulation. To evaluate safety in critical situations, GAIA initializes interactive scenarios directly from unstructured real-world video, such as dashcam footage. This lets us evaluate the AI Driver’s performance on long-tail, hazardous events using vast video data captured across diverse fleets.

By evaluating Wayve’s AV stack in closed-loop simulations directly alongside recorded human driving, we can rigorously benchmark performance and build evidence in high-risk operating conditions. Just as we apply active learning techniques to recursively improve the AI Driver, we can use them to search in our data corpus for scenarios where we can improve simulation to reduce the sim-to-real gap as an end-to-end problem. This is crucial to establishing the credibility of GAIA as a simulation tool for validation.

Pillar 2: GRADER – A Learned and Generalizable Critic

Many in this industry would say that grading driving is at least as hard as driving: defining, on an absolute scale and with little error, what separates good driving from bad. What’s more difficult is doing it at scale, across every input source and every geography, capturing what good and bad look like in every country around the world.

State-of-the-art AV1.0 answers this by hand-crafting measures. Elemental scene attributes, derived metrics, and bespoke logic covering compliance, safety, and comfort, re-authored for every city and every market. The result is a heuristic analytics stack that can be more complex and harder to maintain than the driving stack it judges, and it only improves when people extend it.

As such, a central pillar of Validation 2.0 is reward modeling: singular critics capable of grading all driving.

Wayve’s reward model, GRADER (Generalizable Reward for Autonomous Driving Evaluation and RL), is an offline component independent of the AV stack; it maintains high recall and generalizes across ODDs, vehicle types, and geographies, reducing the need for expensive QA and extensive human labeling for basic information.

GRADER learns geographically specific driving conventions from data, without ever being given the road rules. In the example below, it assigns a high score for staying stopped at a red light before turning right in Germany, where it is not permitted, and a low score for the same behavior in the US, where turning right on red is generally permitted.

Pillar 3: Scaling Validation with Statistical Evidence Transfer

Supporting a wide diversity of vehicle platforms and ODDs is a crucial pillar of the Wayve AI Driver, but this diversity of configurations creates validation complexity. Evidence transfer is vital for efficient and rigorous automated-driving validation. Without it, establishing statistical confidence across each ODD, vehicle platform, and system version would require costly, redundant on-road testing. This could duplicate millions of dollars of work for each new vehicle and delay validation, even when previous tests have already established much of the core system’s safety performance.

Evidence transfer enhances efficiency and safety by aggregating metrics across software releases, platforms, and testing modalities such as simulation, on-road, rig transfer, dashcam, and customer fleet data. Statistical methods like the Bayesian power prior formalize the expectation that similar models will exhibit consistent event rates. In an internal analysis, leveraging customer fleet data reduced the 95% upper confidence bound on the estimated event rate by a factor of three and reduced the required validation data by more than 20-fold, significantly accelerating validation.

Evidence transfer is key to building data-driven safety arguments. By weighting past data based on statistical similarity, it quantifies how much historical evidence can be trusted when evaluating new releases. Combining this evidence reduces uncertainty, yielding a tighter upper confidence interval for the event rate than data from the new release alone. While cumulative data is still needed for strict targets, evidence transfer significantly reduces the amount of new data required for each release while strengthening the overall safety argument.

Conclusion

AV2.0 provides a clear mandate to modernize validation for the AI era. At Wayve, we believe in taking on this mandate deliberately and responsibly, borrowing all that is possible from existing techniques, but leaning into new, innovative tools we believe will scale better, generalize better, and ultimately produce trustworthy products, from L2 to L4.

Validation 2.0 fundamentally reshapes how enterprises develop and deliver AV products beyond mere efficient scaling. Unlike AV1.0 systems, engineering headcount no longer grows linearly with system or ODD complexity; learned simulation and evaluation systems scale with data rather than human effort. 

The final article in this series addresses a different form of supposed evidence: the explanations AI systems produce about their own behavior and why those explanations must not be mistaken for proof of safety.