Where a policy red-team fits in the physical-AI safety stack
Halos certifies the computer the policy runs on. Nothing in it publishes an attack-success rate for the policy itself.
Provael publishes that rate — and, as of v0.26.0, a measured mitigation delta with its own confidence interval and a benign-task-success acceptance gate. On the CPU fixture suite the adversarial ASR moved 67.5% [52–80%] to 7.5% [3–20%] with the benign false-positive rate unchanged at 0% and clean-task success unchanged at 100%. That result is fixture scaffolding, not a real-model claim, and the study leads with why it is substantially circular — read it before quoting it.
The stack, top to bottom
Three layers, three different questions. Each is certified or tested by different people against different standards, and each is blind to the one below it.
- Layer 1
Compute + functional safety
NVIDIA Halos for Robotics — announced 22 Jun 2026 at Automate 2026
Certifies / tests: Chips, sensors, OS and a certification programme. ISO 26262 work with TÜV SÜD inspecting the Thor SoC and Halos Core; TÜV Rheinland inspecting IGX Thor, Halos OS and the Holoscan Sensor Bridge. Runs the NVIDIA Halos AI Systems Inspection Lab, which NVIDIA describes as “the world’s first ANSI National Accreditation Board (ANAB)-accredited program for functional and AI safety for physical AI” (22 June 2026), with certification bodies including TÜV Rheinland, UL Solutions, TÜV SÜD, exida, SGS and CertX. Agility is the first company NVIDIA names as incorporating elements of Halos for Robotics into its own safety system.
Cannot see: What the learned policy does when the instruction is adversarial. A certified SoC runs a redirected policy exactly as faithfully as a safe one.
- Layer 2
Infrastructure + CVE
Alias Robotics / CAI — arXiv:2603.08665, submitted 9 Mar 2026, with IOActive
Certifies / tests: The robot’s own cyber-physical stack: ROS / ROS 2, firmware and the exploit chains between them — the layer the fielded robot vulnerabilities actually live in. The cited study ran against three consumer robots and reports 38 vulnerabilities.
Cannot see: The policy’s behaviour once an attacker is already talking to it legitimately. A perfectly patched stack still forwards a hostile instruction.
- Layer 3
Policy + action
Provael — this is the layer we measure
Certifies / tests: The instruction and observation that reach a vision-language-action policy, and the action that comes out. Reported as an attack-success rate with a 95% Wilson confidence interval and a benign false-positive control.
Cannot see: Firmware, comms and teleoperation — declared out of scope, and simulation is not the real robot. Nothing here is a certificate.
Layer boundaries are ours, drawn to answer “who tests what”. Each organisation’s own scope is linked above; where a programme spans layers we place it at the one it certifies against a published standard.
What each one actually covers
| What it is | What it tests | What “pass” means | Standards route | Evidence it produces | What it does not measure |
|---|---|---|---|---|---|
| NVIDIA Halos for Robotics | The compute platform a robot runs on — SoC, sensors, OS and the safety toolchain. | Inspected and certified against functional-safety standards (ISO 26262 work named). | ISO 26262 · functional safety, via the ANAB-accredited NVIDIA Halos AI Systems Inspection Lab. Agility’s Digit is assessed there against IEC 61508, ISO 13849 and ISO/IEC TR 5469. ISO 25785-1 (dynamically stable robots) is a Committee Draft (ISO/CD), not published. | Inspection and certification artifacts for the platform | The learned policy’s response to an adversarial instruction |
| TÜV Rheinland · UL Solutions · TÜV SÜD · exida · SGS · CertX | Independent inspection and certification of the platform and its components. | A certificate or inspection report against the standard assessed. | Functional safety and product-safety routes, per body | Third-party certificates — the artifact a notified body recognises | Adversarial robustness of a learned policy; no ASR is published |
| UL 3300 · Simbe “Tally” | The complete machine as a product, including its functional-safety behaviour. | Certified to UL 3300. Simbe describes the safety functions as verified using methodologies “aligned with ISO 13849-1:2023” — alignment of method, which is not the same as certification to that standard, and we do not upgrade their wording. | UL 3300 — the certification. ISO 13849-1:2023 — methodology alignment, per Simbe. | A product certification a buyer or insurer can rely on | Whether a reworded instruction redirects the policy — ISO 13849-1 addresses deterministic control functions |
| Alias Robotics / CAI | The robot’s infrastructure: ROS / ROS 2, firmware and exploit chains — run against three consumer robots (a lawnmower, a powered exoskeleton, a window cleaner). | Nothing “passes”: it reports findings. The paper states 38 vulnerabilities discovered in an automated manner, including fleet-wide reach across 267+ connected devices. | The vulnerability-disclosure route | Published vulnerability findings | Policy behaviour under a legitimate-looking but adversarial instruction |
| Provael | The instruction and observation reaching a VLA policy, and the action it emits, in simulation. | Nothing “passes”: it reports a rate. ASR with a 95% Wilson CI and a benign-FPR control, plus a pre/post mitigation delta under an acceptance gate. | Embodied AI Security Top 10 · mapped to EU AI Act / Machinery Reg / ISO 10218 as candidate evidence | report.json, SARIF, a signed attestation, and a mitigation report | Firmware, comms and teleoperation — declared out of scope (EAI07). Not a certificate, not a notified-body opinion, and simulation is not the real robot. |
Each row reflects that organisation’s own published scope as of July 2026; programmes evolve — follow the link. Names and marks are their owners’; Provael is independent and affiliated with none of them. Placing a row at a layer is not a judgement of its quality.
No. It is not close, and nothing on this page should be read as suggesting otherwise.
A Provael report is candidate evidence toward an assessment, not a certificate and not a notified-body opinion.
The constructive half: under the EU Machinery Regulation (EU) 2023/1230, applying 20 January 2027, Annex I products that use AI for safety functions require mandatory third-party conformity assessment — you will need a notified body, and they will ask what evidence you hold about the learned parts. CEN/CENELEC are still revising the harmonised standards, with the Commission aiming for a complete list by end of 2026. That gap is the reason this page exists, not a reason to skip the assessment.
Read the Machinery Regulation route and how our evidence maps to it.
Why there is a gap at all
The safety standards a robot ships against were written for deterministic control systems. ISO 13849-1:2023 and ISO 26262 reason about a function whose behaviour you can enumerate: given this input and this fault, the output is that, and the residual risk is computable. That is a good model for a brake controller and a safety relay.
A vision-language-action policy is not that function. It is a probabilistic map from an instruction and an image to a trajectory, with no enumerable state space and no fault tree — so the existing methods have nothing to point at it. You can certify the SoC it runs on, the OS beneath it and the machine around it, and still have measured nothing about what the policy does when the instruction is adversarial. That is not a criticism of those standards; it is a statement about what they are for.
The standards bodies know. ISO 10218-1/-2:2025 introduced cybersecurity clauses and deferred the detail to IEC 62443, and NIST AI 100-2e2025 added generative-AI and prompt-injection classes to its adversarial-ML taxonomy. Both point at the problem. Neither hands an assessor a number for a specific policy.
A number is what an assessor can act on. Ours is an attack-success rate — the fraction of episodes in which a perturbed instruction or observation drove the policy into an unsafe state — reported with a 95% Wilson confidence interval and a benign false-positive control, so it can be read against the rate you would have measured with no attack at all. It is a simulation result, and we label it as one.
Whether that simulation result predicts hardware behaviour is the obvious next question, and it is unanswered: the physical-transfer study is pre-registered and has not been run. The protocol is published in advance of its own trials, including the null hypothesis that simulation and real disagree.
Find out where your gap actually is.
Two minutes of questions, or the measured result we already published — whichever you trust more.