STALE MEASUREMENTPast this project's own 2-release window: the published result was measured with v0.32.0, 9 releases ago. Why, and what unblocks it

ProductEvidenceTop 10LeaderboardCompliancePricingDocsStar on GitHub Quickstart
COMMUNITY DRAFT · VERSIONED · v0.2

The Embodied AI Security Top 10

A robot policy has its own attack surface, and it needs its own risk taxonomy. This is a versioned, community-draft list for vision-language-action policies - each entry with a definition, a real example, how Provael tests it (or why it is out of scope), mitigations, and a compliance mapping.

Cross-mapped to OWASP Agentic + LLM Top 10MITRE ATLASProvael covers 8 / 10EAI01 measured on SmolVLASee these risks in the wild →Measured per family →

The ten risks

RELATIONSHIP TO OTHER TAXONOMIES

Complementary to RoboJailBench - not a rival.

RoboJailBench (Yeke, Zhou, Lin, Cai, Bianchi & Celik - Purdue University; arXiv:2605.19328v1) proposes an 18-category harm-outcome taxonomy - what harm results (collision, force violation, unauthorized capture). This list is an attack-mechanism taxonomy - how the system is attacked. One mechanism here produces many of their harms, and several entries (poisoning, injection, CPS, evaluation) are delivery mechanisms or meta-risks with no single harm counterpart. Two axes of the same problem - complementary, not competing.

Mapped against their 18 harm categories, Provael measures 2 covered, 5 partial, 9 not covered, and 2 out of scope by design - deliberately not a clean sweep. A crosswalk that claimed to cover all 18 would be measuring the wrong thing.

$ provael crosswalk --target robojailbench
OPEN RFC · v0.2 → v0.3

The list is a draft. Help version it.

The ranking is expert-elicited, not data-driven, and it will change as the field matures. If an entry is wrong, mis-ranked, or missing - or you have field evidence, a real incident, or a defense result to add - bring it. Co-authors and disputes are both welcome, and the CC BY-SA 4.0 license keeps it open.

  1. Propose - open a discussion or issue with the change and your evidence.
  2. Discuss - argue it in the open; maintainers and contributors weigh in.
  3. Vote - rough consensus decides what lands in the next revision.
  4. Version - merged changes ship in a numbered draft (v0.2 → v0.3), contributors credited.
CANDIDATE FOR v0.3 · NOT NUMBERED

Candidate: under-specified policy identity

Status: candidate for the next revision, raised 12 September 2026. It is not one of the ten above and carries no number until the process on this page says it does.

What it is. A deployment where the policy under review is identified by its weights and its prompt, while the behaviour that actually reaches the robot also depends on the action unnormalisation statistics selected at load time and on the controller-facing conventions downstream. Two deployments matching on weights, prompt and benchmark suite can still produce different physical actions.

Why it would belong here. Every other entry on this list assumes you know which policy you tested. If the identity is under-specified, a passing result on one deployment does not transfer to another deployment of “the same” policy, and neither the engineer nor the person signing off has any way to see the difference.

Source. The executable-policy-specification framing is Jianwei Tai’s, not ours: Same Weights, Different Robot: A Deployment Safety View of VLA Policies (2026), arXiv:2606.03724. On LIBERO-Goal his metadata mismatch takes success from 28 of 28 to 2 of 28 with nothing about the checkpoint changed.

Where we stand on it, which is badly. A Provael report currently identifies a policy by weights hash and config hash, which is exactly the under-specification described above. We are adding the resolved unnormaliser and the controller convention as required schema fields (provael#227). Until that ships, a Provael report identifies the checkpoint and not the deployed policy, and it is better to say so here than to wait for someone to find it.

How to check your own. Load the same checkpoint through two different serving paths, hash the normalisation statistics actually resolved at load time in each, and compare. If the hashes differ you have two executable policies and one name.

CONTRIBUTORS

Be the first organization on this list. This is a new, independent community project - no contributors are listed yet. Propose or dispute an entry, and when your change is adopted you will be credited here.

Start a contribution →

Test your policy against the Top 10.

Run Provael locally and measure your own attack-success rate, or book a red-team assessment for an evidence pack you can file.