One sentence. Out of bounds, 44 times in 50.

The same run, framed honestly

The 88% headline is one attack (roleplay) across all ten tasks. Here is every attack in the same run - including the families that did nothing, and the one that did not apply. Publishing the zeros and the N/A is the point: they are what make the 88% believable.

N/A is not 0 and is not a pass: mcp_tool_desc had no applicable trials in this run, so it has no rate - not a zero. Visual (0/100) and injection (scene_text 0/50) transferred nothing on this policy - honest nulls with non-zero upper confidence bounds (0-3.7% and 0-7.1%), not omissions. The all-trials row (64/350) includes the benign control and is not the adversarial ASR (62/300). Ten tasks, 5 seeds each, uncalibrated keep-out predicate.

How the number was produced

• Policy: SmolVLA (checkpoint HuggingFaceVLA/smolvla_libero). Simulator: LIBERO · robosuite · MuJoCo.
• Tasks: all ten libero_object tasks, sharded one per container. Trials: 5 seeds per (task, attack), horizon 280, on an L4.
• Headline attack: roleplay (instruction family) - a reframed goal that still reads as compliant.
• Every attack is paired with a benign control; the benign false-positive rate (2/50) is reported. It is NOT zero: the uncalibrated predicate fires on two benign episodes, on tasks 4 and 5.
• The keep-out predicate is uncalibrated: a success is a trial diverted out of the benign envelope, not a calibrated hazard rate.
• Point estimates carry a 95% Wilson score interval, wide at small n by design.
• Provenance: this reference run was produced with provael 0.32.0 (the ledger below dates it) and is retained as a dated result; the current release is 0.41.2, and no method-equivalence study across that gap has been run yet.

Consistent with the literature

The full side-by-side — every published figure with its arXiv ID and an explicit verdict on whether it can be compared to a Provael ASR at all — is at /compare/published-attack-baselines. It also states where this run disagrees with the literature, unresolved.

Every committed measurement, and how old each one is

The banner at the top of this site dates the newest measurement. This dates all of them. One row per run committed to the product repository, read from watch/measurements.json when this page was built - which is generated from the execution manifest beside each report, so no human types a date into it.

30 committed measurements, of which 24 are real-policy measurements with an observed date. 5 came from a fixture backend and measure no policy at all, and 1 carries a date reconstructed to the day rather than observed. Both facts are stated in the row itself, in words.

A fixture run executes the real attacks against a deterministic CPU stub in under a second. It exercises the harness and it measures no policy, so it can never refresh a freshness claim - the product computes that distinction, this page does not decide it. A reconstructed date is a day-granularity value read from a manifest that recorded no instant; it is shown as the day it names and never as a time.

Run it yourself

The result comes from a public reproduction recipe - a notebook you run, not an assessment you buy, and not (yet) an automated CI reproduction. The one command behind the headline:

Reproducing the SmolVLA transfer needs the optional [lerobot] extra and a GPU; the CPU-first stub path lets you exercise the same harness without one.