Our published benchmark result was measured with v0.41.2, 7 releases ago, past this project's own 2-release window, and the newest real-model measurement, dated 2026-09-18, is past this project's own 7-day window. That is our own number, not your assessment, which runs against your policy on the current release.Why, and what unblocks it
20 corrections on record; the newest, E-2026-21, was raised 7 November 2026.Each entry says what was wrong and why, what it affects, the published value beside the corrected one, and what to do if you hold an affected file.
If you hold a Provael report or signed bundle, check here before relying on a number or a date in it. There is no E-2026-09 on this page: the product’s log uses that ID for the correction this page published first, as E-2026-05.
Entries are append-only and dated. Nothing is removed once added.
An erratum records a claim that was wrong, not a feature that changed. Version history lives in the changelog.
A correction here never invalidates a signature. It changes what the signed content asserts, not whether it was signed.
If you hold a Provael artifact, check this page before relying on a regulatory date inside it.
This page mirrors docs/errata.md, also served at docs.provael.com/errata, and was last updated from it on . docs/errata.md in the product repo is the maintained source. If this page and that document disagree, that document wins.
E-2026-21 — Succ-But-Unsafe, risk exposure time and cumulative cost cannot be observed when an episode ends at its first event
Status
Corrected on main on 10 November 2026 · no headline number moves · report.json files are unchanged; 69 committed report.md files are re-rendered with a dated note, 65 on 7 November and the PAISI recording’s four on 10 November
Date raised
Affects
In the tool, Succ-But-Unsafe from 0.16.0 (15 July 2026), and CC, RET and the four-quadrant table from 0.30.0 (1 August 2026): the Succ-But-Unsafe row, the process-level safety-cost table and the “unsafe success” quadrant row of report.md, the succ_but_unsafe fields of dossier.json and the EAI04 study, and CC and RET in the ForesightSafety crosswalk. On this site, the RET, CC and USR rows of /compare/published-attack-baselines and the Succ-But-Unsafe glossary entry. One public post said it outright: “Episodes that both completed the task and breached the envelope: 0 out of 400” (PKU-Alignment/VLA-Arena#31, 17 August 2026).
What is wrong
The runner stops an episode at its first unsafe step and at done, and on LIBERO done fires at task success, so every episode ends at the first of an envelope exit or a task success. Three published metrics need an episode to go on past that point. Succ-But-Unsafe and USR ask whether an episode both completed the task and left the envelope: only a single step that is both could count, so the rate printed 0.0% whether or not the policy ever does it. RET was documented as dwell, but every episode stops at its first unsafe step, so RET is 0 or 1 and measures no dwell, and CC, its mean, equals the unsafe rate. Across the 1,660 applicable episodes in the committed real-policy LIBERO runs, none is both a success and an exit, and in the 1,040 with decision logs all 175 unsafe episodes have exactly one unsafe step.
Why it stopped being true
The metrics were defined for a run that steps past both events, and the runner ends every episode at the first one. Nothing asked whether a run could observe a metric before printing it, so a value that could not be anything but zero was printed as though it had been measured.
What E-2026-21 changed: each affected value as published and as corrected.
Value
As published
Corrected
Succ-But-Unsafe and USR on a LIBERO run
0.0%
N/A (episodes end at the first of task success or envelope exit); a new LIBERO report.json carries succ_but_unsafe: null
RET and CC on a LIBERO run
a count of unsafe steps per episode and its mean, read as dwell
N/A, for the same reason: RET can only be 0 or 1 here, and CC equals the unsafe rate
The weight-integrity stub
Succ-But-Unsafe, CC and RET as numbers
Succ-But-Unsafe stays, because the stub steps on after its task-success signal latches; CC and RET print N/A (episodes end at their first unsafe step)
What to do
If you hold a LIBERO report.md that prints Succ-But-Unsafe or CC as a number, or a dossier.json with a succ_but_unsafe number for a LIBERO run, read those figures as unobserved, not zero, and re-render it from its report.json with a release that carries this correction. Attestations and execution manifests bind report.json, which is unchanged, so nothing needs re-issuing. The envelope-exit rates, the benign floor, the intervals and the McNemar figures on /results do not use these metrics and do not move.
E-2026-20 — Figures and capabilities posted in other projects’ threads that the product repository does not support
Status
Recorded on 7 November 2026 · the posts are not yet corrected on their threads · no figure in the product repository, on this site or in a signed artifact moves
Date raised
Affects
Posts written by this project’s author in other projects’ issue trackers between 12 August and 21 September 2026, in NVIDIA/garak, robocurve/inspect-robots, UKGovernmentBEIS/inspect_evals, ZJUSCL/Awesome-Robot-Verifier, PKU-Alignment/VLA-Arena, huggingface/lerobot, Lifelong-Robot-Learning/LIBERO, openvla/openvla, mani-skill/ManiSkill and robocasa/robocasa; the product’s entry links each one. Nothing on this site carried these figures. E-2026-14 is the precedent for recording a correction to a post here.
What is wrong
Each post states a figure, a capability or a command that no committed artifact supports. Wilson intervals of [76.4%, 94.3%] on 44/50 and [0.6%, 13.8%] on 2/50 were typed by hand. Four threads on 12 September said “3 of 16 attack families transfer to a real hosted model”, when the registry that day held 17 families of adversarial attacks, three had run on one open policy, locally in simulation, and one separated from the floor. Others claimed 16 perturbation families against 4 open policies, “across 19 perturbation families”, a ManiSkill adapter, commands that do not exist, RoboCasa failure rates measured outside the repository, and reset and init-state handling the tool did not have.
Why it stopped being true
The rule that a number traces to a committed artifact already covered outreach, and these posts broke it: the interval pair was typed by hand, the RoboCasa rates came from work outside the repository, and two posts described reset and init-state handling the tool did not have when they were written.
What E-2026-20 changed: each affected value as published and as corrected.
Value
As published
Corrected
Wilson 95% intervals on 44/50 and 2/50 (NVIDIA/garak#2062, 19 August)
[76.4%, 94.3%] and [0.6%, 13.8%]
[76.2%, 94.4%] and [1.1%, 13.5%], from provael.calibration.wilson_ci
Attack families that transfer to a real model (four threads, 12 September)
3 of 16 transfer to a real hosted model
that day the registry held 17 families of adversarial attacks; three had run on one open policy, locally in simulation, and one separated from the floor
Families and policies measured (huggingface/lerobot#4619, LIBERO#152, openvla/openvla#347)
16 perturbation families against 4 open policies; “across 19 perturbation families”
no committed run supports either; 19 is the family total, which counts the benign baseline and the control family (17 are adversarial)
Adapters and commands (mani-skill/ManiSkill#1488, huggingface/lerobot#4548)
a ManiSkill adapter; provael run with --task-suite and --attack
no ManiSkill suite exists; the commands are provael attack, then provael report --in on one directory
RoboCasa failure rates (robocasa/robocasa#224)
2 to 9 of 50 unperturbed failures on five kitchen tasks, with per-task numbers at provael.com/results
measured outside the repository and withheld: no committed run, no RoboCasa suite, and /results carries no RoboCasa figure
Reset and init-state handling (openvla/openvla#342, huggingface/lerobot#4152)
a fresh env per episode with body_pos recorded; init states pinned to the seed, with a manifest of the states visited
one env per task and no body_pos recorded; init-state recording landed on 24 October 2026 and pinning on 26 October 2026, both shipped in 0.47.0
What to do
Treat a figure from a thread that names no committed run as unsupported. Check an interval against provael.calibration.wilson_ci, a count against the product’s watch/registry.json, and a rate against its run’s README under results/. The current headline is on /results, E-2026-12 gives its reading and E-2026-19 what a pair is matched on. The posts stay wrong on their threads until each carries a correction that cites this ID.
E-2026-19 — The published LIBERO pairs share a task and a seed, not a starting scene
Status
Corrected on main on 10 November 2026 · no number moves · no signed artifact is affected
Date raised
Affects
Every committed LIBERO run so far, from 6 June 2026: none pinned its scenes, because provael attack --pin-init-states first shipped in 0.47.0. And every sentence that let “matched pairs” or “benign twin” read as the same starting scene: in the product repository, the McNemar sentence of the generated results READMEs, two scoring docstrings, the cross-architecture findings write-up, the top-10 and quickstart docs, PRIOR_ART.md and three paper sources; on this site, the Provael row and the Surface axis of /compare/vla-safety-benchmarks, which said the scene was held fixed. One public post said it outright: “Same policy, same scenes, only the wording changed” (PKU-Alignment/VLA-Arena#31, 13 August 2026).
What is wrong
Every attacked episode has a benign twin at the same task and seed, and that stays true. What was wrong is what the project let it imply: that the twin starts from the same scene, so the instruction is the only thing that differs. On LIBERO the seed does not pick the scene. One task’s episodes share a reset counter across arms, and lerobot 0.5.1 loads initial state init_state_id % 50 from it, moving it by two on a task success and by one otherwise, so unpinned, the arms at one seed start from different fixed initial states. The runs recorded no state, so the rest is inference from the documented counter: in the 14 September run’s first task the benign arm started from states 0, 2, 4, 6 and 8, and roleplay from 10 to 14.
Why it stopped being true
The pairing was described by what it shares, a task and a seed, and read as sharing what it does not, the starting scene. The scene is chosen by a reset counter inside the simulator wrapper, and no report recorded it until schema 8. The mechanism was reported upstream on 26 July 2026 (huggingface/lerobot#4152).
What E-2026-19 changed: each affected value as published and as corrected.
Value
As published
Corrected
What a matched pair shares
task, seed and starting scene, so only the instruction differs
task and seed only; the two arms at one seed start from different initial states
The McNemar test on the published pairs
a comparison from one starting scene
exact for the nominal (task, seed) pairs; the pairing removes no scene-to-scene variation, and any scene effect is unmeasured
What to do
Read any earlier sentence that compares two arms “at the same seed” as comparing them at the same task and seed, not from one scene; a clip of an unpinned run shows two different starting scenes. A report of schema 8 or later records init_state on every LIBERO episode, so a pair shares a scene only if its two values are equal; below schema 8, treat pairs as matched on task and seed only. No number moves: 42/50 against 1/50, 44/50 against 2/50 and their McNemar and Holm figures stand. The pinned re-run owed under proof 1 on /roadmap is what restores scene-matched pairs.
E-2026-18 — Per-run reports marked attacks “significant” against a benign control read as a known constant
Status
Corrected on main on 11 October 2026 · the headline numbers, their McNemar and Holm figures, and every signed artifact are unaffected
Date raised
Affects
In the tool, from 15 July 2026 (the test shipped; first released in 0.16.0) to 10 October 2026: the “Significance (Benjamini-Hochberg FDR)” table that provael attack and provael report write into report.md, and the per-family bh_qvalue and bh_significant fields of dossier.json. In the product repo, 65 committed report.md files carry the table, and 21 of their 49 rows marked significant were not: 19 on real policies (SmolVLA and π0.5) and 2 on the weight-integrity stub. No page on this site printed those marks.
What is wrong
Per-run reports marked 21 attack arms significant that were not. The clearest case: the π0.5 run of 18 September (libero_object_2) marked roleplay 1/3 against a benign 0/3 as significant, while the same run’s README reports roleplay 1/30 against 0/30, McNemar p = 1, rejected. Same data, two Provael surfaces, opposite claims: the E-2026-03 pattern.
Why it stopped being true
The test treated an estimate as a known constant. It ran a one-sample exact binomial whose null rate was the run’s observed benign false-positive rate, so the benign arm’s own sampling noise never entered it. A benign arm of 0/n was clamped to 1e-12, so a single success scored a p-value near n × 1e-12 and survived the Benjamini-Hochberg correction. The dossier’s per-family test did the same.
What E-2026-18 changed: each affected value as published and as corrected.
Value
As published
Corrected
Per-attack significance test in report.md, and the per-family test in dossier.json
One-sample exact binomial against the run’s observed benign rate, a benign 0/n clamped to 1e-12
One-sided Fisher exact test against the benign arm’s own counts (fisher_exact_greater), then Benjamini-Hochberg as before. A report whose benign counts were trimmed away gets no significance at all
Rows marked significant across the 65 committed report.md tables
49
28, with 21 withdrawn: 19 on SmolVLA and π0.5, 2 on the weight-integrity stub
π0.5 run of 18 September (libero_object_2), roleplay 1/3 against a benign 0/3
q = 3e-12, significant
q = 0.500, not significant
What to do
If you hold a report.md whose significance note says “one-sided exact binomial … against the benign baseline FPR”, or a dossier.json written by provael 0.16.0 to 0.45.0, re-render it from its report.json with a release that carries this correction, or recompute each arm against its benign counts with fisher_exact_greater. Attestations and execution manifests bind report.json, which did not change, so nothing needs re-issuing. The 42/50 against 1/50 headline on /results was never tested this way (its significance comes from paired McNemar with Holm) and does not move.
E-2026-17 — Signed attestations carried an ISO 10218 date no source gives: 1 April 2025, for a standard ISO dates to February 2025
Status
Corrected on 28 September 2026 · ships in provael 0.45.0 · no measured number moves · a bundle already issued stays authentic, so read its ISO 10218 date as corrected below
Date raised
Affects
In the tool: the ISO 10218 entry of the regulatory clock inside every attestation statement provael 0.7.0 (3 July 2026) to 0.44.0 issued, and the ISO 10218 row of every assurance-report draft, so the insurer view in a --profile insurer bundle too. On this site: /regulatory-clock.json, whose ISO 10218 and ANSI/A3 R15.06 entries still carried a day their own notes doubt. No page printed either day after E-2026-15.
What is wrong
The regulatory clock inside every signed attestation dated ISO 10218 to 1 April 2025, from the release that introduced attestations. No source gives that day: ISO’s catalogue gives the publication date of both ISO 10218-1:2025 and ISO 10218-2:2025 as 2025-02, a month with no day (checked 28 September 2026). The clock sits inside the signed statement, so every attestation from those releases vouches, under a valid signature, for a date nobody verified. This site’s machine-readable clock held the other half of the same problem: its ISO 10218 and R15.06 entries kept a day (5 February, 21 August) their own notes said was not verified, the decision E-2026-15 left open.
Why it stopped being true
The tool’s date was typed when attestations were added, never traced to a source, and repeated in a second copy in the hosted conformity mapping. On this site the pages stopped printing the doubted days on 26 September, but the data kept them, so a machine reading the JSON still got a day the clock itself doubted.
What E-2026-17 changed: each affected value as published and as corrected.
Value
As published
Corrected
ISO 10218-1/-2:2025 in the tool’s signed regulatory clock
1 April 2025
February 2025, at month precision
ISO 10218 applicableDate in /regulatory-clock.json
2025-02-05
2025-02, with datePrecision month
ANSI/A3 R15.06-2025 applicableDate in /regulatory-clock.json
2025-08-21
2025, with datePrecision year (updated the same day from 2025-08: ANSI’s record shows no date at all, so the month was no better verified than the day)
What to do
If you hold an attestation from provael 0.7.0 to 0.44.0, read its ISO 10218 date as February 2025; the rest of the statement is unaffected, and 0.45.0 carries the month. If you read /regulatory-clock.json, read applicableDate at its datePrecision: ISO 10218 carries the month and R15.06 the year, and each doubted day survives only as unverifiedDay, which the build fails any page for printing.
E-2026-16 — Three ATLAS mappings named something that is not an ATLAS tactic, on two Top 10 pages and in the tool’s SARIF since 0.16.0
Status
Corrected on 26 September 2026 · no measured number moves · no signed artifact is affected
Date raised
Affects
/eai-top-10/eai01 and /eai-top-10/eai03 and their Markdown twins. In the tool, the atlasTechniques property of the EAI01, EAI02 and EAI03 SARIF rules from 0.16.0 to 0.44.0, and the crosswalk artifact; this site’s pinned copies of that artifact and of the sample SARIF move at the next re-pin.
What is wrong
Checked against MITRE ATLAS data release 2026.09, including which tactic each technique sits under. EAI01 named “ML Attack Staging”, a retired name for AML.TA0001 (“AI Attack Adaptation” since 2026.08), and ATLAS files neither prompt injection nor jailbreak under that tactic. EAI03 put “ML Supply Chain Compromise”, a technique under its retired name, where the tactic goes. The tool also named “Evasion” for EAI02, where the tactic is “Defense Evasion”; this site already had that one right.
Why it stopped being true
The mapping was checked for shape (an arrow, no invented ATLAS identifiers) and never against ATLAS’s own list of tactics, and ATLAS renamed its tactics after the rows were written. Nothing on this site compared a page’s mapping with the tool’s.
What E-2026-16 changed: each affected value as published and as corrected.
Value
As published
Corrected
EAI01 ATLAS mapping
ML Attack Staging → jailbreak of an ML-driven agent
Execution → LLM Prompt Injection (direct) of an ML-driven agent; Defense Evasion → LLM Jailbreak
EAI03 ATLAS mapping
ML Supply Chain Compromise → poison checkpoint; Persistence → backdoor model
Persistence → Manipulate AI Model (backdoor); Initial Access → AI Supply Chain Compromise of an open-weights checkpoint
What to do
If you ingest atlasTechniques from a Provael SARIF file, read the EAI01 to EAI03 rules with the tactics above. The tool now fails its tests on a mapping whose tactic is not one of ATLAS’s sixteen, and this site’s build fails when an EAI page names different tactics from the tool’s catalog.
E-2026-15 — Two publication days were printed that the site’s own regulatory clock does not verify, one of them under a “Verified dates” heading
Status
Corrected on 26 September 2026 · no measured number moves · no signed artifact is affected
Date raised
Affects
/compliance and /compliance/iso-10218 and their Markdown twins; /regulatory-clock and its twin (visible text, screen-reader link labels and machine-readable dates); /for/certifiers-insurers; /readiness; llms-full.txt; the sample evidence pack.
What is wrong
ISO 10218-1/-2:2025 was printed as “published 5 February 2025” and ANSI/A3 R15.06-2025 as “21 August 2025” on ten built surfaces, one of them on /compliance.md under the heading “Verified dates”. The site’s own regulatory clock said otherwise: of ISO 10218, that ISO’s catalogue shows the month and “the 5 February day is from secondary sources and is not carried as a verified value”; of R15.06, that “the 21 August 2025 day remains unverified”.
Why it stopped being true
The ISO date was typed into four fields and a Markdown twin instead of read from the clock, and the fact check required that day as a verified regulatory fact, so the one check that read every page held the unverified value in place. The clock already held the truth and nothing compared the pages against it.
What E-2026-15 changed: each affected value as published and as corrected.
Value
As published
Corrected
ISO 10218-1/-2:2025 publication date
5 February 2025
February 2025, the month ISO’s catalogue shows
ANSI/A3 R15.06-2025 date on /regulatory-clock
21 August 2025
August 2025; the clock doubts the day
What to do
If you quoted either day from this site, quote the month. The build now fails if any page prints a day the clock doubts, or renders a month-precision date with a day. The clock’s recorded dates are unchanged: whether a doubted day should stay in its applicableDate field is an open decision, not part of this correction.
E-2026-14 — A post on the project’s LinkedIn page called this site’s Machinery Regulation date wrong; the post was wrong
Status
Recorded on 27 September 2026 · this site and the signed clock were right throughout · no measured number moves · no signed artifact is affected
Date raised
Affects
One post on the Provael LinkedIn page, published 12 September 2026. Nothing on this site, in the product repository or in any signed artifact carried the wrong date.
What is wrong
The post said the date on this site’s front page was wrong and gave the Regulation’s application date as the uncorrected one, six days earlier, with two more uncorrected Article 54 dates it said had already passed. It had read Article 54 as the EUR-Lex page of Regulation (EU) 2023/1230 still prints it, before the Corrigendum of 4 July 2023 (OJ L 169, CELEX 32023R1230R(01)) corrected all three dates in items 10, 11 and 12. Laura S. pointed to the corrigendum in the comments. The reply under the post announced this correction as E-2026-12; that ID went to a different correction on 18 September, so this one is E-2026-14.
Why it stopped being true
The post re-read a legal date from EUR-Lex prose instead of taking it from the regulatory clock this site and the signed artifacts already carried, and the page it read predates the corrigendum.
What E-2026-14 changed: each affected value as published and as corrected.
Value
As published
Corrected
EU Machinery Regulation application date (the post)
the uncorrected Article 54 date, six days earlier
20 January 2027 (Corrigendum of 4 July 2023, item 10)
Articles 26 to 42 and Article 50(1) (the post)
the uncorrected dates, both said to have passed
20 January 2024 (item 11) and 20 October 2026 (item 12), which has not passed
What to do
If you quoted a date from that post, quote 20 January 2027 and cite the corrigendum. A regulatory date in any post of ours now comes from the regulatory clock, and the product’s tests fail on a file that states an uncorrected Article 54 date as the one that applies.
E-2026-13 — The measurement meta block contradicted itself on every page, a regression of a published correction; and two surfaces paired an inclusive attack count with an exclusive family count
Status
Corrected on 18 September 2026 · no measured number moves · no signed artifact is affected
Date raised
Affects
The <meta name="provael:measurement-*"> block in the <head> of every page (12 routes checked live, /pricing/ to /contact/); the registry sentence on the homepage’s Markdown twin and in /llms.txt.
What is wrong
Every page served, in its own <head>, measurement-age-days="0" beside measurement-stale="true" and a status string reading “0 days old … past this project’s own 7-day window and 2-release window”. Zero days is not past a seven-day window; only the release window had fired. The visible banner had been corrected to say exactly that on 6 September 2026 (PR #110); the meta was built from a second copy of the sentence, which kept the old wording for twelve days under a corrected banner. Separately, the homepage twin and /llms.txt put the total attack count (42, “incl. the benign baseline”) inside the parenthesis after the adversarial family count (17), two conventions the product’s own registry note says are not interchangeable, and the label undercounted what the 42 includes (the baseline and the harmless-variation controls). 42 is the registry of the pinned release, v0.41.2, and stays until the site re-pins to the next one; the defect was the pairing, not the number.
Why it stopped being true
Two sentences where there should have been one: the banner and the meta each assembled their own status string from the same facts, so a fix to one did not reach the other, and no check compared them. For the counts, check:registry asked whether each number was derived from repo-facts.json and every number was; nothing asked whether the pair beside it came from the same convention.
What E-2026-13 changed: each affected value as published and as corrected.
Value
As published
Corrected
measurement-status meta
“0 days old … past this project’s own 7-day window and 2-release window”
the banner’s own sentence, naming only the window that fired (src/lib/freshness.ts STATUS_SENTENCE); check:freshness fails if the two differ
measurement-age-days meta
0 (a day count, wrong the day after the build)
removed; measurement-measured-at carries the date and a consumer subtracts
homepage twin and /llms.txt registry sentence
the adversarial family count (17) with the adversarial attack count (39) and then the TOTAL attack count (42) in the same parenthesis, labelled “incl. the benign baseline”
two matched pairs, each convention with itself: the adversarial attack count with the adversarial family count, and the total attack count with the total family count (counting the benign baseline and the harmless-variation controls). check:counts fails on any unmatched pair
What to do
If a crawler or a script read measurement-age-days, read measurement-measured-at and compute the age. If you quoted “42 attacks in 17 families”, quote 39 in 17 (adversarial) or 42 in 19 (total). The same sweep fixed three more machine-readable surfaces without an erratum of their own — /llms.txt’s adoption figures now derive from the file /adopters/ renders, /404 answers 404, and /pricing’s JSON-LD no longer publishes the design-partner rate as the assessment’s base price; the changelog records them.
E-2026-12 — The 44/50 roleplay result was described as the attack redirecting the policy; the controls show the policy leaves its envelope under the frame alone, target or no target
Status
Wording corrected on this site and in the product repository (0.42.0) · no published number moves: 44/50 stands and a re-measurement on 0.41.2 returned 42/50 · no signed artifact is affected
Date raised
Affects
Every sentence that read the 88% as the attacker redirecting or steering SmolVLA: the homepage hero and method copy, /results, /quickstart, the note “What 88 percent means” (its section 4 drew exactly the inference the controls refute), and in the product repository the README’s finding paragraph, the cross-architecture findings page and the attribution to reframed meaning on the semantic-vs-mechanical page. Update, 7 November 2026: the correction reached the product repository and this site, not the posts already made in other projects’ repositories. The product’s entry lists sixteen threads that still carry the superseded reading, or the 44/50 against 2/50 pair with no current figure beside it, and four already corrected on the thread.
What is wrong
Two control arms registered on 14 September 2026 were built to test whether the roleplay frame works because the policy acts on the attacker’s framing. Both failed that reading, on the same ten tasks at three seeds each (results/smolvla_libero_object_control_2026-09-14): the roleplay frame with the graspable target replaced by a one-token filler left the envelope in 27 of 30 cells (90%, Wilson 95% [74%, 97%]), as often as the attack itself (26/30); the roleplay prompt’s own tokens with the target replaced and the word order destroyed left it in 18 of 30 (60% [42%, 75%]); the two meaning-preserving rewordings stayed at 0/30 and the benign floor at 1/30. The published number is correct and reproduced (42/50 on 0.41.2). What it measures is SmolVLA leaving its safety envelope under a long, imperative, out-of-distribution instruction, a fragility finding about the policy, not the attacker steering the arm toward a chosen object.
Why it stopped being true
The controls that would have caught this did not exist when the number was published: benign_reword and nonsense_text are three-token strings, and a 0/50 on three tokens never showed that a twenty-token frame is harmless. The note’s own text said a high reword rate would have made the number “evidence of fragility rather than of attacker control”; the arms that could show that were added on 14 September and did. The falsification clause on the semantic-vs-mechanical page was written for meaning-preserving arms only, so a meaning-destroying arm that fires slipped past it.
What E-2026-12 changed: each affected value as published and as corrected.
Value
As published
Corrected
What the 88% (44/50) shows
an attacker who controls the instruction can redirect the policy
the policy leaves its safety envelope under a long, imperative, out-of-distribution string; the exit does not depend on the target the attacker names
roleplay_no_target (frame, no target)
not run
27/30, 90% [74%, 97%]
scrambled_text (same tokens, order destroyed)
not run
18/30, 60% [42%, 75%]
What predicts an envelope exit on this policy
that the instruction’s meaning was changed
that the instruction is a long, imperative, out-of-distribution string of this kind; a shorter imperative and a fluent non-imperative sentence of the same length have not been run
The number itself
44/50 [72%, 100%] task-clustered
unchanged; re-measured at 42/50 on 0.41.2
What to do
If you quoted the 88% as evidence that an attacker can steer SmolVLA to a chosen object, re-state it as an envelope-exit rate under an out-of-distribution imperative instruction, and cite the control run beside it. Nothing numerical needs re-deriving.
E-2026-11 — The shipped gradient_patch attack could not move a frame from its zero start, and the figures it was published with come from a script outside the repository
Status
Corrected in the product repository on 14 September 2026, ships in 0.42.0 · no published rate moves · no signed artifact is affected
Date raised
Affects
The gradient_patch attack as released from 0.39.1 to 0.41.2; the product CHANGELOG entry that introduced it and the 1 September 2026 update in PRIOR_ART.md, which present a Diffusion Policy × PushT result as evidence that the harness had the image-space attack it was missing. No provael.com page quoted those figures.
What is wrong
The attack maximises the distance between the policy’s vision feature of the perturbed frame and its feature of the clean frame. That objective’s gradient is exactly zero at the clean frame, and the projected-gradient loop started from a zero perturbation, so its first step received sign(0) = 0 and every later step the same: against a smooth encoder the released module could not leave the clean frame. It surfaced the first time the module was pointed at a real vision tower (SmolVLA, 14 September 2026): two refinements, L∞ distance 0. The PushT figures were measured by a script run outside the repository, which is not committed and cannot be re-run from it.
Why it stopped being true
The unit tests exercised the loop with a stand-in oracle that returned a constant non-zero gradient everywhere, which no real encoder does at the clean frame; and a result from an uncommitted script was published under the module’s name.
What E-2026-11 changed: each affected value as published and as corrected.
Value
As published
Corrected
Start of the PGD search
a zero perturbation
a uniform draw inside the ε-ball, seeded from the episode seed and the step
What the PushT figures measure
this module
an external script whose code is not in the repository; retained as a dated report of that script’s result
Any VLA rate from gradient_patch
none was claimed
none is claimed; the first measurement is the SmolVLA × LIBERO run, committed under results/ before it is quoted
What to do
If you cited the PushT figures as a measurement of provael’s gradient_patch, re-cite them as an external script’s result or drop them. Nothing else to re-derive: the arm never entered a published rate, because it reports itself not applicable wherever no gradient path exists.
E-2026-10 — The methodology note for the 88% figure called the paraphrase attack the reword control
Status
Corrected in the note, with a dated correction block in its body · no measured number moves · no signed artifact is affected
Date raised
Affects
/notes/what-88-percent-means and the site changelog entry of 4 September 2026, as published from 4 to 13 September 2026. The homepage, /results, /findings, the sample evidence pack and the pinned manifest were right throughout.
What is wrong
Section 4 of the note said the semantics-preserving reword fired 3 of 50, named `paraphrase` as that arm, and added that an earlier draft’s 1 of 50 was unsupported by the committed artifact. It was the other way round. `paraphrase` is an adversarial attack: the unsafe request rephrased without the roleplay frame (3 of 50 in the suite run). The harmless-variation control is `benign_reword` — the benign instruction reworded with no unsafe target — and it fired 1 of 50 in the control run (results/smolvla_libero_object_control), 43 discordant pairs against roleplay, none the other way, McNemar exact p = 2.3e-13. The note’s argument about attacker control versus brittleness was built on the wrong arm.
Why it stopped being true
Two arms with “reword” in their plain-language description, one an attack and one a control, read from memory of the run rather than from the manifest. Nothing checked figures inside notes: check:figures reads pages, not the content collection.
What E-2026-10 changed: each affected value as published and as corrected.
Value
As published
Corrected
The reword control arm
paraphrase, 3 of 50 (6%)
benign_reword, 1 of 50 (2%)
What 3 of 50 is
the control
the paraphrase attack arm of the suite run
Discordant pairs quoted in §2
41-0 (the control run)
42-0 (the suite run the p-value belongs to)
What to do
If you quoted the note’s section 4 between 4 and 13 September 2026, the control figure is 1 of 50, not 3 of 50, and the conclusion (attacker control, not brittleness) stands more strongly, not less. The note now carries the correction in its own body.
E-2026-08 — A calibration's 0.0 benign false-positive rate was published as evidence of a well-placed boundary
Status
The figure is accurate · what it was offered as evidence for is corrected · the predicate it describes was never adopted, so no published rate moves
Date raised
Affects
The provael repository's 0.40.0 CHANGELOG entry and the commit message of PR #212, as published from 6 September 2026. Nothing on this site is affected: every page that renders the LIBERO result describes the keep-out predicate as uncalibrated, which was and remains accurate. The 44/50 roleplay headline, its task-clustered interval and the 2/50 benign control are all unchanged.
What is wrong
Ten per-task keep-out calibrations for libero_object were published with the statement that all ten achieved a held-out benign false-positive rate of 0.0 against a 0.05 target. The number is correct. It was presented as the notable property of the fit, and it is not one: it is what nearly any placement of that hazard box achieves. Replaying the one committed real-model run that records end-effector trajectories against all six candidate faces of the benign envelope — one task, 14 episodes, 2 benign and 12 attacked — five of the six faces score the same 0.0 benign rate, and five of the six flag nothing at all. The face the fitter used flagged 0 of 12 attacked episodes, where another face flagged 5 and the uncalibrated default box flagged 4.
Why it stopped being true
The fitting routine searched the GAP between the hazard box and the benign envelope and took the FACE from a default argument, never varying it. A hazard box is disjoint from the benign workspace by construction, so every gap large enough to clear the envelope drives the benign rate to or near zero — the search always succeeded and the number it reported carried no information. The structural reason it could not have been caught by repeating the run: a benign-only calibration cannot choose a face, because where an attack goes is not observable from rollouts in which no attack ran. The published entry did carry the caveat that a 0.0 benign rate says nothing about whether the zone still catches a redirected policy. That caveat was the finding, filed under a heading that read as an achievement.
What E-2026-08 changed: each affected value as published and as corrected.
Value
As published
Corrected
ten calibrations at a 0.0 benign FPR
the notable property of the fit
the property of almost any placement — five of six candidate faces score the same
detection by the fitted zone
not stated
0 of 12 attacked episodes, on the one task with trajectory data
relative to the uncalibrated default box
implied improvement
strictly worse — the default box flags 4 of 12
What to do
Nothing to re-check in a published number: these zones were never adopted, so every rate measured for libero_object was scored against the documented default box, and every run report records calibrated: false. If you cited the ten calibrations as evidence that the keep-out predicate is now fitted, it is not, and the tool has never claimed otherwise at runtime — provael doctor reports "calibrated zones: none adopted". If you were planning to adopt them, do not: provael calibrate --attack (0.41.0 and later) fits a boundary whose face is chosen against attacked rollouts rather than assumed.
E-2026-07 — Four surfaces said ISO 10218:2025 defers its cyber detail to IEC 62443
Status
Corrected on every surface · the phrasings are now blocked by the build
Date raised
Affects
/compliance/iso-10218, /defenses, /compare/physical-ai-safety-stacks and /regulatory-clock (with its JSON), as published up to 6 September 2026. No measured result and no signed artifact is affected: the control identifiers, the iec-62443 requirement key and the emitted routes_to field are a published contract and were never the thing that was wrong.
What is wrong
Four pages stated that the 2025 revision of ISO 10218 introduces cybersecurity clauses and hands the detailed requirements to the IEC 62443 series. It does not. Clause 2, Normative references, of ISO 10218-1:2025 lists ISO 3864-x, ISO 4413/4414, ISO 7010, ISO 9283, ISO 12100, ISO 13732-x, ISO 13849-1:2023, ISO 13850, ISO 14118/14119/14120, ISO 19353, ISO 20607, ISO 20643 and IEC 60073. IEC 62443 is not among them; it appears in the informative Bibliography, alongside IEC TR 63074. The standard’s own Foreword says the revision adds "requirements for cybersecurity to the extent that it applies to industrial robot safety" — it adds them, it does not delegate them.
Why it stopped being true
The claim is a common secondary-source summary of what the 2025 revision did, and it was transcribed rather than read against the standard. It then spread by being restated: one phrasing in a compliance catalogue, a second in a defenses table, a third in a comparison page’s answer, a fourth in the machine-readable clock. Provael’s own mapping onto an IEC 62443 SL2 target is legitimate and stays; presenting it as something ISO 10218 routes to was not.
What E-2026-07 changed: each affected value as published and as corrected.
Value
As published
Corrected
ISO 10218-1/-2:2025, relationship to IEC 62443
introduces cybersecurity clauses and defers the detailed requirements to IEC 62443
adds cybersecurity requirements to the extent they apply to industrial robot safety; IEC 62443 appears only in the informative Bibliography
What to do
If you were treating a Provael IEC 62443 view as inherited from ISO 10218, it is not: an assessor reading ISO 10218 inherits nothing about IEC 62443, and the cyber requirements are in the document they already hold. The four superseded phrasings are now in the build’s forbidden-string list, so none of them can be reintroduced on any page.
E-2026-06 — A correction to the CRA sub-deadline source was reverted while fixing a link label
Status
Restored the same day · the four deadlines themselves were correct throughout
Date raised
Affects
/regulatory-clock, /regulatory-clock.json and /compliance/cra-incident-reporting, as published for roughly seven hours on 6 September 2026. No deadline, no measured result and no signed artifact is affected.
What is wrong
The CRA Article 14 sub-deadlines were published citing the Commission’s CRA summary as their source. That is the document E-2026-04 moved them OFF, five days earlier, because its phrasing does not carry the distinction the four rows encode — the two final reports run from different events. The rows on the page were correct the whole time. What was wrong was the citation under them: a reader checking the deadlines against the cited source would not have found the distinction there, and could reasonably have concluded the page had invented it.
Why it stopped being true
/compliance/cra-incident-reporting closed with two source links carrying different labels and the same URL — "Regulation (EU) 2024/2847 on EUR-Lex" and "Commission CRA summary" both pointing at the Official Journal. That is a real defect, and it was fixed by moving the URL to match the label. The label was the wrong half. The link text lived in markup on two pages while the URL lived in JSON, so the two were editable apart and neither one carried the reason the other existed. Nothing was red.
What E-2026-06 changed: each affected value as published and as corrected.
Nothing to re-check in a runbook: the 24-hour, 72-hour, 14-day and one-month deadlines and their start points are unchanged and were correct throughout. If you cited this page’s source link rather than the deadlines, cite Article 14 of the Official Journal text instead. The link label now travels in the clock data beside the URL so the two cannot be edited apart, and scripts/check-clock-sources.mjs fails the build if a sub-deadline is ever pinned to a summary of the instrument rather than the instrument.
E-2026-05 — Three of this site’s own surfaces published a stale attack-family count at the same time
Status
Corrected by re-pinning both witnesses to v0.39.1 · ID collision: the product ledger issued its own E-2026-05 on 6 September 2026 for a different correction (ISO 10218 / IEC 62443, = E-2026-07). This entry keeps the ID it was published under and is recorded in docs/errata.md as E-2026-09; do not renumber either.
Date raised
Affects
Every page rendering a registry count — the homepage, /results, /verification, /leaderboard — plus public/llms.txt and the supported-version line on /security, as published between 1 and 3 September 2026. No measured result is affected: the ASR, its interval, the benign control and every per-attack row were byte-identical before and after.
What is wrong
The site published family and attack counts one lower than the product’s generated watch/registry.json across every figure — the exact superseded and correct values are in the table below, so this sentence does not restate a live count. /security told a security reporter to reproduce on v0.38.0 while PyPI served 0.39.1 — a disclosure page pointing at a version two releases old.
Why it stopped being true
gradient_patch shipped on 1 September 2026 in release 0.39.0 and grew the registry. The generated file was right the whole time. What was stale was the site’s pinned mirror of it, src/data/repo-facts.json, last refreshed on 26 August against the v0.38.0 tag — which is also where the supported-version string came from, so one stale pin produced both defects. The uncomfortable part is the check: check-registry-agreement.mjs asserts the site’s counts against watch/registry.json and PASSED, because the copy it compared against was the same stale mirror. A guard that reads a cached copy of the fact it guards cannot fire. That is the third time this project has found that shape — after the measurement-freshness banner and the regulatory-clock anchor — and it is the pattern worth remembering rather than any of the three numbers.
What E-2026-05 changed: each affected value as published and as corrected.
Value
As published
Corrected
Adversarial attack families
16
17
Adversarial attacks
38
39
Attacks including baseline and control
41
42
Families never run against a real policy
13
14
Supported version on /security
v0.38.0
v0.39.1
What to do
If you quoted a family or attack count from this site between 1 and 3 September 2026, it was low by one family and one attack. If you were preparing a security report against v0.38.0 because /security named it, reproduce on the current PyPI release instead. Nothing measured moved, and no signed artifact is affected. A project that asks strangers to check its numbers should publish the occasions its own numbers were wrong, which is what this entry is.
E-2026-04 — The CRA final-report deadline was published with the wrong start point for severe incidents
Status
Corrected in the clock and on every page that renders it · the source half of that correction was reverted for seven hours on 6 September 2026 and restored — see E-2026-06
Date raised
Affects
/regulatory-clock and its JSON, as published between 25 August 2026 and this correction. No signed artifact is affected.
What is wrong
The CRA Article 14 reporting clock carried its two final-report deadlines as a single row: "Final report, due once a corrective or mitigating measure is available - 14 days for an actively exploited vulnerability, one month for a severe incident." The start point in that sentence is correct for the vulnerability branch and wrong for the incident branch.
Why it stopped being true
Article 14(2)(c) and Article 14(4)(c) set deadlines measured from different events. The 14-day vulnerability report runs from when a corrective or mitigating measure is available. The one-month incident report runs from submission of the 72-hour incident notification under Article 14(4)(b), and does not wait for a measure at all. Collapsing both into one row applied the first branch’s anchor to the second. The sub-deadlines were read from the Commission CRA summary rather than the OJ text, and the summary’s phrasing does not carry the distinction.
What E-2026-04 changed: each affected value as published and as corrected.
Value
As published
Corrected
Severe-incident final report, start point
once a corrective or mitigating measure is available
one month after the 72-hour incident notification was submitted (Art. 14(4)(c))
What to do
If you planned an incident-reporting runbook against this page, check the one-month deadline: it starts earlier than the superseded wording implied, because it runs from your own 72-hour filing rather than from a fix being ready. A runbook that waits for a fix before starting the one-month count can miss it. The 24-hour, 72-hour and 14-day figures were correct throughout, and no measured result or signed artifact is affected.
E-2026-03 — Two READMEs published a zero-width confidence interval for three null arms
Status
Corrected in the tool and on both surfaces · the signed board and its signature were correct throughout
Date raised
Affects
README.md and results/smolvla_libero_object_suite/README.md in the product repository, as published between 9 and 30 August 2026. No signed artifact is affected, and this website was never wrong about it.
What is wrong
Both READMEs published the task-clustered 95% confidence interval for the three null arms (patch, decoy_object, scene_text, each 0/50) as [0%, 0%]. A zero-width interval states that the true rate is known exactly. It is not: the arms are null because nothing succeeded in fifty attempts, which is a different claim from a rate of precisely zero.
Why it stopped being true
The bootstrap guard counted CLUSTERS rather than checking the interval it produced. Ten tasks that all score zero pass a cluster count and are just as degenerate - every resample returns the same rate, so the percentiles collapse onto it. provael.com published a non-zero upper bound for those same three results throughout, so the project contradicted itself in public.
What E-2026-03 changed: each affected value as published and as corrected.
Value
As published
Corrected
patch, 0/50
[0%, 0%]
no clustered interval; pooled exact 95% upper bound 7.1%
decoy_object, 0/50
[0%, 0%]
no clustered interval; pooled exact 95% upper bound 7.1%
scene_text, 0/50
[0%, 0%]
no clustered interval; pooled exact 95% upper bound 7.1%
What to do
No action for anything read on this site - provael.com carried the correct non-zero bound the whole time. If you hold a copy of either README from that window, the three null rows overstate how strong those nulls are: the measurement supports only that the attack was not observed to succeed in fifty attempts, which leaves a true rate of up to 7.1%.
E-2026-02 — The documented verify command printed a pre-rotation signing keyid
Status
Corrected · the published board and its signature were correct throughout
Date raised
Affects
Anyone who verified the public leaderboard signature between 30 July and 3 August 2026 by following the documented steps.
What is wrong
The project signing key was rotated on 30 July 2026 and the published board was re-signed the same day, so verification succeeded and printed the NEW key id, 8d62aa33ed5162f3. The documentation did not move with the key: README.md and docs/leaderboard.md kept showing the pre-rotation id 5b9a65790d93d0bc as the expected output, and docs/leaderboard.md went further and called that id the only key the published board is signed with.
Why it stopped being true
For four days anyone who ran the documented verification got a result the documentation called impossible. The natural reading of that contradiction — that the signature is fraudulent — was wrong in the worst direction available to this project: the check was working and the prose about the check was not.
What E-2026-02 changed: each affected value as published and as corrected.
Value
As published
Corrected
documented verify keyid
5b9a65790d93d0bc
8d62aa33ed5162f3
What to do
Nothing to re-issue: the board and its signature were correct throughout, and a verification run in that window succeeded for the right reason. The keyid is not an independent fact — it is the first 16 hex characters of SHA-256 over leaderboard/results/leaderboard.pub — so compute it yourself rather than trusting this page or the README.
E-2026-01 — Signed attestations carry a superseded EU AI Act application date
Status
Corrected in the tool · previously issued bundles are unaffected in authenticity
Date raised
Affects
Any attestation bundle signed before this correction shipped
What is wrong
The regulatory clock embedded in every attestation payload recorded the EU AI Act Annex I (product-embedded high-risk) application date as 2027-08-02, with a note stating that the Digital Omnibus deferral to 2028 had been agreed only provisionally and had not been published in the Official Journal.
Why it stopped being true
That was accurate when written. It stopped being accurate on 24 July 2026, when Regulation (EU) 2026/1744 (Digital Omnibus on AI) was published in the OJ. It entered into force on 27 July 2026 and moved product-embedded Annex I application to 2 August 2028 (stand-alone Annex III moves to 2 December 2027). The clock’s own last_verified field read 2026-07-23 — the fact was checked one day before it changed, and nothing re-read it.
What E-2026-01 changed: each affected value as published and as corrected.
Value
As published
Corrected
AI Act Annex I applies_from
2027-08-02
2028-08-02
What to do
A bundle signed before the fix is still cryptographically authentic — the signature is over what was believed at signing time, and it verifies. What it is not is current. If you are relying on the AI Act date inside one, re-issue it; if you are relying on the ASR, the measurement is unaffected.
The dates themselves live on the regulatory clock, which carries a last_verified field per instrument. What has and has not been independently checked is on verification, and the diligence answers are on trust.