Qualification report

Rule: FAILED if any declared limit is measured and exceeded; otherwise INCOMPLETE if any declared limit lacks an admissible measurement; otherwise QUALIFIED. No budgets: NO_VERDICT. Limits are inclusive. Budgets file examples/demo/budgets.yaml, canonical SHA-256 b5e97b4d08db33796d0ee0534a931ad3b6eb7a269cefdfeac8901757d19bf793.

Redacted for publication: directory paths are shortened to the repository and the host name is removed. Every SHA-256, version, measurement and verdict is unmodified.

Run id

20261002T203931Z-ethos-u55-128-32c372d2

Created (UTC)

2026-10-02T20:39:31+00:00

Harness

gauntlet-harness 0.1.0, git 4cac1b03489f93edd8af0bf3372799332f749e0d-dirty

Source tree

8101e52c132c2c67869694ea815cfccfec1720d141d03ce95f495553c396f28e

Host

<redacted>, Darwin 25.6.0 arm64

Host CPU

Apple M2, 8 logical

Target

ethos-u55-128: Arm Corstone-300 FVP 11.27_42 (Cortex-M55 + ethos-u55-128, Shared_Sram): simulator

Target firmware

ML Embedded Evaluation Kit 26.06 inference runner, dynamic-load build (TFLM), sha256 4e9cb5975997e418698392f577b55445524fcb4bfbe78ba24409186c565fc0ec

Hardware

adapter has never run against physical hardware; TARGET executes on the Arm Corstone-300 FVP 11.27_42, a simulator

Inputs unchanged

yes: SHA-256 identical before and after the run

Files

Measurement conditions

Captured (UTC)

2026-10-02T20:39:31+00:00

Clock pinned

N/A: simulator, no physical clock

Clock

simulated: no physical clock. NPU counts are simulated cycles; the application reports a 32 MHz MPS3 core clock

DVFS

N/A: simulator

Power

N/A: simulator

Thermal at start

N/A: simulator

Source

Arm Corstone-300 FVP 11.27_42, cycle-approximate; container localhost/gauntlet-fvp:11.27_42, 6 FVP instances in parallel

Execution settings

GOLDEN

onnxruntime 1.30.0, CPUExecutionProvider, 1 thread, sequential, graph optimizations disabled

SCHEME

ai-edge-litert 2.2.0, op resolver BUILTIN_REF, 1 thread

TARGET

Fast Models [11.27.42 (Dec 9 2024)] in container localhost/gauntlet-fvp:11.27_42 (sha256:c25ef359d5b40aec723d055c625c386ae2f1ff5ed3425b773438cc43bf804882); ML Embedded Evaluation Kit 26.06 inference runner, dynamic-load build (TFLM); compiler: ethos-u-vela 5.2.0; ran 500 of 500 inputs

Input binding

float input bound to integer input tensor as clamp(round_half_away_from_zero(x / scale) + zero_point, dtype range), using the artifact's declared input quantization parameters

Seed

0

Bootstrap

2000 resamples, 95% CI, percentile bootstrap, paired resampling of eval inputs

Latency trials

50 timed, 5 warm-up discarded

Tapped inputs

first 512 inputs for intermediate tensors; all inputs for model outputs

Toolchain: ai-edge-litert 2.2.0, ethos-u-vela 5.2.0, numpy 2.5.3, onnxruntime 1.30.0, pydantic 2.13.5, python 3.12.7, pyyaml 6.0.3.

• Toolchain counts (vela cpu operators = 1, vela npu operators = 40) are taken after its internal operator decomposition and differ from the 9 source-graph operators classified here.
• UNSUPPORTED is not assigned by this adapter: Vela places every operator it cannot accelerate on the CPU, and whether the CPU runtime registers a kernel for it is set by the application's TFLite Micro op resolver, which is not observable at compile time.

The larger gap at the model output is quantization damage (GOLDEN→SCHEME), 21.3× the other: GOLDEN→SCHEME RMSE 0.003327, SCHEME→TARGET RMSE 0.0001563.

Largest SNR decrease between consecutive comparison points in GOLDEN→SCHEME: pool2 → conv3 (−6.73 dB).

• 9 comparison points were declared in the points file.

Saturation

Range-limit hits in SCHEME: 5 of 8,746,000 observed output elements, in 2 of 9 quantized operators (500 inputs).

NOT MEASURED the Arm Corstone-300 FVP is a simulator and produces no wall-clock inference latency; NPU cycle counts are reported separately as npu.* metrics and are not latency

NPU performance counters

NOT MEASURED Roofline position: requires measured external-memory traffic and bandwidth on the device. Compile-time traffic estimates do not establish whether this workload is bandwidth-bound or compute-bound.

Thermal at start

N/A: simulator

DVFS state

N/A: simulator

Power mode

N/A: simulator

Throttle onset

NOT MEASURED sustained-load runs are not implemented in this build

Compared with run 20261001T023700Z-ethos-u55-128-32c372d2 (2026-10-01T02:37:00+00:00), matched on most recent earlier run with the same budgets canonical SHA-256 and the same target. Artifact unchanged: 32c372d247606287… → 32c372d247606287…. Verdict FAILED → FAILED.

• GOLDEN is a reference convention (ONNX Runtime FP32, graph optimizations disabled), not ground truth; another FP32 implementation would differ slightly.
• Per-layer comparison is valid only at fusion boundaries. A tensor that does not exist in a column is marked NOT_COMPARABLE; no alignment is inferred.
• Saturation counts are range-limit hits on operator outputs; clamps on accumulators inside kernels are not observable and are not counted.
• The ethos-u55-128 adapter has never run against physical hardware.
• ethos-u55-128 executed on Arm Corstone-300 FVP 11.27_42 (Cortex-M55 + ethos-u55-128, Shared_Sram): simulator, not silicon. TARGET outputs come from Arm Corstone-300 FVP (functional model), which computes the outputs the silicon would; accuracy and decision changes from it count toward the verdict. It does not model timing faithfully: NPU counters are cycle-approximate, are not latency, and are not admissible. No latency, power, or memory figure in this report comes from silicon.
• ethos-u55-128: compiler cycle, SRAM, flash, and traffic figures are compile-time estimates, not measurements, and are not admissible against declared limits.