The release gate for AI on hardware
Your model gets quantized and compiled before it reaches the device. Gauntlet measures exactly what that changed — and whether the compression or the silicon is to blame.
Runs inside your network · pass / fail / incomplete exit codes for CI · 197 automated tests
We run the same evaluation set three ways and compare them pairwise.
Your original FP32 model, run deterministically. Ground truth.
The quantized model on reference kernels — the compression mathematics alone, no vendor code.
The compiled artifact on the vendor toolchain, running on your board — in your enclosure, at your ambient. The only place thermal and bandwidth are real.
Compilers silently reassign operations they can't place. The model still returns correct numbers, so every accuracy test passes — while the op runs on a CPU core and quietly eats the latency budget you sized the chip for. We read it straight out of the compiler's own output.
9 operators · 8 accelerated · 1 fell to the CPU. Vela's own words: Unsupported opType — an L2_NORMALIZATION, on Ethos-U55. The build was blocked with exit code 1 before a board existed. Read the full report
Chip vendors will time your model for free. Gauntlet makes the release decision: pass or fail against the limits you set, the cause of any drop, and a record your safety team can keep.
The gate runs in your CI on your own board. Exit 0 ships, 1 blocks, and 2 means a limit went unmeasured, so nothing passes by accident.
A new compiler or SDK version? Rerun and diff against the last qualified build before it reaches a customer.
Find out whether the compression or the chip caused it, and which operations quietly left the accelerator.
Qualify your own model on each candidate, against the same limits, before you commit the bill of materials.
Hashed, reproducible, redaction-safe reports, built to go into an ISO 26262 or IEC 62304 file.
Qualify your customers' models on your own silicon before every compiler release.
Every number in a report traces back to inputs you can hash and a run you can repeat, and 197 automated tests keep it that way.
Pass, fail or incomplete against limits you declare before the run, with exit codes your CI already understands.
Every operator marked accelerated, fell back to the CPU, or refused, read straight from the compiler's own output.
At least 30 timed trials, bootstrap confidence intervals, and per-input answer flips, because two models can score the same and still disagree.
A SHA-256 of every model, artifact and dataset, plus toolchain versions, so any run can be reproduced and audited.
The whole toolchain in one image: the same compiler and kernels, re-runnable years later from its digest.
Paths and machine names come out before a report leaves your bench; every hash, measurement and verdict stays intact.
Priced per product line, never per run, so you can gate every build.
Three places. A full pilot on your model and board, in exchange for a case study, a quote, and a couple of hours a week from one of your engineers.
Fixed price, about four weeks. We connect Gauntlet to your chip and toolchain, qualify your model against your limits, and hand over the report and a regression baseline. Founding-customer rate.
Per product line. The gate in your CI on your own bench, unlimited runs, regression history, and adapters kept current as vendor toolchains change.
The same licence, for your own silicon. No competitor data, no rankings, and no say in what we publish: that is what keeps the verdict worth having.
No. You run Gauntlet's pinned container on your own bench, against your own eval set. The only thing that comes back is the report, and only if you send it.
Today, Arm Ethos-U through the Vela compiler, executed on Arm's simulator for demos, plus a host stand-in for development. NVIDIA Jetson is next. We add a target when a customer needs it, and every report says exactly what the target was.
For a pilot we connect Gauntlet to your board and toolchain. If your hardware hasn't arrived, we can start on our own bench, and the report says which measurements are ours.
About four weeks, at a fixed price. Runs take minutes; the time goes into your toolchain and agreeing your limits, which most teams have never written down.
A benchmark tells you how fast a model runs. Gauntlet tells you whether this exact build should ship, and if it shouldn't, whether the compression or the chip is to blame.
We ship you a pinned container. It runs on your bench, against your eval set, on your board — and the only thing that comes back is the report. No upload, no data room, no security review. We're working with a small number of teams shipping neural networks onto embedded silicon.
Or email adyaan@gauntletlab.com
GAUNTLET