After a transcription test returns more errors than you hoped, changing the parameters is an appealing next step. Perhaps the recognizer needs an explicit language. Perhaps it is skipping quiet speech. Perhaps preserving fillers will bring it closer to the reference.
We tested those possibilities after our full-episode podcast benchmark. The follow-up made 59 additional requests, covering eight configurations and a repeated baseline. The most promising automatic-language candidate disabled the voice-activity gate.
We did not promote it. Its confirmation improvement was small and uncertain, while selected-name F1 fell enough to fail our predeclared rule.
We make Podtyper. This is our own evaluation of grok-voice-transcribe-2.0 on a public English corpus. It does not establish the best configuration for every language, recording condition or application.
What the settings change
We used the parameters documented in xAI’s speech-to-text reference.
language supplies a language bias. format changes the written presentation of items such as numbers and requires a language. filler_words controls whether fillers are retained. vad_threshold sets the voice-activity gate: a lower threshold can admit quieter speech, but can also admit noise; zero disables the gate.
Those controls affect different parts of the task. A number written differently may change text matching without changing the recognized quantity. Preserving a spoken filler may improve agreement with a verbatim reference while making an edited-reference score worse.
That is why we did not judge the experiment using whichever reference happened to produce the nicest result. Parameter selection used a fixed primary view: English-normalized WER against the verbatim reference. We retained the original strict scores, the edited-reference views and exact-spelling vocabulary checks as well.
Three development episodes, eight configurations
The development recordings came from Podcasts in Color, Skidmarks Show and Drinkin’ Bros. Each received the eight configurations below and one independent repeat of the default request.
| Configuration | Parameters added to the original request |
|---|---|
| Defaults | None |
| English | language=en |
| English, formatted | language=en, format=true |
| Fillers | filler_words=true |
| Low VAD | vad_threshold=0.1 |
| VAD off | vad_threshold=0 |
| Fillers, low VAD | filler_words=true, vad_threshold=0.1 |
| English, formatted, fillers, low VAD | All four controls above |
“Defaults” means we omitted those optional controls, not that we hard-coded every server-side default. All conditions kept the same complete source audio and original metadata keywords, with diarization off, no cleanup and no inference retries.
The three episodes multiplied by nine requests produced 27 development calls. The repeated default request was useful because it exposed ordinary response variability. We did not get to assume every small change was caused by a parameter.
The English-only conditions were diagnostic. They could help us understand the behavior of English material, but they were not eligible to become a global default for a multilingual product.
Why VAD-off went to confirmation
On the development recordings, default normalized verbatim WER was 10.74%. VAD-off produced 10.52%. The independent default repeat scored 10.91%.
These results made VAD-off worth checking, not worth shipping immediately. We froze the candidate before confirmation, rather than continuing to adjust it until the next set looked good.
We required more than a lower point estimate. A global change needed at least a 0.2 percentage point reduction in the primary confirmation WER, a show-cluster bootstrap interval entirely below zero, and a selected-name F1 decline no larger than 0.02.
Those thresholds were fixed in the protocol. They prevented us from redefining success after seeing a small result.
The confirmation result
We compared the frozen candidate with fresh default requests across all sixteen recordings, adding 32 calls. The primary confirmation view excluded every episode from the three development shows: eleven recordings from nine other shows.
These recordings had already appeared in the original benchmark. They were held out from parameter development, but they were not a fresh unseen corpus. The distinction limits how broadly we can generalize.
| Confirmation measure | Fresh defaults | VAD-off candidate |
|---|---|---|
| English-normalized verbatim WER | 8.02% | 7.99% |
| Strict verbatim WER | 9.66% | 9.62% |
| Strict nonverbatim WER | 10.73% | 10.67% |
| Selected-name F1, strict verbatim checklist | 0.873 | 0.793 |
The primary paired difference was −0.037 percentage points, where a negative value favors the candidate. The 95% show-cluster bootstrap interval was −0.089 to +0.043 percentage points. We used 10,000 resamples of the nine confirmation shows, with seed 20261027.
The run was complete and all confirmation pairs were present. The three performance gates failed. We therefore kept the existing defaults.
Why the other scores still matter
A tiny primary-score gain can coexist with a meaningful regression in a selected task. In this case, the name result gave us a reason to be particularly cautious. Our metadata study had already shown that names needed their own precision and recall checks.
The all-sixteen-recording confirmation view also stayed available. On that view, normalized verbatim WER was about 8.30% for fresh defaults and 8.29% for the candidate. That includes the development shows, so it is not a substitute for the primary confirmation set.
Changes to filler handling and formatting are similarly easy to overread. Strict matching can improve because the output style moved closer to the reference. The WER explainer shows why a changed text score alone does not demonstrate better recognition of the substantive words.
One implementation detail worth checking
When sending streaming multipart uploads to xAI, option fields need to precede the file field. The documentation notes that fields after the file may be ignored. A parameter experiment is difficult to interpret if the service never received an option in time to apply it.
Our request builder was arranged accordingly. It validates optional controls and keeps omitted settings distinct from explicit false or 0. Zero matters here because it is the actual VAD-off value; treating it as a missing value would quietly run a different experiment.
This is a request-construction detail, not an additional accuracy result. It belongs in the reproducible setup because the setup determines which experiment was run.
What we would test next
A new collection of recordings would tell us more than repeatedly tuning this one. Useful additions would include independently annotated quiet passages, overlap, different accents and other languages. We would keep quotation checks and correction time as separate measurements if those became evaluation targets.
For now, the evidence supports a narrower conclusion: these parameter changes did not justify a global default change under the rule we set. It does not prove that defaults are optimal for every recording, or that a caller with a specific audio condition cannot benefit from a different setting.
The settings reproduction kit includes the frozen protocol, request plans, scorer, responses and decision. The compact series data supplies the plotted values. Re-running live inference incurs provider charges; reading the retained results and rebuilding the figures does not.
We would rather publish a failed promotion than turn a development improvement into a recommendation it did not earn. That leaves the next experiment with a baseline we can still explain.