Do Keywords Improve Podcast Transcription? We Checked Names, Not Just WER

When a transcript gets a guest’s name wrong, the obvious fix is to give the recognizer the spelling beforehand. The episode title and description often contain exactly that information.

We tried it across sixteen complete podcast recordings. Metadata hints recovered more selected name occurrences: 43 out of 57, compared with 31 out of 57 without hints. But the output also contained more name mentions that did not match the human reference alignment.

The useful answer is therefore more specific than “keywords improve accuracy.” In this test, they improved selected-name recall and reduced selected-name precision. The overall word error rate barely changed.

We make Podtyper and ran this paired experiment ourselves. It uses the same recognizer responses as our full-episode benchmark, not a separate set of recordings selected for this article.

What we sent before transcription

When metadata is available, Podtyper extracts vocabulary from the show title, episode title, description, author and guest metadata. Those terms are sent to the recognizer before it processes the audio.

This is a recognition hint, not a replacement transcript and not a later instruction to rewrite whatever the model heard. The distinction matters: a correct spelling in the output still needs to correspond to the speech.

The run used two conditions for every recording. One had no keyterms. The other used terms derived from the frozen episode metadata. The audio, model and baseline settings stayed the same. We did not add terms after inspecting a failed transcript.

The xAI speech-to-text documentation describes keyterm as a bias toward particular vocabulary. A bias is the right way to think about it: it can make one interpretation more likely, without turning metadata into evidence about what was said.

We counted occurrences, not just unique names

Suppose a name appears five times. Getting it right once should not count as complete success.

Our checklist was frozen before transcription. An occurrence receives credit when the complete phrase is spelled correctly and aligns with the human reference under the fixed scoring rules. The main table uses Rev’s nonverbatim references.

The checklist includes names such as Rebekah Taussig and Gidi Chamdi, and terms such as autocross and intuitive eating. These are selected phrases, not an exhaustive named-entity annotation of every episode. First-name and full-name entries can overlap. Fifty-seven reference occurrences do not mean fifty-seven different people or fifty-seven independent trials.

Recall improved. Precision fell.

Recall asks how many reference occurrences were recovered. Precision asks how many checklist-name mentions in the output were correctly aligned.

For names, the paired figures were:

Showing recall alone would make the result look cleaner than it is. Hints recovered twelve additional aligned occurrences, but unmatched output occurrences rose by eleven.

We have not independently adjudicated each unmatched mention against the audio. A disagreement with a reference alignment can reflect several things, including a reference issue or alignment behavior. Calling all fourteen hallucinations would go beyond the audit.

For a reader, though, the tradeoff is already relevant. A transcript can contain more correctly spelled names while still needing careful checking of where those names appear.

The overall transcript score did not reliably improve

Our primary strict nonverbatim WER moved from 11.37% without context to 11.43% with metadata. The paired difference was +0.060 percentage points, with a show-cluster bootstrap interval of −0.030 to +0.203 percentage points.

That interval includes zero. The run does not establish a consistent improvement in overall WER.

The supplementary normalized verbatim scores were similarly close: 8.30% without context and 8.29% with metadata. We should not turn a tiny numerical difference into a claim that the full transcript became more accurate.

There is no contradiction between these findings. Names are a small part of the word count. A feature can recover some important names without noticeably shifting the aggregate, and it can introduce other disagreements at the same time.

What about the episode description specifically?

We also made title-only requests for three recordings, so we could compare a smaller context source with fuller metadata.

That is a diagnostic check, not a general result about descriptions. Three recordings are too few to support a claim that adding a description reliably improves transcription. The full sixteen-recording comparison tests metadata hints as a package; it does not isolate the contribution of every metadata field.

Descriptions remain a sensible source of spellings when they are accurate and relevant. This experiment gives us a reason to test that more carefully, rather than a reason to promise an improvement for every show.

How to prepare useful context

If your transcription tool accepts vocabulary hints, start with information you knew before reading the transcript: the guest’s name, the company or project being discussed, and a few technical terms that are likely to occur.

For example, an episode description about a conversation with a fictional researcher named “Mara Voss” might supply that name and the episode’s specialist vocabulary. That is an illustrative example, not a measured result or a person from this dataset. Adding every colleague, product and rival organization to the list would give the recognizer more competing suggestions to work with.

A practical sequence is to keep the original metadata, record the exact hint list, run the transcription, then check names against the relevant audio. If you compare settings, freeze the hints for both runs. Otherwise you will not know whether a change came from the recognizer settings or from a better vocabulary list.

The same discipline applies when comparing apps. Test a common condition without hints, then disclose each tool’s context features separately. Giving one product a carefully curated glossary and another only raw audio measures two workflows, not a shared baseline.

The improvement we can actually claim

For this collection and selected checklist, metadata increased correctly aligned name occurrences from 31 to 43. It also reduced name precision and did not reliably reduce overall WER.

That is useful evidence for a context feature. It is also a reason to judge context using more than one number. The scoring section of our full benchmark explains the differences between scoring views, and the downloadable results include the denominators behind the charts.

The original scoring kit preserves the request plan and metadata. There are no manually repaired name spellings hidden behind these scores.