{
 "_about": "Public test set behind the transcriber head-to-head (reports/market/h2h-transcriber-2026-11-18.md): public-domain, synthetic (Kokoro TTS) and CC-BY 4.0 (Multilingual LibriSpeech) audio with references. Re-run any transcriber on it and score with Whisper's normalisers + jiwer.",
 "clips": [
  {
   "id": "clean_tts_a",
   "category": "clean",
   "language": "en",
   "seconds": 54.45,
   "reference": "Good morning, and welcome to the weekly operations briefing. This week we have three topics on the agenda. First, the warehouse move that was postponed in the spring. Second, the new returns policy. Third, a short update on hiring. Please hold your questions until the end of each section, and keep an eye on the chat for links to the documents we discuss. Let's start with the warehouse. The lease on the old building runs out at the end of November, so the move has to be finished before then. The new site is about twenty minutes further north, which means longer routes for the morning drivers. We looked at three options. We could keep both sites open for a month, move everything over a single weekend, or move one product line at a time. The single weekend is the cheapest, but it carries the most risk. If anything goes wrong on Saturday, customers feel it on Monday.",
   "reference_words": 158,
   "recipe": "Kokoro TTS, script lines from 0, voices ['af_heart', 'am_michael']"
  },
  {
   "id": "clean_tts_b",
   "category": "clean",
   "language": "en",
   "seconds": 53.81,
   "reference": "Moving on to the returns policy. Customers currently have fourteen days to send an item back. Our competitors mostly offer thirty. The proposal is to extend our window to thirty days for all items except perishables and custom orders. Finance modelled the change and expects a small rise in returns, offset by higher conversion on the product pages. The customer service team will need new scripts, and the website terms have to be updated before the change goes live. From the support side, I want to flag something. When we changed the delivery promise last year, the volume of calls doubled for about two weeks, mostly from people asking whether the change applied to orders they had already placed. If we announce the new returns window, we should say clearly whether it covers existing orders. A single sentence in the email would save us hundreds of calls.",
   "reference_words": 147,
   "recipe": "Kokoro TTS, script lines from 4, voices ['bm_george', 'af_bella']"
  },
  {
   "id": "clean_tech_names",
   "category": "clean",
   "language": "en",
   "seconds": 59.52,
   "reference": "Welcome back to the Kestrelyn Labs engineering review. I'm Oleksandra Wyrzykowski, and with me is Tjark Ihlenfeld from the platform team. Today we cover the Zephyrine migration. Last month we moved the Quorvex scheduler off the old Brindlemoor cluster and onto Vantablock storage. Tjark, how did the cutover go? Thanks, Oleksandra. The cutover took about four hours. Anselmo Przybylski wrote the rollback plan, and Ngozi Adeyemi ran the load tests from our Llanfyllin office. We hit one problem: the Quorvex workers lost their leases when Zephyrine restarted. We fixed it with a patch to the Hraunfoss driver. Good. For next quarter, Kestrelyn Labs will retire Brindlemoor completely. Ngozi and Anselmo will own the Vantablock capacity plan, and the Llanfyllin team will document the Hraunfoss driver. If you have questions, send them to Tjark Ihlenfeld.",
   "reference_words": 134,
   "recipe": "Kokoro TTS, 15 invented/rare proper nouns, no vocabulary prompt"
  },
  {
   "id": "reader_sonnet_db",
   "category": "accented",
   "language": "en",
   "seconds": 54.81,
   "reference": "My mistress\u2019 eyes are nothing like the sun;\nCoral is far more red, than her lips red:\nIf snow be white, why then her breasts are dun;\nIf hairs be wires, black wires grow on her head.\nI have seen roses damask\u2019d, red and white,\nBut no such roses see I in her cheeks;\nAnd in some perfumes is there more delight\nThan in the breath that from my mistress reeks.\nI love to hear her speak, yet well I know\nThat music hath a far more pleasing sound:\nI grant I never saw a goddess go;\nMy mistress, when she walks, treads on the ground:\n    And yet by heaven, I think my love as rare,\n    As any she belied with false compare.",
   "reference_words": 123,
   "recipe": "LibriVox reader 'db', Sonnet 130, poem only (6.1-60.9 s of the source)"
  },
  {
   "id": "reader_sonnet_gw",
   "category": "accented",
   "language": "en",
   "seconds": 58.32,
   "reference": "My mistress\u2019 eyes are nothing like the sun;\nCoral is far more red, than her lips red:\nIf snow be white, why then her breasts are dun;\nIf hairs be wires, black wires grow on her head.\nI have seen roses damask\u2019d, red and white,\nBut no such roses see I in her cheeks;\nAnd in some perfumes is there more delight\nThan in the breath that from my mistress reeks.\nI love to hear her speak, yet well I know\nThat music hath a far more pleasing sound:\nI grant I never saw a goddess go;\nMy mistress, when she walks, treads on the ground:\n    And yet by heaven, I think my love as rare,\n    As any she belied with false compare.",
   "reference_words": 123,
   "recipe": "LibriVox reader 'gw', Sonnet 130, poem only (14.4-72.7 s of the source)"
  },
  {
   "id": "reader_sonnet_jtm",
   "category": "accented",
   "language": "en",
   "seconds": 50.48,
   "reference": "My mistress\u2019 eyes are nothing like the sun;\nCoral is far more red, than her lips red:\nIf snow be white, why then her breasts are dun;\nIf hairs be wires, black wires grow on her head.\nI have seen roses damask\u2019d, red and white,\nBut no such roses see I in her cheeks;\nAnd in some perfumes is there more delight\nThan in the breath that from my mistress reeks.\nI love to hear her speak, yet well I know\nThat music hath a far more pleasing sound:\nI grant I never saw a goddess go;\nMy mistress, when she walks, treads on the ground:\n    And yet by heaven, I think my love as rare,\n    As any she belied with false compare.",
   "reference_words": 123,
   "recipe": "LibriVox reader 'jtm', Sonnet 130, poem only (12.4-62.9 s of the source)"
  },
  {
   "id": "reader_sonnet_ks",
   "category": "accented",
   "language": "en",
   "seconds": 51.16,
   "reference": "My mistress\u2019 eyes are nothing like the sun;\nCoral is far more red, than her lips red:\nIf snow be white, why then her breasts are dun;\nIf hairs be wires, black wires grow on her head.\nI have seen roses damask\u2019d, red and white,\nBut no such roses see I in her cheeks;\nAnd in some perfumes is there more delight\nThan in the breath that from my mistress reeks.\nI love to hear her speak, yet well I know\nThat music hath a far more pleasing sound:\nI grant I never saw a goddess go;\nMy mistress, when she walks, treads on the ground:\n    And yet by heaven, I think my love as rare,\n    As any she belied with false compare.",
   "reference_words": 123,
   "recipe": "LibriVox reader 'ks', Sonnet 130, poem only (8.8-59.9 s of the source)"
  },
  {
   "id": "reader_sonnet_mac",
   "category": "accented",
   "language": "en",
   "seconds": 43.04,
   "reference": "My mistress\u2019 eyes are nothing like the sun;\nCoral is far more red, than her lips red:\nIf snow be white, why then her breasts are dun;\nIf hairs be wires, black wires grow on her head.\nI have seen roses damask\u2019d, red and white,\nBut no such roses see I in her cheeks;\nAnd in some perfumes is there more delight\nThan in the breath that from my mistress reeks.\nI love to hear her speak, yet well I know\nThat music hath a far more pleasing sound:\nI grant I never saw a goddess go;\nMy mistress, when she walks, treads on the ground:\n    And yet by heaven, I think my love as rare,\n    As any she belied with false compare.",
   "reference_words": 123,
   "recipe": "LibriVox reader 'mac', Sonnet 130, poem only (10.3-53.4 s of the source)"
  },
  {
   "id": "noisy_pink_0db",
   "category": "noisy",
   "language": "en",
   "seconds": 54.45,
   "reference": "Good morning, and welcome to the weekly operations briefing. This week we have three topics on the agenda. First, the warehouse move that was postponed in the spring. Second, the new returns policy. Third, a short update on hiring. Please hold your questions until the end of each section, and keep an eye on the chat for links to the documents we discuss. Let's start with the warehouse. The lease on the old building runs out at the end of November, so the move has to be finished before then. The new site is about twenty minutes further north, which means longer routes for the morning drivers. We looked at three options. We could keep both sites open for a month, move everything over a single weekend, or move one product line at a time. The single weekend is the cheapest, but it carries the most risk. If anything goes wrong on Saturday, customers feel it on Monday.",
   "reference_words": 158,
   "recipe": "clean_tts_a + pink noise at 0 dB SNR"
  },
  {
   "id": "noisy_music_10db",
   "category": "noisy",
   "language": "en",
   "seconds": 53.81,
   "reference": "Moving on to the returns policy. Customers currently have fourteen days to send an item back. Our competitors mostly offer thirty. The proposal is to extend our window to thirty days for all items except perishables and custom orders. Finance modelled the change and expects a small rise in returns, offset by higher conversion on the product pages. The customer service team will need new scripts, and the website terms have to be updated before the change goes live. From the support side, I want to flag something. When we changed the delivery promise last year, the volume of calls doubled for about two weeks, mostly from people asking whether the change applied to orders they had already placed. If we announce the new returns window, we should say clearly whether it covers existing orders. A single sentence in the email would save us hundreds of calls.",
   "reference_words": 147,
   "recipe": "clean_tts_b + synthesised music at 10 dB SNR"
  },
  {
   "id": "noisy_phone",
   "category": "noisy",
   "language": "en",
   "seconds": 54.45,
   "reference": "Good morning, and welcome to the weekly operations briefing. This week we have three topics on the agenda. First, the warehouse move that was postponed in the spring. Second, the new returns policy. Third, a short update on hiring. Please hold your questions until the end of each section, and keep an eye on the chat for links to the documents we discuss. Let's start with the warehouse. The lease on the old building runs out at the end of November, so the move has to be finished before then. The new site is about twenty minutes further north, which means longer routes for the morning drivers. We looked at three options. We could keep both sites open for a month, move everything over a single weekend, or move one product line at a time. The single weekend is the cheapest, but it carries the most risk. If anything goes wrong on Saturday, customers feel it on Monday.",
   "reference_words": 158,
   "recipe": "clean_tts_a through a phone channel (300-3400 Hz, 8 kHz, 8-bit mu-law)"
  },
  {
   "id": "noisy_reader_pink_5db",
   "category": "noisy",
   "language": "en",
   "seconds": 47.12,
   "reference": "My mistress\u2019 eyes are nothing like the sun;\nCoral is far more red, than her lips red:\nIf snow be white, why then her breasts are dun;\nIf hairs be wires, black wires grow on her head.\nI have seen roses damask\u2019d, red and white,\nBut no such roses see I in her cheeks;\nAnd in some perfumes is there more delight\nThan in the breath that from my mistress reeks.\nI love to hear her speak, yet well I know\nThat music hath a far more pleasing sound:\nI grant I never saw a goddess go;\nMy mistress, when she walks, treads on the ground:\n    And yet by heaven, I think my love as rare,\n    As any she belied with false compare.",
   "reference_words": 123,
   "recipe": "LibriVox reader 'njm', Sonnet 130 poem only + pink noise at 5 dB SNR"
  },
  {
   "id": "nonen_de",
   "category": "non-English",
   "language": "de",
   "seconds": 50.58,
   "reference": "denken sie soeben weilten meine gedanken bei ihnen in adelaide und ich w\u00fcnschte mir sie herzaubern zu k\u00f6nnen nun der zauber ist gelungen lachte m\u00fcnchhausen da bin ich und was mich herf\u00fchrt also bei ihren technischen kenntnissen und ihrer erfindungsgabe auf diesem gebiet glaubt der lord keinen besseren ingenieur und kapit\u00e4n f\u00fcr sein weltschiff finden zu k\u00f6nnen als sie das stimmt aber h\u00f6ren sie er schreibt seine entdeckung hebe die trennenden r\u00e4ume des weltalls auf und gestatte der reisen nach dem mond nach den planeten vielleicht gar in die fixsternwelt",
   "reference_words": 90,
   "recipe": "Multilingual LibriSpeech test (german), one speaker, consecutive utterances",
   "mls": {
    "speaker_id": "4705",
    "utterances": [
     "4705_13109_000000",
     "4705_13109_000001",
     "4705_13109_000002"
    ],
    "source": "http://www.archive.org/download/wunderwelten_1911_librivox/wunderwelten_01_mader_64kb.mp3"
   }
  },
  {
   "id": "nonen_fr",
   "category": "non-English",
   "language": "fr",
   "seconds": 50.25,
   "reference": "pendant le second si\u00e8cle je fis serment d'ouvrir tous les tr\u00e9sors de la terre \u00e0 quiconque me mettrait en libert\u00e9 mais je ne fus pas plus heureux dans le troisi\u00e8me je promis de faire puissant monarque mon lib\u00e9rateur d'\u00eatre toujours pr\u00e8s de lui en esprit non ta mort est certaine dit le g\u00e9nie choisis seulement de quelle sorte tu veux que je te fasse mourir le p\u00eacheur le voyant dans la r\u00e9solution de le tuer en eut une douleur extr\u00eame non pas tant pour l'amour de lui qu'\u00e0 cause de ses trois enfants dont il plaignait la mis\u00e8re o\u00f9 ils allaient \u00eatre r\u00e9duits par sa mort la nuit suivante appela sa soeur quand il en fut temps si vous ne dormez pas ma soeur lui dit-elle je vous prie en attendant le jour qui para\u00eetra bient\u00f4t de continuer le conte du p\u00eacheur",
   "reference_words": 142,
   "recipe": "Multilingual LibriSpeech test (french), one speaker, consecutive utterances",
   "mls": {
    "speaker_id": "1406",
    "utterances": [
     "1406_1028_000000",
     "1406_1028_000001",
     "1406_1028_000002"
    ],
    "source": "http://www.archive.org/download/les1001nuits_tome1_0711_librivox/1001nuits1_010_galland_64kb.mp3"
   }
  },
  {
   "id": "nonen_es",
   "category": "non-English",
   "language": "es",
   "seconds": 54.55,
   "reference": "y las almas buscando alg\u00fan alivio se revuelven ansiosas y hacen el mundo que as\u00ed resulta ser del dolor obra el dolor o la nada quien tenga coraz\u00f3n venga y escoja cuando calla el dolor se oye a la muerte las alas tenebrosas batir en los profundos cual si fuesen las olas del mar de la ilusi\u00f3n en que los seres sin rumbo bogan donde se mecen fr\u00e1giles barquillas las fugitivas formas echar fuera la vida y acallar las dom\u00e9sticas cuestiones con el huero fragor de las pol\u00edticas no hagas caso \u00e1 los hombres que se juntan y gritan hojas sus gritos son que el viento lleva mientras en silencio su dolor radica",
   "reference_words": 113,
   "recipe": "Multilingual LibriSpeech test (spanish), one speaker, consecutive utterances",
   "mls": {
    "speaker_id": "11266",
    "utterances": [
     "11266_10604_000000",
     "11266_10604_000001",
     "11266_10604_000002"
    ],
    "source": "http://www.archive.org/download/poesias_1805_librivox/poesias_038_unamuno_64kb.mp3"
   }
  }
 ]
}