5 ms·

Looks very cool. One thing I have been looking for, which this doesn't seem to cover (at least I didn't see any mention of IPA in the model documentation), is a

• shenberg 3mo ago
  We take a lot of shortcuts when speaking, it's actually much harder to transcribe phonemes than to transcribe words, even when aware of the language being spoken. Some models have been trained for the task (e.g. look at https://huggingface.co/spaces/KoelLabs/IPA-Transcription-EN https://huggingface.co/spaces/KoelLabs/IPA-Transcription-EN ), but the error rate is really high.
  • rmunn 3mo ago
    There are, broadly, two kinds of audio recordings that linguists want to transcribe. One is native speakers telling traditional stories, where they're speaking naturally and taking the natural shortcuts (such as "wanna" and "gonna" in English). The other is native speakers reading words (or short example sentences) very carefully and distinctly, so that the linguist can listen to the recording over and over to learn how to pronounce the word right. In those recording, they'll say "want to" and "going to" rather than "wanna" and "gonna". Thanks for the pointer; I'll check out that model and see if it handles the "slowly and carefully" type of recording better than the "natural speaking" type. (And depending on what kinds of errors the model makes, even the recordings where it makes errors can prove useful: for example, a linguist studying regional variations in speech would want the model to produce the IPA for "gonna" rather than "going to").
  • uoaei 3mo ago
    Dialects degenerate phonemes that would otherwise occupy identity relations between different utterances of the same "word" (word/concept mutable hyperobject as is the standard in any socially-relevant spoken language) which would be a bit of irony in this thought experiment since common knowledge dictates that more data samples must be present in the dataset (not less, as in rarely-spoken languages) to associate separate pronunciations of utterances representing the same underlying concept. However very-rarely-spoken languages probably don't have distinct dialects since so much focus is put on mutual intelligibility with the few members of the group that remain fluent in that language. It's not outside the realm of possibility that small speaking communities nonetheless fractionate into dialectical specialities but that seems increasingly unlikely as the fervor for preserving/recognizing dying languages increases, and global instant communication continues to become more commonplace. Example: Schwabisch is wild and would be phonetically transcribed very differently from Hochdeutsch which is its ostensible language progenitor (technically more a cousin than an ancestor in the lineage of language evolution), but if the goal is merely to focus the model purely on phonetic transcription then you can add additional post-processing layers which map sounds to core concepts shared across dialects for actual translation. But I like your idea of interacting with the intermediate elements to familiarize yourself at least with the phonetic patterns, we humans are still thinkers enough to infer patterns of grammar and semantics from these building blocks just as we have done for the entire history of the species/lineage before written representations of language came along (relatively late -- evidence of script cropped up only once civilization had centralized to a sufficient degree to make economics non-local and non-trivial). tl;dr the big words: it's not til you collect enough spoken samples of the dead(ish/dying) language being spoken that the local idiosyncracies are discovered, luckily linguists are smart enough to probably anticipate and certainly post-process language snippets to grasp the common structures for this or that given language.
    • rmunn 3mo ago
      > ... very-rarely-spoken languages probably don't have distinct dialects ... That's true if you mean "very rarely spoken" literally, as in even the native speakers don't get to use it very often. But many languages aren't widely spoken (such as only in a certain geographical area, which sometimes is only a single village, or other times a small number of villages). But inside that area, they are frequently spoken. And you might be surprised how many of those small-geographic-area languages still have distinct dialects. For example: my wife (a linguist) did her master's thesis on the pronunciation of a language with about 7,000 speakers, and identified how many distinct dialects there were. (Which is why I know a little bit about this). She recorded native speakers from all 13 (I think it was 13, but it might have been 14) villages where the language was spoken, and found five different dialects, which she grouped into two "main" dialects. (Think American vs British in the English language, with subdivisions into Midwest, New York, and New England accents and so on, and you'll have the right general idea — though these dialects were closer to each other in sound than Midwest vs New York). I'd have to go reread her thesis to give you any more details. But this was a language that was only spoken in a small geographic area, but it was frequently spoken, because that was the main language of those villages. (The country's official national language is what the kids learned in school, but some of the people, mostly those 60 years old or older, hadn't gone to school, because the first government school in their area was only built 60 years ago -- so they only spoke their minority language, and not the country's language, and their kids had to translate for them if they had to leave their village and go shopping in a major town).
• sipjca 3mo ago
  Largely this is out of scope for the library, mainly because I’m not aware of many models supporting this. but if there are models which support this would be happy to support
• verst 3mo ago
  I would love such a model. My wife's family is Iu Mien which is a sub group of the Dao/Yao Chinese ethnic minority. Mien is its own language but most speakers are essentially illiterate. I'm good with language but there simply isn't a course or any books for learning the language. Not much in writing to begin with given the high illiteracy rate. I would love to build a translation system - project Hail Mary style :)
  • yorwba 3mo ago
    FWIW someone wrote their PhD thesis on the Iu Mien language: https://opal.latrobe.edu.au/articles/thesis/An_Iu_Mien_grammar_a_tool_for_language_documentation_and_revitalisation/21854526 https://opal.latrobe.edu.au/articles/thesis/An_Iu_Mien_gramm... It's not a textbook, but that doesn't mean you can't try to use it as a textbook. For example, there are some thoroughly-analyzed example texts. Also, the acknowledgments section mentions the Iu Mien Literacy Project, which has published a short course on the very basics, as it turns out: https://iumienliteracyprojects.com/resources/ https://iumienliteracyprojects.com/resources/ I suspect the 925-page PDF contains some more hints to additional resources like that. :)
    • verst 3mo ago
      Unfortunately there are two sub groups of Iu Mien with somewhat different vocabulary as well - and my wife is from the other group. I had found this pdf document previously - thanks for the pointer! I'll see what I can do with it.
• simsla 3mo ago
  Automatic Phoneme Recognition (APR). There are some models that do this, but they're only so-so.
• KingMob 3mo ago
  I learned IPA for Thai, and as part of that, I also read that a lot of professional linguists still find IPA too limited. IPA seems very comprehensive from my amateur perspective, but apparently a lot of modern linguists still extend it or roll their own.
• msm_ 3mo ago
  It sounds like the only way this would make sense is if such model knew the range of sounds it expects to "hear". There's a lot of possible sounds that IPA knows about, but world languages only use a fraction of them at once. Think English dark and light "l" (ball/light) or aspirated "p" (pin/spin) - some languages contrast them, while in english the difference is not meaningful. Or maybe linguists are actually interested in having maximally faithful IPA representation and manually normalizing it? You are clearly way more knowledgeable about that topic than I, so I'm curious what you think.
  • rmunn 3mo ago
    The linguists I know are not necessarily a representative sample... but they're mostly interested in just that: maximally faithful IPA representation. They want to know if the speaker switches back and forth between aspirated p and unaspirated p on the same word, because that tells them something about the language — that aspirated consonants are not meaningful. Linguists studying the sounds of a language, its phonology, often want to find "minimal pairs", words that differ by only a single sound. For example, din and tin in English. You record a native speaker saying both words, and telling you their meaning, and then you play back either recording A or recording B to other native speakers and ask them which word it is. If they can identify the word every time, then you've found two sounds that are meaningfully distinct in this language. (Some languages don't distinguish the d and t sounds, but English does). But if the native speakers go 50/50 on which word it is, or ask to hear it in a sentence for clarification because it could be two or three different words, then you've found a pair of sounds that this language does not distinguish. (Note that you're playing the words in isolation, because sentence context might make it obvious which one it is, e.g. you can't tell if an English speaker is saying their or there until you hear more words of the sentence). So yes, the linguists that I know (who, again, are not necessarily a representative sample) are interested in as faithful an IPA representation as they can get, because that inconsistent transcription will give them many clues about the language. It still all has to be checked, because that switching back and forth between aspirated and unaspirated p (for example) could have been an artifact of a poor-quality microphone not picking up the aspiration, or a windy day causing aspiration sounds that the speaker never said, or the speech-to-text model making a mistake. But I watched my wife listen to the same two-second recording on loop over and over, trying to be certain of which sound she was hearing in the middle of the word. Double-checking the output of the model would (in most cases) only require listening to the audio once or twice, not half-a-dozen times like she typically did while researching her thesis. At the time, LLMs were not really a thing yet, but if she were doing her thesis today I bet a speech-to-IPA model would have saved her quite a lot of time — but only if it output every distinction, even the ones not meaningful in the target language. The "maximally faithful" representation, as you put it.
• akreal 3mo ago
  This field got reactivated recently, some new models were released (in the order of release, but the last one seems the best at the moment): 1. ZIPA https://github.com/lingjzhu/zipa https://github.com/lingjzhu/zipa 2. POWSM https://huggingface.co/espnet/powsm https://huggingface.co/espnet/powsm 3. PhoneticXEUS https://github.com/changelinglab/PhoneticXeus https://github.com/changelinglab/PhoneticXeus I would be curious to know how to help people to use these models, or what kind of tasks they could be applied to.