4 ms·

> I've been told in school that English has spelling rules. Excluding a couple hundred irregularities, native speakers (and simple rule-based algorithms) can a

by retrac 5y ago

> I've been told in school that English has spelling rules. Excluding a couple hundred irregularities, native speakers (and simple rule-based algorithms) can accurately predict the pronunciation of 85 - 90% of common words with most of the errors being minor issues about vowel reduction to schwa and stress placement. For the incredulous, here's 56 table-based search-and-replace rules that get you to about 85%: https://www.zompist.com/spell.html https://www.zompist.com/spell.html Various other authors have their own slight variations on this, but typically it comes to 40 - 70 algorithmically-applicable rules. I wish more students got taught some variation on spelling rules like that, because they're much closer to the real rules of English spelling. It's only slightly worse than French, who we ripped off most of the modern orthography from. (Such table-based rules for French get above 95% pretty easily.) The thing is that, in both languages, this is a decidedly one-way process. Guessing how something is spelled from the sound is hopeless.

FabHK 5y ago
Right, 85% for English sounds plausible. For Spanish, I'd venture much closer to 100%. For Chinese, around two thirds of characters contain a phonetic component. In other words, English is around half way between Spanish and Chinese :-) Oh, and in Spanish it is bijective: if you know the pronunciation, you pretty much know the spelling also.
- ralferoo 5y ago
  The counter example for Spanish is b/v which is pretty much the same sound in most dialects nowadays, and it seems increasingly common to see misspellings online, e.g. baca for cow.
morsch 5y ago
Would be 99% for Standard German with its quasi deterministic pronunciation. Going from sound to spelling isn't hopeless, either.
miki123211 5y ago
Getting to 80% may be easy, but getting to 100% is close to an AGI problem. Words like read (present vs. past tense) or Nice (the adjective vs. the French city) can be pronounced in two different ways. Same with an address / to address, a present / to present etc. Determining which way to use is impossible without understanding the text. You can get pretty far by using some simple heuristics. For example, when the word read is preceded by "have", you almost always want to use the past tense pronunciation, but if it's preceded by "to", you want the present tense. At some point, though, you will hit the "time flies like an arrow" vs. "fruit flies like a banana" problem, and simple sentence analysis will not be enough. Another issue with English is that it likes to mash words together, and most pronunciation rules are not ready for that. While "Black Lives Matter" might be easy to pronounce, #blacklivesmatter is not. Getting this wrong sometimes leads to hilarious results[1]. I'm blind and I use a speech synthesizer daily, so I'm familiar with most of these issues. I haven't found a single piece of software that actually solved them well. This is still much better than languages like Hebrew, though, as they require you to manually insert vowels in the appropriate places, a task modern computers are not ready for. [1] https://www.boredpanda.com/worst-domain-names/?utm_source=google&utm_medium=organic&utm_campaign=organic https://www.boredpanda.com/worst-domain-names/?utm_source=go...
- retrac 5y ago
  Yes, I was thinking of speech synthesizers when I wrote my comment :) Closing the gap is devillish and requires increasingly complex grammatical analysis and more and more rules. One of the big barriers for early speech synthesizers was enough RAM/ROM to cram all the text-to-phonetic transformation rules. I believe that modern approaches mostly use deep learning. And definitely still aren't perfect. I don't use speech synthesizers regularly personally as a tool, but synthesized speech is common enough that I do hear obvious stress errors in synthesized speech every day. Probably because English orthography is most opaque about stress, which basically doesn't exist as far as the writing system is concerned.
  - aasasd 5y ago
    > synthesized speech is common enough I'm curious: where do you encounter much synthesized speech? In my ‘fairly general’ usage (if there is such a thing), I only hear it in YouTube tutorials when the author is totally reluctant to both use their voice and do on-screen text.
lordnacho 5y ago
I caught up with my old French teacher, who also taught German in high school. He told me there was value in doing dictations in French, but in German he just stopped because it was so simple. Kids would barely ever get anything wrong. I have to say I agree, having done a bit of both. English has enough weird spellings that kids seem to have to practice it a lot. 85-90% isn't actually all that high, you're going to run into a weird one pretty soon if you're writing an essay.
andruby 5y ago
> Excluding a couple hundred irregularities, native speakers can accurately predict the pronunciation of 85 - 90% of common words That sounds pretty bad actually. You're saying most _native_ speakers can't predict the pronunciation of 10-15% of common words? I wasn't great in Spanish at school, but the teacher was mostly correct that "you write it like it sounds". Much more so than English at least. (for context: I'm native Dutch/French, but speak mostly English, and know a bit of Spanish and Portuguese)

by retrac 5y ago

FabHK 5y ago

Right, 85% for English sounds plausible. For Spanish, I'd venture much closer to 100%. For Chinese, around two thirds of characters contain a phonetic component. In other words, English is around half way between Spanish and Chinese :-) Oh, and in Spanish it is bijective: if you know the pronunciation, you pretty much know the spelling also.

ralferoo 5y ago
The counter example for Spanish is b/v which is pretty much the same sound in most dialects nowadays, and it seems increasingly common to see misspellings online, e.g. baca for cow.

morsch 5y ago

Would be 99% for Standard German with its quasi deterministic pronunciation. Going from sound to spelling isn't hopeless, either.

miki123211 5y ago

Getting to 80% may be easy, but getting to 100% is close to an AGI problem. Words like read (present vs. past tense) or Nice (the adjective vs. the French city) can be pronounced in two different ways. Same with an address / to address, a present / to present etc. Determining which way to use is impossible without understanding the text. You can get pretty far by using some simple heuristics. For example, when the word read is preceded by "have", you almost always want to use the past tense pronunciation, but if it's preceded by "to", you want the present tense. At some point, though, you will hit the "time flies like an arrow" vs. "fruit flies like a banana" problem, and simple sentence analysis will not be enough. Another issue with English is that it likes to mash words together, and most pronunciation rules are not ready for that. While "Black Lives Matter" might be easy to pronounce, #blacklivesmatter is not. Getting this wrong sometimes leads to hilarious results[1]. I'm blind and I use a speech synthesizer daily, so I'm familiar with most of these issues. I haven't found a single piece of software that actually solved them well. This is still much better than languages like Hebrew, though, as they require you to manually insert vowels in the appropriate places, a task modern computers are not ready for. [1] https://www.boredpanda.com/worst-domain-names/?utm_source=google&utm_medium=organic&utm_campaign=organic https://www.boredpanda.com/worst-domain-names/?utm_source=go...

retrac 5y ago
Yes, I was thinking of speech synthesizers when I wrote my comment :) Closing the gap is devillish and requires increasingly complex grammatical analysis and more and more rules. One of the big barriers for early speech synthesizers was enough RAM/ROM to cram all the text-to-phonetic transformation rules. I believe that modern approaches mostly use deep learning. And definitely still aren't perfect. I don't use speech synthesizers regularly personally as a tool, but synthesized speech is common enough that I do hear obvious stress errors in synthesized speech every day. Probably because English orthography is most opaque about stress, which basically doesn't exist as far as the writing system is concerned.
- aasasd 5y ago
  > synthesized speech is common enough I'm curious: where do you encounter much synthesized speech? In my ‘fairly general’ usage (if there is such a thing), I only hear it in YouTube tutorials when the author is totally reluctant to both use their voice and do on-screen text.

lordnacho 5y ago

I caught up with my old French teacher, who also taught German in high school. He told me there was value in doing dictations in French, but in German he just stopped because it was so simple. Kids would barely ever get anything wrong. I have to say I agree, having done a bit of both. English has enough weird spellings that kids seem to have to practice it a lot. 85-90% isn't actually all that high, you're going to run into a weird one pretty soon if you're writing an essay.

andruby 5y ago

> Excluding a couple hundred irregularities, native speakers can accurately predict the pronunciation of 85 - 90% of common words That sounds pretty bad actually. You're saying most _native_ speakers can't predict the pronunciation of 10-15% of common words? I wasn't great in Spanish at school, but the teacher was mostly correct that "you write it like it sounds". Much more so than English at least. (for context: I'm native Dutch/French, but speak mostly English, and know a bit of Spanish and Portuguese)