3 ms·

Inferring rules from shed loads of examples instead of coding them explicitly is only superior for big language models because machines are faster at processing

by yorwba 4y ago

Inferring rules from shed loads of examples instead of coding them explicitly is only superior for big language models because machines are faster at processing examples than humans are at implementing a comprehensive set of rules for the machines to follow. So it's a trade-off between slow and expensive human labor vs. fast and comparatively cheap machine labor. When it's a human learning a language, that speed difference doesn't apply. When the choice is between a slow human processing a lot of examples vs. a slow human processing a small number of examples plus a description of the rule governing them, it's no longer so obvious that the first approach is better.

wokwokwok 4y ago
> it's no longer so obvious that the first approach is better. That’s an easy arm chair argument to make. I won’t argue it because, because there’s plenty of literature about the topic of immersion vs book learning for languages. I’ll just tldr; I spent a lot of time on this, and I am convinced you are just flat out wrong. …but, instead of arguing over a difference in opinion, how about this? Given that in digital products space is not at a premium, why not, instead of one or two contrived examples of each particle, have 50 examples of usage of each? It’s easy. It’s addresses the problem of rule based approach vs examples by having both. …because, a trivial example and a rule is quite useless for real world purposes; and if you cant make a bunch of different examples, you don’t understand the particle and shouldn’t be trying to teach it to people. I mean, here’s a specific example: > Just like the particle に (ni), へ (e) is used to indicate destination or direction. They are translated as to in English and are often interchangeable. > e is also used as a term to which we turn our actions. > Also in this case it can be replaced by に. Are they totally interchangeable? Nope. When do you use which? It depends. Guess how you figure that out? Examples. In many cases, both in English and Japanese rules become long lists of examples and specific cases when they are made complex enough to encapsulate the full range of when to use when. Language models show categorically that predictive models of language work well. Anecdotally, the same approach works well for people too. Why not offer both approaches? There’s totally a place for rules too. Let people pick what they want to use, try both and pick what works for them?
- ad404b8a372f2b9 4y ago
  I'm not sure what qualifies his argument as an arm-chair argument relative to yours. Grammar is a shortcut to a basic understanding, the primary complaint of users of Duolingo is the lack of grammar lessons, and having to infer rules solely through examples. I don't know of anyone who is arguing for a single approach to language learning either. Pretty much everyone who has ever worked through a textbook recommends supplementing with real content, examples, media, conversations and so on. It doesn't invalidate the use of a textbook focused on grammar. As for language models they're pretty much entirely irrelevant to human learning.
  - wokwokwok 4y ago
    Language models show that language is predictive, not rule based. Every bit of research in the last 30 years shows this. If you want to ignore that and use rule based approaches, that’s fine, but you’re a) wrong and b) it doesn’t work. /shrug Of course people want rules, they think it’s a shortcut to learning. …but there is no shortcut to learning. You have to do the hard work.
    - yorwba 4y ago
      > Language models show that language is predictive, not rule based. Training language models by making them predict missing words in a text is a trick to optimize their ability to extract information from text. That doesn't mean that making humans do the exact same task is the optimal way to learn a language. And it definitely doesn't mean there are no rules. Millenia of linguistic research have succeeded in identifying a lot of rules in every language. You don't have to learn them explicitly, but it speeds things up a bit. If you think there are no shortcuts to learning, you're either blind to the shortcuts you used, or you missed out on a lot. Like dictionaries. They just tell you what a word means in a language you can understand! That saves a lot of time compared to having to figure out everything from context. The Wiktionary entry for the particle に https://en.wiktionary.org/wiki/%E3%81%AB#Particle https://en.wiktionary.org/wiki/%E3%81%AB#Particle lists eight different uses and has a note on the different nuance of ヘ. Do you think this entry is completely useless? If someone looks up に, what information should they get?
ajconway 4y ago
The rules of a natural language are not prescriptive, but descriptive. Studying the rules may work if the language's grammar is similar to that of one's native language.
Bakary 4y ago
The answer usually comes from experience. Once you've learned a language through immersion and deliberate practice, you can never go back to the classroom setting. Language learning gets obfuscated by the profitable industry that has sprung up around it, but in the end it's clear which method works the best.

by yorwba 4y ago

wokwokwok 4y ago

> it's no longer so obvious that the first approach is better. That’s an easy arm chair argument to make. I won’t argue it because, because there’s plenty of literature about the topic of immersion vs book learning for languages. I’ll just tldr; I spent a lot of time on this, and I am convinced you are just flat out wrong. …but, instead of arguing over a difference in opinion, how about this? Given that in digital products space is not at a premium, why not, instead of one or two contrived examples of each particle, have 50 examples of usage of each? It’s easy. It’s addresses the problem of rule based approach vs examples by having both. …because, a trivial example and a rule is quite useless for real world purposes; and if you cant make a bunch of different examples, you don’t understand the particle and shouldn’t be trying to teach it to people. I mean, here’s a specific example: > Just like the particle に (ni), へ (e) is used to indicate destination or direction. They are translated as to in English and are often interchangeable. > e is also used as a term to which we turn our actions. > Also in this case it can be replaced by に. Are they totally interchangeable? Nope. When do you use which? It depends. Guess how you figure that out? Examples. In many cases, both in English and Japanese rules become long lists of examples and specific cases when they are made complex enough to encapsulate the full range of when to use when. Language models show categorically that predictive models of language work well. Anecdotally, the same approach works well for people too. Why not offer both approaches? There’s totally a place for rules too. Let people pick what they want to use, try both and pick what works for them?

ad404b8a372f2b9 4y ago
I'm not sure what qualifies his argument as an arm-chair argument relative to yours. Grammar is a shortcut to a basic understanding, the primary complaint of users of Duolingo is the lack of grammar lessons, and having to infer rules solely through examples. I don't know of anyone who is arguing for a single approach to language learning either. Pretty much everyone who has ever worked through a textbook recommends supplementing with real content, examples, media, conversations and so on. It doesn't invalidate the use of a textbook focused on grammar. As for language models they're pretty much entirely irrelevant to human learning.
- wokwokwok 4y ago
  Language models show that language is predictive, not rule based. Every bit of research in the last 30 years shows this. If you want to ignore that and use rule based approaches, that’s fine, but you’re a) wrong and b) it doesn’t work. /shrug Of course people want rules, they think it’s a shortcut to learning. …but there is no shortcut to learning. You have to do the hard work.
  - yorwba 4y ago
    > Language models show that language is predictive, not rule based. Training language models by making them predict missing words in a text is a trick to optimize their ability to extract information from text. That doesn't mean that making humans do the exact same task is the optimal way to learn a language. And it definitely doesn't mean there are no rules. Millenia of linguistic research have succeeded in identifying a lot of rules in every language. You don't have to learn them explicitly, but it speeds things up a bit. If you think there are no shortcuts to learning, you're either blind to the shortcuts you used, or you missed out on a lot. Like dictionaries. They just tell you what a word means in a language you can understand! That saves a lot of time compared to having to figure out everything from context. The Wiktionary entry for the particle に https://en.wiktionary.org/wiki/%E3%81%AB#Particle https://en.wiktionary.org/wiki/%E3%81%AB#Particle lists eight different uses and has a note on the different nuance of ヘ. Do you think this entry is completely useless? If someone looks up に, what information should they get?

ajconway 4y ago

The rules of a natural language are not prescriptive, but descriptive. Studying the rules may work if the language's grammar is similar to that of one's native language.

Bakary 4y ago

The answer usually comes from experience. Once you've learned a language through immersion and deliberate practice, you can never go back to the classroom setting. Language learning gets obfuscated by the profitable industry that has sprung up around it, but in the end it's clear which method works the best.