6 ms·

Team uses Ohio Supercomputer Center to translate lesser-known languages

54 points by breck 9y ago

nategri 9y ago
Awesome! I had an account on OSC for some of my graduate research. Fantastic resource.
- antognini 9y ago
  As did I! I ran a lot of gravitational dynamics experiments on OSC in my graduate student days.
itchyjunk 9y ago
I wonder if they are using deep learning models similar to Google. If so, I wonder if googles Tensor processing units would have been computationally more efficient. Tangentially, can a neural network that has learned most of human languages somehow some up with a better language?
- glup 9y ago
  What do you mean by better? More communicatively efficient? Less ambiguous? Easier to learn? No NN has developed human-like domain general facility with language, but even without doing that we can think about how variants of existing natural languages perform on the above objective functions.
YeGoblynQueenne 9y ago
>> Schuler’s team is working to build a Bayseian sequence model based on statistical analysis to discover a given language’s grammar. It is hypothesized this parsing model can be trained to learn a language and make it syntactically useful. I'm not sure who is "hypothesising" this but the consensus in the field of grammar induction [1] has long been that natural language learning from examples is intractable [2]. I also see some really funny claims about "rapid grammar acquisition" from just 20,000 sentences. For comparison, in English that'd be about 2.8 MB of text (assuming sentences with an average of 25 words and words with an average of 5.5 characters). So, what exactly do you need a supercomputer for, if you're training on just 3 ish MB? Not to mention- 2.8 MB might be OK to train a simple statistical model, but for grammar induction? Oh, you can forget about that! I don't see any results either- just some "excitement" about how the thing "seems to be yielding positive results". So- where are they? Sorry but I have to say that this article is really a bit rubbish. ________________ [1] Grammar induction is thee sub-field of machine learning that researches the possibility of learning languages from examples. [2] Oh, I don't know- try Colin de La Higuera, Grammatical Inference - Learning Automata and Grammars. You won't find any source disagreeing on the intractability issue.
- jnbiche 9y ago
  I had the exact same reaction to this article. Either they've poorly reported on the project, or the project investigator is hoping to make what would be a quantum leap in the state of the art. I am extremely skeptical that this will provide any kind of actionable disaster relief language materials at any time in the next 5 or even 10 years. It's fine and worthy to do this kind of research as basic research, but to market it (and fund it?) as some kind of "developing countries/disaster relief" project seems borderline deceptive.
  - glup 9y ago
    Regarding your second point, #grantsmanship. Unfortunately it's quite hard to get funding for basic research without arguing it will help (=health intervention, diagnostics) or hurt (=help kill the bad guys, or at least manage them). Regarding your first point, this isn't so nutty. Bayesian approaches to PCFG induction are getting increasingly sophisticated. It's not grammar identification in the limit, it's coming up with a probabilistic linguistic representation that improves your predictions (all models are wrong, these are just more useful than having no expectations regarding linguistic structure).
- glup 9y ago
  Of note, children demonstrate pretty robust grammatical expectations for specific syntactic phenomena after hearing comparable number of utterances, with conflicting evidence regarding the availability or reliability of negative evidence. Of course the proofs you cite from formal work are unassailable (=math), but it's possible that "learning" tolerates more error than identification in the limit (see PAC learning for example), and that developing a facility with natural language has more to do with matching the distributions over strings from the adult grammar than inferring the correct grammar (identifying which utterances are in the language, vs. those that aren't). Child language induction is often used as the model for zero-resource natural language processing. This seems to be the direction Dr. Schuler is coming from: http://aclweb.org/anthology/C/C16/C16-1092.pdf http://aclweb.org/anthology/C/C16/C16-1092.pdf.
  - 9y ago
thechao 9y ago
I do GPU driver development, and I love GPUs, but articles like this, which are extended statements of the form “GPUs are faster than CPUs” is just ignorant, or a sales pitch. A lot of times I see this process: 1. CPU code is slow; 2. Port code to GPUs; it’s super-fast! 3. Backport low-cross-communication parallel codebase to CPU: it's even faster! That is, the GPU port forces developers to think about fine- and large- scale threading granularity, and to design low-communication algorithms. However, having done this, the same algorithm runs better on the CPU than the GPU, because the GPU is starved of bandwidth.
- zeusk 9y ago
  /offtopic I'm interning with a GPU driver development team next summer but have no experience with Graphics programming or Shaders. I have some experience with university level OS course and a past internship where I worked on linux but not much else. Can you recommend some resources on GPU from the point of view of an operating system/driver programmer? I tried some of the AMD and Nvidia architecture sheets but were not basic enough for me to grasp and moreover they were oriented towards extracting performance from the respective architectures.
  - thechao 9y ago
    The only API facing GPU programming I’ve ever done is the first dozen-or-so of the NeHe tutorials back in 2003. And lots of single triangles & quads. I’ve written lots of software rasterizers. If you want to understand modern graphics HW, then try to write a fully programmable (shader based) software rasterizer in less than 500 loc of C/++. It’ll really bring to clarity the stages, what they do, and why they’re included. Hint: use function pointers for the shaders.
ryacko 9y ago
When will Codex Seraphinianus be broken?

54 points by breck 9y ago

nategri 9y ago

Awesome! I had an account on OSC for some of my graduate research. Fantastic resource.

antognini 9y ago
As did I! I ran a lot of gravitational dynamics experiments on OSC in my graduate student days.

itchyjunk 9y ago

I wonder if they are using deep learning models similar to Google. If so, I wonder if googles Tensor processing units would have been computationally more efficient. Tangentially, can a neural network that has learned most of human languages somehow some up with a better language?

glup 9y ago
What do you mean by better? More communicatively efficient? Less ambiguous? Easier to learn? No NN has developed human-like domain general facility with language, but even without doing that we can think about how variants of existing natural languages perform on the above objective functions.

YeGoblynQueenne 9y ago

>> Schuler’s team is working to build a Bayseian sequence model based on statistical analysis to discover a given language’s grammar. It is hypothesized this parsing model can be trained to learn a language and make it syntactically useful. I'm not sure who is "hypothesising" this but the consensus in the field of grammar induction [1] has long been that natural language learning from examples is intractable [2]. I also see some really funny claims about "rapid grammar acquisition" from just 20,000 sentences. For comparison, in English that'd be about 2.8 MB of text (assuming sentences with an average of 25 words and words with an average of 5.5 characters). So, what exactly do you need a supercomputer for, if you're training on just 3 ish MB? Not to mention- 2.8 MB might be OK to train a simple statistical model, but for grammar induction? Oh, you can forget about that! I don't see any results either- just some "excitement" about how the thing "seems to be yielding positive results". So- where are they? Sorry but I have to say that this article is really a bit rubbish. ________________ [1] Grammar induction is thee sub-field of machine learning that researches the possibility of learning languages from examples. [2] Oh, I don't know- try Colin de La Higuera, Grammatical Inference - Learning Automata and Grammars. You won't find any source disagreeing on the intractability issue.

jnbiche 9y ago
I had the exact same reaction to this article. Either they've poorly reported on the project, or the project investigator is hoping to make what would be a quantum leap in the state of the art. I am extremely skeptical that this will provide any kind of actionable disaster relief language materials at any time in the next 5 or even 10 years. It's fine and worthy to do this kind of research as basic research, but to market it (and fund it?) as some kind of "developing countries/disaster relief" project seems borderline deceptive.
- glup 9y ago
  Regarding your second point, #grantsmanship. Unfortunately it's quite hard to get funding for basic research without arguing it will help (=health intervention, diagnostics) or hurt (=help kill the bad guys, or at least manage them). Regarding your first point, this isn't so nutty. Bayesian approaches to PCFG induction are getting increasingly sophisticated. It's not grammar identification in the limit, it's coming up with a probabilistic linguistic representation that improves your predictions (all models are wrong, these are just more useful than having no expectations regarding linguistic structure).
glup 9y ago
Of note, children demonstrate pretty robust grammatical expectations for specific syntactic phenomena after hearing comparable number of utterances, with conflicting evidence regarding the availability or reliability of negative evidence. Of course the proofs you cite from formal work are unassailable (=math), but it's possible that "learning" tolerates more error than identification in the limit (see PAC learning for example), and that developing a facility with natural language has more to do with matching the distributions over strings from the adult grammar than inferring the correct grammar (identifying which utterances are in the language, vs. those that aren't). Child language induction is often used as the model for zero-resource natural language processing. This seems to be the direction Dr. Schuler is coming from: http://aclweb.org/anthology/C/C16/C16-1092.pdf http://aclweb.org/anthology/C/C16/C16-1092.pdf.
- 9y ago

thechao 9y ago

I do GPU driver development, and I love GPUs, but articles like this, which are extended statements of the form “GPUs are faster than CPUs” is just ignorant, or a sales pitch. A lot of times I see this process: 1. CPU code is slow; 2. Port code to GPUs; it’s super-fast! 3. Backport low-cross-communication parallel codebase to CPU: it's even faster! That is, the GPU port forces developers to think about fine- and large- scale threading granularity, and to design low-communication algorithms. However, having done this, the same algorithm runs better on the CPU than the GPU, because the GPU is starved of bandwidth.

zeusk 9y ago
/offtopic I'm interning with a GPU driver development team next summer but have no experience with Graphics programming or Shaders. I have some experience with university level OS course and a past internship where I worked on linux but not much else. Can you recommend some resources on GPU from the point of view of an operating system/driver programmer? I tried some of the AMD and Nvidia architecture sheets but were not basic enough for me to grasp and moreover they were oriented towards extracting performance from the respective architectures.
- thechao 9y ago
  The only API facing GPU programming I’ve ever done is the first dozen-or-so of the NeHe tutorials back in 2003. And lots of single triangles & quads. I’ve written lots of software rasterizers. If you want to understand modern graphics HW, then try to write a fully programmable (shader based) software rasterizer in less than 500 loc of C/++. It’ll really bring to clarity the stages, what they do, and why they’re included. Hint: use function pointers for the shaders.

ryacko 9y ago

When will Codex Seraphinianus be broken?