6 ms·

When I was studying Computational Linguistics I kept running into the unspoken question: given that Google Translate already exists, what is even the point of a

• hnfong 4y ago
  I learnt some very basics of computational linguistics since it was related to a side project. I kept wondering why people were spending huge amounts of resources into tagging and labelling corpora of thousands of words, while to me it seems that in theory it should be possible to feed wikipedia (of a certain language) into a program and have it spit out some statistically correct rules about words and grammar. I guess the same intuition led to these new AI technologies...
  • hnbad 4y ago
    I think a huge part is that computational linguistics still chases the idea of a universal language model, which may simply not be possible. I haven't followed the science in general linguistics but something feels off when most of the information ends up being tagged onto nil particles (i.e. parts of speech present neither in utterances nor written language and not affecting intonation or otherwise being detectable except by contrasting the structure with related languages).
    • hnfong 4y ago
      In a sense the model is universal. It's just a 100GB (give or take) neural network. And apparently (or so I heard, I think) feeding transformer models training data of Language A could improve its ability to understand Language B. So maybe there's something truly universal in some sense.
  • vkazanov 4y ago
    The secret is that there are no grammars in our brains. Rules are statistical, not precise. Rules, idioms are fluid and... statistical. We're a bit more specialised than these new models. But that's it, really.
    • xp84 4y ago
      ^ This. I think the more we internalize the fact that we're also basically LLMs, the more we'll realize that there likely isn't some hard barrier beyond which no AI can climb. If you watch the things kids who are learning language say, you'll see the same kinds of slip-ups that belie the fact that they don't yet understand all the words themselves, but nobody thinks that 2-year-olds aren't people or thinks they will never learn to understand these concepts.
      • oska 4y ago
        Internalising the idea (not 'fact') that 'we're basically LLMs' will only take you to a place of deep sterility, delusion and nihilism.
        • vkazanov 4y ago
          Why? Do you prefer magical "handwavy" thinking? A lot of things just fall in place if you accept an obvious fact that the brain is just a powerful enough function with certain inputs and outputs. Neurons are not that complicated. There are interesting consequences here though: no stable grammars, no truly uniform thinking, no singular way to understand people.
          • oska 4y ago
            The 'handwavey' thinking here is from those who state as a 'fact' that the human brain is an LLM
            • vkazanov 4y ago
              Well, I didn't say a brain is just an LLM. It's more complicated than just this. But what LLM showed is that it works using the same simple building blocks + specialised parts of the brain: memory, constant relearning and world input, etc. So in this particular case of LLMs we've managed to optimise our way around having specialised brain structures using a powerful enough math function. Next steps require improvements in how the model is trained, how we can reduce the amount of training data, what additional machinery might be necessary, etc... But, damn it, this very thing that makes humanity possible - our language - is solved now. Natural language is a solved problem now. That very thing that makes complex societies possible - it's done. This is the fact. And it is crazy. EDIT: typos
  • sp332 4y ago
    English Wikipedia is the largest. Wikipedia in other languages would be less useful.
  • tkgally 4y ago
    I've just started testing GPT-4 on translation from Japanese to English, but it seems to blow Google Translate out of the water. It was particularly good with a novel excerpt. I encourage people to try it with a variety of languages and texts to see if those results hold up.
    • mtlmtlmtlmtl 4y ago
      It would be useful to know whether you know both languages or just English. Not trying to poke holes, just clarifying.
      • tkgally 4y ago
        No problem. My first language is English, but I worked as a Japanese to English translator for twenty years and have written books in Japanese. My personal website is linked from my user page.
        • mtlmtlmtlmtl 4y ago
          Thanks. Refreshing to see one of these claims made by someone actually qualified to verify them! I might have to play with some Norwegian <-> English translation at some point. Should be even more effective than for Japanese due to the common germanic origins.
• lisasays 4y ago
  Given that Google Translate already exists, what is even the point of all of this? Because for the other 20 percent it's plainly -not- good enough. It can't even produce an acceptable business letter in a resource-rich target language, for example. It just gets you "a good chunk of the way there." And there's no evidence that either (1) throwing exponentially more data at the problem with see matching gains in accuracy or (2) this additional data will even be available.
  • jjoonathan 4y ago
    Yeah... Google Translate is still occasionally translating good/item as "baby" on taobao. "Return Defective Baby" was hilarious for a year or two, but that was ~8 years ago IIRC, and now it just stands as a reminder that Google Translate still has a considerable way to go.
    • JohnFen 4y ago
      Indeed. Google Translate is just barely useful. Whenever I use it to translate to English, what I get is generally poor. It's good enough to understand the gist of what the original text said, but that's about it. Fortunately, most of the time, understanding the gist is enough.
    • mtlmtlmtlmtl 4y ago
      Google translate lets you correct a translation, right? Did people stop doing this at some point? Maybe after the advent of massively addictive social media, people more often ended up screenshotting it and sharing it for likes instead of correcting it. I'm sure Google have stats on this but no idea whether they're public.
      • humanizersequel 4y ago
        I haven't seen the option to correct a translation in years.
        • simongray 4y ago
• dogcomplex 4y ago
  Ask a toddler how the world works and you'll get a very similar response. It is entirely likely the 80%-of-human-intelligence barrier is not a "dead end" but merely a temporary limitation until these models are made to hone their understanding and update over time (i.e. get feedback) instead of going for zero-shot perfection. The GPT models incorporating video should start developing this "memory" naturally as they incorporate temporal coherence (time) into the model. The fact we got this far through brute force is just insanely telling. This is a natural phenomena we're stumbling upon, not something crafted by humans. Also - fun fact, the Facebook Llama model that fits on a Raspberry Pi and is almost as good as GPT3? Also basically brute force. They just trained it a lot longer and it shrunk the model. Food for thought.
• esperent 4y ago
  Google translate works amazingly will on languages with a similar grammar (or at least, it works so on European languages, which I have the experience to judge). However, translation of more distant languages is pretty terrible. Vietnamese to English is something I use Google translate for everyday and it's a mess. I can usually guess what the intended meaning was but if you're translating a paragraph or more it won't even be able to translate the same important subject words consistently throughout. Throw in any kind of slang or abbreviations (which Vietnamese people use a lot when messaging each other) and it's completely lost.
• leroy-is-here 4y ago
  I personally think that humans easily apply structure to language that doesn’t really exist. In fact, we restructure our languages daily, as individuals, when communicating verbally and through text. We make up words and shorthands and abbreviations and portmanteaus. But I think the brain simply makes connections between words and things and the structure of speaking those words is interpreted like audio or visuals in our brains — just patterns to be placed. Really, words, utterances by themselves, carry meaning. Language is just a structure for _us_, so to speak, that we agree on for ease of communication. I think this is why probabilistic models do so well: the ideas we all have are mostly similar, it really is about just mapping from one kind of word to another, or kind of phrase to another. Feel free to respond, I’m most certainly out of my depth here.
  • acid__ 4y ago
    Hrm, true, no one’s ever gotten fluent in a language just by reading a textbook.
• nl 4y ago
  > Computational Linguistics I kept running into the unspoken question I've done a lot of work in NLP and the times when computational linguistics has been useful is very rare. The only time I shipped something to production that used it was a classifier for documents that needed to evaluate them on a sentence by sentence basis for possible compliance issues. Computational linguistics was useful then because I could rewrite mulit-clause sentences into simpler single clause sentences which the classifier could get better accuracy on. > And here was Google Translate being "good enough" for 80% of all use cases using a "dumb" statistic model that didn't even have a coherent concept of what a language is. I assume you are aware if Frederick Jelinek quote "Every time I fire a linguist, the performance of the speech recognizer goes up"?[1] That was in 1998. It's been pretty clear for a long time that computational linguistics can provide some tools to help us understand language but it is insufficiently reliable to use for unconstrained tasks. [1] https://en.wikipedia.org/wiki/Frederick_Jelinek https://en.wikipedia.org/wiki/Frederick_Jelinek
• tootie 4y ago
  I know only the bare basics of NLP and AI but isn't Google basically just a specialized case of LLM? Translate and Search work on the same principle that all you need to do is regression analysis on a huge pile of human behavior. Early search engines worked so hard to try to understand content and user intent and got thumped by a comparatively naive heuristic and a giant pile of data.
• satvikpendem 4y ago
  > But it will always be a 80% solution because it's an evolutionary dead end: it can't know things, we have only figured out how to trick it into pretending that it does. At the margin, these are equivalent (Chinese room, and all). I wonder if humans also learn similarly then retroactively tell themselves they actually do know things instead of just containing experiences encoded in their neurons (and whether that is any different than a neural network encoding trained "knowledge" in its neurons, too). This is the semantics of epistemology, at the end of the day.
• WA 4y ago
  To take it further: if 80% is good enough and you gotta do some work anyways on the output of LLMs, maybe all the extrapolations like "just wait 10 years and most jobs are doomed" are exaggerated. It’s not unlikely that LLMs hit a wall, because they inherently lack any sort of logic and reasoning. Which of course is a good thing to make sure many people get to keep their jobs.