4 ms·

I learnt some very basics of computational linguistics since it was related to a side project. I kept wondering why people were spending huge amounts of resour

• hnbad 4y ago
  I think a huge part is that computational linguistics still chases the idea of a universal language model, which may simply not be possible. I haven't followed the science in general linguistics but something feels off when most of the information ends up being tagged onto nil particles (i.e. parts of speech present neither in utterances nor written language and not affecting intonation or otherwise being detectable except by contrasting the structure with related languages).
  • hnfong 4y ago
    In a sense the model is universal. It's just a 100GB (give or take) neural network. And apparently (or so I heard, I think) feeding transformer models training data of Language A could improve its ability to understand Language B. So maybe there's something truly universal in some sense.
• vkazanov 4y ago
  The secret is that there are no grammars in our brains. Rules are statistical, not precise. Rules, idioms are fluid and... statistical. We're a bit more specialised than these new models. But that's it, really.
  • xp84 4y ago
    ^ This. I think the more we internalize the fact that we're also basically LLMs, the more we'll realize that there likely isn't some hard barrier beyond which no AI can climb. If you watch the things kids who are learning language say, you'll see the same kinds of slip-ups that belie the fact that they don't yet understand all the words themselves, but nobody thinks that 2-year-olds aren't people or thinks they will never learn to understand these concepts.
    • oska 4y ago
      Internalising the idea (not 'fact') that 'we're basically LLMs' will only take you to a place of deep sterility, delusion and nihilism.
      • vkazanov 4y ago
        Why? Do you prefer magical "handwavy" thinking? A lot of things just fall in place if you accept an obvious fact that the brain is just a powerful enough function with certain inputs and outputs. Neurons are not that complicated. There are interesting consequences here though: no stable grammars, no truly uniform thinking, no singular way to understand people.
        • oska 4y ago
          The 'handwavey' thinking here is from those who state as a 'fact' that the human brain is an LLM
          • vkazanov 4y ago
            Well, I didn't say a brain is just an LLM. It's more complicated than just this. But what LLM showed is that it works using the same simple building blocks + specialised parts of the brain: memory, constant relearning and world input, etc. So in this particular case of LLMs we've managed to optimise our way around having specialised brain structures using a powerful enough math function. Next steps require improvements in how the model is trained, how we can reduce the amount of training data, what additional machinery might be necessary, etc... But, damn it, this very thing that makes humanity possible - our language - is solved now. Natural language is a solved problem now. That very thing that makes complex societies possible - it's done. This is the fact. And it is crazy. EDIT: typos
            • oska 4y ago
              I don't share any of the excitement that ppl like you appear to be feeling, and I recoil from the grandiose claims being made by people who, in my opinion, are being fooled by the 21st century equivalent of a ventriloquist's doll. (I did feel excitement while following the development of AlphaZero and its played Go matches, but that was because it was revealing greater depths and beauty in the human created game of Go. And I maintain some interest in following the development of self-driving, particularly by Tesla.) With regard to LLMs I can see how they could be useful. I think more particularly useful when they work from a constrained corpus, so the user can know what they're drawing from (and thus the limitations of that knowledge base). The example site that been posted by its maker to HN [1] where you can ask questions against a particular book is a good one for showing the use of the tool I think. But it's just a tool and it's not in any way a breakthrough in our understanding of ourselves, of cognition or anything like that. I think the people who are making these claims can't distinguish science fiction from actual reality. They are fantasists and I think they are leading themselves and others into delusion. [1] https://portal.konjer.xyz/ https://portal.konjer.xyz/
• sp332 4y ago
  English Wikipedia is the largest. Wikipedia in other languages would be less useful.
• tkgally 4y ago
  I've just started testing GPT-4 on translation from Japanese to English, but it seems to blow Google Translate out of the water. It was particularly good with a novel excerpt. I encourage people to try it with a variety of languages and texts to see if those results hold up.
  • mtlmtlmtlmtl 4y ago
    It would be useful to know whether you know both languages or just English. Not trying to poke holes, just clarifying.
    • tkgally 4y ago
      No problem. My first language is English, but I worked as a Japanese to English translator for twenty years and have written books in Japanese. My personal website is linked from my user page.
      • mtlmtlmtlmtl 4y ago
        Thanks. Refreshing to see one of these claims made by someone actually qualified to verify them! I might have to play with some Norwegian <-> English translation at some point. Should be even more effective than for Japanese due to the common germanic origins.