5 ms·

Geez people. Please learn to identify hypes, but most importantly the causes of hypes. LLM is fundamentally information processing technology. Not the path to

Geez people. Please learn to identify hypes, but most importantly the causes of hypes. LLM is fundamentally information processing technology. Not the path to AGI, not an emerging sentience. The reason "this time" feels so amazing is that the unwashed masses suddenly got access to new information processing technology in a context where their tools been stagnant for decades. Not because it was not possible, but because there was no money in it. To understand my argument and its implications, humor me for a moment and imagine a universe in which everyone was already using linux computers and for a decade now published papers of ML/DL were available for people to use. So there were various crowdsourced indices and models of all sorts, which people incrementally embedded in their information processing workflows. In that universe there would be no room for the delirious reaction we have here. It would be incremental evolution of search, knowledge bases, algorithmic content generation etc. What we have experienced instead in these past decades is information tool starvation. These incrementally improving tools, while nothing but known and ultimately mundane algorithms were not available except within a tiny elite. In fact, people's information processing capability arguably declined as the desktop platforms got downgraded, adtech toxic waste covered the information landscape etc. What is happening now is that a socioeconomic and artificially induced scarcity is now being broken (for reasons that require serious piecing together of events). So while on the surface this hype is as distastful as any illustration of human lemming behaviour, there is enormous silver lining if we succeed to read its causes. These tools are here, have been here for a while and they can be inserted into our ever growing information processing toolkit. The risks are there to match the opportunities. The biggest risk of them all is precisely what has led to the current situation. Technology not diffusing normally, but controlled by gatekeepers.

Swizec 3y ago
Your post boils down to “This is the next iPhone moment”. The tech existed but was clunky and unavailable. Now it’s good and accessible enough to be used by most people. This is how technological revolutions happen! Edison didn’t invent the lightbulb. But he made it useful and invented power distribution. That’s the important part. Consumers don’t care about technology that exists in a lab that they cannot even begin to fathom how to use. Mind you last year even experts didn’t think LLMs would be able to do what they’re doing right now for at least another 5 to 6 years. This also is in part a side-effect of the hype and extra attention. All this attention causes an acceleration.
bmacho 3y ago
Why not? I believe we are this close: >< to an AGI, and chatGPT will turn into an AGI soon. It probably should rely on its neurons less, and use symbolic reasoning more. That, and learn and grow, either maintain a fact database, or retrain its neurons or add more neurons for new facts.
deleted 3y ago
[deleted]
idopmstuff 3y ago
> LLM is fundamentally information processing technology. Not the path to AGI, not an emerging sentience. This logic doesn't follow - AGI will be, fundamentally, information processing technology, just as the human brain is, fundamentally, information processing technology.
- sledgehammers 3y ago
  Well said. > "Not the path to AGI, not an emerging sentience". These are HUGE claims. Define AGI. Define sentience. What is the path and why isn't LLM on it? Right now those words are only dangerous misinformation - originating from a brain trying to protect its shaking world view. A reminder: you can talk with computers now. Think about that for a moment.
  - oxfordmale 3y ago
    LLM are mathematical models that, given your question, return a sequence of words based on a very complex probability model. The many examples on the internet show that ChatGPT doesn't understand most concepts. For instance, it doesn't know what a book is; it just knows what humans say about books. Finally, OpenAI has already stated there is a limit to how far the current AI models can progress. ChatGPT 5 will eventually be released but will be hitting the limits. A paradigm shift is needed to move to the next phase of AI.
    - divan 3y ago
      > LLM are mathematical models that, given your question, return a sequence of words based on a very complex probability model. How is this different from how humans respond? Our model could be just some orders of magnitude more complex. Or do you think there some _fundamentally_ different things are going on in the human brain?
      - oxfordmale 3y ago
        You know what a book is and can reason about it. As a human you can provide answers that go beyond the knowledge you have ingested. For example, if you ask ChatGPT if books can be fluorescent it says no. However, as an adult you know someone somewhere has made a book with fluorescent images, as it is a cool thing. You are combining knowledge from two different fields (books + fluorescence) and establishing the likelihood of someone being able to combine them.
fullstackchris 3y ago
Finally someone on HN who understands the technology (surprisingly few) and the socioeconomic undercurrent and risks. Absolutely no risk of terminator here, just ignorance masquerading as such.
- stevenhuang 3y ago
  Pride comes before the fall, and the AI comes before humility - https://news.ycombinator.com/item?id=35879657 https://news.ycombinator.com/item?id=35879657
retrac 3y ago
We have spent the last seventy or more years working on machine translation. All the way back in the 1950s, big money went into it. Decades of hard work, mathematical models of human language, all manner of study, enormous bilingual corpuses of text with phonetic annotation, programmed in general-knowledge databases, fuzzy reasoning algorithms. The amount of work put into it is quite staggering, in hindsight. I remember the cutting edge in the 1990s - SYSTRAN for example, could with some significant human guidance and a limited context domain, translate technical material sometimes usefully. All of that work has been rendered moot by deep learning. All of it. A machine can, simply with the correct deep learning algorithm and mass exposure to language plus a few bilingual texts, learn an algorithm for translation. It does so automatically, no verb conjugation algorithms, no general knowledge databases, no expert systems with fuzzy reasoning, no parsers, not like a specifically-designed old-school translator had. And yet these deep-learning systems are vastly superior to the old school architectures, completely supplanting them a couple years after their development. It is the same story in many other areas. Chess, Go? They learn to play chess and go better than any AI designed specifically to do so. Image classification? Better than the previous 60 years of work on machine vision, and again, accidentally falls out of it. Speech recognition? An algorithm to write a bad poem? Well, we now have an algorithm to find an algorithm to write you that bad poem, if you want it. That's the thing. These are algorithms to solve very tricky problems, and we didn't have to discover, find, or otherwise create the algorithm. The machine did it for us. I am not sure I'm communicating it well, but to me that's probably the most significant advance since the computer. It was understood - theoretically - that this was possible for a long time; but personally at least, I assumed it would forever require more data and compute than could be realized.
- Izkata 3y ago
  > It is the same story in many other areas. Chess, Go? They learn to play chess and go better than any AI designed specifically to do so. Take GP's "stagnant for decades" with tech and turn it into "stagnant for centuries" with these. When it first started I remember professional Go players talking about just how big a shakeup it was.
  - deleted 3y ago
mmcgaha 3y ago
Even if the core difference is accessibility, that is huge. I see AI in 2023 as a pivotal moment similar to 1977 with personal computers, 1994 for the internet and 2007 for mobile computing. All of these technologies had a history before their pivotal moments but when the time was right things changed fast. Having easy access to the technology is starting to change the way I think about what is possible for computing. One of the skills that has helped me though my career is understanding the scope of what a computer can do or automate. Most of the time it isn't that the thing cannot be done but that the cost associated with it is too high. I recently started solving an issue that would have taken hundreds of hours of human intervention but now I can do it with ten bucks and two hundred lines of python.
- antifa 3y ago
  2023 is the year where I personally benefit from LLMs, and might spend money on them as casually as others spend on video streaming, but the pivotal moment could still be a year or few away.
Izkata 3y ago
> Not because it was not possible, but because there was no money in it. When one of the earlier GPT models was released, didn't they straight-up say "none of us expected it to work this well"? I think it really was closer to "not possible" than "no money" - but more like "no one thought it was possible".
zamnos 3y ago
The problem with calling it hype is the broad stroke that's painted about all discussions of its abilities, simply by using that word when it's not all hype. A key part of the definition of hype is the exaggeration of importance and benefits. There are absolutely some out there out there that are exaggerating ChatGPT's abilities, but what are those exaggerations and what are actual abilities that anybody with an OpenAI account can go a verify (true or not) for themselves? It's an exaggeration to say that GPT-4 can solve all programming problems, but it's easily verifiable that, for example, it's able to solve easy level leetcode problems, and what's impressive on top of that it's able to give an answer to those problems practically before I've finished even reading the problem. That's not hype or a claim by me that you have to take on faith, anybody can copy and paste a problem description and get an answer. It's fair to point out that it's so-so at medium problems, and downright unable to do hard problems, but, well, most humans can't solve hard level problems. So what's hype, and what's not? It seems to think/reason/understand better than a human child. It's better than some humans with its command of the English language. It has broad knowledge of many subjects beyond what most individual humans know about. It sometimes has poor recall of that knowledge, and some of it is wrong, but it accepts corrections (even if it's unable to commit those corrections back to long-term memory) with far more grace than most humans.
jasmer 3y ago
"LLM is fundamentally information processing technology. Not the path to AGI, not an emerging sentience." I don't agree with this premise. 'This Time Is Different' because the LLMs have enabled AI to cross a material threshold. The AGI hype is overdone, but AI promise, unlike just 3 years ago, will be fulfilled across a number of industries. A lot of things are going to change. Notably, I wonder if 'AI companies' will doing much of it, rather 'AI will be utilized'. One thing is that 'help and instructions' are finally going to work for a broad array of applications.

Swizec 3y ago

Your post boils down to “This is the next iPhone moment”. The tech existed but was clunky and unavailable. Now it’s good and accessible enough to be used by most people. This is how technological revolutions happen! Edison didn’t invent the lightbulb. But he made it useful and invented power distribution. That’s the important part. Consumers don’t care about technology that exists in a lab that they cannot even begin to fathom how to use. Mind you last year even experts didn’t think LLMs would be able to do what they’re doing right now for at least another 5 to 6 years. This also is in part a side-effect of the hype and extra attention. All this attention causes an acceleration.

bmacho 3y ago

Why not? I believe we are this close: >< to an AGI, and chatGPT will turn into an AGI soon. It probably should rely on its neurons less, and use symbolic reasoning more. That, and learn and grow, either maintain a fact database, or retrain its neurons or add more neurons for new facts.

deleted 3y ago

[deleted]

idopmstuff 3y ago

> LLM is fundamentally information processing technology. Not the path to AGI, not an emerging sentience. This logic doesn't follow - AGI will be, fundamentally, information processing technology, just as the human brain is, fundamentally, information processing technology.

sledgehammers 3y ago
Well said. > "Not the path to AGI, not an emerging sentience". These are HUGE claims. Define AGI. Define sentience. What is the path and why isn't LLM on it? Right now those words are only dangerous misinformation - originating from a brain trying to protect its shaking world view. A reminder: you can talk with computers now. Think about that for a moment.
- oxfordmale 3y ago
  LLM are mathematical models that, given your question, return a sequence of words based on a very complex probability model. The many examples on the internet show that ChatGPT doesn't understand most concepts. For instance, it doesn't know what a book is; it just knows what humans say about books. Finally, OpenAI has already stated there is a limit to how far the current AI models can progress. ChatGPT 5 will eventually be released but will be hitting the limits. A paradigm shift is needed to move to the next phase of AI.
  - divan 3y ago
    > LLM are mathematical models that, given your question, return a sequence of words based on a very complex probability model. How is this different from how humans respond? Our model could be just some orders of magnitude more complex. Or do you think there some _fundamentally_ different things are going on in the human brain?
    - oxfordmale 3y ago
      You know what a book is and can reason about it. As a human you can provide answers that go beyond the knowledge you have ingested. For example, if you ask ChatGPT if books can be fluorescent it says no. However, as an adult you know someone somewhere has made a book with fluorescent images, as it is a cool thing. You are combining knowledge from two different fields (books + fluorescence) and establishing the likelihood of someone being able to combine them.

fullstackchris 3y ago

Finally someone on HN who understands the technology (surprisingly few) and the socioeconomic undercurrent and risks. Absolutely no risk of terminator here, just ignorance masquerading as such.

stevenhuang 3y ago
Pride comes before the fall, and the AI comes before humility - https://news.ycombinator.com/item?id=35879657 https://news.ycombinator.com/item?id=35879657

retrac 3y ago

We have spent the last seventy or more years working on machine translation. All the way back in the 1950s, big money went into it. Decades of hard work, mathematical models of human language, all manner of study, enormous bilingual corpuses of text with phonetic annotation, programmed in general-knowledge databases, fuzzy reasoning algorithms. The amount of work put into it is quite staggering, in hindsight. I remember the cutting edge in the 1990s - SYSTRAN for example, could with some significant human guidance and a limited context domain, translate technical material sometimes usefully. All of that work has been rendered moot by deep learning. All of it. A machine can, simply with the correct deep learning algorithm and mass exposure to language plus a few bilingual texts, learn an algorithm for translation. It does so automatically, no verb conjugation algorithms, no general knowledge databases, no expert systems with fuzzy reasoning, no parsers, not like a specifically-designed old-school translator had. And yet these deep-learning systems are vastly superior to the old school architectures, completely supplanting them a couple years after their development. It is the same story in many other areas. Chess, Go? They learn to play chess and go better than any AI designed specifically to do so. Image classification? Better than the previous 60 years of work on machine vision, and again, accidentally falls out of it. Speech recognition? An algorithm to write a bad poem? Well, we now have an algorithm to find an algorithm to write you that bad poem, if you want it. That's the thing. These are algorithms to solve very tricky problems, and we didn't have to discover, find, or otherwise create the algorithm. The machine did it for us. I am not sure I'm communicating it well, but to me that's probably the most significant advance since the computer. It was understood - theoretically - that this was possible for a long time; but personally at least, I assumed it would forever require more data and compute than could be realized.

Izkata 3y ago
> It is the same story in many other areas. Chess, Go? They learn to play chess and go better than any AI designed specifically to do so. Take GP's "stagnant for decades" with tech and turn it into "stagnant for centuries" with these. When it first started I remember professional Go players talking about just how big a shakeup it was.
- deleted 3y ago

mmcgaha 3y ago

Even if the core difference is accessibility, that is huge. I see AI in 2023 as a pivotal moment similar to 1977 with personal computers, 1994 for the internet and 2007 for mobile computing. All of these technologies had a history before their pivotal moments but when the time was right things changed fast. Having easy access to the technology is starting to change the way I think about what is possible for computing. One of the skills that has helped me though my career is understanding the scope of what a computer can do or automate. Most of the time it isn't that the thing cannot be done but that the cost associated with it is too high. I recently started solving an issue that would have taken hundreds of hours of human intervention but now I can do it with ten bucks and two hundred lines of python.

antifa 3y ago
2023 is the year where I personally benefit from LLMs, and might spend money on them as casually as others spend on video streaming, but the pivotal moment could still be a year or few away.

Izkata 3y ago

> Not because it was not possible, but because there was no money in it. When one of the earlier GPT models was released, didn't they straight-up say "none of us expected it to work this well"? I think it really was closer to "not possible" than "no money" - but more like "no one thought it was possible".

zamnos 3y ago

The problem with calling it hype is the broad stroke that's painted about all discussions of its abilities, simply by using that word when it's not all hype. A key part of the definition of hype is the exaggeration of importance and benefits. There are absolutely some out there out there that are exaggerating ChatGPT's abilities, but what are those exaggerations and what are actual abilities that anybody with an OpenAI account can go a verify (true or not) for themselves? It's an exaggeration to say that GPT-4 can solve all programming problems, but it's easily verifiable that, for example, it's able to solve easy level leetcode problems, and what's impressive on top of that it's able to give an answer to those problems practically before I've finished even reading the problem. That's not hype or a claim by me that you have to take on faith, anybody can copy and paste a problem description and get an answer. It's fair to point out that it's so-so at medium problems, and downright unable to do hard problems, but, well, most humans can't solve hard level problems. So what's hype, and what's not? It seems to think/reason/understand better than a human child. It's better than some humans with its command of the English language. It has broad knowledge of many subjects beyond what most individual humans know about. It sometimes has poor recall of that knowledge, and some of it is wrong, but it accepts corrections (even if it's unable to commit those corrections back to long-term memory) with far more grace than most humans.

jasmer 3y ago

"LLM is fundamentally information processing technology. Not the path to AGI, not an emerging sentience." I don't agree with this premise. 'This Time Is Different' because the LLMs have enabled AI to cross a material threshold. The AGI hype is overdone, but AI promise, unlike just 3 years ago, will be fulfilled across a number of industries. A lot of things are going to change. Notably, I wonder if 'AI companies' will doing much of it, rather 'AI will be utilized'. One thing is that 'help and instructions' are finally going to work for a broad array of applications.