5 ms·

The problem is, human intelligence is likely also based on a similar advanced chat bot setup. While GPT-4 only performs as good as top-10th percentile of human

The problem is, human intelligence is likely also based on a similar advanced chat bot setup. While GPT-4 only performs as good as top-10th percentile of human students taking an exam (a professional in the field can do much more than this), it is notable that as a generalist GPT-4 would outperform such professionals. And GPT-4 is much faster than a human. And we have not yet evaluated GPT-4 working in its optimal setting (access to optimal external tools). And we have not yet seen GPT-5 or 6 or 8. So, get ready for an interesting ride.

krainboltgreene 4y ago
> The problem is, human intelligence is likely also based on a similar advanced chat bot setup. This is so wildly wrong and yet confidently said in every techbro post about LLMs. I beg of you to talk to an expert.
- startupsfail 4y ago
  Like what expert? And who are you exactly to state that this is wrong, that boldly? Are you an expert? How many neuroscience and psychology papers have you read? Do you have any children? Have you trained any LLMs? Have you worked with reinforcement learning? Or how many computer science papers have you read during last two decades?
  - krainboltgreene 4y ago
    Lets assume for a moment that I haven't read any papers in those fields, that I don't have any childrens, that I haven't trained any LLMs or worked in "reinforcement learning", or even read any computer science papers in the last 20 years (the answer to 90% of that is yes): I don't have to be an expert in physics to know that pastors can't levitate, regardless of what they claim. You're mad that I'm calling you out, I get it, but you gotta understand after the 200th time of seeing this unfounded sentiment bandied about I'm not phased.
    - startupsfail 4y ago
      This statement is a theory and it is not a widely accepted or a proven one. Yet, I do see it in my lab research notebooks on generative AI (that date at ~2017). I think it is a good theory. Haven’t seen anything that contradicts it badly so far… If you haven’t done the above, I’d suggest doing it. It’s fun and gives a good perspective :)
    - theaiquestion 4y ago
      >Lets assume for a moment that I haven't read any papers in those fields, that I don't have any childrens, that I haven't trained any LLMs or worked in "reinforcement learning", or even read any computer science papers in the last 20 years (the answer to 90% of that is yes): I don't have to be an expert in physics to know that pastors can't levitate, regardless of what they claim. ....what? You're saying to assume you know nothing about a field but to assume your claim is correct? You said to "talk to an expert" - who should I talk to? What should I read about here? If there's something I'm missing I want to correct it, I simply can't say "well this random guy commented and said I'm wrong, better change my understanding of this topic." > you gotta understand after the 200th time of seeing this unfounded sentiment bandied about I'm not phased. All you've said is "I've disagreed with everyone on this topic, and while I have no information to offer other than it's 'common sense'". That does nothing to either improve our understand, or further a conversation, it's literally just saying "you're wrong and I'm right" with no elaboration.
- HyperSane 4y ago
  The fundamental truth is we really have no idea how the human brain works.
jliptzin 4y ago
But where is the imminent danger? It is still limited in many ways. For example, it can be turned off or unplugged. Is it because CAPTCHAs won’t work anymore? That sounds like a problem for sites like Twitter that have bot problems. Is it because it may replace people’s jobs? That comes with every technological step forward and there’s always alarmist ludditism to accompany it. Is it because bad people will use it to do bad things? Again, that comes with every new technology and that’s a law enforcement problem. I don’t really see what the imminent danger is, just sounds like the first few big players trying to create a regulatory moat and lock out potential new upstarts. Or they’re just distracting regulators from something else, like maybe antitrust enforcement.
- pixl97 4y ago
  You didn't understand who the actual luddiets were, but don't worry, I have a feeling we'll get our chance.
- josephg 4y ago
  There are two big concerns: 1. GPT-8 or something is able to do 70% of people’s jobs. It can write software, drive cars, design industrial processes, build robots and manufacture anything we can imagine. This is a great thing in the long term, but in the short term society is designed where you need to work in order to have food to eat. I expect a period of rioting, poverty, and general instability. All we need for this to be the case is a human level AI. 2. But we won’t stop improving AIs when they operate at human level. An ASI (artificial superintelligence) would be deeply unpredictable to us. Trying to figure out what an ASI will do is like a dog trying to understand a human. If we make an ASI that’s not properly aligned with human interests, there’s a good chance it will kill everyone. And unfortunately, we might only get one chance to properly align it before it escapes the lab and starts modifying its own code. Smart people disagree on how likely these scenarios are. I think (1) is likely within my lifetime. And I think it’s very unlikely we stop improving AIs when they’re at human levels of intelligence. (GPT4 already exceeds human minds in the breadth of its long term memory and its speed.) That’s why people are worried, and making nuclear weapon analogies in this thread.
  - 4y ago
inciampati 4y ago
Alas, if it could only remember and precisely relate more than 4k or 8k or 32k or 64k words... And if only scaling that context length weren't quadratic... Indeed, we would really expect an AI to be able to achieve AGI. And it might decide to do all kinds of alien things. The sky would not be the limit! We have more than 100 trillion synapses in our brains. That's not our "parameter" count. It's the size of the thing that's getting squared at every "step". LLMs are amazing, but the next valley of disillusionment is going to begin when that quadratic scaling cost begins to rear its head and we are left in breathless anticipation of something better. I am not as worried, I guess, as your average AI ethicist. I can hope for the best (I welcome the singularity as much as the next nerd), but quadratic isn't going to get easier without some very new kinds of computers. For those to scale to AGI on this planet it's questionable if they'll have the same architecture we're working with now. Otherwise, I'd expect a being whose brain is a rock with lightning in it to have take over the world long, long ago. Earth has plenty of both for something smart and energy efficient to have evolved in all these billions of years. But it didn't and maybe that's a lesson. That all said, these LLMs are really amazing at language. Just don't ask them to link a narrative arc into some subtle detail that appeared twice in the last three hundred pages of text. For a human it ain't a problem. But these systems need to grow a ton of new helper functionality and subsystems to hope to achieve that kind of performance. And, I'll venture that kind of thing is a lower bound on the abilitites of any being who would be able to savage the world with it's intellect. It will have to be able to link up so, so many disparate threads to do it. It boggles our minds, which are only squaring a measly 100T dimension every tick. Ahem.
- startupsfail 4y ago
  You can only hold around 7 to 10 numbers in your mind well, in your working memory. Let me give you a few: 6398 5385 3854 8577 You have 1 second, close your eyes and add them together. Write down the result. I’m pretty sure that GPT-4 at its 4k setting would outperform you. [The point being, we have not seen what even GPT-4 can do in its optimal environment. Humans use paper, computers, google, etc. to organize their thoughts and work efficiently. They don’t just sit in empty space and then put everything into the working memory and magically produce the results. So imagine now that you do have a similar level of tooling and sophistication around GPT-4, like there is present around humans. I’m considering that and it is difficult to extrapolate what even GPT-4 can do, in its optimal environment.]

krainboltgreene 4y ago

> The problem is, human intelligence is likely also based on a similar advanced chat bot setup. This is so wildly wrong and yet confidently said in every techbro post about LLMs. I beg of you to talk to an expert.

startupsfail 4y ago
Like what expert? And who are you exactly to state that this is wrong, that boldly? Are you an expert? How many neuroscience and psychology papers have you read? Do you have any children? Have you trained any LLMs? Have you worked with reinforcement learning? Or how many computer science papers have you read during last two decades?
- krainboltgreene 4y ago
  Lets assume for a moment that I haven't read any papers in those fields, that I don't have any childrens, that I haven't trained any LLMs or worked in "reinforcement learning", or even read any computer science papers in the last 20 years (the answer to 90% of that is yes): I don't have to be an expert in physics to know that pastors can't levitate, regardless of what they claim. You're mad that I'm calling you out, I get it, but you gotta understand after the 200th time of seeing this unfounded sentiment bandied about I'm not phased.
  - startupsfail 4y ago
    This statement is a theory and it is not a widely accepted or a proven one. Yet, I do see it in my lab research notebooks on generative AI (that date at ~2017). I think it is a good theory. Haven’t seen anything that contradicts it badly so far… If you haven’t done the above, I’d suggest doing it. It’s fun and gives a good perspective :)
  - theaiquestion 4y ago
    >Lets assume for a moment that I haven't read any papers in those fields, that I don't have any childrens, that I haven't trained any LLMs or worked in "reinforcement learning", or even read any computer science papers in the last 20 years (the answer to 90% of that is yes): I don't have to be an expert in physics to know that pastors can't levitate, regardless of what they claim. ....what? You're saying to assume you know nothing about a field but to assume your claim is correct? You said to "talk to an expert" - who should I talk to? What should I read about here? If there's something I'm missing I want to correct it, I simply can't say "well this random guy commented and said I'm wrong, better change my understanding of this topic." > you gotta understand after the 200th time of seeing this unfounded sentiment bandied about I'm not phased. All you've said is "I've disagreed with everyone on this topic, and while I have no information to offer other than it's 'common sense'". That does nothing to either improve our understand, or further a conversation, it's literally just saying "you're wrong and I'm right" with no elaboration.
HyperSane 4y ago
The fundamental truth is we really have no idea how the human brain works.

jliptzin 4y ago

But where is the imminent danger? It is still limited in many ways. For example, it can be turned off or unplugged. Is it because CAPTCHAs won’t work anymore? That sounds like a problem for sites like Twitter that have bot problems. Is it because it may replace people’s jobs? That comes with every technological step forward and there’s always alarmist ludditism to accompany it. Is it because bad people will use it to do bad things? Again, that comes with every new technology and that’s a law enforcement problem. I don’t really see what the imminent danger is, just sounds like the first few big players trying to create a regulatory moat and lock out potential new upstarts. Or they’re just distracting regulators from something else, like maybe antitrust enforcement.

pixl97 4y ago
You didn't understand who the actual luddiets were, but don't worry, I have a feeling we'll get our chance.
josephg 4y ago
There are two big concerns: 1. GPT-8 or something is able to do 70% of people’s jobs. It can write software, drive cars, design industrial processes, build robots and manufacture anything we can imagine. This is a great thing in the long term, but in the short term society is designed where you need to work in order to have food to eat. I expect a period of rioting, poverty, and general instability. All we need for this to be the case is a human level AI. 2. But we won’t stop improving AIs when they operate at human level. An ASI (artificial superintelligence) would be deeply unpredictable to us. Trying to figure out what an ASI will do is like a dog trying to understand a human. If we make an ASI that’s not properly aligned with human interests, there’s a good chance it will kill everyone. And unfortunately, we might only get one chance to properly align it before it escapes the lab and starts modifying its own code. Smart people disagree on how likely these scenarios are. I think (1) is likely within my lifetime. And I think it’s very unlikely we stop improving AIs when they’re at human levels of intelligence. (GPT4 already exceeds human minds in the breadth of its long term memory and its speed.) That’s why people are worried, and making nuclear weapon analogies in this thread.
- 4y ago

inciampati 4y ago

Alas, if it could only remember and precisely relate more than 4k or 8k or 32k or 64k words... And if only scaling that context length weren't quadratic... Indeed, we would really expect an AI to be able to achieve AGI. And it might decide to do all kinds of alien things. The sky would not be the limit! We have more than 100 trillion synapses in our brains. That's not our "parameter" count. It's the size of the thing that's getting squared at every "step". LLMs are amazing, but the next valley of disillusionment is going to begin when that quadratic scaling cost begins to rear its head and we are left in breathless anticipation of something better. I am not as worried, I guess, as your average AI ethicist. I can hope for the best (I welcome the singularity as much as the next nerd), but quadratic isn't going to get easier without some very new kinds of computers. For those to scale to AGI on this planet it's questionable if they'll have the same architecture we're working with now. Otherwise, I'd expect a being whose brain is a rock with lightning in it to have take over the world long, long ago. Earth has plenty of both for something smart and energy efficient to have evolved in all these billions of years. But it didn't and maybe that's a lesson. That all said, these LLMs are really amazing at language. Just don't ask them to link a narrative arc into some subtle detail that appeared twice in the last three hundred pages of text. For a human it ain't a problem. But these systems need to grow a ton of new helper functionality and subsystems to hope to achieve that kind of performance. And, I'll venture that kind of thing is a lower bound on the abilitites of any being who would be able to savage the world with it's intellect. It will have to be able to link up so, so many disparate threads to do it. It boggles our minds, which are only squaring a measly 100T dimension every tick. Ahem.

startupsfail 4y ago
You can only hold around 7 to 10 numbers in your mind well, in your working memory. Let me give you a few: 6398 5385 3854 8577 You have 1 second, close your eyes and add them together. Write down the result. I’m pretty sure that GPT-4 at its 4k setting would outperform you. [The point being, we have not seen what even GPT-4 can do in its optimal environment. Humans use paper, computers, google, etc. to organize their thoughts and work efficiently. They don’t just sit in empty space and then put everything into the working memory and magically produce the results. So imagine now that you do have a similar level of tooling and sophistication around GPT-4, like there is present around humans. I’m considering that and it is difficult to extrapolate what even GPT-4 can do, in its optimal environment.]