3 ms·

How can you be confident that by scaling up the models their reasoning skills will not surpass humans

K0balt 3y ago
Because, ultimately, the models are modelling human reasoning. Not only that we do not have any evidence to support the idea that reasoning has a higher quality than we can apply to it using logical processes. It can be done faster, it can be done many times concurrently but ultimately 1 || 1 !& 1 is going to be have the same answer. The whole idea of superintelligence except as a measure of speed or quantity seems flawed on its face. And if we are to call speed or concurrency superintelligence, then we have already hit that mark some time ago. AI wins at games because of time compression, not because of an inherently superior logic. They just have time to consider more potential outcomes.
- seanhunter 3y ago
  AI research went off the idea of modelling human reasoning a long time ago. To the extent that current models reproduce reasoning in any sense comparable to a human it is purely for the purpose of user interface. For example if you ask a transformer model like chatgpt to solve some problem and explain its reasoning step by step it will give you a reasonable facsimile of a human thought process, but if you’ve read the transformers paper, you know that its actual process is tokenize -> do a bunch of matrix math -> decode the result into words (entirely different to how a human reasoning process works). Although things like neural nets were clearly inspired by biology they work completely differently to a biological brain. It would be closer to truth to say that the models use linear algebra and optimization to improve performance at specific tasks. For that reason, the whole debate about whether or not superintelligence is possible/has been achieved etc boils down to an argument about the definition of intelligence (as Turing predicted so long ago).
  - K0balt 3y ago
    I don’t think it’s a foregone conclusion that human reasoning doesn’t operate on the basis of statistical prediction of the next most probable “token” at its most granular level. Humans are certainly capable of hallucination in the LLM sense, and without training we often struggle to produce (or even outright fabricate) the rationale behind our “conclusions”. We stopped intentionally modelling human reasoning because we have no clue how it works. We roughly copied the physical devices then figured out how to slap them together with an algorithm that makes them behave similarly to human thought, when trained with a massive quantity of cultural-linguistic-memetic data. It should not be surprising that if you take a bucketful of engine parts and keep messing with them until they kinda work as an engine that you will probably wind up with something along similar lines as the original intention of the parts. Frankly, it would be quite surprising if you didn’t. Biological neural networks are fundamentally linear algebra processing device that integrate data into functions through training, so the fact that we understand the process as linear algebra is actually an argument that the process is similar. As to whether “superintelligence” can exist, I think we are in exactly the same page there. It is a matter of definition. Personally I doubt the existence of a fundamentally superior system of logic type of superintelligence, but certainly a device can made to iterate faster and flawless memory and instant, accurate calculation is going to beat the hell out of my notebook and HP28s lol. I suspect that superintelligence can be (has been?) achieved in some respects, but in the same sense that a room full of ten educated people with regular computing tools is “superintelligence”.
    - seanhunter 3y ago
      > Biological neural networks are fundamentally linear algebra processing device that integrate data into functions through training, so the fact that we understand the process as linear algebra is actually an argument that the process is similar. Do you have a reference for this? My understanding of biological neural networks is very different from this. Specifically I find it really hard to believe that a biological brain is a linear algebra processing device.
      - K0balt 3y ago
        Matrix multiplication with a different function and weights for each input/output. That’s the primary operation. There are many other operational modes overlaid that we typically do not model with mainstream models, but they primarily act as broad modifiers (suppression, gain, etc) we understand very well how it works, for example, when we get a few brain cells together to play pong or doom. We can transcribe the functionality of these simple biological networks, once trained, onto models and they function as expected. The simplified computational model we use for neural networks closely models the primary mechanism, but lacks self reinforcement systems, biochemical moderators, self directed connectivity modifications, etc. Basically we do the training with a separate process for most of the systems in common use, in contrast with biological systems which self organise and train in-situ.

Because, ultimately, the models are modelling human reasoning. Not only that we do not have any evidence to support the idea that reasoning has a higher quality than we can apply to it using logical processes. It can be done faster, it can be done many times concurrently but ultimately 1 || 1 !& 1 is going to be have the same answer. The whole idea of superintelligence except as a measure of speed or quantity seems flawed on its face. And if we are to call speed or concurrency superintelligence, then we have already hit that mark some time ago. AI wins at games because of time compression, not because of an inherently superior logic. They just have time to consider more potential outcomes.

seanhunter 3y ago
AI research went off the idea of modelling human reasoning a long time ago. To the extent that current models reproduce reasoning in any sense comparable to a human it is purely for the purpose of user interface. For example if you ask a transformer model like chatgpt to solve some problem and explain its reasoning step by step it will give you a reasonable facsimile of a human thought process, but if you’ve read the transformers paper, you know that its actual process is tokenize -> do a bunch of matrix math -> decode the result into words (entirely different to how a human reasoning process works). Although things like neural nets were clearly inspired by biology they work completely differently to a biological brain. It would be closer to truth to say that the models use linear algebra and optimization to improve performance at specific tasks. For that reason, the whole debate about whether or not superintelligence is possible/has been achieved etc boils down to an argument about the definition of intelligence (as Turing predicted so long ago).
- K0balt 3y ago
  I don’t think it’s a foregone conclusion that human reasoning doesn’t operate on the basis of statistical prediction of the next most probable “token” at its most granular level. Humans are certainly capable of hallucination in the LLM sense, and without training we often struggle to produce (or even outright fabricate) the rationale behind our “conclusions”. We stopped intentionally modelling human reasoning because we have no clue how it works. We roughly copied the physical devices then figured out how to slap them together with an algorithm that makes them behave similarly to human thought, when trained with a massive quantity of cultural-linguistic-memetic data. It should not be surprising that if you take a bucketful of engine parts and keep messing with them until they kinda work as an engine that you will probably wind up with something along similar lines as the original intention of the parts. Frankly, it would be quite surprising if you didn’t. Biological neural networks are fundamentally linear algebra processing device that integrate data into functions through training, so the fact that we understand the process as linear algebra is actually an argument that the process is similar. As to whether “superintelligence” can exist, I think we are in exactly the same page there. It is a matter of definition. Personally I doubt the existence of a fundamentally superior system of logic type of superintelligence, but certainly a device can made to iterate faster and flawless memory and instant, accurate calculation is going to beat the hell out of my notebook and HP28s lol. I suspect that superintelligence can be (has been?) achieved in some respects, but in the same sense that a room full of ten educated people with regular computing tools is “superintelligence”.
  - seanhunter 3y ago
    > Biological neural networks are fundamentally linear algebra processing device that integrate data into functions through training, so the fact that we understand the process as linear algebra is actually an argument that the process is similar. Do you have a reference for this? My understanding of biological neural networks is very different from this. Specifically I find it really hard to believe that a biological brain is a linear algebra processing device.
    - K0balt 3y ago
      Matrix multiplication with a different function and weights for each input/output. That’s the primary operation. There are many other operational modes overlaid that we typically do not model with mainstream models, but they primarily act as broad modifiers (suppression, gain, etc) we understand very well how it works, for example, when we get a few brain cells together to play pong or doom. We can transcribe the functionality of these simple biological networks, once trained, onto models and they function as expected. The simplified computational model we use for neural networks closely models the primary mechanism, but lacks self reinforcement systems, biochemical moderators, self directed connectivity modifications, etc. Basically we do the training with a separate process for most of the systems in common use, in contrast with biological systems which self organise and train in-situ.