4 ms·

Software can't execute itself. It must be executed by an external agent and given some kind of input. That will be true regardless of whether you consider the s

by shawnz 5y ago

Software can't execute itself. It must be executed by an external agent and given some kind of input. That will be true regardless of whether you consider the software to be a true AGI or a mere language model. So what does it mean exactly to "start a conversation"? Are you saying that it should be able to give a useful output with a blank input? I don't see why a language model can't do that.

simonh 5y ago
In principle it might be possible for a language model to develop true general intelligence. However there's no mechanism in GTP and similar systems that makes such a development necessary. In fact it seems likely that the development of internal structures that might develop into the faculties needed by an AGI would, in their intermediate state, make it worse as a language model. Evolution sometimes faces such development gaps, where to develop a capability that would ultimately grant improvements, it would have to go through intermediate phases that would make it worse. Language models are optimised in a very specific way and trained to solve a very narrow problem, compared to the problems faced by physical beings. So while I cant honestly say no, a language model can never do that, equally there's no actual reason to believe one ever would.
- shawnz 5y ago
  Was there a mechanism in the evolutionary system of biological life on Earth that makes the development of higher intelligence necessary? How did we develop higher intelligence if it's not possible to get there though a series of progressive improvements? Furthermore I would argue that intelligence is a necessary development in order to create the most sophisticated possible language model, for the reasons described above. Anything less will not be able to perfectly emulate the conversations of other intelligent beings.
  - simonh 5y ago
    I agree the best possible language model would possess general AI, but you can't go from no intelligence to general AI in one step. It seems likely to me that the intermediate levels of intelligence such a system might develop early on might actually make it a worse language model. The evolutionary path from dumb language model to language model with human level AI might well lie through an unhappy valley in which the fledgling AI makes more mistakes and is easier to catch out than a pure language model. In which case, a system would never take that evolutionary path. This is a problem with systems optimising towards a single specific function. It constrains their optimisation paths.

by shawnz 5y ago

simonh 5y ago

In principle it might be possible for a language model to develop true general intelligence. However there's no mechanism in GTP and similar systems that makes such a development necessary. In fact it seems likely that the development of internal structures that might develop into the faculties needed by an AGI would, in their intermediate state, make it worse as a language model. Evolution sometimes faces such development gaps, where to develop a capability that would ultimately grant improvements, it would have to go through intermediate phases that would make it worse. Language models are optimised in a very specific way and trained to solve a very narrow problem, compared to the problems faced by physical beings. So while I cant honestly say no, a language model can never do that, equally there's no actual reason to believe one ever would.

shawnz 5y ago
Was there a mechanism in the evolutionary system of biological life on Earth that makes the development of higher intelligence necessary? How did we develop higher intelligence if it's not possible to get there though a series of progressive improvements? Furthermore I would argue that intelligence is a necessary development in order to create the most sophisticated possible language model, for the reasons described above. Anything less will not be able to perfectly emulate the conversations of other intelligent beings.
- simonh 5y ago
  I agree the best possible language model would possess general AI, but you can't go from no intelligence to general AI in one step. It seems likely to me that the intermediate levels of intelligence such a system might develop early on might actually make it a worse language model. The evolutionary path from dumb language model to language model with human level AI might well lie through an unhappy valley in which the fledgling AI makes more mistakes and is easier to catch out than a pure language model. In which case, a system would never take that evolutionary path. This is a problem with systems optimising towards a single specific function. It constrains their optimisation paths.