4 ms·

I think you make a lot of assumptions that you should perhaps reexamine. > Human cognition was basically bruteforced by evolution-- why would it be impossible

I think you make a lot of assumptions that you should perhaps reexamine. > Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? Here are some of your assumptions: 1. Human intelligence is entirely explicable in evolutionary terms. (It is certainly not the case that it has been explained in this manner, even if it could be.) [0] 2. Human intelligence assumed as an entirely biological phenomenon is realizable in something that is not biological. And perhaps this one: 3. Silicon is somehow intrinsically bound up with computation. In the case of (2), you're taking a superficial black box view of intelligence and completely ignoring its causes and essential features. This prevents you from distinguishing between simulation of appearance and substantial reality. Now, that LLMs and so on can simulate syntactic operations or whatever is no surprise. Computers are abstract mathematical formal models that define computations exactly as syntactic operations. What computers lack are semantic content. A computer never contains the concept of the number 2 and the concept of the addition operation even though we can simulate the addition of 2 + 2. This intrinsic absence of a semantic dimension means that computers already lack the most essential feature of intelligence, which is intentionality. There is no alchemical magic that will turn syntax into semantics. In the case of (3), I emphasize that computation is not a physical phenomenon, but something described by a number of formally equivalent models (Turing machine, lambda calculus, and so on) that aim to formalize the notion of effective method. The use of silicon-based electronics is irrelevant to the model. We can physically simulate the model using all sorts of things, like wooden gears or jars of water or whatever. > I'm not buying the whole "AI has no agency" line either; this might be true for now, but this is already being circumvented with current LLMs (by giving them web access etc). [...] As soon as profit can be made by transfering decision power into an AIs hand, some form of agency for them is just a matter of time, and we might simply not be willing to pull the plug until it is much too late. How on earth did you conclude there is any agency here, or that it's just a "matter of time"? This is textbook magical thinking. You are projecting a good deal here that is completely unwarranted. Computation is not some kind of mystery, and we know at least enough about human intelligence to notes features that are not included in the concept of computation. [0] (Assumption (1), of course, has the problem that if intelligence is entirely explicable in terms of evolutionary processes, then we have no reason to believe that the intelligence produced aims at truth. Survival affordances don't imply fidelity to reality. This leads us to the classic retorsion arguments that threaten the very viability of the science you are trying to draw on.)

soiltype 1y ago
I understand all the words you've used but I truly do not understand how they're supposed to be an argument against the GP post. Before this unfolds into a much larger essay, should we not acknowledge one simple fact: that our best models of the universe indicate that our intelligence evolved in meat and that meat is just a type of matter. This is an assumption I'll stand on, and if you don't disagree, we need to back up. Far too often, online debates such as this take the position that the most likely answer to a question should be discarded because it isn't fully proven. This is backwards. The most likely answer should be assumed to be probably true, a la Occam. Acknowledging other options is also correct, but assuming the most likely answer is wrong, without evidence, is simply contrarian for its own sake, not wisdom or science.
- lo_zamoyski 1y ago
  I don't know what else I can write without repeating myself. I already wrote that even under the assumption that intelligence is a purely biological phenomenon, it does not follow that computation can produce intelligence. This isn't a matter of probabilities. We know what computation is, because we defined it as such and such. We know at least some essential features of intelligence (chiefly, intentionality). It is not rocket science to see that computation, thus defined, does not include the concepts of semantics and intentionality. By definition, it excludes them. Attempts to locate the latter in the former reminds me of Feynman's anecdote about the obtuse painter who claimed he could produce yellow from red and white paint alone (later adding a bit of yellow paint to "sharpen it up a bit").
  - ACCount37 1y ago
    What. Are you saying that "intentionality", whatever you mean by it, can't be implemented by a computational process? Never-ever? Never-ever-ever?
    - lo_zamoyski 1y ago
      If you don't know what intentionality is, a commonplace term in the philosophy of mind, then I suggest you take some time to better acquaint yourself with the subject matter. This is probably the major blow against any kind of AI fantasy, and you don't do yourself any favors by wading in the shallows and confidently holding to misinformed opinions. So, one last time. A computer program is a set of formal rules that takes one sequence of uninterpreted symbols and produces another sequence of uninterpreted symbols. Adding more rules and more uninterpreted symbols doesn't magically cause those symbols to be interpreted, and it cannot, by definition.

soiltype 1y ago

I understand all the words you've used but I truly do not understand how they're supposed to be an argument against the GP post. Before this unfolds into a much larger essay, should we not acknowledge one simple fact: that our best models of the universe indicate that our intelligence evolved in meat and that meat is just a type of matter. This is an assumption I'll stand on, and if you don't disagree, we need to back up. Far too often, online debates such as this take the position that the most likely answer to a question should be discarded because it isn't fully proven. This is backwards. The most likely answer should be assumed to be probably true, a la Occam. Acknowledging other options is also correct, but assuming the most likely answer is wrong, without evidence, is simply contrarian for its own sake, not wisdom or science.

lo_zamoyski 1y ago
I don't know what else I can write without repeating myself. I already wrote that even under the assumption that intelligence is a purely biological phenomenon, it does not follow that computation can produce intelligence. This isn't a matter of probabilities. We know what computation is, because we defined it as such and such. We know at least some essential features of intelligence (chiefly, intentionality). It is not rocket science to see that computation, thus defined, does not include the concepts of semantics and intentionality. By definition, it excludes them. Attempts to locate the latter in the former reminds me of Feynman's anecdote about the obtuse painter who claimed he could produce yellow from red and white paint alone (later adding a bit of yellow paint to "sharpen it up a bit").
- ACCount37 1y ago
  What. Are you saying that "intentionality", whatever you mean by it, can't be implemented by a computational process? Never-ever? Never-ever-ever?
  - lo_zamoyski 1y ago
    If you don't know what intentionality is, a commonplace term in the philosophy of mind, then I suggest you take some time to better acquaint yourself with the subject matter. This is probably the major blow against any kind of AI fantasy, and you don't do yourself any favors by wading in the shallows and confidently holding to misinformed opinions. So, one last time. A computer program is a set of formal rules that takes one sequence of uninterpreted symbols and produces another sequence of uninterpreted symbols. Adding more rules and more uninterpreted symbols doesn't magically cause those symbols to be interpreted, and it cannot, by definition.