5 ms·

Two concepts of intelligence and neither have remotely anything to do with real intelligence, academics sure like to play with words. I suppose this is how they

Two concepts of intelligence and neither have remotely anything to do with real intelligence, academics sure like to play with words. I suppose this is how they justify their own existence; in the absence of being intelligent enough to contribute anything of value, they must instead engage in wordplay that obfuscates the meaning of words to the point nobody understands what the hell they're talking about, and confuses the lack of understanding of what they're talking about for the academics being more intelligent than the reader. Intelligence, in the real world, is the ability to reason about logic. If 1 + 1 is 2, and 1 + 2 is 3, then 1 + 3 must be 4. This is deterministic, and it is why LLMs are not intelligent and can never be intelligent no matter how much better they get at superficially copying the form of output of intelligence. Probabilistic prediction is inherently incompatible with deterministic deduction. We're years into being told AGI is here (for whatever squirmy value of AGI the hype huckster wants to shill), and yet LLMs, as expected, still cannot do basic arithmetic that a child could do without being special-cased to invoke a tool call. How is it that we can go about ignoring reality for so long?

bdbdbdb 9mo ago
I keep coming back to this. The most recent version of chatgpt I tried was able to tell me how many letter 'r's were in a very long string of characters only by writing and executing a python script to do this. Some people say this is impressive, but any 5 year old could count the letters without knowing any python.
- deleted 9mo ago
  [deleted]
- williamcotton 9mo ago
  How is counting not a technology? The calculations are internal but they happen due to the orchestration of specific parts of the brain. That is to ask, why can't we consider our brains to be using their own internal tools? I certainly don't think about multiplying two-digit numbers in my head in the same manner as when playing a Dm to a G7 chord that begs to resolve to a C!
- armchairhacker 9mo ago
  The 5-year old counts with an algorithm: they remember the current number (working memory, roughly analogous to context), scan the page and move their finger to the next letter. They were taught this. It's not much different than ChatGPT being trained write a to Python script. A notable difference is that it's much more efficient to teach something new to a 5-year old than fine-tune or retrain an LLM.
  - notahacker 9mo ago
    A theory behind LLM intelligence is that the layer structure forms some sort of world model that has a much higher fidelity than simple pattern matching texts. In specific cases, like where the language is a DSL which maps perfectly to a representation of an Othello gameboard, this appears to actually be the case. But basic operations like returning the number of times the letter r appears in 'strawberry' form a useful counterexample: the LLM has ingested many hundreds of books explaining how letters spell out words and how to count (which are pretty simple concepts very easily stored in small amounts of computer memory) and yet its layers apparently couldn't model it from all that input (apparently an issue with being unable to parse a connection between the token 'strawberry' and its constituent letters... not exactly high-level reasoning). It appears LLMs got RHLFed into generating suitable Python scripts after the issue was exposed, which is an efficient way of getting better answers, but feels rather like handing the child really struggling with their arithmetic a calculator...
satisfice 9mo ago
Intelligence is not just about reasoning with logic. Computers are already made to do that. The key thing is modeling. You must model a situation in a useful way in order to apply logic to it. And then there is intention, which guides the process.
- anonymous908213 9mo ago
  Our computer programs execute logic, but cannot reason about it. Reasoning is the ability to dynamically consider constraints we've never seen before and then determine how those constraints would lead to a final conclusion. The rules of mathematics we follow are not programmed into our DNA; we learn them and follow them while our human-programming is actively running. But we can just as easily, at any point, make up new constraints and follow them to new conclusions. What if 1 + 2 is 2 and 1 + 3 is 3? Then we can reason that under these constraints we just made up, 1 + 4 is 4, without ever having been programmed to consider these rules.
  - satisfice 9mo ago
    Executing logic is deductive reasoning. But, yes, I get it. There are also other layers of reasoning, and other forms. For instance, abductive and inductive inference.
anonymous908213 9mo ago
Addendum: > With recent advances in AI, it becomes ever harder for proponents of intelligence-as-understanding to continue asserting that those tools have no clue and “just” perform statistical next-token prediction. ??????? No, that is still exactly what they do. The article then lists a bunch of examples in which this in trivially exactly what is happening. > “The cat chased the . . .” (multiple connections are plausible, so how is that not understanding probability?) It doesn't need to "understand" probability. "The cat chased the mouse" shows up in the distribution 10 times. "The cat chased the bird" shows up in the distribution 5 times. Absent any other context, with the simplest possible model, it now has a probability of 2/3 for the mouse and 1/3 for the bird. You can make the probability calculations as complex as you want, but how could you possibly trout this out as an example that an LLM completing this sentence isn't a matter of trivial statistical prediction? Academia needs an asteroid, holy hell. [I originally edited this into my post, but two people had replied by then, so I've split it off into its own comment.]
- n4r9 9mo ago
  One question is how do you know that you (or humans in general) aren't also just applying statistical language rules, but are convincing yourself of some underlying narrative involving logical rules? I don't know the answer to this.
  - anonymous908213 9mo ago
    We engage in many exercises in deterministic logic. Humans invented entire symbolic systems to describe mathematics without any prior art in a dataset. We apply these exercises in deterministic logic to reality, and reality confirms that our logical exercises are correct to within extremely small tolerances, allowing us to do mind-boggling things like trips to the moon, or engineering billions of transistors organized on a nanometer scale and making them mimick the appearance of human language by executing really cool math really quickly. None of this could have been achieved from scratch by probabilistic behaviour modelled on a purely statistical analysis of past information, which is immediately evident from the fact that, as mentioned, an LLM cannot do basic arithmetic, or any other deterministic logical exercise in which the answer cannot be predicted from already being in the training distribution, while we can. People will point to humans sometimes making mistakes, but that is because we take mental shortcuts to save energy. If you put a gun to our head and say "if you get this basic arithmetic problem wrong, you will die" we will reason long enough to get it right. People try prompting that with LLMs, and they still can't do it, funnily enough.
djoldman 9mo ago
Many people would require an intelligent entity to successfully complete tasks with non-deterministic outputs.
messe 9mo ago
> Probabilistic prediction is inherently incompatible with deterministic deduction Prove that humans do it.
dcre 9mo ago
I just don’t think the question is about determinism and probability at all. When we think, our thoughts are influenced by any number of extra-logical factors, factors that operate on a level of abstraction totally alien to the logical content of thought. Things like chemical reactions in our brains or whether the sun is out or whether some sound distracts us or a smell reminds us of some memory. Whether these factors are deterministic or probabilistic is irrelevant — if anything the effect of these factors on our thinking is deterministic. What matters is that the mechanical process of producing thought is clearly influenced (perhaps entirely!) by non-rational factors. To me this means that any characterization of the essence of thinking that relies too heavily on its logical structure cannot be telling the whole story.
jimbokun 9mo ago
Inductive logic is not synonymous with intelligence.
ACCount37 9mo ago
How many humans are you disqualifying from "real intelligence"? Logic isn't exactly natural to wet meat.
- anonymous908213 9mo ago
  Virtually all humans demonstrate the ability to reason about logic within a few years of development. Anyone who can conduct arbitrary arithmetic, such as correctly determining that X + Y = Z without having seen and memorised the result beforehand and without using a calculator, is reasoning about logic and therefore intelligent. Some are more intelligent than others and can handle more complex reasoning; my theory is that differences in intelligence are a result of brains setting a threshold limiting energy consumption rather than letting one thread consume all system resources. The threshold differs based on environment; in an environment that rewards correct reasoning, more energy will be allocated for it, whereas in an environment where correct reasoning is not important, less energy will be allocated as it is not beneficial or necessary for survival. This has compounding effects over the course of one's life, as the more energy that is allocated to building pathways between neurons conductive to reasoning in a lifetime, the more complex the reasoning that can be conducted.
  - ACCount37 9mo ago
    No one can "conduct arbitrary arithmetic". It's a problem that goes to infinity and you're throwing it at humans who very much don't. Even in your own mind, "5 + 25" and "7 + 27" take two different routes based on different pattern-matched heuristics. You don't have a single "universal addition algorithm". You have a bag of pattern-matching features, and maybe a slow unreliable fallback path, if you have learned any. Which is why "5000+5000" is trivial but "4523+5438" isn't. The problem with your "theory" is a common one: it's utter bullshit. It says "here's what I think is true" but stops short of "why", let alone "why it fits reality better than the alternatives" or "here's what this theory allows you to do". It's like claiming that "all matter is phlogiston". It was entertaining when people were doing that in Ancient Greece, but by now, it's just worthless noise.
    - 9mo ago

bdbdbdb 9mo ago

I keep coming back to this. The most recent version of chatgpt I tried was able to tell me how many letter 'r's were in a very long string of characters only by writing and executing a python script to do this. Some people say this is impressive, but any 5 year old could count the letters without knowing any python.

deleted 9mo ago
[deleted]
williamcotton 9mo ago
How is counting not a technology? The calculations are internal but they happen due to the orchestration of specific parts of the brain. That is to ask, why can't we consider our brains to be using their own internal tools? I certainly don't think about multiplying two-digit numbers in my head in the same manner as when playing a Dm to a G7 chord that begs to resolve to a C!
armchairhacker 9mo ago
The 5-year old counts with an algorithm: they remember the current number (working memory, roughly analogous to context), scan the page and move their finger to the next letter. They were taught this. It's not much different than ChatGPT being trained write a to Python script. A notable difference is that it's much more efficient to teach something new to a 5-year old than fine-tune or retrain an LLM.
- notahacker 9mo ago
  A theory behind LLM intelligence is that the layer structure forms some sort of world model that has a much higher fidelity than simple pattern matching texts. In specific cases, like where the language is a DSL which maps perfectly to a representation of an Othello gameboard, this appears to actually be the case. But basic operations like returning the number of times the letter r appears in 'strawberry' form a useful counterexample: the LLM has ingested many hundreds of books explaining how letters spell out words and how to count (which are pretty simple concepts very easily stored in small amounts of computer memory) and yet its layers apparently couldn't model it from all that input (apparently an issue with being unable to parse a connection between the token 'strawberry' and its constituent letters... not exactly high-level reasoning). It appears LLMs got RHLFed into generating suitable Python scripts after the issue was exposed, which is an efficient way of getting better answers, but feels rather like handing the child really struggling with their arithmetic a calculator...

satisfice 9mo ago

Intelligence is not just about reasoning with logic. Computers are already made to do that. The key thing is modeling. You must model a situation in a useful way in order to apply logic to it. And then there is intention, which guides the process.

anonymous908213 9mo ago
Our computer programs execute logic, but cannot reason about it. Reasoning is the ability to dynamically consider constraints we've never seen before and then determine how those constraints would lead to a final conclusion. The rules of mathematics we follow are not programmed into our DNA; we learn them and follow them while our human-programming is actively running. But we can just as easily, at any point, make up new constraints and follow them to new conclusions. What if 1 + 2 is 2 and 1 + 3 is 3? Then we can reason that under these constraints we just made up, 1 + 4 is 4, without ever having been programmed to consider these rules.
- satisfice 9mo ago
  Executing logic is deductive reasoning. But, yes, I get it. There are also other layers of reasoning, and other forms. For instance, abductive and inductive inference.

anonymous908213 9mo ago

Addendum: > With recent advances in AI, it becomes ever harder for proponents of intelligence-as-understanding to continue asserting that those tools have no clue and “just” perform statistical next-token prediction. ??????? No, that is still exactly what they do. The article then lists a bunch of examples in which this in trivially exactly what is happening. > “The cat chased the . . .” (multiple connections are plausible, so how is that not understanding probability?) It doesn't need to "understand" probability. "The cat chased the mouse" shows up in the distribution 10 times. "The cat chased the bird" shows up in the distribution 5 times. Absent any other context, with the simplest possible model, it now has a probability of 2/3 for the mouse and 1/3 for the bird. You can make the probability calculations as complex as you want, but how could you possibly trout this out as an example that an LLM completing this sentence isn't a matter of trivial statistical prediction? Academia needs an asteroid, holy hell. [I originally edited this into my post, but two people had replied by then, so I've split it off into its own comment.]

n4r9 9mo ago
One question is how do you know that you (or humans in general) aren't also just applying statistical language rules, but are convincing yourself of some underlying narrative involving logical rules? I don't know the answer to this.
- anonymous908213 9mo ago
  We engage in many exercises in deterministic logic. Humans invented entire symbolic systems to describe mathematics without any prior art in a dataset. We apply these exercises in deterministic logic to reality, and reality confirms that our logical exercises are correct to within extremely small tolerances, allowing us to do mind-boggling things like trips to the moon, or engineering billions of transistors organized on a nanometer scale and making them mimick the appearance of human language by executing really cool math really quickly. None of this could have been achieved from scratch by probabilistic behaviour modelled on a purely statistical analysis of past information, which is immediately evident from the fact that, as mentioned, an LLM cannot do basic arithmetic, or any other deterministic logical exercise in which the answer cannot be predicted from already being in the training distribution, while we can. People will point to humans sometimes making mistakes, but that is because we take mental shortcuts to save energy. If you put a gun to our head and say "if you get this basic arithmetic problem wrong, you will die" we will reason long enough to get it right. People try prompting that with LLMs, and they still can't do it, funnily enough.

djoldman 9mo ago

Many people would require an intelligent entity to successfully complete tasks with non-deterministic outputs.

messe 9mo ago

> Probabilistic prediction is inherently incompatible with deterministic deduction Prove that humans do it.

dcre 9mo ago

I just don’t think the question is about determinism and probability at all. When we think, our thoughts are influenced by any number of extra-logical factors, factors that operate on a level of abstraction totally alien to the logical content of thought. Things like chemical reactions in our brains or whether the sun is out or whether some sound distracts us or a smell reminds us of some memory. Whether these factors are deterministic or probabilistic is irrelevant — if anything the effect of these factors on our thinking is deterministic. What matters is that the mechanical process of producing thought is clearly influenced (perhaps entirely!) by non-rational factors. To me this means that any characterization of the essence of thinking that relies too heavily on its logical structure cannot be telling the whole story.

jimbokun 9mo ago

Inductive logic is not synonymous with intelligence.

ACCount37 9mo ago

How many humans are you disqualifying from "real intelligence"? Logic isn't exactly natural to wet meat.

anonymous908213 9mo ago
Virtually all humans demonstrate the ability to reason about logic within a few years of development. Anyone who can conduct arbitrary arithmetic, such as correctly determining that X + Y = Z without having seen and memorised the result beforehand and without using a calculator, is reasoning about logic and therefore intelligent. Some are more intelligent than others and can handle more complex reasoning; my theory is that differences in intelligence are a result of brains setting a threshold limiting energy consumption rather than letting one thread consume all system resources. The threshold differs based on environment; in an environment that rewards correct reasoning, more energy will be allocated for it, whereas in an environment where correct reasoning is not important, less energy will be allocated as it is not beneficial or necessary for survival. This has compounding effects over the course of one's life, as the more energy that is allocated to building pathways between neurons conductive to reasoning in a lifetime, the more complex the reasoning that can be conducted.
- ACCount37 9mo ago
  No one can "conduct arbitrary arithmetic". It's a problem that goes to infinity and you're throwing it at humans who very much don't. Even in your own mind, "5 + 25" and "7 + 27" take two different routes based on different pattern-matched heuristics. You don't have a single "universal addition algorithm". You have a bag of pattern-matching features, and maybe a slow unreliable fallback path, if you have learned any. Which is why "5000+5000" is trivial but "4523+5438" isn't. The problem with your "theory" is a common one: it's utter bullshit. It says "here's what I think is true" but stops short of "why", let alone "why it fits reality better than the alternatives" or "here's what this theory allows you to do". It's like claiming that "all matter is phlogiston". It was entertaining when people were doing that in Ancient Greece, but by now, it's just worthless noise.
  - 9mo ago