5 ms·

> A model can only present that intelligence. It cannot create it, and it is not aware of it. There is no objective-thought function. That's just like, your op

> A model can only present that intelligence. It cannot create it, and it is not aware of it. There is no objective-thought function. That's just like, your opinion man. I see both sides of this argument make statements like this that are clearly opinion and not backed up by facts. The truth is for now, we can't say yay or nay on whether the model has reasoning abilities, creative abilities, or a coherent internal world view. It's still a debate which is why there are academics like those in the OP trying to come up with theories that make predictions and then running experiments. It's still early days. If you were to ask my opinion, I'd say these current models are probably closer to the stochastic parrot than an AGI, but there is enough there to start wondering, especially if we have years of improvements coming down the pipeline. These aren't easy questions, and anyone espousing certainty now is doing themselves and others a disservice.

2024throwaway 3y ago
Do you understand how these models work under the hood? It's literally just picking the next most likely word, based on some fancy math. There is nothing that can create new insights or anything approaching intelligence.
- sowbug 3y ago
  "The statement that P=NP is [...]" (true or false) The article uses the concept of irony and the word "because" as examples of fairly sophisticated next-word predictors. You can imagine others that require even more understanding.
- doctoboggan 3y ago
  Yes, I understand how these models work under the hood as well as any lay person can. I worked through Andrej's video series and built my own toy model. Your argument is missing the whole concept of emergence, which is what is at question here. How does emergence come about, under what circumstances, and what is possible in those circumstance? It's an open question. Following your logic, one could argue that physics is basic and follows well understood rules. We can do some fancy math to predict what will happen in a given particle interaction. How would these basic particles ever give rise to life and minds with reasoning ability? Well you and I are an existence proof that basic rules when followed on massive systems can give rise to emergent behavior.
  - 2024throwaway 3y ago
    Following your logic, the million monkeys hammering randomly on typewriters that eventually produce the full works of Shakespeare, did so not through pure randomness, but by actually understanding the plight of Romeo and Juliet and the motivation behind Hamlet. Edit since I can't reply: "I don't like your answer, therefore you aren't engaging in good faith." lol
    - doctoboggan 3y ago
      I am aware of the thought experiment you are referring to, but if I witnessed a group of monkeys write the full works of Shakespeare in my lifetime then yes, I would actually consider it something worthy of investigating. However I can see you are not engaging in this conversation in good faith so I don't think there is anything else to discus with you.
      - mistermann 3y ago
        > However I can see you are not engaging in this conversation in good faith... What if this "categorize and ignore/forget" is causing humans to miss something extremely important in their midst? Bad faith is certainly possible here, but also possible is that this person genuinely believes what they say.
      - thomastjeffery 3y ago
        > However I can see you are not engaging in this conversation in good faith so I don't think there is anything else to discus with you. From my perspective, it seems pretty clear that you are the one arguing in bad faith. People are allowed to disagree with you. This person presented a thoughtful critique of your argument. It made plenty of sense to me.
    - two_in_one 3y ago
      > the million monkeys hammering randomly on typewriters that eventually produce the full works of Shakespeare, did so not through pure randomness, but by actually understanding the plight of Romeo and Juliet and the motivation behind Hamlet The funny thing 'yes'. Million monkeys cannot do it by randomly typing. They cannot even randomly type at all. Each monkey is intelligent creature, they will follow some patterns. If they did manage to produce Shakspeare they definitely understood something. Today it's easy to programmatically simulate 1m monkeys typing randomly 10 keys per second. Try it ;)
- ohwellhere 3y ago
  I think this critique would only be fair if we more completely understood how we ourselves work under the hood. The "fancy math" is a bunch of non-linear functions that encode something derived from series of tokens. I believe we're still exploring what the derived vector spaces "mean" for LLMs, with research inspecting smaller and simplified models to try to make sense of it all. It's clear that LLMs can "remix" in interesting ways. "ChatGPT, write me a Shakespearean-style sonnet about how LLMs work." (https://pastebin.com/FwzqWJ5W https://pastebin.com/FwzqWJ5W) It's not clear to me that our creativity and intelligence is not the same or a similar process. Beyond that, I'm not convinced there are definitions of meaning, knowledge, creativity, intelligence, etc, that are both useful and don't refer solely to the output. If I need a solution and none exists yet and something creates it, I will call that creative irrespective of process. But I don't know anything about anything. :)
  - thomastjeffery 3y ago
    Your argument is centered on a lack of understanding. How could anyone possibly argue that God and Heaven are fiction? No one can prove they don't exist! I prefer to approach such assertions with skepticism. There is no reason for me to believe the attestation is true, so I may as well believe it to be false. People remix expressions all the time. Any good training corpus will contain countless examples. If a statistical model can recognize the general semantic patterns that are present in human-remixed expressions, then it should be able to repeat them. So that's a thing that both humans and LLMs can do. Does that mean that LLMs are interesting and useful? Yes! Does it mean we are equivalent? I don't think so. If I could point out specifically what an LLM can't do, then I would be 99% of the way toward building something that can.
- shanusmagnus 3y ago
  These arguments are exhausting. Do you understand how neurons work under the hood? And yet, hook billions of them together in interesting ways, and look what happens.
- pillefitz 3y ago
  That doesn't sound too dissimilar from what human brains do, if you replace "math" by "spiking neural network"
- z7 3y ago
  "Emergence occurs when a complex entity has properties or behaviors that its parts do not have on their own, and emerge only when they interact in a wider whole." https://en.wikipedia.org/wiki/Emergence https://en.wikipedia.org/wiki/Emergence
- ryandvm 3y ago
  I find the "it's just picking the next most likely word" argument to be not very convincing. If anything it makes me wonder if that's not all there is to my own intelligence. Hell, try writing a paragraph backwards and tell me that you're not "just picking the next most likely word" as well.
  - nomel 3y ago
    My saddest observation is that I don't even have control over what comes out next, especially when talking/typing fast. "I" observe myself speaking, more than anything. Sometimes, as I'm talking, I think "that's a good point!" because I didn't consciously make it, it was just part of this "magic stream" that I'm listening to as much as the person I'm speaking to is listening. I also find letting that stream flow is when I'm at my sharpest. Same with problem solving. I point my intention at something and answers come out, through "intuition" and "creativity" which are almost entirely autonomous/subconscious. This makes me fear getting old, where this magic machine probably fades, almost certainly leaving me pointing my intention, and starting up sentences, without much happening after. I'm afraid I'll be left with my conscious self, which I don't find as entertaining.
  - thomastjeffery 3y ago
    It's able to perform that task because it has been fed examples. Essentially, it's the examples themselves that know how to do the task.
- randomdata 3y ago
  Not a whole lot different than what I do as a human. I can do it a lot faster, though, allowing more exploration into the second, third, and fourth most likely word at the same time. Nobody wants to wait years for ChatGPT to finally reply, so shortcuts have to be taken. I have never discovered any new insights, but it is apparent that the single most important attribute in discovering a new insight is time spent focused on finding a new insight. Undoubtedly, insight simply comes from executing on even more permutations than regular Joes like me take the time for until the right combination finally lands.
- QuantumGood 3y ago
  I think one of the arguments here is that we do not know how thinking in the human brain works, so we can't say that something else doesn't do something similar.
chlorion 3y ago
Not all of it is an opinion. The LLM does not have the ability to tell you what it knows and what it doesn't know about a subject. In order to get information out of it, you need to present it with a pattern that matches with something it was trained on. An example, is that an LLM can tell you how many programming languages it knows, how much it knows about them (which one it understands best), and how many it doesn't know. If you ask it a question like "how to do X in python", it will have been trained on something that roughly matches that and give a response, but if you ask "what things do you know how to do in python", and it simply is not capable of giving a response. I'm not sure that this means it doesn't have some form of understanding of things, and I do believe that AGI is possible to create in general with sufficient technology, and I do realize that our brains may work in a similar way, but I think LLMs are missing something when it comes to understanding and creating that we have not yet figured out.
- thomastjeffery 3y ago
  I think that something - or at least one of them - might be objectivity. An LLM has all of the what and none of the why.
- altruios 3y ago
  well... what we have an an LLM doesn't is state and feedback and restraint, a sense of time. We have an internal state of some kind, an LLM cheats having a state by having every input be the entire conversation + one new input. making the context grow overtime making that state unsustainable. The other thing an LLM is missing is proper feedback, it receives text chunk prompts and output text chunks. since it's output will never 'change the llm's internal state', it can not have a feedback loop. It can mimic one with the trick of refeeding messages as context, but that's untenable after a certain size input. Another is restraint, we human's don't have to respond to everything. we can choose to ignore input entirely, leave people on read, etc... LLM's respond to each input regardless of it is appropriate, and they only respond to stimuli (maybe we do as well, just subtle real world effects). last is a sense of time, which I don't see how the LLM can manage at all. It has no clock. Give an LLM a rich state (okay, maybe rename it too), have it be 'always on', give it a camera and microphone feed for inputs, give it a robot arm to control (pass the output of that back into the model for feedback), give it a second brain to decided to talk or not, and give it a clock too... and what comes out would look leagues better than current AI.
thomastjeffery 3y ago
> That's just like, your opinion man. Yeah, that's why I literally said so. You are welcome to disagree with me. My opinion isn't the whole point I intended to make here. I think there is too much focus on "wow look at the emergent behavior of this LLM!", and not enough focus on "where could this specific behavior come from, and why?" The second question may hold the key to both understanding what we have, and making something better.

2024throwaway 3y ago

Do you understand how these models work under the hood? It's literally just picking the next most likely word, based on some fancy math. There is nothing that can create new insights or anything approaching intelligence.

sowbug 3y ago
"The statement that P=NP is [...]" (true or false) The article uses the concept of irony and the word "because" as examples of fairly sophisticated next-word predictors. You can imagine others that require even more understanding.
doctoboggan 3y ago
Yes, I understand how these models work under the hood as well as any lay person can. I worked through Andrej's video series and built my own toy model. Your argument is missing the whole concept of emergence, which is what is at question here. How does emergence come about, under what circumstances, and what is possible in those circumstance? It's an open question. Following your logic, one could argue that physics is basic and follows well understood rules. We can do some fancy math to predict what will happen in a given particle interaction. How would these basic particles ever give rise to life and minds with reasoning ability? Well you and I are an existence proof that basic rules when followed on massive systems can give rise to emergent behavior.
- 2024throwaway 3y ago
  Following your logic, the million monkeys hammering randomly on typewriters that eventually produce the full works of Shakespeare, did so not through pure randomness, but by actually understanding the plight of Romeo and Juliet and the motivation behind Hamlet. Edit since I can't reply: "I don't like your answer, therefore you aren't engaging in good faith." lol
  - doctoboggan 3y ago
    I am aware of the thought experiment you are referring to, but if I witnessed a group of monkeys write the full works of Shakespeare in my lifetime then yes, I would actually consider it something worthy of investigating. However I can see you are not engaging in this conversation in good faith so I don't think there is anything else to discus with you.
    - mistermann 3y ago
      > However I can see you are not engaging in this conversation in good faith... What if this "categorize and ignore/forget" is causing humans to miss something extremely important in their midst? Bad faith is certainly possible here, but also possible is that this person genuinely believes what they say.
    - thomastjeffery 3y ago
      > However I can see you are not engaging in this conversation in good faith so I don't think there is anything else to discus with you. From my perspective, it seems pretty clear that you are the one arguing in bad faith. People are allowed to disagree with you. This person presented a thoughtful critique of your argument. It made plenty of sense to me.
  - two_in_one 3y ago
    > the million monkeys hammering randomly on typewriters that eventually produce the full works of Shakespeare, did so not through pure randomness, but by actually understanding the plight of Romeo and Juliet and the motivation behind Hamlet The funny thing 'yes'. Million monkeys cannot do it by randomly typing. They cannot even randomly type at all. Each monkey is intelligent creature, they will follow some patterns. If they did manage to produce Shakspeare they definitely understood something. Today it's easy to programmatically simulate 1m monkeys typing randomly 10 keys per second. Try it ;)
ohwellhere 3y ago
I think this critique would only be fair if we more completely understood how we ourselves work under the hood. The "fancy math" is a bunch of non-linear functions that encode something derived from series of tokens. I believe we're still exploring what the derived vector spaces "mean" for LLMs, with research inspecting smaller and simplified models to try to make sense of it all. It's clear that LLMs can "remix" in interesting ways. "ChatGPT, write me a Shakespearean-style sonnet about how LLMs work." (https://pastebin.com/FwzqWJ5W https://pastebin.com/FwzqWJ5W) It's not clear to me that our creativity and intelligence is not the same or a similar process. Beyond that, I'm not convinced there are definitions of meaning, knowledge, creativity, intelligence, etc, that are both useful and don't refer solely to the output. If I need a solution and none exists yet and something creates it, I will call that creative irrespective of process. But I don't know anything about anything. :)
- thomastjeffery 3y ago
  Your argument is centered on a lack of understanding. How could anyone possibly argue that God and Heaven are fiction? No one can prove they don't exist! I prefer to approach such assertions with skepticism. There is no reason for me to believe the attestation is true, so I may as well believe it to be false. People remix expressions all the time. Any good training corpus will contain countless examples. If a statistical model can recognize the general semantic patterns that are present in human-remixed expressions, then it should be able to repeat them. So that's a thing that both humans and LLMs can do. Does that mean that LLMs are interesting and useful? Yes! Does it mean we are equivalent? I don't think so. If I could point out specifically what an LLM can't do, then I would be 99% of the way toward building something that can.
shanusmagnus 3y ago
These arguments are exhausting. Do you understand how neurons work under the hood? And yet, hook billions of them together in interesting ways, and look what happens.
pillefitz 3y ago
That doesn't sound too dissimilar from what human brains do, if you replace "math" by "spiking neural network"
z7 3y ago
"Emergence occurs when a complex entity has properties or behaviors that its parts do not have on their own, and emerge only when they interact in a wider whole." https://en.wikipedia.org/wiki/Emergence https://en.wikipedia.org/wiki/Emergence
ryandvm 3y ago
I find the "it's just picking the next most likely word" argument to be not very convincing. If anything it makes me wonder if that's not all there is to my own intelligence. Hell, try writing a paragraph backwards and tell me that you're not "just picking the next most likely word" as well.
- nomel 3y ago
  My saddest observation is that I don't even have control over what comes out next, especially when talking/typing fast. "I" observe myself speaking, more than anything. Sometimes, as I'm talking, I think "that's a good point!" because I didn't consciously make it, it was just part of this "magic stream" that I'm listening to as much as the person I'm speaking to is listening. I also find letting that stream flow is when I'm at my sharpest. Same with problem solving. I point my intention at something and answers come out, through "intuition" and "creativity" which are almost entirely autonomous/subconscious. This makes me fear getting old, where this magic machine probably fades, almost certainly leaving me pointing my intention, and starting up sentences, without much happening after. I'm afraid I'll be left with my conscious self, which I don't find as entertaining.
- thomastjeffery 3y ago
  It's able to perform that task because it has been fed examples. Essentially, it's the examples themselves that know how to do the task.
randomdata 3y ago
Not a whole lot different than what I do as a human. I can do it a lot faster, though, allowing more exploration into the second, third, and fourth most likely word at the same time. Nobody wants to wait years for ChatGPT to finally reply, so shortcuts have to be taken. I have never discovered any new insights, but it is apparent that the single most important attribute in discovering a new insight is time spent focused on finding a new insight. Undoubtedly, insight simply comes from executing on even more permutations than regular Joes like me take the time for until the right combination finally lands.
QuantumGood 3y ago
I think one of the arguments here is that we do not know how thinking in the human brain works, so we can't say that something else doesn't do something similar.

chlorion 3y ago

Not all of it is an opinion. The LLM does not have the ability to tell you what it knows and what it doesn't know about a subject. In order to get information out of it, you need to present it with a pattern that matches with something it was trained on. An example, is that an LLM can tell you how many programming languages it knows, how much it knows about them (which one it understands best), and how many it doesn't know. If you ask it a question like "how to do X in python", it will have been trained on something that roughly matches that and give a response, but if you ask "what things do you know how to do in python", and it simply is not capable of giving a response. I'm not sure that this means it doesn't have some form of understanding of things, and I do believe that AGI is possible to create in general with sufficient technology, and I do realize that our brains may work in a similar way, but I think LLMs are missing something when it comes to understanding and creating that we have not yet figured out.

thomastjeffery 3y ago
I think that something - or at least one of them - might be objectivity. An LLM has all of the what and none of the why.
altruios 3y ago
well... what we have an an LLM doesn't is state and feedback and restraint, a sense of time. We have an internal state of some kind, an LLM cheats having a state by having every input be the entire conversation + one new input. making the context grow overtime making that state unsustainable. The other thing an LLM is missing is proper feedback, it receives text chunk prompts and output text chunks. since it's output will never 'change the llm's internal state', it can not have a feedback loop. It can mimic one with the trick of refeeding messages as context, but that's untenable after a certain size input. Another is restraint, we human's don't have to respond to everything. we can choose to ignore input entirely, leave people on read, etc... LLM's respond to each input regardless of it is appropriate, and they only respond to stimuli (maybe we do as well, just subtle real world effects). last is a sense of time, which I don't see how the LLM can manage at all. It has no clock. Give an LLM a rich state (okay, maybe rename it too), have it be 'always on', give it a camera and microphone feed for inputs, give it a robot arm to control (pass the output of that back into the model for feedback), give it a second brain to decided to talk or not, and give it a clock too... and what comes out would look leagues better than current AI.

thomastjeffery 3y ago

> That's just like, your opinion man. Yeah, that's why I literally said so. You are welcome to disagree with me. My opinion isn't the whole point I intended to make here. I think there is too much focus on "wow look at the emergent behavior of this LLM!", and not enough focus on "where could this specific behavior come from, and why?" The second question may hold the key to both understanding what we have, and making something better.