5 ms·

> "The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time." I always find this point a bit od

by jshen 3y ago

> "The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time." I always find this point a bit odd because humans aren't "optimized" to be correct either. > "LLMs have no understanding of the underlying reality" I struggle with this one because I see both sides of it. I was making a prompt the other day and gave a CSV file as an input and told the LLM if could only answer with values from one column and it did exactly as I asked. It's hard for me to see things like that and not believe it has an understanding at some level.

km3r 3y ago
There is a difference between general AI and generative AI. Humans can be creative and create vast fictional worlds with made up technology and magic. They can also optimize for facts and pass a BAR exam. LLM like have some form of understanding of the world, not sure if its anywhere near comparable to our understanding though. But they are still not general enough to really focus on facts. They can get close but the nature of statistics means there isn't hard checks to truth.
- happytiger 3y ago
  I think the next step logically is to see how the human brain functions. We have a similar model where certain parts of our brain are hopelessly useless by themselves as they are so overly specialized in one thing, but work in harmony to create network effectiveness. And we only just [mapped it for the first time](https://phys.org/news/2023-10-scientists-generate-single-cell-atlas-primate.html https://phys.org/news/2023-10-scientists-generate-single-cel...) so now much of what we are learning can potentially be applied to our “digital twins” in coming years. I imagine a lot of progress with AI will involve similar networks, with governance providing evolutionary paths and guidance for multiple concurrent goals. There are also lots of issues with training data and memory limits that make LLMs weak at continuity. Holes or weaknesses in the training data might “come through” in the behavior of hallucination. Where do you think general AI is going right now that we should be looking at? I’m overwhelmed by how large this field has become in the last five years and am always interested in what others know.
pixl97 3y ago
>I always find this point a bit odd because humans aren't "optimized" to be correct either. It depends which subsystem. The older the system in the body the less likely it is to have a high error rate, otherwise we'd die from cancer at a much higher rate or injure ourselves far more often. Of course this also depends on the definition of 'correct', if the system never changed we'd never evolve. >> "LLMs have no understanding of the underlying reality" This statement has always bothered me because it really depends on what you mean by 'underlying reality'. How many layers are we talking about? What does understanding mean? Because at the end of the day, humans don't really understand 'underlying reality, we just have a particular set of input devices we take in information and do some transformations on it... Where is the understanding happening?
- jshen 3y ago
  I'm talking about all of the cognitive science which seems to point to the fact that we often care more about fitting in with our group that being correct. Group cohesion seems to have been selected for. Edit: there is also loads of evidence that our brains are wired to make incorrect decisions in many cases. Loss aversion is one example.
tikhonj 3y ago
Humans have evolved to communicate based on some sort of shared mental model and shared intentionality—human speech generally has some sort of more-or-less coherent semantics by its very nature. While it's hard to reason about what goes on inside the neural network, there are too many examples of LLMs outputting not-even-wrong nonsense to conclude they have a similar internal model, even assuming that having an internal model at all would make sense in this context!
- jshen 3y ago
  Have you read social media? I see an overwhelming volume of "not-even-wrong" nonsense from humans all the time.
pdonis 3y ago
> humans aren't "optimized" to be correct either Humans at least have a concept of "being correct", even if we don't always set that as our primary goal when communicating. LLMs don't even have a concept of "being correct". That would require having a concept of an "external world" that text refers to, which LLMs don't have. All they have is the text in their training data.
- famouswaffles 3y ago
  >LLMs don't even have a concept of "being correct" They clearly do as plenty research indicates. You've just decided not to accept this. Pure confirmation bias in action.
  - pdonis 3y ago
    > They clearly do as plenty research indicates. I have seen plenty of researchers claim this. What I have not seen is actual support for such a claim.
    - famouswaffles 3y ago
      There's plenty support. You just need to know how to read.
      - yladiz 3y ago
        Please link to published papers that prove what you’re positing.
        happytiger 3y ago
        To tack on, I would love to read more research on this topic if there is relevant research to read. I have not read much compelling evidence though I have seen a lot of researchers conjecturing and some inappropriately reaching (and anthropomorphising the living hell out of ai in the process sometimes). There’s a lot to learn and I don’t think anyone can keep up with all the papers coming out so let’s keep things positive and work together to learn as much as we can og_kalu. Thanks for letting me tag this comment on the end of thread. This is HN. We’re all nerds here anyways and nobody can keep up with everything in a field moving as fast at AI is these days. :)
danmaz74 3y ago
Humans aren't optimized to be correct, but they are optimized to survive and reproduce themselves in the real world. That's why we develop an understanding of the real world, while LLM can, at most, develop an understanding of human language and the constructs that human language can create. But without the napping that we have between that language and our experience of the world.
- happytiger 3y ago
  It gets very interesting when we start providing digital input of analog source data in film, photograph, sound and speech. We recently started providing perception inputs, just like you’re talking about. Obviously it’s nascent and not exactly a revolutionary statement, but it’s interesting to see the progression towards our experience. I’m also interested in how the analog computing revolution is going to converge with AI in coming years, as we are making huge inroads into analog computing and that means cheap ubiquitous sensor inputs right in time for AI to start maturing.
- gaganyaan 3y ago
  LLMs develop a model of the world, experienced through human-generated text. The same as I've never been to China, but I have an internal model of China due to experiencing it through human-generated text.
  - danmaz74 3y ago
    There is a huge difference between your model and an LLM model in your example. Your model "knows" that China has rivers (of which you have direct experience), coastlines (same), trees, mountains etc. etc. An LLM has no tangible experience of any of the concepts included in the text.
    - gaganyaan 3y ago
      Both my internal model and an LLM are built from sensory data. My model knows what a river is from having experienced it through sensors that an LLM doesn't currently have, but don't expect that limitation to last. Also, given how many words humans have written describing every part of the experience, LLMs can generate a pretty good understanding of it as-is.

by jshen 3y ago

km3r 3y ago

There is a difference between general AI and generative AI. Humans can be creative and create vast fictional worlds with made up technology and magic. They can also optimize for facts and pass a BAR exam. LLM like have some form of understanding of the world, not sure if its anywhere near comparable to our understanding though. But they are still not general enough to really focus on facts. They can get close but the nature of statistics means there isn't hard checks to truth.

happytiger 3y ago
I think the next step logically is to see how the human brain functions. We have a similar model where certain parts of our brain are hopelessly useless by themselves as they are so overly specialized in one thing, but work in harmony to create network effectiveness. And we only just [mapped it for the first time](https://phys.org/news/2023-10-scientists-generate-single-cell-atlas-primate.html https://phys.org/news/2023-10-scientists-generate-single-cel...) so now much of what we are learning can potentially be applied to our “digital twins” in coming years. I imagine a lot of progress with AI will involve similar networks, with governance providing evolutionary paths and guidance for multiple concurrent goals. There are also lots of issues with training data and memory limits that make LLMs weak at continuity. Holes or weaknesses in the training data might “come through” in the behavior of hallucination. Where do you think general AI is going right now that we should be looking at? I’m overwhelmed by how large this field has become in the last five years and am always interested in what others know.

pixl97 3y ago

>I always find this point a bit odd because humans aren't "optimized" to be correct either. It depends which subsystem. The older the system in the body the less likely it is to have a high error rate, otherwise we'd die from cancer at a much higher rate or injure ourselves far more often. Of course this also depends on the definition of 'correct', if the system never changed we'd never evolve. >> "LLMs have no understanding of the underlying reality" This statement has always bothered me because it really depends on what you mean by 'underlying reality'. How many layers are we talking about? What does understanding mean? Because at the end of the day, humans don't really understand 'underlying reality, we just have a particular set of input devices we take in information and do some transformations on it... Where is the understanding happening?

jshen 3y ago
I'm talking about all of the cognitive science which seems to point to the fact that we often care more about fitting in with our group that being correct. Group cohesion seems to have been selected for. Edit: there is also loads of evidence that our brains are wired to make incorrect decisions in many cases. Loss aversion is one example.

tikhonj 3y ago

Humans have evolved to communicate based on some sort of shared mental model and shared intentionality—human speech generally has some sort of more-or-less coherent semantics by its very nature. While it's hard to reason about what goes on inside the neural network, there are too many examples of LLMs outputting not-even-wrong nonsense to conclude they have a similar internal model, even assuming that having an internal model at all would make sense in this context!

jshen 3y ago
Have you read social media? I see an overwhelming volume of "not-even-wrong" nonsense from humans all the time.

pdonis 3y ago

> humans aren't "optimized" to be correct either Humans at least have a concept of "being correct", even if we don't always set that as our primary goal when communicating. LLMs don't even have a concept of "being correct". That would require having a concept of an "external world" that text refers to, which LLMs don't have. All they have is the text in their training data.

famouswaffles 3y ago
>LLMs don't even have a concept of "being correct" They clearly do as plenty research indicates. You've just decided not to accept this. Pure confirmation bias in action.
- pdonis 3y ago
  > They clearly do as plenty research indicates. I have seen plenty of researchers claim this. What I have not seen is actual support for such a claim.
  - famouswaffles 3y ago
    There's plenty support. You just need to know how to read.
    - yladiz 3y ago
      Please link to published papers that prove what you’re positing.
      - happytiger 3y ago
        To tack on, I would love to read more research on this topic if there is relevant research to read. I have not read much compelling evidence though I have seen a lot of researchers conjecturing and some inappropriately reaching (and anthropomorphising the living hell out of ai in the process sometimes). There’s a lot to learn and I don’t think anyone can keep up with all the papers coming out so let’s keep things positive and work together to learn as much as we can og_kalu. Thanks for letting me tag this comment on the end of thread. This is HN. We’re all nerds here anyways and nobody can keep up with everything in a field moving as fast at AI is these days. :)

danmaz74 3y ago

Humans aren't optimized to be correct, but they are optimized to survive and reproduce themselves in the real world. That's why we develop an understanding of the real world, while LLM can, at most, develop an understanding of human language and the constructs that human language can create. But without the napping that we have between that language and our experience of the world.

happytiger 3y ago
It gets very interesting when we start providing digital input of analog source data in film, photograph, sound and speech. We recently started providing perception inputs, just like you’re talking about. Obviously it’s nascent and not exactly a revolutionary statement, but it’s interesting to see the progression towards our experience. I’m also interested in how the analog computing revolution is going to converge with AI in coming years, as we are making huge inroads into analog computing and that means cheap ubiquitous sensor inputs right in time for AI to start maturing.
gaganyaan 3y ago
LLMs develop a model of the world, experienced through human-generated text. The same as I've never been to China, but I have an internal model of China due to experiencing it through human-generated text.
- danmaz74 3y ago
  There is a huge difference between your model and an LLM model in your example. Your model "knows" that China has rivers (of which you have direct experience), coastlines (same), trees, mountains etc. etc. An LLM has no tangible experience of any of the concepts included in the text.
  - gaganyaan 3y ago
    Both my internal model and an LLM are built from sensory data. My model knows what a river is from having experienced it through sensors that an LLM doesn't currently have, but don't expect that limitation to last. Also, given how many words humans have written describing every part of the experience, LLMs can generate a pretty good understanding of it as-is.