3 ms·

Humans aren't optimized to be correct, but they are optimized to survive and reproduce themselves in the real world. That's why we develop an understanding of t

Humans aren't optimized to be correct, but they are optimized to survive and reproduce themselves in the real world. That's why we develop an understanding of the real world, while LLM can, at most, develop an understanding of human language and the constructs that human language can create. But without the napping that we have between that language and our experience of the world.

happytiger 3y ago
It gets very interesting when we start providing digital input of analog source data in film, photograph, sound and speech. We recently started providing perception inputs, just like you’re talking about. Obviously it’s nascent and not exactly a revolutionary statement, but it’s interesting to see the progression towards our experience. I’m also interested in how the analog computing revolution is going to converge with AI in coming years, as we are making huge inroads into analog computing and that means cheap ubiquitous sensor inputs right in time for AI to start maturing.
gaganyaan 3y ago
LLMs develop a model of the world, experienced through human-generated text. The same as I've never been to China, but I have an internal model of China due to experiencing it through human-generated text.
- danmaz74 3y ago
  There is a huge difference between your model and an LLM model in your example. Your model "knows" that China has rivers (of which you have direct experience), coastlines (same), trees, mountains etc. etc. An LLM has no tangible experience of any of the concepts included in the text.
  - gaganyaan 3y ago
    Both my internal model and an LLM are built from sensory data. My model knows what a river is from having experienced it through sensors that an LLM doesn't currently have, but don't expect that limitation to last. Also, given how many words humans have written describing every part of the experience, LLMs can generate a pretty good understanding of it as-is.
jshen 3y ago
"[humans] are optimized to survive and reproduce ... in the real world." I think about this difference a lot, and a fair bit of AI/ML attempts to reproduce this process to some degree.

happytiger 3y ago

It gets very interesting when we start providing digital input of analog source data in film, photograph, sound and speech. We recently started providing perception inputs, just like you’re talking about. Obviously it’s nascent and not exactly a revolutionary statement, but it’s interesting to see the progression towards our experience. I’m also interested in how the analog computing revolution is going to converge with AI in coming years, as we are making huge inroads into analog computing and that means cheap ubiquitous sensor inputs right in time for AI to start maturing.

gaganyaan 3y ago

LLMs develop a model of the world, experienced through human-generated text. The same as I've never been to China, but I have an internal model of China due to experiencing it through human-generated text.

danmaz74 3y ago
There is a huge difference between your model and an LLM model in your example. Your model "knows" that China has rivers (of which you have direct experience), coastlines (same), trees, mountains etc. etc. An LLM has no tangible experience of any of the concepts included in the text.
- gaganyaan 3y ago
  Both my internal model and an LLM are built from sensory data. My model knows what a river is from having experienced it through sensors that an LLM doesn't currently have, but don't expect that limitation to last. Also, given how many words humans have written describing every part of the experience, LLMs can generate a pretty good understanding of it as-is.

jshen 3y ago

"[humans] are optimized to survive and reproduce ... in the real world." I think about this difference a lot, and a fair bit of AI/ML attempts to reproduce this process to some degree.