3 ms·

Do you think an average human from 1850 transported to the modern world would fare better than an LLM?

b112 1y ago
This isn't a serious question, is it? A human would immediately start learning, and understanding the differences. Forming new memories. People learn every, single day. An LLM would never have a single new memory, nor understand a single change had happened. Without being retokenized via new data it trains on, nothing of its symbolic world view would even change in the tiniest bit. Telling it so in a context window isn't the LLM learning, that's gone the second the context window is gone. In 5 years, 10 years, 1M years, that LLM would be precisely the same. It would view the world as 1850. Forever. Meanwhile, being told "It's 2025!" would immediately update the human's expectations. They would see the changes around them, and start learning. Note that initial reaction isn't relevant here, a person would eventually deal with it, and move forward. You have posted a link, in another response, where someone is working on dynamically updating an LLM. However that is not how current LLMs work, thus invalid in any attempt to refute how I describe LLMs currently. Further, until we see how this works -- precisely, it may be a non-starter. It may not work as described. It may be hype. It may work, but still require retraining if large amounts of symbolic data needs to change. There's a reason it takes massive farms of hardware to tokenize, and I'm skeptical that the quoted twitter link reforges all relationships. Again, we'll see.. and this is something required for an LLM to actually "learn" about changes to how tokens are weighted.
- hatthew 1y ago
  For long term memory, LLMs can learn new information by training on it. It would need a lot of training data, but it will get there eventually. For short term memory, they can store information in their context window. Yes the context window is finite, but they can still be very large, and there exist techniques to fit more information within that window. For medium term memory, there's not one single solution, but anecdotally it seems like solutions exist. My workplace has a chatbot that can ingest documents and save slack conversations, and then use the knowledge from those to answer questions indefinitely into the future. I haven't looked into how it works, but empirically it functions well across thousands of people for more than a year. Between a human and an LLM transported 200 years into the future, a human will certainly have better critical thinking skills, but I'd bet an LLM system designed for the task will probably be able to more quickly learn to have coherent and symmetric conversations about the world with modern humans. Overall, there's a lot that's unclear, and I think I'm pretty pessimistic about progress towards AGI, but I don't think we can deny that LLMs have something that is, at minimum, the best substitute for human intelligence that exists.

b112 1y ago

This isn't a serious question, is it? A human would immediately start learning, and understanding the differences. Forming new memories. People learn every, single day. An LLM would never have a single new memory, nor understand a single change had happened. Without being retokenized via new data it trains on, nothing of its symbolic world view would even change in the tiniest bit. Telling it so in a context window isn't the LLM learning, that's gone the second the context window is gone. In 5 years, 10 years, 1M years, that LLM would be precisely the same. It would view the world as 1850. Forever. Meanwhile, being told "It's 2025!" would immediately update the human's expectations. They would see the changes around them, and start learning. Note that initial reaction isn't relevant here, a person would eventually deal with it, and move forward. You have posted a link, in another response, where someone is working on dynamically updating an LLM. However that is not how current LLMs work, thus invalid in any attempt to refute how I describe LLMs currently. Further, until we see how this works -- precisely, it may be a non-starter. It may not work as described. It may be hype. It may work, but still require retraining if large amounts of symbolic data needs to change. There's a reason it takes massive farms of hardware to tokenize, and I'm skeptical that the quoted twitter link reforges all relationships. Again, we'll see.. and this is something required for an LLM to actually "learn" about changes to how tokens are weighted.

hatthew 1y ago
For long term memory, LLMs can learn new information by training on it. It would need a lot of training data, but it will get there eventually. For short term memory, they can store information in their context window. Yes the context window is finite, but they can still be very large, and there exist techniques to fit more information within that window. For medium term memory, there's not one single solution, but anecdotally it seems like solutions exist. My workplace has a chatbot that can ingest documents and save slack conversations, and then use the knowledge from those to answer questions indefinitely into the future. I haven't looked into how it works, but empirically it functions well across thousands of people for more than a year. Between a human and an LLM transported 200 years into the future, a human will certainly have better critical thinking skills, but I'd bet an LLM system designed for the task will probably be able to more quickly learn to have coherent and symmetric conversations about the world with modern humans. Overall, there's a lot that's unclear, and I think I'm pretty pessimistic about progress towards AGI, but I don't think we can deny that LLMs have something that is, at minimum, the best substitute for human intelligence that exists.