3 ms·
I hear people say this kind of thing but it confuses me. 1. What does inherit limitations mean? 2. How do we know something is an inherit limitation 3. Is it
by space_fountain 2y ago
I hear people say this kind of thing but it confuses me.
1. What does inherit limitations mean?
2. How do we know something is an inherit limitation
3. Is it a problem if arguments for a particular inherit limitation also apply to humans?
From what I’ve seen people will often say things like AI can’t be creative because it’s just a statistical machine, but humans are also “just” statistical machines. People might mean something like humans are more grounded because humans react not just to how the world already works but how the world reacts to actions they take, but this difference misunderstands how LLMs are trained. Like humans LLMs get most of their training from observing the world, but LLMs are also trained with re-enforcement learning and this will surely be an active area of research.
- mbesto 2y ago> 1. What does inherit limitations mean? One of many, but this is a simple one - LLMs are only limited to knowledge that is publicly available on the internet. This is "inherit" because thats how LLMs are essentially taught the information they retrieve today.
- space_fountain 2y agoBut this isn’t an inherit limitation is it? LLMs can be trained with private information and can have large context windows full of private info