4 ms·
I think the way to swallow this bitter pill is to acknowledge they can "generalize" because all human knowledge is actually a relatively "small" finite distribu
by bilater 2y ago
I think the way to swallow this bitter pill is to acknowledge they can "generalize" because all human knowledge is actually a relatively "small" finite distribution that models are now big enough to pattern match on.
- gmueckl 2y agoCalling human knowledge small is hyperbole. I cannot get any LLM even close to giving accurate answers related to the things I know. They simply do not know what I, a single human being, knows. That's simply because I'm a subject matter expert on somewhat niche topics. There are easily hundreds of thousands of people like me out there. There's simply no way an LLM can even train on all of that because each bit of true expert knowledge necessarily comically underrepresented in any possible training set.
- deleted 2y ago[deleted]
- whattheheckheck 2y agoWhere you you instruct others to go to find out more about those niche topics?
- ashirviskas 2y agoNice try, AI company AI bot /s Though I'm not even sure about "/s", it is more than feasible to build such a bot that would gather quality information sources.
- UncleEntity 2y agoMaybe there's a way to reduce the dataset for a LLM to learn to reason down to the smallest possible set and then apply the vast knowledge of humankind on top of that? I mean, if it can reason about and process the data as it ingests it?
- Davidzheng 2y agoAnd another way is that the human brain is a relatively "small" circuit that models are now big enough to model ;)