4 ms·
I think LLMs are gonna be the "jumpstart" of more general AGI prototypes over the next couple years. The large corpus of text gives them a general basis of log
by atleastoptimal 4y ago
I think LLMs are gonna be the "jumpstart" of more general AGI prototypes over the next couple years.
The large corpus of text gives them a general basis of logical patterns, which can then be pruned iteratively in simulated environments.
- notnaut 4y agoSo you think the logical patterns found in human language might also be similar enough to the logical patterns found in other systems that these LLMs have a jumpstart in figuring those out? We already have such a poor idea of how these things seem to understand so much… it’ll be a wild day when cancer is cured and we have absolutely no idea why.
- SanderNL 4y agoIt feels like some type of Heisenbergian deal. You can solve what you want, but you cannot know how at the same time.
- atleastoptimal 4y agoIMO a sufficiently advanced AI should be able to do a full analysis of its neural architecture and explain it and break down its functionality.
- astrange 4y agoThere is no capability in an LLM to do this and we don't know how to build an "AI" that is not an LLM. (Just deciding that your LLM has magic powers because you've put it in a category called "AI" and you've decided that category has said magic powers is what that guy Wittgenstein was complaining about in philosophical problems. Besides, intelligence doesn't mean all your thoughts are automatically correct!)
- bambataa 4y ago> Besides, intelligence doesn't mean all your thoughts are automatically correct!) This is the true bitter lesson for HN.
- mrbungie 4y agoThat would be even more intelligent than the majority (or even all, YMMV) of humans.
- astrange 4y agoYou can't "cure cancer" because "cancer" is a label for different things with different causes. If your brain cells all have the wrong DNA there's no way to get different brain cells.
- notnaut 4y agoI was using the phrase sort of idiomatically, but fair enough!
- rimliu 4y agoGive computers the alphabet. It contains all the texts. Imagine the possibilities.
- RandomLensman 4y agoMy experience with ChatGPT is that it cannot reason from the data it has, i.e., it cannot take abstract concepts it can write about and use them on something it doesn't have the solution for. So not sure how the logical patterns really can evolve into something closer to AGI there. Maybe other LLMs? The inability to do math properly is really limiting, I think.
- lucubratory 4y agoI haven't yet seen an example of an LLM failing at math that cannot be very easily solved by having the LLM use a calculator, much as all humans do with math of any significance. It needs to use a calculator more often, but that's a completely negligible compute cost.
- RandomLensman 4y agoI meant more abstract math, e.g., construct something that needs math to construct it. Take a concept like risk neutral pricing and get it to construct a replicating portfolio for something not totally trivial (i.e., with not a lot of solved examples on the web). Fails for me.
- lucubratory 4y agoThat's fair, I've seen that it can really struggle coming up with novel algorithms. I am curious on if there is more improvement on that front in future models, because even its current performance at algorithmic manipulation is far, far better than e.g. GPT-3.