3 ms·
If Karpathy truly believes that LLM's could be extended into trillion-dollar AGI, and had a workable plan how this might be achieved, then why is he not raising
by HarHarVeryFunny 2y ago
If Karpathy truly believes that LLM's could be extended into trillion-dollar AGI, and had a workable plan how this might be achieved, then why is he not raising money to do it, or working at a company pursuing AGI? His own background is more around vision (both PhD and at Tesla) than SOTA transformers...
It's interesting to see what Noam Shazeer (largely responsible for the transformer design) did with it after leaving Google.. not pursuing AGI but instead entertainment chatbots (character.ai).
AGI may well be achieved in our lifetimes (next 50 years), but 10 seems short considering that it's already been 8 years since "attention is all you need", and so far that's all we've got! Maybe o1 is a baby step beyond transformers (or is it really just an agent?), but I think we need multiple very large steps (transformer-level innovation) to get closer to AGI.
If one wants to appeal to authority, then how about LeCun, privvy to SOTA LLM research at Meta, yet confident this is entirely the wrong approach to achieve animal/human intelligence.