5 ms·
Frankly, we may be barking up the wrong tree with LLMs. Sure they deliver novel and very marketable results, hence the insane funding, but I can’t help but feel
by TerrifiedMouse 3y ago
Frankly, we may be barking up the wrong tree with LLMs. Sure they deliver novel and very marketable results, hence the insane funding, but I can’t help but feel it’s really just a parlor trick and there is a yet undiscovered algorithm that can actually deliver the AGI that we seek - AGI that reliable and precise like fictional AIs.
- jrockway 3y agoI think a lot of progress in the field of AI comes from random breakthroughs rather than evolution of existing ideas. For that reason, I think AGI will probably be another random breakthrough, and not just an LLM with more parameters. Good if you're NVidia, bad if you're OpenAI. The random breakthrough could happen at Google, or it could happen in some professor's basement. There is no way to know, and no way to throw money at the problem and guarantee a return.
- TillE 3y ago> bad if you're OpenAI Just a few days ago there was a (somewhat cryptic) report about OpenAI developing a model that could do simple math. I'm certain they're pursuing quite a lot of research that's not directly related to LLMs. I don't think we're heading towards AGI any time soon, but I can imagine a complex system in the next couple years which uses an LLM for text generation, but offloads its serious "thinking" on to predictable, specialized subsystems. It's easy to imagine a lot of possibilities for code in particular.
- jrockway 3y agoI look at it like finding a lost treasure or something like that. The government expedition with a 1 trillion dollar budget might find it first... or it might be in some guy's attic and his kid came over and happened to notice it. Spending infinite money on research increases your chances, but it's still a chance.
- Belomolo 3y agoThe protein folding project was basically solved due to money and ml. Alpha fold. The leap that area took thanks to it was a leap. It's more than just something potentially better
- TerrifiedMouse 3y ago> Good if you're NVidia, bad if you're OpenAI Not necessarily good for Nvidia. If the new algorithm is branch heavy, don’t parallelize well, … etc. and operate better on CPUs rather than GPUs, it will be Intel, AMD, and ARM that enjoy the windfall.
- jrockway 3y agoGood point!
- ImHereToVote 3y agoIt would be a very poor AGI if it didn't create a company that does AGI ASICs in the first few days. Minutes?
- actionfromafar 3y agoA cat is AGI. (Well, not artificial, but you get my point.) It couldn’t design ASICs in minutes.
- jrockway 3y agoI hope that computerized AGIs also get the zoomies at 3AM.
- ImHereToVote 3y agoA cat is a very narrow form of intelligence.
- actionfromafar 3y agoTrue. And yet so incredibly wider than any AI.
- ImHereToVote 3y agoI can't use my cat to make a painting in the style of Van Gogh. Nor can I use my cat write an email to my engineering team that I'm laying of 45% of them, in corporate speak. Nor can I use my cat to refactor a class into a more functional pattern. Hell I can't even get my cat to hunt mice. He's a cute fucker though.
- deleted 3y ago[deleted]
- oblio 3y ago> there is a yet undiscovered algorithm that can actually deliver the AGI that we seek - AGI that reliable and precise like fictional AIs. Nobody wants to hear this, but AGI could very well be the next invention after practical cold fusion and faster-than-light travel. Yes, we're moving faster, but does anyone know how long the road is? A space rocket is moving very quickly (50k kmph?), much faster than any other technology such as airliners, maglev trains, but the closest star is still several light years away.