3 ms·
Even if LLMs are a local maxima, given that we're even asking that question implies that we are closer to AGI than caveman who didn't have GPUs, even if we don'
by fragmede 3y ago
Even if LLMs are a local maxima, given that we're even asking that question implies that we are closer to AGI than caveman who didn't have GPUs, even if we don't know exactly how much closer. Unless it takes us 12,000 years to get to AGI, which, it could, since we don't have it yet, but assuming we get AGI before 12,000 years are up, which is when the end of the Paleolithic era was, then yes, we can say we are closer to AGI than cavemen, regardless of if LLM or GPUs are how it comes about. If it never does, then this comment will age poorly in 12,000 years, but I'm okay with that.
- dmitrygr 3y ago> given that we're even asking that question implies that we are closer to AGI than caveman "Am I close to having gills and breathing underwater?" The fact that I asked, changes nothing.
- joelfried 3y agoYou don't think having functional computers being capable of trillions of operations per second is even the slightest bit of an improvement over use of literal stones? I am genuinely curious: What do you imagine would be the kind of thing that would be a meaningful step towards AGI? I have no idea when we'll get there for real, but it seems a pretty big assertion that nothing invented in the last 150 years even helped. So what do you think would help?
- dmitrygr 3y agoOk. I’ll rephrase. In caves we were 0% there. Now we’re 0.00001% there :). Maybe. May need a few more zeroes. What we have helps probably, but hard to be sure how much. Maybe we need 1e40 FLOPS for AGI. In that case going from 1FLOP (human in a cave) to 1e14 FLOPS (a GPU now) is irrelevant. I’ll believe we’re closer when we have a computer solve a novel problem that is not a simple pattern match to a similar solution. When a computer can reply to “write me a JIT translator from ARMv5 to ARMv7M” with working code. That takes actual thought and we’re not even close.
- fragmede 3y agohttps://chat.openai.com/share/2ae7883d-3df2-40ab-9184-11ad4a4ee5a9 https://chat.openai.com/share/2ae7883d-3df2-40ab-9184-11ad4a... doesn't result in actually working code, but we're clearly closer than cavemen banging rocks together.
- dmitrygr 3y agoThat covers 0.001% of the work - the part that my polit-sci-major no-programming experience sister just explained to me given the prompt. Now do the hard parts :)
- fragmede 3y agoThe hard parts are just an implementation detail. Let's say a human took 15 minutes to draft all that up. It's about 800 words, and given a typing rate of 50 wpm, it would take that long to just type that in, so that seems reasonable. If that's only 0.001% of the work, it means it would take 3 years to finish the project. 3 years to write a JIT from ARMv5 to ARMv7M? I think your numbers might be off :) btw, your polit-sci-major no-programming experience sister seems pretty clever if she was able to understand all that. I ran it past a few cavemen I came across and they didn't quite understand the nuances surrounding runtime environment management.
- dmitrygr 3y ago> just an implementation detail Try it sometime :)
- fragmede 3y agoThe question wasn't if I could do a JIT compiler but if the LLM could. Which, there are limitations to the technology, but my opinion is it would go quite far at our current level of technology. Presumably you don't, bit you've shown no evidence that it can't. Whereas I've shown that it's able to decompose the problem and start dotproducting the problem.
- deleted 3y ago[deleted]