4 ms·
Many big names in the industry have long advocated for the idea that LLM-s are a fundamental dead end. Many have also gone on and started companies to look for
by Etheryte 1y ago
Many big names in the industry have long advocated for the idea that LLM-s are a fundamental dead end. Many have also gone on and started companies to look for a new way forward. However, if you're hip deep in stock options, along with your reputation, you'll hardly want to break the mirage. So here we are.
- fallingfrog 1y agoI have some idea of what the way forward is going to look like but I don't want to accelerate the development of such a dangerous technology so I haven't told anyone about it. The people working on AI are very smart and they will solve the associated challenges soon enough. The problem of how to slow down the development of these technologies- a political problem- is much more pressing right now.
- fallingfrog 1y ago[flagged]
- ACCount37 1y agoYou're being downvoted for displaying the kind of overconfidence that people consider shameful. Everyone in ML has seen dozens to thousands of instances of "I have a radical new idea that will result in a total AI breakthrough" already. Ever wondered why the real breakthroughs are so few and far in between?
- fallingfrog 1y agoI know how it looks. I shouldn't have said anything- it was pointless. At any rate someone else will figure out what i know soon or already has.
- philipwhiuk 1y agoNostradamus 2.0
- fallingfrog 11mo agoNah im just the kind of autistic that is good at solving specific kinds of problems that can be worked out through visualizing data structures
- WhammyBammy 1y ago[dead]
- thunky 1y agoIf you're so confident that your discovery will be found by someone else soon why not just share it right here and get the credit you seem to want? Otherwise why even bring it up in the first place?
- fallingfrog 11mo agoNo reason to risk making the humanity killing doomsday machine appear even a minute sooner, and besides, you wouldn't listen. I said something because im a fool who cant keep her mouth shut. It was reasons of emotional catharsis having no rational basis.
- deleted 1y ago[deleted]
- chriswarbo 1y ago> I have some idea of what the way forward is going to look like but I don't want to accelerate the development of such a dangerous technology so I haven't told anyone about it. Ever since "AI" was named at Dartmouth, there have been very smart people thinking that their idea will be the thing which makes it work this time. Usually, those ideas work really well in-the-small (ELIZA, SHRDLU, Automated Mathematician, etc.), but don't scale to useful problem sizes. So, unless you've built a full-scale implementation of your ideas, I wouldn't put too much faith in them if I were you.
- ACCount37 1y agoFar more common are ideas that don't work on any scale at all. If you have something that gives a sticky +5% at 250M scale, you might have an actual winner. Almost all new ML ideas fall well short of that.
- fallingfrog 1y agoIf someone else comes along and makes the exact claim I just made, I won't believe it either
- ACCount37 1y agoDid you try any of your shit at any scale at all?
- fallingfrog 1y ago999999 times out of a million you'd be right. But, I shouldn't have said anything.
- ajjahs 1y ago[dead]
- titzer 1y agoLet me guess, you have a truly marvelous demonstration of this proposition which this margin is too narrow to contain.
- fallingfrog 1y agoWell it kind of sucks honestly because im never going to get any sort of recognition, but that's partly by choice and partly because I don't have the right personal connections. I 100% understand why all of you think I'm a significant fool. Thats ok. Like i said, im burying the idea. But its obvious enough that someone else will discover it soon enough.
- titzer 1y agoYou literally see no irony here? You are doubling, tripling down on some idea you won't even vaguely describe to people, but you are so utterly convinced that it's right you don't appear to even admit the possibility that your idle thoughts could be wrong, or just silly. Frankly it sounds delusional and you should question yourself more often. If you're so afraid of being right that you won't even write the idea down to have another person look at it for fear it will somehow set the world on fire because of its rightness, then you should seriously consider talking to a therapist. If that comes across as an ad hominem, it isn't, and it isn't meant as an insult or a dismissal of your idea--you've actually offered no idea. Frankly, you should be a little embarrassed as you've done nothing but grouse about how smart you are, and people react negatively to that. Puffing gets us nowhere except drawing attention to yourself and gave us no new ideas or thoughts to discuss, except your ego. I'd recommend you question yourself more and talk to some people you trust about whatever it is you think you discovered.
- fallingfrog 11mo agoUgh why do you care so much? Leave me alone. The big insane statement i made was "i have some idea of what the next bit of advancement will look like". Transformers are a special case of a much more beautiful general principle ok? Go away
- 11mo ago
- yahoozoo 1y agocringe
- fallingfrog 11mo agoReject subtlety Embrace cringe
- wild_egg 1y agoThey're a dead end for whatever their definition of "AGI" is, but still incredibly useful in many areas and not a "dead end" economically.
- bigbuppo 1y agoWell, except for that needing a 40 year bond for a 3 year technology cycle thing.
- hoherd 1y ago"It is difficult to get a man to understand something when his salary depends upon his not understanding it" and "never argue with a man whose job depends on not being convinced" in full effect.
- red75prime 1y ago> Many big names in the industry have long advocated for the idea that LLM-s are a fundamental dead end. There should be papers on fundamental limitations of LLMs then. Any pointers? "A single forward LLM pass has TC0 circuit complexity" isn't exactly it. Modern LLMs use CoT. Anything that uses Gödel's incompleteness theorems proves too much (We don't know whether the brain is capable of hypercomputations. And, most likely, it isn't capable of that).
- jimbokun 1y ago"Fundamental dead end" strikes me as hyperbolic. Clearly they could be an import part of an "AGI" system, even if they're not sufficient for building an AGI in and of themselves?
- tim333 1y agoI figure it's more like steam engines and flight. While steam engines were not suitable for aircraft, experience building them could carry over to internal combustion engines. I imagine something like that with LLMs and AGI.
- jamesoofou 1y agoHow can u look at progress in LLMs and think "this is mirage"
- NamlchakKhandro 1y agoBecause the depth of concept is so deep, and if you're just beginning then it's going to take a while for that statement to illuminate how obvious it is. For those who've been sniffing this since early 2010, it's so blindly obvious they've already dropped llms on the floor and moved onto deeper alternative research. For the rest of us, we're still catching coke bottles from the sky and building places of worship around them
- rsbrown 11mo agoName some? Because YeCun's break is the first big name I've seen strike out in a major fashion away from the LLM trajectory.