4 ms·
AI is such a huge expectations dichotomy. For those of us used to the continual disappointment that pre-LLM AI was, the current crop of LLM's are amazing, mind
by bryanlarsen 3y ago
AI is such a huge expectations dichotomy. For those of us used to the continual disappointment that pre-LLM AI was, the current crop of LLM's are amazing, mind blowing things. We start raving about them, so other people take a look expecting that modern LLM's are the greatest thing since sliced bread. They're not quite that, so we get HN comments complaining that AI sucks.
- fennecfoxy 3y agoEh I think people's expectations for things they don't understand are always overblown. One thing I always find funny is the general expectation that machine learning models are both incredibly generalised and designed based on the way biological systems work, but should also be 100% perfect and never be wrong just like a machine and NOT like a biological system, those things are mutually exclusive; even the best, smartest most physically capable humans will still sometimes spill their coffee, yet we expect coffee-bot 2024 not to do this. Certainly machines can be much better at a task than humans are, but if that tasks requires generalisation then it's still gonna fuck up from time to time. Machine learning programs may think a dog is actually a cat sometimes, but afaik they ain't ever called their teacher "Mum" yet.
- _5ygc 3y agoI think the problem is that they aren’t mind-blowing, but an improvement. I resent some LLM implementations on principle, but decided to give these code helpers a try. What I found was they’re reasonably bad, and I kept telling them the solution doesn’t work, only to be presented with a little tweak. So I don’t see the point of outsourcing my thinking, I’d rather remain intelligent and do the search/try/tweak on my own, instead of pretending a half-assed LLM is genius. That doesn’t mean they don’t have good use cases, or aren’t an improvement on previous tech. But we definitely should stop calling them mind-blowing. Jaron Lanier had long ago predicted we’d willingly downplay human intelligence to pretend AI was… I.
- ryanklee 3y ago> I think the problem is that they aren’t mind-blowing, but an improvement. You seem to be doing what GP is pointing out. GP's claim is that the relative improvement itself is mind-blowing, not that the tech is mind-blowing in an absolute sense. I tend to agree: much of the detraction hangs on current-state rather than a probable potential-state informed by recent relative advancements. In other words, many proponents are chuffed because of that potential; not because actuality. Likewise, skeptics are reserved because of the actuality, and not because of potential, for whatever reason (there are at least a few main ones, I think).
- mandevil 3y agoSee, I'm very skeptical that we can guess at the potential, because so much of it depends on research and figuring out better ways to do things going forward, and successful research programs are very unpredictable. I feel like people are taking Moore's Law, which is definitely a real thing, and thinking that everything else is going to advance like semiconductors did, and I just am not seeing it in any other field. It's not true in software development (where gains, such as they are, are more linear than exponential) its definitely not true in rockets or civil engineering or steel or anything like that. So I am afraid a whole lot of people are expecting Moore's Law type improvements in AI, when really AI advances more like punctuated equilibrium: a sudden dramatic improvement, then a long period of consolidation and stasis, then another sudden dramatic improvement, often in a totally different unpredictable area. But I've just been keeping tabs on AI since the hot way to do it was Expert Systems back in the 1990's, and I'm aware of its history since Norbert Weiner wrote Cybernetics back in 1947, and this seems to be a repeating pattern: a single major breakthrough (in this case, honestly, the combination of large quantities of data with NN's- with driving and natural language being the two easiest to get, and so the most prominent examples) followed by a lengthy fallow period where not much appreciable progress happens, then another breakthrough, often orthogonal to where earlier breakthroughs happened.
- finnthehuman 3y ago> We start raving about them, so other people take a look expecting… If that’s what you want to call it. I see hypemen overpromising and product underdelivering. And when pressed about specifics, attempts to drown queries in jargon or an ass-covering retreat to treating it like it’s just a tech demo not intended to be used for anything ever.
- lukeschlather 3y agoIt looks like it's capable of anything, so it's easy to point out that it's pretty bad at certain things, but that's an easy way to miss the things it's actually very good at. Also even if it's bad at most things it's still often good enough to be useful. And it seems likely that with an order of magnitude better hardware it might be good at some things that it seems really bad at now. So yes, it's a tech demo for a lot of things that aren't quite ready, and it's also very useful as it is.
- cm2187 3y agoAnd management consultants solving every problem in the world by adding a “AI” box in their powerpoint flow diagram.
- chasd00 3y ago> And management consultants solving every problem in the world by adding a “AI” box in their powerpoint flow diagram. i work at a large consulting firm everyone knows and am seeing this first hand. I'm not on the bandwagon until i see real money in the bank from large AI projects succeeding. It's not happening yet. I not a naysayer but am still very skeptical.
- antisthenes 3y agoYep, just like "crypto" before that, and "big data" before that and "social media presence" before that and so on and so forth...
- bryanlarsen 3y ago
- baz00 3y agoThey do suck unconditionally. They're like having a mediocre person working with you that you have to verify and validate all the time that sucks up a ton of money and electricity and generally does bad things to the environment. Edit: I haven't even started on the social and political impact this has either.
- IshKebab 3y agoYou missed the point. And you're also completely wrong. Copilot is extremely useful and in general a huge timesaver. Yes if you tell it to write an entire program it will get it wrong and you'll spend some time verifying things. But that's not a sane way to use it. As a very clever auto-complete it's fantastic. It's also pretty great at getting past "blank page syndrome". Even if what it spits out is wrong it's still helpful to get you started.
- baz00 3y agoNo I didn't miss the point, not even slightly. You didn't read mine and are blinded by the insane virtues. Do you really think it's fine blowing 400 watts because you can't be arsed to think or do not have the creative intelligence to get over the blank page syndrome and have to lean on a crutch?
- hazard 3y ago> Do you really think it's fine blowing 400 watts because you can't be arsed to think or do not have the creative intelligence to get over the blank page syndrome and have to lean on a crutch? Yes, I think it is absolutely 100% fine.
- Leherenn 3y agoI don't know if I'm doing something wrong, and admittedly I have only tried the free version of ChatGPT (3.5), but it basically never works for me. Not even for relatively simple things. From my past history: "Is "192.168.1.4" included in the subnet "192.168.0.0/16"?" -> No "Check whether a widget overflows in flutter" -> returns a function that cannot be made to works even with a lot of massaging (uses stuff that does not exist) "Write a parser for this multiline format in C++ (describe format)" -> parser only read first line Admittedly a trick one: "Can you give me a C++ function to merge 2 uint32_t and one uint16_t into a unique uint64_t?" -> happily gives an answer Sometimes it is salvageable, and sometimes it can provide ways I did not consider to solve a problem (though the proposed solution is usually broken), but usually I would have been faster to do it myself than to try to fix whatever it gives me. I have basically given up on it, except for some generic "how would you solve problem X?", and when I see people talking about it, it feels like a totally different world.
- YeGoblynQueenne 3y ago>> For those of us used to the continual disappointment that pre-LLM AI was, the current crop of LLM's are amazing, mind blowing things. Aren't you overgeneralising a bit? Not even I would say that CNNs for image classification, or Deep-RL for board game-playing are a "continual disappointment" and they certainly predate LLMs. Are you talking about NLP? Even Neural Turing Machines were quite capable in language pairs with large parallel corpora (and similar linguistic structure). Basically, what do you mean by "continual disappointment"? What I'm aware of is an incessant hype crescendo that crashing over everything like a relentless wave.
- nvm0n2 3y agoPre-2022 most AI was disappointing not because it was unimpressive but because it near exclusively took the form of blog posts or university papers that announced something interesting but unusable. Either because it was merely an academic curiosity (AlphaGo) or because the tech wasn't being released for public usage at all, and wasn't really easy to replicate externally either, or because there was no obvious way to apply it to normal problems. So we all got used to this constant year-in-year-out stream of "amazing" "breakthroughs" that ended up being a thats-cool followed by a shrug. RLHF trained GPT3 and then DALLE/Midjourney/Stable Diffusion changed all that. Suddenly AI not only got good, but the field broke loose of the inane and insane obsession with pseudo-safety that had been holding it back. Now the rest of us can use it without dropping $5M on a GPU cluster and hiring a dozen researchers first. AI is no longer a disappointment.
- bdavbdav 3y agoI’m not sure about this pre-LLM AI being disappointing take. What are we considering as AI? AI encompasses all manner of fields. The face detection in my iPhone, nest doorbell works fantastically. The DSG in my car learns how I shift, fraud detection in finance is heavily driven by ML and typically very effective. It may not be helpful to you, but product recommendations and targeted advertising are ML driven and incredibly effective. They may not be exciting uses, but they’re incredibly helpful.