10 ms·
I'm tired of LLMs. Enough billions of dollars have been spent on LLMs that a reasonably good picture of what they can and can't do has emerged. They're really
by Animats 2y ago
I'm tired of LLMs.
Enough billions of dollars have been spent on LLMs that a reasonably good picture of what they can and can't do has emerged. They're really good at some things, terrible at others, and prone to doing something totally wrong some fraction of the time. That last limits their usefulness. They can't safely be in charge of anything important.
If someone doesn't soon figure out how to get a confidence metric out of an LLM, we're headed for another "AI Winter".
Although at a much higher level than last time. It will still be a billion dollar industry, but not a trillion dollar one.
At some point, the market for LLM-generated blithering should be saturated. Somebody has to read the stuff. Although you can task another system to summarize and rank it. How much of "AI" is generating content to be read by Google's search engine? This may be a bigger energy drain than Bitcoin mining.
- jajko 2y agoI'll keep buying (and paying premium) for dumber things. Cars are a prime example, I want it to be dumb as fuck, offline, letting me decide what to do. At least next 2 decades, and thats achievable. After that I couldnt care less, I'll probably be a bad driver at that point anyway so switch may make sense. I want dumb beautiful mechanival wristwatch. I am not ocd-riddled insecure man trying to subconsiously immitate much of the crowd, in any form of fasion. If that will make me an outlier, so be it, a happier one. I suspect new branch of artisanal human-mind-made trademark is just behind the corner, maybe niche but it will find its audience. Beautiful imperfections, clear clunky biases and all that.
- mattgreenrocks 2y agoWhat dumb cars are you looking at?
- spitfire 2y ago1963 Jaguar E-Type
- lobsterthief 2y agoI want a dumb car with airbags
- spencerchubb 2y agoLLMs have been improving exponentially for a few years. let's at least wait until exponential improvements slow down to make a judgement about their potential
- bloppe 2y agoThey have been improving a lot, but that improvement is already plateauing and all the fundamental problems have not disappeared. AI needs another architectural breakthrough to keep up the pace of advancement.
- Animats 2y agoYes. Anything on the horizon?
- bloppe 2y agoI'm not as up-to-speed on the literature as I used to be (it's gotten a lot harder to keep up), but I certainly haven't heard of any breakthroughs. They tend to be pretty hard to predict and plan for. I don't think we can continue simply tweaking the transformer architecture to achieve meaningful gains. We will need new architectures, hopefully ones that more closely align with biological intelligence. In theory, the simplest way to real superhuman AGI would be to start by modeling a real human brain as a physical system at the neural level; a real neural network. What the AI community calls "neural networks" are only very loose approximations of biological neural networks. Real neurons are subject to complex interactions between many different neurotransmitters and neuromodulators and they grow and shift in ways that look nothing like backpropagation. There already exist decently accurate physical models for single neurons, but accurately modeling even C. elegans (as part of the OpenWorm project) is still a way's off. Modeling a full human brain may not be possible within our lifetime, but I also wouldn't rule that out. And once we can accurately model a real human brain, we can speed it up and make it bigger and apply evolutionary processes to it much faster than natural evolution. To me, that's still the only plausible path to real AGI, and we're really not even close.
- __loam 2y agoFunnily enough, bitcoin mining still uses at least about 3x more power that AI at the moment, while providing less value imo. AI power use is also dwarfed by other industries even in computing. We should still consider whether it's worth it, but most research and development on LLMs in corporate right now seems to be focused on making them more efficient, and therefore both cheaper and less power intensive, to run. There's also stuff like Apple intelligence that is moving it out to edge devices with much more efficient chips. I'm still a big critic of AI generally but they're definitely not as bad as crypto which is shocking.
- illiac786 2y agoDo you have a nice reference for this? I could really use something like this, this topic comes up a lot in my social circle.
- __loam 2y agohttps://blog.giovanh.com/blog/2024/08/18/is-ai-eating-all-the-energy-part-1-of-2/ https://blog.giovanh.com/blog/2024/08/18/is-ai-eating-all-th...
- Ferret7446 2y agoHow do you measure the value of bitcoin, if not by its market cap? Do you interview everyone and ask them how much they're willing to pay for a service that allows them to transfer money digitally without institutional oversight/bureaucracy?
- __loam 2y agoThe amount of power being used to support a system that can do 5 transactions a second is disgusting.
- Ferret7446 2y agoAs opposed to what other system that can do a single transaction in any time frame without individual or organizational interdiction?
- wrycoder 2y agoJust wait until they get saturated with subtle (and not so subtle) advertising. Then, you'll really hate them.
- mhowland 2y ago"They're really good at some things, terrible at others, and prone to doing something totally wrong some fraction of the time." I agree 100% with this sentiment, but, it also is a decent description of individual humans. This is what processes and control systems/controls are for. These are evolving at a slower pace than the LLMs themselves at the moment so we're looking to the LLM to be its own control. I don't think it will be any better than the average human is at being their own control, but by no means does that mean it's not a solvable problem.
- linsomniac 2y ago>also is a decent description of individual humans A friend of mine was moving from software development into managing devs. He told me: "They often don't do things the way or to the quality I'd like, but 10 of them just get so much more done than I could on my own." This was him coming to terms with letting go of some control, and switching to "guiding the results" rather than direct control. The LLMs are a lot like this.
- theamk 2y agoYour friend got lucky, I've seen (and worked with) people with negative productivity - they make the effort and sometimes they commit code, but it inevitably ends up being broken, and I realize that it would take less of my time for me to write the code myself, rather than spend all the time explaining and then fixing bugs. The LLMS are a lot like this.
- latexr 2y ago> I agree 100% with this sentiment, but, it also is a decent description of individual humans. But you can understand individual humans and learn which are trustworthy for what. If I want a specific piece of information, I have people in my life that I know I can consult to get an answer that will most likely be correct and that person will be able to give me an accurate assessment of their certainty and they know how to accurately confirm their knowledge and they’ll let me know later if it turns out they were wrong or the information changed and… None of that is true with LLMs. I never know if I can trust the output, unless I’m already an expert on the subject. Which kind of defeats the purpose. Which isn’t to say they’re never helpful, but in my experience they waste my time more often than they save it, and at an environmental/energy cost I don’t personally find acceptable.
- latexr 2y ago> They're really good at some things, terrible at others, and prone to doing something totally wrong some fraction of the time. (…) They can't safely be in charge of anything important. Agreed. If everyone understood that and operated under that assumption, it wouldn’t be that much of an issue. Alas, these guessing machines are marketed as all-knowing oracles that can already solve half of humanity’s problems and a significant number of people treat them as being right every time, even in instances where they’re provably wrong.
- bschmidt1 2y agoI love how people are like "there's no use case" and there's already products on shelves. I see AI art everywhere, AI writing, customer support - already happened. You guys are naysaying something that already happened people already replaced jobs with LLMs and already profit due to AI. There are already startups with users where you provide a OPENAI_API_KEY, or customers where you provide theirs. If you can't see how this tech is useful Idk what to tell you, you have no imagination AND aren't looking around you at the products, marketing, etc. that already exists. These takes remind me of the luddites of ~2012 who were still doubting the Internet in general.
- anon7725 2y agoThere is a difference between “being useful” and living up to galactic-scale hype.
- bschmidt1 2y ago[flagged]
- anon7725 2y agoYou’re certainly able to sus out everything about my worldview, knowledge level and intentions from my one-sentence comment. The only thing that LLMs are at risk of subverting is the livelihood of millions of people. AI is a capital intensifier, so the rich will get richer as it sees more uptake. About copyright - yeah, I’m quite concerned for my friends who are writers and artists. > You'll get left behind with these takes, it's not smart. If you don't care about advancing technology or society then have fun being a luddite, but you're on the wrong side of history. FWIW I work in applied research on LLMs.
- bschmidt3 2y ago[flagged]
- lukev 2y ago
- seandoe 2y agoTotally agree on the confidence metric. The way chatbots spew complete falsities in such a confident tone is really disheartening. I want to use AI more but I don't feel I can trust it at all. If I can't trust it and have to search for other resources to verify it's claims, the value is really diminished.
- naming_the_user 2y agoIs it even possible in principle for an LLM to produce a confidence interval given that in a lot of cases the input is essentially untrusted? What comes to mind is - I consider myself an intelligent being capable of recognising my limits - but if you put my brain in a vat and taught me a new field of science, I could quite easily make claims about it that were completely incorrect if your teaching was incorrect because I have no actual real world experience to match it up to.
- theamk 2y agoRight, and that's why "years of experience" matters in humans. You will be giving incorrect answers, but as long as you get feedback, you will improve, or at least calibrate your confidence meter. This is not the case with current model - they are forever stuck at junior level, and they won't improve no matter how much you correct them. I know humans like that too. I don't ask them questions that I need good answers too.
- datahack 2y agoIt’s probably generally irrelevant what they can do today, or what you’ve seen so far. This is conceptually essentially Moore’s law, but about every 5.5 months. That’s the only thing that matters at this stage. I watched everyone make the same arguments about the general Internet, and then the Web, then mobile, then Bitcoin. It’s just a toy. It’s not that useful. Is this supposed to be the revolution? It uses too much power. It won’t scale. The technology is a dead end. The general pattern of improvement to technology has been radically to the upside at an increasing pace parabolically for decades and there’s nothing indicating that this is a break in the pattern. In fact it’s setting up to be an order of magnitude greater impact than the Internet was. At a minimum, I don’t expect it to be smaller. Looking at early telegraphs doesn’t predict the iPhone, etc. Optimism is warranted here until it isn’t.
- jiggawatts 2y agoSpeaking of the iPhone, I just ugpraded to the 16 Pro because I want to try out the new Apple Intelligence features. As soon as I saw integrated voice+text LLM demos, my first thought was that this was precisely the technology needed to make assistants like Siri not total garbage. Sure, Apple's version 1.0 will have a lot of rough edges, but they'll be smoothed out. In a few versions it'll be like something out of Star Trek. "Computer, schedule an appointment with my Doctor. No, not that one, the other one... yeah... for the foot thing. Any time tomorrow. Oh thanks, I forgot about that, make that for 2pm." Try that with Siri now. In a few years, this will be how you talk to your phone. Or... maybe next month. We're about to find out.
- al_borland 2y agoThe issue with appointments is the provider needs to be integrated into the system. Apple can’t do that on their own. It would have to be more like the roll out of CarPlay. A couple partners at launch, a lot of nothing for several years, and eventually is a lot of places, but still not universal. I could see something like Uber or Uber Eats trying to be early on something like this, since they already standardized the ordering for all the restaurants in their app. Scheduling systems are all over the place.
- rldjbpin 2y agoLLMs are to AI what BTC is to blockchain, let me explain. blockchain and no-trust decentralization has so much promise, but grifters all go for what got done first and can be squeezed money out of. same is happening with LLMs, as a lot of current AI work started with text first. they might still lowkey be necessary evils because without them there would not have been so much money or attention flowing in this way.
- agubelu 2y ago> blockchain and no-trust decentralization has so much promise I've been hearing this for the past 5 years, yet nothing of practical use based on blockchains has materialized yet.
- Jommi 2y agoyou dont think an open finance network that's accessible for anyone with an internet connection is useful? your westernness is showing go ask SA or Africa how useful it is that they arent restricted by insane dictatorial capital controls anymore
- throw45678943 2y agoIndeed. Decentralised currency is at least a technology that can power the individual at times, rather than say governments, big corps, etc especially in certain countries. Yes it didn't change as much as was marketed but I don't see that as a bad thing. Its still a "tool" that people can use, in some cases to enable use cases they couldn't do or didn't have the freedom to do before. AI, given its requirements for large computation and money, and its ability to make easily available intelligence to certain groups, IMO has a real potential to do the exact opposite - take away power from individuals especially if they are middle class or below. In the wrong hands it can definitely destroy openness and freedom. Even if it is "Open" AI, for most of society their ability to offer labor and intelligence/brain power is the only thing they can offer to gain wealth and sustenance - making it a commodity tilts the power scales. If it changes even a slice of what it is marketed at; there are real risks for current society. Even if it increases production of certain goods, it won't increase production of the goods the ultra wealthy tend to hold (physical capital, land, etc) making them as a proportion even more wealthy. This is especially true if AI doesn't end up working in the physical realm quick enough. The benefits seem more like novelties to most individuals that they could do without where to large corps and ultra wealthy individuals the the benefits IMO are much more obvious with AI (e.g. we finally don't need workers). Surveillance, control, persuasion, propaganda, mass uselessness of most of the population, medical advances for the ultra wealthy, weapons, etc can now be done at almost infinite scale and with great detail. If it ever gets to the point of obsoleting human intelligence would be a very interesting adjustment period for humanity. The flaw isn't the technology; its the likely use of it by humans and their nature. Not saying LLMs are there yet or even if they are the architecture to do this, but agentic behaviour and running corporations (as OpenAI makes its goal on their presentation slides to be) seem to be a way to rid many of the need for other people in general (to help produce, manage, invent and control). That could be a good or bad thing, depending on how we manage it but one thing it wouldn't be would be simple.