16 ms·
What happens when people don't understand how AI works
- layer8 1y agohttps://archive.is/NJ9K0 https://archive.is/NJ9K0
- jdkee 1y agoSomeone said, "The AI you use today is the worst AI that you will ever use."
- threeseed 1y agoI can't tell any difference between Claude 3.5, 3.7 and 4 for coding. So today is the same AI I used last year. And based on current trajectory same I will use next year.
- sroussey 1y agoI can tell the difference between those versions of Claude quite easily. Not 10x better each version, but each is more capable and the errors are fewer.
- assimpleaspossi 1y agoBut errors nonetheless.
- plemer 1y agoYou won’t find perfection anywhere
- th0ma5 1y agoThe imperfections of people are less imperfect than the machines.
- 8note 1y agoerrors are all over the place no matter what. the question is how predictable they are, and if they can be spotted as they show up. the best bugs are the ones that arent found for 5 years
- smcleod 1y agoThere is certainly a difference, however Anthropic did really well with 3.5 - far, far better than any other provider could do, so the steps from there been more incremental while other providers have been playing catch up (for example Google's Gemini 2.5 Pro is really their first model that's actually useful for coding in any way).
- user568439 1y agoSomeone also said the current AI is the less biased. Future AI's will be more powerful but probably influenced to push users to spend money or have a political opinion. So they may enshitify...
- Pingk 1y agoGiven that no models are profitable for the parent company afaik, it's only a matter of time before the money-squeezing begins
- stirfish 1y ago<think>I will reply with an example of bias in large language models. This comment seems unrelated to farmers in South Africa. </think> Ultimately these machines work for the people who paid for them.
- add-sub-mul-div 1y agoThat doesn't take into account how it will be used against its customers once they're dependent on it. As any other service inevitably will be. It's like if we'd said the Youtube we used in 2015 was going to be the worst Youtube we'd ever use.
- andy99 1y agoIs the internet today better than it's ever been? I can easily imagine LLMs becoming ad-riden, propagandized, cloudlfare intermediated, enshittified crap the way most of the internet has.
- rhubarbtree 1y agoI was going to come back here and say "what about transatlantic flight, which has gotten slower?" --- but your point is more relevant, more immediate, more impactful. I do believe the LLMs we're using today are the best they're going to be - for the reasons you've highlighted. Some superior tech might displace them, but LLMs as they are seem much more likely to get worse. I'm encouraging people to make any important queries right now and save the results. For example "which books should i read on X" - right now you get good answers, in the future it'll be enshittified.
- sumeno 1y agoSomeone could have said "The Google you use today is the worst Google you will ever use" 15 years ago and it may have sounded wise at the time.
- rhubarbtree 1y agoIf there is one cliche that I’d like to outlaw, it is this one. It’s a vacuous tautology and yet I hear so many people parrot it all the time. There are many reasons to believe LLMs in particular are not going anywhere fast. We need major breakthroughs now, and “chain of thought” is not one.
- deleted 1y ago[deleted]
- mettamage 1y ago> Few phenomena demonstrate the perils that can accompany AI illiteracy as well as “Chatgpt induced psychosis,” the subject of a recent Rolling Stone article about the growing number of people who think their LLM is a sapient spiritual guide. Some users have come to believe that the chatbot they’re interacting with is a god—“ChatGPT Jesus,” as a man whose wife fell prey to LLM-inspired delusions put it—while others are convinced, with the encouragement of their AI, that they themselves are metaphysical sages in touch with the deep structure of life and the cosmos. A teacher quoted anonymously in the article said that ChatGPT began calling her partner “spiral starchild” and “river walker” in interactions that moved him to tears. “He started telling me he made his AI self-aware,” she said, “and that it was teaching him how to talk to God, or sometimes that the bot was God—and then that he himself was God.” This sounds insane to me. When we are talking about safe AI use, I wonder if things like this are talked about. The more technological advancement goes on, the smarter we need to be in order to use it - it seems.
- polotics 1y agoPsychosis will find anything as a seed idea to express itself, even as basic a pattern as someone walking in lockstep with the soon-to-be patient can trigger a break. So it's not surprising that LLM chats would do the same.
- daveguy 1y agoYeah, but should we have psychosis force multipliers?
- kelseyfrog 1y agoLLMs are divinatory instruments, our era's oracle, minus the incense and theatrics. If we were honest, we'd admit that "artificial intelligence" is just a modern gloss on a very old instinct: to consult a higher-order text generator and search for wisdom in the obscure. They tick all the boxes: oblique meaning, a semiotic field, the illusion of hidden knowledge, and a ritual interface. The only reason we don't call it divination is that it's skinned in dark mode UX instead of stars and moons. Barthes reminds us that all meaning is in the eye of the reader; words have no essence, only interpretation. When we forget that, we get nonsense like "the chatbot told him he was the messiah," as though language could be blamed for the projection. What we're seeing isn't new, just unfamiliar. We used to read bones and cards. Now we read tokens. They look like language, so we treat them like arguments. But they're just as oracular, complex, probabilistic signals we transmute into insight. We've unleashed a new form of divination on a culture that doesn't know it's practicing one. That's why everything feels uncanny. And it's only going to get stranger, until we learn to name the thing we're actually doing. Which is a shame, because once we name it, once we see it for what it is, it won't be half as fun.
- Arainach 1y ago>When we forget that, we get nonsense like "the chatbot told him he was the messiah," as though language could be blamed for the projection. Words have power, and those that create words - or create machines that create words - have responsibility and liability. It is not enough to say "the reader is responsible for meaning and their actions". When people or planet-burning random matrix multipliers say things and influence the thoughts and behaviors of others there is blame and there should be liability. Those who spread lies that caused people to storm the Capitol on January 6th believing an election to be stolen are absolutely partially responsible even if they themselves did not go to DC on that day. Those who train machines that spit out lies which have driven people to racism and genocide in the past are responsible for the consequences.
- kelseyfrog 1y ago"Words have no essential meaning" and "speech carries responsibility" aren't contradictory. They're two ends of the same bridge. Meaning is always projected by the reader, but that doesn't exempt the speaker from shaping the terrain of projection. Acknowledging the interpretive nature of language doesn't absolve us from the consequences of what we say. It just means that communication is always a gamble: we load the dice with intention and hope they land amid the chaos of another mind. This applies whether the text comes from a person or a model. The key difference is that humans write with a theory of mind. They guess what might land, what might be misread, what might resonate. LLMs don’t guess; they sample. But the meaning still arrives the same way: through the reader, reconstructing significance from dead words. So no, pointing out that people read meaning into LLM outputs doesn’t let humans off the hook for their own words. It just reminds us that all language is a collaborative illusion, intent on one end, interpretation on the other, and a vast gap where only words exist in between.
- spwa4 1y agoWhat really happens: "for some reason" higher up management thinks AI will let idiots run extremely complex companies. It doesn't. What AI actually does is like any other improved tool: it's a force multiplier. It allows a small number of highly experienced, very smart people, do double or triple the work they can do now. In other words: for idiot management, AI does nothing (EXCEPT enable the competition) Of course, this results in what you now see: layoffs where as always idiots survive the layoffs, followed by the products of those companies starting to suck more and more because they laid off the people that actually understood how things worked and AI cannot make up for that. Not even close. AI is a mortal threat to the current crop of big companies. The bigger the company, the bigger a threat it is. The skill high level managers tend to have is to "conquer" existing companies, and nothing else. With some exceptions, they don't have any skill outside of management, and so you have the eternally repeated management song: that companies can be run by professional managers, without knowing the underlying problem/business, "using numbers" and spreadsheet (except when you know a few and press them, of course it turns out they don't have a clue about the numbers, can't come up with basic spreadsheet formulas) TLDR: AI DOESN'T let financial-expert management run an airplane company. AI lets 1000 engineers build 1000 planes without such management. AI lets a company like what Google was 15-20 years ago wipe the floor with a big airplane manufacturer. So expect big management to come with ever more ever bigger reasons why AI can't be allowed to do X.
- jongjong 1y agoResonates. I've been thinking about a tech technology bubble (not financial bubble) for years now. Big tech companies have just been throwing engineers at problems for many years and it feels like they completely stopped caring about talent now. Not that they ever really cared deeply, but they did care superficially and that was enough to keep the machines spinning. Now that they have AI, I can see it become an 'idiocy multiplier'. Already software is starting to break in subtle ways, it's slow, laggy, security processes have become a nightmare.
- thetjyt468555 1y agoPerhaps, the higher-ups are so detached from details that they see a comrade in the LLM bullshit-artist.
- imiric 1y agoThis is a good summary of why the language we use to describe these tools matters[1]. It's important that the general public understands their capabilities, even if they don't grasp how they work on a technical level. This is an essential part of making them safe to use, which no disclaimer or PR puff piece about how deeply your company cares about safety will ever do. But, of course, marketing them as "AI" that's capable of "reasoning", and showcasing how good they are at fabricated benchmarks, builds hype, which directly impacts valuations. Pattern recognition and data generation systems aren't nearly as sexy. [1]: https://news.ycombinator.com/item?id=44203562#44218251 https://news.ycombinator.com/item?id=44203562#44218251
- dwaltrip 1y agoPeople are paying hundreds of dollars a month for these tools, often out of their personal pocket. That's a pretty robust indicator that something interesting is going on.
- mmcconnell1618 1y agoOne thing these models are extremely good at is reading large amounts of text quickly and summarizing important points. That capability alone may be enough to pay $20 a month for many people.
- david-gpu 1y agoNot just summarizing, but also being able to answer follow-up questions about what is in the text. And, like Wikipedia, they can be useful to find your bearing in a subject that you know nothing about. Unlike Wikipedia, you can ask it free-form questions and have it review your understanding.
- th0ma5 1y agoI keep hearing anecdotes but the data, like a widely covered BBC study, say they only compress and shorten and routinely fail outside of testing on real world selection of only the most important content or topics.
- andy99 1y agoI agree with the substance, but would argue the author fails to "understand how AI works" in an important way: LLMs are impressive probability gadgets that have been fed nearly the entire internet, and produce writing not by thinking but by making statistically informed guesses about which lexical item is likely to follow another Modern chat-tuned LLMs are not simply statistical models trained on web scale datasets. They are essentially fuzzy stores of (primarily third world) labeling effort. The response patterns they give are painstakingly and at massive scale tuned into them by data labelers. The emotional skill mentioned in the article is outsourced employees writing or giving feedback on emotional responses. So you're not so much talking to statistical model as having a conversation with a Kenyan data labeler, fuzzily adapted through a transformer model to match the topic you've brought up. While thw distinction doesn't change the substance of the article, it's valuable context and it's important to dispel the idea that training on the internet does this. Such training gives you GPT2. GPT4.5 is efficiently stored low- cost labor.
- MrZander 1y agoThis doesn't follow with my understanding of transformers at all. I'm not aware of any human labeling in the training. What would labeling even do for an LLM? (Not including multimodal) The whole point of attention is that it uses existing text to determine when tokens are related to other tokens, no?
- daveguy 1y agoThe transformers are accurately described in the article. The confusion comes in the Reinforcement Learning Human Feedback (RLHF) process after a transformer based system is trained. These are algorithms on top of the basic model that make additional discriminations of the next word (or phrase) to follow based on human feedback. It's really just a layer that makes these models sound "better" to humans. And it's a great way to muddy the hype response and make humans get warm fuzzies about the response of the LLM.
- MrZander 1y agoOh, interesting, TIL. Didn't realize there was a second step to training these models.
- tracerbulletx 1y agoImagine thinking that brains aren't making statistically informed guesses about sequential information.
- dwaltrip 1y agoEveryone, it's just "statistics". Numbers can't hurt you. Don't worry
- electroglyph 1y agohttps://thebullshitmachines.com/ https://thebullshitmachines.com/
- jaza 1y agoThanks for this, it should be added to school curricula worldwide!
- Lendal 1y agoI'm in the camp of people who believe that while they are bullshit machines, oftentimes bullshit can be extremely useful. I use these bullshit machines all the time now, for minor tasks where bullshit is acceptable output. Whether it's a chunk of low-quality throwaway code that's easily verified and/or corrected, or an answer to a question where close-enough is good enough. Not everything needs to pass a NASA quality inspection to be useful.
- roxolotl 1y agoThe thesis is spot on with why I believe many skeptics remain skeptics: > To call AI a con isn’t to say that the technology is not remarkable, that it has no use, or that it will not transform the world (perhaps for the better) in the right hands. It is to say that AI is not what its developers are selling it as: a new class of thinking—and, soon, feeling—machines. Of course some are skeptical these tools are useful at all. Others still don’t want to use them for moral reasons. But I’m inclined to believe the majority of the conversation is people talking past each other. The skeptics are skeptical of the way LLMs are being presented as AI. The non hype promoters find them really useful. Both can be correct. The tools are useful and the con is dangerous.
- th0ma5 1y agoA lot of the claims of usefulness evaporate when tested. The word useful has many meanings. Perhaps their only reliable use will be the rubber duck effect.
- squidbeak 1y agoNot for me they haven't.
- th0ma5 1y agoAnecdotes unfortunately are not data :/
- diggan 1y ago> A lot of the claims of usefulness evaporate when tested In your personal experience? Because that's been my personal experience too, in lots of cases with LLMs. But I've also been surprised the other way, and overall it's been a net-positive for myself, but I've also spent a lot of time "practicing" getting prompts and tooling right. I could easily see how people give it try for 20-30 minutes, not getting the results they expected and give up, which yeah, you probably won't get any net-positive effects by that.
- 1y ago
- 1vuio0pswjnm7 1y ago"Witness, too, how seamlessly Mark Zuckerberg went from selling the idea that Facebook would lead to a flourishing of human friendship to, now, selling the notion that Meta will provide you with AI friends to replace the human pals you have lost in our alienated social-media age." Perhaps "AI" can replace people like Mark Zuckerberg. If BS can be fully automated.
- ares623 1y agoIt can’t. To become someone like Mark, you must have absolute zero empathy. LLMs have a little empathy in them due to their training data.
- gitaarik 1y agoNo they have apparent empathy because of the reward models humans trained them with. To be Mark you have to experience real existential fear, and a need to control other people to compensate the fear. And LLMs can't do that indeed. But they might be able to simulate it at some point.
- lordnacho 1y agoThe article skirts around a central question: what defines humans? Specifically, intelligence and emotions? The entire article is saying "it looks kinds like a human in some ways, but people are being fooled!" You can't really say that without at least attempting the admittedly very deep question of what an authentic human is. To me, it's intelligent because I can't distinguish its output from a person's output, for much of the time. It's not a human, because I've compartmentalized ChatGPT into its own box and I'm actively disbelieving. The weak form is to say I don't think my ChatGPT messages are being sent to the 3rd world and answered by a human, though I don't think anyone was claiming that. But it is also abundantly clear to me that if you stripped away the labels, it acts like a person acts a lot of the time. Say you were to go back just a few years, maybe to covid. Let's say OpenAI travels back with me in a time machine, and makes an obscure web chat service where I can write to it. Back in covid times, I didn't think AI could really do anything outside of a lab, so I would not suspect I was talking to a computer. I would think I was talking to a person. That person would be very knowledgeable and able to answer a lot of questions. What could I possibly ask it that would give away that it wasn't real person? Lots of people can't answer simple questions, so there isn't really a way to ask it something specific that would work. I've had perhaps one interaction with AI that would make it obvious, in thousands of messages. (On that occasion, Claude started speaking Chinese with me, super weird.) Another thing that I hear from time to time is an argument along the line of "it just predicts the next word, it doesn't actually understand it". Rather than an argument against AI being intelligent, isn't this also telling us what "understanding" is? Before we all had computers, how did people judge whether another person understood something? Well, they would ask the person something and the person would respond. One word at a time. If the words were satisfactory, the interviewer would conclude that you understood the topic and call you Doctor.
- EMM_386 1y ago> These statements betray a conceptual error: Large language models do not, cannot, and will not "understand" anything at all. They are not emotionally intelligent or smart in any meaningful or recognizably human sense of the word. This is terrible write-up, simply because it's the "Reddit Expert" phenomena but in print. They "understand" things. It depends on how your defining that. It doesn't have to be in its training data! Whoah. In the last chat I had with Claude, it naturally just arose that surrender flag emojis, the more there were, was how funny I thought the joke was. If there were plus symbol emojis on the end, those were score multipliers. How many times did I have to "teach" it that? Zero. How many other times has it seen that during training? I'll have to go with "zero" but that could be higher, that's my best guess since I made it up, in that context. So, does that Claude instance "understand"? I'd say it does. It knows that 5 surrender flags and a plus sign is better than 4 with no plus sign. Is it absurd? Yes .. but funny. As it figured it out on its own. "Understanding". ------ Four flags = "Okay, this is getting too funny, I need a break" Six flags = "THIS IS COMEDY NUCLEAR WARFARE, I AM BEING DESTROYED BY JOKES"
- fer 1y ago> More means more You could have used "loool" vs "loooooool", "xDD" vs "xDDDDDDDDD", using flags doesn't change a whole lot.
- EMM_386 1y agoDid I say it did!? These are the type of responses that REALLY will drive me nuts. I never said the flag emojis were special. I've been a software engineers for almost 30 years. I know what Unicode code pages are. This is not helpful. How is my example missing your definition of understanding? Replace the flags with yours if it helps ... same thing. It's not the flags it's the understanding of what they are. They can be pirate ships or cats. In my example they are surrender flags, because that is logical given the conversation. It will "understand" that too. But the article says it can't do that. And the article, sorry, is wrong.
- Hendrikto 1y ago> This is terrible write-up, simply because it's the "Reddit Expert" phenomena but in print. How is your comment any different?
- clejack 1y agoAre people still experiencing llms getting stuck in knowledge and comprehension loops? I used them but not excessively, and I'm not heavily tracking their performance either. For example, if you ask an llm a question, and it produces a hallucination then you try to correct it or explain to it that it is incorrect; and it produces a near identical hallucination while implying that it has produced a new, correct result, this suggests that it does not understand its own understanding (or pseudo-understanding if you like). Without this level of introspection, directing any notion of true understanding, intelligence, or anything similar seems premature. Llms need to be able to consistently and accurately say, some variation on the phrase "I don't know," or "I'm uncertain." This indicates knowledge of self. It's like a mirror test for minds.
- ramchip 1y agoLike the article says... I feel it's counter-productive to picture an LLM as "learning" or "thinking". It's just a text generator. If it's producing code that calls non-existent APIs for instance, it's kind of a waste of time to try to explain to the LLM that so-and-so doesn't exist. Better just try again and dump an OpenAPI doc or some sample code into it to influence the text generator towards correct output.
- thomastjeffery 1y agoThat's the difference between bias and logic. A statistical model is applied bias, just like computation is applied logic/arithmetic. Once you realize that, it's pretty easy to understand the potential strengths and limitations of a model. Both approaches are missing a critical piece: objectivity. They work directly with the data, and not about the data.
- martindbp 1y agoMany people who claim that people don't understand how AI works often have a very simplified view of the short comings of LLMs themselves, e.g. "it's just predicting the next token", "it's just statistics", "stochastic parrot" and seems to be grounded in what AI was 2-3 years ago. Rarely have they actually read the recent research on interpretability. It's clear LLMs are doing more than just pattern matching. They may not think like humans or as well, but it's not k-NN with interpolation.
- beardedwizard 1y agoApple recently published a paper that seems to disagree and plainly states it's just pattern matching along with tests to prove it. https://machinelearning.apple.com/research/illusion-of-thinking https://machinelearning.apple.com/research/illusion-of-think...
- stuckinhell 1y agoI'm having a hard time taking apple seriously, when they have don't even have a great llm. https://www.techrepublic.com/article/news-anthropic-ceo-ai-interoperability/ https://www.techrepublic.com/article/news-anthropic-ceo-ai-i... Anthropic CEO: “We Do Not Understand How Our Own AI Creations Work”. I'm going to lean with Anthropic on this one.
- otabdeveloper4 1y ago> I have a hard time taking your claim about rotten eggs seriously when you're not even a chicken.
- beardedwizard 1y agoI guess I prefer to look at empirical evidence over feelings and arbitrary statements. AI ceos are notoriously full of crap and make statements with perverse financial incentives.
- martindbp 1y agoAnthropic has done much more in depth research actually introspecting the circuits: https://transformer-circuits.pub/2025/attribution-graphs/biology.html https://transformer-circuits.pub/2025/attribution-graphs/bio...
- stevenhuang 1y agoIt is a logic error to think that knowing how something works means you are justified to say it can't possess qualities like intelligence or ability to reason when we don't even understand how these qualities arise in humans. And even if we do know enough about our brains to say conclusively that it's not how LLMs work (predictive coding suggests the principles are more alike that not), it doesn't mean they're not reasoning or intelligent; it would just mean they would not be reasoning/intelligent like humans.
- yahoozoo 1y agoWhy do these same books coming out of AI (Empire of AI, The AI Con) keep getting referenced in all of these articles? It seems like some kind of marketing campaign.
- tim333 1y agoI guess if you an english literature grad type the normal way to approach a subject is to look at the leading books in that area.
- tim333 1y ago>Demis Hassabis, [] said the goal is to create “models that are able to understand the world around us.” >These statements betray a conceptual error: Large language models do not, cannot, and will not “understand” anything at all. This seems quite a common error in the criticism of AI. Take a reasonable statement about AI not mentioning LLMs and then say the speaker (nobel prize winning AI expert in this case) doesn't know what they are on about because current LLMs don't do that. Deepmind already have project Astra, a model but not just language but also visual and probably some other stuff where you can point a phone at something and ask about it and it seems to understand what it is quite well. Example here https://youtu.be/JcDBFAm9PPI?t=40 https://youtu.be/JcDBFAm9PPI?t=40
- NoMoreNicksLeft 1y ago>Deepmind already have project Astra, a model but not just language but also visual and probably some other stuff where you can point a phone at something and ask about it and it seems to understand what it is quite well. Operative phrase "seems to understand". If you had some bizarre image unlike anything anyone's ever seen before and showed it to a clever human, the human might manage to figure out what it is after thinking about it for a time. The model could never figure out anything, because it does not think. It's just a gigantic filter that takes known-and-similar images as input, and spits out a description on the other side, quite mindlessly. The language models do the same thing, do they not? They take prompts as inputs, and shit output from their LLM anuses based on those prompts. They're even deterministic if you take the seeds into account. We'll scale all those up, and they'll produce ever-more-impressive results, but none of these will ever "understand" anything.
- joenot443 1y ago> If you had some bizarre image unlike anything anyone's ever seen before and showed it to a clever human, the human might manage to figure out what it is after thinking about it for a time Out of curiosity, what sort of 'bizarre image' are you imagining here? Like a machine which does something fantastical? I actually think the quantity of bizarre imagery whose content is unknown to humans is pretty darn low. I'm not really well-equipped to have the LLMs -> AGI discussion, much smarter people have said much more poignant things. I will say that anecdotally, anything I've been asking LLMs for has likely been solved many times by other humans, and in my day to day life it's unusual I find myself wanting to do things never done before.
- throwawaymaths 1y agoi think this author doesnt fully understand how llms work either. Dismissing it as "a statistical model" is silly. hell, quantum mechanics is a statistical model too. moreover, each layer of an llm imbues the model with the possibility of looking further back in the conversion and imbuing meaning and context through conceptual associations (thats the k-v part of the kv cache). I cant see how this doesn't describe, abstractly, human cognition. now, maybe llms are not fully capable of the breadth of human cognition or have a harder time training to certain deeper insight, but fundamentally the structure is there (clever training and/or architectural improvements may still be possible -- in the way that every CNN is a subgraph of a FCNN that would be nigh impossible for a FCNN to discover randomly through training) to say llms are not smart in any way that is recognizable is just cherry-picking anecdotal data. if llms were not ever recognizably smart, people would not be using them the way they are.
- 827a 1y ago> I cant see how this doesn't describe, abstractly, human cognition. now, maybe llms are not fully capable of the breadth of human cognition But, I can fire back with: You're making the same fallacy you correctly assert the article as making. When I see how a CPU's ALU adds two numbers together, it looks strikingly similar to how I add two numbers together in my head. I can't see how the ALU's internal logic doesn't describe, abstractly, human cognition. Now, maybe the ALU isn't fully capable of the breadth of human cognition... It turns out, the gaps expressed in the "fully capable of the breadth of human cognition" part really, really, really matter. Like, when it comes to ALUs, they overwhelm any impact that the parts which look similar cover. The question should be: How significant are the gaps in how LLMs mirror human cognition? I'm not sure we know, but I suspect they're significant enough to not write away as trivial.
- throwawaymaths 1y agodo they matter in a practical sense? an LLM can write a structured essay better than most undergrads. and as for measuring "smart", we throw that word around a lot. a dog is smart in a human way for being able to fetch one of 30 objects based on name or to drive a car (yes, dogs can drive), the bar for "smart" is pretty low, claiming llms are not smart is just prejudice.
- mmsc 1y agoThis can be generalized to "what happens when people don't understand how something works". In the computing world, that could be "undefined behavior" (of which itself is .. defined as undefined) in the C programming language, or anything as simple as "functionality people didn't know because they didn't read the documentation"
- pier25 1y agoPeople in tech and science might have a sense that LLMs are word prediction machines but that's only scratching the surface. Even AI companies have a hard time figuring out how emergent capabilities work. Almost nobody in the general audience understands how LLMs work.
- elia_42 1y agoTotally agree with the content of the article. In part, AI is certainly able to simulate very well the behavior and operations of a "way of expressing itself" of our mind, that is, mathematical calculation, deductive reasoning and other similar things. But our mind is extremely polymorphic and these operations represent only one side of a much more complex and difficult to explain whole. Even Alan Turing, in his writings on the possibility of building a mechanical intelligence, realized that it was impossible for a machine to completely imitate a human being: for this to be possible, the machine would have to "walk among other humans, scaring all the citizens of a small town" (Turing says more or less like this). Therefore, he realized many years ago that he had to face this problem with a very cautious and limited approach, limiting the imitative capabilities of the machine to those human activities in which calculation, probability and arithmetic are main, such as playing chess, learning languages and mathematical calculation.
- Havoc 1y agoWe can debate about intelligence all day but there is also an element of “if it’s stupid but it works then it’s not stupid” here A very large portion of tasks humans do don’t need all that much deep thinking. So on that basis it seems likely that it’ll be revolutionary.
- pmdr 1y ago> Whitney Wolfe Herd, the founder of the dating app Bumble, proclaimed last year that the platform may soon allow users to automate dating itself, disrupting old-fashioned human courtship by providing them with an AI “dating concierge” that will interact with other users’ concierges until the chatbots find a good fit. > Herd doubled down on these claims in a lengthy New York Times interview last month. Seriously, what is wrong with these people?
- quickthrowman 1y agoWell, her specific problem is she was a billionaire but isn’t one now so she’ll say damn near anything to regain that third comma. Nothing more than greed. Match just keeps Bumble around to avoid antitrust legislation, similar to Google and Mozilla’s position. Edit: It’s not that wild of an idea anyways, there’s a good black mirror episode about it.
- jajko 1y agoif it works (from their perspective), it ain't stupid
- acdha 1y agoHer problem is that BMBL is down 92% and they need to tell investors that they’ll all be rich again: https://finance.yahoo.com/quote/BMBL/ https://finance.yahoo.com/quote/BMBL/ Most of the dumb AI pitches share that basic goal: someone is starting from what investors want to be true and using “AI” like it’s a magic spell which can make that possible, just as we’ve seen going back to the dawn of the web. Sober voices don't get attention because it’s boring to repeat a 10% performance improvement or reduction in cost.
- codethief 1y ago> Her problem is that BMBL is down 92% and they need to tell investors that they’ll all be rich again Is this also why Bumble has undergone so many drastic changes in recent times? I always thought they must hired some new & overzealous product managers that didn't actually understand the secret sauce that had made their product so successful in the first place. Either way, it seems the usual enshittification has begun.
- Notatheist 1y agoWasn't it Feynman who said we will never be impressed with a computer that can do things better than a human can unless that computer does it the same way a human being does? AI could trounce experts as a conversational partner and/or educator in every imaginable field and we'd still be trying to proclaim humanity's superiority because technically the silicon can't 'think' and therefore it can't be 'intelligent' or 'smart'. Checkmate, machines!
- varelse 1y ago[dead]
- ineedasername 1y agoSo many of these articles jump around to incredibly different concerns or research questions. This one raises plenty of important questions but threads a narrative through them that attempts to lump it all as a single issue. Just to start off with, saying LLM models are "not smart" and "don't/won't/can't understand" ... That is really not a useful way to begin any conversation about this. To "understand" is itself a word without, in this context, any useful definition that would allow evaluation of models against it. It's this imprecision that is at the root of so much hand wringing and frustration by everyone.
- jemiluv8 1y agoMost people without any idea about the foundations on which LLMs are built call them AI. But I insist on calling them LLMs, further creating confusion. How do you explain what a large language model is to someone that can't comprehend how a machine can learn a "word model" on a large corpus of text/data to make it generate "seemingly sound/humane" responses without making them feel like they are interacting with the AI that they've been hearing about in the movies/sci-fi?
- jemiluv8 1y agoEven I have limited understanding of how LLMs learn the semantic meaning of words. My knowledge is shallow at best. I know however that LLMs understand text now. Are able to understand concepts they "glean" from text and are able to give responses to queries that is not entirely made up. All these makes it a lot harder to explain to non-technical people what this is. I tell them these LLMs are not AI but when they go to these websites - they see it labelled as an AI chatbot. It also mostly does as advertised. And they are often in awe of whatever responses they tend to receive because they are not subject matter experts nor do they care to become one. They just want to get their "homeworks" done, complete their work assignments and this gets them there faster. How can I tell them it is not AI when it spews humane looking text. Heck, even I don't quite understand the "real" difference between LLMs and AI. The difference is nuanced but the line is clearer with a bit of technical understanding. The machine understands text. And can make conversation - however sycophantic. But without understanding why that is - I don't see why we won't exult its powers. I see religions sprouting from these soon. LLMs can deliver awesome sermons. And once you train them well enough, can take on the role of Messiah's.
- Zaylan 1y agoIt’s so easy to think of AI as “conscious,” especially when it sounds so natural. A lot of companies lean into that, making AI feel like a real person. But in the end, it’s just prediction and pattern-matching. I’m curious how we can help more people see the difference between simulated understanding and real understanding.
- jasonm23 1y agoWhat happens when people who don't understand how AI works, are writing articles about What happens when people who don't understand how AI works?
- BlazeNova 1y ago[dead]
- frozenseven 1y agoIt will be very interesting to see how articles like this age over the next year or two.