3 ms·

Transparency won't help a lot from a technical standpoint (seems more like a solution to a legal issue than a technical one). I can't trust LLMs because they ju

Transparency won't help a lot from a technical standpoint (seems more like a solution to a legal issue than a technical one). I can't trust LLMs because they just...recombine text by probabilities and aren't deterministic. I get incorrect information every time I ask them a thing, and it's incorrect in different ways every time. The only things they seem to get consistently correct are very widespread facts that are much faster and trivial to google. I still find it funny we managed to get image generation working so much better than text.

ToucanLoucan 3y ago
So much of the odd almost cultish community around LLMs seems to just be people who really want to be at the ground floor of the Next Big Thing who are so wildly biased into this being that next big thing that they will spend all their time, all their energy, not just on other people but on themselves, convincing themselves over and over that their LLM girlfriend really does love them, that their LLM assistant is going to be the next DaVinci, that their generated art is so much better than anything else. More than anything it makes me sad. ANY amount of critical thinking would tell you all of this is not true, which isn't to say there's NO USE AT ALL for this technology, it certainly exists and has it's applications, and I also do more or less believe someday we'll create digital intelligence, but at the same time... ChatGPT is not that. DALL-E is not that. These systems are interesting and they have uses but they are not emergent intelligence, they don't know anything, they just assemble words from massive probability matrices and then the people who read those words ascribe meaning to them that is far, far beyond what originated them. In this way it's not so dissimilar from any garden variety religion, it's just religion for people who think they're too smart to fall into the trap of motivated reasoning and magical thinking.
- RodgerTheGreat 3y ago
  I think it's under-appreciated how much LLMs harvest the natural, human tendency to generously ascribe meaning, subtext, and intent to text they read, glossing over flaws and small mistakes so long as the overall "thrust" seems reasonable enough. In a sense, LLMs have reinvented cold-reading from first-principles and created the cleverest Hans of them all.
  - adamrezich 3y ago
    We really should have figured out anthropomorphization and how to overcome it before we got to the "AI age".
    - pixl97 3y ago
      Heh, I don't think you've had enough philosophy under your belt to realize how incredibly far we are from figuring that out... Or to say, "It's Plato's cave all the way down"
    - RodgerTheGreat 3y ago
      One of the worst-case scenarios I can presently imagine for LLMs is a world so drowned in semi-plausible nonsense that new generations teach themselves to simply ignore the printed word, subconsciously categorizing it as noise, and in so doing essentially un-invent literacy.
      - wizzwizz4 3y ago
        Then, maybe, we can re-invent libraries. (https://xkcd.com/1909/ https://xkcd.com/1909/)
      - fzzzy 3y ago
        Literacy isn't required if a computer can read any text. That is a huge amount of the brain that could be applied to something else.
        ToucanLoucan 3y ago
        I genuinely haven't in some time read an opinion so horrifying as this. Well done.
        RodgerTheGreat 3y ago
        Do you think it would have the same psychological impact if you had to ask your cell phone to read it to you?
- floren 3y ago
  "You must have been using GPT 3.5..."
- Me1000 3y ago
  You seem to be making a point in good faith so I would like to give you a slightly different perspective. I'm not entirely sure what you mean by "cultish community", from my perspective there are a few distinct communities around LLMs, all focusing on different aspects, all excited about different things. One common theme across all the groups though is that they used an LLM for the first time and their mind ran wild with the possibilities. That first moment when the LLM does something better than you expected, or even completely unexpected. I think most people understand that their imagination might be overactive in that moment. But it's a rare feeling to be surprised by a new technology (at least for me) these days. On the other end, we have social media platforms where being a pessimistic curmudgeon ends up getting the likes and shares. And it's just easier to be a pessimistic curmudgeon; the vast majority of ideas never work as well in the real world as they do in your head. I'm just as guilty of this as anyone else. But the real problem is that it puts us into tribes. As someone who is very excited about what LLMs are going to bring to our futures, when I see someone post on Mastodon or HN, or wherever, I become defensive and my monkey brain feels the urge to push back. In particular because I think the criticisms generally voiced are not well reasoned or thought out. Your own post has a tone of dismissal, painting a lot of people, all of whom excited about different things, as a cult who is obsessed with their LLM girlfriend. I would agree that anyone today trying to draw some deeper meaning from the outputs of these systems are probably worthy of dismissal, but I don't think that's the vast vast majority of people who are excited about LLMs. And it makes _me_ sad that the extremists are the ones that get to suck all the oxygen out of the conversation. We're in the beginning days of this new technology. LLMs are good at doing things traditional software isn't, and bad at doing a lot of things computers are traditionally good at. Natural language answer engines and sex bots might have been some of the first obvious applications of LLMs, but I'm willing to bet there are a lot more undiscovered use cases out there. Simon Willison has some great advice, which is for newcomers to try to break the LLM as quickly as they can, get it to lie to you, or do something wrong. Test its limits. That's part of the process! We're going to need some time to figure it all out and make these systems work well for us. I'm a technologist, and exploring this technology is exciting.
  - ToucanLoucan 3y ago
    I mean I think the biggest problem with the LLM community is that a sizable portion of not a majority of it consists of the exact same opportunistic "ground floor" people as just jumped off of cryptocurrency as the "thing they refuse to shut up about" which is all well and good when it's an exciting new thing that might change the world, but gets notably more insufferable when it's obviously just the newest route that person sees to getting greater amounts of money and/or social media clout, and they clearly do not know anything about it beyond it's potential to do those things. The colloquial term for that, I think, being "grifter." And to be clear, LLMs are not unique in their ability to draw in those types of insufferable people: see the aforementioned comment about crypto, it was the last big one, and before that probably dropshipping. But I digress. And I bring all that up to say: no, the vast majority of people, I don't think expect the computer to come to life and tell them it loves them. I think the vast majority, in fact, don't know a fucking thing about LLMs beyond maybe the rote copy/pasted code it takes to bring one into existence, or if we're being honest, more likely, the websites to put their credit card information into to gain access to one for their use, and that shift in base assumptions I think explains why they speak so incoherently: they do not understand it in any depth, and thereofre, they might think the computer will come to life, because they don't know much about computers in general and to a layman, what an LLM does can indeed look like a vague imitation of life. Like, I mean this in the nicest way possible though just by virtue of what I'm going to say, it is going to sound mean, but: tons of the really big pro-AI hype people just, clearly, bluntly, full disclosure, do not know shit about LLM. Quite a large slice of that pie also don't know shit about technology in general, or seemingly, much of anything beyond a business degree? But irrespective of that, to the wider world who aren't in this and don't participate in the groups at hand... those are your representatives, by default. The attention economy has produced them and you have my most sincere sympathies for that.
chongli 3y ago
I still find it funny we managed to get image generation working so much better than text. If you care about veracity then image generation works about as well as text. Frequently you can find details of the image that are just bizarrely wrong, such as hands or food or other basic things. It's the same basic problem: there's no intelligence behind what it's doing, it just regurgitates mostly realistic-seeming pixels that are pretty good at fooling the casual viewer. Really, it's like those moths with eyespots on them: good at fooling the brain's heuristics but obviously not real.
- falsaberN1 3y ago
  Try looking at things from my angle: A few errors in an image can be not much of a big deal (with modern tools, the mistakes are within human margin of error on average), but errors in delivery of textual data such as facts, dates or code can be far more severe and subtle. There are ways to work around or reduce the shortcomings of image generation, and the quirks you mention have drop-in solutions for local installs, but you can't quite fix wrong facts in text automatically, or the context going off-rails. It can be much harder to catch than a hand missing a finger, too. It's also worth mentioning you can run a heavily customized Stable Diffusion setup at home with fairly modest hardware with satisfactory results if you know what you are doing, but anything you can run at home for LLMs in the same hardware is dog slow and actually kind of terrible.
  - gamacodre 3y ago
    I think this is still just a difference in how the output is used. You're presenting text generation as factual and image generation as artistic. It could be reversed - no one will care if a fantasy story gets some in-milieu "facts" wrong, but a blueprint or architectural reference coming out of Stable Diffusion could ruin someone's year.
brookst 3y ago
s/LLMs/people

ToucanLoucan 3y ago

So much of the odd almost cultish community around LLMs seems to just be people who really want to be at the ground floor of the Next Big Thing who are so wildly biased into this being that next big thing that they will spend all their time, all their energy, not just on other people but on themselves, convincing themselves over and over that their LLM girlfriend really does love them, that their LLM assistant is going to be the next DaVinci, that their generated art is so much better than anything else. More than anything it makes me sad. ANY amount of critical thinking would tell you all of this is not true, which isn't to say there's NO USE AT ALL for this technology, it certainly exists and has it's applications, and I also do more or less believe someday we'll create digital intelligence, but at the same time... ChatGPT is not that. DALL-E is not that. These systems are interesting and they have uses but they are not emergent intelligence, they don't know anything, they just assemble words from massive probability matrices and then the people who read those words ascribe meaning to them that is far, far beyond what originated them. In this way it's not so dissimilar from any garden variety religion, it's just religion for people who think they're too smart to fall into the trap of motivated reasoning and magical thinking.

RodgerTheGreat 3y ago
I think it's under-appreciated how much LLMs harvest the natural, human tendency to generously ascribe meaning, subtext, and intent to text they read, glossing over flaws and small mistakes so long as the overall "thrust" seems reasonable enough. In a sense, LLMs have reinvented cold-reading from first-principles and created the cleverest Hans of them all.
- adamrezich 3y ago
  We really should have figured out anthropomorphization and how to overcome it before we got to the "AI age".
  - pixl97 3y ago
    Heh, I don't think you've had enough philosophy under your belt to realize how incredibly far we are from figuring that out... Or to say, "It's Plato's cave all the way down"
  - RodgerTheGreat 3y ago
    One of the worst-case scenarios I can presently imagine for LLMs is a world so drowned in semi-plausible nonsense that new generations teach themselves to simply ignore the printed word, subconsciously categorizing it as noise, and in so doing essentially un-invent literacy.
    - wizzwizz4 3y ago
      Then, maybe, we can re-invent libraries. (https://xkcd.com/1909/ https://xkcd.com/1909/)
    - fzzzy 3y ago
      Literacy isn't required if a computer can read any text. That is a huge amount of the brain that could be applied to something else.
      - ToucanLoucan 3y ago
        I genuinely haven't in some time read an opinion so horrifying as this. Well done.
        RodgerTheGreat 3y ago
        Do you think it would have the same psychological impact if you had to ask your cell phone to read it to you?
floren 3y ago
"You must have been using GPT 3.5..."
Me1000 3y ago
You seem to be making a point in good faith so I would like to give you a slightly different perspective. I'm not entirely sure what you mean by "cultish community", from my perspective there are a few distinct communities around LLMs, all focusing on different aspects, all excited about different things. One common theme across all the groups though is that they used an LLM for the first time and their mind ran wild with the possibilities. That first moment when the LLM does something better than you expected, or even completely unexpected. I think most people understand that their imagination might be overactive in that moment. But it's a rare feeling to be surprised by a new technology (at least for me) these days. On the other end, we have social media platforms where being a pessimistic curmudgeon ends up getting the likes and shares. And it's just easier to be a pessimistic curmudgeon; the vast majority of ideas never work as well in the real world as they do in your head. I'm just as guilty of this as anyone else. But the real problem is that it puts us into tribes. As someone who is very excited about what LLMs are going to bring to our futures, when I see someone post on Mastodon or HN, or wherever, I become defensive and my monkey brain feels the urge to push back. In particular because I think the criticisms generally voiced are not well reasoned or thought out. Your own post has a tone of dismissal, painting a lot of people, all of whom excited about different things, as a cult who is obsessed with their LLM girlfriend. I would agree that anyone today trying to draw some deeper meaning from the outputs of these systems are probably worthy of dismissal, but I don't think that's the vast vast majority of people who are excited about LLMs. And it makes _me_ sad that the extremists are the ones that get to suck all the oxygen out of the conversation. We're in the beginning days of this new technology. LLMs are good at doing things traditional software isn't, and bad at doing a lot of things computers are traditionally good at. Natural language answer engines and sex bots might have been some of the first obvious applications of LLMs, but I'm willing to bet there are a lot more undiscovered use cases out there. Simon Willison has some great advice, which is for newcomers to try to break the LLM as quickly as they can, get it to lie to you, or do something wrong. Test its limits. That's part of the process! We're going to need some time to figure it all out and make these systems work well for us. I'm a technologist, and exploring this technology is exciting.
- ToucanLoucan 3y ago
  I mean I think the biggest problem with the LLM community is that a sizable portion of not a majority of it consists of the exact same opportunistic "ground floor" people as just jumped off of cryptocurrency as the "thing they refuse to shut up about" which is all well and good when it's an exciting new thing that might change the world, but gets notably more insufferable when it's obviously just the newest route that person sees to getting greater amounts of money and/or social media clout, and they clearly do not know anything about it beyond it's potential to do those things. The colloquial term for that, I think, being "grifter." And to be clear, LLMs are not unique in their ability to draw in those types of insufferable people: see the aforementioned comment about crypto, it was the last big one, and before that probably dropshipping. But I digress. And I bring all that up to say: no, the vast majority of people, I don't think expect the computer to come to life and tell them it loves them. I think the vast majority, in fact, don't know a fucking thing about LLMs beyond maybe the rote copy/pasted code it takes to bring one into existence, or if we're being honest, more likely, the websites to put their credit card information into to gain access to one for their use, and that shift in base assumptions I think explains why they speak so incoherently: they do not understand it in any depth, and thereofre, they might think the computer will come to life, because they don't know much about computers in general and to a layman, what an LLM does can indeed look like a vague imitation of life. Like, I mean this in the nicest way possible though just by virtue of what I'm going to say, it is going to sound mean, but: tons of the really big pro-AI hype people just, clearly, bluntly, full disclosure, do not know shit about LLM. Quite a large slice of that pie also don't know shit about technology in general, or seemingly, much of anything beyond a business degree? But irrespective of that, to the wider world who aren't in this and don't participate in the groups at hand... those are your representatives, by default. The attention economy has produced them and you have my most sincere sympathies for that.

chongli 3y ago

I still find it funny we managed to get image generation working so much better than text. If you care about veracity then image generation works about as well as text. Frequently you can find details of the image that are just bizarrely wrong, such as hands or food or other basic things. It's the same basic problem: there's no intelligence behind what it's doing, it just regurgitates mostly realistic-seeming pixels that are pretty good at fooling the casual viewer. Really, it's like those moths with eyespots on them: good at fooling the brain's heuristics but obviously not real.

falsaberN1 3y ago
Try looking at things from my angle: A few errors in an image can be not much of a big deal (with modern tools, the mistakes are within human margin of error on average), but errors in delivery of textual data such as facts, dates or code can be far more severe and subtle. There are ways to work around or reduce the shortcomings of image generation, and the quirks you mention have drop-in solutions for local installs, but you can't quite fix wrong facts in text automatically, or the context going off-rails. It can be much harder to catch than a hand missing a finger, too. It's also worth mentioning you can run a heavily customized Stable Diffusion setup at home with fairly modest hardware with satisfactory results if you know what you are doing, but anything you can run at home for LLMs in the same hardware is dog slow and actually kind of terrible.
- gamacodre 3y ago
  I think this is still just a difference in how the output is used. You're presenting text generation as factual and image generation as artistic. It could be reversed - no one will care if a fantasy story gets some in-milieu "facts" wrong, but a blueprint or architectural reference coming out of Stable Diffusion could ruin someone's year.

brookst 3y ago

s/LLMs/people